https://github.com/DeNA/ChainerPruner https://chainer.connpass.com/event/106292/

1. © DeNA Co., Ltd.
Chaienr
Channel Pruning
December 15, 2018
Tomohiro Kato
AI System Dept.
DeNA Co., Ltd.

2. © DeNA Co., Ltd.
2

3. © DeNA Co., Ltd.
3

4. © DeNA Co., Ltd.
n u o PV
n ) d d
n rRpR h D C )
n ( A 18 8 l cg a st
n ) 2 8 e nyiMLNm T I
n (728 D 1 01 D 8
n )8 8 7e A 8 I
n ) 2 8 e nyi ( /
n A 01
4

5. © DeNA Co., Ltd.
n CV
5
https://www.slideshare.net/ShotaSuzuki2/mlops-shibuyasynapse-4-125602323
https://dena.ai/work6/

6. © DeNA Co., Ltd.
n Deep Learning …
HW
n Channel Pruning
ChainerMeetup →
6

7. © DeNA Co., Ltd.
Channel Pruning
n Channel Pruning
Deep Learning
Weight CPU/GPU /
n Optuna Pruning ChainerPruningExtension
7

8. © DeNA Co., Ltd.
n ChainerCV ResNet50 channel pruning
8
MacBook Pro CPU, Chainer v5.1.0

9. © DeNA Co., Ltd.
demo
9

10. © DeNA Co., Ltd.
ChainerPruner
n https://github.com/DeNA/ChainerPruner
n Channel Pruning
n Channel Pruning
n Under development
10

11. © DeNA Co., Ltd.
ChainerPruner
n Pruning
Channel Pruning
n Pruning
Channel Pruning
11

12. © DeNA Co., Ltd.
Dynamic Pruning
n Pruning
Weight
n Pruning
Weight shape
1/0 0
Iteration
n Pruning load
:benri:
12

13. © DeNA Co., Ltd.
Mask and Rebuild
n Weight -step
Mask: Weight
Rebuild: Weight
n Mask
L2 N-%
chainerpruner.Mask
13

14. © DeNA Co., Ltd.
Rebuild mechanism
n Channel Pruning
n Conv – BN – FC Conv Pruning Bn FC
Bn, FC input channel Conv output channel update
n
n Conv - BN - Pool - ConvDW – FC Link Function
ChainerPruner
n
14

15. © DeNA Co., Ltd.
Rebuild mechanism
n
15

16. © DeNA Co., Ltd.
Serializers
n Pruning Weight
n Link Pruning Weight
16

17. © DeNA Co., Ltd.
Utility
n
https://github.com/belltailjp/chainer_computational_cost
wrapper
n pruning
n / /
17

18. © DeNA Co., Ltd.
n ChainerPruner
https://github.com/DeNA/ChainerPruner
Channel Pruning
n Future work
ChainerX support
Pruning
Reporter tensorboardX
Chainer
18

19. © DeNA Co., Ltd.
19
EOF