20210226 Oracle Code Night 講演 機械学習入門：ディープラーニングの基礎から転移学習まで

1. 2021 2 26
~

2. 2 Copyright © 2021 Oracle and/or its affiliates.
•
•
• (NN)
•
•
•
• NN
•
•

3. 3 Copyright © 2021 Oracle and/or its affiliates.
•
•
•
• Etc.
?%

4. 4 Copyright © 2021 Oracle and/or its affiliates.
•
•
•
• K-means
• SVM
• etc.
•

5. 5 Copyright © 2021 Oracle and/or its affiliates.
: Wikipedia
https://ja.wikipedia.org/wiki/%E7%A5%9E%E7%B5%8C%E7%B4%B0%E8%83%9E
• ニューロン：脳の神経細胞
• シナプス：ニューロンと他のニューロンとの接合部分
• シナプス結合：ニューロンから別のニューロンへ信号が伝達される状態
• 活性化：ニューロンが受け取った刺激の大きさが特定の大きさを超える
と、次のニューロンへと信号を伝達する。この状態を活性化と呼ぶ。
• 人工ニューロン(ノード)：ニューロンに相当。重みとバイアスを持つ。
• エッジ：シナプス結合に相当。ノード間を接続する。
• 重み、バイアス：シナプス結合の強度。ノード間接続時の信号の大きさの
計算に利用される。
• 活性化関数：ノード間の信号の大きさを計算するための関数。次ノードへ
伝える値を決める関数。

6. 6 Copyright © 2021 Oracle and/or its affiliates.
( )
𝑦 = 𝑓 ෍
𝑖=0
𝑛
𝑤𝑖𝑥𝑖 + 𝑏
𝑥0w0 + 𝑥1 w1 +… +𝑥𝑛𝑤𝑛 + 𝑏 =
b
input x w
n
𝑥0
𝑥1
𝑥2
w0
w1
𝑤2
y
𝑥𝑛
input
output
𝑏
0 ⇒ 1
0 ⇒ 0
• ⇒ 1 ⇒ 0
• ⇒ 1 ⇒ 0
• ⇒ 1 ⇒ 0
input output
• ⇒ 1
• ⇒ 0
→ 1
→ 0
→ 1
→ 2
→ 1
→ 3
→ 1
→ 3
→ 7
1*2 + 0*1 + 1*3 +3 = 8
7 ⇒ 1
7 ⇒ 0
)

7. 7 Copyright © 2021 Oracle and/or its affiliates.
• (
)
•
• (
)
• (
)
•
𝑥0
𝑥1
𝑥2
w0
w1
𝑤2
y
𝑥𝑛
input
output
𝑏
)
65
80
70
A
45
60
98
B
65
55
80
C
65
70
63
1
3
2
40
38
43

8. 8 Copyright © 2021 Oracle and/or its affiliates.
ReLU
Tanh
Sigmoid
Step
• 0
0 0
1
• NN (※
)
• 0 1
• 1
0
• -1 1
• 1
-1
• 0
0 0
• NN

9. 9 Copyright © 2021 Oracle and/or its affiliates.
ReLU
Tanh
Sigmoid
Step
• 0
0 0
1
• NN (※
)
• 0 1
• 1
0
• -1 1
• 1
-1
• 0
0 0
• NN
( ) ( )

10. 10 Copyright © 2021 Oracle and/or its affiliates.
Softmax
•
•
• ( )
(※ Sigmoid )
• 0 1
• 1
•
(※ )

11. 11 Copyright © 2021 Oracle and/or its affiliates.
Softmax
ReLU
ReLU 0.8
Softmax
ReLU
ReLU 0.2
1
0
Cat or Tiger ?
( )
Cat
Tiger
𝑥0
𝑥1
𝑥2
𝑏 𝑏 𝑏
𝑏
𝑏
𝑏
w
w
w
w
w
w
w
w
w
w
w
w
w
w
[Cat, Tiger] = [1, 0]
100 pixel
60
pixel
w ( )
( )
( ) w ( )

12. 12 Copyright © 2021 Oracle and/or its affiliates.
= ( )
= ( ) (w)
Softmax
ReLU
ReLU 0.8
Softmax
ReLU
ReLU 0.2
1
0
Cat or Tiger ?
( )
Cat
Tiger
𝑥0
𝑥1
𝑥2
𝑏 𝑏 𝑏
𝑏
𝑏
𝑏
w
w
w
w
w
w
w
w
w
w
w
w
w
w
[Cat, Tiger] = [1, 0]
100 pixel
60
pixel
( )

13. 13 Copyright © 2021 Oracle and/or its affiliates.
( )
w
( )

14. 14 Copyright © 2021 Oracle and/or its affiliates.
( )
≒(( ) – ( ))2
)
• 100
• A 90
• B 120
)
• ⇒ [Cat, Tiger] = [1, 0]
• A ⇒ [Cat, Tiger] = [0.8, 0.2]
• B ⇒ [Cat, Tiger] = [0.2, 0.8]
A
(1 x log0.8 + 0 x log0.2)
(log0.8)
(0.09)
0.09 ⇒
≒ log(( ) x ( ))
B
(1 x log0.2 + 0 x log0.8)
(log0.2)
(0.69)
0.69 ⇒
A
(100 - 90)2
100 ⇒
B
(100 - 120)2
400 ⇒

15. 15 Copyright © 2021 Oracle and/or its affiliates.
(Stochastic Gradient Descent, SGD)
(w) (w) (w)
( ) )
(w)

16. 16 Copyright © 2021 Oracle and/or its affiliates.
Stochastic Gradient Descent(SGD) w
momentumSGD SGD
AdaGrad SGD
RMSprop AdaGrad
AdaDelta SGD
Adaptive Moment Estimation(Adam) momentumSGD RMSprop
(SDG)

17. 17 Copyright © 2021 Oracle and/or its affiliates.

18. 18 Copyright © 2021 Oracle and/or its affiliates.
CNN ( )
• 28x28 /
• 6
•
0 9 0 9
• 28x28 /
• 1
•

19. 19 Copyright © 2021 Oracle and/or its affiliates.
CNN
784
2
16 10
768
1
16
0
1
2
8
9
8
784 786
10 0 9 10
768 x 1
input
input
input
input

20. 20 Copyright © 2021 Oracle and/or its affiliates.
•
•
•
•
•
•
•
•
W
W W

21. 21 Copyright © 2021 Oracle and/or its affiliates.
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
•
( )
• w (
)
• w

22. 22 Copyright © 2021 Oracle and/or its affiliates.
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w
w

23. 23 Copyright © 2021 Oracle and/or its affiliates.

24. 24 Copyright © 2021 Oracle and/or its affiliates.

25. 25 Copyright © 2021 Oracle and/or its affiliates.
: http://image-net.org
: https://cifar.ca/
• ImageNet
• 160 x 160 pixel
• 140
•
• CIFAR-10
• 32 x 32 pixel
• 5
•
• MobileNetV2
• 1000
• 10
W
W
W
※MobileNetV2by Google)
NN
?%