Mais conteúdo relacionado Semelhante a TENSORFLOW深度學習講座講義(很硬的課程) 4/14 (20) Mais de NTC.im(Notch Training Center) (7) TENSORFLOW深度學習講座講義(很硬的課程) 4/146. ⼈人腦 vs 電腦
• ⼈人腦優勢:
– 影像、聲⾳音
– 語⾔言
– ⾃自我意識(⾃自決⼒力)
– …
• 電腦優勢:
– 數學運算
– 記憶(儲存)能⼒力
– …
16. ⾮非監督式學習
機器學習模型
Beijing is the capital of China.
As China's capital, Beijing is a large and vibrant city.
Tokyo is the capital of Japan.
As Japan’s capital, Tokyo is a large and vibrant city.
…….
資料
結果
24. 訓練資料
X
Y
-0.47241379 0
-0.35344828 0
-0.30148276 0
0.33448276 1
0.35344828 1
0.37241379 1
0.39137931 1
0.41034483 1
0.44931034 1
0.49827586 1
0.51724138 1
…. ….
26. 修正模型
• Error function : Cross Entropy
E(h(X), Y ) =
1
m
(
mX
i
y(i)
log(h(x(i)
)) + (1 y(i)
)log(1 h(x(i)
)))
h(x(i)
) ⇡ 0 and y(i)
= 0 ) E(h(X), Y ) ⇡ 0
h(x(i)
) ⇡ 1 and y(i)
= 1 ) E(h(X), Y ) ⇡ 0
h(x(i)
) ⇡ 0 and y(i)
= 1 ) E(h(X), Y ) ⇡ 1
h(x(i)
) ⇡ 1 and y(i)
= 0 ) E(h(X), Y ) ⇡ 1
31. nout = 1
nout = 0.5
nout = 0(0,0)
x2
x1
模擬神經元
nin = w1x1 + w2x2 + wb
nout =
1
1 + e nin
nin = w1x1 + w2x2 + wb
nout =
1
1 + e nin
w1x1 + w2x2 + wb = 0
w1x1 + w2x2 + wb > 0
w1x1 + w2x2 + wb < 0
1
0
32. ⼆二元分類:AND Gate
x1
x2
y
0
0
0
0
1
0
1
0
0
1
1
1
(0,0)
(0,1)
(1,1)
(1,0)
0
1
n
20
20
b
-30
y
x1
x2
y =
1
1 + e (20x1+20x2 30)
20x1 + 20x2 30 = 0
34. ⼆二元分類:XOR Gate
n
-20
20
b
-10
y
(0,0)
(0,1)
(1,1)
(1,0)
0
1
(0,0)
(0,1)
(1,1)
(1,0)
1
0
(0,0)
(0,1)
(1,1)
(1,0)
0
0
1
n1
20
20
b
-30
x1
x2
n2
20
20
b
-10
x1
x2
x1
x2
n1
n2
y
0
0
0
0
0
0
1
0
1
1
1
0
0
1
1
1
1
1
1
0
35. 類神經網路
x
y
n11
n12
n21
n22
W12,y
W12,x
b
W11,y
W11,b
W12,b
b
W11,x
W21,11
W22,12
W21,12
W22,11
W21,b
W22,b
z1
z2
Input
Layer
Hidden
Layer
Output
Layer
39. 初始化
• 將所有的W隨機設成-N~N之間的數
• 每層之間W的值都不能相同
x
y
n11
n12
n21
n22
W12,y
W12,x
b
W11,y
W11,b
W12,b
b
W11,x
W21,11
W22,12
W21,12
W22,11
W21,b
W22,b
z1
z2
42. Error Function
J = (z1log(n21(out)) + (1 z1)log(1 n21(out)))
(z2log(n22(out)) + (1 z2)log(1 n22(out)))
n21
n22
z1
z2
nout ⇡ 0 and z = 0 ) J ⇡ 0
nout ⇡ 1 and z = 1 ) J ⇡ 0
nout ⇡ 0 and z = 1 ) J ⇡ 1
nout ⇡ 1 and z = 0 ) J ⇡ 1
43. w1
w0
Gradient Descent
w21,11 w21,11 ⌘
@J
@w21,11
w21,12 w21,12 ⌘
@J
@w21,12
w21,b w21,b ⌘
@J
@w21,b
w22,11 w21,11 ⌘
@J
@w22,11
w22,12 w21,12 ⌘
@J
@w22,12
w22,b w21,b ⌘
@J
@w22,b
w11,x w11,x ⌘
@J
@w11,x
w11,y w11,y ⌘
@J
@w11,y
w11,b w11,b ⌘
@J
@w11,b
w12,x w12,x ⌘
@J
@w12,x
w12,y w12,y ⌘
@J
@w12,y
w12,b w12,b ⌘
@J
@w12,b
(–
@J
@w0
, –
@J
@w1
)
53. Example : Binary Classification
n
w1
w2
1
b
y
x1
x2
x1
x2
y
y =
1
1 + e x1w1+x2w2+b
x_data = np.random.rand(50,2)
y_data = ((x_data[:,1] > 0.5)*
( x_data[:,0] > 0.5))
模型 資料
54. Example : Binary Classification
n
w1
w2
1
b
y
x1
x2
y =
1
1 + e x1w1+x2w2+b
訓練後
55. Tensorflow
x_ = tf.placeholder(tf.float32,[None,2])
y_ = tf.placeholder(tf.float32,[None,1])
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
y = tf.nn.sigmoid(tf.matmul(x_,w)+b)
cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1- y_) * tf.log(1 - y) )
optimizer = tf.train.GradientDescentOptimizer(0.1)
train = optimizer.minimize(cross_entropy)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
for step in xrange(500):
sess.run(train,feed_dict={x_:x_data,y_:y_data})
print sess.run(cross_entropy)
sess.close()
Computational
Graph
Session
56. Computation Graph
# placeholder
x_ = tf.placeholder(tf.float32,[None,2])
y_ = tf.placeholder(tf.float32,[None,1])
# variable
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
# operations
y = tf.nn.sigmoid(tf.matmul(x_,w)+b)
# error function
cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1-
y_) * tf.log(1 - y) )
# trainer
optimizer = tf.train.GradientDescentOptimizer(0.1)
train = optimizer.minimize(cross_entropy)
# initalizer
init = tf.initialize_all_variables()
57. Placeholder
0.70828883
0.27190551
0.89042455
0.63832092
0.11332515
0.00849676
0.73278006
0.37781084
0.292448
0.09819899
0.9802261
0.94339143
0.36212146
0.54404682
……..
……..
0!
1!
0!
0!
0!
1!
0!
…!
x_ y_
x_ = tf.placeholder(tf.float32,[None,2])
y_ = tf.placeholder(tf.float32,[None,1])
59. Matrix Multiplication
y = tf.nn.sigmoid(tf.matmul(x_,w)+b)
0.42905441
-‐0.43841863
w
x_
0.70828883
0.27190551
0.89042455
0.63832092
0.11332515
0.00849676
….
….
0.184686
0.1021888
0.04489752
….
tf.matmul(x_,w)+b
b
0!
0.70828883
*
0.42905441
+
0.27190551
*
-‐0.43841863
+
0
0.89042455*
0.42905441
+
0.63832092*
-‐0.43841863
+
0
0.11332515*
0.42905441
+
0.00849676*
-‐0.43841863
+
0
….
61. Error Function
E(h(X), Y ) =
1
m
(
mX
i
y(i)
log(h(x(i)
)) + (1 y(i)
)log(1 h(x(i)
)))
cross_entropy = -tf.reduce_sum(y_*tf.log(y) + (1- y_) * tf.log(1 - y) )
0.54604071
0.52552499
…
0!
1!
…!
y_ y
1.4331052
-tf.reduce_sum(y_*tf.log(y))
64. Computation Graph
• Initializer
init = tf.initialize_all_variables()
w
b
w = tf.Variable(tf.random_uniform([2,1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1,1]))
0.42905441
-0.43841863
0!
65. Session
# create session
sess = tf.Session()
# initialize variable
sess.run(init)
# gradient descent
for step in xrange(500):
sess.run(train, feed_dict={x_:x_data,y_:y_data})
# fetch variable
print sess.run(cross_entropy, feed_dict={x_:x_data,y_:y_data})
# release resource
sess.close()
67. Run Operations
for step in xrange(500):
sess.run(train, feed_dict={x_:x_data,y_:y_data} )
the Node in
Computational
Graph
Input
Data
0.70828883
0.27190551
0.89042455
0.63832092
0.11332515
0.00849676
0.73278006
0.37781084
……..
……..
0!
1!
0!
0!
…!
x_data y_data
68. Run Operations
print sess.run(cross_entropy, feed_dict={x_:x_data,y_:y_data})
the Node in
Computational
Graph
Input
Data
0.70828883
0.27190551
0.89042455
0.63832092
0.11332515
0.00849676
0.73278006
0.37781084
……..
……..
0!
1!
0!
0!
…!
x_data y_data
Results
2.4564333
70. Demo : Binary Classification
https://github.com/ckmarkoh/ntc_deeplearning_tensorflow/
blob/master/intro/binaryClassification.ipynb
77. About the Speaker
• Email: ckmarkoh at gmail dot com
• Blog: http://cpmarkchang.logdown.com
• Github: https://github.com/ckmarkoh
Mark Chang
• Facebook: https://www.facebook.com/ckmarkoh.chang
• Slideshare: http://www.slideshare.net/ckmarkohchang
• Linkedin:
https://www.linkedin.com/pub/mark-chang/85/25b/847
77