第19回 全脳アーキテクチャ若手の会 勉強会 カジュアルトーク(2016/11/30)で発表したスライドです。

1. Nextremer Co., LTD. Confidential Document
AI
19
2016 11 30
Twitter: @parc_b
1

2. Nextremer Co., LTD. Confidential Document
Nextremer
■ Nextremer
2
MINARAI
AI Samurai
AINOW PPAP AI
Samurai Nextremer

3. Nextremer Co., LTD. Confidential Document
■
■
■
■
3

4. Nextremer Co., LTD. Confidential Document
■
■
–
–
–
–
–
– etc.
4

5. Nextremer Co., LTD. Confidential Document
( )
–
–
(2015). Vol.2
(pp. 6-17)
5

6. Nextremer Co., LTD. Confidential Document
n-gram
■
■
■ ( )
■ 1. 2. 3.
– P( | ) = 0.1 ←
– P( | ) = 0.2 ←
– P( | ) = 0.7 ←
– 2
6

7. Nextremer Co., LTD. Confidential Document
(n-gram )
–
–
–
■ P( )
■ P( )
■ P( )
■ P( )
Graham Neubig.
7
P(wi) =
c(wi)
P
˜w c( ˜w)
= 4.12 ⇤ 10 23
= 3.53 ⇤ 10 24
= 6.53 ⇤ 10 25
= 6.53 ⇤ 10 27

8. Nextremer Co., LTD. Confidential Document

9. Nextremer Co., LTD. Confidential Document
Recurrent Neural Network
■ ( )
–
■ LSTM(Hochreiter & Schmidhuber, 1997) RNN
9
Christopher Olah. Neural Networks,
Manifolds, and Topology colah's blog
(RNN)
Christopher Olah. Understanding
LSTM Networks colah's blog

10. Nextremer Co., LTD. Confidential Document
RNN
■ Andrej Karpathy. The Unreasonable Effectiveness of
Recurrent Neural Networks Andrej Karpathy
blog
10
/*
* Increment the size file of the new incorrect UI_FILTER group information
* of the size generatively.
*/
static int indicate_policy(void)
{
int error;
if (fd == MARN_EPT) {
/*
* The kernel blank will coeld it to userspace.
*/
if (ss->segment < mem_total)
unblock_graph_and_set_blocked();
else
ret = 1;
goto bail;
}
segaddr = in_SB(in.addr);
selector = seg / 16;
setup_works = true;
for (i = 0; i < blocks; i++) {
seq = buf[i++];
bpf = bd->bd.next + i * search;
if (fd) {
current = blocked;
}
}
rw->name = "Getjbbregs";
bprm_self_clearl(&iv->version);
regs->new = blocks[(BPF_STATS << info->historidac)] |
PFMR_CLOBATHINC_SECONDS << 12;
return segtable;
}

11. Nextremer Co., LTD. Confidential Document

12. Nextremer Co., LTD. Confidential Document
Sequence to Sequence Learning with Neural Networks(Sutskever et al., 2014)
■
■ encoder decoder 2 RNN
■ encoder
■ decoder
12

13. Nextremer Co., LTD. Confidential Document
Google’s Neural Machine Translation System: Bridging the Gap between Human
and Machine Translation(Yonghui Wu et al., 2016)
■ Google
■
– 8 LSTM
■ attention(D Bahdanau et al., 2014)
– encoder
■
wordpieces(Mike Schuster & Kaisuke
Nakajima, 2012)
■ Melvin Johnson et al., 2016
13

14. Nextremer Co., LTD. Confidential Document
Neural Conversational Model(Oriol Vinyals & Quoc Le, 2015)
■
■ Sutskever et al., 2014
■
– IT help desk
troubleshooting
– open-domain movie
transcript
14

15. Nextremer Co., LTD. Confidential Document
■ Hochreiter, S., and Schmidhuber, J. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
■ Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim
Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz
Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant
Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff
Hughes, and Jeffrey Dean. Google’s neural machine translation system: Bridging the gap between human
and machine translation. arXiv preprint arxiv:1609.08144, 2016a.
■ D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate.
arXiv preprint arXiv:1409.0473, 2014.
■ Schuster, M., and Nakajima, K. Japanese and Korean voice search. 2012 IEEE International Conference on
Acoustics, Speech and Signal Processing (2012).
■ Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda
Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, Jeffrey Dean (2016), Google's Multilingual Neural
Machine Translation System: Enabling Zero-Shot Translation. arXiv preprint arxiv:1611.04558, 2016.
■ Oriol Vinyals and Quoc V. Le. A Neural Conversational Model. In International Conference on Machine
Learning: Deep Learning Workshop, 2015.
■ (2015). Vol.2 (pp. 6-17)
■ Graham Neubig.
■ Christopher Olah. Neural Networks, Manifolds, and Topology colah's blog
■ Christopher Olah. Understanding LSTM Networks colah's blog
■ Andrej Karpathy. The Unreasonable Effectiveness of Recurrent Neural Networks Andrej Karpathy
blog