1.
Keisuke Fukuda
Preferred Networks, Inc.
PFNにおける研究開発
深層学習からMN-3開発，そして社員の働き方
2022/10/19 融合情報学特別講義Ⅲ

2.
自己紹介
2
● 福田圭祐 Keisuke Fukuda
○ 東京工業大学(Tokyo Tech)
○ Interests：
■ High Performance Computing（HPC）
● Perform large-scale parallel & distributed computing on supercomputers
■ Joined PFN in Apr. 2017
■ Distributed / Paralell Deep learning, performance optimization

3.
Introduction to
Preferred Networks
Making the real world computable

4.
Our Vision
4
We make cars, robots, and other devices more intelligent by fusing software and hardware in a sophisticated
manner. By making devices intelligent enough to adapt to continuously changing environments and conditions, our
world becomes computable through real-time sensing of the physical world.
We do not compete in familiar territory, but rather take on ambitious technological challenges. By leveraging the
latest technologies, we want to advance the frontiers of knowledge and discover the world of the future.
Making the real world computable.
With our innovative and essential technologies,
we venture into the unknown.

5.
Company information
5
Manufacturing Logistics
Transportation Bio & Healthcare
Personal Robot Entertainment
Founded March 2014
Directors
CEO Toru Nishikawa
CER Daisuke Okanohara
CTO Ryosuke Okuta
Located
Tokyo, Japan (HQ) ​
Burlingame, CA., US
(Preferred Networks America, Inc.)​
Number of
Employees
270+ Engineers & Researchers
(October, 2020)​

6.
2021, 2020
● No.1 on Green500 list of the world’s
most energy-efficient supercomputers
2019
● Prime Minister’s Award, Nippon Venture Awards
2018
● Grand Prize37th NIKKEI Product and Service Excellence Award
● Open Source Data Science Project Award, ODSC East 2018
2017
● Japan-U.S. Innovation Awards「Emerging Leader Award」
● FT ArcelorMittal Boldness in Business Awards
● METI Minister’s Award, Nippon Venture Award
2016
● 1st Annual JEITA Venture Awards
● Forbes JAPAN’s CEO OF THE YEAR 2016
● 「1st place - Most innovative startup」
Awards
6

7.
We develop practical
applications of cutting-edge
technologies
Preferred Networks (PFN) develops practical
applications of deep learning and other cutting-
edge technologies in order to solve real-world
problems that are difficult to address with existing
technologies.
Our Focus
7

8.
Our Capabilities
8
Deep Learning
World class researchers
focusing on deep learning
Expertise
Wide range of deep expertise from
robotics to genomics to
computational chemistry
World class computational
resources designed for deep
learning application
PrivateSuper
Computer
Software
In-house developments of OSS and
hyperparameter tuning library to
accelerate software development

9.
● PFN collaborates with world-leading corporations and organizations to drive innovation in a wide range of
fields. We aim to build long-term relationships with our partners to create new innovations that lead to
creation of new businesses
Our Business
Cutting-edge technology
x
Computational resources
Business challenges
x
High quality data
Creation of
new businesses
Software Applications
x
Intellectual Property
R&D
projects
Profit
sharing
Partnering
Company

10.
Our Values
10
Preferred Networks is a young, yet
rapidly growing company
Our Values are what make us different
As PFN members, we question:
what should we do and not do?
who are we and what do we consider important?
To answer these questions, we came up with the
four statements as our code of conduct, or PFN Values

11.
● Employees: 300+ (270+ engineers & researchers)
● Top Management + Corporate Officers = 12
● Each team consists of an Engineering Manager + members
○ Many members belong to multiple teams concurrently
○ Slack-based communications, most channels are open
● Working style under COVID-19 era
○ WFH (Work-from-home) by default
○ Slack / Zoom / Google Meet / Jamboard / Mural
● We are exploring a new workign style for post COVID-19 era
How we work in PFN
11
Teams (EM + 3-10 ppl. each)
Corporate Officers (9)
Top Management (3)

12.
Our developments to date

13.
Industry automation powered by deep learning technologies
13
Autonomous learning for bin-picking robot.
The robot gathers data by trial and error and
learns the place where it is likely to pick the
piece up by using deep learning (as of
December 2015).
https://youtu.be/ydh_AdWZflA

14.
@ICRA 2017 voice Recognition + object picking
14
“Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions”
arXiv:1710.06280
• ICRA is a top-tier conference on robotics
• Best Paper Award on Human-Robot Interaction
• Technologies:
• Visual recognition
• Natural language processing (NLP)
• The robot can understand ambiguous words:
• ”The Teddy bear”
• ”The brown fluffy stuff”
https://youtu.be/_Uyv1XIUqhk

15.
Factory and plant operation control using deep learning
15
PFN is working with ENEOS (formerly JXTG
Nippon Oil & Energy Corporation) on a joint
research project regarding optimization and
automation of oil refineries.
An oil refinery is a very complex system consisting
of hundreds of processes and thousands of sensors
and actuators.
Because of its massive production scale of
petrochemical products, an improvement of a
fraction of a percent of productivity delivers
significant cost reductions.
By leveraging PFN’s deep learning technology, the
joint venture aims to automatically control and
optimize large and complex plant equipment for
more efficient use of energy resources.
ENEOS’ Kawasaki Refinery. ML-based control model can be used to keep plant machine operation
stable against unknown external disturbance
DL-based Digital Twins for advanced automation and optimization

16.
16
https://matlantis.com

17.
17
https://petalica-paint.pixiv.dev/index_ja.html
ロボット系エンジニアが、サイドプロジェクトとして開始→正式プロジェクトへ

18.
Crypko™: High-quality Anime Character Generation and Design
18
Deep learning can
revolutionize the
entertainment industry
PFN’s technology Crypko uses state-
of-the-art generative models, a branch
of techniques in deep learning, to
generate a potentially infinite set of
unique, high quality characters not
contained in the training data.
Furthermore, it can fuse several
characters into new characters,
inheriting their distinctive features
Crypko’s character fusion. From the two characters on the top row, Crypko can generate characters on the bottom
row that inherit distinctive features of the input characters For more information please visit our entertainment page:
https://preferred.jp/en/projects/entertainment/

19.
Playgram™ / Playgram typing™: Programming education for kids
19
Virtual, high-quality learning experience in Computer Science
PFN has developed Playgram™, a programming education app primarily targeting students in elementary
school and above. PFN has teamed up with Yaruki Switch Group (YSG), Japan’s leading education group with a
diverse range of programs and over 1,700 schools, to build a programming course package using Playgram.
Beginning August 2020, YSG will first pilot the package in three schools in the Tokyo area, both in classrooms and online.
Developed by PFN’s software engineers at the forefront of artificial intelligence technologies, Playgram incorporates the K-12 Computer
Science Framework, a U.S. guideline for computer science education. The app will be available in Japanese at launch
For more infromation, please visit our Playgram website: https://playgram.jp/
Bridges the gap
between visual and
text-based coding
Rich 3D interface that
inspires creativity
Adaptive learning
system and user-
friendly tutorials

20.
Optuna™: Automation for Hyper-parameter Tuning
20
Optimize Your Optimization
An open source hyperparameter optimization framework to automate hyperparameter search
In deep learning, it is essential to tune hyperparameters since they control how an algorithm behaves.
The precision of a model largely depends on tuning a large number of hyperparameters, including training iterations, neural
network layers and channels, learning rate, batch size, and so on. Optuna, an open source technology developed at PFN,
automates this trial-and-error process of optimizing the hyperparameters. It automatically finds optimal hyperparameter
values that enable the algorithm to give excellent performance. Optuna can be used for any black-box optimization problems.

21.
Consistent contributions to Research
21
Examples of recent PFN publications in top-tier conferences (in 2022)
● [J. Chem. Inf. Model.2022] “Molecular Design Method Using a Reversible Tree Representation of Chemical
Compounds and Deep Reinforcement Learning”
● [BMVC2022] “Multi-View Neural Surface Reconstruction with Structured Light”
● [NeurIPS 2022] “Unsupervised Learning of Equivariant Structure from Sequences”
● [NeurIPS 2022] “Decomposing NeRF for Editing via Feature Field Distillation”
● [Lung Cancer 2022] “Machine Learning-based Exceptional Response Prediction of Nivolumab Monotherapy
with Circulating MicroRNAs in Non-Small Cell Lung Cancer”
● [ICAIF 2022] “Theoretically Motivated Data Augmentation and Regularization for Portfolio Construction”
● [ICAIF 2022] “Efficient Learning of Nested Deep Hedging using Multiple Options”
● [ROMAN 2022] “F3 Hand: A Versatile Robot Hand Inspired by Human Thumb and Index Fingers”
● [Nature Communications 2022] “Towards universal neural network potential for material discovery applicable
to arbitrary combination of 45 elements”
● [Physical Review Research 2022] “Power Laws and Symmetries in a Minimal Model of Financial Market
Economy”

22.
Our research & platform

23.
Total of 2,560 GPUs
Total 200 PFLOPS
Listed No.1 in Japan amongst private entity
1 PETA FLOPS =
1,000 trillion
Floating-point Operations
Per Second
Our Infrastructure
23
MN-Core MN-Core Board x 4
CPU Intel Xeon 8260M 2way （48 physical cores）
Memory 384GB DDR4
Storage Class Memory 3TB Intel Optane DC Persistent Memory
Network
MN-Core DirectConnect（112Gbps） x 2
Mellanox ConnectX-6（100GbE） x 2
On board（10GbE） x 2
MN-3 specs
Deep learning processor MN-Core
Supercomputer designed for deep learning application
MN-1 MN-2 MN-3
For more information please visit: https://projects.preferred.jp/supercomputers/en/

24.
The MN series: PFN’s in-house supercomputers
24
 MN-1a (Sep. ’17〜)
━ 1024 NVIDIA Tesla P100 + IB FDR
━ Peak 19.1 Peta FLOPS (SP)
━ #227 in Top500 Nov. 2018
 MN-1b (July. ’18〜)
━ 512 NVIDIA Tesla V100 + IB EDR
━ Peak 57.3 Peta (tensor) Flops
 MN-2b (July. ’19〜)
━ 1024 NVIDIA Tesla V100 + IB EDR
━ 128 Peta (Tensor) Flops
 MN-3 (Nov. 20〜)
━ We’ll later!

25.
0
10
20
30
40
50
60
70
Time
[min]
Training time of ResNet-50 (90 epochs) on ImageNet
Achievement on MN-1a: ImageNet in 15 minutes
25
2018 July
2018 Nov
2017 Nov
arXiv: 1711.04325
Extremely Large Minibatch SGD: Training ResNet-50
on ImageNet in 15 Minutes
2018 Nov

26.
Achievement on MN-1b: PFDet in OIC 2019
26

27.
● Google AI Open Images - Object Detection Track
○ Competition using Largest-class image dataset
○ 12 million bounding boxes, 1.7 million images
○ 454 competitiors
○ Approx. 500GB (annotated subset)
● Object detection: much harder than object recognition task
Achievement on MN-1b: PFDet in OIC 2018
27

28.
https://tech.nikkeibp.co.jp/atcl/nxt/column/18/01006/101000005/
28

29.
Simulation ✕ AI
29

30.
○ c.f. 「演繹から帰納へ〜新しいシステム開発パラダイム〜」丸山宏, PPL2018 招待講演
○ 特別なものではなく、実装手法の1つとして広く使われるようになっていくのでは？
AIはコンピューターサイエンスのコア技術になっていく
AIが向いている場面 AIが不向きな場面
• データが大量（or 生成可能）
• 誤差が許容される
• 現象が複雑／原理が不明
• シミュレーションが困難／計算量多い
• 法則・原理が一定
• 予測が目的
• データが少ない
• 厳密さが必要
• 演繹的プログラミングが可能
• シミュレーションが容易／手法が確立
• 過去から未来が予測できない
• メカニズムの理解が目的
⇒困難なタスクは
計算パワーで解く
⇒計算パワーが無いと
戦えない
Conventional Programming
従来のプログラミング
演繹的プログラミング
(Deductive programming)
Machine Learning
機械学習
帰納的プログラミング
(Reductive programming)

31.
Simulationとは：
● 現実世界の物理法則を数式でモデル化し、計算機上で計算によって再現・予測
する
● 流体、天体、気象、機械設計、材料化学、・・・
Simulationの課題
● 複雑すぎる現象・Multiphysics（ex. 構造連成計算、気象）
● 計算量の爆発
Simulation
31

32.
● これまで深層学習の実用化はデータが容易に入手可能な分野（ウェブ、バーチ
ャル）に限られていた。
● 今後、実世界の問題に深層学習を導入していくためにはシミュレーション利用
が不可欠である
● データが21世紀の石油と言われる中で、そのデータ自身を作れるシミュレーシ
ョンを揃えていくことが重要となる
● またシミュレーション自体も深層学習を利用することで劇的に高速化、多様化
を達成できる
今後シミュレーションが重要となる
32

33.
SimulationとAI は相性が良い
33
Simulationの中でも難しいとされているものに対して、AIを 組み合わせて互いに補い合う
AIが向いている場面 AIが不向きな場面
• データが大量（or 生成可能）
• 誤差が許容される
• 現象が複雑／原理が不明
• シミュレーションが困難／計算量多い
• 法則・原理が一定
• 予測が目的
• データが少ない
• 厳密さが必要
• 演繹的プログラミングが可能
• シミュレーションが容易／手法が確立
• 過去から未来が予測できない
• メカニズムの理解が目的
Simulationが向いている場面 Simulationが不向きな場面
• 少ない物理法則から、モデル化可能
• 保存則などを厳密に維持可能
• メカニズムの理解・予測の両方
• 現象が複雑・原理が不明なものは難
• 計算量が爆発する
Simulationが深層学習を助ける
• 網羅的なデータを入手可能
• ラベルを作るのが難しい場合もラベル
付が可能
• 最適化、強化学習に必要なWhat-If分析
が可能
深層学習がSimulationを助ける
• シミュレーションの高速化
• データからシミュレーションを学習す
る
• データ同化、パラメータ推定を助ける

34.
34
https://matlantis.com
AI x Simulationの事例（１）

35.
● 2022年度夏季インターンシップの成果（東京大学・助田さん）
● 気象シミュレーションは、シミュレーションの中でも特に難しい分野
○ 観測データが少ない（観測機器の制約）
○ 計算量が多い
○ 現象が複雑
● このテーマでは、「計算量が多い」という課題に着目して、スパコンで実行されるシミュレーターを
省メモリで模倣計算することにチャレンジ
数値シミュレーションデータの低次元潜在空間における時間発展ダイナミクスの学習
35
AI x Simulationの事例（２）
Preferred Networks Tech Blog “数値シミュレーションデータの低次元潜在空間における時間発展ダイナミクスの学習”

36.
このあと続く
Deep Learningのための専用プロセッサ「MN-Core」の開発と活用
金子 紘也 Hiroya Kaneko
● PFNにとっての計算能力の位置付け
● 代表的なDeep Learningの高速化手法
● なぜ今プロセッサ開発なのか？
● MN-Coreの概要
● 開発チームの働き方
● 最近の成果