Artificial general intelligence research project at Keen Software House (3/2015)

1. Artificial Intelligence Research
at Keen Software House
Technical Report

2. Shortly about AGI
• Artificial General Intelligence
- Autonomous agent
- Able to perceive and change its environment
- Able to remember, reason and plan
- Adaptable and able to learn
- Able to communicate

3. What to use for AGI?
• Classical AI?
- Symbolic architectures
- Inference machines, expert systems
- Planners and solvers, STRIPS
• Artificial neural networks?
- Spiking Networks
- FFN, RNN
- DeepNets
• Multi agent systems?
• All of them!

4. Suitable tool for experiments
• Rapid model prototyping
- Integrate existing model
- Create (or recreate) new model
• Model insight
- Rich GUI & Visualization possibilities
- Model structure view (oriented graph?)
- Runtime view & execution control
• Heterogeneous architecture
- Connect different models together
- Able to use various hardware
• Parallel execution
- GPU based solution
- Cluster solution

5. Existing tools & inspiration
• Graph of connected modules
- ROS
- Matlab / Simulink
- Maya material editor
- Nengo (Eliasmith)
• Specialized libraries (modules)
- Caffe, OpenCV, cuBLAS, cuDNN,
- ROS modules

6. Our solution – Brain Simulator
• Model structure
- Nodes, tasks, memory blocks, worlds
• Model view
- Graph view (model structure)
- Observers (model data)
• simple, numeric, 3D, custom
• Experiments & debugging
- Model parameters exposed to GUI
- Adjustable observers
- Simulation control
• Parallel computing
- CUDA (Intel Phi support in progress)
- Multi GPU support

7. Brain Simulator – screenshots

8. Brain Simulator – modules
• Implemented modules
- Feed-forward nets (FFN, RNN, convolution nets, auto associators)
- Self-organizing networks (SOM, GNG, K-means …)
- Vector symbolic architectures (HRR, BSC)
- Hierarchical temporal memory (spatial & temporal poolers)
- Spiking networks & STDP
- Computer vision (filters, segmentation, tracking, optical flow)
- Hopfield network, SVD, SLAM, PID, Differential evolution and many
others
• Imported modules
- Caffe, BLAS, BEPU Physics, Space Engineers, Gameboy emulator
• Planned modules
- Deep learning & RBMs, Hierarchical Q-Learning

9. BS Screenshots – SOM

10. Development methodology
• Iterative/agile approach
- Early implementation and experiments
- Separated experiments with mockup parts
- Milestone oriented (global model iterations)
• Separated experiments (proofs of concept)
- Data representation, memory models, temporal data encoding
- Learning strategies, goal inference, action selection
- Spatial awareness, visual working memory, navigation
- Computer vision
• Milestone examples
- 6-legged robot agent (integration test)
- Breakout/Pong game (reinforcement learning & vision test)
- Autonomous agent game (PacMan, Nethack)

11. Example 1 – walking robot
• Physical world emulation
- Connected to Space Engineers game
- 6-legged robot body
- Runtime visual data processing & body control
• Learning from mentor
- Hardwired movements
- Learning body state associated with high level
movement commands
- Simple vision to action associations
- Totally supervised system

12. Video of 6-legged robot

13. Example 2 – Pong / Breakout
• Pong / Breakout game
- From bitmap to buttons
- Reinforced learning (reward and punishment)
- Image processing towards object tracking
- Vector symbolic architecture
- Goal states extraction
- Action learning & action selection
• Existing solutions
- Not Q-learning (DeepMind and others before them)
- Modular, engineered system
- Better insight (faster learning?), sacrificed flexibility

14. Pong / Breakout model

15. Visual Processing

16. Pong / Breakout model

17. Pong / Breakout BS inspection

18. Future work
• Next milestone – 2D egocentric game
- Advanced visual working memory
- Navigation & inner spatial representation of environment
- Environment variables extraction, hierarchical Q-learning
- Multiple goals and motivations, goal chaining
- Motoric systems (bipedal balancing)
• Future milestones
- Same model playing different games
- Same model instance playing different games
- Motoric systems (command sequences unrolling & execution)
• Computing platform improvements
- Brain Simulator release (with remote module repository)
- HPC solution
- Unix systems release

19. The end
• You can invest in AI companies
• Every $1 invested today will return 1,000,000 times
• Join our team – we are always hiring
• AI Programmers / Researchers
• SW Engineers / Architects
• PR Manager / Evangelist
• Follow us:
• http://blog.marekrosa.org/
• http:// www.keenswh.com/
Thank you.
Questions?