This document discusses using the AnyLogic simulation software together with reinforcement learning. It provides an example of using AnyLogic to model traffic light control and training a reinforcement learning agent using RL4J to optimize the traffic light policy. Key points covered include:
- AnyLogic allows building simulation models that can integrate with reinforcement learning libraries like RL4J.
- A traffic light control simulation was built in AnyLogic and used as the reinforcement learning environment to train an agent's policy for optimizing traffic flow.
- The trained policy was able to improve traffic flow compared to the base simulation and fixed optimization approaches.
- Learned policies from simulation and reinforcement learning can be deployed in real systems to enable adaptive autonomous decision making.
3. Currently VP. Engineering @ Skymind
• Leading RL Applications
• Previously:
• Assistant Manager @ JBS
• Intern Researcher @ Panasonic
Eduardo Gonzalez
| WHO AM I
3
@wm_eddie
https://qiita.com/wmeddie
https://wm-eddie.info
4. ● Builds AI infrastructure for operating models in
production
● Allows model access from cloud, server,
desktop, and mobile
● Providing tooling for models such as revision
history and accuracy monitoring over time
● Created the widely used open-source AI
framework Deeplearning4j, powering AI for
large enterprises globally, from banking to
telecom
PRODUCTS
SKIL:
ML and DL
Model Server
| ABOUT SKYMIND
4
5. Skymind’s team has contributed millions of lines of code to Open Source
| OPEN SOURCE CONTRIBUTORS
5
6. Deep Learning, A Practitioner’s Approach
● Written by Adam Gibson (CTO) and Josh Patterson (Contributor)
● Published in 2017
● Good fundamentals for deep learning and the DL4J framework
● Many Graphics come from the book
| BOOK
6
7. Deep Learning and the Game of Go
● Written by Max Pumperla, Deep Learning Engineer @ Skymind
● Published in 2019
● Shows how to go from 0 to an entire AlphaZero style Go bot
● Introduces Deep Learning and Reinforcement Learning from
scratch.
| BOOK
7
9. AnyLogic is a multi-modal simulation modeling
software that is capable of doing system
dynamics, agent-based and discrete event based
simulations.
It is a de facto standard in the industry and is
used by almost all of the Fortune 500.
| ANYLOGIC
AnyLogic models can be exported into a Java
application and deployed to customers.
10. AnyLogic models are extended with Java so you can create custom agents or experiments.
Exported applications are Java libraries and can be integrated into and leverage data from Enterprise
applications and Excel.
| ANYLOGIC DETAILS
11. DL4J includes RL4J, a reinforcement library for Java. It can be used
inside AnyLogic without friction.
Reinforcement Learning was a main theme of the AnyLogic ’19
Conference. Skymind collaborated closely with AnyLogic for workshops
and panel discussions.
| WHY ANYLOGIC + SKYMIND
19. • Lots of NP-Hard problems exist in Simulation
• Current Optimization techniques are not able to do anything
• A good enough solution is better than no solution
• And better than hand written heuristics
| WHY REINFORCEMENT LEARNING
