A set of modular design patterns that can describe a large number of neuro-symbolic architectures from the literature. Corresponding paper is at https://arxiv.org/abs/2102.11965
Modular design patterns for systems that learn and reason: a boxology
1. Modular design patterns for
systems that learn and reason:
a boxology
Frank van Harmelen, Annette ten Teije (V1)
Vrije Universiteit Amsterdam
+ Michael van Bekkum, Maaike de Boer, André Meyer (V2)
TNO Netherlands
(https://arxiv.org/abs/2102.11965)
Creative Commons License
CC BY 3.0:
Allowed to copy, redistribute
remix & transform
But must attribute
1
2. Increasingly broad concensus in AI
(“the third wave”)
The next progress in AI will be driven by systems
that combine neural and symbolic techniques
Position papers by
Marcus, Lamb & Garcez, Darwiche, Pearl, Kautz, …
Keynotes at AAAI17, IJCAI18, IJCAI19, AAAI20,….
(“proof by authority” )
2
4. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
4
5. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
300.000 medical definitions
40 years of effort,
10.000 updates every years 5
“knowledge acquisition
bottle neck”
6. Strengths & Weaknesses
10M training images
Symbolic Connectionist
Construction Human effort Data hunger
Scalable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
4.8M training games
6
“sample inefficiency”
7. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
worse with
more data
worse with
less data
7
“sample inefficiency”
“combinatorial
explosion”
8. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
8
“black box problem”
9. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
quality
generality 9
10. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
10
Class: 793
Label: n04209133 (shower cap)
Certainty: 99.7%
“out of distribution generalisability”
11. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
11
13. So we started reading…
• 3 years of weekly reading group ≈ 75 papers
• 3x 8-week seminar with 15 students ≈100 papers
13
It was a mess…
Lot’s of techniques, tricks, ideas, methods, math
No structure, no guidance, no map, no theory
23. Symbols in, symbols out
• Inductive Logic Programming
• Probabilistic Soft Logic
• Markov Logic Networks
• ….
23
24. Intermezzo: Symbol or data?
24
“A classical machine learning system: ”
“What the <0.70, 1.17, 0.99, 1.07> is a Symbol?”
Istvan Berkeley, Minds & Machines, 2008.
1. a symbol must designate an object, a class or a relation in the world
(= the “interpretation” of the symbol)
2. symbols can be either atomic or complex,
(= composed of other symbols according to compositional rules
3. there must be system of operations that, when applied to a symbol,
generates new symbols, that again must have a designation.
cat
25. Symbolic prior (informed ML)
P(cushion|chair) >> P(flower|chair)
25
See survey of 100+ systems in Von Rueden et al, Learning, 2019
cushion
32. From symbols to data and back again
Knowledge Graph completion
Rolling
Stones
Angi Beat It
Michael
Jackson
Publish_song
32
Angi
Rolling
Stones
Publish_song
From:
Predict:
ML ML
36. Goal 1: Create some structure
in the huge number of proposals
for combining learning and reasoning
Goal 2: Create modular architectures
Contribution:
A set of re-usable architectural patterns
for modular systems that learn and reason
Next steps:
• Formalise informal diagrams as pre/post-conditions
• Implement informal diagrams as a code library
• Generate diagrams via a grammar
(and predict unexplored patterns)
36