1. The True
Arti
fi
cial Intelligence Revolution
Eric Postma
Cognitive Science & AI, Tilburg University
Data Analytics, Jheronimus Academy of Data Science, ’s-Hertogenbosch
XR COLOMBIA & NETHERLANDS 2020, November 19
7. Symbolic AI (1957-2000)
• Emphasis on formal logic and symbolic representations
• Neuroscience is irrelevant
• Focus on high-level cognition
(reasoning, problem-solving, …)
8. “Statistics is irrelevant to AI”
Deterministic inference rules operating on explicit knowledge representations
19. Alternative approach
1. Train the network (randomly initialise weights: Wr)
2. Remove super
fl
uous structure (i.e., small weights)
3. Fine-tune Retrain the network by initialising with Wr
This does work…..
20. Over-parameterisation
• After pruning, subnetworks can do the job
• These subnetworks are typically between 1%-15% of the original size
• Require considerably less training time
• Perform at or around the same level
This raises the question: Why is over-parameterisation useful?
23. Conclusions
• The huge number of connections incorporates the solution
• The main challenge is to
fi
nd it (needle in a haystack)
• In nature, this happens during early development