ICLR 2020: Yoshua Bengio and the Nature of Consciousness
In this episode of Machine Learning Street Talk, Tim Scarfe, Connor Shorten and Yannic Kilcher react to Yoshua Bengio’s ICLR 2020 Keynote “Deep Learning Priors Associated with Conscious Processing”. Bengio takes on many future directions for research in Deep Learning such as the role of attention in consciousness, sparse factor graphs and causality, and the study of systematic generalization. Bengio also presents big ideas in Intelligence that border on the line of philosophy and practical machine learning. This includes ideas such as consciousness in machines and System 1 and System 2 thinking, as described in Daniel Kahneman’s book “Thinking Fast and Slow”. Similar to Yann LeCun’s half of the 2020 ICLR keynote, this talk takes on many challenging ideas and hopefully this video helps you get a better understanding of some of them! Thanks for watching! Please Subscribe for more videos! Paper Links: Link to Talk: https://iclr.cc/virtual_2020/speaker_7.html The Consciousness Prior: https://arxiv.org/abs/1709.08568 Thinking Fast and Slow: https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555 Systematic Generalization: https://arxiv.org/abs/1811.12889 CLOSURE: Assessing Systematic Generalization of CLEVR Models: https://arxiv.org/abs/1912.05783 Neural Module Networks: https://arxiv.org/abs/1511.02799 Experience Grounds Language: https://arxiv.org/pdf/2004.10151.pdf Benchmarking Graph Neural Networks: https://arxiv.org/pdf/2003.00982.pdf On the Measure of Intelligence: https://arxiv.org/abs/1911.01547 Please check out our individual channels as well! Machine Learning Dojo with Tim Scarfe: https://www.youtube.com/channel/UCXvHuBMbgJw67i5vrMBBobA Yannic Kilcher: https://www.youtube.com/channel/UCZHmQk67mSJgfCCTn7xBfe Henry AI Labs: https://www.youtube.com/channel/UCHB9VepY6kYvZjj0Bgxnpbw 00:00:00 Tim and Yannics takes 00:01:37 Intro to Bengio 00:03:13 System 2, language and Chomsky 00:05:58 Cristof Koch on conciousness 00:07:25 Francois Chollet on intelligence and consciousness 00:09:29 Meditation and Sam Harris on consciousness 00:11:35 Connor Intro 00:13:20 Show Main Intro 00:17:55 Priors associated with Conscious Processing 00:26:25 System 1 / System 2 00:42:47 Implicit and Verbalized Knowledge [DONT MISS THIS!] 01:08:24 Inductive Priors for DL 2.0 01:27:20 Systematic Generalization 01:37:53 Contrast with the Symbolic AI Program 01:54:55 Attention 02:00:25 From Attention to Consciousness 02:05:31 Thoughts, Consciousness, Language 02:06:55 Sparse Factor graph 02:10:52 Sparse Change in Abstract Latent Space 02:15:10 Discovering Cause and Effect 02:20:00 Factorize the joint distribution 02:22:30 RIMS: Modular Computation 02:24:30 Conclusion #machinelearning #deeplearning