This is a combined slide/speaker video of Yoshua Bengio's talk at NeurIPS 2019. Slide-synced non-YouTube version is here: https://slideslive.com/neurips/neurips-2019-west-exhibition-hall-c-b3-live

This is a clip on the Lex Clips channel that I mostly use to post video clips from the Artificial Intelligence podcast, but occasionally I post clips from other lectures by me or others. Hope you find these interesting, thought-provoking, and inspiring. If you do, please subscribe, click bell icon, and share.

Lex Clips channel: 
https://www.youtube.com/lexclips

Lex Fridman channel:
https://www.youtube.com/lexfridman

Artificial Intelligence podcast website:
https://lexfridman.com/ai

Apple Podcasts:
https://apple.co/2lwqZIr

Spotify:
https://spoti.fi/2nEwCF8

RSS:
https://lexfridman.com/category/ai/feed/

Connect with on social media:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman

Thank you so much for inviting me for this talk, and thank you so much for being so numerous today.

But what's interesting in my mind is that these things are linked together in a really interesting way.

At least that's what I'll try to convince you about.

So the title talks about system 2 cognition, which is about high level cognition.

And that's connected to consciousness, as I'll explain.

It's also connected to issues that are very important today in machine learning about generalization out of distribution and it's also connected to the notion of agency and reinforcement learning and how agents face problems that standard static machine learning hasn't been looking at enough already.

So deep learning has made amazing progress in the last few years and decades.

And some people think that it might be enough to take what we have and just grow the size of the data sets, the model sizes, computer speed, just get a bigger brain.

I think that we are missing out in qualitative ways in order to approach human-level AI.

We have currently machines that learn in a very narrow way.

They need much more data to learn a new task than our human examples of intelligence.

They need humans to provide high-level concepts through labels.

And they still make really stupid mistakes.

They're not very robust to changes in distribution.

There are adversarial examples and things like this.

And so some people, contrary to the first ones that I mentioned, think that we have to start afresh and invent something completely new to face these challenges, and maybe go back to classical AI to deal with things like high-level cognition.

What I'll try to tell you about today is that I think there is a path to go from where we are now, expanding the abilities of deep learning to approach these kinds of high-level cognition questions of system 2.

So I talked about these 2 things, system 1 and system 2.

I really got introduced to these concepts by reading the amazing book of Daniel Kahneman, which I suggest you to look at, Thinking Fast and Slow.

And he talks about these 2 kinds of cognitive tasks.

System 1 are the kinds of things that we do intuitively, unconsciously, that we can't explain verbally, and that we can, in the case of behavior, that are habitual.

This is what current deep learning is good at.

So if you're driving a car and going back home to a place you know, you don't need to pay a lot of attention to the road, you can talk to the person near you.

On the other hand, if you're in a new city and you don't know the way, maybe somebody told you directions, but you have to really pay attention to every corner.

You have to read signs, you have to be on the lookout, and if somebody tries talking to you, you ask them to, you know, please let me drive.

So what's going on there is that you're generalizing in a more powerful way.

And you're doing it in a conscious way that potentially you could explain.

So if I ask you to add 34 and 56, you can do it in your head.

But it's going to be sequential and slow compared to a calculator.

The kinds of things that we do with system 2 include programming.

We can plan, we can reason, we can use logic.

Usually these things are very slow if you compare to what for some of these problems computers could do.

And these are the things that I'm going to argue that we want future deep learning to do as well.

So here are some things that I think we are missing in order to approach human level AI with deep learning.

We need to work better on out of distribution generalization and transfer, so when you learn a new task, you wanna be able to do it with very little data.

We want to be able to handle these high-level cognitive tasks that I mentioned, including using these tasks to force the learner to discover the kind of high level representations which deep learning was supposed to be about in the first place.

Deep learning was about learning multiple levels of representation, with the highest level supposedly corresponding to the kind of concepts we manipulate with language.

And many of these concepts have to do with causality.

The kinds of concepts we manipulate with words tend to be cause or effect.

And once you start talking about causality, you have to also talk about the agent perspective, something that the mainstream of machine learning hasn't paid enough attention to.

Not just because we wanna solve reinforcement learning problems, but because we wanna have machines that understand the world, that build good world models, that understand cause and effect, and that can act in the world in order to acquire knowledge.

So a big theme of my presentation today is that these different questions are not independent research directions.

When you make progress in 1, you can make progress in the other, and understanding how they're linked can help us plan a path for our research.

So, let me talk a little bit about consciousness and the kind of functionalities that I think we need to add to machine learning.

I'm going to start by talking about the need for having machine learning systems that can handle changes in distribution and why it's particularly important for agents because of non-stationarities in their environment.

Then I'm going to talk about the building blocks that I think can help us get there.

In the last few years, maybe unnoticed, we have added things to the deep learning toolbox in particular attention mechanisms which I think are actually the key to move to the next stage that I'm talking about.

And in this part of the talk I'll also tell you a little bit about how much work has been done by our colleagues in cognitive neuroscience to better understand the human side of the equation.

Then I'll talk about several priors that may be linked to consciousness.

So the main theme here is that there's an advantage for human beings to have these high level cognitive abilities.

And so these advantages, you can think of them as assumptions about the world.

So usually we talk about priors, or you can think of them as regularization.

So the first 1 that I'll talk about is the idea that the joint distribution between the high-level concepts can be represented by a sparse factor graph.

Any joint distribution can be represented by a factor graph, but the sparse 1 is a different story.

And then I'll talk about another prior that has to do with how the world changes.

And this is gonna connect with the notion of agency.

So the fact that most of the time when things change in the world, it's because agents, like people, do something.

So we intervene, we do things, And the hypothesis that I'm proposing with other people is to consider those changes as localized in some abstract space.

And if we use these hypotheses I'm going to show through some of the recent work we've done, you can actually discover relationships that are just causal relationships between variables.

Yeah, I'm going to also talk about meta-learning, which is connected to the problem of learning out of distribution.

And the final bit of my presentation is going to be more about the detailed architectural structures that we should explore to introduce the kind of compositionality that system 2 processing requires.

And in particular, I think we should be moving towards neural nets that can not just operate on vectors, but also operate on sets of elements, sets of vectors, sets of objects that are pointable, that can be referred and operated on by dynamically recombined modules.

Now, I'm going to go a little bit more detail on each of these points.

Let's start with changes in distribution.

So the classical framework for machine learning is based on the hypothesis of IID, independently and identically distributed data.

That means the test data has the same distribution as the training data.

And it's very important because if we didn't have that assumption, then we would not be able to say anything about generalization.

Why would a function learned on some data work on some other data, right?

Now, unfortunately, this assumption is too strong.

And so in practice, what many people do in industry or in academia is they take, when they collect data, they shuffle it.

And here, to make it i.i.d., and here I want to quote my friend Leon Boutou who in his ICML keynote this year said something like, nature does not shuffle data, we shouldn't.

And the reason why we shouldn't is because when we do that, we destroy important information about those changes in distribution that are inherent in the data we collect.

And instead of destroying that information, we should use it in order to learn how the world changes.

So this is important because there are, for example, rare events like black swan events that are highly unlikely to happen but could have severe consequences.

And this question of out-of-distribution generalization really breaks the IID hypothesis.

But If we discard it, we need to replace it by something else.

That's why I've been talking about priors.

1 of the important ones is going to be, how is the test distribution going to be related to our training distribution if it's not the same?

So let's talk about out-of-distribution generalization.

What I mean by out-of-distribution generalization is essentially the phenomenon of a learner being able to generalize in some way to a different distribution.

It doesn't say anything about how we do it.

Well, if you are a learning agent, so agent means actions, right?

So it's a learning system that's embedded in some environment.

You are almost always facing non-stationarities for several reasons, right?

There are changes in distribution due to the actions of the agents.

The actions of other agents, the fact that I'm moving, that We're looking at different times, different sensory signals, different sensors, different actuators, different goals, different policies, all kinds of things are changing.

And as you heard from Blaise's talk this morning, once we start looking at multi-agent systems, it gets even more complicated.

You can't even talk about optimization, as he was explaining.

But certainly, from the point of view of each agent, you don't have a stationary distribution anymore.

And if you think about a child learning, for example, their world is changing all the time, their body is changing all the time.

And so we need systems that are gonna be able to handle those changes and do things like continual learning, lifelong learning, and so on.

This has been a longstanding goal for machine learning, but I think we haven't yet built the solutions to this.

And 1 of the crucial elements, according to me, in order to be successful in this, and I'll come back at the end of this presentation much more on this, is introducing more forms of compositionality.

It means being able to learn from some finite set of combinations about a much larger set of combinations.

We already get that from distributed representations.

Distributed representations are really at the heart of why neural nets are working.

They were introduced by Geoff Hinton in the early 80s.

And in the last few years, we actually have theory that helps us see why you get an exponential advantage, potentially at least, if you make the right assumptions, if we're making the right assumptions about the world in terms of compositionality, in terms of being explained by a number of different variables and factors, then distributed representations can be exponentially advantageous.

Because essentially, Once you've trained a bunch of features, you can generalize to new combinations of those features.

This is what a single hidden layer already gives you.

Now if you have a deeper network, you also get compositionality because each layer gets composed with the next 1.

And we've also shown that gives another exponential advantage.

Now I think there are other forms of compositionality.

And the 1 that we know best is the 1 you find in language.

And linguists have been talking about this for a long time.

They call it systematicity or systematic generalization.

So this opens the door to better powers of analogy, of abstract reasoning.

And that remains to be done in machine learning, I think.

So systematic generalization really is aiming at out-of-distribution generalization and fast transfer, but it's about how we do it.

It's about the idea that we can get that by dynamically recombining existing concepts.

So it could be language, but it could be in other settings.

Like in the picture here from Lake et al.

2015, we invent a new type of vehicle by combining properties of different vehicles.

But what's interesting about this is that dynamic systematicalization allows potentially to generalize to combinations that have 0 probability under the training distribution.

It's not just that they're not present in the training data, they would have 0 probability under the training distribution.

So as an example, if I tell you a science fiction scenario, clearly you haven't lived something like this in your life, but you can still imagine it.

If you've been living all your life in some continent or some city and then you move to a completely different place and you have to drive in this unknown city, as the example I gave earlier, you are also doing a form of systematic generalization.

Unfortunately, current neural net architectures are not that good at doing that.

And it's been shown through several papers and experiments, starting in particular with the work of Lake and Baroni.

More recently, the work we did at Mila, led by Badano and collaborators, presented by EICLR.

Currently, we have a paper that's going to be on archive probably tomorrow or the day after on CLEVR, which is a data set for visual question answering, where there are combinations of the linguistic concepts that are present in the questions that just don't come up in the way that the data is generated.

And the current methods, when you ask them to answer these kinds of questions fail completely, whereas the human wouldn't even realize that these combinations are not present in the data.

OK, and then 1 question people may come up with is how what you're proposing is different from the classical AI program of symbolic logic and so on?

Well, I think there are a number of reasons why these classical AI programs had trouble.

And in the work that we need to do in order to achieve system 2 performance.

We want to make sure that those systems will be able to generalize efficiently and in large scale.

The concepts that we want to learn need to be grounded with system 1, in low-level perception and action.

We want to keep the power of generalization of the distributed representations.

We want to make sure that the kind of search that is involved in things like reasoning and planning can be done efficiently, whereas the classical approaches require a huge number of explorations of many trajectories of how things could unfold or how you could combine concepts, rules, and so on.

And finally, we need to make sure we build systems that can properly handle uncertainty in the world, And machine learning has been doing a pretty good job at that up to now.

But we want to achieve these extra goals that we're not doing very well at, like systematic generalization that I explained, factorizing the knowledge, or at least some of it, that the learner is acquiring in small, exchangeable pieces to get this combinatorial advantage that I've been talking about.

And that includes being able to manipulate variables, things that you do naturally in programming and in logic formulations, dealing with instances that are associated with more general categories, if you want, and having references and indirection, things that don't seem natural in the neural net world.

But as I'll try to convince you, we now have actually built the tools for doing that in deep learning using attention mechanisms.

Okay, so let's talk about attention and consciousness.

Attention is about doing computation in a focused way.

We're going to sequentially focus computation on 1 or a few elements at a time.

And we realized in around 2014 that this was extremely powerful and was the reason why we were able to get a breakthrough in machine translation.

When you produce the next word in English when you're trying to translate from French say, you want to really focus the computation on just the right few words in the French sentence that are relevant to do the translation.

So we introduced a particular form of attention called content-based soft attention, which is very convenient because you can backprop through it and so you can learn it.

In other words, we can learn where to attend.

And so the way it works is the computation being done at the next level is going to use as input a selection from the previous level of computation.

And that selection is going to be a soft selection.

So we can take a convex weighted combination of value vectors from the previous level.

And these convex weights are coming from through a softmax that is conditioned on each of the elements.

So for each of the elements, we're going to see how well they match the context to see on which 1 the attention should be focused.

We're considering all the possible elements in some set, and we're computing a score for each of them in order to decide on which 1 or which ones we're going to put attention.

And there's been recent work in cognitive neuroscience showing that attention should be thought of as an internal action, right?

So the way that your brain is attending is very similar to the way that your motor system is deciding to move your arm.

And so we want to learn these attention policies.

I mentioned machine translation, but essentially, today's NLP systems, state-of-the-art systems, all rely on attention.

Look at all the work based on transformers and their variations.

They're also at the heart of memory-extended neural nets.

We had a paper last year and more ongoing work on how attention connected to memory can also unlock the problem of vanishing gradients.

And as I'll mention, attention also allows to change neural nets from machines that are processing vectors to machines that are processing sets, and in particular, sets of key value pairs.

We can think of attention as creating a dynamic connection between 2 layers.

Whereas in a traditional setting, the connections are fixed.

Here, we kind of pick which of the inputs is going to be sent to whatever module we're considering that uses an attention mechanism.

But from the point of view of the receiving module, there is a problem.

It gets this value, which is 1 of those in the set of input elements, but it doesn't know where it's coming from.

And so what we're doing with attention mechanisms is, in addition to the value, we have some concept of key.

In other words, a kind of identifier for where the value is coming from.

Currently, we're using those keys to decide which element should get the attention.

But that key is also sent to the next level.

And so downstream computation can know what the value it's getting is, what it is, what it's coming from, what kind of object it is, what kind of type it is.

So you can think of this as creating a name for these objects and creating a form of indirection.

And as I said, now we have these systems of operating on sets.

Because the attention mechanism doesn't care of the order in which we're putting these elements in the first layer.

It just picks 1 according to how well it matches some kind of learned policy.

But the information about the relative position of these elements in your brain or in a neural net architecture doesn't matter anymore.

Okay, now let's connect to what our friends in neuroscience and cognitive science are doing.

In our community, the C word, consciousness, is still kind of taboo, but in their community, it's not anymore.

They've made a lot of progress in understanding several aspects of consciousness.

And there are a number of theories, but many of them are related to what's called the global workspace theory, which originated from Barthes, and there's a lot of very good, important improvements to it from Stan Dehaene and collaborators.

And what this theory says is that what's going on with it with consciousness is there's a bottleneck of information.

Some elements of what is computed in your brain gets selected and then broadcast to the rest of the brain and influencing the rest of the brain.

So this is related to short-term memory where the things that have been selected are available and conditions heavily perception and action.

So it also gives rise naturally to the kind of system 2 abilities that I've been talking about.

All right, so how is machine learning going to do something?

I mean, what's the connection with machine learning?

Machine learning could be used to help brain sciences better understand consciousness.

But what we are understanding of consciousness could also help machine learning develop better abilities.

So first of all, the work that's been done in neuroscience in general is based on fairly qualitative descriptions of these functionalities that we think are associated with consciousness.

And what machine learning can do is help us formalize in a way that's more mechanistic, what these exactly means.

And then that could feedback the research in neuroscience in order to provide more specific tests that could be done of these theories.

Of course, for me at least, 1 of the motivations is also to get rid of the sort of fuzziness and magic that surrounds the word consciousness.

And I think machine learning is in a good position to provide a justification for these particular forms of computations in a sense of why have they evolved?

What kind of computational and statistical advantage are coming with these particular forms of computations.

And of course, once we understand these things, we also want them in our learning agents.

So consciousness is closely related to language, Because the way that we know that somebody is conscious is by asking them to report what they're thinking about.

So this is the direct way that we know about consciousness.

And that means that there's a very strong link between your thoughts, things that you're conscious of, and language.

That 1 can be translated to the other fairly easily, although there's a loss of information when you go from your thoughts to language.

It also means that there's a connection between system 1 and system 2 here, because those high-level concepts that we communicate with language are anchored in the system 1 sort of intuitive system that connects your brain to the rest of the world through perception and action.

And I think that's 1 of the big important direction for natural language research.

We really want to explore things like grounded language learning where we don't just learn from texts.

We learn from environments which involve perception, action, the perception-action loop through the environment, and allow a learner not just to get sort of patterns of sequence of words, but also what they refer to as its understanding, potentially implicit, of how the world works.

And I refer you here to some work we've done recently, published in the last iClear, on grounded language learning, something that I called Baby AI.

OK, so now we are at the part of my presentation when I'm going to talk about the kinds of priors, the kind of structures, the kind of assumptions and regularization that we could use in order to encourage our learning systems to do a good job at out of distribution generalization and the kind of conscious processing abilities that I've been talking about.

And the first 1 I want to talk about is the sparse factor graph assumption I mentioned.

I called that in a 2017 paper, the consciousness prior.

See all Lex Clips transcripts on Youtube

Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)