Charles Isbell is the Dean of the College of Computing at Georgia Tech. Michael Littman is a computer scientist at Brown University. Please support this podcast by checking out our sponsors:
- Athletic Greens: https://athleticgreens.com/lex and use code LEX to get 1 month of fish oil
- Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savings
- MasterClass: https://masterclass.com/lex to get 2 for price of 1
- Cash App: https://cash.app/ and use code LexPodcast to get $10

EPISODE LINKS:
Charles's Twitter: https://twitter.com/isbellHFh
Charles's Website: https://www.cc.gatech.edu/~isbell/
Michael's Twitter: https://twitter.com/mlittmancs
Michael's Website: https://www.littmania.com/
Michael's YouTube: https://www.youtube.com/user/mlittman

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
2:27 - Is machine learning just statistics?
6:49 - NeurIPS vs ICML
9:05 - Data is more important than algorithm
14:49 - The role of hardship in education
23:33 - How Charles and Michael met
28:05 - Key to success: never be satisfied
31:23 - Bell Labs
42:50 - Teaching machine learning
53:01 - Westworld and Ex Machina
1:01:00 - Simulation
1:07:49 - The college experience in the times of COVID
1:36:27 - Advice for young people
1:43:19 - How to learn to program
1:54:43 - Friendship

CONNECT:
- Subscribe to this YouTube channel
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/LexFridmanPage
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

The following is a conversation with Charles Isbell and Michael Whitman.

Charles is the dean of the College of Computing at Georgia Tech, and Michael is a computer science professor at Brown University.

I've spoken with each of them individually on this podcast, and since they are good friends in real life, we all thought it would be fun to have a conversation together.

Quick mention of each sponsor, followed by some thoughts related to the episode.

Thank you to Athletic Greens, the all-in-one drink that I start every day with to cover all my nutritional bases, 8 sleep, a mattress that cools itself and gives me yet another reason to enjoy sleep, masterclass, online courses from some of the most amazing humans in history, and Cash App, the app I use to send money to friends.

Please check out the sponsors in the description to get a discount and to support this podcast.

As a side note, let me say that having 2 guests on the podcast is an experiment that I've been meaning to do for a while.

In particular, because down the road, I would like to occasionally be a kind of moderator for debates between people that may disagree in some interesting ways.

If you have suggestions for who you would like to see debate on this podcast, Let me know.

As with all experiments of this kind, it is a learning process.

Both the video and the audio might need improvement.

I realized I think I should probably do 3 or more cameras next time as opposed to just 2, and also try different ways to mount the microphone for the third person.

Also, after recording this intro, I'm going to have to go figure out the thumbnail for the video version of the podcast, since I usually put the guest's head on the thumbnail, and now there's 2 heads and 2 names to try to fit into the thumbnail.

It's a kind of a bin packing problem, which in theoretical computer science happens to be an NP hard problem.

Whatever I come up with, if you have better ideas for the thumbnail, let me know as well.

And in general, I always welcome ideas how this thing can be improved.

If you enjoy it, subscribe on YouTube, review it with 5 Stars on Apple Podcast, follow on Spotify, support on Patreon, or connect with me on Twitter at Lex Friedman.

And now, here's my conversation with Charles Isbell and Michael Littman.

You'll probably disagree about this question, but what is your biggest, would you say, disagreement about either something profound and very important or something completely not important at all?

I don't think we have any disagreements at all.

So 1 thing that you sometimes mention is that, and we did this 1 on air too, as it were, whether or not machine learning is computational statistics.

And in particular, and more importantly, it is not just computational statistics.

Oh, because it's- Well, you can't be wrong now.

If it were just the statistics, then we would be happy with where we are.

That's why it's computational statistics.

I agree that machine learning is not just statistics.

What is the computational and computational statistics?

Does this take us into the realm of computing?

It does, but I think perhaps the way I can get him to admit that he's wrong is that it's about rules.

It's about symbols, it's about all these other things.

It's not just a random variable that you choose and you have a probability distribution.

I think you have a narrow view of statistics.

Okay, well then, what would be the broad view of statistics that would still allow it to be statistics and not say history that would make computational statistics okay?

Well, okay, so I had, my first sort of research mentor, a guy named Tom Landauer, taught me to do some statistics.

And I was annoyed all the time because the statistics would say that what I was doing was not statistically significant.

And I was like, but, but, but, and basically what he said to me is, statistics is how you're gonna keep from lying to yourself.

It is a way to keep yourself honest in a particular way.

I'm just going to bring it back to rules.

Even regular statisticians, non-computational statisticians, do spend some of their time evaluating rules, right?

Applying statistics to try to understand, does this rule capture this?

I mean like hypothesis testing kind of thing?

Like I feel like the word statistic literally means a summary, like a number that summarizes other numbers.

But I think the field of statistics actually applies that idea to things like rules, to understand whether or not a rule is valid.

Because I think it's useful to think about a lot of what AI and machine learning is or certainly should be as software engineering, as programming languages.

To put it in language that you might understand, the hyperparameters beyond the problem itself.

The hyperparameters is too many syllables for me to understand.

practice of machine learning is different than the practice of statistics.

Like the things you have to worry about and how you worry about them are different, therefore they're different.

Right, at a very little, I mean at the very least.

It doesn't mean that statistics, computational or otherwise aren't important.

I think that we could think about game theory in terms of statistics, but I don't think it's very as useful to do.

I mean, the way I would think about it, or a way I would think about it is this way.

But I don't think it's as useful to think about chemistry as being just physics.

It's useful to think about it as chemistry.

The level of abstraction really matters here.

So I think it is, there are contexts in which it is useful.

And so finding that connection is actually helpful.

And I think that's when I emphasize the computational statistics thing.

I think I want to befriend statistics and not absorb them.

Here's the A way to think about it beyond what I just said, right?

So what would you say, and I want you to think back to a conversation we had a very long time ago, what would you say is the difference between, say, the early 2000s, ICML and what we used to call NIPS, NIRPS, is there a difference?

A lot of, particularly on the machine learning that was done there?

Michael knows this better than me because of course he's significantly older than I am.

But the point is, what is the difference between ICML and NeurIPS in the late 90s, early 2000s?

I don't know what everyone else's perspective would be, but I had a particular perspective at that time.

Which is I felt like ICML was more of a computer science place, and the NIRPS was more of an engineering place, like the kind of math that happened at the 2 places.

As a computer scientist, I felt more comfortable with the ICML math, and the NIRPS people would say that that's because I'm dumb.

And that's such an engineering thing to say.

I agree with that part of it, but I do it a little differently.

Actually, I had a nice conversation with Tom Dietrich about this.

I put it a little differently, which is that ICML was machine learning done by computer scientists.

And NURBS was machine learning done by computer scientists trying to impress statisticians.

Which was weird because it was the same people, at least by the time I started paying attention, but it just felt very, very different.

And I think that that perspective of whether you're trying to impress the statisticians or you're trying to impress the programmers is actually very different and has real impact on what you choose to worry about and what kind of outcomes you come to.

I think computational statistics is a means to an end.

And I think that really matters here in the same way that I don't think computer science is just engineering or just science or just math or whatever.

Okay, so I'd have to now agree that Now we agree on everything.

The important thing here is that, you know, my opinions may have changed, but not the fact that I'm right, I think is what we just came to.

Right, and my opinions may have changed and not the fact that I'm wrong.

This happens to us sometimes, we're sorry.

How does neural networks change this, just to even linger on this topic, change this idea of statistics, how big of a pie statistics is within the machine learning thing.

Like, because it sounds like hyperparameters and also just the role of data.

You know, people are starting to use this terminology of software 2.0, which is like the act of programming as a, like you're a designer in the hyperparameter space of neural networks and you're also the collector and the organizer and the cleaner of the data and that's part of the programming.

So how did, on the NeurIPS versus ICML topic, what's the role of neural networks in redefining the size and the role of machine learning?

I can't wait to hear what Michael thinks about this, but I would add 1.

I think there's 1 other thing I would add to your description, which is the kind of software engineering part of what does it mean to debug, for example.

But this is a difference between the kind of computational statistics view of machine learning and the computational view of machine learning, which is I think 1 is worried about the equation as it were.

And by the way, this is not a value judgment.

But the kind of questions you would ask, you start asking yourself, well, what does it mean to program and develop and build the system?

It's a very computer science-y view of the problem.

I mean, if you get on data science Twitter and econ Twitter, you actually hear this a lot with the economist and the data scientist complaining about the machine learning people, well, It's just statistics, and I don't know why they don't see this.

But they're not even asking the same questions.

They're not thinking about it as a programming problem.

I think that that really matters, just asking this question.

I actually think it's a little different from programming in hyperparameter space and collecting the data.

But I do think that that immersion really matters.

So I'll give you a quick example of the way I think about this.

Michael and I have co-taught a machine learning class, which has now reached, I don't know, 10,000 people at least over the last several years or somewhere thereabouts.

And my machine learning assignments are of this form.

So the first 1 is something like, implement these 5 algorithms, KNN and SVMs and boosting and decision trees and neural networks and maybe that's it, I can't remember.

When I say implement, I mean steal the code.

You get 0 points for getting the thing to work.

I don't want you spending your time worrying about getting the corner case right of what happens when you are trying to normalize distances and the points on the thing, and so you divide by 0.

However, you're going to run those algorithms on 2 data sets.

Well, a data set's interesting if it reveals differences between algorithms, which presumably are all the same, because they can represent whatever they can represent.

And 2 data sets are interesting together if they show different differences, as it were.

You have to justify their interestingness, and you have to analyze them in a whole bunch of ways.

But all I care about is the data in your analysis, not the programming.

And I occasionally end up in these long discussions with students.

I copy and paste the things that I've said the other 15,000 times it's come up, which is, they go, but the only way to learn, really understand is to code them up, which is a very programmer, software engineering view of the world.

If you don't program it, you don't understand it, which is, by the way, I think is wrong in a very specific way, but it is a way that you come to understand, because then you have to wrestle with the algorithm.

But The thing about machine learning is it's not just sorting numbers, where in some sense the data doesn't matter.

What matters is, well, does the algorithm work on these abstract things, 1 less than the other?

And so, as a result, you have to live with the data and don't get distracted by the algorithm per se.

And I think that that focus on the data and what it can tell you and what question it's actually answering for you, as opposed to the question you thought you were asking, is a key and important thing about machine learning and is a way that computationalists, as opposed to statisticians, bring a particular view about how to think about the process.

The statisticians, by contrast, bring, I think I'd be willing to say, a better view about the kind of formal math that's behind it and what an actual number ultimately is saying about the data.

And those are both important, but they're also different.

I didn't really think of it this way, is to build intuition about the role of data, the different characteristics of data, by having 2 data sets that are different and they reveal the differences and the differences.

That's a really fascinating, that's a really interesting educational approach.

The students love it, but not right away.

They love it at the end, Not at the beginning.

I feel like there's a deep, profound lesson about education there.

That you can't listen to students about whether what you're doing is the right or the wrong thing.

Well, as a wise, Michael Lipman once said to me about children, which I think applies to teaching, is you have to give them what they need without bending to their will.

Your whole job is to curate and to present Because on their own, they're not gonna necessarily know where to search, so you're providing pushes in some direction and learn space.

And you have to give them what they need in a way that keeps them engaged enough so that they eventually discover what they want and they get the tools they need to go and learn other things off of.

What's your view, let me put on my Russian hat, which believes that life is suffering.

What do you think is the role of, we talked about balance a little bit, what do you think is the role of hardship in education?

Like, I think the biggest things I've learned, Like what made me fall in love with math, for example, is by being bad at it until I got good at it.

So like struggling with a problem, which increased the level of joy I felt when I finally figured it out.

And it always felt with me with teachers, especially modern discussions of education, how can we make education more fun, more engaging, more all those things?

Well, from my perspective, it's like you're maybe missing the point that education, that life is suffering.

See all Lex Fridman transcripts on Youtube

Charles Isbell and Michael Littman: Machine Learning and Education | Lex Fridman Podcast #148