That the idea of kind of letting the algorithm start exploring as well, I definitely think, this is a much narrower statement, I definitely think that kind of algorithmic auditing for different types of unfairness, right?

So like in this gerrymandering example, where I might want to prevent not just discrimination against very broad categories, but against combinations of broad categories, you quickly get to a point where there's a lot of categories, there's a lot of combinations of end features, and you can use algorithmic techniques to sort of try to find the subgroups on which you're discriminating the most and try to fix that.

That's actually kind of the form of 1 of the algorithms we developed for this fairness gerrymandering problem.

But I'm, you know, partly because of our technology and our sort of our scientific ignorance on these topics right now, and also partly just because these topics are so loaded emotionally for people that I just don't see the value.

I mean, again, never say never, but I just don't think we're at a moment where it's a great time for computer scientists to be rolling out the idea like, hey, not only have we kind of figured fairness out, but we think the algorithms should start deciding what's fair or giving input on that decision.

I just don't, it's like the cost benefit analysis to the field of kind of going there right now just doesn't seem worth it to me.

That said, I should say that I think computer scientists should be more philosophically, like should enrich their thinking about these kinds of things.

I think it's been too often used as an excuse for roboticists working on autonomous vehicles, for example, to not think about the human factor or psychology or safety in the same way like computer science design algorithms they've been sort of using as an excuse.

And I think it's time for basically everybody to become computer scientists.

I was about to agree with everything you said except that last point.

I think that the other way of looking at it is that I think computer scientists, and many of us are, but we need to weigh it out into the world more, right?

I mean, just the influence that computer science and therefore computer scientists have had on society at large, just like has exponentially magnified in the last 10 or 20 years or so.

And before when we were just tinkering around amongst ourselves and it didn't matter that much, there was no need for sort of computer scientists to be citizens of the world more broadly.

And I think those days need to be over very, very fast.

And I'm not saying everybody needs to do it, but to me, the right way of doing it is to not to think that everybody else is gonna become a computer scientist, but I think people are becoming more sophisticated about computer science, even lay people.

You know, I think 1 of the reasons we decided to write this book is we thought 10 years ago, I wouldn't have tried this just because I just didn't think that sort of people's awareness of algorithms and machine learning, you know, the general population would have been high and I mean, you would have had to first, you know, write 1 of the many books kind of just explicating that topic to a lay audience first.

Now I think we're at the point where lots of people without any technical training at all know enough about algorithms and machine learning that you can start getting to these nuances of things like ethical algorithms.

I think we agree that there needs to be much more mixing, But I think a lot of the onus of that mixing needs to be on the computer science community.

Yeah, so just to linger on the disagreement, because I do disagree with you on the point that I think if you're a biologist, if you're a chemist, if you're an MBA business person, all of those things you can, like if you learned a program, and not only program, if you learn to do machine learning, if you learn to do data science, you immediately become much more powerful in the kinds of things you can do.

And therefore, literature, like library sciences, so you were speaking, I think, I think it holds true what you're saying for the next few years, but long-term, if you're interested, to me, if you're interested in philosophy, you should learn the program.

Because then you can scrape data, and study what people are thinking about on Twitter, and then start making philosophical conclusions about the meaning of life.

I just feel like the access to data, the digitization of whatever problem you're trying to solve, it fundamentally changes what it means to be a computer scientist.

I mean, a computer scientist in 20, 30 years will go back to being Donald Knuth style theoretical computer science and everybody will be doing basically, they're exploring the kinds of ideas that you're exploring in your book.

Yeah, I mean, I don't think I disagree enough, but I think that that trend of more and more people in more and more disciplines adopting ideas from computer science, learning how to code, I think that that trend seems firmly underway.

I mean, you know, like an interesting digressive question along these lines is maybe in 50 years, there won't be computer science departments anymore because the field will just sort of be ambient in all of the different disciplines.

And you know, people will look back and, you know, having a computer science department will look like having an electricity department or something.

It's like, you know, everybody uses this.

I mean, I do think there will always be that kind of Knuth style core to it.

But It's not an implausible path that we kind of get to the point where the academic discipline of computer science becomes somewhat marginalized because of its very success in kind of infiltrating all of science and society and the humanities, et cetera.

What is differential privacy, or more broadly, algorithmic privacy?

Algorithmic privacy more broadly is just the study or the notion of privacy definitions or norms being encoded inside of algorithms.

And so, you know, I think we count among this body of work, just, you know, the literature and practice of things like data anonymization, which we kind of, at the beginning of our discussion of privacy say like, okay, this is sort of a notion of algorithmic privacy.

It kind of tells you, you know, something to go do with data.

But, you know, our view is that it's, and I think this is now, you know, quite widespread, that it's, you know, despite the fact that those notions of anonymization, kind of redacting and coarsening, are the most widely adopted technical solutions for data privacy.

They are like deeply, fundamentally flawed.

And so, you know, to your first question, what is differential privacy?

Differential privacy seems to be a much, much better notion of privacy that kind of avoids a lot of the weaknesses of anonymization notions while still letting us do useful stuff with data.

So by anonymization, I'm kind of referring to techniques like I have a database, the rows of that database are let's say individual people's medical records, okay, and I want to let people use that data, maybe I want to let researchers access that data to build predictive models for some disease, but I'm worried that that will leak, you know, sensitive information about specific people's medical records.

So anonymization broadly refers to the set of techniques where I say, like, okay, I'm first going to, like, I'm going to delete the column with people's names.

I'm going to not put, you know, so that would be like a redaction, right?

I am going to take ages and I'm not going to like say your exact age.

I'm going to say whether you're 0 to 10, 10 to 20, 20 to 30.

I might put the first 3 digits of your zip code, but not the last 2, et cetera, et cetera.

And so the idea is that through some series of operations like this on the data, I anonymize it.

Another term of art that's used is removing personally identifiable information.

And this is basically the most common way of providing data privacy, but that's in a way that still lets people access some variant form of the data.

So at a slightly broader picture, as you talk about what does anonymization mean when you have multiple database, like with a Netflix prize, when you can start combining stuff together.

Is that notions of anonymization, removing personally identifiable information, the kind of fundamental conceptual flaw is that these definitions kind of pretend as if the data set in question is the only data set that exists in the world or that ever will exist in the future.

And of course, things like the Netflix prize and many, many other examples since the Netflix prize, I think that was 1 of the earliest ones though.

You know, you can re-identify people that were, you know, that were anonymized in the data set by taking that anonymized data set and combining it with other allegedly anonymized datasets and maybe publicly available information about you.

For people who don't know, the Netflix prize was, what was being publicly released as data.

So the names from those roles were removed, but what was released is the preference or the ratings of what movies you like and you don't like.

And from that combined with other things, I think forum posts and so on, you can start to figure

Yeah, I mean, in that case, it was specifically the internet movie database, where lots of Netflix users publicly rate their movie preferences.

And so the anonymized data and Netflix, when can you, it's just this phenomenon, I think we've all come to realize in the last decade or so is that just knowing a few apparently irrelevant, innocuous things about you can often act as a fingerprint.

Like if I know what rating you gave to these 10 movies and the date on which you entered these movies, this is almost like a fingerprint for you is in the sea of all Netflix users.

There were just another paper on this in Science or Nature about a month ago that, you know, kind of 18 attributes.

I mean, my favorite example of this is it was actually a paper from several years ago now where it was shown that just from your likes on Facebook, just from the, you know, the things on which you clicked on the thumbs up button on the platform.

Not using any information, demographic information, nothing about who your friends are, just knowing the content that you had liked was enough to, in the aggregate, accurately predict things like sexual orientation, drug and alcohol use, whether you were the child of divorced parents.

So we live in this era where, you know, even the apparently irrelevant data that we offer about ourselves on public platforms and forums, often unbeknownst to us, more or less acts as a signature or fingerprint.

And that if you can kind of do a join between that kind of data and allegedly anonymized data,

So is there hope for any kind of privacy in a world where a few likes can identify you?

So differential privacy basically is an alternate, much stronger notion of privacy than these anonymization ideas.

And it's a technical definition, but the spirit of it is we compare 2 alternate worlds.

Okay, so let's suppose I'm a researcher and I want to do, you know, I—there's a database of medical records and 1 of them is yours.

And I want to use that database of medical records to build a predictive model for some disease.

So based on people's symptoms and test results and the like, I want to build a model predicting the probability that people have disease.

So this is the type of scientific research that we would like to be allowed to continue.

And in differential privacy, you act, ask a very particular counterfactual question.

1 is when I do this, I build this model on the database of medical records including your medical record.

And the other 1 is where I do the same exercise with the same database with just your medical record removed.

So basically, you know, it's 2 databases, 1 with N records in it and 1 with N minus 1 records in it.

The N minus 1 records are the same and the only 1 that's missing in the second case is your medical record.

So differential privacy basically says that any harms that might come to you from the analysis in which your data was included are essentially nearly identical to the harms that would have come to you if the same analysis had done been done without your medical record included.

So in other words this doesn't say that bad things cannot happen to you as a result of data analysis, it just says that these bad things were going to happen to you already, even if your data wasn't included.

And to give a very concrete example, right, you know, like we discussed at some length the study that, you know, in the 50s that was done that created the, that established the link between smoking and lung cancer.

And we make the point that like, well, if your data was used in that analysis, and you know, the world kind of knew that you were a smoker because, you know, there was no stigma associated with smoking before that, those findings, real harm might have come to you as a result of that study that your data was included in.

In particular, your insurer now might have a higher posterior belief that you might have lung cancer and raise your premiums, so you've suffered economic damage.

But the point is, is that if the same analysis has been done with all the other N-1 medical records and just yours missing, the outcome would have been the same.

Your data was an idiosyncratically crucial to establishing the link between smoking and lung cancer, because the link between smoking and lung cancer is like a fact about the world that can be discovered with any sufficiently large database of medical records.

Yeah, so that's showing that very little harm is done.

What is the mechanism of differential privacy?

So that's the kind of beautiful statement of it.

What's the mechanism by which privacy is preserved?

So it's basically by adding noise to computations, right?

So the basic idea is that every differentially private algorithm, first of all, or every good differentially private algorithm, every useful 1, is a probabilistic algorithm.

So it doesn't, on a given input, if you gave the algorithm the same input multiple times, it would give different outputs each time from some distribution.

And the way you achieve differential privacy algorithmically is by kind of carefully and tastefully adding noise to a computation in the right places.

And to give a very concrete example, if I want to compute the average of a set of numbers, the non-private way of doing that is to take those numbers and average them and release a numerically precise value for the average.

In differential privacy, you wouldn't do that.

You would first compute that average to numerical precisions.

You'd add some kind of 0 mean, you know, Gaussian or exponential noise to it, so that the actual value you output is not the exact mean, but it'll be close to the mean, but it'll be close, the noise that you add will sort of prove that nobody can kind of reverse engineer any particular value that went into the average.

How many algorithms can be aided by adding noise?

Yeah, so I'm a relatively recent member of the differential privacy community.

My co-author Aaron Roth is, you know, really 1 of the founders of the field and has done a great deal of work and I've learned a tremendous amount working with him on it.

But I must admit the first time I saw the definition of deferential privacy, my reaction was like, well, that is a clever definition and it's really making very strong promises.

And my, you know, I first saw the definition in much earlier days, and my first reaction was like, well, my worry about this definition would be that it's a great definition of privacy, but that it'll be so restrictive that we won't really be able to use it.

Like, you know, we won't be able to compute many things in a differentially private way.

So that's 1 of the great successes of the field, I think, is in showing that the opposite is true, and that, you know, most things that we know how to compute absent any privacy considerations can be computed in a differentially private way.

So for example, pretty much all of statistics and machine learning can be done differentially privately.

So pick your favorite machine learning algorithm, back propagation and neural networks, you know, cart for decision trees, support vector machines, boosting, you name it, as well as classic hypothesis testing and the like in statistics.

None of those algorithms are differentially private in their original form.

All of them have modifications that add noise to the computation in different places in different ways that achieve differential privacy.

So this really means that to the extent that, you know, we've become a, you know, a scientific community very dependent on the use of machine learning and statistical modeling and data analysis, we really do have a path to kind of provide privacy guarantees to those methods, and so we can still, you know, enjoy the benefits of kind of the data science era while providing, you know, rather robust privacy guarantees to individuals.

a slightly crazy question, but if we take the ideas of differential privacy and take it to the nature of truth that's being explored currently.

So what's your most favorite and least favorite food?

I'm not a real foodie, so I'm a big fan of spaghetti.

But is 1 way to protect your preference for spaghetti by having an information campaign, bloggers and so on, of bots saying that you like cauliflower.

So like this kind of the same kind of noise ideas.

I mean, if you think of in our politics today, there's this idea of Russia hacking our elections.

What's meant there, I believe, is bots spreading different kinds of information.

Is that a kind of privacy, or is that too much of a stretch?

I have not seen those ideas, you know, that is not a technique that to my knowledge will provide differential privacy.

But to give an example, 1 very specific example about what you're discussing is, there was a very interesting project at NYU, I think led by Helen Nissenbaum there, in which they basically built a browser plugin that tried to essentially obfuscate your Google searches.

So to the extent that you're worried that Google is using your searches to build predictive models about you, to decide what ads to show you, which they might very reasonably want to do.

But if you object to that, they built this widget you could plug in, and basically whenever you put in a query into Google, it would send that query to Google, but in the background all of the time from your browser, it would just be sending this torrent of irrelevant queries to the search engine.

So out of every thousand queries, let's say, that Google was receiving from your browser, 1 of them was 1 that you put in, but the other 999 were not.

Okay, so it's the same kind of idea, kind of, you know, privacy by obfuscation.

It's also, I was actually talking to somebody at 1 of the large tech companies recently about the fact that, you know, just this kind of thing, that there are some times when the response to my data needs to be very specific to my data, right?

Like I type mountain biking into Google, I want results on mountain biking, and I really want Google to know that I typed in mountain biking.

And so I think there's sort of maybe even interesting technical questions around notions of privacy that are appropriate where, you know, it's not that my data is part of some aggregate like medical records and that we're trying to discover important correlations and facts about the world at large, but rather, you know, there's a service that I really want to, you know, pay attention to my specific data yet I still want some kind of privacy guarantee.

And I think these kinds of obfuscation ideas are sort of 1 way of getting at that, but maybe there are others as well.

So where do you think we'll land in this algorithm driven society in terms of privacy?

So sort of China, like Kai-Fu Lee describes, you know, it's collecting a lot of data on its citizens, but in the best form, it's actually able to provide a lot of, sort of protect human rights, and provide a lot of amazing services.

And it's worse forms that can violate those human rights and limit services.

So algorithms are powerful when they use data.

So as a society, do you think we'll give over more data?

Is it possible to protect the privacy of that data?

So I'm optimistic about the possibility of balancing the desire for individual privacy and individual control of privacy with kind of societally and commercially beneficial uses of data, not unrelated to differential privacy or suggestions that say like, well, individuals should have control of their data, they should be able to limit the uses of that data, they should even, you know, there's fledgling discussions going on in research circles about allowing people's selective use of their data and being compensated for it.

And then you get to sort of very interesting economic questions like pricing, right?

And 1 interesting idea is that maybe differential privacy would also, you know, be a conceptual framework in which you could talk about the relative value of different people's data, like, you know, to demystify this a little bit.

If I'm trying to build a predictive model for some rare disease and I'm trying to use, I'm going to use machine learning to do it, it's easy to get negative examples because the disease is rare, right?

But I really want to have lots of people with the disease in my data set, okay?

And so somehow those people's data with respect to this application is much more valuable to me than just like the background population.

And so maybe they should be compensated more for it.

And so, you know, I think these are kind of very, very fledgling conceptual questions that maybe we'll have kind of technical thought on them sometime in the coming years.

But I do think we'll, you know, to kind of get more directly answer your question, I think I'm optimistic at this point from what I've seen that we will land at some, you know, better compromise than we're at right now, where again, you know, privacy guarantees are few, far between and weak, and users have very, very little control.

And I'm optimistic that we'll land in something that provides better privacy overall and more individual control of data and privacy.

But I think to get there, it's again, just like fairness, it's not gonna be enough to propose algorithmic solutions.

There's gonna have to be a whole kind of regulatory, legal process that prods companies and other parties to kind of adopt solutions.

And I think you've mentioned the word control a lot, and I think giving people control, that's something that people don't quite have in a lot of these algorithms.

And that's a really interesting idea of giving them control.

Some of that is actually literally an interface design question.

Sort of just enabling, because I think it's good for everybody to give users control.

Except that you have to hire people that are good at interface design.

I mean, the other thing that has to be said, right, is that, you know, it's a cliche, but you know, we, as the users of many systems, platforms, and apps, you know, we are the product.

The customer are advertisers and our data is the product.

So it's 1 thing to kind of suggest more individual control of data and privacy and uses.

But this, you know, if this happens in sufficient degree, it will upend the entire economic model that has supported the internet to date.

And so some other economic model will have to be, will have to replace it.

So the idea of markets you mentioned, by exposing the economic model to the people, they will then become a market.

And you know, this is not a weird idea, right?

Because there are markets for data already.

It's just that consumers are not participants in them.

There's sort of publishers and content providers on 1 side that have inventory, and then they're advertising on the others.

And Google and Facebook are running, pretty much their entire revenue stream is by running two-sided markets between those parties, right?

And so it's not a crazy idea that there would be like a three-sided market or that, you know, that on 1 side of the market or the other, we would have proxies representing our interests.

It's not a crazy idea, but it would, it's not a crazy technical idea, but it would have pretty extreme economic consequences.

Speaking of markets, a lot of fascinating aspects of this world arise not from individual humans, but from the interaction of human beings.

You've done a lot of work in game theory.

First, can you say what is game theory and how does it help us model and study things?

Yeah, game theory, of course, let us give credit where it's due.

It comes from the economists first and foremost, but As I've mentioned before, like, you know, computer scientists never hesitate to wander into other people's turf, and so there is now this 20-year-old field called algorithmic game theory.

But, you know, game theory, first and foremost, is a mathematical framework for reasoning about collective outcomes in systems of interacting individuals.

You know, so you need at least 2 people to get started in game theory, and Many people are probably familiar with prisoners dilemma as kind of a classic example of game theory and a classic example where?

Everybody Looking out for their own individual interests leads to a collective outcome.

That's kind of worse for everybody Then what might be possible if they cooperated, for example.

But cooperation is not an equilibrium in Prisoner's Dilemma.

And so my work in the field of algorithmic game theory more generally in these areas, kind of looks at settings in which the number of actors is potentially extraordinarily large, and their incentives might be quite complicated and kind of hard to model directly.

But you still want kind of algorithmic ways of kind of predicting what will happen or influencing what will happen in the design of platforms.

So what to you is the most beautiful idea that you've encountered in game theory?

I mean, you know, I mean, technical answers to that, of course, would include Nash's work just establishing that, you know, there's a competitive equilibrium under very, very general circumstances, which in many ways kind of put the field on a firm conceptual footing, because if you don't have equilibrium, it's kind of hard to ever reason about what might happen since there's just no stability.

So just the idea that stability can emerge when there's multiple

Not that it will necessarily emerge, just that it's possible, right?

It means like the existence of equilibrium doesn't mean that sort of natural iterative behavior will necessarily lead to it.

Yeah, maybe answering slightly less personally than you asked the question.

I think within the field of algorithmic game theory, perhaps the single most important kind of technical contribution that's been made is the realization between close connections between machine learning and game theory, and in particular between game theory and the branch of machine learning that's known as no regret learning.

And this sort of provides a very general framework in which a bunch of players interacting in a game or a system, each 1 kind of doing something that's in their self-interest will actually kind of reach an equilibrium and actually reach an equilibrium in a pretty, you know, a rather, you know, short amount of steps.

So you kind of mentioned acting greedily can somehow end up pretty good for

And you know, stability or equilibrium by itself is not necessarily either a good thing or a bad thing.

So what's the connection between machine learning and the ideas of equilibrium?

I think we've kind of talked about these ideas already in kind of a non-technical way, which is maybe the more interesting way of understanding them first, which is, you know, we have many systems, platforms, and apps these days that work really hard to use our data and the data of everybody else on the platform to selfishly optimize on behalf of each user.

So, you know, let me give I think the cleanest example, which is just driving apps, navigation apps like, you know, Google Maps and Waze, where, you know, miraculously compared to when I was growing up at least, you know, the objective would be the same when you wanted to drive from point A to point B, spend the least time driving, Not necessarily minimize the distance, but minimize the time.

And when I was growing up, the only resources you had to do that were maps in the car, which literally just told you what roads were available.

And then you might have half hourly traffic reports just about the major freeways, but not about side roads.

And now we've got these apps, you pull it out and you say, I wanna go from point A to point B.

And in response kind of to what everybody else is doing, if you like what all the other players in this game are doing right now, here's the route that minimizes your driving time.

So it is really kind of computing a selfish best response for each of us in response to what all of the rest of us are doing at any given moment.

And so, you know, I think it's quite fair to think of these apps as driving or nudging us all towards the competitive or Nash equilibrium of that game.

Now you might ask like, well, that sounds great.

Well, you know, it's, it's known both in theory and with some limited studies from actual like traffic data that all of us being in this competitive equilibrium might cause our collective driving time to be higher, maybe significantly higher than it would be under other solutions.

And then you have to talk about what those other solutions might be and what the algorithms to implement them are, which we do discuss in the kind of game theory chapter of the book.

But similarly, on social media platforms or on Amazon, all these algorithms that are essentially trying to optimize our behalf, they're driving us in a colloquial sense towards some kind of competitive equilibrium.

And 1 of the most important lessons of game theory is that just because we're at equilibrium doesn't mean that there's not a solution in which some or maybe even all of us might be better off.

And then the connection to machine learning, of course, is that in all these platforms I've mentioned, the optimization that they're doing on our behalf is driven by machine learning.

You know, like predicting where the traffic will be, predicting what products I'm gonna like, predicting what would make me happy in my news feed.

Now in terms of the stability and the promise of that, I have to ask, just out of curiosity, how stable are these mechanisms that you, game theory is just, the economists came up with, and we all know that economists don't live in the real world.

Sort of what's, do you think when we look at the fact that we haven't blown ourselves up from a game theoretic concept of mutually shared destruction, what are the odds that we destroy ourselves with nuclear weapons, as 1 example of a stable game theoretic system?

Just to prime your viewers a little bit, I mean, I think you're referring to the fact that game theory was taken quite seriously back in the 60s as a tool for reasoning about kind of Soviet, US nuclear armament, disarmament, detente, things like that.

I'll be honest, as huge of a fan as I am of game theory and its kind of rich history, it still surprises me that, you know, you had people at the Rand Corporation back in those days kind of drawing up, you know, 2 by 2 tables and 1, the row player is, you know, the US and the column player is Russia, and that they were taking seriously, I'm sure if I was there, maybe it wouldn't have seemed as naive as it does at the time.

seems to have worked, which is why it seems naive and silly.

See all Lex Fridman transcripts on Youtube

Michael Kearns: Algorithmic Fairness, Privacy & Ethics | Lex Fridman Podcast #50