“Ground Truth” is our new series featuring conversations with the brightest minds in AI and machine learning.

We're really excited tonight to launch Ground Truth, Arthur's semi-weekly speaker series where we'll bring in some of the brightest minds from AI, ML, policy, the tech ecosystem at large, investing, and others.

This is a space, obviously, that doesn't move very quickly.

When I think about what AI and ML and policy around it looked like 10 years ago, I often question my life as a professor, which actually is why I'm here in industry now, having left that job to join, well, this space where nothing ever moves, well, except for every week or every day where we see it in the news.

So Today to kick this off, we're really lucky to have an expert in an area that I think is really undergoing an inflection point between academic research and industry and also governmental adoption.

So we're honored, honestly, to have Rachel Cummings here.

She's an associate professor at Columbia University, homed in the IE and operations research department, but obviously with a strong affiliation with computer science, AI, and ML.

And she also co-leads the Cybersecurity Research Center there, which I believe is in the Data Science Institute, which is another 1 of these sort of multidisciplinary tech-focused institutes.

Prior to that, she was an assistant professor at Georgia Tech, and prior to that, had a PhD from Caltech and a number of other sort of listed accomplishments from Northwestern and, I believe, USC.

So from the academic side, we love accolades.

She's gotten all the sort of big ones that a junior or mid-career researcher would get, NSF Career Award, DARPA Young Faculty Award, Apple Privacy Preserving Awards, JPMC, Google, Mozilla, et cetera.

I can keep listing these, but we're not here for that.

Her work focuses on, and you'll learn more about this tonight, data privacy.

And so that is a broad space that combines machine learning, economics, econ CS, optimization, statistics, public policy, communication, and so on.

And so you'll hear a little bit tonight as well about her work in sort of the policy space.

So she's on the advisory board of, for example, the ACM US Public Policy Council's Privacy Committee and their future privacy forum.

So without further ado, we'll get a 25-ish minute talk from Rachel.

We'll have a Q&A session, and then we can take some more questions from the crowd.

Thank you, John, for perhaps the kindest and friendliest intro that I've ever received, truly.

So as John mentioned, my work is sort of at the interface of academia, industry, policy, and so I'm excited that I can be here tonight and share all 3 of those.

1 disclaimer that I always give in my talks is that I have a minor speech impairment, so if I pause briefly, I'm probably not just daydreaming.

So my talk today will be about how do you communicate technical guarantees to people who may be math phobic.

And so I know that's not the people in this room because you're here.

But in general, we oftentimes have to communicate to people who are not like engineers and mathematicians and so on.

And how do we communicate things without their eyes glazing over as we talk about PDFs and so on.

This is a joint work with a fantastic team, Gabe, Gabum, Kapchuk, Priyanka, Nani Akara, Alyssa, Red Miles, Mary Ann Smart, all of them are from different universities and come from different backgrounds.

And so I'm especially proud of the fact that we are a very multidisciplinary team coming from very different strings to solve this big complex problem.

So I'll be talking about how do you communicate specifically like a differential privacy.

In this room, how many folks have heard that phrase before?

So my first 2 slides, they're all about, oh, this is so confusing.

So I'll start by talking about how do you explain differential privacy in a room like this.

But I want you to think about, as I talk about it, what would not work if you were explaining this to your grandparents or something.

And differential privacy was defined in 2006 by Dwork, McSherry, and E.C.

If I take a database D containing data coming from n people, and I plug that into my favorite randomized algorithm that will give me out a PDF of the possible outputs of this algorithm instantiated on this particular database.

And our privacy guarantee says, if I were to change 1 person's data, say that I change Alice into being Javier.

And I then plug this new database that is the same except for 1 person's data into my same randomized algorithm, I'm going to get out a different curve.

And our question is, how close are these curves, in the worst case, over all possible databases, all possible Alice, and all possible Javier?

And we'll measure a closeness of this curve as being a bound on the ratio, in the worst case, point-wise over these 2 curves.

So now you might think, if you haven't heard of it before, which I guess most everyone has, you might think, how does this correspond to privacy?

You're talking about bounds of ratios of some output PDFs.

So now imagine that we had the point on the blue line that was produced by this algorithm.

And then someone who was an adversary tried to figure out, do we produce this output because of Alice's data or because of Javier's?

And they cannot, information theoretically, make any inferences beyond the amount allowed by this bound on the ratio.

So this is how our bounds translate into a thing that feels kind of like privacy.

So if we have an algorithm M mapping from n tuples of data.

Think of this as being data coming from n people.

It's going to be called an epsilon, differentiate private if it is the case, that for all pairs of neighboring databases, d and d prime, they're the same except for 1 person's data.

And for all possible things in my output space, I'm going to output that thing with about the same probability under these 2 different databases.

So effectively, bounding this kind of like a maximum amount, 1 person's data, can affect the results of the computation.

And because we're talking about randomized algorithms, we bound it in this strange probabilistic way.

So some nice properties in the last how many years?

We have developed a ton of different algorithms that will give you both privacy as well as accuracy for a task of interest across problems like machine learning, statistics, optimization, economics, and so on.

And these tools are also used in practice by some pretty big name organizations.

The question becomes, if you want to convince users that you're doing it and it's great, and they should feel good about this, which they should, how do you do this?

And I will say in this case, this like a particular billboard actually is describing differential privacy, which you might notice is pretty different from the way that it was presented on the last couple of slides.

So we want to ask how do we explain differential privacy in a way that is both easy to kind of understand, but also captures the core principles of the privacy guarantee that is provided?

Now, if I were to modify this to be a little bit more technically precise, I would say what happens on your iPhone generates a random signal that's correlated with your data that does leave your iPhone, but that doesn't look as nice on a billboard.

And the reason that we care about this is because we want to empower users to make informed choices about their data.

And we want to be able to really offer them choices.

Do you want to participate in this differentially private computation?

And these are things we are just not equipped to even begin asking in a way that we can expect reasonable, informed responses back.

And I'm going to have a two-part talk today.

And I know this is a short talk, so I'm going to be skipping a lot of the details, but I link all the papers at the bottom.

And so first parts is just about communicating the definition.

What does it promise, and what does it not promise?

Because it doesn't promise nothing leaves your iPhone, even though the billboard says so.

So our question is basically, how does the way differential privacy is described impact users' expectation of privacy against certain information disclosures?

And like in the paper, but not mentioned today, we also think about what are the type of information disclosures that are important as we ask this question, and then also how does the way in which we describe differential privacy affect people's willingness to share data as well?

I'm happy to share these answers offline.

So the plan of attack, which is also a plan for the next few slides is we first went out and we looked at how is a differential privacy described in the real world?

It's been talked about in industry press releases, It's been talked about in academic papers.

It has been talked about in journalism and tech spaces.

And so we wanted to see, how do people describe it in a sentence?

Then we went out And we then did vignette-based surveys to ask people about a particular data sharing scenario.

And then we asked them, given a particular scenario and a particular privacy description, would you share what type of like a price expectations do you have based on this?

And then we ask them things and then we like evaluated which of these like a differential privacy descriptions are the most effective.

So the first part is we found there were roughly 6 themes corresponding to these privacy definitions.

And so we grouped things into these 6 topics, and for each of them we developed 1 single canonical differential privacy description in a way that really captured some of the core essence.

And I won't bother reading them all to you but I'll tell you some of these core themes.

It says things like differential privacy is a gold standard.

Number 2 talked about the fact that it used a randomized statistical noise.

Number 3 talked about the fact that it enabled analyses through a previously impossible while still protecting individual data.

Number 4 relied on a trust of some kind of institutions and things like you know companies including Uber and Google use it.

Strange enough math was it like trusted entity.

We saw lots of articles being like ah yes it's very mathematical so you should trust it.

Number 5 was about how it reduced risk to the user if they were to participate in this analysis.

And number 6 is 1 that we took straight out of a differential privacy academic paper, solo authored by Dwork.

And we thought, if this is the way we describe it amongst ourselves, Does this work?

We expected not, but it's good to test hypotheses.

And we gave them a vignette, and I'll give you an overview because I know this is hard to read.

But so we said, imagine that you work in the banking industry in a particular city.

All of your friends also work in the banking industry in the same city.

1 of your friends has a part of a salary transparency initiative and they want everybody to share their salaries so that we can measure this.

There's a website, and you can input information onto the web form about your job title and your salary.

And then we add, to reduce the intrusion into personal privacy, your friend says that they will use a tool that is called a differential privacy.

And then we introduced 1 of our 6 privacy definitions here, and then we also had a seventh, a no privacy baseline.

We had a second vignette that was very similar.

We were asked about kind of like, I'm sharing your health record with a nonprofit medical research institute, just to see if things like salary versus health care make a difference.

And so then with this, we asked a very short survey.

And aside from all of our standard demographics and attention checks and so on, we asked them 2 kind of core questions.

And the first 1 is kind of like, Would you share your data?

And then we also asked them, which of the following information disclosures would you expect to be possible?

And we gave them 6 that we had identified in a previous survey.

And these are kind of like Hackers, law enforcement with a warrant, organization collecting your data.

So this could either be the kind of like a transparency initiative or this like a medical research group.

And the analyst is the person working there organizing your data, and graphs is about kind of like a graphs and charts, like a computed base on your data, will they leak information about you and your information, and then finally like a third-party sharing.

And so we asked them, which of these following disclosures would you expect to be possible?

And if you're thinking, technically anything is possible because like, you know, differential privacy is a randomized notion and so like, you know, possible versus probable are very different.

And in that case, hold on for part 2 of the talk.

So the results are that we found very, very heterogeneous effects.

And there's no 1 single answer that worked all the time.

In fact, we found that different explanations caused people to have different privacy expectations.

And this was even different across different people.

And so there's no single like, this 1 always works for this group, and this 1 doesn't.

Just kind of different people like different things, which is not the most satisfactory answer, but that's the data.

So in this picture, so this shows basically kind of like we have on the left-hand side, On the left-hand side, we have all of our different privacy description methods.

In the columns, we have our different privacy protections.

And the colored columns indicate there was a significant difference at the 5% level in people's privacy expectations relative to the no privacy baseline.

So colored cell means a kind of like a comparing against, so again this 1 in the top left, we have the kind of like unsubstantial, causes people to expect privacy against hacks significantly more than the no privacy baseline causes people to expect protection against hacks.

So we kind of don't know what to make of this.

Again, there's no single thing that works all the time.

We can say things like talking about risk does indeed talk to things everybody wants to hear, apparently, and it makes them expect stronger privacy protections.

Similarly, talking about the kind of algorithmic techniques used, some things that were not very great, just unsubstantial and trust.

They really didn't make people expect very strong privacy protections.

And similarly, we see that almost all of these cause people to expect stronger protections against things like analysts and graphs and charts based on their data.

Nobody expected privacy against the kind of the co-ordination itself and very few, only risk caused people to expect a privacy against law enforcement.

And the question is, what do we do with this information?

We mostly just see a lot of heterogeneity in the responses, and that's not particularly actionable.

Maybe you want to do personalized privacy.

But importantly, we're not just thinking about does it make you expect stronger protections, we're actually asking does it make you expect correct protections?

And before we talk about like a correctness, we have to like a distinguish between between like a 2 different privacy models, central versus local.

So in the central model, think of something like the census.

They're going to collect all of your data, store it, look at it, analyze it, and then they're going to publish something about it.

And the thing they publish is going to be private.

On the other hand, local model is basically doing kind of like an on-device privacy, on-device noising.

And so this is things like Chrome, Chrome on-device, iPhones on-device privacy.

We're going to kind of add noise to your data before you share anything about it.

And by the way, That's why the billboard was about what happens on your iPhone stays on your iPhone.

And so of course, in the central model, there exists a database that can be hacked, subpoenaed, shared, and so on.

And so as we talk about possibilities, all these things are more possible.

In the local model, there's not a single database containing everyone's data, so you can't really even access it, hack it, share it, and so on.

So using this, we can define our ground truth with respect to these 2 different models.

And so false means that it's not possible.

And there's a little asterisk, which is for a typical implementation.

And these don't necessarily have to be true.

For example, Uber implemented differential privacy in a central model because it didn't trust its employees.

And so in that case, that's their problem.

And so in this case, there was protection against the organization and its employees.

So we can go back, and we can track, if I were to take everybody's answers about whether or not they expected privacy protections, I can grade those same answers against 2 possible ground truths.

And for each of these, we have our 6 possible privacy explanations, as well as a baseline.

On the bottom, we have scored, what is a fraction of responses that were correct?

So remember, these are true-false questions.

And in the central model, we see people are getting them roughly half right.

So they're just the same as random guessing.

In the local model, they are getting true false questions 25% of the time.

And this told us more work is needed here.

So this is where I switch to questions of this part.

And so in fact, this part will be shorter because it's a very much work in progress.

So I don't have as many answers here, but I have some exciting things that are happening.

So there's 2 important things that we thought were really missing as we talked about a differential privacy definition.

1 of them, of course, is local versus central.

We see people are just pretty wrong about that, and that's not good.

And then also this notion of the epsilon.

So we have some work in progress on both of these questions.

And they're basically written the same in math.

But as we saw, they give very, very different real-world privacy protections.

So we have ideas and these all end in a like question mark.

We have just finished pilots but we don't have any results yet.

So you might think about privacy nutrition labels.

In the sense of inner food, we don't need to know chemical compounds and processing.

I just have to know how many calories, how many grams of protein, and so on.

And somebody somewhere decided, these are the correct, like a small number of details that are important, and I can communicate that.

Everything else isn't important to the end user.

So you might think about privacy nutrition labels, maybe this, as you compare in the central model versus local model.

And you want to compare kind of like a with privacy protection without.

I personally don't like them because it makes a central model look very weak.

And so there's so many things that are possible in the central model.

But those things are things like hacks or like in a law enforcement subpoenaing you, like a third-party sharing.

Those requirements are totally different from like a differential privacy, but that's about your like security practices and your legal policy.

And so that's not differential privacy's fault.

You might model things instead in terms of data flows, who can see what and what part of the process.

I personally like these, and I think they're the most clear and the most precise.

However, they have not done very well in our pilots, so we're probably not going to proceed with them.

On to part 2 of this question, how do you explain epsilon?

So also, if I talk about how likely is it for your information to be leaked through some graph or some chart, well, that depends on the epsilon.

And of course, talking about epsilon doesn't really capture people's intuition about privacy.

It's a bound on the log of the worst case Bayesian update of an adversary's posterior.

This is not how people reason about privacy.

So maybe we have to think about implications, given some epsilon, what's a thing that you might care about?

And 2 things that we have ideas about is you might want to bound how much more likely is a bad thing to occur as a result of this analysis of your data?

Or it might be, here's some observations of an output of the algorithm.

And so we call these things odds-based versus sample-based.

And 1 challenge is that people are just remarkably bad at reasoning about risk and low probability events.

And this is just like, how do you explain low probability events in this way?

So we have our 2 different privacy explanations.

1 is saying if there's a certain chance of a bad thing happening to your data, then I can plug in some numbers based on the epsilon.

And those are the ones that are kind of odds based.

How's like, here's 5 samples, how do you feel about them?

So we developed, So we did like a vignette based survey again But we said you work at a company and they want to know is your manager adequately supporting you and you think like no That guy sucks, but you're afraid of saying it because of like, you know retaliation and the manager might get back to you, and so on.

And so then we gave them a privacy explanations with various epsilon.

We evaluated a bunch of things, which I'll skip.

See all Arthur transcripts on Youtubetab

Ground Truth