Tim Wong, the global public policy lead on AI and machine learning for Google, discusses his role in shaping Google's position on various AI-related issues. He works with governments, regulators, and civil society to address concerns about job displacement and ensuring fairness and non-discrimination in AI systems. Wong also collaborates with product teams and researchers to keep them informed about political developments. He provides examples of policy challenges, such as addressing bias in machine learning systems and navigating the trade-offs between collecting diverse data and privacy concerns. Wong emphasizes the importance of interrogating data and understanding the knowledge generated by machine learning systems. He also discusses the impact of AI on automation and how the research community is exploring various research questions that could shape the implementation of AI technologies. However, he notes that certain policy issues, such as fairness in AI, are not considered serious problems in the research field, which can limit progress in those areas.

- The implementation of certain technologies is limited by a gap between product understanding and deep research.
- Domain knowledge will be a crucial skill in the future of machine learning.
- The interface and collaboration between humans and machines is still an open question.
- Cloud services and machine learning APIs are reducing the need for extensive machine learning expertise.
- One area of interest is the ability to perform machine learning with less data, such as one-shot learning.
- The combination of virtual reality and AI can be used to teach robots without extensive physical setups.
- Northern Europe is leading the way in experimenting with AI and automation models.
- Governments are curious about AI and seeking a better understanding of its implications.
- Some governments are passing regulations, such as the GDPR's right to explanation for automated decision making.
- The UK and the Obama administration have produced reports on the risks and opportunities of AI.

- The speaker discusses the need to focus on concrete risks and near-term problems in AI, rather than only considering the scenario of robots taking over.
- They highlight the intersection of machine learning and art, with projects like Google's AI Experiments and Magenta exploring artistic possibilities.
- The speaker mentions the emergence of unexpected things in AI, such as machines learning rudimentary encryption without needing to be programmed for it.
- There is a discussion about how AI may change the way computer science is taught, with machine learning becoming a more central focus.
- The potential for AI to be distributed and integrated into everyday life is mentioned, with examples like AI-powered music collaboration and web app development.
- The speaker suggests that there is room for competition in the AI industry, with cloud platforms and the decreasing need for large amounts of data becoming important factors.
- The importance of good interfaces and experiences around AI technology is emphasized, as well as the need to consider how AI becomes effectively integrated into products.
- The speaker raises the question of why there is a focus on similar AI implementations, such as chatbots, and suggests that AI as a marketing position plays a role in shaping our perception of AI.

- There are incentives to market AI as personified personalities, but the most powerful AI applications may not come in that form.
- Facebook, for example, does not advertise itself as AI, but it heavily relies on AI for its services.
- There is a growing interest in small-scale machine learning projects that solve specific daily problems.
- Machine learning is becoming more cost-effective compared to traditional coding for certain tasks.
- There is potential for on-device training and processing to reduce latency and improve user experience.
- Security in the machine learning space and visual representation of neural nets are two areas that need more attention.
- Recommended resources for learning about machine learning include Ian Goodfellow's textbook on deep learning and books on the history of AI and cybernetics.

- The speaker introduces himself as Tim Wong and provides his website and Twitter handle
- He clarifies that he is not the Korean pop star with the same name
- He expresses gratitude and thanks the host, Craig, for having him.

- Tim Wong, the global public policy lead on AI and machine learning for Google, discusses his role in shaping Google's position on AI-related issues.
- He works with governments, regulators, and civil society to address concerns about job displacement and ensuring fairness and non-discrimination in AI systems.
- Wong collaborates with product teams and researchers to keep them informed about political developments.
- Policy challenges include addressing bias in machine learning systems and navigating the trade-offs between data collection and privacy concerns.
- Wong emphasizes the importance of interrogating data and understanding the knowledge generated by machine learning systems.
- The impact of AI on automation and the research community's exploration of AI technologies are discussed.
- Certain policy issues, such as fairness in AI, are not considered serious problems in the research field, limiting progress in those areas.

Tim Hwang is the Global Public Policy Lead on AI and Machine Learning for Google - https://twitter.com/timhwang

Read the transcript here: http://blog.ycombinator.com/at-the-intersection-of-ai-governments-and-google-tim-hwang

YC's podcast is hosted by Craig Cannon - http://twitter.com/craigcannon

And we are live from Tim Wong's apartment in San Francisco.

I think the easiest way to do this is just to introduce yourself.

So thanks for having me on the show, Craig.

I'm a global public policy lead on AI and machine learning for Google.

It's a combination of a couple of different things.

On 1 hand, I work a lot with sort of governments and regulators and civil society, trying to figure out actually what Google's position should be on a whole range of issues, everything from whether or not machine learning is going to take all the jobs to whether or not we can make sure that these systems are fair and non-discriminatory.

And then internally, I work with product teams and researchers to kind of keep them apprised of what's happening on the political scene worldwide.

Does that mean traveling around and meeting with people?

Yeah, it's a lot of meeting with people, actually.

So we end up kind of talking with people from a whole range of different kind of sectors and a whole range of different backgrounds Particularly because AI is you know, this kind of emerging technology a Lot of what we're doing is kind of just trying to assess like how different parts of society are thinking about it

Mm-hmm And so I think the with AI you and then policy on AI You've kind of like nested to obscure things that people don't really know what you're talking about.

So could you just back up a little bit and explain what doing policy for Google actually means in the context of AI?

So I think the really interesting thing about AI is basically that a lot of the modern techniques in artificial intelligence, if you even asked people a decade ago, they would have told you, this is never going to be a thing.

And it really has kind of exploded in a completely unexpected way in the last few years.

And so really a lot of the challenge has been like, OK, everybody's kind of wrapping their heads around even what the business impact of the technology is going to be.

But there's increasingly a lot of people trying to figure out what the social impact of the technology will be.

And I would say policy really sits at that interface between these really cool technological capabilities that are coming about and then what society in general is going to do about it.

And so what would be a tangible example at Google of a policy that you guys have worked on to figure out?

Sure, so there's a couple of really interesting problems that we've been working on very closely.

So 1 of them is this question about fairness in machine learning systems.

And for example, to give you 1 really concrete challenge we've been thinking a lot about is, in order to de-bias a system, once a machine learning system is behaving in a biased way, 1 way of trying to deal with it is collecting more diverse data.

of the big problems is when you do that, you end up collecting lots and lots of data about minorities, which raises all these really interesting questions around privacy and then what have you.

And that ends up being a really interesting problem because it's both a technical challenge, which is like, can you collect an adequately diverse data set?

But on the other hand, also this policy question, which is, what is society comfortable with you collecting?

And that ends up being a really interesting trade-off that you have to navigate if you're interested in these problems.

Are you going doing user interviews with people or is it just guessing?

Part of it's actually working with people who know.

It turns out that issues of privacy, particularly minority privacy, are like not new problems.

And so a lot of our work is actually like talking with people who are like experts in that space, right?

People have worked on bias and discrimination questions on the past and a lot of data scientists and trying to get them to talk to 1 another.

Because I think right now what we're really trying to do is kind of bridge these sort of human values on 1 hand with a lot of what's happening on the technological side.

And so if I'm a company and I'm like, I can't afford a policy guy like Tim, and I will be dealing with large amounts of data that may or may not discriminate against people.

Are there any obvious no-go's that you would tell someone?

Well, I think it's to be sure that you're interrogating the data.

I think that's 1 important place to start.

I think 1 of the interesting things about machine learning is that there's lots of potential points of failure.

I think every single interesting point of failure is being investigated right now.

1 of the most common problems is just that you don't adequately think through your data.

The machine does what the machine does, which is trying to optimize against your objective function that you give it.

And it will often maximize in ways that you don't expect.

And that is, in fact, part of the problem.

1 of the examples that I always think about is, we have this project that we released, it was called Deep Dream.

And 1 of the problems in computer vision is trying to figure out what the computer actually thinks it sees when it looks at an image.

And so you go through this process and you basically, the whole process, you show it an image, you ask it, what do I have to do to this image to make it look more like what you think, for example, a sandwich looks like.

And you edit the image slightly and you keep repeating this process until you show out what is the ideal thing that the computer thinks it is.

It turns out that when you ask it to reveal what it thinks a barbell looks like, barbells always show up with human arms attached to them.

And so that's a really interesting problem, because you've trained barbells on photos that always have someone holding the barbell.

And so it ends up learning this completely bad representation.

I mean, a big part of it is just like the consciousness around like, oh, that can happen.

And like, how do you interrogate your data set to make sure it doesn't have those problems.

And you guys are doing some interesting stuff around adversarial data, right?

So I mean, I think adversarial examples and generative adversarial networks are like some of the hottest points in the research right now.

It's almost become a joke that there's so many, what they call GANs out there right now.

So it's a very particular way of setting up machine learning.

But adversarial examples lead to these really fascinating results where you know You can take a picture of a panda and that's a classic example And you edit a couple of the pixels and it like basically like the computer will be like, yep That's definitely a giraffe and it still looks like a panda to humans, right?

And so what data are you seeding into that image to make it think it's giraffe?

Well, a lot of it I think is basically you're editing particular pixels within the image that we know will set off the machine to behave in certain ways.

So because it turns out basically that We always assume that a computer will see the same thing that we do Just based on the visuals, but how we process is actually completely different from from machines This researcher David Weinberger did this awesome article recently, which is basically trying to argue that like, you know machine learning It's it's generating knowledge but 1 of the most interesting things about it is that it's generating knowledge in maybe a way that is completely different from the way our human brains work.

And that ends up being a really interesting challenge, is how do you understand the knowledge that you're getting, and how do you understand the reasoning behind the knowledge that you're getting from machine learning systems.

Well, maybe that's a sensible segue into like how people are investigating the impact of AI as it relates to like automation and what humans are good at doing and what computers are good at doing.

And so when you travel around, you meet with people, you meet with different countries.

How are people gauging the effects of automation and AI right now and its effects over the next decade?

Yeah, I think, so it's an evolving picture, right?

And I think right now, I think everybody is just surprised at all of the things that machines can do that we thought that humans were going to be good at for the foreseeable future.

So like Go is the canonical example, but there's all sorts of really interesting kind of like reasoning and other things that like machines are engaging in now.

And so 1 thing I always tell people is basically that everybody always wants to think about AI as if it were like this huge meteor just crashing into the earth where they're like, what do we do when the AI arrives, right?

And it just like, it doesn't, it just then turns out that it doesn't work like that, right?

And in fact, like what we really need to get to is like thinking about like how particular, you know, technical capabilities will map onto the economy.

And that's what a lot of the work is happening on right now.

And so, yeah, let's go into some examples.

So for example, 1 really interesting question is this adversarial examples, which is basically, everybody always assumes that, OK, if it can be automated, it definitely will be automated, right?

But that's like a fallacy because in certain cases, like you may really worry about the security of your systems, right?

So if someone, for example, can like hold up a photo and cause like a security camera to be like, oh, it's definitely Tim, open the door.

That ends up being a real reason why you would not necessarily want to implement a machine learning system for access control, for instance.

So that's actually really interesting because that means that if we don't solve that research problem, that means that we will be limited in the kind of domains that machine learning enters into.

And I think that's what we're really interested in right now is like, what are these kind of gateway research questions that if we got through, would like totally change the nature of like who, when and why someone would implement this stuff.

And so are those things collecting the interest and the momentum of the research community?

Because I can see a certain direction where it becomes incredibly product focused, right?

Where I'm a researcher, I'm incredibly talented.

Figuring out if the security camera is going to work with an adversarial network maybe might not be of highest interest

Is that blocking people or is the general concept enough?

I mean, I think right now it's a little bit unevenly divided.

It turns out that research interest is not necessarily policy relevant interest.

And so in some cases they're overlapping, right?

So I think there's a lot of interest in adversarial examples There's a lot of interest in like what are attacks?

Essentially that you can put on these machines to get them to behave in ways that you don't expect That seems to be a place where like security which is like very much a policy interest Mm-hmm will map on quite nicely to security as a research interest.

I was talking to a machine learning researcher the other day who was basically like, look, I could not, in good faith, advise a grad student to work on machine learning fairness issues.

Because it's just not considered a serious problem in the field.

And like, that's just like, that has less to do with like the field and more like the norms of the field.

And then that, that ends up being a big issue, right.

We don't have coverage on certain types of things in practice may actually really limit, like where these technologies are implemented.

Well, I think it's a material issue right now.

There's a gap between product understanding and actual deep research.

Everybody's always like, so what skills do we need to teach people in the future because of machine learning?

And I think 1 enormous skill will be domain knowledge, because coming up with a technical capability is just 1 part of this huge picture, which is just like, okay, so then how do we actually introduce automation in a way that makes sense to people?

And so my personal prediction is that interface and how we effectively collaborate with machines, particularly with these 2 new types of models, how that effectively done is still a big open question and will seem to be increasingly in higher demand as you suddenly have access to these capabilities.

So what I've been wondering then is, does, for example, TensorFlow or any 1 of the machine learning APIs, does that become the new AWS for products or do people have to build their own to create like a defensible company?

I mean, I think there's still like, so I mean, like cloud services will have the same impact on the economy that they always have.

And I think this is 1 interesting thing is all these companies are now competing for offering cloud ML services.

And the upshot of that basically is that the amount of, you don't need a PhD in machine learning to get all the benefits from machine learning.

And I think that will shape the space for sure.

So then what are the other areas, aside from the first 1 we talked about, for automation and work?

Well, so I think the other thing we're really interested in, and I'm really interested in, is is it possible to pull off machine learning with less and less data.

And so there's a couple examples of that, but 1 of them is one-shot learning, where people are basically working on the ability to teach machines, but a much smaller number of examples.

Now, that actually has a really big impact on the game, because that means that you can implement machine learning effectively in situations where it's really expensive to collect lots of data.

There's also 1 really cool interface between VR and AI that's happening right now, where the whole idea is there's a project called Universe from OpenAI and another project called DeepMindLab, Which basically, imagine you need to teach a robot to get through a maze.

Well, you could have it physically run through that maze millions of times, or you could just have a virtual 3D environment that you cause a computer to run through, and it learns how to do that in virtual space, and then basically you put it into, in practice, in a real robot.

And so that's a really another exciting way we don't necessarily need like an expensive physical setup to collect the data that you need to accomplish tasks in the real world.

So then what like I guess what I'm curious about then is like how are these countries like preparing for this like, you know, again, not a meteor strike, but like perhaps a gradual shift over 20 years, 30 years to a very different world than what we have right now.

Yeah, and I think you're right now, you're seeing like a bunch of different ideas out in the space.

You know, some, for example, like basic income, right?

Universal basic income, which would like fundamentally reshape, you know, like the social contract and how we think about doing, for example, like welfare in a whole number of countries.

And like, So you see proposals like that.

I think you see a number of proposals that are more focusing on education.

So what are skills that people would need in the space?

That ranges everything from everybody needs to be a programmer to, oh, well, we need to really encourage computational thinking, which is the ability to work effectively with data.

And so there's a couple of different options out there.

Some of the more interesting ones that I've heard of that are a little bit more obscure.

So some people have said, oh, well, maybe we need automation insurance.

in the future, your employer will provide you with a contract that says, if your job turns out to be replaced by AI at some point in the future, we'll pay out at some kind of rate.

So people are experimenting with lots of options right now.

I think what we actually need in this space is more experimentation.

So for even proponents of basic income, a lot of them will tell you, like, we actually don't know in practice what this would look like if it were actually rolled out at any level of scale.

And so, like, I mean, it's cool seeing YCR and a couple other places, like, experiment with this.

And so where is the traction happening, then, with all of these experiments?

It seems very limited, but is it all in Northern Europe?

Or I know there's a basic income study in India at this point.

Who seems to be focusing most on this area?

Yeah, we're seeing a lot of different countries engage in this.

I think Northern Europe is kind of leading the way in terms of their willingness to kind of experiment with some of these models.

And I think they've got a couple of things going for them.

On 1 hand, I think they have a skilled labor force that is relatively expensive.

So I think that they are seeking and excited about AI in large part because it's a prospect to bring, for example, manufacturing back to the country, because it allows them to compete on the same footing as other countries that have offered labor for much lower costs.

I think the other thing that's also encouraging a lot of experiments is that they have a lot more coordination between government, industry, and labor, which is making it more possible to experiment with these sorts of things.

So I think in a really interesting case, it turns out that maybe Northern Europe is actually a little bit ahead in its ability to kind of experiment, understand some of these programs.

And then as a like Google or Alphabet as like this international institution at this point, how are you guys thinking about interacting with different countries as this happens?

Yeah, so We're investigating at the moment, right?

So the question is who on the research side should we be working with?

And what are kind of programs that we could support that will help us give a better handle on this picture?

Right, because I think like, look, ultimately it's a technology company, right?

And So we know that we don't have all the talents necessary to evaluate what is a proper social welfare program.

But on the other hand, we do think it's actually really important that we encourage a better societal understanding of how to deal with these technologies.

And so I think we're very much in the mode of like, how can we support this?

And I think that's partially through potentially resources, but also potentially like expertise as well.

Like, if you want to know anything about machine learning, we got people who can tell you about that.

Now we have to marry that up with people who have a good understanding of how this will impact society either through economics or otherwise.

And do you ever feel like the information you're disseminating is like guiding the conversation and guiding the future?

Or like people are like playing into the game like it's intentional?

I mean, I think like, you know, I think it's easy, particularly in the Valley, to be like, oh my God, these big companies.

But we're only 1 part of a much larger, larger picture about what's happening in the economy.

We talk about AI and automation, but we might also want to talk about demographic shifts happening in the economy.

What's it mean that we have an aging workforce?

Or what's it mean that we have falling workforce participation in the United States?

Those are actually trends that are almost as large as what someone comes up with in a lab and presents at a machine learning conference.

And so I think it's actually really important that we look at this all in a bigger perspective.

And so what do you guys do to keep that in mind?

I imagine you just have a whole policy team to manage that

Kind of what we're responsible for is keeping track of a lot of this stuff and getting a better understanding of who is researching in this space.

Because as I said, I think we're still really early on in this technology.

Again if you had asked someone 10 years ago whether or not neural nets were going to be a thing, they'd be like, yeah, I don't know, it probably wouldn't work, right?

But if we're at a phase right now where suddenly it has become technically real, I think now that understanding is just starting to percolate out to a bunch of other fields who are like, okay, well, I guess now we've got to assess what's going on.

And so do you see companies and organizations and countries locking their gates because they're scared, because it feels new, it's obviously massively hyped, but there's also some reality behind it.

I mean, I think by and large, what we're seeing is that a lot of governments are just really curious.

They actually want a better understanding of what's going on.

So in many cases, I think what we're seeing is people asking, what is happening in the technology?

So I think the phase of what to do about it is still on its way.

So you give them the PowerPoint deck and they're like, oh, okay I kind of get how this works and then they go home, you know, whatever to like Japan and they're like, okay No, you think about it?

Yeah, I think so I mean this is how government progresses right is like I think like they ask questions to get information and there's like a long process figuring out what you do around it.

But that isn't to say there isn't laws and other regulations being passed that have relevance for machine learning.

So 1 of the most interesting aspects of the GDPR, which is a new privacy regulation in Europe, is the potential for what they call a right to explanation.

So the idea is for certain kinds of automated decision making, it might be so significant as to require or give citizens the right for that system to be able to produce some kind of human understandable explanation for what it's doing.

And like that raises all sorts of interesting challenges about like how you actually pull that off.

And so like I would say that I don't want to make it sound like no governments are taking action, but I think like that's that's the beginning part of it, right?

Like and I think like by and large, the stance of most governments have been to like understand what's going on

Do you think someone's doing it particularly well now?

Yeah, I mean I was really excited by some of the stuff happening out of the UK So last year they actually did a report that was on kind of like giving a an account of like the risks and opportunities From artificial intelligence and I think there's like a really good account of that So and then last year with the under the Obama administration, there was a really good report that he did as well on the topic

okay, and so like Can you go some specific on that?

Yeah, sure So, I mean I think the what we at least have it in the, what we least had in the US case, right, was basically a report that like really focused in on like, okay, what are the real concrete risks here?

And part of the idea was to pivot away from discussions that were just like, okay, the main thing we've got to talk about here is whether or not robots are gonna destroy us, right, like decide to take over, right?

Which I agree is like kind of an interesting scenario to consider, but like, you're right, like there's a lot of like core near term problems that need to be dealt with.

And I think that was 1 thing that he did that was very useful.

aside from the stuff we've talked about, what do you find to be particularly exciting both like here, like at a local Bay Area level as far as like research and then at global international research level moving this stuff forward?

So I think there's 2 things that I find really interesting right now.

1 of them is the intersection of machine learning and art.

So largely, this is a technology we've been using to solve pretty pragmatic things, which is how do we ensure that we can adequately recognize like cats in photos?

But like, what's really interesting is a bunch of people are kind of playing around right now with the intersection between like, oh, could I use this for like artistic purposes?

Google has this project called AI Experiments, which is a lot of small things like this which demonstrate the artistic possibilities of technology.

We also have another program called Magenta, which is looking into machine learning in music and whether or not there's ways of creating better creative collaboration between humans and machines on that front.

And have you experimented with it personally?

There's 1 project which is basically like a melody generator.

Like you play some notes on a piano and then the computer will play alongside

And so you kind of like improvise with the computer, which is super cool.

There's another project called Marodercam, which

which is you take a couple of photos of things in the room and it produces this boppin' electronic dance hit that uses the words of the objects in the room as like a rhyming, you know, set of lyrics.

See all Y Combinator transcripts on Youtube

At the Intersection of AI, Governments, and Google - Tim Hwang