This is a video about the most famous problem in Game Theory, the Prisoner’s Dilemma. Head to https://brilliant.org/veritasium  to start your free 30-day trial, and the first 200 people get 20% off an annual premium subscription.

If you’re looking for a molecular modeling kit, try Snatoms – a kit I invented where the atoms snap together magnetically – https://ve42.co/SnatomsV   


▀▀▀
A massive thank you to Prof. Robert Axelrod and Prof. Steven Strogatz for their expertise and time. 

To read more about Prof. Axelrod’s Passion for Cooperation visit: https://ve42.co/Axelrod2023 

A massive thanks to the wonderful Nicky Case. Nicky’s “The Evolution of Trust” game was a huge inspiration for this video. We highly recommend you play this excellent game yourself, over at: https://ncase.me/trust/ 

A huge thank you to those who helped us understand and fact check different parts of this topic - Dr. Christian Hilbe, Dr. Vincent Knight, Dr. Jelena Grujic, Prof. Andreas Diekmann, and Dr. Alexander Stewart.

▀▀▀
References:
Excellent game on the evolution of trust by Nicky Case - https://ve42.co/Case2023  
Summary of Axelrod’s work by This Place - https://www.youtube.com/watch?v=BOvAbjfJ0x0 
How to outsmart the Prisoner’s Dilemma by TED-Ed - https://www.youtube.com/watch?v=emyi4z-O0ls&pp=ygUScHJpc29uZXIncyBkaWxlbW1h 
Tit for Tat by radiolab - https://ve42.co/T4T   
The Golden Rule by radiolab - https://ve42.co/GoldenRule   
Axelrod, R. (1984). The Evolution of Cooperation.
Dawkins, R. (2016). The selfish gene. Oxford university press.
Poundstone, W. (1992). Prisoner's Dilemma. William Poundstone.
Nowak, M. A., & Highfield, R. (2011). Supercooperators. Edinburgh: Canongate.
Binmore, K. (2007). Game theory: a very short introduction. OUP Oxford.
Northrup, L. & Rock, D. (1966). The Detection of Joe I. - https://ve42.co/JOE1 
Prisoner’s dilemma, Wikipedia - https://ve42.co/WikiPD
Prisoner’s Dilemma, Stanford -  https://ve42.co/StanfordPD 
Flood, M. M. (1952). Some experimental games. - https://ve42.co/Flood1952 
Historical nuclear weapons stockpiles, Wikipedia - https://ve42.co/WikiNWS 
Goodwin, I. (1998). The Price of Victory in Cold War - https://ve42.co/Goodwin1998 
Cold war: How it happened. - https://ve42.co/CW2014 
Axelrod, R. (1980). Effective choice in the prisoner's dilemma. Journal of conflict resolution, 24(1), 3-25. - https://ve42.co/Axelrod1980a 
Axelrod, R. (1980). More effective choice in the prisoner's dilemma. Journal of conflict resolution, 24(3), 379-403. - https://ve42.co/Axelrod1980b 
Axelrod, R., & Hamilton, W. D. (1981). The evolution of cooperation. science, 211(4489), 1390-1396. https://ve42.co/Axelrod1981 
Stanislav Petrov, Wikipedia - https://ve42.co/WikiSP 
Wu, J., & Axelrod, R. (1995). How to cope with noise in the iterated prisoner's dilemma. Journal of Conflict resolution, 39(1), 183-189. - https://ve42.co/Wu1995 
INF Treaty - https://ve42.co/INF 
START Treaties - https://ve42.co/START 
START I, Wikipedia - https://ve42.co/WikiSTART 


Images & Video:
RAND Historical images via rand.org - https://ve42.co/RAND  
Golden Balls - https://www.youtube.com/watch?v=S0qjK3TWZE8
Zotti, G., et al. (2021). The Simulated Sky: Stellarium for Cultural Astronomy Research - https://ve42.co/Stellarium  
Newspapers from 1980s via Newspapers.com  – https://ve42.co/Newspapers  
Decommisioned nuke image via The Moscow Times - https://ve42.co/MT2012 
Soviet inspection image via Bulletin of the Atomic Scientists - https://ve42.co/Krzyzaniak2019  
Decommissioning nuclear weapon via ShareAmerica - https://ve42.co/Kaufman2014 


▀▀▀
Special thanks to our Patreon supporters:
Adam Foreman, Amadeo Bee, Anton Ragin, Balkrishna Heroor, Bernard McGee, Bill Linder, Burt Humburg, Dave Kircher, Diffbot, Evgeny Skvortsov, Gnare, Jesse Brandsoy, John H. Austin, Jr., john kiehl, Josh Hibschman, Juan Benet, KeyWestr, Lee Redden, Marinus Kuivenhoven, Mario Bottion, Max Maladino, Meekay, meg noah, Michael Krugman, Paul Peijzel, Richard Sundvall, Sam Lutfi, Stephen Wilcox, Tj Steyn, TTST, Ubiquity Ventures


▀▀▀
Directed by Casper Mebius
Written by Casper Mebius, Derek Muller, Petr Lebedev, and Ashley Hamer
Additional research and fact checking by Gregor Čavlović and Will Wood
Edited by Peter Nelson
Animated by Fabio Albertelli, Ivy Tello and Alondra Vitae
Illustrations by Jakub Misiek
Filmed by Derek Muller
Produced by Casper Mebius, Derek Muller, Gregor Čavlović and Han Evans

Additional video/photos supplied by Getty Images
Music from Epidemic Sound
Thumbnail by Peter Sheppard

This is a video about the most famous problem in game theory.

Problems of this sort pop up everywhere, from nations locked in conflict, to roommates doing the dishes.

Even game shows have been based around this concept.

Figuring out the best strategy can mean the difference between life and death, war and peace, flourishing and the destruction of the planet.

And in the mechanics of this game, we may find the very source of 1 of the most unexpected phenomena in nature, cooperation.

On the 3rd of September, 1949, an American weather monitoring plane collected air samples over Japan.

In those samples, they found traces of radioactive material.

The Navy quickly collected and tested rainwater samples from their ships and bases all over the world.

They also detected small amounts of cerium-141 and yttrium-91.

But these isotopes have half-lives of 1 or 2 months, so they must have been produced recently.

And the only place they could have come from was a nuclear explosion.

But the US hadn't performed any tests that year, so the only possible conclusion was that the Soviet Union had figured out how to make a nuclear bomb.

This was the news the Americans had been dreading.

Their military supremacy, achieved through the Manhattan Project, was quickly fading.

This makes the problem of Western Europe and the United States far more serious than it was before and perhaps makes the imminence of war greater.

Some thought their best course of action was to launch an unprovoked nuclear strike against the Soviets while they were still ahead.

In the words of Navy Secretary Matthews to become aggressors for peace.

John von Neumann, the founder of game theory, said, If you say, why not bomb them tomorrow?

If you say today at 05:00, I say why not at 01:00?

Something needed to be done about nuclear weapons and fast.

In 1950, the RAND Corporation, a US-based think tank, was studying this question.

And as part of this research, they turned to game theory.

That same year, 2 mathematicians at Rand had invented a new game, 1 which, unbeknownst to them at the time, closely resembled the US-Soviet conflict.

This game is now known as the Prisoner's Dilemma.

If you both cooperate, you each get 3 coins.

If 1 of you cooperates, but the other defects, then the 1 who defected gets 5 coins and the other gets nothing.

And if you both defect, then you each get a coin.

The goal of the game is simple, to get as many coins as you can.

Suppose your opponent cooperates, then you could also cooperate and get 3 coins.

Or you could defect and get 5 coins instead.

But what if your opponent defects instead?

Well, you could cooperate and get no coins or you could defect and at least get 1 coin.

So no matter what your opponent does, your best option is always to defect.

Now if your opponent is also rational, they will reach the same conclusion and therefore also defect.

As a result, When you both act rationally, you both end up in the sub-optimal situation getting 1 coin each when you could have gotten 3 instead.

In the case of the US and Soviet Union, this led both countries to develop huge nuclear arsenals of tens of thousands of nuclear weapons each, more than enough to destroy each other many times over.

But since both countries had nukes, neither could use them.

And both countries spent around $10 trillion developing these weapons.

Both would have been better off if they had cooperated and agreed not to develop this technology further.

But since they both acted in their own best interest, they ended up in a situation where everyone was worse off.

The Prisoner's Dilemma is 1 of the most famous games in game theory.

Thousands and thousands of papers have been published on versions of this game.

In part, that's because it pops up everywhere.

Impalas, living in between African woodlands and savannas are prone to catching ticks which can lead to infectious diseases, paralysis, even death.

So it's important for impalas to remove ticks and they do this by grooming.

But they can't reach all the spots on their bodies and therefore they need another impala to groom them.

Now, grooming someone else comes at a cost.

It costs saliva, electrolytes, time and attention, all vital resources under the hot African sun where a predator could strike at any moment.

So for the other impala, it would be best not to pay this cost.

But then again, it too will need help grooming.

In other words, should they cooperate or defect?

Well, if they only interact once, then the rational solution is always to defect.

That other Impala is never going to help you, so why bother?

But the thing about a lot of problems is that they're not a single prisoner's dilemma.

Impalas see each other day after day, and the same situation keeps happening over and over again.

Because instead of playing the prisoner's dilemma just once, you're now playing it many, many times.

And if I defect now, then my opponent will know that I defected, and they can use that against me in the future.

So what is the best strategy in this repeated game?

That is what Robert Axelrod, a political scientist, wanted to find out.

So in 1980, he decided to hold a computer tournament.

He invited some of the world's leading game theorists from many different subjects to submit computer programs that would play each other.

Axelrod called these programs strategies.

Each strategy would face off against every other strategy and against a copy of itself.

And each matchup would go for 200 rounds.

That's important and we'll come back to it.

Now Axelrod used points instead of coins, but the payoffs were the same.

The goal of the tournament was to win as many points as possible, and in the end, the whole tournament was repeated 5 times over, to ensure the success was robust and not just a fluke.

Axelrod gave an example of a simple strategy.

It would start each game by cooperating and only defect after its opponent had defected twice in a row.

In total, Axelrod received 14 strategies and he added a 15th called Random, which just randomly cooperates or defects 50% of the time.

All strategies were loaded onto a single computer, where they faced off against each other.

It starts off by cooperating, but if its opponent defects just once, it will keep defecting for the remainder of the game.

It also starts by cooperating, but then it just copies what the other player did on the last move.

Then around 10% of the time, Joss gets sneaky and defects.

There was also a rather elaborate strategy called Grass Camp.

This strategy works The same as Joss, but instead of defecting probabilistically, Grass Camp defects in the 50th round, to try and probe the strategy of its opponent and see if it can take advantage of any weaknesses.

The most elaborate strategy was name withheld, with 77 lines of code.

After all the games were played, the results were tallied up and the leaderboard established.

The crazy thing was that the simplest program ended up winning.

A program that came to be called tit for tat.

Tit for tat starts by cooperating and then it copies exactly what its opponent did in the last move.

So it would follow cooperation with cooperation and defection with defection.

But only once if its opponent goes back to cooperating.

When tit for tat played against Friedman, both started by cooperating, and they kept cooperating, both ending up with perfect scores for complete cooperation.

When tit-for-tat played against Joss, they too started by cooperating, but then on the sixth move, Joss defected.

This sparked a series of back and forth defections, a sort of echo effect.

Okay, so now you've got this alternating thing, which will remind you of some of the politics of the world today, where we have to do something to you because of what you did to us.

And then when this weird program throws in a second unprovoked defection, now it's really bad, because now both programs are going to defect on each other for the rest of the game.

And that's also like some of the things that we're seeing in politics today and in international

As a result of these mutual retaliations, both tit-for-tat and Joss did poorly.

But because tit-for-tat managed to cooperate with enough other strategies, it still won the tournament.

I imagined initially it'd be sort of like computer chess where you need a pretty

Axelrod found that all the best performing strategies, including tit-for-tat, shared 4 qualities.

First, they were all nice, which just means they are not the first to defect.

So tit for tat is a nice strategy, it can defect, but only in retaliation.

The opposite of nice is nasty, that's a strategy that defects first.

Out of the 15 strategies in the tournament 8 were nice and 7 nasty The top 8 strategies were all nice and even the worst performing nice strategy still far outscored the best performing nasty 1 The second important quality was being forgiving.

A forgiving strategy is 1 that can retaliate, but it doesn't hold a grudge.

It retaliates when its opponent defects, but it doesn't let defections from before the last round influence its current decisions.

Friedman, on the other hand, is maximally

that it pays to be nice and forgiving came as a shock to the experts.

Many had tried to be tricky and create subtle nasty strategies to beat their opponent and eke out an advantage.

Instead, in this tournament, Nice Guys finished first.

Now, tit-for-tat is quite forgiving, but it's possible to be even more forgiving.

Axelrod's sample strategy only defects after its opponent defected twice in a row.

Now that might sound overly generous, but when Axelrod ran the numbers, he found that if anyone had submitted the sample strategy, they would have won the tournament.

After Axelrod published his analysis of what happened, or circulated it among these game theorists, he said, now that we all know what worked well, let's try again.

So he announced a second tournament where everything would be the same except for 1 change, the number of rounds per game.

See, in the first tournament, each game in the last round is, then there's no reason to cooperate in that round.

Of course, your opponent should reason the same, and so they should also defect in the last round.

But if you both anticipate defection in the last round, then there's no reason for you to cooperate in the second to last round, or the round before that, or before that, and so on all the way to the very first round.

And so in Axelrod's tournament, it was a very important thing that the players didn't know exactly how long they were going to be playing.

They knew on average it would be 200 rounds, but there was a random number generator that prevented them from knowing with certainty.

Yeah, if you're not sure when it ends, then you have to kind of keep cooperating because it might keep going and you need might need them on your side.

For this second tournament Axelrod received 62 entries and again added random.

The contestants had gotten the results and analysis from the first tournament and could use this information to their advantage.

Some thought that clearly being nice and forgiving were winning qualities, so they submitted nice and forgiving strategies.

The second camp anticipated that others would be nice and extra forgiving.

Therefore they submitted nasty strategies to try to take advantage of those that were extra forgiving.

It would defect on the first move to see how its opponent reacted.

If it retaliated, Tester would apologize and play tit-for-tat for the remainder of the game.

If it didn't retaliate, Tester would defect every other move after that.

And once again, tit-for-tat was the most effective.

Similarly, in the bottom 15, only After the second tournament, Axelrod identified the other qualities that distinguish the better performing strategies.

The third is being retaliatory, which means if your opponent defects, strike back immediately.

Always cooperate is a total pushover, and so it's very easy to take advantage of.

Tit for tat, on the other hand, is very hard to take advantage of.

The last quality that Axelrod identified is being clear.

Programs that were too opaque, that were too similar to a random program, you couldn't figure them out because they were so complicated.

It was very hard to establish any pattern of trust with a program like that because you couldn't figure out what it was doing.

I mean, the other programs it was playing couldn't figure them out.

And so they would end up more or less defaulting to thinking every turn is like the last time I'm going to see you, so I might as well defect.

What to me is mind-blowing about this is that these 4 principles being nice, forgiving, provocable, and clear is a lot like the morality that has evolved around the world that is often summarized as an eye for an eye.

It's not turn-the-other-cheek philosophy.

Tournament, it only came 24th in the second tournament.

In the repeated prisoner's dilemma, there is no single best strategy.

The strategy that performs best always depends on the other strategies it's interacting with.

For example, if you put tit for tat in an environment with only the ultimate bullies of always defect, then tit for tat comes in last.

I wanted to see whether, for example, that tit for tat did well because it did well with with really stupid rules that didn't do well at all themselves, that basically took advantage of people.

So he ran a simulation where successful strategies in 1 generation would see their numbers grow and unsuccessful ones would see their numbers drop.

In this simulation, the worst performing strategies quickly shrink and go extinct, while the top performing strategies become more common.

Harrington, the only nasty strategy in the top 15, first grew quickly.

But then as the strategies it was preying on went extinct, Harrington's numbers also quickly dropped.

This shows a main benefit of this simulation because it tests how well a strategy does with other successful strategies.

After a thousand generations, the proportions are mostly stable and only nice strategies survive.

Again, tit for tat comes out on top, representing 14.5% of the total population.

Now, this process may sound similar to evolution, but there is a subtle difference, which is that in this case there are no mutations.

So it's actually an ecological simulation.

But what if the world you started in was different?

Except there's a little cluster of tit-for-tat players that live in some kind of nucleus, and they get to play with each other a lot because they're geographically sequestered.

They will start building up a lot of points, and also, because that translates into offspring, they'll start to take over the population.

So in fact Axelrod showed that a little island of cooperation can emerge and spread and eventually will take over the world, which is fantastic.

How can cooperation emerge in a population of players who are self-interested, who are not trying to be good because they're good-hearted.

You could be looking out for number 1 for yourself and your own interests, and yet cooperation can still emerge.

Some argue that this could explain how we went from a world full of completely selfish organisms, where every organism only cared about themselves, to 1 where cooperation emerged and flourished.

From impalas grooming each other to fish cleaning sharks.

Many life forms experience conflicts similar to the prisoner's dilemma, but because they don't interact just once, both can be better off by cooperating.

And this doesn't require trust or conscious thought either, because the strategy could be encoded in DNA.

As long as it performs better than the other strategies, it can take over a population.

But there was 1 aspect that his original tournaments didn't cover.

What happens if there is a little bit of random error in the game, some noise in the system?

For example, 1 player tries to cooperate, but it comes across as a defection.

Little errors like this happen in the real world all the time.

Like in 1983, the Soviet satellite-based early warning system detected the launch of an intercontinental ballistic missile from the US.

The Soviet system had confused sunlight reflecting off high-altitude clouds with a ballistic missile.

Thankfully, Stanislav Petrov, the Soviet officer on duty, dismissed the alarm.

But this example shows the potential costs of a signal error, and the importance of studying the effects of noise on those strategies.

And as you mentioned that this came up in the Cold War, I mean, it could actually be life and death of the whole planet, we could annihilate human civilization.

So these are not games in any kind of trivial sense.

It's just the term that is used by mathematicians and theorists.

When tit for tat plays against itself in a noisy environment, both start off by cooperating.

But if a single cooperation is perceived as a defection, then the other tit-for-tat retaliates, and it sets off a chain of alternating retaliations.

And if another cooperation is perceived as a defection, then the rest of the game is constant mutual defection.

Therefore, in the long run, both would only get a third of the points they would get in a perfect environment.

Tit for tat goes from performing very well to performing poorly.

Well, you need a reliable way to break out of these echo effects.

And 1 way to do this is by playing tit for tat, but with around 10% more forgiveness.

So instead of retaliating after every defection, you only retaliate around 9 out of every 10 times.

This helps you break out of those echoes while still being retaliatory enough to not be taken advantage of.

And so we also ran the tournament with with noise and generosity.

My favorite example is tit for tat does really well but it could never do better than the player it's playing with.

By design, all they can do is lose or draw.

And yet when the results of all interactions are tallied up, they come out ahead of all other strategies.

Similarly, always defect can never lose a game.

But overall, it performs extremely poorly.

Because for many people, when they think about winning, they think they need to beat the other person.

In games like chess or poker, this is true, since 1 person's gain is necessarily another person's loss.

To win, you don't need to get your reward from the other player.

Only in real life, the banker is the world.

It is just up to us to find those win-win situations and then work together to unlock those rewards.

From 1950 to 1986, the US and Soviet Union had trouble cooperating, and both kept developing nukes.

But then, from the late 80s onwards, they started reducing their nuclear stockpiles.

They too had learned how to resolve conflict Rather than making an agreement to abolish all nuclear arms at once and essentially turning it into a single prisoners dilemma they would disarm slowly a small number of nukes each year, and then they'd check each other to see that they had both cooperated, and then repeat the year after, and the year after that, all along checking to ensure mutual cooperation.

In the more than 40 years since Axelrod's tournaments, researchers have continued to study which strategies perform best in a variety of environments.

In doing so, they varied everything from payoff structures to strategies to errors and more.

Some even allowed the strategies to mutate.

While tit for tat or generous tit for tat doesn't always come out on top, Axelrod's main takeaways still hold.

Be nice, forgiving, but don't be a pushover.

Can I ask you, why did Anatole Rappaport submit tit for tat?

Well, the reason was because I asked him to.

See all Veritasium transcripts on Youtube

What Game Theory Reveals About Life, The Universe, and Everything