Let's examine the developments this week in the world of AI through the lens of predictions made just two years ago. It is hard to contextualise just how crazy the times we are living in without seeing what people thought would be happening now.

From the Bard upgrade in vision and spoken language, to Claude 2, Mustafa Suleyman's latest predictions for Inflection AI to xAI reported promises of $200 million to those leaving DeepMind and OpenAI, this brief video tries to cover a lot. And not to forget the FTC's exhaustive investigation OpenAI and Sam Altman's response on twitter.

A Brief History of AI: https://www.amazon.co.uk/Brief-History-Artificial-Intelligence-Where/dp/1250770742/ref=sr_1_1?crid=VCZ0CVPTUSO9&keywords=A+brief+history+of+AI&qid=1689587065&sprefix=a+brief+history+of+ai%2Caps%2C76&sr=8-1
Bard: https://bard.google.com/
PaLM 2 Technical Report: https://arxiv.org/pdf/2305.10403.pdf
Bard Languages: https://support.google.com/bard/answer/13575153?hl=en
PaLI: https://sites.research.google/pali/
GPT 4 Story: https://www.reddit.com/r/ChatGPT/comments/120oq1x/i_asked_gpt4_to_write_a_book_the_result_echoes_of/
GPT 4 RC: https://openai.com/research/gpt-4
Let's Verify Step by Step: https://arxiv.org/pdf/2305.20050.pdf
FTC Investigation: https://www.washingtonpost.com/technology/2023/07/13/ftc-openai-chatgpt-sam-altman-lina-khan/
Sam Altman Tweet: https://twitter.com/sama/status/1679602638562918405?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet
2021 MATH Forecast: https://bair.berkeley.edu/blog/2021/10/14/forecasting/
Metaculus AGI: https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/
Claude 2: https://poe.com/Claude-2-100k
Million Dollar AI: https://www.technologyreview.com/2023/07/14/1076296/
I, Robot Clip: https://www.youtube.com/watch?v=siHfHUm3HGE
xAI $200 million: https://www.semafor.com/article/07/14/2023/elon-musk-threw-nine-figure-promises-at-top-ai-researchers

https://www.patreon.com/AIExplained

People are starting to get used to the breakneck pace of AI, so I wanted to show that developments in just the last few days fly in the face of what was predicted only a couple of years ago.

Starting with the upgrades to BARD, then a snapshot of Claude II, the all-encompassing open AI investigation and inflection AI's predictions of a self-guided, million-dollar making AI coming in just 2 years.

But I'm going to start with this page from a book released in 2021 called A Brief History of AI.

Look at the tasks it says are nowhere near solved.

And at the bottom it says at present we have no idea about how to get computers to do the tasks at the bottom of the list.

I'm not arguing that these are solved, but this year we are getting pretty close.

Check out the second 1 down, human-level automated translation.

I asked the new bard to write a poem about machine translation and here it is.

But then I asked, now translate it into Spanish.

Of course, that's nothing new, but listen to the quality of the text-to-speech model used to read out this poem.

I don't know about you, but that is starting to sound awfully human like.

Now yes, it could do this before for English but now it can do it for even Swahili.

And I know some people will say don't we already have Google Translate?

But Palm 2, which is the model behind BARD, outperforms Google Translate.

I covered this in the original Palm 2 video, but the conclusion was, we observe that Palm 2 improves quality both over the original Palm and Google Translate.

Okay, that is pretty insane, but what about the next 1?

Interpreting what is going on in a photograph.

Well, I gave Bard the meme that you can see and I said, interpret what's going on and it said this.

The image shows a pizza that has been shaped to look like the Death Star.

Already, that is so savvy to me that it knows it's a pizza despite it being so strangely formed and it can interpret that the toppings make it look like the Death Star.

Of course, as a bonus, it can read the text and therefore understand the meme.

It says it's humorous because the death star is a symbol of death and destruction compared to a pizza which is about food and enjoyment.

A quick bonus tip by the way is that you can scroll to the end of the response and click this modify response button and then you can adjust the output.

For example, I'm going to make it do something shorter and you are going to see a shorter version of this interpretation.

And here it is apparently reduced by about 25%.

Now that Google Lens is incorporated into BARD, I use it daily on my walks to answer questions about things around me.

Like maybe I see a butterfly and I ask what type of butterfly it is or a plant.

And 1 thing to caution you on is that if it sees a human face, it will block that out and not answer the question.

And 1 more crazy thing that I found that I'm curious if anyone else found this is that I took a photo in my local park and it actually recognized the location of the park and it was not a nationally let alone internationally recognized park at all.

Now that didn't work every time but it is something you might want to try on your next walk or run.

And you might have noticed that that's quite similar to the 1 second from last which is interpreting a work of art.

Now again I'm not saying that's solved and there may be some reverse image search going on here.

I asked, write a haiku about where the stairs are going in this work of art.

Famously, this sketch is about the stairs going nowhere by Escher.

And it wrote an almost perfect haiku about the fact that the stairs are going nowhere.

Let's now take a break from that book and look at what professional forecasters predicted that AI would be capable of for the math dataset.

For this particular benchmark in 2021 they predicted a score of 21% in 2023 and 52% in 2025.

And the predicted trend would hit 80% only in 2028, 5 years from now, 7 years from then.

Well, regular viewers of my channel might know that GPT-4 can already get 78% of problems from that dataset correct already today.

And you can see here that there is still quite a lot of room for further improvement.

And honestly, this is without Code Interpreter or Wolfram Alpha.

So I actually ran hundreds of experiments of my own using GPT-4 with Code Interpreter on the math dataset and it was getting more like 86% correct.

But back to predictions from 2021, notice 2 of the other categories that are apparently nowhere near solved.

Understanding a story and answering questions about it and writing an interesting story.

Here is a 112 page novel written fully by GPT-4.

Now I think it meets the definition of being interesting if not human level.

But honestly, when a model can be fine-tuned on an author's work, I think it will get very, very close.

My own prediction is that we are less than a year away from a human level novel.

Not As good as the best of course, but fooling many readers.

But here is where I can explore a bit with Claude II which can take in 100, 000 tokens.

I was lucky to get early access to Claude II and ran hundreds of my own experiments which I will talk about more in future videos.

But for this video, remember the book's predictions about answering questions about such a novel or story.

Well, I inputted that entire GPT-4 generated novel and I said, find 10 sentences whose vocabulary could be made less generic.

And if you look at the suggestions from Claude too, it does indeed make the vocabulary more exciting.

Definitely not perfect, still quite generic, but we have words like crystalline, ethereal and inoxorable.

Again, I think we're less than a year from a high quality full-length novel being produced using 1 prompt.

And if you wanted a more technical test of reading comprehension, don't forget that GPT-4 got 99th percentile in the verbal section of the GRE, which includes reading comprehension.

I managed to get 170 on this test and to be honest when I saw GPT-4 get 99th percentile that was a huge wake-up call for me.

A true story actually is that when I saw that result I decided to make covering AI my full-time job.

And I think each person will go through their own such moment when GPT-4 or GPT-5 or Gemini crosses some benchmark to do with their profession.

It's like I didn't wake up much when it could create basic art because I'm not an artist.

But when these models come onto territory that you yourself know about, that's when you realize how powerful they are.

And I think a lot of people have gone through that journey.

This is Metaculous, if that's how you pronounce it, and their predictions about AGI.

And look what the predictions were 2 years ago, around the time of that book.

And this was as late as October or November of 2021.

You can see that they thought AGI would be 20 or more years away.

That is quite a change in 2 years and I can understand why.

Mustafa Suleiman, the founder of Inflection AI, goes further.

They are building towards their own AGI and they plan to ask it to go make 1 million dollars.

Of course it would need a strategy, it would need to research and design products, etc.

Maybe with a little bit of human approval, but the work would all be done by an AI.

And he says something like this could be as little as 2 years away.

And those of you who watched my previous video on superintelligence will find that quite a contrast to the idea that superintelligence is 10 to 20 years away.

I don't know about you guys but I think if an AI can make money in this way, the entire world economy will be transformed rapidly.

There is a curious possibility though that such an AI wouldn't be released to the general public.

Of course, I was following the FTC investigation into OpenAI.

As you might expect, I read all 20 pages of the investigation multiple times.

In fact, Sam Altman openly grumbled that the FTC request started with this leak, enabling me to read about it.

It is actually a crazy document and it feels to me a bit like some sort of strip search.

Like they want all internal communications about the propensity of ChattyPT to produce inaccurate statements or reveal personal information.

And they say at the top, do not delete any emails relating to this.

In other videos, I've covered how any model can be jailbroken.

And I've also covered how Sam Altman said that it might be 2 years before models stop hallucinating.

So if the FTC ends up fining OpenAI or the other companies billions of dollars as they've fined other companies before.

I can see 1 result being much more reticence from these companies over publicly deploying their models.

Of course, let me know in the comments whether you think that's a good thing or a bad thing.

Speaking of money though, I can't leave out that this week was the start of xAI.

Elon Musk was able to promise up to $200 million as a signing bonus for some of the AI researchers that joined him.

If companies are willing to promise that much to researchers willing to defect from deep mind or open AI, I do find it hard to see the march to superintelligence slowing down much.

So let's end with that famous scene from iRobot, although perhaps this time with a different ending.

Can a robot turn a canvas into a beautiful masterpiece?

Thank you so much for watching and have a wonderful day.


Speaker 0: People are starting to get used to the breakneck pace of AI, so I wanted to show that developments in just the last few days fly in the face of what was predicted only a couple of years ago. Starting with the upgrades to BARD, then a snapshot of Claude II, the all-encompassing open AI investigation and inflection AI's predictions of a self-guided, million-dollar making AI coming in just 2 years. But I'm going to start with this page from a book released in 2021 called A Brief History of AI. Look at the tasks it says are nowhere near solved. And at the bottom it says at present we have no idea about how to get computers to do the tasks at the bottom of the list.

I'm not arguing that these are solved, but this year we are getting pretty close. Check out the second 1 down, human-level automated translation. I asked the new bard to write a poem about machine translation and here it is. I'm not going to judge the poem. But then I asked, now translate it into Spanish.

Of course, that's nothing new, but listen to the quality of the text-to-speech model used to read out this poem. Here's a snippet. I don't know about you, but that is starting to sound awfully human like. Now yes, it could do this before for English but now it can do it for even Swahili. And I know some people will say don't we already have Google Translate?

But Palm 2, which is the model behind BARD, outperforms Google Translate. I covered this in the original Palm 2 video, but the conclusion was, we observe that Palm 2 improves quality both over the original Palm and Google Translate. Okay, that is pretty insane, but what about the next 1? Interpreting what is going on in a photograph. Are we nowhere near solving that?

Well, I gave Bard the meme that you can see and I said, interpret what's going on and it said this. The image shows a pizza that has been shaped to look like the Death Star. Already, that is so savvy to me that it knows it's a pizza despite it being so strangely formed and it can interpret that the toppings make it look like the Death Star. Of course, as a bonus, it can read the text and therefore understand the meme. It says it's humorous because the death star is a symbol of death and destruction compared to a pizza which is about food and enjoyment.

A quick bonus tip by the way is that you can scroll to the end of the response and click this modify response button and then you can adjust the output. For example, I'm going to make it do something shorter and you are going to see a shorter version of this interpretation. And here it is apparently reduced by about 25%. Now that Google Lens is incorporated into BARD, I use it daily on my walks to answer questions about things around me. Like maybe I see a butterfly and I ask what type of butterfly it is or a plant.

And 1 thing to caution you on is that if it sees a human face, it will block that out and not answer the question. And 1 more crazy thing that I found that I'm curious if anyone else found this is that I took a photo in my local park and it actually recognized the location of the park and it was not a nationally let alone internationally recognized park at all. Now that didn't work every time but it is something you might want to try on your next walk or run. And you might have noticed that that's quite similar to the 1 second from last which is interpreting a work of art. Now again I'm not saying that's solved and there may be some reverse image search going on here.

I asked, write a haiku about where the stairs are going in this work of art. Famously, this sketch is about the stairs going nowhere by Escher. And it wrote an almost perfect haiku about the fact that the stairs are going nowhere. Let's now take a break from that book and look at what professional forecasters predicted that AI would be capable of for the math dataset. For this particular benchmark in 2021 they predicted a score of 21% in 2023 and 52% in 2025.

And the predicted trend would hit 80% only in 2028, 5 years from now, 7 years from then. Well, regular viewers of my channel might know that GPT-4 can already get 78% of problems from that dataset correct already today. And you can see here that there is still quite a lot of room for further improvement. And honestly, this is without Code Interpreter or Wolfram Alpha. So I actually ran hundreds of experiments of my own using GPT-4 with Code Interpreter on the math dataset and it was getting more like 86% correct.

But back to predictions from 2021, notice 2 of the other categories that are apparently nowhere near solved. Understanding a story and answering questions about it and writing an interesting story. Here is a 112 page novel written fully by GPT-4. Now I think it meets the definition of being interesting if not human level. But honestly, when a model can be fine-tuned on an author's work, I think it will get very, very close.

My own prediction is that we are less than a year away from a human level novel. Not As good as the best of course, but fooling many readers. But here is where I can explore a bit with Claude II which can take in 100, 000 tokens. I was lucky to get early access to Claude II and ran hundreds of my own experiments which I will talk about more in future videos. But for this video, remember the book's predictions about answering questions about such a novel or story.

Well, I inputted that entire GPT-4 generated novel and I said, find 10 sentences whose vocabulary could be made less generic. And if you look at the suggestions from Claude too, it does indeed make the vocabulary more exciting. Definitely not perfect, still quite generic, but we have words like crystalline, ethereal and inoxorable. I love that word myself. 1 of my favorites.

Again, I think we're less than a year from a high quality full-length novel being produced using 1 prompt. And if you wanted a more technical test of reading comprehension, don't forget that GPT-4 got 99th percentile in the verbal section of the GRE, which includes reading comprehension. I managed to get 170 on this test and to be honest when I saw GPT-4 get 99th percentile that was a huge wake-up call for me. A true story actually is that when I saw that result I decided to make covering AI my full-time job. And I think each person will go through their own such moment when GPT-4 or GPT-5 or Gemini crosses some benchmark to do with their profession.

It's like I didn't wake up much when it could create basic art because I'm not an artist. But when these models come onto territory that you yourself know about, that's when you realize how powerful they are. And I think a lot of people have gone through that journey. This is Metaculous, if that's how you pronounce it, and their predictions about AGI. And look what the predictions were 2 years ago, around the time of that book.

They were late 2030s, even early 2040s. And this was as late as October or November of 2021. You can see that they thought AGI would be 20 or more years away. What do they think now? Well, 2026.

That is quite a change in 2 years and I can understand why. Mustafa Suleiman, the founder of Inflection AI, goes further. They are building towards their own AGI and they plan to ask it to go make 1 million dollars. Of course it would need a strategy, it would need to research and design products, etc. Etc.

Maybe with a little bit of human approval, but the work would all be done by an AI. And he says something like this could be as little as 2 years away. And those of you who watched my previous video on superintelligence will find that quite a contrast to the idea that superintelligence is 10 to 20 years away. I don't know about you guys but I think if an AI can make money in this way, the entire world economy will be transformed rapidly. There is a curious possibility though that such an AI wouldn't be released to the general public.

Of course, I was following the FTC investigation into OpenAI. As you might expect, I read all 20 pages of the investigation multiple times. In fact, Sam Altman openly grumbled that the FTC request started with this leak, enabling me to read about it. It is actually a crazy document and it feels to me a bit like some sort of strip search. Like they want all internal communications about the propensity of ChattyPT to produce inaccurate statements or reveal personal information.

And they say at the top, do not delete any emails relating to this. In other videos, I've covered how any model can be jailbroken. And I've also covered how Sam Altman said that it might be 2 years before models stop hallucinating. So if the FTC ends up fining OpenAI or the other companies billions of dollars as they've fined other companies before. I can see 1 result being much more reticence from these companies over publicly deploying their models.

Of course, let me know in the comments whether you think that's a good thing or a bad thing. Speaking of money though, I can't leave out that this week was the start of xAI. Elon Musk was able to promise up to $200 million as a signing bonus for some of the AI researchers that joined him. If companies are willing to promise that much to researchers willing to defect from deep mind or open AI, I do find it hard to see the march to superintelligence slowing down much. So let's end with that famous scene from iRobot, although perhaps this time with a different ending.

An imitation of life. Can a robot write a symphony? Can a robot turn a canvas into a beautiful masterpiece? Klar, ich kann. Kannst du auch?

Thank you so much for watching and have a wonderful day.

See all AI Explained transcripts on Youtube

Bad AI Predictions: Bard Upgrade, 2 Years to AI Auto-Money, OpenAI Investigation and more