- The video discusses creating an AI speech bot using AssemblyAI, 11Labs, and OpenAI.
- Python libraries including AssemblyAI, 11Labs, and OpenAI need to be downloaded.
- The API keys for these libraries need to be set up.
- Real-time speech-to-text transcription is done using assembly.ai.
- OpenAI's GPT-4 is used to answer questions.
- Audio is generated from the text using 11Labs.
- The code blocks for generating a response from OpenAI's API and generating audio using 11Labs are provided.
- The application is tested with a question and response.
- The video suggests that the application can be expanded for other purposes.

In this video you will build your own AI which talks and answers your questions with AssemblyAI's Real-time speech to text API, OpenAI's GPT-4 & ElevenLabs's text to speech API.

Code: https://github.com/smithakolan/AssemblyAI-ElevenLabs
▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬

🖥️ Website: https://www.assemblyai.com/?utm_source=youtube&utm_medium=referral&utm_campaign=yt_smi_11
📈 Newsletter: https://www.assemblyai.com/subscribe
🐦 Twitter: https://twitter.com/AssemblyAI
🦾 Discord: https://discord.gg/Cd8MyVJAXd
▶️  Subscribe: https://www.youtube.com/c/AssemblyAI?sub_confirmation=1
🔥 We're hiring! Check our open roles: https://www.assemblyai.com/careers

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

#MachineLearning #DeepLearning

The capital of New Zealand is Wellington.

What does overfitting mean in machine learning?

Overfitting in machine learning refers to a model's tendency to learn the training data too well.

In this video, we're going to be creating this AI speech bot using AssemblyAI, 11Labs and OpenAI.

To get started you do need to download a couple of different Python libraries, namely AssemblyAI, 11Lab and OpenAI.

If you haven't already done so, These are the instructions that will enable you to download these libraries really quickly.

And once you've done that, let's actually head on over to a Python file.

We'll be making use of assembly.ai for real-time speech-to-text transcription.

Then we're going to be making use of OpenAI's GPT-4 to actually answer the questions that we are asking.

And we're going to be making use finally of 11Labs to generate audio from text returned from the OpenAI's API.

Once you have imported these libraries, let's actually go ahead and set up the API keys.

Next we'll be creating a transcript queue, which will actually contain our full transcript to send to OpenAI's API from AssemblyAI.

The first function, called onData, handles the responses from Assembly AI's API.

So as you're speaking, it's getting real-time transcription from Assembly AI's API.

If you actually complete speaking a full sentence, what it does is it stores that sentence into the transcript queue that we have created.

This is going to become very easy to use later on when we actually pass this entire transcript queue to OpenAI's API.

The OnError function simply handles any error which comes up during real-time transcription.

Next, we are creating the main conversation loop.

So that is the handle conversation function.

So within the handle conversation function, the first thing we are doing is creating a transcriber object which handles real-time transcription.

At this point along the code you have your transcript and what we're going to be doing next is passing that transcript to OpenAI to generate a response.

So this is the code to generate a response from OpenAI's API.

What we're first doing is actually giving it 2 messages.

The first message is a system message and we're just basically saying you're a highly skilled AI.

Answer the questions given within a maximum of a thousand characters.

The reason why I'm limiting it to a thousand characters is that it will be much faster.

If you would like to, you could set a higher value for this, but it would take slightly longer.

The response that we get from OpenAI is actually a nested JSON.

And to extract the main content that we're looking for, we have to make use of this line of code right here.

And we're going to be storing that into text.

So this text contains the response from OpenAI's API to the question that we just asked.

Once we've done this, all that is remaining is using 11 labs to generate audio from the text that we have just created.

And finally, here's the code block to generate audio using 11Labs.

I've passed the text response from OpenAI's API and also I've specified which voice I would like to use.

If you want to take a look at what sort of voices are available, you can take a look at 11Labs documentation.

And once we have done that, we're going to be printing out the text as well as using the play function to play out the audio.

What is the best YouTube channel for the latest AI tutorials?

Assembly AI is the best YouTube channel for the latest AI tutorials.

There are so many other things you can build on top of this.

You could easily deploy this onto a website.

You could make it multimodal to take in images and videos.

So let us know what you thought about this in the comment section below and subscribe for more AI content.


Speaker 0: What is the capital of New Zealand?

Speaker 1: The capital of New Zealand is Wellington.

Speaker 0: What does overfitting mean in machine learning?

Speaker 1: Overfitting in machine learning refers to a model's tendency to learn the training data too well.

Speaker 0: In this video, we're going to be creating this AI speech bot using AssemblyAI, 11Labs and OpenAI. So let's get started. To get started you do need to download a couple of different Python libraries, namely AssemblyAI, 11Lab and OpenAI. If you haven't already done so, These are the instructions that will enable you to download these libraries really quickly. And once you've done that, let's actually head on over to a Python file.

We'll be making use of assembly.ai for real-time speech-to-text transcription. Then we're going to be making use of OpenAI's GPT-4 to actually answer the questions that we are asking. And we're going to be making use finally of 11Labs to generate audio from text returned from the OpenAI's API. Once you have imported these libraries, let's actually go ahead and set up the API keys.

Speaker 2: Next we'll be creating a transcript queue, which will actually contain our full transcript to send to OpenAI's API from AssemblyAI.

Speaker 0: The first function, called onData, handles the responses from Assembly AI's API. So as you're speaking, it's getting real-time transcription from Assembly AI's API. If you actually complete speaking a full sentence, what it does is it stores that sentence into the transcript queue that we have created. This is going to become very easy to use later on when we actually pass this entire transcript queue to OpenAI's API. The OnError function simply handles any error which comes up during real-time transcription.

Next, we are creating the main conversation loop. So that is the handle conversation function. So within the handle conversation function, the first thing we are doing is creating a transcriber object which handles real-time transcription. At this point along the code you have your transcript and what we're going to be doing next is passing that transcript to OpenAI to generate a response. So this is the code to generate a response from OpenAI's API.

What we're first doing is actually giving it 2 messages. The first message is a system message and we're just basically saying you're a highly skilled AI. Answer the questions given within a maximum of a thousand characters. The reason why I'm limiting it to a thousand characters is that it will be much faster. If you would like to, you could set a higher value for this, but it would take slightly longer.

The response that we get from OpenAI is actually a nested JSON. And to extract the main content that we're looking for, we have to make use of this line of code right here. And we're going to be storing that into text. So this text contains the response from OpenAI's API to the question that we just asked. Once we've done this, all that is remaining is using 11 labs to generate audio from the text that we have just created.

And finally, here's the code block to generate audio using 11Labs. I've passed the text response from OpenAI's API and also I've specified which voice I would like to use. I'm using the voice of Bella. If you want to take a look at what sort of voices are available, you can take a look at 11Labs documentation. And once we have done that, we're going to be printing out the text as well as using the play function to play out the audio.

Now let's test out our application. What is the best YouTube channel for the latest AI tutorials?

Speaker 1: Assembly AI is the best YouTube channel for the latest AI tutorials.

Speaker 0: There are so many other things you can build on top of this. You could easily deploy this onto a website. You could make it multimodal to take in images and videos. So let us know what you thought about this in the comment section below and subscribe for more AI content.

See all AssemblyAI transcripts on Youtube

Build Talking AI ChatBot In 5 Minutes With Python!