Curious about OpenAI's newly-announced fine-tuning API?

In this brief video, I'll showcase how to apply fine-tuning to OpenAI's GPT 3.5 model, using an obscure bank reconciliation file format as an example. Discover how this API empowers developers to enhance the model's capabilities for specific tasks.

The gpt-fine-tuner tool is available here: https://www.npmjs.com/package/gpt-fine-tuner

The fine-tuning announcement can be found here:
https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates

00:00 Demo
00:16 Intro
00:35 What is fine tuning?
1:11 Fine-tuning process
3:13 Another Demo
4:06 Summary

GPT 3.5 doesn't know this file format, so when I ask it to parse it, it just returns a guess.

The amounts are wrong, the data isn't formatted well, the types are wrong, and it thinks some of the non-transaction lines are transactions.

With a fine-tuned model, it actually does the right thing.

My name is Christian, and this is Code and Stuff, a place where I write code and do stuff.

OpenAI just recently announced the availability of fine-tuning in GPT 3.5.

In this video, I'm going to demonstrate a real fine-tuning job, and I'll be using a new command line tool I built just for this called GPT FineTuner.

Large language models are trained to be general-purpose helpers, but don't specialize in any field.

Sometimes there's just personality nuance that needs to be taught, while other times there are prompts that are too tricky to write or formats that hasn't been exposed to.

By giving a model some examples, we can show it how we want it to act.

This updates the weights and lets it learn something new.

For this demo, I'm going to teach you how to parse a file format called BAI2.

It's what banks use to communicate transaction settlements with each other.

But the same approach could be used for any fine-tuning use case.

The fine-tuning process is pretty simple.

We give the API a file that looks like this, where each line represents a message, which could be a system prompt, it could be a user input, or it could be the assistance response.

I've constructed 10 examples where this file format is properly parsed.

10 examples are the minimum for the OpenAI fine-tuning API.

I expanded 1 of these just to show what it looks like.

So I have my system prompt, parse the provided file, summarize it with these fields.

In the user field, I say here's the file, and in the response I show what I want the output to look like.

I'll take this file and turn it into a fine-tuning job.

That's where the command line utility comes in.

This job will run for a few minutes and at the end, a new model becomes available in the Playground and in the API.

I give it this JSONL file and I tell it that I want the suffix of the name of the model to be BAI2 just so I can keep track of it better in the playground.

It's going to upload the training file, and then we have to wait for them to do some pre-processing on the file.

Now that the pre-processing is done, the fine-tuning job has been created.

So I'll take this fine-tuning job ID and run npx gpt fine-tuner show, and then that job ID.

And we can see that it created the fine-tuned job but it hasn't actually run yet.

We can also run gpt fine tuner ls to show all of the jobs in our account.

In this case I have 1 that I ran earlier and the 1 that is currently running.

So I'm gonna let this wait a few minutes and then show the progress that it makes along the way.

After a few minutes it looks like there's been some progress on the model.

Its training loss has gone down and its accuracy has gone up.

It looks like the job finished in about 10 minutes, and now we have a new model ID.

I've got a brand new BAI2 file right here, and I'm going to select our brand new model, BAI2, and let's run it.

And just like that it's extracting the fields properly.

It understands that the amounts are number of cents, it understands there's a currency, it understands that the sender and receiver fields exist, and it's not trying to create transactions out of lines that aren't transactions.

We've actually taught it how to read a BAI2 file, which is incredible.

If we compare that to the baseline 3.5 model, which is just making some guesses, we can see that it truly is way better.

In the standard 3.5 model, it's just trying to infer based on some context cues, but it doesn't really understand what's going on.

Meanwhile in the fine-tuned model, we actually get real results.

Fine-tuning a model can give it new skills or help you show a solution instead of shoving a description into the limited context.

Fine-tuning has a per 1000 token cost, and inference with a fine-tuned model currently costs about 8 times as much as a generic GPT-3.5.

Let me know how you end up fine-tuning GPT-3.5.

And don't forget to do the YouTube stuff, liking, subscribing, and leaving comments.

Your feedback will help me share more things in the future.

This has been Code and Stuff, thank you for watching!


Speaker 0: GPT 3.5 doesn't know this file format, so when I ask it to parse it, it just returns a guess.

Speaker 1: The amounts are wrong, the data isn't formatted well, the types are wrong, and it thinks some of the non-transaction lines are transactions. With a fine-tuned model, it actually does the right thing. My name is Christian, and this is Code and Stuff, a place where I write code and do stuff. OpenAI just recently announced the availability of fine-tuning in GPT 3.5. In this video, I'm going to demonstrate a real fine-tuning job, and I'll be using a new command line tool I built just for this called GPT FineTuner.

Link in the description. But first, what is fine-tuning? Large language models are trained to be general-purpose helpers, but don't specialize in any field. Sometimes there's just personality nuance that needs to be taught, while other times there are prompts that are too tricky to write or formats that hasn't been exposed to. This is where fine-tuning comes in.

By giving a model some examples, we can show it how we want it to act. This updates the weights and lets it learn something new. For this demo, I'm going to teach you how to parse a file format called BAI2. It's what banks use to communicate transaction settlements with each other. But the same approach could be used for any fine-tuning use case.

The fine-tuning process is pretty simple. We give the API a file that looks like this, where each line represents a message, which could be a system prompt, it could be a user input, or it could be the assistance response. I've constructed 10 examples where this file format is properly parsed. 10 examples are the minimum for the OpenAI fine-tuning API. I expanded 1 of these just to show what it looks like.

So I have my system prompt, parse the provided file, summarize it with these fields. In the user field, I say here's the file, and in the response I show what I want the output to look like. I'll take this file and turn it into a fine-tuning job. That's where the command line utility comes in. This job will run for a few minutes and at the end, a new model becomes available in the Playground and in the API.

So I'm running npx gpt-finetuner create. I give it this JSONL file and I tell it that I want the suffix of the name of the model to be BAI2 just so I can keep track of it better in the playground. So I'll go ahead and run this. It's going to upload the training file, and then we have to wait for them to do some pre-processing on the file. Now that the pre-processing is done, the fine-tuning job has been created.

So I'll take this fine-tuning job ID and run npx gpt fine-tuner show, and then that job ID. And we can see that it created the fine-tuned job but it hasn't actually run yet. We can also run gpt fine tuner ls to show all of the jobs in our account. In this case I have 1 that I ran earlier and the 1 that is currently running. So I'm gonna let this wait a few minutes and then show the progress that it makes along the way.

After a few minutes it looks like there's been some progress on the model. Its training loss has gone down and its accuracy has gone up. It's actually learning this file format. It looks like the job finished in about 10 minutes, and now we have a new model ID. Let's go try it out in the playground.

I've got a brand new BAI2 file right here, and I'm going to select our brand new model, BAI2, and let's run it. And just like that it's extracting the fields properly. It understands that the amounts are number of cents, it understands there's a currency, it understands that the sender and receiver fields exist, and it's not trying to create transactions out of lines that aren't transactions. We've actually taught it how to read a BAI2 file, which is incredible. If we compare that to the baseline 3.5 model, which is just making some guesses, we can see that it truly is way better.

This is meaningless. In the standard 3.5 model, it's just trying to infer based on some context cues, but it doesn't really understand what's going on. Meanwhile in the fine-tuned model, we actually get real results. Fine-tuning a model can give it new skills or help you show a solution instead of shoving a description into the limited context. It does come at a cost though.

Fine-tuning has a per 1000 token cost, and inference with a fine-tuned model currently costs about 8 times as much as a generic GPT-3.5. Let me know how you end up fine-tuning GPT-3.5. And don't forget to do the YouTube stuff, liking, subscribing, and leaving comments. Your feedback will help me share more things in the future. This has been Code and Stuff, thank you for watching!

See all Code & Stuff transcripts on Youtube

Fine-Tuning GPT 3.5: From Generalist to Specialist