Model Distillation For ChatGPT: OpenAI Tutorial For Cost-Efficient AI
Let's become an AI prompt engineer📅 2024-10-04
Transcript
[00:00] in today's video we're going to learn how to get better outputs with artificial intelligence and that is model distillation so in this video I'm going to show you from start zero all the way to start done how to do this in your workflow sound good let's jump in to give some context of what this even does here is a evaluation so I provided some test data here and as you can see it failed most but it did pass one so let me show you how to do this and to take a step back we can identify what's a failing or a succeeding version of an output from a Chad gbt prompt this allows us to get better outputs at scale
[00:30] let's proceed I'll make sure to leave this in the description down below so you can walk through step byep on how you can do it yourself the first thing we need is to get a data set EG this was the input we got this is the output we got how do we train it to get better outputs now what is really cool like extremely cool is that if you add this buling here which is store equals true this allows you to actually get real inputs and outputs that you can start evaluating this will show up in your sidebar here at chat completions the data will start showing up here and you'll simply hit evaluate as shown with
[01:00] this image here where it's like a tech support type of chat gbt bot here we are able to take all like the real inputs and outputs and hit evaluate but in this video I'm going to show you how to do it with some test data and the reason I'm going to show you how to do it with some test data is because some of y'all might just be like I want to see what's up here I want to check it out I want to see how to do it let me show you how to do it therefore let's go and begin we're going to come over here to evaluations in evaluations we're going to hit create once we're in create here we're going to hit import test data now let me show you how to create some test data together I suggest you to use something like vs code cursor AI some type of IDE this is going to allow us to create the test
[01:31] data that's relevant which is a Json L file go ahead and launch your IDE this is cursor Ai and what you're going to do is hit command new control new create a new file here once you created a new file here let's go and just save name it whatever you want what is put test data and then we're going to put dot Json L this is extremely important that it has to be this file type for this to work hit save there we go so far so good we have the correct format now I've gone ahead and pasted some test data I've done in the past which is is creating Marv which is a factual chatbot that is
[02:02] also sarcastic I'll see about putting this like in a Google drive folder and I'll put in the description down below and allowing you just to download this test file so you can use it yourself so if that exists just check the description it should be like test data click it Google Drive Link download so make sure you leave a like it's free for the free value but the idea here is this if you're familiar with prompting and structuring prompts when it comes to API and specifically open AI typically what happens is that we have a system prompt we provide the system prompt so we want to have repetition on what the underlying bot is specifically going to do here so Marv is a factual chat bot
[02:33] that's also sarcastic then the next part of the prompt here is typically the user and what the user says like the user input EG what's the capital of France and then finally the output the assistant and from this output we have Paris as if everyone doesn't know that already for test data and when you are testing this kind of logic go ahead and make sure you have 10 different rows here it's also extremely important that when you save this file you save it and you don't have like oh there's 16 lines I'm going to save it it import it no no no no no no you will get an error make sure it's 10 hit command save and now
[03:05] you have your test data obviously your test data can be anything whether you're making a customer support bot whether you're making a Storyteller bot the name of the game here is the idea is we want whatever the input comes in this is the specific output we like that's why in this context we're giving like general questions like what's the capital of Texas and then giving sarcastic answers so we are training it to be sarcastic factual bot now that we have that test data here let's go and import here we go I have the test data uploaded test data Json click it hit import now that we
[03:36] have it imported and ready to go the next thing we can do here is either generate responses which I'll probably do in a future video so make sure to subscribe here to check that out in this video I just want to show you how to test with it how to evaluate now we come to adding testing criteria this is going to show us whether this is a good output or output based off our preferences we're going to add with this we have a bunch of pre-built options here such as factuality sediment test quality for us we're going to check out sediment here see if it picks up on the sarcastic part hit sediment I'm going to provide the item messages as what we're going to be analyzing and then what we're grading
[04:07] with is very important here so you wouldn't want want to choose gbt for mini because the whole point of what we're doing here is we're going to use a higher level model gbt 40 ow and mini ow and preview to grade a lower level model like gbt 40 mini or like 3.5 if you're still using that Etc then in the context of sediment we can give either positive neutral or negative for a passing grade I have no clue how AI interprets sarcasm so I'm going to see what it says if the passing grade's negative hit add following this we can go ahead and
[04:37] estimate the underlying cost to the amount of tokens that are going to be expended based off this there we go and we're going to go hit run so we got a results here and it's actually pretty funny here so the idea behind sarcasm is like that could be interpreted as negative and as you see like 60% of the results were negative so I guess our barometer in this context is very much so if it's failing EG the Romeo and Juliet question or how many in the solar system it wasn't sarcastic enough therefore we need to restructure the output so it's more sarcastic and would pass this test that's the idea here though which is really cool is our
[05:09] ability to put in a ton of inputs and a ton of outputs imagine in the context that you put that flag that we saw earlier we actually get real data getting flowed in and we have like a 100 inputs and 100 outputs that we can then test to see for whatever we want so in this context it's sediment and we got a 60% which is like a d we need to up our prompts that is how we start leveraging the new ability provided by open AI here if you want to learn how to start prompting with Chad gbt in a more effective way make sure to check out this channel here I believe two days ago 3 days ago I show you how to
[05:40] successfully prompt with the most advanced model o1 preview so if that interests you either click my face down there or just type in Corbin Brown 01 I'll see you in the next video we got a 60% we didn't fail or I guess some context that could be failing those are random videos that's my face I'll see you in the next video