Training OpenAI Assistant API with Web Datasets: BrightData Tailors Data to Your Use Cases
Other Videos📅 2024-06-01
Transcript
[00:00] in today's video let's check out a web scraper that's powered by artificial intelligence that's going to allow us to collect data and then we're going to take that data put it into an llm and get valuable information for our business so we're going to be using a web data scraper like bright data we're going to scrape a Amazon category for the reviews we're going to then take that data that is relevant for our business load it into an llm like open ai's Assistant start communicating with it get valuable information and
[00:30] everything about the board let's jump into today's video Welcome Back y'all today's video is sponsored by bright data but I think they're super cool as I actually just got off a call with a law firm that was asking for a specific pain Point like this now what they wanted was the ability to scrape public domain government sites for information in regards to contracts but bright data could be past that it can be used in a lot of different contexts now one context we're probably all familiar with is how online e-commerce vendors get the lowest prices on all their products the
[01:01] way they'll do this is they'll they scrape the data from all their competitors create these data sets compare them and then provide the lowest prices to their customers now your next question probably is Corbin how the heck do I leverage this in my business I believe the best way to leverage this in your business would be getting relevant data for your Niche that you can Proctor using an AI language model like Chad gbt and assistance API find relevant answers that can help you out in a big big amount of data so we're going to do that today my plan is to show you how to connect the the data we're going to scrape into an assistance API
[01:33] communicate with it and get value let's go ahead and jump into today's video I'm going to go ahead and leave a link to Bright data in the description down below if you use that link you get $10 worth of free credit to start using bright data products let's go ahead and begin the sign up process and it is free to start so let's go ahead and start free trial we're just going to Simply click Start free trial with our free trial here all you have to do is provide your information or you can continue with Google or GitHub and proceed now before we jump any further here know two major things about this platform first
[02:04] one you can only scrape data that is publicly available so anything behind a login screen we won't be able to get access to and the second thing is this platform is free to use right off the bat so you can get comfortable with it and see if this platform actually provides enough value for you to pay for it knowing all this let's go ahead and do today's example so in today's example I'm going to take the place as an Amazon Seller and trying to find a hot new product to sell in order to do so we're going to come over to the web data section and you can already see there is a lot of data sets that we can choose from I go to data set Marketplace and
[02:36] you can go through all of this right but in this video we're going to go over specifically Amazon and checking out the data sets relevant to this scrolling through here we can see there is a ton of data though across the board and what this data can be used for now is grabbing a data set like this like I'm about to show you downloading it and exploring it with artificial intelligence rather than a bunch of command FS a bunch of reasoning that you would have to do in the old ways so in this video let's go ahead and jump over to Amazon products once we're in Amazon products here we have a couple different
[03:06] records relevant to Amazon so we have best sellers top rating products we'll go ahead and choose Amazon's best sellers here once you found a data set that you like and you want to test with you can follow along on this video here we're going to go ahead and make sure we choose the option of Json why are we choosing Json Corbin that's because of the fact that this allows us to put it into an llm with our assistant and actively communicate with it so it can actually read the data so we're going to go ahead and hit download sample as a Json I went ahead and downloaded it as a CSV as well just to show you how much
[03:37] data we're dealing with here before the ability to load it into a chat bot like assistance we would have to go ahead and Cipher through this and come to our own conclusions which could be very excessive as this is a ton of data to try to read and understand coming over to the open AI dashboard we're going to be able to start talking to this data with no code which is super cool so let's go ahead and name or assistant here we're say Amazon researcher once we name it we're going to go ahead and come down here we'll use the new model chadt 40 for the best most comprehensive
[04:08] answers here we're going to make sure we turn on file search and then attach our Json file so we're hit plus files here next we'll hit click to upload currently uploading the products here now if you chose a CSV file and try to upload it it's going to say like not supported file type so keep that in mind Json is basically the language that the AI will read it in we're going to go ahead and hit attach here so now that we've attached it we're going to see it right here we can also find it within our storage so here we go this is other
[04:38] stuff I've stored throughout this dashboard here but this is where you'll find the file itself we're come back to our assistant you'll see it is indexing still so it's loading the data in so it's readable and let's go ahead and create our instructions here now the instructions are probably the most important part to get the most effective answers here and let's go ahead and begin let's start off by giving some context here so we're going to say context we are reading a data set of Amazon best sellers so in your specific use case what is the data set you're reading that's the first thing we're going to set up here with the assistant
[05:09] so it understands contextually what the data is then we're going to ask it what to do with the data why we care and how to operate context we are reading a data set of Amazon's bestsellers we are trying to find patterns on what the best type of product is when it comes to I'm going do semicolon here and now I'm going to go ahead and list off relevant columns that I care about that I think would be important for the data set to return use
[05:39] your discretion here what's important in the data that you want to point out to the AI model obviously it will look through everything but this is what we're going to point out together we're going to do final price going to do categories going to do department and top reviews so we'll do final price notice how I'm spelling it how it shows up in the data this is important so it is very crystal clear to the AI what's going on here we can also just command C command V control C control V come back over here command V paste and let's go ah and do a little bit more here so I'm going to expand this instructions as we
[06:09] can get pretty crazy here so far we have context we are reading a data set of Amazon's best sellers what are we reading then we're identifying specific columns within the data set that we have deemed important for our questioning so for us we're doing Final prices categories manufacture and top review furthermore based on all the data answer relevant questions I may have as a Amazon Seller ready to list a new product H period here go back and let's go and proceed here let's go and test
[06:41] this out we may need to change some settings in order for it to be better when handling temperature and top P I would only edit the temperature part you might be asking Corbin what is temperature what is top P that is how random the answer is when it comes to consistency basically they did a ton of tests and they found that a lower temperature gives you more consistent answers B scale less creative in a higher temperature gives us way more creative answers so if your answers are kind of get a little crazy there I would go ahead and lower the temperature so I'm going to do a 7 temperature here and we'll say create now that it's been
[07:12] created we're going to go ahead and hit playground and here's what's amazing about open ai's dashboard and connecting it with bright data is our ability to actually just start using it and conversating with the data that we have I think the most important question to first ask your assistant is to see if it even sees the data correctly so let's go and ask a question I'm going to say what file do you have access to this is just to ensure that like it's actually looking at the correct data here it should say something along the lines to Amazon maybe give us more information it's important that you say running retrieval that does show us that it is
[07:43] basically retrieving the data that we've attached earlier in this video and from our directions here you can see it's already taking the initiative to proceed so we're going to say Amazon researcher based on the data set of Amazon's best sellers here are some patterns and insights on the type of products concerning final price categories manufacturers and top reviews and it's already G here so taking all that information that we have here and it's just printing it out and here we go what is really cool as well as you'll notice is that if you have multiple data sources that you connect to the assistant API it will tell you where it
[08:13] referenced that piece of data so right now we only got one data source so of course every single one where it references it is going to be from the same data source but in theory if you had multiple data sources you would become aware of where it's grabbing that specific data what specific file you uploaded if you have multiple Json files that you were downloading from Bright data now You' be able to identify specifically where that point came from so I'm going to let this keep generating here it is genuinely going through a ton of data so this may take a couple seconds for it to work while this keeps
[08:44] generating let's go over the output that it's given us based off what we provided first thing we asked for to do was to find patterns for final price here's the patterns it found s Electronics products like USB network adapters and external hard drive docking stations range from 69 999 to $28.99 indicating affordability and high demand for practical Tech accessories health and safety covid-19 tests are priced at 1588 reflecting the essential nature of a high turnover for medical supplies and arts and crafts and sewing heat transfer vinyl records are priced around $8.99
[09:16] showing the craft supplies maintain a moderate price point while being popular DIY Enthusiast after we've internalized and read all this data we're going to go ahead and see what the next best product to sell based off of it is so let's go to jump over to categories based off the data as well is identified Electronics Arts crafts and sewing and health and safety to be very popular it seems like the product we're going to create together is probably going to be around Electronics as sometimes you can make really small pieces of Hardware that sell well as requested is identify the three major manufacturers for top selling products which is ID duster ssk
[09:47] htvront and finally something that could be very relevant for us when creating our listing has to do with positive feedback on utility products like the ID duster compressed air duster received High Praise for their effectiveness and cleaning Electronics which is a crucial factor of potential buyers let's actually gut check that let's see if that piece of data is actually true so what we're going to do is command C this I'm going to copy this I'm going to come over to a CSV to command F command V all right so we found our product that we were referencing here let's see if the review is aligned with that so I'm going
[10:19] to scroll all the way over here and it looks like it says works as advertised which is probably one of the best type of reviews to get as it shows that the advertisement itself is true to the product itself nice airf flow like all the other pressurized air canisters it does get very cold and touched when used but the big thing here is it does show that it works as advertised which is a big thing for a lot of consumers in this context scrolling down here we have our final piece here so recommendations for a new Amazon product listing category selection it goes over the three major categories like earlier pricing strategy
[10:49] goes over $6.99 to $28.99 manufacturing credentials I kind of just wanted to see how it would identify that and it probably just did a quantification of basically this is how many listings are from this manufacturer and so on and then highlighting top reviews showcase customer reviews that emphasize ease of use Effectiveness durability to attract more buyers let's push this a little bit further here we're going to say okay great can we make a product for the electronic category based on the data what would be
[11:20] the best product give a name five key points I know for Amazon listings there's like those lot of five bullet points five key points description pricing and more so we have done our research based off the research we understand that one of the top categories is electronics so we're going to be expecting electronic product we're also going to be expecting the price for it to say is six is somewhere between $6.99 $28.99 if it doesn't that means it's not probably following the data so we'll prove that right now and lastly we're going to get some extra
[11:51] copyrighting cuz what's super cool is based off our entire data sheet here we have a ton of copywriting and we already know these are top performing products therefore the copyrighting we see for the descriptions is obviously good enough to get enough conversions so to be a little bit more clear there I'm going to say five key points and descriptions and pricing and more all based on the relevant data let's hit run there we go it came out pretty fast here so the product proposal is smart Clean Pro compressed air duster and vacuum 2 in one it gives us the key points dual
[12:22] functionality eco-friendly high performance portable and lightweight multiple attachments that's always good an entire description pricing between the pricing that we had under $28.99 the categories the manufacturer the top review highlights the features and so on let's go and personalize this a little bit more we're going to say our brand is dust BS gone add that to the relevant copy now if I was actually going to proceed and do this I would go through the process of finding a manufacturer maybe through Alibaba and kind of use
[12:54] the copywriting that we just based off of this data so that's leveraging bright data in the context of creating a new listing off of best sellers but this can be leveraged way past that so whether your business is a very specific Market or Niche that you want data on and ask questions for to make the product selection SLS services you offer to be better there's also a bunch of other stuff that you may not even be aware of such as LinkedIn people's profiles that's interesting you can compile a bunch of people and get information on individuals one that might be
[13:25] interesting to a software company is that we can get Yelp business reviews which is super cool cool furthermore if you're looking through the entire Marketplace of all the data that they have available for sale you don't like any of it what you can do is actually request certain data sources to be scraped you get custom data sets that may be relevant to your business now I just got off a call this morning for a law firm and I can tell you right off the bat that a custom data set that they would probably request is going to be off of Government websites and relevant information when it comes to case law
[13:55] that can be found on the internet lastly if this sounds cool and you want to learn more you can go ahead and jump into a call with your account manager and they'll fill you in with everything bright data can do for you in this day and age we already heard a ton of stuff in the media when it comes to good data creates good llms so use bright data if you want to get some good data that you can start training your llms or your different artificial intelligence use cases as now we know that the best types of answers are trained on the best type of data I'll see you in the next video
[14:26] this playlist right here goes over different productivity tools you can start adding to your business right away that's a random video based off your clicks on YouTube that's my face I'll see you in the next video