How To Use Claude Computer Use Agent For Beginners (anthropic, claude 3.5 sonnet)
Let's learn how to use Claude Code📅 2024-10-25
Transcript
[00:00] here's our prompt download a cute photo of a cat all right all right so let's go to Google Images y'all oh this is so cool all right go ahead and search something I need a cat come on let's go clae okay there we go okay okay I'm I'm doing nothing my hands are here I'm doing nothing oh this is awesome oh okay is there going to be a pop up here or is it actually going to get this is it going to get it come on y'all hit download oh oh yes we got it we got it we got it I thought we got paid wall we have our cat there it is y'all AI can now control our computer in this video I'm going to show you how to set up this new artificial intelligence from Claude
[00:31] so it starts working on your computer the best showcas this technology we're going to be using Docker along with zap to show you a really cool use case as you probably already saw in that very intro of this video this stuff is pretty cool y'all so this thing can take complete control of our computer because this is so early days they have advised us to put this in a virtual machine and what that basically means is that obviously you have your laptop or desktop but we're going to use a virtual machine so if anything goes crazy like the AI turns on us then we are not going to have issues with like losing us privacy information like anything on our
[01:02] laptop cuz it's in a virtual machine I'll make sure to link the documentation down below and today's video is sponsored by zapier zapier has contacted me and they're just like let's make a bunch of cool videos together so you're going to see more content around these topics let's go and jump into today's video on how we set up this computer API or they call it computer use beta step number one download Docker the reason for Docker is that this is going to allow us to run this in a virtual machine it cost no money you don't even have to make an account you just sign in as you'll see here we're going to be doing some fun examples together but I wanted to do one major your use case to see if it's able to fill out a form so
[01:32] we're going to be using zap your interfaces here now think about it this is just to graze the imagination in context this could be filling out a lead form this could be scraping data online which I'll probably dive more into later on let's just see if this Tech can even fill out of form it's a dog form once you've installed Docker you're going to be able to launch the application if you just launched it you can just say continue as guest if you don't want to make an account yet therefore part one is done we will create a VM here or basically give AI the ability to act like it's on a computer and computer stuff I'll make sure to leave a Google doc with the instructions down below of
[02:03] the steps I do in today's video or alternatively just provide you with bump ups.com which can just read the video and give you the steps outlined in text format the next thing you need to do is make sure you go to Google type in anthropic API once you've logged in and set up your account go to get API Keys two major things you should know first major thing for you to do this you're going to need to set up your billing I think when you create an anthropic account you get like $5 of free credits so that's super cool second major thing is that this is where the API keys are located but all you really need to do is is simply hit create key once you hit create key put in a name that you're going to be able to recognize and then
[02:34] select your workspace hit add once you've done that copy that put in your notes just have it for later reference I'll make sure to put all the commands I use today in the description down below but let's get this VM started here to get started I'm going to go ahead and use the documentation provided by anthropic here I'm going to scroll down and go to computer use reference implementation click this don't worry you don't have to understand everything that's shown here but all you need to know is that basically this is the code that's going to allow us to run this okay so with this though we're going to need to put this in a VM I'm going to take a step back as if you've never coded or never even seen a terminal command before let's make sure you can
[03:04] even do the steps and the actions I'm about to do right now so first off just see if you have git installed and what git is is just it's connection to GitHub as you just saw there if you don't see that you don't have it installed therefore make sure you install home brew first and this is going to allow you to basically do these kind of commands that you're about to see later on in this video that line right there I'll of course try to remember to put in the description once you do that you can do Brew install git hit enter and then all you need to do to confirm that you have git is do Git Version if you see that you're going to go oh and also this is just a terminal window so on Mac or Windows or Linux just type in terminal
[03:35] and this will pop up with all that out of the way step one we're going to clone that git we're going to take the code I'm going to paste this line I get this message just because of the fact that I was already playing around with it but you'll get a message that makes it a clone within the directory next thing we want to actually navigate within this GitHub directory to the specific part that's going to allow us to run this VM we're going to do CD and thropic quick starts and then within quick starts this is where we want to go to we want to go to the computer use demo so we'll do CD computer use demo and then we're in the right area so what you should see on your terminal is whatever you're using
[04:06] so from me coffee Fuel and then computer use demo and just to make this a little bit more simpler all we're doing is simply going into a folder in a folder and being like hey this is the code that I can run to get the AI to do stuff on my computer which they plainly call it a computer use demo next thing we're going to do is we're going to load in the variables we want to Associated such as our API key and how we're going to run this locally in our machine or just basically run this on our laptop top we're going to do touch. EnV this is going to create a file called EnV this is where we're going to store a key so I'm going to paste this here but go ahead and just copy it from the
[04:36] description from this video you're going to do two things anthropic API key then your anthropic API key the one that we just got earlier in this video and then the second line do p 8080 this is where we're going to run this so I'm going to go and do this but I can't show you CU then my API key will be lost so far so good on to the next so now that we set that all up make sure you're still in computer use demo this is very important if it doesn't say that you're in the wrong area you're lost you need directions from here we're going to Simply paste in Docker build T Cloud computer use demo all this is doing is
[05:08] creating an image which we'll then be able to use in our VM so we're going to hit enter here and this is going to take a while if this is your first time for me I think it took around maybe 3 minutes when creating this image originally don't worry about all that you're not getting hacked when I first created this it took like 3 five minutes but now since I've already done it it only takes like 10 seconds his next terminal line is what's going to actually allow us to run the container within our our Docker here now don't worry it looks a little crazy you can always copy and paste from the description down below but this is it right here so we're going to load in our EMV file we're going to load in the demo
[05:38] we're going to load in the ports that we're going to be associating with this local demo here we're going to enter and boom it shows up right here if you want to verify you did all the steps that you just saw there correctly all you need to do is do Docker PS and then the output should look like this if it doesn't something might have messed up but regardless if you see it right here you're probably good okay we set it up in our VM so we don't destroy your computer beta let's check it out show all three ports go to 888080 no I'm not a robot click it this is going to open up a new tab we are loading in we have connected so this is super cool y'all so we are officially in
[06:10] our VM right now we need to do a couple things first thing we're going to do is that sometimes it'll ask again for us to provide an API key here so we're going to go and open this right here and obviously this is going to be in control of your computer but in a VM so it's not going to be able to access files that are like on your desktop so don't worry uh this can only access stuff that's found within the VM right here which is like like Windows like 2009 I don't know we'll go ahead and open this sidebar here and put in that API key we had earlier once it's pasted hit enter and we are good to go y'all we officially
[06:41] have our chat so we can do anything here anything Corbin anything so let's try this out first use case let's just see how this interacts and just as one other side not here toggle screen control on and off what that means is that if it's on I can't do anything or sorry if it's on I can do stuff but if it's off then if I'm clicking around here I can't do anything so I guess just to prove to you that I'm not doing anything when this demo is happening I'll click around a little bit watch this we're officially using computer use API in a VM I've shrinked down here's our prompt let's AI controller computer download a cute
[07:12] photo of a cat enter so what's going to happen is we're going to get what it's thinking what it's doing here oh and then the actual sorry I said oh because it makes a little pop noise that you can't hear it turned off on purpose but then all the stuff that it's doing like actually on your computer shows up here this is cool stuff stuff so this is the future and we're getting to the point now y'all where yeah this is very simple use case in the sense of downloading a photo of a cat but whoa whoa whoa whoa could you imagine what we can do in the
[07:43] future in the sense of just leaving your computer on and then AI does a very complex task Next Level stuff so let's check it out so right now it's on cute cat pictures and we can see what it's thinking of over here so the way it's kind of navigating is it'll take a photo right here and then like kind of search a new tab here okay so it's it basically keeps running into an issue where maybe it can't find a very specific image that's available for download this is so cool y'all so we're having an issue right now where it's trying to find an image that's downloadable so let's guide it a little bit I'm going turn screen on so I can just close up this browser
[08:13] we're going to start from Ground Zero again we may need to guide it a little bit more I mean this is all early days so I'm going to try this download a cute image of a cat from Google Images just right click and save let's see if this helps help me out I need a picture of a cat I'm going to turn off screen control here we go here we go here we go all right all right so let's going to Google Images y'all oh this is so cool this is actually so cool this is it's about to be crazy y'all all right go ahead and search something I need a cat come on let's go Claude okay there we go there we go okay
[08:48] okay okay come on oh this is so cool y'all I'm I'm doing nothing my hands are here I'm doing nothing oh this is awesome come on can you do it can you do it for me don't visit the site oh you visited the site oh no let's see if they can track back here as I said before this is all beta so you're not going to see like the most perfect showcasing like this is all beta but this is is so cool
[09:18] whoa okay come on you almost have it I might need to do some route where I just do a screenshot oh this is going at it y'all when it runs into walls like pay walls it's like no I'm going to find you something else don't worry okay I'm I'm going to let this keep going y'all I'm going to see if it can actually do this okay so it's getting smarter I think I may have needed to include in the prompt free image of qat because a lot of stuff's pay walled okay royalty free images we're getting somewhere y'all come on Claude you got it you're a
[09:51] couple clicks away okay don't payall me just hit download hit the big button that says download oh okay is there going to be a popup here or is it actually going to get this is it going to get it come on y'all let me know in the comments if you think he's going to get it come on hit download oh oh yes we got it we got it we got it I thought we got paid wal okay this is O all right this is the future y'all let me stop this please AI stop this is the future and this is no joke
[10:22] so we're going to turn it back on wow that is so cool all right y'all we actually got it if you go right here make come to these little the little hamburger go to downloads we have our cat there it is y'all we officially let AI download a picture of a cat okay now the real test let's see if this can fill out that form all right so this is like really I'm really interested here y'all I have no clue on the like the top level of this okay we got the dog form y'all oh this is so cool okay so I'm gonna copy this
[10:53] so we got our form here dog breed how many dogs in the image does this dog seem happy I wanted to ask an open-ended question and just see what the heck it's going to come up with then we have the ability to upload the file there I went ahead and say okay please to fill out this form with the requested inputs of dog breed amount of dogs and the image and if the dog seems happy then upload the dog image and hit submit gave the form URL it seems like the the way you prompt computer use API is a little different you have to be very more specific because it's like actually doing actions within your computer which is super cool enter I have no clue this is going to work y'all here we go here we go here we go so there does seem to
[11:24] be actual limitations within the API itself I think for security reasons it doesn't owed to submit forms yet but let's see if it can fill out the form at least and then all we have to do is hit submit okay please just fill out the form then also I realized I didn't ask it to get a dog image so we'll see if it's smart enough tole that okay so it does seem like there's heavy restrictions around computer use which makes sense I think they're trying to mitigate Bots and Bot spamming so if someone use this API to like actually fill out forms you know hit the little thing that says like are you a human that kind of thing let's see if you can actually just provide the data then then I can paste over hit enter it really
[11:55] doesn't want to give me this it actually goes as far as like gatekeeping it by saying oh you're you're just using this for the form let's try a new chat let's go ahead and try again I'm going to try a different strategy here so if you want to reset that chat I'm going to go ahead and just say stop selected items here and then I'm just going to hit run again once it's stopped let's hit that little play button again and get going so I was running into issues with restarting it so I'm going to make sure I leave these terminal commands below y'all got to understand this is all beta what this is going to do is this is going to remove the container so we can kind of reinitialize it Docker stop CLA Docker RM Claud hit enter once you do that I already did it it's going to remove that
[12:26] container and then you're going to run that other command I showed earlier hit run again here we go 80808 with my API key readed here I'm going to try a new strategy we got a little options here so let's ask it to put all these options and answers in a text edit I want you to download an image of a free dog then answer these questions in a text file what's the dog breed how many dogs are in the photo and does the dog seem happy hit enter we are live again Firefox is open I do Wonder longterm how they're going to get around users leveraging this in the context of filling out forms they may be spammy right there must be a
[12:56] lot of security setup right now for that okay we got some happy look dogs I think this is the site with a pay wall though so it's going to have to try to find an alternative solution here unless it's smart enough to right click now SL screenshot interesting also we can see the the type of actions it's taking in the sense of like what it's doing Mouse move right click oh the issue is that it keeps clicking the sponsored images and those aren't the free ones that's why it keeps getting stuck in this little Loop here it is cool though that you can just let it keep running and it just keeps trying like it doesn't give up which is really nice and part of this probably
[13:27] has to do with the fact that you need to have better prompts when asking these kind of questions so like when it has context like this I can go and stop it and I'm going to just be like no no no let's go to Google Images I'm going to try just go to Google image to get the dog image don't try to download sponsored images let's see what's up I don't know if I should have added that sponsored images part it might get really confused we'll see first off let's see if it even listens to me for Google images okay it listens to me go and search it you know what oh yeah here we go so the prompt is key y'all The Prompt is key oh okay
[14:00] although I don't know if it's saving the dog photo oh this is hilarious I don't know if that was the dog photo I think it might have messed up oh look at that it didn't even need to show me like it uh typing out stuff okay so it created the dog analysis text file as well all right let's check this out I'm actually curious what it's going to do though because I think that's a blank photo let's see all right so let's come up here let's go to downloads is this good it was actually good it did download the dog photo let's see the
[14:32] text there we go here's a text file and then nice little air cuz it's in beta let me hit stop here let's see what's up here y'all all right so we got the breed border Ki mix number dogs one happiness level very happy dog appears to be running joyfully through a field of yellow flowers with an open Smile expression typical of happy dogs all right solid and then this was the original image that I grabbed here I think we can confirm therefore now we can fill out this form like it's nothing
[15:03] p over border collie it was one and we got the confirmation it was very happy download the image uploaded and hit submit we've successfully submitted a form with the help of AI once you're done make sure to pause it and you're going to go you can run AI on your computer now so that example shows us automating stuff locally on our computer which is super cool but as you probably already know we can automate basically anything we want with zappier when it comes to software these little automations that you can set up leave and let them go really cool stuff like automatic email replying with AI automatic data input with AI when it comes to spreadsheets Etc that concludes today's video all relevant commands
[15:34] directions and all the resources can found in the description down below make sure you leave a like it's completely free and I'll see you in the next video AI controls our computer download a cat download a dog those are two random videos that's my face I'll see you the next video