I taught a three-day bootcamp this week, and the last activity I had planned was to create a simple agent with python. I had only successfully done this myself that morning, between the hours of 3:00 and about 5:30 AM. It was incredibly basic, but at least it was actually agentic (it had tools and used a large language model to decide which tool to use for the job). I had gone down the ChatGPT assistants application programming interface (API) route originally, only to discover you have to bind the context you upload to the tool you want to use. Hardly agentic. I also discovered that the assistants API had been deprecated, but it doesn’t say than in the user interface, only when you work with the API. So that was an annoying bust. I lost interest in doing it with the OpenAI developer stack.
A Morning Before Class
So that was my morning before class. At that point, I switched to and clarified to Claude my specific objective: “I am not a developer, but I do have a high-powered Mac Pro. I want to create a super simple mortgage agent that connects to the OpenAI API”.
I hadn’t yet given up on OpenAI on the backend (that came later) since it was, at the time, what I knew the best. Then I went on a journey, using just the Claude UI to get there. Two and a half hours later, I had met my objective. I even managed to deploy it securely using a git repository, a .env file for handling secrets, and streamlit. Go me!
I did have a modest heart attack when I made a rookie mistake and put the API key directly in the code and then put that code into Claude. Claude went with it for a while and ultimately terminated my session. At first, I didn’t know why, then I realized what I had done and freaked out a little bit. But no worries, I then learned how to revoke API keys. Honestly, I went on a little revoking mission at that point because “Only the Paranoid Survive” (Andrew Grove superfan).
I felt confident that if anyone got this far in the class I was teaching, they would be able to create a similar app if they were smart and had hustle. A few got this far but elected not to try. We spent a lot of the day building applications in Replit and these three students elected to build OpenAI integration instead of built the python agent. Totally get it, no judgement here. We built so many applications and added so many users that Replit reached out to me to see if I wanted to have a conversation about what we were doing.
A Ridiculous Machine and Claude Code
But now I’m on a mission to get a much more tactile understanding of agents. I originally bought this ridiculously powerful machine so I could run at least llama 70b, which slowed to a crawl on my previous and now dead laptop. I decided to figure it out. I also decided I had hit the limit of what I could do in Replit and on my own and wanted to try Claude Code.
It took me a while to get Claude Code installed even though I already had a bunch of the prerequisite due to my earlier agent journey. It was not completely trivial for a non-developer.
My first few apps were… meh. And they were slow. Which didn’t make sense. I found myself with a need for speed. I went on a journey to first optimize my llama install with ollama. Ultimately, I was able to achieve pretty good performance, at least according to Claude.
A Successful Failure
Now I wanted something really useful, and a steppingstone to the agent I have in mind. My first application was a successful failure. I wanted a retrieval augmented generation (RAG) based solution that would let me search my active documents folder on One Drive and let me engage conversationally. Yeah, that idea was not good. Too slow, not enough space for a bigger model and too many files to work with. If only Microsoft Copilot was even semi-useful. I appreciated Claude following up with me to let me know that unfortunately I could not run a 256b parameter model on my machine. I am maxed out at 70b. First world problems indeed. I decided to create just a simple ragbot, but one that was blazing fast.
That’s where I was for a few hours. I have at least gotten my front end to deploy and I can upload documents. Now I’m debugging. I seem to have a corrupted database, and my first test was entirely too big (I see you entire Fannie Mae seller servicer guide). That was a mistake, I should have started much smaller. Such a rookie, good thing this is just a hobby and I have a very accomplished team that knows what they are doing most of the time.
And that’s part 1 of my frustrating-but-still-fun journey with my Mac Pro and llama 70b. I feel semi-but-not-too-uncomfortable with Claude Code, I feel an ok level of comfort with terminal, I feel not-at-all comfortable with Docker, I have learned a ton, and the next class I run this is DEFINITELY on the agenda.
Part 2 - Success!
This post blew up a little bit so I thought I would update on my progress. I have achieved success! I was able to get my ragbot up and running. It’s blazing fast and I even added a capability to engage with the 50 state attorney generals web pages to look for updates, store updates, and engage with those. Go me! I then moved on to install and deploy stable diffusion for image gen, which proved a bit difficult, but I had success there too. Thanks for all the engagement.
By Tela G. Mathias, Chief Nerd and Mad Scientist at PhoenixTeam, CEO at Phoenix Burst
