joshmlewis

I've been building promptslice.com as a way to create, test, and optimize prompts + tools + context. I needed it when building agents for other use cases so I built it. Unlike Langsmith and some of the other prompt editors, this has a first class editor for your prompt with "Cursor style" chat where you can ask it to make tweaks, overhaul sections, optimize ambiguity, etc.

Same thing happened to me except I wasn't able to stop it. I still receive pictures of mail going to an old PO Box I had.

There have been a lot of these clickbait articles the past few months about how a company "hacked" MCP of <insert well known company>. They always get upvotes and do well but the take away is what you said at the end of the day.

This is more of a model that VSCode would integrate, not something for end users to use.

I'm going to test implementing this for my project https://promptslice.com and see how it does with text based edits. I assume it will do ok.

I'm also really curious about the XML tool calls in the documentation. I have not heard of this being the norm for tools like Cursor. Is that still the case? I feel like I'm pretty in the know about this stuff but must have missed that trend.

I have not ever actually hit the rate limit as far as I'm aware. I have gotten the "approaching Opus usage limit" a couple times but have not hit it.

yes the latent space podcast I believe

I've been wondering this too. They said on a podcast that the average usage is around $6 a day in credits but I really question that. If that's the case though and they do have a lot of Pro and higher tier subscribers that might be making up the difference.

For a reference point, it says my max session limit in the past was ~337,492 tokens and I have the Max20 plan and 99% use Opus.

My total tokens used since I started using Claude Code on May 27th was 1,374,439,311 worth around $3397.34.

Yes this is one of the techniques apps can use. You vectorize the tool description and then do a lookup based on the users query to select the most relevant tools, this is called pre-computed semantic profiles. You can even hash queries themselves and cache tools that were used and then do similarity lookups by query.

For coding at least, I don't bother with anything less than the top thinking models. They do have their place for some tasks in agentic systems but time is money and I don't want to waste time trying to coral less skilled models when there are more powerful ones available.

It's a skill that takes some persistence and trial and error. Happy to chat with you about it if you want to send me an email.

Yes this is what I mean by conciseness with o3. If prompted well it can produce extremely high level quality code that blows me away at times. I've also had several instances now where I gave it slightly wrong context and other models just butchered a solution with dozens of lines for the proposed fix which I could tell wasn't right and then after reverting and asking o3, it immediately went searching for another file I hadn't included and fixed it in one line. That kind of, dare I say independent thinking, is worth a lot when dealing with complex codebases.

I have not used them but I feel like there are tools like Replit, Lovable, etc that are for that audience. I totally agree IDE's are intimidating for non-technical people though. Claude Code is pretty cool in that way where it's one command to install and pretty easy to get started with.

Claude Code isn't a model in itself. By default it routes some to Opus 4 or Sonnet 4 but mostly Sonnet 4 unless you explicitly set it.

I have the Max $200 plan so I set it to Opus until it limits me to Sonnet 4 which has only happened in two out of a few dozen sessions so far. My rule of thumb in Cursor is it's worth paying for the Max reasoning models for pretty much every request unless it's stupid simple because it produces the best code each time without any funny business you get with cheaper models.

Yeah, exactly. For everyone who might not know, the chat apps add lots of complex system prompting to handle and shape personality, tone, general usability, etc. IDE's also do this (with Claude Code being one of the ones that are closest to "bare" model that you can get) but at they are at least guiding it's behavior to be really good at coding tasks. Another reason is using the Agent feature that IDE's have had for a few months now which gives it the ability to search/read/edit files across your codebase. You may not like the idea of this and it feels like losing control, but it's the future. After months of using it I've learned how to get it to do what I want but I think a lot of people who try it once and stop get frustrated that it does something dumb and just assume it's not good. That's a practice and skill problem not a model problem.

What languages do you use it with and IDE? I use it in Cursor mainly with Max reasoning on. I spent around $300 on token based usage for o3 alone in May still only accepting around 33% of suggestions though. I made a post on X about this the other day but I expect that amount of rejections will go down significantly by the end of this year at the rate things are going.

o3 is still my favorite over even Opus 4 in most cases. I've spent hundreds of dollars on AI code gen tools in the last month alone and my ranking is:

1. o3 - it's just really damn good at nuance, getting to the core of the goal, and writing the closest thing to quality production level code. The only negative is it's cutoff window and cost, especially with it's love of tools. That's not usually a big deal for the Rails projects I work on but sometimes it is.

2. Opus 4 via Claude Code - also really good and is my daily driver because o3 is so expensive. I will often have Opus 4 come up with the plan and first pass and then let o3 critique and make a list of feedback to make it really good.

3. Gemini 2.5 Pro - haven't tested this latest release but this was my prior #2 before last week. Now I'd say it's tied or slightly better than Sonnet 4. Depends on the situation.

4. Sonnet 4 via claude Code - it's not bad but needs a lot of coaching and oversight to produce really good code. It will definitely produce a lot of code if you just let it go do it's thing but it's not the quality, concise, and thoughtful code without more specific prompting and revisions.

I'm also extremely picky and a bit OCD with code quality and organization in projects down to little details with naming, reusability, etc. I accept only 33% of suggested code based on my Cursor stats from last month. I will often revert and go back to refine the prompt before accepting and going down a less than optimal path.

It's called attachment theory and does show up in more than just romantic relationships.

It's nothing groundbreaking nor particularly exploitive about MCP itself (although I have my thoughts on MCP), it's just a clever use of prompt injection and "viral" marketing by saying MCP was exploited. As I build agentic systems I always keep the philosophy of assume whatever you give the agent access to can be accessed by anyone accessing the agent. Never trust the LLM to be doing access control and use the person requesting the LLM take action as the primary principal (from a security standpoint) for the task an agent is doing.

This article does make me think about being more careful of what you give the agent access to while acting on your behalf though which is what we should be focusing on here. If it has access to your email and you tell it to go summarize your emails and someone sent a malicious prompt injection email that redirects the agent to forward your security reset token, that's the bad part that people may not be thinking about when building or using agents.

Typically the left lever is the clutch, right lever is the front break, and your right foot has a lever for the rear break. Not sure what the setup was for the record breaking motorcycle though.

Playing devils advocate here, it's because removing a function is not always as simple as deleting the lines. Sometimes there are references to that function that you forgot about that the LLM will notice and automatically update for you. Depending on your prompt it will also go find other references outside of the single file and remove those as well. Another possibility is that people are just becoming used to interacting with their codebase through the "chat" interface and directing the LLM to do things so that behavior carries over into all interactions, even perceived "simple" ones.

It's become very commonplace for there to be a dozen replies on viral posts where users all @grok to ask if the post is real or for more context. It's almost more work to compose a reply asking Grok about it than it is to just click the Grok button and it give you more context that way. I don't get it.

I gave it a research task to go and find certain government portals within a specific region. It found around 12 of them successfully although it would only do 4-5 at a time and I had to prompt it to keep going. When I asked it to put all of the links in one plaintext list with the full url not hyperlinked, it absolutely could not. It just kept repeating the same list with hyperlinks over and over. It doesn't seem to be setup quite yet to do the kind of data gathering where you can have it collect data and then also refine it in the same chat. Otherwise I'm still impressed by the workarounds it would do if it got stuck. Some of the portals are a pain to find.

Are you willing to share what you are using for data sources? Particularly the renter, net worth, etc aspects as that is not in the publicly available data out there.

Agreed, this should be the main link instead of giving AppSignal free publicity. The original is very much a light reworking of the original. It has the same content order and everything.

It means being able to make travel itineraries public / shareable for other people to use for their own.

It's fascinating they used Excel in the product photos instead of Numbers. I wonder who actually uses Numbers these days? Or any of the Apple "productivity" apps for that matter.

He talks about supplements sometimes but I don't ever feel like he pushes them and usually says you can get them wherever but I suggest here. I don't agree with everything he says but it's clear he puts a ton of time and research into the episodes he does and I don't blame him for trying to make money. So many of his guest episodes contain invaluable knowledge and that's a fine trade off for me.