mesmertech

Yea I can believe all these. I've personally have been having issues on these points:

"200 OK, no answer" - insane that openrouter's main feature is literally a fallback and streaming doesn't support 200 no content to fallback to another provider or smth.

"rate-limit by IP"... now it kinda makes sense why deepseek v4.1 flash rate limits me on prod but never seems to happen on local. Makes you have to basically pin Deepseek as provider, since I've never had 429 error on them

hopefully this come in openrouter api cause I'm not signing up for a specific provider's specific api platform, and have yet another thing that can bill me.

Also honestly if you have a thing that works, I'd say don't even share it. Its too easy to build a clone and start copying, which ofc was also possible a few years back but then it took actual effort and someone had to maintain it.

The only reason I'm sharing these is cause AI image generators are already like thousands in the market, so I mean if someone wants to go through the hell, I don't think its really that easy a market to succeed in anyways.

And here's my list of things that didn't work out: https://mesmer.tools/

Pretty basic ones, I got into AI images cause of that viral AI avatars thing that levelsio made 2-3 years back and have just been iterating on that idea(even though I think if I went into the AI text side of things, I'd have been much bigger at this point):

https://admakeai.com - Recently got this to 800 MRR , kinda trying to make it the main thing since the LTV on it is the highest. So if I could just figure out ads, this has the highest potential

https://aieasypic.com - First project that I made that made money from subs vs making money from ads, whole reason I continued to indiehack. Its kinda dying but core users still bring in 3-4k/mo , altho actual MRR is 1k/mo

https://bestphoto.ai - Made this after I got frustrated from the bad decisions I made on above, basically a tech rewrite + clone + simplification of aieasypic. Currently makes the most but at a lower margin than aieasypic. Not gonna say exact numbers, but lets say 5k+/mo

Announcement for the appearance of a commit, thats where we're at

Why are you guys not on Openrouter? I assume you'd get way more volume that way no?

Or does openrouter have like a specific contract you have to sign with them and requirements or smth? https://openrouter.ai/moonshotai/kimi-k3#providers

And on creativity at least visually, Gemini 3.1 pro is somehow still up there. But its really hindered by its inability to use tool calls effectively or make a long term plan.

My personal benchmark for new models has been to compare video making skills with something like remotion. Usually reveals if they have any "taste" or outside the box thinking.

I'm starting to not trust any "benchmarks" when it comes to frontier models at least. As an example Sol feels the most "gets stuff done" but has zero taste, or any capability to surprise.

And for frontier models I go one step ahead and try to recreate a complex animation video, with the ability for the model to review its own work. And at this Fable is still the top one. Ex: https://www.youtube.com/watch?v=uDAeAuYyl0E (recreation of Claude announcement video) and https://www.youtube.com/watch?v=cSsVNtGPOIg (recreation of a fireship video). Sol did something similar but you can instantly tell its AI slop from very small things, and it just has no narrative or thought put into the writing.

https://mesmer.tools/benchmarks/ai-video-generation , I usually put basic ones here.

Hadn't tested out any of the image models for training loras since Flux 1 dev, so I was curious which one is the current best. Results were pretty interesting

Models tested: Ideogram v4, Flux.1 Dev, Flux.2 Dev, Klein, Krea 2, Z-Image

I'm on max $200, I had this idea for around a month since Fable got banned. For the usage, not exactly sure since I had like 4 agent heavy tasks running at the same time.

I think personally you can do first half of this with the direction with fable and then render + checking using sol since thats what I ended up doing. Guesstimate I'd say it takes around 10-20% of the 50% fable usage

Had some fable usage to waste yesterday cause my reset was gonna happen. Ended up burning all my usage so had to finish the last part with Sol itself.

Imo Remotion based videos feel better to watch than just actual GenAI videos, like Seedance.

the exact prompt for reference:

"I wanna make a plan for making good fireship videos automatically. so my thinking is, you need to first need to find a good way to insert memes like gifs, short videos, images stuff like that. another part of a good fireship video is the voice itself, so you need to find a way to voice clone fireship's audio(probably download a sample video and use our existing higgs audio api, check airoleplay and search "higgs" for the api endpoint details and the api keys and stuff if you need). another part of it is the animations and creative text animations and extra touches that he puts, maybe for this you could analyze a couple of his vids in general using gemini 3.1 pro with openrouter in detail and you can ask multiple questions abut a video to explain in detail exactly what happens.

now remember you're the orchestrator and this is likely going to be a very long task so you need to be using your context very carefully, and assign various research tasks to opus or fable subagents depending on the complexity.

the video itself should be about claude fable 5 and make another one for gpt 5.6 sol. some more reference for how we recreated antoher vidoe with a single prompt before: '/Users/test/Documents/openmotion/recreate-video-user-prompts.md'"

As long as they have the best model they can afford to lose goodwill.

People who don't wanna spend too much on LLMs and are trying to optimize whats subsidized even on the Max plans are customers Anthropic is honestly better off without.

Ok thats a one month clock to the next Opus model at least, so thats a silver lining to a meh model.

Spoiler: more than half of it is just Meta ads spending cause its just addictive.

Seems really good at frontend work, and as a result on remotion programmatic videos. Not the best yet, thats still Gemini 3.1 pro(trained on actual videos) or Fable, but often better than what Opus can come up with

https://mesmer.tools/benchmarks/ai-video-generation

noticed it cause of ad manager, the main facebook site being down is kinda weird tho. I don't remember when was the last time that happened

Overall an improvement over Opus 4.8, but I'd still say Gemini 3.1 Pro has more of an artistic vision even tho it fails tool calls and writes buggy code sometimes.

Ik almost everyone is interested just in the SWE stuff, but this has been a good eval for me to think about how big the model is, how "creative" it is for generating new ideas etc.

More results from fable, with comparisons for Gemini, opus and some open source models: https://mesmer.tools/benchmarks/ai-video-generation

I think gpt 5.6 is coming out today so might wanna wait

/model claude-opus-4-8

seems to work but idk why they never set it so you can see it in the /model list.

"what model are you

I'm Claude Opus (claude-opus-4-8), running in Claude Code."

Yep sorry was just pulling it out my rear, not like a market trend that nearly every enterprise uses Anthropic or Openai models for coding or that Anthropic has had such ridiculous growth that they're 10x-ing year over year

My point was that even openrouter, the one place people who are looking for open source SOTA models go to, doesn't definitively have opensource models at the top. Esp considering quite a lot of the closed models usage is through AWS, GCP , Azure etc, probably dwarfing the usage on openrouter by a huge factor

As long as closed source is 6 months ahead in terms of current difference. Although this is hard to figure out using simple percent based coding benchmarks, you def. notice it when you're actually trying to do a long task. Even simple things like UI "taste" is enough for me to use opus instead of 5.5 though even though 5.5 is strictly better for anything that doesn't have a UI, ie backend, scripts, making agent workflows etc

As long as closed models are 6 months ahead I won't be switching from them to prev. 6 month SOTA open source models. Maybe its just a different calculation if you're in a job, but as an indiehacker I'll take any edge I can get

Ofc again, can be convinced to switch if there's however a clear speed difference, like 5x+ for a open source sota even if it was SOTA for 6 months ago

Based on current market for LLMs I'd say my use of "you" in the general is fine. Even openrouter which doesn't capture all of the SOTA closed models but nearly all of opensource model usage has Opus as 1st(on last week) on "Programming" category and 3rd in overall rankings

https://openrouter.ai/rankings

Cost for the value delivered. Like if you offered the current SOTA open source models at $0.1/M, I still think I'd be using Opus or 5.5 at $30/M. Or say GPT 5 which was released Aug 25, I don't think I'd use it for coding for even $0.1. I'd def find other uses for it(translations, agentic workflows, prompt guards etc), but for coding I don't think I'd ever completely switch to a SOTA open model

Unless ofc there was an actual speed difference, only reason I'd be willing to go with a worse model couple of percent worse than current best model is if the speed was at least 5x higher. Looking forward to kimi k2.6 offered publicly by Cerebras

For coding you always want to go with the best model in the category, not something that would be the best model if we went 1 year back which GLM 5.1 is, and I'm saying that as a big fan of GLM cause I run a translation site where GLM is good enough for the price.

Most of the money right now is in coding. Openai and Anthropic just have to be 6 months ahead of SOTA open source models and they'll capture most of the enterprise and dev market

If nothing else this blog did give me the idea that I should split my $200 claude max plan into two $100 CC max and $100 codex plan, esp because Claude is now offering 1.5x weekly limits so its the 5x usage is now more like 7.5x usage.

I'm just hoping they release Mythos soon now that it seems like they have enough compute to do promotions like this

I think that was a typo on my end, its "/model claude-opus-4-7" not "/model claude-opus-4.7"

I'm on the max $200 plan, so maybe its that?