scottcha

Very cool paper and some interesting things for local serving. Though its a little apples-to-oranges we do publish live energy stats for all models on our service here https://portal.neuralwatt.com/energy-pricing in case you are interested in what this looks like on the cloud side. FWIW DSV4.1 flash is really getting popular due to its IPW.

Some of the items like model routing, if you do it per request instead of per session, can break down on the cloud from an energy and cost POV since one of the best things you can do for both is to maintain the KV cache which both reduces time component of energy and the quite expensive prefill energy.

I am keen on the future where we have local/cloud hybrid serving which is cache aware. I do think that could be the best use of energy resources for AI.

Neuralwatt | https://neuralwatt.com | REMOTE | Seattle or Denver/Boulder metros | Full-time

Hiring: 2 Engineers + Director of Sales & Operations

Energy is becoming one of the biggest constraints on AI infrastructure. Neuralwatt is building software to help datacenters get more useful AI compute and revenue from every kilowatt.

We're a VC-backed early-stage startup working on GPU optimization, AI infrastructure, energy-aware control systems, and the Neuralwatt Cloud.

$180k to $220k DOE + equity

You'll work on: Rust and Python, GPU, inference, and datacenter optimization, Production AI infrastructure, Energy-aware scheduling and control, Neuralwatt Cloud

Director of Sales & Operations

We're looking for someone with datacenter industry experience to build sales channels, partnerships, and customer relationships, support company operations, and help drive growth. Comfort using modern AI and agent tools is required.

Location: Remote-first, but you must be based in the Seattle or Denver/Boulder metro areas.

No visa sponsorship available.

To apply: hiring@neuralwatt.com Subject: HN Hiring (<your metro>/<engineering or sales>)

Include your resume, GitHub or relevant work profile, and a short note on why you're interested in AI and energy.

We use vllm as it generally has the best ecosystem support. Parameters are largely dependent on what type of requests you are serving (concurrency, input/output ratios, cached hit patterns). We've never had a limitation at the tokenizer step. Limitations at peak tend to manifest more on slower time in vllm doing prefill or decode though we actively try and minimize this.

I run an AI platform and we need to tokenize fast and early to make a lot of decisions on the subsequent steps (things like routing, rate limiting and such). Its really important to do this efficiently even though its not a large % of total end to end time for the request.

Hi, I'm co-founder of Neuralwatt. While there aren't other providers selling by energy we also produce the same tokens stats you get at others (input, output, cached) and you can compare using those. The CO2 number is really just multiplying the energy by the CO2-intensity at the time/place we served the request. So everything is actually measured and can be derived externally.

The main observation when you compare to token prices our input/cached energy is much lower than the equivalent token prices while the output energy is generally higher. One of the reasons this is a bit cheaper, especially for agentic systems, is that in an fully built/cached session your input & cached to output ratio is really massive input to very small output (multiple orders of magnitude usually). The other thing we do is we actively work to optimize the system around tokens/joule with the goal of making the most energy-optimal system.

Happy to answer any other questions.

They do have a growing amount of Scope 1 emissions (emissions from their on site sources) which originally was primarily on site diesel but due to grid interconnect delays have been growing number of on site gas turbines.

This certainly wouldn’t be necessary with adequate generation and transmission capacity.

Thanks for the feedback! Our primary focus is charging by energy, for token pricing we really just try to be close to the market. That being said I'll take a look at our token pricing to see if we need an update there https://portal.neuralwatt.com/energy-pricing Generally our users get much lower cost on energy than token pricing though on a typical request with a high prefix cache hit the input, cached costs is very small and the output energy cost is higher.

We definitely don't have any intention to obfuscate and in fact we actually try and provide more data than any other provider out there about both an individual request, as well as the fleet behavior. Since we tend to focus directly on our energy pricing and optimizing that the issue is likely where the ROI lies on energy optimization versus token optimization (totally correlated but we have other levers to reduce energy while keeping token counts the same).

Hi I'm the CTO of neuralwatt, would love to hear your feedback on what your experience was. Feel free to email me scott@neuralwatt.com. Also for GLM5.2 we run the FP8 quantization at 1M context which is a common deployment target.

Yes way better. We host both and while qwen3.6 is over 100tps we usually can do glm around that too.

I use glm5.1 plus pi with a few customized skills and am very happy with it. I hadn’t touched my Claude 5x plan for a couple of weeks but opened it back up in Claude code when fable was released and did a few tasks and still was happy to return to glm/pi.

I use claude code and pi.dev side by side most days and i'm mostly choosing pi for most work in last couple of weeks.

Pretty cool idea, but whats the stack behind this? As 15-25 tok/s seems a bit low as expected SoA for most providers is around 60 tok/s and quality of life dramatically improves above that.

I think there was a clarification posted on Reddit that said Claude Agents SDK didn't apply for now.

I use OpenCode and have just started using Nanoclaw with ClaudeCode (my coworker has a post coming on this) and sometimes ClaudeCode with Claude Code Router. I do a range of small to complex work with these but I also do drop back in to Claude Opus for some really complex things where I want it to be more autonomous.

Mine are pretty unique since we optimize the energy for and run an inference service api so forces me to dogfood alot of different options.

Yes GLM5 and KimiK2.5 are pretty close replacements for sonnet.

I switch between Claude Code (Opus/Sonnet) and Qwen (OpenCode, OpenClaw) multiple times throughout the day and Qwen 3.5 is really nice. I do also use KimiK2.5 and GLM5 pretty often too and I'm starting to get a sense that the agent tool is becoming a little more important than the model with these level of models. As long as tool calling and prompt quality is all configured correctly by the provider.

We offer multiple SOA models at https://portal.neuralwatt.com at very generous pricing since we have options to bill per kWh instead of per token. Recipes for your favorite tools here: https://github.com/neuralwatt/neuralwatt-tools

I actually built this analysis while I worked at Microsoft so I 100% agree. Doing the work at the platform level is the way to go and you can actually make a significant impact with this kind of approach. The other value of this that's not obvious is that doing it client side ends up touching all the grids/generators in the world outside of the market based accounting that tends to drive the datacenter carbon impact analysis.

There have been a few questions about the state of Show HN lately. Was actually interested in this post but I see all the OPs responses to questions are Dead? I do see its a new account but I don't really see anything egregious or against policy for these.

That is a pretty good article although the one factor not mentioned that we see that has a huge impact on energy is batch size but that would be hard to estimate with the data he has.

We've only launched to friends and family but I'll share this here since its relevant: we have a service which actually optimizes and measures the energy of your AI use: https://portal.neuralwatt.com if you want to check it out. We also have a tools repo we put together that shows some demonstrations of surfacing energy metadata in to your tools: https://github.com/neuralwatt/neuralwatt-tools/

Our underlying technology is really about OS level energy optimization and datacenter grid flexibility so if you are on the pay by KWHr plan you get additional value as we continue to roll new optimizations out.

DM me with your email and I'd be happy to add some additional credits to you.

Neuralwatt | https://neuralwatt.com | REMOTE (US – Seattle/Denver/Boulder metros only) | Full-time | $180k–$220k DOE Energy is the #1 constraint in new datacenter buildouts. Neuralwatt is reshaping AI compute around energy efficiency to maximize revenue per kilowatt. We’re a VC-backed, early-stage startup building optimization tools for AI, HPC, and datacenter workloads.

We're hiring 2 experienced founding engineers to help architect our core systems and work directly with customers.

What you'll do:

Technically: Architect critical datacenter infrastructure - Write Rust and Python - Measure real-world energy impact - Design state-of-the-art AI-led optimizations

Non-technically: Help build the business and win customers - Present at conferences - Develop marketing and company materials

Requirements:

- 5–10+ years of software development experience

- Thrive in ambiguous, outcome-driven environments

- Experience working closely with customers

- Clear communication and strong leadership

- Familiarity with LLM/AI infrastructure

Location: Remote-first, but we meet regularly in Seattle/Denver metro areas.

To apply: Email: scott@neuralwatt.com Subject: HN Hiring Include: - Resume - GitHub profile - A short note on why you're interested.

Please note: At this time, we are unable to offer visa sponsorship.

Neuralwatt | https://neuralwatt.com | REMOTE (US – Seattle/Denver/Boulder metros only) | Full-time | $180k–$220k DOE

Energy is the #1 constraint in new datacenter buildouts. Neuralwatt is reshaping AI compute around energy efficiency to maximize revenue per kilowatt. We’re a VC-backed, early-stage startup building optimization tools for AI, HPC, and datacenter workloads.

We're hiring 2 founding engineers to help architect our core systems and work directly with customers.

What you'll do:

Technically: Architect critical datacenter infrastructure - Write Rust and Python - Measure real-world energy impact - Design state-of-the-art AI-led optimizations

Non-technically: Help build the business and win customers - Present at conferences - Develop marketing and company materials

Requirements:

- 5–10+ years of software development experience

- Thrive in ambiguous, outcome-driven environments

- Experience working closely with customers

- Clear communication and strong leadership

- Familiarity with LLM/AI infrastructure

Location: Remote-first, but we meet regularly in Seattle/Denver metro areas.

To apply: Email: scott@neuralwatt.com Subject: HN Hiring Include: - Resume - GitHub profile - A short note on why you're interested

I’ve asked that question on linked in to the Cerebras team a couple times and haven’t ever received a response. There is system max tdp values posted online but I’m not sure you can assume the system is running in max tdp for these queries. If it is the numbers are quite high (I just tried to find the number but couldn’t find it but I had it in my notes as 23kw).

If someone from Cerebras is reading this feel free to dm me as optimizing this power is what we do.

Turns out there is multiple publications associating tachycardia and other heart symptoms with long covid. https://pmc.ncbi.nlm.nih.gov/articles/PMC8356730/

Maybe I’m a statistical anomaly or maybe I just don’t know the baseline occurrence rate for this stuff but I have 3 close acquaintances two of which are this persons age or younger with similar symptoms (tachycardia, though to a lesser degree) going on. Is there data on the incidence rates for this stuff and has it been increasing since 2021?

The are many great things about Aurora, here are a few as I've been using it since it came out. 1. Its open source & open weights and free to use non-commercially. 2. Its configurable to easily fit on my local gpu for development purposes. 3. I've also gotten great engagement from the repo owners.

Yeah, that was my first thought. I actually wrote a blog post a few weeks ago modeling the point at which agent recursion really gets out of control. https://www.neuralwatt.com/blog/agent-bedlam-a-future-of-end...

Shameless plug . . . I run a startup who is working to help this https://neuralwatt.com We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.

My Grandparents lived in a very small farming town (pop 500) and word would get around town when chicks had arrived and she would take us down there to see them.