Hacker News Reader

vrmon Who's afraid of Chinese models?

good luck running a 2.4T model on any local hardware. it’s not gonna happen. the arrow is to specialized hardware at least for the smartest models

vrmon Building durable workflows on Postgres

TBH it's intended only for internal use (we don't even publish it as a crate at this point) so I don't particularly mind it being low-key. But I appreciate it!

vrmon Building durable workflows on Postgres

If you don't need a ton of throughput I think `absurd` (and our Rust derivative `durable`) are very nice options that keep the client side extremely simple. It's also lightweight enough that a coding agent can keep the entire thing in its head easily and just run queries to look up state as needed.

vrmon Building durable workflows on Postgres

Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable. It has been quite stable and extensible but of course you need to be very careful with the SQL implementations. Hope this is interesting to readers here.

vrmon Making deep learning go brrrr from first principles (2022)

It’s really not a concept you can express in idiomatic Python very easily. This comes from the actual generated assembly involving copies from global GPU memory into registers (slow, bandwidth saturates quickly) and back in between the cosines. If you can avoid the intermediate roundtrip that cuts the cost approximately in half.

vrmon Formal Verification Gates for AI Coding Loops

One question I have here: I think this type of thing would be trivial to do in Rust with constructors, private fields, and newtypes. What am I getting on top of it?

vrmon Databricks is raising a Series K Investment at >$100B valuation

that is earnings (net income) not revenue (top line) so these are wildly different and incomparable numbers

vrmon Ask HN: How can ChatGPT serve 700M users when I can't run one GPT-4 locally?

a 6:1 parameter ratio is too small for specdec to have that much of an effect. You'd really want to see 10:1 or even more for this to start to matter

vrmon Diffsitter – A Tree-sitter based AST difftool to get meaningful semantic diffs

This is neat! I think in general there are really deep connections between semantically meaningful diffs (across modalities) and supervision of AI models. You might imagine a human-in-the-loop workflow where the human makes edits to a particular generation and then those edits are used as supervision for a future implementation of that thing. We did some related work here: https://www.tensorzero.com/blog/automatically-evaluating-ai-... on the coding use case but I'm interested in all the different approaches to the problem and especially on less structured domains.

vrmon Reverse Engineering Cursor's LLM Client

if you haven't check out our repo -- it's free, fully self-hosted, production-grade, and designed for precisely this application :)

https://github.com/TensorZero/tensorzero

vrmon Reverse Engineering Cursor's LLM Client

I definitely see different prompts based on what I'm doing in the app. As we mentioned there are different prompts for if you're asking questions, doing Cmd-K edits, working in the shell, etc. I'd also imagine that they customize the prompt by model (unobserved here, but we can also customize per-model using TensorZero and A/B test).

vrmon Reverse Engineering Cursor's LLM Client

we're doing the latter! Cursor lets you configure the OpenAI base URL so we were able to have Cursor call Ngrok -> Nginx (for auth) -> TensorZero -> LLMs. We explain in detail in the blog post.

vrmon Reverse Engineering Cursor's LLM Client

wireshark would work for seeing the requests from the desktop app to Cursor’s servers (which make the actual LLM requests). But if you’re interested in what the actual requests to LLMs look like from Cursor’s servers you have to set something like this up. Plus, this lets us modify the request and A/B test variations!

vrmon AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms

We're working on an OSS industrial-grade version of this at TensorZero but there's a long way to go. I think the easiest out of the box solution today is probably OpenAI RFT but that's a partial solve with substantial vendor lock-in.

vrmon AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms

This is very neat work! Will be interested in how they make this sort of thing available to the public but it is clear from some of the results they mention that search + LLM is one path to the production of net-new knowledge from AI systems.

vrmon Show HN: Autarkie – Instant grammar fuzzing using Rust macros

would it be possible to fuzz an arbitrary JSON schema with this? I’ve been looking for such a lib for a while now

vrmon Spinlaunch with Basic Physics [pdf]

OP here: I saw the spinlaunch video and got really excited that this was completely tractable in simplified form with basic physics. So I did the math expecting to see a huge savings in fuel mass and....it was basically negligible. I'm curious if anyone on HN (maybe even from Spinlaunch!) could explain where I go wrong. Otherwise you might find it interesting to read. Thanks!

vrmon The Truth About Nuclear Fusion Q Factors

Yeah so the paper I linked on contextual bayesian optimization (https://papers.nips.cc/paper/2019/hash/7876acb66640bad41f1e1...) does a combination of controlling for beta and optimizing linear MHD stability --- which is part of the problem. One of our collaborators has been working precisely on disruption prediction and mitigation but it's on most of our minds: https://www.pppl.gov/news/2021/artificial-intelligence-helps....

vrmon The Truth About Nuclear Fusion Q Factors

will repost my comment from the other one:

Papers I would recommend from our collaboration on control of normalized plasma pressure: https://papers.nips.cc/paper/2019/hash/7876acb66640bad41f1e1...

plasma profile transport modeling: https://iopscience.iop.org/article/10.1088/1741-4326/abe08d/...

hybrid dynamical modeling of gross plasma quantities: https://arxiv.org/abs/2006.12682

uncertainty quantification for plasma dynamics: https://arxiv.org/abs/2011.09588

It's still early days for this work and for us but we're looking at pushing reinforcement learning in methods and engineering to solve this problem.

vrmon The Truth About Nuclear Fusion Q Factors

I need to redo my website, just getting into the more public part of my PhD. Papers I would recommend from our collaboration on

control of normalized plasma pressure: https://papers.nips.cc/paper/2019/hash/7876acb66640bad41f1e1...

plasma profile transport modeling: https://iopscience.iop.org/article/10.1088/1741-4326/abe08d/...

hybrid dynamical modeling of gross plasma quantities: https://arxiv.org/abs/2006.12682

uncertainty quantification for plasma dynamics: https://arxiv.org/abs/2011.09588

It's still early days for this work and for us but we're looking at pushing reinforcement learning in methods and engineering to solve this problem

vrmon The Truth About Nuclear Fusion Q Factors

like I said above, we certainly hope so! It has been slow progress so far but applying modern ML / control techniques to tokamaks is one of the truly exciting applications of the current generation of AI in my opinion. Biased because this is literally what I do all day

vrmon The Truth About Nuclear Fusion Q Factors

I'm a PhD student in robotics at Carnegie Mellon working on exactly this. It's extremely challenging for a few reasons:

- the dataset is a mess. The experiments that have been conducted on the tokamak that we have access to were done for very many different reasons and under many different configurations of the machine so there is not a clear method for disambiguating what dynamical changes are due to differences in the system vs underlying dynamical truths

- the simulators available are very slow and not that accurate

- the physics is hard enough that it's not possible to develop a controller in closed form (obviously)

This implies that we need a version of reinforcement learning or model-predictive control that is substantially more robust and sample-efficient than currently exists. We're working on that but obviously it's an open research problem.

vrmon A Brand New Interview with David Foster Wallace

https://www.nytimes.com/2006/08/20/sports/playmagazine/20fed...

If you have any interest in tennis (or not, honestly), I'd say start here before buying another book. Read it as a kid and then had a magical revelation years later when I connected this piece to the man itself.

If you like it, then hop onto the rest of his work from nonfiction to short-stories to IJ if you have a spare couple months.

vrmon CS 522: Machine Learning Approaches to Decode the Human Genome

dude nice job getting exposure with this!

vrmon BeyondCorp: How Google Ditched VPNs for Remote Employee Access

As far as I recall, you can download code to your desktop but not your laptop, unless you’re doing something like iOS development that would be difficult to do over say ssh -Y.

vrmon Is GitHub down or just me?

just checked: it works at Stanford

vrmon CS 20SI: Tensorflow for Deep Learning Research

She's honestly one of the most amazing people I've ever met. Go chip!!