Hacker News Reader

red2awnon Tongyi DeepResearch – open-source 30B MoE Model that rivals OpenAI DeepResearch

The moat of OpenAI is 1. internal knowledge they've built over the last few years building front tier models 2. their talent 3. the ChatGPT brand (go ask a random person on the street, they know ChatGPT but not Claude or Gemini)

red2awnon Character.ai to bar children under 18 from using its chatbots

They are not a profitable company at all. They only started monetizing this year and the customer base is not a fan of it at all.

red2awnon AdapTive-LeArning Speculator System (ATLAS): Faster LLM inference

A lot of optimizations in LLMs now are low hanging fruits inspired by techniques in classical computer science. Another one that comes to mind is paged KV caching which is based on memory paging.

red2awnon Who needs Git when you have 1M context windows?

Technically it doesn't have to be since that part of the context window would have been in the KV cache and the inference provider could have thrown away the textual input.

red2awnon GPT-2 implementation in Modular MAX

I am learning to write LLM pipelines using the Modular MAX inference framework. As a starting point I got GPT-2 working after reading through "The Illustrated GPT-2", Karpathy's nanoGPT codebase and existing models in the Modular repo. The MAX framework does require a lot of boilerplates and not designed to be very flexible, but you do gain awesome performance out of the box.

red2awnon Defeating Nondeterminism in LLM Inference

The blog post is about LLM non-determinism in the context of serving at scale (variable batch size). The page you link is only about run-to-run determinism implicitly assuming a fixed batch size.

red2awnon Defeating Nondeterminism in LLM Inference

Conceptually setting temperature to be >0 doesn't actually introduce any non-determinism. If your sampler is seeded then it will always choose the same next token. Higher temperature only flattens the logit distribution.

red2awnon ML needs a new programming language – Interview with Chris Lattner

I don't think they were intentionally rug pulling the community. More likely they started out thinking they can build a python superset, realised it wasn't a good idea so quickly pivoted to the current design. Making Mojo more Python-like is now on their long term roadmap (https://docs.modular.com/mojo/roadmap/#phase-3-dynamic-objec...).

red2awnon Anthropic raises $13B Series F

If it only takes 30k can't you bootstrap and built it yourself? Or even just work on it on the side alongside your day job.

red2awnon Are OpenAI and Anthropic losing money on inference?

It also didn't take into account a lot of the new models are reasoning models which spits out a lot of output tokens.

red2awnon Mira Murati’s AI startup Thinking Machines valued at $12B in early-stage funding

It is literally in the article:

> "We're excited that in the next couple months we will be able to share our first product, which will include a significant open source component and be useful for researchers and startups developing custom model," CEO Murati said in a post on the X social media platform.

red2awnon All You Need Is 4x 4090 GPUs to Train Your Own Model

> Architecture Advantages: Enhanced ray tracing, Shader Execution Reordering, and DLSS 3 technology for improved efficiency.

This jumps right out as written by AI, these features have nothing to do with training LLMs.

red2awnon OpenAI spent $10M on chat.com URL

"Chat (formerly ChatGPT)"

red2awnon Unit tests as documentation

Better yet, use doc test as featured in Python [1] or Rust [2]. This makes sure your documentation examples are always up-to-date and runnable.

[1]: https://docs.python.org/3/library/doctest.html

[2]: https://doc.rust-lang.org/rustdoc/write-documentation/docume...

red2awnon PixelVerse t1 – CoT prompting outperforms flagship LLMs

prompt: how many "r"s in the word "raspberry"?

response: There are 2 "r"s in the word "raspberry".

red2awnon Features I'd like to see in future IDEs

The helix editor [1] allows you to navigate and select code objects, I use Alt-o all the time which expand the current selection to the parent syntax node. As mentioned by sibling comments, there are also a bunch of diff tools that are syntax aware instead of operating on plain text. All of these are backed by tree-sitter.

[1]: https://helix-editor.com/

red2awnon How Uber tests payments in production

Stopped reading after this

> The reason I know this is because I’ve built and maintained systems that handle close to 100,000 payments a day.

That's 1.16 payments per second.

red2awnon Preliminary Post Incident Review

> How Do We Prevent This From Happening Again?

> Software Resiliency and Testing

> * Improve Rapid Response Content testing by using testing types such as:

> * Local developer testing

So no one actually tested the changes before deploying?!

red2awnon Codestral Mamba

A very good primer to state-space models (from which Mamba is based on) is The Annotated S4 [1]. If you want to dive into the code I wrote a minimal single-file implementation of Mamba-2 here [2].

[1]: https://srush.github.io/annotated-s4/

[2]: https://github.com/tommyip/mamba2-minimal

red2awnon Apple's new AI leaves out more than 90% of current iPhone users

It is exactly what this is, pleasing the shareholders. From Tim Cook himself: https://youtu.be/pMX2cQdPubk?t=146

> It became clear that people wanted to know our views on generative AI. So we decided to embrace it and call it Apple Intelligence.

red2awnon Apple's AI Strategy in a Nutshell

The article seems to be based on misinterpreted information:

> Apple’s model has extremely low latency (0.6 milliseconds to first token), outperforms similar sized Phi and Gemini models from Microsoft and Google.

From [1] it is 0.6 millisecond per prompt token so unless the prompt is one token the latency would be higher.

> Apple’s server-side models running in the Private Cloud Compute are apparently quite near the GPT-4o in terms of quality

The benchmarks from [1] never mentioned GPT-4o, the best model they compared to for the server model is GPT-4-0125. If their model almost matches GPT-4o they wouldn't need to integrate with ChatGPT.

[1]: https://machinelearning.apple.com/research/introducing-apple...

red2awnon Alacritty – A fast, cross-platform, OpenGL terminal emulator

kitty is being rewritten in Rust from C.

red2awnon Show HN: A simple, open-source Notion-like avatar generator

The first one I got was holding a knife...

red2awnon Meet the $4 Billion AI Superstars That Google Lost

It is a Blockchain project. https://near.org/

red2awnon Coming Soon: Fedora for Apple Silicon Macs

Just use the Rectangle app, problem solved. It would be nice to have it built in, but it is not a big deal.

red2awnon {n} times faster than C

I experimented with different optimizations and ended with 128x speedup. The improvement mainly comes from manual SIMD intrinsics, but you can go a long way just by making the code more auto-vectorization friendly as some other comments have mentioned. See:

https://ipthomas.com/blog/2023/07/n-times-faster-than-c-wher...

red2awnon Mark Zuckerberg: “We’re moving very slowly” (2012)

Probably cloned from China.

red2awnon How an obscure OTC-traded derivative from the 80s took over crypto

Through "funding". If the perpetual future's price is above (below) spot's price, long holders pay (receive) short sellers an interest at a regular interval (every 8 hours at most crypto exchanges).

red2awnon Ask HN: What's a good program for calisthenics for beginners that you have tried

The Armstrong Pullup Program [0]. It is designed specifically to increase your pull up numbers. I went from being able to do ~10 (straight) pull ups to 17.

[0]: https://armstrongpullupprogram.com/