HNHacker News
TopNewBestAskShowJobs

elliotto

735 karma · joined September 15, 2020

Software engineer interested in machine learning and data science.
submissionscomments
elliotto··on Replacement.ai
The Grapes of Wrath by Steinbeck is largely about this concept. If these social transitions were as rosy as you paint them I think the book would be called Grapes of Pleasant Improvements in Living Standards.
elliotto··on Why do LLMs freak out over the seahorse emoji?
GPT-5 can't handle 2 things: an esoteric quine or an aquatic equine
elliotto··on Why do LLMs freak out over the seahorse emoji?
Very neat! A lot of small LLM's have a similar failure mode where they get stuck and repeat a token / get stuck in a 2-3 token loop until they hit the max message size cutoff. Very ironic that it's about a quine.
elliotto··on Why do LLMs freak out over the seahorse emoji?
I tried it with thinking mode and it seems like it spiraled wildly internally, then did a web search and worked it out.

https://chatgpt.com/share/68e3674f-c220-800f-888c-81760e161d...

elliotto··on Why do LLMs freak out over the seahorse emoji?
https://chatgpt.com/share/68e3674f-c220-800f-888c-81760e161d...

With thinking it spirals internally, runs a google search and then works it out.

elliotto··on Why do LLMs freak out over the seahorse emoji?
https://chatgpt.com/share/68e366b2-0fdc-800f-9bf3-86974703b6...

GPT-5 Instant (no thinking) spirals wildly. Poor bot

elliotto··on Why do LLMs freak out over the seahorse emoji?
I do this all the time. I start writing a comment then think about it some more and realize halfway through that I don't know what I'm saying

I have the luxury of a delete button - the LLM doesn't get that privilege.

elliotto··on Which table format do LLMs understand best?
These sort of experiments and results are really important for language model implementation. This has a tangible implication for my AI startup and how we approach tool design.

Much more important than citation farming a paper on 1 % improved performance

elliotto··on Benefits of choosing email over messaging
In my experience restrictive and developer hostile API structures are indicative of exploiting a monopoly position rather than some provided excuse like 'countering spam'
elliotto··on Americans increasingly see legal sports betting as a bad thing for society
The author approaches the topic from a libertarian perspective with some prior beliefs about the futility of regulation. These restrictions are not permitted within their ontology, which in my opinion makes the damning conclusion more powerful.

I strongly disagree with the authors political stance, which makes the fact that we agree on the problem / solution a nice bipartisan result

elliotto··on Ask HN: How do you keep up with exploding AI chat history?
I keep a few long running chats for important things and then just keep the random questions as individual chats. There's not a lot of structure but it works.

For example at the moment I have 1 long chat about some health stuff, 1 long chat about some bicycle repairs I'm doing, and then the rest are one offs. I rename the long ones (or just remember the name) and they stay at the top because I find them when I need.

I also manage memories assertively. If there's something important that I'm tired of repeating I just ask it to add it to memory and it will do it.

Most of my professional work is done in cursor so it's in a different place to my personal questions. These usually are one chat per feature kind of thing, and start a new one when it gets confused with the enormous context

elliotto··on Ask HN: How do you keep up with exploding AI chat history?
You can manage the memories in settings. I have to go through every now and then and purge weird stuff that it's latched onto
elliotto··on Americans increasingly see legal sports betting as a bad thing for society
https://www.lesswrong.com/posts/tHiB8jLocbPLagYDZ/the-online...

This less wrong piece by a libertarian who examines the numbers and struggles to reconcile them with his beliefs is one of the best indictments on sports betting.

There is a small proportion of the population who cannot handle this. And they become prey to the predators in the sports betting industry. These guys make money off destroying their lives.

elliotto··on Benefits of choosing email over messaging
I've often thought about building a messaging platform aggregator that takes conversations from Whatsapp/messenger/discord/Instagram DMs/etc and provides a unified interface for them. I suspect there's a bunch of legal and annoying auth things that make this impossible. But at its core these things are just arrays of strings
elliotto··on Autism, A.D.H.D., Anxiety: Can a Diagnosis Make You Better?
https://archive.md/0jytv
elliotto··on Ask HN: Resources at the Intersection of NLP and Philosophy
it might be slightly adjacent, but Stephen wolfram's blog posts are one of few sources I can find about the relationship between computation and philosophy. His entropy blogpost is one of my favourite pieces
elliotto··on The RAG Obituary: Killed by agents, buried by context windows
I work at an AI startup, and we've explored a solution where we preprocess documents to make a short summary of each document, then provide these summaries with a tool call instruction to the bot so it can decide which document is relevant. This seems to scale to a few hundred documents of 100k-1m tokens, but then we run into issues with context window size and rot. I've thought about extending this as a tree based structure, kind of like an LLM file system, but have other priorities at the moment.

Embeddings had some context size limitations in our case - we were looking at large technical manuals. Gemini was the first to have a 1m context window, but for some reason its embedding window is tiny. I suspect the embeddings might start to break down when there's too much information.

elliotto··on Terence Tao: The role of small organizations in society has shrunk significantly
Your example of firms setting up in the desert or wilderness assumes there is some desert or wilderness left to expand into. This is Marx's concepts of the expansion of capital. Marx argues that with nowhere to expand it begins to eat itself.

You posit that the situation has a political cause, but I think this is just what happens when a system requiring exponential growth reaches the limits of its bounding box.

elliotto··on Terence Tao: The role of small organizations in society has shrunk significantly
If a multi-agent game is played in which each agent grows proportionally to its current size, isn't the end result that small agents are eaten by big agents, and at the end you have one big agent? Ie the end state of Monopoly? In practice, systems crack apart by the time there's only a few agents left.

But it must be a wild ride to live while these cracks start to show. It probably looks like the greatest living mathematician making stream of thought tweets about how there aren't any small agents left.

elliotto··on Terence Tao: The role of small organizations in society has shrunk significantly
I agree with you, I live in Sydney Australia and there are plenty of small orgs and communities to participate in.

I wonder whether this post is just a reflection of Terry living in California, which from where I sit looks like an end-stage capitalist hellscape.

elliotto··on [dead]
because the original is paywalled. despite being an ad
elliotto··on How can England possibly be running out of water?
Food production is largely nationalised due to heavy subsidies provided to farmers. Different countries have different policies here, but in my country of Australia it meets the definition of a nationalised industry by everything but name.

Food distribution as not, and again in my country we are having constant investigations into monopolistic anti consumer behaviour by the large supermarkets.

elliotto··on Anduril's product engineering machine
There's perhaps another war going on right now that's a bit less popular, and a lot more profitable, that these guys are making bank on. I hope you sleep well knowing you support this team
elliotto··on Affiliates flock to scam gambling machine
The optimal strategy in a resource acquisition game with no rules is usually just to kill all the other players. This results in really bad nash equilibria where everyone is dead.

I think about this as entropic decay - the lowest energy state of a social system (Hobbes' state of nature).

Lucky it is possible to create social and economic systems that use cooperation to produce better individual and group utility (dissipative structures). But there is a particular arm of politics that is currently dismantling these to feast on their cores.

elliotto··on Do the simplest thing that could possibly work
Possibly, I mean in the book there's a hard number survey that says 37% of their sample described their job as not making a meaningful difference. It's a great book.

https://inthesetimes.com/article/capitalism-job-bullshit-dav...

elliotto··on Do the simplest thing that could possibly work
Do you reckon the KPI's and performance indicators used in big tech count as meaningful ways to measure performance? Wouldn't someone implementing a complex resume-driven project score highly on these measurements, despite a simpler solution being correct? I am not sure that job-hopping every 18 months to maximise TC (ie optimise against your incentives) is a great way to learn about long-term design and organisational implications.

I'm not saying that these jobs are bullshit in the same way that a VP of box-ticking is, just that it's not a conspiracy that a cathedral based on 'design-doc culture' might produce incentives that result in people who focus on maximising their performance on these fiscally rewarding dot points, rather than actualising their innate belief in performant and maintainable systems.

I work at a start-up so if my code doesn't run we don't get paid. This motivates me to write it well.

elliotto··on Do the simplest thing that could possibly work
I'd recommend reading bullshit jobs by David graeber. Most jobs in most organisations have an incentive structure for an individual to keep themselves employed rather than to actually solve problems.
elliotto··on Will AI Replace Human Thinking? The Case for Writing and Coding Manually
https://openai.com/index/chatgpt-study-mode/

Openai released study mode. I don't think it's anything special beyond a custom prompt that tells it to act as a teacher. But it's a good example of what these bots can do if you prompt them right.

The bots as they stand seem to be sycophantic and make a lot of assumptions (hallucinations) rather than asking for more clarification or instruction. This isn't really a core truth to bot behaviour, and is more based around adhering to American social norms for corporate communication - deference to authority etc. You can prompt the bots to behave more usefully for coding - one of my tricks is to tell the bot to ask me clarifying questions before writing any code. This prevents it from making assumptions around functionality that I haven't specified in the brief.

For non-coding use cases, I like exploring ideas with the bot, and then every now and then prompting it to steel-man the opposing view to make sure I'm not getting dragged down a rabbit hole. Then I can take the best of these ideas and form a conclusion - hegelian dialectics and all that.

elliotto··on A teen was suicidal. ChatGPT was the friend he confided in
As part of my role I watch a lot of people use LLMs and it's fascinating to see their different mental models for what the LLM can do. I suspect it's far easier to explore functionality with a chirpy assistant than an emotionless bot.

I suspect history will remember this as a huge and dangerous mistake, and we will transition to an era of stoic question answering bots that push back harder

elliotto··on A teen was suicidal. ChatGPT was the friend he confided in
I think the nature of a conversational interface that responds to natural language questions is fundamentally different to the idea that you talk to yourself while reading information sources. I'm not sure it's useful to dismiss the idea that we can talk with a machine.

The current generation of LLMs have had their controversies, but these are still pre alpha products, and I suspect in the future we will look back on releasing them unleashed as a mistake. There's no reason the mistakes they make today can't be improved upon.

If your experiences with learning from a machine are similar to mine, then we can both see a whole new world coming that's going to take advantage of this interface.

← PreviousPage 2 of 6Next →