eggdaft

What would you like to know?

Fine tuning can take a fraction of the resources required for training, so I think the original point stands.

Do they really need “free RLHF”? As I understand it, RLHF needs relatively little data to work and its quality matters - I would expect paid and trained labellers to do a much better job than Joey Keyboard clicking past a “which helped you more” prompt whilst trying to generate an email.

I’ve achieved everything I set out to in life, at least everything I have control over and set out to do when I was in my 20s.

I’ve added a new ambition after some thought, not sure if I’ll manage to squeeze it in.

But I see the second half of my life now as predominantly about “service”. I think it’s important for everyone to have a life, to do the things they love, to follow some ambition or passion and take care of themselves. But we then need to ensure we help others do the same.

So my focus now is on my parents, family members, my partner, and society. I’m very grateful I had the health and opportunity to build and see what I wanted. Now it’s payback. I still do selfish things - I have to in order to stay sane - but the focus is on service.

People at Apple didn’t earn much money? Jonny Ive?

If by “optional” you mean “optimal” then no, a GA is not guaranteed to find the optimal solution in the general case.

Working with universities won't slow your pace of development, because you can work directly with users on that.

But go-to-market, after the initial delay, uni's will drive sales fast at scale.

Great point. Someone needs to replace this measure.

Words? Minutes? Number of edits? Eg

Free - try 10 minutes active editing a month, great for trying it out

Light use - 120 minutes a month, perfect for jotting down a few things daily

Pro - 600 minutes a month, write an entire essay by voice

Ultra - unlimited. Make voice editing your main workflow and work 10 times faster

I’d consider dentistry first. It’s still an open market in terms of SaaS, and they tend to have the same computer sat there all day constantly switching between the patient and the machine.

Suggestion for go-to-market: if you haven’t already done so, try to sell this direct to universities. They are an absolute pain to deal with, but they have various obligations to show how they are supporting dyslexic students etc and this fits the bill perfectly.

Their lead times are long so I’d start establishing trials now, partner with a university on agreement of a discount and use them to develop the software.

Once you’ve got a few uni’s onboard, you can rapidly expand and they are very unlikely to churn as you’re serving such an important niche.

I know a very similar product that has had huge success doing this.

Classic Hacker News moment. Thanks Rob, love everything you do! Your recent SaaS Playbook is awesome.

Sounds like you might want to start a lifestyle business. You should read and listen to everything by Rob Walling, especially his “stair step approach”. Wish someone had told me this at your stage.

For meeting founders, find someone in the area you’re interested in and build stuff with them. Also consider using the YC founder network, but they may be too ambitious for you.

Lifestyle businesses are probably less dependent on cofounders for success.

You’ll need to work hard at your tech skills if they’ve atrophied. The good news is, this part is incredibly fun.

I think also in the west you can’t be as blatant. There are checks and balances. Every murder invites journalistic investigation, so plausible deniability is important should things get out of hand.

Citation needed for your first statement, certainly doesn’t rhyme true with what I hear in the UK.

As for mocking suicide… I think you’d have to talk to a professional about what they do it. Given David Kelly, Epstein, and many other high profile cases, it does seem to be as difficult as you imagine.

I’m actually really amazed by LLMs and think the world is going to change dramatically as a result.

But the “you won’t need to code” reminds me “you won’t need to learn to drive”.

It’s the messy interface with the real world in both cases that basically requires AGI.

If AGI is just a decade off then, yep, I won’t need to code. But a decade is a long time and, more importantly, we’re probably more than a decade away.

And even if it is “just round the corner”, worrying about not needing to code would be worrying about deckchairs on the titanic. AGI will probably mean the end of capitalism as we know it, so all bets are off at that point.

It’s wise to hedge a little but also realise that to date AI is just a coding productivity boost. The size of the boost depends on how trivial the code is. Most of the code I write isn’t trivial and AI is fairly useless at that, certainly it’s faster and more accurate to write it myself. You can get a 50% boost if you’re writing boiler plate all day, but then you have to wonder why you’re doing that in the first place.

The Pragmatic Programmer explains why plain text is the way to go.

Biggest issue with binary is the difficulty reading / manipulating it long-term.

Not sure why downvoted, maybe because it’s a bit of a personal attack on me.

Well, we can say everyone has motives but that does not mean some incentives are more misaligned than others. So it’s not really a counter argument to say “everyone has motives”.

I wouldn’t say it’s my “feelings”, but my common sense, which is mostly derived from wider reading.

Quite often we wish to overcomplicate subjects to make them seem sophisticated. But many topics just don’t lend them to the complex analysis of say, high energy physics, they’re just much more prosaic. It’s ok to say “someone who lives near the sea has accessed to an additional food source and that’s probably beneficial” (as a random example) without having to write a thesis on it.

I defer to Charlie Munger on this topic, as he expresses it so well.

That’s a fair point. I skimmed it and got the gist of their approach, but I didn’t deep dive. It’s very hard to do what they’re trying to do rigorously. The real problem lies in the formulation of the question. It also doesn’t feel like they’ve “steelmanned” the original thesis, which is suspicious (tbf it’s rare in most of academia to see folks genuinely try to disprove their own position).

No, just I have a lot of experience of academia so I’m wary of motives.

Diamond’s ideas seemed pretty intuitive to me. The idea that geography is the primary shaper of civilisations seems _very_ likely to be true in general. I’ve read other books on the impact of geography on military defence and how that shapes geopolitics, for example.

Having worked in academia for many years, when I see academics attack a book such as GGS my instinct is to question their motives. Academics can be very egotistical and territorial. A priori, it is very difficult to imagine a situation where academics would say to any book like Diamond’s “oh yes this is a really great point and we need to learn from this guy” and very very likely to say “this guy doesn’t know anything about (less important factors that I know a lot about)! What an idiot! He even gets the date of this event wrong, proving I am clever and he is wrong.”

So when I hear about academics attacking something that seems to be fairly obviously correct at least at a very general level, my instinct is not “oh, yes, this guy must be completely wrong” but rather “this guy is probably mostly right but of course there will be a lot of other factors and it will be very easy to pick apart his book if you so wish, especially if your ego relies on you finding fault with him.”

I’m not so easily dissuaded that GGS isn’t mostly right that geography and in particular agricultural factors are the broad strokes of history, rather than the less permanent and smaller scale “Gaussian noise” of cultural events that intuitively are likely to mostly even out over time.

I definitely do this a lot. I would say I’m somewhat self-centred, but not a sociopath!

I think I just make a connection with what they’re saying and then share my experience to commiserate, and to find wisdom/learnings from that experience to see if they apply.

That’s my conscious assessment anyway. But I feel like I shouldn’t do it.

Anyone have tips to avoid this? How do I prevent myself from relating their experience to my own?

I feel that being able to completely ignore your own experience (or even opinion) and adapt purely to what they need at this moment is true selflessness, and it’s something I’ve always struggled with.

(Ironic comment I know).

Was BS actually of any practical use? It just seems to be a fantasy like most maths in economics.

It’s interesting to note what we can learn from the use of tables for layout.

Tables have border, padding, and a cell-based layout arranged in columns. At a stretch they and their cells need white space around them, ie “margins”.

Tables are a long established element in layout.

And it turns out tables actually contain this wisdom, that all elements should work a bit like table cells: they should follow the box model (fairly obvious to anyone working in typography) but also that they should be arranged in columns (much less obvious and the foundation of grid layout and bootstrap).

It’s a natural evolution to come across some layout problems, solve them with tables (as any typesetting system is likely to support tables), and then realise that you need a new facility that has some of the properties of tables but not all of them.

CSS has discovered this. Oddly, some systems have not - for example, word processors have not unified elements under a box model etc.

Aside: this is exactly the kind of query that ChatGPT is so good at. If you haven’t tried “I’m trying to remember the name of this thing…” please could you see if it works?

MS growth has been driven by multiple factors.

Lost me at why OpenAI not being floated makes any difference to my analysis?

Yes. This article demonstrates the problem of trying to reason by simple earning analysis alone. You can have to consider the dynamics of the businesses involved.

OpenAI being worth $100B doesn’t seem ridiculous at all to me. There is a lot of risk investing in such a stock, but given it has gone from nowhere to over $1B in revenue in such a short time, and it has a very strong brand position, is partnered with one of the best run tech companies, has a unique training pipeline, has fantastic connections to important institutions, is the first mover, and has an array of impressive products, you would expect that 1B to easily reach 10B, and reaching revenue of 100B doesn’t seem far fetched.

Taking another analysis: if they manage to charge $50 per business user, which isn’t too far from current charges and not that different from say what Microsoft costs, and is a small amount to recoup in productivity gains, then they need 150 million or so users. Office has double that. And of course we can see many other revenue sources, like education.

Their biggest challenge will be competition, and that makes it highly uncertain. It’s a very _risky_ stock, but the potential upside is very high. It is not a bubble if the price is driven by expectations of earnings growth.

I agree you’re right in that the current tech stack is bad. But the answer is to fix that, and I think (very) slowly we will.

JS was a terrible language but nowadays it is pretty good in some areas, passable in others, with some horror shows left in. The ecosystem and tooling are the real problems. I don’t understand why these issues haven’t been addressed by Google et al.

Native applications are just so much more expensive to develop and maintain. Given the nuances of different platforms, it’s also very difficult to keep different versions in sync. People expect collaboration as table stakes, which makes that task even harder.

I agree accessibility is a problem.

Ironically the Java applet had the right idea, as a compromise between our positions, but wrong time and maybe wrong language? Its Swiss cheese security didn’t help either.

Yeah the backlash against SPAs doesn’t make any sense. Server side rendering is fine for documentation etc. but full applications don’t want to be simulated on a server, client-server makes complete sense.

Think they replied to your comment accidentally

I’d roll back a bit on meat. It can be very fatty, particularly red meat. And it does significantly raise risks of various diseases, including cancer. If you’re going to eat it, I’d be very selective about the types and sources you consume.

Humans didn’t exist millions of years ago…