4,164 karma · joined March 1, 2008
I play a weird part at work near AI. I use it all the time but I’m the first person to warn everyone that it’s absolutely not trustworthy. No matter your prompt, the data, the guidelines built into it, the output is fundamentally flaky. But I use it while knowing that and working around it. Making the process reliable is a big part of my focus, and usually that means minimising the part the LLM plays. Checks and balances live where things are predictable.
If it doesn’t work, you’ve tried and now you know, and you probably built a bunch of skills you can use.
I had quite a few people who said “I’ve never bought anything ever but I love your app so much I’ve paid for it.” Incredibly gratifying.
Although check out Patio11’s advice on charging more. Customers who spend more give you less shit.
All you’ll get are sluggish companies designed to suck up the funds. I used to work for such a company. The business model seemed to be whatever would fund hours of researchers time, and travel money for those smart enough to write papers for whatever conference was in a country they hadn’t visited yet. Targeting EU funds was top priority and they could do it by creating some arbitrary “consortium” that generated enough credibility with adjudicators. I hated the waste and the lack of actual goals.
“Thank you for your request. Can you walk me through the steps you’d use to do this manually? What things would you watch out for? What kind of number ranges are reasonable? I can propose an algorithm and you tell me if that’s correct. The admins have set up guidelines on how to reason about customer and purchase data. Is the following consistent with your expectations?”
- greybeard who is trying his hardest to keep up
I assume a useful goal would be to guide development of the system in coordination with experts, test it, have the AI explain all trade offs, potential bugs, sense check it against expected results etc.
Taste is hard to automate. Real insight is hard to automate. But a domain expert who isn’t an “analyst” can go extremely far with well designed automation and a sense of what rational results should look like. Obviously the state of the art isn’t perfect but you asked about goals, so those would be my goals.
I managed to add one startup and so far it’s done very well, but it was an exceptional case and the global CEO wanted the functionality. But it used MongoDB and ops team didn’t have any skills, so rather than learn one tiny thing for an irrelevant data store they added cash to use Atlas with all the support and RBAC etc etc. They couldn’t use the default Azure firewall because they only know one firewall, so added one of those too. Also loaded with contracts. Kept hiring load down, one number to call, job done. Startups cost is $5-10k per year. Support BS about $40k. (I forget the exact numbers but it dwarfed the startup costs.)
Startups are from Venus, enterprise are from Jupiter.
This might mean the $10/month is the best. Depends entirely on how it works for you.
(Caps obviously impact the total benefit so I agree there.)
Enterprise view: delegate AI environment to Databricks unless you’re a real player. Market is too chaotic, so rely on them to keep your innovation pipeline fed. Focus on building your own core data and AI within their environment. Nobody got fired for choosing Databricks.
This is already impossibly hard. Approximately zero people commenting would be able to win this battle in Sam’s shoes. What would they need to do to begin to have a chance? Rather than make all the obvious comments “bad evil man wants to get rich”, think what it would take to achieve the mission. What would you need to do in his shoes, aside from just give up and close up shop? Probably this, at the very least.
Edit: I don’t know the guy and many near YC do. So I accept there may be a lens I don’t have. But I’d rather discuss the problem, not the person.
The point, please focus on this point, is that it’s not a done deal that all accept free will. There exists a set of people who are smart and believe the opposite. I’m arguing against “everyone should think what I think”.
Edit: although to be honest if you convinced me otherwise I’d be 100% happy! “Most philosophers” (or my own “lots of smart people”) isn’t an argument I am moved by but I’ll go read up. My personal view is that the disappearance of time at the speed of light is incredibly powerful. I don’t feel we’ve explored the implications of that nearly enough. It either means the future impacts the past or super determinism is correct. Is there an alternative?
There are a lot of smart people who are less sure that our perceptions reflect reality. Free will, time, consciousness, space all feel very real. But when you interrogate them enough, those fall away. Go fast enough and your time slows down. Go as fast as a photon and time and distance disappear entirely. Yet we’re sure they’re real. At one point we absolutely knew the sun revolved around us. We could see it doing that daily.
One smart person on free will below. There are so many others. There’s certainly no consensus that “our experience = how things are”.
https://www.samharris.org/blog/the-illusion-of-free-will
The point is that our perception doesn’t provably equal “truth”. To make absolute statements you need to be a lot more sure and be more correct than all the smart people who say “not sure yet”.
Not feel. Not think. Not have a strong vibe. Proof.
I don’t have to prove anything here, because I’m not the one making an assertion. My thought experiment demonstrates a situation where an entity thinks it’s conscious but isn’t. Humans think we’re conscious but can’t prove it.
Argument 2:
Could we design a machine that is absolutely sure it’s conscious, but is not? If so, what differentiates us from it? Open question, I’m not sure. But I’d be surprised if we can be permanently sure.
It’s still better to have the principles than not have them, just keep balance in mind.
[1] for you, that is. People who don’t share your principles might not like your choices and you might be making their life difficult with your lack of balance. See religion, politics, etc.
There is certainly a contribution in improving how the body of work is represented but we treat the “AI” as the smarts, when really it’s a lens on the collective knowledge we have built. You can make the lens better, bit not claim ownership of the body of work. Right now that’s what’s happening, with a few edge cases for artists and publishers.
And by all I mean the AI companies owe a huge debt to all humans who wrote or designed or drew anything. The vast majority of the benefit of this technology relies on volume: the billions of pages and lines of code we wrote for other humans, but have now been repurposed. This technology relies on bulk, which was mainly unprofessional or freely given content, by those who intended it for other humans. It was not 100% built only on the output of the few who charge for their exquisite words or designs, even if their output is higher quality.
Alternatively, let the AI companies go for it but everyone who uses any kind of AI should understand that they’re standing on the shoulders of the millions of developers and nonprofessional writers whose work has now been repurposed. Not the few artists and journalists. So those artists and journalists should both refuse to contribute to, and use, AI.
* I’ve written very little of this useful content, but would be happy to pay my share to those that have built what we have. I also turn off training on my content, but I pay a lot for models. Feel free to help me think through this with comments of your own.
LLM’s are just so happy to generate enough tokens that look right ish. They need so many examples driven into them during training.
The map is not the territory, and we’re training them on the map of our codified outputs. They don’t actually have to survive. They’re pretty amazing but of course they’re absolutely not doing what we do, because success for us and them look so different. We need to survive.
(Please can we not have one that really wants to survive.)
It feels like I only have 5% of the control, and then it goes into a self-chat where it thinks it’s right and builds on it’s misunderstanding. So 95% of the outcome is driven by rambling, not my input.
Windsurf seems to do a good job of regularly injecting guidance so it sticks to what I’ve said. But I’ve had some extremely annoying interactions with confident-but-wrong “reasoning” models.
I’d be extremely surprised if other countries meekly do what Trump wants. There are many options on the table, and change is more likely in a crisis.
Eg how to get non technical people to know when they’re using the civilian version.
Alternative crazy universe: Just use the tech that was created for the government and does all the right things.