what value do we get from AI?
It's unlikely I would have had the confidence to do it from YouTube alone, specific diagnostic help and a full diagram to work off of was extremely helpful. I had it prepare an SVG of the whole assembly with all the measurements and parts labeled.
That’s the thing with AI - its responses sound plausible enough to non-experts but time and time again I see experts in any given field being able to identify AI content by pinpointing subtle but crucial errors. That’s one of its dangers - it gives you enough confidence to shoot yourself in the foot.
Also, ask people in the trades to review each other's jobs. They will harshly criticize each other too for missing basic things and then go on to vehemently disagree. As an outsider it doesn't mean much that an expert found some fault. They always find something to nitpick.
Yes, but with a human worker there is a chain of accountability. With AI, there is none.
Maybe, but that's part of the experience of learning. I plan to maintain pools for the rest of my life, if I made an oversight which costs me down the line then the lesson will be that much more memorable.
This is an above-ground pool with a pump and a filter, the stakes are relatively low. In the absolute worst case I could rip it all out and pay a pro to do it for the price I was quoted.
I guess what you're talking about is sanitizer, in my case we use chlorine. I test it every time we swim, but I wouldn't have needed an LLM for that. It's very straight forward to maintain pool chlorine, my Dad taught me that when I was 13.
The previous system was also installed by a non-professional and was mostly tubes. It leaked to all hell and looked generally redneck and awful.
First step was draining the pool and doing a nice deep clean. Then I ripped out all the original plumbing until it was just the pool outlets, pump, and filter.
I arranged it all and measured the dimensions. I fed the figures along with a tonne of photos and explanation to GPT. I spent a while talking pros/cons and landed on a design which lined up with what I'd seen on YouTube. I had it prepare me a full shopping list of PVC, tools, cements, etc. all linked to a local pool dealer. I picked it up the next day.
The PVC was all cut with a chop-saw then primed and cemented together. I found this part easier than I would have expected. I put a layer of TigerFlex hose between the PVC manifold and the pump/pool/filter inlets so it had some tolerance.
We've been swimming in it all summer, no issues whatsoever so far.
I expect that it's good for common use cases that's well documented. But then, so are a lot of other approaches.
Yeah, I would imagine pinball machines are at least an order of magnitude more complicated to maintain than above-ground pools are. That situation sounds really annoying, I feel for you.
Pool technology has been more stable, and there are more people out there writing well-informed content for LLMs to extract and present as their own.
I know very little about pinball machines, out of curiosity, do some come with any amount of schematics or diagrams?
https://www.youtube.com/playlist?list=PLv0jwu7G_DFVAUoqtVxFV...
I would love to put them on a kiosk in the museum.
The internet has a lot of scans of wildly varying quality. The AI industry's hunger for data means there has been a lot of progress in OCR and data extraction, so I have notions of taking something like PaddleOCR and trying to turn the scans into a cohesive reference site that will work well on phones, etc. But I'm not sure when I will get to it.
If people out there are interested in working on a project like that, let me know. Email me at william@theflip.museum.
The quality it is today, is the worst it's ever going to be.
Bubbles are not a great time to form intuitions. WebVan [1] and Kozmo [2] also seemed to herald a new age. Decades later, brick-and-mortar grocery stores and convenience stores are still doing fine.
> The quality it is today, is the worst it's ever going to be.
A few years ago it would have been insane to say that you got help from Google’s I’m Feeling Lucky button to fix your outdoor plumbing.
The quality it is today, is the worst it's ever going to be.a.k.a. the third? fourth? AI summer
This was for a largish (40,000L) above-ground pool with no hookup to my home's plumbing system. The water was all pumped in from a water truck. The previous system was also installed by a non-professional and was mostly tubes. It leaked to all hell and looked generally redneck and awful.
First step was draining the pool and doing a nice deep clean. Then I ripped out all the original plumbing until it was just the pool outlets, pump, and filter.
I arranged it all and measured the dimensions. I fed the figures along with a tonne of photos and explanation to GPT. I spent a while talking pros/cons and landed on a design which lined up with what I'd seen on YouTube. I had it prepare me a full shopping list of PVC, tools, cements, etc. all linked to a local pool dealer. I picked it up the next day.
The PVC was all cut with a chop-saw then primed and cemented together. I found this part easier than I would have expected. I put a layer of TigerFlex hose between the PVC manifold and the pump/pool/filter inlets so it had some tolerance.
We've been swimming in it all summer, no issues whatsoever so far.
I find it telling that the highest praise for LLMs comes from people using it for something where they admittedly have very little domain knowledge. Domain experts usually mention major caveats. I've been testing them on subjects where I already understand the problem well, and I've yet to see any outputs that would make me trust them on things I don't already know.
Anyway, my pool's looking great and I gained some new skills. I probably could have gotten there with books and YouTube alone, but having another tool at my disposal made me a bit more confident.
"LLMs seem good at things you are not good at."
So, if you've never touched PVC before, LLM sounds plausibly competent--it may actually be or it may not be, but you'll walk away from it thinking you learned something. If you are a professional plumber and ask an LLM the same thing, the output will more look flawed and possibly dangerous.
Same for software writing: If you're not a good software developer, you probably think an LLM is great and writes much better code faster than a human developer can. But if you are a good software developer, LLM output is slop and requires huge rework to be passable.
I am not a mechanic by trade, but I know nearly everything there is to know about working on an ICE car. I did paint cars professionally for a bit.
LLMs are absolutely full of garbage advice, when I try to use it for troubleshooting. However, people who don't know anything about cars are telling me it helped them fix issues. I am assuming their issues were maybe surface level and something I would just know without even looking at any manuals, because when I use it for complex problems it just doesn't work for me.
well to be fair, it's sounds about as competent as your average homedepot employee. He's wasn't doing something super complicated, cutting and gluing PVC for an above ground pool is very common and doesn't require a plumber. I used some youtube videos to fix my dishwasher, i didn't need a professional service agent from the manufacturer I just needed some pointers.
as for software writing, for standard everyday enterprise app work which is typically just CRUD and moving data around it works fine. That kind of software does not need to be a highly tuned work of art to meet the requirements.
Why were you not confident with literal how to videos and documentation, but became confident when a chatbot generated probable text?
https://siliconreckoner.substack.com/p/terence-tao-on-machin...
To me it seems like people who are confident in their expertise generally find it useful, even if it's imperfect
For instance, in the domain of software engineering: I would not trust it to implement a major architectural change, or a groundbreaking, complex new feature. I would trust it more (but not completely) on something like a refactoring that may touch thousands of lines in a fairly mechanistic way, but that was a little too-complicated for simpler tools likes regexes. While that's kind of a nifty use, I think it's fair to say that non-LLM software purpose-built for such tasks can probably do the same thing more effectively for less real cost (meaning the currently-subsidized real cost of all the training and inference power burn, etc)
But the education system - at least for me, 20ish years ago - was very much tiered or broken up into classism: if you were highly intelligent or a good learner you'd go to advanced schools where you'd get higher level math, latin, etc. If you were "dumb" you'd get taught how to do woodworking and masonry. At best I was taught how to use a figure saw, drill press safety measures, and how to patch an inner tube (welcome to the Netherlands, this is very important. Or, was, it's much simpler and cheaper to just buy a new inner tube nowadays).
The fellow who'd done it originally is my neighbor and he's a retired teacher. There's a lot of ways to learn this stuff - my one takeaway has been that it's much easier than people make it out to be.
Except, for people who were gently motivated, that information was already pretty well democratized by libraries. You could trivially go to the library, and get whatever books were published on a topic, even if the only copy was on the other side of the country. Tons of the famous names from previous decades got their start teaching themselves things from a book in a library. It was very common in the technical churn of the 20th century that a new project at work meant you went to the library and grabbed books on a brand new topic and self-taught. This for example is how some programmers in the 90s developed 3D engines.
There was even a short period of human history where it was common to pay a few thousand dollars for a family encyclopedia. I got my start reading an 80s encyclopedia, focusing on the more technical tomes, before I found Wikipedia. The drive to access and learn information lead to me learning about computers in a time and place where a formal education on the subject was unavailable to me. I owe my career to it.
After the existence of Ebay, a few hundred dollars could populate a shelf with the standard reference books and material for nearly any interest. All it took was a willingness to look for books, buy them, and sit down and read them.
Similarly, the internet did the same since the 90s. Specifically, it allowed for non-physical clubs to supplement the fact that not everyone lived in Silicon Valley and could access those rich clubs on niche topics. But special interest magazines were already providing some of that functionality.
The primary filtering LLMs do is provide new access and ability to people who are far too lazy and unmotivated to do the real work necessary to learn about something without being literally spoon fed.
This helps explain why the primary thing LLMs have done is increase the noise floor of information, and explains differing sentiments. People willing to put in minimal effort to learn things had zero issue learning new information in the previous regime, so aren't that impressed when an LLM regurgitates the wikipedia intro paragraph or summarizes a popular reference. They already read that. They note that the LLMs confidence is often unwarranted, and they get reasonable results because they have foundational understanding of the domain and already know which pitfalls and problems to be concerned about, and how to prompt the LLM to make the right choices.
For people who largely are unwilling to take minimum effort to learn something new, of course LLMs feel magical, because Wikipedia level introductions to topics are magic to people who aren't already seeking them out. Of course, the question is, what in the world was previously stopping you from learning new things?
[0] https://tomtilley.net/projects/pvc/
(found via https://wiby.me/)
I'm not sure if the sites you linked would have been immediately very helpful in my specific case. You know this is for a pool, right?
The first is called "Everyday Uses for PVC Water Pipe" and has some cool ideas like using PVC for Wiimote holders or tridents, but I don't think that's super relevant for this project.
The second is a great forum which I'm already familiar with, but again, the topic is household plumbing and from a brief skim, none of the topics mention pools. I think mine would have been out of place.
I appreciate you trying to help!
People have been DIYing swimming pools for decades with the help of… books:
* https://www.amazon.com/COMPLETE-GUIDE-SWIMMING-CONSTRUCTION-...
Where do you think ChatGPT got its information from?
Perhaps with good reason?
The problem as usual is the mass of rich people trying to profit off of them, not the technology itself.
And apparently is more than adequate to its gulled users.
If ever there were three adjectives which do NOT apply to generative AI...
select l.id, l.name
from locations as l
where l.type = 'amusement_park'
and exists (
select id from locations as l2
where l2.type = 'rv_parking'
and distance(l2.geo, l.geo) < $nearby_distance
)
The hard job is collecting the data.
Only a novice would look at ai code and say “wow this is good”.
For a lot of problems "quick and good enough" is all that is required. I've used it a lot for managing my Home Assistant setup. Has saved me countless of hours.
In principle I could've done it myself, but I never would have, the time investment required to learn it wouldn't have been worth the value I get from it.
Note that I am talking about frontier models and only the last 6-12 months. Opus was really the breakthrough point for me.
A pain plate of rice is bad food. But would you prefer starving too death? Or for others to starve?
Or saying that a car is a better vehicle than a bicycle. That is probably true, but many times for many people a bicycle is all they can get.
When you discuss design and architecture first, and write that out in a design doc or something along those lines, it works quite well for the most part.
And 9 out of 10 times when it produces some poor results, just asking "is this really a good approach?" or just stating "This code makes me very sad" it will most of the time do a really good job of analysing why that code is bad and how to improve it.
And you should know that this is not accurate. But at this point this has devolved into a dick measuring contest, which I refuse to do. Have a good day.
Funnily enough, I've had a somewhat mixed-to-hostile response when trying to upstream the vibecoded fixes, so I suspect using an LLM to fix broken open-source software (that human maintainers don't have the time to fix themselves, nor the humility to accept an LLM-authored fix) will become more of a thing going forward too.
Oh, it's also been identifying a bunch of patterns in sales data for my business that has been increasing monthly profit consistently since last November (around $4,000 USD, every month, cumulatively so far with no sign of slowing down - could easily be $10k/mo in increased gains by end of financial year).
I tend to respond quite well to AI-authored or assisted PRs to my project, but to be fair we maybe only get 3-5 PRs in a good month.
Thinking it through a little bit is probably coming from having bounties on a few issues that might be contributing to my negative experience.