The barriers to AI engineering are crumbling fast
blog.helix.ml
blog.helix.ml
I have been running the 32B parameters qwen2.5-coder model on my 32G M2 Mac and and it is a huge help with coding.
The llama3.3-vision model does a great job processing screen shots. Small models like smollm2:latest can process a lot of text locally, very fast.
Open source front ends like Open WebUI are improving rapidly.
All the tools are lining up for do it yourself local AI.
The only commercial vendor right now that I think is doing a fairly good job at an integrated AI workflow is Google. Last month I had all my email directed to my gmail account, and the Gemini Advanced web app did a really good job integrating email, calendar, and google docs. Job well done. That said, I am back to using ProtonMail and trying to build local AIs for my workflows.
I am writing a book on the topic of local, personal, and private AIs.
I actually store the data in the EXIF as well, but the nice thing about having a database is that it's significantly faster than attempting to search hundreds of thousands of images across a nested file structure, particularly since I store a great deal of media on a NAS.
This is the prompt I've been using.
"Create a structured list of all of the people and things in the image and their main properties. Include a section transcribing any text. Include a section describing if the image is a photo, comic, art, or screenshot. Do not try to interpret, infer, or give subjective opinions. Only give direct, literal, objective descriptions of what you see."
Good luck with that. The only thing that I found that works is using gbnf to force it, which slows inference down considerably.
Still, I would really prefer everything running under my own control.
I did a quick and dirty prototype with Claud for this, but it returned everything with an offset and/or scaled.
Would be a killer app to be able to auto-fill any form using OCR.
Computer use is also not very good at it (often mis-clicking for example).
I'm guessing this will work flawlessly within 6 months to a year or so, but it doesn't seem ready yet.
i argue that its useful to have these shorthands to quickly understand what people do. its not so nice to just be default suspicious at a perhaps lower-technical-difficulty job that nevertheless a lot of comapnies want and a lot of people do
But we think it is, due to our learned industrious—which is the opposite of learned helplessness. We associate difficulty with reward. And we feel discomfort when really valuable things become very easy. Strange times.
Catchy job title.
A bad one is a SQL analyst whose been promoted too much.
I think there's not a ton of political palatability for realizing most of their projects are like one API and a sane use of some SQL away.
But none of that is relevant to my point, which is largely linguistic. If a person is doing a thing, then they are "a person who is doing the thing."
If a person is engineering, then they're an engineer by definition. That's not saying they're a good or bad engineer, just that's what they're doing.
There is a role for certifications, of course! But those certifications are intended to mark a lower boundary of ability, nothing more. Rather than overloading "engineer" to indicate that a person has a certification, I'd advocate calling them something like a "certified engineer". That would be correct and accurate, and wouldn't exclude a whole lot of great defacto engineers.
Let someone call themselves whatever they want. If they can do the job they were hired for then... who cares?
In my experience when people say things like this what they're just projecting insecurity
Well, most are called "project manager" now. But it would still be a giant red flag, just like the project manager job title or even worse, using PM so you don't know exactly what it means.
I'm only suspicious if they don't simultaneously and eagerly show me their Github so that I can see what they've accomplished.
Sometimes this industry is a lot like the "finance" industry: People struggling for credibility talk about it constantly, everywhere. They flex and bloviate and look for surrogates for accomplishments wherever they can be found. Peacocking on github, writing yet another tutorial on what tokens are and how embeddings work, etc.
That obviously doesn't mean in all cases, and there are loads of stellar talents that have a strong online presence. But by itself it is close to meaningless, and my experience is that it is usually a negative indicator.
But if you mostly mean in the sense that they don't have a fancy GitHub profile with a ton of followers... I agree, that does seem to be the case.
LinkedIn on the other hand... I sincerely can't imagine networking on LinkedIn being a fraction as useful as networking... Well, at work. For anyone that has a decent enough resume LinkedIn is basically only useful if you want a gutter trash news feed and to be spammed by dubious job offers 24/7. Could be wrong, but I'm starting to think you should really only stoop to LinkedIn in moments of desperation...
there is an entire industry of devs who work on meaningful projects, independently or for an employer who solve amazing problems and do amazing sh*t none of which is public or will ever be public.
I have signed more NDAs in my career than I have commits in public repos :)
I'm younger, but not new to programming (started in the early 2000s.) Mentality-wise, I like working on software because I like solving problems, but I only really do a day job to make a living, not to scratch an itch, so I continue to work on software outside of work hours. I basically don't care what happens once my software reaches the Internet. I publish some open source software, but not as a portfolio; just for the sake of anyone who might find it useful. I don't really advertise my projects, certainly don't go out of my way to do so.
I basically have GitHub for roughly two reasons:
- I use a wide variety of open source software. I like to fix my own problems and report issues. Some projects require GitHub for this.
- May as well take advantage of the free storage and compute, since everyone else is.
Honestly, my GitHub is probably a pretty bad portfolio. It is not a curation of my best works, and I disabled as much of the "social" features as possible. It's just code that I write, which people may take or leave.
Earlier in my career maybe I cared more, but I honestly think it's because I thought you were supposed to care.
I have worked with some stellar programmers from many different places, probably more than I really deserved to. I don't know for sure if any of them didn't have GitHub accounts, but some of them definitely did have GitHub accounts, barren as they may have been.
Someone like you is extremely experienced and skilled, and has a reputation in your industry. You started working before it was normal and trivial to build stuff in public. Such activities were even frowned upon if I recall correctly (a friend got fired merely for emailing a dll to a friend to debug a crash; another was reprimanded for working on his own projects after hours at the office, even though he never intended to ever sell them).
That you have a reputation means posting work publicly would be of little to no value to your reputation. All the people who need to know you already do or know someone reputable who does.
https://www.hanselman.com/blog/dark-matter-developers-the-un...
I joined my current employer a bit over a month ago at this point and legal still hasn't authorized me to open a GitHub account or authorized me to use my personal account to report issues.
Damn. You signed a contract that prevents you from ever publishing your own code? I guess everyone in the comments who are against have something to show in public all have security clearance or something. : ^ )
I have removed my profile some time ago. While I agree with you in general, I have to say that initially LinkedIn was actually not bad and I did get some value out of it. It is a little harder now to not have it, because initial interviews scoff at its absence ( it apparently sends a negative signal to HR people ), but an established network of colleagues can overcome that.
I guess it is a little easier for people already established in their career. I still am kinda debating some sort of replacement for it, but I am honestly starting to wonder if github is just enough to stop HR from complaining.
Any usage beyond that, for "networking" or "social sharing" is terrible.
Still have no idea how anyone can even hope to use the news feed there though. It's just a seemingly random torrent of garbage 24/7, with the odds of you getting any traction being virtually non existent.
As a candidate I think LinkedIn is absolute trash - and toxic with all the ghost jobs and dark patterns. Feels like everyone else has the same opinion, or at the very least it is a common one. But when I have 50 candidates to review for 5 open positions, LinkedIn is going to give me great insight that I cannot really get anywhere else. I keep my profile current in the hopes that I'm not doing it for LinkedIn, but for the person researching me to see if I am a valid candidate for their role.
You truly know how to align yourself with hype cycles?
my barometer for penetration is how often the non-tech people talk about it, e.g. goofball uncle didn't buy a drone, but he went hard on BTC. if he's still holding he probably made money recently, too.
I hope there will still be room for devs in the future.
If a model goes sideways how do you fix that? Could you find and fix flaws in the base model?
There's a lot of tooling out there making this accessible to someone with a solid full-stack engineering background.
Training an LLM from scratch is a different beast, but that knowledge honestly isn't too practical for everyday engineers given even if you had the knowledge you wouldn't necessarily have the resources necessary to train a competitive model. Of course you could command a high salary working for the orgs who do have these resources! One caveat is there are orgs doing serious post-training even with unsupervised techniques to take a base model and reeaaaaaally bake in domain-specific knowledge/context. Honestly I wonder if even that is unaccessible to pull off. You get a lot of wiggle-room and margin for error when post-training a well-built base model because of transfer learning.
Is it just me? Why are people using them? I feel like objectively they look like fake garbage, but obviously that must be my subjective biases, because people keep using them.
Some people can recognize these shortcomings and simply don't care. They are fundamentally nihilists for whom quantity itself is the only important quality.
Either way, these hero images are a convenient cue to stop reading: nothing of value will be found below.
If you don't like such content. But I would say don't judge a book by its cover.
In a world where anyone can ask an LLM to gish-gallop a plausible facsimile of whatever argument they want in seconds, it is simply untenable to give any piece of writing you stumble upon online the benefit of the doubt; you will drown in counterfeit prose.
The faintest hint that the author of a piece is a "GenAI" enthusiast (which in this case is already clear from the title) is immediate grounds for dismissing it; "the cover" clearly communicates the quality one might expect in the book. Using slop for hero images tells me that the author doesn't respect my time.
Premise 1 : anyone can get an LLM to churn out "content" in minutes.
Premise 2 : reading "AI" slop is a poor way to spend one's time.
Premise 3 : someone who thinks those images are ok is likely to have a poorly developed capacity to differentiate between actual expertise and confident-sounding blah-blah-ing.
Conclusion : abort at the first sign that someone thinks that that vaguely confident-seeming world of half-truths and vibing is in any way acceptable.
Your argument, on the other hand: it got traction on HN, therefore... something. Traction, or viewership, or ability-to-attract-attention, does not equate to value. And yes, I'm stating this as a general truth.
I agree, with some nuances. Premise 2 goes also without the word AI in the sentence. Premise 3, there's no way to solve the fact that confident AI-or-not bla bla convinces people.
If one finds an article has bla bla vibes, be it from weirdly looking pics or from sounding overly optimistic, it is everyone's choice to skip (which makes sense!) or continue to look for any value or even entertainment. Do you stop watching a movie always when it doesn't have the correct vibes from the beginning?
Indeed convincingness does not equal to value.
The reason I reacted to the comment was largely its patronizing vibes.
Reminds me of the image attached to Karpathy's (one of the founding members of openAI) twitter post on founding an education AI lab:
https://x.com/karpathy/status/1813263734707790301
I just don't understand how he didn't take 10 seconds to review the image before attaching it. If the image is emblematic of the power of AI, I wouldn't have a lot of faith in the aforementioned company.
If you're going to use GenAI (stable diffusion, flux) to generate an image, at least take the time to learn some basic photobashing skills, inpainting, etc.
Last time I worked on my laptop on a trestle table in the forest at dusk it looked almost exactly like this.
It is like standing in front of a Zara, and wondering why people are in that shop, and not in the Versace shop across town. Surely, if you cannot afford Versace, you rather walk naked?
https://en.wiktionary.org/wiki/rompt
Remove both the P's and you get rømt, which is Norwegian for sour cream.
Insert your own joke here about taking the Ps, it's bound to be better than what I'll come up with.
Nah, I'm struggling too, so urine good company.
There’s one silent "p" in the word "prompt"—right at the beginning! The “p” isn’t pronounced, but it sneaks into the word anyway.
It feels like it can't possible be true, but on the other hand, I'm probably due for having my confidence in my English completely shattered again by learning another weird word's real pronunciation, so maybe this is it.
Also, I too asked ChatGPT and it told me that In the word "prompt," there are no silent P's. All the letters in "prompt" are pronounced, including the P.
English is full of these consonant encounters and they are an easy way to tell if someone is not fluent.
Most of the ML candidates I see now are all "working with LLMs". Most of the ML engineers I know in the industry who are actually shipping valuable models, are not.
Cool, you made a chatbot that annoys your users.
Let me know when you've shipped a fraud model that requires four 9's, 100ms latency, with 50,000 calls an hour, 80% recall and 50% precision.
On that note, I'm surprised the precision / recall for fin models are 80% / 50%
If humans were as bad as LLMs at basic math and logic, we would consider them developmentally challenged. Yet this constant insistence that humans are categorically worse than, or at best no better than, LLMs persists. It's a weird, almost religious belief in the superiority of the machine even in spite of obvious evidence to the contrary.
Clearly, humans also need training in these things and almost no one figures out how to do basic things like long division by themselves. Some people sometimes figure things out, and more importantly, they do so by building on what came before.
The difference between humans and LLMs is that even after being trained and given access to near everything that came before, LLMs are terrible at this stuff.
Ineffective seems harsh.
They do. But not 100%, at least not all times. Like a normal person.
You might say this is about Helix being small and trying to break into a crowded market, but OpenAI and Google offered similar contests / offers that asked users to submit ideas for LLM applications. Considering how many LLM sample apps are either totally useless ("Walter the Bavarian, a chatbot who gives trivia about Oktoberfest!") or could be better solved by classical programming ("a GPT that automatically converts currencies to USD!), it seems AI developers have struggled to find a single marketable use case of LLMs outside of codegen.
So individual apps don't need to do anything to have AI.
Basically what chatgpt did for chatbots, but at app level. There are lots of apps that take a long time to master. But the average joe doesn't need to master them. If I want to lightly edit some photos, I know photoshop can do it, but I have no clue where that specific thing is in the menus, because I haven't used it in 10 years. But it would be cool to type in a chat box "take all the pictures from my sd card, adjust the colors, straighten the ones that need it, and put them in my Pictures folder under "trip to the sea". And then I can go do something else for the 30-60 minutes it would have taken me to google how to do all of that, or script something, etc.
The ideea of an assistant that can work like that isn't that far-fetched today, IMO. The apps need to expose some APIs, and the "os" needs an language -> action model capable enough to handle basic stuff for average joes. I'd bet good money sonnet3.5 + proper APIs + a bit of fine-tuning could do it today for 50%+ of average user cases.
I could go on and on.
Copy paste is great until you literally dont know where you are copy and pasting
But that's akin to web devs of old that stitched up some cruft in Perl or PHP and got their databases wiped by someone entering a SQL username. Yes, it kind of works under ideal conditions, but can you fix it when it breaks? Can you hedge against all or most relevant risks?
Probably not. Don't put it your toys into production, and don't tell other people you're a professional at it until you know how to fix and hedge and can be transparent about it with the people giving you money.