hubraumhugo

It will be interesting to see how all the companies in the document processing space adapt as OCR becomes a commodity.

The best products will be defined by everything "non-AI", like UX, performance and reliability at scale, and human-in-the loop feedback for domain experts.

You can get your HN profile analyzed by it and it's pretty funny :)

https://hn-wrapped.kadoa.com/

I'm using this to test the humor of new models.

As someone building in this space, we've found that raw OCR accuracy is just one piece (and it's becomming a commodity).

The real challenge is building reliable and accurate ETL pipelines (document ingestion from web, OCR, classification, validation, etc.) that work at scale in production.

The best products will be defined by everything "non-AI", like UX, performance, and human-in-the loop feedback loop for non-techies.

Avoiding over-reliance on specific models also helps. With good internal eval data and benchmarks, you can easily switch or fine-tune models.

I've seen many "HN for X" projects for various niches now, and they all suffer from the chicken/egg problem of getting a critical mass of participants.

Isn't game development already discussed here?

I actually built a side project that categorizes front page articles so I can filter for topics. Here's an example for recent gamedev content: https://www.kadoa.com/hacksnack/d57360e8-1eb1-4800-a711-f0d5...

Should work again.

It takes the last 75 posts and comments for performance reasons. Going to improve this.

fixing, look like we're hitting token limits :(

To everybody reading this and having doubts about starting your own venture: DO IT!

Yes it's a lot of risk, yes most startups fail, and yes you could make more money at FANG. However, this is one of the most exciting times to build a startup, thanks to AI.

The big winners of the all the AI advancements (especially OS) are devs and AI startups:

- No more vendor lock-in

- Instead of just wrapping proprietary API endpoints, developers can now integrate AI deeply into their products in a very cost-effective and performant way

- Price race to the bottom with near-instant LLM responses at very low prices are on the horizon

It's as if your product automatically becomes better, cheaper, and more scalable with every major AI advancement. This leads to a powerful flywheel effect: AI improves product -> better product attracts more users -> users generate more data

Besides the product perspective, AI also allows you to operate your startup a lot more efficiently.

Will we see a single-person unicorn soon? Very likely not.

Will we see very successful small and bootstrapped startups? Yes, already the case.

The arms race between AI bots and bot-protection is only going to get worse, leading to increasing infra costs while negatively impacting the UX and performance (captchas, rate limiting, etc.).

What's a reasonable way forward to deal with more bots than humans on the internet?

The /archive page is a great overview of all his posts: http://www.aaronsw.com/weblog/archive

I'm having a bit of a deja-vu with this since I tried to build the same a few years ago, see my Show HN from back then: https://news.ycombinator.com/item?id=24707407

Some challenges I faced:

- Warranties have many nuances depending on the country and the fine print

- Monetizing this is hard. I see your using affiliate links, but this will hurt the trust of your users.

- Only very few users are willing to write valuable in-depth reviews. I tried to collect data on how products perform over their lifetime (https://www.looria.com/reviews), but never reached a critical mass.

- Astroturfing is a major issue. I aggregated many reviews from Reddit, but even there it's getting worse.

Happy to chat or help out with data if you're interested :)

I love the idea of having a plug-in spreadsheet on every website, so I can work with the data without having to do any scraping or programming.

Unfortunately, it looks like the project was abandoned after the initial talk and paper in 2020.

I assume that we'll get there soon with small LLMs running directly in the browser (like Gemini nano) that will make it easy to query/structure website data.

It's pulling the 30 most recent comments + submissions. Any suggestions for a more representative way?

Kadoa | Senior Software Engineer | Remote (Europe) | Full-Time | https://kadoa.com

We are building the foundational infrastructure for unstructured data ETL.

Our platform enables any organization to instantly extract, transform, and integrate unstructured data (websites, documents, emails, etc.) - without code or maintenance overhead.

You’ll work at the intersection of AI, data engineering, and software development, shipping code with real-world impact at industry-leading customers.

There are two official roles, but we welcome conversations with anyone interested in joining the team.

- Senior Software Engineer: https://www.kadoa.com/careers/senior-software-engineer

- DevOps Engineer: https://www.kadoa.com/careers/devops-engineer

Please email us at (adrian at kadoa dot com) and mention HN in the subject line.

Any good alternatives for "adopt: Blank Spaces app" on Android?

Ciechanowski is likely the best content producer of our time, absolutely fascinating reads. Imagine having such a person as a teacher - he could probably excite students about any scientific topic.

I'd love to spend my time working on such articles when I'm retired :)

Meta continues to overdeliver. Their goal from the start was to target and disrupt OpenAI/Anthropic with a scorched earth approach by releasing powerful open models.

The big winners: we developers.

I can't think of any other product that makes it to the HN frontpage so consistently. I guess Kagi really hits the bullseye in terms of HN as a target audience.

Given the recent advances in vector-based semantic search, what's the SOTA search stack that people are using for hybrid keyword + semantic search these days?

Congrats on the launch! A quick search in the YC startup directory brought up 5-10 companies doing pretty much the same thing:

- https://www.ycombinator.com/companies/tableflow

- https://www.ycombinator.com/companies/reducto

- https://www.ycombinator.com/companies/mindee

- https://www.ycombinator.com/companies/omniai

- https://www.ycombinator.com/companies/trellis

At the same time, accurate document extraction is becoming a commodity with powerful VLMs. Are you planning to focus on a specific industry, or how do you plan to differentiate?

I'm curious - what does the AI coding setup of the HN community look like, and how has your experience been so far?

I want to get some broader feedback before completely switching my workflow to Aide or Cursor.

I've seen quite a few YC startups working on AI-powered RPA, and now it looks like a foundational model player is directly competing in their space. It will be interesting to see whether Anthropic will double down on this or leave it to third-party developers to build commercial applications around it.

Has anyone seen AI agents working in production at scale? It doesn't matter if you're using Swarm, langchain, or any other orchestration framework if the underlying issue is that AI agents too slow, too expensive, and too unreliable. I wrote about AI agent hype vs. reality[0] a while ago, and I don't think it has changed yet.

[0] https://www.kadoa.com/blog/ai-agents-hype-vs-reality

We've been working on AI-automated web scraping at Kadoa[0] and our early experiments were similar to the those in the article. We started when only the expensive and slow GPT-3 was available, which pushed us to develop a cost-effective solution at scale.

Here is what we ended up with:

- Extraction: We use codegen to generate CSS selectors or XPath extraction code. This is more efficient than using LLMs for every data extraction. Using an LLM for every data extraction, would be expensive and slow, but using LLMs to generate the scraper code and subsequently adapt it to website modifications is highly efficient.

- Cleansing & transformation: We use small fine-tuned LLMs to clean and map data into the desired format.

- Validation: Unstructured data is a pain to validate. Among traditional data validation methods like reverse search, we use LLM-as-a-judge to evaluate the data quality.

We quickly realized that doing this for a few data sources with low complexity is one thing, doing it for thousands of websites in a reliable, scalable, and cost-efficient way is a whole different beast.

Combining traditional ETL engineering methods with small, well-evaluated LLM steps was the way to go for us

[0] https://kadoa.com

As an engineer who became a founder, I cannot recommend the book enough. Whether you're improving your landing page, writing your pitch deck, reaching out to customers, or developing your company and product strategy, communicating effectively in writing is a very crucial skill.

Invest in improving your writing skills. It will pay dividends in every aspect of your business.

Curious to hear some real-world use cases for fine-tuning and what your setup looks like in terms of:

- frameworks

- hardware

- foundation models (OS vs 3rd party)

- evaluation & training data

Did the performance gains justify the investment for fine-tuning?

Very cool to see more YC hard tech startups emerging [0]. These are the kind of moonshot projects I love to see getting funded (instead of just SaaS and AI).

[0] https://www.ycombinator.com/companies/industry/hard-tech

You mention validation and schema guarantees as key features for high accuracy. Are you using an LLM-as-a-judge combined with traditional checks for this?

Fascinating how traditionally very complex and hard ML problems are slowly becomming commodities with AI:

- transcription

- machine translation

- OCR

- image recognition