Comment by hliyan | Hacker News Reader

Comment by hliyan | Hacker News Reader

Parent Full thread

I can provide a data point for what the article calls pseudo productivity: I extensively use LLMs as semantic search engines or expert systems (but not as agents). Recently I asked one how to consume a Google Pub/Sub topic using Python (context: I come from an C++/Java/JS background with some Python knowledge). The LLM gave me a good intro and some code. As it usually happens, I had a few follow up questions/clarifications, and then had to clarify the intent behind the code I requested. After a few relatively effortless rounds of back and forth, I got what I needed. It felt productive. But looking at the clock, about 20 minutes had passed without me even realizing it. Then I went and looked at the official overview page for the Google Pub/Sub Python client. It had everything I needed (including the code), in a more condensed, well-structured form. I could probably have arrived at the same outcome in 5-10 minutes. The only difference was that the latter method required some focus/discipline.

I'm wondering whether this is what they call pseudo-productivity: a lot of low-friction back and forth that feels productive, and perhaps even enjoyable, but in objective terms, takes longer?

This is a very common effect and I don't want to be defending LLMs here. But I've seen the same study with CLIs - people using them feel more productive but take longer than using GUIs.

What I want to say is that it's very situational and it's likely good to focus on the average. Using LLMs as docs are bad when good docs exist, but if you aren't sure if they do, it's a gamble. A much better approach would be to have somebody pre-create and edit the docs with an LLM for each service with bad docs.

Only when your situation isn't covered would it make sense to create new docs.

> people using them feel more productive but take longer than using GUIs

I hope that this isn't the case for me poking around in vim, using ctags etc. But sadly it may be true.

You are good with that 20 mins :) I wrote a maxscript for 3Dmax to speed up some tasks, and i got stuck in coding. Then I asked the AI for debug, then to write a snippet, etc...I lost 3-4 days. At the end I found a developer on Fiverr who fixed my script for $20 :)

That's something that is really really common in other contexts as well. For example lower level languages and especially more verbose languages make one feel a lot more productive. Another example is over-engineered infrastructure and especially cloud infrastructure that somehow make me feel very productive, because you have to think about certain details and things can feel puzzly where just running a service with a service file or init script on a random server might get you just as far and provide a lot less surface for things to go wrong.

I think another set of related effects might when people switch programming languages. There two major things tend to happen. Rewrites of something they now understand way better and having a clean slate. As well as - in case of new programming languages - way less historical bagged, 15 ways of doing the same way, deprecated tools, lots of the "new way" code in dependencies and "old ways".

What I mean with that is that there are a lot of overlooked things going on. And humans in general are really good at mistaking moving a problem somewhere else as not having to deal with that problem. Sometimes that is the case, sometimes even moving things to another apartment or being able to move work to a free coworker is a worthwhile investment even if it adds overhead. But it's also really easy to forget about how you didn't make issues disappear but just moved the issue somewhere else.

I think psychology plays a much bigger role in many of these things than technology does.

These are just examples. I don't argue against any of these things, also because whether they are worthwhile depends a lot on context. However, I do think that LLMs aren't the first example of that happening.

Reply on news.ycombinator.com