Similar to what others have said, we have had big changes in our systems, thanks to LLMs. Some examples of what we have done:

  - Rewrote a data extraction and PDF bounding box algorithm - LLM provided the tooling to visualize the output of algorithm and find the right rules for our needs.
  - Migrated an old SciBERT model that was bundled into a 6GB docker container that needed GPU, with to an ONNX based inference container about 900MB running on CPU.
  - Setup a K3S cluster to replace our Nomad cluster.
  - Optimized an algorithm that used SQLite with better indexes and optimized querying with some 60-80% performance gains.

We have reduce resource usage in specific areas of the application drastically. These were all possible before, but LLMs provided the tooling to iterate and deliver it in time and cost that seemed prohibitive before.

Now are the end users going to see the benefit and is the product magically better? Well no. The chicken answer is I am not involved in that side to know. But a more realistic answer is, user experience and product fit is not something LLMs can solve. That's still upto the humans to figure out and I think that's where this "nothing has improved" feeling comes from.