Those prompts should be renamed as hints. Because that's all they are. Every LLM today ignores prompts if they conflict with its sole overarching goal: to give you an answer no matter whether it's true or not.
I like to think of them as beginnings of an arbitrary document which I hope will be autocompleted in a direction I find useful... By an algorithm with the overarching "goal" of Make Document Bigger.
(As an engineer it’s part of your job to know if the problem is being solved correctly.)
You invoke "engineering principles", but software engineers constantly trade in likelihoods, confidence intervals, and risk envelopes. Using LLMs is no different in that respect. It's not rocket science. It's manageable.
Software engineering is mostly about dealing with human limitations (both the writer of the code and its readers). SO you have principles like modularization and cohesion which is for the people working on the code, not the computer. We also have tests, which is an imperfect, but economical approach to ensure the correctness of the software. Every design decision can be justified or argued and the outcome can be predicted and weighted. You're not cajoling a model to get results. You take a decision and just do it.
At its heart that all engineering principles exist to do. Allow us to extract useful value, and hopefully predictable outcomes from systems that are either poorly understood, or too expensive to economically characterise. Engineering is more-or-less the science of “good enough”.
There’s a reason why computer science, and software engineering are two different disciplines.
> Software engineering is the application of an empirical, scientific approach to finding efficient, economic solutions to practical problems in software.
> The adoption of an engineering approach to software development is important for two main reasons. First, software development is always an exercise in discovery and learning, and second, if our aim is to be “efficient” and “economic,” then our ability to learn must be sustainable.
> This means that we must manage the complexity of the systems that we create in ways that maintain our ability to learn new things and adapt to them.
That is why I don't care about LLMs per se, but their usage is highly correlated to the wish of the user to not learn anything, just have some answer, even incorrect, as long as it passes the evaluation process (compilation, review, ci tests,..). If the usage is to learn, I don't have anything to say.
As for efficient and economical solutions that can be found with them,...
I’ve personally found them extremely useful to test and experiment new ideas. Having an LLM throw together a PoC which would have taken me an hour to create, in less than 5mins, is a huge time saver. Makes it possible to iterate through many more ideas and test my understanding of systems far more efficiently than doing the same by hand.
For the rest of us less fortunate, LLMs can be a fantastic tool to sketch out novel modules quickly, and then test assumptions and interactions between them, before committing to a specific high level design.
Not really. It's just that there's a lot of prior works out there, so I don't need to do experimentation when someone has already done it and describe the lessons learned. Then you do requirement analysis and some designs (system, api, and ux), plus with the platform constraints, there aren't a lot of flexible points left. I'm not doing research on software engineering.
For a lot of projects, the objective is to get something working out there. Then I can focus on refining if needs be. I don't need to optimize every parameter with my own experiments.
I’m currently dealing with a project that involves developing systems where the existing prior art is either completely proprietary and inaccessible, or public, but extremely nacient and thus documented learnings are less developed than our own learnings and designs.
Many projects may have the primary objective of getting something working. But we don’t all have the luxury of being able to declare something working and walk away. I specifically have requirements around long term evolution of our project (I.e. over a 5-10 year time horizon at a minimum), plus long term operational burden and cost. While also delivering value in the short term.
LLM provide are an invaluable tool for exploring the many possible solutions to what we’re building, and helping to evaluate the longer term consequences of our design decisions, before we’ve committed significant resources to developing them completely.
Of course we could do all this without LLMs, but LLMs substantially increase the distance we can explore before timelines force us to commit.
> Most of my coding is fully planned to get to the end. The experiment part is on a much smaller scale (module level).
I would seem that these statements taken together mean you don’t experiment at all?
Ah I think we’re finally getting somewhere. My point is that you can use LLM as part of that research process. Not just as a poor substitute for proper research, but as a tool for experimental research. It’s supplemental to the normal research process, and is certainly not a tool for creating final outputs.
Using LLMs like that can make a meaningful difference to speed and quality of the analysis and final design. And something you should consider, rather than dismissing out of hand.
What's the alternative?
If your C compiler invents a new function call for a non-existent function while generating code, that's usually a bug.
If an LLM does, that's... Normal. And a non-event.
What other engineering domain operates on a fundamentally predictable substrate? Even computer science at any appreciable scale or complexity becomes unpredictable.
An engineer doesn't just shrug and pick up slag because it contains the same materials as the original bauxite.
We’re basically in the stone ages of understanding how to interact with synthetic intelligence.
But attempts to integrate little understood things in daily life gave us radium toothpaste and lead poisoning. Let's not repeat stone age mistakes. Research first, integrate later.
It’s reasonable to scope one’s interest down to easily predictable, simple systems.
But most of the value in math and computer science is at the scale where there is unpredictability arising from complexity.
But a lot of the trouble in these domains that I have observed comes from unmodeled effects, that must be modeled and reasoned about. GPZ work shows the same thing shown by the researcher here, which is that it requires a lot of tinkering and a lot of context in order to produce semi-usable results. SNR appears quite low for now. In security specifically, there is much value in sanitizing input data and ensuring correct parsing. Do you think LLMs are in a position to do so?
In the hands of an expert, I believe they can help. In the hands of someone clueless, they will just confuse everyone, much like any other tool the clueless person uses.
Are you Insinuating that dealing with unstable and unpredictable systems isn't somewhere engineering principles are frequently applied to solve complex problems?