I like this reasoning. It is shortsighted to say that LLMs aren’t well-suited to something (because we cannot tell the future) but it is not shortsighted to say that LLMs are well-suited to something (because we cannot tell the future)
Ever try to get ChatGPT to play scrabble? Ever try to describe the board to it and then all the letters available to you? Even its fancy pants o1 preview performs absolutely horrible. Either my prompting completely sucks or an LLM is just the wrong tool for the job.
It’s great for asking you to score something you just created provided you tell it what bonuses apply to which words and letters. But it has absolutely no concept of the board at all. You cannot use to optimize your next move based on the board and the letters.
… I mean you might if you were extremely verbose about every letter on the board and every available place to put your tiles, perhaps avoiding coordinates and instead describing each word, its neighbors and relationships to bonus squares. But that just highlights how bad a tool an LLM is for scrabble.
Anyway, I’m sure schematics are very similar. Maybe somebody we will invent good machine learning models for such things but an LLM isn’t it.
To a nail, every hammer has a purpose.
The main problem is an optimal decomposition of the big project into a collection of interconnected modules and in defining adequate interfaces between modules.
This is not difficult when the purpose of the project is to just take an older project and make some improvements to it, when a suitable structure is already known, but it is always the main difficulty when a really new problem must be solved.
I have yet to see any example when a LLM can be used to help even in the slightest way to solve such an example of "divide et impera" for something novel, where novel by definition means that the training set has not contained the solution for an identical project.
There is pretty much no relationship between the 2-dimensional or multi-dimensional structural graph of the interconnected modules, together with the descriptions of their matching interfaces, and the proximity or frequency of tokens in the description of the circuit by a hardware design language. So there is little that a LLM could use to generate any HDL program for an unknown circuit.
What a LLM could do is only after a good designer has done the difficult job to decompose the project into modules and define the interfaces. When given a small module with its defined interfaces, a LLM might be able to find some boilerplate code to speed up the implementation of the module.
However, any good designer would already have templates for the boilerplate code and I can not really imagine how a LLM could do this faster than a designer who just selects the appropriate templates and pastes them into the module.