No I think that is pretty much it- it is deep learning after all, the scope of the training data limits the model.
It's so good with code because not only is there a TON of code in the data set, there is a TON of data of tutorials and examples and documentation EXPLAINING the code!
I've long been amazed at the computer programming world at the amount of content programmers write about... programming. When it comes to other engineering fields, people don't write nearly as much on the internet (and, ofter, are not allowed to write as much, publicly) about the topics. So of course these large language models are simply not going to be able to produce super detailed analysis of those fields.
There is something a bit more needed - Meta did a deep model recently with training specifically more on scientific papers, and it is able to 'talk the talk' better but unfortunately the 'understanding' just isn't there.
I think at the end of the day, programming languages are languages after all, thus work well with these tools, but there is something harder about scientific reasoning and interdisciplinary planning that is going to take something a bit more. This is the main goal of various research into bring symbolic methods back into interface with deep learning models.