Hacker News Reader

Comment by greg | Hacker News Reader

Engineer on the team here. Some of the interesting challenges of doing ChatGPT for circuit design were:

1) How to communicate the circuit connections to ChatGPT in the text prompt. We ended up just using the standard netlist format. Apparently it knows about netlists from its training data. It also knows about datasheets.

2) How to get the needed accuracy and precision for questions like "calculate the resistance needed between component A and B". For that, we adopted the popular langchain framework to integrate specific "tools" into the AI for specific recognizable problems.

View on HN

Out of curiosity, I noticed in the video that it recommended for a voltage regulator either an LM7805 or an LM1117 - either a super inefficient 5V regulator that has no business in new designs, or a 1.2V low-dropout 800mA regulator that is probably not appropriate to the other components it recommended (i.e. an LM7805 -> LM1117 would be 36% efficient at best).

It then also recommends a LiPo battery with a nominal 3.7V which obviously wouldn't be compatible with its LM7805 recommendation.

It also recommends an ESP8266 which is a 3.3V part so neither of the recommended regulators, nor the battery alone, would be appropriate.

Also, it recommends a 168MHz STM ARM micro - but it already has an ESP8266 with GPIO.

I'm not sure what it's suggesting I build but it doesn't really sound like a working environmental radiation logger ;)

How should I think about this?

I find comments similar to yours about ChatGPT all over the internet. I finally took the time to ask that some questions over the last few days. All the answers it gave were mundane and expected from any click bait web site you can find. I Googled the same questions and found similar answers.

Some personal questions it refused to answer or give suggestions because "only you can decide" was the response.

I'm very suspicious of all this. I'm thinking the only real benefit--some may say--is the chat aspect. You can carry on a conversation rather than having to determine a proper Google search over and over again.

Which makes me think there is some hype involved.

You’re probably not using GPT-4, the internet is being flooded with takes about GPT-3.5’s quality like this simply because the author doesn’t know there is a meaningful difference with the new model.

When I go here: https://openai.com/product/gpt-4 it says "try on chatgpt plus" and sends me to the same page I log in with and that's where I've been testing it.

Be sure that you select GPT-4 from the drop down list of models. For each new chat, it reverts back to default GPT-3.5.

I don't see any dropdowns for such a thing.

GPT-4 still confidently makes up sources for wrong answers and throws subtle mistakes (the obvious mistakes aren't as big a nuisance) into output.

This isn't to say gpt-4 isn't cool or impressive or a development to watch and learn about and be excited about, but I frequently see criticism dismissed as "you must be using 3.5" while I find 4 still costs more time than it would have potentially saved.

Of course this is possible but usually criticizing a dismissal like this as being wrong comes after it has been proven wrong.

If GPT-X would just shut up if it doesn't know something it would already be 10x more useful than it is right now.

Isn't that the core issue? Its model doesn't really "know" what is real or made up.

> Which makes me think there is some hype involved.

There's _a lot_ of hype.

It's a good tool if you forget about any "intelligence" and just think of it as a tool. We don't expect search engines to be 'intelligent', we expect them to return results we are interested in, in an efficient manner.

In the case of this and similar tools, it is a text generator. It will generate _something_ based on your input. In many cases, it will make up stuff because it has to generate something. Note how it will not ask follow-up questions to 'understand' you (because it doesn't understand anything). You have to apply judgment and ask the follow up questions yourself.

One thing I found these tools to be useful is to mitigate the "blinking cursor on an empty document" paralysis. I just asked the "Pulumi AI" to generate some code that, while not really correct to my needs, is a good starting point for modification.

If you ask it to generate something from you, other than just a pre-made response that you could google, then it's more useful.

GPT 4 is a massive improvement in this regard.

Copilot specifically asked me to provide the capacitance value for the capacitor closest to the comment pin in response to my question about optimal resistance for a resistor. Once I provided the value (by setting it in the tool, not by typing it in the chat) and asked it to try again it thanked me for the new info and gave back a good resistance value and how it arrived at it.

In this case it seems to be giving examples of common ICs that are voltage regulators - these answers to high level questions are pretty vague by default.

We've had good results by asking follow up questions like "what power supply IC could I use to power a 3.3v microcontroller from a 3.7V nominal LiPo battery over the whole battery voltage range?".

I asked that question and I got: LM3940 and AMS1117-3.3 (LDO, high drop out but probably ok for low current - one isn't in stock but it doesn't have stock info yet), and TPS62203 and ADP2108 (bucks, both seem appropriate). These answers should get better as Copilot gets access to more and more tools.

btw I hope my comment didn't come across as too negative, like many folks I'm still trying to frame my own thinking around this kind of tooling. Thanks for the follow-up.

Not at all. I thought it was salient and very much what I wanted to know about. It seems there will be a heck of a lot of comments saying 'this is so awesome!' and this isn't a field where I'm able to look at the answer and immediately see the list of howling errors and hallucinations, so I'm very glad somebody can.

It's all very well expecting the hallucinations, but specificity beats Ludditeism every day. My gut reaction was 'ho boy, I'd better not trust this' but I didn't have the domain knowledge to understand why the GPT advice was dangerous. You did.

We are all good

I look forward to following your work. I don't know what the ideal integration of generative AI and hardware design is, but I do feel like there are opportunities. Excited to see what y'all cook up.

Thanks!

I believe we are extremely early…and so this is just a first baby step

Much more to come from us in the coming weeks

My god.. if you gave it real time stock updates to adjust BoM answers to availability that would be amazing!

Brace for impact!

We have a bunch of features that didn’t make the cut for todays launch but should be available shortly

Why is the battery alone not appropriate for the ESP8266? It runs just fine off a lithium battery.

Does your module maybe have a built-in VRM? The datasheet said the operating voltage range was 2.5-3.6V. A LiPo battery has a nominal voltage of 3.7V and a fully-charged voltage closer to 4.2V. That's definitely out of spec without a VRM.

It doesn't, but it's very tolerant of 4.2. I think the most I've run it with was 5V.

I absolutely believe it works for you, but I wouldn't ship that to customers, and I wouldn't expect what is ostensibly an authoritative reference tool to recommend it at all.

I’m with you on not shipping out-of-spec designs. Maybe this is a good place for a reverse-polarity protection diode!

I mean the more fundamental problem with the power supply is it recommending a "USB-to-Serial converter for powering the device." (See second example image.)

I'm overall bullish on AI systems but I'm not sure this space has enough detailed context to train. Much if this knowledgeis in-house/proprietary. We'll see how fast these are able to improve over time.

How do you deal with designs whose netlist or part list exceeds the context size limit?

It’s indeed a challenge

For many applications such as talking to pdfs you can use a vectors db like pinecone

But that doesn’t work well for schematics because for of the interesting use cases you need access to the whole thing.

Here is what’s been working for us

1. Extreme compression and summarization of the data we dynamically put in the context 2. Leverage the extreme amount of world knowledge the cutting edge models have

Do you do any fine tuning of the model for circuit analysis and other hardware design literature?

I'm curious if you fine-tuned a GPT model with specific hardware-related examples. Based on the OpenAI docs it seems like a pain to get enough good prompt:response pairs for fine-tuning (A few hundred minimum).

Reply on news.ycombinator.com