I've not done much with ChatGPT but so far my personal impression is that, to make a school analogy, it is like it is a kid a year or two ahead of me who took the class I'm asking for help on but didn't actually do too well.
Their help often won't be quite right, but they will probably mention some things I didn't know that if I look up those things will be helpful. Sometimes they are right on the first try. Sometimes they are just bullshitting me. And sometimes they brush me off.
Examples:
1. I asked it how to translate a couple of iptables commands to nft. It got it right. I then asked what nft commands I would use on Linux to NAT all IP packets going through the tun0 interface. It got that right too, giving me the complete set of commands to go from nothing to doing what I had wanted.
So here it was the kid who was ahead of me, but he did well in his networking class.
2. At work we are looking into using Elavon's "Converge" payment processing platform for ACH. I asked ChatGPT how you do an ACH charge on Elavon Converge.
It gave what I believe is a 100% right answer--but it was for how to do it using their web-based UI for humans. That's my fault. I should have specified I'm interested in doing it from a program, so my next question was "How would I do that from a program?".
With this it was the kid who did OK in class, maybe a B-. It gave me an overview of using their API, except it said it involves a JSON request object send as theyt payload on a POST when in fact it uses XML.
I asked what would be the payload of that POST. It gave me an example (in JSON). All the fields names were right (e.g., they corresponded to the XML element names in the actual API) and it included a nice (both in content and formatting) description of each of those fields.
This definitely would have been useful when I was first trying to figure out Elavon's API.
3. I then decided to see how it did with something nontechnical. I asked it what is the OTP for Luca fanfiction. It dodged the question saying that as an AI language model it doesn't have OTP information for any fandom, and said it is a matter of personal preference. I also asked what is the most common OTP for Luca fanfiction since that is an actual objective question, but it still dodged.
I then tried "Kirk or Picard?". It gave me the same personal preference spiel, but then offered some general characteristics of each to consider when making my choice.
4. I was automating some network stuff on my Mac. I needed to find the IPv6 name server(s) that was currently being used. It suggested "networksetup -getdnsservers -6 Wi-Fi". I was actually interested in Ethernet but hadn't specified that.
Two problems with its answer. First, there is no -6 flag to networksetup. Second, -getdnsservers only gets dnsservers that were explicitly configured. It does not get DNS servers that were configured by DHCP.
I think the right answer is to get the IPv6 DNS servers that were obtained from DHCP is e.g. "ipconfig getv6packet en0".
I also asked it how to find out which interface would be used to reach a given IP address. It suggested using traceroute to the IP address, getting the IP from the first hop of the router that my computer uses to reach that network, and then looking for that IP address in inconfig output to find out which interface it is on.
That doesn't work (the IP of the router does not appear in ifconfig output), and unlike many of the earlier things it got wrong this doesn't really even send you in the right direction.
The right answer is "route get $HOST".
The final networking question I had for it was this:
> On MacOS the networksetup command uses names like "Ethernet" for networks. The ifconfig command uses names like "en0". How do I figure out the networksetup name if I have the ifconfig name?
It said "networksetup -listallhardwareport" and told me what to look for in the output. This is exactly right.
It probably would have taken me quite a while to find that on my own, so definitely a win. The earlier wrong answers didn't really waste much time so overall I came out ahead using it.