Going one deeper, how does the communication work on a physical level? I’m guessing the wires of the PCI Express bus passively propagate the voltage and the CPU and GPU do “something” with that voltage?
The system agent then receives the memory operation and looks at the system address map. If the target address is PCI-E memory, it generates a PCI-E transaction using its built-in PCI-E controller. The PCI-E bus is actually a multi-lane serial bus. Each lane is a pair of wires using differential signaling (https://en.wikipedia.org/wiki/Differential_signalling). Bits are sent on each lane according to a clock by manipulating the voltages on the differential pairs. The voltage swings don't correspond directly to 0s and 1s. Because of the data rates involved and the potential for interference, cross-talk, etc., an extremely complex mechanism is used to turn bits into voltage swings on the differential pairs: https://pcisig.com/sites/default/files/files/PCI_Express_Ele...
From the perspective of software, however, it's just bits sent over a wire. The bits encode a PCI-E message packet: https://www.semisaga.com/2019/07/pcie-tlp-header-packet-form.... The packet has headers, address information, and data information. But basically the packet can encode transactions such as a memory write or read or register write or read.
At the physical level we're literally pushing the limits of semiconductor physics as an industry. :-) There's all sorts of tricks, like 8b10b encodings[1] to address error handling and recovery, and then different ways to itself transmit the PCI signal. You can, for example, encode a PCIe signal on an optical cable. :-) The bus itself is divided into "lanes" which can be aggregated into various "widths" - so you'll see x1, x4, x8, x16 etc. This compounds the problem, in that each lane is negotiated. See this video[2] for what appears to be a nice overview. I haven't watched the video in depth, but clicking around, this seems reasonable to get a sense of what's what.
There's a good reference on the rest of the protocol here: https://www.mindshare.com/files/ebooks/pci%20express%20syste... . Put bluntly - it's amazing that anything works at all, technology is as cheap as it is, ubiquitous as it is, and continues to get cheaper, and more powerful on a daily basis.
If you want your mind blown - look up how modern WiFi works, and what we take for granted today. Beam forming. [3] 256-QAM encoding.[4]
Source: I wrote firmware for very, very large servers at one point in my career. Code that I wrote still (AFAIK) is still running, and writing a a 32-bit test pattern into a register once every second to validate that a particular chip is working ok. That test pattern? 0x4D494B45, which is "MIKE" in ASCII. :-)
Math (and science) is amazing. :-)
[1] https://en.wikipedia.org/wiki/8b/10b_encoding
[2] https://www.youtube.com/watch?v=EHkuzkNWXFk
We’ve come pretty far from the telegraph, haven’t we?
Or maybe not, one does not need all the details, so often just scaled concepts :)
https://en.m.wikipedia.org/wiki/Universal_asynchronous_recei...
Edit: Wait it is really already QAM over PCIE? Yeah then UART is a gross simplification, but maybe still a good one to start with depending on knowledge level?
This is an old spec. I think it’s like equivalent to QAM-512 for PCIe 6
Intersymbol interference likewise applies to pretty much any high-speed digital transmissions. At high frequencies, you have to worry about things like the signal reflecting off the other end of the circuit trace, which creates inter-symbol interference.
QAM is a modulation technique. It specifies how symbols of one or more bits are represented as analog waves on the wire. PCI-E does not use QAM. It simply represents 0s and 1s high and low voltage swings--what's called Pulse Amplitude Modulation. Other modulation techniques encode symbols by varying the amplitude and phase (or both) of a carrier wave. QAM works by modulating the amplitude of two carrier waves 90 degrees out of phase (e.g. sine and cosine) based on a data signal and summing them together: https://www.techtarget.com/searchnetworking/definition/QAM
The different versions of PCIe use a different encoding, so it's hard to sum it all up in a couple sentences in terms of what the voltage does.
The A pins would be a binary representation of an address, and the D pins would be the binary representation of data.
A couple of other pins would select behavior (read or write) and allow handshaking.
Those pins were connected to everything else that needed to talk with the CPU on a physical level, such as RAM, I/O devices, and connectors for expansion. Think 10-base-T networking where multiple nodes are physically modulating one common wire on an electrical level. Same concept, but you have many more wires (and they're way shorter).
Arbitration logic was needed so things didn't step on each other. Sometimes things did anyway and you couldn't talk to certain devices in certain ways or your system would lock up or misbehave.
Were there "switches" to isolate and select among various banks of components? Sure, they are known as "gate arrays" - those could be ASICs or implemented with simple 74xxx ICs.
Things like NuBus and PCI came about - the bus controller is directly connected and addressable to the CPU as a device, but everything else is connected to the bus controller, so now the new-style bus isn't tied to the CPU and can operate at a different speed and CPU and bus speed are now decoupled. (This was done on video controllers in the old 8-bit days as well - to get to video RAM you had to talk to the video chip, and couldn't talk to video RAM directly on some 8-bit systems).
PCIE is no longer a bus, it's more like switched Ethernet - there's packets and switching and data goes over what's basically one wire - this ends up being faster and more reliable if you use advanced modulation schemes than keeping multiple wires in sync at high speeds. The controllers facing the CPU still implement the same interface, though.
And on some buses, some or all A/D pins where multiplexed, so transaction divided to address set part and data read/write part.
Similar to old school modems if the line is noisy it can drop to lower “baud” rates. You can manually try to recover higher rates if the noise is gone but it’s simpler to just reboot.