Let me guess: i7 with hyperthreading? -> 55%/12 = ~4%
Got it. Why my MBP feels like burning when running zoom then?
If only one core is used, percentage of a core used is a misleading measure, because the CPU will boost the speed of that core, and have it use a way larger part of the power budget for the CPU (because the other cores do not need the power, and modern CPUs, especially in laptops, are almost always limited to some overall power usage limit by how much their cooling system can do)
That's not quite how SMT works. Half of the core is not sitting idle when running a 1 thread workload on a 2 thread core. Rather the core will be working at mostly full tilt.
Also if only one core is active, it can be put into a higher than normal power regime.
As far as I know, 100 % in the Activity Manager is one hyperthread fully busy, regardless of clock. It doesn't seem too unlikely that Zoom takes that from the equivalent API, divides it by the number of seen virtual cores to convert it to some measure of "overall CPU load", and reports it as usage. It's not a good measure, but an explanation. (And as you point out boosting happens, so the "55%" reported by Activity Manager is already not all that good)
Zoom doesn't have to do much computing while in a meeting – except for video encoding, which is usually handled by the GPU. That's why "GPU Time" is high (as shown in Activity Monitor) and why your MBP gets hot.