The binary is that big on Linux because the debug symbols are embedded in the executable. You need to strip them.
-p:Configuration=Release -p:DebuggerSupport=false
FWIW, I did also try the following, but found they had basically no impact on output size: -p:TrimMode=link
-p:EnableUnsafeBinaryFormatterSerialization=false
-p:EnableUnsafeUTF7Encoding=false
-p:EventSourceSupport=false
-p:HttpActivityPropagationSupport=false
(Tested on .NET Core 3.1.5)38MB is a pretty typical size for a real-world, self-contained, assembly-trimmed package, and the size comes from the runtime itself, plus the core assemblies that weren't trimmed out.