Even if you solve the memory issues you will still be constrained by the NIC.
Most of these SBC's run ethernet over USB, so the performance is still going to be shit.
Most of these SBC's run ethernet over USB, so the performance is still going to be shit.