Most internet-based VoIP services (read: full telephone providers, not just chat providers) use the same exact G.711 codecs used in the digital portions of traditional telephone lines for normal calls.

In theory you can run anything over this that could run over a long distance PSTN link.

In practice most modem technologies do not handle jitter and packet loss well at all, and quirks which a human would not even notice will trip up a data call. It still works, I have clients running credit card machines on VoIP lines all day, but the longer the call and the higher the bandwidth the more likely something is to go wrong. A short 9600bps data exchange from a card machine is a lot easier than a 30 page 33.6k fax which is a lot easier than a 56k modem on a PC.

There's a special method called T.38 that allows VoIP providers with compatible endpoints to convert digital fax calls back to a data stream instead of audio which makes it a lot more tolerant of delay and jitter as well as slightly more tolerant of loss. Something similar could hypothetically be done for data modems but I haven't seen it.