It's important to note that LPG (liquefied petroleum gas, mostly propane) is very much
not the same thing as CNG (compressed natural gas) from an engineering perspective.
CNG tanks have to keep much higher pressures and are therefore quite a bit more expensive. As a previous commenter noted, $2k-$3k just for the tank is about right, at least for a new, light unit. If you're willing to go the solid steel route, you can probably still get something old and heavy [1] for under $1k, but good luck finding capacity at under $100/GGE (gasoline gallon equivalent).
Lastly, there's the safety question. Currently, CNG cars have remarkably good real-world crash experience. I don't believe there's a single tank burst or fire in a collision [2]. I'd be curious to know if other countries, with more lax regulations, have as good safety statistics.
TANSTAAFL
LNG (liquefied natural gas) is also quite different from LPG. I've looked into it, if only as an emergency reserve fuel tank. Turns out that to store it indefinitely requires refrigeration (or, presumably, absurd amounts of insulation), else it'll boil off, escaping out the pressure relief valve installed in the tank for just such an eventuality. Fortunately, like a pot of water on a stove, it doesn't all boil off at once, so there's little risk of leakage if one always uses enough vapor to keep the pressure down.
[1] To the tune of hundreds of pounds. This is less likely to be a concern for converting trucks, but I notice a fuel economy difference carrying just one adult passenger in my Civic GX.
[2] I haven't looked this up in the last 3 years or so. Corrections/updates are appreciated.
Edit: added paragraph on LNG.