There are advantages and disadvantages to internal combustion engines.
Compared to external combustion engines (steam engines), ICEs have much quicker start-up and response times. Steam engines require building up a head of steam, and actually exhaust the working fluid (water) over time, which is why steam railroads had water towers every so often.
ICEs have been improved over more than 130 years, and scale from a few CCs of displacement to many cubic meters -- the smallest ICEs could fit in the palm of your hand, you could stand in the largest and not reach the sides or top of the cylinder.
Liquid hydrocarbon fuels are exceptionally energy dense by both weight and volume. This makes certain classes of use very difficult to substitute for: commercial passenger and freight aircraft, overland truck transport, and powered marine shipping really have few tractable alternatives. Aircraft would all but disappear without liquid hydrocarbons (gasoline for piston engines, kerosene for gas turbines), trucks would probably be replaced with electrified rail, and shipping would return to wind power. Optimists might suggest nuclear marine powerplants, but I find that unlikely; ships are lost at far higher rates than is commonly realised: about 200 every decade, and existing trials of both military and nonmilitary nuclear marine power have proven it expensive and nonviable for all but the most demanding instances -- aircraft carriers and submarines.
They're also quite stable in storage (proved over hundreds of millions of years), relatively safe to handle (no respiration or exceptional contact protection required), and their combustion products are mostly benign: CO2 (I'll get to this) and H20 with scant quantities of carbon monoxide, sulfer and nitrogen oxides, and other contaminants or partial combustion products, all of which can be greatly mitigated with combustion and exhaust controls and treatment.
The CO2 exhausted isn't a problem in itself but for the, um, slight problem that in the past 200 or so years humans have returned to the biosphere carbon sequestered over several hundreds of millions of years. Which turns out to be a rather considerable problem.
But it's only a problem where what you're burning are fossil fuels. There exist several options, though all much more expensive and/or constrained than present fossil fuels, for creating synthetic hydrocarbon-based liquid fuels. These also have challenges. Biofuels are intractable at scale given natural limits on plant production: what's called "HANNP" -- the human appropriation of net primary production -- or the photosynthetic ceiling. There's only so much plant growth which occurs and humans already consume much of it (40%, biofuel replacement of fossil fuels would consume another 20%, see Jeffrey S. Duke's "Burning Buried Sunshine" (2003), PDF available online, for more on this.
Another prospect: sequestering carbon from the biosphere, and combining it with hydrogen, electrolised from water. The US Navy and national energy labs (especially Brookhaven), as well as M.I.T., have researched this for the past 50 years. It's proven expensive and hasn't been scaled past very low production (a few litres), but does work. It raises costs of fuel from one unit of input energy per 20-40 units made available to two units per one provided -- that's a 40-80x increase in the real cost of fuel.
Not cheap, but it might still be our best option.