Getting rid of the heat was the biggest problem. Rockets which launch vertically don't spend much time in the atmosphere. Aircraft which accelerate horizontally do. Hypersonic in-atmosphere travel means having to get rid of huge amounts of heat with no place to dump it. Running liquid hydrogen fuel over the vehicle exterior for cooling was considered and rejected for the SR-71. (Many rocket engines run the cryogenic fuel and oxidizer through cooling channels in the engine bell before using it as propellant. That's not a totally unreasonable idea.)
The X-30 was supposed to have a big cabin, which would have to be kept cool. Only a small part of the SR-71 is kept cool; the pilot is in a space suit. Even cooling the pilot was hard. Remember, at hypersonic speeds in atmosphere, there's no place to dump the heat; you have to bring your cold with you.
The only real advantage of this approach over rockets which get out of the atmosphere as soon as possible is that you don't have to carry oxidizer. That improves the mass fraction (rockets to orbit are > 90% fuel). Not enough to be a win, though.