Yet whole applications are written around it and they work perfectly well. Not saying that is an argument that it is somehow a good idea, just that while it might looks bad, it also works ok when done properly.
That's circular reasoning. The objection is that it only works well in certain circumstances. Mostly in fairly simple applications with either small teams or rigid conventions to enforce clarity and prevent bugs.
I don't think it is a circular reasoning. There's always the good and the bad way to use something, even C++ exceptions.
I'm not sure your example is a counterexample either. Implementing part of a fairly simple runtime (like Lua's) using longjmp is an example of a good use of longjmp, in my opinion.
If abstracted away they don't necessarily cause that many problems. Sure, it'd be nicer if there were a better alternative, but often there's just none. And no C++ isn't always an alternative. Many older platforms have only horribly bad C++ compilers, certainly not ones where you want to rely on exception handling.