The fact they are typically referred to as exceptions has nothing to do with the fact that they are exceptional or not. Exceptions, in an abstract sense, are a language feature that allows exiting from functions via an alternative mechanism to traditional 'return' statements.

The main advantage (just picking reasons off the top of my head) is that you can write your code as if it will certainly succeed, then have a separate block of code for handling failure cases. This makes code a lot more legible and draws explicit attention to failure case handling. Some languages make a distinction between 'checked' and 'unchecked' exceptions (e.g. Java) whereby checked exceptions must be handled at the call-site, whereas unchecked exceptions can be caught anywhere in the call stack (or not at all, in which case the thread or program will terminate). It is often conventional to forego checked exceptions and only use unchecked exceptions to provide flexibility to the caller in where they handle the exception. Some languages such as Kotlin remove the concept of checked exceptions from the language entirely.

Languages like C which don't have exception handling force the programmer to use return code checking to verify if a method call has succeeded. This is fine, but as I alluded to above, can add noise to code and diminishes flexibility.

One example of the power of unchecked exceptions is e.g. an exception mapper in a REST server. This allows the programmer to throw an unchecked exception at any point in the server application code and this will (if not handled by the application code explicitly) get caught by the exception mapper. It will then map this exception to an appropriate HTTP response code and return it to the API user.

An argument against exceptions is that there is a large 'overhead' to handling them. I've yet to see a scenario where an application performance bottleneck is being caused by exceptions. YMMV.