I wouldn't call myself a person with a mathematical background, but there are those people who believe it's just fine. [0] I don't have enough knowledge to debate that, but it would seem to disprove "basically nobody". Zero is a convention, like NaN or Inf are conventions.
A problem that Gleam has here is that the Erlang runtime does not have NaN or Inf in its float type (or integer type for that matter). It could be represented with an atom, but that would require an atom and a float having the same type in Gleam, which is not something the type system can do (by design). The operator could, in theory, return a Result(Float, DivisionByZeroError), but that would make using it very inconvenient. Thus zero was chosen, and there is an equivalent function in the stdlib that returns a result instead, if you wish to check for division by zero.