If both builds turn on debug info, why wouldn't they still be the reproducible?
For example, debug information for several languages I've worked with embed the full path to where the source code file was originally compiled. This can easily vary across two machines in a way that when debug information is turned on, the builds are different, but when turned off they are the same.
however debug symbols and "debug mode" are two different things.. at least in debian packages are generally compiled with debug symbols on, then separated into a separate package automatically and stripped into the final package as a separate step.