Perhaps. Maybe our brains are just an algorithm and data, maybe (for all I know) you are a bot, maybe we are all a simulation.

I think these sort of arguments (assuming you meant the first - that brains are just maths) that equate our thinking with a neural network are hard to deny but also we have thousands of years of history that have established that suffering is real and conscious, whilst debatably imaginary, is at least a consideration.

I therefore argue we must, for example, maintain a massive distinction between the morals of turning off a human brain vs turning off a neural network.

The calculations and output of the neural network could be achieved with pen and paper (albeit it might take years). I doubt a human brains workings could ever be calculated that way.