Your sigma is a bijection on rational functions of a, b, not on the evaluations of those functions at particular values of a and b. In lambda notation:*

sigma f = \a b -> f (sigma a) (sigma b) = f b a

That means sigma f = \a b -> b - a. On the other hand, g := \a b -> 2. So sigma g = g.

Looking again at your equality:

> sigma f(5, 3) = sigma (5 - 3) = sigma 2 = f(3, 5) = -2

We should read sigma f(5, 3) as (sigma f)(5,3) = (3 - 5) = -2. Note: sigma f is not equal to sigma (5 - 3), because f is not the same rational function as "5 - 3"!

* This is an abuse of notation, because a and b are bound variables inside the lambda. In this case, being more precise would probably be less clear.