Inside the continental US, if you fly a small not-turbine-engine-powered airplane yourself and only go to rural airports with no tower, it is actually legal to have no transponder (see below, 91.215(b), 91.225(b)).

But, if the transponder is installed, you are legally required to turn it on at all times no matter where you are (see 91.215(b)(3), "all controlled airspace" means everything above 700-1200ft AGL (depending on exactly where)).

So for this to be legal, you can't just turn if off: you have to actually remove it from the airplane. Whether that is legal additionally depends on the airplane.

https://www.law.cornell.edu/cfr/text/14/91.215 https://www.law.cornell.edu/cfr/text/14/91.225