Well, if it is a large leak, the smell should make it obvious when it is a really bad idea, and you'll have no trouble hearing such a leak. If you're trying to find a tiny leak this actually seems to work well and is relatively safe due to the speed at which natural gas dissipates. IMO it is somewhat less safe when using LP gas, since LP gas is heavier than air, and can pool near the floor. Remember, until recently, gas water heaters, and other appliances had a partially exposed pilot light, so any significant leak could potentially find that, but rarely did.

source: I've somewhat nervously watched old-timers do this. I've also accidentally discovered leaky welding torches this way.