To clarify, the gamma energy from K-40 in bananas is around 1.5 MeV, which is dangerous to humans. It's not the most dangerous, but it is dangerous in significant amounts (which are much much higher than what is in a banana).
A BED is usually taken as 1 micro-Sievert. Sieverts are a unit for measuring the biological effect of radiation, hence they are normalized to adjust for how harmful the given type of radiation is. A dose of 1 Sievert represents a 5.5% chance of eventually developing cancer. 1 Sievert is 10000000 BEDs.
The part that is a bit misleading about the BED is that it's not the same as the dose you get from eating a banana. This is because of how potassium is absorbed in the body; all the K-40 in the banana isn't staying around in your body. The BED is the dose you would be exposed to for absorbing the equivalent amount of K-40 in the same number of bananas. Bananas are fine for putting small doses or activities into perspective since the biological effects are negligible, but are not really great for comparing against more significant doses.