Very true - I'll admit that while I knew that peanuts are legumes not nuts, I didn't know that the others you mentioned were not nuts. I learn something new every day (and my son has a severe allergy to many of them - though not all - so I should know these things!).
And while I know my son can safely play with conkers, we most certainly have not tried to eat one!