No, it really isn't. "Sugar" is not a poison. "A very large dose of sugar" is a poison. It's a very important distinction. Small or moderate amounts of sugar do not cause problems, and in fact give the organism more energy to work with. If you take the position that the substance is a poison simply because it's possible to hurt yourself if you take ridiculous amounts of it, then the term 'poison' becomes meaningless. Everything then becomes a poison, and it's then completely redundant to label something a poison.

Besides, sugar is the primary fuel for your brain; without sugar, we don't exist. It's a funny sort of poison that is necessary for our existence.