You may not be aware of this, but the dairy and egg industries kill an insane amount of animals. I too was an ethical vegetarian for years before discovering that and reevaluating my dietary ethos.
You make an excellent point about those industries - I originally had a comment about them above, but dropped it as it was getting a bit (more) off topic. My point at the time is that I'm not ethically against the actual act of eating eggs or consuming dairy, but you are correct that those industries, even many of the "friendly" ones, do things that I'm not ethically okay with.
The hardest part about becoming a vegetarian was actually admitting to myself that my actions AREN'T ethical by my own definitions, and that I would have to accept that while not giving up on the idea of improving. Kind of like how we are about charity - we all spend money on our own frivolities that we would spend to help someone in need were they in front of us...but we try very hard to not have such issues in front of us.