Uhhh... what? Where did you get your PhD, and what did you do after? Most graduating PhDs end up in a research group as postdocs, with everything above them in the org chart. Even if you get a tenure track job right after graduating, you'll still have to answer to a lot of people.
As a tenure track professor, you don't really answer to many people. You have to beg for money (grants), constantly prove yourself (publications), and always be recruiting (students). Of course you also have to teach classes, do community service, and participate in department politics and initiatives. I'm not counting these as "answering to people," however, because none of them involve someone telling you to do something. They're all activities you must decide to do yourself, and then it's completely up to you how to do them. Is this what you meant by answering to lots of people, or something else?
The relevant academic joke is: "It's the best job ever! You can do whatever you want with your 80 hour weeks!"
[1] In CS, from my experience, postdocing is usually a 1-year "reset your brain" time before taking an academic job. I know in other fields it's often a mandatory many-year grunt-work experience.
Yes, this is what I meant. I dropped out of my PhD (CS as well), but when I was a student my advisor (who was still an assistant professor at the time) was spending most of his waking hours in department meetings, writing grants, finding a way to make $RANDOM_OLD_PROFESSOR happy, and so on. To me it seemed very much like he had 10 million voices telling him what to do (of course he got to choose what he would actually do and what he wouldn't, but it didn't feel to me like he was much more "free" than he would have been in an industry job).