One problem I fought was that a lot of people expected handholding from me, or wanted me in their meeting to "feel important." There really is value to having your manager act as a gatekeeper.
Another problem I had was that less ambitious people in other departments (support, qa,) would throw their work over the wall to me. I eventually had to get involved with triage to train the other managers to push back on this.