Legally.
The law where I am is that human traffic directors (police, sheriffs, construction crews, etc) take precedent over what's in the book. If they didn't, nobody could ever get around a fire truck at a fire.
I just ignored him and went around when it seemed safe.
And that's where you get a ticket. Plenty of cops around here sitting at the exit of construction zones waiting for people just like you.
That's not breaking the law though.
I get that nobody's perfect and you might do it by mistake (going 35 on a unmarked road that was 30 max or something), but the way you write your comment makes it sound like it's something one does in a deliberate way?
Drivers like that shouldn't be on the road.
If there's an obstruction in the middle of my lane then obviously I'm going to deliberately go around it. Have you ever driven a car?