Yes, but unless all cars have speed limiters in them at those speeds, the drivers go faster regardless.
Basically, you set the speed limit at the speed most of those drivers are already naturally driving at, so that the people who do mind the speed limit will travel at the same speed of travel as everyone else.
And then, of course, obviously there will be outliers.