I don't have time for a complete post (and I don't have permission to publish examples from the datasets), but here is some "quick" reply.
Once you see examples from the datasets, lots of problems come to mind (and to todo list a bit later). This is a quite expensive problem to tackle with.
Some requirements:
(1) you need datasets from various places, various weather conditions and various situations,
(2) everything must work in realtime,
(3) equipment is expensive (cameras, cars, gas, ...),
(4) error has to be minimal (we are talking about human lifes).
(and this is just part of requirements)
Student labs fail at money part, companies fail at lack of time (again, money; you need lots of time to deal with extreme number of situations and produce error prune product -- and nobody guarantees that you'll manage to do that).
And now, some problems:
- Everything has to work in realtime which means more than 30 FPS in average (you need to aim for higher average speed so you don't get lags in complex scenes). Computer vision algorithms aren't usually realtime, for example, for simple task as object detection one of the best realtime algorithms is Viola-Jones which is more than 10 years old and patented. In newer days there are some breakthroughs in this area, but quite small if you take ten-year gap (and we are talking about quite simple problem -- object detection).
- Then, the datasets. You need a lot of them. Different places, different times. All you can do is to beg someone for it, pay a lot for it or make an contract with traffic companies (for the product that you don't know will it work well; and good luck if you represent a student lab).
- Now, the data. Take a ride during the different weather conditions and times of day or year. You'll be ok, but from CV point of view, you'll meet hundreds different problems. Disorted view during the rain, big balls of light during the night, different types of cars (cars with trailers, bikes at the back, motorbikes, ...), damaged road, damaged traffic control, traffic accidents, ... you get the point.
- Then, think of number of things that you need to take care of -- traffic signs (very hard problem! specialy if you want to ride on local roads where some of signs are only partially visible), traffic fixes, etc.
Some of solutions are taking only some of the problems and tacking with them (more in way of alerting the driver). I'm not familiar with different sensors that deal with some of these problems, but maybe some of them aren't so expensive (today you can buy a car which can park itself (or that was just R&D showcase)).
Anyway, this is extremely interesting problem and it deserves us to fight with it. Unfortunately, from business side it looks like a big gamble.