This is why e.g. go pro uses a hybrid approach with an embedded rtos for the camera stuff and then virtualized
Linux on top of this for less critical work where you want to use a stack e.g.http streaming. 3 yrs back I worked in a team which looked at booting Linux for a camera device in < 100ms and couldn't find a solution.