It's unlikely that they would drive by remote control as feel is so important - a racing driver relies on his backside as much as his eyes. Drivers use physical sensation to feel the limit of grip and the balance of the car, something that would be very different to control remotely.
You could potentially go much faster without drivers, as cornering speeds are artificially limited by the rules to maintain driver safety. The severest corners in F1 induce 5G of lateral force. Inexperienced drivers often spend later laps of a race resting their head against the side of the cockpit, as their neck muscles are exhausted.
In a completely free racing formula, cars would have vast amounts of mechanical grip and downforce, weigh next to nothing and corner so fast that no human could survive a lap.
I agree the driver still has a lot of influence. But, if drones can fly around in the skies then a pilotless F1 car is not so unrealistic. Fans liking robo F1 cars no one really knows for certain but it would be very interesting to see.