It depends on the bike and on your condition. A normal cheap road bike weighs about 12 kg and an e-bike 22-24 kg. So riding the e-bike without e-assist is like riding a normal bike while being 10kg overweight (20 pounds maybe?) or like loading a bike with 10kg of cargo. It's doable, but is a lot heavier.
But there are also top of line electrified road bikes which are almost (almost!) like the road bikes they use in Tour de France but with an engine strapped onto them. They weigh about 11-13 kg and you'd have no trouble riding them without the assist. Of course those aren't cheap...
40 mile trips (60km?) are doable on e-bikes, but you'd have to conserve the battery. Like turning off the assist on flat terrain and only using it for riding uphill. Since you have a display indicating how much charge there's left you won't all of a sudden run out of power.