The CPU can not remember an infinite number of branches. Also, many branches will increase code size. With exceptions the unwind tables and unwind code can be placed elsewhere and not take up valuable L1 cache.
I suspect a modern CPU has a branch instruction saying "This branch will never be taken except in exceptions, so assume this branch is not taken". But I must admit I haven't seriously looked at assembly language for some time.
(EDIT: yes, modern CPUs including x86 and ARM allow the programmer/compiler to hint if a branch is expected to be taken).
> Also, many branches will increase code size.
I'd like to see some data on that. Of course branches take code size, but how much is that percentage-wise? I suspect not much.