Generators allow one to easily transform code from non-buffered to buffered. Consider this example:
for y in x:
do(z)
Now, x may be a collection that was eagerly evaluated in previous steps, let's say a list, but then you've discovered that this list is too big to fit in memory, and you want to generate it in manageable fixed-size chunks, so you replace the code that created x to make x a generator. You don't have to touch the code above -- it will work the same way because generators have the same interface as collections.Another use: generators are used to implement async / await. I personally find this idea ridiculously stupid, but a lot of people (and especially those who don't understand what it does) like it a lot. Yielding mechanism, which is a feature of generators, is the one that's used to communicate / switch between co-routines (tasks) of asyncio.
Another aspect, besides bufferization is that you might want to delegate control over how much looping you want to do to a separate chunk of code. I.e. you may want to separate the generation of elements (hence, generator) from eg. filtering them, or transforming them in some way, or reducing them etc. If you didn't have this ability, you'd have to generate the entire collection upfront, and if your computation takes multiple steps that you'd like to separate into different code chunks, you'd have to also generate intermediate collections, even though, potentially, you don't need some of the elements in those collections. Consider, for example:
def powers(start, end, power):
return (x ** power for x in range(start, end))
def flt(a, b, c):
return a + b == c
def disprove_flt(upto, upto_power):
for power in range(upto_power):
for a, b, c in zip(powers(1, upto - 2, power), powers(2, upto - 1, power), powers(3, upto, power)):
if flt(a, b, c):
return a, b, c
return None, None, None
Which is, of course not a correct way to search for the counterexample to Fermat's last theorem, but I tried to find a popular enough subject so that the example was easier to follow.An exercise to the reader: rewrite the code above in such a way as to eliminate "upto" and "upto_power", i.e. to search until a counterexample is found (or indefinitely).
Practically? Yes, generators are often more memory-efficient, and thereby often more compute-efficient.
An optimizing compiler would need to know what actions are pure in order to rewrite code to be lazy. Python allows so much dynamism, I can only see that happening with a very sophisticated tracing JIT.