The Python yield keyword explained
stackoverflow.com
stackoverflow.com
What is a metaclass in Python - http://stackoverflow.com/questions/100003/what-is-a-metaclas...
Understanding Python decorators - http://stackoverflow.com/questions/739654/understanding-pyth...
Thanks for the feedback. Because of comments like these, I decided to spend more time at training people for a living, and I'm loving it.
So don't underestimate the power of web comments, it can literally change the way people live.
def y():
yield 1
yield 2
for i in y():
print iNot Yield as in the common multithreading command to allow another thread to run.
PS. Anyone know why they chose such a confusing name?
For example:
def createGenerator():
print "aaa"
mygen = createGenerator()
aaa
Whereas
def createGenerator():
print "aaa"
yield
mygen = createGenerator()
To emphasise this point, it is the existence of the yield keyword which causes the function to return a generator when it is called - it will not execute any of the code. This is why the following results in an error:
>>> def hello(j):
... if j>2:
... return "big number"
... elif j<2:
... yield "small number"
A more detailed explanation can be found here http://docs.python.org/2.5/ref/yieldexpr.html
This seems a little weird syntactically though because it feels like the interpreter is somehow reading ahead in the program. It also means you don't necessarily know that this is a lazy function unless you read to the end.
A more intuitive syntax might be something like:
def lazy createGenerator():
print "aaa"
yield "bbb"Also, Python has precedent for this scanning behavior. One of the fundamental features of Python is that the scope of a variable is determined by the presence of an assignment statement in a function. For example:
a = 1
def f():
print a # UnboundLocalError occurs here
a = 2
f()
A specific syntax for generators would in my view be generally superior, containing the same number of new keywords (generator now replacing yield) and providing enhanced clarity.
generator hello(n):
for i in range(n):
return i
def first10(*iterables):
n = 0
for iterable in iterables:
for item in iterable:
yield item
n += 1
if n == 10:
return first10 = lambda iterables : itertools.islice(itertools.chain(*iterables), 0, 10)> To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is bit tricky :-)
(I do agree that a formal education is valuable, even if you already think (or know) you're good at programming. I learned a lot from my course. There's a lot to be said for being forced to learn a bunch of stuff, and when you're under 25 it's not like it's going to hurt.)
College is (most of the time) American for university.
What does that mean/how does it affect me?
It means that only people who have specifically turned "showdead" on can see your posts. No one can reply to you posts directly. Your upvotes do nothing and don't count. But crucially, to you, everything seems completely normal.
And howd that happen?
An administrator decided that your first ever comment didn't fit the tone of this site (which it didn't, but well, you were new), so they decided to instantly and silently ban your account forever. So you have been commenting, for a year, and basically no one has seen your posts. I feel really bad about that and I hate this objectionable practise.
No, that is just something HN does.
Roughly there are three fundamental control transfer operations: conditional jumps, subroutine calls, and coroutine calls. A subroutine is a slave, the caller is a master. With coroutines, the caller and callee are peers.
Coroutine calling is more fundamental and easier to program, but it isn't available in C.
The theory is about continuations. A continuation is just "the rest of the program". You can think of it as the program counter (PC). Subroutine calling works by passing a continuation to a routine, when the routine is finished it invokes the continuation. This is done by pushing the program counter on the stack, and the subroutine popping it with a return statement.
Coroutines work by exchange of control. When you call a coroutine you give it your current continuation to call when its ready. But also you do not call the coroutine at the beginning. You call it where it last left off: at its last continuation point.
A set of coroutines are usually called fibres. They represent interleaving of control. You can emulate them with pre-emptive threads and locks, but coroutines are synchronous and non-premptive.
Felix and Go both make heavy use of coroutines (called fthreads and goroutines). Iterators as in Python are a special case.
In general on today's badly designed CPU's you have to think of coroutines as requiring stack swapping. Threads do this which is why you can emulate coroutines with threads.
By far the most well known coroutine scheduler is .. the Operating System. Its basically a coroutine of applications. This is why you can read and write to files: the real world is event driven but callbacks are impossible to program with. So the operating control inverts the events by stack swapping so your application can be written as a master.
You think you're calling the OS, and the OS thinks its calling you. You're both masters. That's coroutines.
Python (and Felix) both have yield, but that's a special case. In general the model is reading and writing channels: yielding is just writing the "sole" channel and getting a function result is just reading it. A channel is basically a "place to swap stacks".
Citation needed.
For coroutines, each coroutine needs its own stack. That means you have to have a dynamic memory system baked into the language. And maybe garbage collection too.
> on today's badly designed CPU's you have to think of coroutines as requiring stack swapping
I can't think of an implementation of yield (let alone general coroutines) that doesn't require a separate stack for each coroutine. I admit that I haven't learned very many of the stranger forgotten architectures that are out there, so I might be blinded by the limitations of a somewhat conventional experience.
But I'm also thinking it might even be provable that each coroutine needs its own stack: Think of a program that has m generator functions, where f_1 calls f_2, f_2 calls f_3, ..., f_{m-1} calls f_m. Each of these subroutines creates n copies of its next-level generator, and steps those generators and yields to the parent unpredictably (for example, depending on input from a user-supplied file). It seems like if m and n are large enough, you'll have no choice but to resort to swapping stacks.
Why? Just statically give every co-routine it's own stack.
More formally, I'm pretty sure you can write a program that always halts, but for every pair of large numbers N, K > 0, there are at least K different input values that result in (1) at least N coroutine invocations being simultaneously active, and (2) for each of those K different input values, those invocations finish in a different order.
If civil engineers had a forum site and someone asked "how do I calculate the load a simple beam", people would say, what the fuck are you doing in your job if you don't know that? Who is employing you and why? Where did you go to college? Who was your tutor because next time I'm at a dinner at my college I'm going to ask him how the fuck did they pass you.
Our current trend of saying college doesn't matter really starts to be a problem when professional programmers (yeah I'm assuming that the guy in the question is) are asking basic questions like this. This shows that we do need college education for programmers.
It also irritates me - this guy could have taken the time to learn all these basics, but for whatever reason he thought it didn't matter, and now he's paying for it.
So assuming that only well-educated programmers can have questions about a programming language is asinine.
My university education didn't cover coroutines either.
Why do you look down on all these people?
And by the way, I'd rather work with fellow programmers who don't know what a coroutine is than with an insufferable elitist like your posts make you out to be.
Generally curiosity and willingness to learn is worth more.