If you're used to this, it's nice and readable.
Or you can do,
return filter(None,tokens)
Not obvious, but giving None is like giving "lambda x : x" to filter().