Hacker News Reader

Comment by asgard1024 | Hacker News Reader

I think what makes Python unique and successful as a programming language is a strict pragmatism (rejection of any ideology) and rather conservative choices in features to add (that is, don't add everything that someone suggest, wait until we can see it's a good idea and it cannot be readily composed from existing parts).

Scheme, OTOH, has some ideological choices. The whole notion of programmable programming language that comes with Lisp tradition (which includes s-expressions). The emphasis on teaching and small libraries. Attempt to somewhat model mathematics in numerical tower. These design choices, while interesting, are going against practicality.

I am big fan of mathematics, Common Lisp, Haskell. There is so much beauty and elegance. And yet, when I need to get something done quickly, I will just use Python because it drops the mathematical purity without prejudice, just like a physicist working on an impossible problem.

View on HN

> is a strict pragmatism (rejection of any ideology)

What? Python is very pushy on its ideology of "there should be only one way to do it", which is an exact opposite of a Lisp complete lack of ideology: "do it any way you like, turn Lisp into any language imaginable".

> rather conservative choices in features to add

Which is only ok if your language is extensible, while Python is most definitely not anywhere near an extensible language.

What I mean is that Python tries hard to find a compromise solution between various ideologies. Whether or not that is an ideology I would leave up to Bertrand Russell; I think it isn't.

Regarding extensibility, again, there are people who argue against it and have an opposite opinion to Lispers. Python tries to find a middle ground, conservatively. It's not super extensible like Lisp or Forth, but still quite a lot compared to many mainstream languages.

> there are people who argue against it

These are the people who don't know any better, arguing out of ignorance, not from some valid ground.

> Python tries to find a middle ground, conservatively.

How exactly? I do not see any extensibility features in Python, none at all. Some dirty hacks with rewriting the compiled bytecode in runtime do not count.

> These are the people who don't know any better, arguing out of ignorance, not from some valid ground.

That's very unfair. Let's take Lisp-style macros. There are good arguments against macros in programming languages. For example, keeping the amount of primitives small and core language simple. I don't agree with these arguments (I am pro-macros), but it's not just ignorance.

> I do not see any extensibility features in Python, none at all.

Not sure what you mean, but macros are not the only way to extend a programming language.

One particularly common method is called "functions" (if you have ever coded in assembler, you will know what luxury that is). Then you have classes, operator overloading, metaclasses, decorators, native C interface, AST parser in the standard library..

Not sure what more, aside from macros, you could want regarding extensibility. Python is more extensible than C and Java, which are the most common programming languages in use.

> Let's take Lisp-style macros. There are good arguments against macros in programming languages. For example, keeping the amount of primitives small and core language simple.

Wait, that's the argument for including Lisp-style macros as a primitive feature of the language: by doing that, you reduce the number of primitives needed, keeping the number of primitives small and the core language simple.

With macros, you can have fewer primitive control structures in the core language, and build other control structures that are desired on top of the existing ones + macros.

So, while there may be arguments for keeping macros out of programming langauges, the one you've presented is, instead, pretty much the main argument for having Lisp-style macros.

I don't want to get into this discussion, because, as I said, I disagree with these arguments; I just wanted to point out they exist and are valid.

Anyway, I already outlined the argument in another of my comments, where I talk about set theory and integrals - the "axiomatic" primitives are different bunch than "useful" primitives. So trying to keep language small may lead to different decisions based on which primitives you want to keep.

So for instance (as a simple example) map and filter can be built with recursion, but why bother with learning how to do it, if I can just call them from the standard library? (Or in case of Python, use generator comprehension.)

Another example would be complaint below, that Python doesn't have proper closures. Valid complaint or not, Python has classes, and you can pretty much build one from the other.

> they exist and are valid.

Uhm, no, so far not a single valid non-straw-man argument against macros.

It's important to mention that macros do not affect the rest of the language design choices in any way whatsoever. The underlying language can be literally anything, macros can be added on top of any imaginable combination of design decisions.

In case of a python-like language, see Converge, for example.

> It's important to mention that macros do not affect the rest of the language design choices in any way whatsoever.

I think you're wrong. It affects how people use the language, and how the community develops around it, and other things. Perhaps if people would use macros as a sort of last resort thing, then it wouldn't matter, but from my experience, they don't. It's probably most visible in the Forth community - every implementation of Forth is subtly different from the others, and by design! Chuck Moore even said that the Forth standard goes against the very idea of Forth.

I now work on a large codebase in language with macros (SAS), and while it's powerful, it also causes problems; the code where any calculation can take place during compilation is harder to read, sometimes, and they cannot be easily translated to other language, for example.

The truth is, the programming community moved from macros, which are powerful and easy to implement, to other abstractions, less powerful but more specific to problems that people usually face (such as OOP). Many languages from 70s and 80s had macros, but today it's not so common. This transition is a little bit similar to transition from imperative to functional style, which will probably happen in the future.

> It affects how people use the language, and how the community develops around it, and other things.

It affects the use (in very positive way), but does not affect the design.

But, what is true, is that a majority of people using macros do not understand how to do it properly and do not follow the right discipline. What is needed is to promote such discipline instead of spreading lies that "macros are dangerous".

And no, there is a very strong trend back to metaprogramming at the moment: Julia, C++, D, Clojure, Rust, all that stuff. All the other methods of implementing abstractions (OOP and the other pathetic things) have a very pronounced maximum of the level of abstraction, while there is no limit when metaprogramming is used to implement hierarchies of eDSLs.

> It affects the use (in very positive way), but does not affect the design.

It does, because the usage was one of the design criteria for Python. That's what I mean by "Python being conservatively non-ideological"; they don't want to add features that are controversial in the programmer's community.

> But, what is true, is that a majority of people using macros do not understand how to do it properly and do not follow the right discipline.

Maybe; maybe you don't want to accept that there are good reasons against. I accept it, even if I disagree; you can do the same.

In any case, opinion how should people be programming (e.g. use macros), is ideology. One ideology can be, in practice, better than the other, but it still is an ideology. Python tries to avoid the controversial ones and only adds those that have significant consensus.

> What is needed is to promote such discipline instead of spreading lies that "macros are dangerous".

You should be a little more honest with yourself and admit there is a grain of truth in it. Things of great power are almost always dangerous. Heck, even the Lisp community is divided whether or not the macros should be "hygienic".

> there is a very strong trend back to metaprogramming at the moment: Julia, C++, D, Clojure, Rust

First of all, I think macros and template metaprogramming is a different feature. So unless D or Rust have macros, you end up with Julia and Clojure.

Second, I am talking about languages used in the industry; in the 80s, macros were in widely used languages such as assembler, C, PL/I, Common Lisp, SAS, M4, Forth.. It's simply not what we see today. It may change in the future, but given the general trend in programming languages to add constraints (smartly) rather than remove them (I mentioned FP), I think it's unlikely (just to remind you - I actually like macros, at least in CL).

Finally, you can do (equivalent of) template metaprogramming in Python (mostly, templates are way to circumvent the type system, which is already done deal in Python). You can also define DSLs in Python pretty easily, especially with __call__() methods. The bottom line, for me, is that Python has lots of small niceties (mostly in the standard library) that collectively overweight the advantages of macros.

> maybe you don't want to accept that there are good reasons against

I'm a scientist. Before I accept something I need to see it first. I have not seen a single non-straw-man argument against macros yet. Not a single one. Either vague scaremongering without any specifics, or attacks on something that is totally unrelated to the macros.

> Things of great power are almost always dangerous.

Macros combined with the proper discipline are by orders of magnitude less dangerous than pretty much any other programming practice in existence. That's the whole point - our aim as engineers is to eliminate complexity. Macros are the ultimate way of doing so.

> So unless D or Rust have macros, you end up with Julia and Clojure.

Rust got proper, Lisp-style macros. D metaprogramming is Turing-complete, but yes, quite a bit behind, now even less powerful than the modern C++.

> C, PL/I, Common Lisp, SAS, M4, Forth

Of this list, only CL and Forth had proper macros.

> that collectively overweight the advantages of macros.

Nothing can overweight the infinite (literally) list of advantages of macros. With macros, every single semantic property you can imagine is available to you at no cost. Without macros - only the limited set of things that the language designers decided to give you.

Python's design choices (imperative, untyped, interpreted, etc.) are just as ideological as Lisp's s-expressions.

I would argue not. These paradigms actually encompass other models, and it that sense, they are pragmatic conservative choices (in a sense, "minimal").

You can program functionally in Python. It doesn't really gets in way of doing that, which cannot be said about functional languages, they are often designed to prevent you programming imperatively.

Dynamically typed languages can be understood as having only one type, so it's in a sense a type system that encompasses all the other type systems. For fans of typing, Python added optional type annotations, again, as a very conservative choice.

Interpreted - again, it's probably easier to write interpreter than compiler. Once you get into compiling, you have to make some more choices, which Python conservatively forfeits.

That is not to say that these choices aren't cultural (for example, most people are more familiar with infix notation, although even here there may be pragmatic reasons - the exactly right amount of parentheses). Just that it tries to encompass various tastes and be conservative about it.

> it's probably easier to write interpreter than compiler

It's not easier. Interpreters are much more complex and much less robust than compilers.

Über fast JIT virtual machines, maybe. Even bytecode interpreters are likely simpler because while you still have to compile to the bytecode, it is generally simpler and higher level than x86 or ARM assembly. Direct interpreters (based on the AST) are slow, but dead simple.

What makes you think interpreters are more complex?

> Direct interpreters (based on the AST) are slow, but dead simple.

Still far too complex, and, what is worse, very convoluted. You cannot split such an interpreter into series of very small and easy to understand steps, it must do everything at once, in a single huge recursive visitor. Compare it to a simple compiler which will rewrite one distinct aspect of a language at a time.

Another awful property of interpreters is that you can't strip them away from the runtime, it must be woven into all the parts of the interpreter and cannot be abstracted away. While compiler can be isolated from all the runtime issues which should be implemented separately (or simply reused).

I don't think compilers are easier to write than interpreters, but you should try and find the source code for some of the basic compilers from the eighties. The difference between (lex, parse, execute) and (lex, parse, generate assembly) are very small, if you are willing to forego optimizations. A compiler can even use thenexact same data structures as the interpreter to store variable names and their values, and use subroutines to look up and store values.

Fast? No, but it will be faster than that direct interpreter (some Basics of the '80s scanned a linked list to find target lines of goto's and gosub's. If you manage to get rid of that, tight loop can get a lot faster soon)

I've written a few interpreters during my (almost) 30 years of programming; never a compiler. The thought of 1) machine code generation and 2) optimizations scares me every time.

> The thought of 1) machine code generation and

Don't do it. Compile into a higher level representation and reuse and existing machine code generator (LLVM, even plain C), or compile into a higher level VM with threaded code implementation (Forth and alike). Although even a full compiler with a native codegen is still much simpler than an interpreter.

> 2) optimizations scare me every time

As if you don't have to optimise in an interpreter. Not to mention that even an unoptimised compiler would produce much faster code than your average "easy" interpreter.

Compilation is nothing but a sequence of trivial term rewriting passes. Each can be as simple as you want, doing one trivial thing at a time. Easy to understand, easy to write in a fully declarative way, easy to reason about, easy to debug and maintain. You don't even need a Turing-complete language to do it.

And an interpreter is always a pile of convoluted mess, leaking multiple abstractions in between layers (if you're lucky enough to even get any distinct layers at all). I always wondered how people can even compare interpreters to compilers.

Hmm. Interesting way of looking at it ("term rewriting passes"). I'll have to think about it, thanks.

You may want to take a look at Nanopass: http://www.cs.indiana.edu/~dyb/pubs/nano-jfp.pdf

> You can program functionally in Python.

Python doesn't even have closures.

> Interpreted

Languages are neither interpreted nor compiled. Implementations are.

Python doesn't even have closures.

The usual reason people give for believing this is the inability to mutate closed-over variables.

If that's the case, then a strictly-immutable FP language similarly "wouldn't have closures" and thus could not be FP. Problem solved!

You can mutate closed-over variables in Python. Here's an example....

def some_function(val):

    def say_stuff():
        print("message is: {}".format(val))

    def mutate_me(new_mutation):
        nonlocal val
        val = new_mutation

    say_stuff.mutate_me = mutate_me
    return say_stuff

if __name__ == '__main__':

    f = some_function("hello")
    f.mutate_me("goodbye")
    f()

> Python doesn't even have closures.

So what's this?

    >>> x = 1
    >>> def f(y): return y + x
    ... 
    >>> f(2)
    3

EDIT: Note that my example works even if the definition of x and f appear within the definition of another function. If closed-over variables must be mutable then Haskell doesn't have closures!

I believe you are confusing closures with scope.

Closures define the set of variables whose lifetime extends the function they are in because of their use in another function.

For instance:

    ;; Function that returns a function that counts even numbers
    (define (even)
      (let ((x 0))
        (lambda () (set! x (+ x 2)) x)))
    (define count (even))
    (count)  ; → 2
    (count)  ; → 4

In Python:

    def even():
      x = 0
      def add():
        x += 2
        return x
      return add
    count = even()
    count()  # → UnboundLocalError: local variable 'x' referenced before assignment

That said, Python does support a less convenient style of closures — it works if the variable is a container (such as a list, a dictionary or an object).

Python supports closures even when the variable isn't a container. For instance:

    def add_double(n):
        x = n * 2
        def add(y):
            return x + y
        return add
    
    f = add_double(1)
    f(2) # -> 4

What python versions prior to 3 didn't support is rebinding a variable from an enclosing scope; not because of any fundamental feature of the implementation, but because there was no syntactic way to differentiate between a assigning to a variable in an enclosing scope, and creating a variable in the current scope. So x += 2 is short for x = x + 2, which python interprets as creating a new variable x in the current scope and then trying to access that variable before it's been assigned.

Python 3 introduced a syntax to specify that you wanted to assign to a variable from an enclosing scope, the 'nonlocal' keyword. So your example would be:

    def even():
      x = 0
      def add():
        nonlocal x
        x += 2
        return x
      return add
    count = even()
    count()  # → 2
    count()  # → 4

So C has closures now?

    #include <stdio.h>
    
    int x = 1;
    
    int f(int y) {
      return x + y;
    }
    
    int
    main(argv, argc) {
      printf("f(2): %i\n", f(2));
    }

The reason they are called closures is because they close over the, lexical, environment where the function was defined.

Python doesn't have lexical scope, hence it doesn't have closures.

"Python doesn't have lexical scope"

Can you explain why https://en.wikipedia.org/wiki/Python_syntax_and_semantics#Cl... says:

> Python has had support for lexical closures since version 2.2. ... Python's syntax, though, sometimes leads programmers of other languages to think that closures are not supported. Variable scope in Python is implicitly determined by the scope in which one assigns a value to the variable, unless scope is explicitly declared with global or nonlocal.

Just been reading up on this. Python has sort of read only closures where the free variables are stored against the functions. You can't alter a variable inside the closure like you can in JS. Personally, I don't mind - I'd rather have a better idea of which scopes are changing which information, but I don't think they're considered "true closures."

More over here - http://stackoverflow.com/a/23558809/171450

This is incorrect for at least python 3.x

I.e.,

  Python 3.4.3 (default, Apr  7 2015, 08:05:21) 
  >>> def mk_generator(initial):
  ...    var = initial
  ...    def increment(count=1):
  ...       nonlocal var
  ...       var += count
  ...       return var
  ...    return increment
  ... 
  >>> a = mk_generator(3)
  >>> b = mk_generator(10)
  >>> a(1)
  4
  >>> b
  <function mk_generator.<locals>.increment at 0x1040a7d08>
  >>> b()
  11
  >>> b()
  12
  >>> a(3) 
  7

Reference to a global, shared variable.

Also, Python is strictly typed.

Recently I've been musing the aphorism "Quantity has a quality all of its own". I think something similar applies here too. Python goes to such extreme lengths to be (seen as) pragmatic that I can't help but conclude that "Pragmatism has an ideology all of its own".

> pragmatism (rejection of any ideology)

Pragmatism is an ideology, not the rejection of any ideology. (Perhaps more generally, pragmatism is an umbrella name for a class of ideologies which are all about maximizing some objective utility function, they differ in what utility function they are maximizing.)

When we want to get something done, we typically reach for what is most familiar.

Python wasn't my first programming language. And I think I gave a fair chance to other languages. But there come these moments where you are like "oh, this would be really simple in imperative code". Ideology gets in the way; to claim the opposite is just kidding yourself.

I probably gave the most chance to Common Lisp. It's a very pragmatic language too, unfortunately (largely for historical reasons), the standard library didn't keep the pace. It's not just a matter of what you can do, it's also the design. Python standard libraries almost always put the user first and the purity second.

For instance, Python would never add versions of functions remove-if, remove-if-not, delete-if, delete-if-not. That's mathematical purity (in if vs. if-not) and letting the onus of choice on user (in delete-if vs. remove-if) instead of conservatively making reasonable default.

At the same time, Common Lisp library lacks things like function to strip whitespace from string; you can do it, but Python has it as a function. It's just hundreds of little things like these that make Python more pleasant to work with.

I genuinely want to like Lisp more than Python (I am learning Clojure about now, which is on par with Common Lisp when it comes to pragmatism). But sometimes it is just a little more frustrating, because of ideology and not putting user first when it comes to API design.

If you do not declare your ideology upfront you will still end up having one, only implicit and ad hoc.

This does not apply to Python, however. It does have a explicit, if not entirely serious, statement of its ideology in The Zen of Python. (For those unfamiliar, read http://c2.com/cgi/wiki?PythonPhilosophy. It has The Zen and some discussion.) Python takes its ideology pretty seriously, too, as evidenced by how much the words "pythonic" and "unpythonic" used in discussions involving the language. In discussions around, e.g., Java you will find no terms of that sort.

I already explained that below, but let me elaborate. (But beware I am not a big fan of Zen of Python, precisely because I think it's misunderstood.)

The thing is these rules do not really give much guidance on how to design a programming language. These don't hint whether or not the language should be functional, or should have small library, or should have classes, and bazillion other things that programming language designers tend to argue about.

We can pretty much agree on these rules. And that's the point, these rules are all about making tradeoffs.

Maybe you will like Hy. It's a Python import hook that allows you to load files with a syntax similar to Clojure into Python. It also has an interpreter, so it can work as a standalone language or just dropped in Python projects. And it's not just the syntax, you can of course use Lisp macros!

https://github.com/hylang/hy

Heard about it recently, haven't used it, but certainly is an interesting project!

I used it in a Django project for specifying the urls, to have an easier time parsing them to build client-side validators.

I'm sure there are better solutions to that though, and I never got around to actually generating the validators.

The notion of a programmable programming language is an implementation of the mathematics of computation. It's as ideological as addition. COBOL allowed self rewriting code...probably because all software is [when we're at the right level of abstraction] is a mechanism for a computer to rewrite its instruction set.

A programming language that does not allow self rewriting code can only come about in one of two ways: a lack of cleverness on the part of its designer or a willful decision to disallow certain types of computations. Willful decisions are driven by systems of beliefs and values. One person's beliefs are an ideology to someone who doesn't hold them.

In my opinion, Python's practicality stems from its library ecosystem and that ecosystem is an accidental artifact of the epoch in which it was born and grew. The internet and open source made sharing of libraries practical. The idea that Java would be the one true language in enterprise and the economics of MatLab in academia and the public failure of Lisp as a startup in the 1980's, all helped.

But "Python" is fungible with "Javascript" in any it-just-lets-me-get-things-done argument without too much problem. More people use Javascript to just get shit done than any other language.

Python doesn't reject ideology. It's ideology is "minimize required background knowledge and learning curve".

Python is so ideological it is not even funny. It has a crippled lambda because Guido couldn't find a syntax he liked for it. It doesn't have tail call elimination because Guido is under the, mistaken, impression that they inhibit backtraces. The zen of python spouts explicit is better than implicit as a marketing slogan to cover up a design wart with Python's lolscope. It only had local and global scope. Python Scope is still broken[0] btw. But don't worry, it is not a bug because it is documented. (╯°□°)╯︵ ┻━┻. The only reason Python passes as a language with good design is because people compare it to PHP or Javascript.

How is the numerical tower impractical? Having to check for overflow/underflow after every addition is what is impractical in my book.

> I am big fan of ... Common Lisp ... There is so much beauty and elegance.

Idk where to people get the idea that Common lisp is 'beatiful and elegant' (in opposition to practical I may add). It is eminently practical. Have you seen loop and format (which btw Python has adopted with some glaring _practical_ omissions like list iteration ~{~}.)? The whole anything but nil is true to allow functions like digit-char-p to return a number instead of t is another practical design choice. Do you think there is anything elegant in modifying the readtable? I could go on and on. CL is a practical language that, apparently due to ESR's nirvana bullshit quote, people who haven't used it have a misconception about its design.

But back to the point, Scheme's design (ideology if you want) is to play to pick features that synergize as to allow discard others as they can be expressed with the previous features/concepts. You don't need iteration, as it is a special case of recursion. You need iterators, you have map, which is just a function the user can define, etc. And this is relevant to teaching computation as it allows the teacher to waste as little time teaching a language and allows him to focus on CONCEPTS. The amount of scheme knowledge needed to finish the SICP can be taught in 15 minutes (It doesn't use macros). All the difficulty lies in the concepts it deals with. Getting the language out of the way allows to cover a lot of ground, for example in the SICP one implements a Virtual Machines and a compilers targeting it.

Now if you want to argue that nowadays the student needs to learn how to pipe software together and has less need for concepts that is another thing altogether[2], but saying that Python is practical (as opposed to ideological) and Scheme's 'ideological' choices hampers the students learning is bullshit.

[0]: https://bugs.python.org/issue24321#msg244357

[1]: https://twitter.com/whitequark/status/604088954816716800

[2]: Of which I'd quote Dijkstra: "It is the university responsibility to give society not what it wants but what it needs."

> The only reason Python passes as a language with good design

..because ability to compromise is a good design, in a sense.

I kinda agree about the scope though, but I don't think it's a big deal. Sometimes having a hierarchy is overrated, and two levels of names is good enough.

But you will find design mistakes in all languages. Every designer errs. Python doesn't actually have that much warts because the designer is quite conservative.

> Idk where to people get the idea that Common lisp is 'beatiful and elegant'

Maybe from the paper that introduced Lisp, of which CL is descendant? I am aware about CL's warts, but these are result of Lisp's success rather than its original design goals.

> Scheme's design (ideology if you want) is to play to pick features that synergize as to allow discard others as they can be expressed with the previous features/concepts

I understand. But the reason for that is that people consider it beautiful when you start from small subset of features. It doesn't however reduce the need to know the more practical concepts. Like in mathematics. The fact that we can build everything from set theory doesn't mean we don't need to know about integrals.

Some universities skip the basics (set theory) and teach only the practical concepts (integrals). Perfectly fine in my book.

> Now if you want to argue that nowadays the student needs to learn how to pipe software together

I have said it on HN before, it depends on your goals. If you're teaching future programmers/computer scientists, sure, starting from Scheme or even Lambda calculus is better. Because you can assume people will do the real work in other languages. But if you teach a general non-programmer audience to program (like scientists), why not teach them just one language, which they can use in practice?

"Python Scope is still broken[0] btw"

Yes, it's working as designed, and I see how it can be considered broken, but it's not due to a scope issue. The "except X as x" is specified (with Python 3.x and PEP 3110 at https://www.python.org/dev/peps/pep-3110/ ) as doing a "del x" at the end of the except clause, and that was done for garbage collection reasons.

I've found in the past couple of years Go has fallen into that pragmatism bucket for me. If I need to throw something together quickly it's a good tool for the job thanks to the simple syntax, decent tooling and significant standard library.

I found Go quite obstructive compared to Python

ideology: things I don't like

pragmatism: things I like

Reply on news.ycombinator.com