Hacker News Reader

Comment by junke | Hacker News Reader

> Now, compare that to what happens when I search for the same in Python. I get this:

> https://docs.python.org/2/library/re.html

I clicked and I saw the kind of technical documentation I love to read: precise, with examples. There is an introduction to Python's flavor of regular expressions, which highlights the potential problems (backslashes). This is not targetted at novice programmers, though.

> That’s the generic re library page. I have to scroll down to see which function I need.

Did you notice the table of contents on the left? You might want to try Ctrl+F. But you might have to scroll down too, I don't deny it. You have to read, too.

> And even then, all I get is a hundred words of generic description. No examples.

Ctrl+F "example" highlights the table of contents. There is an example for "findall".

> I then have to Google Python regex findall examples, and go through a dozen blogs written in 2008.

http://lmgtfy.com/?q=regex+findall+example#

Not really, there are plenty of recent posts (stackoverflow, etc). And having articles from 2008 would not even imply that they are incorrect. In order to see how old your post is, I have to look at the source, but even though it is quite recent, it contains factual errors.

View on HN

I think this is the exact type of comments that the author comes out against.

You aren't putting yourself in the shoes of a beginner.

Let's say I'm a beginner, and I want to just experiment with the language a bit (I've already read some tutorials or am not interested) - since I'm just trying to get the feel of the language and ecosystem, I'm not trying to get a deep understanding of the language and tools - I want to match a regex.

In the C++ and PHP examples he provided, the basic example is easily found - it's basically two actions - search in google and scroll to the example.

What you are suggesting takes several more steps - ctrl-f for a text that you might get wrong (you've searched for sample instead of example for instance), iterate until you get the right place, etc...

And that's assuming you even realize that you should run ctrl-f, why should you?

The main issue though is that if I'm a beginner I'm doing this kind of searches several times a minute - to have to stop and realize what to look for every time is extremely frustrating.

I've worked with Ruby,Python, Java and Javascript - python's (and django's) documentation is extremely verbose and rarely straightforward compared to the other ones.

You aren't putting yourself in the shoes of a beginner.

The real issue you and the author both run into is assuming that the wants, needs and learning styles of all people everywhere are identical. There's a good argument to be made for expanding the variety of styles and types of documentation available in order to cater to more ways of learning, but not a good argument to be made for suppressing or removing existing documentation since that boils down to "all right, let's cut off those people to focus on these people".

One big example that's not covered, but relevant, is how many (free, so potentially competing with the author's book) workshops or online complete guides to learn Python or Python-related tools have been popping up the past couple of years. Having an experienced coach who walks you through everything your first couple of days and gets you acclimated is a huge deal for some people (while others prefer to read on their own in an undirected way), but isn't accounted for in the article.

> You aren't putting yourself in the shoes of a beginner.

How do you know if I am putting myself in the shoes of a beginner or not? And what kind of beginners are we talking about? Are they comfortable with a browser? Do they already know how to program something? You can't expect a single document to be relevant for all kind of people. The reference documentation has a clear target audience, which apparently conflicts with what you expect from it.

> And that's assuming you even realize that you should run ctrl-f, why should you?

> (you've searched for sample instead of example for instance)

Okay, so you are talking about absolute beginners, with no experience whatsoever with programming. And the C++ and PHP pages are somewhat useful for them?

> I've already read some tutorials or am not interested

> Let's say I'm a beginner [...] I want to match a regex.

Now, you are saying that you already have experience with programming? Google "python tutorial regex" gives plenty of results, the first of them being currently "Regular Expressions HOWTO" (https://docs.python.org/2/howto/regex.html).

Absolute beginners need a teacher, a mentor, or a step-by-step book, or a very gentle tutorial, or an online REPL. That's fine. You can't please everybody in the reference documentation.

And what kind of beginners are we talking about? Are they comfortable with a browser? Do they already know how to program something?

I'll add to that, "Are they even computer-literate in any way?" I'd worked at a college a few years ago, and it is sometimes shocking how little the newcomers know about just using and troubleshooting basic problems with computers, nevermind learning how to write programs for one... and IMHO it's only gotten worse. Knowledge of things like basic filesystem concepts ("What's a directory? How do paths work?") in particular seem to be disappearing.

Quoting myself from a blog post i wrote when i was learning Python two and a half years ago [1]:

Let’s say i want to know how to do the Python equivalent of PHP’s foreach. The docs about the for statement say this:

    The for statement is used to iterate over the elements of a sequence (such as a string, tuple or list) or other iterable object:

    for_stmt ::=  "for" target_list "in" expression_list ":" suite
          ["else" ":" suite]

Right, so what does that mean? What’s ::=? What’s an ‘expression list’?

    The expression list is evaluated once; it should yield an iterable object. An iterator is created for the result of the expression_list. The suite is then executed once for each item provided by the iterator, in the order of ascending indices.

“Ascending indices”? “Suite”? This might be clear for a CI professor, but it’s not for the mere mortals that read the docs and want to get stuff done.

Things haven't gotten much better since then.

[1]: https://www.haykranen.nl/2014/02/12/saying-hello-to-python-s...

You were reading the wrong docs for a newbie. The language reference is intended for people who want a very concise and precise technical document, not an introductory explanation. If you read the very beginning, the language reference is very clear about what it is and isn’t:

> This reference manual describes the syntax and “core semantics” of the language. It is terse, but attempts to be exact and complete. The semantics of non-essential built-in object types and of the built-in functions and modules are described in The Python Standard Library. For an informal introduction to the language, see The Python Tutorial.

You should have been looking at the Python Tutorial instead.

https://docs.python.org/3/tutorial/controlflow.html

https://docs.python.org/3/tutorial/datastructures.html#loopi...

Another nice example of what the author complains about, a "it's your own fault" reply whenever someone complains that the official language docs are hard to understand.

Personally I almost never find any use from the docs whenever they come up in a google search, its almost as bad as those expert-exchange hits that used to appear. The core issue being that pythons docs are only a language reference for the vast majority of the standard library, and the size of the stdlib bein one of pythons strengths.

Personally, as a 16 year old (or possibly younger, it was a long time ago) with no internet connection and little programming experience I found the Python user documentation very helpful. That possibly had something to do with me actually reading the tutorial rather than dismissing it as beneath me or ignoring it altogether and attacking the person who brought it up.

Well, the discussion is about the docs recently available online, so you finding a python book helpful in the times when no python shell autocompleted is a bit outside the topic at hand.

I'm talking about exactly the same documentation and the same tutorial that's available online (well, obviously it was a much older version of it). Only difference is that the offline version is a directory of local HTML files rather than a website. The user experience is pretty much the same with both. Still pretty handy even in the StackExchange era, though it'd be more useful if Google didn't keep finding the wrong version of the docs (2.x when I'm using 3.x or vice-versa).

It’s not “it’s your own fault”, but rather “you should read the document you’re looking at, and think critically, instead of expecting that every google hit is going to be precisely what you are looking for right off the bat, with no need to scroll or read”.

If the author had gone on the Freenode #python IRC channel or the Python Users mailing list (or..., or...) and asked “Hi, I’m a newcomer trying to understand Python loops, what’s the best resource?” someone could have linked him to the tutorial. Or if he’d done a google search and tried skimming a few hits, he would have found it.

It’s hopeless to expect that every single document in the world will be tailored for every reader.

There’s a real value to a concise and precise technical document aimed at folks who need something closer to a specification than to an introductory explanation. It would be net-negative to remove the language reference from the internet, or clutter it up with a bunch of information that can easily be written somewhere else.

Imagine someone wanted to understand the basics of how levers and pulleys work, and they came across a quantum mechanics textbook, which wasn’t comprehensible at their current level of understanding. Should the reader blame the book, or keep looking for a more appropriate source?

Sure, in a perfect world i would read the complete tutorial before i get started on a new language.

But it's not a perfect world, i want to get stuff done. I have twenty years of programming experience, i don't need to learn what a for loop is (the first link) and i don't want to read a whole document just to find a reference to the items() function (second link). I just want to know what the equivalent of foreach() is in Python.

Sure, a document with a 'very concise and precise' technical description might be useful to some, just for looking up things it's not. The current alternative is Googling StackOverflow posts.

Sure, in a perfect world the documentation would guess who I am and adapt accordingly, be both highly detailed when I need it or informal when I don't.

But it's not a perfect world, I want to get stuff done. I have twenty years of programming experience, I know how to find the information I need.

Sure, the whole language explained in one place for all kind of people from beginners to experts would be nice. But I have Internet. I type "foreach python" in Google and I see a StackOverflow post.

> I type "foreach python" in Google and I see a StackOverflow post.

That's kind of indicative right there.

Indicative of what?

Are you trying to say that a the docs should have "Python for X programmers" sections, foreach (no pun intended) X in {Perl, PHP, Haskell, Lisp, Go, Javascript, C++, ...}?

That would be nice, but what is even better is a totally different resource that aggregates idioms for all languages:

https://www.rosettacode.org/wiki/Loops/Foreach

That the first result is a stackoverflow question about the language rather than a documentation page, is indicative that getting started with the language is not so straight-forward, even factoring in documentation.

If you're googling 'python foreach', you're probably at the beginner level, and the fact that a documentation page is not the first result means that the beginners documentation is not very discoverable.

The fact that there's a not-closed, highly-upvoted, highly-ranked stackexchange question about this very basic issue in existence /at all/ is what one might call a "documentation smell".

The documentation contains "for-each" and "for each", but sadly not "foreach". Why should "foreach" be in the documentation? There is no "foreach" keyword in Python. If you are looking for "foreach", someone else will look for "python do until", "python fold", "python switch" and maybe another one for "python macros" or "python transducer".

Some of those searches are legitimate, because "foreach" is quite used outside of Python and "switch" should be expected to be defined in every language. However, the documentation states what the language is, not what it isn't.

The goal of a reference documentation is like a dictionary: you define your words w.r.t. the other words defined in your language; this is already quite a complex task. But you don't mention words from other languages because that would start to be overly confusing and a burden to maintain (now you have to check how other languages evolve). There are other resources to make connections between languages.

> Why should "foreach" be in the documentation?

To make it easier to find "for", "for each", and "for-each"?

How come you know "foreach" in the first place, this is not even a correct english term? You learn it from another language? Wonderful, but why should every keyword from every programming language ever designed be present in the Python docs?

I never argued for every for every keyword from every programming language. But like I said, that there's a stackoverflow question about this one, I think, is indicative.

And this one is special because?

With all due respect, if you have 20 years of programming experience and can't grok BNF, it might be worth stopping attempting to learn new languages and learn BNF instead. Given than BNF defines how Python (and many other languages) is parsed, it's well worth learning.

Actually, the folks with 20 years of experience in other languages are the ones who most need to read the Python tutorial.

The python code I read from PHP/Java/C++/... “career experts” tends to be worse than the python code I read from people who have only been programming for 2–3 years, because they never bother to learn Python idioms or style, but just keep on writing code in the same old style in every language.

It’s certainly possible to dive into a new programming language and start writing nominally working code right away, filling in with a google search whenever you get stuck.. That’s not a way to ever end up with any kind of basic competence in the language though. It also doesn’t give you much of a leg to stand on for complaining.

The Python tutorial is short and breezy, a fast read which will put you well on your way to understanding the language. I’d recommend it for anyone who plans to spend more than a day or two reading or writing Python code.

Regarding the docs layout and "having to scroll down to see which function", I also don't quite get it. From my perspective, it's a very nice trait of Python docs compared to PHP docs, that the former actually explain how to use the various parts of the API together, before giving descriptions for every specific API. It's simply a much better way to digest it and learn to use it without making a mess (speaking of which and PHP...).

Coming from statically typed languages, I've really struggled with the Python docs, because they aren't really clear with what is returned from a lot of functions, and which arguments are allowed. I end up finding an example where it is being used, then deducing from what they do with the result what was actually returned.

Contrast this with the ugly-but-functional JavaDocs, which can be auto-generated, and easily shows all return types and arguments for every publicly exposed function and member variable. I understand why this can't be done automatically in Python, but at least they could put the time in to make it clear in their standard libraries.

To provide a counterexample, it takes me forever to figure out how to do something after reading Java/Cpp docs, but I instantly know how to do it in Python after reading the docs.

Typical workflow for Java: This function takes in these object types as inputs... Then you have to wonder how to get those object types so you scour through 10 more pages of docs to figure that out. You end up creating a factory of a factory of a factory to make that input object only to find out that it has to be further preprocessed by sending it somewhere else.

Typical workflow for Python: read docs, try out an example in the interpreter by passing a basic object type to the function (such as string or dict), copy code from interpreter into text editor. You are done.

I really love Python docs. It takes me seconds to find what I want and figure out how to use it. I hope they don't change anything.

That's more to do with coding style. I abhor reflection and factories, so if I write Java I don't use them. That said, you can end up writing a RandomNumberGeneratorFactory in Python just as easily if you are trying to achieve the same scaling and re-usability requirements that the Java program is.

The issue I run into in Python is that I need to call something which takes * args or * * kwargs, and the only way I know any of the arguments exist at all is because of some snippet of code on github, probably by the person who implemented the feature in the first place. Even if the argument is named I don't know if it should be a magic number which is defined in some enum-like class, a boolean, or a string to describe the name of what I want and pray that the library maintainers used the same capitalisation and localisation that I think is most logical. In fact, without looking at example code, I literally have no clue how to do anything.

As such, with Python I end up doing copy-paste programming which is dependant on my internet speed and how many other people had the same problem already. With C/C++/Java I tend to do a more build-it-from-scratch style programming, driven mostly I think by the style of the documentation.

> I abhor reflection and factories, so if I write Java I don't use them.

But factories etc are idiomatic Java. You can avoid it in the code that you write, sure, but how do you avoid it in the third-party libraries that you use?

I've been lucky enough to use Java for fun, not work, and as such I have the time to write things the way I feel they make the most sense.

The advantage of Python is that you do not need to think about type labels: "If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck."

For example, the builtin print() function accepts any object with write(string) method as the file parameter. There are ABCs classes that capture the "duck" essence but it would be an unnecessary pessimization to require a io.TextIOBase instance as the file parameter in this case. Don't write Java in Python. Go's io.Writer would work in this case (the type label as the documentation and the exact specification) but if you don't mind thinking about type labels and/or believe that the type of bugs that can be found by a compiler worth the trouble instead of just writing tests and expressing directly what you want the code to do then using a statically typed language make sense.

Though there are type hints (proposed by BDFL) in the recent Python (to enable better static analysis, IDEs code completion and refactoring) https://docs.python.org/3/library/typing.html If its usage is not limited (e.g., through Python culture convention); it could be a regression in Python evolution. If your API is so enormous that you need type labels; you should split it. Python code should communicate the intent to a human first over a compiler/static analysis tool.

Here's a static typing programmer perspective (how it feels) on dynamic languages "What is the appeal of dynamically-typed languages?" https://gist.github.com/non/ec48b0a7343db8291b92

Wholly agreed about not writing Java in Python, and I do write idiomatic Python. The problem is that I don't know what methods the returned objects have available, and as such whether they are a duck or a tiger. The examples show that it has a mouth and eyes, but I need to know whether it has wings or claws, and Python's docs don't make that clear.

Can I index the returned object, or only iterate over it? Is it mutable? Is it a dict or a tuple of (key,value) tuples? Did the author think it was worth it to make its own class for the return type, because they want the indexes where the value was found as well as the metadata?

Python, to a degree, brushes over these problems by being able to print out most types of data and then figure out where the problem is, but TBH I'd prefer to write the correct code to begin with, and the docs don't give me enough information to do so. As such, Python turns into a dodge-the-error-driven-programming, and I feel much less productive than I could be.

Yes, duck-typing gets you part of the way, but it doesn't let you pretend type doesn't exist entirely.

I feel like using Go's interfaces would be more powerful for a duck-typing language than having type-annotations.

If you don't know whether a method returns an iterator or a mutable/immutable sequence (regardless of the specific type) then the documentation is incomplete (it is a bug). If it is in stdlib; you could submit a documentation patch for a *.rst file on https://bugs.python.org/ See https://docs.python.org/devguide/docquality.html

"The problem is that I don't know what methods the returned objects have available, and as such whether they are a duck or a tiger."

I don't get this... Why not do the following:

    In [1]: import requests
    In [2]: resp = requests.get('http://www.google.com')
    In [3]: resp.
    resp.apparent_encoding     resp.iter_lines            
    resp.close                 resp.json                  
    resp.connection            resp.links                 
    resp.content               resp.ok                    
    resp.cookies               resp.raise_for_status      
    resp.elapsed               resp.raw                   
    resp.encoding              resp.reason                
    resp.headers               resp.request               
    resp.history               resp.status_code           
    resp.is_permanent_redirect resp.text                  
    resp.is_redirect           resp.url                   
    resp.iter_content

Now you can try each of those things and explore their methods.

Because I'm not always scripting - sometimes the work I'm doing takes 10 minutes of runtime to get to the end and I want to know what I'm writing before I run it.

If Python wants to remain a scripting language forever then the docs are sufficient. But if it wants to graduate, and it seems a lot of people think it already has, then the libraries need to be better documented.

    >>> import requests
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ImportError: No module named 'requests'

Now, what was your question?

Python also uses autogeneration for most of the documentation, but the documentation itself is not fully autogenerated. You write your documentation and you say to Sphinx: put the generated documentation for this module here.

IMO, this is something positive. You can decide the structure of your documentation and include only the most relevant references.

For example, this part of the documentation for the re module is autogenerated from code: https://docs.python.org/2/library/re.html#module-contents

Agreed, but my point is that argument and return types are badly documented, and in Python autogen doesn't help this.

Usually there are examples for everything. Sometimes you have to scroll down, though. What the function returns is usually stated in the description. But I know what you are saying, actually most 3-party libs do document types.

IMHO, python docs are ok. If you struggle with them as a beginner I don't see what would change as an advanced programmer.

I am an advanced programmer.

I shouldn't have to read code snippets to figure out if a function returns a tuple or a 'group' which can be iterated over, but can't be indexed. This is something which would be fixed if the documentation included types, because then I could check what all the allowable operations on 'group' are directly without searching for a code snippet which does what I want or just running it and praying I got it right.

The author has a vested interest in making people believe they can't figure it out themselves. The article is an advertisement for their book/videos.

The author also has experience teaching beginners (presumably), giving them a perspective that experienced programmers very easily forget and/or ignore.

On the contrary, I learned python starting with the docs, and I am very picky, if something is difficult to understand I am the first to complain, but I did find python docs to be okay, considering I started learning the language from the docs itself.

They might not be the best, but they aren't what the author says like 'we just can't understand shit'

I learned python purely from an offline copy of the official docs, back in 2001 or so. It was obviously doable, since I did learn Python, but it wasn't particularly easy and I remember starting to really like Microsofts documentation (eg their VB docs).

Python's docs have improved, but they're not amazing. Then again, neither are the docs for many other languages. For example, I'm a big fan of Clojure, but many of the docs there have me really appreciating Python's docs ;-)

I think once you get used to Pythons docs, they're pretty easy to follow, although I agree that type information on arguments/return values would be great and I also largely work from examples unless I'm already familiar with a library and am just looking for reference material.

That's what my point it, the author, in my opinion is wrong about the docs, since even I learned the language entirely from an offline html copy of the docs.

Yeah, I wasn't disagreeing with you. Just adding my own anecdote.

Yes, I know that :-) Sorry for being unclear in my comments!!

> he kind of technical documentation I love to read: precise

Nothing precise about it. Method signatures don't even include a clear description of what they can f return! I have to go and fish this infos from the text blob below. Even JavaScript gives you a damn return values section for every method: https://developer.mozilla.org/en/docs/Web/JavaScript/Referen... . I can't expect a type signature like in OCaml or Haskell (yeah, H sometimes gives you only a type signature which is even worse...), but still.

The cool thing about Python is not the high quality documentation. It's that you can still be productive in it despite it being a dynamic language with such a horrible and fuzzy documentation :) Guido really did a fine language design job.

The "text blob" always starts explaining precisely what kind of return value to expect under which circumstances. The return value's type can change depending on the context, and using plain text is somewhat easier to write and read than trying to come up with a dedicated notation.

See findall[0], for example. The structure returned depends on whether the regexp has capturing groups or not. A specialized notation could be designed to represent that, but as far as I am concerned, the textual string is good enough.

Just for fun, the return type could be expressed as:

      [string N]                 (A)
    | [(string M) N] : (M > 1)   (B)

    - Case A if the regexp matches N times without overlapping and without capturing any group, or capturing a single group.
    - Case B if the regexp matches N times without overlapping and, for each match, captures a tuple of M groups, as strings.

... where [type size] denotes a list of size "size" satisfying the type description "type". Likewise, (type size) is a tuple of "size" elements, all of which satisfying the same type. Or you could write t1 * t2 like in other languages to define cross-products.

[0] https://docs.python.org/2/library/re.html#re.findall

Your thinking is correct and precise, but I hope you never take a job of documentation writer for any project I will ever have to work with :)

My example of documenting re.findall all would be smth like this:

    Signature:
      re.findall(pattern, string, flags=0) -> [str] or [(str,)]

    Arguments: ...

    Returns:
    - [str] -- List of Strings matching the pattern if the pattern has no capturing groups, or only one capturing group
    - [(str,)] -- List of Tuples of Strings, one string for each captured group, if the pattern has more than one capturing group

    Examples:
      re.findall(r"h(\w{0,10}o)", "hello hoolo water")
      # -> ['ello', 'oolo']

      re.findall(r"h(\w{0,10}o)", "hello hoolo water")
      # -> ['ello', 'oolo']

      re.findall(r"(h)(\w{0,10}o)", "hello hoolo water")
      # -> [('h', 'ello'), ('h', 'oolo')]

    Details: ...put here some deatailed explanations about behavior with overlapping etc....

This is the kind of documentation that really spares me a lot of time. Your explanation may be correct but I'd have to read it 5 times and then open a REPL to do a small experiment and confirm that it means what I thought it means...

(I now I don't have the right to bitch about this though, the correct thing to do would be to donate some of my time and actually improve the existing docs instead of commenting on hn...)

> Your thinking is correct and precise, but I hope you never take a job of documentation writer for any project I will ever have to work with :)

I agree, that was terrible.

> My example of documenting re.findall all would be smth like this

The hard part is to have everybody agree on the documentation mini-languge.

https://www.python.org/dev/peps/pep-3107/

https://www.python.org/dev/peps/pep-0484/

Googling about Python regular expressions still returns the “Regular Expression HOWTO” near the top of the results. https://docs.python.org/3/howto/regex.html

That’s pretty much the best succinct introductory page about regular expressions (applying to whatever language) online, and has been for 15+ years. After finishing that page, a beginning python programmer should be ready for the library reference and/or a more serious book.

(No idea why the author here thinks that examples from 2008 are such a terrible thing. Nothing has changed about Python regular expressions in ages.)

Overall, the Python docs are amazing. Not perfect by a long shot, but miles ahead of PHP or C++. Pretty much the best programming language documentation I’ve ever used (but I admit there are many programming languages I’ve never tried). Perhaps that’s partly down to the language itself making a lot more sense than many alternatives. The docs don’t have to do as much work to paper over brokenness.

His take on the community also bears no resemblance to my experience. People on python mailing lists, IRC, local meetups, Pycon, etc. have unfailingly been friendly and welcoming and willing to bend over backwards to help newbies.

I remember first learning about NLTK, not knowing the first thing about Python, or anything at all about natural language processing.

And I could just "get it". My lack of Python knowledge did not get in the way because it was like reading pseudocode. The language is so beautifully designed for the majority of the cases where you do any kind of algorithmic/math work.

In fact, it may have been too beautifully designed. :-) From what I can see from comment threads [1], most people seem to want to use NLTK even if there are a lot of complaints about how it is too slow for production, and how there are much better alternatives for NLP when it comes to performance. I think people are attracted to it because of its simplicity.

[1] https://news.ycombinator.com/item?id=7658977

I am a Ruby guy, but we had a project where we built a forking socket server in Python, I was really impressed with the documentation.

https://docs.python.org/2/library/socketserver.html

https://docs.python.org/3/library/socketserver.html

Reply on news.ycombinator.com