ForgePlucker starts using BeautifulSoup | Hacker News Reader

ForgePlucker starts using BeautifulSoup

svn.gna.org

14 pointsjgrahamc7 comments

Context: http://www.jgc.org/blog/2009/11/parsing-html-in-python-with....

Can you please let go of this topic and that person ? I'm pretty sure most people don't want to see a respected HN member sink to his level.Thanks.

Upmodded you for the first part, but the "that person" and "sink to his level" parts remind me of junior high. Can we just stick to code here? Parsing HTML is nasty enough without these catfights.

If I'm reading those two files correctly, he's trying to use BeautifulSoup, but doesn't ever import it from anywhere... I think the snotty kid/poor fool was right, this is poor code.

No, it is imported from htmlscrape.py

http://svn.gna.org/viewcvs/forgeplucker/trunk/forgeplucker/h...

  # This doesn't use Beautiful Soup (yet) because there's a Berlios case
  # BS may not be able to cope with. We'll check sometime.

I wonder if he's having issues with 3.1.0 which frankly never worked for me (as clearly stated by the BS developer) when used on normal web sites.

Have you tried following the instructions on the BeautifulSoup site[1]? The HTML parser it depended on was removed from the Python standard library, and its replacement doesn't work as well at processing bad markup. You should install html5lib[2] and use it for parsing:

    import html5lib
    from html5lib.treebuilders import getTreeBuilder

    parser = html5lib.HTMLParser(tree=getTreeBuilder("beautifulsoup"))
    soup = parser.parse(open("mydocument.html"))

[1] http://www.crummy.com/software/BeautifulSoup/3.1-problems.ht...

[2] http://code.google.com/p/html5lib/

Reply on news.ycombinator.com