Hacker News Reader

Comment by mherrmann | Hacker News Reader

I don't get why people still use XPaths, CSS selectors or HTML IDs to identify elements, even when they are "recorded". Please please please just use my https://github.com/mherrmann/selenium-python-helium instead. It makes so much more sense.

View on HN

Let me have the opportunity to really thank you for this it has been my goto library for scraping, while allowing also selenium syntax, the idea behind it is really great. Also the “elements below …” very nice

Please make the switch to selenium 4, so that it is kept up to date with the rest of the selenium ecosystem e.g. undetectable browsers

I haven't found anything that made me really need to try other things, but the fact that it's just a wrapper and sounds like it handles nested iframes better means I'm trying this next time I need to make a quick selenium script. Thank you

This is the idea behind Testing-Library from what I understand.

https://testing-library.com/

Why wouldn't you use CSS selectors? They are unambiguous, concise and most importantly, a standard. They should be the conventional way to refer to elements.

The CSS structure of a web site is an implementation detail and on a global average, CSS selectors break much more often than user-visible labels.

> and most importantly, a standard.

A standard for what, exactly? Selecting HTML elements? You could say the same about XPath.

Both are implementation details tightly coupled to the code instead of being driven by the “human” elements of the UI, such as the visible text, labels, etc.

The CSS selector isn't the problem, it's how you identify it. Many sites have dynamically named selectors, including class names. Even depending on ordering is fraught. Learning how to create robust selectors is about 1/2 the battle in writing a good Selenium/Puppeteer/Playwright/etc script. (been doing this as a major part of my day job for about 14 years)

Helium seems great! Handling popups sounds really clever, how do you pull that off?

Playwright is another, really good alternative

It is, but the selector issues are the same. (though it does make it much easier to execute scripts in the context of the page when standard selectors don't work)

It is, but the selector issues are the same

I disagree. Playwright's locators are pretty powerful compared to standard css/xpath selectors.

For example it includes layout based selectors.

https://playwright.dev/docs/other-locators#css-matching-elem...

This looks great, thanks

Reply on news.ycombinator.com