Using XPath in 2023 | Hacker News Reader

Using XPath in 2023

denizaksimsek.com

My personal homepage in the 2000's was a simple XML document, automatically translated with XSLT. IE supported that using <?xml-stylesheet?> pre-processor tag. When you tried to view its source, you'd only see a weird XML markup. Surprised a few I'm sure as this predates Firebug and similar DOM inspectors :)

I actually liked XML + XSLT + XPath combo a lot, and kept using them with my projects at Microsoft.

I did the same, and then allowed logged in users to define their own stylesheet. Thought it was pretty cool at the time.

This is still supported in the like of Chrome/Safari. It just sucks as they never upgraded past XSL/T 1.0.

I wouldn't bet on this support to continue forever. There were calls to drop this support.

Thankfully it should be possible to emulate the whole thing with JS snippet, so old websites probably would work with minimal modifications.

One of the first places I worked ended up creating quite a nice framework where components would ask for data that was all put in a single SHAPE query, and the view layer was just XSLT (which is a perfectly nice functional language once you understand it). Was pretty productive.

I tried to go that path in 2010-ish but discovered browser support was not great, so I dropped it.

I really liked it.

I still use, to this day, XSLT on my website:

1/ To transform its Atom feed to pretty-looking HTML, if you access it from a browser: https://paul.fragara.com/feed.xml

2/ Similarly, to embed the feed dynamically on the homepage, using JavaScript XSLTProcessor (https://developer.mozilla.org/en-US/docs/Web/API/XSLTProcess...)

This works on all major browsers (and even on IE!) The XSL sheet is here: https://gitlab.com/PaulCapron/paul.fragara.com/-/blob/master...

In another context, I’ve leveraged XSLT 2.0, this time in a build process, to slightly transform (X)HTML pages before publishing (canonicalize URLs, embed JS & CSS directly in the page, etc.): https://github.com/PaulCapron/pwa2uwp/blob/master/postprod.x...

I wrote an XSLT stylesheet that would turn any page into a pretty-printed source view of the HTML just by including it via the <?xml-stylesheet?> tag. That was during the years I worked at a publishing company that stored, generated and edited everything via a combination of XML, XSLT, XPath, XInclude and XQuery, and now I'm working somewhere else that generates documents I do miss the flexibility of that combination... unfortunately it's hard to sell anything XML-related to people who've never seen it used effectively.

A major pain point of using XPath in isolation (not embedded in XSLT or something else) is those damn namespace bindings. Ugly enough as xmlns: pseudo attributes, in the absence of XML you'd have to use freaking XPointer bindings (as in "xmlns(bla=uri)//bla:xpathexpr/bla:following[@here]"), interpret HTML as XHTML with implicit namespaces, and similar tricks I thought HTML5 left behind for good. Bit surprised htmx falls into the XML nostalgia trap.

Namespaces always seemed to be the ugliest part of the XML ecosystem - I know why they were required, but they always seemed to be horrible to work with.

I just can't fathom why they chose to use URLs--with a protocol--to define namespaces? By all means use the DNS, but bar.foo.example.com would have been such a better choice than http://example.com/foo/bar!

Even the original designers of XML (those responsible for subsetting XML from SGML, and those responsible for dropping the ball by adding XML namespaces on top ;) aren't happy with it [1].

As to why URIs, I completely agree. I think URIs/URLs were introduced by TBL as arguably the major feat of HTML over generic SGML along with <a> and so phenomenally successful every W3C mechanism of name resolution and namespacing had to make use if it; RDF is littered with namespaces as well.

[1]: https://blog.jclark.com/2010/01/xml-namespaces.html

That's quite fascinating - shows how you can have a design process where every step is completely reasonable but the end-result can still be horrible!

As you say, its piggybacking on the dns's uniqueness guarantee. Urls do at least provide and obvious way to have multiple namespaces per domain. But I agree that it's confusing.

I once had the misfortune to have to use a third-party xml-based webservice whos developers thought that the xml namespace had to match the url used to access it. So their production and test systems had completely different namespaces. They maintained multiple xsd schemas for each system that only differed by namespace.

When I finally managed to explain namespaces to their "chief web architect" he got all embarrassed.

This was back in .net 4.x days, and I was generating serialisation classes from their schemas - so I had to create two different dlls and load the appropriate one at runtime. Very aggravating.

good grief

The W3C has all their specs, as far as I know, use Uniform Resource Identifiers (URIs), and most use the subset of Uniform Resource Locators (URLs). There seems to be a strong sentiment against using other types of identifiers, based on my experience on the Extensible Resource Identifier (XRI) technical committee in OASIS. So the chance that they'd create a new one for XPath was probably very unlikely, in my opinion. Even non-URL URIs might have been better for XPATH namespaces.

As an aside, I haven't used XPath in a few years, but use JSONPath / JSONPointer on multiple days each week, and highly recommend them. I find XML to be great for many things, but I find JSON much easier to work with.

The idea is that the URL can do something, like link to specification or something. However, there is nothing mandatory the URL has to do whatsoever.

I think in practice you can treat it as an opaque blob and if you want to just declare a namespace as "bar.foo.example.com" I think it would work just about everywhere. There's enough legacy, but still in use, namespaces that aren't full legal URIs that libraries will never be able to start enforcing it, and since there's effectively no utility in forcing them to be URIs anyhow I doubt it'll suddenly become an issue when all the libraries decide to become compliant. Given that it's a bit bizarre to have a URI to something that doesn't have to even so much as exist, I can't see the standards bodies every getting very fussy about this either. Backing it down to just being "some agreed-upon blob value" would be backwards compatible anyhow.

"However, there is nothing mandatory the URL has to do whatsoever"

To be honest I think that just added to the confusion - the idea of using a URI that was just a unique name and not something that actually referred to anything wasn't the easiest thing to appreciate.

The worst thing is not the existence of namespaces, but that are are relatively often implemented incorrectly.

As an example of things that sometimes break:

* namespace redefinition: you have xmlns:ns1 on a parent node and xmlns:ns1 with different value on a child node.

* same namespace on sibling nodes (happens often with streaming): multiple child nodes have a xmls:ns1 with some value. That is only valid for that node.

* some processors expect namespaces only on a specific node (I think it was MS Navision ~ decade ago that failed when declaration wasn't on a specific node, but on a child node)

Indeed. There is an XML formatted file that is important to the engineers at my company. Sometimes they're namespaced; sometimes not, so just I remove the namespace if it's there. It wasn't like the people who defined the namespace gave us any usable format definition to work with anyway. The only thing worse than XML is an XML definition that's only half-implemented.

    raw = Nokogiri::XML(File.read(uploaded_file).gsub(/ xmlns="[^"]\*"/, "")) { |config| config.noblanks }

I am yet to see a system that uses namespaces 100% correctly, especially larger systems. XMPP is closest but they still have the stream tag defined as carrying a namespace, but as being the "same" stream tag without specifying it to be so, last I recall.

The annoying thing is namespaces are not that complicated. They're probably five times more complicated than they need to be because you're always dealing with some library or system's half-assed bizarre implementation of them instead of the rather simple spec.

My biggest issue with XML namespaces is how they work with attribute names. I worked with XML enough but I have no idea how it really works. Every time I had to Google it or trial it.

My opinion is that XML would be much better without attributes. A lot of things would be simpler. Attributes are nice, but they add whole another dimension of problems everywhere.

In an XML document using namespaces, all tags and attributes are named with the tuple (namespace, name).

Tags are always in the empty string namespace unless an explicit namespace prefix is given, in which case the tag's namespace in the one corresponding to the prefix.

That's it. That's all there is to namespaces and attributes. The idea is that the namespace would be redundant to the namespace on the tag, because the attribute's meaning is already 100% dependent on what tag it is on anyhow.

The big key to working with namespace documents in general is you have to understand that the names are now tuples (namespace, name), and that the string prefix you may see like "ns1" is an abbreviation, not the namespace. The namespace is the thing declared as the attribute of the xmlns tag. Get that right and most of the rest follows. What almost everything gets wrong is failing to treat names as (namespace, name) correctly, and trying to treat names as a single string like in non-namespaced document, and then writing in whatever misconceptions their particular implementation wrote in to get around the fundamental impossibility of handling a 2-tuple with an internal datastructure that is only a 1-tuple.

> Tags are always in the empty string namespace unless an explicit namespace prefix is given, in which case the tag's namespace in the one corresponding to the prefix.¶ That's it. That's all there is to namespaces and attributes.

This is at best misleading (or worse: misled—and wrong). The semantics of the following snippet is completely different:

    <foo:bar baz="xyzzy"/>

... from this:

    <foo:bar foo:baz="xyzzy"/>

(Assume the "foo" prefix is defined and mapped to a namespace.)

This is before even getting into the fact that unprefixed elements will belong to the default namespace (despite being written with no "explicit namespace prefix").

> The namespace is the thing declared as the attribute of the xmlns tag.

The "xmlns tag"? No.

> I know why they were required

Have you ever seen tag name clashing in a XML document?

I can vaguely remember working on systems in the finance industry that combined vendor XML, various internal XML schemas and transformed these into XSL-FO documents for rendering through multiple passes through XSLT.

If I had to guess the chances of there being some element names clashing without namespaces would have been fairly high.

> Bit surprised htmx falls into the XML nostalgia trap.

As the article states, the only tag that can't be searched for via CSS selectors is the new "hx-on".

And my initial thought about this is (which I'm open to reconsider), it seems like you probably shouldn't go looking for elements based on their event model. Show me every <div> with a certain value in its class attribute? Yes of course. But find me every <div> with a certain behavior on mouseUp? This isn't something you'd do in an OO language, you'd look for objects with a certain property value, not a specific event handler, which would likely be private anyway.

I'd also note that, for the same reason, standard HTML as well as React, etc have the same inability as HTMX here. The article notes that with HTMX you can't use CSS selectors to find "hx-on(something)" tags, the same handicap prevents you from searching for any element with an "on(something)" attribute, i.e. onClick, onMouseUp, onKeyPress etc.

Maybe I haven't thought through this entirely but I don't see any problem here.

(edit: to be clear, of course you can CSS-select a specific tag like onClick, and you can also select a specific HTMX tag like hx-on:click. What the article notes that you cannot do is search via wildcards or "starts-with" in CSS, i.e. hx-on:*, but you can't do on* either)

A common source of frustration is that XPath itself doesn't provide a means of binding a namespace prefix with a namespace. It relies upon mechanisms provided by the hosting language or library. After helping scores of devs with such problems on Stack Overflow, I took time to write a canonical answer addressing this issue:

How does XPath deal with XML namespaces?

https://stackoverflow.com/a/40796315/290085

All those devs who skipped xpath/xquery/xslt from the bad old days of xml are going to get a panic attack.

IME their opinions are split between trauma and nostalgia.

Xslt and xml apis could do neat things. I didn't mind the era

Starting to see more use of JSON Schemas in my job and can't help but think ... "we had something for this previously".

jq is the XSLT/XPath of JSON.

Xquery, for me, was and remains a core tool for dealing with XML specifications of surreal complexity that verge on madness. BaseX is the "Microsoft Access" xquery application, while eXist is sort of like a full framework, with package management, deployment, and that sort of thing. Other query languages might be more cutting edge, but they either 1) have a lot of stuff I don't need, or, more likely, 2) require a more permissive InfoSec setup than I am typically allowed. "Docker and any other form of virtualization are not permitted on ANY company network regardless of circumstances". Well, ok then.

Generally the next stop after xquery, for me, is text mining, either on R+Python or on Orange ML. If a miner doesn't cut it, then LLM shenanigans.

Also, xpath? It's pretty great. XQuery? Does the job. XSLT? Ok, so NOW that's the feeling of a panic attack. I've been doing XSLT for literal decades, and I still don't know what I'm doing when wrenching on a giant pile of FOP generating funhouse madness. When I am tagged into a data transformation job, I always stress that xquery is the right tool, rather than a confounding nested directory of XSLT using different parsers and different passes like a figure-8 interstate off-ramp. For FOP, though, there's really just one game in town for that. Although, having said that, me and a bunch of others are doing our damndest to show that what you're trying to do with XSLT/FO can be done way way way easier with CSS and Paged Media (either via Paged.js or Vivliostyle or any of the other zillion PMM implementations). The downside is you have to wrench some CSS yourself, but honestly, that's probably going to be easier than wrenching on DocBook-XSL or the DITA-OT or one of the MIL-STD XSL packages.

The problem with XSLT is how incredibly verbose it is, but maybe that's just the problem with XML. jq is to JSON as XSLT/XPath is to XML, which shows you can have pithiness.

> maybe that's just the problem with XML

It’s the problem from having built XSLT out of XML, that was completely unnecessary.

And then with XSLT being so verbose, having cheaped out and made the current node (“.”) so implicit. And the confusion from the dual use of templates as both match/patch and function constructs (it would have worked fine as a shortcut, but it makes grokking how things fit much harder than necessary).

I wouldn’t say jq is really comparable to xslt though, xslt has much more transformative flexibility. It’s closer to xquery.

> I wouldn’t say jq is really comparable to xslt though, xslt has much more transformative flexibility. It’s closer to xquery.

jq isn't structured around data schema transformations, it's true, so writing complex transforms is not as easy as with XSLT, but it is very much possible. Essentially it's a reduction over [possibly paths to] elements of interest in `.` updating the reduction state with the new schema. You can organize your code into functions that do much what XSLs do.

Glad to hear that someone else thinks of XSLT the way I do. I had to write some to deal with converting DocBook XML to LaTeX (building on dblatex, but adding some specializations), and besides being verbose (as another commenter here says), I found it virtually impossible to debug. I'd much rather write in Prolog.

"XSLT? Ok, so NOW that's the feeling of a panic attack."

Right, so here's the secret decoder ring of XSLT: Underneath all the complexity, it isn't doing ANYTHING you can't do in your language of choice armed with an XPath library. And it is often incapable of doing even some rather simple things you can do in your language of choice armed with an XPath library.

XSLT is just a terrible programming language. That's all it is. All of the magic is in the XPath part; once you've selected the nodes you're working with, XSLT is a horrifyingly awful way of manipulating them into doing what you want them to do.

XSLT is the intersection of the worst parts of declarative programming with the worst parts of functional programming, wrapped up in one of the worst ways of serializing a programming language. What confuses some people even to this day is that they see "declarative programming" and "functional" and even "standardized serialization" and accidentally credit XSLT with the benefits of such approaches, and then if XSLT doesn't work they blame themselves for failing declarative functional programming in such a wonderful serialization format. They're wrong. It's XSLT failing them, whose origin is also people who thought if they just create something declarative and functional and serialized through XML they were guaranteed to be producing something good because those things are just so Platonically good on their own that they couldn't possibly produce something useless and broken, so it was not necessary to analyze the resulting abomination to see whether it actually fulfilled the goals, because it simply by definition fulfilled the goals by virtue of being declarative and functional and in the bestest serialization ever.

Perhaps there is a declarative, functional XSLT-inspired language that could be written that would be as good as the people bedazzled by the buzzwords think XSLT is. (Though there's no world where such a language is helped by serializing into XML; serializing a language for manipulating XML into XML is actually the worst choice possible because of the nested encoding you inevitably produce!) However, in the meantime, you don't really need to wait around for someone to produce it because it turns your favorite general purpose language equipped with an XPath library is already 90%+ of the way there.

While I share your disdain for XSLT, the way I look at the XSLT in a project I maintain is that it’s a straightforward (albeit labor intensive, whether manual or automated) refactor away from being directly ported to a more maintainable FP implementation… whenever I eventually have the bandwidth and buy in to do it.

It’s not so much that XSLT is a terrible langauge, it’s a terrible syntax. At the end of the day, it’s just a really awkward way to express some pretty basic functions. With a bit of care, it’s easy to imagine transforming arbitrary XSLT to any arbitrary language expressing the same semantics. This is especially true because XSLT itself is so limited in capability.

That's sort of been my secret suspicion. Seriously, when a customer asks me to make some XSLT to turn a pile of XML into a bunch of spreadsheets/tables/whatevers, I try to sit them down and list out the many ways that are - from the standpoint of efficiency . . AND functionality . . AND "retain your sanity"-ness - superior, vastly, sometimes exponentially so. Xquery is one of those, but if the output is more complex than a read of delimited data, then your rec "Functional Language X plus Xpath" is the Way to Go.

Please don't tell anyone I said this, but the hardcore XML people can be a little culty. At untold meetings, I'd walk through all the functionality that they're getting - from a staggeringly gigantic XML system, priced in millions of dollars - and demonstrate, one bullet point after another, how the same functionality is available from a few lines of Asciidoc/ReStructuredText/<insert_your_tech_here>. Then, at the end of everything, I always have to contend with the last one: "But it's not XML". To which I shrug my shoulders: I can't solve existential problems. Why is this specific technology so important? My theory is that there's some very deep and possibly poisoned incentives at work here, which I don't really want to unpack because it would make me sound nasty, but the most innocent of these is the Sisyphean drive to impose labels on the chaos of business and on natural language. I think it might be a neurosis of the industry, but I've seen it outside of aero-def-landia too. Who knows? Maybe someday I too will rue the day I ever turned from the True Faith.

I had that skip until two years ago when I had to use it to parse the OpenVAS API output. I wish I never had to put a stop to that skip, I hated every second of my life working with XPath.

XML had a bad rap and was certainly abused. But the wealth and quality of the tools for working with it is really unmatched even today.

It's also the only widespread format that can deal with both rich text documents (à la HTML) and complex, structured data (what JSON is good for). It's golden when you need to add tons of complex annotations to a text document.

<rant>

JSON is a pain in the ass for organizing and parsing structured data, particularly arrays of things.

It's so much more verbose and disassociates an object's definition from it's type. Let's say you have parents with children with names.

   <parents>
     <parent name="Bob">
       <child name="Alice" age="12" />
     </parent>
   </parents>

The root object is 'parents', so you know a bunch of 'parent' elements are going to follow. When you read the <parent> object, you also know it's a parent because the tag say so. You don't even need a <children> element because it's fine to have a list of <child> elements directly after <parent>.

Now here's what I think a typical JSON equivalent would be:

  {
     "parents": [
        {
          "name": "Bob",
          "children": [
            {
              "name": "Alice",
              "age": 12
            }
          ]
        }
     ]
  }

Ok, so whitespace aside, it's less verbose, but look at all the info that's missing.

What "type" is the root object? You'd say "parents", but how did you find that out? You have to know _a priori_ that a field called 'parents' would have to be there. Not a big deal on the root object because it's usually special, but how about a single parent?

Look at the 'parents' array. The only thing hinting at the fact that { "name": "Bob" } is a parent is the fact that it is part of an array, that's attached to the 'parents' field of the parent object. You have to do 'upwards' lookups to determine what this object is. The object itself doesn't have that information. Same thing with { "name": "Alice" }. How do you know that's a child object? You don't. You have to do an upwards lookup.

Now you might say "just tag the elements with their type so you can keep track of what these objects are". Let's try that:

  {
    "type": "parents",
    "parents": [
      {
        "type": "parent",
        "name": "Bob",
        "children": [
          {
             "type": "child",
             "name": "Alice",
             "age": 12
          }
        ]
      }
    ] 
  }

Sweet, now we're reaching data representation parity, but if you get an object like this from a third party service, how do you validate it without kicking in the logic to process each element? You'd have to have a 'dry run' version of your logic.

In fact, how do you formally describe the structure of these objects to another service so that the service could guarantee that it is only generating valid objects in the first place? XML Schema was a solid solution for that. JSON Schema had no support anywhere last time I looked at it. Where is it now? It looks like you could use it if you wanted to, but afaict most people generate fly-by-the-seat-of-your-pants JSON objects in code and no 'formal' validation is happening, other than the reply code from the service when the object is actually sent (if you think about it, that's just "testing in prod")

I think version 1 of XSLT, XPath and so on where pretty simple solutions to working with structured data, but people went overboard with trying to shoehorn XML into solving problems best suited for imperative code, so you got XSLT 2.0 (want for loops? no.), XQuery and XPath 2.0 abominations, various weird xlink solutions, imperative code in tags: <script>function foo() { }</script> which introduced a second syntax, and so on.

I understand why the XML world of nonsense had to be stopped, but we threw the baby with the bath water.

Don't get me wrong, I like JSON, but I also feel like we collectively took a step back and opted for 'the javascript of structured data representation'.

Maybe the rise of thick, JavaScript heavy clients had a lot to do with it? XML was never easy to work with in JavaScript, which is a shame, seeing as it has the same roots as HTML. I blame the DOM API - it has always been tedious to use in any language that implemented it.

ActionScript had that nice built-in @ syntax for selecting nodes and first-class support for XML in the language (E4X?). How we killed that first-class language support only to turn around and rediscover it in half-baked form as transpiled JSX is beyond me.

</rant>

XML was a very good idea badly implemented to an absurd level.

A XML-lite that ditches all the noise and stupidity, but keeps the annotated text with user-defined sum-typed tags, and a type specification would be better than any standard we have today.

"XML-lite"

XML's creation (or definition) in the early 1990s was motivated mainly by a desire for an SGML-lite.

And it was a great goal, as it made SGML borderline usable.

Domenic Denicola (aka the man who ruined promises) probably will as well.

https://github.com/whatwg/dom/issues/67

That made me chuckle.

For those not familiar with the promise design controversy:

http://brianmckenna.org/blog/category_theory_promisesaplus

https://github.com/promises-aplus/constructor-spec/issues/24

https://github.com/promises-aplus/promises-spec/issues/94

I guess sometimes Worse is not Better?

How did he ruin it?

EDIT: Thanks, sibling!

I use xpath literally every time I need to look for something on a webpage before resorting to scraping.

I am the opposite. The only time I use xpath is for scraping.

What / how do you use it? I've only ever used xpath for scraping, and I don't know how it can be useful outside of scraping.

End to end testing comes to mind, for UI testing.

I think you can use it in the browser's inspector.

Yes, you can use `$x("//a/path")`, with (at least) Chrome & Firefox.

See https://devhints.io/xpath & https://devtoolstips.org/tips/en/evaluate-xpath/, and in this thread: https://news.ycombinator.com/item?id=36755041

And you can do the same for CSS selectors with $$('foo')!

These are still really useful in Jest/Playwright tests when you don't have an easy CSS selector or ID/class to choose from. XPath is super powerful, especially for dynamic pages/apps where the DOM isn't necessarily predictable but the relative positions of items (like a card in a list) are.

XPath is also good if you think of it in adversarial terms (i.e. QA). I don't care what you div is - I care what text is on the page or whether something is saying what I expect under a title. That's where I got familiar with it (writing automation frameworks for selenium/appium for ff/chrome/ios/android) and it did a great job at it - regardless of drivers' performance. When your eyes parse a webpage, you don't really care whether you're at a div or a paragraph or basically anything else - it's the software's job (the dev's job) to think of it for you. Is it nested inside something with this accessibility marker? What do I start to read first? How many levels deep do I have to look? And so on and so forth.

I'm looking forward to those new things in htmx because it seems like it extends what we know of html. And I can't yet quite say if that is going to stick (by being integrated into IDEs, work with frameworks easily, be the go-to way to do this and that, etc). Time wil tell, but it's cool to see new proposals to improve the status quo out there.

Using xpath in a test on a dynamic and unpredictable DOM is a painful road to flaky, brittle tests. If you're writing tests for a system you have no control over maybe you have no choice, but if you do then I'd recommend changing the output to be something robust and testable.

>Using xpath in a test on a dynamic and unpredictable DOM is a painful road to flaky, brittle tests.

sure, but more powerful and thus more able to handle situation than depending on css selectors.

Also consider Robula+ https://tsigalko18.github.io/assets/pdf/2016-Leotta-JSEP.pdf

https://tsigalko18.github.io/assets/pdf/2014-Leotta-ISSREW.p...

css selectors

I wasn't recommending those either. roles or testids are much more robust than xpath and selectors.

Robula+

I had a very brief glance over the paper and while it looks a lot better than plain xpath it still has some of the problems that I think make tests flaky. One of the suggested examples is "[contains(text(),'123456789')]" where '123456789' is an input field with that value. If the example text is changed the path will break. It's testing the content of the field, not what the field is* or what the field should do. An alternative would be a selector like 'getByRole('button', {name: 'Telephone'})' (getByRole comes from Jest, but more testing libraries have something like it) means you can change practically everything about the input field (xcept the aria-label or aria-labelledBy value and the test won't break. And, as a bonus, your app will end up more accessible.

I love XPath, I'd say that alongside of regular expressions, SQL and Excel functions, they helped in my IT career a ton.

You can simply use this in the browser console to get a friendly XPath API:

    $x("//my/selector")

Very handy for testing XPath expressions for Selenium or Playwright tests, if there is no data-testid attribute.

Interesting! What are the limits of `evaluate()` in browsers? I see it is available in 95% of users' browsers [1], but is it consistent in its implementation of XPath? How does its performance compare to `querySelector[All]`? Might be interesting to see if JS libraries that do a lot of DOM searching could get some perf gains. Maybe they already utilize evaluate?

[1] https://caniuse.com/?search=evaluate

CSS is designed to be very fast, and because of that design choice it's less expressive. XPath is not going to beat CSS in speed, but it allows you to move through the document in any direction (up and down in the DOM tree of elements, forward/back among siblings in the DOM tree, and forward and backward in document appearance order).

We use XPath at work from CSS with custom plugins, and use XPath in JavaScript for targeting elements that otherwise wouldn't be straightforward to select with CSS.

Another cool thing XPath does is have awareness of the text content of elements! //li[contains(.,"example")] would target all <li> elements with text content containing "example".

FWIW CSS now has “:has” which provides a form of general purpose predication, although afaik it still doesn’t have a :contains (that was proposed for CSS3 or something but I don’t think it got accepted).

A huge part of XPath’s power though, and something you AFAIK can’t do in browsers, is extensibility.

For instance selecting an element on the basis of a class is absolute hell in XPath 1.0 (and not great in 2.0 either, XPath 3.1’s `contains-token` finally made that not hell). But server-side you don’t care because pretty much all implementations allow installing your own functions so you can add your own `contains-token` or even `has-class` predicate and be on your way.

I used XML/XPath/XQuery in the early 2000s to define a custom binary protocol. It was so fantastic to have a single source of truth, and use transformations to generate both document (spec) and C serialization/deserialization code.

I have never experienced that again. :(

Sadly, like any popular technology, it experienced its fair share of use and abuse (people wanted XML database, XLST, XML in Scala, etc.). It's a much-maligned tech, and with a few tweaks to the robustness, it could be pretty nice (see https://annevankesteren.nl/2007/10/xml5).

I'd take it over YAML any day. And I speak from a YAML parser implementer perspective. Making a 1.2/1.3 YAML parser is hard. The number of edge cases and states is absurd, while the specification needs to be on HTML/XML level.

E.g. When I started working on YAML parser I didn't expect to be fixing errors in spec. But here I am.

I did the same thing but used Java as the initial code generation target back around 2010 (for a personal project). Since the network protocol wasn't controlled by me, this approach made keeping up with changes trivial.

XPath is very impressive and expressive when one just looks outside the xml/html world. An abstract XPath query selector only needs a tree enumerator with nodes reporting the name and (optionally) some attributes. One of such application is querying against Abstract Syntax Trees [1] of programming languages. The name here is the name of the rule of then node. A further step is to have semantic relationships understood. In this case something like //IDENT[@owning_class='CFooBar'] is possible

[1] https://en.wikipedia.org/wiki/Abstract_syntax_tree

Not XPath, but for folks interested in querying (rather than walking) syntax trees for arbitrary nodes, this is also a cool feature of tree-sitter[1]. It uses a scheme-like syntax, and it’s impressively efficient.

And in terms of XPath, for folks using a JS stack, fontoxpath[2] supports a DOM facade adapter interface which allows for querying any arbitrary tree-like structure, so it could certainly handle the same use case.

1: https://tree-sitter.github.io/tree-sitter/using-parsers#patt...

2: https://github.com/FontoXML/fontoxpath

I'm doing this with css to do syntax highlighting & formatting for [my editor](https://coil-editor.netlify.app/). I also wrote some linters like using undefined variable names in like 5 lines of code using query selectors, this was before I learned more about xpath, so I imagine it'll only be more powerful

I'm using Playwright [1] (dotnet version) to test my web app, and I mainly use XPath to identify elements on a page. I found it to be to the best choice in my use case, mainly because it is composable. I can define an XPAth to a table `tableXPath` and then reuse it to identify a cell in the table like eg $"""{tableXPath}//td[text()="my value"]""", and I can further reuse that to identify a sibling cell which might trigger an action.

1: https://playwright.dev

I built a CMS out of an XML document DB called eXist and XSLT / XPath etc back in the day because I was convinced it was the future. It was *%#^ cursed is what it was.

I was looking at eXist and Berkeley DB XML back in the day to store XML docs and query them with XPath. I never liked XSLT, but XML + XPath 1.0 (not 2.0) was pretty convenient for organizing and querying data.

back in the day where every OTA (online travel agent) and airlines use XML for their API, we had to integrate them in an API gateway in order to unify their API schema and booking flow.

we wrote a small package[1] (using pugixml) to transform XML to JSON using a custom Xpath template syntax. Make our job much easier.

[1]: https://github.com/tuananh/camaro

I use XPath on DOM when I want to find the n-th parent from a particular node. This is very easily done with XPath vs. selectors.

lets say you have

    <body>
    <div><!-- you want this one -->
      ok
      <div id="the-child"></div>
    </div>
     <div><!-- you dont want this one -->
      nok
      <div id="the-wrong-child"></div>
    </div>
    </body>

css:

    div:has( > #the-child)

xpath:

    //div[@id="the-child"]/..

isnt that kinda the same? I think I'd even consider the css version to be easier to understand at a glance.

If you've got full control over the Dom it'd be much better to just set an id attribute on the element in question though, as getElementById is still the best performing option

Now do that when you want the 8-th parent in a complex document.

I only did depth=1 because the syntax doesnt change, i still don't see a difference to speak of

css:

    div:has( > \* > \* > \* > \* > \* > \* > \* > #the-child)
    (backslashes added by hn because of formatting conflict)

xpath:

    //div[@id="the-child"]/../../../../../../../..

I've recently had some fruitful interactions with jq. I found its functional nature quite satisfying.

Now, everytime I've to scrape data from XML from the command line, I wish I had a tool to translate XML to json so that I could use jq on it, instead of having to wrangle with xmlstarlet.

https://github.com/kislyuk/yq#xml-support

I wrote about my frustrations with modern day XPaths here - https://dev.to/nikhilverma/xpaths-in-the-modern-age-1gah

Reminds me to check out SXPath [1] at some point.

[1]: https://okmij.org/ftp/Scheme/xml.html#SXPath

There is also a JSON analog which if found quite handy sometimes to extract some values deeply nested in an API response. Its also natively supported by some databases that have JSON datatypes like mysql.

I read the post and also the docs, but still don't understand where or why one would use "hx-on". Can someone clarify?

hx-on is the general js API from htmx, it's essentially a wrapper/shortcut for creating a js event handler. The htmx docs provide some examples

https://htmx.org/attributes/hx-on/

I had found that. It has the "what" but not the "where or why"

Because HTML's on* attributes are not generic, they are hardcoded in the spec, you can't use them to handle arbitrary events. hx-on is an attempt to fill this gap for simple cases, when managing event listeners manually or resorting to _hyperscript or alpine is too much

> it addresses a specific shortcoming of HTML: the on* attributes do not support arbitrary events

> it is much less featureful & useful than, for example, @Alpine_JS, but if you have very basic needs for event handling it can help avoid another dependency

https://nitter.net/htmx_org/status/1679966199667478531#m

> it addresses a specific shortcoming of HTML: the on* attributes do not support arbitrary events

Yes, I saw that. But what does it have to do with XPath?

XPath is implementation detail, used for searching of attributes with a give prefix, and this article is just a public note of how it was done in htmx

Got it. Thanks.

You mean htmx in general?

The author of htmx wrote a rationale (when and when not to use it). https://htmx.org/essays/when-to-use-hypermedia/

Good article, but I saw no mention of hx-on

No thanks

Reply on news.ycombinator.com