Semantic Segmentation of Websites

github.com

23 pointskroggen4 comments

This is not meant to be dismissive, but is from a place of curiosity: What is the purpose of this?

A few ideas come to mind. Most of them would use this as a pre-processing step to guide something valuable but a bit tricky to automate:

1. Detect meaningful redesigns in competitor sites

2. Power a more robust dynamic scraping solution

3. Summarize common layouts (in your industry, outside, ...)

4. Monitor sites known to A/B test to see which specific changes work for them, and aggregate those results across many inputs

5. Add it to your test pipeline, checking that modern browsers render your site roughly the same semantically (maybe more robust but still valuable compared with screenshot tests and whatnot)

...

In general, anything that would benefit from automating a task made easier by knowing the visual structure of a page should be a great candidate.

a startup from our university [1] successfully builds on something like this for various automation tasks.

[1] https://www.askui.com/

Neat, could be useful for turning mockups into html. I wonder if you could also generate some reasonable ids/classes, with the classes matching on related objects (in a list, etc).

Reply on news.ycombinator.com