Developing for the Semantic Web
smashingmagazine.com
smashingmagazine.com
First, the software technology scales very poorly unless the size of your data model is trivial. The semantic web is a type of graph search problem with all of the hard algorithm and data structure challenges that entails. Most interesting semantic web applications require data models operating at scales where existing graph platforms have pathological performance characteristics. The gap between capability and requirements is several orders of magnitude. You can't throw hardware at it; when I was first hired to do CS research on scaling the semantic web, I was working on literal supercomputers.
Second, the human element of the semantic web doesn't scale. Semantics are contextual and subjective. As the number of people contributing to the data model increases, the consistency and quality of the data model decreases. This becomes an extremely expensive global coordination problem at scale that may be unsolvable. It is broadly recognized that you need a single, global arbiter of semantics in any scalable system just to maintain semantic consistency, which basically means strong AI.
If we have the technology to solve these two problems, I suspect the Semantic Web will no longer be interesting.
The human element defies any solution other than social coordination such as a specs registry or org. At some point the software needs to understand each other and that requires people to work together. A technical solution around schemas can assist the coordination work, but it can't replace it.
EDIT to add: Since we can't remove human coordination, I question the value of arbitrary schema composition when it incurs the huge DX cost of using a URL to represent every attribute.
Interesting ideas, would highly recommend checking it out!
What if the unscalability of the semantic web were a feature rather than a bug?
The current political and economic fallout of the big data web that we all know and love arises precisely due to the ungodly effects of network size on power laws. E.g., the rich get richer and the poor get poorer on a global scale.
Why not a more semantic and a more decentralized web.
Although it's primarily known for allowing apps to store their users' data in a place under their users' control, for better or worse, it's built on many SemWeb technologies.
And even though I'm still somewhat disillusioned about the Semantic Web (like Aaron Schwarz, I used to be enthusiastic about it way back when), but there certainly is a chance it will be interesting again when your personal (potentially private) data can be part of the Semantic Web as well. For example, if they'd have written their reviews in a Solid-based website, linking them to MusicBrainz would be a relatively small step.
- There are several organisations behind Solid that whose raison d'etre is Solid. I'd argue that Mozilla possibly pulled the plug for Persona too early, specifically, without first adding support for it in Firefox, thereby creating massive incentive for website owners to support it. But then, it wasn't their core focus, and there was no return on investment for them, so I can see why they cancelled it.
- Ironically, a wider focus. The incentive for website owners to support Persona would have been that it would be easier for potential users to join - although they already got most of that benefit by allowing sign-in with Facebook, and most people did not actually use Persona. Solid, on the other hand, has several incentives to support it. It can give organisations access to data that the user is actively curating, rather than having to provide it to different organisations individually, with it getting outdated the moment they shared it. Additionally, given how, for many organisations, data is increasingly (post-GDPR and all that) a liability and not their primary competency, outsourcing management of user data to the user themselves makes sense. And finally, there will hopefully be more people with a Solid Pod, especially in specific markets - for example, IIRC the government of Flanders was looking at giving every citizen a Pod.
- It can lean on the marketing benefits of it being TimBL's main focus, which may or may not be stronger than being one of Mozilla's projects.
- Depending on your definition of success, it can be as successful as Mastodon/the Fediverse is. As opposed to Persona, people can and do write apps specifically to support Solid, because it's more than just a login.
Just off the top of my head, and again, personal views. But hopefully that gives some insight.
Let's imagine the web has lots of annotated, machine-readable data. You still need a client to explore that data, right? The browser is not going to know how to make sense of the relationships between your data points. How is the end user going to consume the information, a web app? As the dominance of companies like Facebook, Google and Twitter have shown, the average user doesn't care about de-centralization. They care about a good user experience.
Let's take the recipe scenario presented in the article. What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead.
Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
that's pretty much exactly what they were doing with freebase (https://developers.google.com/freebase/guide/basic_concepts)
Building a tool or service on top of the semantic web is out of reach for most developers because of inconsistencies and complexities, unless they use the tools provided by the giants – and that goes directly against the principles of the semantic web.
The exception here is niche use cases and academia working with huge datasets, where the semantic web seems to be incredibly useful.
http://client.linkeddatafragments.org/
> How is the end user going to consume the information, a web app?
https://github.com/linkeddata/rdflib.js
There's many more RDF libraries out there
> What's to stop Google from ingesting all of this semantic data, and then using their army of software devs to integrate the results into Google's search results? Google has invested in natural language processing that can allow their web crawlers to make semantic sense of written text. I'm sure they would love it if a swath of volunteers did this for them instead.
That's already what Google does with Schema.org
https://www.searchenginejournal.com/technical-seo/schema/
is a result I pulled up on DDG
> Sure, some small, indie shop could use the data too. But Google can integrate with their translation engine, their home speakers and Android. They'll make the experience nice enough that your aunt looking for a scone recipe won't want to leave their ecosystem, even if a more free, de-centralized alternative is available.
Sure but without some way for smaller players to get access to the same data that the giants have, there's no way to compete. There's a reason Google, FB, LI, et al have spent as much energy as they have siloing their data, cutting off interop, and consolidating their services. Technical or product or user experience superiority means nothing when you don't have the data; a polished, quick search engine means nothing if it has no results to search through.
If you are old enough then you will remember a time when SEO wasn't given a name yet, and mostly consisted of spamming the head tag with untrue META tags. The entire idea of the semantic web is basically a hypothetical extrapolation of the assumption "what if META didn't fell victim to lies?"
But that idea quickly evolved away from that web use case full of deliberate deception and evolved into technology for interoperable knowledge representation. The "web" use case quickly turned into the science funding equivalent of spamming META tags: systems (decision-makers) that were still falling for it where considered so far behind that influencing them with that method did not even feel like lying.
Interoperable knowledge representation isn't valuable because you might want to plug together random knowledge bases (you wouldn't), but because it would be an enabling technology to build one knowledge base on top of another (or on top of multiple others). This would be particularly valuable when modeling is done in short-lived iterative steps with changing teams, organisations and goals, like the quick iterations of student generations that each push ahead their niche of science with some miniscule contribution to progress. With sem-web tech, one generation of knowledge modelers would be able to do their work on top of the work of those who came before them, without being restricted to the technology of those earlier generations. Think of the "web" in "semantic web" as a big spelling error of "stack".
(source: did my diploma during peak semantic web, shoveling OWL from a Java DOM into a Lisp server to brute force maximal consistent subsets of an inconsistent superset in fantastically un-scalable ways)
I've been exploring semantic technologies from the cold and incredible practical angle of "my own needs" :-). The RDF model offers a lot even when keeping data for myself: I can use RDF to infer new things from existing data and visualize relationships. An RDF triplestore can be used to store pretty much anything, and works well for CRUD. RDF is about graphs, so a triple store can be used to model documents too, since trees are a subset of all graphs.
RDF4J has really nice documentation and supports a lot of modern features. The getting started guide is great [1], and there's also a super friendly gitter room to ask questions [2].
1: https://rdf4j.org/documentation/tutorials/getting-started/
I have read this sentiment countless of times, but I don't quite understand it. Maybe because I am not familiar with the semantic web. Can you please suggest some reading list for a person who is unfamiliar with semantic web to better understand these issues?
The gist of it is to provide a way to model data as a graph, and make it almost trivial to exchange data between parties. A simple example is, if you and I want to share, say, user data, and you have a "people" table and I have "Users", and my fields are "first_name" "last_name" and yours "given" and "proper", and so on, how do we match the same user across tables? RDF provides a solution for that.
The "pie in the sky" part is that, from this basic building block, a whole host of other technologies can be built, some of them dealing with artificial intelligence, machines being able to _infer_ more information from what you have already said [2]. Inferences can work best when there's more data, that's were the ability to mix your data with third party databases comes handy.
Solid adds even more stuff to the table. In their words: derived from "social linked data", is a proposed set of conventions and tools for building decentralized social applications based on Linked Data principles. I think "decentralized" is the keyword here. I haven't even started looking at it :-)
What solution does RDF provide for that? I know of `owl:sameAs`, but even if it were feasible to look up those statements and reason about them (assuming someone reliable went to the effort to actually define them, and did so correctly), that already breaks down if I have to match a single field "name" (or, say, vcard:fn) with two fields "first name" and "last name" (or, say, foaf:givenName and foaf:surName). So as far as I know, there's still no practical solution - or is there?
To be able to say that two subgraphs describe the same thing, at some point, a mapping is needed between the different sub graphs (for users could be, say, a common schema:email statement). A good practice is to use some existing ontology when modeling (say, FOAF or schema.org), this can fix the problem of having to map fields since both sources would already be using the same conventions.
I think what's normally done is some sort of normalization pass or passes after ingesting the triples, so at the end of the process your app can always use your preferred property (say vcard:fn) or just ignore the subgraph in the worst case. There are a number of tools to perform this kind of normalization.. I know of SPARQL construct queries but I've seen others out there (rml.io).
- Having to create your own mapping for every piece of data of every divergent data source, with a not-insignificant amount of work involved?
- Those mappings inevitably containing mistakes/different interpretations that result in invalid data?
- Some of the data not being amenable to lossless mapping in the first place?
In other words: is "RDF provides a solution for that" not overselling it a bit?
But I think the integration story is way better than when integrating other data models, say, relational databases.
The other aspect is that URLs being the ids for statements, I can peek at the implementation of a 3rd party data model and use that _in advance_, instead of integrating later.
For instance, when I write my blog I can use the URLs of existing wikidata entities to model my own entities. If wikipedia entities used a relational model, the IDs of these entities would be opaque and could change at any time, so I wouldn't be able to do that.
In order to bootstrap the process, authorship tools need to produce semantic markup automatically on the behalf of users. Unfortunately the content silos do not want to do that. They're happy to keep the data to themselves and run Snowflake / big data queries on them all day long. They just don't want you doing the same. It's no benefit to them if your friend or interest graph escapes their hold.
This is a difficult chicken and egg problem.
A good angle of attack might be for the distributed web folks to take an interest. ActivityPub, Mastodon, and Matrix/Riot/(whatever they rebranded to) would be beneficiaries of schema ingestion. The more they can mark up the data they produce, the more useful it becomes. A tooling ecosystem can grow around semantic data.
Technically open standard APIs in a distributed web are already semantic, it may just be that their schemas don't map well to queryable triple stores. But that's just a transformation away. RDF is just a format for expressing your ontology in an open and compatible way others can reference and build upon.
Where did you get that notion? Just take a look at the internet this last decade.
What people crave now is a semantic web that is connected to a community of computers able to very quickly verify and distinguish what’s real and what’s disconnected from the truth.
When the data is your USP, sure. But there's a ton of data that's not.
I was experimenting with validating some data through the SHACL version of schema.org [1]. To be fair my findings are not much more than "anecdata" for now, since a proper benchmark should use a bigger dataset, allow the JVM to warm up, etc... But from a quick comparison, the RDF4j implementation was on my tests from 2 to 3 times faster than the "reference" one [2] when validating some small in-memory data.
While the original vision seems to have failed from a lack of incentives, the idea behind RDF as way to model connected data and its many serialisation formats is actually quite helpful. It seems to have found a niche in academia when it comes to sharing and integrating large graph datasets. I just consider RDF and most of the other semantic web standards as a framework for modelling and serialising graph data. It's pretty good at that.
What do you suggest as an alternative to RDF?
1: http://www.snee.com/bobdc.blog/2018/04/reification-is-a-red-...
2: https://blog.liu.se/olafhartig/2019/01/10/position-statement...
Interesting to see it back again. It had many problems but was always an interesting concept.
They only consider a website from their viewpoint – a human viewing the website through browser. The HTML (including microdata, JSON-LD, etc) is terrifying for most of them. So at best, their web designer builds in some semantic web standards but the data can be inaccurate or become out of data quickly because the website owner doesn't even know it's there.
Even big companies have adopted semantic web standards in such a confusing and inconsistent way that it's difficult to rely on the data being available in an expected format, so that a developer can build something on top of it.
Unless semantic web standards can be adopted by small businesses in a simple way (point, click and type) I don't think they will ever be widely adopted.
Finally, I don't think it's efficient to hide the 1kb of machine usable data inside 99kb of HTML designed for browser rendering.
Shameless plug: A small team and I have built an alternative to the semantic web, to store machine readable data in DNS. It can be populated by simple user friendly online forms.
Explainer and summary: https://www.num.uk
Best way to play around with it: https://app.numserver.com/tools/editor/add
What comes to mind is people training AI to parse web pages and mark them up to indicate different types of data inside of them. Such as the recipe example.
So someone who authors a WordPress plugin for recipes could incorporate it. Or, and this is more scalable but obviously very difficult, it may be possible to create plugins for web frameworks that automatically inject the semantic markup using AI. Maybe.
Then there could be another AI program that converts natural language queries into semantic knowledge queries.
But I think that AI could play a big part in rolling it out. I mean, if you had the AI systems without the semantic web, for open systems you would still want a way for them to share data unambiguously.
We can have semantic social networking now, https://github.com/patchworkuniversity/radiojade
In addition spam will continue to be a significant problem.