Publishing JSON-LD for Developers
datalanguage.com
datalanguage.com
According to the Wikipedia article [0] this is used by Google Knowledge Graph, but there's no other examples. Are there open source RDF processors? What are they used for?
Previously I've used json-schema for documenting, validating, and testing a REST API. The tooling wasn't very good, but it worked alright. As I understand it, JSON-LD wouldn't help at all with this.
The fact that JSON-LD is a W3C standard while json-schema isn't doesn't constitute an argument against it. Standards regularly show up and die without ever gaining adoption. Maybe instead of a vacuous remark on JSON-LD offering more than json-schema with respect to linked data you could actually muster some concrete example which highlight these alleged benefits?
I'm not an expert on this subject, but isn't it worrying that third-party links can change or go down without notice? Ideally if I'm hosting a context I'll want to replicate my whole dependency graph and provide a mirror in order to reduce dependencies on external systems. However, just because I want to host my own mirror doesn't mean I want it to be treated as a different kind of entity. It seems like something such as content-addressable storage [1] could be relevant here.
[0] https://en.wikipedia.org/wiki/JSON-LD
[1] https://en.wikipedia.org/wiki/Content-addressable_storage
For example: 1. How do you distinguish IDs from one system with those of another? RDF solves this. 2. How do you distinguish property names across systems? RDF solves this. 3. What if a entity originates in system A which you don’t control but you want to add a property to it in your system? RDF solves this.
JSON-LD is simply one way to serialize RDF data that makes look like what you would typically expect from an API that returns JSON. I.e. it’s developer friendly
In addition to the above, RDF is also an extraordinarily flexible/schemaless data model if you store your data as triples rather than as documents. There is definitely a steep learning curve to linked data, and I’ve had a hard time explaining it’s value over the years, but I think it’s worth the time to learn, understand, and leverage.
All of these benefits exist before you get into things like ontologies, which allows you to add additional value on top of what I outlined above.
> If you have multiple systems with data, and you want to merge the data together into a unified whole for some reason
This isn’t just about “big data” or “data warehousing” or such OLAPy concerns. It’s really about the bread and butter of modern information systems.
“Get some ID from service A, use it to query service B.” Does that sound familiar? That’s data integration for you. That’s what RDF / Linked Data is about.
JSON-LD is JSON for Linking Data.
When you're passing semantic web data around or maybe storing it temporarily in a text file (rather than in a triple-store) one of the formats you can use is JSON-LD. So that covers when you use it; the next question is why you use it over one of the other interchange or at rest formats. And generally the answer to that is that you're using Javascript as your language and you hate XML :)
Anyway, less facetiously here is the blurb from the website!
“Linked Data empowers people that publish and use information on the Web. It is a way to create a network of standards-based, machine-readable data across Web sites. It allows an application to start at one piece of Linked Data, and follow embedded links to other pieces of Linked Data that are hosted on different sites across the Web.”
“JSON-LD is a lightweight Linked Data format. It is easy for humans to read and write. It is based on the already successful JSON format and provides a way to help JSON data interoperate at Web-scale. JSON-LD is an ideal data format for programming environments, REST Web services, and unstructured databases such as CouchDB and MongoDB.”
This is at a lower level of abstraction, but... I care simply because it's Google's preferred encoding for web publishers to communicate metadata to them:
https://developers.google.com/search/docs/guides/intro-struc...
https://groups.google.com/forum/m/#!topic/mozilla.dev.platfo...
It may be read as an argument against making namespaces a ubiquitous part of JSON, in the same way as they are ubiquitous in XML (even though not part of the base XML spec). It’s a valid argument against getting to the point where namespaces pop up in every JSON tutorial, and every JSON serializer takes a namespace map, and every user of JSON has to deal with them.
But JSON-LD (as RDF) uses namespaces to solve a particular problem (merging data from disparate sources). Solutions involve costs, that’s normal. You might weigh this cost against the expected payoff and decide it’s not worth it for you — but the article doesn’t do that.
1. What benefit does this format give that other schema systems do not?
2. The `@context` just seems like a way of defining variables. As best as I can tell, the use case leads to one only using each of these "variables" once. What's the benefit?
3. What benefit does RDF in any format provide to the average web developer?
Sorry if these questions are asinine. It's just that the article bemoans the fact that this hasn't hit mainstream web development yet, to my mind, does nothing to encourage its use.
2. Context is for shortcuts, all properties can be expanded to full names but these are qualified and long.
3. Benefit over what? RDF is for describing things with machine readable language.
2. Okay, so you either put the long data in the `@context` or in the original data. I suppose it may improve readability but for a data interchange format (which I believe this is), it seems wasteful.
3. Benefit in general. Why would I use RDF at all? What is any of this for? I can describe my data just fine with normal JSON. Why should I design my JSON in this way?
As for practical uses JSON-LD is used by Google to scrape things from the internet, and check if they got a Place here, Person or maybe a Business. Activitypub that powers Mastodon is also JSON-LD.
Probably (almost) none, otherwise in its 20 years of existence it would have been used somewhere. Heck, even Semantic Web scholars actually don't use it for their own personal websites...
I think one problem though is that RDF is a solution searching for a problem to solve. It's clear it was designed from some abstract space instead to be a solution evolving from real use cases. And that's why most people have a hard time understand it, and even more putting it in practice.
For example, even basic RDF concepts such as a resource are hard to grasp. You have an IRI, great, but should it be dereferenceable? If yes, what it expected to be returned: a document in various formats or more RDF triples? If not (URN case), what to do with it? All these problematics are not addressed in the spec itself but must be understood before even starting to implement a hello world.
There is interesting projects such as DBpedia. Another problem of RDF is social and pertains to its advocates: trying to impose the technology everywhere without taking in account the audience needs nor the limitations of the technology itself is not going to work great. See the suggestions made during this workshop to improve the technology: https://www.w3.org/2009/12/rdf-ws/ In 9 years (half of its live), nothing was addressed.
I agree with your other points though, people have a hard time understanding it, and the community hasn’t really shown the value. My hobby project is an effort to show the value, but it’s not ready to share yet.
And yes, there’s a lot of confusion things (URI vs URL, dereference or not) that the community hasn’t done a good job articulating a happy path. There’s another you didn’t mention, there’s an impedance mismatch between triples and resources. One is property oriented, and the other entity oriented, and they dont’ always play nice together.
They are not, those are good questions. The fact that you've been able to make and deliver successful products for years without knowing or worrying about RDF or JSON-LD should make you suspicious of JSON-LD and it's value proposition.
I've been hearing about for years and yet the development world is pretty happy moving ahead without it. I've looked at RDF and JSON-LD some years back when designing a web API shrugged and ignored it.
Now sure, it's nice. It's like having a car with chrome plated rims instead of regular rims. It's better, shinier, prettier but not enough for me to bother to spend time (money) to get it.
Your complaint is as if you looked at the Sublime Text configuration file, which is in JSON, and said "I liked JSON better when you didn't have to configure anything".
JSON-LD isn't designed to replace JSON, it's designed to replace all the awkward formats for RDF.
Seeing some knee-jerk negativity in other replies, I should emphasize: nobody is under any obligation to use JSON-LD just because you use JSON! It's not an upgrade to JSON, it's an upgrade to RDF.
Spec: https://github.com/edn-format/edn/blob/master/README.md
Grammar: https://github.com/antlr/grammars-v4/blob/master/clojure/Clo... (EDN is a subset of Clojure).
Sorry if this is too fanboy, I just need to assume people aren’t aware when I see this big pile of mud that is JSON.
If the space intrigues you, I would suggest starting with a graph database instead.
I also find it strange to call it dead when companies like google and facebook are using it for metadata on content to power their core products. Again, a graph database does not solve that.
PS: and yes, a graph database is a good start to open your mind.
<%= WebSite.new(
name: 'Texas Public Law',
url: 'https://texas.public.law',
)%>
...and you'll see this in the HTML: <script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "WebSite",
"name": "Texas Public Law",
"url": "https://texas.public.law"
}
</script>
This DRY's up my code - I just encode the right way of forming JSON-LD and the correct objects and attributes in one place, the library. (That JSON-LD is from my live site, https://texas.public.law)Details:
First I created a small library which wraps Rails ActiveModel Validations so they can be used in any Ruby object. And I supply a new validator, TypeValidator.
https://github.com/public-law/validated_object
With this building block, I can create classes for each Schema type I care about. And they strictly enforce their attributes:
class Organization < SchemaType
attr_accessor founding_date
validates founding_date, type: Date
end
That's a little redundant, but to a Rails developer, it's immediately clear what's going on. The full Organization code: https://github.com/public-law/schema-dot-org/blob/master/lib...I'm not using Ruby-RDF at all. I looked into the current Ruby meta-data and strong typing work, and found it all too complex for my simpler use case:
I'm creating websites, and I want to embed correct JSON-LD-formatted schema.org in a rock-solid way: (1) Correct syntax, and (2) correct semantics. I've got the information as Rails models and objects, and I need to convert this to HTML/JSON-LD.
So all of these objects, like Organization above, recursively creates proper markup in response to #to_s:
<%= @an_organization %>
...instantiated from that class above, will result in this in the generated HTML page: <script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "Organization",
etc.What do you think of Spira https://github.com/ruby-rdf/spira , does it complement what you are doing in any way?
And for generating the website metadata, that info is coming from the Rails models.
Data structures aren't marked-up text. And that one time that XML was supposed to be used for marked-up text (XHTML), it wasn't even good at that either.
Main issues are:
- a literal node (strings which carry the useful information for an human) cannot be the source of an edge, so you end up creating "resources" nodes and pushing content as leaf of the graph. Access to its is thus done with a lot of indirections.
- no annotations, which mean you cannot represent simple graph construct such as a weighted edge.
- blank nodes, needed to join more than two piece of data which, are anonymous so they are difficult to reuse. That's sad because a graph is supposed to make data link each other. Coupled with issue 1, it mean most linguistic resources graphs are actually closer to trees than real graphs.
- reliance on the web infrastructure (IRI, DNS, web servers, ...) which is a big dependency with a lot of gotchas and not fitted for every uses cases (self contained data for example)
- performances is abyssal. I tried one of the dictionary build on RDF, and some query took up to 40 seconds (!) to return no result. Even bloated web dictionaries perform under 2 seconds.
Also, the second sentence of the article is totally false. I read the history of JSON-LD by one of its main author [1], and it does not originated in the RDF community. It originated elsewhere, and then there was RDF people included in the loop.
Also, nodes can host complex structure but do not expose it to the graph (edges are made only between nodes) and edge can be of n-arity. The system is regular enough to implement simple and complex lexicographic constructs easily as well as to use the graph in an actual application.
Issue with RDF et al if that before you start even working on your real data, you have to come up with an ontology, that is to describe the world in categories, which is not easy. Another problem is that a lot of projects want to define standards that works with everything. That's not realistic. So I took an approach where you can learn from your domain by working directly on your data, make something useful with it, and then expand it you need it. KISS.
I know that some of my collegues are using JSON-LD as the interchange format for this kind of thing.
Prolog won't come with reasoners for the description logic profiles supported by OWL 2 (EL, RL), but can be used to implement such reasoners if needed.
> reliance on the web infrastructure (IRI, DNS, web servers, ...)
This is the whole point of RDF. It is what enables Linked Data.
> I read the history of JSON-LD by one of its main author [1], and it does not originated in the RDF community.
Not sure what you mean exactly, but here’s from http://manu.sporny.org/2014/json-ld-origins/:
> JSON-LD started around late 2008 as the work on RDFa 1.0 was wrapping up. We were under pressure [...] to come up with a good way of programming against RDFa data.
JSON-LD definitely was always an RDF serialization, created by people intimately familiar with RDF and the Semantic Web.
Exactly. That's my exact point. So, now, my problem is other scholars asking "why don't you use our model X or Y based of RDF" when the only thing I care about is the graph part. And I don't use RDF because of the points I exposed earlier. I should I've put emphasis that I'm working in a given field (lexicography), and that the RDF issues I face may be irrelevant for some other use cases.
As for the JSON-LD origin I stand correct.
> JSON-LD is trying to make RDF sexier (more exactly said, RDF try to pimp itself by using JSON-LD) but as it is just RDF with a new serialization syntax, it share every of its issues.
JSON-LD is _only_ a serialization format. Look, here are _nine_ others. https://help.poolparty.biz/doc/developer-guide/general-infor... Choose the one that best suits your needs. And using language like sexier and pimp, words fail me.
> a literal node (strings which carry the useful information for an human) cannot be the source of an edge, so you end up creating "resources" nodes and pushing content as leaf of the graph. Access to its is thus done with a lot of indirections.
What you mean here is that in the [S, P, O] resource, where S is the subject and P is the predicate and O is the object, one can't make the Subject a string literal. No you can't. Big deal. Whatever language and language framework you're using should have an abstraction for unpacking string literals, and if it doesn't you can roll your own. The semantics of RDF are that literal values are Objects. This is because in natural language we tend to say things like [User5000, is called, "titanix2"] and not ["titanix2", is the name of, User5000]. Sorry if you don't like it, that's the convention: https://www.w3.org/TR/rdf11-concepts/#section-Graph-Literal
> no annotations, which mean you cannot represent simple graph construct such as a weighted edge.
No idea what this means
> blank nodes, needed to join more than two piece of data which, are anonymous so they are difficult to reuse. That's sad because a graph is supposed to make data link each other. Coupled with issue 1, it mean most linguistic resources graphs are actually closer to trees than real graphs.
I didn't get how blank nodes worked originally and made myself look like a fool when I questioned how they worked on a mailing list. Such is life. They actually make sense for what they do once you wrap your head around them. Rather than me explaining them to you here I urge you consider how they work again: https://www.w3.org/TR/rdf11-concepts/#section-blank-nodes
> reliance on the web infrastructure […]
Well, it's not called the semantic web for nothing! Also, it's been pointed out to me that one can use URNs instead of URLs if you don't want to use the semantics of the HTTP schema. But you know what it seems to be working for pretty much everybody else and how expensive is it to but a domain these days?
> performances is abyssal. I tried one of the dictionary build on RDF, and some query took up to 40 seconds (!) to return no result. Even bloated web dictionaries perform under 2 seconds.
Without further details I'd have to presume you're talking about SPARQL? Yes some queries are slow but if you're dealing with millions or billions of triples then that's inevitable. I've discovered that you can optimize SPARQL once you start digging into it. No it's not easy, but it's not exactly simple tech either. Without details about the triple-store you were querying against and the query you used there's nothing more I can help you with. Anyway, your complaint is like you having a slow query on some particular relational database and you blame the entirety of SQL, wow.
For annotations, you can read[1]. What I meant however a really simple graph construct: information on edges. In the Neo4j lingo that is called a property[2]. There absence in RDF make it difficult to represent situations such "Paris"---distance:54---"Rennes". I'm visibly not the only one it's itching[5].
Yes you can use URN, be some projects doesn't support them. Check out the linguistics data cloud[3] for an example with such a requirement. Moreover, renting a domain (you can't buy a domain name) may be cheap but why putting this burden on yourself if you can avoid it? Even Microsoft mess up with them[4] for time to time. I read some papers of 2 years old and all the endpoints and links where dead. So yes, if you're not doing a web project, no need for a web format. Sadly this is what is done in my field.
Anyway I'll be curious to see something you built with RDF, if any. I failed to find your work besides two papers that aren't related to the subject.
[1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352...
[2] https://graphaware.com/neo4j/2013/10/24/neo4j-qualifying-rel...
[3] http://linguistic-lod.org/
[4] https://www.theregister.co.uk/2003/11/06/microsoft_forgets_t...
So create your own serialization format suited to your own needs.
> I don't really understand the point you try to make about literals. Mine is that because literals cannot be the origin of an edge, they form sinks. And that's very annoying for building a graph (of strings) because you then need to create proxy nodes for data all the way up.
If by origin you mean what RDF calls subject then yes, as I said, the RDF data model does not support subject literals. That is by design. What you could do is create a resource that encapsulates strings in the way you want a use that for both the subject and object. Don't think them as proxy nodes, think of it like you now have the ability decorate your string literals in an extensible way.
> For annotations, you can read[1]. What I meant however a really simple graph construct: information on edges. In the Neo4j lingo that is called a property[2]. There absence in RDF make it difficult to represent situations such "Paris"---distance:54---"Rennes".
PREFIX distance: <http://www.my_cool_domain.fr/prop/distance>
[wdt:Q90 distance:354 wdt:Q647]
Because Paris and Rennes are already in Wikidata, why not use them? Create your own namespace, call the distance property "distance" when you're prefixing it, reserve numerals in this property to signify geographical magnitude in km. 354 between Paris and Rennes, not 54?> I'm visibly not the only one it's itching[5].
Two wrongs don't make a right. Considering that link was calling for this in 2010 suggest to me that you oughtn't hold your breath. Also them self-describing as an expert is crass.
> Yes you can use URN, be some projects doesn't support them.
Well, that is not the fault of RDF now is it?
> Moreover, renting a domain (you can't buy a domain name) may be cheap but why putting this burden on yourself if you can avoid it?
If you can't afford €10 per year or the institution where your project is located can't afford it then you have bigger problems than imaginary RDF restrictions.
> Anyway I'll be curious to see something you built with RDF, if any. I failed to find your work besides two papers that aren't related to the subject.
I've used Wikidata and DBpedia to build a corpus of philosophers and their philosophical texts without too much of a problem once I wrapped my head around how RDF and SPARQL work.
Paris on Wikidata: https://www.wikidata.org/wiki/Q90
Rennes on Wikidata: https://www.wikidata.org/wiki/Q647
[1] https://www.safaribooksonline.com/library/view/practical-rdf...
Imho, JSON-LD is a piece of crap (because it is based on JSON, that is flawed by design). So I would definitely promote the usage of RDF/N3 (and its N3.parse()) as the way to go when you need to transmit typed graphs from server to client.
Note: N3 is a dialect of RDF, that is super easy to serialize and deserialize, that builds graphs of objects in memory (which JSON cannot do, because it can only describe trees). And that (almost always) follows a normalized vocabulary for types, IDs and labels.
(as you can see, nothing very complex :)
{ [ x:firstname "Ora" ] dc:wrote [ dc:title "Moby Dick" ] } a n3:falsehood .
PS: yep, there are blank nodes here. PPS: very few parsers take this into account, I agree. But I am pretty sure N3.parse() does.