Hacker News Reader

Comment by x-complexity | Hacker News Reader

I'll be the first to admit this: All I'm seeing here is the JSON version of the semantic web.

https://en.wikipedia.org/wiki/Semantic_Web#Example

Fundamentally, it suffers from the same problems that the semantic web suffers from as well, primarily the top of my concerns being link rot & scenarios involving at least one link in those examples being inaccessible/unrecoverable.

For the Semantic Web / JSON-LD to work, the infrastructure behind it needs to permanently keep all of the necessary sources that a file relies on (i.e. permaweb). Otherwise, the entire knowledge graph suffers from a left-pad style situation, wherein the removal of a critical piece of definition(s) will cause the unparsability of all documents reliant on it for definitions.

View on HN

Yes, JSON-LD is just a different serialization format for RDF.

For anyone looking for a solution to the link rot problem (as well as versioning and dependency management), they might be interested in Plow[0].

We built it due to frustration with exactly those drawbacks of the semantic web/data space. At the core it builds on a model very similar to Cargo, with a public index and checksummed artifacts, that allows for easy mirroring for use-cases where all transitive links always need to be resolvable.

We are also looking to build out some tooling in the future that makes keeping up those properties easier, such as a linter that ensures that e.g. all referenced concepts in the dependency tree have a definition. (We already have a primitive version of that, but that's just on a single package level.)

[0]: https://plow.pm/

What's the advantages of plow over ipfs?

IPFS alone only gives you content addressing. So you'll need at the minimum some higher-level IPLD structure that allow you to express structures such as an ontology, etc.. That can be based on the Plow model, if you want.

If you want a workable ecosystem that enables many decentralized parties to collaborate, the model of not-too-big-not-too-small packages that are interconnected via abstract dependencies (embodied by SemVer ranges), is still the gold standard to achieve this in my opinion. And breaking the mold of traditional monolithic slow-moving ontologies with high technical barriers to publish is one of the main motivating factors for building Plow (together with building a stable ontological layer that you can build other software on).

I actually have worked on bridging the gap between semantic data tech and IPFS in the past[0] (both on a fine-granular per-concept level and a more coarse-grained ontology level), and I can just say that there are a ton of additional challenges if you want to do it right (and semtech is already challenging enough as it stands today).

Conceptually, the infrastructure parts that make up Plow (the index and artifact store) are also flexible enough that you could distribute them via IPFS.

[0]: https://github.com/rlay-project

It also comes from the exact same line of development at W3C[1]. LD, or “linked data”, is W3C’s attempt at rebranding the “semantic web” (and, despite my general dislike for rebranding attempts, seems more descriptive). This is the first time I see LD expanded as “linking data” , though.

[1] Manu Sporny [JSON-LD 1.0 editor and CG chair], “JSON-LD and Why I Hate the Semantic Web” (2014), http://manu.sporny.org/2014/json-ld-origins-2/ (down right now but that looks temporary?). Unfortunately, he did not participate in JSON-LD 1.1 (as much?), and the spec once again returned to its RDF-jargon-filled equilibrium.

Yep it’s just hypermedia. And as we know, cool URLs don’t change. Of course there are hypermedia solutions to even changing URLs, but that would require investing in using the actual platform rather than constantly rediscovering parts of it.

Is "left-pad style situation" (aka the Azer/Kik/NPM drama) the formal name for this class/type/definition of system fragility? What would this be an example of, formally, in math or systems design or whatever?

The reason I ask, is that I see it literally all the time, sometimes as a design goal, which drives me absolutely up the wall. "Let's make a centralized repository (CIR) of all acronyms, and then everyone can reference the CIR". Ah yes, sure, great idea and the first time someone changes Air Conditioning to Alternating Current ALL YOUR DOCUMENTS CHANGE[1].

If "left-pad style" has a formal name, I might be able to offer a counter-argument in the form of math.

[1] Now, obviously, the CIR problem does have a solution, I'm aware of that, but it's not a solution that lies in the domain of the document markup - it's a change process problem, which is in another domain, which means extra eyeballs and red tape. So I guess the broad definition, I am looking for, is what is wrong with these sorts of functionalities that break out of their domain like this? What do you call that?

Agreed, there's an overly-optimistic assumption that other people will go out of their way to maintain referential integrity for applications that they don't know about.

I think it would be better if it was done in a content addressed way so that you can trivially host the bits you're relying on without having to also be an authoritative source for them.

Once we have that figured out we can socialize it onto the users. Like either pay $1 for access or pin whichever nodes in the app's dependent knowledge graph hash to your user ID mod 1024. Having helped with your 1/1024th share of the hosting, you get access for free.

Maybe it's not "link rot," it's just information that's no longer relevant. Design your systems to be able to handle that. And you'll be able to interop and reason with all kinds of data.

Except it's highly relevant and gone anyway.

Designing a system to operate with partial data (or backing it up everywhere) has pretty hard tradeoffs when you are dealing with a system the size of the internet, any piece of content can go from a few bytes to terabytes.

So we add a DHT and use magnet links. The permanence problem is then at least distributed and the semantic graph is no longer a collection of single points of failure.

Reply on news.ycombinator.com