Hacker News Reader

Comment by haberman | Hacker News Reader

> JSON would do nicely as well.

Yep, some friends of mine did this with JSON, but didn't make the schema explicit like I mean to: https://github.com/benbernard/RecordStream

> Better yet, unify the shell with a virtual machine that is used to implement the OS, and have everything available as 1st class Objects.

Please no. This is the Microsoft PowerShell approach, where everything is a .NET object. Once you start dictating representations of objects, you are dictating far too much about the implementation of individual pipeline nodes.

View on HN

How can your tools all accept the same kind of structured data without dictating its representation? I don't get it.

Allow a shell variable to control output record and field separators. Default to space and newline if nothing is specified.

ORS=: OFS=, ls -l # rw-r--r--:1:uname:gname:1025:Jun 1:somefile.txt,...,...

Add a specifier for dates, usernames, groupnames, etc. DSF="%Y-%m-%d" ls -l

When I talk about a "representation," I mean an in-memory format. For example, the "representation" of an HTML tree is the DOM.

Yes, you have to agree on a serialization format (JSON, Protocol Buffers, etc), but that's not the same thing. From a serialization format you can represent the data however you see fit in your process. For example, a C++ user might represent a string as a std::string object whereas a Python user would represent it as a native Python string.

The VM-based approach (like PowerShell) defines an in-memory tree representation, namely .NET objects. This means that you can't really interoperate with this stack unless you use .NET too, since you don't have an easy way of converting .NET objects to your own objects.

Yes, but you could also just use IronPython and script in Python.

The more decisions you force on your users, the more reasons you give them to choose some other technology, and the less future-proof you are. For example, .NET is only really usable on Windows.

Just a nit: I think protocol buffers include representations, not just serialization formats. You need to have the schema of the proto to parse it correctly, know which fields are required, repeated, etc. Am I understanding you correctly?

Schema isn't representation the way haberman is using it. He means something like implementation, which protobuf has in many languages in various VMs and can be easily ported to more.

It's true that many Protocol Buffer libraries include representations, but these are for convenience; Protocol Buffers are defined in terms of their serialization format and schema.

I don't follow. Isn't schema the same as representation? It's the equivalent of the DTD for an XML document.

To be super concrete, I can't read a file containing protos without knowing their type, what fields they contain, etc. I can however read JSON just fine without knowing the precise schema being encoded.

Why does the protocol define the objects? Shouldn't the object metadata should have enough information to transform the output? E.g. object is a URL vs protocol is a URL

Reply on news.ycombinator.com