Hacker News Reader

Comment by temporarely | Hacker News Reader

I've always felt stderr should have been stdmeta.

p.s.

Well, actually more completely, something like this:

                  +---------+
    [meta-in] --> |         | --> meta-out
                  | p r o c | 
     input    ==> |         | ==> output
                  +---------+

View on HN

You might like this proposal:

https://unix.stackexchange.com/questions/197809/propose-addi...

The idea is that some output is metadata (such as ps headers) and some is data. With stdmeta we could differentiate between the two.

Powershell has seven "output streams": https://learn.microsoft.com/en-us/powershell/module/microsof...

That's awesome, I wish it was more commonly the case.

I bet you'd also be onboard with files having data forks and resource forks too ... ?

TBH, it's a great idea, but history proved that we apparently prefer a single stream of data and solving all the problems it brings ...

We don't have a single stream - that's the point. stdout and stderr are already different streams.

Right, I was alluding to the original Mac's filesystem, with separate data + resource forks, requiring all sorts of hacks to transfer files to and from them. Due to all the trouble of working with that across other platforms, Mac eventually gave that up at a filesystem level, and sprinkled ".DS_Store" files everywhere.

TCP/IP streams are bidirectional, but there is a limited way of sending "out of band" data, though it is not used as much. It would have been nice if the stdout/stderr multiple streams extended to TCP/IP networking and even HTTP messages too.

> TCP/IP streams are bidirectional, but there is a limited way of sending "out of band" data, though it is not used as much.

It's not real "out of band" data: that's something wholly invented by the Unix socket API. TCP itself just has an "urgent pointer", which addresses some byte further in the data stream that the receiver doesn't have yet, with the intent that higher-level protocols could use it as a signal to flush any data up to that pointer to observe whatever the urgent message is. There's nothing in the protocol itself to actually send a message separately from the rest of the stream.

Among conceptually Unix-like OSes, at least one tried to do something along these lines: see http://www.bitsavers.org/pdf/apollo/SR10/011021-A00_Using_Yo... PDF pp 151 and following.

Thanks. (page 151 for those interested)

This is interesting: MIMO channels to a process. Single stdin/stderr/stdout is effective for a single OS process, but with so much pulled up to user land (e.g. workers via green threads) maybe it makes sense to introduce multichannel i/e/o.

Sure. Make metadata out-of-band rather than in-band, so that the ungovernable mess of Unix-standard plain ol' text streams is replaced by structured data.

So, well then: allowing programs to consume and emit JSON - is this progress ?

JSON is hardly the greatest structured format, but nearly anything is better than Unix text streams.

I think having a plain byte stream is actually better, in a worse is better sort of scenario.

A plain byte stream can be easily aligned to work with any future or past encoding fashion. Consider the situation if them that designed unix had not been so aggressively minimal. We would probably be complaining how streams had to be ASN1 encoded and how much a pain it is to define the schema for what should be a simple ad-hoc data transfer.

As it stands, you can put whatever object format you want on top of the stream. I think it is the same with the files. I am sort of pleased we are not stuck with some obsolete no longer relevant, screwball structured format from the 70's that all our file have to conform to. instead our file are a simple range of bytes and we can impose whatever structure on them that we want.

I'm thinking that if a range of XML markups can delimit and separate out metadata (e.g. HTML head v body), then heck so can JSON. Maybe not prettily.

To me the advantage of XML in this would be CDATA segments, that can represent (almost) any text (that isn't the CDATA terminator string) without requiring text escape hell. And if some sort of JSON supported said CDATA, well, it is no longer JSON.

Why not more than 4? Another thing CL did "better":

https://www.lispworks.com/documentation/lw50/CLHS/Body/v_deb...

https://www.lispworks.com/documentation/lw50/CLHS/Body/v_ter...

Sure, it is arguably cleaner to explicitly isolate error condition from general process meta-data. So in my OP diagram you can add an errout coming down from the proc-box. I've used this actual pattern in-process to hook up pipelines of active/passive components. However, there is no sense (imo) to propagate the errout of P(n) to P(n+1), so 2-in, 3-out.

p.s. That is pL(n).stderr -> pE.stdin, where pL is the 'business logic' and pE is the system's error processing aspects. I.e. the error processing component's stdin is the stderr of the logical processes (Lp), so there is a uniform process model applicable to both logical and error processing elements of the pipeline.

The issue is how to do this within the limits of line terminal interface (CLI). In code (as in in-process chaining) that aspect is a non-issue.

so... named pipes/named file descriptors?

Reply on news.ycombinator.com