The patented IBM multi-pipe: the evolution of Unix pipes | Hacker News Reader

The patented IBM multi-pipe: the evolution of Unix pipes

9fans.net

144 pointsdexen36 comments

Just to clear some things up. The idea was invented by Doug McIlroy who said that we should screw computers together like garden hoses. This was in the 60s. They made unix pipes as a simple compromise. I wanted to make "screen" for plan 9 and because plan 9 has no TTY, you can do it just with buffered muxed pipes. It was a lucky insight into something cool. I invented nothing, I just tried to keep working in the great tradition of unix and share the implementation of ideas. By the way, ants.9gridchan.org is actually more interesting than all this patent bullshit. I'd rather have people take an interest in my current work than make a huge soap opera out of this. Advanced Namespace Tools for Plan 9 are pretty cool I think.

Mycroft, don't be too hasty. I remembered Eric and others working on this for a long time, and indeed with a little bit of searching I found an email Eric sent back in May 2009 linking to a page which led me to http://graverobbers.blogspot.com/search/label/brasil

In those articles, and in the email thread, he discusses running commands across multiple machines using methods which I believe are recognizable as multi-pipe like. This article predates your "time traveling". Let it go.

IBM employees are strongly encouraged to submit things for patents. You do something interesting, you write it up quickly and send it to the patent lawyers, they do the rest. Maybe eventually you find out it has been approved. I'm not saying it's good, but it's the IBM corporate culture and this is not an especially unusual situation.

I think I have stated clearly that I personally do not take credit for these ideas. I believe that Doug McIlroy expressed everything important related to pipes, hubs, and everything else. I believe that IBM's patent in this case is indefensible completely regardless of any work that I have personally done - but that my personal interests in this area called my attention to this patent.

As I hope I made clear, I think the idea of "we can mux the readers and the writers" is not anything that should be patented, and neither is the details of implementing this idea.

It's this sort of attitude, "well, it's the lawyers' fault, you can't blame the engineers" that helps enable this sort of garbage. Yes, you absolutely can blame the engineers. They choose to work at IBM and choose to stay there even after they learned about the patent happy nature of the company. Same goes for every other patent happy company. You can't continue to patent everything under the sun if you don't have engineers finding it for you. "Just following orders" hasn't been a good excuse longer than software has been a significant field.

> You can't continue to patent everything under the sun

I see what you did there

> By the way, ants.9gridchan.org is actually more interesting than all this patent bullshit.

But the patent bullshit is why your software is worthless. It can't be used because someone claims ownership of the very concept it's built on.

The original announcement/prior art: http://9fans.net/archive/2009/07/278

"All functionality is implemented using a simple abstraction called a Hub, which is an attempt to expand the traditional unix pipe | to have multiple inputs, multiple outputs, and the ability to connect and disconnect from the data stream dynamically."

I'm pretty sure we had parallel readers and writers in Parsytec's PARIX environments back in the early 90s (I would have been using it from around 92-94). This ran on transputer hardware and had a 'parallel shell' which supported odd piping syntax to describe a network of processes that it would farm out on the hardware.

Fairly niche, and I don't have a manual to hand, and I didn't use that aspect of it except as a toy - the Solaris workstations we had sitting around were much faster and the extra though you had to put in to parallelise shell scripts wasn't really justified.

What a nightmare. Everything that is wrong with software patents is illustrated here. First and formost in my mind is that this is a obvious extension to existing software.

"I believe the idea of a Multi-pipe is a natural progression of basic unix pipes, and this amazing Patented Invention of IBM's is something that I think everyone should know about."

Seems pretty bold to put "natural progression" and "amazing patented invention" in the same sentence.

It's sarcasm in its clearest form.

I'm not into details of IBM patent, but it seems to me that the general idea doesn't feel that much innovative, given that we already have numerous projects achieving similar goal.

It's even bolder to use a time machine to send software back in time. Of course Plan 9 has a time machine built in, it's just that people can't get used to rio(1) and the lack of readline support, so they tend not to get far enough into the OS to avail themselves of it.

The author is out of luck: the idea to send source code back in past to circumvent patents was itself patented some time ago...

You mean like this? http://en.wikipedia.org/wiki/Tee_%28command%29

Can you insert "tee" into a running pipeline? Can you insert an additional writer to a running "tee" process? I think not. Like you I imagine there's an old way to do these things, but I think that "tee" is not quite it. Maybe you could write or find something similar to "tee" that forks or joins upon receiving certain signals.

> Can you insert [X] into a running pipeline?

Yes [0]. It needs a bit of tooling to be more user friendly, but even the gdb stuff is pretty self-explanatory.

[0]: http://etbe.coker.com.au/2008/02/27/redirecting-output-from-...

tee can do barriers and reduces? Wow!

Prior art isn't going to matter pretty soon: https://news.ycombinator.com/item?id=5379944

Publicly disclosed prior art still matters. But now, first to file ALSO matters in litigation. (making hidden notebooks that show the other party invented it two years before you did less interesting)

No, being the first to invent won't matter.

Before: I invent something, document internally that I invented it, but tell noone. A year later, you invent the same thing and apply for a patent. Since I invented it first, I could challenge your patent.

Now: Since you filed first, you automatically win

What doesn't change: I invent something and publicly disclose the idea, giving a presentation at a conference, putting a product on the market, putting code on github, etc. You are not able to patent that. In the US, I have a year to patent it, but will have difficulties patenting it in the rest of the world.

Note, this is a simplification as IANAL, but prior art does still matter.

In case there is doubt, language from the bill is below:

‘(a) Novelty; Prior Art- A person shall be entitled to a patent unless--

‘(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention;

So basically, first to file allows the filer to "steal" the unpublished work of others. Independent parallel invention should be a defense against infringement, or better yet, evidence of obviousness.

These are the claims:

"What is claimed is:

1. A method, in a data processing system, for group communications using a MULTI-PIPE synthetic file system, the method comprising: creating, by a master application in the data processing system, a multi-pipe synthetic file in a plurality of multi-pipe synthetic files in the MULTI-PIPE synthetic file system, wherein the master application indicates a multi-pipe operation to be performed; writing, by the master application, a header-control block to the multi-pipe synthetic file specifying at least one of a multi-pipe synthetic file system name, a message type, a message size, a specific destination, or a specification of the multi-pipe operation by writing to a specific offset; opening, by at least one other application in the plurality of applications participating in the group communications, the multi-pipe synthetic file in the MULTI-PIPE synthetic file system; signaling, by the MULTI-PIPE synthetic file system, a MULTI-PIPE file system module in an operating system of the multi-pipe operation; and implementing, by the MULTI-PIPE file system module, the multi-pipe operation as identified by the master application, wherein the master application and the at least one other application in the group communication either read or write operation messages to the multi-pipe synthetic file, wherein the master application writing a message that is longer than an atomic I/O buffer size of the multi-pipe synthetic file is delivered using the header-control block with the message size specifying a total size of the message and as a contiguous unit to at least one other application, wherein the messages are signaled to the MULTI-PIPE files system module, and wherein the MULTI-PIPE synthetic file system module performs an appropriate action based on the mode identified by the master application.

2. The method of claim 1, wherein the multi-pipe synthetic file opened by the at least one other application is limited to only being used in the mode directed by the master application and wherein the at least one other application is not able to send any control messages.

3. The method of claim 1, wherein the mode that the at least one other applications open the multi-pipe synthetic file in is dependent on the multi-pipe operation requested by the master application.

4. The method of claim 1, wherein the MULTI-PIPE file system module is implemented as at least one of user space file servers, embedded within the operating system itself as built-in file systems, or a set of dynamically loaded modules.

5. The method of claim 1, where the MULTI-PIPE synthetic file system is constructed to allow allocation of new instances of the plurality of multi-pipe synthetic files via user action, either by leveraging existing file-system interfaces or by using a synthetic control file which the plurality of applications sends commands to allocate new instances of the plurality of multi-pipe synthetic files.

6. The method of claim 1, wherein any application writing a message with a header-control block is directed to a specific enumerated reader queue where the message will be received by a specific application when read from the multi-pipe file.

7. The method of claim 1, wherein the master application writing a broadcast message to the multi-pipe synthetic file triggers a copy of the broadcast message to be delivered to all applications in the group communications reading from the multi-pipe synthetic file by encoding a broadcast flag in the header-control block.

8. The method of claim 1, where different ones of the at least one other application are isolated from one another by the MULTI-PIPE synthetic file system enabling application appropriate multiplexing without the limitations of native file system I/O multiplexing of the operating system and wherein, in non-enumerated multi-pipe synthetic files, individual ones of the at least one other application reading from the multi-pipe synthetic file receive complete messages in a round robin fashion from the master application without fragmented messages being intermixed.

9. The method of claim 1, where the master application changes the multi-pipe synthetic file system name of the multi-pipe synthetic file by writing to the header-control block of the multi-pipe synthetic file and wherein the master application clears the specification of the multi-pipe operation of the multi-pipe synthetic file by writing to the header-control block of the multi-pipe synthetic file.

10. The method of claim 1, wherein: all of the at least one other application opening the multi-pipe synthetic file are blocked until a first predetermined number of the at least one other application are blocked as specified by the master application in the header-control block written to the multi-pipe synthetic file by the master application, all of the at least one other application reading the multi-pipe synthetic file are blocked until a second predetermined number of the at least one other application reading the multi-pipe synthetic file are blocked as specified by the master application in the header-control block written to the multi-pipe synthetic file by the master application, all of the at least one other application writing the multi-pipe synthetic file are blocked until a third predetermined number of the at least one other application writing the multi-pipe synthetic file are blocked as specified by the master application in the header-control block written to the multi-pipe synthetic file by the master application, the master application writing a control message to the multi-pipe file synthetic file is blocked until a fourth predetermined number of the at least one other application have closed the multi-pipe synthetic file as specified by the master application in the header-control block written to the multi-pipe synthetic file by the master application, and the master application writes a reduction control message to the multi-pipe synthetic file, a subsequent read on the multi-pipe synthetic file by the master application will block until all of the at least one other applications which have opened the multi-pipe synthetic file for writing to the multi-pipe synthetic file write a value to the multi-pipe synthetic file forming a set of values and the MULTI-PIPE file system module reduces the set of values and returns a result of the reduction to the master application and wherein all of the at least one other applications receive the result of the reduction from the MULTI-PIPE file system module."

Or here:

    http://www.google.com/patents/US8380765

Anyone got a nice summary?

Do you think this should be patentable?

I think there's a nice case of posterior-prior art.

It's a parody, with the author providing prior art vs. the patent in question.

s/parity/parody/ perhaps?

What do you think should be patentable -- the practice of being extremely sarcastic about IBM filing a patent on something that someone else invented and published a few years earlier? Or multi-pipes?

this guy is a looper :)

We need a generic pipe for internet services. Yes there is IFTTT and open-source clones, but they are not generically useful like the unix pipe on the command line.

    curl http://example.com/some/api/resource/json | sort | uniq | whatever?

what good is it going to do to sort JSON? You need something like http://stedolan.github.com/jq/

jq - great tip - thanks for that!

netcat comes pretty close...

tcputils from Debian does precisely that, allowing you to extend a UNIX pipe over a TCP connection. It's not generally available as a package in other distros, but you can easily grab the source from Debian and compile it yourself.

Consequently, tarballing to stdout and piping over tcputils is by far the fastest and most efficient way to send a large number of files over a network.

Bash actually supports /dev/tcp/HOSTNAME/PORT for this kind of thing. Ironically, that's disabled in the Debian bash package.

Reply on news.ycombinator.com