Palantir: an operating system for data analysis
blog.palantirtech.com
blog.palantirtech.com
I can see why the title might irk you, but if you read the post, we're not claiming that it's another operating system to list besides Windows, Unix, OS X, and VMS, but rather that we've built an abstraction that has the same power, robustness, etc. for analytic tasks that an operating system does for simple processes.
The history of computing is one of building abstraction upon abstraction -- it's not like we built the OS and we're done.
The post is attempt to put into words an explanation of what it is that we do that other engineers can understand. It's an attempt to explain by analogy.
Nothing in there is being used as a buzzword. We build actual ontologies. Our system is an actual abstraction on top the data. It's operation follows the same pattern as an operating system.
Not only that, I wouldn't call 'abstraction' or 'operating system' a buzzword. (I'll grant you that 'ontology' is bit overused at times, but we don't use it for buzzword compliance, we use it because it actually describes the data models that we build for each instance. But you know, even 'synergy' means something and can be used in a way that's correct?)
The picture of the internet is in the section referencing how the computer operating system has brought us into the information age -- hardly a a non-sequitur. The rise of computing power and the labor savings that good abstractions like the operating system gives us have actually shaped our modern world.
I'm sorry you think the piece is fluffy. I'm a software engineer who helped build the system. I interact with operating systems every day. I ran the piece by a whole slew of engineers in the company to make sure that I wasn't off my rocker in what I was describing, from the analogy about operating systems to why our system has incredible potential for the future. It's not gratuitous wanking, I promise you.
(Maybe the title just needs to be softened? Make the analogy more clear rather than making it seem like we're declaring our system to be a 'operating system' in the classic sense of the phrase?)
I think the problem I had was the piece read to me like it was arguing an equivalence not analogy.
>We build actual ontologies. Our system is an actual abstraction on top the data.
Pretty much all software does this. Perhaps implicitly, but it all does on some level. For example, edlin uses an ontology composed of concepts like a byte, a file, a line, a file handle, standard i/o, character codes to characters, etc. It abstracts the data from a bit stream into characters and characters into a line and lines into files, etc. Is edlin an "operating system" for editing files? I don't think anybody would argue that was case.
The same with Palantir, it's a software application (or set of applications) that succeeds in using an ontology for object and relationship semantics, it abstracts away the data handling drudgery, and automates lots of the search and correlation work. But make no mistake, it's an application.
The analogy I'd draw would be to claims of Google building a web OS (discounting Chrome OS which is still vaporware). http://googlesystem.blogspot.com/ Google is building a lot of interesting and useful apps, but the apps taken together != an Operating system. In this case "Operating System" has become an unfortunate buzzword.
Palantir is a particularly interesting application, full of interesting and compelling capabilities, but Palantir's code does not sit in BIOS at the CPUs entry point memory address, booting up upon power-on. It's not handling and abstracting raw input data from hardware, carefully monitoring various interrupts into mouse movement and other I/O interaction, it's not handling task scheduling or memory management, or has a kernel.
Like any app, it's relying on those services as provided by its OS.
Abstraction likewise is abused as a buzzword and to imply "simplicity". "Oh, you don't have to know assembly, you can use Java, it uses abstraction!" or "you don't have to know how to use a command-line, just use Windows, it's an abstraction!" But Palantir's abstraction on its data, while notable and interesting, does not make it an OS. And user pattern don't make something an OS. Even if you can write different components for Palantir using a common API (like your mapping component), that doesn't make Palantir an OS, it just makes it an application with different "stuff" in it. Is MS-Excel an OS because I can use it to balance my checkbook, as a lightweight database, parse data and play games? http://www.excelgames.org/
One fortunate thing is that Palantir really does use a proper ontology, so claims to such a thing make more sense than clams that edlin does (or heck, I've noticed Analyst's Notebook's marketing copy has stated to abuse this word also).
I mean this constructively, you guys make cool stuff. But it sometimes becomes possible to become too enamored in the revelry of your own creation, to think of it as a mini-alternate universe with a different set of natural laws you can bend to your will ;) I know I've done it. I'm happy for the willingness to engage.
To see the interview, it seems you'll have to go to the front page, charlierose.com, click the 'archive' tab on the flash interface and scroll down to 08/11/09.
-- To charlierose.com site devs, linkable content - good.