Steve Yegge and Grok
bsumm.net
bsumm.net
For many years I made a living working on static analysis tools and that included a lot of compiler construction stuff. Unfortunately not all languages are equally amenable to static analysis.
Take C++ for example. On one hand C++ is statically typed and that helps in static analysis and refactoring. On the other hand, its semantic is so horribly complicated that it makes almost impossible to write static analyzers for it and useful refactoring tools for C++ are almost non-existent. Pointer aliasing and macroprocessor just add insult to injury.
Python on the other hand is very clean language, no macroprocessor, but it is dynamically typed which makes reasoning about code very difficult.
Java (and after it, C#) managed to strike very nice balance, that is why tooling ecosystems for these languages are very good. The road to good tools starts with carefully designed language.
I think nobody really understands the pitfalls of language tooling better than the Jetbrains guys.
But then almost all the fights he fights everyday are precisely against these multiple techniques that make code analysis, even when simply greping for the place where a thing has been set, a nightmare. (Just imagine you want to know why ab == 2, and it is so because in another line of another file you have eval("a" + x + " = 2"), and because of another remote line of code x happens to equal "b".)
So, we can be optimistic and hope Grok will work. But another possibility is that Yegge's sweet-and-sour rant is the sign that it is unbearable for him to see his "software political belief" to be the cause of the failure of his gigantic project.
The fact that he is talking publicly about an internal project, apparently without authorisation, may mean he is looking for the emergency exit sign over the door...
From everything said publicly it seems like "grok" isn't a secret for Google, the team members just haven't talked about it much because it is so likely to fail or at least fall short of its goals given how ambitious it is.
The fact that the project wouldn't be explicitly secret is perfectly believable, just look at Go or Dart which have been developed fully in the open. Like those projects, grok seems like the sort of thing that could potentially help Google and everyone but isn't seen as a competitive advantage for advertising/social, so it is perfectly reasonable to believe they aren't trying to keep it a secret.
Whether it will work or not, this is a truly audacious and remarkable vision.
For any given C++ refactoring (rename method + swap args), propagate that refactoring automatically to every caller of the refactored function, such that upgrading to a new version of the library is automatically refactored correctly.
Parse this Python function and emit it as C, for optimization. Parse this C function and emit it as Python, for readability.
Cross-layer data-flow analysis - so you could track that a lock that's taken out in Java and released in C++ code would never produce deadlocks.
Isn't this similar to what Microsoft is trying to achieve with Project Roslyn [1]? Was someone at Microsoft inspired by Yegge's vision and decided to implement it for the .Net Common Language Runtime?
[1] http://www.microsoft.com/en-us/download/details.aspx?id=2774...
[1] http://blogs.msdn.com/b/ericlippert/archive/2012/06/08/persi...
I feel a bit obligated to point out that co-incidental invention/discovery is very possible, especially when trying to solve a problem large numbers of smart people would have exposure to.
I with he were more knowledgeable on IDE's (IDEA, Eclipse, Visual Studio, XCode) so he had a better idea of what modern developers expect from their tools.
I'm hoping Grok will succeed, but I wouldn't be surprised if in the coming year, it gets completely canceled.
Also worth noting: he is pretty knowledgeable about IDEs too. Why do you think he isn't?
Finally, I wouldn't count on Grok being canceled.
"IDE as a service" in the style of Grok (or Ensime for something currently available) puts Emacs on a level playing field with any IDE.
I am a (sometimes reluctant) polyglot programmer. Sticking with a single IDE product line (I use IntelliJ, RubyMine and PyCharm for some consistency across Clojure, Java, Ruby, and Python development) helps a lot. Eclipse is another great alternative for a unified development platform.
At SAIC in the late 1980s, we had a small team put together for a job - we were all Lisp hackers and we had a great dev community built around sharing Emacs configurations. A lot of fun but our adhoc tooling would not scale up to a Google or Facebook size engineering infrastructure.
And I'm with you on "hacker". I don't have a problem with the word itself, but its so overused in these circles to the point that its irritating every time I see someone try to stuff yet another everyday activity under the hacker term.
It's at that point that you regex it to something else. [0]
I won't stop using hacker; because it has history. But I do laugh when I see it get applied to everything, it's sad, in a humorous sort of way.
At least the IT community hasn't ruined wizard yet.
(Actually the above isn't what "grok" originally meant, but I think it means that now.)
But water was so tied up to everything in their culture that what it really meant was virtually anything. In particular it meant to really "get" what something meant, or what someone else was saying.
---
"Now take this one word: 'grok.' Its literal meaning, one which I suspect goes back to the origin of the Martian race as thinking, speaking creatures—and which throws light on their whole 'map'—is quite easy. 'Grok' means 'to drink.' "
"Huh?" said Jubal. "But Mike never says 'grok' when he's just talking about drinking. He—"
"Just a moment." Mahmoud spoke to Mike in Martian.
Mike looked faintly surprised and said, " 'Grok' is drink," and dropped the matter.
"But Mike would also have agreed," Mahmoud went on, "if I had named a hundred other English words, words which represent what we think of as different concepts, even pairs of antithetical concepts. And 'grok' means all of these, depending on how you use it. It means 'fear,' it means 'love,' it means 'hate'—proper hate, for by the Martian 'map' you cannot possibly hate anything unless you grok it completely, understand it so thoroughly that you merge with it and it merges with you—then and only then can you hate it. By hating yourself. But this also implies, by necessity, that you love it, too, and cherish it and would not have it otherwise. Then you can hate—and (I think) that Martian hate is an emotion so black that the nearest human equivalent could only be called a mild distaste."
Mahmoud screwed up his face. "It means 'identically equal' in the mathematical sense. The human cliché, 'This hurts me worse than it does you' has a Martian flavor to it, if only a trace. The Martians seem to know instinctively what we learned painfully from modern physics, that the observer interacts with the observed through the process of observation. 'Grok' means to understand so thoroughly that the observer becomes a part of the process being observed—to merge, to blend, to intermarry, to lose personal identity in group experience. It means almost everything that we mean by religion, philosophy, and science—and it means as little to us as color means to a blind man."
It's broad in the human (as opposed to Heinlein's Martian) sense because we engage with religion, philosophy, and science in a largely intellectual way. Yes, even wingnut fundamentalists. They're all highly linguistic things; most of our more mystical traditions have fallen by the wayside for one reason or another.
In the Martian sense, that mystical perspective is far more useful: it unifies their physical reality of transcendent Old Ones and life that can be grown through singing and so on. The presence of a single word signals that they've collapsed all of these things down to a simple concept; the lack of a human analog simply says we haven't.
So to make a long thing short, the word cuts to the heart of a conceptual area that we can but dance around with all of our words and qualifications and half-understandings in the fields of religion, philosophy, and science. The desire to know divinity, to know oneself, to know the universe: these can be streamlined into a single concept, to grok.
Thus why Heinlein goes on to have Mike explain his pantheistic "God is that which groks". The closest English word I know of is not "understand" but rather "suffuse" or "permeate". To me, the reason the word survives the novel as a mere stronger version of "understand" is because most people hand-waved the novel's walk-through of religion as a boisterous critique rather than as a considered and valid approach to life, the universe, and everything.
Yet you do not seem to mind the word "geek".
You know when you read or hear something that is contradictory, or something that is just painfully wrong, and you can almost feel your psyche twinge as it recognizes the error? I get that feeling every time I see this word. Perhaps that's just me.
EDIT: Another article on the analyzer by this author, in English. http://yinwang0.wordpress.com/2010/09/12/pysonar/
EDIT: Found another gem from the same author: http://yinwang0.wordpress.com/2012/04/09/reinvent-y/
> Today I found that I forgot how to derive the defintion of the Y combinator. I learned it several years ago from an online article, but now the search term “Y combinator” gives me all the news about startups (sigh).
Given N langauges and M editors / IDEs, total toolchain effort is N x M… Any toolchain support for this number of systems is non-trivial. [SNIP] How do you solve matrix problems like this? Use a hub and spoke model.
Indeed, every problem in computer science can be solved by yet another layer of indirection.
(and no, not putting every imaginable layer of indirection in there already is not flunking CS 101, like the author suggests. I hope he never becomes an architect somewhere).
The usual dangers of this extra layer of indirection are performance and configuration complexity. In fact, platforms like Eclipse have added many such layers already, and we can feel that (sluggish UI performance, conflicting plugins, random crashes).
I really hope Grok will work as well in practice as it could in theory.
We can just look at how hard it is to compare programming languages through common benchmarks is to do to realize what a bad idea this is.
Making the infrastructure to understand a programming language shareable between all the ways to edit/refactor/analyze code in that language is far from dumb. It's something Eclipse/IntelliJ/VisualStudio should have done a long long time ago. Heck I once attempted to pick apart the tooling in Eclipse into something I could use headless without booting the entire thing and the code was so tightly integrated into their GUI I gave up in disgust. I mean this is architecture 101. Clear API's and division of labor should have been on everyone's mind but instead the whole thing is a pile of spaghetti code.
Even something as simple as a CLI interface to your IDE's language support features so I could integrate them into editor of my choice would have been smart. But no one did that. I work with 4 different languages minimum every day at work. I'm not going to context switch my editor/IDE for each one of them. For this reason I'm cheering Yegge on like mad.
(Full disclosure: I work at google same as Yegge. I've had these opinions for longer than I've worked there though.)
Even just one. The Python RefactoringBrowser takes tends of seconds just to make a mess of the whole thing. It's a nice effort, but it fails on both reliability and speed.
At any level above the trivial, the code we write (or work with) is very often not a direct set of instructions for the machine to carry out. Here, I don't simply mean a high-level language is not machine code or assembly. I think it's safe to assume that we all get that much at least.
Rather, it's a lot like everyday conversation, but with Lewis Carrol's Humpty Dumpty character. There will be common (and less-than-common) cultural references, along with a whole bunch of private definitions for things. Actually understanding the conversation means not only having those cultural references in common[1], but also having access to the private definitions[2] (such as "glory" meaning "a nice knock-down argument").
So the text on the screen (or in the file) is not the actual code, or even a complete description of the code, that the machine will run. It's only a sort of short-hand representation, a language we created along the way to talk about the instructions (even if, at the basic level of the language facilities, the operators and keywords are directly translatable). Simply working with the text content, then, as one would do with a simple parser, or as a human programmer would do with a single text file in isolation, is insufficient; the person or tool would have to have a much deeper understanding of the implications of the text in order to make any real sense of it.[3] For a tool to offer useful help (beyond the simple matter of altering text), it has to be able to "understand" what the text means, and that , in turn, means looking beyond the simple arrangement of text characters.
[1] It might help to think about the Darmok episode of Start Trek TNG. Library calls are very much like the metaphorical language of the Tamarians, in that they make perfect sense if we know the "story" being referenced, but not knowing means either the reference is meaningless or can even be misleading.
[2] That applies as much to preprocessing directives like constant definitions as it does to things like function/method calls.
[3] We do this is natural languages as well. Every time you've run across a cant, an argot or a jargon specific to one field of endeavor, you either understand it because you are familiar with the displaced vocabulary, or you're left in that uncomfortable state of knowing (or thinking you know) what most of the words mean, but failing entirely to understand what's being said.