IDEas: Tools for Coders
omegaortega.com
omegaortega.com
Now for the latter - that is great, and everyone has different strengths and weaknesses. And that is just a UX issue.
For the former however, I don't know why I need to use different IDEs for different languages, or even different IDEs for different transforms to the same codebase in the same language (again, for the automated bits).
It would be so much better for the world if there was some common protocol between user environment and code transformer. That way I could buy some of JetBrains awesome plugins and have my trusty Vim or Emacs environments use them. I could just keep using IntelliJ with MS' plugins for C# (and resharper of course). And so on.
For stuff that really needs a special UI, we are in the age of the web browser. Provide a localhost server in your tool and make the reports etc live in a browser window, big GUI IDEs can provide a webkit that can link back to their own editors, but text editor users could still get the useful output.
This "everyone does their own thing" without even pretending interoperability is getting old. We've been doing this coding thing for a little while now - we should be better at tooling that plays nice.
I feel that Software Engineering/Science is lacking a body of knowledge: something which is basically required in other professional industries (architecture, medical, physics, electrical engineering), etc.
Software Engineering/Science is at a disadvantage because there is no limitation placed on us by nature: an EE can't simply change the brightness of the sun to meet a requirement. We can.
To create an industry standard plugin is really saying we need to create a common way to communicate information between systems and sub-systems in software. I feel we have not really figured out the best way to do this yet. Our industry is still young.
I've put a lot of thought (about 6 years now) into a way to standardization communication between software systems and even parts of software. It's a hard problem to crack.
> common protocol between user environment and code transformer
I feel that once we crack the problem of communication between software parts, this will come naturally.
Wow, you can change the brightness of the sun to meet a requirement?
BTW, there are quite a few limitations placed on you by nature: uncomputable numbers, NP-completness, ...
Not really, NP-completness is only important if you want perfect solutions, which aren't necessary. Uncomputable numbers have no usage in real world scenarios.
Finally it is important to note that while a physical turing machine cannot exist due to lack of infinite tape, real machines are also more powerful than turing machines, for instance by having a real time clock and communication abilities.
For these reasons saying what we can and cannot do is quite complicated. For almost every situation possible, the only answer that is negative about capabilities is "it will take a while".
Sure, why not? sun.brightness = sun.brightness * 2.0f;
In fact, a lot of games play with physics to get cool behavior. I can adjust the "constant" G in my game to be something other than 9.8 m/s to make it interesting.
One moment the G vector may be down and then suddenly it flips up. Why not?
HTTP?
It's not perfect, but it seems to be what our industry has settled on as the universal way for pieces of software to talk to each other.
If you are thinking in terms of the payload, JSON seems to be filling that role.
Windows PowerShell communicates with typed objects. COM components and .NET components are a Windows standard way of communicating between software parts. COM has been around for a long time.
Between computers, TCP/IP is a standard way of communicating. But without good documentation of the protocol adding an HTTP layer on top makes it easier to understand. But without good documentation of the protocol RESTful APIs makes it easier to understand. But without good documentation of the protocol WADL...
However I do agree about HTTP. Because I imagine there are more devs that understand HTTP than higher or lower protocols.
Parameters are, for basically all languages, the means by which information is communicated between sub-systems.
The thing is, it is really hard to standardize the communication between sub-systems because sub-routines can vary widely in how they are defined (the possible combination of parameters and types is quite high). In fact, you can end up with thousands of different sub-routines: even in the same program. Each one unique and thus "non-standard".
This is the problem we need to crack. How do we create frameworks that don't lead to thousands of specialized interfaces: each one making the framework that much harder to use.
> api`s for systems that negotiate a communication protocol dynamically
I think the first step is to find a very easy way to describe communication between sub-routines/processes. Once people can hook things up and compose the interaction of software with foreign/unknown systems (all in real time while the software is running) then we can get systems to start doing it dynamically (run on sentence but ya).
Just what we need - now there will be N+1 incompatible protocols!
(shamelessly stolen from xkcd)
If you look at Xcode, it saw a significant improvement when it started using clang to build its indexes. That is purely a C, C++ and Objective-C front end to LLVM.
Xcode basically requires a significant portion of a compiler to provide a decent coding environment.
What you are suggesting is almost to create a compiler "protocol" that can handle all languages. I am not sure how much value you would get from such a protocol.
That it's a command-line application also means it's "start up and throw away" which is easier to get working for most users than daemon based stuff. Typically once it's configured you don't touch it again. This allows people to build on each other's work which can lead to higher levels of abstraction and more human-oriented tooling.
The alternative is everybody having to write Haskell parsers and analysis tools over and over again before getting to the parts that actually make their editor/IDE different. The Clojure community has a similar ethos of tool sharing and reuse.
End users do not generally invoke these command line tools themselves, their editors/IDEs integrate them and offer the native trimmings the users would expect.
Why are you being so hostile? You're not making me want to share anything I know. I'm talking about the kinds of ecosystems that let people like you (that work on live-editing environments) focus on the parts that matter to you and you're behaving like a prick.
We were talking about how it would be nice to have a standard for interoperability IDE plugins, and you step in by saying Haskell gets it right by...not doing it at all. Unix was a great C dev environment, you had ed (later vi and emacs) for editing, and a bunch of command line tools for doing everything else. We tried to evolve that interoperability into Unix, but users wanted more tightly integrated tooling (hence the term "IDE"), and the unix shell model no longer cut it. Going back to the past won't help us much, especially when our tooling is increasingly interactive and not batch, where the former are very hard to write and standardize compared to the latter!
I'm sorry if my tone was hostile sounding. But as someone who works on IDEs full time, this rosy-tinted glasses belief that we were doing it the right way 30 years ago quickly annoys me.
I frankly don't buy the entire interop premise, the IDE is the language! How much do C and Haskell share in common? Not much beyond the foundation level, and people aren't really complaining about that, so why worry if IntelliJ doesn't share an architecture with Eclipse? Haskell has no popular IDE as far as I can tell; the community just isn't into them (though some exist, like Leksah), which is quite odd to me, as they have all that static type information sitting around!
My point was that super fancy IDEs and plain old text editors alike could reuse the same tooling and not have to reimplement things like language parsing, analysis, and refactoring tooling.
Libraries like haskell-suite and command-line tooling like ghc-mod have succeeded in that. That ghc-mod happens to be a command-line application is utterly irrelevant to the end-user unless they want to go poking around.
And uh, you know, some people do sometimes.
Want to go poking around that is. Some people like owning and operating their tools.
I'm not saying "Hail Unix", I'm saying, "Hail IDE, Emacs, and Vim hackers not needing to write the same libraries over and over". The specific implementation is irrelevant.
Also if you refuse, on account of bias, to understand why people might find such a design (command-line application invoked from different programming environments to augment functionality) then you are damning yourself to be ignorant of how other people think and work.
This reminds a lot of the hatred some people have for the Twilight series. It's not a product designed to appeal to somebody like me, but clearly there's something to it because it has resonated with millions of people.
Is it really worth it to say "ugh yuck" and not even pause to reflect on what it is about this thing you dislike (Unix-style tooling) that so many other people have stuck with for decades? Are you feeling insecure because the average hacker these days is using a Mac or Linux machine?
>I frankly don't buy the entire interop premise,
I guess you can be that way if you want, but Emacs, Vim, Sublime Text and Eclipse all have Scion integration. Scion isn't quite like ghc-mod, note that I was talking about a general attitude that all IDE tooling should be reusable across multiple environments in Haskell, not just command-line applications.
Scion in this case is:
https://github.com/nominolo/scion
Scion is a Haskell library that aims to implement those parts of a Haskell IDE which are independent of the particular front-end. Scion is based on the GHC API and Cabal. It provides both a Haskell API and a server for non-Haskell clients such as Emacs and Vim.
You'll note that it's not actually a command-line app like ghc-mod. I use ghc-mod because it's "simple" and easily inspectable/dumpable for it's output at the terminal.
I know Emacs and vim users to both use ghc-mod.
buildwrapper mostly replaced Scion and is used by Leksah, Yi, EclipseFP (all IDEs). Buildwrapper provides its functionality via JSON.
Leksah, Yi, and EclipseFP are the primary IDEs Haskell users use, IF they're not using FPComplete's dilly which is web-based and is using the same Haskell IDE tooling as everybody else.
> Haskell has no popular IDE as far as I can tell
I don't really know that "IDEs are popular in Haskell"
but "Haskell has popular IDEs" is true.
Further, while I can't speak for vim users, the way most advanced Emacs users interact with their programming languages, especially Haskell, is an interactive hybrid IDE environment usually built on a REPL.
> How much do C and Haskell share in common?
The fuck are you talking about? I've only been talking about tool-sharing in the Haskell community this whole time.
> so why worry if IntelliJ doesn't share an architecture with Eclipse?
It's troubling because it means labor is wasted writing the same tooling (IDE) for the same language (Java). Hackers should be bothered when their colleagues are writing redundant code.
Maybe that's not an ethos at Microsoft or MSR, but it is in the various open source communities I've been in.
>But as someone who works on IDEs full time, this rosy-tinted glasses belief that we were doing it the right way 30 years ago quickly annoys me.
You clearly have not used Emacs or any of the Haskell tooling I use.
While some Emacs users still use things like etags/ctags, I do not. I have language-aware search/go-to-definition faculties just like any IDE in Emacs. The same goes for most other things I give a damn about. We have the same faculties as anybody else, we simply refuse to give up control over our tools because we don't buy into Microsoft's dystopian digital neo-feudalism.
I don't let that difference of vision cause me to act like a fucking prick to a perfect stranger on HN though. You are representing Microsoft and MSR - and all I've gotten from this experience is that Microsoft employees are totally out of touch with the work anybody else is doing. Unprovocatedly unpleasant to boot.
>...has no popular IDE as far as I can tell; the community just isn't into them (though some exist, like Leksah), which is quite odd to me, as they have all that static type information sitting around!
Haskell users aggressively leverage the types for secondary purposes, you're just not a Haskell user and are therefore ignorant to it. You're still wrong about Haskell users not using IDE-esque workflows, you just don't understand how that can manifest in a different form than being a Visual Studio customer.
>I frankly don't buy the entire interop premise,
I've conclusively demonstrated aggressive tool-sharing across different toolsets in Haskell, even with most people upgrading from Scion to buildwrapper when it became mature. It's rare for open source communities to keep up with each other that well.
> Unix was a great C dev environment, you had ed (later vi and emacs)
The proximity of mentioning ed and vi/emacs in the same sentence is not as clever as you think it is and really just shows how ignorant you are to how vimmers and Emacs users work these days.
My Emacs environment of today would be utterly alien to myself-10-years-ago. Or even 3-5 years ago. Or even 2 years ago. A lot of evolution can happen when you've been improving the same tooling over and over for multiple decades.
Emacs is nearly as old as Bill Gates's DUI arrest (1976 and '75). Do you really think people are using it the same way now as they were then?
I mean, for one thing, it starts up quickly now >:)
You should really try immersing yourself in how Emacs users work these days, you really have no idea at all. It could use a lot of work, so could everything, and there are some things that some IDEs for some languages will do better...but it bears more resemblance to those IDEs than to ed of all things.
The interactive REPL oriented workflow is something IDEs still haven't gotten right though. Pity that.
Parsing and analysis are great examples: if you accept that batch is the best that can be done, they are completely reusable in lots of different contexts. As soon as you want something interactive, standardization and integration become much more tricky (I know directly from experience in writing interactive parsing and analysis tools).
> Also if you refuse, on account of bias, to understand why people might find such a design (command-line application invoked from different programming environments to augment functionality) then you are damning yourself to be ignorant of how other people think and work.
My early 90s self completely agrees with you. My current self is much more interested in non-batch interactive tooling, the mainstream is also.
> Maybe that's not an ethos at Microsoft or MSR, but it is in the various open source communities I've been in.
Along with much of the OO PL community, please don't forget (though we divide into static easy-to-tool languages and dynamic hard-to-tool languages).
> You clearly have not used Emacs or any of the Haskell tooling I use.
I was a heavy emacs user in the 90s (from about 92 to 2003 or so). It was a fine text editor, I would have never called it an IDE though.
> My Emacs environment of today would be utterly alien to myself-10-years-ago. Or even 3-5 years ago. Or even 2 years ago. A lot of evolution can happen when you've been improving the same tooling over and over for multiple decades.
Please educate me. How about your own workflow: does your emacs IDE provide code assist (I think Haskell Eclipse does right?), does it provide a nice debugger?
> You should really try immersing yourself in how Emacs users work these days, you really have no idea at all.
I used Emacs for 10 years in the 90s, early 00s, I don't miss it at all. Language-aware editing with code completion has been a huge win in PL and I don't see us going back. But these kinds of tools are complicated and not as interoperable as the old text pipe-based batch tooling was in the past.
And Visual Studio is in fact pretty handy.
Anyway, Haskell has an Eclipse plugin. EclipseFP, which integrates with some standalone programs.
But Sean's implicit claim that Power she'll style object passing is good...ew.
Bizarre.
ghc-mod's command line application has a "to lisp list" argument so that Emacs can parse the output data natively. I found that rather cute.
I know who Sean is, I still found the hostility unnecessary.
These ideas are all basically the first thing anyone comes up with when dreaming up a new IDE.
The big problem of IDEs is that they're a lot of work to make and polish very well. Most developer environments are either in a terminal, to get rid of the hard UI stuff, are an editor with good scriptability, to get rid of the hard IDE stuff, or are backed by huge slow moving corporations.
The one really big innovation that is going to blow life into innovative IDE development is scriptable GUI, aka Javascript+HTML. LightTable isn't the first and isn't going to be the last of awesome new IDE environments for which implementing ideas like this is going to be a piece of cake.
So hold on to your hats, we're definitely going there.
Indeed.
Use of icons was one of the topics in an April fools' joke from Bjarne Stroustrup [1, 2].
[1] http://www.stroustrup.com/whitespace98.pdf
[2] https://www.cct.lsu.edu/~hkaiser/spring_2012/exercises/week6...
This has been a neat-o feature which slowly became nice-to-have, but it is fast becoming a requirement. For example, the JetBrains products (Idea, Resharper, etc.), and also what Google is doing with clang to help them manage their large C/C++ codebase.
We already compile/interpret our languages. So we need to open up that "machine understanding" to programmatic access, which IDE's and other tools can then use to do all kinds of nice things.
But it has been on beta for almost 3 years I think. I wonder if it will ever go forward.
In an ideal project, of course, your code be so beautiful--so loosely coupled, so free of side effects--that you could in fact focus on a function in perfect isolation. But few of us are lucky enough to live in such a world.
I think that for now the best way for me, to continue developing my product, is to continue working isolated from the world. At this stage is probably still early, but if anyone is interested to know more, and think that can provide something, I am open to considering possible partnership. Unfortunately at this point, I am not willing to share online any of the progress done.
"Task context management in Mylyn is based on the idea that your interaction with a system can be transformed into a degree-of-interest model, where each element in the system is weighted according to its relevance to the task at hand. This degree-of-interest weighting forms the context of all the elements relevant to the task. The task context can then be used to focus the UI by highlighting the most important elements, filtering the unimportant ones, and allowing you to perform operations on elements of interest "
I use IntelliJ (or pycharm) myself but some people swear by Mylyn. Not having used Eclipse myself, the screenshot does't really tell much: http://www.eclipse.org/mylyn/images/mylyn-3.1-screenshot.png
This article can be boiled down to: Project idea: better IDEs.
It's in the same vein as comments like: Business idea: nuclear fusion.
They just aren't productive / valuable.
If say Henry Ford said: Project idea: Use energy from chambered oil combustion with pistons to convert direction of force into a spinning axis to create an horseless vehicle. (I'm obviously not a car expert) Then we would be talking about a valuable idea.
My viewpoint is simple: Ideas are not innately valuable, of all the ideas in the world, only a subset of them are valuable. Obviously you don't know what current ideas will be valuable in the future, but I have a heuristic: for an idea to be valuable, it must be a specific (implementation details included (batteries included)) idea.
I do agree that implementers are good though. As I mention in the post, I do have one more idea that I didn't share that I am going to build. I'll comment here again when I'm ready to announce it.
Actually, if that's a route worth taking I'd rather see LaTex than HTML.
Or just Markdown.
Computer programmers are one of the most dismal professionals on that planet. How many so called programmers/coders can even write a simple parser or a compiler? Less than one percent I suspect. I should know because I am one of them. After dabbling away with Aho and Ullman somewhere in the 80s or 90s I haven't approached the subject since.
Ever since Smalltalk came out it has been downhill all the way, and it is only now a lot of the current IDEs have caught up with it and in the last few years have begun to extend the concept.
I believe that:
1. Learning to understand and write parsers and interpreters should be one of the first things taught in computer science even before the whole gamut of data structures and algorithms have been studied.
2. Being able to develop IDEs should part of every computer science course. They may even be developed to help illustrate the data structures and algorithms mentioned above. Unlike most professions computer scientists are the ones most capable of building the tools for creating, analyzing and illustrating their work, so why are they the ones that as far as we can tell use the shittiest tools available to most professions? Honestly I blame Sun and IBM for this. The moment Java came out everything went down hill. Then the web came, then 'scripting' , then everybody decided that they could and should 'build' everything using Notepad, VIM or Textmate.
3. The tools are there, Smalltalk guys have been using and developing their own tools for ages, only the targetted graphical environments have been split up and the OS owners, Microsoft in particular keeps blowing hot and cold as to which one system developers should target. IDE and graphical tools development is so interactive that the idea of using traditional compiled languages to develop them is simply absurd unless you use very fast compilers and computers. It is time they went 3D as well may be using some Blender, Unity3D or some other gaming toolkit to bring them up to date. 2D text and graphics are boring.
4. The bottom line is IDEs should be something developers ought to create or collaborate to create themselves. The big corporations should stay out of that, ie the decisions shouldn't be commercial decisions. It seems that a sad immaturity of computer programming as a profession has led to development tools being dictated by clueless profit seeking corporate bosses. They should be the decisions of developers guilds which should not be linked with corporate interests of any kind. You know what you want, just do it yourselves.
5. What else? Yeah, get your parsing skills together, and don't develop it in Javascript unless you plan to develop your own version of Typescript or Dart later on when subtle bugs begin to creep into your cherished IDE.
PS. Sorry for the trolling at the top. I just couldn't help myself.
As a result, every developer wouldn't need to know how to write a parser/compiler, but rather would need to master the usage of the API the compiler-as-a-service offers. (For example, imagine JetBrains Idea for Java or Resharper for C# as a suite of open source, api-driven libraries).
Then, we could have IDE's with all kinds of features. In addition, we could write programs that transformed or modified our code bases.
This lets the "hard work" of compiler/parser building be centralized to a few developers, but spreads the benefits around to most everyone.
Since this is already happening in some languages (JetBrains, what Google is doing with clang for C/C++, and apparently Rosyln from Microsoft), it seems like only a matter of time before its power becomes apparent and it becomes a requirement for a language to be taken seriously.
I see no reason why Python and Ruby couldn't expose the compiler's AST as a service as well (obviously it would be limited by metaprogramming and dynamic typing ... perhaps in those cases we'd need "interpreter as a service" ... hmmm).
Oh no you are not wriggling out of that one. Every developer should know how to write a parser and a compiler, not full fledged ones, just enough to implement a small language. That is in essence what computing is about, composing and instantiating formulae, with a full grasp of its semantics. To call your self a computing professional without knowing about parsing is like calling yourself a medical professional without knowing anatomy. This can only happen in the world of computer programming. It is the very essence of what we do, and how such a situation exists is an indictment of the computing profession, its educationists and educators.
It is the very soul of computing.
We might get away with it in the eyes of the public who don't know better because we produce the results they want, internally we shouldn't kid ourselves.
What I really meant to say was, everyone would not need to actually write a compiler/parser (regardless of whether they know how to or not). To achieve the goal of having a bunch of nice new IDE features, we don't need everyone writing parsers and compilers. We can achieve that goal by having a few developers write really good compiler-as-a-service offerings.