Beyond Light Table
chris-granger.com
chris-granger.com
I mean Light Table is a great IDE but it just needs so much done. It feels like 95% complete in every corner. The vim keybindings are buggy, Python plugin lacks proper virtualenv/custom python support, performance is not quite there either... I forget all the issues I encountered back when I used it but there has been very few releases since and it's super concerning to just see what was once an extremely active project slowly go in maintenance mode.
Please, quell my fears. You guys put together an amazing Kickstarter and used to put out so much in terms of updates and feature updates; I don't see that anymore.
With all that said, best of luck with Eve, sounds awesome.
... Edit: I think these graphs are pretty telling. https://github.com/LightTable/LightTable/graphs/code-frequen...
This is a simple truth.
For my part I think they've delivered on the spirit of their Kickstarter. They released a very usable editor that does some amazing shit. It doesn't have every feature in the Kickstarter video, but that's because reality kicked in and they had to prioritize, and found that some of the features didn't work out as well when put to other users.
LT fills multiple spaces in the area of IDEs:
- An extremely-programmable IDE (there's no other one like it)
- A lightweight generic IDE (there's REALLY no other one like it). I almost wouldn't call it an IDE but it's more than a text editor... it's right in the middle of the two and it's great to have something fill that gap, especially for languages that don't always need full-blown IDE support (like python and clojure)
It'd take a long time to explain the differences but emacs is far from lightweight, it's a beast. LT is very barebones. And LT is written in HTML/CSS/CLJS which makes it stylable and programmable as such, a real advantage over emacs.
Emacs is an IDE, first and foremost, to Emacs Lisp itself. Other Lisps are close. It's an excellent editors for other languages, and a good enough IDE for some languages. See my C/C++ guide in Emacs: http://tuhdo.github.io/c-ide.html.
Fyi, people wrote parsers, chat and email clients in Emacs that LT is still dreaming.
etags -- it's just an nightmare, full stop. It only gets you where you want to go half the time at most. And then because the GUI is so primitive you can't do basic things like click on a symbol and have it jump you to the definition, you're stuck using the keyboard interface designed for work in a terminal.
An etags that actually worked properly is one of the most basic things that could improve my productivity monumentally.
CEDET -- last used about 5 years ago, gave up because on larger files/projects it would just hang my editor for a few seconds as it processed.
Variable/symbol completion -- see etags.
I'm not doing a very good job communicating my dislikes / pain points because I'm not very good at communicating. Please do not respond with a "but these aren't problems because..." because they are problems for me, I just do a poor job of communicating.
I gave up on emacs for Java development for the reasons you cited, but I still use it for REPL style development (Jython, Ruby Motion, XQuery, as a few current examples).
And use GNU Global instead of etags. Finally, keyboard boost productivity many times over the mouse.
Google Chrome, or any modern browser for that matter, has a much better way of doing things. Imagine for a moment that I was browsing and editing code in Google Chrome instead. A symbol would be a link, and when I ctrl+clicked on it, it would open in a new frame. That would be an interface I could get behind.
For completion, it ought to work like the IDE I used back when I was writing Scala. A little tooltip would pop up with completions. You could scroll through them with the mouse or keyboard, and click or ENTER the one you want and it would paste in. Or you could ignore it completely and continue typing. This sort of UI is so much better than popping up a new buffer in the frame. But in Emacs UI, everything is a buffer and there are only three places they can (or usually do) go: in a new frame, in a new window, or in the minibuffer. Only that last one is usable.
GNU global is not a solution. It produces marginally better output than etags, but still requires you to actually run it whenever you make changes to the codebase. Or at least it did back when I tried it as an etags replacement.
There's nothing elite about the keyboard. It's just the keyboard enable higher productivity when it comes to write code.
For completion, it ought to work like the IDE I used back when I was writing Scala. A little tooltip would pop up with completions. You could scroll through them with the mouse or keyboard, and click or ENTER the one you want and it would paste in. Or you could ignore it completely and continue typing. This sort of UI is so much better than popping up a new buffer in the frame. But in Emacs UI, everything is a buffer and there are only three places they can (or usually do) go: in a new frame, in a new window, or in the minibuffer. Only that last one is usable.
In Emacs you have tooltip as well. For example, company-mode shows a completion popup like this: http://tuhdo.github.io/static/c-ide/company-clang-company-gt...
Completion using a window buffer is still vastly superior because it can handle large collections. For example, here is how I filter completion candidates with Helm: http://tuhdo.github.io/static/c-ide/semantic-boost-demo.gif. Helm works even if you have 99999 candidates (not joking) to filter out. Here is how I quickly filter to a file and enter the file in Linux kernel source tree: http://tuhdo.github.io/static/helm_projectile.gif , very quick and simple; in contrast, the file search dialog box in typical IDEs is very complex with many checkboxes and text fields. How do you cope with completion popup with contains thousands of candidates in it?
GNU global is not a solution. It produces marginally better output than etags, but still requires you to actually run it whenever you make changes to the codebase. Or at least it did back when I tried it as an etags replacement.
No, ggtags: https://github.com/leoliu/ggtags or helm-gtags: https://github.com/syohex/emacs-helm-gtags automatically updates your tag database when you save your file. The tags produced by GNU Global is A LOT better than ctags for C/C++.
Maybe you can look at a few demo screenshots in my C/C++ guide: http://tuhdo.github.io/c-ide.html
Any single tool is going to be an incremental improvement (at best) on programming, because the source of the issues is really the model itself. My suspicion is that programming is going to change substantially over the next decade, because while we've been worried about machine learning, rails, and hadoop, the notion of the machine changed out from under us. We're already starting to hit a wall as a result of that - reasoning about highly concurrent systems with the programming models we have now is nearly impossible. If we continue pushing LT for that way of thinking, it's unlikely to make much of a difference. Instead, there's an opportunity to embrace that change and take the ideas of LT to their logical conclusion: what would a stack built for today's machine that is meant to be used by humans look like? We stumbled on a solution as we did more and more user testing of LT and in the interest of self preservation we ran with it.
More practically, though, incremental improvements are hard to sell - not just to VCistan, but to programmers as well. Hell, even orders of magnitude improvements are hard to sell to programmers. We'd have no way of sustaining ourselves if we couldn't find some avenue to eventually turn this into a business - we looked and couldn't really find a reasonably quick way to do that. Matching the millions of man hours that have been poured into vim/emacs and so on is just not in the cards for a couple of guys with not much funding. Plus it seems programmers are increasingly less interested in paying for anything in general, so that didn't make our prospects all that attractive. With Eve there's a much clearer path towards creating a sustainable business that is able to continue towards the ridiculous goal we set out with from the beginning: to empower the world's creators.
We use LT every day. Lots of other people do too. It fills an important part of the ecosystem and we hope it continues to infect other tools and organizations who have significantly greater resources than a couple of guys in SF. But either way we will do our best to make sure that it continues. Unfortunately, though, it just can't be our primary focus - both because I think that would be focusing on the wrong part of the problem and because at the end of the day we have to pay ourselves and there's just no way for us to do that.
I still think that LT is going to slowly peter out, as there isn't enough momentum to dethrone existing IDEs / upcoming actively developed contenders.
That said, I think this level of transparency about where your collective heads are at is great. While some people may still be annoyed (particularly those who donated to the kickstarter) I applaud your willingness to change paths and do what you think needs to be done to succeed.
I'm excited to start playing with Eve.
a) More "traditional" languages aren't going anywhere, at least for the next few decades, however successful Eve is.
b) The idea of really connecting to the running program has a huge amount of potential, even in these languages. Less potential than Eve? Perhaps, but I don't believe for a second that we've hit that wall yet (and I wholly intend on proving that with continued work on the Julia plugin).
Leaving Light Table unfinished would be a travesty, even if it is only an incremental improvement – but it sounds like you'll at least treat it as an important side project, which is something. Best of luck with Eve, anyway.
I fully support your ideas with Eve.
Every time I dig into how Xerox PARC systems worked, my own experience with Smalltalk and Oberon, and then look for what we have, I always think we could have had so much better developer experiences if those technologies had succeeded in the market.
Good luck with your efforts.
A lot of people are interested in the project, would like to contribute to it, and would be happy to take over where you left off. However, this is difficult to do due to lack of documentation and any support from yourself.
You can read the experiences of one person trying to contribute here (https://groups.google.com/forum/#!topic/light-table-discussi...) and here (https://github.com/JonyEpsilon/gorilla-repl/issues/132).
Since you're not going to be actively working on the project, why not involve the community find individuals or an organization that would continue developing and maintaining it.
Surely, this is not too much to ask and it would be a responsible way to transition from the project.
I will certainly not stand in the way of the community and we're more than happy to help maintain.
Simply dropping the code on GitHub and saying that you're glad for people to get involved is not a sufficient level of support at this point. This might've worked when the project was younger and smaller, but it's just too big for people to be able to dive into now. The links I provided are a prime example of the problem.
Since you found the time to make a blog post about Eve, surely you could also make a post stating that you're looking to hand over Light Table and you'd like to help facilitate the transition.
Whenever I've used it I've felt like a small amount of effort in those aspects will allow it to continue to progress in the hands of the plugin community.
I certainly didn't expect the team would be "revisiting some of the foundational ideas of computing" so soon.
LightTable right now is a proof of concept. A very good one, but still — it has a long way to go until it can become an everyday tool. Let's not leave it hanging there. If you continue on this path, you will leave behind a trail of half-baked tools, each one a great concept, but none of them actually useful.
FWIW, I agree with you. This is our path, though, and the funding and the people we've brought on to work with us prove that. We're locked in to Eve and we think it will ultimately have far greater and for more lasting impact than LT ever could.
So the best I can say is that we're all in on Eve (both personally and now as a company) and I think that's very important for this to have a chance to succeed.
I backed Light Table and enjoy it (though it is nowhere near my daily editor) but I'm excited about the shift in focus. Sometimes great ideas do not pan out to great products for a variety of reasons, and it would be fiscal suicide for them to keep the focus on Light Table if the team truly does not believe they will be able to reach the product/market fit they were hoping for. This is business, not charity after all.
To put it another way, Eve sounds like a really interesting solution to a hard problem and I'm really excited that this team is trying to tackle it. Stoked to see what the future brings.
I can understand having ran out of the original kickstarter money, but after enteing Ycombinator I'd expect them to still have resources to avoid being forced to pivot
(trivia/disclaimer: I'm a kickstarter backer and Light Table was my main editor for some months)
It's a company full of smart people backed by smart investors, if there was a way to make LT work at scale I'm sure they would still be working on it.
I was taught that when someone gives you even one dollar in consideration of something that you've promised, that taking the money is tantamount to an oath.
Light Table essentially proves the lie, since it is pretty far from the video. From where I sit, Chris and his team broke their oath and I won't be burned twice.
If Andreesen Horowitz believe that computing can be revolutionized for $2.3M, it's their cash to burn. Not surprised at all that they didn't turn to Kickstarter for funding.
The word is out.
And, while Light Table is a polished little editor, it's hard to call Light Table an IDE without offering an apology. Light Table certainly is not what was promised.
In fact, Light Table was my first experience really being burned by a Kickstarter project. Nice that Chris Granger wants to reimagine computing, but he should finish what he starts.
Who would gift his crew with $2.3M given their track record? Atom, which did not promise an IDE has done more work on it's editor in a shorter time than Chris and crew. Buyer beware!
I was very much interested in LightTable. If I had given money to the kickstarter fund I would be very ticked to find out that those funds were being redirected to a different project (and I would not give funds to those individuals again).
I am also trying to be helpful by pointing this out (not just taking a cheap shot).
Also, my gut reaction to this is "LightTable is effectively dead". Again, not trying to be mean -- just basing this on a long history of following the industry.
All that being said, it does seem that the LightTable folks are trying to be transparent about all of this, and that much is refreshing.
If they are successful in producing something very useful, my hat is off to them. But I have to admit that it sounds kind of "pie in the sky" to me in this announcement. Please feel free to prove me wrong :-D
While that's good to know, I think that the implicit upside gain hoped for from Kickstarter that underlies a lot of decisions to fund through it (especially for software) is that acheiving funding means not just that the thing gets delivered, but that an effort is made to build a sustainable business model is built around it and support the user community. The perception that, having delivered the kickstarted product and spent a couple years on it the team would abandon it and move on to a very different kind of focus (even if in the same broadly drawn domain) understandably reduces the perception that that kind of upside potential is around for a future effort.
Now, one can argue that that kind of upside expectation is somewhat irrational in the first place, of course.
It would be a shame for that to be the case. The amount given was indeed much more than reasonably used up if it supported 3 people for 2.5 years (especially in SF).
Looks like it's time to dive into Emacs...
Like Unix you can't grok all of it (ever), but you won't ever need all of it. Just learn more of it as you go along. See it as a trip :-)
And if you need your Emacs Clojure development environment right now, have someone make a VM for you.
...and customized it enough to have an environment comparable to what other editors ship out of the box
wc -l ~/.emacs.d/init.el
241 /home/dario/.emacs.d/init.el
ls -l ~/.emacs.d/emacs.org
-rw-r--r-- 1 user user 67417 Oct 1 17:01 ~/.emacs.d/emacs.org
The ctrl+spacebar and search way that Light Table works, while not the fastest, was still preferable to holding down multiple modifier keys.
All that being said, the source of my hand pain might actually have to do with my neck and back, as working on those areas has substantially relieved my pain. So perhaps that will no longer be a barrier to emacs usage in a few months. :)
I've never used the defaults[1] for long. First by making my own custom keybindings, then by making keybindings compatible to what most others applications use and a couple of years ago by switching to evil-mode. I already had vi/Vim experience so that was pretty optimal.
[1] Except some really basic ones that I also use in the shell: C-a, C-e, C-r, etc.
and there's also vim-fireplace:
https://github.com/tpope/vim-fireplace
By all means, if you don't want to use emacs then you don't have to!
If it truly is "a great IDE" and "95% complete in every corner", yet it dies because the users (who all happen to be coders!) don't rally around it and contribute that last 5%, what does that mean?
And just because all of the users are devs doesn't mean they're devs who have time to dedicate to building an IDE while they could use something else like KDevelop, PyCharm, Sublime, vim, or hell even VS.
This seems to misunderstand fundamentally what it means to be a professional programmer. Knowing some programming language isn't it, its a mode of thinking and analysis. I think that with programming things get confused in people's minds because there is less of a clear professional distinction between construction and design than in physical engineering, but learning languages and syntax is the trivial part of programming already. Eliminating the hard part of the requirement to become a professional programmer is an AI complete problem, because it means you are getting the computer to do your thinking for you.
Many people who try to make programming better starts out with tools like IDEs or analysis tools, maybe proceeds to actual new languages, and ends up tilting at the windmill of transforming programming by replacing it with something that eliminates the need for professional programmers entirely -- and while we tend to get useful tools out of the other step, that last one is pretty consistently a place where we get vague hopes and pretty demos and, if anything of lasting use, salvaged pieces that end up as analysis, etc., tools for professional programmers.
I'm not saying this isn't interesting or that I don't want to see where it goes, but the description provided of both the goal and how the people promising they can deliver it have "found the way" by "studying the past and revisiting some of the foundational ideas in computing" is something I've heard from some group or another every few years for the last several decades. What I don't see is why this time is different.
I think success for this project looks more like a supercharged spreadsheet than a simplified programming language. There will be many programs that won't be feasible to write in it, but I think they could carve out an area wherein there are many useful programs that can be. If that area is large enough, that would be a big deal.
Data can be a complex thing to understand. By forcing data to be explicitly displayed in tables, users no longer need to "read source code" as just a casual glance at a chart or a table leads to significant comprehension about what's going on.
I would say that that's almost precisely backwards: compared to typical plain-text code, spreadsheets make intermediate results more obvious, while obscuring the steps. (Things like, e.g., scala worksheets or ipython notebooks bridge this gap in a way which makes both the steps and the results clear.)
There are many programs that aren't feasible to write in a spreadsheet, and yet people have written them regardless.
In college, one course I took got groups of students to participate in a networked, multi-user business simulation that was written in Excel.
It was slow as hell, and Excel probably wasn't the best tool for the job, but it was probably just what the domain experts who created it were familiar with. Despite its lack of performance, it did its job admirably.
I think many people would be absolutely shocked at the size and scope of applications written in Excel using VBA. So in that respect, perhaps a supercharged spreadsheet would result in many more feasible applications than most of us, as developers, would first imagine.
Five years earlier, Prof. Harel wrote a book, Come, Let's Play[2], in which he explored the role that temporal logic[3] could have in the building of reactive software systems (e.g. desktop GUI applications, among many others). If you visit Harel's full list of publications[4] and do an in-page search for the terms "LSC", "behavioral" and "scenario", you can follow the links to a number of papers published since the early 2000s, related to the application of temporal logic in the process of software design.
It's possible that my connection of Granger's "eve" to Harel's work is completely mistaken; and if so, I apologize for jumping the gun.
[1] http://www.wisdom.weizmann.ac.il/~harel/papers/LiberatingPro...
[2] http://www.wisdom.weizmann.ac.il/~playbook/
Short: You provide the goals and the constraints, then through a back and forth dialogue with the computer you describe and clarify the problem (and also your understanding). You keep on negotiating and providing more constraints until it simply finds a possible solution (the problem domain is now computable) or the best fitting solution now satisfies your needs (in time).
The hardest parts are probably: 1. Collecting, developing the constraint solvers. Should we imagine that collection more as platform of user contributed (programmed/trained) packages? 2. Developing the dialouge that is not just textual but visual (in both ways) (notepad vs IDE vs Theater(?!) don't forget macros/acting out!)). Showing is always faster then writing and gives one better cues on the closures/framework of the actual discussion.
[1] http://scattered-thoughts.net/blog/2014/07/21/imperative-thi... [2] https://gist.github.com/jamii/8957881c8eaa4035f4ae
https://speakerdeck.com/mtrimpe/graphel-the-meaning-of-an-im...
http://en.m.wikipedia.org/wiki/Subsumption_architecture
http://en.m.wikipedia.org/wiki/Behavior-based_robotics
Kodu was based on this model, and it worked very well.
Right, he's talking about replacing the need for professional programmers. And, in doing so, he focuses on not needing the guy that knows a particular language, rather than not needing the guy that can think through a structure a logical approach to solving the problem.
Its all pretty vague, anyway, and the further clarification on the thread sounds like the goal is the yet another app builder (the supposed gap left by VB6 is specifically cited as the motivation) for the web, of which there are many existing now, all with similar pitches about letting "normal people" (or business experts) build line-of-business apps without hiring a professional programmer.
That's a pretty narrow way of reading what I was saying. What I meant is the one guy who knows how to program and I just happened to pick Python because that's the common case in science. Do you really think a mathematician, biologist, or chemist can't think through a logical approach to solving a problem? I certainly believe they can, the problem is then in translating that to the computer. Right now, that would mean knowing (pick your programming language) and translating that logical solution. We want to make that way, way easier, to the point where it's like using the tools you already do to do this stuff (Excel).
> Its all pretty vaguem, anyway, and the further clarification on the thread sounds like the goal is the yet another app builder
We've bootstrapped the editor, parts of the incremental compiler, parts of the platform, the domain editors, etc. If you can do that in an "yet another app builder", I'd definitely love to see it. We're picking one domain to focus on because that's how you grow a product, but at the end of the day, if it doesn't suit my needs as well, I certainly can't call it a "better programming" in general.
No, then again, I think that mathemeticians, biologists, and chemists now, with existing tools, often program computers to solve problems without being "professional programmers". (And not just with Excel and similar tools.)
> We've bootstrapped the editor, parts of the incremental compiler, parts of the platform, the domain editors, etc. If you can do that in an "yet another app builder", I'd definitely love to see it.
I think all of that is significant, and I didn't intend to minimize that.
> We're picking one domain to focus on because that's how you grow a product
Insofar as I have concerns with the way the short-term focus is described, its that:
1) There isn't a clear description of what unique thing Eve intends to the table in that initial focal market, and
2) The initial focal market, as it has been described, seems quite crowded, rather than a gap that needs filled.
A simple example of this is SQL. You shouldn't need to be a pro engineer to run some simple queries.
Another example is Mathematica. Scientists and engineers want to solve their domain problems, can learn a little bit of stuff, but shouldn't have to learn about build tools or d3 just to plot some data and solve some equations.
Whether Eve gets the balance right and is useful remains to be seen, but there is plenty of room at the bottom for empowering non-engineers.
Right now, though, that thinking and analysis mostly comes with very little assistance from the computer. We are pleasantly surprised when a compiler manages to guess what we meant when we made a mistake.
I think there's a lot of room to help with the process of thinking rigorously. New models of programming could conceivably allow someone to start down the path of building, and then have an interaction with the programming environment in which it helps to find ambiguities or special cases.
I'm a terrible artist; I wouldn't try to make mockups if there weren't sites like Balsamiq. Even if I tried, they would be too rough to serve as any kind of spec. The right tools allow more (not all) people to succeed in trying their hand at design. Of course they will never be a "real designer", but hopefully they can accomplish more of their goals without help.
I think better programming tools and paradigms can have the same effect.
This is a great analogy, but unfortunately I think it undermines your point somewhat.
Almost anyone can become a great artist or designer, but what it takes is practice, years of practice, and learning from others who have gone before and are working now. There are no short cuts and tools are really the least important element in any discipline, be that design or programming.
I challenge this claim. I don't think just anyone can become a great artist or designer and if you think this I think you miss a great deal when it comes to evaluating these skills. Maybe anyone can become great at Photoshop and the other various tools, but if you don't find yourself enjoying designing, and you haven't had an artistic trait that others have recognized in you early on and you're not making much headway now, you probably want to find out what other talents you might have.
I think that if you enjoy designing you should pursue it. I think wanting to be good at something can lead to a high degree of skill because the motivation will drive you there. Not everyone has that motivation and so not everyone can be a great artist or designer. I think the same thing can be said of engineering. People who go into this field to get a 9-5 job and hate the work will struggle competing against people who love what they do and would do it even if they weren't paid for it.
I'm a designer, trained as an artist ;) I do agree motivation is key though.
I think what worries me about the hyperbole on this linked page is that it implies all problems software solves are fundamentally similar (they're not), or that all software shares certain fundamentals (it does of course, but at such a low level they are hard to abstract). The range of problems it says they want to tackle is huge:
They want to make a better excel
They want to make a powerful, intuitive, database UI
They want to make a tool scientists can use to crunch and visualise data
They want to make a game programming language for kids
They want to make hypercard with modern programming languages backing it
They want to make a tool for professional programmers to focus on the hard problems
Any one of those is an admirable and difficult goal - trying to make all of those things at once is a good recipe for starting something and never finishing it.
That's not a reason not to try to improve programming practice with something radically better, but I think they'd be far better to pull a Buchheit here and focus on one small set of customers and make them very happy, instead of over-promising and under-delivering.
The rest comes later, though we do believe it all flows out of a common model for programming in the same way that all of those things could come from any general purpose language.
Will be great to see what you come up with, but I think the sales pitch here is overstating what you can reasonably achieve. Everyone has had dreams of revolutionising their discipline by inventing new tools, but it's very rare for tools alone to lead to a new paradigm.
> but it's very rare for tools alone to lead to a new paradigm
I think the distinction is that a tool helps you do something more efficiently while still generally thinking about the problem in the same way. What we're proposing is a very different way of thinking about the problem, which is what I think starts to define a paradigm. That being said, this is largely semantics and not a very valuable thing to argue about: paradigms come to be by the accident of use, so we'll find out when we launch and build a community around Eve.
These are basically all the same thing. Excel is generally used as a database / data cruncher. A relational DB with a similarly accessible learning curve could prevent many of the errors that riddle most spreadsheets. We add incrementally updated views to handle the data manipulation / crunching side (relatational aggregates are sufficiently expressive to handle most statistics / ML problems - http://db.cs.berkeley.edu/cs286/papers/bismarck-sigmod2012.p...).
> They want to make hypercard with modern programming languages backing it
Papers like https://github.com/papers-we-love/papers-we-love/blob/master... and http://db.cs.berkeley.edu/papers/eurosys10-boom.pdf show how you can handle interactivity in a relational model. We have enough of this working that we bootstrapped the IDE.
> They want to make a tool for professional programmers to focus on the hard problems
Most of what I do as a professional programmer is not building abstractions or designing systems, but wrestling with incidental low-level detail, lack of visibility and tools that don't talk to each other. I've had enough experience with Eve to be convinced that I would be more productive.
> ... over-promising and under-delivering.
Obviously we are suffering from http://lesswrong.com/lw/kg/expecting_short_inferential_dista... since we have been thinking about this and working with it for months and all you have to go on is a few blog posts. I don't expect to convince you that we are right, but hopefully I can convince you that we have at least thought it through. We aren't trying to boil the ocean here - we are building a small set of tools that are applicable to lots of different problems. We've picked one problem to focus on for the first assault
If they incorporate all that into one program, it will be (an unusable mess OR so damn complex that a small percentage of users will ever tackle it) AND it will quickly fall behind whatever bleeding edge they're wanting to stick to.
For evidence, I give you: Eclipse, the ultimate do-everything IDE. It's awesome until you realize how much unmaintained cruft there is in there. Which doesn't mean you can't get things done with it, but don't expect bleeding edge on a lot of things that started out that way.
Its still a challenge -- getting the UI right especially -- but its not particularly hard to see from the cited research how a tool built on that foundation that could do what they are saying, with the only specialization to the separate goals being UI affordances rather than separate basic functionality.
In order to get a job as a junior full stack web developer, you're going to need to know the following:
- HTML, CSS, Python/Ruby, Django/Rails, javascript, jQuery, SQL, Git, basic unix/linux
- Possibly Angular, design patterns, testing
Whereas in terms of CS questions you're likely to be asked, you're going to need to know:
- Basic data structures, algorithmic complexity, basic concurrency patterns
- Possibly some knowledge of common algorithms
Whereas all the programming stuff can easily take a year or more to learn, all the CS stuff you need to know to get a job as a professional developer is maybe a couple weekends worth of watching YouTube videos. I realize that CS knowledge sometimes becomes more important at higher levels depending on your career track, but I think you're underestimating the difficulty of learning the APIs versus learning the CS stuff as far as becoming a professional developer goes.
I beg to differ. Some of the biggest frustrations are languages and syntax for people learning to program. For a professional programmer learning another language is like learning another car for a mechanic. They all look the same after a while. But trying to teach someone how an engine works is not trivial.
You seem to be implying that we currently have tools that provide a suitably natural and frictionless medium for such thinking and analysis. But what if our current languages/tools are more like Roman numerals and there's still the computing version of Arabic numerals out there to be found?
- a style of analytical thinking and formal reasoning
- syntax and semantics of 1 or more programming languages
- interpreting the debug and error reporting capabilities of their environment
- version control
- text editing
- one or more build systems
- a mental model of computing machines and networks
Outside of academic environments you need to learn all of these thing simultaneously. If we can collapse away a few of these things with minimal loss of productivity we make programming much accessible.
1. By bringing investors on board, and promising Hacker News they're going to change the world, they're basically asking to be stressed out all the time. This is not conducive to innovation.
2. There's a fundamental tension between creating something usable by Joe in accounting and something that is cutting-edge from a technical perspective (highly concurrent etc). 90% of line-of-business programs use less than 10% of CPU, and approximately 0% need high concurrency. Joe in accounting is not going to build the next WhatsApp, and every moment you spend figuring out multi-core whatever, you're not thinking about how to help Joe make his Cash Flow Prediction Tool look slick.
3. From a marketing perspective, creating a new category of software is an enormously difficult undertaking. There are two related problems here:
A. You have to find potential customers. Because nothing like Eve exists, the market does not exist. I.e. the world's "creators" don't all go to the same conferences, visit the same websites, and hang out in the same IRC channels. Light Table at least was addressing an existing market, i.e. "programmers who use X/Y/Z", so you could actually find customers, and they could tell their programmer friends about it.
B. After you find potential customers, you have to educate them, i.e. explain the potential benefits and teach them how to use the darn thing. This requires a major investment from both you and from your interested customers. It is much, much easier to sell something that relieves the customer's pain than something that promises him "superpowers". 95% of "creators" have only vague and incorrect notions about programming, so you'll have a hell of a time explaining the product. Let's say you want to target Joe in accounting: What will the advertisement actually say? "Do you miss VB6?"
4. The fact that the LT team is giving up on Light Table when it's still the proverbial "couple of guys" is not a good sign. From a marketing perspective, Light Table had every advantage in the world: tons of publicity / viral videos, a clearly defined market, and an exciting product that got people talking. The fact that they're throwing in the towel tells me, well, they quit too early, and they'll quit on Eve when the going gets tough.
To be more specific, I think Chris's fears about the market for Light Table are totally unfounded. To wit:
* "The competition has millions of man-hours invested". Who cares? On a day-to-day basis I execute maybe 3% of the code paths in my editor. In my view, Light Table is a classic opportunity to offer 10% of the functionality of the bloated competition but really knock that 10% out of the park. Look at e.g. Pixelmator versus Photoshop or [toot toot] Wizard versus SPSS.
* "Programmers don't pay for software". I'm sorry, but this is a poor excuse, and if I were your drill sergeant, and this were an 80s movie, I'd be screaming in your face right now. Programmers are a wealthy and growing segment. They'll pay for stuff if it helps them get their job done (GitHub, TextMate, books, conferences, heck Visual Studio). Not only that, with the product's positioning, you have a great opportunity to sell to people who want to be programmers, the "ski pants" market, if you will. If you read the comments in this thread, you'll notice that your users are complaining about the Light Table's lack of polish and they clearly recognize the need to pay someone to provide that polish.
In sum, I think the LT team should have worked themselves into a crying, bleeding, starving mess to make Light Table a commercial success, and then pursued bigger ideas once they had the business experience under their belt. Even if it took a few years, they'd be in a much better position both financially and psychologically to tackle a Grand New World-Changing Product once they had the routine down from their Small Life-Changing Editor.
As an indie-app-developer-whatever myself, I would kill for the kind of publicity and market opportunity that Light Table had. I wish Chris & Co. all the best with Eve, and I truly hope they succeed. But based on what I'm reading, I don't think they don't deserve to.
I was waiting for LT to become a tool I can recommend to beginners. It's far from being ready, and it now appears they just gave up. If LT was too hard for them, what are they even talking about with this Eve thing?
You have to do all these things because if you don't the editor is not usable for editing programs. The vast majority of the code in LT is devoted to basic text editing. Just reaching the same level of polish as, say, gedit would take many more man-years of work. All of that work has to be done before we can spend time on actual innovation.
Or we could just make a language with a sane serialisation format. Eve already supports live coding, incremental compilation as you type, globally unique identifiers for every piece of code and time-travel debugging. We also have algorithms sketched out for concurrent editing, commit-as-you-type (ie the version control system stores every keystroke) and causal history for data (why/why-not provenance). We got to do all that in a few months because we didn't have to fuck around building a text editor.
> "The competition has millions of man-hours invested". Who cares? On a day-to-day basis I execute maybe 3% of the code paths in my editor
This is the Microsoft Office argument and the problem is that no one uses the same 3%. If they did, sure it'd be "dead simple", by that of course I mean take many, many years of work. Sublime which attempts to do far less than we wanted to took 5 years to get to today and I'm not sure it really is a commercial success capable of supporting 3+ people.
> "Programmers don't pay for software". I'm sorry, but this is a poor excuse
It's far from an excuse, it's the truth. It doesn't matter if developers as a class have a lot of money. They don't actually spend it. I don't know a single person who has ever personally paid for Visual Studio. Github is a service and the vast majority of their money comes from companies not individuals. How many people run stuff only on the heroku free tier?
Or here's a direct example: when we launched the kickstarter people \freaked\ out about us pricing it at $50. We had to drop it all the way to $15 and folks still complained endlessly about it. It is well known that short of a service targeted at companies, it is near impossible to sell developer tools. After all, it's a group of people who look at a piece of software and say "I can build that".
If there's some secret, we're missing it. There's an entire organization [0] that has grown up around trying to help dev focused companies figure this stuff out. It's led by James Lindenbaum (co-founder of Heroku, invested in us), he knows his stuff and he couldn't figure out a way to make this work out either.
[0]: http://heavybit.com/
Building a consultancy company around a technology product is probably on of IT's most classic business models.
From what I understand something like Eve is begging for a hybrid product/consultancy approach...
Putting personal preferences aside it would by far be the most logical path towards building a big business out of it.
People pay for stuff that is useful to them, the problem with light table is people don't understand why it is useful to them until they think about it/look at it. You should have completely ignored every complaint related too money.
VS costs hundreds of dollars (going into the thousands for MSDN). And it is used by hundreds of thousands of programmers.
If product A results in a increase in productivity people will pay for it.
According to https://sublime.wbond.net/stats there are roughly 2.5M sublime text users. If even 10% of them paid for it, that would be 250,000. Assuming 10% is a little high, let's do 1% would would be 25,000.
Even if those numbers are wrong (which they may be I have no idea how he gets those numbers, and if everyone is using Package Control) there can very easily be hundreds of thousands of paid licenses.
How does Eve compare to something like Autodesk's design script?
> By bringing investors on board, and promising Hacker News they're going to change the world
a16z constitutes the majority of our investment, and Chris Dixon at a16z is rather well equipped to help us tackle this problem. It has also given us access to a rather impressive network of very smart people who can help us. Yes, there is a trade-off in that our investors have a say in what we do, but given the small number of investors, and who they are individually, there is a low probability of micromanagement from them. Secondly, we never promised Hacker News that we're going to change the world; we said we have some ambitious ideas, and that we wanted to pursue them. Might they change the world? Sure. Might we also die trying? Absolutely. That's a risk inherent to... well, everything. Certainly a lot of things worth doing.
> The fact that the LT team is giving up on Light Table/I think the LT team should have worked themselves into a crying, bleeding, starving mess
The last time we had office hours with pg about LT, we didn't exactly leave with a warm fuzzy feeling about continuing work on the project. But we did, for another year, spending the remainder of the Kickstarter money and approaching the end of the runway, staying as lean as possible, and continuing to develop a product that pg, James Lindenbaum, et al, all regarded as a nonviable business. We may have to agree to disagree on how plausible it is to make an open source IDE a commercial success, but there was plenty of crying and bleeding before we came to work on Eve.
At that point, it became what Jamie aptly described as a matter of return on effort. There isn't much sensibility in spending countless hours to solve the wrong problem, and running out of money didn't strike us as the optimal way to continue working on a problem we genuinely cared about. Instead, once LT became open source, we forged ahead into an area that Chris Dixon called a "Vietnam" of software, and while Eve may be particularly difficult, it brought us new funds precisely so we don't have to give up.
> I truly hope they succeed
Thank you, so do we :)
I don't see anyone offering to pay for polish. Can you point to these instances?
I've been active in the LT community for most of a year and don't recall _one_ offer to pay for any bug fix or feature. If anyone has such a request, please make it in LT's github issues or mailing list and I'll consider looking into it.
It sounds like Eve will essentially be a visual programming language, similar to the many others that have come across Hacker News.
Chris - is Eve a visual programming language? If so, what makes it better than all the others that have tried and failed?
I didn't want to believe it, but it looks like HN is highly biased toward anything that has some VC behind it, flocking to it like it's suddenly going to change the world.
Considering who sponsors HN, the biases of the community it draws aren't that surprising.
Most of the designers (outside of web to be sure, since I don't live in silicon valley) I know don't use textual formats, that is for developers. Much of the content is too dynamic to work with textually without substantial programming.
What's the big deal? There is an entire family of conventions and tools that have been built up over the years to support coding in text: editors, version control, diff tools, patches, linters, code generators, naming conventions, meanings of special symbols, etc.
It's definitely technically possible to change the paradigm entirely, but how does one avoid reinventing twelve wheels?
To paraphrase, "I know, you won't write my language in flat text files. And now you have a dozen problems."
What am I missing?
If we were talking about any other coding problem and I suggested you keep all your data in a complicated text serialisation spread across several hundred text files and resolve conflicts by diffing the files line-by-line, I would be laughed out of the room. The vast majority of work in any IDE, including Light Table, is just working around the fact that plain text is a terrible storage mechanism for complex, concurrently edited data.
That's not to say that you shouldn't use text to write or view the program or that we should all be coding by drawing pretty pictures and connecting lines. Just that we should keep the code in a sensible, networked data-store with support for concurrent editing and ACID transactions and then the way you choose to present and edit it can vary from person to person.
All the stuff I've done here [1] is plain text. I'm sure you have other good reasons for avoiding text, but these aren't that strong; text is mostly orthogonal to the real problems of building live systems.
[1] http://research.microsoft.com/en-us/people/smcdirm/managedti...
> If we were talking about any other coding problem and I suggested you keep all your data in a complicated text serialisation spread across several hundred text files...
Except developing code in a team of size greater than one is a distributed computation problem. Distributed persistence mechanisms are basically a bunch of files spread across a system. And they deal with some of the same problems (availability, versioning, reconciliation, replication, etc.). Speaking of which, these are problems are typically solved through tools (git, etc.) that presume textual languages.
3. To be more concrete:
Q: How do I post code samples to sites like StackOverflow?
Q: How do I create patches from and apply patches to my codebase?
Q: Does git (or some other offline editing mode) work?
...some of the answers may include statements like, "Well, wiring and metadata is all XML (or something) under the hood, so use your existing merge tools." In that case, we're dealing with two things: a language and a configuration spec. We're not really talking about a system that's friendly to coders after all.
...again, these are technically solvable problems. I just think people are vastly underestimating the level of effort needed to get to feature parity with the current way of doing things.
The best way to prove me too skeptical is to develop a decent-sized project across a decent-sized team, which is something non-text development tools (including Excel and Photoshop) haven't been particularly good at. My bet is that this won't happen and that at best Eve will be a successful enterprise tool for domain-specific problems that can be tackled by small teams (again, like Excel, Photoshop, LabView, etc.).
Not that there's anything wrong with that. Adobe is a successful company.
People who worked with that used to love it and I'm fairly certain that Eve's model would (be able to) work quite similarly.
Professional programming is about managing complexity. It's about putting thousands of man-hours into one single project with a dozen of different people without anyone's head exploding.
Light-table could have potentially been a huge help in the constant battle with managing complexity.. It's an area that has need some serious development (virtually nothing substantial has been added to IDEs in the past decade). But instead you're going to create a beginner friendly monster.
This is the opposite of progress
The model we have is significantly better than what we ever showed with Light Table. For example, being able to organize "code" by the actual functionality it's related to instead of files is something that happens by default in our world. You can actually create queries to show any view of the rules that exist in our world that you want (show me everything related to this button, that has been changed in the past month..)
Our model is entirely about preventing people's heads from exploding, taming complexity through a very simple set of primitives that seamlessly work together.
We're tremendously excited about the potential we have with Eve and everyone we've talked to about it so far leaves with their head reeling. There's been an undercurrent brewing in the programming community that we're approaching a sort of transformational point in the way software ends up built. We hope to be part of that transformation.
I don't talk about it a ton in the post, but Eve isn't just about giving the Excel generation a seat at the table, but it's also explicitly designed to get us back to what we do best: solving problems. There are some pretty amazing properties that fall out of simplifying things as much as we have: concurrent by default, a real shot at auto-parallelization, a debugger that can tell you when, why, and how a value came to be, diffs not of code but of the output, the chance for true real-time collaboration on software, introspection into every part of the system, and so on. We'll be talking more about how it works over the next couple of months, but I can promise you now: there's some exciting stuff coming for all of us.
As always, in the interest of sharing with the community who helped us get here, I'm happy to answer any questions!
My one comment is to please make eve keyboard oriented from the start. I really don't want to use a mouse. I should be able to navigate every entity logically with a keyboard. For example, I should be able to hit a key to select available link endpoints and another to connect them.
I like visual, but I love keyboard oriented a lot more. Dragging and scrolling are way too overused ui interactions.
Don't worry, we'll make sure it works whether you use the mouse, a keyboard, maybe even a touch screen or voice. :)
Just say 'NO'. & best luck :)
At NDC Oslo this year there was a keynote by Luke Wroblewski on user experience on the web. There's a video on Vimeo: http://vimeo.com/97305189
The key takeaway was that a significant percentage of people are using their smart phones with either only their thumb or their forefinger. Many people don't even have a computing device with a traditional keyboard and full-size screen. That talk got me thinking about this question: what would programming look like if you only use your thumb?
I don't think you can write off these interfaces, rather I think we may need to start to really think about separating creation of core components of software and then orchestrating the interaction of these components as a different aspect to developing software.
If you're wondering how programming with just your thumbs might work, look at Touch Develop from Microsoft Research.
1. Powerful keyboard-based navigation/manipulation will be extremely helpful.
2. RDBMS-style tuples seem weird to me; why not objects? Quite natural to a layperson, IMO. If a "cell" can hold an object ({name: "Brown shoes", price: { amount: 254, currency: "USD"}} rather than just a scalar value like $2.54) a spreadsheet can be a lot more powerful.
Best of luck.
Is it planned that users of Eve will routinely construct new or modify existing domain-specific editors to the task at hand?
Since first focus of Eve is on web apps (and it's hosted on javascript, correct?), will it cover both client and server applications?
Will there be an FFI?
If I remember correctly, you're planning to use datalog+time+constraints and plan to use materialized views extensively, is it so? Will ability to work with changes as a stream and aggregate over them in more interesting ways than "the last always wins" be builtin in Eve?
Have you tried to show a prototype to children and how do they respond in learning such system if so? What was the reaction of academic friends who were not too familiar with standard forms of programming (like you did with the first prototype of Aurora)?
If I'll ask more questions will it be all right and not too much trouble? :)
Not sure at this point. For now we are focusing on the core interactions and we'll build out the rest once we've put tools in peoples hands and seen how they work.
> What domain-specific editors are you planning to include in public release? Something for UI construction on the web?
The table, rule and function editors for managing data and logic. A UI editor, hopefully with some constraint-based layout model. Various debugging/tracing/understanding tools.
> Since first focus of Eve is on web apps (and it's hosted on javascript, correct?), will it cover both client and server applications?
It will have to. Not sure yet how that will work. We can do websocket-style stuff at the moment but we will want to provide something simpler on top of that.
> Will there be an FFI?
Yes. You will be able to add new functions / datatypes (like http://www.postgresql.org/docs/9.1/static/xtypes.html), new table/index data-structures (like http://www.postgresql.org/docs/9.1/static/gist-intro.html) and new query-plan/data-flow nodes.
The editor and compiler are mostly written in Eve. The core language is tiny but very extensible - since execution order is not specified explicitly but determined from data-flow you can drop new logic in anywhere to add new compiler optimisations or new editor commands.
The language is also very data-centric - this week I was playing with programs that modify their own code by writing to the ast tables, triggering the incremental compiler to update the dataflow plan.
> If I remember correctly, you're planning to use datalog+time+constraints and plan to use materialized views extensively, is it so? Will ability to work with changes as a stream and aggregate over them in more interesting ways than "the last always wins" be builtin in Eve?
We're still trying out different models of time. The ideal semantic model is append-only, never forget. Implementing that on a real machine needs some restrictions to prevent exploding. Dedalus (http://db.cs.berkeley.edu/papers/datalog2011-dedalus.pdf) attaches epochs to each facts and only allows rules to query the current or last epoch. Edelweiss (http://db.cs.berkeley.edu/papers/vldb14-edelweiss.pdf) allows querying any epoch but uses static analysis to figure out when data can be safely deleted without changing the results.
> Have you tried to show a prototype to children and how do they respond in learning such system if so? What was the reaction of academic friends who were not too familiar with standard forms of programming (like you did with the first prototype of Aurora)?
No children yet. We test things on adult non-programmers regularly. That lead to a couple of surprising changes like getting rid of nesting / scoping (no matter what we did visually, people just couldn't figure it out).
> If I'll ask more questions will it be all right and not too much trouble? :)
Keep 'em coming.
> That lead to a couple of surprising changes like getting rid of nesting / scoping (no matter what we did visually, people just couldn't figure it out).
I worry about that -- I can see where its probably a huge gain for initial usability but my concern with it is that it shows signs of one of the big problems with spreadsheets in that they make it easy for non-programmers to start with something, and add complexity to it until not only they can't maintain it, but calling in a professional programmer doesn't make it maintainable either, and you essentially need to do a ground-up reanalysis and rewrite to get it into a state where it is maintainable with the complexity it has grown to.
Instead of having a set of rules to map the human-readable names, we generate unique ids for everything at edit time and tag them with the human-readable name. So when you type "foo", you get an intelligently sorted auto-complete box asking you which foo you mean.
This is one of the advantages of not directly editing the program text - we can insert all kinds of useful metadata directly into the AST.
> ...they make it easy for non-programmers to start with something, and add complexity to it until not only they can't maintain it...
Like most things, the problem can't be solved with technology alone. We can improve the tools to make it easier to organise code and to make it easier to understand other peoples code but we will also have to find ways to scale up the teaching aspect. It's the 'pit of success' - we have to design the experience so that people naturally end up doing the right thing. That includes building a community where people learn from each other and are exposed to better ways of structuring their thoughts.
> This is one of the advantages of not directly editing the program text - we can insert all kinds of useful metadata directly into the AST.
I suppose I'll have to see how it works when there is something available to poke on, but that description doesn't allay my concern about it making a programming environment that makes writing easy, but reviewing, understanding, and maintaining previously-written programs hard in much the way that spreadsheets do.
If the information needed to disambiguate references is buried behind the scenes and not readily apparent in the primary interfaced used for editing, that seems problematic.
True, but most databases are very limited components of the program(s) that use them, which do use namespacing, not the complete platform for the program.
That said, I'm quite interested in getting a hold of eve.
It would be interesting to hear how much of EVE code has it taken to implement compiler or editor. And is there any interesting bottlenecks you would need to optimize out before release?
And how was the experience of writing a lot of EVE code? Do you feel yourself a lot more productive? Have you become accustomed to a new way of writing software very fast and does it feel natural to you now?
Regarding models of time. I'm a bit familiar with Dedalus and I've skimmed over Edelweiss. I'm interested in how would you use Edelweiss in non-distributed settings (I assume Eve is not targeted at that kind of applications right now). Would you try to use it to maintain transactional visibility during re-flows of data across views and afterwards discarding dead data?
I remember you're mentioning datomic in a podcast, so I think you'd want to keep dead data at least for some time, to have an ability to run time-travel queries later. Is this correct?
Would it be possible for developers to control when facts should be garbage collected? For example, CORFU log (which is a good implementation of event log abstraction with flexible views above it) allows clients to control trimming http://muratbuffalo.blogspot.ru/2014/09/paper-summary-tango-... (also I've had a good experience implementing underneath a similar interface for a queue system, it lead to a similar range compression optimization as in Edelweiss).
All the details you've already mentioned make me dream at night wishing a sooner release of Eve to experiment with it. I hope your next answers won't make me kidnap you and beg for an earlier access :)
The current iteration of the editor is 350 lines of our text-based Eve dsl plus 450 lines of helper js which will eventually become our UI and IO libraries.
Stratification is about 30 lines. There used to be more going on in the compiler but now that we use a constraint solver for all the query plans there isn't much left for it to do. http://p2.berkeley.intel-research.net/papers/EvitaRacedVLDB2... has some interesting ideas that we will be exploring later.
> And is there any interesting bottlenecks you would need to optimize out before release?
We've managed to get pretty surprisingly far with really dumb code. We are halfway through a big rewrite so the current build of the editor is using a runtime that is missing half of it's features. Each table is stored in a big array, we do sequential scans for constraint propagation, the incremental evaluation has not been ported yet and we don't have any Edelweiss gc. It can still recalculate and rerender in 20-40ms. That's too slow for fluid animation but plenty fast enough button clicks etc.
The lesson here is that computers are really fast and we squander most of that speed on unnecessary runtime abstraction. I'm pretty excited to see how fast it will get when we have real indexes and incremental evaluation.
> And how was the experience of writing a lot of EVE code? Do you feel yourself a lot more productive? Have you become accustomed to a new way of writing software very fast and does it feel natural to you now?
There are ups and downs.
The main upside I've noticed is how much time I normally have to spend worrying about where to put data, how to access data, how to keep data consistent and what order to do things in. In Eve you just put it in a table and let the runtime worry about the rest. I feel like I now spend a lot more time thinking about the actual problem instead of shepherding data around.
The main problem we're wrestling with at the moment is identity. Since the editor is built in the Edelweiss style as a pure function of the input events, we have to be able to generate stable, unique ids for anything that has a continuous identity. In the Bloom model we could just generate a random UUID and store it. At the moment we build ids by mashing together strings but I feel like the language should be helping us out here.
There are also a whole bunch of problems that come down to immaturity. We don't have a debugger yet so we rely on console dumps a lot. The compiler doesn't have much error checking so bugs in the editor tend to percolate a long way before surfacing symptoms. Implementing integrity constraints is pretty high up on my todo list right now.
> I'm interested in how would you use Edelweiss in non-distributed settings (I assume Eve is not targeted at that kind of applications right now).
There is a section in Out Of The Tarpit where the author argues that the ideal way to specify an application is as a purely functional view over all the inputs it has ever received. He then concedes that in practice you are forced to throw things away to avoid running out of memory. Reading the Edelweiss paper convinced me that we could program as if we were living in Mosely's ideal world and then separately specify a garbage collection strategy. It's just another instance of separating specification from optimisation which I believe is key to managing complexity.
This is probably the most risky part of our plan but we can always fall back to the Bloom model instead. The actual language implementation doesn't really vary between them.
I'm working on something similar but much more constrained myself [1], but initiatives like Brandom Bloom's EClj and Reid McKenzy's KISS are already on their way to building the foundation for what you need with Eve but within the Clojure format/community.
Taking that route would also give you a natural path to keep LightTable alive and going and eventually get it bootstrapping/self-hosting.
[1] https://speakerdeck.com/mtrimpe/graphel-the-meaning-of-an-im...
I've been programming professionally for about three years. When I got my first job, I spent days setting up ruby/rails/zsh/vim/git. I was seriously dumbfounded. I could not believe that programmers (in 2011) actually worked this way. And I was even more perplexed that the main tools for reasoning about my code were print statements, browser reloading, and unit tests (so, so many unit tests). This is "the pain we forgot", except it's recent enough that I've never forgotten it.
Thanks for your work. I'm looking forward to a better programming for my kids. And hope to support any way I can.
[0]: http://pryrepl.org/
I keep thinking what we need is for the structure of programs to be something saner than the one-dimensional byte stream that we keep pretending programs aren't by splitting them into lines, functions, classes, and files. Tables aren't fundamentally different than those other constructs, in the same sense that programming languages aren't fundamentally different from each other - BF is Turing complete and all that. But usability and learning curve, as someone said on here, really matter, and tables have a far more clean and rigid structure than a mess of braces.
The most difficult problems in programming aren't the low level algorithms, they're the social problems along the lines of how do you involve large numbers of new programmers in a project without ruining it, or how do you empower users to create their own features without starting over and building a custom rules engine?
Light Table was announced to change the way we're writing code, too. But the execution didn't quite match the vision.
In the continuum of programming tasks, tasks best for spreadsheets are pretty far down the scale. But these tasks are very common and an improved tool could have a very big impact.
After grad school I did real time process control programming for semiconductor manufacturing. Complex machines, which couldn't be slowed down, were programmed in assembly language and had many simultaneously moving parts, all under the control of software. Programming, done in assembly language, was a mess. It was hard, very hard, we had to build all of our own tools, including the operating system that ran on the minicomputer running our software. The mechanical engineers were very talented and built amazing machines, but they thought that the understood the "hardware" so they should be the ones to write the software to run it. Of course, this didn't work out at all. It's easy to believe that programming shouldn't be that hard.
Computation can be intrinsically difficult in ways that new tools don't help. Consider a simple problem like suffling a deck of cards. Having a tool that makes a shuffle easy to code doesn't make it a fair or efficient shuffle. On the otherhand, most students of Computer Science would know the pitfalls of shuffling by the time they got through a couple of years of grad school. One can't expect that even intelligent people will understand that the a fair shuffle has to pick one of the exactly N! possible shuffles with uniform probability and that the following loop doesn't do it (for a number of reasons):
(* Randomly rearrange each card *)
for ctr := 1 to 52 do begin
random_number := random(51)+1;
tmp := card[random_number];
card[random_number] := card[ctr];
card[ctr] := tmp;
end;If you look at the research papers they cite, its not so much that they are aiming at a better spreadsheet which will , as a side effect, revolutionize programming, but a better programming model than those currently deployed which will, as a side effect, revolutionize spreadsheets.
My initial post was highly skeptical, and I still think that there's an uphill battle here. But I think that they have a good foundation to build a tool that both eases and empowers "casual programming" (including the kind of things that would be done in spreadsheets and that people don't and won't think of as programming) and empowers professional programmers to focus on the hard part of professional programming.
That said, its far from the first improvement of this general description that's been tried -- even with a good foundation -- and while I think its by far the best foundation I've seen for it, most of the key application domain areas have "good enough" solutions that its going to be hard to get people out of. (I am increasingly convinced that the initial casual-programmer web app focus makes sense, though, because while there's lots of established incumbents in that space, there's no dominant player to displace, so while it will may take a lot of polish to get decent uptake there, its an easier fight than going straight for Excel out the gate, and better reach than focussing on pro programmers as the initial focal market.)
I'm curious about the target audience and the drive to make programming "easier". Sure--the tools can help, but it doesn't necessarily help a user solve the problem correctly. Is the audience programmers? Is it for replacing programmers with software power users? Or am I totally missing the point, that this is a totally new way of solving problems in a language agnostic way?
But, I agree with you, the former-- changing the fundamental computation model-- is the more exciting (and harder) of the two.
https://news.ycombinator.com/item?id=7489865
Or did you arrive at a use case for temporal logic from another avenue?
Congratulations, by the way, eve sounds exciting!
Still, even though their goal (and the goal of similar projects) is my obsolescence, I wish them luck. It will probably be a massive overall good. But it's also a reason why I think all the current frenetic activity about teaching more people to program will end up being the wrong thing.
I'll bet this won't happen. If you can get non-programmers to produce what we think of as real software now, I am sure whatever insight you made will make programmers create much better, more complicated software.
I am sure they will do much better than MS Access. But still, years ago I had to work on a couple of projects where the spec was some system in Access that the domain experts had coded up. What a mess! - Really, you have no idea. They eventually could not fix the bugs themselves which was why the new system was being written. So there will always be room for people who can organise and abstract things.
Similarly, basic computational literacy should include being able to knock together a simple CRUD app or running a simulation. That doesn't mean that Joe Plumber is going to be building complex distributed systems any time soon. It will certainly ease the interface between professional programmers and users though. Rather than having to provide complete appliances where every possible user action is accounted and coded for ahead of time, you can just supply tools that the user applies however they like.
The Unity marketplace looks a bit like this. It turns out there are lots of people who have the desire to make a game and enough skills to handle basic scripting but not enough to build their own physics engine. In the marketplace you can buy level editors, animation engines, pathfinding algorithms etc and glue them together.
I have a hard time seeing what small niche Eve will dominate, and how it's at least 10x better than any existing solution. Looking at the examples they give:
> a scientist doesn’t have to rely on the one person in the lab who knows python
Scientist these days pretty much have to pick up a programming language (Python, R, Matlab, etc.) throughout their undergrad/early grad studies. If you're going to target that niche, not only are you fighting against the huge investment professors and labs have made in existing tools (along with the small libraries/frameworks they built themselves), but you also have to provide an equivalent quality ecosystem for the weird data formats some very specific niches use, for the lab equipment, and so on. If you wanted to deliver a 10x improvement on the existing state of computation in research labs, you'd have to spend a lot of time focusing on exclusively that.
> a child could come up with an idea for a game and build it in a couple of weekends
They already can, using Scratch or the myriad similar tools available which will probably remain better because they were designed with game design in mind.
> your computer can help you organize and plan your wedding/vacation/business.
People already use MS Office, Google Docs, etc. for these tasks. Unless Eve brings a 10x improvement to those tools, how can they ever hope to displace them? My dad has been using the same version of Excel for the past 10 years, I have a hard time imagining people like him switching over.
In his lectures, Peter Thiel spends some time talking about all the solar startups that emerged more or less at the same time in the late 2000s, only to fail miserably. While the buzz and excitement for solar power companies was there, in reality they could not deliver on what startups need to deliver to thrive: a product solving a real problem for a niche market that's at least 10x better than the existing solution in place. I feel like we're in a similar space with all those "learn to code" and "new programming tools" companies that have been steadily emerging.
No doubt, the CS academic and HCI researcher that I am is excited by Eve- I'm sure they've done their research and that it would be a really neat product. The entrepreneur in me is however extremely doubtful.
> Unless Eve brings a 10x improvement to those tools
We know the bar is pretty universally an order of magnitude and so we've planned accordingly. Our primary target here is selling a super power: you couldn't do this at all before, and now you can. Joe in accounting doesn't have the opportunity to build things to support himself (short of studying programming), but with Eve he could do everything from automate several of his workflows, to building a dashboard, to building dynamic reports for his bosses, to creating internal tools to track and measure things.
There is a fairly clear gap in our ecosystem around these kinds of programs and they represent what I would argue is the vast majority of programs that should be written, but aren't because of the cost.
There's also the opportunity to just make websites an order of magnitude easier to build for programmers and we're getting there. I was able to write an entire editor for this thing in itself in a week for one of our prototypes - there's some real potential there.
There's plenty of people throwing together simple LOB apps (including, particularly web apps) now -- with a wide variety of different tools -- without spending a decade becoming a professional programmer. And using similar workflows as they did when VB6 was around. Visual designers with minimal code didn't go away when VB6 did, and the place where there are the most competing ways of doing this is on the web.
> Joe in accounting doesn't have the opportunity to build things to support himself (short of studying programming), but with Eve he could do everything from automate several of his workflows, to building a dashboard, to building dynamic reports for his bosses, to creating internal tools to track and measure things.
That's exactly the pitch for lots of things that are on the market now (to pick one among many, without any particular intention to claim its the best in the field, Zoho Creator).
They've created LightSwitch. LightSwitch is also a great example for the usual outcome of using RAD tools: You often come pretty soon to the point where the provided abstractions are lacking some functionality or performance. The customer / user doesn't care how the software is built, it's compared to the greatest there is.
A product with some interesting ideas in this area is Calca: http://calca.io/
Good luck with Eve. I am sure you have read it already, but the paper "Out Of The Tar Pit" by Ben Moseley and Peter Marks might be of interest.
http://shaffner.us/cs/papers/tarpit.pdf
The first half is pretty dry. Around Section 7 they get into describing a non-standard way to look at application design. Their language design ideas might fit well with the goals of Eve.
I expect Eve will have an open architecture? So that those of us that like to plumb the depths can play too.
Most of these "programming for the masses" projects inadvertently throw up road blocks to experienced programmers. GUI's where you have to use the mouse and click 10 times to create a function, complicated formats that don't lend well to source control, etc.
With what we saw with Light Table, I am hopeful you will easily sidestep these sorts of issues.
It seems like the interaction between spreadsheets and programming is an oft-explored area - it will be interesting to see what this evolution brings.
* Networked / multiuser. Excel deals poorly with concurrent access to data and reacting to events. Most of the users we talk to have problems with sharing spreadsheets and moving data through pipelines. We aim to provide a shared database so that one users computations can easily depend on another users data and can respond to changes or new data.
* Web-native. The web is unfortunately the only distribution platform with any reach. That's one of the things that killed VB6.
* Not ugly. It may be shallow, but people don't like to use ugly software and will perceive it as being harder to use (eg http://www.ergonomicsclass.com/wp-content/uploads/2011/11/Tr...)
That said, the forms/3 HCI research is excellent and their recent work on preventing errors (http://eusesconsortium.org/wysiwyt.php) has been especially educational.
I'm working on a database-backed spreadsheet product too, and it sounds like we have very similar ideas about the weaknesses of the current spreadsheet model.
Our output product is still 'rich' spreadsheets rather than apps, though, and while we have a programmable VM and data processing language our language looks much more Excel than prolog or SQL. We also were really interested in the forms/3 research during our early development which is what prompted me to post.
Imagine a world where everyone has access to computation
without having to become a [..] programmer
I'm a long term Java programmer who recently learned Python. Python is sensational. We should encourage everyone to know a language like Python, not pass it off as the domain of "professional programmers".
My father is a microbiologist. When he started his postdoc at OHSU, they actually had a information artist guy in the department (apologies, I have no idea what the profession's name actually is). When you were writing a paper, you'd go to the graphics office/dept/whatever and give him your data, which he'd then lay out in some huge post board or something, then take a picture of it and shrink it down. Clearly much more time consuming than what Excel can accomplish for you now. I think around that same time they were just starting to get computers installed everywhere. By the time my dad left OHSU a few years later, this person was working in the mail room, his profession no longer existed (or at least wasn't particularly employable anymore). Everyone was now making their own charts, graphs, visualizations or what-have-you using their computers.
I, being a programmer, always thought about this as a success story. Interestingly enough, my dad has a different take on it. His perspective is that he used to be a scientist, now he's a scientist and a shitty artist. The way he sees it, its now his responsibility to learn this myriad of tools (photoshop, illustrator, Canvas hehe), whereas before his sole focus was more so on "the science". I certainly see it on my end, he's much better at Photoshop than I am, and at the same time knows a bit of HTML to get by, etc etc. However the reality isn't that the time to make these graphics has magically dropped to zero, its just that is become simple enough to not justify having an entire art department handling it, but hard enough to be annoying to him. He certainly spends a lot of time being angry at Photoshop now, which seems to be perfectly acceptable for a professional designer, but a strange consequence of being a scientist. Another result is that you now have someone not trained in data visualization trying to express his findings (vs possibly having someone who has a degree in that making suggestions about how to best present it).
Its interesting to try to reconcile these results. Its quite possible that there is no lesson here and its simply a spectrum of trade offs, no perfect scenario. Certainly everyone agrees Microsoft Word is a clear net positive, maybe visualization is just less of a win. Or maybe this is just because we are in an intermediate step, at some point things will be so easy that you 1) don't need a separate department, 2) it takes almost no time and 3) it inspires you to create really good visualizations without having much background in that. Who knows.
On the other hand some people would argue that this is actually displaying a deeper educational problem, where if there was more interdisciplinary teaching, then every scientist would have some grasp of data visualization and then its just a matter of having a better Photoshop. As time goes on I become more and more skeptical of this approach however. It seems strange to "empower" people when you are actually burdening them with additional responsibility. Since you can do everything yourself, it now becomes "your fault" for not being good at everything.
"A human being should be able to change a diaper, plan
an invasion, butcher a hog, conn a ship, design a
building, write a sonnet, balance accounts, build a
wall, set a bone, comfort the dying, take orders, give
orders, cooperate, act alone, solve equations, analyze
a new problem, pitch manure, program a computer, cook
a tasty meal, fight efficiently, die gallantly.
Specialization is for insects."If I have to do my stuff, I can stop blaming others. This is good for me and for the others. Even if the information artist guy made better visualizations, the people at the lab could blame him for not doing them as they envisioned, and the guy himself could not put the best of his effort on doing things, because they weren't made directly for him, so the blame could would be probably justified.
But, well, it is better to step away from this world of giving responsibility to others and then blaming them, into a world in which everyone do their best and don't blame anyone. Even if the final work is a little worse, it would be acknowledged that it was done by no specialist.
Ok, I don't know exactly what is my argument here, so I'll just stop.
Sure.