“The Mess We're In” by Joe Armstrong at Strange Loop [video]
youtube.com
youtube.com
Here are some of the problems Joe posed:
- bugs in software like Open Office, Keynote, grunt
- code not being commented
- computers booting slowly
- computers using too much energy
- code being written for efficiency rather than readability
This seems more like a disconnected list of gripes, plus a completely unrelated list of things he currently finds neat to think about. Which is totally fine, but I don't think it makes a particularly great talk.
Indeed. So there can be a market for companies that take on liability when serving commercial customers using F/OSS code that they have audited and that they feel exposes no more risks than they can bear. The original authors should definitely not be liable if they label their code as alpha or beta quality and do not wish to be exposed at all. They are doing a service to society. But once you aim your code at being used in production by entities that can suffer vast losses if your code turns out to be defective (in other words, if you sell your stuff to a business or private person) then you should be liable for those damages, or at a minimum you should insure against those damages.
Compare software to for instance the engineering profession to see how strange this software anomaly is.
It's not merely accepting liability, there are a whole slew of changes that need to come before this, and frankly, I doubt most people would pay for that. Indeed, if people want to be covered now, they can be. They just have to pay for it.
Computers and software are the way they are because of the set of tradeoffs that the market rewards.
It's certainly possible to write software with fewer bugs, that consumes fewer CPU cycles, memory, starts faster etc...: but it does less. So far, most people and businesses prefer software that does more at the cost of slower boot times, more CPU usage, and a few more bugs.
Six months down the line, when my not so bug free code eats your data I will point to that line in my EULA that says I'm not liable. Nobody will care, after all it is your data that got lost, not theirs. The fact that your EULA does not have that line and that you offer a warranty does not count for anything until someone would be willing to pay a premium. The only people that would like to pay that premium are the ones that lost their data...
So it's an industry phenomenon. Imagine extrapolating this to buildings. Engineers claim their buildings will stand. Those engineers that talk nonsense will be sued out of business. But if they could disclaim responsibility they would continue to happily practice their borked trade and as a rule people would suffer from this. And so engineer became a word that actually meant something.
But in software 'engineer' is roughly equivalent to 'can hold keyboard without dropping it'.
It'd be annoying if Things for iOS crashed and lost all of my data. It'd be horrifying if flight control software crashed and all aboard a plane were killed. It stands to reason that some software is and should be held to higher standards than other software. It probably doesn't make sense that all software should be held to the same high standard, as it is extremely time- and resource-consuming to ship avionics software. Do folks really want to dish out a few $thousand for a copy of Things for iOS?
And some companies do already take responsibility for open source software. In aerospace development, we routinely use GNU software that has been thoroughly inspected and certified as good by companies that accept many thousands of dollars to stand behind it. (Of course, if we were to upgrade from their GNU Foo 2.1.0 to the FSF's copy of GNU Foo 2.2.0, then all bets are off.)
No, actually, it looks a lot worse: given the same time and developers, the bug free software will do way less than the buggier software. That, or at feature parity, the bug-free software takes more time and/or requires more people, so arrives later or costs more.
I don't have any direct experience, but I suspect there are niches here and there where the market and/or regulations put a premium on no bugs. Avionics? Some categories of medical software?
You can only tell good quality software from bad quality software by auditing the code, not by observing the software from a users perspective (unless it refuses even to perform the basics).
- Is the only reason why he touches upon the limits of computation and computing efficiency that it secures distributed hash tables and that there is space for improvement in terms of energy consumption, respectively?
- It seems contradicting that he advocates biologically inspired systems and lowering entropy at the same time. Aren't biological systems even messier than current computer systems?
- Wouldn't the 'condenser' very likely require AGI to be of any use for us?
You have a function, it codes for a gene. Only then you have it's anti-sense translation, which can also code for a gene. And then you have post-translational processing, which takes that gene-product and makes into any number of other things. And then you have DNA binding proteins which effect readability, so that gene can code for a different gene when the normal start/stop codons are made accessible or inaccessible further up the DNA strand. And then the whole program also grinds to a halt if you remove any of the "junk" because the junk is used to control execution (translation) speed and inhibit the program from self-destructing (cancer).
Capitalism. Nobody gets paid to think about the big problems.
At ~28:00 is he saying that the optimal computing device could operate 27 orders of magnitude more efficiently that what we use today?
At ~36:00, finally something to make me sit up. "content addressable store".
* https://en.wikipedia.org/wiki/Chord_(peer-to-peer)
* https://en.wikipedia.org/wiki/KademliaKurzweil has pointed out that one can think of the 10^15 state changes per second going on inside a 1Kg rock with no outside energy input as a computation device.
Ok, so at 28 minutes, he's referring to some quantum mechanical lower bound to change a bit. I imagine the rock is using that really tiny amount of energy from the outside environment.
"content addressable store" or cam (content-addressable-memory) is pretty well-known in cpu-arch, n/w equipment etc. domains. hashtables are s/w counterparts :)
Always impressed by Joe. Programming since the 60s and still programming, still writing, giving talks. He is a great role model. I wish I would be programming and be just as excited about it when I am at his age.
It's a fundamental problem - good well maintained tests help but this is cultural not technical problem.
---
# JRFC 27 - Hyper Modular Programming System
(moving it over to https://github.com/jbenet/random-ideas/issues/27)
Over the last six months I've crossed that dark threshold where the desire of building a programming language has become an appealing idea. Terrifyingly, I _might_ actually build this some day. Well, not a language, a programming _system_. The arguments behind its design are long and will be written up some day, but for now I'll just dump the central tenet and core ideas here.
## > Hyper Modularity - write symbols once
### An Illustration
You open your editor and begin to write a function. The first thing you do is write the ([mandatory](https://code.google.com/p/go-wiki/wiki/CodeReviewComments#Do...) [doc comment](http://golang.org/doc/effective_go.html#commentary) describing what it does, and the type signature (yes, static typing). As you write, [your editor suggests](http://npmsearch.com/?q=factorial) lists of functions [published to the web](npmjs.org) (public or private) that match what you're typing. One appears promising, you inspect it. The editor loads the code. If it is exactly what you were going to write. You select it, and you're done.
If no result fits what you want, you continue to write the function implementation. You decompose the problem as much as possible, each time attempting to reuse existing functions. When done, you save it. The editor/compiler/system parses the text, analyzes + compresses the resulting ASG to try to find "the one way" to write the function. This representation is then content addressed, and a module consisting of the compresses representation, the source, and function metadata (doc string, author, version, etc) is published to the [(permanent) web](http://ipfs.io/), for everyone else to use.
### Important Ideas
- exporting a symbol (functions, classes, constants, ...) is the unit of modularity (node)
- system centered around writing _functions_ and writing them once (node)
- stress interfaces, decomposition, abstraction, types (haskell)
- use doc string + function signatures to suggest already published implementations (node, Go)
- content address functions based on compressed representations
- track version history of functions + dependents (in case there are bug fixes, etc). (node)
- if a function has a bug, can crawl the code importing it and notify dependents of bugfix. (node, Go)
- use static analysis to infer version numbers: `<interface>.<implementation>` (semver inspired)
- when importing, you always bind to a version, but can choose to bind to `<interface>/<implementation>` or just `<interface>`
- e.g. `factorial = import QmZGhvJiYdp9Q/QmZGhvJiYdp9Q` (though editors can soften the ugly hashes) (node + ipfs)
- all modules (functions) are written and published to the (permanent) web (public or private)
- when importing a function, you import using its content address, and bind it to an explicit local name (`foo = import <function path>` type of thing)
- the registry of all functions is mounted locally and accessible in the filesystem (ipfs style)
- _hyper modular_ means both to "very modular" and "modules are linked and on the web"
Note: this system is not about the language, it is about the machinery and process around producing, publishing, finding, reusing, running, testing, maintaining, auditing, bugfixing, republishing, and understanding code. (It's more about _the process of programming_, than _expressing programs_). This means that the system only expresses constraints on language properties, and might work with modified versions of existing languages.
I tend to think it also has connections with a recent discussion here: https://news.ycombinator.com/item?id=8308881
As a concrete example, if I say (picking a syntax at random; underscore denotes where my cursor is):
add :: int -> int -> int
def add(a, b)
"""add two numbers"""
_
proj1_add :: int -> int -> int
def proj1_add(a, b)
"""returns sum of arguments"""
a+b
proj2_sub :: int -> int -> int
def proj2_sub(a, b)
"""returns difference between a and b"""
a-b
It allows for bringing up the worst ten percent of code without limiting the top ten. It is far far better than the Java idea of making it hard to shoot your foot off with the language.
Somewhat related to this, I have been annoyed by the way apps scatter information and have been working to find a way of managing the mess.
http://productionjava.blogspot.in/2014/07/the-broken-web.htm...
and
http://productionjava.blogspot.in/2014/08/coding-can-be-puni...
"Computers are becoming a big environmental threat. They're using more energy than air traffic."
Is this actually true? Sure, the average person spends a lot more time on a computer than in a plane, but still it seems crazy that they'd be comparable. Or at least the comparison isn't very relevant, because the Internet can significantly reduce people's need to travel.
I was doing some preliminary analysis for a small project recently, and considering various frameworks and tools. Eventually I realized I could implement what was needed using four JSPs producing static html, with a bit of styling in CSS. No AOP, no injection framework, no JavaScript. And no explicit differentiation between device types.
The resulting application will start up quickly -- which is important when running in a PaaS environment -- and should work on any browser, including weird old stuff like Lynx. Less butterfly. More rat.
- Symbiam was coded in C++
- GCC moved to C++
- Clang and LLVM are done in C++
- Mac OS X drivers are written in Embedded C++ subset
- Windows team is moving the kernel code to be C++ compilable
- JPL is moving to C++
- AAA game studios have dropped C around PS3/XBox 360 timeframe
- It is the only common language available in all major mobile OS SDKs
Decision to move Windows to C++ was announced by Herb Sutter in one of his Microsoft talks and the Windows 8 DDK was the first to support C++ in kernel code.
JPL going C++ was covered at their CppCon 2014 talk.
GCC switched to C++ as implementation language in 2012.
In any case, I would say, all languages with native code compilers can be used in relation to energy efficient computation.
It is just that C and C++ are the only ones on the radar of developers (and managers) looking for mainstream languages with native compilers.
Let's ballpark it (fill in better numbers if you have them). Assume that a person takes an average of one 5,000 km trip by plane per year. ATW, a Boeing 737-900ER uses 2.59 liters of fuel per 100 km per seat, so that would be about 130 liters of fuel. Also ATW, Jet A-1 fuel has an energy density of 34.7 MJ/l, so about 4,500 MJ for the trip. A watt is one joule per second, and there are about 31 million seconds in a year, so the continuous power output equivalent for the plane flight would be about 145 watts. So, if all your computers put together are consuming more than ~150 watts, they're consuming more than the equivalent of a 5,000 km plane flight over the course of a year.
This is a very rough estimate, but it doesn't appear unreasonable on its face that his statement could be true.
However he also mentions we probably can't do a lot more with computer to reduce their power consumption.
It depends entirely on how much they get used! An SUV that gets driven once a year contributes less carbon than a Prius that is driven all day every day.
Comparing the aggregate numbers is obviously the only way to compare computers and airplanes anyway.
$ cp hash://<somehash> .
There was URN https://en.m.wikipedia.org/wiki/Uniform_Resource_Name many moons ago that is still used. A URN resolver is a software library that could convert that identifier to a URL.
URLs aren't much different from URNs but they actually specified a default resolution algorithm that everyone could fall back on. They were more successful because there was less need to separate identifiers and locators than originally thought, though it's still a debatable point whether the results are intuitive (eg. HTTP URLs for XML namespace identifiers which may or may not be dereferenceable).
HTTP URLs took advantage of DNS as an existing globally deployed resolver, coupled with a universally deployed path resolver (the web server) the rest was history. You could create a URL scheme called "hash" but it would be hard to see how you could design a standard resolver unless it was one big centralized hash table in the sky - you still would need to, at the very least, map objects to IP addresses.
They do, but that does not mean there should not be other ways to access data. Hashes are universal and unambiguous. There should be a way to retrieve a file given its hash.
> You could create a URL scheme called "hash" but it would be hard to see how you could design a standard resolver unless it was one big centralized hash table in the sky - you still would need to, at the very least, map objects to IP addresses.
There would be an underlying P2P protocol that cp would use. On the other hand, cp doesn't even use FTP or HTTP so maybe that's too much to ask.
Maybe with curl or wget, then.
I'm not sure you've thought through the complexity of what you're asking for.
Hashes require (a) a hash function everyone agrees to, (b) a way to resolve them to an IP address.
Unless you synchronized all global hashes across the Internet on everyone's computer (the git hashed project model -- which we know doesn't scale beyond a certain point unless you bucket things into independent hashes you care about), you'd basically have to do something like Hash://ip_address/bucket/hash or hash://bucket/hash if you want to give a monopoly to one IP address that manages giant hash in the sky
Which is back to URLs and HTTP, and no different from say Amazon S3
Theoretically speaking, isn't it possible to create a virtual BitTorrent FUSE filesystem?
It's not like you're going to type that in. You're going to copy and paste it from somewhere. So it's just as good to use
http://releases.ubuntu.com/14.04.1/ubuntu-14.04.1-server-amd...
as
hash://b4ed952f6693c42133f73936abcf86b8
In either case, your computer can do whatever it takes to get the file. With a useful URL, you'll have a reasonable notion about what's coming down and whether it matches your intentions.
Without that, the very natural question is, "Did I get the thing I wanted?" For example, it would be easy to paste the wrong hash code.
There are other benefits, like real-time binding. A hash is going to point to a particular sequence of bits. But you may not want a particular file, but rather the best current mapping from an idea to a file. E.g., if Ubuntu discovers a an issue with their released ISO, they can make a new one and replace what gets served up by the URL.
I wouldn't. I'd make a symbolic link.
Basically the current directory/names structure would be an abstract layer above the hash-based system.
> aria2c magnet:?xt=urn:btih:1e99d95f....
Edit: Strange Loop is a multi-disciplinary conference that aims to bring together the developers and thinkers building tomorrow's technology in fields such as emerging languages, alternative databases, concurrency, distributed systems, mobile development, and the web.
Strange Loop was created in 2009 by software developer Alex Miller and is now run by a team of St. Louis-based friends and developers under Strange Loop LLC, a for-profit but not particularly profitable venture.
I had to stop watching when a laptop was compared to a black hole.
I'm sure the laymen are impressed though.
> Please comment if you downvote. Or even better; just comment.
So here you go: you are utterly clueless, before you write comments like these do your homework or you will get downvoted a lot.
Or even better; just don't comment. Until you've done your homework, that is. It just increases the noise and does not add to the conversation at all.