hp

An alternative to C++ in this situation is to have a C core or set of C libraries, with bindings up to something like JavaScript or Lua. World of Warcraft, Emacs, Firefox are a few popular examples of that architecture. GNOME 3 works this way too.

It's pretty important to have good automation for the C-to-high-level conversion, for example GNOME has a gobject-introspection to do this, and Firefox has XPCOM. Otherwise it becomes too tedious and bug-prone to be gluing two languages together by hand all the time.

An old blog post http://blog.ometer.com/2008/08/25/embeddable-languages/

I suppose node.js could be considered another example of this approach, by writing the super-tiny-and-fast event loop and http parser in C and then putting all the application logic in JavaScript.

it seems to allow duplicate keys but drops all but one. I have no idea if this is correct since the JSON spec doesn't really cover it. personally I think the spec should ban it because it means you need to parse the closing brace for an object before you know you have the right value for any field. also it means you need to keep a dictionary-like representation in memory to detect the dups, a list or stream of pairs wouldn't consolidate dups.

a nitpick for sure but several times I've wished the JSON spec had something to say here.

A standing desk eliminated my arm and wrist pain, for whatever reason. I think I move around more when standing and maybe tend to rest some body weight on my arms when sitting.

Don't know what to tell you. I was in the room and writing the code on a lot of this, and what you're saying doesn't correspond to the whys and the whats that were on the whiteboard at that time.

"It is hard to have two smart, outspoken, and opinionated people work on the same piece of code."

But all the stuff discussed here - dbus, gstreamer, EWMH, GNOME, etc. - has had dozens (e.g. EWMH, dbus), hundreds (e.g. gstreamer) or even thousands (e.g. GNOME, Fedora, Ubuntu) of contributors. And that's not counting all the people that build on top of those things, it's only counting the ones who contribute to them directly.

"it is infinitely easier to create in Linux than it is to fix"

I've always found in open source that it's harder to find people to create, than to fix. I mean yeah, there's a background noise of a thousand 1-person projects being born and dying every day. But the big projects with momentum are full of dedicated people primarily interested in incremental change.

Most of the technologies we're talking about here are in the range of 6-12 years old, with no significant overhaul or replacement in that time. For perspective, Firefox (as "Phoenix") appeared 9 years ago, and Mac OS 10.0 is 10 years old. It feels tough to argue that Linux is moving faster than Apple, Microsoft, Google, web tech, etc. It's relatively stable as OS's go.

Sure, Solaris and IRIX are (were?) even older and there was prior art on all sorts of fronts. If you'd like to argue that the original Linux desktop efforts should have copied more from those: you're probably right on some of the specifics. It's easy to say this or that could be slightly better if you look at a huge piece of software like a full Linux distribution. What counts is the software that exists, not the software we all coulda woulda shoulda written.

There were a few hundred people who probably worked on or around Linux desktop IPC back then, and I think zero argued that SUNRPC was a good option. Maybe it was, and someone could have showed up to prove it in code. They did not. Instead, a number of other systems were coded and tried (MICO, ORBit, DCOP, IPC-over-X11, even SOAP), and in the end dbus caught on as a working solution. By that time everyone had a lot of hard knocks and knew what problems they were trying to solve. All the solutions people tried worked fine for sending a message. That was not what differentiated these approaches. The problems to solve included things like how to cross boundaries between systemwide daemons and user session; how to discover, activate, and track other apps and daemons; licensing issues; a least-common-denominator implementation that all the projects were willing to use; security model; etc. At some point dbus cleaned up everybody's ad hoc hacks and experiments, and now Linux is pretty uniform about using it and has been for years. Is it perfect? Not at all. It was just the first thing to be good enough and it stuck.

If someone comes along and does something legitimately better and worth switching to, then I'm sure Linux will do so, and take a lot of heat for it too.

"So rather than point out how wrong he is, ask 'what is he trying to say?' and deal with that."

Well, I think he's trying to say what he says, which is "please don't write software which requires any of the Gnome/KDE and DBus API. Writing X11 programs with xcb and proper RPC APIs like SUNRPC or Thrift should be more than good enough."

This is nonsense.

The idea to use raw xcb rather than GTK or Qt or HTML: come on. You'd spend months getting to the point where you had crappy buttons and scrollbars working. Replicating user-expected and mandated functionality provided by the toolkits is a multi-year task to do _poorly_. You'd never, ever finish writing your app (and it'd suck, too).

On the IPC front: you'd be adding yet another way to do it and thus more complexity. It's fine to say SUNRPC should have been chosen in 2001, but it wasn't, and rewriting hundreds of apps today is nuts. Whatever your dbus annoyances, you could solve them in one place and fix the whole system.

More importantly, most of the newfangled (= 6-12 years old) crazy ideas that this post complains about, exist for some good reasons that the author of the post doesn't seem to be aware of. You could certainly build a system _involving_ SUNRPC or Thrift that would work. But you'd have to innovate on top with an understanding of the problem space. And what's the end-user benefit of that, at this point in time?

I'd argue it's a big old zero.

But if someone shows that there's enough benefit, I hope a new idea wins on the merits (and the running code).

yeah, it's a tradeoff. If the software does more, then the software is more complex... OK, but, sometimes it's nice that it does more. People are used to other systems (iOS, Android, Windows, MacOS) and those are setting the bar pretty high. They are all extremely complex systems that do a lot.

Everyone wants a simple system... as long as it has just this one thing that they need... and this one other thing...

This author seems to feel there was some way in which the software could do everything it does and there would be no downsides... you know, here and there in some detail it's probably true that the tradeoff is wrong. But that's just saying "all software could be better" or "all software has bugs" or something - true, but not an actionable insight.

I get the guy's frustration. But you know, there's no need to wrap the emotion up in non-factual hypotheses about source code that one is not familiar with.

Software sucks. We all know it. Using your imagination to diagnose why isn't going to get anyone anywhere ;-)

There probably are some improvements possible if we all go look at the source and get the real info.

This article is not well-informed. I worked on or sat next to people who worked on a lot of the stuff mentioned. So you can take me as biased or as having a clue or both as you wish.

A general point, the changes described here have been over the course of something like 15 years. So the article seems to be making a "stuff keeps changing!" point... but we are talking about over 15 years. Think about changes to hardware, the Internet, etc. over that time. And most indicators are that the Linux desktop has moved much too slowly compared to say Windows, Mac, Android, and iOS.

Some examples of errors:

"So the Gnome developers wanted to reduce the complexity of their protocol as well and started working on a protocol which was supposed to join the advantages of DCOP and CORBA. The result was called the Desktop Bus (dbus) protocol. Instead of complete remote objects it just offers remote interfaces with functions that can be called."

This is false on several levels. dbus was mostly a kind of cleanup of DCOP for general use, with no intent to "join the advantages of CORBA" which were essentially none. I can make no sense of "instead of objects it offers interfaces" - it has both objects and interfaces, and pretty much can implement the same kind of API that DCOP does (I believe KDE even did that). Basically this paragraph doesn't mean anything I can relate to the actual technology.

"APIs to abstract the uses of OSS, esound and ALSA: gstreamer for Gnome and Phonon for KDE"

This is wrong. GStreamer is for making graphs of elements, where elements are decoders, encoders, effects, filters, etc. and can be both audio and video. There is one kind of element ("sound sink") that does abstract sound output, as you would imagine. There are some other elements that use sound APIs too. But GStreamer is not the same thing as a sound API like ALSA, in any way shape or form. It's for building multimedia _apps_, sort of a media toolkit.

Moreover, the main reason to replace the older tech here (OSS, esound) was just that it didn't work very well and didn't support a lot of the things sound cards do. It's not like keeping that old stuff was an option, since it could barely play beeps.

"it is no longer possible to run the system without a graphical user interface"

I'm just not sure what planet that's on. There sure are a lot of headless Linux servers out there in the world, and it's pretty obvious that the large Linux distributions care about this intensely.

Re: NetworkManager, if it's somehow needed when headless and not configurable headless, that would be considered a bug by all involved. Just a matter of tracking down the details and reporting them if they have not been. All the Linuxes aspire to (and in my experience do) support headless operation.

"they don't implement the original X11 protocol directly and rely on so-called window manager hints."

This sentence is total word salad. X11 has had window manager hints for two decades. What's new is "extended window manager hints" which are some new hints in the same spirit ... in order to do new things. They don't "wrap" anything, so "directly" is just gibberish. Kind of like how CSS 2.0 isn't the same as CSS 1.0, you know? This complaint is equivalent to bitching because you can't use IE5 on the modern web anymore. The protocols are documented, and you have to use an implementation that implements something from within the last 5 years. The extended window manager hints range from 6 years old to 10 years old, so that's how old a crap we're talking about.

An almost exact translation of this claim to the web is: "they don't implement the original CSS 1.0 directly and rely on so-called CSS 2.0 properties" ... see how that makes no sense?

"Writing X11 programs with xcb and proper RPC APIs like SUNRPC or Thrift should be more than good enough."

This 100% misunderstands why dbus is used. The first goal of dbus is not to send a message from process A to process B; it's to keep track of processes (help A find B, have them each know when the other goes away). The messaging is important but in many ways secondary.

Overall, the article doesn't understand the big picture of why all this new stuff was needed. I think there's one big reason: dynamic change. The old ways of doing things almost all involve editing a text file and then restarting all affected applications. But to implement the UIs that people expect (as you'd find on iOS, Android, Windows, Mac), everything has to be "live"; you change a setting in the dialog, and the whole system immediately picks up on the change. You unplug a cable, everything notices right away. etc. The daemons are because so many pieces of dynamically-updated live state are relevant to more than one process or application. That's why you have a "swarm of little daemons" design. And guess what: some other OS's have the same design.

That's (at least one of) the major problems being solved. And the author here gives no indication he knows it exists, let alone his proposed alternative approach.

I sort of get the inspiration for the article: Linux has been trying to keep up with modern UI expectations without having enough staffing for that really, and certainly regressions have been introduced and there have been bugs and things that could have been better. On the 6-month distribution release cycles, users are going to see some of that stuff. It's software, people. And it's understaffed open source software to boot. So yeah, legitimate frustration, shit changes, sometimes it breaks. I get it.

But there's no need to wrap that frustration up in pseudo-knowledge as if it were a technical problem, or say inane things about getting back to the "unix way"; if someone could show up and make the desktop UI stuff behave well with the "unix way" they would have done it. Or maybe they did do it, and the critics understand neither the problem requirements nor the "unix way." Just saying.

Yeah, it sucks. It's pretty hard to fix though, one of those bugs where the multi-week or even multi-month effort to fix it probably isn't ever going to make sense to anybody. (Say IDLE in particular is bothering you; you could probably spend a very short time just hacking it to truncate that string before putting it in the widget, vs. the likely gigantic task of fixing the widget. Everyone is making that same calculation whenever they get annoyed by this.)

Back in the era this is talking about, I ported the Tk text widget over to GTK+ which became GTK's still-used text widget (GtkTextView).

I think the author is right that it was a mistake to ignore Linux. It didn't have the desktop user marketshare, but it did have the developers that were doing a lot of the work on open source code.

Even at that time (just before GTK+ 2.0, around 2001), expected toolkit features had jumped ahead of Tk quite a bit; in porting the widget I think I added model-view separation, pixel-based (vs. line-based) scrolling, international text layout, input methods, accessibility, lots of optimizations, support for non-XPM images... likely more I can't remember.

Tk lacked all that stuff, and various GTK-related companies and Troll Tech (Qt) were investing in building all of it for GTK and Qt. That led to a situation where only those two toolkits, that had been modernized, were acceptable to lots of decision-makers who were deciding what to ship with Linux distributions or GNOME or KDE. Nobody could afford to go modernize Tk also, it was easier to just get all the apps on one or two toolkits. The modernization work was being funded or volunteered-for with Linux in mind, nobody cared to do it for open source toolkits on Windows or Mac. Swing was probably the only cross-platform toolkit that had the same modernization work done. (Ignoring "wrapper" toolkits like wxWindows or SWT that have whatever modernization is found in the toolkit they wrap.)

The Tk text widget was a solid piece of work, I mean, it's an accomplishment to write code good enough that it makes sense to port it to another toolkit. The core btree data structure from Tk is in GTK to this day with extensions but no real major overhaul: http://git.gnome.org/browse/gtk+/tree/gtk/gtktextbtree.c I think Ousterhout may have written it.

It's an interesting data structure if you like that sort of thing. The older GTK text widget had various operations that were O(n), and so people kept writing O(n^2) or worse code accidentally; with the new one, one of the goals was to try to keep everything below O(n) so it was harder for developers to hose themselves. The core of this was the btree from Tk. One place we declared defeat was in very long lines; some stuff is still O(n) where n is the line length, and that's hard to fix without massive overhaul. The btree is about storing lines and related metadata so that things aren't O(n) in number of lines. There's also a useful trick in http://git.gnome.org/browse/gtk+/tree/gtk/gtktextiter.c where an incrementing "change stamp" is used to keep an iterator valid even though the iterator caches some pointers that can become invalid as the btree changes.

Sorry for the "used to walk both ways in the snow"-flavored post ;-)

That said, I'm not sure UI matching the implementation is the problem with git's UI; it's more that git's UI is inconsistent, has poorly-chosen names for commands/options, and has bad default behaviors.

Elijah has a nice detailed breakdown of the EasyGit UI rationale as well: http://people.gnome.org/~newren/eg/git-eg-differences.html

I've been using EasyGit for years, it's very good and addresses exactly the complaint made in this article. Plus it's a one-file script so it's simple to drop it onto whatever computer you're using.

One source of Scala's design is Java interoperability, much as C++ has to live with its C legacy. This compromise may also be why people are able to use Scala in practice, though; they need to talk to Java libraries or need the performance of a language that maps straightforwardly to the JVM.

Scala is a functional/object-oriented hybrid, making it more complex than a purist language in either mold. It always lets you do things in a Java-like way - you can use it as "Java with less boilerplate" and touch almost nothing Scala-specific. Then it adds functional programming alongside.

To me this feels very natural; I like objects for the big-picture structure, but I like to write algorithms and manipulate data in functional style. If you need raw performance in some hotspot, write a Java-like while loop with mutable state; otherwise, write something nice and high-level (and the JVM will still be much faster than a "scripting language" however you define it, e.g. http://blog.j15r.com/2011/12/for-those-unfamiliar-with-it-bo...).

Scala does fix some Java warts that are legitimately complex or confusing in their own right. For example, primitive and boxed types are less strongly separated; there's no "static", just nice syntax for singleton objects; collections are 100x nicer with far less noise; a nice multiple inheritance design; covariance eliminates a bunch of nasty hacks; better ways to specify access controls; there's a decent way to factor out exception handling; case matching is _awesome_; and _so_ much less boilerplate in general.

I'm not sure the static vs. dynamic religious war can ever be resolved, but static types feel less broken and less verbose in Scala than in Java. Java makes you lie to the type system, or do something unnatural, much too often. Scala hasn't cured every such situation, but it's cured a lot of the most common ones, and greatly reduced the need for manual type annotations.

Scala gives you the conciseness of Ruby, but with static type checking, higher runtime performance, and interoperability with existing Java code.

Some tradeoffs of static type checking remain, such as compilation times.

People do go on wild goose chases trying to push the language farther than it's ready to go. I've done it myself. I agree with the article that there are lots of areas to improve and appreciate the constructive write-up.

But on the other hand, the perfect shouldn't be the enemy of the good. I certainly would not choose to go back to Java, even as I'd love to keep seeing Scala get even better.

I found it essential to read Martin Odersky's Programming in Scala book - if you haven't then I recommend it. It makes many things clear that can be hard to pick up using only the free online resources.

There's also a new docs site and it's getting better all the time: http://docs.scala-lang.org/

To me, a big advantage of Scala's "enrichment" over monkey-patching in Ruby or JS is that it isn't global. That is, you have to import the enrichment. Another code module in the program won't be unexpectedly affected by it.

In practice, I almost never use monkey-patching in dynamic languages because it's too dangerous. While in Scala there are cases where enrichment won't work, you can always just write a regular function in those cases, and there are lots of cases where enrichment _does_ work...

On Heroku, you can use any Scala version that SBT will use. I even used parallel collections in the sample app: http://devcenter.heroku.com/articles/scaling-out-with-scala-...

Play support came out a while ago: http://blog.heroku.com/archives/2011/8/29/play/ One trick with play-scala is that the default welcome page is broken because it depends on a module that's only enabled in dev mode, and Heroku runs play in prod mode. So replace the default welcome page with a real page.

The announcement today is for SBT-based projects. Anything you can build with SBT should be possible to use. You have to figure out how to honor the Heroku configuration variables (namely PORT for the port to listen on) and you have to figure out how to get an executable command, for example with https://github.com/typesafehub/xsbt-start-script-plugin

We put up two examples today, the Finagle one and the larger "web words" one: https://github.com/typesafehub/webwords/tree/heroku-devcente...

Web words example shows how to run Jetty, and you should be able to drop various web frameworks into Jetty.

If you follow the "how to run it on Heroku" section in the README here: https://github.com/typesafehub/webwords/tree/heroku-devcente... then you would get something like this running app here: http://webwords.herokuapp.com/

That's pretty much what it does ;-)

Some people call this a "Platform-as-a-Service" or PaaS.

hey, I was just idly chattering about C and the way people in general often approach it. Not intended to be a review of an unwritten book or imply that you plan to approach it in any particular way.

I am a bad ass of course. But I thought it was relevant to the comment that I've written a lot of C.

I was just going off on a generic tangent, obviously I don't know what your book will be like. Just talking about C since it's 1am and vaguely on-topic.

Maybe "low level" is a bit ambiguous.

I certainly agree that C is most appropriate when you are "low in the stack," just above the operating system and maybe implementing something like a virtual machine. I don't make a habit of writing stuff in C for no reason and the vast majority of programming these days isn't and shouldn't be in C (or C++ for that matter).

However, "low in the stack" is different from "low level" like "I refuse to use modern practices" or "I get to omit half the letters from my function names." You can be coding an on-the-metal kind of thing and still think about it in a high level way.

Agreed, but it's still sort of a weird case, I think. The considerations related to things like memory allocation, performance, concurrency, internationalization, security, IO, etc. are pretty different in the kernel.

Ohloh says I've changed at least half million lines of C code (https://www.ohloh.net/accounts/rhp/positions/total) Play me a tiny violin ;-)

What kinda bugs me is that whenever people go to teach C, they make out like it _has_ to be a low-level exercise, as if writing in C suddenly means you can't use abstract data types or object-oriented style or name your functions properly or have Unicode support.

For example, people teach libc string APIs like scanf() and strtok(), which should almost never be used. (See http://vsftpd.beasts.org/IMPLEMENTATION for one take.) Instead, use http://git.gnome.org/browse/glib/tree/glib/gstring.h or write your own like http://cgit.freedesktop.org/dbus/dbus/tree/dbus/dbus-string....

If you're going to display user-visible text, you are pretty much required to link to GLib or another Unicode library, since libc doesn't have what you need (unless you want to use the old pre-unicode multi-encoding insanity).

Don't use fgets() and other pain like that, use g_file_get_contents() perhaps, or another library. (g_file_get_contents is in http://developer.gnome.org/glib/stable/glib-File-Utilities.h...)

You need help from a library other than libc to deal with portability, internationalization, security, and general sanity.

Maybe more importantly, a library will show you examples that in C you can still use all the good design principles you'd use in a higher-level language.

I told someone to "use a string class" in C recently for example, and they said "C doesn't have classes" - this is confusing syntax with concepts.

C requires more typing and more worrying about memory management, that's all. It doesn't mean that all the best practices you know can be tossed.

There's a whole lot to be said about how to write large, maintainable codebases in C, and it can even be done. It's not something I would or do choose to do these days, but it can be done.

One other thought, two of the highest-profile C codebases, the Linux kernel and the C library, have extremely weird requirements that simply do not apply to most regular programs. However, a lot of people working in C or writing about C have experience with those codebases, and it shows.

It would be pretty simple conceptually (maybe not practically) to make node.js work in an actor-like way, here's a piece of toy code I wrote that does it for JS (not node, but no reason the same couldn't be done for node): http://blog.ometer.com/2010/11/28/a-sequential-actor-like-ap...

By "actor-like way" here I just mean a code module ("actor") sees one thread (at a time), and the runtime takes care of the details of scheduling threads when a module has an event/message/request to process. Also I guess avoiding callbacks. But you could be more Erlang-ish/Akka-ish in more details if you wanted.

node.js punts this to the app developer to instead run a herd of processes. In most cases that's probably fine, but in theory with one process and many threads, the runtime can do a better job saturating the CPU cores because it can move actors among the threads rather than waiting for the single-threaded process an actor happens to be in to become free. The practical situations where this comes up, I admit, are probably not that numerous as long as you never use blocking IO and are basically IO-bound. (Only CPU-intensive stuff would cause a problem.)

btw this has been hashed out to death on the node.js list: http://groups.google.com/group/nodejs/browse_thread/thread/c...

if nobody could show their portfolio, then nobody would get an advantage from doing so.

but since people can show it in the programming world, you are at a disadvantage if you don't.

if a civil engineer had some way to show what they could do, I'm sure it'd give them an advantage.

You'd go with a graphic designer with a portfolio you could look at every time, right? Say the graphic designer's previous employer didn't let them put stuff in the portfolio -- I don't know, maybe they only worked on NSA internal marketing -- that's the designer's problem. They aren't going to say "no fair," they're going to find a way to do some work to put in a portfolio.

Wouldn't have to be open source work, just work you could show.

It's tough to hear, but the goal of a new employer is not to give us what we deserve for working hard and being skilled.

The employer's goal is to maximize their chances of hiring someone good.

If a hiring manager has two people that seem about the same but they can see the code one of those people wrote, it's a no-brainer to go with the one who has a portfolio.

Agree with you that some lame open source patch doesn't matter. But if someone's done significant work in public, or even has non-open-source code they're able to share, ignoring that would be an insane choice for the hiring manager to make.

As your parents may have mentioned, life isn't fair.

As the ad in the in-flight magazine says (I think I remember right), you don't get what you deserve, you get what you negotiate.

The impact on your visibility, personal brand, and ability to find future roles should be a factor when you sign up for a job. Just as you'd consider your job title, for example.

If you're signing up for something you can't talk about or show off -- worse, if you're signing a contract that says you can't do anything that you _can_ talk about on the side -- then you'd better be sure you're getting paid extra, or in some other way getting compensated.

From what I've seen, working on super-secret trading software for Wall Street _does_ pay a lot more than working on an open source project, on average.

If the price is that you have to use references and other means to show future employers what you can do, then that's the tradeoff.

If you're not getting paid much, have no spare time, aren't allowed to code in your spare time, etc. then those are some items for the "cons" column that might nudge you to look for something new, all else equal...

* One practical question is whether you can do a _useful_ subset of your stuff as open source... if the open source part is worthless without the "proprietary icing" then nobody will care about it, so you get no value from open source.

* A related practical question of course is whether there are customers who will still want to buy the proprietary icing even though there's an open source useful subset.

* How hard is it for competitors to replicate your proprietary icing? Is the community likely to replicate it?

* One of the main competitive advantages of most open source companies is that the key upstream developers behind the project work for the company. Do you have these developers? If so others will be at a disadvantage when it comes to offering support and services.

* Do you want a business that is based on license fees or one based more on support and services?

* Do you have a good way to market the product and get a lot of people interested, other than through an open source project?

* Do you want to spend time on community building and ensuring outside contributors can be productive? Can you put development discussions in the open?

How many companies have this problem? The most common problem in projects I've worked on has been the lack of someone getting the word out. As a developer, a number one thing I'd be looking for in a cofounder would be someone who can network and publicize - and make a good impression while doing so!

It's tempting to say that you can ignore within-tech-industry networking and only talk to customers, but ... it's pretty important in a lot of situations (funding, partnerships, serendipity, hiring) that the tech community has heard of your company. And they can be some of your first customers, too, in a lot of cases, since they're interested in trying new things for their own sake. And you can get advice. And find employees.

I bet most tech companies need to network more in addition to talking to customers more. Coding is what comes naturally...

I agree with the impulse to split stocks/bonds about equally: http://blog.ometer.com/2010/11/10/take-risks-in-life-for-sav...

But, the 2x leveraged fund is a Bad Idea. These just don't make sense or do what you think beyond the 1-day horizon. If you want a long-term leveraged bet on oil, one better way is to buy a long-term call option ("LEAP") on an ETF. That will work a lot better than the 2x fund. Not saying you really want a long-term levered bet on oil, but if you did, the option is a better approach.