Hacker News Reader

jdefargeon On Disk IO, Part 3: LSM Trees

By the way, there's an interest research on designing a new file system based on LSM tree that leverages SSD's sequential writing, as it is mainly motivated by SSDs' shortcomings with random writes: https://www.usenix.org/legacy/event/fast/tech/full_papers/Mi...

jdefargeon On Disk IO, Part 3: LSM Trees

You are absolutely right. Just an addendum: there are modern B-Trees that are optimised to leverage the cache lines, part of the so called cache oblivious data structures ( http://erikdemaine.org/papers/CacheObliviousBTrees_SICOMP/pa... ). A cache oblivious B-Tree is the basis of Percona's TokuDB storage engine: https://www.percona.com/doc/percona-tokudb/ft-index.html

jdefargeon James Golick has died

A big loss to computing, a terrible loss to family and friends. My sincere condolences. :'-(

jdefargeon Results of the GitHub Investigation

Exactly! I had the curiosity to check JAH's github profile too, and it raised more suspicious towards her than I had previously imagined. For a fair comparison, let's see a couple of female github repos (both frontend and backend):

https://github.com/pamelafox?tab=repositories

https://github.com/kellabyte?tab=repositories

https://github.com/gwenshap?tab=repositories

See? Any of those smart women above has much more quality code in their repos with regular activity, even though not any of them works at github, that is, they push code on their free time.

Finally, let's see a repo by another harassed geek girl: https://github.com/adriarichards?tab=repositories

Um... I definitely see a pattern here screaming too loud not to take notice.

jdefargeon Julie Ann Horvath Describes Sexism and Intimidation Behind Her GitHub Exit

"Julie Ann, Adria Richards and Shanley enter a bar: the bar just explodes because it's a symbol of the patriarchy."

But, seriously, I know various female programmers that I could trust not only my project, but my LIFE to. And as in the case of men, those are usually the low profile, highly productive, people. I doubt any "famous" developer this days in spite of gender/race/whatever. As with any hyped profession, there are a LOT of impostors trying to succeed without basic qualifications in computing. Hint: those are the ones who shout the loudest, have the highest number of followers on Twitter, but have very little, if any, REAL code on Github or any other public repositories. So... when the masks start to fell off they usually bail out their jobs loudly and pointing fingers to preserve their public personas. I doubt this is not the case of Julie Ann.

"You can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time.", Abraham Lincoln

jdefargeon Flappy Bird creator, Dong Nguyen, taking down game; "I can't take this anymore"

The optimal number of moves follows this recurrence relations: T(1) = 1 T(N) = 2 * T(N-1) + 1

jdefargeon Ask HN: Why is ExtJS not getting any traction?

TL;DR: because ExtJS sucks.

Long answer:

1. The learning curve is really steep. I mean, really really steep. There's a critical shortage of (good) books, video, and tutorials on the web. I guess ExtJS' learning resources should be 0.1% of jQuery's. The examples on Sencha site were still using the 3 version last time I checked;

2. If you need to customize ExtJS, or need something slightly more sophisticated than CRUD screen, then you are pretty much on your own. You'll spend a large amount of time trying to "fight" the framework and your chances you succeeding are very low;

3. It's expensive and its open source license has restrictions;

4. Sencha seems to hide and obscure the access to the learning resources so that you are pushed towards their paid support.

5. It's slow;

6. The error messages (when they show up!) are cryptic.

7. ExtJS was re-designed for version 4, but some things like the Model objects are cumbersome and suffer many usage limitations;

8. It will never be adopted as a general purpose solution on Internet facing webapps for the reasons exposed above;

jdefargeon Benchmarks of Cassandra, HBase, VoltDB, MySql, Voldemort and Redis

Why they didn't tested MongoDB or Riak? Is Voldemort really more relevant than Riak or MongoDB? WTF?!

Sorry, boys, but this paper looks like the result of a ill performed research job, done in a hurry of getting published.

jdefargeon Why I Left Google

Java was viewed as being "good enough"; alternatives like Scala and Clojure were not considered.

If a company enter the what-is-the-hottest-language-of-the-week game it can easily drag itself in a downward spiral of death. Just because it boils down to taste! And it leaves a lot of hurted feelings around the way. Language is a lot about maintenance 10 years down the road.

Scala and Clojure each have significant design flaws, in my opinion, and neither would have been a significantly better choice.

Couldn't agree more! Be it a startup or a mega-corp, VPs should make their minds upfront and stay on their path until something amazingly better comes down the road. C'mon, if Google had chosen Scala, for example, they could have produced clean, concise code, FP-oriented software at the price of sluggish compilation times (many wasted minutes), lots and lots and lots of generated bytecodes, and binary incompatibility (ouch!). Jump into the trendy language badwagon and you find yourself nowhere pretty soon. I know of at least two prominent startups in Bay Area who are switching of Scala and adopting the old-fashioned Java, for many reasons, but fondness of OO or lack of vision are not among them. And a third startup is stealthily switching to Clojure. On the other hand, you have C/C++.

jdefargeon F1 - The Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business

F1 is a successor of Megastore, even with relation to high write/read latencies. From bits leaked here and there by Google, one can safely assume they put highl value on cross datacenter consistency nowadays even if it means to hurt write ratios so much -- I recommend this video of Google IO 2011: http://www.youtube.com/watch?v=rgQm1KEIIuc (from minute 35:40 on). Megastore and F1 present abysmal latencies because they synchronously write to 3 data-centers at least before acknowledging the client side through Paxos or 2PC.

Only an idiot can assume that F1/Megastore is a drop-in replacement to MySQL (or Cassandra, jbellis!), but those guys invented BigTable, and battle tested it, long before writing the papers so they know where their priorities lie nowadays. On the other hand, I am highly curious about Spanner, the successor of BigTable, that powers F1.

jdefargeon F1 - The Fault-Tolerant Distributed RDBMS Supporting Google's Ad Business

It's an industrial presentation. As such, there's no paper about it, just the talk given at SIGMOD and the slides. When Google unveiled Megastore, F1 predecessor, it did it as an industrial presentation too. Years, and many improvements later, they published a paper on Megastore.

jdefargeon Loop: a programming language for the JVM inspired by Haskell and Ruby

Good to know there are newer takes on JVM based languages! I like its minimalist design as opposed to being a kitchen sink language. Good luck!

jdefargeon Aussies' fix for 'stagnated' email

Java, Guice, jQuery, and JAX-RS, at least. Dhanji has been deeply involved on those technologies for a long time.

jdefargeon What's new in Scala 2.10

I don't see any candidate either, and it will not be Scala, for sure. I think Scala will continue to be a pretty interesting language, used in pretty cool places, for a smart and enthusiastic community, as Haskell or Clojure, but it will not replace Java as the major JVM language not even in a million years. Smart engineers will develop in Scala, Assembly, Lisp, whatever, but it's the average developer who accounts for the use of a language worldwide. And Scala will loose those developers from the start.

I also agree that Java will be dead by 2014, but not as COBOL. It will be dead as C language is dead. Java is the C of the XXI century.

jdefargeon What's new in Scala 2.10

Please, Scala maintainers, stop adding features. You'll not please every developer in the world and it's not cool to be a kitchen sink. Again, just stop!

jdefargeon The Pirate Bay Will Stop Serving Torrents

Pirate Bay is the BEST torrent site ever created. It's a pity they are switching to Magnet. :( It would be so cool if they provided both (Magnet and Torrent)...

jdefargeon Damien Katz' interview on abandoning Apache CouchDB

CouchDB has always been the least competitive alternative in comparison with other (better designed?) NoSQL systems like HBase, Cassandra, Riak, Redis, and MongoDB. The progress of those other systems has been steady -- during the last few years, CouchDB has lost its position as the major nosql document store to MongoDB -- while CouchDB's community trudged on uncertainty and bad design decisions. It was a matter of time until CouchDB just vanished.

Damien Katz is just assuming the above reality and restarting from scratch . He's always stated that he was investing a lot of time, and sacrificing many things, so that he could support his family AND work on something relevant. Nevertheless, I dislike this strategy of abandoning a project by "donating" it to the open source community. Just call it quits, for God's sake!

jdefargeon The Rest of the Story (Yammer & Scala)

Coda's opinions, even if leaked, were highly appreciated. This guy has invested a lot of time on Scala so it's in better position to say what he thinks about the language. In the end, it should help to fix imperfections and help the community.

jdefargeon What Does It Take To Get A Job At Google? (Infographic)

There are some traits that can be easily observed on Googlers and Xooglers. Particularly, if you meet at least two of the requirements below then they are definitely going to screen your resume more carefully:

1. Holds a BS/MSc/PhD from a famous US university (Stanford, Yale, Berkeley, etc) or a PhD in a known university.

2. Has publications or is a well renowned CS researcher/professor/veteran (including famous progr. lang. gurus)

3. Is/was the committer of a famous open source software

4. Has invented algorithms or data structures or programming language

5. Has worked on a major company like Microsoft, Oracle, or Intel;

6. Has been the finalist of programming competitions like ACM, TopCoder, etc.

jdefargeon Brin on Yegge's post: "I stopped reading it after the first 1,000 pages"

LOL, Brin was right on target. I ask myself how on earth can I guy write SO MUCH of a blog post and still have time to get things done. If Yegge is not not able to summarize his POVs in one page or two then it's better not to try at all.

jdefargeon Oracle does NoSQL

Oracle is doing a NoSQL as a strategic move. They'd done this before with the acquisitions of BerkeleyDB's and Coherence's companies. But their cash cow will continue to be their ubiquitous RDBMS for many years to come. You know, just in case the world wakes up one day and falls in love with NoSQL, they will have something to sell under this brand.

I deal with passionate Oracle RDBMS users on a daily basis and (very good) technologies as BerkeleyDB and Coherence are nothing but esoteric toys they will hardly use anytime soon. So will be this NoSQL database that will probably be given away as open source.

jdefargeon Building Polyglot (Scala/Erlang) Systems With Scalang

update: An older Cliff Moon's talk about polyglot systems (Erlang and Scala) is available at http://www.infoq.com/presentations/Building-Polyglot-Systems...

Disclaimer: in spite of thinking that Akka is a very cool framework, I think it can quickly become something like Spring, that is, a humongous framework that tries to solve everybody's problems.

jdefargeon Building Polyglot (Scala/Erlang) Systems With Scalang

Their claim is: Erlang is good for building scalable distributed systems. Scala has actors too, but Erlang is more than actors (a lot more!). It's a language created to build distributed systems and highly scalable servers.

On the other hand, Scala is good for data crunching and mutable/immutable data processing. And you'd have a hard time dealing with mutable data on Erlang. As they are building a highly scalable data analytics system, it makes a lot of sense to couple the two languages. But I guess Akka is good enough to the other 98% of the use cases you may ever find. ;)

jdefargeon How Google's build system works

Rumours has Blaze, Google's build tool the article describes, is being open sourced. At least this information was floating around early 2011.

jdefargeon Wikipedia: Making of Doom

If I remember well, the biggest problem with NeXT was that it was quite expensive! According to the "Master of Doom" book, Carmack pay around U$ 10,000 for one workstation in 1991.

jdefargeon Wikipedia: Making of Doom

+ one here too. Very good book, very nice narrative.

jdefargeon Grok cassandra's data model

The Facebook guys who wrote Cassandra have heavily drawn its design and terminology from Google's Bigtable. By the way, Column Family (CF) in BT makes a lot more sense because the compression of data,as well as disk storage locality, is made on a per-CF basis. They have even filled a patent about this (http://bit.ly/ooop2s).

Oddly enough, BigTable's terminology seems fits more naturally in the classic concepts than Cassandra's. Maybe it's the result of Dynamo's design choices (DHT, etc) that got into the mix or new concepts like SuperColumn.

jdefargeon Grok cassandra's data model

This column-family/column/super-column lingo that Cassandra pulls out just makes it harder to understand its data model. In fact, it's quite simple:

Keyspace: a hash table that holds your application data. Okay, the table is distributed among nodes (i.e.,a DHT), but it's still a hash table;

Row: an entry in the above hash table where each value is composed by a collection of "column-families".

Column Family: a key-value table (I avoid to call it a hash table because I don't remember if it's implemented as such). A better name for this thing would be 'Attribute Set'.

Column: it's a key-value pair (with timestamp). Thinking about it as a column just blurs the concept. Better name: 'Attribute'.

Note: it's possible to have a different set of attributes on a per-row basis (for the same Column Family), so this concept of 'column' breaks quite easily.

Super-column: key-value pair where the value is yet another key-value table! Better(?) name: 'Super-Attribute'.

Then Cassandra data model is in fact a nested set of key-value tables while dynamo's model is flat (just one level hash table). Oh! Last but not least, it's not a column-store. It's on-disk storage is row-oriented.

jdefargeon Stonebraker trapped in Stonebraker 'fate worse than death'

Unfortunately, there are many facets of this discussion that stumble accross each other.

1) Stonebraker is an industry troll that likes to shamelessly advertise his current products while pointing 'problems' with the state of the art. And he usually emits clueless opinions about subjects he doesn't deal with.

2) In spite of Stonebraker, voltdb engineers are doing a serious job, and the product has its value;

3) It's virtually impossible (and high risky) to change the infrastructure of any company the size of Facebook/Google. As any software engineer research shows, it'll take years and lots of money to change a large code base and deploy other data management system. Of course, Facebook is using HBase now for new products, but the current MySQL based systems should account for millions of lines of code. It's worth to spend time developing new products or refactoring old code that is working?

4) Thousands of MySQL shards is a hell on earth, for sure. We should praise Facebook engineers for being able to manage it efficiently.