Systems Programming at Twitter
monkey.org
monkey.org
On slide http://monkey.org/~marius/talks/twittersystems/#16:
case class Stock(ticker: String, price: Double) {
def <(other: Stock) = price < other.price
}
val goog = Stock("GOOG", 675.15)
val aapl = Stock("AAPL", 604.00)
aapl < goog == true
goog < aapl == false
I wonder if this kind of method would work
def ==(other: Stock) = price != other.price
def ==(other: Stock) = self != other
and it was invoked like this
appl == goog
You should download the Scala IDE and play with a feature called the "Worksheet".
See here: https://github.com/scala-ide/scala-worksheet/wiki/Getting-St...
Isn't this ability a bit dangerous or worrisome? It means that for every Scala package you use you need to review the code. It does make a good prank though :)
I wouldn't say it was particularly dangerous, but it is a massive pain to deal with (it also means you can google for any help with a particular method).
To me it shows two things:
* The author doesn't know what system programming is;
* Future, promises and generally speaking asynchronous operations are I/O management 101. It doesn't paint your company in a good light to exhibit these techniques as "how we do things".
Am I too pedantic?
No, it's that systems programming is not what it was. The term now also applies to large scale server side systems, not just drivers and lower level work. See also how Go was presented as a "systems language" for these same uses.
>Future, promises and generally speaking asynchronous operations are I/O management 101. It doesn't paint your company in a good light to exhibit these techniques as "how we do things".
I/O 101? You'd be surprised. Take a look around the industry, and even HN posts, and you'll find out this is far far from the truth.
Heck, back when I was in CS I had one teacher who was saying "use Java applets for web forms, its more secure since you can't right click to view the source to see the field names in order to forge a request". Somehow most of his students ended up on government contracts..
Composable futures and promises are still fairly new in use; I would wager our systems are some of the largest using the techniques described therein.
While this talk focused on composable IO and services, there is much more to it. Finagle (https://github.com/twitter/finagle) is where the rubber meets the road. Its interfaces are described in these terms and it makes great use of their power.
Also, as with any talk, it is difficult to convey the message using only the slides.
Generally speaking futures and promises are new labeled "as is" but asynchronous I/O really isn't and is actually a requirement if you want to write something efficient.
I understand that slides convey a small percentage of information and I really hope I didn't come up as snarky.
That being said, when you say "I would wager our systems are some of the largest using the techniques described therein" I think you should visit a stock exchange. ;)
As for us, we've stopped using futures and promises for asynchronous operations as we've found faster way to manage operations (our system is written in C++11).
The best is to design your system in a way that you never have to wait/synchronize at all.
I think you've perhaps misunderstood: the futures described in the presentation are not at all like, say for instance std::future or java.util.concurrent.Future - that's why I called them "composable futures". `flatMap` is the key to great power.
Our systems, using this variant of a "future", do not wait or synchronize at all, as you describe.
Another thing that I highlight a lot in the presentation itself, but does not come through in the slides is that this doesn't really have anything to do with asynchronous IO at all, it has to do with composing your operations from smaller operations. It's possible to use asynchronous IO (as we do—with asynchronous operations throughout, and with no waiting), but that is not necessary. The important bit is this: composing larger operations from smaller ones leads to inherently modular code; code that composes well within the context of larger systems. That's the important bit. That it is also easy to implement a performant execution system for this is gravy.
"Please learn Scala, we can't hire people that already know Scala.
Please?
Learn Scala."
The last presentation of any substance I saw re: Twitter Engineering was their cute BitTorrent based deployment stack.
Which is really important for largely distributed systems. Google wrote a little about it when they introduced their global run-time performance tracing tool called Dapper[3] in "Dapper, a Large-Scale Distributed Systems Tracing Infrastructure"[4]"
Modern Internet services are often implemented as complex, large-scale distributed systems. These applications are constructed from collections of software modules that may be developed by different teams, perhaps in different programming languages, and could span many thousands of machines across multiple physical facili- ties. Tools that aid in understanding system behavior and reasoning about performance issues are invaluable in such an environment.
[1] http://monkey.org/~marius/talks/twittersystems/#52
[2] http://monkey.org/~marius/talks/twittersystems/#63
[3] http://highscalability.com/blog/2010/4/27/paper-dapper-googl...
I don't know enough Scala to say with certainty but that seems to be an error:
Seq(1,2,3,4) flatMap { x => Seq(x, -x) } == Seq(1,-2,2,-2,3,-3,4,-4)
I was very happy back then when scala shows up.. but with time i think computer languages should just get out of your way.. be simple as possible..
nowadays im more a Dennis Ritchie/Ken Thompson fan.. they had make simple tools with C and Unix, that just worked..
This is also the Go language school of thought.. Languages must get out of our way, get things done in the simplest way possible..
Functional language people , are more passionate about proving some theoretical points like using immutability all way down.. even if this will blow out your memory resources, or make your code longer..
i think language designers need to be more pratical, and i like when the simplicity way of thought like the one seen in Go comes around the table, to make discussion a little bit more interesting..
We nee more simplicity, work less, so we can be more with our children or family.. and spending less time on our computer.
(The author seems to be shoe-horning a lot of different concepts into flatMap via traits. I count several different Monads in there.)
val f: Int => Int = { x => x*2 } f(123) == 246
aapl < goog == true
;-)