[1] "Scalability! But at what COST?"
423 karma · joined December 7, 2015
[1] "Scalability! But at what COST?"
DB2 BLU: https://researcher.watson.ibm.com/researcher/files/us-ipandi...
Wildfire "v0", the "failed" HTAP project: https://researcher.watson.ibm.com/researcher/files/us-ytian/...
Wildfire "v1", the shared-nothing architecture over cloud storage (EventStore 1.0?): http://cidrdb.org/cidr2017/papers/p123-barber-cidr17.pdf
DB2 EventStore (EventStore 2.0?): http://www.vldb.org/pvldb/vol13/p3299-garcia-arellano.pdf
Re: my “dilution” comment, i was not thinking about DEs. I was thinking about non-technical people (management, law, accounting, ...) that have been creeping up the Fellowship lately. Also the bluewashed executive that were given a fellowship package to seeeten the pot.
Both levels may reside in different pay bands, but they are both at the executive level. And DEs cannot become fellows, and vice versa (some exceptions to the rule may have existed).
[1] https://en.wikipedia.org/wiki/Ordinance_of_Villers-Cotter%C3... [2] https://en.wikipedia.org/wiki/Language_policy_in_France#Fren...
Then he woke up one morning wanting to reboot his life's work for his class and, undeterred by the fact the original targeted processor did not exist anymore, settled to write his own computer architecture on a Xilinx Spartan.
He then successfully ported the Oberon system and compiler to this architecture to demonstrate to his students.
Just wow.
It was UDP.
> Which is to say, you're saying "detrimental impact on performance" and yet this seems to be a significant win over BSD sockets.
The point I was trying to drive home is that, while 30% overhead reduction compared to BSD socket is nothing to laugh at, a fully user-space UDP/TCP network stack combined with memory-mapped buffer sharing usually gives performance improvement measured in the "x" and not in the "%".
Now, there is not much in terms of experiment protocol in their material so its very hard to tell. You mention uncompressed video so they could be sending humongous frames, which I doubt as no one ever would send raw frames over the network (that would be several MB per frame for a 720p front camera). But if that is the case then the data copy operation becomes the predominant bottleneck and getting rid of one may justify the improvement. But that is not a realistic scenario.
The more realistic scenario is that they sent compressed delta-frames across the network (H264 or HEIF), which would then considerably reduce the transferred payload size. In that scenario, data copy is not the predominant overhead anymore and 30% overhead reduction is underwhelming, telling me that they still are calling expensive operations like syscalls/uIPC on the critical data path.
I am not doubting they have some sort of user-space TCP implementation. I am trying to understand how much of TCP they moved out of the kernel. From what I can gather they still have enough of it in the kernel such that the related syscall/uIPC overhead does not allow more than the 30% performance improvement announced.
[0] https://devstreaming-cdn.apple.com/videos/wwdc/2017/707h2gkb...
Also, the 30% less overhead is underwhelming. I would have expected much better improvement than that. That being said, if their measurement includes the encryption layer then the I/O benefit may be overshadowed by the encryption overhead.
Still, pretty cool stuff to make publicly available.
For the doubters and the disbelievers that have been wondering what is the relevance of IBM in this day an age: this. This is what IBM is all about.
And it's not just about PFLOPS; each node has 1/2 terabyte of memory, globally addressable across the entire cluster using RDMA over Mellannox 200Gb/s EDR.
It's also P9: 44 cores per node; but most importantly each node drives a couple of V100 through NVlinks, which allows the GPU to share the system's main memory.
At that very moment, losing ground, de Gaulle flees France for Baden-Baden to regroup with General Massu. After that meeting de Gaulle decides to resign, leaving the room to the banker Pompidou, much more open to the Anglo-saxon “weltanshauung” that the General spent his life fighting.
The absolute number is really meaningless. I remember a Ph.D prep talk about salary expectation in grad school in France mentioning $105K in the US, and I remember thinking “wow!”. What they don’t tell you is that a third of this will go to taxes, half of the reminder will go to housing, and because of the outrageous deductible and out-of-pocket caps of most corporate medical insurances you can be hit anytime with a $2K or $3K medical bill, which can be quite common if you have kids.