moab

Thanks for this great comment. What kind of workloads are people trying to run on sensor data that arrives at such a high velocity? Time series analysis? Anomaly detection? I wish I had a better idea of what kind of specific problems users you've run into are trying to solve, which fail on the existing software stack.

Yes, there's a few papers over the last few years on this, starting with this paper: https://www.cis.upenn.edu/~sanjeev/papers/soda14_disjoint_se...

Here is a more recent paper (from this year) analyzing concurrent union find with random linking: https://arxiv.org/pdf/2003.01203.pdf

If he was suspended for that, it is truly a sad day for USC. There seems to be 0 malicious intent; he's explicitly referring to "that that that" before saying it in Chinese. This is a common way of saying "um..." in fast speech. He also seems like a good professor. It's too bad some student found the need to complain when it is obvious there is no bad intent meant.

That's really unfortunate. I agree that the $$$ and accounting help make sure the system is not abused in the usual case. Unfortunately jerks are unavoidable, and even if the manager knew the true story they could ignore it to emphasize the contributions of their own reports.

It's hard to imagine a promo system that rewards hard work, where feedback comes from ones peers, and only promotes a fixed number of people where these kind of zero-sum issues don't appear.

Not my experience of Google, at least having worked there as an intern multiple times. All of the peer bonuses on our teams and affiliated teams have been for people going out of their way to help out with some project even if it was not part of their day-to-day work. I can only speak for the research-y side of Google, which in my experience seems to avoid most of these problems probably due to having managers that also do research and write papers. Having a separate academic status system other than your internal work position probably helps in this regard.

Does the OP have links to any benchmarks? Specifically, what kind of ingestion rates can one expect with a modest number of machines? Does it support a single-machine (shared-memory parallel) environment? What kind of algorithms are supported?

It would be good to add some information about the features/capabilities on the homepage. Right now the blurbs make vague statements like "high throughput", which could be 1000 edge updates/sec or 10M.

This is a very cynical and bad faith reading of the OP. I don’t see how knowing about HoTT or category theory gives one away as an absurdly pre-prepped kid who is shoehorned into academic success by tiger parents. I went to a Bay area high school full of specifically that kind of kid, and cannot think of a single one that knew what curry- was or had heard of prolog. Of course all anecdata. But let me tell you that the pre-prepped kids are not writing about wanting to do type theory research.

It’s a shame because the OP sounds like someone who would thrive at an mit/cmu, but likely got screwed due to an opaque system optimizing for a completely unknown objective (let me be cynical now—-the objective is optimizing us news rankings by only accepting top gpa/sat that have high yield, a subset of overprepared foreign students that will pay full ticket, and a scattering of “african americans” ie wealthy nigerians).

The system is bullshit. The only thing us universities have going for them is the professors and the fact that you can drop by their office and start a research relationship by extending your hand in good faith.

Yes, but sites like: https://github.com/iamadamdev/bypass-paywalls-chrome https://sci-hub.tw/ exist and make it trivial to get around draconian publishing practices. I'm not advocating not paying for journalism---it's just that not everyone realistically can, and paywalling them seems like a bad solution.

A more plausible reason, at least to me, is the increasing specialization of fields and lack of cross-field communication.

Also, ideas seem to be flowing plentifully at least in math and TCS, where old and hard conjectures are still being proved with astonishing regularity.

By the argument you make below ("The brains are different.") you could use this argument to deny the humanity, or capability of feeling pain of anyone that isn't yourself. This argument quickly slides into solipsism, which is not a useful philosophical position to take in a world inhabited by agents that pass our individual Turing tests.

It's a great and tough question, and to be totally honest I don't have a fully formed view on this. It does seem like a life lived on a farm with humans could have significantly less suffering than a live in the wild. If the animal is (i) not being used for some byproduct, e.g., being repeatedly inseminated for milk (ii) has enough volition, or freedom to roam, and reproduce in some (possibly controlled) manner, it seems like such a life could be quite good for an animal. There are probably many small family farms that keep animals for pleasure and not to raise for meat that practice this sort of ethical farming.

One argument against this sort of intervention comes from my experience living with our family dog (we've had him for five years). He is a calm and intelligent dog who I feel has had a good life, but I feel sometimes that it is a very restrictive one. He can't do whatever he wants, and is separated from other creatures like himself. My feeling is that his life was worth living, but there is an element of guilt when I think about how free he is, and once he passes away I don't think I will keep a pet in a future family I start.

It's very hard to say what these creatures who cannot speak up for themselves really want, and whether we should interfere with their lives to prevent suffering that's inherent in nature.

Do you have any thoughts on this? Please let me know if you have any books/resources to recommend. I'm planning to read Singer's Animal Liberation soon, which will probably discuss this question.

I'm actually working toward a sort of "orthodox"-Jainism in which you only eat substances that plants are actively trying to give to other animals to help them propagate themselves (so eating fruits/bellpeppers, etc. are all ok, but killing a plant to consume it is not).

Anyway I understand that talking about animals and plants in this fashion is not a popular position on HN, and folks have and are likely to downvote me.

There are now clear environmental and animal rights reasons to avoid red meat and dairy. Unfortunately, animal rights motivations don't seem to cut it for most of my colleagues and friends, who are by and large highly educated. Arguments about whether cows are "alive" or "feel pain" the same way humans do inevitably conclude with an acknowledgement that they probably do in all likelihood, but too bad, they taste good/produce animal products we like, and thus it sucks to be them. Or "too bad our ancestors have been doing this for hundreds of thousands of years".

Thankfully, most are swayed by environmental arguments and have eliminated their red meat consumption. Far fewer have given up milk, probably out of long-standing habits of milk consumption with cereal/coffee.

Just like the choice of whether we hurt or inflict pain and enslavement on a fellow human (a choice which has taken centuries to reverse), there's a choice of whether we enslave another sentient species for the sake of our tastebuds and convenience. I hope the decline of the milk industry continues, and that more of these farmers can successfully switch into more ethical and planet-friendly sources of revenue.

"It’s only a threat to people who have a compromised immune system because of age, diabetes or something like that."

Yeah forget those people.

/s

Thanks for the reply. I would be very interested if you could point me towards any private companies providing these kind of database solutions. I am particularly interested in their story around concurrent ingestion + analytics workloads. This is a tricky problem that most existing works solve by either doing stupid things a la. having a single-version system, or by using lots of synchronization + complex version-friendly concurrent data structures. Solutions I have seen in academia are usually tested well below the 1TB of data range, and I don't imagine they scale to the kind of data sizes you mention.

The fact that the research is being done outside of academia and not being published is pretty sad imo. Hopefully those in the know decide to publish at least tech-reports/whitepapers once they have a share of the market.

Could you provide some references for this work on caching and schedulers?

I’m also curious what application domains are storing close to petabytes on a single machine with only a few terabytes of dram.

I found it fun to read this after reading this other post that made the rounds today about AI automating most programming work and making program optimization irrelevant: https://bartoszmilewski.com/2020/02/24/math-is-your-insuranc...

There's an interesting thought.

Can you elaborate? I’m not familiar with the current state of graphics research. Can a high end GPU today do this, and if not, how far are the fastest high-quality algorithms from realtime today?

Can you elaborate on why? As someone who works on parallel/concurrent algorithms and data structures a single-threaded system design seems anathema to most research over the past decade (maybe excluding H-Store, which uses local single-threadedness in an interesting way).

Thanks for sharing these. It's funny, I had exactly that series by Naval in mind earlier. I think the database tweet is really useful (reminds me of a lecture slide).

To the author's point, yes I remember the main themes/movements in the Steinbeck books I have read. I remember them for the Dostoevsky and Tolstoy I have read. I remember them for most good books I have read, even if the reading happened in high school, or 10+ years ago. Good books are almost like a lived-experience. Yes, they are not as "creative" as sketching, or writing an essay, but reading can also require creativity on the part of the reader.

I find it hard to believe that something deeply insightful can be compressed into 140 characters, or even a collection of tweets. I've read tweet-chains from twitter pundits, and though these longer form tweets do have insight, it's still more like getting an elevator pitch from a smart person than actually getting a deep understanding of the reasoning that leads to said opinions.

Basically, my feeling from using twitter is that it is a venue for expressing opinions in clever/snarky ways, and reading other people's clever/snarky opinions and feeling amused. But I might be very wrong, or following the wrong people. Do you have any examples of tweets that are highly information-dense and making insightful points?

Delany's writing raises the perennial question of whether we can separate the art from the artist (the view in the current climate seems to be an emphatic "no", see for example the recent Times article about Nolde's paintings being removed from Merkel's office [1]). For the obvious comparison of Nabokov, I'm inclined to view Lolita separately from the author and his personal beliefs. If Nabokov held views similar to Humbert, he kept those thoughts to himself, and I don't think anyone reading Lolita can charge Nabokov of advocating those ideas, although it seems really popular to do so in literary circles today.

I am not sure the same case about "a lack of advocacy" can be made for Delany or his oeuvre.

[1] https://www.nytimes.com/2019/04/10/arts/nolde-nazi-exhibitio...

Because that's the radicalized ideology that pops up most often on 4chan. If you know of a board with a contingent of alt-left/identity-politics I would be interested to know. As you said yourself, the case of the second shooter seems to be related to Twitter, not the chans.

The "bad" boards frequently leak, and you do see highly political or racist posts from time-to-time on the regular boards. A sign that the community is functioning properly is regulars on /out/, /lit/, etc. calling out (read mocking) ethno-nationalist threads, or race-baiting for what they are.

I would not lump 4chan into this discussion. I'll concede that some boards on the site are more polemical than others and promote alt-right ideologies without a doubt, but there are many other interesting boards and people on 4chan that are not captured by the broad strokes you're outlining. I know people that browse /fit/, /lit/, /mu/ and /out/ just to name some boards that are perfectly reasonable individuals that don't lionize shooters or hold other alienated views.

Hi Richard,

What applications/problems are you solving using hypergraphs? There is definitely a dearth of high performance hypergraph processing engines/systems.

Even if the speed is identical to how fast you are thinking of words to type, there are plenty of applications (prevent RSI, accessibility for amputees).

I don't think anyone is trying to be mean/overly critical with their comments here. I hope you keep writing/submitting to HN.

Absolutely agree. Unfortunately this kind of "always optimizing" viewpoint seems to have spread like a virus over this past decade. It is sometimes difficult to remember that spending time to read/understand things slowly is not a waste of time, but an important way to appreciate the work of a master in a way that speeding through it at 1.5x on an audiobook simply will not provide.

Do we speed-walk through the Louvre to see the masterpieces as quickly as possible?