HNHacker News
TopNewBestAskShowJobs

dwenzek

604 karma · joined September 21, 2014

submissionscomments
dwenzek··on Introducing Arc: Outliers Wanted
This is the topic of the first week https://arc.sequoiacap.com/#main ;-)
dwenzek··on Using rubbish to make clean drinking water
This page [1] tells the story of Timothy Kayondo and his Eco Mobile Purifier.

[1) https://www.southampton.ac.uk/namrip/news/2021/02/africa-alu...

dwenzek··on Durability and Redo Logging
Thank you for this excellent post! Also thanks to jorangreef for his detailed comment!

It's a real pleasure to read such clear and thorough posts and discussions.

For me, the key insights are:

* to `fsync` before reading,

* the analogy between batching writes and a train between two stations.

dwenzek··on Implementing a File Pager in Zig
I see this as a good follow-up after this thread on the use of mmap to implement a database[1].

Indeed, there is a first implementation using mmap [2] to be compared with a full-fledged implementation [3].

[1] https://news.ycombinator.com/item?id=29936104

[2] https://ayende.com/blog/195588-B/implementing-a-file-pager-i...

[3] https://ayende.com/blog/195713-B/implementing-a-file-pager-i...

dwenzek··on Are you sure you want to use MMAP in your database management system? [pdf]
Thank you for these counter-arguments. It's good to have them to make up your own mind, especially when recognized experts use a mocking tone "you will no dare think the contrary".
dwenzek··on Even with Agile and Scrum waterfall will sneak in
I see as the key point of this post the following point:

> I believe we urgently need a new process for working remote.

I do believe the same. Working remotely forces agile processes to be adjusted. Information radiators are no more the same. I can no more notice that 3 guys are arguing since an hour in their corner. I can no more notice that 1 guy is struggling alone with his compiler. It's more difficult to agree on a design on the whiteboard. The pace of our days has changed too, with more strict alternation between meetings and individual activities.

We need other channels, more written communication, and a bit more of planning. In no case a whole up-front plan. Definitely not. But the minimum of planning that will help to cope with the distance and the changed pace.

dwenzek··on Foundations of Dawn: The Untyped Concatenative Calculus
How this semantics is extended to handle named multi-stacks?

For instance, how is interpreted the main example of the introduction [1]?

    {fn f => {spread $x $y $z} {$z drop}
        {$y clone square pop} {$x square pop} add {$y abs pop} sub
    }
Notably, how to interpret the second use of $y that makes sense only because the former y value has been duplicated in the first $y call?

[1] https://www.dawn-lang.org/posts/introducing-dawn-(part-1)

dwenzek··on You need to write in order to think well
I agree that intuition and thinking often precede speaking and writing. However writing is a great tool to clarify and elaborate our thoughts. The point is not to be a talented writer but to take the most of our ideas, to make them stronger before we can convey them to others and this can be done using a different medium.
dwenzek··on The problem of effects in Rust (2020)
Thanks for the link. Indeed a nice programming language proposal. I love the opinionated set of design principles https://flix.dev/principles/.

There is also a former thread on HN: https://news.ycombinator.com/item?id=25513397

dwenzek··on Rustdoc résumé
I'm missing the integration tests ;-)
dwenzek··on “Design Patterns” Aren't (2002)
> How can you distribute responsibility for design through all levels of a large hierarchy, while still maintaining consistency and harmony of overall design?

This how the author summaries the intent of Christopher Alexander's work. I have not read "A Pattern Language" so it's difficult to say if this is accurate or not. However this is such an important issue in software engineering that I would welcome any ideas to reduce the problem.

I agree when the author that the GoF addressed a lower level problem. However, I'm also disappointed by these slides that provide not hints for a better high-level approach. The GoF patterns form a language for developers speaking to developpers. We also need a language across roles.

dwenzek··on The Toit Programming Language
Exactly the kind of PL ideas I'm looking for. Discovering concatenative languages, like Forth, opens a door to a new world.
dwenzek··on The Toit Programming Language
What is a modern programming language?

I like discovering new programming languages or more specifically new approaches to programming. I'm looking for new landscapes these languages can help me to explore. And when I say new I mean something I was not aware of, not necessary something recent. This is why I'm both surprised and disappointed when the claim is to be modern without specific novelties.

dwenzek··on Fuzz me wrong – How QuickCheck destroyed my favourite theory
Indeed, the identity is only required to deal with an empty input.
dwenzek··on How Not to Teach Recursion
With a cyclic definition, one simply fall back precisely to the starting point, while with recursion one applies a similar process process again and again but each time on a smaller scope up to a point the response is known.
dwenzek··on A Relational Model of Data for Large Shared Data Banks (1970) [pdf]
Thank you. This is the article I remember.

And yes, I have been fooled. This is a lesson to only cite (and even remember) carefully read articles!

dwenzek··on A Relational Model of Data for Large Shared Data Banks (1970) [pdf]
What is really amazing is that this paper has been rejected in the first place (among others).

http://www.fang.ece.ufl.edu/reject.html

I once read this story in either ACM or IEEE but failed to find the exact reference. It was an editorial relating famous papers rejected by the journal. Does anyone have this reference?

dwenzek··on Thesis: Partial State in Dataflow-Based Materialized Views
This has already been posted [1], but I do think that this video deserves a better audience.

Jon Gjengset presents his work [2] on materialized views, cache maintenance, and incremental updates, in the context of web databases. His main contributions are around partial state materialization of the hot subset of the data with downstream propagation of the updates and upstream requests to create the missing entries on-demand. These ideas have been implemented with Noria [3], a MySql compatible database designed for mostly-read workloads. The performance measured on lobsters-like workload is really impressive.

[1] https://news.ycombinator.com/item?id=24853783

[2] https://jon.thesquareplanet.com/papers/phd-thesis.pdf

[3] https://github.com/mit-pdos/noria

dwenzek··on Testing Database Engines via Pivoted Query Synthesis [pdf]
I find really impressive the contrast between the simplicity of the approach and its effectiveness.

* The idea is to generate tests by picking a random row in a database and by crafting random queries which are expected to return that row.

* Sure the details are more complex, but this idea is really effective. The authors found 121 bugs in SQLite, MySQL, and Post-greSQL.

This is even amazing that a table with a single column and 3 rows can be used to detect a bug on SQLite, a so widely used database.

dwenzek··on The Expression Problem
There is also that video [0] discussing object algebras as a solution to the expression problem.

[0] "Who's Afraid of Object Algebras?" https://www.infoq.com/presentations/object-algebras/

dwenzek··on The database I wish I had
Indeed, Irmin is a library to implement data stores which can be freely cloned, updated and merged along a model similar to git.

It separates the content from the actual store. So it can run as well on a file-system, on a git repository, in memory or in the browser (See for instance https://github.com/talex5/irmin-js).

As git, Irmin uses 3-way merge but applied to arbitrary values and not only files. As described here http://gazagnaire.org/pub/FGM15.pdf and generalized here https://kcsrk.info/papers/oopsla19-mrdt.pdf

dwenzek··on Keeping CALM: When Distributed Consistency Is Easy
> CRDT's are about sharing data, whereas this CALM theory is only talking about making a local decision.

Both CRDT and CALM are about sharing data to make a local decision which is globally consistent.

Both use an order relation over datasets to modelise the concept of "adding more info to some partial input or output".

I would say that the difference is on their focus. CALM determines the frontier where no coordination is required and provides general criteria (no need either to retract former output nor to hear everything there is to hear nor to know all the participants). CDRT provides a mean to meet this criteria. By taking the least upper bound of former partial results, CDRT ensures that the outcome is growing.

dwenzek··on What every software engineer should know about Apache Kafka
I fully agree that this "stream-table duality" is misleading or at least quite vague.

The diagram juxtaposing a chess board (the table) and a sequence of moves (the stream) seems enlightening but actually hides several issues.

* An initial state is required to turn a stream into a table.

* A set of rules is required to reject any move which would break the integrity constraints.

* On the reverse direction, a table value cannot be turned into a stream. We need an history of table values for that.

Hence, the duality, or more accurately the correspondence, is not between stream and table but between constrained-stream and table history.

Even if these issues are not made explicit, the Confluence articles and the Kafka semantics are correct.

* "A table provides mutable data". This highlights that stream-table-duality is relative to a sequence of table states.

* "Another event with key bob arrives" is translated into an update ... to enforce an implicit uniqueness constraint on the key for the tables.

dwenzek··on What every software engineer should know about Apache Kafka
> Kafka is a message broker. It’s not a database and it’s not close to being a database.

Kafka is somehow a database, even if specialized on logs of events. For a similar discussion, I would refer to Jim Gray "Queues Are Databases" https://arxiv.org/pdf/cs/0701158.pdf.

dwenzek··on Stop trying to make hard work easy
On the contrary, I really found this article insightful.

Yes, there are few concrete pieces of advice to tackle the "internal triggers" of distraction. These are to be found in ourself and any tricks, like turning off all of the notifications, will only address the surface of the problem. Any concrete advice would risk here to conceal the truth that that hard word is not easy.

I like the path proposed by the article.

* searching in oneself for the deep obstacles toward our goals. boredom? loneliness? fatigue? uncertainty?

* clarifying our priorities into a concrete daily schedule;

* imposing ourselves a routine on what matters

* and even using pacts with ourselves.

Looking closely at my own current habits, this helped me to better understand what I already do well and what I need to improve.

dwenzek··on Saltpack – A modern crypto messaging format
The https://saltpack.org/ web-site links to a insightful post about authenticated encryption with associated data (AEAD) and the shortcomings addressed by saltpack: https://www.imperialviolet.org/2015/05/16/aeads.html
dwenzek··on Show HN: Micro-mitten – Research language with compile-time memory management
It's refreshing to see new approaches for memory management exploring notably the power of static analysis at compile time. I will take the time to read this dissertation!

Just a question. It reminds me previous works on static inference of stack-allocated regions. What are the relationships, if any?

* [A Simplified Account of Region Inference](https://hal.inria.fr/file/index/docid/72527/filename/RR-4104...)

* [MLton regions](http://mlton.org/Regions)

dwenzek··on The naked truth about writing a programming language (2014)
Great list!

I specifically agree that the triple (transformation between data representations + ability to define where the code should execute + built-in profiling) can be a game-changer. It would help to solve the tension between clarity and performance by decoupling the what from the how.

dwenzek··on Is a Dataframe Just a Table? (2019) [pdf]
Reading the sections 2.2.3 (about joins) and 3.5 (about code comprehension) took me aback! I came with a positive a priori for dataframes, but changed my mind. SQL is far to be perfect, but at least there are sound foundations!
dwenzek··on The Build is Always Broken
Unison (https://www.unisonweb.org/) is a new programming language designed around a really neat idea that removes the necessity of builds. Unison’s core idea is that code is immutable and identified by its content. A function name can be changed with absolutely no recompilation nor risk to break another piece of code.
← PreviousPage 2 of 6Next →