Hacker News Reader

gz09on Make Ubuntu packages 90% faster by rebuilding them

we're using it in feldera and it's been giving us (slightly) better performance than jemalloc which is what we've used before mimalloc

gz09on IO Devices and Latency

Nice blog. There is also a problem that generally cloud storage is "just unusually slow" (this has been noted by others before, but here is a nice summary of the problem http://databasearchitects.blogspot.com/2024/02/ssds-have-bec...)

Having recently added support for storing our incremental indexes in https://github.com/feldera/feldera on S3/object storage (we had NVMe for longer due to obvious performance advantages mentioned in the previous article), we'd be happy for someone to disrupt this space with a better offering ;).

gz09on Billion Cell Spreadsheets with Rust

the code for it can be found here in case anyone is looking for it https://github.com/feldera/techdemo-spreadsheet

gz09on Billion Cell Spreadsheets with Rust

Thanks for posting this. Let me add some background for this demo we built to showcase incremental computation with Feldera (SQL compiled to do incremental evaluation). https://github.com/feldera/feldera

The gist of it is that if you update a cell, we incrementally update the spreadsheet which means we will only emit a minimal amount of changes for the cells affected by your update. The nice thing about it is that this is something that Feldera does automatically (and it would do that for any SQL that you end up writing, so it doesn't have to be a spreadsheet, but a spreadsheet is a nice example that everyone understands and knows about).

There is a more detailed explanation in this video https://www.youtube.com/watch?v=ROa4duVqoOs if you're interested what's going on under the hood -- or if you prefer reading about it we have an article series that goes over all the parts of the demo

1. Feldera SQL (gets compiled to Rust) https://docs.feldera.com/use_cases/real_time_apps/part1

2. API server (Rust/Axum hosted on fly.io) https://docs.feldera.com/use_cases/real_time_apps/part2

3. egui web Client (Rust compiled to WebAssembly) https://docs.feldera.com/use_cases/real_time_apps/part3

gz09on DBSP: Automatic Incremental View Maintenance for Rich Query Languages

You can find a Rust implementation of DBSP at https://github.com/feldera/feldera

The crate is also on https://crates.io/crates/dbsp

gz09on RocksDB: Not a Good Choice for a High-Performance Streaming Platform

Hi, you might be thinking of another key-value store (FoundationDB maybe?). RocksDB can be linked as am embedded library so it doesn't need to run in a separate address space.

gz09on At 50 Years Old, Is SQL Becoming a Niche Skill?

Coincidentially, SQL is having a "revival" in the streaming world... feldera, materialize, arroyo and many others are all betting on SQL as the language to express computation on moving data.

gz09on We Built a Streaming SQL Engine

neat! I sent you an email

gz09on We Built a Streaming SQL Engine

Very nice! Are you using the dbsp implementation from the paper authors for this (e.g., https://github.com/feldera/feldera/tree/main/crates/dbsp)?

gz09on Feldera – a more performant streaming database based on Z-sets

Hi,

> I wonder if it lives up to the hype.

We do think so! (disclaimer: I'm a co-founder at Feldera)

To give some more background: We are trialing feldera with several industry/enterprise partners from different domains. Our core team also built differential datalog (https://github.com/vmware/differential-datalog) in the past. And while ddlog is used quite successfully in products today, we believe the many lessons we learned with ddlog will help us to build an even better continuous analytics platform. FYI our code is open-source at https://github.com/feldera/feldera if you'd like to try it out.

Also feel free to join our community slack channel (https://www.feldera.com/slack/) if you have more questions.

gz09on On programming without malloc

If you don't want to use malloc in a kernel, that is fine. However, you should at least consider having something similar. This can be as simple as finding the available memory in your system and writing a frame or slab allocator. It will make your life much easier. It would be interesting to see how easy it is to integrate your memory manager directly with the rust language.