davgoldin

FDB seems to outperform, and out-scale most, if not all?

According to an old report, FDB can do around 75k transactions per core [0].

MySQL on same CPU (all cores) can do about 8k [1].

[0] https://apple.github.io/foundationdb/performance.html

[1] https://www.anandtech.com/show/8357/exploring-the-low-end-an...

it is in tokyo

Just something I got used to do. My right palm is always conveniently hovering above the right Ctrl, Alt and L, Enter keys, making those combos easy and instant.

After hitting Ctrl+L, I usually do one of these: 1) "Enter" to discard current page; 2) "Alt+Enter" to open in new tab; 3) "Ctrl+Enter" to open in new tab but keep focus on current page (e.g. read later); and 4) "Esc" if I got an answer from the address bar (math, currency, history, already open page, etc).

Looks nice, I wish you best of luck! I personally haven't seen my "new tab" page in a very long time. What I'm doing instead: in current tab hit Cmd+L to focus on address bar, type query/address hit Option+Enter to open resulted page in a new tab. Skipping a bunch of clicks and the "new tab" page. Should be Ctrl+L Alt+Enter on Windows.

Why read only? What's stopping this engine from using (for example) FoundationDB as storage?

Congrats, looks like a lot of hard work!

Could I swap storage engine with own key value storage e.g. rocksdb or similar?

The article makes some hard to digest claims, for example:

> According to six Israeli intelligence officers

Not 1 reservist, or 2 retired officers, or 3 contractors, but 6 active serviceman - whose day to day job is to figure out how to hide secrets.

There are more statistically impossible statements.

I thought not much: https://github.com/tauri-apps/tauri/issues/579

I'm on light mode by default. If you look at the "The fullstack database" text you'll see things emanating from it outwards. Fast, and barely visible, hence the initial confusion. In any case, it's not a problem at all, just caused me doubt my vision for a moment :)

Just to be on the same page, I'm talking about top section of https://www.triplit.dev/

Your (honestly) very nice homepage made me almost book a visit to ophthalmologist, eye floaters galore!

Congrats! Many moons ago the lack of undo/redo was the main blocker. Has this been added?

I share the frustration! I have to open lots of small and large CSV files daily, with multiple encodings. Usually I'm juggling between Excel, Numbers, and Sublime text editor - depending on which one would open that file faster.

Naturally, I installed Modern CSV seconds after seeing this post.

I'm on MacBook M2, using trackpad 1) the vertical scroll is slow and un-mac-like, unnatural; and 2) horizontal scroll basically doesn't work, it scrolls a column or a half on a full swipe, but it does improve somewhat when there's less content in the cells.

I admire your work, please keep on going, there's a huge need!

There's a demo video at https://twitter.com/igorsushko/status/1684645290132000768

Perfect timing!

I thought Google Marketplace's page was much more useful than your home page. The screenshots and video helped quite a bit!

NSFW! Don't open the (expired?) domain.

We've got MySQL 8 running on high-end-ish consumer hardware, with 150 tables ranging from thousands to billions of rows. Largest table is a bit over 5 billion rows and 520GiB. Parallel network backup takes about 3 hours.

I'm on a fairly recent Macbook Air 2020 (Intel i7). Hovering over the top menu, a full-page animation badly stutters to reveal a few text links. It seems I'm barely getting a few frames per second. Worse yet, the overlay gets stuck in an open position, blocking the content, until I refresh the page, and scroll down avoiding the menu. That's in Chrome. Safari is slightly smoother, but the animation still obviously stutters.

A trick to buy new hardware? \s

The case is "LZMOD A24 V3" - found it on caseend.com - there are smaller ITX cases, but I wanted to fit in standard components only, and not to mess with custom PSUs (for example).

The rest of the components are:

Board: AsRock Rack X570D4I-2T (2x 10GBe and IPMI!)

NVMe: 2TB Transcend TS2TMTE220S TLC

SSD: 2x 8TB Samsung 870 QVO

PSU: Seasonic SSP-300SUB (overkill, went for longevity)

CPU Cooling: Thermalright AXP-100 Series All-Copper Heatsink with Noctua NF-A12x15 PWM

Exhaust fans: 2x INEX AK-FN076 Slimfan 80mm PWM

On the air intake side, there's a filter sheet that I replace (or vacuum) once in a blue moon - the insides are still pristine after running for over a year now.

Interesting thing about cooling: one of those cases has a PSU with custom made cabling (reduced cables by about 90%). I was hoping it will reduce the temperatures a bit. Surprisingly there was basically no change. At full load all keep running at around 70 celsius.

Important: in such a small case, if you want silence you'd better disable AMD's "Core Performance Boost". This will make the CPU run at its nominal frequency, 3.4GHz for 5950x, otherwise it'll keep on jumping to it's max potential, 4.9GHz for 5950x, which will result in more heat, and more fan noise.

This looks very promising. I basically could print an enclosure to specifically fit my home space. And easily print a new one when I move.

More efficient use of space compared to my current silent mini-home lab -- also about 2U worth of space, but stacked semi-vertically [1].

That's 4 servers each with AMD 5950x, 128GB ECC, 2TB NVMe, 2x8TB SSD (64c/512GB/72TB total).

[1] https://ibb.co/Jm1SX7d

500 in Tokyo, Japan

Nice page title: "Error 500 (Server Error)!!1"

I can hear my lizard brain saying "this time it's different, China can't be the next superpower" - but I have nothing concrete to support this thought, except maybe a few weak attempts like: Chinese language is hard; it's a communist country where the law isn't "the law"; this time there's Bitcoin...

I'd put cleaning the mailbox in second place behind filling forms. They get filled up daily with flyers and paper ads of all kinds. It's so bad that most buildings have a dedicated trash can, just for these. Right next to the mailbox area.

These two resources helped me a lot in understanding how to setup FoundationDB processes correctly:

https://nikita.melkozerov.dev/posts/2019/06/building-a-found...

https://forums.foundationdb.org/t/roles-classes-matrix/1340/...

One of PingCap's (company behind TiKV and TiDB) founders once did his own FoundationDB review [1] - he concluded: "the performance is cool". I'm assuming at that time FDB's performance was superior to TiKV.

[1] https://medium.com/@siddontang/benchmark-foundationdb-with-g...

From own experience, I'd say FDB can handle it all, we've got 20% read 80 write during peak hours, and reverse: 80 read 20 write other time.

Without doing your own tests, here's per core numbers, from which you can extrapolate (e.g. via CPU mips) towards your own hardware: https://apple.github.io/foundationdb/performance.html#throug...

FDB is ACID as is shipped, you don't need to turn knobs to make it such. Toughest part is to figure out classes/roles of the system. Here are a couple of good starting points: https://nikita.melkozerov.dev/posts/2019/06/building-a-found... https://forums.foundationdb.org/t/roles-classes-matrix/1340/...

FDB's Directory layer provides all you need to create and edit nested paths. What's left to develop is a file chunking and assembly part, and statistics if needed.

The only reason you need chunking is because FDB has very clearly defined limits in their documentation, and one of those limits is value size - it can't exceed 100kB, and should be kept below 10kB for best performance.

For statistics like folder byte count, you could use FDB's atomic increment operation.

Here's a good starting point (not mine): https://forums.foundationdb.org/t/object-store-on-foundation...

I'll be happy to answer specific question if you got any.

Only great things to say about FoundationDB. We've been using it for about a year now. Got a tiny, live cluster of 35+ commodity machines (started with 3 a year ago), about 5TB capacity and growing. Been removing and adding servers (on live cluster) without a glitch. We've got another 100TB cluster in testing now. Of all the things, we're actually using it as a distributed file system.

We've tried Ceph, GlusterFS, HDFS, MinIO and some others, and eventually decided on a custom FDB solution. It's a breeze to setup, and seems to eclipse others in performance [0] and reliability - Kyle (aphyr) the author of Jepsen series on distributed systems correctness, said: "haven't tested foundation in part because their testing appears to be waaaay more rigorous than mine." [1]

The way we use FDB, if anyone is interested, is we simply split files into small chunks (per FDB data design recommendations), and store all file's & folder's meta data in FDB such as byte count, create/access/write times, permissions, and a lot more. Folders are handled by the builtin Directory layer [2].

[0] https://apple.github.io/foundationdb/performance.html

[1] https://web.archive.org/web/20150312112552/http://blog.found...

[2] https://forums.foundationdb.org/t/whats-the-purpose-of-the-d...

My previous startup (~2014) had a similar problem: PBs of data, with millions of mixed clients accessing it at close to real-time speeds. The biggest difference is that we needed to do real-time processing before delivering the content. We needed storage capacity balanced with CPU and RAM.

We ended up buying lots of Supermicro's ultra dense servers [1]. That's a 3U box, containing 24 servers that are interconnected with internal switches (think: 1 box is a self-contained mini cloud). Each server has (cheap config) 1 CPU 4 Xeon cores, 32GB ram, 4TB disk.

Those were bought & hosted in China, and IIRC price tag was around $20k USD per box. That's 96TB per 3U, or >1.2PB and ~$200k per rack. We had a lot of racks in multiple datacenters. These days capacity can be much larger, e.g.: 6TB disk, 144TB per 3U and >1.8PB per rack.

We've tried Ceph, GlusterFS, HDFS, even early versions of Citus, and pretty much everything that existed and was maintain during that time. We eventually settled on Cassandra. It required 2 people to maintain the software, and 1 for the hardware.

Today, I would have done the same hardware setup, mainly because I haven't had 1 Supermicro component fail on me since I bought them first in early 2000s. Cassandra would've been replaced by FoundationDB. I've been using FoundationBD for awhile now, and it just works: zero maintenance, incredible speeds, multi datacenter replication, etc.

Alternatively, if I needed storage without processing, but with fast access, I'd probably go with Supermicro's 4U 90 bay pods [2]. That'd be 90*16TB, 1.4PB in 4U, or ~14PB per rack. And FoudnationDB, no doubt.

As a fun aside: back then, we also tried Kinetic Ethernet Attached Storage [3]. Great idea but what a pain in the rear it was. We did however have a very early access device. No idea if it's still in production or not.

[1] https://www.supermicro.com/en/products/system/3U/5038/SYS-50...

[2] https://www.supermicro.com/en/products/system/4U/6048/SSG-60...

[3] https://www.supermicro.com/products/nfo/files/storage/d_SSG-...