Hacker News Reader

> Nothing is streaming

Except CouchDB entire REST API, where nearly every endpoint streams result row by row, which is great.

> Performance (and likely security, due to age of the engine) in this subsystem is extremely poor

As of security: wrong guess. Go compare number of critical CouchDB CVE for last, say, 5 years, with any other DB you want.

As for query server: indexing is slow for the reasons you pointed out. Requesting persisted index is ok. Taking in account CouchDB persists everything, it‘s not in-memory DB (which is again great), I‘d say it‘s even fast.

> Performance in this (Erlang) area also appeared to be proportional to the code size of the map function

Performance in this area depends much more on how branchy is the json doc being processed. It‘s not about CouchDB itself, it‘s about json parser used. There might be a situation when you have better perf using JS views.

> Nothing about this is relaxing.

Indeed. But then it just works, for years, with zero maintenance even for replications – the result I‘ve never even nearly achieved with any other DB I used.

> Except CouchDB entire REST API, where nearly every endpoint streams result row by row

I'm talking about the internals of couchjs.

> As of security: wrong guess. Go compare number of critical CouchDB CVE

I'm talking about Spidermonkey 1.8.5, which is seven years old. Multiple CVEs have been reported including a full RCE, which was patched in later versions. If you believe these are patched in libmozjs185, I'd love to see a link to the source package showing where the fix was applied. Because I can't find it.

Further: https://www.cvedetails.com/vulnerability-list/vendor_id-45/p.... Another full RCE in CouchDB.

> Performance in this area depends much more on how branchy is the json doc being processed

Citation needed. There's an issue specifically about this in the CouchDB issue tracker if you'd like to read more.

> As for query server: indexing is slow for the reasons you pointed out.

Right. It is not performant. That's my point.

Curious what kind of database sizes you're working with too; you won't see this stuff with a few hundred documents.

Correction: the library is too old for that specific CVE to be exploitable. There may be others, there may not be. I don't know – and that's actually the problem.

The chief complaint here is that it's very hard to reason about the security of seven year old software that distros are pulling out of their package repositories citing security issues (e.g. Alpine Linux). Advisories tend to say "versions before X", and you're left to either wonder or read tickets/code. Not everyone wants to sign on for that, and it's work that isn't going toward writing your app.

> I'm talking about the internals of couchjs.

_list functions stream, and was specifically designed to be able to stream. However, streaming does not help much in this case, moreover too small chunks dramatically reduce already awful _list perf.

Taking this in account I see no value for views to stream. Sending all emitted KVs for the doc to Erlang bits in one turn seems both much more predictable and safe.

> Multiple CVEs have been reported including a full RCE, which was patched in later versions

Last CVEs has nothing to do with Spidermonkey. BTW they may be patched without upgrade, with 5LOC long design document in Erlang.

The main reason of last CVEs is inconsistency in parsing improper JSONs by Erlang parser. Namely, most JSON parsers process '{"abc":1, "abc":2}' as {abc:2}, but old jiffy parses it as {abc:1}. BTW severe inconsistencies in parsing JSON are pretty common across implementations, please read http://seriot.ch/parsing_json.php for more details.

> Citation needed. There's an issue specifically about this in the CouchDB issue tracker if you'd like to read more.

I‘ll give you no cite, sorry, because we discovered the effect during internal tests. Reason is simple: accessing values in deep branchy JSONs is generally faster in JS, because it‘s native format. JSON in Erlang is a monster like {[{<<"abc">>, 1}]} for {abc:1}, which, when has a lot of levels and nodes at each level, performs bad for selecting random node.

> Curious what kind of database sizes you're working with too

We sometimes measure number of docs with M postfix ) Not very often, however. In my humble opinion, if you plan to have CouchDB with, say, 100M docs in a single bucket, you probably chose wrong solution.

BTW, same for large kitchen DBs for buckets with, say, 10K docs.

> I‘ll give you no cite, sorry, because we discovered the effect during internal tests

We discovered the issue in internal tests and reported it upstream where it was confirmed; there's nothing to discuss here.

> BTW they may be patched without upgrade, with 5LOC long design document in Erlang.

I don't have to hand-patch other database systems. Further, as of now there are no functioning packages for multiple versions of Ubuntu. Multiple competitors do not have any of these problems.

> Last CVEs has nothing to do with Spidermonkey

It was a full RCE that didn't have vendor packages ready in time.

Sorry, but using seven year old language runtimes is daft. It might be fine for you, but it's not appropriate in environments where you care about security and performance, or care about the overhead of making your team reason about these things unnecessarily.

> We sometimes measure number of docs with M postfix

Yeah, we did this in 2003 on commodity hardware, and even it built indexes faster than CouchDB builds map/reduce indexes. Fix the external view server protocol or be honest with people and kill it off – the status quo is unacceptable.

Finally, here's a five year old issue that's still open admitting what I just explained: https://issues.apache.org/jira/browse/COUCHDB-1743

100M docs? Of json format? On commodity hardware? In 2003?

Hahaha, hello long waited friend from parallel universe!

Easy enough to convert on the way out, using a mature database system without these performance flaws. The app got done and performed well, and we weren't at the mercy of an unresponsive community that leaves seven year old dependencies in critical paths.

Keep going; it'll bite you eventually. Don't say you weren't warned.

For balance: there is a newer query/index system called Mango in Apache CouchDB 2.0+, that IIRC is internal and doesn't rely on any external view server. It wasn't in 1.7.1, though, so if you're coming from there, it's very much a "switch query APIs to get tolerable performance" situation.

Also for balance: it's unclear if there are CVEs. There may not be any that are exploitable due to the age of the library (i.e. the bugs landed post-release). However, it's also very tricky to rule these out one by one, and the dependency is slated for removal from some distros.

There may be no material security issue. But the age of the software does not instill confidence, and newer engines are likely far more performant.

> You can write map/reduce views in Erlang itself as an alternative, but that engine is disabled by default due to security concerns – it's "not sandboxed".

I get the impression the author(s) of this DB were still at the stage of needing to brow-beat the infrastructure into submission, or somesuch :S

One of the stated goals on their Spidermonkey replacement issue (N.B. it still won't fix the view server protocol performance, but at least it'll be a modern engine) is "Get my C skills back out of the closet", and will require https://github.com/jquery/esprima.

It's depressing.

> You can write map/reduce views in Erlang itself as an alternative,

Or you can use a query language mostly similar to what MongoDB uses http://docs.couchdb.org/en/2.0.0/api/database/find.html

> If you make a change to a single view in a design document, the database system will rebuild all indexes in that design document unconditionally,

That's a feature. If they don't usually change together put them in a separate design documents.

> and manage those documents yourself.

It's a document database. So managing documents yourself is kind of what you do. Isn't it?

> he data itself is stored in a single file per shard, opened in O_APPEND mode, effectively serializing all writes on the shard.

So have more shards? A lot of databases use append only mode. There is a reason for that. Having uses the db for a good number of years, it had never lost data. Even during power outages or brutal restarts. That's more than I can say about other databases.

> no shared memory for IPC,

Shared memory IPC doesn't magically solve problems but it is a minefield of bugs.

> use of mmap AFAICT.

Large blocks of allocated memory use malloc instead of brk so memory allocation uses mmap. For file IO mmap isn't a panacea either. If data is not there, reads will still go to the disk. If it uses regular reads through data should end up in the page cache

> kswapd active,

Hmm, seeing kswapd being active is not something I look forward on a production server. From what I remember I have seen page cache utilized correctly and that's what should matter. Not kswapd.

> Transactional behavior: no, aside from a single bulk write feature

bulk docks is not a transactional feature. The database is not CP database so it doesn't have transactional feature.

> But worst of all: every time it starts up, it tells me "it's time to relax". Nothing about this is relaxing.

Yeah that is silly never liked that. From what I remember it was a DB that was built when MySQL ruled the open source DB world and many of the current DBs were not there. It was actually pretty relaxing being able to insert regular json object in it, index them, have master to master replication (which most dbs still don't support), have a nice built in web dashboard to inspect and query documents.

> Or you can use a query language mostly similar to what MongoDB uses

Yes, provided you started on 2.0, as I specifically stated in a reply.

> That's a feature. If they don't usually change together put them in a separate design documents.

Unnecessary computation and I/O is not a feature. They should hash the views individually. A database that makes me do extra work to keep it from doing unnecessary work is not helpful.

> So managing documents yourself is kind of what you do. Isn't it?

This doesn't have anything to do with the unnecessary index rebuilds. CouchDB has all of the information it needs to avoid this unnecessary computation and I/O.

> So have more shards

More shards spending time serializing and deserializing JSON instead of doing useful work. Throwing shards at inefficient code might help, but it doesn't address the underlying problem.

> Shared memory IPC doesn't magically solve problems but it is a minefield of bugs.

Lots of database systems manage to use it just fine. You probably use one of them. And it does solve a problem: it avoids serializing and deserializing all of your data down a pipe.

> Large blocks of allocated memory use malloc instead of brk so memory allocation uses mmap.

Yes, with MAP_ANONYMOUS. I'm not talking about anonymous mappings.

> Hmm, seeing kswapd being active is not something I look forward on a production server.

For swap? Sure. For file-backed mmap pageout guided by madvise? I actually do. It means the VMM is being used for its intended purpose.

> bulk docks is not a transactional feature

It was an example of the one way to achieve an atomic write of multiple documents, which comes in very handy for any nontrivial application.

I'd honestly be fine with them just deprecating the entire external view server protocol – but as it stands now, it's a supported feature, the performance documentation in 5.2.5 of the manual is nonsensical, and it performs very very poorly due to excessive serialization/deserialization and inefficient IPC.