The history of Berkeley DB
queue.acm.org
queue.acm.org
BDB was impressive in that you’d file a bug with them & they would have a patch for you by the next day. If you did the same thing with Oracle, stars would have to align even to get an ack.
I think that’s because they kept BDB pretty simple. I happened to sit in a meeting with Margo Seltzer where the (later distinguished) engineer from Amazon was asking for new features. I think Margo must have said no to every feature request because it didn’t align with their roadmap. I’ve never seen anyone do that. It was impressive to watch.
Then again, Oracle acquired Sleepycat in the end.
We used BDB at Amazon way back in the beginning days. It was the first DB the company used, before Oracle and before any in-house systems were written. Shel wrote the code to suck the info we got from Baker & Taylor (basically Books In Print) into a BDB, and that was the Amazon bibliographic database, from day one.
He also knew either Keith or Margot and had a few conversations with them along the way about some performance tweaks that we either wanted or had developed (can't recall which).
Just to clarify, the engineer in that meeting was Peter Vosshall. I wanted to quote the sentence about Amazon rather than the person.
Also, I was a newbie at the time (probably still am) & I consider myself lucky to be in the meeting. They probably let me in on the meeting because I didn’t mind carrying the pager as much. :)
I was there. He was my boss. It was miserable.
The company got bought out, mostly as acqui-hire for the scientific staff.
The main tech was by a Stanford prof. and one of his ex-grad students. An X-Windows/IRIX-only(!) based tool for molecular modeling. It added support for integrating bioinformatics analyses, gathered from public web servers. Which was cool, but companies wouldn't buy it because sending proprietary data to public sites as a big N-O.
Many companies had similar resources in-house. The program could call out to awk or shell as a configuration language for doing those searches. But our customers were all switching to Perl and didn't care for those languages. So, late in the cycle we added Perl integration support.
There was only one person who could really work on the core tool. (I remember fixing ~10,000 compiler warnings, nearly all because it used a home-brew GUI based on void* function pointers.) There had been a push to port it to the Mac, but that took a year and the fork was unmergeable.
We developed an intranet server product for basic bioinformatics tools in perl that would mirror the core bioinformatics databases, process them for FASTA, BLAST, and text searches. Mind you, this was 1998, so CGI.pm and a home-brew templating language I threw together.
We needed this because we were ramping up science consulting (I don't recall why any more), and needed our own internal tools because proprietary data.
So, Mike came in as VP of engineering, with a product that was hard to maintain, nor could it quite pivot to the new hotness of bioinformatics, nor pivot to the new hotness of the web. (Get this: there was a proof-of-concept version that used a persistent version of the tool running on the developer's desktop, in an off-screen display buffer, which presented a clickable image map to the user. Each click would round-trip another screen shot to the running program.) And without the ability to really hire the people to get out of that hole, because funding was running out.
On the organization level, there was also then-new CEO, who sales and marketing referred to as a tornado, because she would come in and everything would get blown around and mixed up. Not a good boat to be in.
This was also Mike's first VP-level management job, so rather like jumping into the deep end.
Mind you, I was a fresh-behind-the-ears 27 year old, with only 2 years of professional experience in an academic lab. I know little about what the politics was with upper-management. I can agree that it was miserable.
Going back to Sleepycat, I actually evaluated BDB vs GDB vs. a few other technologies for one of our projects, and ended up having problems with all of them. Only after I left, when I met Mike after he started with Sleepycat, did I point out that I he probably should have told me about his connections with BDB. He agreed. ;)
pretty sure mike was a vp at britton-lee hardware database company during the mid 1980s in los gatos. i was there.
Wouldn't he have been rather young to be a VP?
For more complicated setups, sqlite also works wonder.
A post processing tool would read the profiler data and create a BDB file with support for extracting call graphs and topN sort of analysis.
The final GUI was implemented in Visual Basic since other developers would not use the TUI/CLI based tools in console.
The next project used BDB to store file system metadata on embedded NAS storage. We implemented a fast ‘find’ like service based on file metadata (stat fields) stored in BDB with support for user defined file metadata.
So much fun to implement this stuff and see it provide so much value.