nemild

The other question is how many upvotes are "sponsored" upvotes.

Thanks so much, fixed!

Thanks for the kind words. Most thoughtful engineers I've met are believers in right tool for the job and thinking in tradeoffs, not good and bad - and so I'm a big fan about digging into MongoDB's value and understanding your projects' needs.

MongoDB has matured as well over the last decade, and it's a testament to the hard work of their engineering team.

You may like this old post that I wrote which echoes the broader points I make in this series about thoughtfully making eng choices and having better eng debates:

https://www.nemil.com/musings/betterdebates.html

My own research doesn't back up that this "unsafe write" was done by 10gen for benchmarking reasons, but rather due to an early expected use case (from a footnote in part 1 of my series):

Waiting for writes was off by default:

> This "unchecked" type of write is just supposed to be for stuff like analytics or sensor data, when you're getting a zillion a second and don't really care [if] some get lost [or] if the server crashes.

This was rarely the typical use case in most startups - even though the defaults were based on it for a long time.

It’s unclear to me how long this was the top Google result, and it was earlier in Mongo’s life (2009). This may be one of the benchmarks that led to anger at competing NoSQL vendors - and seems to me like a mistake rather than a malevolent effort.

https://www.nemil.com/mongo/1.html#fn2

_____________________

MongoDB's CTO has also mentioned that if he could go back and change anything it would be this early default, as his earliest customers valued it - but he quickly realized that it caused issues for others.

(Throughout this series, my goal has been to be tough, but fair to both sides)

Thanks for sharing. It'd be really helpful for others to understand the use cases that made MongoDB a good choice for you in 2010.

Part of what I've felt is missing in the debate is where early MongoDB versions were a good choice, and where they were a poor choice. For example, I've seen it used in a number of places where transactions are really important (a number of fintech companies) - and I'll assume this was not a need for your team?

And as I've stressed in this series, my goal is not to argue that MongoDB is a poor choice, especially as it has matured. Rather, it is to argue that we have to pick the right tool for the job - and can't blindly follow hype or marketing.

Survival? A planned IPO?

(Author here) You can read parts 1 and 2 of the three part series:

- Part 1, Why Did So Many Startups Choose MongoDB: https://www.nemil.com/mongo/1.html

- Part 2, Startup Engineers and Our Mistakes with MongoDB: https://www.nemil.com/mongo/2.html

You can see most of the notes from my interview with MongoDB's CTO, Eliot here:

https://news.ycombinator.com/item?id=14804765

And the interview notes related to MongoDB's marketing are somewhere in this HN post:

https://news.ycombinator.com/item?id=15124316

(Author here) As part of this three part series, MongoDB’s CTO, Eliot Horowitz, was gracious enough to spend two hours chatting with me.

In our discussion, we touched on 10gen’s early marketing strategy, which I've combined with notes from my research:

- 10gen’s Marketing Focus: Eliot noted that much of 10gen’s marketing message was meant for large enterprise CTOs and engineers who made database decisions. If you’re trying to build a database company, this is where most of the money is. But many startup engineers I knew didn’t realize this, and seemed to think the message applied to them. 10gen’s explicit focus on sponsoring hackathons and and targeting startups also encouraged these issues.

- Anger on HN: Eliot and I disagreed where some of the MongoDB anger on HN comes from. In his view, a fair bit of it stems from competitors and their supporters. In my view, much of the anger came from 10gen’s fanciful marketing message that outstripped the product in the early days. I believe that if the marketing message had been more thoughtful and the product more mature when the marketing ramped up, the community anger would have been much less (but it likely would have significantly hurt MongoDB's adoption, which is a challenging problem).

- 10gen’s Marketing “Strategy”: Eliot argued that 10gen didn’t have much of an early marketing strategy. In my view, their marketing team made some smart decisions that really set them apart from competitors. First, MongoDB’s Javascript DSL, JSON data store, and onboarding experience were critical differentiators - and their product was early to market. Their marketing team then used this as they built the MongoDB user groups/conference network, pitched NoSQL and the MEAN stack as the future, and brought industry allies to their side.

- Engineers and Marketers: We debated how much role CTOs should play in dev tool marketing. At 10gen, Eliot noted that he was rarely involved, instead focusing on engineering and product. My own view is that engineers should be involved in the marketing message in highly technical products - and have some input into marketing. I also believe that marketing a database is very different from other tools in the engineering stack such as a frontend framework.

Much of this debate stems from the differential objectives of engineers (making the right decisions for their teams) and marketers/founders (convincing customers to use your product, sometimes at any cost).

I let Eliot know that I would support him in sharing any follow-up thoughts, with the hope of spurring a thoughtful debate in our community.

Finally, I won't argue that 10gen's marketing and product was enough on its own to explain the growth amongst startups - the usability of MongoDB was a key reason for it's success in startups. Less discussed also was the marketing undertaken by training programs, bootcamps, and conferences (see the marketing around MEAN that inundated Hacker News and Reddit in the early-mid 2010s, as NodeJS was growing).

This thinking is a common mistake I see from teams that think only about tech.

User acquisition is critical in the success of marketplaces. Usability is often ignored too.

Even today, if you create an eBay (granted, a much stickier marketplace than Kickstarter) with no fees, it's not as simple as throwing the doors open and expecting everyone to switch.

You can read more here, in a different example I dig into (Slack vs IRC): https://www.nemil.com/musings/oss-and-slack.html

If you ever want, I'm happy to have the debate with you (email in profile). If you really want to compete, you're going to need to be aware of what it'll really take to win, not just decentralized/blockchain and no fees.

(And as a nit, I can also build a website in a week that takes 2% credit card vig without blockchain - sure I know you'll argue that I can raise the price later, but I'm still not going to win against Kickstarter without something more than just a cheaper price)

Am I right in thinking that all forms of meat contain heme iron? And in what dosage is this "disastrous" to the body?

If you're interested in this topic, I did lots of data analysis to show the nature of the skew in death coverage vs actual deaths in the New York Times:

https://www.nemil.com/s/part2-terrorism.html

https://www.nemil.com/s/part3-horror-films.html

My own view is that much of the mismatch between coverage and reality can be explained by media economics, where reader interest and media incentives encourage the coverage of extreme events. And while my data focuses on traditional media, I wouldn't be surprised if social media has a much greater skew.

I'm pretty disappointed by this title.

I know that it might be said in fun - and is an easy way to get clicks, but it feeds into the narrative of new, shiny tech. As the article points out at the end, there are very real engineering tradeoffs with GraphQL - and the answer isn't as easy as REST is dead, anachronistic technology that no engineer should consider (the XML analogy felt particularly inflammatory).

Kelly and I were chatting about GraphQL, and his post might be a more thoughtful engineering post: http://kellysutton.com/2017/01/02/do-we-need-graphql.html (the title - Do we need GraphQL? - is at least the way I'd expect engineers to approach the problem, where he discusses the tradeoffs)

In my MongoDB series, I point out a past case where Free Code Camp fed the hype, by telling engineers that the reason everyone had to learn the MEAN stack was due to employability in the software industry: https://medium.freecodecamp.org/the-real-reason-to-learn-the...

(you had to dig into the article to realize that their argument was more nuanced, and that they taught SQL first before MongoDB)

A number of "engineering" posts are not written as a thoughtful engineer might, and are in many ways marketing for the products sold (like a training program or code camp).

I had a similar reaction.

(I’m the author of this series)

Eliot Horowitz (HN: @ehwizard), MongoDB’s current and founding CTO, reached out after my last post - and spent two hours providing feedback last week in Palo Alto. It was an expansive discussion, and Eliot was reflective and eager to understand the perspectives I had heard. He noted how much it mattered to him what HN thought.

I left with tremendous empathy for the challenges of building a venture-backed database company - while we also disagreed in a few key areas. As engineers, I hope we continue to have thoughtful, if spirited discussions, like Eliot and I had - where we both are open to being wrong and have a desire to understand differing viewpoints.

In the interest of length, I didn’t write the whole interview into my series, but wanted to share key parts of our discussion on HN:

- NoSQL: I felt that NoSQL was overhyped in the early 2010s - and that 10gen’s marketing claims were overblown. Eliot argued that many of NoSQL’s benefits have been realized and that 10gen’s early marketing accurately reflected the changes to come. For example, while he doesn’t love the term NoSQL for its impreciseness, he feels that both the JSON-like data structure and horizontal scaling are here to stay and were in fact the key changes NoSQL led to (not the SQL DSL). In that sense, he argues that Amazon RDS is a form of NoSQL today - and that NoSQL had a powerful impact on the roadmap of existing SQL databases.

- Data Loss: I generally have stayed away from talking about the controversial examples of MongoDB data loss in the earliest days (there are several of HN posts that note this). I know personally of only one team that lost data, but it’s always hard to understand if this was a database issue - or their own mistakes. I did ask Eliot explicitly about this, and he said that these were exceedingly rare.

- Defaults: I feel like it’s playing with fire to set bad defaults in a database - with numerous data breaches due to 10gen’s early decisions on authentication, remote login, and encryption (see for example, https://snyk.io/blog/mongodb-hack-and-secure-defaults/ ). For auth, Eliot argues that developers need to take responsibility for exposing MongoDB on public servers - and that the SLA for a self-hosted instance is different than a managed instance (at minimum, I have issues with users having their data exposed to the world through no fault of their own). He disagreed with 10gen’s decision to turn on auth by default in later self-hosted versions once MongoDB ignored remote connections by default (but thought this was the right choice for the managed Atlas service). (But before 2014, the default behavior was no auth - and accepting all remote connections, see https://snyk.io/blog/mongodb-hack-and-secure-defaults/ ; Eliot notes that this took a while because changing the default would have caused issues for existing customers)

I do have concerns when 10gen explicitly targets junior developers (to be fair, he could never have predicted the growth of Node and the interest in the backend for frontend engineers). What he says makes sense say 20 years ago, but with 25% of new software engineers coming from coding bootcamps with non-engineering backgrounds, I worry that defaults matter ever more in dev tools (and even seasoned engineers may mess this up, if they’re coming from a database with different defaults). We discussed analogies like seat belt lights versus the responsibility of passengers to know better. He also argued that waiting to get all this right - not just auth - would impact database innovation, while I think there’s a balance that gets us a lot of the low hanging fruit (like security).

- Mistakes with MongoDB: Eliot felt that certain posts (such as Sarah Mei’s popular HN post about Diaspora: https://news.ycombinator.com/item?id=6712703 ) misunderstood how to architect MongoDB. Regardless of the particulars, I think that new technology is especially susceptible to issues like this until the community develops broader knowledge. The failure states are well known in many relational databases and there is a broad base of knowledge about how best to architect relational data models. As startup engineers, we have to weigh tradeoffs of new vs old tools, with some new tech being game changing for startups (see PG’s much cited post about the benefits of new languages like Python: http://www.paulgraham.com/gh.html ) while many others are hyped new tech that damages startup productivity.

- SLAs are hard in databases: Early products make choices that customers get used to (unacknowledged saves), and you can’t quickly just change the default behavior, esp in a database. 10gen’s early customers loved not waiting for the writes, and only later did 10gen realize that this was an issue for others (the default is very different in nearly every other database). To migrate their first customers without dislocation, they had to hold off on changing the default behavior for longer than they wanted to. In Eliot’s view, their competitors would unfairly argue that this default was a way to juice benchmarks, hoping to stoke anger at MongoDB and cut into their growth. (I tend to have sympathy for 10gen’s perspectives, with the controversial, mistaken benchmark that I referred to in part 1 looking to me like an honest mistake)

Generally, he felt that the issues with MongoDB were few and far between - and that the benefits of MongoDB was game changing for so many startups. Many of them would not have survived without MongoDB.

I’ll add some more notes from the interview in part 3, where I go into 10gen’s early marketing. I also let Eliot know that I would be happy to share/publicize any broader responses/critiques he writes, so that we can have a thoughtful debate that benefits others (and he can point out issues in my arguments).

(Apologies in advance if I’ve made mistakes in representing Eliot’s views)

I can relate. I wrote an essay on this a while back:

https://www.nemil.com/musings/shinyandnew.html

Excerpt: Pity the modern web or mobile app developer on Hacker News. As a full stack startup developer in 2012, you were building websites using Ruby/Rails on the backend, Backbone/Coffeescript/Underscore on the frontend, while deploying your app with Capistrano (or the relevant Python analog). By 2013, you'd transitioned to Node/Express/Mongo on the backend, Grunt/Ember on the frontend. In 2014, you'd switched whole hog over to the MEAN stack (now with promises), but experimented with Koa and considered a shift to Go (after the Express core contributor made the jump).

In 2015, you're using Gulp/ES2015/React on the front end, Express/Go on the backend, React Native over the native mobile languages, and slowly transitioning your system over to microservices with Docker. Soon, you'll be transitioning to Phoenix, figuring out if Angular 2 is the right choice - and maybe even plotting a world where Go can work on Android and where an open sourced Swift might fit into your stack. (I’m clearly exaggerating for effect, though this is a fair representation of what is popular in HN headlines)

I wrote this series over several months and am happy to answer questions.

Part 2 will dive into the benefits of Mongo along with the mistakes that some teams made with Mongo. Part 3 will dig into Mongo's marketing strategy and how it influenced uptake. Sign up here to be notified: http://eepurl.com/cxs5Zr

As any reflective engineer, it's important for me to provide a thoughtful perspective backed by evidence. This can be a fraught discussion and I personally know teams that truly value Mongo, and other teams that had tremendous issues. I want to provide color about the early years of Mongo (not today). My key point is that as engineers, we have to get beyond hype, thinking critically and choosing the right tool for the job.

For anyone needing context, last year I wrote a Learn X in Y about Solidity:

https://learnxinyminutes.com/docs/solidity/

Please note that it is fairly old at this point (contributors welcome) - so this is just to give you a sense of the syntax and key concepts.

Instagram was financially a steal for FB, in retrospect. And no, you can't just say "they paid this much for download, that is dumb". There is a lot of nuance, for example:

- Does the company need it to survive or succeed?

- Can the acquiring company better monetize or grow the asset?

- Are the growth trends exponential, such that a huge amount of other people may soon be using the product?

- Is there some strategic advantage to the team - or product category expertise?

- Is the product easily defensible? (e.g., strong network effects)

I can't speak to this particular acquisition, but there are a number of examples where heavily "overpaying" may actually be a steal in the long term (and of course, the opposite).

I like your proposed title, "Another 'Don't Cargo Cult' article" on its own seems dismissive, when the content seems quite useful for many engineers (the acronym need more work, though).

And you can make good money while coding and working with technology. You don't have to be a C level executive or work primarily on the business side.

If so, I would focus on entrepreneurship. You don't need an MBA - and it may even be counterproductive for the earliest stages of entrepreneurship (coming from someone who has both an eng degree and MBA).

Still, hard to say 100% without knowing your background and goals.

Nadia Egbal (github) has a great list of financial strategies for open source:

https://github.com/nayafia/lemonade-stand

I'll give you one piece of advice: beware premature scaling. If you have a headful of ideas, I'd focus on building something people want, not thinking about scale or other technical optimizations.

If you're an engineer like me, we'll often overweight tech choices, rather than the product and user acquisition decisions that matter in a social network.

You might like Merchants of Doubt which goes deep into false equivalency in media coverage.

One thing that's a bit weird is that the donate link is not over SSL (and it says Powered by Stripe).

Vinod Khosla (Sun Microsystems, Khosla Ventures) is a big fan of this piece. You can see his version here (has his notes in the margin):

http://www.khoslaventures.com/wp-content/uploads/What_makes_...

As someone who's done a few entrepreneurial ventures and also has an MBA, the causal reasoning is indeed the default in most MBA programs.

A few weeks back, I wrote a piece about assessing and setting your values as an engineer:

https://www.nemil.com/musings/software-engineers-and-ethics....

I was surprised we didn't discuss this more and I wanted to help someone early in their career think through it.

(I also got some good feedback from YC's Paul Buchheit who helped coin "Don't be evil" in the early days of Google)

Want to drop me a note (see my HN profile). I've done these ancestor maps in the past in other cases, and have some ideas.

For those interested in converting PDF tables into CSV, there's also Tabula ( http://tabula.technology/ )

(Used by many journalists to analyze the data in PDFs)

[OP] I spent some time categorizing death coverage in the first pages of The New York TImes compared to actuality over a 20 month period. I’ve been intrigued about why so many make threat assessments based on news coverage, and wanted to see if the “sampling” of news was close to representative.