Startup Technical Diligence Is a Waste of Time
codingvc.com
codingvc.com
1. Many of my clients are unsophisticated when it comes to technology, regardless of whether they buy a "tech company" or not. Yes this includes both PE and VC clients. When a PE firm makes an investment in a tech company (and take 51+% of the company) they need to be confident the technology will hold up. About 90% of my work is PE, with the other being 10% in VC (mostly Series A +). Believe it or not, many commercially successful companies do not always use "today's world of SaaS tools, APIs, and cloud infrastructure". And even if they do, just because you use AWS doesn't mean you've setup a VPC correctly, you have multi-az setup, or sensitive data isn't encrypted at rest. These can have potentially costly (i.e. lose customers / brand recognition) consequences. (case in point -> https://news.ycombinator.com/item?id=11999712 )
2. If you're a VC, and especially an early stage one, then no, due diligence isn't necessary. At all in fact (including finance, etc). You're thesis isn't aligned with careful analysis, it's spray and pray, so diligence isn't a worthwhile endeavor. I wonder if something prompted the author to write this or if it's just a random rambling. I thought it was pretty well understood that doing diligence at the angel/seed stage was almost unanimously worthless.
3. Tech due diligence at the later stage of VC, if done right, has more to do about scalability of operations and processes then it does about the specific tech. When you put $50M into a company, spending $50k is a pretty good safeguard.
4. Tech diligence, by the right partner, could have certainly identified risks at Theranos or uBeam (assuming there are some there), but that's precisely what it is - risk. I'm pretty confident the investors in these companies understand the risk, they simply just don't care, because the potential outcome will return their whole fund and some. Do this 10 times and one bet is bound to pay off.
5. I always try to remind entrepreneurs, VCs are finance professionals, not tech professionals. While some may have made their way into VC because of their previous technical ability, there are also many who do not. And just like when you stop coding for 6 months and get "rusty", so do previously technical able VCs who get older and out of touch.
I think you're off a bit though. There is a place for diligence at early VC stage, but it is almost entirely about the people.
That said, they could also just randomly choose without telling anyone and justify actions after the fact. But, as you might guess nobody who does that is going to say they did it that way.
The basic problem is if your going to randomly pick then why should I pay you anything?
PS: One fund did chose to fund every YC company which is close to a random choice.
Alternatively, a common approach when getting a lot of applications is to simply randomly look at half and ignore the other half unless you can't find enough good options.
So, in the real world you would be able to talk up various points. However, I don't think you can convince a politician to use your random methods unless he can sneak a few choices in.
http://www.npr.org/sections/money/2016/05/20/478883658/episo...
So, we're roughly in the same line of business only the customer relationships are reversed (10% PE, 90% VC).
What I'm writing here is from a European perspective.
(1) strongly agreed
> (2) If you're a VC, and especially an early stage one, then no, due diligence isn't necessary.
I don't agree with that, not because it is 'bad for business' but because not doing tech DD at all will dramatically increase the number of bad investments. And with the size of A rounds at the moment it will not take a whole lot of those to materially impact fund performance.
Some due diligence is required, at almost every level of investment, and the easiest way to get to that is to take some minimum amount or a percentage out of the budget to be invested and to use that for a check just prior to actually investing. The bigger the investment, the more complete the process will have to be. Especially when LPs money is on the line.
The reason why this is important is not because the money will kill you after a few bad investments. That's a really important reason, but not the only one and in my opinion not the most important one.
Bad investments eat up partner time. That's the one thing a VC is hard pressed to make more of. More money is usually easily found if that is what is needed to fix something. But partner time is a limited asset and it does not take a large number of flakey companies to significantly impact the ability of a fund to do further investments and to eventually make the fund an underperformer. It can even impact later funds. So the fewer the number of bad investments/time consuming investments the better for a fund.
Early stage (A rounds) require a bit of work, but even at the angel/seed stage it's worth it to take a (very short!) look at the tech to make sure the people involved know what they're doing and that there is no fundamental flaw.
Obviously this should not turn into a full blown DD and legal and financial are a waste of time. But a quick once-over can help weed out the bad stuff.
(3) Strongly agreed.
And I'd like to add security to your list as well as some review of how they got where they are as well as general team quality. I've seen some pretty horrific stuff over the years in this respect (CEOs and CTOs that had absolutely no idea of what their job entailed).
(4) I still wonder what the story is there.
Spot on that it is all about risk, but the multipliers in the US being what they are you're allowed to take more risk there than in the EU, where the multipliers are structurally lower.
(5) I don't see it your way. Some of the VCs that I work with have 'rusty old guys' who are also the sharpest tacks when it comes to identifying the real issues underlying a problem. All that experience counts for something and even if the tech changes the lessons learned remain.
Of course, being one of those 'rusty old guys' myself that could be a biased view ;)
Cool, let's merge and take over everything then :)
> (2) If you're a VC, and especially an early stage one, then no, due diligence isn't necessary.
Sorry, I shouldn't have been so blanket with that statement (and you could probably tell, I'm a proponent of doing diligence regardless). If you're an angel investor and doing $10-25k deals then I would comfortably say it might be a waste of time, but if you're doing $1M seed deals, I definitely agree with your viewpoint.
> All that experience counts for something and even if the tech changes the lessons learned remain.
I don't disagree. My point here rather is that many people (i.e. entrepreneurs) see VCs as these more-than-mortal tech gods, which in reality, they are probably more fallible then you and I.
Deal.
> My point here rather is that many people (i.e. entrepreneurs) see VCs as these more-than-mortal tech gods, which in reality, they are probably more fallible then you and I.
Oh, that I'll happily agree with. They are - as a rule - more banker or record label than tech gods and they're certainly not more than mortal. All human failings present and accounted for.
"Our biggest obstacle to growth, right now? We still can't seem to attract candidates with strong CS fundamentals. Maybe we should crank up HackerRank hazing process from 3 hour-long sessions, up to 5. Because I mean, just look around -- aren't we amazing? At this critical juncture for our venture, surely we can't afford to dilute our crack team with mediocre talent."
That is total speculation though.
Here's how we're looking at what we're doing though. It's viable with a fairly low tech approach, and that's a good start. But we can use technology to take something that "works" and turn it into something that purrs.
It's like the phone switching terminals of old. You'd have an operator and you'd tell them who you want to speak to, and they'd plug the right wires in. It "works". Bell made money on that. They had the right product, strong sales people, and they had a business model. But to think even for a second that they could compete with that nowadays?
There's some irony that VCs investing in the industry that owes its life to computing and telecommunications are saying "Eh, the technology you're using doesn't matter".
But I do think that technology isn't the bottom layer of Maslov's hierarchy of startups. The existence of a market need is at the bottom, then whether your product is able to address this need, then whether you're able to reach that market, then whether you can get paid for doing so.
These layers are filters. It's not surprising that most startups fail on the ones that come first. That doesn't mean if you're shooting to be Uber, or Dropbox, or Facebook, or Google, that you tech doesn't need to be on point. And the best point to start using sane tech practices is, well, as early as possible, preferably once you've validated your other assumptions.
That's a very good! So it's 5% of startups that posted a post mortem (and it didn't disapear into ether), rather than 5% of all startups. These two things are very different things.
> we can use technology to take something that "works" and turn it into something that purrs
Well put. However, nowadays, the tech itself is often not the key bit in making the difference between 'works' and 'purrs'. Of course, there are exceptions, bigger and smaller ones. More often, it's understanding the problem, the users, and finding the fit. You can build a product with tons of technical debt and wtfs that gains some traction and then spend resources on improving it. However, it doesn't work the other way round. It often takes an experienced engineer to say 'right, let's start some cleaning up' at the right time. However, if your product doesn't solve any problems, no one is willing to pay for, etc, brilliant tech solutions are not going to help it.
> start using sane tech practices is, well, as early as possible, preferably once you've validated your other assumptions
Totally! It's just that it's often hard to tell when it's the right time.
There's definitely some sampling bias from the post-mortem article. Alas, this is the the most relevant public data that I could find. I do have a good (anecdotal) first-hand dataset: I've worked closely with about 50 portfolio companies at this point, and I've tracked a few dozen more that my fund didn't invest in. Maybe 5 out of the 50 had real technical risk, and 2 or 3 struggled and/or failed due to technical issues. For the remaining companies, the challenges have always been market-related, hiring-related, funding-related, and so on.
"Homeowners' insurance is a waste of money. 99% of the time your house doesn't burn down!"
You don't buy insurance because you expect your house to burn down. Similarly, you don't perform due diligence because you expect to find something. You do it to catch those rare 1 out of 20 events where something is f*cked up.
"Due diligence" does not mean making a judgment call on whether three designers can build, scale, and operate a tech company. Due diligence means diving into the details of a company you are going to fund and checking, among other things, that the things they've said are valid and that there are no skeletons in the closet.
Don't get me wrong: due diligence sucks for all parties involved. If you are investing your own money, you have the right to do as little work as you want and skip it, but when you are entrusted as a fiduciary of someone else's capital -- and paid for the job -- it comes with responsibility.
Given that time is a factor and given that I have 2-4 hours to spend with the companies I'm most interested in before making a final investment decision, I've gotten the most value out of spending 95% of that time on non-tech diligence. YMMV.
I'm coming more from a founder perspective though, and I think there's some dangers with the mindset as a founder, and I see it with a lot of people in the chinese startup community.
Whilst as an investor, one might get to play a bit fast and loose with technology choices, I believe that as a founder doing so is suicidal. I see startups here struggling to hire PHP programmers because they made the decision to go with PHP because they heard it was easy, and later on realized that that may have been a mistake. They try to smooth it over with funding later. PHP programmers over here have the same reputation. They learn it because they thought it was easy, and that there's a lot of companies desperate for that skill. It's a match made in hell.
Similar boneheaded decisions, preventable with a week of due dilligence reading, are made by founders here all the time. One thing I found interesting is that the people making these kinds of calls are always working on C-list ideas. It's always derivative businesses in markets that just don't care. They need to get their MVP out fast and half baked because what they're working on should have died on the drawing board. They're desperate for investor money because they think it will patch over major errors they've made up to that point.
Some of these ideas are salvageable. It usually requires restructuring on both the business side and the technology side, preferably to the point where they make sense as an integrated system. One guy I was working with, his startup is trying to create a site for farmers and rural Chinese to sell things on that, and that is so interesting, Alibaba jumped into the ring recently. Now he has a dinky site that does not solve anyone's problems. It's a website, which is greatly hindered by the fact that most villages have a single computer, and the people who wanted to trade with each other would sit at that computer, together.
I suggested to him that he change things up a little. Small farmers do suffer from information asymmetry with the people they're selling to. Technology wise, most of them have non-smart phones, but China Unicom is rolling out mobile coverage to rural China, because that's their market tomorrow. So, provide informational and automated brokering services over a website on the buyer end, and an SMS gateway on the seller end. Ship the FAQ on actual paper. Allow farmers to post their wares with a formatted text message, have offers coming in an hour later, goods shipped the day after. Give them a reason to upgrade to smartphones, pack in more features on the app once data becomes cheap. Do agricultural futures 5 years down the line, in one of the hungriest economies in the world.
The part where technology makes this interesting is that it allows you to reach a market segment that is unreachable over conventional means. Innovation is the edge startups have over large, well funded enterprises. Founders ignoring that side are blindsiding themselves to half their business.
So a team working on such a startup idea is unlikely to be seeking venture funding. Instead, they're probably a university professor and a couple grad students, or just the grad students, or an eccentric genius in a garage. When they have something that works, then they'll seek venture funding to turn it into a company, but at that point the idea has already been significantly de-risked (and they'll often go to a "name brand" VC, since the de-risking of the business makes them attractive to pretty much all investors). If they never get to the point where they have something that works, then they go back to their research or get a job, and you never hear about them.
The idea that "even great engineers" might take on technical debt is insane. In a startup situation, great engineers take on technical debt with the greatest of glee and at every opportunity, as the primary goal is to find product market fit. Bullshitting about scalability, maintainability and frameworks is counterproductive and maturely, one understands that any software will be almost totally rewritten whatever happens: take this example by John Carmack to start a prototype environment for rapid iteration, explicitly laying out ways in which it should be rewritten if successful https://groups.google.com/forum/#!msg/racket-users/RFlh0o6l3...
Many good engineers stay in their comfort zone of some part of the software lifecycle. Seems like the author was trying to assess people imagining their situation was that of big company software project going in for a third generation rewrite with little uncertainty about what it should do. This is not the situation startups are generally in, and great engineers in this situation adapt; they pivot without getting stuck on old conceits, and have a knack of understanding which signals to heed to get to that glorious market fit where suddenly reliability is valuable. Definitely they can be assessed on ability.
Just to clarify a little bit, since I didn't make this explicitly clear in the article: there are certainly questions about the technical side that are useful.
One example from the blog post is trying to understand if the tech team's experience lines up with what they're building. If the founders are 2 infrastructure engineers who are trying to build a social app, or if they're mobile devs who are trying to build a streaming database, then that's a yellow or red flag.
Another example: I've gotten a lot of value out of drilling in a tiny bit when people make bold technology claims. If someone says they've developed cutting edge ML, they rarely expect an investor to ask about it. When I ask them what "cutting edge" means, if I hear words like linear regression or decision trees, then my bullshit detector goes off -- but not for technical reasons. It's not bad at all to use simple ML models, what is bad is misrepresenting your tech, because then I start wondering what else you're misrepresenting.
But for most (seed) VCs that I've met, technical diligence means asking an engineer they're friends with to talk to a CTO they're thinking of investing in. The engineer asks about system design, infrastructure decisions, deployment environment, coding practices, etc. and then tells the VC if they think the tech stack is robust. (VC: "Ah, they're using AWS and Redshift? That sounds modern. Thumbs up!") For many, many startups (SaaS, mobile apps, ecommerce, etc.) I think this type of diligence is basically useless for almost all seed stage companies.
While it's a tough pill to swallow for many engineers, there's a lot of truth in it.
The company I work for (Bored Panda) is a pretty good example of it. I was the first full time engineer, and the company already had 20+ millions of monthly visitors and ~2m facebook followers. Surely, there was tons of technical debt to clean up and uptime was... really poor :)
into what? :-)
Also, I believe, we contributed significantly to the coloring-books-for-artists trend which was quite a thing in late 2015 and sold loads and loads of copies, because I think we were the first major publisher to publish a story about them.
Not something world changing, but it's good to know that I contributed in one way or another to actually help some artists :)
But thinking like this is how Theranos crashed and burned. Their funders needed to ask hard questions, and didn't.
You mean they got to that level of visitors and followers without a full time engineer to help make it all work?
Or was it something like multiple part time engineers?
But for the first year or two, while it was gaining initial traction, it was just wordpress installation on shared hosting, with a free wordpress theme. The founder is tech savvy but not an engineer. And it's a bootstrapped business, the typical born-in-the-college-dorm.
* No homebrewed tech allowed. It's too "complex"
* We should only use off-the-shelf libraries and SDKs because bar-to-entry isn't a real investor risk ("Just trust me, I'm a charismatic CEO who played golf a few times with the best friend of the college roommate of the guy who helped the guy close Series A for Baidu")
* Soft skills, soft skills, soft skills! Smiling > coding
* I don't know what technical debt is, but I'm sure we can safely rewrite our core product from scratch in between Series B and Series C. Product owners, customer support teams, and quality assurance re-on-boarding is a money problem, not a leadership problem.
* Security audits? I have no idea what those are, but lets incorporate IoT into our product suite somehow. That seems hot right now to VCs.
Bubbles. Bubbles everywhere.
Yeah the gig I had got taken over by some guys who said "Why are you doing all that yourself? You can get off-the-shelf stuff for free!" Literally the first words out of my mouth were "We didn't write our own code because we were not smart enough to use open solutions. We had product and performance requirements that could not be met." They gave lip-service to me for some months before letting me go. A year later they still are an order of magnitude below our previous performance/quality/feature levels.
"Well technically yes, but..."
"Good, cut it"
.. 2 years later ..
"How could we possibly have been hacked! We had security right?!"
.. points behind at ten miles of cut corners ..
The above happened on almost every project I have worked for over the years.
Q: "Can we take feature x out of the plan?"
A: "We can't release the product/project without it"
Q: "Can we then add 2 more months to the release?"
A: "Absolutely impossible, the client will not accept it"
You are correct, the reason is severe (and systemic) lack of communication:
Product-roadmap planning either does not involve technical personnel at all, or at least not until the very last minute, when you are brought in to essentially rubber-stamp the plan. Saying "no" takes a lot of courage at this stage, you are seen as incapable/negative.
The underlying problem here is that we are seen as mere technicians/workers, not planners or strategists (we are partially to be blamed for this, but that's a separate topic).
In fact, we're not negotiating. We're describing the situation to the best of our understanding. Indeed, when often these situations have started to feel like negotiations, I've stopped and emphasized that this is not a negotiation, this is my very best estimate as to what I need to get the job done, nothing more, nothing less. Sometimes this gets through to them.
But, its magical open-source! Which means anything can be fixed. Wave that wand and objections disappear.
The new bosses made the mistake of thinking our app was the product. But it was the behaviors it facilitated that was the product. They dummied up a web app that looked like what we'd been doing. But it didn't perform or scale anything like what we'd had. Which will probably kill them in the end.
Mgr: "Joe, we need to get this new FrobNoz done. This week instead of next."
You: "Sure Mgr, no worries."
You: "Hey Frank, you're working overtime to make this happen. Or else you're fired."
Frank tries, then quits. FrobNoz doesn't get done in time. You get fired. Your Mgr gets fired. Your company implodes. ;)
Or alternatively:
Mgr: "Joe, we need to get this new FrobNoz done. This week instead of next."
You (gives cool, direct look to Mgr)
You: "We can do that. But we need to pull resources from Alpha, Beta, or Gamma to make that happen. Which of these three is least important to you?"
Mgr: "Beta can wait, get this done."
Result: You have the people who can do it, (ideally) without excessive overtime. Hopefully the lack of Beta doesn't cause company implosion.
Anyone who says their Mgr demands all of it and won't make resource decisions, has not developed their "cold, direct look" (thousand yard stare?) enough.
Just about anything is possible if you're willing to throw enough money and time at it.
This problem is compounded when managers don't meet the minimum technical skill bar to rationally assess the tradeoffs presented to them by engineers.
If anything, good managers help ensure that this communication channel remains as bullshit-free as possible. Excellent managers actively elicit honest feedback, listen to it carefully, evaluate the risk / benefit tradeoffs, and insulate their reports from political consequences. Unfortunately, very few managers are excellent, which is a big part of why many engineers imagine management as this useless floppy vestigial apparatus.
(I say this from personal experience as a consultant: I learned really, really quickly exactly what managers do for you once I no longer had any.)
Same is true in the Program Manager/Tech Program Manager/Producer roles.
Use open solutions where you can, and don't use them where you can't.
[1] That is in no way denigrating startups that do that. If you can solve people's problems by making a better form then that is a Good Thing.
They should probably ask themselves what sort of defensible competitive advantage they're building, though.
With many founders I know, the answer is "none". This is probably necessary in the very early seed stage - it's hard to build something unique and difficult and still get to market - but it's disturbing how many companies end up with $50M in funding and the answer is still "none". Probably a good indicator of a shake-out in the SaaS world in the near future.
You can't build the next Facebook anymore because everybody's friends are already on Facebook.
Knowledge is the very detailed and insightful understanding of what their industry needs at the moment, and where it needs to move to in the longer term. Founders who build those businesses are typically experienced in their industry and are solving a pain they've seen firsthand. That is exceptionally hard to compete with if they're good. Just because that knowledge can be codified in a 'simple' CRUD app doesn't make it any less valuable.
The brand advantage is comes from being the first mover. If the industry knows your company, trusts that you've proven to have an impact on their bottom line (or their competitors bottom line), then you're probably in a good position to retain customers and build a market. Competition is not a problem for a business that's intent on making a market $100m bigger rather than trying to take $100m of business from someone else.
Neither of those advantages require any code.
If you're a religious language defender (bleh, why are you using Java, Perl, etc.), you're not going to be objective. Different languages are useful for different problem spaces. Somethings are starting to show their maturity.
For example, I use to write software that ran on OpenWRT based tiny/embedded systems in Python, because python-mini was present and I could install it, my dependencies and everything I needed in less that 16MB of space. There weren't many options for that footprint other than C, but that was 2011. If I wrote it again today, I'd have to seriously consider using Rust instead.
Being a good engineer/architect requires a lot objectivity. It involves working at different shops with different frameworks, languages and design decisions and filtering through all of that. It involves constantly searching for "I really hate <x>" and "<x> is a piece of shit" every time you find something you really like and have pushed towards your production release, just to see what issues other people have and if their problems are things you may run into. By the same token, when you use something you hate, you want to do the opposite searches as well (e.g "I really love Powershell").
This article is really written from a business perspective. There are a lot of businesses that survive their startup incubation period with terrible design. Some even become profitable. Good luck keeping your developers though.
Meanwhile if you use that "off the shelf" thing and introduce a few hacks to make it work for your use cases, all I have to learn is those hacks and why they're there. The rest is already documented and maintained by another company who built this dev tool.
Or more likely it is someone's open source project on GitHub, and is just as poorly documented as something in-house would be.
- current dev status (hasnt been updated in 6 months? probably shouldnt touch it)
- community review (is anyone else using it, hows it working for them?)
- limitations (this one is pretty obvious, technical limitations/benchmarks/etc)
- code quality (unless its a third party service you pay for you should be able to see if the code is worth a crap pretty easily).
- vendor lock (if we need to change, how easily can we change? do they use open source protocols/standards that other tools understand?)
You need to think about things like the quality of existing third-party solutions, and how well they will fit into your codebase. You might find a third-party library that is 90% of what you need, but there just isn't a good way to get to the other 10%, and so sometimes even in those cases it makes sense to roll your own.
Other times I see people rolling their own versions of stuff, and it's clearly much worse than readily available libraries. You just have to look at the specific situation to determine the right thing to do.
"What kind of cyber security does your server have?"
Speaking to me in buzz words is a huge red flag.
The fact that you do not speak their dialect does not make their communication any less meaningful. If anything, it means you will always be working for some one-guy or another, instead of being the one-guy.
Serf: But we don't own it.
King: You need to improve your soft skills.
Serf: Sir, we don't own it.
King: You need to realize that you're going to starve if you don't start farming now.
Serf: Sir, how much of the food will you take for working on your land?
King: Don't worry about the agrarian debt or vendor lock. Think of me as a land-as-a-service provider.
Serf: But...
King: I already have deals with farm tools to offer you at a discount! $40 a month with $0.01 per pound of food created! Also, friendly reminder of soft skills..
Serf: Well, sir... I... I guess my farm has yet-to-be-proven...
You can't differentiate your product if you are the product. Tech service startups are just renters of digital property they can never own. They're the street shops of the internet.
King: here's some steel for you to use. ...
Not all of it is accessible (other planets) or fertile (desert)
Not all virtual land is the same, either. Some virtual land is a vendor doing streaming-as-a-service. Some is payment processing-as-a-service. Containers cannot help you regarding the leased farming equipment. No matter what land you move you, you'll still need those third-party services.
This is it, totally agree with that.
* Product must scale. You are not allowed to write code that will scale before we actually need to scale.
One startup I consulted with - was able to replace a broken e-commerce platform ($M+ in 18 months) in one week with a shopify platform and some custom development.
Some startups ARE technology driven (I bet the Theranos investors wished they did more tech diligence).
I find a light / informal discussion about technology to be really helpful. What's been built gives a sense of the real (vs stated) tech skills on the founding team (we all tend to reach for what's most familiar). For example, if the first version is built on the Microsoft stack, you might want to know that.
Finally, code meant to be thrown away always hangs around longer than expected.
The article specifically addresses this:
"There are, of course, some companies with real technical risk: SpaceX, Zoox, Rigetti Quantum Computing, etc. But for a typical consumer app or SaaS tool, technical risk is low enough to be ignored."
And of course it never hurts to ask a few questions - you can quickly find out if the technical team is on-the-level without examining their entire commit history (and as he suggests, on-the-level can mean lots of different things in the context of an early stage software company, not just "10X engineer").
I think his point is that obsessing over technology when the technical challenges are insignificant compared to the business challenges is silly, and likely to lead to lost investment opportunities.
I don't see how the premise of the post can coexist with the phenomenon of clamoring to "only hire the best" and obsessing over e.g. HackerRank interview submissions or something.
Also, what of acqui-hires? I've seen and been part of some big corporate acquisitions in which a company buys a start-up that is not doing good work and has a stupid product, solely to get their 10-person engineering team and then drop the other employees. In which case that start-up succeeding mostly by not giving a shit about any actual business problem and just hiring a group of excellent engineers.
No, I don't think so. Just because you're not doing novel computer science or engineering doesn't mean you aren't doing interesting work, or that your work isn't valued. I think the point is that from an investor's perspective you aren't going to learn much from poking around the code and saying "why'd you use this library rather than that library?"
I can offer my own personal perspective on the value of developer talent as a quasi-technical (I'd say on a good day I'm maybe a 0.9x engineer, most days more like 0.5x :) co-founder of a SaaS company that definitely isn't doing any ground-breaking computer science. To me, the value of a top-flight engineer isn't that they are going to overcome a problem so hard that no "normal human" (like myself) could ever solve it. It's that they are able to look at the fairly vanilla problems we are facing (usability, scalability, etc.), see better solutions than I see, implement them faster and more robustly, and avoid pitfalls that are obvious to them but aren't obvious to me (because I lack the experience to spot them). Maybe we could hire someone cheaper to do the same work, but it would be half as good, take twice as long, and be four times harder to build on in the future.
So to me, having top-notch developers is a business issue, not a computer science issue. Let's face it, even at companies that are doing novel computer science, most of the work that most engineers do is going to be pretty mundane from a CS perspective: Google may be doing amazing work at the boundaries of technology, but someone still needs to maintain the mouseover behaviors in Gmail...
Which is not to say that development work not at the boundaries of technology is inherently uninteresting! I think most people who code (even sub-par coders like myself) get off on a lot of things other people would consider pretty boring, like (in my case with some work I'm doing right now) figuring out a reasonable stop-gap way to manage state when creating a new object that requires putting some hooks into multiple third-party APIs. This is not a novel problem, but there are a lot of ways to solve it, and it made me happy to implement one that is simple and fails in a reasonable way even if it isn't maximally flexible/robust.
Of course, not all founders/managers/executives "get it". And those that don't may opt to just hire the cheapest while paying lip service to hiring the best. But I think (shamelessly tooting my own horn) that people with some experience in the weeds, or with personal exposure to the range of engineering talent out there in the world, recognize hiring sub-par talent is a false economy.
And of course there are all sorts of other factors that go into choosing where to work: can I stomach working with these people every day, does the job fit with my personal life, is the executive team technically illiterate and prone to giving us impossible tasks, etc.
TL;DR: Engineers can be highly valued even when their work isn't "novel" to the point of requiring technical DD at the time of an early stage investment (there are those of us out there who recognize the value of craft), technical work doesn't have to be groundbreaking to be interesting, and making the choice to hire top-notch talent is usually a business decision, not a technical one.
In which case that start-up succeeding mostly by
not giving a shit about any actual business problem
and just hiring a group of excellent engineers.
It doesn't mean the tech isn't valued, but that whether or not it's a marvel of elegance and cleanness is less important than whether or not it works and delivers sufficient value early on.
So you should go into a startup being prepared to do what it takes do meet business needs, and be less concerned about building something clean.
If you can do both, great. If your quest for technical elegance sacrifices functionality in the short term, in a startup that can often kill the business.
I've been part of discussions like that more often than I care, where e.g. some engineer wanted to go off and spend six months cleaning up a codebase that was ugly but functional, but where what mattered to the business was getting more functionality in place even if it meant having to spend 10x the amount to clean up the codebase a year later.
Sooner or later you pay the price, but paying the price later can often be far cheaper, because you may, e.g., be paying with ongoing revenue instead of paying with seed money or early stage VC funding, which has a very high cost.
This is different in a more established business, where cost of capital is less likely to decline as rapidly, which makes it more likely that you get into scenarios where deferring technical debt may actually be a net negative.
In a startup I'd value a developer that is sensitive to business needs and willing to take calculated shortcuts (as long as they also make the business aware when they do) over a technical perfectionist any day.
I feel there's a huge "repricing" of the value of early-stage tech employment occurring as capable developers realize just how shitty the standard 90k + .1% job offer for 50-60 hours/wk, little autonomy, and zero chance for promotion is, in a city where apartments cost 3400/mo, gas is 3.50/gal and income taxes are 9.4%. (And the city wants a "tech tax" on payroll of companies)
I worked in this kind of environment for a few years, gained some great experience/connections, and enjoyed it to a large extent, but probably wouldn't do it again.
Good, experienced American developers are realizing that $90K + 0.1% equity is a really shitty deal relative to what the rest of the tech industry pays, and they're giving up startup work for AmaGooAppBook.
At the same time, entrepreneurs are realizing that they don't actually need a good engineer to build v1 of their product, and so they're relying more on international contractors sourced through UpWork, Alibaba, etc. Most startups I know actually have entirely-international dev teams now, and hiring your first local engineer is considered a milestone like getting funded (and often comes after getting funded).
I suspect that the real losers in this are entrepreneurs - the startup field has become so competitive lately that there's likely to be a shake-out where the vast majority of startups fail, and many founders are paying for these outsourced dev teams out-of-pocket.
This is yet another example of what I call the "Hollywood-ization of tech", where the industry (we?) are trying to sort out the relative value of being attached to something big vs. money vs. relationships.
I think our industry is maturing. I see it all around me. The industry-wide career ladder is starting to solidify, we're getting more concerned about credentials/pedigree, salaries are going way up, people are getting more concerned about their "careers", etc. I don't know that it'll ever get to the point of, say, law or banking, but it's a hell of a lot closer than it was 15-20 years ago, when there was much more of a "homesteading" feel to everything, and the suggestion that software could be a prestigious career choice was the last thing on anyone's mind.
To be clear, I don't know if any of this is good or bad, but it's definitely happening.
I think you have to frame "losers" in a financial context.
Let's assume the hypothetical "entrepreneur" of your comment is an ambitious 25-30 year old male (they usually are), then the relevant question becomes, "am I better off being an entrepreneur or working for someone else". I think that's as much a question about the value of autonomy vs. money. I have little doubt most people, given that they could work at say, Goldman, would be better off financially doing that than trying to do a tech startup. So in that sense, they've definitely "lost".
Then there's the investor's perspective...a company that takes a $100k investment and turns it into $1M for the investor is a pretty good outcome. And I think there's a lot of potential for wins on that scale, but that's not how things are being done today. Instead, money is piling into OKish opportunities by the truckload, because every investor has the hubris to think their deal flow is as good as Greylock/a16z/USV/KPCB, etc. They'll win because they have their pick of what deal to do, and there certainly are great companies; there just aren't enough to go around to all the low-tier family offices and rich-doctor syndicates, and other "fools" who are piling into tech equity.
The point is, I think you have to frame win vs. loss in terms of opportunity cost. If the broad market declines 10% and you only lose 2%, that might be a "win" even though, objectively, it's a "loss".
Yes, absolutely, good developers probably shouldn't bother working for a typical consumer app or SaaS tool. If your skills are technical then you want to be working on something where the technical part is the hard part.
> Also, what of acqui-hires? I've seen and been part of some big corporate acquisitions in which a company buys a start-up that is not doing good work and has a stupid product, solely to get their 10-person engineering team and then drop the other employees. In which case that start-up succeeding mostly by not giving a shit about any actual business problem and just hiring a group of excellent engineers.
Depends how successful they were. Often such an outcome is a loser for the VC.
In my opinion, and having helped dozens of startups (friends or work), this is the biggest mistake I've seen. Engineers love to make things that scale, while business logic imply to leave it for when it is actually needed. I'd say it's #1 waste of time in a startup (in Spain). Also, when doing so mostly it's done without measuring anything, just doing it "for the sake of it".
Alas, in shockingly stark terms, as a VC he is quite definitely, necessarily, in very strong terms not looking for "most" startups.
Instead, he is looking for exceptional startups. How exceptional? A few or so each decade.
What is true for most startups is no doubt close to irrelevant for the exceptional startups he is looking for.
Sure, due diligence is not important for lemonade stands, but it was just crucial for, say, the Lockheed SR-71. And for any startup that is using new, advanced, original, powerful, valuable technology, proprietary intellectual property, barrier to entry, technological advantage, due diligence stands to be crucial for, say, estimating the exit value of the company. Sure, a lemonade stand may have traction growing rapidly right away, but it will never have much exit value.
Moreover, don't do due diligence on just the code or the architecture but also on the crucial core technology, e.g., some original applied math. We're talking theorems and proofs here, guys, maybe with some pure math grad school prerequisites missing among nearly all chaired full profs of computer science. Did I mention technological barrier to entry?
Uh, again, we are looking for the exceptional, and that may look different from the "most".
Does anyone have the data to do the math and see what the difference in ROI is between startups in general and startups which a) should have technical due diligence done and b) which have or would have passed tech due diligence? Maybe we could call these something like "tech startups" to distinguish them from the others. :)
It would be good to see how much delta exists between these two different pools of startups. This would be a fairly easy filter for VCs to apply.
In the meanwhile, pretend you are the Johns Hopkins University Applied Physics Lab (JHU/APL) working for the US Navy, and they have explained that for their nuclear armed, submarine launched intercontinental ballistic missiles they have one heck of a navigation problem. They tried inertial navigation but are not pleased with it.
So, some physics guys thought about some satellites sending out signals, the Doppler shift, etc. Some back of the envelope physics was some convincing due diligence. The system was built and deployed as planned, and a receiver on the roof of the lab routinely navigated itself within one foot.
No question. No doubt. Worked great the first time.
Disclosure: Have to track the satellites quite carefully and update the data on their orbits frequently, and the JHU/APL had a group doing that. At one time I was a programmer, on the fast Fourier transform with passive sonar data, in that group and heard the stories.
There are many more examples where some such applied math/physics, etc. worked great. The due diligence was carefully done, and the batting average for the projects was terrific.
Instead of empirical pattern matching, just do due diligence on such projects one at a time. That's what the US DoD has long done.
E.g., look at that truck with some long tubes on the back the US just deployed to South Korea. Each tube has a missile that can get up to altitude like right now and hit an incoming warhead. So, given several warheads and several missiles, how to assign missiles to the warheads? There's some nice applied math there. And don't have to wait for a pattern of nuclear wars to know that the math will work.
Maybe the lack of enough data for pattern matching is the bad news, but, then, the good news is that same lack of data -- we're talking greenfield here, guys!
To heck with the pattern matching: Instead pick a problem, one that with the first good or a much better solution enough customer/users will care enough about to yield spectacular revenue and earnings. Then, use applied math, physics, etc. to get the crucial core of such a good solution. Write the software. Go live. Get publicity. Please the target audience. Smile on the way to the bank. The core tech permits better or first good solutions to challenging problems, gives crucial technological advantage, barrier to entry, proprietary intellectual property, etc.
But, as for the JHU/APL project for the US Navy and many more US DoD projects, have to do the due diligence one project at a time. Sorry 'bout that.
So just be careful how you define 'iterate' and how much emphasis you put on 'quick'.
This is in stark contrast to sustaining technology, which tends to require expensive teams of scientists and engineers to make incremental improvements to an existing business line, making it palatable only to large organizations with deep pockets. Samsung can afford to have its physicists to find tighter SSD data packing. Google can afford statisticians to tune their search algorithm with complex models. Startups simply can't, nor should they!
I've always conjectured that this is why vetting the team is considered such a critical element. For instance, I have a Ph.d. and so I've found people just assume I know what I am doing in terms of tech and algorithms. This can be a disadvantage when I want honest feedback, actually.
Most VCs are much better equipped to discuss the business related aspects than anything technical. In fact, I would be hard pressed to evaluate most start-up technology outside my fields of study, because it becomes so specialized so quickly.
If you're buying a startup, chances are you're buying into the idea and the team much more than the current technical solution, and if you're right the margin for error on the cost side are huge, so you can afford taking the risk of having spend extra cleaning up a messy code base.
Apart from rare areas where solving a particularly hard problem is part of the value, the underlying tech is a distraction if the team is good and the idea is good.
But if you're buying a "boring" established business with a moderate growth rate for its market position and revenue, whether or not you'll end up spending 20% on top of the acquisition cost over the next year fixing crumbling infrastructure suddenly matters a lot more.
Intel Capital brought over their Wall of Faces (mostly technologists) for a "tech DD meeting". They asked almost no tech questions. All the questioning went to our business model, transition from custom proto to volume mfg, and building protectable IP. Our tech PowerPoint stack remained unrun, and I was very glad we'd stayed up all night doing 5-year financial pro formas.
I.C. invested. But I was surprised at how little they vetted our technology before signing the terms sheet.
I would suggest this isn't addressed in post-mortems because most of the time customers won't or can't tell you why they chose a competitor, sales doesn't know or doesn't communicate that to engineering, or engineering is too dysfunctional to act on the information and has a vested interest in justifying their own decisions.
Sometimes it isn't even tech debt per-se, just bad decisions that make implementation of critical features much more expensive... sometimes so expensive everyone feels it would take too long to implement so it isn't done.
Later on in a company's life, however, technical diligence becomes very important. If anything, VCs should do a better job of it. Countless companies have struggled, stalled or failed because they didn't become aware of or address technical issues early enough and scaled the business side too far out in front of what could be managed technically. This gets your company upside down fast, and in very difficult to unwind ways. Customers become dissatisfied at instability and lack of rapid improvements. Internally, strife between revenue owners and technical owners can easily increase as tech tries to balance delivering new funcitonality and addressing old technical debt.
After you find product-market fit and start really scaling-- ie after the early stage-- technical diligence should be a much higher priority than it currently is, for both VCs/investors and for company operators within their own orgs. Better technical assessments of issues and opportunities would serve most startups very well.
i/ An startup wants to enter as an automatic security escrow in a Bitcoin multi-signature wallet (please forget for a moment this is about a sector with too much hype). If you do a basic analysis you will realize that multisig wallets are well supported but there is a missing link: there is no standard out-of-band way to setup these wallets and you should do many manual steps. These manual steps add a lot of friction for user onboarding and undermine the startup success.
ii/ A security startup wants to integrate their app inside Microsoft Outlook via an add-on. They are confident because they look at the API and it seems you can extend Outlook but they discover very late that a specific feature can't be achieved.
iii/ An startup wants to virtualize Windows applications (e.g. VMware ThinApp) using binary instrumentation and/or filtering drivers. They will find that some few but critical applications (e.g. legacy Java applets) require an ad-hoc solution for specific issues that significantly increases the development costs.
This is one more example of how what's good for the VC (with a large portfolio) is not necessarily good for the startup (with a portfolio of one)
1. Early version of a product especially when in hurry with features delivery are going to be "temporarily definitive" implementation. As soon a feature reached the production the code has to be considered "legacy code"
2. Technical debt is a choice, and when you have too much of it you enter in a spiral of death.
3. The quote from Donald Knuth is only a poor excuse to slack and to justify bad development habits. Writing efficient code most of the time doesn't require more time than not (choose the right container or the right algorithm for example).
4. It's a so generic statement that adds no info at all.
"For >95% of startups [...] That means technical resources are rarely the cause of success or the reason for failure."
that's the best statement of the whole post that more or less sounds like this:
"In Africa 90.3% of people die due to: nutritional causes, hearth disease, cancer and diabets, why send over there good doctors then?"
I would argue that the former is always important.
I agree that the latter is less important, but with some caveats. The number of engineering hours put in during the first year is going to be a tiny fraction of the second year, etc, so tech evaluations will change drastically over time. The author argues that this makes them a slight waste of time.
However, I've been very successful in using rough technical diligence in the early rounds as a proxy for measuring the decision making abilities of founders. This is hard to measure, so I still find technical diligence useful. You have to know the constraints they are working under, and not hold technical debt/differences of opinion (like homebrew vs stock) against them, the purpose is different.
Just my 2 cents.
Google won early by having the best algorithm. Facebook's early competitor Friendster died due to technical issues. It seems less clear at what stage companies like Uber and AirBNB faced substantial technical risk? I'd guess Slack and DropBOx had significant technical risk in early stages. I suppose WeWork probably had less technical risk.
I suppose the early prototype's code may not be totally indicative of whether the company can overcome the technical risk. Still, I think you need to evaluate whether the founding team can deal with the technical risk.
Technical due diligence isn't about looking for beautiful. It is about looking for an ability to manage technical risk. This includes an ability to hire and manage good engineers.
I would add though that the impact the technical founder has on the design of the product and the quality of execution is immense and often hidden behind the non-technical founder selling the company to VCs. Good technical founders do a lot more than code. If I were a VC, I would look for teams with technical founders that have skills extending into product management, people management/recruiting, and operations.
The case for not acquiring technical debt is when a business fails because it can't pay down the debt or fails because it is bitten by the debt in some way.
What would this look like in practice?
For a young startup to be successful, it needs to offer evidence to investors that additional funding will lead to exponential revenue or market share growth. Those are the hard things.
Building code without taking on technical debt is actually easier than building it with just the right amount of technical debt for the business's uncertain future.
In my opinion, the kind to avoid is sloppy, non-strategic technical debt, such as what you often get when developers are burnt out and need to complete aggressive milestones so that management will be satisfied with the rate of development. This debt tends to break modularity and lead to bad system design, whereas stategic debt can simply mean building a quick and dirty black box for desired functionality that has clearly defined interfaces that can likely remain once the debt is removed and will not necessitate a rewrite of other modules.
Similarly, management debt can be strategic if it teaches young or inexperienced managers useful lessons that they grow from and apply as the company grows. But management debt that results in burnout and high turnover also gets rid of the valuable lessons and lets someone with slightly more experience make some of them again.
Just hire at least one professional for the startup. He/she will simply not be able to do complete garbage. That's meaning of "professionalism" at the end, right?
Good Experienced Professional Engineer also understands real life compromises. For him/her it is not that hard to make initial architectural decision that allows to implement something fast but has a foundation for future expansion. Usually it takes about couple of days for the person that "been there, seen that" to make good architectural decision.
Startups: "We are not so rich to buy cheap things". Well, at least CTO / architect.
If an early stage team is building a massively complex microservices based product, using a new type of database and deploying it via kubernetes — where a Rails CRUD app on Heroku would easily satisfy requirements for years. Then someone seriously needs to ask the question as to whether using bleeding edge tech is necessary, and whether it will impact the ability to iterate and hire.
Common questions are what would happen if the data storage failed/datacenter went dark and do you have an up do date list of all libraries used, yes including copypasted stuff out of stack overflow.
Due diligence != code audit. Sure if you get in that level detail for the td then it's a waste of time, but otherwise knowing that the tech team is not sitting on a landmine it's kinda important
But as far as microservices, its totally possible to do that in a way that doesn't add a bunch of complexity.
As far as new databases versus relational, a new database can make it easier to iterate (no/easier schema migration) and hire (can't prove this but I believe there are many people like me who got sick of relational dbs after many years and are just happy to deal with different types of problems.)
Something like kubernetes if done correctly can really help smooth things over because many apps have specific external dependencies which without a tool like that are hard to automate 100% and so create drag for new hires especially.
Startup technical diligence needs to check for a minimum level of ability and beyond that, ability to find the right direction to build in.
The idea that what product you build is not the responsibility of the technical team is almost certainly why you think there's no point in doing technical diligence.
I agree, the first software produced is just a proof of concept and will be re-written. I even believe the only way to write good code is to build it fast in some intransitive language and then throw it away and do it over in something compiled.
So I agree with this post except, to become great, you need to know upfront you are just proving the idea and you will do Technical Diligence of the final product and NOT SELL CRAP.
From my experience working in startups, this is never the case. Crap code persist because rewriting it would take time, they would rather spend on new features. Feature creep sets in very fast.
He is using the quote wrong. A scalable product and an MVP are build entirely differently. Both scalable product and MVP can have technical debt, but I wouldn't say the scalable product is the optimized version of the MVP.
The devil/fun is always in the details. Deprive me of that, I will never work for you.
High-tech startups still exist but they are not the norm.
I'd be curious to see how he arrived at that number. In my experience poor technical execution manifests itself as other problems, like slow time to market, poor user experience, or even building the wrong product.
> Early versions of a product are often prototypes that are intentionally meant to be rewritten or heavily refactored in the near future.
Cool, but surely a check of basic physics is in order for certain ideas? (see: uBeam)
> Because getting a product in the hands of users is a top priority, even great engineers will intentionally take shortcuts and accumulate technical debt in order to launch sooner.
Agreed for the most part. I see technical debt as incurring a price to get to market faster. Sometimes the price is too high, though I'd wager 9 times out of 10 it's the product-market fit that would kill a company faster.
> The ability to scale with success is important, but designing products for high scalability from Day 1 is usually a mistake. ("Premature optimization is the root of all evil." -- Donald Knuth)
Sure, but I don't think this is technical diligence necessarily. To me, the diligence would be researching whether or not the product/service could scale, either by seeing other examples of similar companies, understanding how AWS works, understanding manufacturing processes, etc.
> "CTO" is an ambiguous title at small companies. > Because code is likely to be temporary and the CTO's role is likely to change quickly, it's not obvious what should be measured during technical due diligence.
Ok... but can one not measure or research the person (and team's) ability to create technology or manage those who can?
> (An aside: in addition to tech diligence having questionable value, many seed stage founders consider it overbearing and passively resist it. Given that engineering resources are scarce and that there are plenty of investors who write $100k+ checks after one or two short meetings, founders often prioritize "lighter-diligence investors" instead of submitting to an hour or two of technical grilling.)
Jesus Christ I need to move to California already.
> in today's world of SaaS tools, APIs, and cloud infrastructure, most startup ideas don't have significant technical risk. That means technical resources are rarely the cause of success or the reason for failure.
That's actually a very fair point. In the specific domain of web product companies that are leveraging the heavy lifting of others, a few Business Bros can get very far.
But does this cover ">95% of startups" ?
> A quick skim of CB Insights' collection of 150+ startup post-mortems reveals that only ~5% of post-mortems referenced a lack of technical ability/execution.
Yes but did none of these startups have any technical due diligence? Maybe they did, and that's why the rate is so low.
> But for a typical consumer app or SaaS tool, technical risk is low enough to be ignored.
Sure, just double-check that the engineers on the team did more than a coding bootcamp.
> On the recruiting side, a technical founder needs to be someone who can do recruiting and someone whom other engineers want to work for. This is especially true in today's hiring market, which is extremely competitive. The best ideas won't go far if a company can't attract enough talent to launch a product.
This sounds like a great reason to vet the technical prowess of the team.
> Notably, these tests don't require having a technical background.
Oh come on now. A GitHub filled with shit code and poor technical decisions on past projects would completely escape you folk.
> Instead, they require the ability to read a resume, common sense, EQ, and a few reference checks.
Reference checks maybe. Everything else you can get duped. Even reference checks.
> If you're a seed investor and you're doing tech diligence on a company, you're most likely wasting your time and the founders' time.
Perhaps for your narrowly defined web product startups, which apparently are 95% of startups. But when someone starts talking to you about drones delivering packages, all of a sudden it might be nice to be able to draw some free body diagrams and verify the feasibility of a payload claim.
> A well-built product that doesn't solve a problem will always be inferior to an ugly, buggy product that addresses a burning need.
Sure, but I don't think this logically leads to the conclusion "technical diligence is a waste of time".
Overall a pretty meh post. Focuses on a narrow subset of startups for which, it's mostly true, the tech is a commodity. I don't even consider web products to be "technology" anymore quite frankly.. Not until you need to scale some sort of real-time computing with millions of users and build your own datacenters.