Hacker News Reader

Comment by hoodoof | Hacker News Reader

"Hard tech startups

Some people think YC only funds software startups."

Kinda strange that software is not seen by YC as "hard tech".

I think there's lots of hard tech in software but the way venture capitalists/angel investors tell it, you should only ever build software that can be made in a weekend and then iterated upon.

The outcome of this is the X thousandth dog walking social network for uber drivers staying at AirBNB - i.e. stuff that you can build in a weekend and iterate on. And this is why demo days are full of such lightweight software applications - people are actively discouraged from taking the time to build something large scale and, well, "hard".

Sometimes it makes sense to make a big bet on building some complex software that might take months or years to build, with lots of moving parts, but once complete solves a big problem. That is software that is "hard tech".

View on HN

There are plenty of hard tech software startups. But I see how unclear this is; fixing now. Thanks.

I agree wholeheartedly. I think that treating software as dichotomous with "hard tech" (instead of orthogonal) is a profound mistake that really proliferates, not just through VCs, not just through the industry, but through our entire culture. So, so many (especially older) hardware guys I know don't see software as a real field, and I think this is a big part of the reason why.

And much of the discussion about technical debt, about "the web toolchain sucks", or "operating systems suck", or X, Y, Z, they all suck... so often, I think that's just a different way of saying "we have this really hard problem, and we've spent decades piling really innovative partial solutions to it on top of other really innovative partial solutions, but nobody has come along with something that really, really works." And it's a tremendous problem.

The response I've seen in the past is, "We love those ideas, they're just more appropriate for, say, YC research." But for hardware, the bar is automatically different: it's understood that there must be some degree of longer-term research and development that goes into a product. And I think that's completely untrue: both require experimentation, both require feedback, both require upfront work to get right. And to be honest, this misunderstanding probably stands at the core of the strongest arguments I've ever heard against the maturity of the software industry.

As someone in the hard tech space I can tell you that it's not completely irrational on their part. When you look at exits and probabilities of exits, the types of software companies you describe make up the bulk.

The biggest (by val) technology startups that we know of (Uber, Lyft, AirBnB, Snapchat, Dropbox, Pinterest) are by no means hard tech. Even the public ones aren't hard tech (Facebook, Linkedin, Twitter). So that's what these investors look to as the types of investments that make sense for them.

The question is, do hard tech companies have similar probabilities of outcomes or bigger?

I'd say Google was hard tech

Google was working, using Stanford resources, before it was funded. Low investor technical risk. Also, Google's founders had patent coverage of their idea.

DropBox was hard-tech when it came out; they reverse engineered the Finder so that your DropBox folder would appear like a normal Mac folder. They may seem like a commodity now because everybody's doing stuff like that (and in many cases, APIs have appeared that make it trivial), but when they first launched, they did stuff most people assumed was impossible. Twitch/Justin.TV was also hard-tech when it came out, though now the hard parts are built into every iPhone.

Other than that, I largely agree. The last software hard-tech company to strike it really big was Google. There've been a number of open-source projects doing what I'd consider hard-tech software, though - Bitcoin, git/Mercurial/Darcs, Bittorrent, TensorFlow, etc.

(My definition of hard tech, as applied to software, is "Software where you need to use scientific-method trial-and-error to build core pieces of the product." If you can read an online tutorial or reference manual and build the product, it's not hard tech. If you need to poke around at things, observe the responses, and build your own model of how things work, it is.)

My definition of hard tech, as applied to software, is "Software where you need to use scientific-method trial-and-error to build core pieces of the product."

I think that's a great definition actually and fits with how ML systems are built.

"DropBox was hard-tech when it came out; they reverse engineered the Finder so that your DropBox folder would appear like a normal Mac folder. "

This, IMHO, is a really bad example.

This is pretty much just a few days of sitting in GDB for the right engineer[1]. Now, maybe it require people experienced with debugging tools, but it's really not "hard tech". Now, productionizing it so it works on all versions, yeah, a bit trickier. but again, none of this is at the level of basically "understanding how to make custom bacteria that do a thing", etc. If this is the example you mean for "did stuff people assumed was impossible", then i strongly disagree.

""Software where you need to use scientific-method trial-and-error to build core pieces of the product.""

This, IMHO, is way too low a bar. By this definition, the clang compiler we built for windows is "hard-tech". While it requires time and energy and trial and error, that is not hard, in the same way the dropbox stuff is not hard.

It is known that it is possible, and requires the reasonable application of good engineering skill. That engineering skill may often involve the scientific method trial-and-error, but you know you will eventually get there.

The same is true of dropbox, and in particular, your finder example. The only thing unknown is the timeline, and even that you can take a reasonable stab at if you have good enough engineers.

[1] I did it before they did, and i wasn't even the first. Plenty of people have made this happen :)

I like the xkcd definition "I'll need a research team and five years" with the implication made explicit -- "and we might not figure it out."

Yeah, this is pretty much what i'd say qualifies as hard software tech

Unlikely Google would've qualified then, but I would certainly have put Google into the hard software tech category.

If we're talking about Google before Brin's PhD thesis, I think it would have qualified. It was not at all clear back then that using backlink data would yield more useful results than mere textual analysis of page content. One can definitely imagine a scenario where you try to build a search engine based on going down the rabbit hole of natural language processing as the key feature and then end up with something that doesn't work all that great.

Perhaps I'm just not as conversant in reverse-engineering as some of the people here, but my understanding is that if a key part of your product relies on patching somebody else's software for which you don't have the source code, this is also fraught with potential dead-ends and uncontrollable risks. What if they're using ASLR? What if they change the functions involved in the next version? What if the functions you're trying to patch have side effects that you can't afford to ignore?

That's why I prefer to put the dividing line at "must figure out things by poking at them rather than by reading documentation". The definition of "research" can be pretty vague - is a security team poking at a product conducting research? How about a UX team trying to figure out how their users behave? A search team doing language modeling? All of these would count in my head, and if a startup built their product around one of these results I would consider it "hard tech", but evidently not everyone agrees.

I think Research = Observe-Theorize-Experiment works. Reverse engineering is not research because someone already has the answer.

> Reverse engineering is not research because someone already has the answer.

Someone potentially had the answer at one point. That person/organization may be dead/defunct, or the knowledge may otherwise be lost to time.

Yes, but an answer exists.

...an answer always exists, even in pure scientific research. Nothing is inexplicable.

Why not? The research just happened in grad school [1] before the startup.

[1]: http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.109.4...

"Research team and 5 years" != "Two [1] grad students and 2.5 years".

Dropbox, BTW, was 4-5 engineers and 1.5 years.

[1] Actually more like 4-5, Larry & Sergey had help.

Google's other big innovation was building with commodity machines instead of high-end servers. That definitely puts them in the hard software tech category.

I'm personally sympathetic to "I'll need a research team and five years" missions. I think both university research and xkcd are great! But it's a critically different from YC's definition that:

Hard tech = "There is doubt that the technology can be built at all."

Though many pieces of technology face huge doubts, what's key is often a team can get to a working prototype or partial release in way less than 5 years! (E.g. Dropbox.) YC is exactly the type of environment to refocus a 'research style' team exclusively on demonstrable progress.

The problem with the xkcd definition (if attempted in a startup) is very few research teams can continue to fundraise for 5 years without a product or significant prototype.

A partial solution to the "doubted" tech is often good enough to build a great company.

>This is pretty much just a few days of sitting in GDB for the right engineer[1]. Now, maybe it require people experienced with debugging tools, but...

As someone sort-of familiar with gdb (but not extensively so) I have no idea how I'd do that. Can you point me in the right direction?

I don't use a Mac, but assuming Finder will detect that if its view is of a folder and items are added to the folder elsewhere, Finder's view will be updated to reflect this:

Use dtrace and create a lot of such events. They're presumably using kqueue or some event mechanism to be notified when the file arrives. Do this with many file types if they look different in Finder. Somewhere in there should also be a read that corresponds to the dirent. You can break on these things.

attach the debugger and create the events. Step through the code to find when these things are read. Attempt to discern how what is read differs between file types. Do stuff like make files with conspicuous attributes (e.g., file size), because it's easier to correlate from traces. The data is probably a file containing file metadata somewhere.

This is probably mostly looking at the bytes coming off the read. dtrace makes this easy because you can trigger it to set a flag when the kqueue event fires and then just dump bytes and locations from file reads/opens. If it's more integrated into the OS Finder would have to have its own special syscalls to read stuff off inodes or whatever. You'd be able to see those happening too.

Once you think you know how it works, give it a try. Rinse and repeat.

Now it may be you have surprises here and there and it's kind of annoying, but I'd be surprised if I couldn't do it.

How about things like

* jvm -- it wasn't clear you could build a vm that was fast enough to compete with c++;

* c2 compiler inside the jvm -- also not clear you could do this, fast enough, or one that optimized enough to care about

* azul systems / zing -- could you build a gc that tolerates terabit allocation rates w/o stop-the-world pauses?

Sorry, no, JVM was not even innovative, let alone "hard tech". There were lots and lots of previous examples of doing the same (and better) than JVM, see for example the Smalltalk and Lisp world.

IMO, the only thing on your list that comes close is the Azul GC which, in my limited understanding, actually advanced the state of the art.

Microsoft (operating system, SQL database, programming languages / environments) and Oracle (databases, partial credit for programming language (Java)) could be considered hard-tech software companies with outcomes (so far) that are many multiples of those you mentioned.

Those ones (along with Google and Microsoft that are arguable hard tech) were founded prior to web 2.0. Since about 2002 no hard tech software startups achieved any scale- they are sort of all "moving up the stack". When they emerged (Android Inc) they are promptly swallowed. In the mid-2000s there were Yelp (from the software perspective, a simple website with Lucene backend and Youtube - a simple website with Flash player and ffmpeg transcoding command). It's probably a sign of consolidation of the industry.

Using Sama's definition of a hard-tech product or business as "where there is doubt that the technology can be built at all", a bunch of successes come to mind.

1. Tesla (Market Cap ~$29B w/$14B+ in pre-orders) 2. SpaceX (Valued north of $10B) Elon's companies faced massive doubts about tech feasibility, both when started and at basically every-point since.

3. Oculus. Sold for $2B in March 2014, including $1.6B in FB stock that's now worth 2x more.

4. Machine Zone The messaging platform for Game of War/Mobile Strike is very "hard tech". (Google/AWS still lack equivalents and MZ was kinda forced to take a "time out" to develop it.) In 2014 the WSJ said they where fundraising at close to $3B and now their cash flow powers global TV add campaigns.

5. Cruise. "Hard Tech" and a big win for YC.

6. Zoox. They started w/a prototype and have made huge progress.

Many new business lines at public companies faced huge doubts about tech feasibility and are now clear home runs:

7. Nvidia Every new chip is "hard tech" and they've grown massively in the last decade.

8. iPhone/Apple + Android Inc. Smartphones are really, really, "hard tech". So many doubts on both the tech and product side are now forgotten!

9. Google (mentioned by others) In addition to search/adds/Android, I'd say maps/gmail/infrastructure/android are all examples of "hard tech". Each was viewed as "hard tech" when released. (Contrast Google Maps w/Apple's disastrous maps launch years latter.)

10. AWS If you had pitched AWS as a startup, in late 2003, you would have faced lots and lots of technology skepticism. A phenomenal success.

11. Amazon Echo Hard tech, took a massive team to develop. Doing great as a product.

Crossing "the valley of death" in commercializing new tech is really hard. But there are vast success!

Edit: I'm awful at this reading thing. Please see the replies to this comment for a correct definition of "hard tech". My original comment follows.

I think "hard tech" in this instance means "technology in the form of hardware" or "technology that is physically manifest" and not "technology that is difficult."

I think the latter is the intended meaning actually. AI/machine learning seem to be thrown into the "hard tech" bucket in VC jargon.

There's a definition from Sam in the footnote:

> [1] I use 'hard tech' to mean a startup is whether there is doubt that the technology can be built at all.

Edit: I'm glad he defined it for this post. In general though, there's definitely ambiguity around the term.

No. From the footnote: " I use ‘hard tech’ to mean a startup where there is doubt that the technology can be built at all."

From the footnote in the original post:

[1] I use ‘hard tech’ to mean a startup where there is doubt that the technology can be built at all.

This conflicts with the way he uses the term "hard tech" to differentiate from software startups, as if the two are mutually exclusive:

It’s relatively easy for a software startup to take short cycle times and low-costs to an extreme, but hard tech founders are often surprised by how effectively they can do this, too.

What I think he is really referring to as "hard tech" is the move from the consumer electronics and office automation markets to the grittier, less-visible industrial verticals: Automotive, industrial, energy & utilities, aerospace & defense, etc. Verticals which have generally been dominated by a GE/Siemens/Boeing/ABB/Schneider etc. due to the high R&D costs needed to realize a certified, generally safety-critical solution in a highly bureaucratic environment.

Dropbox is down for a few hours? We'll have a few pissed customers. The ADAS system in a customer's car failed momentarily? They'll be lucky to survive and massive lawsuits/loss of business could ensue.

As an aside he should really define key terms up front to avoid this sort of confusion and to allow us to focus on his main point: YC is trying to move into and disrupt this lucrative but hard-to-broach space.

Exactly. I think that itself actually an interesting "hard tech" problem: how do you move fast, iterate, and be first-to-market without killing anyone?

I don't think you do. You want to be number 2, and swoop in when the time is right.

You do not move fast and iterate if you care at all about the cost of failures. Period.

I did say it's a hard tech problem... :)

(But seriously, at least in theory, there are sort of obvious things you can do to isolate anything failure-critical behind statically or dynamically checked contracts, which then allows rapid iteration outside of the small set of things that are truly failure-critical. But you need those contracts trustworthy enough that you can iterate without worry, flexible enough that you can iterate on economically important sub-components, and applicable to a system or set of systems where this iteration gives you the trump card in a billion dollar(s) industry. As I said, it's "hard tech" -- which almost by definition means some people will think it's impossible.)

The first line of the post explicitly makes a distinction between "software" and "hard tech".

I really challenge the accepted wisdom of "build something tiny and iterate" as being the only way to build software.

Games can be really complex 'hard-tech', but compare Elite Dangerous's small+ iterative to Star Citizen's big+all-at-once approaches.

Even with things like nuclear energy and particle colliders I'd guess those who succeeded probably started small and iterated.

Of course there are many approaches to build software [1]. I also read that comment as more of a philosophy for product development than software development.

[1]: https://en.wikipedia.org/wiki/Software_development_process#A...

The quote you put isn't what is said on the blog post. I assume they edited it after your comment, Thanks! Adding that straightforward adjective removes the ambiguity.

> I use ‘hard tech’ to mean a startup where there is doubt that the technology can be built at all.

> Some people think YC only funds straightforward software startups.

Yes the original post has been edited.

I too found it a bit strange that the list didn't include software, or at least some specific types of software. I can assure you though that YC does invest in this and doesn't exclusively look for software that can be built in a weekend. My startup, Pachyderm [0], is a pretty good example of this that they funded in W15. A ton of other hard tech software startups have come out of YC including: Docker, CoreOS, Memsql and RethinkDB. In general I think it's one of their more successful verticals, although probably not quite as successful as consumer software.

In general I think you are right that VCs have the bias you're talking about though. Not all VCs but many. It's not totally irrational, software that can be built in a weekend has fewer risks associated with it and can have just as much upside. However there are many very problems that are worth solving and can only be solved with complicated, hard tech, software. Investors with this bias cut themselves off from this opportunity.

[0] https://www.pachyderm.io

My idea for a startup weekend was such a hard-tech/software problem that lost out to a couple of social weekend apps. While a lot of people got behind my idea and we put together a strong presentation, the lack of a finished app sank us. How do you compete with something launched on mobile in a weekend when your at power points and mocks?

The judging criteria for Startup Weekend is 3 parts: Business Model, Customer Validation, Execution: http://www.techstars.com/content/community/startup-weekend-j...

If your mocks were sufficient for gathering user feedback, that may be okay. I've certainly seen groups with no written software place in the top 3, off of designer mocks alone.

Another point on judging criteria is that it varies widely based on who your judges are too. Those categories are pretty broad with plenty of room for interpretation intentionally.

If the judging panel is less technical, you'll have to sell your idea differently. I've seen cool tech hackathon projects fail because of bad presentations, either because they're ineffective or they just didn't practice. It's kind of less than ideal that you're judged for the whole weekend based on how well you can sell in a 2-minute pitch, but then again it's not all that different from the real startup world.

Edit: I guess with the premise being "startup [in a] weekend", I shouldn't have expected my idea to win even with the team I picked up. Perhaps a startup month would work?

I've been an organizer of several hackathons and participated in Startup Weekend. The judging guidelines vary across organizers, theme, and city. In general, Startup Weekend events focus on business model generation and customer validation, with building an actual product following.

And remember, it's a weekend: the "winners" typically don't turn the project into an actual startup.

I think of like in school, you can learn because you want to know things or you can learn to the test because you want the mark to move on.

Would you rather build something hard or build something more likely to win? From my experience, it's not the same apps that do both. I also don't think one approach is necessarily better than the other, just personal preference, plus also depends on the team dynamic you end up with. For example, I wouldn't try to solve a hard tech problem on a team with mostly non-tech people because their expectations can be unrealistic. But if my team is 3 experienced startup engineers that can iterate, well why the hell not?

It's always better to develop most of the software with real users using it. So although the goal may be huge, start by building a piece that does something in a month and get real users using it, so the rest of the project can proceed with a compass.

"It's always better" - words like this make me think "unchallenged accepted wisdom".

Accepted wisdom remains accepted until eventually people start to realize that it's not entirely correct.

As someone who has gone through YC, I think a lot of the kind of absolute talk you are pointing out has to do with consistent messaging and providing general advice to an entire group of people who might not have the experience to understand for themselves what the best way to proceed is.

Obviously in reality, things are a lot more nuanced than 'always', but when your goal is to help as many companies at once, where the degree of experience varies, it often help to simplify the problem. In one-on-one situations, the partners can give you more personalized insights and in general are quite insightful.

There is considerable experience with doing large software projects in one shot. Many corporate and government software projects are done, or at least used to be done, that way. Results were pretty bad.

It's hard to be positive of causation, but correlation (long time to initial customer contact <=> software doesn't make customers happy) is pretty strong.

If you want to challenge it, you could try to deliver a successful large multi-year software projects with no prior customer engagement, and document the process. People can then make informed comparisons against the accepted wisdom.

Sure, one counterpoint, or at least nuance, is when you give too much focus to the wrong users.

Then wrong users can oversteer your product. For example, BigCo wants Feature X and will pay $100k/year for it, you add it but it complicates the experience for others and delays the rest of your roadmap.

Another is knowing when your users are wrong, they request Feature Y, but really what they want is Benefit Z.

Of course there's also nuance around how exactly you define minimum viable product (i.e., the part built before you start exposing it to users). I've seen experienced non-startup business people describe an entire app with every feature that they envision, then claim that's the MVP. It can be a hard model to adjust to.

Software have a poor image due to lack of perfectionists in the software industry. Buggy software are happily released. Correct software are very hard to build. I wonder how many have ever existed.

buggy software makes money

perfectionists don't

This should be a plaque somewhere.

So that it could be pointed to people who start wondering why almost everything in software industry is utter garbage built upon a pile of shit.

Yay, business interests.

One of the consequences of that being tons of cheap disposable software jobs everywhere, crappy recruitment process, good engineers literally drowning in the sea of the above.

No kidding. Just ask the Japanese - They were known for going through their software with a fine-tooth comb before releasing, but they started to realize how far behind the Americans they were (and arguably still are).

Buggy (and bloated by extension) software never has my money at least. I never trust, for example, my battery life and my mobile traffic to software houses that are not exactly perfectionist.

Perfectionism should become a skill, not just a characteristic.

I don't think endemic poor software quality is about "lack of perfectionists." It's mostly about perfectly rational cost/benefit analysis. A lot of software is valuable for reasons other than code quality (e.g. network effects). In this case it makes complete sense that businesses would rather be first to market than produce something of the highest technical quality.

There are other issues at play, of course. In cases where quality and business profitability are strongly aligned - think avionics - then there is plenty of high quality software.

> It's mostly about perfectly rational cost/benefit analysis.

I disagree.

Looks at any exploit database. A large percentage of the exploits on any typical day are the sorts of mistakes that a well-educated developer wouldn't make, or that best practices and modern tooling would prevent.

That's not even a cost/benefit problem, because in many cases there's no additional cost to doing things the right way. It's a culture and education problem[1].

[1] by which I do not mean traditional university education, fwiw.

I'm not so sure we do disagree :)

At the risk of splitting hairs... there are costs associated with finding developers who have this education and/or fostering a company culture where security best practices are important.

If software companies were seriously punished for compromising PII, then I imagine we'd see a change.

Maybe engineers are making a rational cost/benefit analysis too. In that perfectionism doesn't pay more or lead to promotion. Actually it can be harmful to call things out in a corporate environment.

> Kinda strange that software is not seen by YC as "hard tech".

Could the reason be that pure software patents have a weaker status than hardware patents and/or are more difficult to enforce?

I read it as tech that is hard to build because of barriers to entry and an assumption that there isn't a good way to iterate rapidly.

If and only if, a market for such a software still exists and continues to grow when you deliver the product.

I have a hunch it is because many people, myself included, see a lot of software development getting replaced by generalized AI in the next 10 years.

Building these AI systems is itself a form of hard software development.

(I disagree on the premise, BTW. I saw a number of successful machine-learning systems at Google, and the most successful were always the ones that combined machine-learned functions - sometimes many of them - with traditional algorithms. AI is a tool that extends the reach of software into new problem domains, it's not going to replace software itself.)

What makes you think that we'll see generalized AI this soon? Is there some kind of progress beyond machine learning that I missed?

Hard = hardware, physical. Not "difficult"

FTA: "I use ‘hard tech’ to mean a startup where there is doubt that the technology can be built at all."

Reply on news.ycombinator.com