Premature optimization | Hacker News Reader

Premature optimization

blog.samaltman.com

60 pointsjordn10 comments

This is proper advice and if you're in a position to adhere to it, do.

However, most of the time, you are more likely in a position where you try to convince somebody (who contracts you to do something for him) to adhere to it. My pet peeve is signup forms[0]. Trying to convince people that their product is the thing to obsess about seems straight forward. But that's also their most sensitive spot, psychologically. So most likely, they're projecting all of their efforts around precisely that part and any attempt to change that is met with incredible resistance.

I have yet to see a case where the words "we're loosing business, because" is followed by "people don't need or want what we offer". In fact, if I were to check, I'm pretty sure that in every single instance of "loosing business, because" a quick glance at the site in question would result in thinking "well, because who would want that?".

[0] https://news.ycombinator.com/item?id=3809186

From that comment: "If your website gives out free gourmet food and massages, you can literally require people to fill out ten page forms and ask for confirmation via standard mail - your signup form will still convert users like crazy. At the opposite end of that scale, there are forms like the recent April Fools joke 'Google Nigeria' - the simplest possible form: Enter your Credit Card details and be done."

Definitely agree with this. Currently, I am running a side project with a friend (which we hope to turn into a company). On our roadmap, we have Product/Market Fit and after that, Product Optimization.

We have released mobile apps for iOS and Android but they are 100% mobile web views. We want to be able to prototype quickly and figure using web views is the best way to do this (no approval waits, cross platform). Registration conversion rates are bad and the UX is what you get with a non-optimized mobile web view.

Despite that, the data we are gathering from initial testing is valuable. We've been able to gather that users find the base product interesting but our engagement at the end of the funnel is not as high as we like.

For us, we consider moving from web to native would be the highest form of optimization but we want to get the product right first.

That said, there are some problem domains where better optimization is the only way to differentiate. Eg, I'm a short span away from releasing version 0 of Wellposed's numerical linear algebra product/library, and this is a product where if your code isn't at least as fast and usable as the alternatives, no one cares.

Thankfully My code seems to be >10x faster on the problems that matter, and trending to be substantially more extensible and usable than alternatives.

If "optimization" of some sort isn't fundamental to your business, you don't have a real business or value prop for someone.

OP is talking about the completeness-type of optimization, whereas you're talking about the effectiveness-type of optimization; OP touches upon this:

> "This is great, but only if the business [in your case, library] is already working."

in my case... theres no business without optimization :)

but I suppose you're right and I'm being a tad literal! Comes with having optimization on the brain all day1

I Only semi-agree.

To truly understand if your product has hit the mark, you need to have the metrics/analytics to be able to assess those factors, look for tipping points, know what to remove etc. Sure, don't do premature micro optimization.. but massive optimization to get product market fit is still a form of optimization that needs to be done, for example, A/B testing different landing pages to discover what people are actually looking for, each page with different subsets of 'features'.

It takes a lot of data to get good A/B test results and you can't run all these tests at the same time and in parallel. A lot of newer startups simply don't have the traffic or data available to optimize like crazy. I think the key is just to focus on your core metrics, like engagement for a consumer product or whatever your metric is for a B2B company (contracts, revenue, etc.).

I've worked with many startups to help them build a culture of data-driven decision making and I'll say from my experience that you don't need massive data to run highly valuable A/B tests. Also, with proper segmentation techniques, you can run many tests in parallel.

Fair enough. My experience is more on A/B testing high trafficked paid lead gen pages, and I understand you can still make conclusions with less data (doesn't need to be massive). But I do agree with the author of the original post that it can be easy to worry about all these different A/B tests that aren't that illuminating in the grand scheme of things when you haven't even got a solid product.

Yep, we agree there, the advice of the author is spot on. I think too many startups, and large companies for that matter, think optimization is going to fix all the problems that are inherent in their product/business by just blindly slaping optimization vendor A's tags on the site and money just falls from the sky. Just doesn't happen like that.

Reply on news.ycombinator.com