Hacker News Reader

Comment by kilroy123 | Hacker News Reader

Great work. When I loaded the page I even thought to myself, oh shit this is fast.

One huge problem, with most large platforms for e-commerce stores, they are terrible at speed optimization!

For example, I have a store on Shopify and it's basically impossible to do most of what you explained. With Shopify, you can customize HTML / SCSS files with dynamic content. All of which gets compiled on their back-end. You have no access to the final compiled HTML and markup. You're sadly limited in what you can do.

It seems you need to have a custom store to really be able to optimize this hard-core.

View on HN

This is largely incorrect. All the front end customizations that this article describes are possible with Shopify because you have complete control over the HTML, JS, and CSS that the platform spits out. While true that you can't "access" the final compiled HTML to edit it, the templates that power your theme are very close to that output, and not hard to change if you're familiar with HTML/CSS. The Javascript optimizations are all completely possible as well but you either have to pick a theme that has been optimized or implement them yourself in your own theme. Shopify also supports AMP and responsive themes (because it's just HTML).

It is true that with Shopify or similar platforms you aren't able to make backend choices or implement caching like the article has described, and yes it's true that all the themes in the theme store aren't perfectly optimized for page load time -- but the question really is do you want to do any of this stuff yourself? Shopify has invested years in their platform to make your website as fast as possible for as little money as possible. The themes that Shopify publishes in the theme store undergo huge amounts of performance testing to ensure that you aren't losing money from latency. The platform comprises a world class CDN, and Shopify doesn't charge for bytes through it. The platform uses many layers of backend caching at the data access, logic, and rendering layers to make your shop as responsive as possible if anyone has been to it before. Shopify powers some of the biggest sales on the internet that truly dwarf Good Morning America. The platform is also geographically redundant such that an entire datacenter can fail with only a moment of visible impact to you and your customers. If what I'm saying is true and you don't actually sacrifice that much flexibility on the Shopify platform, why would you choose to have to implement all the above?

Another angle is operations: How many people do you think they have monitoring it and ready to respond to any incidents? Shopify serves an order of magnitude more traffic than Baquend saw at the very peak of this occurrence every single minute of the day with a better response time on average.

The Shopify platform took an army of engineers 10 years to build so it would seem unlikely that Baquend has all the same features and stability. How much do you think Baquend charged Thinks to build this? Is it more than $29 / month?

Source: work at Shopify and built many of these systems.

I never said Shopify isn't good. I pay for it and use it. I'm just saying there are limitations by using a third party service and not having complete control of the stack.

Plus you haven't rolled out AMP just yet.

https://engineering.shopify.com/225104708-shopify-merchants-...

Plus, you could always use Shopify's JavaScript SDK if you need more control. We use it with a static Middleman site hosted on Netlify, which is a great combination: You get a performant backend, easy to use e-commerce administration and all the freedom to create your own site.

Another option would be Snipcart, which looks very strong as well but lacked advanced inventory and product management when we had to decide between Shopify and Snipcart.

> It is true that with Shopify or similar platforms you aren't able to make backend choices or implement caching like the article has described, and yes it's true that all the themes in the theme store aren't perfectly optimized for page load time -- but the question really is do you want to do any of this stuff yourself?

You're right, that is surely a thing you do not want to do yourself. This is why Baqend does all the browser and CDN caching, TTL estimation, query invalidations and so on automatically.

In terms of the frontend if you do not want to rely on a stock theme - and many larger ecommerce site won't - then you will have to do these optimizations yourself. Fortunately with good tooling available as outlined in the article.

I think Shopify is in fact a great improvement over systems like Magento. But if Shopify is well-enginneered (which I'm convinced of), how do you handle sudden load spikes in the database? Do you reshard and migrate data in real-time? How does Shopify scale the application tier? How can tenants run scalable custom backend code? What happens if a database or server node crashes?

The advantage of Baqend is that its inner workings are published in research papers and blog posts, so people can judge the performance and scalability not only through benchmarks and real-world examples like Thinks but also by looking at the design.

In contrast to Shopify, Baqend is backend-as-a-service which means customers have full freedom on how they design their site or shop. They do not have to live with a server-generated HTML that they cannot change - they can for exmaple easily build a React/Angular single page applications, run server-side JavaScript code, do custom queries, build a native app around the same business logic and APIs, subscribe to websocket-based change streams etc.

While I respect that it took you 10 years to build Shopify, this is usually not an indicator for scalability and a low-latency design. Often such projects end up with solutions like MySQL sharded at application-level, intermingled technology stacks and legacy systems.

Disclaimer: I'm the Baqend CEO and its caching approach is my PhD thesis.

> then you will have to do these optimizations yourself

For the optimizations you listed they actually all still apply with a custom theme. The author doesn't have to do anything. Shopify caches all generated HTML, serves all assets from a CDN, and controls expiry without using TTLs because we know when anything changes. The stuff custom theme authors do have to do is things like bloat reduction / post-first-paint behaviour addition / css pruning etc etc. We're considering shipping mod_pagespeed for all customers too which would help automate a bunch of this stuff.

> how do you handle sudden load spikes in the database

Most of our read load is served by a datablob caching layer[1] so the DB is doing mostly stuff only it can do, writes, and we then have a carefully tuned schema that allows for massive throughput. We also buy really expensive databases -- we just took shipment of over a ton of database hardware in one datacenter in preparation for Cyber Monday. If we do hit our limit, we're able to throttle the number of people in the checkout while maintaining a good experience for customers.

> Do you reshard and migrate data in real-time

Yes

> How does Shopify scale the application tier

We shard by shop and build failure domains called pods that are self contained versions of each piece of Shopify that can failover independently between infrastructures.

> How can tenants run scalable custom backend code

Using Shopify Scripts. They're only available in the domain of carts and checkouts right now, but are rolling out to all our backend domain models in time. See https://help.shopify.com/api/tutorials/shopify-scripts

> What happens if a database or server node crashes?

We have automated failover in place at almost every level of the system, so unless we've screwed up somewhere, no one notices!

> they can for example easily build a React/Angular single page applications, run server-side JavaScript code, do custom queries, build a native app around the same business logic and APIs, subscribe to websocket-based change streams etc

Did the Thinks store end up using any of these features of the platform? I am sure lots of apps need these things, but for web storefronts a lot of this seems like overkill. Shopify also supercharges you when you want to move beyond just a web storefront: we have a mobile SDK for building native apps, connectors to let you sell on Amazon, Facebook, and anywhere you can inject HTML, a world class mobile administration app, a POS solution, a massive app store for other integrations, etc etc etc.

> this is usually not an indicator for scalability and a low-latency design

Again, sub 100ms response time and an order of magnitude more traffic every minute than the peak the Thinks sale saw... that's pretty good!

I'm trying to get at the fact that building a web store from scratch, be it with a handy backend as a service or not, is going to be really hard in comparison to using Shopify. You sacrifice some small amount of flexibility, but you gain so so so much more by buying into the ecosystem. I still think it is unwise to try to compete with other platforms who's core competency is commerce.

1: https://github.com/shopify/identity_cache

they could do even better. I mean something is really bad here, since their uncached css from frankfurt to my house should not take over 100ms. and they also serve images, css and js from the same domain. and they download the text's as a extra request. and somehow http2 doesn't kick in.

Reply on news.ycombinator.com