The next step in ecommerce – replatform with APIs and micro front ends
stackoverflow.blog
stackoverflow.blog
The movement towards "headless", API-first platforms has a lot of steam. So much so that older platforms (e.g. Salesforce) are jumping on the bandwagon to stay relevant in the market.
That said, I know there are a lot of startup-focussed readers on HN. This trend is for larger companies who want to really own their customer experience, including their own frontend. You'll have more flexibility, and using an API-first platform will be a lot less work than writing your own backend, but... it's still more work.
If you are small, you need a quick MVP etc. going with a Shopify or BigCommerce is the faster way to start selling. Once you've got a good stream of revenue and you're feeling these platforms are too rigid, looking for an API-first platform is a good idea!
I’ve come into working on multiple different headless sites developed by others who then walked away and left an entirely non-technical team to sort it out.
A well-built Shopify theme works great and plenty fast for even medium sized companies.
Since then Shopify has updated a bit (although not all that much, a lot of the issues remain) and we have had a wave of requests in the last year to build in Shopify - largely since none of the local marketing agencies want to touch headless.
It's been a bit disheartening really. I've been trying to get to the bottom of the marketing issues but I haven't been able to get clear answers on what they are.
Shopify also doesn't have amazing support for headless (double login on checkout is a big one), but with the rollout of their own Oxygen/Hydrogen platforms I've been waiting to see if they manage to solve these issues for their own solutions.
I’ve found though that often when clients say they need something that only a headless solution can provide, they actually don’t truly need it. And once the actual requirements are focused, a well built Shopify theme is plenty sufficient.
Maybe we are working at a different scale of companies though.
The main advantage of headless to me is the ability to get the speed of the site way up. But the nature of e-commerce is to add tons of pixels and other junk so it can defeat the advantages (and much harder to talk them out of adding pixels sadly…)
Our scale is a bit unusual, its more like high end small scale than large scale, think more like an architect designed house rather than a commercial building. As a result every site has some fairly unique features and Shopify often struggles with those. It's getting tricker to balance over time as the bar for a 'basic' store keeps being raised though.
If you were small, you don't even get the privlidge of an inventory file. Vendors don't have time to deal with small guys, and it can be super difficult to get started.
Playing with crypto, and NFT's, I realized this was what I wish was the norm. The indexers were close to real-time. The chain I was using (AVAX) had a finality of 4 seconds, and gas is cents.
What i'm going to describe isn't a solution to a problem the big guys have. They had enough sway with their vendors to get good systems. But for small guys like me, open access to automated systems would have been a god send. My dream system would be something like this:
Think of this happening in a subnet (an app specific blockchain that doesn't share space with other apps, so scale is not affected by other apps)
1. A Trusted supplier mints a NFT for each of their SKU's (yeah trust and crypto don't go together ideologically, that's a purity problem crypto needs to get over itself, sometimes trust is useful... if you try to build the whole thing trustless, it's not better than traditional systems).
2. These NFT's are listed in any NFT marketplace that cares to list them. (with subnet-to-subnet messaging, you don't need to duplicate the contracts... so shouldn't have major scale issue. This is a brand new feature, so i'm not super sure though.)
3. I as a small scale vendor list a sku on my normal site. I can take payment as normal, but then I can call the NFT's index to get real-time inventory (even though I have no preexisting relationship with the vendor)
4. I get an order, I buy the NFT off the marketplace, and burn the NFT. My customers details are provided in the NFT burn contract. An app specific precompile let's me transmit those details to the vendor without making it public on the block chain (with proof it received it correctly).
5. After the NFT is burned, an ERC20 token is dropped in my stores wallet. After I get enough of these, I can trade them in for a volme rebate. This let's the small vendors get access to effectively what is a volume discount.
I suppose this could be a SaSS app, but there's a bunch of side benefits to crypto, such as defi, tooling etc. But mainly vendors can customize their own terms if they own the full contract chain.
The first white papers I read about crypto emphasized the "trustless" part of it. It's a selling point. They exchanged human trust for computational trust, only now people are realizing the limits of computer trust (namely, anything that isn't digital isn't covered).
What we're looking for now is a way to actually entangle digital assets with physical assets (hence the rise of NFTs, which purport to be a physical thing, or at least the closest thing to).
And you cant trust random people to create the token or to reconcile (lie) about the physical/digital correspondence. Entangling digital with physical seems to devolve to just a database with a trusted authority administering it. I don't see any other way.
Yep! this was my realization when I first started thinking about the idea. You can't computationally trust the movement of physical stuff. Some amount of trust is going to slip in somewhere. It's better to insert the trustless transactions in the places where the relationship is purely digital. In my example above, you have a large vendor which you (as a retailer) trust has inventory to back up every NFT they minted, and you trust to ship relatively reliably. You can't avoid that trust (though you could build mechanisms to increase your trust like 3rd party audits or something). But the relationship with hundreds of small retailers could be built trustlessly. So it's more like strategic trust as opposed to pure trustlessness.
Do you mean you're selling as an intermediary and aren't holding stock yourself? What sort of industries does that happen in? I'd have thought for most B2C sales you'd hold inventory yourself (that's always been one thing putting me off starting an online shop).
Not sure if this is what you're talking about, but drop shipping (you're a proxy store) and Print-On-Demand (you sell t-shirts but they're made and shipped to the customer by the order from a different company - again you're just a proxy) are very popular.
There's less margin but you remove all of the cost/risk of logistics and inventory
Risk isn't removed, only externalized. In this case you externalize it to the manufacturer. They know this and are happy to take on the risk... for a price.
That's why the kind of post-facto bulk rebates pitched in GP make no sense.
The Big Big bulk discount the big guys get is the "real" price, and drop shippers get smaller/no discount because they have to pay at least part of the risk premium induced by the drop shipping business model.
I guess what I don't understand about this is the incentive structure for the vendor.
If the TAM opened by providing small guys with inventory files is big enough, just put a CSV in an S3 bucket or provide an API that's "free until abusive". No need for middleware and you can probably pay for decades of bandwidth before the middleware's cut comes even close to break-even. I work with a few small vendors, and this is basically what they do. No NFTs, blockchain, or other nonsense needed. File hosting+CSV solved the problem over a decade ago.
This part also doesn't make any sense to me:
> After I get enough of these, I can trade them in for a volume rebate. This let's the small vendors get access to effectively what is a volume discount.
The real question is: how many of these vendors even want to work with the small guys?
In B2B, nontrivial volume discounts are justifiable due to a downstream entity's ability to guarantee demand to the upstream supplier. It's a cash-flow thing. If a downstream consumer can guarantee they will buy a certain amount of product, I can take that to the bank (sometimes literally) and invest in growing the business. The reduced volatility and guaranteed demand is worth a 20% discount because it allows me to invest in growth with reduced risk.
Small businesses with JIT inventory and bursty sales are literally the exact opposite of what a small manufacturer wants. As a manufacturer, why would I give a bulk discount to the least attractive type of consumer?! It'd be literally paying for the privilege of taking on someone else's risk... I can understand why my counterparty would love post-facto bulk rebates, but why the heck would I as a manufacturer give them the pleasure of screwing me!?
It's more than JUST inventory. It's inventory, orders, and volume discounts. I don't deny that you could build a bunch of s3 buckets, and API's to solve these problems. Of course you can, we've been doing it for 2 decades. But it's not a great solution. I've worked with them, it sucks. As a retailer I need to automate each one independently. My web3 solution gives a single interface for multiple vendors for each function. I believe my solution is simpler. It may not sound simpler because the tech underneath is somewhat complicated. But from a user point of view (as a retailer, and as a vendor) it's WAY easier. I think it's also SIGNIFICANTLY less work for the vendor to implement.
> Small businesses with JIT inventory and bursty sales are literally the exact opposite of what a small manufacturer wants.
Does a vendor care if it's 1 large business or many small businesses in aggregate? It does if it has to deal with all the small guys. The main problem with dealing with many small vendors is the management of it all. This is my solution to making it trivial, so they can get the additional reach, without having to deal with a bunch of small entities.
My goal is to make it easier for small businesses to compete with big business (as in retailers). I think this technology can solve some of the problems that previously made it not worth the time for vendors.
I see. That makes more sense.
The web3 angle seems like a red herring.
> It may not sound simpler because the tech underneath is somewhat complicated. But from a user point of view (as a retailer, and as a vendor) it's WAY easier. I think it's also SIGNIFICANTLY less work for the vendor to implement.
Easier than bespoke, sure. But a web2 platform custom-built for this purpose would surely be easier than web3. Both to implement and to sell to vendors. No?
> Does a vendor care if it's 1 large business or many small businesses in aggregate?
Well, you definitely care about your counter-party's credit risk and the cost to you of assessing that risk!
Cost of assessment: having one large counter-party vs. a ton of smaller counter-parties is a substantive difference even if they're identical credit risks.
Actual risk: In practice a good number of those small counter-parties are small shops that might keep close to zero capital in their business. Particularly in the case of eg drop-shipping. A typical drop shipper is literally more risky than the end consumer! So counterparty's credit risk is wildly different. A bunch of worthless LLCs running on razor thin margins with very little cash in the bank vs. a large well-capitalized company with significant assets.
> My goal is to make it easier for small businesses to compete with big business (as in retailers). I think this technology can solve some of the problems that previously made it not worth the time for vendors.
Aggregating demand is a useful role for an intermediary, but the real product is financial services rather than information services. Playing the role of either auditor or guarantor for aggregated larger purchasing promises made by smaller entities would certainly be a real financial service for which you could charge a reasonable fee (ie take a cut of the discount). But still the vendor doesn't care. They just treat you as the large well-capitalized client. And your customers are the smaller consumers.
Even better, if your goal is to promote competition rather than just make money: publish an open standard that can be implemented by anyone. Might not be much more than a specific set of columns that a CSV has to have to be considered an OpenVendorMagick(tm) inventory endpoint.
It’s important to not just choose the correct tool for the job, but also be aware of the scale of tool you need.
There are hammers, and then there are pneumatic powered nail guns, which one do YOU need?
Tools at the CommerceTools level address “large company” problems first and foremost, some of which are real and others imaginary, often to the detriment of the customer experience.
This isn’t a criticism of anyone’s technology but more of an observation of how things are very different when the customer is a billion dollar enterprise.
Startups can have great success going API-First, but it's contingent on working with the right company.
I'm one of the maintainers of MageOS (a fork of Magento) as well as a maintainer of Daffodil (a monolithic Angular PWA framework - not Microfrontends yet).
APIS are definitely the way to go but, while that's a great next step, what we're really facing is a standards issues.
Some have tried.
Schema.org as a standard is overly complex and each platform has their own API definition.
Microfrontends can be great as well, but ecommerce has a particularly special problem of requiring extremely performant SSR, so that's always a critical and complex piece to get correct.
However, this article is clearly an ad.
I haven't seen Magento - under Ebay, under Adobe, under anyone address what I think is the biggest issue facing them - total cost of ownership in comparison to the competition.
Is mageOS mostly just a maintainence mode, keep this alive till everyone can switch off (like the M1 forks that exist) or do you actually envision trying to solve some of the challenges magento is facing as a community and platform?
The goal is to migrate management of the codebase out of Adobe's hands back into the community's. Community PRs, triage, etc.
We're currently working on getting the entire CI pipeline setup in Github Actions (if you're interested in contributing, we could really use friendly faces, it's quite an undertaking). Once we're past that, we have goals that are in progress:
DevX:
1. Correct the dep graph of MageOS to be a legitimate DAG and then use nx/bazel/make, etc to compute the build graph. 2. Create codespaces/gitpods/devcontainers for easier day 1 devs. 3. Rip GraphQl out into a separate package space. 4. Create smaller more use-case specific metapackages so that you're not forced into all of Magento or nothing. 5. Maintain backwards compatibility with existing Magento packages.
UX: 1. Significantly improve overall performance.
MerchX: 1. Improve Merch documentation and make Merch docs easier to consume.
Generally to your question, we're not interested in maintenance, we're interested in taking on the burden of being a better development effort alongside a better merchant experience.
That said, this is not a sales pitch. The problem is hard, but we'd like to prove that we're harder.
The backwards compatibility is really a challenge, but I think more importantly dumping Knockout and Luma and all that insanity and going towards the Hyva route, or going more-headlessy with a good starter theme would encourage people enourmously.
Magento's frontend cluster-F is the reason we moved away from it. It was great to have such a good pricing engine, integrations pieces, currency support etc, but all of that skillset which is magento's core capability being a really well rounded online OMS is overshadowed by long and expensive turn arounds on frontend work.
Re:frontend, you're absolutely correct. Magento's frontend is a sad state. MageOS is actively looking for ways to move Luma/Blank into a separate installable package so that devs are not forced to see it at every turn.
In terms of Headless, there are a bunch of different heads at this point, but none have yet solved what I call the "upgrade-safe theme" problem.
I like what we've done with Daffodil since it doesn't impose any of those limitations that come with themes, while still handling a lot of the complexity that devs face when building ecommerce stores.
If you can make a name for yourself, you can do very well. Especially for Magento, I love the fact that I can make small change, deliver it as a composer package, and then (if the code is good enough) see the results on many brands within a few months.
It's hard to get that close-to-customer visibility anywhere else as most "cool" large scale FOSS projects are dev focused, which means your impact on the lay user is less obvious.
Aside from the frontend flexibility, one major advantage of building API-first in an ecommerce context is that it can allow different parts of the system to be developed and maintained independently. This can make it easier to update and extend individual components without affecting the overall system - which is often a headache with current systems such as Shopify, Magento etc.
IMO, One of the most interesting nuances in the space is the focus shift between OS and closed-source solutions in this space. The problem for larger companies (+$50M) is that they seek to build something their tech teams can take full ownership of and easily inject custom logic into. This is not as easy with proprietary solutions even if they are headless and API-first. Instead these businesses end up looking into either building something fully custom or seek out open source solutions like Medusa (JS/TS), Sylius (PHP) or Spree (Ruby) to use as a foundation to build a custom setup from.
In general though, the headless / API-first trend definitely seem to be something that a lot of companies are exploring at the moment.
The trend I see continuing is the commidification and standardization of ecommerce platforms, with Shopify and BigCommerce continuing to chomp away at former giants like Magento (now Adobe Commerce). For most companies, the TCO of an ecommerce platform is an important driver in the decision making process. Companies often balk at the demands of shopify (around what you can and can't do) but then come to terms with it and select shopify as their platform.
For anyone with real budgets (like Nike.com) - totally custom, or building on a practically custom (Adobe) site, there may be moves towards the headless trends, but for the most part, the people (SMB) I deal with do not care about the technology, they just want a short time to launch and a low cost. So for me, I've seen a shift towards to low/no-code platforms and low cost. In the headless space I think fabric is particularly interesting, though I do think Adobe has a chance to do it correctly, but overall, I've sat in on several pitches for going headless and the total cost in just doesn't seem to be worth it.
Just about everything is going to AEM with elastic path/ some other ecomm apis and headless frontends (mostly React, namely NextJS, some unfortunately on Gatsby).
I'm not sure what is happening with all the old Magento platforms since Adobe took it over, but I don't think we do much work in Magento any more.
Having talked to adobe reps a bunch it's also they have no idea how things work and how all their tech fits together as Adobe continues to gobble up disparate technologies.
Wondering where to focus
Of course, everyone in the space is scrambling to say they're headless, because commercetools has absolutely changed the conversation in the ecommerce space, but it's not some magic bullet. It's an architectural decision that should be informed by business needs, e.g. supporting multiple regions, multiple brands, or sharing one backend between your website, app, and in-store experiences. Handily, it does serve as a salve to front-end issues inherent to the all-in-one approach that was so dominant a decade ago. By decoupling the frontend from platforms like Shopify Plus or Salesforce, you get a taste of the speed and freedom that comes with a modern front-end, but also quickly run into no-go zones where even in 2022, there's no API access to important backend features - because there never will be, because that messes with the business model of these platforms.
On the bright side, even taking legacy monolithic platforms like that headless will make it easier to migrate to a more robust API-first backend solution when the time comes.
It certainly makes the most sense with the Nikes and the AT&Ts of the world. Substituting something like commercetools in for their in-house ecommerce systems offsets a huge amount of technical debt, and lets internal teams focus on being more effective at doing whatever makes Nike different from Puma. And while a digital transformation to a MACH approach sometimes has a higher up front cost, companies are clamoring to get away from the millions they spend in maintenance on versioned legacy stacks like Hybris and Salesforce.
The benefit of moving to headless to me can be summed up like this: Frontend development strategies, platforms, and libraries are about as ephemeral as a fart. As online OMS systems get as complex as ERPs, it makes little sense to tie OMS capabilities to a specific frontend. That's how Magento 2's frontend got left behind 10 years. They picked something semi-popular and went all in on it (Knockout.JS) and merchants have been paying the price since (literally in terms of $).
Your OMS is going to trend closer to the ERP lifetime, but possible frontend platforms are changing at a rate that is getting shorter, so decoupling allows them to move mostly independently.
To that point though - there's no transcript for the podcast. I try to at least describe some technical merit whenever I write a comment like this on HN, perhaps because I would hope that others would do the same if I were to arrange for sponsored content!
But I'm not going to sit through an hour of content and transcribe a quote vs. skimming a transcript to find something cool to highlight. And this isn't even getting into how the lack of a transcript signals how little the brand prioritizes accessibility!
Brands - there's no excuse not to have a machine-initiated human-edited transcript of any corporate podcast when things like OpenAI Whisper exist. Make sure the partners publishing your content are incentivized to do this as well.
Right now Mirakl is the big incumbent in marketplace APIs, but whether that's the right answer will depend on your company's scale, technical capabilities, and margins.
[0] I keep a folder of examples of unmoderated third-party seller content embarrassing large companies running marketplaces, this one is my go-to example: https://www.independent.co.uk/news/world/americas/ashli-babb...
In turn, depending on your needs and the regulatory regimes you're working under, you may find that you could have saved time and money going with an industrial-grade solution that can seamlessly handle taxes, accounting and revenue recognition, multi-vendor logistics, credit/refunds, reverse logistics, etc. (there's a lot of etc.!), but that could be overkill for your requirements. I've seen SMBs with small digital teams build and run bespoke e-comm solutions handling all their needs (and more effectively than trying to fit a commercial square peg into a round hole), but the more complex and general their use cases, the more trouble they tend to have, especially when they get into multi-stakeholder retail scenarios (e.g., things like splitting a net-90 invoiced order between multiple future dates, vendors and warehouses, and logistics providers, while acting as a subledger that can handle revenue and income recognition properly).
Building a fully custom solution is the most expensive option today and really unnecessary. There is no reason to build your own product management system when so many flexible options exist. Just as I would never recommend building a server and push people towards containerization and the cloud, I recommend finding components that can be leveraged to streamline the custom build.
In terms of high-end headless systems, the market is large and growing. Some leading composable commerce SaaS offerings can offer free tiers and have pre-built front-ends and integrations.
Take a single one of your items: inventory management. There's SOOO much going on here, people look and say "What's the big deal? decrement an integer!" How do you handle payment failures and dunning management? Subscriptions? Bundles? Incomplete carts? Buy online; pick-up in store? blended inventories? multiple locations? The list goes on and the number of interconnected components makes this really hard to solve.
You're building all this, a marketplace and front ends for web & mobile? With a team of 7? I had a team of 23 and struggled to stay on top of all the nuances in just subscriptions, so you're (a) missing important use-cases or (b) way more effective than my former team. Chances are it's somewhere in between.
At this point, the answer is always buy and then build later if necessary. Just as we once questioned building a server vs using the cloud, using pre-built flexible components gets you to market faster.
Just be careful as the market right now has many monoliths and old systems claiming to be "headless" and "composable", but in reality Magento, Salesforce, Oracle, etc. are all expensive to work with and should only be considered for basic needs.
Looking at a marketplace you can consider marketplace specific vendors like mirakl and convictional, but being niche they can be very expensive. I would instead look at composable commerce solutions that are very flexible and can meet your marketplace needs.
As a developer, hosted Shopify is extremely powerful and enables the ability for merchants to create awesome personalized commerce experiences for their customers.
Shopify is great and if you want to start selling in minutes (literally) or do not have complex back-end requirements I highly recommend you use it. (In fact, I believe they have a sale on their plans as of the time of this comment in case you are interested in checking it out.)
Of proprietary solutions Commercetools & Fabric are often mentioned among larger commerce businesses.
Headless = frontend and backend are fully separated; the "promise" was to have the backend be used on multiple projects. You would have a headless backend, then a web frontend project, an iOS one, an Android one, etc without having to rewrite any backend code.
However, the point the platforms are trying to make is that their APIs have been written with the explicit purpose of building a frontend ontop of them later on.
A lot of the old-school platforms have some APIs as well, but they are meant to e.g. import products or export orders. They are not meant to implement a frontend on top of, because they have a frontend engine as part of their monolith, and many functions you need to write a proper frontend will not be available through their APIs.
So the term "headless" means "we have APIs, and they are designed for you to build a frontend on top of".
https://www.howtogeek.com/660841/what-is-a-headless-server/
A “headless” computer system is just one without a local interface. There’s no monitor (“head”) plugged into it. There’s also no keyboard, mouse, touchscreen, or other local interface for controlling it.
[1] https://www.machalliance.com/members
[2] https://dev.to/voucherify/whats-mach-and-how-it-can-make-you...
If you are interested in SaaS, look at Elastic Path.
I see a lot of people recommending the MACH Alliance. That is a marketing organization where members pay in to push their products. Nothing wrong with MACH technology, but worth understanding what it is before going down that road.
The cost to implement and maintain a headless ecomm site is leagues away from a monolithic store.
Plugins that just worked with your Shopify store no longer work and require custom api dev/integration.
Want a new section built or a small thing changed? That’s going to require design and dev.
I’ve spoken to a few larger brands that have actually went back to their old monolithic store after spending hundreds of thousands switching.
The future is headless but not just yet.