Hacker News Reader

Comment by verisimilitude | Hacker News Reader

I can't think of a better way to actually maintain a clear understanding of what's going on at a place like Shopify.

Good memory: Lutke's blog is responsible for a lightbulb moment of mine back in 2007, regarding cache invalidation. The post is long gone, but it was entitled "The Secret to memcached" -- the advice, which is incredibly obvious in retrospect but wasn't to me at the time, is to manage cache invalidation by adding a unique ID to the cached asset, and letting the cache fill and expire items on its own. Going into it, I had assumed I should be removing cached items in the code that was utilizing the cache. I was ignorant. I still am, but less so about memcached. Thanks, Tobias!

View on HN

> The post is long gone, but it was entitled "The Secret to memcached"

The Internet (Archive) remembers:

* https://web.archive.org/web/20120125060458/http://blog.leets...

This technique is the way Rails Fragment Caching works, and enables Russian Doll caching

Those terms should help you find more writing about this technique that I always forget isn’t obvious outside the Rails ecosystem

DHH helped to popularise it with this post: https://signalvnoise.com/posts/3113-how-key-based-cache-expi...

It's been hugely influential to the way I think about scaleable pages. It's a similar idea to the cache-busting techniques well-known with URLs (https://example.com?style.css=1234 etc), but applied to page fragments recursively, where a change in any component also "revs"/"bumps" its parent components.

I disagree completely.

Once you're in a management role, let alone a CEO role, your job not to write software. You defer defer implementation and architecture completely to your team. Your job is to hire and build that team and set high-level company direction.

The more power you have over your co-implementors' careers -- and the more junior they are -- the less comfortable they are standing up to your architectural or technical decisions. This stifling of collaboration can cause you to make poor decisions you wouldn't otherwise, and creates an uncomfortable dynamic.

This is why I don't believe in TLM ("tech lead manager") roles, also, fwiw.

Wanna code? Do it on your own time, Tobi! :)

As CEO or CTO, it;'s crucial you understand your business and if you're in a tech business, you better understand the tech to an appropriate level too. Coding can be a part of that

I’ve only once worked under a manager who also coded, and I had extremely mixed feelings. On the one hand, he was a brilliant engineer who I learned a ton from, and who set a really solid technical direction for the team.

On the other hand, I was personally unhappy a lot of the time. On one occasion he pretty much tore apart one of my designs and I felt fairly humiliated (with reason, as a few other senior engineers confirmed in private conversations).

Did my design deserve to die horribly? I don’t know. The PR had already been accepted, but maybe the accepter was wrong. In any case it felt really crappy to be criticized so heavily by someone who in theory supports me.

You know, it's easy to be an asshole when you're brilliant. But that's why soft skills are important, especially in leadership roles.

I tried my best to be a supporting lead, everywhere i was in that role. I believe in leading by example, but i recognize the necessity to _let_ people do things themselves and not just hog power because "i know best". Let people implement imperfect designs. They'll own it, learn, and fix them in time. Your job is to lead, to give direction and not let total crap be put into prod, not keep people under your "benevolent dictator" heel.

I tend to disagree with the "live and let die" approach. Delegating responsibilities isn’t abandoning responsibilities and when deviation happens, getting hands dirty is good stewardship to me.

The main factor to me is the team size and subsequently the organization of said responsibilities delegation. My personal context sweet spot is around a 50 persons team.

There's also good and bad ways to give feedback. It's important to find ways to give honest feedback that isn't crushing your reports's confidence. It's more of a question of when to step in with an important critique that whether to at all. There are some pieces of code and systems that aren't critical enough or won't be stressed by a junior learning through a bad design decision. And critical system where you absolutely have to step in and make sure everything is in the best shape possible.

50 persons is a small organization, not a team!!

I agree about responsibilities part, however, if you don't let your people grow by simply vetoing the ideas, they won't grow. Discussing and openness to your subordinates' ideas is not a bad thing in my opinion. Letting people make proof-of-concepts and fail is not abandonment of responsibilities. Letting people implement their bad ideas in production is.

> 50 persons is a small organization, not a team!!

Well, I refer to the whole company as a team. This may be my SME bias :)

That's a sign of a good CxO, IMHO - thinking of the whole org as a team. Respect :)

Generally true, but not always true. Shopify seem to be doing just fine with this model. If things change, maybe this model will also change.

You got the point provided that you are concerned about code quality. This might be an area where a CTO lacks skills over time compared to his developers.

Becoming a CTO myself within an 80 developers area, I found it invaluable to code infrequently still. It is quite simple: better decisions, better products.

To me, non-coding CTOs or most CEOs that do not understand Computer Science, it seems that they try to learn a foreign language by sending in their assistant to attend classes.

I mean this quite literally. My team can communicate to me in other terms and a different language than they would have in a simplified ELI5/KPI driven manner. And vice versa, I challenge them to think beyond day to day business.

It is not a lack of trust. It is about better understanding and better helping your team focus on the essential parts. And vice versa.

I believe in dog-fooding, using your own products. That holds for programmers and UI designer, who should use the software they build, but also for architects and CTOs: they should experience the effect of their decisions, and taking part in the design and coding can help.

He’s great. I used his Money (for managing money values as cents in the db), Liquid and DelayedJob in a bunch of projects.

Now I run my own Shopify store, selling my software.

Thanks for sharing this. I've used this (I call it a cache buster) for static assets in a CDN, but never considered it for other things. I'd love to read the article if it can be dug up. It's not clear how that unique ID is managed and am curious to learn more.

You can often go further than that with some database trickery.

We once needed to cache a list of objects that was unacceptably slow serialize the JSON for. We needed to cache them in memcache, but also needed reliable cache invalidation without having to instrument every ingestion point with the logic. Here's what we ended up doing.

1. Every record gets an int field titled "index" or similar.

2. Every time a record is inserted or updated, index is changed to the next value from a sequence

3. The cache key for memcache includes the number of records and the maximum index from all the records returned (including any intermediate objects if joins are involved)

4. Infinite TTL for all cache keys, but the entire cache is LRU guaranteeing that stale records are eventually removed.

This setup works because any conceivable operation done would change the cache key. If new records are inserted or deleted, count and index change. If a record is updated, then the max index changes. Even deleting and adding a new record will change the index, meaning that no matter what happens you'll get a new key and the system will continue to cache appropriately.

From inside of Shopify I’ve heard about a lot of shit stories. For their employees, it may be less ideal to have a CEO messing around with code.

From the outside though, the company does extremely well.

Nonetheless, CEOs are generally not supposed to „make“ but to manage. There has to be a tradeoff somehow.

We call them cache keys. We use a hash of entirely arbitrary data you want a cache item to invalidate on when that data changes.

For example, a cached snippet of HTML for a related products section should invalidate if the product IDs change, so you hash the Item ID with the product IDs and bam, you got a cache key that will let you know when you need to reprocess that snippet.

Put that bad boy as a child of another snippet and you can cache the whole lot. It's cache keys all the way down!

Caveat: not worth the engineering effort for simple systems, just use an FPC and call it a day.

FPC: Full Page Cache

Yep, sorry I should have expanded that.

I think he was an active contributor to RoR in those days

>> I can't think of a better way to actually maintain a clear understanding of what's going on at a place like Shopify.

Talking to customers would be one better way to maintain said clear understanding.

Running your own Shopify store (which Tobias also does) could potentially be another.

Reply on news.ycombinator.com