HNHacker News
TopNewBestAskShowJobs

burke

3,521 karma · joined July 18, 2008

Principal Engineer at Shopify.

I spend a lot of time thinking about developer tooling

submissionscomments
burke··on Mistral "Mixtral" 8x7B 32k model [magnet]
Napkin math: 7x(4/8)x8 is 28GB, and q4 uses a little more than just 4 bits per param, and there’s extra overhead for context, and the FFN to select experts is probably more on top of that.

It would probably fit in 32GB at 4-bit but probably won’t run with sensible quantization/perf on a 3090/4090 without other tricks like offloading. Depending on how likely the same experts are to be chosen for multiple sequential tokens, offloading experts may be viable.

burke··on Nearly Cheating
I mean, the difference between these two is a mathematical technicality. For all practical purposes it’s the same thing.
burke··on Shopify live dashboard
Good point, better not try.
burke··on Shopify live dashboard
I’m not sure what the globe is written but I’m pretty sure the comment is referring to the shopify core platform scaling to that many dollars of throughput, and while, yes, certainly there are peripheral bits in a variety of languages, the core is still very much a rails project.
burke··on Shopify live dashboard
https://www.reuters.com/business/sustainable-business/stripe...

https://www.shopify.com/ca/blog/carbon-removal

burke··on Wegovy's heart benefits due to more than weight loss, Novo says
Rapamycin likely does this.
burke··on .io considered harmful
Legality is a concept defined by the governments doing the things. It's funny to see people react to something they perceive as immoral and just claim it's illegal.

I'm no legal expert but I'm reasonably confident one or both of Florida and the USA would find that to be an illegal transaction. Presumably Britain and Mauritius did not, at the time.

burke··on Finding that lead emissions from aircraft engines contribute to air pollution
This is really not that onerous.

Anyone commercial already has to go in for 100hr inspections all the damn time. The STC is unlikely to cost much relative to that, nor to add much additional hassle.

Anyone private can deal with it, they own their own plane and knew it wasn’t going to be a cheap hobby going in.

G100UL will cost a bit more than 100LL but it seems likely this will change a bit as it becomes more widely used.

I guess it puts people operating midcentury radials that haven’t yet been certified for G100UL in a weird spot.

burke··on Roundup herbicide ingredient connected to epidemic chronic kidney disease
Fluoride in drinking water is a kind of truly marginal level of harm that’s perfectly designed to trigger paranoid libertarians.

A tiny amount of fluoride is helpful for bones and especially teeth. Fluoridated toothpaste covers this need perfectly, but many people don’t use it. Fluoride in drinking water covers this need also but (arguably) brings along small risks of symptoms of slight excess fluoride consumption.

The amounts that are actually put in drinking water do not seem to have any significant negative effects, and it seems reasonably clear that the population positive effects dominate, but being non-consensually subjected to a marginal risk of negative effect in order to achieve a positive outcome for the population is exactly the kind of thing that reliably provokes this kind of reaction from the paranoid libertarian set.

burke··on ACT test scores for U.S. students drop to a 30-year low
If there was some other major thing that came online in 2018, I think it would be easy to rationalize this chart as confirmatory evidence of that too, though I mostly agree.
burke··on Show HN: LLMs can generate valid JSON 100% of the time
You can go pretty deep once you get context free grammars. For example, I'm using torch-grammar (but outlines should be able to do the same thing once CFG support is merged) to not just restrict the format of a generation to a DSL's syntax, but to restrict the keys it updates to valid keys in a known set.

e.g.:

    int_key ::= DQUO ("f" ("e" ("atured-" ("b" ("log." ("p" ("ost_limit" | "a" ...
Obviously, yeah, it doesn't "understand" the content, but that's what the LLM is for. It's remarkable how plausible the generations you can get out of random noise are with a sufficiently-restrictive grammar. Bolting that onto a well-trained LLM is pretty powerful.
burke··on Show HN: LLMs can generate valid JSON 100% of the time
torch-grammar generates a mask per PDA stack... we don't try to compute all the possible stacks. I'm sure there's something smarter that could be done here and you've probably figured it out (though IIRC regular languages don't have the arbitrarily recursive stack problem that you get when you get to context-free languages?) anyway, in practice we spend a few milliseconds on the first few requests building caches and then just apply masks from caches after that.
burke··on Show HN: LLMs can generate valid JSON 100% of the time
Yes! This is closer to the approach I took in my port of llama.cpp's grammar support to PyTorch: https://github.com/Shopify/torch-grammar/blob/main/torch_gra... ... it generates a tensor mapping each PDA stack to a map of which tokens are acceptable from that state. It seems like a much better way to do it than looping over the sampled tokens on each turn.
burke··on Why prepared rotisserie chickens are cheaper than ones you cook yourself
Summary: some combination of:

- they’re smaller chickens and it’s not as much cheaper as it seems

- it’s a loss leader.

burke··on Llama: Add grammar-based sampling
Sure, here it is. It's extremely rough but it works as a starting point: https://gist.github.com/burke/6d035758f7492612e2ad86bb7de2d5...
burke··on Llama: Add grammar-based sampling
I implemented this for PyTorch too at https://github.com/Shopify/torch-grammar. I have a hacked version of text-generation-inference that uses it—happy to share that if it’s useful to anyone.
burke··on Open source Diablo 1 engine – DevilutionX 1.5.0 released
This is a claim by an organization on ownership of a phrase used by humans in common speech. That doesn’t mean it’s what people mean when they say it.
burke··on Open source Diablo 1 engine – DevilutionX 1.5.0 released
The source is open. “Open Source” does not mean the FLOSS-flavour of Open in conventional speech. If you want to fight that battle, sure, it’s not unreasonable, but realize that you’re pushing a sub-consensus view.
burke··on 3M heads to trial in ‘existential’ $143B forever-chemicals litigation
Steel pans—really any pans that just look like metal on the inside—are not made with "forever chemical" coatings.
burke··on Nanoplastic Ingestion Causes Neurological Deficits
I mean the average lifespan in the year 1000 was about 31, so we’re doing at least a few things right.
burke··on No_color
Terminals fortunately make it quite easy to map the basic 8 colors to whatever you like. There are very few people who can’t select 8 mutually distinguishable colours. They could even choose shades of grey!

This is an argument against using 256-color or true color though.

burke··on Shopify will be smaller by about 20% and Flexport will buy Shopify Logistics
This was partially true at points (though we've always done quite a bit better than government), but categorically stopped being true with: https://news.shopify.com/rewriting-the-story-of-compensation -- now it's pretty competitive.
burke··on Ruby's switch statement is flexible
Yeah: it's almost more true to say that `a === b` just means whatever we've decided `case b when a` should check. `===` is almost never used directly.
burke··on Micro- and nanoplastics breach the blood–brain barrier in mice
If all donated blood was screened for trace microplastics, I think we’d have an immediate and total donor blood shortage.
burke··on It's time to put cancer warning labels on alcohol, experts say
This is the thing that's so tricky about... well, I guess the way our world models in general work.

Is drinking bad for you? Yup. Is smoking bad for you? Yup.

Is moderate social drinking bad for you? Likely, but definitely less bad for you than slamming a 6-pack nightly.

Are there other (mainly socially-mediated) health benefits from moderate social drinking? Plausibly? Probably? Occasional social smoking? Probably in some contexts.

If these effects both exist, they can be compared. Do the risk curves intersect? Where? The crossover will land in different places for different people, different circumstances.

burke··on From Ruby to Node: Overhauling Shopify’s CLI for a better developer experience
We’ve been using Go since 2013. We recently started using Rust, but it’s mainly eating away at Go’s usage rather than Ruby’s. Ruby continues to be our first-choice language for most problems.
burke··on The Zen anti-interpretation of quantum mechanics (2021)
> MWI is equivalent to a simpler interpretation that doesn’t require split universes.

That simpler interpretation is also equivalent to a simpler implementation that doesn’t require apparently-arbitrary resolutions of quantum probabilities (MWI). It’s essentially a question of where you hide the “weird”.

It seems likely to me that they’re both wrong, and the in-principle-untestability of all this points to our mental machinery just failing to map intuitive concepts onto this space, but I’m totally agnostic to this. I do find MWI more conceptually appealing.

burke··on Read this post ‘unless’ you’re not a Ruby developer
No, I just think for all ruby is bought into these guard clauses, it's sad that we generally reject the feature that allows putting either the condition or the consequence first, depending on which reads better.
burke··on Read this post ‘unless’ you’re not a Ruby developer
Indeed, but the return value of a guard clause shouldn't be important!
burke··on Read this post ‘unless’ you’re not a Ruby developer
On this note: I’m sad ruby style guides generally ban “and” and “or”.

   raise(“a long error string”) unless valid
This reads much worse, when the string pushes the line near max width, than:

    valid or raise(“..
Of course it’s another idiom to learn but it’s not a difficult one when used simply.
← PreviousPage 2 of 29Next →