HNHacker News
TopNewBestAskShowJobs

lenn0x

248 karma · joined August 13, 2009

Chris Goffinet Louisville, KY

Github: https://github.com/chrisgoffinet ThreeComma: https://threecomma.io

submissionscomments
lenn0x··on Cloudflare outage on June 21, 2022
correct, i saw that too. the outage returned 500/nginx. no version number either on footer. @jgrahamc thought that was strange too as few commenters last night were caught off guard trying to determine if it was their systems or cloudflare. supposedly its been forwarded along.
lenn0x··on `COPY –chmod` reduced the size of my container image by 35%
Came here to suggest just this. Ever since coming across ko, it's been excellent in our CI pipelines.
lenn0x··on Incident at Slack
been seeing 100% failures on webhook event subscriptions too. that's only reason I noticed right away.
lenn0x··on Incident at Slack
Ever since this started, been seeing 100% failure on webhooks for Event subscriptions.
lenn0x··on Trying to get past the 500 nits limit of the MacBook Pro (and failing)
One aspect that I haven't seen considered is how useful increasing the brightness for folks who are disabled like myself. I have a rare eye condition where the amount of light omitted really matters to me. It's one reason why I look at nit brightness each year on phones and monitors. I can say it really helps the more light these devices can output and how it impacts my vision. It's why I try to upgrade more frequently.
lenn0x··on macOS Setup after 15 Years of Linux
Hidden Bar is good as well, that's what I am using.

https://apps.apple.com/us/app/hidden-bar/id1452453066?mt=12

lenn0x··on Canonical Multipass – Ubuntu VMs on demand for any workstation
night and day. it's the reason I sought out something like multipass. I first started with a license for Parallels Desktop and noticed the disk performance to be way better. Once I came around to multipass, I ditched Parallels Desktop because I liked having cloud-init support, so all my workstations can easily spin that up when setting them up on a fresh-install or a new machine. I have my own dotfiles so automating all of that was a win. Also the auto-mounting of the home directory was a nice touch so I could bind volumes like Docker Desktop does out of the box for you. I would say if you aren't doing heavy i/o across binded volumes you might not notice. But for a client of mine I was having Docker do some large C++ compiles of a project that even after bumping up the cores + memory of Docker Desktop, still felt slow. When trying Parallels Desktop and multipass it felt way snappier and compile times were heavily reduced.
lenn0x··on Canonical Multipass – Ubuntu VMs on demand for any workstation
that's how ive been running it on macOS. I put up a guide https://github.com/chrisgoffinet/docker-multipass
lenn0x··on Canonical Multipass – Ubuntu VMs on demand for any workstation
Yes it's been way faster for me over Virtualbox. It's how I run docker now from macOS. I've been testing out the M1 support for few months now.
lenn0x··on Canonical Multipass – Ubuntu VMs on demand for any workstation
I've been running multipass for awhile now to host Docker on macOS, especially on M1. I ended up writing a guide/docs on how to easily get it setup with cloud-init to replace Docker Desktop. Hope this helps anyone.

https://github.com/chrisgoffinet/docker-multipass

lenn0x··on Walmart is using fully driverless trucks to ramp up its online grocery business
i agree they are definitely getting better though. i have memberships to both and have been watching how they have improved quite a lot in last few months. the big reason why i got a membership to walmart+ was due to Amazon going from (1 day shipping for last 3 years) to (4+ days!) in the Louisville market, which makes no sense. We have multiple warehouses in the area. Tons of articles/forums discussing it, so definitely im not an edge case. Strangely enough when testing out the membership types (personal vs business) of amazon they are now just finally giving me 1-day shipping on my business account but not personal (verified using both items). Walmart+ has them completely beat on some items that are usually essentials that you can get at the local store. Plus they are offering same-day shipping for free, since they are coming from the store. I've had multiple times where I ordered something from Walmart+ where it said 2-3 day shipping and within hours, it turned into 'Oh we found it local' and delivered it that day.
lenn0x··on Readme.so – Easiest Way to Create a Readme
I went ahead and created vscode-readme it takes the templates used on readme.so and creates a snippet per section and if you use the readme snippet it combines all the sections https://github.com/ThreeCommaIO/vscode-readme. I just published to the extensions marketplace too.
lenn0x··on We scaled the GitHub API with a sharded, replicated rate limiter in Redis
You are correct that's what it was
lenn0x··on Crafting a high-performance TV user interface using React
Agreed! I was just looking over their web interface, under Playback settings, they at least now have a way to disable Auto Play
lenn0x··on Mean People Fail
Do they? I wonder if it just comes natural to them and they aren't thinking about what they can and can't get away with it.
lenn0x··on Bitly: Lessons Building a Distributed System that Handles 6B Clicks a Month
I can speak from experience on this, home made cache can be much better depending on your avg object sizes. Letting the OS make decisions usually in databases is the wrong choice. The data is cached at page level (4k), if you have many objects randomly spread across your working set it's not going to be as efficient as object caching. Usually the more control you desire the better for the performance of your database. One technique that is used if you want to just let the OS deal with cache/paging of data is you have a dedicated commit log that is written with fsync on every update or periodically depending on your durability requirements.

The OS usually does a pretty good job, but remember it's designed to be generic not for a specific use case. When you really care about performance of a database you tend to bypass what the OS does, and take control bypassing things like the scheduler, cache, etc.

lenn0x··on Manhattan: Real-time, multi-tenant distributed database for Twitter scale
We store more than just tweets :)
lenn0x··on Manhattan: Real-time, multi-tenant distributed database for Twitter scale
Yes, I think the article got that wrong. You are right that Gizzard is eventually consistent. The storage engine for Gizzard is mySQL, so in the event you want to do some node-local indexing, you at least have mySQL's strong consistency, but you won't achieve it in the overall system across keys.
lenn0x··on Manhattan: Real-time, multi-tenant distributed database for Twitter scale
When Manhattan first started, our first use case was this. To support batch (something we talk about in our blog post). Similar to ElephantDB, the biggest difference was we wrote our own storage engine (seadb), instead of using BDBJE. We built the hadoop pipeline service so developers only needed to supply us with sequence files and we did the work for them by watching.

Over time, Manhattan evolved into a fully fledged read/write database that is able to support batch + read/write in the same system. Batch is great for some use cases, but sometimes the cost is too expensive when you factor in how much processing power, storage, etc is needed for that model. We support both of course still we want developers to have the freedom to increase their productivity.

lenn0x··on Keeping Instagram up with over a million new users in twelve hours
Because in-house we have a custom version of memcache. We rewrote memcache's slab allocator, and for some use cases, is better at memory efficiency than Redis.
lenn0x··on Keeping Instagram up with over a million new users in twelve hours
Taskset is fine for the process pinning. Don't forget about hyperthreading, you want to try to keep each thread on each hardware thread. IRQ pinning, see an example script I have:

http://chrisgoffinet.com/pin_network.sh

This will set queues 0-7 to specific smp affinity slots.

lenn0x··on Keeping Instagram up with over a million new users in twelve hours
What kind of instances are you guys running for Redis/memcached? I am a bit surprised on the numbers here, but to be fair I don't do much in the virtualization world. With low cpu overhead, it sounds like you might be saturating the number of interrupts on the network card if it's not a bandwidth issue. Memcache can usually push 100-300k/s on an 8-core Westmere (could go higher if you removed the big lock). Redis on the other hand with pinned processes to each physical core can do about 500,000/s. We (Twitter) saw saturation around 100,000~ on CPU0, what tipped us off was ksoftirq spinning at 100%. If you have a modern server and network card, just pin each IRQ for every TX/RX queue to an individual physical core.
lenn0x··on Bill Nguyen: The Boy In The Bubble
This is my exact sentiment. I stopped reading after midway. It's great this man has so much passion, but I am pretty shocked how he can just continue to throw paint at a wall hoping he picks the right color that people will love. Maybe he should slow down a little and think through what he wants to do and what problem he is out to solve.
lenn0x··on This Sure Looks A Lot Like Facebook’s Project Spartan
why would they remove them? Just update the application with an embedded safari to the HTML5 version?
lenn0x··on One Million Concurrent TCP connections
It's been done before. Here is an article using Erlang and Linux.

http://www.metabrew.com/article/a-million-user-comet-applica...

Part 3 is my favorite. http://www.metabrew.com/article/a-million-user-comet-applica...

lenn0x··on Yahoo employees: Get out now before the vultures descend
It was. Friends of mine who worked on it, eventually got re-org'd and started a project to rewrite it in Java :/
lenn0x··on Twitter: From Ruby on Rails to the JVM [video]
Here is an old presentation from a year ago.

http://www.slideshare.net/nkallen/q-con-3770885

Now, at the time they were doing peak 2000 tweet/s. The fan-out was 1.2M deliveries a second... So if we go with the current 600:1 ratio at 7000/s, that's about 4.2M/s. I actually know it's much higher now since I work there but other things to consider is we have a large data warehouse, search, API, pipelines to external parties for the firehose, logging at terabytes an hour, in-house metric collection doing 3M writes/s, etc.

http://www.scribd.com/doc/59830692/Cassandra-at-Twitter

It add's up very fast.

lenn0x··on What happens after Yahoo acquires you
I am not sure how much farther you can go when it was a SVP executive.
lenn0x··on What happens after Yahoo acquires you
We tried this approach at MyBlogLog. Managements response is: we'll not let you ship and refuse you resources.
lenn0x··on Kevin Rose wants to buy Delicious and make... Digg
I think he is serious
Page 1 of 2Next →