HNHacker News
TopNewBestAskShowJobs

gbrayut

579 karma · joined May 17, 2011

Current: Google Cloud Customer Engineer, Formerly: SRE at Reddit, Edge Platform at WalmartLabs, SRE at Stack Overflow

Mastodon https://hachyderm.io/@GBrayUT Threads https://www.threads.net/@GBrayUT

[ my public key: https://keybase.io/gbrayut; my proof: https://keybase.io/gbrayut/sigs/gjXWe_M0-pf2NttqXdOE_0cjYyuN9inOSy9Ev1DRLIc ]

submissionscomments
gbrayut··on Analyzing the Walmart site performance
I joined the Edge Foundation team @WalmartLabs about a year ago. We help manage the HTTP ingress flow (CDNs, WAF, Tier 1/2 proxies and LBs, etc) for all Walmart eCommerce properties. Our custom proxy also does some basic and not so basic frontend optimizations (js and css compression/minification/bundling, image resizing, url rewrites, etc) and handles a large portion of routing requests to backend servers.

This is a great write up from earlier this year, and from the screenshots it looks like it was before our new site design launched. That was a pretty massive modernization effort that included some site performance improvements. We also now have a sizeable team (100+) focused on some of the other details from the article. Brotli for instance went into A/B testing this month.

There are a lot of moving parts and changing priorities at a company the size of Walmart, and development and deployments can be slower than at smaller shops. But there definitely is active progress on improving our site speed.

And if you like working on any of these kinds of systems we are always hiring!

gbrayut··on IBM acquires Red Hat
https://en.m.wikipedia.org/wiki/WSO2

A previous employer used their identity server for a SSO system. It did what we needed it to and seemed like it was well written.

gbrayut··on IBM acquires Red Hat
"We expect OpenShift to become the dominant brand for Kubernetes."

That seems like a strong statement that I didn't see any backing for in the blog post. Care to expand? I haven't seen much settling in the k8s space. In fact it looks like things are still heating up with many companies pivoting in that direction.

gbrayut··on Flares – CloudFlare DNS backup tool
Neat. Stack Overflow made a similar too that let's you easily push DNS records to multiple providers (ala DNS infrastructure as code)

https://github.com/StackExchange/dnscontrol

https://www.usenix.org/conference/srecon17americas/program/p...

gbrayut··on Dropbox traffic infrastructure: Edge network
I work on an Edge Platform team as part of an Edge Foundation that manages both external CDN and internal Tier 1 WAF/Ingress systems. We do Edge computing at both CDN and Tier 1 layers via tenant plugins running LUA/go. We also have an SDN team building Tier 2 solutions, so basically systems operating at the edge of each layer of the HTTP stack.
gbrayut··on Presidential Alerts
From https://twitter.com/dmelvin3737/status/1047557154729840646

The "Presidential" alert is the level of alert. It's been named this for over a decade, but none have ever been sent.

The levels are :

AMBER alert

Severe alerts

Extreme alerts

Presidential alerts

gbrayut··on Introducing Firefox Monitor, Helping People Take Control After a Data Breach
I wasn't aware of the opt out. Thansk!
gbrayut··on Zoho.com CEO says domain with 40M users suspended for abuse complaint
When I worked as an SRE at Stack Overflow we used name.com for all our domains (and R53/GCP/Azure for DNS). Never had any issues, and worth adding to any short list you come up with.

If you do whois lookups against the top 50 websites you'll see a lot of them use a small set of registrar's. But not all of them accept small businesses.

gbrayut··on Home Assistant: Open-source home automation platform running on Python 3
Glad to hear there are plans to improve it. I'd be willing to do a patreon or monthly donation just for better zwave!

Keep up the great work!

gbrayut··on A dull, resilient stream processor in Go
Interesting! We use NSQ at work to collect logs/messages from various systems and forward them to various endpoints (Kafka, ELK, archives, Prometheus). This looks like a more advanced/flexible system, where as we just use a handful of single purpose daemons writen in go (nsq2kafka, nsq2es, nsqarchive, nsqstream-metrics).

I gave a quick intro to nsq talk at our local go meetup recently. If you haven't used NSQ before I highly recommend giving it a try.

https://docs.google.com/presentation/d/1e9yIm-0aNba_H1gX_u7D...

gbrayut··on GLB: GitHub's open source load balancer
Just a guess, but by using UDP transport for the encapsulated data and configuring the module on the proxy to accept UDP on a wide range of ports, you can pick any port you want (not just the destination port of the TCP stream in the encapsulated packet). And if you are using ECMP with a known hashing algorithm you can then use that UDP port to explicitly spread packets across the RX queues on the proxy servers (gaining better performance).
gbrayut··on GLB: GitHub's open source load balancer
It does but that setup runs into limits on throughput of individual servers, and doesn't have the same drain/fill/failover capabilities discussed in the article.

To be clear, the HAProxy+vrrp+dns is often a better solution, but this describes an interesting design for a load balancing system that can handle many orders of magnitude more traffic and have maintenance without breaking established connections (one of it's core design features)

gbrayut··on GLB: GitHub's open source load balancer
That is exactly what Stack Overflow uses: keepalived to manage a virtual IP between two decent sized baremetal HAProxy servers (w/ bonded 10G nics). Works great and combined with DNS or Anycast based load balancing can scale pretty damn well. Definitely worth investigating as a KISS approach. To quote a recent Atwood tweet:

"if I have learned anything in my career, it is the shocking effectiveness of building ... literally the stupidest thing that could work. (And then iterating on it for a decade.)"

https://twitter.com/codinghorror/status/1026332543153389569

gbrayut··on GLB: GitHub's open source load balancer
Thanks for the details! I'd love to hear more about any data you have on the efficiency of a "second chance" design vs expanding to three or more failover servers. Very curious if a single alternate is enough to cover majority of incidents (xxx out of 1000 events?) Or how frequently you see failures that fall outside the two chance design decisions.

Keep up the great work!

gbrayut··on Ask HN: How is DDoS protection implemented?
Fastly had a good presentation about ddos trends and how they mitigate them at one of their recent Altitude conferences. Video at https://vimeo.com/212305516 and the mitigation stuff starts around 14:45
gbrayut··on Walmart Beats Amazon in $15B Flipkart Battle
FYI: http://oneops.com is the tech stack used for "Application Lifecycle Management of Cloud Based Workloads" at Walmart / Sam's Club / Etc. It was aquired in 2013 but remains open source (not sure who else is using it though). At a high level it is built around Chef solo, and allows you to define application "assemblies" that can then be deployed to various clouds (on prem or public PaaS). Being cloud agnostic means it can shift workloads between different cloud providers with only minor tweaks to the applications.

It works well for a large company with many moving parts and a need for Enterprise wide policies/quotas/etc, but has some rough edges and leaky abstractions. Kubernetes is a more mature and widely used system that has many of the same features, just based around containers instead of VMs.

Disclosure: I work for Walmart Labs on our CDN / Edge Platform team.

gbrayut··on Vim Clutch – A hardware pedal for improved text editing (2012)
I switched from Logitech to Roccata few years ago. The Type mouse has worked pretty well for me, with 14 programmable buttons

https://www.roccat.org/en-US/Products/Gaming-Mice/Tyon/

gbrayut··on Home Assistant is an open-source home automation platform running on Python 3
HA has a bunch of things it will discover... See https://home-assistant.io/components/discovery/ and the list of manually configured components in the top tab.
gbrayut··on Home Assistant is an open-source home automation platform running on Python 3
Yep! I run a network with a dozen or so zwave devices on hassio on an RPi3. See https://home-assistant.io/hassio/zwave/ for details.
gbrayut··on Monitoring Cloudflare's edge network with Prometheus
We store 18-24 months of data in OpenTSDB with good performance and compression, but it can be a pain to manage if you don't have experience working with HBase. InfluxDB has commercial cluster support, and there are quite a few other options. See https://blog.outlyer.com/top10-open-source-time-series-datab... for a good comparison list.
gbrayut··on HTTPS on Stack Overflow: The End of a Long Road
If you like working on these kinds of projects, the SRE team at Stack Overflow is hiring and we allow remote work full time! https://stackoverflow.com/jobs/143725/site-reliability-engin...
gbrayut··on [dead]
Cause was our primary SQL server BugChecked :-( Doing a full internal write up now, and expediting a SQL maintenance window for tomorrow so we can run some memory checks on the system that crashed.
gbrayut··on Ask HN: Who is hiring? (January 2017)
Also https://stackoverflow.com/jobs/remote
gbrayut··on SQL Server on Linux public preview
Yes... They have said that all existing versions in 2016 will have Linux releases including express.
gbrayut··on Ask HN: What have you achieved in 2016?
Presented an ignite talk at a local DevOpsDays event, then co-presented at this year's SQL Summit. This month submitted a session to the new GroupBy conference and just got an email today that it was accepted for January!

We also went on our honeymoon (Turks and Caicos, highly recommended) and vacations in Portland, Seattle, San Francisco, and Florida.

gbrayut··on BigQuery public datasets now include Stack Overflow Q&A
Blog post: https://stackoverflow.blog/2016/12/You-Can-Now-Play-With-Sta...
gbrayut··on Google Cloud Platform HTTP(S) Load Balancers Returning 502 Errors
There was a great talk at SRECon Europe 2015 about how Facebook scales out their load balancers: https://patrickshuff.com/building-a-billion-user-load-balanc...

Shows how many levels of scaling are required to serve traffic at large scale.

gbrayut··on Understanding /proc
Very nice write-up, and a great way to dive deep into an interesting system! But if you plan on maintaining a project like this long term I would recommend using one of the many existing libraries like https://github.com/prometheus/procfs or http://pythonhosted.org/psutil/

There can be a lot of edge cases, and inevitability things will change in the future. Centralising the work of parsing /proc files goes a long way and helps keep things sane for maintenance.

gbrayut··on Ask HN: Alternatives to AWS?
You can run benchmarks using https://cloudharmony.com but it looks like there site is having issues right now.
gbrayut··on This software startup can tell your boss if you’re looking for a job
Interesting that the article mentions Stack Overflow, as back in April we changed our Terms Of Service to explicitly disallow these kinds of scrapers: http://meta.stackexchange.com/q/277369

Of course that doesn't stop it from happening, but at least now we can send notice and take action when we find scrapers.

← PreviousPage 2 of 3Next →