iamsomewalrus

Underrated comment

I use Go as my preferred language and I think the author is mostly right.

There’s no way for me to know or even check what are the possible errors this function can return? Sure, sometimes a comment in the library might be illuminating, but sometimes not.

I agree that errors as values that I can handle at the call site rarely feel useful.

Some of the Is, As ergonomics have improved, but damn if coding agents don’t love to just fmt.error any code it writes. Thus hiding the useful information about the error in a string.

genuinely curious, can you steel man stored procedures? views make intuitive sense to me, but stored procedures, much like meta-programming, needs to be sparingly used IMO.

At my new company, the use of stored procedures unchecked has really hurt part of the companies ability to build new features so I'm surprised to see what seems like sound advice, "don't use stored procedures", called out as a cargo cult.

off the original topic, but on topic for this -

Yes! Gibson recognized they were getting cut out of the vintage market and started making not only the reissues (RIs), but also the limited edition copy-of-famous-person's-guitar. What gets me is that Reissues these days are priced so close to vintage instruments. It's so hard to justify the purchase.

ADHD-haver here. I do the same thing but with music. I created a playlist of albums I've heard hundreds of times. The songs are in album order and when I play the list I play from the start of an album somewhere in that list.

It keeps that part of my brain occupied, but not focus, while I work on the task at hand.

^ I can vouch for this. Worked at Goodreads from about 2013 to 2017 ish. I’m sure my name haunts git blame now.

Oh man, thousands of up votes for a Led Zeppelin reference. This mirrors my experience as a musician. Learning to play gave me the confidence to go into different scenarios and recognize that I had to tolerate being bad first and then I would get it. It's a journey. It's a sacrifice. Strangely, it never felt like a sacrifice at the time.

i helped build this! it's not what i do anymore, and i don't want to go into too much detail, but the TL;DR is that, yes, just walk out (JWO) really does use machine learning models to identify which products customers are picking.

This video from a year ago goes into more engineering depth than promotional videos: https://www.youtube.com/watch?v=S5t6aYhj6pU

the non-obvious challenges of working in this space are that you have to deal with real world constraints that you just don't have to think about in 100% cloud based software. hardware goes down! internet connections go out! electricity goes out! how much processing can you do in store vs out of store?

that's just the tech in the store. what about humans? humans are now in your programming state moving shit around. in-store associates miss-stock items. kids do kid shit. people don't place things back exactly where they found them.

Then you have interesting distributed problems: how do you handle late data? what should be a massively parallel problem is really a graph of interactions that have to be resolved in just the right order so you can generate an accurate receipt.

and you know what's crazy? the vast majority of the time it works! exactly how they say it does: with machine learning models.

bonkers.

This is my experience as a line manager but also a manager of managers.

Hire for fire feels like an urban myth. We do a lot of work to find you, interview you, hire you, and train you. Managing someone out is also a lot of work.

That being said a company as big as Amazon will always have edge cases.

i think this is too pessimistic. via FBA they've also enabled thousands (hundreds of thousands?) of small businesses and a cottage industry of others that have formed around them.

Not sure what the J was for. Internally it’s been IHM since I joined. “Inventory Health Management” is a backronym. I won’t spill the - albeit boring - beans on where IHM actually comes from.

Flippantly, because hardware is cool!

Plugins - software for audio programs - are available but audio engineers are famously persnickety.

I don't have perfect pitch so it's not that I hear a note and see the color it's more like when I think of a note, chord, or mode I see a color in my minds eye. When I'm playing I am aware at a high level the notes I'm playing and their colors.

Key of D is typically black for me. For dorian F-natural is a soft shade of red. The B natural is purple.

F major is red. E-minor is a desert sand color.

Can’t find a reference, but i think it does. I used in the past year for that purpose exactly.

Also, checkout typing.io which has lessons based on open source code.

(Amazon employee but not doing anything related to deliveries)

UPS drivers pee in bottles while out delivering, too.

In another life I worked facilities at a UPS distribution center and my team would have to pick up those bottles :-/

When yah gotta go, yah gotta go.

as always, Miley Cyrus is right.

I use this mapping in my current setup and find it really useful.

I wrapped my head around the ergodox by realizing that you have to use layers. It's not supposed to map 1:1 with a regular keyboard.

I use a modified version of the default symbol layer that's more programmer friendly (I think) and I put the toggle for this layer under my right index finger. I feel like I'm more accurate now because it's not just my right pinky doing all the symbol work.

https://configure.ergodox-ez.com/ergodox-ez/layouts/aN90b/la...

I disagree. I found the book enlightening as I was experiencing growing pains moving to a more senior role on my team (less time for getting things done, more time helping others, etc) and this book helped me realize that it was to be expected.

That may be obvious for others, but it wasn't for me.

I’ll have a blog post out in AWS Big Data in a few weeks but the parent is right in that it’s not terribly difficult.

Should take about an hour with testing to get a Pyspark script together to read in a DynamoDB table and write it out to S3.

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programm...

You’ll then need to crawl the S3 data to add it to your Glue catalog and then you can query it with Athena.

100%. Expect a follow up in the future with a simplified pipeline.

The docs are a good place to start to get a sense of what Athena can do: https://docs.aws.amazon.com/athena/latest/ug/what-is.html

There are no indexes in the traditional MySQL / Postgres sense of the word. You can, however, layout the data to make your querying more efficient. See: https://docs.aws.amazon.com/athena/latest/ug/partitions.html

This architecture is meant for business intelligence purposes, not for oltp queries. You're right, it would be pretty expensive to power a user facing service this way.

However, I read a harrowing / awe-inspring blog post about someone doing just that. So...¯\_(ツ)_/¯

Carrier pigeon transport?

+1 Everything I've learned the hard way with DynamoDB was in the Best Practices documentation the whole time.

Now I've learned to read the Best Practices section of any AWS service before I start implementing. It saves a lot of heartache

You, dear person, win my undying support and affection.

The AWS Glue service provides a serverless Spark environment for running jobs. Here's a link: https://docs.aws.amazon.com/glue/latest/dg/author-job.html

* The default timeout is ~ 48 hours and you pay per Data Processing Unit (DPU) that you've provisioned the Job. * Currently it supports Python and Scala. As far as I'm aware you can't run Java jobs directly, but you can upload JAR libraries and use them in your code.

Re serverless vs dedicated / ephemeral clusters: Like with any serverless runtime environment you are trading convenience (across a few dimensions) for flexibility.

The Glue environment runs in a few limited runtimes and uses a specific version of Spark that you have no control over updating. Given that it's pretty quick to author a job, you can set the required DPU and Glue handles that, and you don't have to worry about sizing the cluster for your data size. For me most of my jobs fit within those constraints.

At some point on the cost curve it may make sense for you to move all of your jobs from Glue into a dedicated cluster on EMR. You may also get there sooner if you need to use specific frameworks or libraries.

The Lambda is included in the DynamoDB exports CloudFormation templates. It's embedded in the file template itself.

Thumbs up on the Glue Dev endpoint. It's been killer. I had trouble setting up a Notebook (I wanted to get fancy with Docker) and I usually use the Python repl link that's provided.

I'm working on a follow up post that removes the Data Pipeline -> Lambda and uses the new Glue DynamoDB integration.

Looking forward to your blog post!

In general, it's not uncommon for startups of the Goodreads vintage to outgrow a simple Rails app and use a more service oriented architecture. We've actually had services for about 5 years now, but we haven't been very vocal about it.

As I mentioned in another comment I, personally, wouldn't advocate for new teams or startups to use a service oriented architecture right out the gate. It's too easy to end up with a distributed monolith (circular dependencies between services, services uptimes that are tightly coupled). Engineers also tend to underestimate the build tooling, observability excellence and discipline you need to make it seamless.