paladin314159

That makes sense, and I figured as much.

I just like to make a pitch for competitiveness being a positive trait whenever this comes up!

I don't know that you need the "dysfunctional" qualification to that statement.

It probably suffers from the usual problem: people want to work at a place that is too good for them (whatever that actually means) and companies want to hire people that are too good for them (again, whatever that means).

This is an interesting discussion. I don't believe collaborative and competitive are actually at odds with each other, think professional sports teams as the most obvious example. Yes, you absolutely want collaborative players who will maximize the potential of the team, but you sure as hell want them to be competitive too.

I'm personally a very competitive person, and I love being part of a team that wins. The collaborative vs competitive mindset feels like a false dichotomy to me.

> But social-media platforms don’t have to be organized around shareability. Instagram, for instance, doesn’t allow links, except a single one in each user’s profile. This dampens self-promotion and slows down the spread of information from the rest of the internet on the platform. It doesn’t have native reposting tools, either. And it is, by pretty much all accounts, a nicer place to spend time online.

While it may have that going for it, wasn't Instagram recently considered the worst social media app for mental health? https://www.cnn.com/2017/05/19/health/instagram-worst-social...

I'm not convinced that dampening self-promotion and discouraging reposting are sufficient conditions for considering one platform to be better than the other.

Amplitude Analytics | San Francisco, CA (SOMA) | https://www.amplitude.com | ONSITE

We help companies build better products by providing analytics for understanding user behavior. We believe that the future of product development is in smart and easy-to-use analytics, not the complex data science stacks or surface-level vanity metrics of today. We're a 100-person company (< 20 engineers), and we raised our Series C last summer. We've gotten incredible traction helping customers like PayPal, IBM, Capital One, Twitter, Microsoft, and Square Enix change the way they build products.

We have many open positions that can be found here (we're growing fast!): https://www.amplitude.com/careers. In particular, we're looking to grow out the engineering team with the following positions:

* Senior Backend Engineer

* Senior DevOps Engineer

* Senior Frontend Engineer

Our tech stack consists of Java, Python, Redis, Kafka, PostgreSQL, Elasticsearch, Docker, SaltStack on the backend and JavaScript, React, Flux, Highcharts, d3 on the frontend. We've got a number of extremely challenging technical problems to solve thanks to being in the analytics space, and we're looking for talented people who are passionate about the intersection of technology and product to help us take the next step.

We only hire the best engineers and pay very competitive salaries. Also, we don't do whiteboard interviews! Bring your laptop, and be prepared to write code the way you expect to on the job -- in a familiar working environment with access to the internet.

If this sounds interesting to you, please reach out to us at careers@amplitude.com or apply directly through https://www.amplitude.com/careers.

AWS launched mobile analytics a few years back: https://aws.amazon.com/mobileanalytics/. I've literally never heard of a customer even considering it, despite the fact that their free tier could accommodate 99% of companies. If you're not selling a commodity, don't be too worried unless AWS considers it a core part of their strategy.

Although I'm not a founder, the rule I espoused in the early days (<20 people) was to not hire for a role unless 50% of a person's time was collectively being spent on that role across the company. This was a great way to be disciplined in determining what was actually a bottleneck for our growth.

We've also had great success using zstd with training. We dump a lot of JSON data into kafka, most of which has a similar schema, and training it easily gave a 2-3x reduction in size over lz4.

Amplitude Analytics | San Francisco, CA (SOMA) | https://www.amplitude.com | ONSITE

We help companies build better products by providing analytics for understanding user behavior. We believe that the future of product development is in smart and easy-to-use analytics, not the complex data science stacks or surface-level vanity metrics of today. We're a 100-person company (< 20 engineers), and we raised our Series C last summer. We've gotten incredible traction helping customers like PayPal, IBM, Capital One, Twitter, Microsoft, and Square Enix change the way they build products.

We have many open positions that can be found here (we're growing fast!): https://www.amplitude.com/careers. In particular, we're looking to grow out the engineering team with the following positions:

* Senior Backend Engineer

* Senior DevOps Engineer

* Senior Frontend Engineer

Our tech stack consists of Java, Python, Redis, Kafka, PostgreSQL, Elasticsearch, Docker, SaltStack on the backend and JavaScript, React, Flux, Highcharts, d3 on the frontend. We've got a number of extremely challenging technical problems to solve thanks to being in the analytics space, and we're looking for talented people who are passionate about the intersection of technology and product to help us take the next step.

We only hire the best engineers and pay very competitive salaries. Also, we don't do whiteboard interviews! Bring your laptop, and be prepared to write code the way you expect to on the job -- in a familiar working environment with access to the internet.

If this sounds interesting to you, please reach out to us at careers@amplitude.com or apply directly through https://www.amplitude.com/careers.

> When asked about being mentioned in the suit, Mr. Gopman, who has drawn attention in tech circles before for criticizing homeless people, said he was not happy about it. “How am I going to get married some day if I have to explain that?” he asked. Upload declined to comment on its former employee.

Really now?

I appreciate and agree with your perspective on putting your money where your mouth is, but Elon is the last person to criticize in that context. He quite literally put all of his money into the problems that he thinks are the most valuable to work on (whether you agree with him or not).

Yeah, that's the real power of DynamoDB, despite all of its shortcomings. It's operationally simple (given you have the money).

> Assuming you know how your data needs to be queried when you create your table, then the hash/range key combination could be the answer to your query problems. However we did not.

DynamoDB's purpose is to allow you to easily scale a known query pattern. I definitely don't recommend it as a first solution when you don't know how the data is going to be accessed. Or when everything fits on one PostgreSQL box.

We've had a lot of success starting out with a big PostgreSQL box and migrating use cases to DynamoDB when a table gets too big and has a query pattern that fits (essentially K/V lookup with some simple indexes).

I recently read On Intelligence, very interesting take on the core principles of the brain. Perhaps someone more knowledgeable can comment on this, but it feels like deep recurrent networks do a pretty good job of capturing the concepts that Jeff Hawkins considers really important: hierarchy/invariance and sequences/prediction. Plus they're really effective in practice!

Amplitude Analytics | San Francisco, CA (SOMA) | https://www.amplitude.com | ONSITE

We help companies build the best products by providing analytics for understanding user behavior. We believe that the future of product development is in smart, fast, and easy-to-use analytics, not the complex data science stacks or surface-level vanity metrics of today. We're a 70-person company (15 engineers), and we raised our Series B last year. We've gotten incredible traction helping customers like Venmo, Udacity, Square, Intuit, Microsoft, Postmates, and Square Enix change the way they build products.

We have many open positions that can be found here (we're growing fast!): https://www.amplitude.com/careers. In particular, we're looking to grow out the engineering team with the following positions:

* Senior Backend Engineer

* Senior DevOps Engineer

* Senior Frontend Engineer

Our tech stack consists of Java, Python, Redis, Kafka, PostgreSQL, Elasticsearch, Docker, SaltStack on the backend and JavaScript, React, Flux, Highcharts, d3 on the frontend. We've got a number of extremely challenging technical problems to solve thanks to being in the analytics space, and we're looking for talented people who are passionate about the intersection of technology and product to help us take the next step.

We only hire the best engineers and pay very competitive salaries. Also, we don't do whiteboard interviews! Bring your laptop, and be prepared to write code the way you expect to on the job -- in a familiar working environment with access to the internet.

If this sounds interesting to you, please reach out to us at careers@amplitude.com or apply directly through https://www.amplitude.com/careers.

Thanks for the wealth of resources. After reading through, the only big concern I have with BigQuery is pricing. I like the model of pay-per-query, but it seems like costs can get out of hand easily.

These two issues particularly worry me:

- No non-date partitioning, which means you'll often end up (in my particular use case) scanning 10-100x as much data as you actually want to.

- Pricing is based on uncompressed data size, which is at least several times more expensive than the alternative (e.g. Athena).

Any thoughts on these? We'll have several hundreds of TB of data, so cost matters.

Many thanks! That's by far the most detailed analysis of Snowflake that I've seen.

In AWS, so we understand there will be some extra costs sending data to BigQuery.

Thanks for responding (and for Redis)! That makes a lot more sense now, would love to see the older commands get new, more powerful versions :)

I really wonder why redis hasn't implemented HMINCRBY. Have run into that issue a bunch of times and the Github issue just doesn't convince me.

It should be there! https://jobs.lever.co/amplitude/c40e24a6-1452-4c5c-9e95-7dc2...

Amplitude Analytics | San Francisco, CA (SOMA) | https://www.amplitude.com | ONSITE

We help companies build the best products by providing analytics for understanding user behavior. We believe that the future of product development is in smart, fast, and easy-to-use analytics, not the complex data science stacks or surface-level vanity metrics of today. We're a 60-person company (15 engineers), and we raised our Series B earlier this year. We've gotten incredible traction helping customers like Venmo, Udacity, Square, Intuit, Microsoft, Postmates, and Square Enix change the way they build products.

We have many open positions that can be found here (we're growing fast!): https://www.amplitude.com/careers. In particular, we're looking to grow out the engineering team with the following positions:

* Senior Backend Engineer

* Senior DevOps Engineer

* Senior Frontend Engineer

Our tech stack consists of Java, Python, Redis, Kafka, PostgreSQL, Elasticsearch, Docker, SaltStack on the backend and JavaScript, React, Flux, Highcharts, d3 on the frontend. We've got a number of extremely challenging technical problems to solve thanks to being in the analytics space, and we're looking for talented people who are passionate about the intersection of technology and product to help us take the next step.

If this sounds interesting to you, please reach out to us at careers@amplitude.com or apply directly through https://www.amplitude.com/careers.

Amplitude Analytics | San Francisco, CA (SOMA) | https://www.amplitude.com | ONSITE

We help companies build the best products by providing analytics for understanding user behavior. We believe that the future of product development is in smart, fast, and easy-to-use analytics, not the complex data science stacks or surface-level vanity metrics of today. We're a 55-person company (15 engineers), and we raised our Series B earlier this year. We've gotten incredible traction helping customers like Venmo, Udacity, Square, Intuit, Microsoft, Postmates, and Square Enix change the way they build products.

We have many open positions that can be found here (we're growing fast!): https://www.amplitude.com/careers. In particular, we're looking to grow out the engineering team with the following positions:

* Senior Backend Engineer

* Senior DevOps Engineer

* Senior Frontend Engineer

Our tech stack consists of Java, Python, Redis, Kafka, PostgreSQL, Elasticsearch, Docker, SaltStack on the backend and JavaScript, React, Flux, Highcharts, d3 on the frontend. We've got a number of extremely challenging technical problems to solve thanks to being in the analytics space, and we're looking for talented people who are passionate about the intersection of technology and product to help us take the next step.

If this sounds interesting to you, please reach out to us at careers@amplitude.com or apply directly through https://www.amplitude.com/careers.

Amplitude Analytics | San Francisco, CA (SOMA) | https://www.amplitude.com | ONSITE

We help companies build the best products by providing analytics for understanding user behavior. We believe that the future of product development is in smart, fast, and easy-to-use analytics, not the complex data science stacks or surface-level vanity metrics of today. We're a 55-person company (15 engineers), and we raised our Series B earlier this year. We've gotten incredible traction helping customers like Venmo, Udacity, Square, Intuit, Microsoft, Postmates, and Square Enix change the way they build products.

We have many open positions that can be found here (we're growing fast!): https://www.amplitude.com/careers.

In particular, we're looking to grow out the engineering team with the following positions:

* Senior Backend Engineer

* Senior DevOps Engineer

* Senior Frontend Engineer

Our tech stack consists of Java, Python, Redis, Kafka, PostgreSQL, Elasticsearch, Docker, SaltStack on the backend and JavaScript, React, Flux, Highcharts, d3 on the frontend. We've got a number of extremely challenging technical problems to solve thanks to being in the analytics space, and we're looking for talented people who are passionate about the intersection of technology and product to help us take the next step.

If this sounds interesting to you, please reach out to us at careers@amplitude.com or apply directly through https://www.amplitude.com/careers.

Had a little bet with a coworker as to whether this was possible... would love to see better solutions!

Rules were: implement the game logic only (start with a 4x4 board, given an input direction, update the board) in ten lines of python, where each line is at most 60 characters wide (including indentation).

That's a great observation. Somewhere along the spectrum of query flexibility you reach a point where pre-aggregation doesn't work anymore. We have a separate column-store based system in place for certain types of queries which we'll almost certainly blog about in the future!

Thanks! We have considered using HLL, and it's a pretty cool algorithm.

For us, however, it's important to get the set members at the end of the day. Amplitude is unique from other analytics products in that we put a lot of emphasis on the actual users that correspond to a data point on a graph -- one of our key features, Microscope, is the ability to view those users, see more context around the events they are performing, and potentially create a dynamic cohort out of them. As such, approximations that don't allow us to get the set members don't quite satisfy our use case.

The raw data is stored directly into S3 every hour, but there are multiple systems that process the data directly from Kafka in order to produce the real-time pre-aggregated views.

The files in S3 are pre-aggregated results keyed by how we fetch them (e.g. there will be a file containing all of the users active on a particular day). What you've described is a pretty accurate description of what happens :)

We don't use Camus; IIRC, it didn't exist at the time that we built most of the infrastructure. We just read data directly out of Kafka using client libraries.