malisper

We didn't apply to YC with the idea of Freshpaint. We were initially a Postgres performance optimization tool.

> but if retroactive event streaming is the only USP I think that should be easy to reproduce for Segment

Retroactivity has been the main differentiator between Heap and it's competitors for seven years now. None of them have been able to successfully copy it yet. Mixpanel launched an autotrack feature, but later shut it down.

> Freshpaint is probably not compliant with GDPR as of now

We are GDPR compliant. In fact, Fitz helped form an industry working group on GDPR with the IAB while working at Quantcast a year ahead of GDPR coming into effect.

Freshpaint works like:

> What I want, past any sort of reasonable business scale, is a constrained set of specific events that I have explicitly decided that I want to track.

and not like:

> here we tracked everything for you, now just sort through it

We collect everything for you behind the scenes, but we will only surface a piece of data if you explicitly ask for it.

This is similar to undo in a text document. While you're editing a document, the entire history is tracked behind the scenes. You don't ever need to pay attention to all of that data unless you explicitly use the undo feature.

> Saw your comment on the Posthog post yesterday. Sounds like you felt the pain of scaling at Heap, which bodes well for FP.

Yep! At Heap I had to build a system to ingest a ton of data and process queries over that data that returned in seconds in real-time. For Freshpaint, we need to ingest the same amount of data, but without the need to query it. I honestly think building Freshpaint has been several times easier than building Heap.

Early on, I had a conversation with Heap's CTO about Freshpaint. He said something funny along the lines of "I wish I could work on this because I would absolutely destroy this problem."

> Segment's great, but super expensive for business models where the # of your tracked users doesn't scale with pricing.

Definitely! This is one of the things Fitz and I noticed before starting Freshpaint. It seems like until now, Segment hasn't had any real competitors. Segment is kind of now known for being overpriced.

They're pretty different tools. If anything, you would use Freshpaint to send data into PostHog.

PostHog is an analytics tool. Once you have data in PostHog, you can run queries against the data to produce various charts and graphs. Other analytics tool include Mixpanel, Amplitude, and Heap. The primary reason you would use an analytics tool is you want to ask questions about your users.

Freshpaint is a data collection tool. We automatically collect data from your site such as clicks, pageviews, etc. You can then use our UI to send that data to different tools. We integrate with the large analytics tools and also a ton of marketing tools like Intercom and HubSpot. There are two reasons you would want to use a tool like Freshpaint:

  1) It's easy to collect data. Freshpaint instruments your entire site, so you don't have to.
  2) It's easy to send the data to different tools. Once a piece of data is in Freshpaint, you can send it in one click to any of your analytics and marketing tools.

Does that make sense?

Freshpaint is separate from Kissmetrics. Freshpaint and Segment are both data collection tools. Out of the box Segment collects nothing, while Freshpaint collects everything.

Let's say you want to send a purchase event to Kissmetrics to do analytics on. With Segment that would look something like:

  1) File a JIRA ticket for an engineer to instrument the purchase event.
  2) Wait a few weeks for the engineer to write and deploy the Segment tracking code.
  3) Wait a few weeks to collect enough data to do your analysis.
  4) Run your analysis in Kissmetrics.

With Freshpaint:

  1) Use our UI to create the purchase event.
  2) Send every purchase event that happened since you installed Freshpaint to Kissmetrics. This is possible because Freshpaint tracks every event out of the box.
  3) Run your analysis in Kissmetrics.

Does the difference make sense?

We're a Segment competitor, not a Heap competitor (we actually support Heap as a destination). A significant number of Heap's users use both Segment and Heap. That's actually one of the big reasons we started Freshpaint. A lot of people would buy Heap for product analytics, but they would still setup manual event tracking for Segment for their other tools. We started Freshpaint so you won't need to setup event tracking at all.

Not right now, although we have plans to build a React Native integration in the future.

> What would be your plan if Segment added “backfill” as a feature to their existing product? Does your “automation” also provide significant advantages over Segment?

I don't think it's possible for them to provide the same kinds of backfills we provide. At least not without completely redesigning their infrastructure. In order to support retroactive backfills, we capture every user action up front. If Segment were to try to do that, they would suddenly find themselves processing an order of magnitude more data than they are today. If you want to support autotrack, you need to design your infrastruture from day 1 with that in mind.

The big difference between us and Segment is we automatically instrument your entire site. As soon as you install Freshpaint, we are automatically capturing every click, pageview, etc, performed on your site. You can create a new event six months after installing Freshpaint, and you'll have access to every instance of that event since you installed Freshpaint. This means you:

1) Don't have to delay launches so you have time to setup event tracking.

2) Don't have to worry about forgetting to setup tracking. A common problem is a company will ship a new feature and realize two weeks down the road that they forgot to track a key event. They then have to setup tracking for that event and wait another two weeks for data to come up. That jus doesn't happen with Freshpaint. Freshpaint collects every action out of the box and gives you access to that data whenever you want it.

It's worked perfectly fine for Heap. Especially since Heap and Freshpaint capture every event out of the the box, if tracking ever breaks due to something like a site redesign, you can recover all the data you missed out on.

> I’d encourage you to bless a “data-freshpaint” attribute as the preferred way to link events to your system.

If you want to go through with this, you can do this already with CSS classes. You can have classes like "freshpaint-login", "freshpaint-checkout", etc.

> IMO the only way this kind of event tracking can be done right is if developers and Product communicate and the devs, who understand the code, put the tracking in the right place.

If you have the resources to do this, I highly recommend you setup manual tracking. In Freshpaint, we give you the ability to fall back to manual event tracking if you want it. The thing is most teams don't have the resources to write tracking code for everything, so a lot of the time it just doesn't get done. We commonly hear it takes 3-6 months to get Segment fully setup, and even then you have to constantly write new tracking code as you launch new features.

You're missing out on the whole retroactive piece! When you install Freshpaint, Freshpaint starts tracking every click, pageview, etc. This means Freshpaint is tracking nearly every event you will ever care about from day 1. At any point in the future, you can send any data Freshpaint has collected to your different tools. In the same way that programmers try to achieve 100% test coverage, Freshpaint achieves nearly 100% "event tracking coverage" out of the box. That's not true at all with Segment and it can take several months to get even most of the events you care about.

Fitz wrote a blog post on the analytics and marketing tools we recommend startups use [0]. It goes through a number of the different tools and why or why not you would want to use them. You should check it out.

[0] https://www.freshpaint.io/blog/yc-stack-recommendations-for-...

> Surely you meant "5TB", not "5GB"?

I meant what I said. You can literally just setup a PG instance and it will work perfectly fine up to a few GB. At that point, you will probably start to see certain slow queries due to bad query plans. All you need to know are the basics of EXPLAIN ANALYZE, create a few indexes. That will get you to ~100GB at which point you will start to have to deploy more serious optimizations like partitioning, denormalization, etc. Once you get to multi-TB postgres instances, you will have to look at ways to horizontally scale your DB. This can be done in the Postgres world with something like Citus, but you would probably also want to look at non-Postgres based alternatives.

> Heap et al have to scale to all of their users combined.

The hard part wasn't scaling the system to handle all users combined. The hard part was designing the system such that when an individual user runs a query, they would get their results back in a reasonable amount of time.

Having every user in a single cluster made this easier because an individual customer could make use of the computer power of a cluster that was sized to fit the data for everyone in it. In other words, if Heap doubled the number of customers, Heap would get twice as fast for everyone. That's not true for PostHog.

> Heap et al have to scale to all of their users combined (as you said, terabytes), we just have to scale to the biggest user.

A decent sized Heap customer had multiple terabytes of data with the largest being well beyond that. You're going to have to figure out how to scale PostHog to that point without the benefits of multi-tenancy.

> Once users get beyond Postgres, we have integrations with databases that can scale well across many hosts, and we provide services around this to help people size their servers correctly.

I think a cluster of servers that could churn through terabytes of data in seconds would be prohibitively expensive for any individual customer to purchase.

I'm curious as to how you plan to scale PostHog to larger users. As the person who scaled Heap, here is my honest opinion of this. I think there is going to be a huge challenge ahead in scaling query performance. This was perpetually a challenge at Heap and was for a long time the main limitation on Heap's growth.

The challenge was tough enough for Heap and PostHog is going to be at a huge disadvantage due to the lack of multi-tenancy. When you use Heap, your data is stored across Heap's entire cluster of machines. When you run a query, that query is ran simultaneously against every single machine in Heap's cluster. Even though your data may be taking up something like .1% of the total disk space, when you run a query, 100% of the disk throughput of Heap's cluster will go to processing your query. It's not an overstatement to say this alone results in a >50x improvement to query performance.

I honestly think Heap wouldn't be possible without multi-tenancy. It's hard enough as is to get queries that process multiple terabytes of data to return in seconds when you have a fleet of dozens of i3s available. I'm not sure how you would do that with a fleet a tiny fraction of that size. If you're curious about Heap's infrastructure, Heap's CTO, Dan Robinson, has given a number of talks on how it works[0][1].

That's not to say that PostHog won't work for anyone. I previously tried (and failed) to start a company based on optimizing people's Postgres instances. One of the big takeaways I had was that no matter how you use it, Postgres will work completely fine as long as you have <5GB of data. I think if you have a modest amount of data, something like PostHog would work perfectly fine for you. Since the Postgres optimization business didn't work out, I wound up pivoting to freshpaint.io which eliminates the need to setup event tracking for your analytics and marketing tools by automatically instrumenting your entire site. Since I started working on it, things have been going a lot better.

[0] https://www.youtube.com/watch?v=NVl9_6J1G60

[1] https://www.youtube.com/watch?v=iJLq3GV1Dyk

See my reply to appwiz: https://news.ycombinator.com/item?id=22280200

> You can have millions of consumers read concurrently from a single SQS queue.

We're using different definitions of "consumer". By consumer, I'm talking about a group of workers that processes the data for one purpose. For example you may have one consumer read from the queue to generate various metrics and a second consumer read from the queue and write to a DB. With vanilla SQS, when you process a message, you need to perform all the tasks simultaneously. With Kinesis and Kafka you can have independent groups of workers (i.e. independent consumers), each performing one of these tasks. Each consumer is able to process the queue at it's own rate. The way Amazon recommends doing this in SQS is to have SNS fan out a single SQS queue to multiple SQS queues. Then you can consume each queue independently[0]. That will multiply your costs by the number of queues you have.

> Messages that are read remain in the queue up to the configured retention period or until a consumer calls DeleteMessage.

I'm talking about retaining a message even if it was successfully processed, on the order of days or weeks. I've used this feature of Kafka before to implement a recovery log. Under normal operation, Kafka writes data to a DB. If the DB goes down, you can quickly recover the last N days of data by going through the data retained in Kafka.

[0] https://forums.aws.amazon.com/message.jspa?messageID=865925

I've never used SQS but IMO it seems inferior to Kinesis or Kafka. The two big reasons are that you can't have multiple consumers read from a single queue and once data leaves the queue, it's gone forever. Both Kinesis and Kafka let you have multiple consumers and configure a retention period for your messages.

> In a couple hours you should be able to setup automatic backups and practice going through the recover process a couple times.

Unfortunately there's a lot more too it than that. You need to handle when the backup job fails or dies, have a process for deleting old backups, etc. Not just that, but if you have multiple Postgres instances, you need to do this work for each machine. I've seen first hand this kind of stuff become a huge distraction. It's often worth it to pay AWS a bit more in exchange to not worry about this stuff.

Although there are a ton of AWS servers, there's only a few core services that I recommend:

  EC2 - You need a server.
  RDS - You need a database.
  S3 - You need to store files.
  Lambda - You are building an API with short lived requests.

These services are all very high quality and are excellent at what they do. Once you get outside of these core services, the quality quickly drops. You're probably better off using the non-AWS versions of those services.

For a few quick examples, you should be using Datadog over CloudWatch, Snowflake over Redshift or Athena, and Terraform over CloudFormation.

I've previously administrated a cluster Postgres instances with a total of about 1PB of data. My recommendation is that you should use RDS unless you have a reason not to.

RDS takes care of tons of administrative tasks such as backups, replication, failover, and database upgrades. Yes you can setup backups yourself, but the on going maintenance is going to be a pain. You need to deal with what happens when a backup fails, have a playbook for restoring from backup, cleanup your old backups, etc. These are tasks that are extremely dangerous if you get something wrong and they are completely taking care of for you by RDS.

> This is the author's failure, not the commenters'.

Putting all the blame on the author is oversimplifying things. Yes the introduction and title could be made better, but that's not an excuse to interpret the post in a way the author obviously didn't intend it to be interpreted.

It seems like most of the top level responses are responding to the title and not the actual argument. The argument Dan is making is that you can easily get relatively good at most activities. This is because most of the people doing those activities aren't actively trying to get to get better.

95th percentile is probably not the best way to put it since there are tons of activities where nearly everyone is trying to actively get better such as competitive sports. Dan's thesis doesn't hold true for those activities and he knows that. That doesn't change the fact that there are a ton of activities out there where you can easily get better than most people by practicing.

> Following the guidance in the article, despite its best efforts, will lead to lots of code.

I don't think that's necessary the case. My goal in being productive is to produce the biggest wins in the least amount of time possible. In many cases you do that by coming up with a better design and writing less code to begin with.

In fact, I wrote another post[0] on how I approach becoming a better programmer. The two key principles I use are learning how to solve existing problems faster and learning to solve new problems.

[0] https://malisper.me/my-approach-to-getting-dramatically-bett...

Sit through a meeting is a bit dismissive. Most of the meeting I had at the time were 1 on 1 meetings. They were more about sharing context with each other and discussing current problems on the team than figuring out complex problems.

> Amount of work done is a poor way to phrase it.

Amount of work done is a poor way to phrase it. I mean something more along the line of "business value produced".

My previous job was leading a team responsible for scaling a Postgres cluster with ~1PB of data. A number of big wins would come from relatively small changes. You can apply the post just as well to finding the biggest wins in the shortest amount of time.

You're telling that to someone who has Lisp in their username :)

There's still a difference between testing while writing it and after writing it that's independent of whether you have a REPL or not.

> One challenge is that there are some things which should distract us:

> 1. Pull Requests which we should review to unblock our teammates.

For PRs, it's unlikely that the person making the PR needs you to review it right this second. It's ok if you finish what you are working on now while you are still in flow and after an hour or so, review the PR.

If your teammate has nothing to do during that time, that's a completely different problem.

There's two main reasons you would want to use auto-track. You should use auto-track if:

- You don't have the resources to setup tracking code.

- You want to look at data that you forgot to track previously. We've heard a lot of stories of people launching a new feature and forgetting to setup tracking on it.

If you have the resources to figure out and instrument everything you want to track ahead of time, then you should go ahead and do so. A manually tracked implementation is more likely to be correct than a autotracked one. Even so, auto-track is still useful as a tool for when you have new questions you want to ask or you realize that you need data that you aren't currently tracking.

You should not use autotrack to perform fishing expeditions where you just look at all the data and try to make sense of it. You should still have some guiding questions and autotrack is a tool that makes it easy to get the data you need to answer those questions.