tnolet

or Checkly's heartbeat check

https://www.checklyhq.com/docs/heartbeat-checks/

- non proprietary

- open source

- run it locally

- SQL like syntax

- tons of plugins

- not by Google

e.g. Virtual hosting as we called it in the Apache days

You saw the small replica Rembrandt made for the dude who commissioned the painting. He wanted one to hang in his home. It’s much smaller than the actual piece, which covers a whole wall.

And indeed, the large one got a chunk cut off at some stage as they had to move it. This was long ago when Rembrandt was not particularly in vogue.

Most SaaS is not about tech. It's about marketing, sales, customer success, operations.

Probably slight a fire sale

- Blameless raised ~$50M - Firehydrant raised ~$30M

seems like Blameless was out of runway.

SVB failed due to them tying up too much cash in long term securities, rate hikes and then a bank run. https://www.economicsobservatory.com/why-did-silicon-valley-...

Yes, they were THE bank for startups, but shitty startups did not cause the bank to collapse.

Feels weird defending VC's but here we go: I'm not sure what your experience is dealing with actual VC's? Are you a founder?

In my experience with some top tier VC's none of the above is true. At least none of the VC partners I work with has ever even suggested "lying to customers", "making up numbers" or anything like that.

Sure, looking bigger than you are is useful. Not airing dirty laundry. Also aggressively going after customers is normal. But this is normal for any business.

Love this book.

It’s also one of the most genuinely scary books I’ve read. There are some scenes in this book that really are frightening.

Maybe House of Leaves is up there too.

Loved this one. If interested in similar scifi short story bundles I can recommend Ted Chiang’s “Exhalation” and “Stories of your Life and Others”. One of these was the basis for the “Arrival” movie.

https://en.m.wikipedia.org/wiki/Ted_Chiang

Moreover, we need to have presence in 20+ regions around the world. This multiplies the hassle of self hosting / bare metal / colo.

Disclaimer: Checkly founder.

When it comes from a vendor you signed up for and you configured the service to send you SMS's on certain events. For us that is when a site or app goes down. Think Pagerduty without the pager.

There are many, perfectly valid non-marketing use cases for link shorteners. We send SMS's to people with a link in them. We shorten them so SMS is less annoying to read. Also, we print links in terminal sessions, same thing.

Also in the market and have the same feeling. The new Volvo EX30 looks very nice in comparison to Model Y and 3.

Day 1 vs day 2. That’s why the SaaS version exists.

I guess you word it better than I did.

As a user it feels very weird to wade into threads like this to find a solution to your problem.

The power of Otel is it being an open standard. But the practice shows the implementation of that standard / spec leads to all kinds of issues and fiefdoms

Yeah. And Otel has actually pretty nice debugging. You just need to set the right environment variable. But on prod it will blow up your logs

Well actually. They (python SDK maintainers) argue their implementation is the correct one according to the spec. See this issue thread for example.

https://github.com/open-telemetry/opentelemetry-specificatio...

There are more. This is a symptom of a how hard it is to dive into Otel due to its surface area being so big.

Sentry is self hostable. https://develop.sentry.dev/self-hosted/

A recent example of OTel confusion.

I could for the life of me not get the Python integration send traces to a collector. Same URL, same setup same API key as for Nodejs and Go.

Turns out the Python SDK expect a URL encoded header, e.g. “Bearer%20somekey” whereas all other SDKs just accept a string with a whitespace.

The whole split between HTTP, protobuf over HTTP and GRPC is also massively confusing.

Can you give an example of the missing semantic conventions?

let me nuance that a bit. 99% of our workload is write heavy and is still on S3. We run monitoring checks that snap a screenshot and record a video. We write that to S3. Most folks will never view any of that as most checks pass and these artefacts only become interesting when things fail.

Enter a new product feature we launched (Visual Regression Testing) which requires us to fetch an image from storage on every "run" we do. These could be every 10sec. This is where R2 shines. No egress cost for us. It's been rock solid and saved us about 60x compared to AWS. Still, we run most of our infra on AWS.

In most cases these traces are shipped over the wire to a vendor. Only that will cost $$. Then, not all vendors have tail sampling as a "free" feature. So, in many cases it's better to not record at all.

Oh, simple answer. The tools you use to inspect those traces just blow up with noise. Like a trace that shows 600+ of file reads that all take less than half a millisecond.

This is all noise when you are trying to debug more common issues than your FS being too slow.

+ also storage cost. Most vendors charge by Mb stored or span recorded.

This deserves an updoot. We reached a level of scale right now at Checkly that all of these things start adding up. We moved workloads off of S3 to Cloudflare R2 because of this.

Hey Checkly founder here, we changed our infra quite a bit over the last ~1 year. Still, it's mostly ephemeral compute. We started actually on AWS Lambda. We are on a mix of AWS EC2 and EKS now, all autoscaled per region (we run 20+ of them).

It seems tiny, but in aggregate this will have an impact on our COGS. You are correct that if we had a fixed fleet of instances, the impact would have been not super interesting.

But still, for a couple of hours spent, this saves us quite some $1Ks per year.

That is awesome. Had no idea this was available as an env var. After diving into OTel for our backend, we also found some of this stuff is just too noisy. We switch it of using this code snippet, for anyone bumping into this thread:

   instrumentations: [getNodeAutoInstrumentations({
      '@opentelemetry/instrumentation-fs': {
        enabled: false,
      },
      '@opentelemetry/instrumentation-net': {
        enabled: false,
      },
      '@opentelemetry/instrumentation-dns': {
        enabled: false,
      },

Hey, I work at Checkly and asked my coworker (who wrote the post) to give some more background on this. I can assure you, we're busy and this was not done for some vanity price!

I was super in the other camp. Could not sleep if something wasn’t finished. Especially for coding. For marketing I found a (good) blog post can take a couple of days.

Yeah. We use that too now but still have my F5 account going