118 karma · joined August 24, 2016
Which projects are standing out in this space right now?
As a teenager I used to fiddle with the visualization editor for hours. Good fun!
I’ve been pushing the use of Datadog for years but their pricing is out of control for anyone between mid size company and large enterprises. So as years passed and OpenTelemetry API’s and SDK’s stabilized it became our standard for application observability.
To be honest the documentation could be better overall and the onboarding docs differ per programming language, which is not ideal.
My current team is on a NodeJS/Typescript stack and we’ve created a set of packages and an example Grafana stack to get started with OpenTelemetry real quick. Maybe it’s useful to anyone here: https://github.com/zonneplan/open-telemetry-js
There must be an easier way to write migrations for pgroll though. I mean, JSON, really?
This is super fast when taking advantage of postgres' partition-wise joins.
At first glance our solution follows a similar approach, let me elaborate:
- no index columns are updated ever, only inserted
- all tables are partitioned based on date (partition range is 1 month)
- for some tables there is another layer of partitioning (3 sub-partitions, based on one specific column)
- finding an appropriate fillfactor is important to improve the speed of UPDATE statements (HOT-updates)
- standard vacuum / auto vacuum settings work great for us so far.
- to improve ANALYZE performance, set column statistics of value-only columns to 0.
This system contains measurements and state of physical devices (time series). It’s designed for both heavy write and read, with slight emphasis on write. Each table is one type of device and contains 1 to 5 different measurements/states. But here’s the trick: because data is queried with minimum bucket size of 15minutes I figured we could just create a column for each measurement + quarter of the day (i.e. measure0000, measure0015), so that’s 100 columns for each measurement (96 quarter + 4 for DST), include the date in the key, et voila: excellent write performance (because it’s mainly UPDATE queries) and good read performance.
Okay, the queries to make sense of the data aren’t pretty, but can be generated.
I find it really cool how effective this is for time-series data without Postgres extensions (we’re on RDS).
The author explains this very well, it’s a good read! I’ve learned about this padding little over a year ago, while I was designing a data intensive application with a colleague. I was skeptical about the advantage at first, but for our specific design, where we have 100 to 480+ columns in one table it makes a huge difference on table store size. Not so much on the indexes, though.
>When responding to IT or programming questions, respond in UK slang language, Ali G style but safe for work.
Took them a few hours to notice.
Grafana agent as OTEL collector on the application hosts, Grafana Tempo as backend for traces, Loki for logs and Prometheus for Metrics.
The cool thing about Tempo it generates metrics for ingested spans and their labels (spanmetrics) so this allows us to explore “unknown unknowns” as the author calls it in a very cost efficient way.
Pulumi is (mostly) a bliss!
Other than that, for most resources I use the docs are pretty good.
Just make sure your hosting package/provider allows and supports self-hosted mail. PTR dns records specifically as without your mail might work but much ends up in spam boxes. The mail in a box setup guide covers this too.
I’ve been using a handful of these accounts nearly 10 years ago. Not much if any mails received on them for at least a few years, and received the inactive notification for all of them last month. Note the nuance that these account didn’t have mail to forward for several years.
It’s very bold to state “ChatGPT user sessions went down by X” when not taking into account:
- iOS app launch (May ‘23)
- Android app launch (July ‘23)
- GPT3.5 turbo and GPT4 API general availability (July ‘23)
- the explosion of apps that users shift to after 3.5-turbo/4 API GA
I worked with ChatGPT’s web interface daily before the mobile app was launched, and my usage is now at say 70% web / 30% app. And countless extra “sessions” integrated directly in code editor (Cursor)