HNHacker News
TopNewBestAskShowJobs

podoman

502 karma · joined February 27, 2018

jay [at] highlight [dot] io
submissionscomments
podoman··on Show HN: We built a ClickHouse-based logging service
This is one of the reasons we’re considering moving off opensearch.
podoman··on Show HN: We built a ClickHouse-based logging service
Oh very interesting. I don't think we're going to build something for that use case specifically. But long term, we do plan to build a metrics product, which you could use to analyze these patterns. It's interesting to hear what people use these sorts of product for. Thanks for sharing.
podoman··on Show HN: We built a ClickHouse-based logging service
Very cool. My mistake for overlooking that. Victoria metrics seems more like a metrics store? Something comparable to influx? Is that right?
podoman··on Show HN: We built a ClickHouse-based logging service
Makes sense. We're going to do #1 for sure. #2 is more difficult because we have to be cognizant of or internal costs. But will share with the team nonetheless. Thank you.
podoman··on Show HN: We built a ClickHouse-based logging service
Sounds good, will update copy accordingly. Thanks for the feedback.
podoman··on Show HN: We built a ClickHouse-based logging service
By default, our hobby deployment reads from a local copy of clickhouse. If you want to point it to a cloud-hosted version, it would be pretty easy to adjust environment variables accordingly.

https://www.highlight.io/docs/getting-started/self-host/self...

If you're interested in doing that, feel free to message us in our community.

podoman··on Show HN: We built a ClickHouse-based logging service
> Which is to say: is there some obvious hook-point to add watcher jobs (as Clickhouse stored procedures, maybe?) to process correlations in the inputs to the system? (Where by "watcher jobs", I mean things like "create a record in a table if a user makes requests across endpoints A, B, C, in that order — with other arbitrary requests from that user in between — in a five-minute sliding window, at some point within 12 hours of the user's registration.")

This is not something we have planned, nor have we heard yet. Is the use case for doing analytics on your web application? Or more of a complex tracing use case that creates new traces as time passes by?

Fwiw, Highlight is designed (in the current state) for basic log search and alerts.

podoman··on Show HN: We built a ClickHouse-based logging service
Whoa. Didn't know folks did that. I always thought of metabase as BI tool, but the table UI is quite nice.
podoman··on Show HN: We built a ClickHouse-based logging service
Thanks for the feedback. Fixed here: https://github.com/highlight/highlight/pull/5085
podoman··on Show HN: We built a ClickHouse-based logging service
That's good feedback. Thanks. What would you prefer we write? Just mentioned "Web" somewhere in the headline?
podoman··on Show HN: We built a ClickHouse-based logging service
Clickhouse's data and control plane are well defined, so many folks end up using S3 (or something like it) as a backing store. From what we've heard, this is what clickhouse cloud does behind the scenes: https://clickhouse.com/cloud
podoman··on Show HN: We built a ClickHouse-based logging service
> Note: hosted CH did not exist yet. We are using Altinity to run our cluster.

It exists now actually. We (highlight) are on hosted clickhouse, which went in GA a few months ago. https://clickhouse.com/cloud

podoman··on Show HN: We built a ClickHouse-based logging service
fwiw, we don't really see a need for more than a handful of containers, even long term. we want to make it easy to self-host Highlight. nonetheless, sentry is a reputable project. they deserve the success they've had so far.
podoman··on Show HN: We built a ClickHouse-based logging service
No worries, and we appreciate it.
podoman··on Show HN: We built a ClickHouse-based logging service
Just added an issue: https://github.com/highlight/highlight/issues/5082

We don't have pointed plans for supporting Elixir, but hopefully this opens the floor for anyone interested.

podoman··on Show HN: We built a ClickHouse-based logging service
Have not seen Victoria! Seems like they don't have a cloud offering though?
podoman··on Show HN: We built a ClickHouse-based logging service
> But I do hope we get a nice open source (self hostable) tool that can do what it claims (error reporting, tracing, logging, metrics) because there's no such thing that does all well at the moment that's mature.

Keep an eye out for Highlight then!

> Couldn't they just reuse other project's sdk or help with OpenTelemetry? Developing yet another batch of sdk for a dozen language seems like wasted time.

We use OpenTelemetry for all of our SDKs. All we do is thinly wrap the SDK so developers don't have to deal with OpenTelemetry internal if they don't want to.

There's a doc on it here: https://www.highlight.io/docs/general/company/open-source/co...

podoman··on Show HN: We built a ClickHouse-based logging service
+1
podoman··on Show HN: We built a ClickHouse-based logging service
Why isn't it logging? You can send us raw logs if you like as well: https://www.highlight.io/docs/getting-started/backend-loggin...
podoman··on Show HN: We built a ClickHouse-based logging service
That's good feedback. We'll update the pricing page accordingly when the time comes. Question: do you worry about usage based billing with respect to surprise charges? Or do you expect a way to limit this?
podoman··on Show HN: We built a ClickHouse-based logging service
In isolation (just our logging product), Loki is comparable. It would be interesting to do a benchmark on Loki and see what a comparison looks like. Beyond logging, we do session replay and error monitoring and tie all of these things together.
podoman··on Show HN: We built a ClickHouse-based logging service
Yes, we do. It's on our public roadmap: https://github.com/orgs/highlight/projects/11/views/1 (marked as "Future Work")

We'll likely get to it by Q3 of this year. We hope that the design choices we're making (Clickhouse, OTEL, etc..) will set us up for supporting tracing easily when we get to it.

podoman··on Show HN: We built a ClickHouse-based logging service
Thanks, appreciate the kind note on the license. And yeah, air is really nice; we've been using it since the beginning. Gives us a "JS hot-reloading" like experience with a server-side app.
podoman··on [tz] Lebanon DST change internally disputed
apologies. edited.
podoman··on [tz] Lebanon DST change internally disputed
"The Lebanese Education Minister issued a statement a few hours ago saying that educational and vocational institutions in the country did spring forward last night and are now operating on summer time... Folks high up in the government seem to be saying that the Prime Minister acted outside his legal authority in announcing the DST delay, and that unless the Council of Ministers convenes and votes to formally adopt the DST delay, the nation should operate as if DST began on March 26 as originally anticipated."
podoman··on Show HN: We’re open-sourcing our session replay tool
Ha, we were very lucky. We reached out to the owner of the username (a developer from france) and he was kind enough to give it up because it was inactive. He didn't ask for any money, but we ended up paying him a bit since he was so generous.
podoman··on Show HN: We’re open-sourcing our session replay tool
> regarding self hosting, how can contributors make sure that you are self-hosting and paying? If code is open and already in your infra there is no way to enforce anything.

The way we're thinking of addressing this is not including the enterprise features in the output binary when you install highlight.io. This makes it a bit harder to actually build the project with all the enterprise features. It it something that folks can take advantage of, though.

podoman··on Show HN: We’re open-sourcing our session replay tool
I think we see "frontend monitoring" in two different buckets: "metrics-oriented" and "incident-oriented". Metrics are about things like lighthouse scores, optimizing your frontend bundle, etc.. which tend to only become relevant for larger companies as only at that point do these things affect conversions to a significant level. Incidents, however, are more about one-off issues happening in your web app, and that' what we're focusing on at highlight.io (at least right now). Examples include customer support issues, bugs that affect a user's experience, etc..

Both are important for larger companies, but for smaller companies, the only thing that we see as relevant are frontend "incidents".

So to answer your question, I'd ask yourself what types of issues are you trying to track down? Does that make sense?

podoman··on Show HN: We’re open-sourcing our session replay tool
Will do. Sounds like other folks on the thread feel the same way, and we’ll keep this in mind.
podoman··on Show HN: We’re open-sourcing our session replay tool
> How does your session replay feature compare to LogRocket

Forgot to reply to this. We've actually had quite a few customers switch off Logrocket because of our support for things like Canvas recording (https://www.highlight.io/docs/general/product-features/sessi...). From a session replay perspective, we have feature parity, i.e. we report everything in the devtools and even report network request/responses. We also support backend error monitoring and pairing those errors with network requests, for example, which is something logrocket doesn't do.

But overall, logrocket is very comparable tool, but as we mature, we're looking to become a more generalized monitoring tool (w/ logging, traces, etc..).

← PreviousPage 3 of 5Next →