Vector – Open-source Rust utility for logs, metrics and events | Hacker News Reader

Vector – Open-source Rust utility for logs, metrics and events

vector.dev

145 pointsweitzj17 comments

Perhaps unrelated, but I appreciate the honest performance comparison graphs on this website. Instead of just showcasing how much better this piece of software is, it also clearly shows that there are cases where its competitors are performing better.

Moet product pages I've seen just want to sell you their product, telling you all about how it's better than everyone else. With graphs like these, I find the product page to be a lot more representative and reliable than other sales pages.

I wish more software developers would do this.

Seems to be really sweet in regards to memory which is one thing that kills me about Logstash / ELK in general. I may have to further evaluate this one as a sane alternative. The one nice thing that ELK has is Kibana though. Wonder how I can generate decent dashboards for this. Would also love to see a database option that is a little smarter. If ElasticSearch crashes or anything it becomes inaccessible for a minute. We lost log data due to instability in ElasticSearch.

https://github.com/timberio/vector/pull/693

Nice, although ClickHouse seems to be a Yandex project. Not sure my boss would appreciate ClickHouse, wonder how does this compare to ElasticSearch though? I'm not looking for 8GB of RAM is not something I can sell my manager.

Check out Altinity - this is US based Clickhouse vendor if your boss needs commercial support and less ties to Russia

Slightly unfortunate naming, as there is already a Vector project in the metrics space.

http://getvector.io/

It seems to be somewhat of a trend to prepend or somehow integrate the language used in naming schemes to ensure you have a somewhat unique name, something like "rvector" would at least make this a lot more searchable, because I suspect most people would search for something like "vector rust" with some additions anyway when looking for this.

The problem with searching "vector rust" is that vectors are one of the most commonly used data structures in Rust and right now searching that will pull up lots of info on Rust vectors (the data structure), not this project

A nice discussion here 2 months ago: https://news.ycombinator.com/item?id=20334779

That's a nice list of sinks and I see the v0.5 Kafka source is already done a few days ago.

1) is there any talk of breaking changes on the road to 1.0?

2) are there any articles about lessons learned from the Timber folks about running Vector at scale?

Hey Lemmox, good questions:

1) Not currently, but it's certainly possible. We don't foresee any major fundamental changes. The most likely change is around transforming. Specifically, how you shape both log and metrics data structures. Right now it's a little too piecemeal and we hope to provide ways to consolidate that more. For example, we have a transform for adding, removing, renaming, and coercing fields. We'll probably introduce a single transform to support that or think about ways to support schemas. 2) Yep! We're organizing our content and education strategy to start sharing this information. Expect to see high-quality guides, blog posts, and comparisons in the next 6-12 months.

Hope that helps!

Terrible name choice...

No comparison with Flowgger? https://github.com/awslabs/flowgger

Impressive! I've been using Fluentd a lot, will look at this.

p.s. Search for "complaince" and fix the typo :-)

Does timber use Elixir for their ingestion pipeline?

Of all the things to name a project these people named it after the most commonly used data structure.

Or one of the most common mathematical terms in programming.

Reply on news.ycombinator.com