Hacker News Reader

Comment by unethical_ban | Hacker News Reader

I just want a quick log ingestion tool that can parse syslog easily and graph/search fields for me.

Setting up a simple log ingestion on Opensearch or ELK felt like a true journey, in a bad way.

It’s surprising how challenging this is for both Elastic and Opensearch. The problem is that it’s all configuration and no convention, so you need to roll everything yourself. There should be prescribed recipes to make this simpler. If you’re using something like opentelemetry you can find help easier but it’s still annoying.

It's possible but you need to buy in to the Elastic ecosystem. Stuff like *beats, logstash, etc, they can configure all sorts of index templates, and ingest pipelines depending on what you've configured it to receive.

These days, getting data in and out of Elasticsearch is quite easy with dynamic field mapping. Its keeping it performant which is tricky.

I think both these tools are more on the easy side of setting up if you follow their guidelines. You can be up and running very quickly. The problems arise when you need some custom logic in processing log files. If you have simple shipping requiremts you can bypass logstash altogether . Elastic and opensearch are not the right tool for application metrics though in my opinion, for that use case just use prometheus and grafana.

Have you tried out Graylog? Their core product does pretty decently at my $DAYJOB.

Reply on news.ycombinator.com