gouthamve

Yes, we have an internal doc where we compared with more providers (including loggly) where we got the pricing right. The line currently mentioned in the doc is a typo.

The rationale still holds and one of the primary reasons we built Loki is to have a an easy to manage scalable "open-source" solution.

I personally have been told to log less at a previous company because of the associated costs of logs and I don't think I utilised anything beyond grep (with a few exceptions). I personally feel the trade-offs are right and a simple greppable/tailable solution that is cheap is missing from the eco-system.

Yes, you're correct. We'll correct the mistake in the doc.

The good thing about this is that the grepping can be parallelised and distributed on to several nodes. Having said that, once you select the relevant metadata right, you should be able to narrow it down enough for the queries to be snappy enough.

While this will definitely be slower than something that indexes the contents, you'll be able to store much more in Loki at much lower costs.

Hi, we don't index the values, but rather we index metadata about the logstream, for example, service name, instance ip, and things like that and not the actual contents of the log lines.

Inside kubernetes the metatadata would be podname, namespace, deployement name, container name, etc...

Hi, the promtail code is also available in the repo. See: https://github.com/grafana/loki/tree/master/cmd/promtail and https://github.com/grafana/loki/tree/master/pkg/promtail

Hi please see the design doc for more inspiration as to why we built it: https://docs.google.com/document/d/11tjK_lvp1-SVsFZjgOTr1vV3...

We're soon coming out with a blog post explaining the motivations and architecture in detail. But yes, the pricing is one of the motivations to build this, and as logs will be stored in S3 or an object store, the cost will several orders of magnitudes less.

Hi, we're currently only exposing an API that we wrote. But we have plans to integrate with other agents such as fluentd for log collection. If the use-case has enough support, we might even be supporting a REST API.

I work with Tom on Loki. When we said we wanted to focus on Kubernetes, we meant we wanted to correlate metrics with logs and focusing on Kubernetes lets us do that well.

Having said that, this will work for any logs as long as you can tag the logs meaningfully. We'll soon be releasing packages for all major distros and journald.