Hacker News Reader

Comment by tyre | Hacker News Reader

Your marketing seems to emphasize storage heavily. Is that the biggest pain point in logging today? If so, my guess would be that retention period could be a bigger differentiator than sheer gigabytes.

What's the killer use case or differentiator (vs. loggly+kibana+elasticsearch for example)? With Slack, it is integrations. If you could do something similar, like a bugsnag + new relic that aggregates all of our logs and notifies us when things happen (either individual incidents or aggregates like "nginx error log above 1 event/10s), you can have all my money.

Another idea would be to standardize transactions across the stack, so I could trace a request from nginx through rails/phoenix, to postgresql, etc. Again, take my money.

A better UI only helps me if I know when to look at it or what I'm looking for.

View on HN

Ding ding ding ding.

It's not the log storage that we care about, it's what we can do with those logs.

I've been through three logging vendors in the last two years (Loggly, LogEntries, now Sumo Logic) and I'm looking to move once again. The secret sauce is not the storage volume; our volume is relatively small: 4.3 GB/day. The secret sauce is the intelligent structuring of unstructured log data, analytics+alerting on the now-structured data, and live tail'ing of logs.

Sumo Logic does an okay job at the parsing/querying/alerting/analytics piece but live tailing is a brand-new feature for them and it has limitations. Sumo Logic's pricing is also frustrating--they don't apply volume discounting until you're pushing something like 500 GB of logs each month. Again, we need affordable analytics; storage is a small piece of the puzzle.

Honestly, our next move may be to an internal "HEK" stack: Heka, Elasticsearch, and Kibana. I've looked at the cost of doing this and it's actually more expensive when you consider what we'd spend to host a big ES cluster internally, but cost is not our #1 consideration, usability is.

The main reason we emphasized storage is because we built it with scaling that in mind, that's all. We've also built it with live tail in mind too. It's an okay differentiator since we believe we can handle the volume with our infrastructure. We also followed up with UI/UX which at the end of the day should make us a platform you can goto, get what you need and go back to work.

Give us a try, even if you don't think you'll use the GBs. We have alerting, field parsing and live tail. Analytics is the only piece we don't have today, but we're working towards it.

I understand. For me, though, scaling is just a small piece of the puzzle and not really a differentiator. To give you an idea of our size, we run about ~500 instances in a private cloud hosted on about 45 dedicated servers at Rackspace. We use OpenStack for stateful services and data storage and we use Kubernetes/Docker/CoreOS for 12FA/microservices. We have many dozens of apps and even with all of that, we're still only pushing about 4-5 GB of logs each day.

The number one most important problem we're trying to solve is the actual parsing of logs during ingestion. We need logs from commonly-used apps & frameworks to be automatically identified and parsed and we need an easy way to set up a custom parser for internal logs that doesn't require crazy hand-written regex. How does LogDNA handle parsing? What apps/frameworks are supported?

We automatically detect common formats using regex during ingestion. Many are based on logstash patterns and we have several of our own (including one that just tries to strip out timestamp + log level). If you use a common format, we can usually add it. If you have a custom format that your app defines, we don't have a way for you to add it yet but it'll likely be a regex as you said. (We can always help you create it if you don't want to).

Give me syslog target, and I'm all yours :)

This is exactly the sweet spot we're focusing on at Scalyr: quick, easy, and powerful parsing, analyzing, and alerting on unstructured logs, including realtime alerts and live tail. We have customers who are infinitely happier after migrating from Sumo Logic.

Congratulations to the LogDNA team on the launch. There are a lot of use cases in this field and it could use some more shaking up.

(Disclaimer: I'm the founder of Scalyr.)

I'm currently using Sumo Logic. I'm pretty happy with the features, but their pricing is pretty frustrating and limits what I can get out of it without multiplying the cost.

Scalyr looks interesting, how does it compare to Sumo Logic feature-wise? For reference, this video has made me switch to them: https://www.youtube.com/watch?v=w5yWfYDQz2Q

We have a pretty strong feature set for ingesting, parsing, and querying / analyzing log (and metric) data. Our key differentiators from Sumo are performance (real-time ingestion, 60-second alerting, extremely fast query execution), pricing, and simplicity / ease of use. We focus on engineering use cases as opposed to business intelligence, so we don't (yet) have all of the group-and-join functions that Sumo provides for creating BI reports.

Speed turns out to be surprisingly important. We've had a lot of customers report that after migrating, usage across the team goes way up, simply because it's much more satisfying to use a tool that gives quick results. See https://www.scalyr.com/case-studies/returnpath for a brief case study.

I'd be happy to discuss in more detail, but I don't want to hijack the thread - please drop me a line at steve@scalyr.com.

Have you looked at Splunk Cloud? Having used both it and the ELK stack, I can say that Splunk lets you easily do way more with your data than ELK does.

http://www.splunk.com/en_us/products/splunk-enterprise/featu...

http://www.splunk.com/en_us/products/splunk-cloud.html

But cost is the problem, isn't it?

I have used/enjoyed scalyr in the past. Pretty good dashboarding, alerting, etc.

Do you really need a large cluster?

We run our ELK stack (elastic search+logstash+kibana+3 days of logs) on a single EC2 ephemeral instance inside an auto-scaling group. If we lose our logs, oh well! Given that we lose this instance every two-ish years, it seems like a reasonable tradeoff.

Well, we use Elasticsearch extensively for our applications and have developed our internal best practices for running it. A single instance is not recommended for ES. If you're totally fine with losing logs, I guess it's okay, but logs are a critical part of production infra for us and so we would either run a larger cluster with data-only and master-only nodes for optimum reliability.

Amazon now offers Elasticsearch as a service; it was a great help for us quickly setting up ELK to test.

Yes, but it's limited to version 1.5.2. Given the backwards-incompatible changes in 2.X, that can be a deal-breaker for many people, sadly.

HEK sounds like: https://github.com/mozilla/MozDef since Mozilla makes Heka

Have you tried that? I wonder how Mozilla likes it

Yup we will be doing some of the items you mentioned. But you gotta start somewhere and launch something. This is just our mvp. :)

I hear you and congrats on the launch!

To be clear, I'm not asking that you have all of this out of the gate. It's that I want to see the vision and get excited about where you're heading. Your early users are investing in your future and taking a bet on what you could become.

Right now, I'm not sure what I should be dreaming about.

You're absolutely right...it's important to clarify the vision. Our vision is to do analytics in the future. Today it's aggregate/search/tail/alerting. I think the beauty of this is that our future is still somewhat unknown and we'll listen to our customers on where we ought to be headed.

For _you_ the unknown future may be exciting, but as a customer it isn't. I'm trusting you with a critical part of my infrastructure and betting that you're the future of logging. If I'm going to make that bet, I have to believe that you have a clear vision and that I'm on board with that.

Obviously internally (as a fellow founder), things are always crazy and you adapt and grow. But your customers need more than the faith that you'll figure it out as you go.

You can do everything you are asking for with Striim. Www.striim.com

I think we're targeting different audiences. We're only really for logging.

Striim's target audience includes people looking for logging too.

I was referring to the aggregation of the logs, including taking columns / data types from different log streams and combining them (in real time) to create new objects. You can then set alerts and build dashboards on the new objects.

Additionally he mentioned standardizing the request across the stack. Since Striim can import the logs from all these types of services and combine them to create a new object that represents the request across the stack he could accomplish his second request as well.

Combine that with the ability to ingest huge amounts of data and query in real time, it sounded like what he was looking for.

Reply on news.ycombinator.com