shinryudbz

It's def possible, but even if that were the case, I'd still be nervous about clicking on the email links. Given the lack of tools for dealing with this issue, I'll assume Google/Facebook haven't seen this problem in a large enough scale yet.

I've been running into a similar-ish problem which involves someone creating a bunch of gmail accounts and linking my account to it. Whenever that happens, Google sends me an email notifying me with an option to remove the linking. However, since I never initiated that action to begin with, I start worrying that the email could be a phishing attempt, so I don't click any of the links. But as a result, I start getting email notifications whenever someone logs to those accounts from random countries on random phones.

Lately they've started creating Facebook accounts with my email. Despite me not verifying the email, Facebook continues to send me login notifications.

Has this happened to anyone? I don't quite understand the attack vector, but my guess is that they're trying to bomb me with notifications and if/when they start realizing that I'm clicking on the links in the notification emails, they can start sending out phishing emails with malicious URLs.

Thank you :) Yes, the use of multivariate analysis was a crucial insight for me, and I'm hoping these ideas will push the monitoring community forward!

So it turns out that these metrics don't exactly follow a Gaussian distribution, so it's hard to get these algos to work right out the box. Additionally, speed was an important component for us (for training and evaluation), so we had to toss out a lot of the fancy, but slower, techniques.

Well, there's a lot of details and I don't think I can cover it all here, but if you're interested in the general framework that I used to approach this problem, take a look at this blog post I wrote last year: https://medium.com/@upal/how-to-use-machine-learning-to-debu.... Let me know if you have any feedback!

Absolutely! Currently all our large deployments are on-prem. Please reach out to me and we can discuss further: upal@overseerlabs.io.

Glad you liked it :) It turns out that modeling operational metrics is a lot harder than I expected, so there was quite a bit of work we had to do to get the algos to work.

Great question!

Being an engineer myself, this was a personal pain point and I wanted to solve it, but the key question was whether or not machine learning would help. Thus, most of the time was spent deploying the tech with early adopters, refining the algos, and trying to better understand the value.

What I learned was that our message resonated with some companies more than others. Working with those guys and getting some proof-points on the value is what kept us going!

Hi everyone, I'm one of the founders of Overseer. We built this tool because we noticed that engineers had to dig through many dashboards when diagnosing an incident. We felt this process could be streamlined through the use of machine learning.

When we first started working on this project over a year ago, we weren't sure if the algorithms would work, or if our insights would be of value to anyone. We were also struggling to figure out how to make it easier for people to try the product without having to change their existing workflow.

Since then, we've made huge improvements to the algorithms, deployed the tech for several large customers, and demonstrated value. Now I'd love to get a bit more feedback from you guys and see if we're going in the right direction!

So here's how the tool works: 1 - We pull down your dashboards from your existing monitoring tool (e.g. Datadog/Wavefront/Librato) using your API key. 2 - We integrate with your PagerDuty account via a Webhook to notify us when an incident has triggered. 3 - When our Webhook is invoked, we will use machine learning to rank your dashboards, rank the metrics on those dashboards, and notify you via Slack/Email of the top dashboards/top metrics on those dashboards to look at.

For this demo, we only expose the Wavefront plugin, and you'll be able to configure it on the initial page.

To integrate with PagerDuty, you'll need a URL to our end point, and we'll need an email address where we can send the analysis. You can configure that by clicking on your user name (on the top right) and doing the following: 1. Clicking on "Generate API Key" and jotting down the generated Webhook URL. PagerDuty will need that. 2. Filling out the "Organization Email" text box. We will send your our analysis there!

Given that we'll be dealing with potentially sensitive data, we reluctantly decided to add a layer of security and have folks register with us first - this allows us to protect your data better. My apologies for the inconvenience.

I'd love to see what the HN community thinks and how we can make it better!

Hi, I'm the founder of Overseer Labs. Companies that are data driven and care a lot about reliability seem to suffer from the "too much data" problem during an incident. Thus, we wanted to develop a tool that would help them sift through their data and help them root cause the problem faster.

Overseer Labs leverages machine learning to model system behavior during normal periods of operations. Afterwards, the trained model is applied in real-time to find strange behavior and then providing a ranking of all your metrics. By correlating these insights with your PagerDuty alert, you will be able to resolve your production problem faster.

We put up a demo of one of our algorithms so you guys can play with it on a small dataset. No algorithm will work perfectly out of the box for all the datasets out there, but if this looks useful for your company, we can tune it for you.

I'd love to get feedback from all the data driven companies here on whether or not something like this could help.

Thanks for the feedback! Right now, we're extracting data from New Relic/Librato, so the code expects JSON format. However, the plan is to write plugins to convert arbitrary data formats to something that the core ML stack can operate on.