bmahmood

Cofounder of 13 Fund, and author of this investment thesis. As entrepreneurs who exited our startups last year, we wanted to give back to the communities that fostered us. So we set up this foundation to invest back in SF and NY with bi-annual grants.

For our first grant, we focused on small business development, specifically restaurant closures. We mined public data, talked to affected restaurants, as well as academics and politicians.

We found some expected causes (falling sales) as well as unexpected (labor flight).

Would be happy to answer questions about our methodology or findings!

Amplitude (YC W12) | Product Engineer | https://amplitude.com/

Amplitude is a product intelligence platform, helping companies use self-serve analytics to make better product decisions. We're one of YC's Top 50 startups, already surpassing $100M in revenue this year serving customers like Twitter, Paypal, Atlassian, & Instacart.

We're now building a new team in our company to build the next $100M product, and looking for our first product engineer. This is an opportunity to work on a startup within a startup, building the foundation for a new frontend app and design system from scratch.

Tech: JavaScript, React, CSS, Python, Node, Postgres, GraphQL

Apply @ https://boards.greenhouse.io/amplitude/jobs/4551252002

Likely unpopular opinion here, but it seems a little unfair to fault YC founders for believing "that the way to win was to hack the test", when the YC application itself would seemingly select for founders that exhibit this behavior with questions like "When have you most successfully hacked a non-computer system to your advantage?"

I'd posit the YC application in and of itself shares some of the facets that the article is critiquing in a "test". The fact there are paid services popping up to review YC applications reminds me of SAT Prep services.

Aww thank you! Our design team is pretty cool. :)

Yep, you're correct that we're using observational studies via a regression to remove confounders and estimate treatment effects. Our confounders are synthetically generated based on the observable variables - we can only make projections of course on digital signals our customers send us (we only use first party data). We are working to incorporate actual experiment data into the algorithm over time as well, to get even closer to the true causal treatment effect.

Thanks for reaching out again! We're prioritizing support for Segment at this time, but hope to add other integrations next year. Our analytics product is completely free, so getting set up on our joint solution with Segment shouldn't be too expensive. :)

Thanks for the great feedback! Yes, some of these limitations expressed in the study are true in the case of ClearBrain - namely we are leveraging observational studies at this time as a prioritized ranking algorithm for which behaviors are most important, but the actual effect sizes themselves may be variable. We're working on improvements, as well as incorporating actual experiment data into our algorithm to make it more accurate over time.

Hi Sean - great point! When I was at Optimizely working on their data science team, we found that on average a test needed 10K-20K unique visitors to reach significance.

We find that this rule of thumb extends similarly to our causal analytics platform. However, we have found that even low-traffic sites are able to get a boost if they are tracking more events on their website (increases the opportunities for signal). Also, our simulations run in minutes on all your historical, rather waiting for weeks for users to be exposed to the test, which speeds up time to insight. If we can not determine significance in our simulation though (due to either sample size or signal), we will designate the projection as a correlation.

Hi I’m Bilal, cofounder at https://www.clearbrain.com . ClearBrain is a new analytics platform that helps you rank which product behaviors cause vs correlate to conversion. Think Google PageRank, but for Analytics.

Our founding team worked on this problem for quite a few years while at Google and Optimizely. We contributed to Google Analytics to analyze historical behaviors in seconds, but observing historical trends merely produced noisy correlations. We built Optimizely to measure true cause and effect through A/B testing, but tests took 4-6 weeks on avg to reach significance, and so it would take years to measure the impact of every single page or feature in an app.

So we asked ourselves, could we estimate which in-app behaviors cause conversion, to complement (not replace) a traditional A/B test? We spent a year in R&D, and built ClearBrain as a self-serve “causal analytics” platform. All you have to do is specify a goal - signup, engagement, purchase - and ClearBrain ranks which behaviors are most likely to cause conversion.

Building this required a mix of real-time processing + auto ML + algorithm work. We connect to a company’s app data via Segment, and ingest their app events in real-time via Cloud Dataflow into a BigQuery backend. When a customer uses the ClearBrain UI to select a specific app event as their conversion goal, our backend will automatically run multiple observational studies to analyze how every other app event may cause that goal. This is done in parallel using SparkML, to analyze thousands of different events in minutes. (more on our algorithm here: https://blog.clearbrain.com/posts/introducing-causal-analyti...)

We’ve had beta customers like Chime Bank, InVision, and TravelBank use ClearBrain to estimate which behaviors and landing pages cause their users to convert, and in turn prioritize their actual growth and A/B testing efforts there.

We’re now releasing the product into general availability in partnership with Segment - available on a free self-serve basis today! We look forward to feedback from the HN community. :)

Apologies! Looks like the site was mid-update when you noted the 404s. It's back live now :)

Thanks for the interest! (Cofounder of Clearbrain here).

The patent covers a combination of statistical techniques and engineering systems we built. The tricky part of this is the infrastructure needed to select confounding variables and estimate treatment effects for thousands of variables at scale in seconds. That was what we filed a patent on.

So weird! Apologies - hope this link below works:

https://angel.co/clearbrain/jobs/177711-machine-learning-eng...

ClearBrain (YC W18) | Machine Learning Engineer | San Francisco, CA | Onsite | https://clearbrain.com

ClearBrain (YC W18) is a startup building the first self-serve predictive analytics platform. We help companies automatically predict and analyze when their users are most likely to convert or purchase. Think a supercharged Google Analytics, based on internal tools our team built at Google, Netflix, and Uber. Fortune 1000 companies use ClearBrain to deliver billions of user-predictions every week and drive double-digit lift in their digital campaigns.

We're a deeply technical team (we were the first engineers on Google Ads and Optimizely), and are backed by early investors in Dropbox and AdMob. We're hiring for machine learning engineers to lead on new cutting-edge products we'll building. We work in Go, Python, Node, Scala, Spark in the backend and React, Javascript, Firebase in the frontend.

For more about the role: https://angel.co/clearbrain/jobs/177711-machine-learning-eng....

If interested, please reach out to grant [at] clearbrain.com - we look forward to hearing from you!

Very excited to see Segment launch their Developer Center in conjunction with this Growth Stack overview. Our company (ClearBrain) was one of the early technology partners to build into their Development Center, and it's been a transformative impact on our business.

As an analytics company, one of the first hurdles to getting a customer successful is onboarding their data. Every customer has heteregenous schemas and then you need to wait weeks to collect enough data to find reasonable results. Segment made this dead simple by providing an API spec with a standardized schema that just took a couple days for us to integrate with. Once integrated, you gain access to 1000s of mutual companies using Segment, who can stream you their data in exactly the same format (a huge win for analytics efforts and data normalization).

Highly recommend other companies to consider integrating with Segment. The ease of integration and access to a platform serving thousands of customers is especially helpful for startups.

ClearBrain (YC W18) | Machine Learning Engineer | San Francisco, CA | Onsite | https://clearbrain.com

ClearBrain (YC W18) is a startup building the first self-serve predictive analytics platform. We help companies automatically predict and analyze when their users are most likely to convert or purchase. Think a supercharged Google Analytics, based on internal tools our team built at Google, Netflix, and Uber. Fortune 1000 companies use ClearBrain to deliver billions of user-predictions every week and drive double-digit lift in their digital campaigns.

We're a deeply technical team (we were the first engineers on Google Ads and Optimizely), and are backed by early investors in Dropbox and AdMob. We're hiring for machine learning engineers to lead on new cutting-edge products we'll building. We work in Go, Python, Node, Scala, Spark in the backend and React, Javascript, Firebase in the frontend.

To learn more about who we are and our engineering culture, check out: https://www.keyvalues.com/clearbrain

For more about the role: https://angel.co/clearbrain/jobs/177711-machine-learning-eng...

If interested, please reach out to grant [at] clearbrain.com - we look forward to hearing from you!

ClearBrain (YC W18) | San Francisco, CA | Onsite | https://clearbrain.com

ClearBrain (YC W18) is a startup building the first self-serve predictive analytics platform. We help companies automatically predict, analyze, and retarget users when they are most likely to convert or purchase. Think a supercharged Google Analytics, based on internal tools used at Google, Netflix, and Uber. Fortune 1000 companies use ClearBrain to deliver billions of user-predictions every week and drive double-digit lift in their digital campaigns.

We're a deeply technical team (we were the first engineers on Google Ads and Optimizely), and are backed by early investors in Dropbox and AdMob. We're hiring across the board from generalist to frontend to machine learning engineers. We work in Go, Python, Node, Scala, Spark in the backend and React, Javascript, Firebase in the frontend.

To learn more about who we are and our engineering culture, check out: https://www.keyvalues.com/clearbrain

For all our open roles: https://angel.co/clearbrain/jobs

If interested, please reach out to grant [at] clearbrain.com - we look forward to hearing from you!

ClearBrain (YC W18) | San Francisco, CA | Onsite | https://clearbrain.com

ClearBrain (YC W18) is a startup building the first self-serve predictive analytics platform. We help companies automatically predict, analyze, and retarget users when they are most likely to convert or purchase. Think a supercharged Google Analytics, based on internal tools used at Google, Netflix, and Uber. Fortune 1000 companies use ClearBrain to deliver billions of user-predictions every week and drive double-digit lift in their digital campaigns.

We're a deeply technical team (we were the first engineers on Google Ads and Optimizely), and are backed by early investors in Dropbox and AdMob. We're hiring across the board from generalist to frontend to machine learning engineers. We work in Go, Python, Node, Scala, Spark in the backend and React, Javascript, Firebase in the frontend.

To learn more about who we are and our engineering culture, check out: https://www.keyvalues.com/clearbrain

For all our open roles: https://angel.co/clearbrain/jobs

If interested, please reach out to grant [at] clearbrain.com - we look forward to hearing from you!

ClearBrain (YC W18) | San Francisco, CA | Software Engineer | Full-time, ONSITE | $120K – $140K, 0.4% – 0.5%

ClearBrain is a startup building the first self-serve AI platform for growth marketing.

We help companies automatically predict and retarget users when they are most likely to purchase. Fortune 1000 companies use ClearBrain’s automated machine learning platform to personalize ads, emails, and push notifications to millions of users every week - as effectively as Uber or Google.

We’re led by the founding engineers of Google Ads and Optimizely’s data infrastructure team, and backed by investors in Dropbox, Optimizely, and AdMob. We’re a deeply technical team who value humility and customer empathy above all. As a group we’re also pretty good at bowling, HQ Trivia, and Rubik’s cubes.

We're looking for engineers across various disciplines (frontend, backend, machine learning).

You can email me directly at bilal@clearbrain.com, or find more info on AngelList https://angel.co/clearbrain/jobs

We're mindful of GDPR and consistently ensuring ClearBrain is compliant with the upcoming regulation, from both how we collect and process user data.

With respect to the points raised in the article - ClearBrain actually does not use deep learning techniques as a basis for our predictive models. Predictions in ClearBrain are based either on logistic regression or decision tree paradigms.

From the beginning of when we approached ClearBrain as well, we wanted to make sure we provided a service that wasn't merely a blackbox. We wanted to provide insight into how the models are performing, so we expose analyses such as feature importance, attribute benchmarks, and indications of which actions are informing the models.

This helps with both interpretability and actionability in customer workflows, but also in some of the GDPR issues noted.

Thanks! Yep, we have a feature called "Benchmarks" which uses a decision tree analysis to identify the thresholds in distinct events that lead to an increase probability towards your conversion goal. We wrote a blog post that expands in more detail on how this works: https://blog.clearbrain.com/posts/discover-your-products-7-f...

Amplitude is definitely on our roadmap as one of the next integrations we're looking to support in 2018!

Great question! It is true that generic engagement/activity metrics tend to be highly correlated to conversion, and the absence of any activity tends to be correlated to churn. We see those features show up often.

But the propensity models built in ClearBrain tend to be more specific. The target variable can be defined as any client or server-side event you've tracked in Segment, or any trait/attribute of your user. As such, common use cases tend to be around predicting conversion events to discrete stages of a user journey - separate models for whom will move from plan type A --> B --> C, etc. So even if a generic engagement metric shows up as highly correlated for these discrete stages, the benchmark in engagement would be different and hence still intuitively helpful to diffrentiate groups of users.

ClearBrain models are primarily based on a logistic regression. We automatically run some parameter tuning like ridge regression and class balancing of your data for feature selection and regularization. We connect directly to your data in Segment, Heap, or Redshift out of the box, so can definitely help with your own data!

Thanks for the comment, but sorry to hear the price seems cost-prohibitive for your company size. :( Our goal is definitely to try and provide a solution that scales with different company sizes, but you're right that below a certain size, it may not make sense to use heavy-weight machine learning tools. Happy to chat at bilal@clearbrain.com though to discuss more about your use cases and see if I can recommend some options or even alternatives!

Nice to meet you as well! We've helped several SaaS startups predict upgrades and reduce churn, so definitely sounds like a great fit.

Shoot me an email at bilal@clearbrain.com and I can help you get started. You can also get set up immediately at https://www.clearbrain.com

ClearBrain | San Mateo, CA | Front-End / Full-Stack Engineer | Full-time, ONSITE | $90K – $140K, 0.75% – 1.5%

ClearBrain's mission is to build a self-service AI to predict any human behavior.

Our first product is a predictive analytics layer to help marketers automatically identify which users will convert or churn, and personalize their marketing in minutes w/o a single line of code.

We're a small, tight-nit, experienced team from Google, Optimizely, Uber, & DraftKings (cofounders were the first SRE on Google Ads, and led data science at Optimizely), and well-funded by investors in Dropbox, Optimizely, and AppDynamics. We're working with mid-market to public companies driving over 40% lift on terabytes of data processed, and are looking for someone to lead our frontend infra and guide product design and direction.

If you're interested in making machine learning and predictive analytics accessible to everyone, we'd love for you to join us.

You can email me directly at bilal@clearbrain.com, or find more info on AngelList https://angel.co/clearbrain/jobs/264140-full-stack-engineer

ClearBrain | San Mateo, CA | Software Engineer / ML / Data / Backend | Full-time, ONSITE | $90K – $140K, 0.75% – 1.5%

ClearBrain's mission is to build a self-service AI to predict any human behavior.

Our first product is a predictive marketing layer that can automatically determine user propensity to buy or churn in minutes, and has helped mid-market to public customers increase conversions by over 40%.

We're a small, tight-nit, experienced team from Google and Optimizely (cofounders were the first SRE on Google Ads, and led data science at Optimizely), and well-funded by investors in Dropbox, Optimizely, and AppDynamics. We're pushing the limits of distributed systems and machine learning, and already working with terabytes of data, while innovating in Scala / Spark / EMR.

We think a lot about optimal matrix design, statistical feature extraction, and making machine learning as self-service and scalable as possible. We'd love for you to join us.

You can email me directly at bilal@clearbrain.com, or find more info on AngelList https://angel.co/clearbrain/jobs/224877-software-engineer

ClearBrain | Software Engineer / Data Engineer / ML Engineer | San Mateo, CA | Full-Time | ONSITE | www.clearbrain.com

ClearBrain is an early-stage startup building a self-service machine learning layer to predict users' propensity to buy/churn. We dabble in Spark, Scala, Go, and Node every day, thinking about optimal matrix design and statistical feature extraction. We're pushing limits of distributed systems to automatically connect disparate datasets across millions of users, and extract predictive insights across billions of events.

We’re led by a team from Google & Optimizely, well funded by investors in Dropbox and Optimizely, and with customers in mid-market to public markets. We’re hiring our first engineers, where you would lead our architecture development across O(terabyte) datasets, build scalable API driver platforms, and design an automated machine learning pipeline.

Please contact us directly at bilal@clearbrain.com if interested!

True, but in an early stage startup (<10 people), you often have to perform the roles of multiple people, which in turn may result in longer hours.

Agree that the # of hours worked in a week is by no means an indication of productivity, and sometimes perhaps a product of a misaligned culture.

However, I do find long hours may result out of necessity for the sheer work involved as well. Especially in the earlier days of a company, you compensate for an initial lack of resources and staff by performing multiple roles.

On the business side, you could easily see your mornings taken up by customer calls (especially if you have an international market), the afternoons spent on ad campaigns and marketing content, the late afternoons on general office management. In the evenings when there's less client interaction, I would spend time on product feedback, some data analysis and metrics reporting for the day/week's past. This could easily extend to a 60-80 hour work week, and I would consider fairly typical of early startups.

As you grow there's less need for sure to work so long, as the company grows and responsibilities become more focused. But overall, I think there are circumstances where a 60-80 hr workweeks are necessary, and not necessarily an indication of a problem.

You are right that clinical trial studies would be much more time intensive and costly. The studies we will be replicating through the Reproducibility Initiative are more preclinical cancer studies, rather than later-stage clinical trials.

We felt the preclinical stage was more important to validate, as much of the research is based on academic studies with over 60% failure rate for replication (see http://www.nature.com/nature/journal/v483/n7391/full/483531a...)

By clinical trials, a lot of the pharma companies have already done initial validation, so extra validation is redundant. We hope a reproducibility system to validate preclinical research can help patient groups, foundations, and industry better identify reproducible oncology targets, and hence the focus on preclinical studies (which in turn are less expensive).