Also are event streams and large connect event graphs like Forge.AI what sells actually useful?
Curious as my phd research potentially has applications for information extraction and event linking. But not entirely sure if those applications are actually valuable
No, like he said the most valuable data has a signal that is independent from other data sets, which generally means something proprietary, and almost always without a long history. Having a cleaned archive of standard data with a long history is valuable but already pretty well served. It would be hard to compete with CRSP.
I’m curious about certain data I’m scraping for a project.
The most valuable data is data that is difficult to gather. Think things like proprietary (i.e. unpublished) industry data. The canonical "sexy" alternative data set sold to hedge funds is counts of cars in retail parking lots from satellite photos.
If the data is compelling and clearly correlates to earnings KPIs, I can tell you from experience that "some data scraped off a few websites" can be salable to the tune of $50,000/quarter. Hedge funds will frequently choose to pay that instead of setting up their own scraping operation, because scraping sucks.
Not all hedge funds, of course. Some do actively try to reverse engineer your dataset. But you probably don't want to work with those anyway.
> The canonical "sexy" alternative data set sold to hedge funds is counts of cars in retail parking lots from satellite photos.
I've personally developed forecasts from (ostensibly) public, scraped sources which beat drone and satellite footage of manufacturing facilities. That one Bloomberg article is not representative; satellite footage sounds sexy but it's not what most alternative data looks like.
Time spent on maintaining a scraping operation is time taken away from optimizing your ETL process and producing actionable research for your trading team. You know how people pay to have their pipes unclogged even when they know how it's done? Same idea.