> If you can source it, so can othersYes, agreed: the longer the data is available, the more likely it is that others catch and start using it. That's a constant battle of course - continually coming up with new sources of data and new ways of getting it before most of the market.
> what makes you think only you have thought about trading on that data which must necessarily be public for you to have it legally?
I don't think I'm the only one who has thought about trading on it; on the contrary, I know there are many parties interested in trading on material information that is only technically public, but very obscure. However, after consistently profiting on this data, I can be fairly well assured that there are relatively few parties with the same informational edge, because the market consensus clearly doesn't price it in until it becomes explicitly widely known.
As I said, I don't need to have certainty that I am literally the only party with the information, so long as it is known by so few parties that the market consensus doesn't price in the sentiment. All material data has a half-life, which decays with time and market awareness. In some cases, the nature of the data or the methodology for sourcing it is so novel that it is conceivable to be the sole party with the information for a short and meaningful time period (where information means the exact same dataset acquired in a categorically similar way, not the downstream sentiment derived from the dataset). But that is a much higher bar, and it's completely unnecessary for profit. It's nice for pride though.
You'll have to forgive me for not going into any greater detail than that, both because any more than this is strictly a trade secret, and because I'd rather not have a competitor for my personal trading operations (or have someone turn around and sell this data, ruining my alpha in the process). The best I can tell you is to look into the analysis of satellite imagery, but with greater human ingenuity and a more specialized approach. That's not going to get you all the way there, but at least that much of the methodology is publicly known thanks to the old article about Walmart parking lots. You can't collect actual information about a company, but there is a lot of room for human creativity in finding strong proxies for that data that the rest of the market is mostly ignoring.