If an agency caches the data but doesn't grep the data for anything related to me has my privacy been violated?If it's stored indefinitely (as they have the ambition to do, and will soon have the ability), it doesn't matter if it is not used today, or in the past. What if at some point in your life you run for parliament, or your business strikes a prominent deal with a foreign power? At that point your records can be taken out and scrutinised by whoever has access to this data at that point in time, and you won't even know, because it's secret. Information has a habit of spreading beyond the intended boundaries, and if it is collected in secret, we have no way of controlling that spread or controlling its use.
I would be a lot more worried if it was a US and not a UK agency.
You should be worried then, because this data is routinely shared with the NSA, seems to be collected under the remit of a joint program, and is probably stored indefinitely by several GCHQ partners where someone is judged to be of interest or enemy of the state/agency, the definition of which will of course vary with the agency.
I would also be a lot more worried if it wasn't GCHQ, but was a police force.
I'd be surprised if access is not given the police on some level, given their participation in counter-terrorism activities, and there's nothing to stop a secret policy change giving more access, you wouldn't even know about it. So you should be worried.
There are other databases in the UK that are a lot more concerning to me.
Given that those databases exist online, and are probably transmitted regularly as backups etc as well as accessed via the internet, you can assume that all that data is also accessible by GCHQ partners and clients. The important difference though is that, as you point out, these databases are public, acknowledged to exist, and controlled by our existing law. If you want to you can challenge the use or collection of the data in a court, sometimes you can stop using these services, or ask to see all information collected on you, but no such options are available for information gathered and shared in the name of security.
What I find troubling here is that the only limit on the indefinite storage of all this material by GCHQ is technical, not legal or moral, and that limit will soon be overcome.