I think it is pretty clear from the paper that the average is over time.
The most detailed (and certainly not vague) description is given in "A Free Energy Principle for Biological Systems", Entropy 2012
> How for example do the 302 neurons of C elegans average?
They don't. An agent can optimize an objective without computing it explicitly.
> Free Energy does not make interesting predictions for neuroscience
What is your objection to my proposed prediction above, that we will find brain models with observable activity that follows the FEP?
Friston also has many papers that explain known facts in terms of FEP. I will grant you those are not predictions but they are consistency arguments.
> none of the progress in AI/ML has come from the Free Energy millieu either
Free energy is a dynamical version of variational Bayes, which has had enormous impact in ML/AI.