What is closed about kaggle that this does differently? I've always viewed that community as extremely open, even allowing hosts the option to open source solutions or not, thus attracting more challenges and competitors.
The about page [1] for this project says,
> With EvalAI, we want to standardize the process of evaluating different methods on a dataset
Standardization often does come from openness, I'm just not sure juxtaposing against kaggle is the right move.