As I said in Rob's thingy, I hope you get the tenure committees and job committees, because they don't have to respect it but they're the ones you have to get to respect
* In the last three weeks, we've had 80 outreach conversations with various stakeholders for Distill. The majority of these have been academic researchers. The response has been extremely positive.
* A number of ML faculty at Stanford / Berkeley / Toronto / Montreal are very excited and supportive of Distill.
* Distill's steering committee consists of recognized leaders in ML and data visualization.
* We've registered with the library of congres / CrossRef, dotting our "i"s and crossing our "t"s to be a serious journal. In some senses, we're more legitimate than some notable venues.
* The largest industry research groups institutionally support Distill.
My sense is that the academic community really wants to have something like this, if it can be done well. At the end of the day, we need to publish outstanding content and demonstrate that we're a high-quality venue.
Yet I don't see how this will readily support possibly cutting-edge work or new research in machine learning that does not have access to visualization development, or these forged connections to Distill to facilitate the development of these visualizations.
So it seems like a likely outcome is that Distill publishes content from well-regarded institutions and increases publicity for that work, to the detriment of a vast bulk of papers which do not have access to the visualization resources to develop Distill-ed versions of their work.
Furthermore, and this is a larger disciplinary issue, but it seems inherently this could end up spotlighting more CS-y machine learning vs statistical learning due to cultural differences between disciplines and differences in computational/web development background in grad students and researchers in both fields. Are there efforts to reach out to statistical associations as well?
Anyway, as a result, I don't see a reason why an alternative-format journal would necessarily fare any worse than conferences have in terms of becoming accepted, if the reviewing standards are high and if it attracts citations.
For the hiring side (more than the tenure side), to some extent, oddly enough, the first-order decision here is in Google's hands. A lot of CS hiring committees nowadays unofficially do a first cut sifting of resumes by typing candidates into Google Scholar and looking at their Google-computed h-index, so what "counts" is basically up to Google.
With new things, what you need is at least one person on the committee to fight and convince the others why this new thing is awesome. As someone who is now on some of these committees, I would put all my weight behind something like this should I encounter it (assuming of course it has the relevant quality).