Edward is officially moving into TensorFlow | Hacker News Reader

Edward is officially moving into TensorFlow

discourse.edwardlib.org

213 pointspepopep44 comments

"Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilistic models, ranging from classical hierarchical models on small data sets to complex deep probabilistic models on large data sets. Edward fuses three fields: Bayesian statistics and machine learning, deep learning, and probabilistic programming."

http://edwardlib.org/

Thank you.

It's unfortunate that people often forget to include context like this in announcements. Thanks for sharing!

The announcement is on edwardlib.org (the URL shared on HN). Wouldn't that be an obvious connection? A project making an announcement about itself on its own website, seems natural to me.

Yes, but it's always nice when a project gives a tldr on their blog about what they are.

The announcement should at least include a link to an explanation on the main project page. Due to only being posted on discourse, there is no direct link to go to the main project page, only by manually editing the URL or googling for Edward. Even the logo in the top-left corner only leads to the discourse overview page.

Edit: Just editing the URL leads to an HTTPS error because the certificate is only valid for *.github.io; you need to use HTTP.

I agree the connection could be smoother to get there... shrugs not really something I'm spending cycles being concerned about.

Even more natural: Edward (probabilistic modeling library in Python) is officially moving into TensorFlow. But I see your point.

It could be parenthesized into the HN title but then stuff like that just gets edited back out :-D

What I meant is that if you have a major announcement about your project that you can reasonably expect to be shared, it's nice to give some context within the announcement.

I think this also applies to people on your mailing list. So often I get announcements sent out to a mailing list I signed up for where I forgot what the project is. Just having a sentence or two with a reminder goes a long way toward keeping me interested.

I totally agree. But in absence of that, I think tweaking headlines or adding parenthesized descriptions is good form, despite being against current HN policy.

I've had the pleasure of working with Edward for the last couple of months, and it's been the first PPL I've felt comfortable building a stable production codebase on top of.

It strikes a great balance between feeling like you're programming with probability distributions, and providing ways of diving under the hood to improve performance when you need to (like tweaking the underlying tf optimizer, or being able to implement your own distribution to use like a native one).

If such a library didn't exist, I would have needed to build my own. Congrats on the move to tf.contrib.

What kind of production stuff are you developing?

I'm a probabilistic programming enthusiast, and I had the impression it's still an open research field.

Time Series forecasting as a service. It's an area where careful treatment of error distributions matters a lot. There's a link in my HN profile, though the page won't be up for another week.

Absolutely! Great stuff.

I've only seen Edward for the first time today, and it looks quite nice.

Could you (or anyone else) comment on why would you choose Edward over something PyMC3? How does it compare?

I had the same question when I was first choosing libraries for my current project!

So I put together this comparison of PyMC3, Edward, and Stan. It's jut doing bayesian inference on the location and scale of data sampled from a normal distribution. Super simple, but highlights the differences between the libraries.

https://github.com/cshenton/normal-comparison

Having worked with Edward, the LOC difference is deceptive. Edward has really solid abstract base classes, so developing on it is much more expressive for larger projects. I unequivocally recommend using Edward.

That's useful, thanks!

Super interesting given that the main Edward competitor, PyMC3, was built on Theano which is discontinuing as of a few days ago.

Hopefully not a trend of everything being subsumed by Tensorflow.

“Everything being subsumed by TensorFlow” doesn’t seem to me like a particularly worrisome outcome. The status quo is “everything being a Python library,” and people generally seem to view that as positive. The PyData movement brought another level of interoperability to the ecosystem, so you can combine elements from Pandas, SciPy, sklearn, etc with good runtime performance and low cognitive load. Bringing everything into TensorFlow abstractions makes more optimizations possible (especially on heterogeneous architectures), and perhaps even more importantly, makes it easy to run end-to-end gradient descent on compositions of parts from across the ecosystem.

"Hopefully not a trend of everything being subsumed by Tensorflow."

Why?

Probably because there is an unfortunate trend when Google enters a product category: Everything around it dies.

https://en.wikipedia.org/wiki/List_of_Google_products

Are you really saying those are all monopolies in their categories?

I don't have anything against Tensorflow, but I've learned over the years that it's good to have competition in numerical libraries and software so there's replication of results. Try your analysis using one library, and corroborate it using another one. When a field is dependent on one major piece of software, it's more susceptible to bugs--a programming bug becomes a misguided line of research for a whole field.

Edward has been a promising addition to the PPL landscape. I actually preferred using it with Theano when I used it but that was a year ago, and it seems to have been developing rapidly. I have mixed feelings about this announcement, although to be honest I don't totally even really understand all the implications of it. In some ways I'm not sure how much Edward incrementally adds above and beyond TF; it has occupied a niche between something like TF and Stan or PyMC which is fine enough but I've sometimes wondered if it was sustainable in the long run. I have appreciated it being around, though, and have hoped it would continue to develop.

Who's the numpy competitor? Are we suffuring due to lack of one?

PyTorch, Theano (rip), TensorFlow itself, Numba/Blaze?

Who's the numpy competitor?

R? Eigen? Neanderthal? HMatrix?

(Yes, none of these are exactly 1:1 equivalent with numpy, but there absolutely are options. And from my point of view, having some options which aren’t tied to Python is healthy).

Thanks for mentioning Neanderthal! http://neanderthal.uncomplicate.org http://github.com/uncomplicate/neanderthal

It aims to have more features, and more speed than numpy, with Clojure, on the JVM + Nvidia + AMD + Intel.

Also relevant here is Bayadera, Clojure/GPU Bayesian modeling lib for the JVM: http://github.com/uncomplicate/bayadera

Thanks, I'd somehow missed Bayadera. Something else to take a good look at!

Bayadera looks very cool. Do you have any examples involving complex hierarchical models, and performance vs Stan or PyMC?

Yes. See https://github.com/uncomplicate/bayadera/tree/master/test/cl...

and

https://www.youtube.com/watch?v=bEOOYbscyTs

https://www.youtube.com/watch?v=TGxYfi3Vi3s

Thanks!

Disclaimer: I maintain this tool.

On the JVM, we're trying to do something like pytorch/numpy on the JVM. We also have python bindings:

https://github.com/deeplearning4j/nd4j

https://github.com/deeplearning4j/jumpy

We've been building this since 2014. Of note is we'll also be able to import TF, pytorch,.. in the next few months.

We're also an eclipse foundation project as of recently.

Numpy was the fusion of competiting numerical libraries for Python, and now it is quite stable.

I think point still applies for new developing software: multiple implementations can corroborate each other or help identify bugs.

Don't forget that numpy was initially a MATLAB competitor, and that we are all a lot better off for it :)

> it's good to have competition in numerical libraries and software so there's replication of results. Try your analysis using one library, and corroborate it using another one.

and then discover that they are all using netlib/lapack under the hood :b

It's built on top of Tensorflow, so it doesn't seem like a terrible idea.

That seems unlikely:

https://research.fb.com/facebook-and-microsoft-introduce-new...

That may be an act of desperation to counteract everything being subsumed by TensorFow.

Does anyone recommend a source akin to “statistical rethinking”, but in Python/pymc/edward?

That book’s balance between theory and practice is remarkable, but im no fan of R :/

Osvaldo Martin has ported (most of?) the book to PyMC3: https://github.com/aloctavodia/Statistical-Rethinking-with-P...

As a frequent user of Edward (big fan), I'm not sure I should be crazy about this - while it probably means that it'll get more frequent updates and love, I liked the researchy vibe that it had and its strong ties to sampling-based methods, etc. which might be lost in a move into contrib.

That being said, Dustin is probably working closely with Google people to do this properly, so it'll probably turn out ok in the end.

The statistician Bob Carpenter has commented that '...I just don’t think the Edward devs care much about doing practical MCMC.'[1]

Anyone got experience of using Edward for serious inference using MCMC?

[1] http://andrewgelman.com/2017/05/31/compare-stan-pymc3-edward...

Came here thinking Edward Snowden was joining the Tensorflow team.

Reply on news.ycombinator.com