Hacker News Reader

Comment by pilif | Hacker News Reader

I'm sure this was initially caused by the updates to crypt_blowfish (to fix the 8bit character issue we talked about a couple of months ago).

Interesting: Their build server has a failing test for crypt: http://gcov.php.net/viewer.php?version=PHP_5_3&func=test...

So I would assume that this should have been caught by running the tests before making the release. Or whoever ran the tests had them passing due to build artifacts of previous builds.

Or there are just too many tests that fail regularly: http://gcov.php.net/viewer.php?version=PHP_5_3&func=test... - when you have 201 failing tests, one more probably isn't going to cause any concern.

View on HN

201 failing tests is really shocking to me. Does anyone know of similar pages for languages such as ruby or python?

Python's test page: http://python.org/dev/buildbot/all/waterfall

And here's one with just Python 2.7 on high-priority platforms: http://www.python.org/dev/buildbot/all/waterfall?category=2....

Perl will not install if any of its tests fail after it's built. I'm surprised that PHP's test suite isn't run on the user's box after building it, allowing end users to "double check" the release process.

I'd imagine the majority of people are installing it via their distro's binary packages, so they're not going to run the tests anyway.

Of course, the packager for the distro should be running tests, and raising some serious questions when stuff fails.

In fact, it is. Just type make test. There are thousands of unit tests and it takes a long time. I generally run it and expect a 98% passing rate. If it's below that, I might have an issue. One or two failing tests will never get noticed.

It seems like they add tests for bugs before actually fixing the bugs. I can see that being useful, but they need to do a better job of marking which bugs are ignorable and which are not.

This is a an open source project - anyone want to volunteer?

Just because it's an open source project does not mean that your contributions will be happily accepted.

Nor does that mean you can't try.

http://rubyspec.org/ is the language "spec" that most ruby implementations of note, jruby/rubinius/mri, use to test that they are compliant.

I couldn't find a page with automatic tests running against each implementations head/master branch. But I do see crypt() like specs in there: https://github.com/rubyspec/rubyspec.git

Here's the CI for MRI (the major Ruby implementation): http://59.106.172.211/~chkbuild/ruby-trunk/recent.html

Reply on news.ycombinator.com