It depends on what you're evaluating it for. Active contributors is generally the most important metric IMO, but there are a lot of projects that break that rule too.

More often than not I see these questions asked in the context of comparing Open Source solutions to B2B products with integration and continuous support contracts; you cannot really "evaluate" Open Source with expectations that high. Getting things to work well takes elbow-grease, and you can't expect free work to materialize and solve every problem for you. You should go into most of these Open Source projects with the assumption it will take some dedicated research on your part to get it working properly.