Hacker News Reader

Comment by otalp | Hacker News Reader

> For instance, in a series of three consecutive tests, the 13- inch model with the Touch Bar ran for 16 hours in the first trial, 12.75 hours in the second, and just 3.75 hours in the third. The 13-inch model without the Touch Bar worked for 19.5 hours in one trial but only 4.5 hours in the next. And the numbers for the 15-inch laptop ranged from 18.5 down to 8 hours.

What could possibly the reason for such drastic variations in the same laptop, over a short period of time, when the testing conditions remain consistent?

View on HN

When they say: "ran for 16 hours in the first trial, 12.75 hours in the second, and just 3.75 hours in the third" they don't mean that each of the three trials consists of the same load.

The mechanism sounds highly random too: "Consumer Reports tests battery life using a real-world test: They turn on the laptop's screen and surf webpages using Chrome. It's not a test based on benchmarking software or watching video that wouldn't require internet access, and the publication said that even recent software updates did not help the problem."

Such tests should be repeatable, with same webpages, durations of visit, etc to have any meaning (and to be able to actually indicate issues with different battery life, and not merely different load).

Doesn't seem fair to base a test on using Chrome when Apple bundles a more power efficient browser with the OS.

Read from the orginal source[1]. They do their laptop tests with default browser, Safari, and got those highly variable results. They then did tests with Chrome and got more consistent results (but still lower than Apple's promised estimates)

[1] http://www.consumerreports.org/laptops/macbook-pros-fail-to-...

This is where it gets more bizarre. They did not use Chrome. According to their website: "We conduct our battery tests using the computer’s default browser—Safari, in the case of the MacBook Pro laptops."

They proceed to find serious aberrations with battery life under Safari.

Then they say, "Once our official testing was done, we experimented by conducting the same battery tests using a Chrome browser, rather than Safari. For this exercise, we ran two trials on each of the laptops, and found battery life to be consistently high on all six runs."

Chrome giving much better battery results than Safari? That is unheard of. It's not a software issue either. I can confirm that with the latest Chrome my Air runs for about a third of the time that it runs with the latest Safari(I have the most recent Sierra).

So this makes very little sense.

Agreed, when I read they got longer results with chrome I did a double take. It's never even been close in my experience so clearly they screwed something up in safari.

Any issue fixable by software is a software issue.

Haha, you've not written many hardware drivers I imagine. Sometimes a software workaround is viable, but that doesn't mean the root cause is not a hardware problem.

That's a crazy high level of variation to blame on relatively small differences in tests.

It sounds like a bug in Safari causing the cpu to kill the battery. Possibly a race condition since it's random. Guessing this will be fixed in an OS X update shortly.

If that were the case it would affect all Macs - it must be some combination of Safari and the new hardware, which I wouldn't be at all happy with if I'd just spent a ton of money on a new Macbook.

>it must be some combination of Safari and the new hardware

Which still doesn't mean it has anything to do with being a hardware bug.

E.g. could just be related to the Safari+Touch Bar combination.

Variance was noted on MBP Escape model (ie, no touchbar).

Do we really know it doesn't affect all Macs? Have they tried to run their exact same tests with the old MBP and the latest OS updates?

From their website:

"Consumer Reports tests hundreds of laptops each year, using identical procedures in highly controlled conditions.

For the battery test, we download a series of 10 web pages sequentially, starting with the battery fully charged, and ending when the laptop shuts down. The web pages are stored on a server in our lab, and transmitted over a WiFi network set up specifically for this purpose. We conduct our battery tests using the computer’s default browser—Safari, in the case of the MacBook Pro laptops.

During the tests, we set each laptop screen to remain on. We use an external meter to set the display brightness to 100 nits—a typical level you might use indoors or out. And, we turn off any automatic brightness adjustment in the laptop’s settings."

While not explicit, it does seem to imply that they conduct the same test multiple times.

OK, now that I've read more than the derivative post, it seems that there's a software glitch in Safari.

They say at the end of the test they use Safari, but got much better results with Chrome:

"Once our official testing was done, we experimented by conducting the same battery tests using a Chrome browser, rather than Safari. For this exercise, we ran two trials on each of the laptops, and found battery life to be consistently high on all six runs. That’s not enough data for us to draw a conclusion, and in any case a test using Chrome wouldn’t affect our ratings, since we only use the default browser to calculate our scores for all laptops. But it’s something that a MacBook Pro owner might choose to try."

This actually bodes well for battery life once Safari is cleaned up. Safari is a much better energy consumption citizen than Chrome on other Mac laptops, and it's very unlikely something about this new hardware will reverse that relationship.

Tests like this are always a choice between two problems, either it is artificial, can be gamed and doesn't represent real world usage, or it's not repeatable and insufficiently controlled. At least one of those criticisms always apply.

First off, the standard test uses the default stock browser, aka Safari, not Chrome. They switched to Chrome as an additional test and found the battery times to be far more consistent. Sounds more like a software issue to me than a hardware one.

Background processes and vastly different webpages visited. That's all I can come up with.

Frankly, I'm not sure either is a valid excuse for such a huge deviation.

If they have some automated setup for doing this web browsing (maybe a selenium that visits websites for 10 hours?)

They should be consistent after a restart and I bet they charge them to full before conducting the next test.

Reply on news.ycombinator.com