There is no mystery or surprise here. It’s basic functional qualification. You buy or make a thermal chamber and cycle release versions of your device before you ship one. This isn’t some uncatchable mystery, they just didn’t test adequately.
Depending on the product size and cost you may also do this to every individual robot off the line. This is not uncommon.
This isn’t “hardware is hard” this is “we thought it was software with screwdrivers.”
When you run a hardware startup, you can only hope for an experienced team that would do everything by the book and implement best practices from the very first production unit. Reality is: that’s a luxury for most hardware startup teams out there.
Typically, there’s a frantic rush to get your device to market that you simply skip, or more likely don’t even have time to think about stuff like climate chamber cycling.
One thing I’m almost sure of: these guys have learned something — the engineer’s way. Good chance there’s a guy there googling climate chambers to ask the CEO for a budget to buy one.
And that, right there, is the difference between a company oriented around myopic management vs a company oriented around robust quality.
Any company trying to build a quality reputation would spec this stuff out AT THE BEGINNING — what are the operational requirements, what loads will they see, in what environments will they run, etc??? Then spec every component, and test the whole lot against those requirements. Sure, this is more like the dreaded "waterfall" vs "agile", but the result is a quality product from the start that has far fewer of these problems (because they did this whole test & fix routine at the prototype or Alpha test stages), rather than showing up with stories like this of how they recovered from customer-reported problems.
If you're telling your customers that they're the Alpha testers because they get early access, fine. If you're selling it as a finished product, then we know your company isn't prioritizing actual quality.
Sounds like Tesla... although it is public knowledge there that you are still alpha testers years after a model was introduced.
> When you run a hardware startup, you can only hope for an experienced team
If you run a hardware startup and fail to acknowledge that places like Alaska or Ontario exist, you fail before even getting close to merely inexperienced. The most charitable word I can think of is "myopic".
I suspect I would be disappointed by the performance of an iPhone that could operate in >125F temperatures (temperatures which I have worked in outdoors for several years)
I’m undecided if they lacked imagination. I work with a lot of electronics that go in oil wells and we have to make different models for different temperature ranges. We usually have to sacrifice a lot of functionality to gain high operating temperatures.
I think it’s possible Apple decided not to serve the markets which need high operating temperatures, rather than simply didn’t even think about the possibility.
It affected me, when I worked outdoors in Saudi Arabia and UAE and even Houston. But I probably would still buy a high performance, high battery life 13 Pro Maxover a hypothetical lower performance 13 ExtremeEnvironment edition.
Operating at 125F ambient would require a whole lot of sacrifice on the power envelope and much greater overall size of the product. I like big phones but my current Pro Max is about as large as I’d like a phone to be.
I was born and raised in a place where the annual temperature range can be 90°K. From -55°C in the winter to +35°C in the summer. That is not even an extreme range, there are well-known places with large populations (>10MM) that can experience >100°K differentials.
Perhaps more importantly, temperature changes of 40°K in less than 24h are not uncommon. Daily deltas of 25°K are experienced several times a year.
If a company builds hardware, and they don't factor in a routine impact of thermal effects, they really have no excuse. Choosing to ignore these can be a valid product development or marketing strategy, but not being aware of them is nothing short of myopic.
My point is that turning things that are unknown unknowns (from the point of view of the company) into known and checked for possibilities takes concerted effort, time, and money. It's easy to look at a problem after it's found and post-facto determine how to find the same problem quicker next time.
I agree that temperature is very basic, low hanging fruit. Especially for a device that seems to be aimed at operation by construction crews. But regardless of where you draw the line on testing environmental factors, you have to draw it somewhere. And so you will still end up with unknown unknowns that escape your QA or debugging process, sending you down the same path of needing to question your assumptions to figure out what's going on.
(Also you're not really giving them the benefit of the doubt here with this assertion that they didn't take temperature into account at all. It seems that they at least looked at the part temperature ranges. And it'd be courteous to assume that they did power dissipation and temperature rise calcs. What they didn't do was component or integration testing at varying temperatures.)
The difference is we did it before the units reached customers.
:)
It reads to me like they didn't properly spec/source components that were appropriate for the weather conditions these robots are likely to see. The fact that this issue was reproducible at refrigerator temperatures is even more shocking. 39F (taken from a photo) is not very cold.
Yeah that's pretty inexcusable. As a comparison, common "industrial" semiconductors are often specified from -40C to 125C.
Range Min Max
Commercial 0 to + 70
Industrial 0 to + 85
Automotive -40 to +105
Extended Automotive -40 to +125
Military -55 to +125
They wasted resources on fixing what they "thought" was the problem. That money would have paid for some nice chambers and other equipment.
Next thread: "We learned our robot doesn't work when sitting in a shipping container at -40C for three months because customs was being difficult."
Thread after: "Hardware is hard! We learnt about EMC compliance."
This is a general problem with all these California-based companies and inventors (especially SV inventors cranking out crowd-funded bike stuff.) They seem blissfully unaware of things like cold weather, water, dirt/mud, and road salt...or combinations of them. I laugh at all those stupid fucking delivery bots because they'll fall apart anywhere there's snow, and get completely stuck on the slightest bit of ice.
For many years, driving a Model S in heavy rain would cause water to get into the drive unit via either seals or vents that weren't sufficiently designed to keep water out. It "totals" the drive unit, causing corrosion of the motor control boards. And Tesla denies warranty claims on such repairs, because of course they do - just like they did on the windows that randomly shattered in parked cars.
Raise your hand if you've owned a car that had problems with water ingress issues affecting its transmission. Or windows randomly shattering.
What's that? Nobody? Exactly.
I have owned a couple of Philips bread makers. Basic ones and expensive ones.
If you make sourdough with them (ie bread) the coating is stripped off the bowl and the stirrer corrodes.
They will deny replacement and claim you sprayed something acid on it. Yes, fermentation is acid, but they don’t believe bread would damage their unit.
If they didn't account for temperature, they certainly didn't account for moisture, and these boards probably don't have any conformal coating on them. I guess its good for the customer that its a robots as a service, but these things are going to end up failing spectacularly in the field. These things are outside at construction sites, they needs to account for the environmental conditions.
That is a total waste of time in a startup with a small number of units shipped.
A component behaving out of spec due to temperature excursions simply isn't that common nowadays. If my system is mostly ADC to digital to DAC (standard for robotics controllers), testing for temperature is a waste until I'm shipping significant volumes.
There is a video of one of the slightly famous YouTubers who has a high voltage thing that fails at the altitude of his lab. The manufacturer did check it for function at the elevation of Denver, but his lab is higher than that. There are limits to how much engineering effort you should put in until you get an actual failure. (Maybe someone can link the video as I can't remember it at this point.)
You can waste infinite engineering effort covering all possibilities. Or you can ship the thing and fix the failures. "Good engineering" is about balancing the two--you need to ship, but you don't want to have too many failures in the field either.
A $5k temperature chamber running over night or weekend would have caught this.
You are right. People are commenting here as if the company didn't know what they were doing. I suspect there is more to the story then what has been shared in a tweet. My guess is that the deployment was done in a place with a temperature differential well outside the component specs/tests and so simply was not known/tested for.