Tesla suffers network outage disabling vehicles’ mobile app
bloomberg.com
bloomberg.com
The Tesla app uses both internet and bluetooth to connect to the car. I would expect that even with no internet connectivity, bluetooth-related functionality would continue to work, and at least today, that is the case.
EDIT: After seeing dawnerd's comment, I just re-opened the app (didn't kill it, it was in background), and now the entry for my car is gone, which means all the bluetooth functionality I mentioned above no longer works. That's one hell of a bug.
[1] https://gist.github.com/darconeous/2cd2de11148e3a75685940158...
Networked software systems generally have completely unknown and unpredictable failure modes.
Do you really want important infrastructure like your automobile dependent on networked software?
When I took delivery of my car, the guy told me to just always keep the NFC card in my wallet. Honestly I thought it was more likely that I would have a dead phone than Tesla would have a network outage, particularly one lasting multiple hours, but I guess my phone charging is more reliable than their network configs. Given how unreliable and flaky I am, it's pretty damning of Tesla's technical chops.
As a comparison, my last car was a BMW. For that car, I had key fobs. I always had to take one of these key fobs in order to unlock and use the car. So going out, I would have one key with which I could operate the car.
Now, because I always carry the key card in my wallet, and I always have my phone, I always have _two_ keys with me. This isn't worse than having only one key, even if one of the keys is failing today due to an internet outage.
Essentially, I'm experiencing the worst case scenario right now, which is that I'm exactly the same situation as when I had my BMW. It's annoying (and as an iOS dev, I Have Questions on how the app came to be architected such that a network outage erases the app's settings). But I'm not worse off
But since Tesla's app can spontaneously stop working, might be worth sucking it up and carrying the card anyway
Never storing it in the first place. Too many apps I’ve worked on have been of the opinion that being on a phone means you always have internet access, so why not eliminate cache invalidation issues by just storing all of your data on a server instead of locally?
Is the automobile actually dependent on this infrastructure, or does that infra just provide some nice perks and extra features? If the car will still lock, unlock, drive, and refuel -- indefinitely -- without the network components, then I wouldn't consider it dependent on the network.
Of course, when adding extra features, we should strive to only make them dependent on the network if they truly require the network as a part of their basic premise. I suspect the Tesla has some extra features that could work fine without the network, but are probably broken because of this, and that's... not great.
"A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." -- Leslie Lamport, 1987 (https://lamport.azurewebsites.net/pubs/distributed-system.tx...)
Tesla strongly markets the idea that "all you need is the phone".
I know that my in-laws with a Tesla don't carry the key cards. They just use their phone.
So either Tesla needs to make a much stronger push for carrying one's key/keycard, or they need to fix the bug that makes the local bluetooth on the app stop working (or both).
Maybe some people will learn an important lesson here and start carrying their cards.
A lost, broken, or dead phone is a common failure case. If you know your phone is your key, then you should be considering what your backup will be. Since you already need to be carrying something with your license, it makes sense to keep a key card with it.
People may primarily use their phone because it's convenient (I definitely do) but that doesn't mean Tesla marketed it that way to me.
So it's necessary to have the ability to have a card on hand anyway, and at that point it's not much of a stretch to have a couple. Either via a phone case or whole wallet, but it's not like someone can (legally) just take their phone with them when going for a drive. Keeping the keycard along with license and credit card is still fairly minimal (phone digital wallet can consolidate a lot of fat card packs beyond those).
I guess the bigger risk is that while I always at least take my wallet with me in the car, I don't always take it with me out of my car while I always have my phone. Might be nice if Tesla had at least an option for some sort of fallback auth mechanism built into the car itself, be it keypad or biometric. The ones in phones are pretty good and are a pretty tiny item on the BOM, in a $35k+ vehicle seems like there should be room for it as an option?
Maybe that's overthinking it though, not like regular cars tend to have those either. Sometimes people just lock themselves out with regular keys too and that's that.
Are you actually sure you know that they don't carry the key card in their wallet? Why wouldn't they? I find this pretty doubtful, it just seems like a pointlessly unnecessary way to risk getting locked out of your vehicle if your phone happens to die or becomes otherwise incapacitated. What possible reason could anyone have for not just keeping the card in their wallet as instructed by Tesla associates upon receipt of the vehicle and the cards themselves? It just doesn't make sense.
Playing devil's advocate: what wallet? As more and more functionality is moved to the smartphone, the less reason to carry a wallet there is. A smartphone can already replace credit cards (Google Pay and similar), cash (several per-country payment systems, the one for my country will be released in a couple of months), and even the driving license! (https://play.google.com/store/apps/details?id=br.gov.serpro....)
In a more humorous note, I've seen many people use their cell phone as their wallet... by putting their identity card and cash inside the aftermarket cell phone cover on the back of the phone.
The phone and keycard are keys. If you lose them, you lose access to the car.
Also, I usually only use my car keys for one purpose. I'm using my phone all the time, for a variety of purposes. I've had my phone stolen from me when I was using it for one of those purposes, and meanwhile, my car keys were still in my pocket.
which you can also check by opening the settings menu on the car screen itself with a couple of clicks or, if you need just the VIN, the driver side door of the car (just like any other car). So, as the parent comment said, using app for this is just a remote convenience function.
There are plenty of other places where you can check your VIN number too, like the insurance card (which you are supposed to have on you at all times while you drive btw)/website or your vehicle registration paper.
The app not opening without network connectivity just seems like one of those, we can acknowledge that without conceding the whole concept and/or ecosystem.
The app is just an additional extra service, it isn't crucial for anything. All of those things you listed that are affected by the app not working can be looked up the exact same way they could be looked up on any other car, no app needed.
I agree with you overall though, it could be improved, but I am struggling to think of a better solution to this that they could implement, aside from caching the vin number and the software version on the client (which i totally agree with, would be a good solution for that specific issue). The rest of the functions like remote controlling the car from the app obviously won't work with that approach, but that's not a trivial problem to solve (if possible at all), and their current solution seems to be very reasonable (talking about them refreshing certs in the background once in a while).
Again, there are ways to deal with this stuff, but my point is just that the app is helpful in more ways than just remote control of the car.
I think they just did that stronger push, albeit involuntary :)
I keep my keycard with me at all times and it's come in handy when my phone died or the app was just being stubborn.
Carrying around the key cards just seems like taking a basic amount of responsibility.
I've been a Tesla owner for quite a while now. I don't recall seeing that in any marketing material. More-so being "strongly". It's pretty silly not to carry the key cards as a backup for the Model 3/Y (S and X use traditional RF key fobs).
They are also only $12 each! https://shop.tesla.com/product/model-3_y-key-card I carry one in my wallet and iPhone case.
Time to add some use cases to the acceptance testing script...
My Nest thermostats on the other hand require Google connectivity to change the temperature even when I'm connected to the home wifi..
Unfortunately it makes buying this stuff a pain because you have to research who owns what and what kind of control you have.
I ended up buying one which can be operated without an app, which there seems to be fewer and fewer.
This is essentially the Google philosophy, and it stands in contrast to the Apple philosophy. Yes, Nest is ex-Apple people and followed this bad pattern before their acquisition by Google, but still.
Apple HomeKit-certified devices don't communicate with Apple. They establish secure local connections and operate locally. For home devices, this makes good sense to me. It means, however, that HomeKit-certified devices have to be powerful enough to establish secure local connections, which the first couple of generations of IoT devices were not.
I'm willing to wait longer and pay extra to get devices that don't require a third-party corporate intermediary. Anything else seems unreasonable to me.
When I lose internet access, or the server is not reachable (and this is super frequent), my floor heating (my only source of heat) stops running, and ventilation starts.
Have to wait for someone to reboot the networked server, but they work only Mon-Fri, 10am to 5pm.
It isn't a common enough problem. Over my 1 year of ownership with almost daily driving, I only had to use the card twice. Once when setting up the car for the first time, the second time when I was dealing with valet parking (for obvious reasons).
It is basically a kind of a failsafe solution, just in case your phone dies or something like that.
And mind you, i don't live somewhere where my car gets constant network connection. In fact, the underground garage of my apt building where I park the car has zero network or cell coverage. It all works just fine.
You don't need to have active internet connection to unlock the car with your phone, it just needs bluetooth and an unexpired certificate from Tesla servers, which syncs occasionally in the background. But until that cert expires, it works perfectly fine with no internet. Never had such a situation happen to me before, despite me opening the Tesla mobile app only once in forever.
That's a bogus comment hovering on FUD.
In no general usage do you need app access to the car via the Internet in order to unlock or drive. My wife just drove one of our cars during the outage and didn't even known it was ongoing.
I would assume that even with the Model 3 using the phone as a key, it just uses LE Bluetooth -- not Internet access via the app.
Are there teslas without physical "key" cards of fobs?
I'd bet that a good number of owners don't always carry them around though.
A lot of newer tech products seem intended to move us to a place where we don't need to carry separate wallets/keys alongside our phones. That's a reasonable goal.
Look at how popular credit card phone cases are!
[1]: https://www.cbc.ca/news/canada/edmonton/tesla-driver-napping...
Tesla sells the fob for $150 which is normal for car keys.
It sells the keycard for $25 which is only $5 more than my office charged to replace a lost access badge.
No
I've seen articles of people using Java cards to create a Model 3 key card that works, but it wasn't possible to clone an existing card. They created a card, then had to use an existing authorized card (or phone) to unlock and turn on the car, then use the car's tablet screen to add the created card to the list of authorized cards.
I was unaware of the outage and did my normal routine obliviously and without ever opening the App. When I read about the outage, I opened the app and it basically gave me the middle finger. For shits and giggles I tried to use the car and it still worked.
So I'm not sure how the Phone talks to the Car but for me, on Android, the App losing it's mind didn't kill my phone's ability to drive the car.
I also carry my card with me at all times in case something happens to my phone. Seeing as how the replacements are only $25, I might pick up one or two just in case.
As other commenters have noted, the problem arises if your Tesla app needs to update its license or whatever this morning. If it did, you're SOL until they fix the bug. If it didn't, like apparently your wife's Tesla app, then you won't experience the problem.
If you live by the sword, you die by the sword.
I just now opened up the App to see if I was affected and it gave the usually "Waking up car" message before kicking me out to a splash screen that says "No Products Found\r\nJoin the Mission"(https://i.imgur.com/n34IKUc.png). When I try to view my account it displays an error (https://i.imgur.com/JnD6gev.png).
And with the App in that state I just checked and the car let me put it in gear. So even without my key(card) it still recognized me.
Idk what's going on but I wouldn't outright dismiss it as FUD.
(update 3:10 EST - app connectivity restore)
So I noticed the app would not connect earlier when I went to lunch so I just hopped in an drove to and from the store. I did use the app to open the frunk but I did not bother to see if it would sync up to the car.
However in the name of science I swiped the app off the phone so I had to restart it. Still could use my car. Now if the app is not loaded I cannot use the car. So I start the app and instantly I can use my car.
Now let me make it clear, at no time has the app forgotten which car I own. I am not sure how that falls off the app if it cannot connect? I am using an iPhone if that matters.
My Mac App, ValetforTesla of course cannot find the car as it needs to get back to the mothership.
On a side note, I had similar issues with my Volt's app when OnStar went on the blink but that car relies on the fob to unlock the car and drive off, all I lost was stats and ability to remote start.
What cars are you accustom to driving? I would say it's average in terms of visibility.
If I bought a car from anyone else, everything I bought the car with will be on the car when I resell it. I'm sure some people will tell me I just need to understand this exciting and innovative new paradigm pioneered by Tesla, but I'd like to think most people can understand how this is anti-consumer behavior and really gross.
Free DLC/software updates that do not have such a check would not be impacted by a loss of network connectivity, but thank you for raising that issue explicitly.
[1] https://electrek.co/2020/08/27/tesla-fbi-prevent-ransomware-...
Just one of a variety of conveniences is currently down.
It doesn't need internet access to unlock it, but it needs periodic internet access to sync this info.
Am I grossly misunderstanding something?
You need a way to remotely disable a phone key.
Imagine a scenario where you realize you've lost your phone, but you don't have physical access to the car to remove the phone as a key.
For example, you've parked your car in long term parking at an airport, then accidentally leave your phone in the airport bathroom and didn't realize it until you were already on the plane. Whoever finds your phone can now steal your car. If you want to use someone else's device to disable your phone's key remotely, the car needs to be able to connect to the Internet to sync the list of authorized keys.
That's not to say that this isn't a problem, but if the only way to fix it is to make access to your car dependent on Tesla's servers, that does not seem like a good trade-off. Especially since if the app can randomly stop working one day, car owners basically required to carry their keys too as a backup, and as discussed, those cannot be revoked.
Additionally, if you can disable your phone key remotely using someone else's phone...technically, a hacker who has phished your credentials can also disable your phone key? Scary.
Nope.
Is that really any different from someone finding your car keys? These days, with every car including a remote, if someone did find your key fob, they could press the panic button on it as they strolled around the parking lot to easily find your car. At least with the Tesla phone key, they'd still have to walk around and try to open up every Tesla on the lot, unless they were able to unlock your phone.
What's the point of having these features in the app if you need to carry a key around all the time due to the app features being unreliable and poorly implemented?
How does the card work if the electronic mechanism to read it fails? Do you then need to also carry around an actual key fob?
My car, like most modern cars, has a keyless entry fob, but if that fails or the battery in the car completely dies, there is still mechanical entry points to get into the car and start the car.
The point is that the key fits in your wallet. It's way nicer to carry, and then the phone gives you the convenience features that you'd usually get from a bulky fob, and much more.
The app is anything but unreliable; in fact the keyless entry and walk-away lock features have worked completely flawlessly for me for more than a year (very uncharacteristic of a feature that relies on Bluetooth). An app malfunction is way less likely than my phone being lost or broken or out of battery; those are the more important reasons to carry the key card.
If the car battery dies then the key card won't open the door. But the car also can't drive, and that seems like the bigger problem. Your ICE car also can't start with a dead battery, so being able to open the door is marginally useful. In both cases you need to fix the battery before you can drive the car. (Yes, there is emergency access to the battery without opening the door.)
Try selling crappy and expensive product with a promise of using the money and expertise to build a better one next time.
Yesterday’s battery day was also a promise to do great things in the future.
Autopilot is something that Tesla sells with the promise of one day becoming self-driving car.
Who else can do that? The others need to put their own money to bet on that and need to deliver good value products with reasonable quality for the money asked.
The BLE phone-as-key works about 95% of the time. Not good enough.
Some parts of the car feel cheap and insubstantial - the doors and front trunk, notably.
These are minor complaints. I agree that it is by far the best all-around everyday car available today. Unless you want a purpose-built 2-seater sports car or convertible, need the cargo capacity or ground clearance of an SUV/truck, or have some other special requirement, the Model 3/Y is a far superior car to everything else on the market.
Tesla cars are the worst-built, worst-furnished cars you can buy at their price points. If you care about comfort, ride quality, etc., go with anything but a Tesla.
But Tesla's batteries are industry-leading. If Tesla were to shed their EV-money pit, they'd easily earn their current valuation just by selling their batteries to everyone in the auto and power industries.
But by keeping their batteries limited to just their small share of the auto market, they've given their competitors reason to do their own research into battery/EV technology, and it's already starting to hurt them in Europe, where Tesla was outsold by other EV brands/models in several countries over the past few months, and that's without their major competitors having yet brought mass-market EVs to market.
I've only been in a properly made Tesla once. And they don't make Teslas that way any more (entirely by hand).
Some BMWs do look and feel like KIAs on the inside, especially their lower-end "beginner" luxury models. (That's not a knock on KIA, they deliberately target the low-end market but make up for it with relatively durable cars.)
And in initial quality surveys while Tesla is bad, other luxury cars are not that great.
JD Power, for example, reports that Tesla owners report the highest number of problems and complaints with their vehicles, which is supported by Consumer Reports' owner satisfaction surveys (but CR finds that Tesla owners "forgive" Tesla for these issues and have high satisfaction ratings despite the numerous problems with their vehicles).
CA investigations have found that Tesla has the highest number of cars of any brand that would be subject to lemon laws (meaning defects serious enough to require the dealership to take the car back).
The idea that Tesla is fundamentally a battery company is just weird. All of their battery chemistry, technology, and capacity is from Panasonic in one way or another.
The battery cell technology in current batteries is all Panasonic (i.e., the chemistry, cell container, etc.), but the battery assemblage is all Tesla. That's the stuff related to assembling the cells together into a battery, adding voltage controls, cooling assemblies, etc.
As a point of comparison: Porsche's Taycan also uses Panasonic EVs, and has even an even better lap-racing performance than any Tesla, because Porsche's battery assemblage includes extra cooling assemblies that allow the car to be used for extended periods of high performance. While a Tesla can do full speed for 1 lap, a Taycan can do full speed until the battery dies.
> Panasonic tolerates the existence of Tesla
Tesla is the biggest buyer of batteries on the planet and Panasonic should be beyond happy that they are the primary supplier.
> If Tesla sheds the EV business then Panasonic will no longer deal with them.
What? If Tesla just decided not to do cars?
> The idea that Tesla is fundamentally a battery company is just weird. All of their battery chemistry, technology, and capacity is from Panasonic in one way or another.
Tesla actually buys cells from Panasonic, LG Chem and CATL. All of these are integrated into Tesla owned and controlled packs that do the overall management.
Also, Tesla works much closer with Panasonic then you think. They are not just a costumer, they have a partnership with Panasonic and they make many decisions together.
Additionally Tesla has a large internal battery group that builds not just battery cells and packs, but actually they build battery manufacturing equipment. They bought the Canadian Hibar system that specialized in electrolyte filling and they are now Tesla exclusive. Tesla Grohmann Automation in Germany is building dedicated battery manufacturing equipment. Tesla has multiple agreements with research universities, including with Jeff Dahn's lab at Dalhousie University. They are part of the Battery 500 consortium. Plus they have a massive internal group on everything from cathode/anode manufacturing, cell design and more.
I agree that it is wrong to say that right now they are a battery company, but that it is 'all Panasonic' is totally wrong. Tesla will be a vertically integrated from cells to EVs.
Ride quality? The Tesla is by far the nicest, smoothest, and most fun ride I've ever experienced.
The in-car tech is pretty awesome too.
If you think Tesla is the smoothest, nicest ride you've ever experienced, then I feel sorry for you. Most luxury cars ride far smoother, and even my 10-year old Camry is a smoother, nicer ride than the Model 3.
Teslas are fun, if you have the opportunity to drive them somewhere you can go fast and safely accelerate. Otherwise, they're quite stiff with the handling.
Their in-car tech is nothing innovative, and as far as usability, it's far less usable than physical controls. I can control pretty much every function in my car blindly. You can't do that on a Tesla (or any other car with pure touchscreen controls).
Aww, thanks.
Maybe we're using different ideas on "stiff handling", but I don't think the handling is stiff enough. My previous car was a Subaru BRZ, which had super tight handling.
I've considered getting a stiffer anti-roll bar installed on my Model 3 to try to bring back a bit of the stiffer handling I liked on my BRZ, but there's only so much you can do when the car weighs 4,000 lbs. Can't exactly make a mid-sized 4-door sedan handle like a 2,800 lb 2-door sports coupe.
pretty much any automotive enthusiast journal disagrees.
Tesla did a lot of things right; ride handling & NVH are not their strengths.
I think you’re reacting to the media presentation. Even though 100s of thousands of people have perfectly fine operation every day, that isn’t going to make the news. The few breakages do.
This is so incredibly underappreciated.
Are their other charging networks? Yes, certainly. But they're either too slow for road tripping, they're spread thin, and/or they're expensive. I can drive my Model 3 from my home in Portland, OR to any anywhere in the USA and never spend more than 30 minutes charging, with most stops being 10-20 minutes every 100-200 miles.
I don't know what a typical supercharger costs as they vary on location, I just know that the one I've used is 29 cents per kWh. So to charge my Model 3's 75 kWh battery to full would cost $21.75, which would give me about 250 miles (It's rated at 299 miles, but you'll never get that unless you're going 55 mph with climate controls off), which is 8.7 cents per mile. If you assume gas is $3.50/gallon, then in terms of price per mile, it's the equivalent to a 40 mpg car.
Other charging networks are usually 40 cents/kWh or more, while charging slower. Some have a ridiculous charge per minute rather than charging per kWh.
Keep in mind that the current massive dip in gas prices is temporary and only caused by COVID causing people to drive less. Once COVID goes away (however long THAT'S gonna take) and people start going out more and taking road trips, prices will return to their $3+.
I bought my Tesla in October. I've had multiple people now ask me "Do you regret buying an EV since gas is so cheap right now?" and I think it's an incredibly short-sighted question.
Besides, if you're buying an EV to save money on gas, then you're either bad at math, or you drive over 1,000 miles per week, because the money savings isn't enough to offset the price difference between an ICE and an EV until you start to reach 200,000 miles or so, where the lower maintenance costs start to also become a factor.
Could you give any examples?
For comparison, I have a Kia Optima SXL, and I find the fit and finish to be beyond any Tesla that I have ridden in. Kia has a strange reputation (despite now being one of the best automakers), and even with this reputation, the car gets many complements and comments. I am also a big fan of buttons, and I simply find having buttons where I know where they're at and how they feel are far superior to navigating a touchscreen. Is the touchscreen cool? Yes, I suppose. Is it better UX or functionality? No, in my opinion. At no point have I ever wished to take all the functionality my car has and have it jammed into a single large touchscreen in the middle of the dash.
I have also ridden in multiple Chinese brands like NIO, BYD, and other more affordable brands. The NIO, upon initial impression, immediately outclassed the Teslas that I have ridden in (Model S and X) in terms of design, especially on the inside. The BYD and others that you find the DiDi drivers driving are all very impressive and affordable such that they easily compete with Tesla. When I was in China, it was "wait, what car is this?!" day after day. We must have saw half a dozen fully electric car models.
The doors unlock automatically when you approach, lock when you walk away. You only need one pedal to drive! Essentially all you need while driving this car is one pedal and the speed limit. THAT is excellent UX. Needing a ton of buttons and nobs is a distraction and unnecessary. This is why it's years ahead of everything else.
Also, I would argue that Nissan was first to release an good enough and affordable EV in 2011.
The concurrence for EVs is strong in Europe now, and Tesla is not the best selling car company anymore. See for example the Norwegian stats : https://elbilstatistikk.no/ That may change when Tesla will make smaller and more affordable vehicles in Europe.
Oh yeah, because we broke the Internet with NAT.
[1] Well, I suppose if you're not near the car, Internet would be needed. Does the car have a publicly reachable IP?
It just amazes me that literally everything has to round trip to the cloud for everything. It's like we forgot that things can connect to each other.
I believe internet is needed because tesla acts as a CA for your phone. That allows you to revoke a device, for instance if it were stolen. My guess is that the phone has authorizations that it renews fairly frequently and in this case apps could renew auths so once the current set expired you couldn't use the phone to unlock, etc. I can't immediately think of a better solution to the problem that allows using a phone for some functions and being able to remotely revoke a device.
If letsencrypt went down we'd similarly see a large portion of sites be inaccessible through browsers after 90 days even though you could directly connect to those servers
Having every car aggregated at one cloud system means if you got into that you could simultaneously attack every single Tesla in the world.
There is no good reason for this. It's just a mixture of "how things are done nowadays" and superstition.
From the Model 3 manual:
CAUTION: Always carry your key card with you in your purse or wallet to use as a backup in case your authenticated phone has a dead battery, or is lost or stolen.
Again, sure maybe people will NOT do this, but you cannot blame Tesla for this. Services fail all the time. The phone as a key is just a convenience. This same silly FUD went around earlier this year as well. Nobody got locked out of their car by a failure of Tesla. They locked themselves out of their own cars.
At what point does the manufacturer take some responsibility for their blatant lies to sell cars?
(Note, both my wife and I own Model 3's and I own $TSLA, so I'm no hater... I do carry a card key in my wallet, but I don't carry my wallet 100% of the time that I need to get into my car.)
Are you serious? Of course you can blame Tesla for this. Their car is a goddamn touch screen on wheels, and it doesn't function.
Hot take: I would rather walk than use my phone to unlock my car, not to mention having my car connected to the internet in the first place. If you're affected by this, you have no one but yourself to blame. This really is the clowniest timeline.