1Password detects "suspicious activity" in its internal Okta account | Hacker News Reader

1Password detects "suspicious activity" in its internal Okta account

blog.1password.com

143 pointssprawl_122 comments

Seems like 1P took the right steps and is being transparent about the incident. It wasn't even an on their systems - but one of their vendors support systems. A lower quality organization would just conveniently not disclose the incident at all - justifying it by saying something along the lines of nothing was breached, it wasn't even our system. I think we should applaud 1P's transparency here. Or am I missing something?

> am I missing something?

This comes immediately after 1P's forced transition away from local app with local storage to Web app with cloud storage, and assurances that their security stance and practices would make a breach unlikely. If they had stuck with the old model, a breach would have no chance of impacting users, but now, we're left scratching our heads and speculating about the true extent of the damage.

I raised exactly this possibility with them when they announced their new model. Their support would not engage with this even as a possibility. Just assertions that everything would be completely secure.

Getting access to this data is the holy grail for attackers - it is preposterous not to have a local-only or "saved on iCloud only" model. Clearly the only reason they removed this ability was the juicy, juicy subscription revenue, which requires them to hold the data.

They may have avoided a breach this time but have they previously been breached? Will they be breached in future? The possibility of each is non-zero.

Needless to say, I'm still using the older version and am planning how to transition once it stops working after an OS update.

> juicy, juicy subscription revenue

The irony is that as a user since at least version 3, I would have easily kept paying a yearly subscription fee just for the same local+sync they had before centralizing. It’s clear that most tech businesses need stable recurring revenue in order to keep doing their best work.

They could have probably done an Amanda Palmer-style patreon (donations fund the ability to make all work public) for individuals/families and a straightforward high-cost enterprise subscription and been just as big if not bigger.

Good point. I would have paid too had they just asked.

> If they had stuck with the old model, a breach would have no chance of impacting users, but now, we're left scratching our heads and speculating about the true extent of the damage.

Well, since 1P clients are not open sourced, you always have to trust that they implement their white paper correctly, this is regardless before or after the transition.

Now, if you do trust them, then you should believe when they say that "IdP is only used for authenticate downloads of _encrypted_ secrets and the decryption only happens on device with a local credential", in which case a breach of IdP still would have no chance of impacting users.

I have a lot of rants about this transition, but the storage location of encrypted data is never something I worry about. In the past it was my personal iCloud/Dropbox accounts, now it's my 1Passowrd.com account. Am I missing something?

> you always have to trust that they implement their white paper correctly

Actually, no - if they implement their whitepaper incorrectly, and I manage to keep my insecurely-encrypted vault blob private, I'm still safe. Bad implementation is only a risk if there is also a data breach. This is defense in depth. Your argument is based on an all-or-nothing model of trust, rather than one where trust can be contextual and partial.

Would you be comfortable uploading your vault somewhere 100% public, rather than behind authentication with iCloud/Dropbox/1P, since it's safely encrypted?

A regular public audit by security firm would help increase confidence in a close source system. In fact, it would help in an open source system too.

It might possibly be a bad idea for everyone to consolidate all of the credentials and all of the auth flow mechanics for all of the things to a small handful of companies.

> It might possibly be a bad idea for everyone to consolidate all of the credentials and all of the auth flow mechanics for all of the things to a small handful of companies.

People have long lost the difference in meaning between "security" and "convenience". They now believe the two are interchangeable.

In the real world they often are—complicated-but-secure processes usually lead to work arounds that are worse than if you had just planned for convenience from the beginning. The classic example of this is the sticky note with the password on it.

Securing a large organization populated by regular human beings is extremely difficult, and is an exercise in balancing theoretical security with convenience.

I don't know how many people believe the two words are interchangeable (vs balancing factors), but one of your worst security nightmares could be when your employees fight against your security team. Making things inconvenience is one way to have it.

> People have long lost the difference in meaning between "security" and "convenience". They now believe the two are interchangeable.

Not sure they're wrong. There are so many IT departments and websites that force dumb practices which are detrimental to both: frequent password changes, required low-entropy recovery question options, etc. And then on the other side, some really convenient flows with reasonable security, e.g. streaming apps that show you a short temporary credential you can copy from your Roku's screen to your signed-in computer/phone rather than requiring you downgrade your permanent password to something easier to enter on the Roku keyboard. So while fundamentally you're right that "security" and "convenience" are in tension, in practice I think the bigger factor is competence and care of the dev and admin teams.

I’ve never met anyone who has made this conflation.

Okta and 1Pass are incredibly well designed and the companies do all of the right things when it comes to security and audit processes.

On the topic of security vs convenience, for someone looking to migrate from Lastpass as conveniently as possible, what are my options now that this event has ocurred ?

> The files the threat actor obtained in the Okta compromise comprised HTTP archive, or HAR, files, which Okta support personnel use to replicate customer browser activity during troubleshooting sessions. Among the sensitive information they store are authentication cookies and session tokens, which malicious actors can use to impersonate valid users.

I know that troubleshooting for pwms is hard, but leaving unencrypted files to access accounts on a server that’s not governed by the same threat-model seems very negligent to me.

Want to know how I detect suspicious activity in my password manager? I have a plaintext bitcoin private key in my password manager as a note. The name is 'bitcoin wallet'. It contains 0.5 BTC. If my password manager ever get compromised, I can reasonably expect the bitcoins to be move from that wallet address.

I then have a BTC node that will send me an SMS if those coins ever move.

Isn't that potentially a $15k detection method?

More like $8k net after taxes, anyways.

Doesn't the US have a tax free allowance for capital gains? In the UK for example you get a 15k allowance annualy

US military is quite impressive.

That needs to be weighed against the likelihood of a compromise (probably low), and the cost of such a compromise (probably quite a bit higher than $15k).

> 0.5 BTC

That's one expensive alert.

The passwords in my manager could potentially cause more financial harm than 0.5 BTC going missing. Everyone has their own price for security. I've also not moved those BTC since 2014 so the price has appreciated considerably.

The point is that there are far cheaper canaries to keep in your coalmine.

More succinctly, this is plain dumb. And especially to tell people about it in public.

No it really isn't dumb at all. Its the basis of a "honeypot".

Museums and galleries etc put their wares on show in public - can you be sure that what is shown is what you think it is or secured as you think it is?

Please don't describe anyone as dumb - its as much demeaning to you as it is anyone else.

High value passwords doesn't mean you need a 0.5 BTC alerting method, though?

You just went from "significant financial harm" to "significant financial harm, and 0.5 BTC".

The idea is anyone who compromised my password manager would likely go for the wallet first since it's as good as cold hard cash. Using the private keys and other secrets stored in my manager would take much more time for an attacker to exact meaningful value.

I would expect the BTC to be moved first and foremost which would hopefully give me enough time to mitigate any other damage that could be caused by the content of my password manager being exposed.

I think they would be more likely to copy all of the data first, in an effort to avoid detection methods like this, then make their move compromising everything in near parallel. At least that is how I would do it.

Would the average attacker, though?

It's a question about not touching an easy $15k, in exchange for a chance at a bigger score.

I'd assume most attackers wouldn't be able to resist securing the low hanging fruit first.

And even if there's a parallel move, it's even less likely they would leverage everything but the $15k, so OP would still receive a realtime indicator of compromise.

From a game theory perspective, it's a pretty compelling trap for OP to get what they want.

Speaking of game theory, there is probably a much lower number that achieves the same goal, though.

Your average attacker might be equally motivated to go for $20k, or $10k, or $5k. $1k, maybe not. $100, probably not. $1, almost certainly not.

There's an interesting game to play in minimizing the cost at no hit to efficiency.

I don't play the minimization game. In 2014 when I started this strategy 0.5 BTC was like 100 bucks. Now that it's 15,000 bucks doesn't make a damn bit of difference. If they spent the time to figure out what the rest of the credentials were worth and exploited them to the maximum extent, they'd be walk away a multi-millionaire. However.... 15k in a plaintext wallet is an easy score and I argue that the vast majority of people who could compromise my password manager would take that in a heartbeat.

What is “the average attacker”? If someone is compromising your entire password manager then that’s far from average and sophisticated

If OP is part of a bigger breach, those data dumps will almost certainly get analyzed automatically and multiple wallets swept at once. Passwords to interesting stuff likely aggregated and then tried. It’s not some script kiddy that browses through the vault 1by1

But OP's point is which will happen first in a breach?

(a) Trivially accessible Bitcoin is stolen or (b) passwords are used to ferret items/info of value out of additional individual sites

For OP's plan to fail, someone has to leave $15k laying on the table, in plain sight and for the taking, while they plan their subsequent moves. Which is why the amount matters.

I personally agree with your thinking.

An intruder will rifle through the top drawers and go for the obvious stuff and let's face it half a BTC is a bit of a shiner. You seem to be able to afford to lose it, given that its loss will trigger the shutters coming down and hopefully allow you to secure the rest of your stuff.

I get that and hopefully that is close to the last resort in your defence in depth approach to security.

I don't think it needs to be 15k in value to entice someone to steal it. You could also get compromised by someone who doesn't know anything about btc or misses that note and you would not know.

You could get the exact same alerting benefit with 1/10 the Bitcoin, or less.

There is no tax penalty for moving bitcoin. You should definitely move most of this elsewhere.

It sounds like you should be using 2FA with a hardware security token rather than setting up a honey pot that may or may not be triggered.

https://blog.thinkst.com/2023/01/swipe-right-on-our-new-cred...

https://canarytokens.org/

https://news.ycombinator.com/item?id=34476507

That sounds like an expensive notification system.

Is that a $15k circuit breaker?

would be just as effective with .05 btc

yeah, that is a lot of money in a developing country

That's a lot of money in any country.

$150 is barely enough to buy a piece of furniture here. Usually you'd be paying over 10 times that just to rent a small apartment. Not a lot of money.

0.05 BTC is $1500 not $150

Debatable - the objective is to play on an attacker's greed and convince them to go for the BTC before any other credentials. Too low of an amount and the other credentials in there might start to look more interesting.

Unless you're being specifically targeted because you're a high up position (ex. CEO) I'd imagine .05 BTC is plenty to attract the attention.

what's your plan if you receive that SMS?

Change the password to the account that has 5 BTC?

I just have a canarytokens credit card stored in it, if anyone tries to auth it I get an alert

That looks like a great approach indeed. https://www.canarytokens.org/generate

Yeah, I like to leave my Rolex Rose Gold GMT out on my nightstand when service people are working in the house to detect suspicious activity in my household. :/

Sorry man, I dunno if this is a weird flex or what, but it's kind of ridiculous to leave $15K of bitcoin as a canary for your password manager. Gotta call a spade a spade.

Yeah, I like to leave my Rolex Rose Gold GMT out on my nightstand when service people are working in the house to detect suspicious activity in my household. :/

i don't think that is a bad idea . It can be a cheaper one or a replica. The idea is it's a small price to pay when being deceived costs far more

It all depends on how costly the fallout from a compromised password manager would be - 15k can be totally reasonable insurance policy if the other credentials in there could give them access to multiples of that?

Why do 15 when $100, $250, or $1k would do the same trick?

Assuming the bitcoin is held as an ‘investment’ anyway, is it really that much worse for it to held on a password manager as opposed to an exchange?

I think he is being sarcastic. Emphasis on the “I think” part.

One should always use caution when using password managers with anything crpyto related.

Sounds like a great idea for a service that manages this automatically for users (but using a more reasonable amount of BTC, like 0.01 BTC or ~$300 worth-- it has to be enough to be worth stealing I suppose!). Then it would automatically do the monitoring of that address and send the user the alert that they should change all their passwords when the coins move. If it happens to just that one account, then its likely the user's account was hacked. If it happens to a bunch of accounts at the same time, it's more likely that the password manager service was hacked. Like a canary with a bounty attached to it.

This exact service exists, and has been featured on HN at least once!

Just spitballing on the idea of this being a service- Wonder if it would work for the canary-creator to also have a bot that watches the btc mem pool, and if it sees the .5 btc from this address being spent, it front-runs it with a transactions that has a much high transaction fee and directs the funds to a new safe wallet. Probably some risk of the bot failing to front-run, but otherwise it would have all the benefits of a $15k canary without the downside of losing all of the money.

Interesting idea. Would also be cool to try to trace back the IP of the first node that announced the transaction to the network so you could try to figure out who the thief is (assuming they aren't using a VPN).

a cheaper way would be to have a highly valuable token in the address without any of the chain’s native asset to pay for moving it

then just alert yourself when the native asset is moved to that address, because then someone is trying to sweep. your node can also send some some of the same asset faster at a higher transaction fee and move all of your tokens somewhere safe

people already do this

mostly as a scam to take the tiny amount of funds that thieves send to try to move the more lucrative bounty

you can take this one step further and have many assets worth sweeping, including assets that merely look like lucrative tokens. one of those is backdoored so that the transfer() function is nonstandard and transfers all the assets out of the attackers address when they try to move yours. or you can at least get just your own assets back if you want to be morally superior, moved to a safe address. this wont work if they dont take your backdoored token though. but all other parts about intercepting your assets before accepted into a block still would.

I don’t know if you though about this but the first thing a hacker would do after gaining access to your 1password vault is export all data.

Why not fill the vault with canary accounts and tokens instead? There’s services that do it for you.

If a SaaS is approximately as unreliable and insecure as self-managed software, the only reason to still choose it would be for liability reasons. You get to legally blame someone else if things go wrong.

I'm curious whether companies have faced this hard reality and decided that buying liability insurance + doing things inhouse is more economical & better for business.

I'm not a lawyer, but I don't think that hiring a SaaS provider shields you from any liability that you would otherwise be subject to. If 1Password were to suffer a massive data breach as a result of this, historical precedent says that there'd be no liability anyway, but if there were liability I can't see them getting out of it by blaming Okta.

Yah, this is why third party risk management is a thing. When I ran sec training, I always hammered home the point that a third party security issue is your issue.

Now, sure, technically there may be circumstances when you can technically/legally shift liability. But your customers don't care - they have the relationship with you. So the third parties problems, are your problems.

> only reason to still choose it would be for liability reasons

That’s not a reason. Haven’t you read any terms of service and user agreements? The vendor never accepts responsibility.

I'm more referring to SLAs and the like.

A base level of competency is expected as well. An SMB with a small staff that sells something non-tech still needs POS, payroll, and other systems and the ability to give employees access to those systems. “Outsourcing security” makes sense for businesses with zero IT staff.

For large companies, however, it seems like a liability, but I would hope an IdP would still be more competent, on average, then internal IT staff (obviously there are tech companies that have needed to deal with this for a long time with success). If a large business’s competency is not tech, there is some likelihood they can’t evaluate the robustness of their IT infrastructure.

> If a SaaS is approximately as unreliable and insecure as self-managed software

IF that were true. No way would it be cost effective at my company to try to internally reimplement 1Password's functionality though. I also would not trust it to be more reliable or more secure than 1Password.

This is an incident report from 1Password that I found more readable (PDF): https://blog.1password.com/files/okta-incident/okta-incident...

Yes that's much better, the original article felt mixed up.

So the culprit seems to have been the session information in the har. It made me wonder a few questions. What were they troubleshooting with Okta that required sending a har over, of their own interaction with Okta. And why are the session lengths so long, wouldn't Okta dogfood and use their own JWTs with limited lifetime?

Ok, we changed to that from https://arstechnica.com/security/2023/10/1password-detects-s... above. Thanks!

Whenever I see the statement "We found no compromise of user data." it reminds me of a meme that it could mean: "We don't have access logs, so we can't tell if any data was compromised." :-)

So it's finally happened at least a tiny bit: one of these corporations to which we have decided to dedicate all authority has had a breach.

Someday it will be much, much worse. Someday someone will manage to breach and take control of a bigger one in a bigger way, and will instantly gain root on a large subset of the entire computing ecosystem. There's a trend of even delegating things like ssh to systems under OIDC control, so I'm not using root metaphorically.

But hey, OIDC is convenient and that's all that matters in computing.

What’s your alternative?

A more portable standardized version of the Apple distributed Secure Enclave sort of thing as 2FA with passwords as the first factor would be great. You could also add something like a Yubikey as an emergency unlock token.

It’d be based on keys you control so there’s no way someone could hack some master database or key authority and own the entire universe. That’s a distinct possibility today.

Plausible scenario: high sophisticated nation state sponsored break at Google with cooperation from inside, used to launch a sudden mass malware infection attack against hundreds of millions of systems.

LastPass had a breach recently where entire vaults where stolen encrypted. Older entries were stored using worse (key derived using KDF with too few iterations) encryption than more recent ones.

Anyone else think it's odd that they scanned the laptop in question with Malwarebytes instead of, you know, a full forensic examination?

It's the go to for a lot of security teams (a fact i'm sure Malwarebytes wishes leveraged into more sales for their actual enterprise product) for routine stuff (random adware etc - no one can afford to run full forensics on all those all the time).

It's odd that they wrote that right out there on an incident report publicly shared and related to such a high profile potential breach though, for something like this it really has to be more of a 1st step triage than a definitive nope nothing wierd here...

I bet there are 100 other Okta customers who saw something similar and are on reporting on it - props should be given to 1Password to publicly talk about this. All this shows how interconnected the cloud world is and there is nothing called absolute security. You want absolute security -> crawl back into the on-prem world.

It was a minor incident, but it does remind me that centralized password managers seem to have an awful amount of concentrated risk.

Or is something like 1Password truly secure at its core, even if an attacker penetrates some layers of access?

Yes, 1Password's end-to-end encryption model (explained at https://support.1password.com/1password-security/) should be secure to this even if an attacker penetrates some layers of access. The model of other password managers may or not hold up though.

With that said, there is a lot of rebuttals to this that begin with "but, that assumes..." that I'm sure some of our fellow HN peeps will point out here :)

A bit light on details but seems "Requested a report of administrative users" was the main outcome disclosed which I assume means further phishing and attack vectors on 1Password admins.

Any other takes?

I’m confused why 1Password publicly reported this if there was no damage.

I'm encouraged by this, and the steps they took to implement additional precautions for the future. I would guess that was the goal of this release.

This is effectively best security practices IMHO.

Bingo.

I appreciate their transparency. I'm not a 1Password customer, but this earned some respect from me.

It's almost a standard practice now, most companies do this.

what's confusing?

1. because people want to know if their for-money proprietary password storage company got hacked 1. because if in the future they actually get owned, "oh yeah, it sorta happened another time also but we didn't say anything" is a terrible look

So a support system's support system is the culprit?

“found no compromise of user data or other sensitive systems, either employee-facing or user-facing”

Does not mean it didn’t happen

No, it does not. But one can’t assert something one doesn’t know. Therefore this assertion is as good as they can do.

Don't outsource something as critical as SSO to internal systems. The end.

Also, don't use 1Password or LastPass. KeePassXC, PasswordSafe, Dashlane, or properly-configured Bitwarden.

What action should a user of 1Password take?

Run

ensure you have some way to bootstrap your world again if 1password data is lost and perhaps consider adding canarytokens.org things to your store.

ie nothing has changed really

Gentle reminder: the absence of evidence is not evidence of absence.

To be fair, evidence of absence is close to impossible in the space of infrastructure and network security.

OTOH every tech CEO knows this and they always say "We have no evidence of compromise" right before they discover evidence of compromise

Full PCAP, process auditing and centralized logs are not only a thing, they have been for decades.

It just simply isn't worth the investment for CIO/CTO/CISO types because it isn't sexy. To say it's impossible is just factually inaccurate.

I know more than a few places doing 40gbps and 100gbps full packet capture for 30+ days. And relatively speaking, the investment isn't that large (for tens of petabytes it isn't as expensive as you might think).

We did this 5+ years ago at a managed hosting company, just for 3 days worth of data. Was still invaluable for figuring out complex events.

As someone who's entire job is to maintain a gigantic qradar cluster (IBM won't sell us larger licenses), I sure hope 1p have to logs to back their claims because I know that it is possible that they do.

> Despite what the expression may seem to imply, a lack of evidence can be informative. For example, when testing a new drug, if no harmful effects are observed then this suggests that the drug is safe.

https://en.m.wikipedia.org/wiki/Evidence_of_absence

Seems a very liberal use of the word “safe” unless I’m misunderstanding. It could mean either the drug is 100% safe, or that our methods of observation were insufficient to find the risk. Safe until proven unsafe and the class action lawsuits start, as it goes with many drugs.

Doesn’t seem like a particularly strong counter-argument, unless the point is that sometimes we humans like to err on the side of recklessness in the name of progress.

I think you are misunderstanding, the article doesn't just say "safe", the article says "suggests it is safe" (the "suggests" part implies not being 100% certain).

> , if no harmful effects are observed then this suggests that the drug is safe

or the harmful effects were missed, and the drug is dangerous

Honestly, it seems to me like we are heading back to a world where everything is self-hosted again. You can't keep a giant central target secure.

Yeah but self hosted means someone out of laziness will expose a port so they dont have to be home to sync or configure wireguard, etc… and/or since all ips are scanned anyway…

Complacency will result in more leaks and less knowledge of them maybe?

I reckon “passwords on a notepad in pen and ink” is safer plus passkeys like yubi.

If someone breaks into your home you got other concerns..

Companies don't force their cloud hosting solutions because it's good for users, they do it because they can make more money. Unfortunately I think things will have to get a lot worse before companies have to reverse course on this.

I actually think if a bunch of companies started hosting their own SSO, we'd hear of a lot more hacks. I'm not sure orgs would put in enough resources to do things properly other than "hey we got keycloak working"

Maybe SSO as such is not the greatest idea?

random IT departments won't do a better job securing IDP than google or microsoft or whatever, self-hosting that stuff will just lead to more, mostly smaller breaches.

simultaneously, Okta seems rather bad at their job of not getting hacked and having proper fucking audit logs

The owner of this website (www.bleepingcomputer.com) has banned your IP address.

Anyone else?

Not me. Are you on a VPN?

This has specifics: https://blog.1password.com/okta-incident/

> Original post by 1Password: https://blog.1password.com/okta-incident/

So, a company that is supposed to protect your most valued secrets and therefore should have paranoid security is using an external support system without any 2FA, and with fully unprotected session tokens/cookies, which in addition appear to have an insane timeout (how else could someone re-use the HAR files?).

Wow.

In general I would regard anyone using a password manager that uses a cloud service and/or phones home to be unreasonable. But even if you believe that this is a good idea, at this point everyone should drop 1Password as they clearly do not have the competence to run such a service.

Reply on news.ycombinator.com