Project Svalbard: The Future of Have I Been Pwned
troyhunt.com
troyhunt.com
I'd hope Troy reconsidered the "just create a business yourself" solution. That could be structured in a way that makes sure the trust Troy earned stays linked to the project. And a bootstrapped company starting from the profitable position I assume HIBP is in now (with the business deals) does not at all have to mean more work for him. He could just offload the work he can't handle anymore to employees.
An acquisition to anyone not as trustworthy as the current solution/the candidates like Mozilla mentioned here would be a disaster mid to longterm.
I'd be surprised if F-Secure made another acquisition so soon.
What makes Troy Hunt any more trustworthy? Do you think he can't make a mistake? What if his operation suddenly can't handle something because of X reason? What if he's breached himself or any of the services he's using break down or worse, provide invalid data or incorrect data? What if user Y searches his site, finds out they aren't vulnerable due to a missed data dump or data dump that isn't been loaded yet, then all of sudden gets compromised? Who's to say his employees won't screw something up.
Troy is right. He can't efficiently do this anymore. From the architecture I've seen, all he's doing is monitoring a twitter feed for new data. What if that twitter feed gets compromised and he just ends up uploading password? If he's dealing with millions of records, there is no way he could "manually verify" if every record was safe to upload, yet he claims he does manually verify them without much elaboration of the process...
Imagine allowing legitimate companies to upload their breaches to the site or maybe other security companies could upload data. It could be so much more accurate. Plus the extra hardware could handle the load and help verify the data being uploaded much better than the current operation.
I trust Troy Hunt more than I trust OP's examples of Facebook and Verizon. I also trust his competence more than I trust theirs. Whose to say that anybody won't make any of the mistakes you mention. FWIW I would doubt he would sell to either of these companies, but it's undeniable that you give up control when you sell and people have made incorrect judgments before.
Nobody is suggesting he continue alone, rather that, if he feels that they're the only two options, he take some venture capital instead of selling the business.
The main reason not to do this is the one that he's given: it may not be the best thing for him personally, and the venture capital plan may be particularly negative for him. I think this reasoning has a lot of merit.
He might not, but 6 months later the company he sells to *might. Or if public could be taken over with little choice in the matter.
As you say, control is gone once he sells.
Troy has basically said nothing about his manual verification process and he says its the worst part of the architecture. There seems to be no mechanism of removing your email from the list once it's added so he'll just keep adding/merging data I guess until it starts giving false positives. He doesn't have the infrastructure to make this scale into a reasonable utility. Even if he did hire employs, he's now delegating responsibility which will introduce new potential judgement holes into the process.
Simply put, it's too big for him. And it has nothing to do with trust. Venture capital is a crap excuse because now there's a profit motive for the service for something that should arguably be non-profit. Venture capital has a track record for producing several, high profile companies that make no profit for years and are compromised in themselves.
The best thing he could do is pass it to Mozilla or some other tech non-profit. It would be even better if it was a government service.
Remember, the founder of Facebook still runs the company. It got big, and look what happened.
Contrary to what you say, Troy has detailed his verification process. There also is an opt out form on the site which will allow you remove your email from the current dumps and future ones.
Troy is talking about somebody acquiring HIBP. This implies he is not necessarily looking to give it away for free. There are already paid aspects of HIBP.
I would have no issue with it going to Mozilla.
I can't tell if you're joking by suggesting he give it to a government or by comparing him to Zuck.
I guess we're at least agreed on Mozilla, who he is already talking to.
And what I was trying to argue is that we shouldn't put so much faith in one man. Whatever he does, it will, more than likely, not be feasible for him to control all himself. Especially with the legal ramifications of storing private data.
And I don't know why you think it's a joke to trust the government with something like this. We trust them with a lot more dangerous things. Considering it's the only entity that can compel a business to do something, it could actually work out if there was ever a law requiring breaches to be reported.
I met with Troy briefly for coffee about 8 to 12 months ago and we chatted a bit about this. I sensed his aversion to growing the biz back then. Seemed like he'd made up his mind. This post from him reinforces that. Even so I feel compelled to post a few thoughts.
Troy is an implementer. I was too. I was a dev guy who started as an ops guy. I really really wanted to build a business and for over a decade I tried to do it myself by writing my own code, doing my own ops, doing my own marketing and so on. It was very very hard, and after many failures and almost financially ruining me, I got to a place where I have an amazing biz and amazing team and I've turned myself into an exec who is no longer doing the day to day implementation, but is leading and coordinating.
This transition is very hard to make for folks like most of the people here - including myself. You have the sense that it's all on you. I need to repeat that in caps because that's how it feels. IT'S ALL ON YOU. I think this deep sense of accountability is what makes great devs and great ops people very good at what they do. But it also is perhaps what leads to burnout.
For an entrepreneur, it really is all on you. That work isn't going to do itself. And so that sense is even more visceral when you're a one man show. Now imagine you're running at the scale of HIBP. Pretty hardcore.
When I made the transition to being a leader and once I had a team behind me, the feeling was a bit like I'd imagine one might feel getting over a traumatic experience. It took a while. I felt like I could breathe again. I never wanted to go back to that place, if I have to be perfectly honest. It's a rough gig.
I think the trouble here is that Troy thinks that scaling HIBP is going to be more of the same. More of everything being on him, more work, more implementation, more accountability, more more more!!!
It doesn't work that way and I'm going to use my own path to growing a team (and regaining my sanity) to describe how it actually does (and can) work.
If one were to not sell HIBP and not raise money but instead grow it yourself into a business, it might work thusly:
1. Immediately work on developing strong cashflow for HIBP. Unfortunately this step is going to take some implementing from Troy. However, with good planning, you can probably hire some help and perhaps even do so in exchange for equity/options if you hire a good lawyer and can structure a cost effective deal. This stage is critical and I'd encourage Troy to get as much advice from other seasoned entrepreneurs as possible. Not folks who have raised VC, but who have actually created cashflow out of thin air. It's a dark art, but many of us know how to do exactly that.
2. Once you launch, it will take a while for the full revenue potential of the business to reveal itself. Cashflow takes a while to kick in and you will take a while to optimize it. e.g. many simply won't know that HIBP now has a paid option. That will take months, perhaps longer. So keep working and wait it out. I've seen this in every single successful cash generating biz I've created. At first it's a trickle, then a stream, then a river, then a wonderful fun and exciting deluge.
3. Once you can demonstrate that the biz is clearly going to grow into something with strong cashflow, you can start making your first hires. I would suggest hiring dev first. At this point you are going to have to do something very difficult. Step back from the coal face and trust your first employee. This was huge for me but thanks to Harvard Biz Review etc writing about this founder dilemma over and over, I was primed and I wasn't going to be the baker that can't get out of the kitchen. So I 100% delegated the job to an amazing person who remains with our team to this day. Once I could hire for ops, did the same. Rinse, repeat. Grow the team.
4. As your expenditures increase, you will need to be very good at managing cashflow. That is because at some point growth will pause. When that happens, if you don't realize that you will run out of money in X months, it will sneak up on you and you will lose the business. It happens every week around the world. Execs take their eye off the cashflow for a few months and byeeeee. Not everyone has the appetite for finance. Some are mildly or even severely allergic. I'm on that spectrum and thankfully my co-founder has a passion for it and happens to be very good at it. This has literally saved our asses and we too went through that growth pause. So if you are allergic, find someone who isn't. This is critical.
Once you do the above, if you build a team you can trust and you are very good at stepping back, finding and motivating talented people and carefully guiding the direction of the biz, things can get weird. You'll see a lot of executives talking about burnout, about how they work 20 hour days and the pressures of being a leader etc. But in your case you'll find that you have more free time and more mental bandwidth to shape the direction of the biz. You'll wake up one morning not sure what to do because you won't have a job anymore. You will have fired yourself from dev, ops, customer service, finance, HR, marketing, blogging and everything else. You'll go "oh shit, what am I supposed to do?"
The answer to this question is really fun: Whatever you and the business want to do. And guess what? You have a CEO who is the company founder and has a ton of energy and bandwidth to continue innovating.
That's pretty much the end of this post. I want to add a few more notes:
Delegating is hard for several reasons: If you're a dev and you have to delegate dev, you need to realize there are developers out there that are better than you and you will need to learn to trust them. You also need to understand that you're firing yourself from a job you are passionate about - a job you have loved and gotten very good at for many years. This is tough.
To scale a biz, you need to continue to delegate, even the things you love doing. Troy loves blogging and he writes epic tomes. But this too will need to be delegated if he wants to run at maximum effectiveness. I know. I did this. It was very hard. But I now have about 5+ writers in our organization and it's freed me up to launch a video podcast which I am already beginning to delegate to a certain extent.
VC is certainly an option, but know that each round you raise will also raise the bar on what success means. Right now you own the biz and success means a team that frees you up and cashflow that pays everyone better than market rate salaries. After the first round, a $20MM exit will be the definition of success. After a B and then C round north of $100MM will become success. And so it goes.
I'd also like to note that HIBP has built an incredible brand and growth. This is very hard to do. As Naval put it in a conversation I had with him not too long ago, it's lightning in a bottle, and I truly think that HIBP is a great example of lightning in a bottle. This won't happen again in Troy's lifetime. And what he has right now makes it very easy to: recruit, hire, retain, get help from other entrepreneurs, find customers, convince them to sign up, convince them to pay, get them to continue to pay, etc. The list of benefits is long. This kind of biz and brand is very hard to create. Troy's personal reputation is sterling and he's one hell of a nice guy. He is young, smart, healthy, well spoken. Seriously, you don't see this very often and it won't happen again, so choose your path wisely if you're reading this Troy.
And finally - and this is really why I'm writing this as a reply to onli's post - because I agree with their sentiment. Have no illusions that once you sell, you 'exit' in a very real sense. You are no longer the owner of the business. You are an employee. I'll also add that M&A folks are VERY good at selling the dream. I was recently at a certain multi-billion dollar company's offices who were trying to buy us. Their offices are based on Lake Washington up here in the Pacific Northwest. The M&A guy actually suggested that once we join their team we can ride to work in our boat. But in his defense, that's his job. Sell the dream. However, in this case I know the reality because I've been here before. Monday morning after you sell your company you will commute to work in a car, sit in a cubicle or office if you're lucky and you will do what you're told to do by the new owners of your business.
You will stare through those bars longing to roam the great plains once again as a free and wild creature in control of your own destiny. Or as Bodhizafa said in the final scene of the original Point Break: You know I can't handle a cage man!
I understand HIBP derives its value from grey-ish hats sharing with Troy any leaked dataset they find because they know him or because of his reputation.
If he leaves, it is not clear to me that his trust and reputation will stay behind with the company running HIBP. The minute HIBP ceases to be the central place for these new datasets to be shared, it ceases to be of any practical use.
Its quite interesting putting in various peoples email addresses to see what sites they are linked to. Maybe once he has made some money out of it, a GDPR claim and financial settlement can be made as he's made no steps to control the data privacy of Europeans.
I'm curious if my naive understanding of this is wrong.
It’s a grey area at the very least.
No it isn't. It covers my data no matter how you got it, with a few exceptions.
EDIT: Please feel free to point to the legislation showing that GDPR only applies to data supplied by the subject.
An operation like this levels the playing field and lets us collectively hold companies to their responsibilities.
"The regulation applies if the data controller (an organisation that collects data from EU residents), or processor (an organisation that processes data on behalf of a data controller like cloud service providers), or the data subject (person) is based in the EU. Under certain circumstances,[2] the regulation also applies to organisations based outside the EU if they collect or process personal data of individuals located inside the EU. The regulation does not apply to the processing of data by a person for a "purely personal or household activity and thus with no connection to a professional or commercial activity." (Recital 18) "
The EU laws apply to people and entities outside of the EU, he is not immune from these EU laws because he is affecting the lives of every European who has an email address in this website.
Wrong, wrong, wrong.
GDPR covers the processing of any data about an identified or identifiable individual.
Do you think GDPR fines go to the person, and not the regulator?
Administrative fines levied by a supervisory authority generally don't go to people who have had their personal data processed illegally, though.
Oh wait: https://www.troyhunt.com/free-course-the-gdpr-attack-plan/ https://www.troyhunt.com/new-pluralsight-course-the-state-of... https://twitter.com/troyhunt/status/1017679101698572295
He's made it pretty clear in the blog post that he intends to stay on and has acknowledged that his reputation plays an important part in making HIBP what it is.
OTOH, it's kind of difficult to begrudge Troy gaining financially from HIBP, since he's spent years building it up and has helped increase security awareness for so many people.
https://www.translatetheweb.com/?from=&to=en&a=https://t3n.d...
Though just realised, they're not that upfront about giving HIBP credit - If I were Troy this would peeve me a bit.
I think they discussed HIBP in the launch announcement: https://blog.mozilla.org/security/2018/11/14/when-does-firef...
It's also in the FAQ: https://support.mozilla.org/en-US/kb/firefox-monitor-faq
Was a little peeved at what seemed like a copy, but I have now realised it is just building on top of Troy's work [1] which is even better because of Firefox's larger reach.
They don't have to have a blinking marque text at the top attributing it to Have I Been Pwned. But they could have mentioned it on the front page somewhere that HIBP is one of their main sources. I trust HIBP, therefore, more value to Firefox Monitor had I known that link.
[1] https://www.troyhunt.com/were-baking-have-i-been-pwned-into-...
If you trust HiBP, you don't really need Firefox Monitor. It was created specifically to reach people that HiBP could not reach.
It makes sense to tie it into the Firefox Account password manager too. Mozilla could leverage Troy's close connections with industry to have Firefox as the recommended secure & open-source option for enterprise clients.
Something that hasn't been touched on as much is the limitations that come with contracts for large commercial companies. Side projects are often expressly forbidden. Yes, Google with give you 20% time to work on your own ideas, but you can't then upload it to your personal website and call it your own - it becomes company property and may never see the light of day. I imagine that Mozilla are more open-minded in that respect. They also have plenty of experience with remote teams, which would work well for his family/travel tradeoffs.
Please, Mozilla - if this opportunity is offered to you, take it.
Maybe an organisation not involved in advertising at any level.
But I, personally, would now trust Mozilla with this, were there to take ownership.
"Troy Approved!"
Mozilla is a good group, they've had missteps but I find them trustworthy and the combination would be pretty trustworthy IMO.
I don't know how much has changed since I left Mozilla, but there were, and probably still are, a number of former employees and volunteer contributors that had a great degree of influence and input on various projects.
Heck, Troy might even be a good potential addition to the Mozilla board of directors at some point in the future.
Since Troy will still be involved, hopefully he can steer things in a direction that benefits everyone - or at least warn the public otherwise.
Another weird thought is that it's the sort of "baseline infrastructure" that should be "governmental" to the internet. Unfortunately the closest I can think to an existing model for that is ICANN and that may not be something to emulate.
We should all do away with password complexity rules (except minimum length) and simply test a large, comprehensive exposed password bloom filter for membership. It's very fast (constant time) and efficient and if the test returns no, then it's safe for a user to select that password.
Here's the code: https://github.com/w8rbt/bp
Also note that this approach satisfies the updated (June 2017) NIST 800-63-3B password vetting guidelines.
IIRC aren't there actually privacy concerns regarding Proton? That may just be FUD.
Plus, I doubt they would maintain the level of transparency we've come to expect from HIBP. They don't seem very... transparent.
This is possibly a step by Troy to mitigate that risk, and given his position I’m surprised he didn’t mention that at all in this post.
It's not a repository or method of communications.
That's all client/requester side, which has been implemented on third party sites/services. There'd be a lot of questions raised if suddenly it required that you use a different technique.
A more subtle and (IMO) more realistic variant would be to backdoor the javascript to capture all input on that site instead.
But you have to ask yourself - who would be the government target, in that case?
They'd have to:
- Have a technically sophisticated target where the government doesn't know their password, and is unable to otherwise break their security (eg forcing Google/Apple/Microsoft/etc to do the work, cloning devices, regular surveilance) - Have that same target also regularly test their passwords against a password strength meter on the public webpage. - Be willing to risk a public leak that this was happening.
I don't think that anyone who meets the first point would be stupid enough to meet the second. I mean, sure, people make plenty of dumb mistakes - but surely not that one, repeatedly.
Leak monitoring would be a service provided by the UNS, not falling to a volunteer, and credential revocation could be automatic and immediate.
I suppose we sort of have that bolted on with OpenID/OAuth, but that's still 'choose a provider' rather than 'this is the one way', with many servers run by different entities, but one 'system'.
1) most people don't want their information public and searchable to that extent
2) most orgs _want_ to silo you in or otherwise control your account
3) the org using x500 still needs to have their own permissions separate from the central directory, which is the harder part of auth[nz], so just rolling your own authn is often easier.
I think you're absolutely right in particular with #2.
But if it had come originally, alongside DNS, 'everything has an address, everyone has an identity', it might've been an unquestionable fact of the internet.
Orgs can't silo you in to their alternate net where they have a more desirable domain name, because it's just not practical or user friendly.
I just think it might have been so for user identity.
Or a more independent company offering it as a standalone service, kinda like Mozilla (Monitor) or even something like Symantec (tho they seem to be bleeding money recently)
> I'll remain a part of HIBP. I fully intend to be part of the acquisition, that is some company gets me along with the project. HIBP's brand is intrinsically tied to mine and at present, it needs me to go along with it.
But for his own well-being over time he needs to delegate and divest himself as the single bus factor for HIBP. But that does not have to happen instantly and can be gradual without affecting the value of HIBP (in money and usefulness for us).
Whoever purchases HIBP also knows this. And as with most acquisitions, they eventually oust the founders. But for it to be successful it is after a long time when it has properly matured into an organisation.
Or Oracle, or any other mega corps that buy and nerf the usefulness of the product.
I went in the dead of winter, so might be a little different from what you might experience in summer. https://www.facebook.com/dheera/media_set?set=a.101010917929...
Note, my comment is not about Troy. Security-wise, I think the trust he carries is well-deserved.
I hope Have I Been Pwned goes to the right people and they do an even better job at moving it forward! Kudos Troy
He's realised he's the single point of failure, can't do it all himself, wants to balance work & family. Doesn't want the work/cost of hiring people and making a business.
So, he's preparing to sell it and there's a wishlist of what he'd like the new owner to do.
Did I get it all?
He did the same thing. Only instead of selling to hackers, he sold our hacked data to companies and governments.
It's legal to sell armor piercing bullets, but marketing them as "cop killers" will not fly.
One option that Troy could have done is to spin up a team / small company that would continue this project - with full control and guidance under his direction. That way, the trust that he has built from everyone at the community will be carried forward as the project progresses and matures further.
This will also allow visibility and transparency knowing that the people who would be working on this project will have access to him and everyone is on board on the direction moving forward.
Lots of companies / venture capitalists would be willing to support this cause which could provide the financing the project will need to be sustained and grow further.
Wonder if he couldn't just bring it in-house?
https://www.troyhunt.com/microsoft-regional-director/ - "I’m not going to work for Microsoft and despite the title of “Microsoft Regional Director”, I’m no more an employee than what I was (and still am) an MVP"
You sure? As far I know, he's an independent now.
>I won’t be working for Pluralsight in the traditional sense of drawing a salary, but rather continuing to work with them on a heap of new things.
EDIT: Serious question, generate hashes out of the leaked logins, store them in a blockchain and provide an interface for lookup via IPFS. Those credentials are considered burned anyway so storing them for ever in a blockchain won't matter.
Being in a blockchain anyone can access the data and use them for example on a registration page.
Especially if privatized.
Blockchain is Very overrated, but it could be useful in keeping data "safe" where the temptation would exist to index or obscure results. Especially where data collection and censoring / disclosure has value to certain markets, i.e. Timed/rated or delayed disclosure, sic.
IDK, it's not impossible, but it's not my wheelhouse either.
I don't see any reuse or value to old databases and hashes being public, so it's missing that purpose to exist or be used/shared. Like a lot of blockchain is. It's not enough to exist, it has to be shared and kept alive. I suppose.
Still, If you look at the way AV and user security is handled, there are potential vectors to prevent or anticipate, especially if the process of disclosure is censored or segregated.
Perhaps also if they proactively lean towards purges or spontaneous negative actions, in order to obscure their intent or actual content / behavior.
HIBP relies on disclosure, and if it were woven into a typical service structure, there would be a temptation to "alleviate" the workload for customers, offering to "feed the beast" with positive results and competitive, defensive tactics against 3rd parties offering a similar product.
Which could segment the disclosure process, so that you would have multiple options, much the way that AV and Malware is handled.
And now you have the same failures as AV and Malware being segmented domains.
The probability of a corporation being incentivized to airbrush a 3rd party listing in a semi-corporate "index" or offering "alternatives" to anxious, very large corporations to disclosure or remediation. Especially if they deal with financial or legal data, or specific disclosure requirements.
And have problems with timely disclosure, or any disclosure.
Imagine if a clearing house for disclosure existed as a Symantec or Kaspersky "Subscription", with tiers of access and disclosure prevention for corporate members, wrapped up in a daily routine app, such as a 2FA/Password manager.
So that a disclosure would be made silently by the subscription service, without disclosing details, or the level of breach, etc. The accounts or corporations breached, would just have their entire client accounts auto-reset and the updated password would be applied to your password manager within a batch process without the user(s), the press, the security agencies, or the hacker(s) being notified.
That, instead of revealing the time period, the hashes of usernames & passwords, or the name of the user, or their IDs, it would just be rotated on a regular basis, and invisibly managed.
Its a concept with some value, ie "paranoid" security features as a service, to prevent or anticipate disaster, sic. But handled via a handshake type batch process of cycling password management.
But this also has potential for occlusion and obfuscation, especially in examples where the breach would be a crime, or need to be disclosed to federal/state/police agencies, etc.
Thankfully, most security policy would prevent this kind of amorphous takeover, but for small businesses and large businesses, having access security taken away and handled by 3rd parties, for convenience, is inevitable.