Hacker News Reader

Comment by mikewarot | Hacker News Reader

I have about 300,000 photos that haven't been scanned by AI (unless someone at Backblaze did it without permission). I'm sure there are lots of other photographers out there who miss Picassa, which Google killed off to push everyone's data to their service. (It did really well in matching faces, even across age, but the last version has a bug when there are multiple faces in a picture, sometimes it swaps the labels)

If there were offline image recognition we could train on our own data privately, could the results of those trainings be merged to come up with better recognition on average than any one person could do themselves with their own photos?

In other words, would it be possible for us to share the results of training, and build better models, without sharing the photos themselves?

View on HN

Absolutely possible.

What I'm building into PhotoStructure is typically called "transfer learning."

https://en.wikipedia.org/wiki/Transfer_learning

PhotoStructure is entirely self-hosted, including model training and application: the public domain base models (trained on huge datasets) are fetched and cached locally.

By design, none of your data (or even metadata) leaves your server.

(I expect to ship this in an upcoming beta next month.)

I want to label all the faces in the photos I've taken since 1997, and save them in the metadata. I'll be glad to run it against my photos. Windows 10, WSL, and/or Virtual Machine with Linux of your choice.

I've got desktop builds for macOS, Windows, and Linux, as well as "headless" builds for Docker and even "directly" via Node.js. Instructions here: https://photostructure.com/install

Nice! Will try this out. Are you planning on taking advantage of in-built neural engines like that in Apple M1 for speeding up object/facial recognition?

I'd like to, but practically speaking, I'm at the mercy of native support in the libraries I'm using. If support is added, though, it's trivial for me to add the switch as a user-definable setting.

Yes, you're talking about federated learning.

https://en.wikipedia.org/wiki/Federated_learning

> If there were offline image recognition we could train on our own data privately...

Apple does all face recognition and image processing stuff on the edge. On your iPhone or Mac.

I wondered why my phone got frighteningly hot while charging sometimes. Then I saw the note after adding some faces manually for it to recognize, which was in the line of "Your phone will update faces when the phone is charging". My all photos are backed up to iCloud, btw.

While possible, only the tech-savvy people would take part in this "collective", which is of-course a minor fraction of the data which Google has access to. This is the same argument as saying that if you care about privacy "just" don't use Google, easier said than done for the vast majority of people on earth.

I am not an expert on the field. But my hope was that this could be facilitated by Transfer Learning. Still don’t know how the scale economies could be achieved. Maybe just out of the sweat and network of passionate people like in the case of open source.

I work in the field. Transfer learning helps get you decent/good models, but the best models remain ones trained on large amounts of data. You may be able to get away with good performance and not great on your task. For some areas you really care a lot about long tail performance (like self driving) that you will need massive dataset. For other areas if your goal is to be the best relative to other large companies you will need a massive dataset.

Transfer learning best use cases are for fast prototypes or for ml tasks that do not need state of the art performance.

If there are hundreds of people with 100,000 photos each, that collectively is a massive training database, with a lot more labels and diversity of subjects.

By keeping the training data itself private, distributed and outsourced, you might be able to get otherwise unachievable levels of performance.

This isn't going to solve the data ownership issues though, since they contaminate the program trained on them (and its blackbox nature only makes it worse)... though I guess that specifically for copyright it's going to depend on the final usage of that tool ?

While interesting, we don't know enough about how models are learning to where we would be able to consider doing this.

There would still need to be a central model (and centralized management thereof) if I understand correctly.

Photoprism, digiKam, shotwell all have image recognition features, with varied levels of sophistication.

You don't encrypt your data before uploading to backblaze?

Oh heck no, I never encrypt data.

I run windows. It can't ever be secure, anyone who wanted to hack me could.

Scrambling the data really makes things worse as any accident requiring recovery of my data is also probably going to lose the encryption key.

The only time I ever lost any significant chunk of data (a persons lifetime set of photos!) was because Windows encrypted data at rest, and thus it couldn't be recovered after a disk crash.

Unless there is some corporate or legal requirement to do so, I'll never encrypt a whole disk, or backup.

Agreed. Everybody talks about encrypting backups like it's common sense, but almost nobody talks about the risks involved with failing to back up the encryption key itself properly. The entire integrity of the backup then depends on that sensitive piece of data, and it's not something that can be openly shared by its nature, or included in the encrypted backup itself. It's even deceptive if your measure for success is restoring the backup to make sure it works properly, because there is now an implicit assumption that the encryption key is still valid and undamaged the next time you restore.

I wish backup tools like Duplicity would warn you about the risks of encrypting backups instead of warning the user if they disable encryption, because encryption has the possibility of rendering all those backups useless when the moment to use them finally comes.

I have a similar feeling that large swathes of my digital life would be rendered permanently inaccessible if 2FA was enabled and my device was rendered inoperable. (That's why I keep meticulous physical backups of emergency keys.) I think 2FA and the like should be considered a tradeoff with its own inherit risks and benefits, instead of a universally better option than randomly generated 80-character passwords alone.

> any accident requiring recovery of my data is also probably going to lose the encryption key.

... why?

i'd hate encrypting too if I threw away all best-practices regarding it -- losing a key with the failed system is a "problem exists between chair and keyboard" type of issue.

Encryption protects your data from yourself, from your adversaries, from serendipitous grey-moral types, and from the prying eyes of over-zealous data-collection conglomerates.

You seem experienced in the field, so I won't presume what your best practices are -- but to be enthusiastic against encryption is a form of cheer-leading that I think I cannot ethically support; the longer I live and the more pervasive companies get to be with their data collection policies then the more powerful and required tools like encryption seem to become.

Reply on news.ycombinator.com