The Best Way to Organize a Lifetime of Photos
wsj.com
wsj.com
It's worth paying for the e-book as he goes into some interesting points, but in a nutshell the core concept is going through your imported photos in three passes: first, quickly pick all the ones that might have some potential merit (e.g. they are in focus and of something somewhat interesting), then go back through the picked ones and quickly rate them from 1-3 stars (1 being "has documentary value", 3 being "potentially a great photo), then take the 3 star ones and spend a bit of time retouching them if needed and giving them 4 or 5 stars if they are really special.
The end result is that you can quickly get down to keeping maybe one in ten photos from a shoot - it makes the whole process feel a lot more manageable and opening Lightroom less scary! Plus, it reduces the amount of photos we store so you need to worry less about the capacity of your cloud solution ;)
Are you a photographer? I'd love to be having that much success, if I end up with three good photos out of 500, I call it a good day.
a) Judgemental - what you might class as a success can differ wildly from what someone else does.
b) Situational - the success rate you get from differing events, shot under different conditions with different equipment, technique, time constraints and expertise etc. etc. can vary hugely, even when judged impartially.
About the only time i ever feel like i can objectively judge a day a good one, is either
1) When i've been to the same event as any of my peers, using comparable gear and felt like i've held my own.
2) Achieved whatever it was i set out to, be that a great picture worthy of putting in a portfolio or sticking on the wall, a satisfied client, a new technique or lesson learnt, a memory captured or just enjoying whatever i did on the back of the excuse i was out with my camera.
I wouldn't worry about the numbers, worring about the numbers would be when you got roped in at the last minute to be the main photographer at best friends wedding and all you had was a kodak brownie with half a roll of film left in it anything nowadays would be a cakewalk in comparison.
Yes, don't stop pushing yourself, but at the end of the day, for me anyway, it's the experiences that count, the pictures are generally just reminders of things i've forgotten, can no longer experience or because the people in them are no longer with us.
Basically everybody has different goals and ambitions with their photography and comparing how many 'good' pictures you get is pointless since everybody defines 'good' so very differently.
Heh, notice I said "photos", not "good photos" ;) Really it depends what kind of photos I'm taking, if it's a set of photos from a holiday then there'll probably only be a handful I would consider sharing publicly, but quite a lot more that I'll want to keep for documentary/sentimental value.
1) After rating the pics (I use color tags in OS X), I copy and paste them into a tmp folder on my desktop.
2) I run an Automator script that took 2 mins to setup and scales all the images to something like max 1024px in any direction.*
3) I drag them into a shared iCloud library
* Why do I resize via automator? It makes the files super lightweight and very quick to appear on all our devices. I still have the originals if I ever want to print, but otherwise this resolution is almost always plenty big enough.
For a couple of years I've been paying SmugMug to host my galleries, but recently finally got annoyed: they are great for pro photographers (who sell photos), but their goals are not aligned with mine.
What I need is fairly simple:
* online site, available at a specific (ONE) URL, that my entire family can bookmark (ONCE),
* login/password required to enter, and I want multiple login/passwords,
* HTTPS (duh),
* no borg-minds harvesting my photos for advertising data, or doing face recognition,
* good image quality, fast and responsive, both on mobile and desktop,
* single gallery index, automatically ordered by date,
* gallery date ranges extracted automatically from photos,
* easy adding of new galleries, preferably just by uploading a directory with a bunch of images exported from Aperture/Lightroom/Photos/Whatever,
* simple text-based search that finds galleries.
Google Picasa, Flickr, SmugMug, whatever you pick — they all fail at this simple task (for various reasons). And some people even use Facebook to share images with family (sigh).
I got so annoyed that I finally wrote my own solution. Which works. Costs me 6EUR/month for a Hetzner.de virtual server, but I can finally have all of the above "features" without any extra annoyance. Plus I get significantly better image quality than any other solution I've seen (mozjpeg kicks ass).
And I don't depend on anyone, in particular on any company's whims or "strategic cutbacks", nor am I "the product being sold".
I find it amazing that it's 2015 and I'm writing software that manages private online photo galleries. I think it doesn't speak well of the current business/tech landscape.
(Incidentally, software to manage your photo collection is a whole another story. After Apple screwed me over with Aperture I am thinking hard about what to do about my photo archives.)
P.S. Any chance you'll open source that software if there are other people in the same position (or create a closed source SAAS product)? P.P.S A product I've found that come close to what you need is iCloud shared photo streams. Did you look at that or does it fall short in some areas?
A lot of your concerns are shared by other people in the free software community. Ideally the solution to these kinds of problems would be decentralized, as a centralized server creates some issues: a) it requires you to acquire and maintain a server and b) it is susceptible to abuse[1]. But there are some technological issues remaining to be solved before decentralization is really available. I think federated systems like this--where multiple people connect to the same server but anyone can spin up their own server--are be a good stand-in for the present.
[1] https://www.gnu.org/philosophy/who-does-that-server-really-s...
You sound like exactly our target customer, so if you've got some time, I'd love to understand what these things are. I'm sure there are nuances here that I'm missing or not fully understanding. Thanks!
I just checked and here's what I wrote to you in our support discussion from 2013 (when you introduced Sharegroups, which I thought would solve my problem):
> "I'm not sure you thought this out — I can't be expected to manually add every new gallery to the sharegroup (which is not an easy process if you have >50 galleries), and then rearrange them manually (something I already did within my folder anyway) using the old interface."
There are a number of problems with SmugMug. The most important one is the amount of work I have to do to add each new event (gallery). I had to fight the category system (I don't even want categories), remember presets (will that one work?), manually rearrange galleries, and perform a number of other chores. And there were minor annoyances, such as that at the end of it all, SmugMug still insisted on placing "Powered by SmugMug" branding on my pages.
For comparison, so that you understand, these are the steps I have to take right now (with my software), starting from a folder full of images exported from Aperture/Lightroom:
* run a script that converts/resizes/prepares images,
* upload the whole resulting directory to my server.
That's it. There is nothing more to do. Galleries get sorted automatically, date ranges are automatic, featured photo is chosen either randomly or based on a "featured" keyword. Galleries are named after directories by default. Galleries are reverse-sorted by date. There are no categories. Family and friends can log in, view pictures, search galleries. Oh, and it also supports retina images, and works very well on mobile devices.
I think if you want to address use cases like mine, you have to realize that those photos are not the center of our lives, they are just a tiny part of life. I am not a pro photographer obsessing over how I want to categorize my galleries and how I will sell my pictures. I want to spend the absolute minimum of time dealing with placing them online.
Also, I love the idea of being able to host the content myself. It might be a little more of a hassle, but then you don't need to worry about the service shutting down or removing features. Making your code available to others would be nice, but if you have a decent implementation it's certainly something people would pay for as well.
As for hosting and business side of things, I thought about providing a service like that, but decided not to. It's a tough business: I don't want to deal with people posting dick pics or other kinds of porn, I don't want to go to jail because someone posts child porn to their account. And I definitely can't match the support level that companies like SmugMug provide. That business is mostly about support.
I might release the software for everyone to use. It's not that complex anyway, you just have to make the right architectural decisions (this is what is really hard). I would probably have to change the thing so that it does everything on the server side, because right now image processing is done on the client (you run a perl script, then upload the directory that it produces).
And to quickly answer those suggesting the GPL — if I do release it, it will not be the GPL. It will be one of the more permissive licenses. I don't want to block people's freedom to do anything they want with the software, and I think the threat of companies "taking stuff closed-source" is no longer a threat at all, because it makes no economic sense to do so. So if I do get around to it, it will be MIT or something similar.
What could actually work well would be an Amazon AMI image, a Docker container, or something similar, that would be very easy to host.
Can you talk about the technical details of your setup?
On the client side, there is a perl script that uses graphicsmagick and mozjpeg. I intend to roll that into the server-side eventually.
I believe the dev team behind it also have a commercially-hosted offering called (maybe) Trove?
I have a simple setup that works well for me: I set my Android phone to not upload photos to my OneDrive Camera Roll unless I am on a wifi connection, and I turn off wifi on my phone when I am hiking, at family dinners - anytime I am taking a bunch of pictures. When I am home, I delete unwanted pictures before turning wifi back on and the pictures sync to OneDrive.
All my picture file names are time stamped and I add descriptions to the some of the file names after the time stamp. Occasional descriptions help find stuff in a time stream.
Every several months I rename my Camera Roll directory to Camera_year_month_day and create a new empty Camera Roll directory in OneDrive. It is easy to find any of my pictures, select groups of pictures and generate share links, etc.
This same scheme used to work fine when I used Dropbox, and converting from Dropbox to OneDrive was easy and maintained this setup.
edit: I use selective sync to device to not have all of my pictures on my devices, relying on the web OneDrive interface to show people pictures without taking up local disk space.
Regardless of the solution, redundant backup is an absolute must for me. I will never depend on 1 solution for any of this no matter what the price.
My current solution is this:
* all photos go to Dropbox about once a month or if there is a large batch (birthdays etc)
* redundant backup to a RAID server in my house (3 TB drives) right now about 50% full
* I also compress really important photos and send them to an FTP server on a VPS with limited web access (no httpd)
For presentation I have a bunch of scripts to import and index in bulk from idevice backups (which catches things received thru MMS from the nanny or family) and convert to webm format or create thumbnails. Once all the metadata is in a db, a simple rails app shows them with paging, date ranges etc.
Would be really nice to have more features like curation, galleries, rotate, and so on like online apps have. But as long as I have full control upstream I'm happy.
It's not 1999 any more, don't use something so awfully unsafe for BACKUPS. Look into sftp if you must.
I'd also like to suggest tarsnap for this sort of cold backup, I don't know what you're paying for a VPS; but my comparisons showed it to be cheaper - which is a good thing considering that it's also really really secure.
Every solution I've attempted to instigate with my wife has been met with resistance for minor technical or minor inconvenience reasons.
Previously, it was an external drive with iPhoto that used what eventually was a merged iPhoto Library on an external drive. The external drive would be plugged in to the RAID 1 NAS periodically to preserve it. Finally, every week or so the NAS would backup to S3.
However, all of that proved too inconvenient or too aggravating.
Apple Photos is working great so far for us (albeit with a tight coupling).
Sure, some of these applications store a copy both online and on disk, but if your application chooses to run amok and delete everything, that's not really going to matter in many cases. I've seen horrifying things happen when iPhoto tries to sync with the cloud before. :-(
Adobe Creative Cloud seems to support 1 collection with 2 people accessing it. Apple Photos is less expensive though, comes out to $5 a month for both of us (20GB + 200GB) vs the 10$ a month for the Lightroom.
On the phones I backed up 2 phones, each with about 10gb of photo/video.
I don't want to be stuck with one software's way of doing things, so here's how I organize them:
- Each trip is organized in a folder with a sortable date down to the month and general placename: "2014-05 London"
- Inside that folder, at the end of every day of shooting, I dump all the photos into a folder with the name of the day of the trip: Day1, Day2, etc. Sometimes I'll use the date for the day instead. It doesn't really matter much.
- Since during the course of a day, I might shoot at several locations, inside each "Day" folder, I organize the photos by location of the shoot or a short description of the event I was shooting, "Walking Around the Shard", "Tate Modern", "East Dulwich" etc.
- I also use a couple different cameras, so inside each of those, I have a folder for each camera: Nikon, Lumix, etc.
- Inside each of those I separate out pictures I took of my family or of me with pictures of sites. I usually upload the "personal" pictures to Google or Facebook so I can access them anywhere.
- So if I want pictures of us from Picadilly circus on the 3rd day of our trip to London (and not the 6th day when we went there again), I go to 2014-05 London/Day 3/Picadilly Circus/Nikon/Us
- If I want Pictures of Bernini's Ecstasy of Saint Teresa from the Santa Maria della Vittoria in Rome, I go to 2011-04 Italy/Day 5/Rome - Santa Maria della Vittoria/Canon
- In each of these, I make a folder called "top" where I copy the photos I think are the best. As I process the photos I make a final selection and put the results in a folder called "done".
- So the best, finished and processed, shots from my last example are 2011-04 Italy/Day 5/Rome - Santa Maria della Vittoria/Canon/top/done
out of maybe 200 photos total at that church, only 2 or 3 might make it to "done".
Even if I can't remember exactly which day I took a shot, the descriptive names are searchable in the file structure and I can usually find a specific photo in a few seconds. It's also self documenting so when I need to update my CV or remember when I took a trip somewhere, just navigating the file system provides all that information.
There's often 1 more step, an agency also buys some of my photos, and their acceptance process is even more stringent than mine, so out of 20 or 30 "done" photos, I might only get 6-10 accepted for professional publication. I put those in a "published" photo under "done".
I've started a new system as well. On January 1st, I take the previous year's photos and put them in a folder with that year. Only the current year's photos stay in my root photography folder. This makes things much tidier even if a little harder to find. So all of the above examples would have one more folder with just the 4 digit year before it.
I make a global backup of my entire collection after every trip or shoot onto a couple other disks, one of which stays disconnected most of the time and in a drawer.
Over the time and many setbacks (lost stuff, not able to find stuff) I have also drifted to a very simple solution: use a file system to organize and store your stuff. This is valid equally for pictures and for music (and documents, and books, etc.). This makes sure you will never loose your stuff. Backups, search, copying etc. is provided by tested and reliable tools either from the OS itself or the standard tools like rsync. Everything else is "on top". For example you can use iTunes, Clementine, Foobar, whatever music library software you want, but keep your mp3s in folders sorted by rating/artist/album/song and you will be able to migrate between the vendors, backup, restore, script conversion (need to compress music and resize photos further for the phone to fit into 128GB limit of the sdxc card). Same thing with the Photos. I use very simple folder structure of Year/Location. Bad shots are just mercilessly deleted (power to the Paper Bin!). Super good ones a copied to Favorits. Because photos are coming from different devices I have given up on filenames for lack of consistency.
And, should the vendor drop your library management software in next 10 years what will you do? With a filesystem oriented approach you can be sure to have it available 50 years down the road, or something better comes around and you will just upgrade.
Here's what I do:
2014
2014-01-22 - Trip to Uruguay
2014-01-23 - Park X
2014-03-30 - New flat
2014-03-30 - Dinner with Mum and Dad
...
2015
...
It keeps very few directories right inside ~/photos, but has lots of granularity when searching for things. Date-Event also makes it easy to find a certain event for which I can't quite recall the date by simply scrolling through.I have folders like
/PHOTOS/_YEAR/YYYY-MM-DD-EVENT_NAME/ ..here are all photos
/PHOTOS/_YEAR/YYYY-MM-DD-EVENT_NAME/best ..here are the photos that are good
/PHOTOS/_YEAR/YYYY-MM-DD-EVENT_NAME/best/web ..photos I share via my website as a generated static html photo gallery
backup is just simple folder backup, synchronization is done via file synchronization tools, viewing via many available picture viewers etc...
Don't get me wrong, the technology is great and it's enabling some really talented people to do really awesome things. But those people spend literally years learning to compose shots, and hours after each photo session poring over thousands of duds to find the few gems. The average person isn't doing that. Let's face it, if you want the sunset in Puerto Rico captured in a photo you're better off letting the professionals do it.
There are other definitions of quality. I have a lifetime of photos too, but my lifetime of photos amounts to about 300 photos taken over 30 years. They're not well-composed. A lot of them were shot on crappy hardware and it shows. A lot of them are pictures I didn't take. But every single one of them takes me back to something I care about and makes me smile. My vacation photos are of me and of people I care about, with things we liked.
I don't need cloud storage to sync those 300 photos between my devices. They don't require organization; they're stored by date and time and that's it--I know where every photo was taken and who is in it by looking at it. The photos are stored on my laptop hard drive, a backup drive, and on my personal server. I sync them with rsync but if that didn't exist there are a dozen other ways that would work as well.
If you're not a professional or trying to be one, then the solution isn't waiting to be invented. The solution is to stop taking so many pointless photos. Is sifting through hundreds of photos really improving your life? Do you really care about all of those photos, or could you figure out before you even take the photo which ones you're going to care about?
And taking all those photos has other downsides too that have nothing to do with poor storage solutions. Trying to capture everything for the future prevents people from living in the present. I see people at concerts all the time watching the entire concert through their phones. Are you really going to rewatch that poorly lit video with its shitty sound later? You're damaging a present experience that could be awesome in exchange for a mediocre future experience that might not happen. Not to mention how this affects other people's experience: if you're tall it's not your fault, but if you're holding a phone between me and the artist I paid to see you're being an asshole.
TL;DR: The best way to organize a lifetime of photos is to stop taking ones that don't matter.
Not sure how much, how far you travel and how various activities are you participating. When I bring my photos from a distant country people are interested to look at them. And different people find different pictures interesting. Some people want to see all the pictures of penguins, while others like to see architecture. Others still happy to see tens of different motorbikes from a trade fare. Often the ones I thought were worth deleting are the interesting ones for the people. So unfortunately deleting them right away (or not taking) is not an option to me.
Also as usual - telling people stop doing things they like is hardly helpful.
I get that my post was long so it makes sense that you didn't read it, but maybe don't respond to posts you didn't read?
Taking photos and then deleting them is not equivalent to not taking them. It takes time and storage, exchanges a present experience for a mediocre potential future experience. I already explained this in my previous post.
> When I bring my photos from a distant country people are interested to look at them. And different people find different pictures interesting. Some people want to see all the pictures of penguins, while others like to see architecture. Others still happy to see tens of different motorbikes from a trade fare.
Are any of these things photos you have to take? Or could you find better versions of these photos, taken by professional photogs, online, and send it to people with a "I saw this"?
> Often the ones I thought were worth deleting are the interesting ones for the people.
Often when I'm taking a photo, I think, "<X friend> would really like to see this." These make up a small fraction of my photos and after I send them to the person I delete them. But the difference here is that on a trip I take 15 photos instead of 10.
All this says about you is that you're not approaching your photo-taking intentionally. None of the objections you've brought up couldn't be solved with 15 seconds of thought.
> Also as usual - telling people stop doing things they like is hardly helpful.
I'm not telling you to do anything. If you want to spend money and time viewing your life through a small screen so that a few people might find some of your photos interesting, go ahead.
People like all sorts of things. Some people like cocaine. But it's easy to get caught up in short-term dopamine-releasing behaviors and sometimes those behaviors do more harm than good even though they feel good. It makes sense to take a step back occasionally and look at the effect of you behaviors you enjoy on your overall happiness.
Also, I find your tone spectacularly offensive.
> You might also be surprised by how little you actually remember, and how much you enjoy being reminded of life's little events perusing pictures later. And postcards are no replacement for pictures of my wife and kids in Hawaii.
I don't disagree with any of this. The problem is not taking pictures, it's taking so many pictures that it causes storage, organization, and experience problems.
> We have about 60gb of photos and videos over the last ten years. I don't spend my time viewing life through a viewfinder.
The first sentence contradicts the second.
> With modern technology, snapping a photo, or a series of photos, is very low friction.
It hasn't been extremely difficult to take pictures for most of my lifetime.
> Also, I find your tone spectacularly offensive.
Stop the presses, we definitely shouldn't be talking about anything that offends anyone!
Getting offended does not make anything better ever--it's just a way of ignoring people you disagree with instead of considering differing opinion. If that's what you want to do, you're only hurting yourself.
I'm not criticizing you or anyone else as a person. In fact I'm not even telling anyone what to do. If you choose pictures and videos over direct experience, that's your choice to make.
I think most people would be happier if they took fewer photos, but I think there's a lot more to happiness than that. If you're happy then there's no need to fix what isn't broken. But if you think you could be happier then there's no reason not to explore conversations about what makes people happy or not.