Deleting the Family Tree – Ancestry.com erased 10 years of correspondence
slate.com
slate.com
This is not one of those situations. Ancestry.com charged as much as $240 annually for its service [1]. Furthermore, the MyFamily.com social network was just one small part of Ancestry.com, meaning the parent company still has the money (and I would think, the actual servers/backups) to make its paying customers whole here. What the OP describes is a seriously incompetent handling of a shutdown...and one of the variety that is most painful for customers.
[1] http://corporate.ancestry.com/press/press-releases/2004/01/m...
I agree fully, their handling of it was insensitive and incompetent and I'll never have anything to do with any of the "Ancestry" products again.
Fortunately we found Spokt.com which (after a few rocky starts) was able to "sort of" import most of the data into our new Spokt account. I feel terrible for those families who didn't and permanently lost a large chunk of their family history.
Yeah, this is a shitty thing to do to any users. Always make sure there's a way for them to export their data. After all, it isn't really yours - they just let you have it because you seemed trustworthy. Don't betray that.
Generally, the genealogy community is very open and sharing. Distant relatives (n cousins nth removed) will spend hours helping you out. Random people will go to local libraries or town halls to pull hard copies of vital records. LDS has one of the largest free genealogy libraries in the world.
Then here comes Ancestry. $240/yr. to use their software, which puts everything you do behind a paywall. Want to share your family tree with others? They have to pay Ancestry for the privilege.
There's serious value in digitized records that might well be worth the subscription fee alone. But these are public records that Ancestry has bought the exclusive rights to digitize. They're aggressively monetizing public data while shutting out competition.
It's great that family history is so easy for the masses, but I'm worried the long-term future is going to be silo'd and less effective in the long run.
What exclusive rights are you referring to? I'm trying to learn more about Ancestry.com.
(If we are going to go by exactly what we said and not the implications of the words we used, though hopefully we can agree that would actually be dishonest.)
https://familysearch.org/blog/en/create-free-account-familys...
The Church is meticulous about good record keeping (which is what makes them so good at genealogy in the first place), and every official member of the LDS church has a unique member identification number, used for things like tracking donations, ordinance records, and, of course, genealogy. Nowadays, the vast majority of these records are digitized and you can access them online through Church-owned-and-operated websites. If you happen not to be in the system yet (maybe you were baptized in the jungles of New Guinea and have only just made it back to civilization), it's pretty simple to meet up with a bishop or clerk who can verify your paper records and tell you what your membership number is, which will let you set up an account on Church websites.
So, to summarize, the Church knows who its members are, and knows how to associate them with online identities. When one of them wants free access to Ancestry.com, it's basically the same process as "Sign in with Google" and such- you tell Ancestry that you're LDS, Ancestry asks the Church's servers if you're telling the truth, and the Church's authentication system teels Ancestry to let you in.
Some copies of the census are digitised already, but the further you go back the less likely that is. And a lot of the information people want originates abroad (e.g. UK, Germany, Italy, France, etc).
I'm working with an organization to collect real estate parcel data for the entire US and open source it. I track every county or district that has parcel data, I built a workflow to get the data from them, manage the ingest of the data, and a separate workflow if they're difficult and I have to FOIA them for it (via muckrock.com). There is a cost, but as always, everything has a cost.
Same deal, but closer to what Distributed Proofreaders for Project Gutenberg does with scanning.
It can be done. Funds and organization would be necessary, but let's not think its some herculean task.
Scanning the images allows people to verify the transcription and to distribute the work of doing the transcription.
Website: http://www.leafseek.com Background article about how the project came to be: http://www.leafseek.com/blog/leafseek-gets-published-in-avot...
I currently work with two major non-profit genealogy groups to put their record collections online for free and easy searches, without handing them over to for-profit groups like Ancestry. So far we're at 700,000+ records online using my code, and the newest "big" addition is the Israeli State Archives agreeing to put their British Mandate period marriage and divorce certificates online via one of the non-profits using the code.
I'm planning on launching the system as a multi-tenant SaaS one of these days, once my kids are a little older. :-)
Business plan?
Data license? (Public domain, creative commons, etc)
I make everything I retrieve publicly available with a copy thrown in the Internet Archive for good measure.
No freemium, only paying clients. BUT a price discount if users allow (1) auto backup of datasets in CSV format to Internet Archive through their S3 system, even if the data is "darked" and not visible (I'm on Archive Team and I also created their "Antecedents" collection of genealogy-related web crawls: https://archive.org/details/archiveteam_antecedents) and (2) they explicitly put their data in the public domain.
Pie in the sky idea: create API for higher tier paying users to allow them to license their data directly to Ancestry, MyHeritage, FindMyPast, etc. Which would flip the pay-for-data model on its ear, allowing users to regain control of their own datasets, and for-profit data brokers to pay them for access.
To that end, I will also be rejecting VC and bootstrapping the venture. Ain't nobody going to be flipping this site or acqui-hiring that data. This is going to be done for the long-term.
No, never. Not Ancestry or anybody else. In this very moment my computer is running my daily back up my Google Drive (customers write stuff there) to my USB disk (I know, I should need a better disaster recovery solution). Not that Google doesn't provide a good service, but I don't trust them to preserve all my data until I need it, possibly tens of years in the future. I better care about it myself.
I consider anything stored "on the cloud" as expendable. If I really care about it I back it up in multiple copies. I have a couple of servers where I send encrypted backups using duplicity.
Is this too much for the average computer/tablet user? Yes, but there won't be a market for proper consumer backup until enough people has been burned by this kind of things and all cloud services will start offering an export procedure.
Meanwhile I think it's worth using self hosted services. For my company I have a self hosted redmine with all the details of my projects.
Is that the best way to proceed? Nope, abject paranoia seems like the only way to preserve data -- assuming there is a nuclear war, have you archived your data on a different continent, have you got a plan for retrieval!?!
Just remember that most cloud services do not do backups, do not have a disaster recovery / BIA plan, do not use ECC memory in network, compute and storage systems. Users must take personal responsibility for ensuring their private data is safe, because wishing is not a winning strategy.
MS Azure uses ECC [2]
Rackspace uses ECC [3]
While compute servers aren't backed up this is by design (use storage). Storage is indeed backed up and replicated. Also most assuredly there are extensive disaster recovery plans.
The cloud isn't for everyone, but FUD doesn't help.
[1] http://aws.amazon.com/ec2/faqs/
[2] http://azure.microsoft.com/blog/2015/03/16/microsoft-azure-u...
And yet, you are referring to pages about redundancy, not backup. Redundancy helps when a disk, server, or perhaps even data center fails. Backups help when your cloud storage provider makes an error and removes your data from redundant storage.
There have been enough incidents to show that cloud computing alone cannot be trusted [1]. Unfortunately, most cloud services are data silos, so you couldn't even backup data if you wanted to.
And then we haven't talked yet about data leaks. Some larger incident than last year's iCloud leak will probably happen sometime. Hopefully then people will realize that client-side encryption is important.
Also I provided only links to ECC proof.
Sure. But the difference is that if you manager your own storage, you know what backups are made, etc. Dropbox claims that backups are made, but these users lost data regardless.
We do not do backups, however, unless you are paying for geo-redundant storage. If you use single site storage, the idea is that we are the backup.
Since the conversation has drifted this way, let me also point out that we have s3cmd and gsutil in our platform, so if you're paranoid about trusting data to google or amazon (and to some degree, you should be) you can easily import/export to an independent cloud platform (rsync.net).
> No, never. Not Ancestry or anybody else
I respectfully disagree: it is natural for people to assume that companies won't callously discard their customers’ data. If it wasn't natural, we wouldn't need to have so much effort going towards education about the terms of service to watch out for and how to make your own copies. Too many people have a mental model for corporations which grossly exaggerates the level of stability and long-term planning.
"Hoping for the best" (or perhaps "trusting that people won't screw me over") is actually an extremely good strategy -- far better than not trusting anyone with anything. I can hire third parties to cook my food, do my taxes, handle my banking, even babysit my children after a background check (where I have to trust the background checking company) and a brief interview. If I couldn't trust people, I simply wouldn't be able to function in the modern world.
So I have a company, who I have paid to store my data. The company has been good so far, I use their services frequently, and I have every reason to trust them (they've had my credit card info, and my family's for years, haven't screwed me over, haven't lost any other data in all this time...) The company is shutting down, and they explicitly tell me that they will let me download all of my data, including text conversations. I download the data, only to open it weeks later and find that the company has failed to do what they said they would. In this one particular case, the strategy of "trust" has failed. However, in the long run, it's made my life a heck of a lot better.
We pick banks, stores and neighborhoods based on reputation. We trust some of them because they proved to be good and we don't trust others (I bet there are areas of the world and maybe of your city where you won't walk alone in the night.)
We get educated to trust according to reputation.
Personally I don't trust much Internet services because so many of them die young. I always wonder how many years any of the services I have to use for work will stay with me and how I can move my data somewhere else, in the few cases it makes sense (example: you can export Trello's data, but then what?) My customers and I use them because they are so convenient but I'm sure that sooner or later one of them will die in the middle of a project and we will be burnt. I have redmine wikis from 8 years ago. I'll have Trello's boards for the projects I'm working on now in 2022?
But I also don't trust much any "normal" company. I expect that they make their interests first, mine maybe. I buy from them but caveat emptor.
You would think that by now a $650 phone would have a lot more then 32gb of storage.... But even if you have a phone that supports an external microSD card, Google makes it difficult to write to it using a 3rd party Android app (stating security as a reason).
There should be a law that prevent companies from keeping root to themselves on devices that you own.
There should be a role here for government, to let people give digital data in permanent archival for historical purposes. But given how poorly governments are preserving their own digital data, I doubt it could actually work.
I think future historians will look back in the same way we do now. We're horrified that people discarded, burned or reused records from every century in history, but technology allows us to record and preserve more every year.
People of the future will probably be very disappointed with us about a great many things that we could have done with our technology, knowledge and resources, but neglected or opted not to do out of short-term financial interests.
But I don't think properly documenting our time period will be very high on that list.
I'd love to be wrong.
If in a few hundred years, there's people, society, culture and at some point again a level of civilisation and technology that allows them to dig back into the past, and the worst thing they'll have to say about our addiction to short-term financial interests is "aww, if only they had documented their time period better", then I ... well, the only likely scenario I can think of is if we also somehow manage to sell out severe-global-brain-damage-in-a-couple-of-centuries for our short-term financial interests (surely right now, there are some very smart people working on this very idea).
No. This is what historians of the future will say if we would indeed meticulously document our current time period, and they could read about our day-to-day lives, important events, culture, media, etc.:
"... they did WHAT?! while the planet was ... and the oceans were still ... they burned WHAT?! in order to be able to ... gosh well at least thanks to our ancestor's meticulous documenting, and millions of hours of video footage we can now theorize what these 'kittens' are supposed to have looked like ... mammals used to have hair?? ..."
No, I think we better start deleting and encrypting our data. To save face.
(small disclaimer: I am half-kidding, of course. While I am not particularly proud of the 'big picture' behaviour of my species in this particular era, we shouldn't actually try and hide our planetary blunders in shame, but indeed document them so that future civilisations can learn from us, and who knows, maybe one day can forgive us)
Well, there is a whole family who will look at hosted services with a much more jaded eye now. Who wouldn't?
Truth is, this kind of garbage devalues everybody. When a large family like this loses out in such a painful way, word gets around. Clearly they are not alone either. People who tend to be into this stuff are very seriously into it. Understandable. It matters.
Future offerings, however well their intent and execution might be, will have to work to overcome the painful experiences. And that is just a waste.
One thing nagging at me though. Did they really just delete it? Seems that having a fall back would make great sense, if anything, for legal reasons.
Or maybe it's better to have it really, really gone. Limit the potentials, so to speak?
From time to time, I export various things. Just ordinary worry, but I'm reasonably informed. Who here isn't?
Perhaps ordinary people really do need to advance to the next level of literacy. "The basics" today are different. In some ways, simpler. In other ways, potentially more complex.
Just saw danso comment. Yeah, my thoughts too. It was possible to treat people right. Do they really just not care?
If this were me, I would have a very hard time sleeping at night, particularly given some options. It was easy enough to set up and make the money...
But have people ever worried? From what I remember from before the cloud became popular, and from people who maintain the old habits, people never really worried, they just kept their precious files in a $10 USB flash drive or 5-year-old HD and then flailed around when it broke.
Perhaps that "safe" needs an addendum to be truthful, "MTBF is 5years at which point usually 2% of drive data is corrupted; this is not a suitable backup solution" or "our cloud storage facility has lost some data from 1% of all our users (2100 users) in the last year; 5.2% of users in the last 5 years" ... now that would be revolutionary.
Also, one other factor that may be important to the discussion is that the USB drive was private. Now somebody hacks a cloud service and your data is potentially in a Bittorrent swarm the next day. Seeing what people put on the net nowadays, I don't think they realize the difference.
tl;dr: educate your loved ones.
And the lack of understanding becomes binary instead of the usual array of ugly, but maybe recoverable scenarios.
With that comes some obligations, IMHO.
This isn't the same thing as replacing a social network service, but the actual trees do capture comments, stories, submissions, etc. from other users.
Having a local copy is better than not, especially one you can back up, but in 50 years it won't help much if the format is too difficult for anything else to read.
(Honest question, though. I don't have this software, so I don't know.)
http://en.wikipedia.org/wiki/GEDCOM
which is a well know standard in the genealogy world.
Now this is impressive. How did you get back that far? I've trouble finding out who my great-great-grandparents were.
It depends on your heritage (I am of northwestern European and American descent, so my records are very good) but Ancestry makes it very easy, to their credit. A few minutes after you've added a new family member, Ancestry begins suggesting historical records and public trees (I try to avoid relying on the latter) that may reveal additional information about them, including who their family members are. You then mark which information to import.
So essentially you would start by inputting the family history you know, and then let Ancestry start making suggestions. It still takes plenty of manual intervention but I've taken their suggestions back to the 1500s in some cases.
I do have to give a special shout-out to the Polish national archives as well as the U.S. National Archives people in Maryland. I was able to deal with the Poles entirely online and request hundreds of records which they reproduced for a reasonable fee in a short time. The U.S. archives also went out of their way to help me. Hell, even the FBI FOIA process worked well and gave me a valuable dossier. The one awesome lead that let me down was the NYPD FOIL request I made. A relative many generations back ran with Al Capone when he was in Brooklyn and was gunned down. The NYPD never caught the killer and supposedly had a hat left at the scene and other evidence. Murder investigations are kept indefinitely so they should still have it in some warehouse somewhere, but they couldn't find it after searching for it manually. (Digitize all the things, people!)
When my family tried to create one on Geni, it brought a few crazy relatives out of the woodwork, and Geni's "growth hacking" strategy of spamming anyone who was invited to join the tree led to angry real-life confrontations between inviters and invitees. Our family tree is maintained a little, but the social aspects died a quick death.
It would be interesting to know why they shutdown the service, presumably not making enough profit, whether anything replaced it?
Perhaps it isn't the cynical decision, rather it's the result of nobody being left to turn on the lights, let alone handle a technical task. Once companies enter shutdown mode, there's frequently nobody left to complain to. I'll be the first to admit that I don't know the specifics here.
There is no way I can believe Ancestry.com doesn't have the technical talent to pull the raw data from the system pretty trivially. Unless it was running on some obscure in-house database backend that doesn't exist anymore and thus can't be read, I don't see why they can't put an engineer on this project for a month [and if it takes them that long...] and at least give people the raw data. Or partner with Spokt or whatever and have them charge a fee to transfer it if they want to cover their costs. People are willing to pay for this data, even if it is incomplete or difficult to digest in the delivered format.
I also can't imagine they don't have a copy of the data somewhere... on some old magnetic backup tape or squirreled away in some DB admin's folders.
Certainly there are legal issues and privacy issues. Are these really insurmountable? If enough people really want their data, I somehow doubt that this couldn't be worked through. And I absolutely cannot imagine that Ancestry just deleted all of it when they shut down. It's probably sitting in cold storage somewhere.