Hacker News Reader

> Not all behavioral data is sensitive.

Maybe not, but I am surprised on a regular basis at inventive new uses for data that I had never before imagined. At this point, I think a safer default is, "all behavioral data is sensitive", until we better understand how all of this stuff works.

> Inflating databases and costing money.

Everything costs money. It is meaningless to point out that something costs money without attaching a number to it. Databases would be inflated by four or 8 bytes per datum; at 500 million rows, you'd add between 8 and 16 GB to a database that is likely very much larger already. I really don't see this being an insurmountable engineering challenge.

> These obligations must be handed down from company to company during acquisitions. A company selling data about to expire will get acquired for a lot less than a company with fresh data. This may in turn cause a series of unforseen consequences in the acquisition market.

This is a feature, not a bug.

> Rather than controlling and manipulating what can and cannot be done, it may be best to just create transparent policies and let the free market converse its way towards a compromise.

One of the central points of the lecture was that there is insufficient competition in some very important parts of the internet right now. Those companies could easily say, "Our policy is that we keep everything forever and there's nothing you can do about it", and it would not affect their business at all because they have no competitors.

Regulations exist in the real world for a good reason, and we're rapidly approaching a point where regulation is exactly the appropriate solution for some of these problems.

>let the free market converse its way towards a compromise.

There's no "free market" that's an ideology (a made-up idea of how the world is that obscures one's thinking).

With government: I have more money than you, and I have friends in Washington. I'll use it to push things my way.

(And lest somebody suggest: "sure, the problem is government", here's the government less version:)

Without government: I have more money and/or power than you, and I have you beat to a pulp, and also spend it to make people go along with my propositions. I'll use it to push things my way.

Usually is a mix of 1 and 2. E.g most Latin American countries, for example, there's not much of a "free market" with regards to their exports/materials because stronger countries force them (with military might, diplomatic pressure, juntas, putting friendly lackeys in power, briding, etc) to go with their way. A powerful country can spend tens of millions of dollars just to promote a favorable candidate in power in a smaller countries (easily recouperated in a day's worth of profits from resource and trade agreements).

And of course with things as a patent system, intellectual property, etc, there's no free market also. The IP owner sets the terms, and you cannot offer the same thing for a reduced price even if you can.

> There's no "free market" that's an ideology (a made-up idea of how the world is that obscures one's thinking).

Everything is a made up idea that obscures one's thinking. I think I have to drive on roads, but really nothing is stopping me. I could get through traffic faster if I just started driving on the shoulder, but the problem is if everyone did that it would be chaos, and would be worse overall. Systems have the potential to create a net good.

Well, the "have to drive on the road" is a "law" or an imperative not an ideology. People know it's a made up convention so that we don't hit each other or depestrians.

People talking about free market, on the other hand, think of it as a real, concrete thing, and even further, that it has this and that properties. Thinking thusly about a made-up thing can have dire consequences -- like when hallucinating on drugs and jumping from a building to avoid a huge snake.

I'm going to have to disagree--how often do you have a "created_at" field in your data?

Moreover, how useful is it to see when data was collected? You pretty much need that to be able to model user behavior over time.

If the deleted data breaks a report, that's in line with what the user wanted--failing that, you should've used a proper design pattern to transparently fill in the gaps.

Let's not pretend that this would actually be anything other than a minor annoyance.

EDIT:

A program must be written that seeks out timestamped data ready to expire and delete it."

Here, let me help, after one second on Stack Overflow:

  DELETE FROM Table WHERE DATEADD(year, 2, CreateDate) < getdate()

There's your program. Anybody want to show how to do this in Cassandra or Mongo? Might as well put that NoSQL map/reduce to work...

I feel i should point out that will not use an index.

DELETE FROM Table WHERE CreateDate < DATEADD(year, -2, getdate())

It doesn't make sense that a company would expire data, it's not illegal, and users are not the customers to companies like Facebook, and Google. Users are the product to advertising.

A more concrete solution would be to own our own data for these kinds of services, and pay for the active development & hosting. I'm all for this, and hope to see business like this in the future, I would definitely pay for Facebook if it were possible.

Some issues I could see with allowing the free market to yield, is allowing interoperability with existing services. Is it illegal to plug Diaspora into Facebook and redirect friends to my service? I'm sure it's against their ToS, and I bet Facebook will do everything in it's power to protect it's product.

Also, credit card fees make it difficult to implement micro transactions. If I charge someone $1, 32.9% will go to fees [1]. I like fiat currencies, but the government should really step in and clean up the infrastructure. Credit card companies are centralized services for our transactions, held in power by the government. Paying with cash is amazingly decentralized, and crypto currencies accomplish the same effect on the internet. If our government designed a better currency for the internet, it would give more power to the free market.

[1] https://stripe.com/us/pricing

> His suggestion that all behavioral data be deleted after a certain period of time means every little piece of data collected must also have a timestamp. Inflating databases and costing money.

I'm not convinced that startups are such an inherent social good that they need to be insulated from the costs of responsible behavior.

> These obligations must be handed down from company to company during acquisitions. A company selling data about to expire will get acquired for a lot less than a company with fresh data. This may in turn cause a series of unforseen consequences in the acquisition market.

Working as intended. Forcing companies to be responsible will cut down on rash, irresponsible acquisitions.

But he also suggests "Let users opt-in if a site wants to make exceptions to these rules". I would think every website would just add a click-through opt-in, undercutting the regulations. How would one prevent this from happening?

The formatting of this article itself is something worth studying. It's brilliantly seductive to read and read all of it.

I couldn't agree more. This had been exactly my thought as well, as soon as I finished reading. For some reason the presence of those photos on the side made me read absolutely every single word. And personally, I haven't paid attention to every single photo, but their mere presence persuaded me to read the paragraphs in their entirety.

It's very intriguing how the first part of your comment captured my thoughts so accurately. (: