'Panama Papers' Offshore Leaks Database | Hacker News Reader

'Panama Papers' Offshore Leaks Database

offshoreleaks.icij.org

108 pointsastdb17 comments

I think this DB was published around a month ago (http://www.thereportertimes.com/panama-papers-icij-offshore-...) and you can get some quick and inefficient import code for Neo4j here https://gitlab.com/berezovskyi/panama4j (really slow, was hacked together during the Neo4j meetup). I will check this DB after the PyCon Sweden today but so far I do not expect anything new (except maybe a bit less "details temporarily withheld"). Here are the SHA256s from the new DB release, nothing seems broken:

  804542be8e6b3896ff3001a49b0486f644598bfdd4cdd76dbb1f0d40c0fa7818  Addresses.csv
  94b4fd9d6578d9d83102cb5d4b139b94e1262d9dbe72f5aa97e9d6502aa94e5d  Entities.csv
  e44b48e248696656854849d2f984dd5e76a2645c28e99451b066c527a96b815b  Intermediaries.csv
  4687edf2bfb23c18e547838bcf4b9bfd36f5f4a960e924be4e4deb193023f115  Officers.csv
  ca91a23b6f6472fc7152e6b3ed4bd343fccc00a142627901485fbaf59c5be739  all_edges.csv

Sizes are the same:

  curl -sI https://cloudfront-files-1.publicintegrity.org/offshoreleaks/data-csv.zip | grep Length
  Content-Length: 37481567

My guess is >99% of this data refers to corporations that are legitimate businesses that are fully compliant with the law. US persons have been required for years to disclose offshore bank accounts (through FBAR reports) and other offshore financial holdings (through IRS Form 8938 on tax returns, cf FATCA). I'm sure there is a team at the IRS that is busy comparing this data set to the IRS database, and woe to anyone that has not disclosed their offshore financial interests.

But it shouldn't be a surprise that people use offshore corporate structures to manage offshore businesses. The main reason this is necessary in the US is that the US taxes corporate profits worldwide, while other countries do not (they have territorial tax systems and do not tax profits earned in other countries). So US corporations have to set up these complex structures to achieve tax rates that are comparable to other developed countries.

Anyone knows why they are not willingly sharing the raw data with law enforcement?

The full database in CSV format (44.8GB) is downloadable via torrent https://offshoreleaks.icij.org/pages/database

Correct me if I am wrong but the way I understand it, what has been made public is a very small part of the data (which 2.6TB),Law enforcement would need access to all the data to investigate leads and prosecute offenders. ICIJ is refusing to give them access.

Where do you get the 44.8GB number from? I see a 37.5MB zip file available for direct download or via BitTorrent.

The torrent does not appear to be working properly (at least for me, I see no seeds and one tracker only that doesn't respond). But The direct download works.

I would bet 45GB is the full raw text of all the docs, and that's not what this is.

It's not a huge file but there's definitely a lot of data there.

This is just a database of the Entities (319,422), Addresses (151,128), Intermediaries (26,643) and Officers (345,646).

They also include an "edges" file to indicate relationships.

It's weird there are no seeds for you. I've been seeing it for ~20 hours or so without issue.

This is the metadata for it, if that helps:

Name: offshoreleaks_data-csv.zip File: torrents/offshoreleaks_data-csv.zip.6ffb15f06e409a58.torrent

GENERAL

  Name: offshoreleaks_data-csv.zip
  Hash: 6ffb15f06e409a58ab7d2a710784e4ef974da782
  Created by: 
  Created on: Unknown
  Piece Count: 144
  Piece Size: 256.0 KiB
  Total Size: 37.53 MB
  Privacy: Public torrent

TRACKERS

  Tier #1
  http://tracker.amazonaws.com:6969/announce

FILES

  offshoreleaks_data-csv.zip (37.53 MB)

At least for the USA, I see a lot of entries for "EL PORTADOR" and "THE BEARER" (essentially translations of each other).

Genuinely curious, does anybody know what these are?

A quick search didn't bring anything up and I'm not very familiar with these matters. Thanks!

Most likely Bearer Shares: http://www.investopedia.com/terms/b/bearer_share.asp

You have to know either the jurisdiction (offshore country) of the company or the name of the person. What would be quite valuable would be a search based on the linked countries. E.g. companies/ppl linked to Italy.

You can do this by leaving the search field blank and just selecting a country, which returns all entities/officers/intermediaries/addresses for a certain country

There is a torrent that lets you download the whole database. Though the structure of it and what the fields represent is not so easy to figure out.

  It is good to see it not only searchable but the raw data as well.  Too bad Greenwald was not this open with the Snowden leaks.

How far to respect the source's wishes regarding controls on the material is a tricky area. In this case, the source was happy with releasing large amounts of raw data. In the NSA leaks, Snowden wasn't, and he imposed a condition on Greenwald and Poitras not to release everything.

Does the journalist have more responsibility to the source or to the global population? Many people disagree with Wikileaks style raw data dumps irrespective of the source's wishes.

In summary, the ethics of leaking is complicated!

> Many people disagree with Wikileaks style raw data dumps irrespective of the source's wishes.

I found it extremely ridiculous how WikiLeaks attacked the Panama Papers leaker multiple times for not leaking full data to the public and accused my organization of intentionally targeting eastern countries because one of our donors is USAID, when, according to the official statement by the leaker that came out a couple of days ago, the leaker tried contacting WikiLeaks multiple times and then decided to pursue another way once he realized that WikiLeaks simply isn't responding to him.

The database available isn't the raw data though. It's just the curated neo4j dataset, in CSV format. About 40MB in total.

Reply on news.ycombinator.com