The US government's open data on Data.gov is currently being scrubbed
old.reddit.com
old.reddit.com
It would be a great start to have a mirror of just the metadata, to at least see what is being removed.
I’m on mobile at the moment so I haven’t been able to dig too deep, but here’s [0] a query that returns the metadata for everything modified since Jan 1st (add more rows at the end if it’s cut off). I don’t think it shows deletes directly, but it can be compared to the current catalog. API documentation here [1]
[0] https://catalog.data.gov/api/3/action/package_search?q=metad...
[1] https://data.gov/developers/apis/
*fixed first url
I was extremely surprised to find out that based on ICE’s annual reports, which are still up, Biden’s administration deported far more people than Trump’s did, especially in 2024 due to rule changes which roughly doubled the number of normal deportations. I was further surprised to find out that there were roughly 3.5 times more deportations under Biden than under the first Trump administration.
I suspect that what’s happening is that leadership under Biden did the sensible thing and very quietly snuck up on priority illegal aliens, whereas with Trump, all the media attention is causing similar people to circle the wagons and proactively take steps to avoid enforcement actions.
Trump loves to talk about trade but when it came to China he put up some token tariffs and announced “I didn’t say it would be easy” and then did nothing the rest of his term.
(This was a surprise to me. I'm grateful that I'm not at risk of being extradited to rot in a chinese prison for reverse engineering some IoT POS.)
The point of TPP was containing China. China wasn't invited. Trump blew it up resulting in a trade bloc designed to exclude China turning into one that excludes us [1].
Trump's voters don't like trade.
[1] https://en.wikipedia.org/wiki/Comprehensive_and_Progressive_...
Without it their incentive was to cut deals to work closer economically with China…
Trump’s isolationist inclinations empowers America’s enemies.
they were mostly ineffective tariffs, in the "tried so hard and got so far but in the end it doesn't even matter" sense, but he wasn't all talk
There were also more deportations the year before Trump's first term than the year of his first term.
Consider the chart of border encounters. This one shows the past through 2023 [0] and in 2024 we had 2.1 million encounters at the southern border [1]. It appears that deportations are a function of border encounters, and border encounters increased substantially during Biden's administration.
0. https://www.nytimes.com/interactive/2023/10/29/us/illegal-bo...
1. https://www.migrationpolicy.org/news/fy2024-us-border-encoun....
It seems like Trump’s biggest problem is, like himself, the people he put in charge are C-players, so getting anything complex done is hard.
I once reported to a guy that set all us minions (product managers) against each other. It was awful.
Really? It was very widely reported. Just wait until you see how many people Obama deported
I think this is an essentially meaningless number until you also include the total number of illegals that were let in during the same period for each administration.
That's a big part of why the Biden administration (on day one) proposed a bill that created a large economic development grant for LatAm while also modernizing border security.
He had the net effect of increasing the amount of time a dangerous immigrant would be inside the country while the system got clogged with all sorts of cases that the American public broadly agrees hardly matter.
It's like a cartoon version of a shitty manager.
https://www.themarshallproject.org/2019/07/16/trump-tried-to...
Maybe we're looking for the wrong effects. Most crime in America is not committed by illegal immigants. Getting rid of violent criminals through immigration courts faster isn't going to result in any crime drop a voter would notice.
The part people will notice is wage inflation. And that's, simply, a numbers game. Furthermore, it's not violent criminals who are competing with Americans for jobs, it's the law-abiding hard-working ones. The ones whose worst offence since crossing the border was having a taillight out.
If you're being Machiavellian, the target is those honest migrants. The ones on dole or committing crimes aren't creating enough problems. Their role is rhetorical. (And honestly, if they get worse, it isn't like people will vote for Democrats. Perverse incentive.)
I tend to default to Hanlon's razor [1]. But it's difficult to call a guy who has won the Presidency twice, in remarkably different situations, an idiot.
It doesn’t take a genius to come up with that one.
Says more about voters and our information environment than it does about Trump.
Actual solutions to complex problems tend to be complex. Fake solutions can get away with being 3 word slogans. The latter has a huge structural advantage in politics, especially in our current information environment where people can’t focus on a single problem for more than 5 seconds at a time.
It is not about actually achieving something.
Break the process and it looks like you have a bigger problem, so it's a political win - if you don't care about actually doing the thing you say you care about.
Are you an ex-wife trans immigrant f us all over?
Thankfully we have archives and youtube, entirely searchable, where from 2008-2012 the democrats were strongly for sending people back home.
I think this is one of those times where two people agree on the evidence but not the resulting position.
It's very easy to fight straw men like all progressives are against all deportation and border enforcement.
Is it ? Outside of the extreme progressive ideologies , what about the progressive platform would be against deportations?
Ive constructed a Sankey diagram of all the flow of visa-less immigration.
Direct citations included.
The biggest loss I see is Organizations - Department of Energy, 5473 to 3647. I also see under Bureaus - Energy Programs, 4347 to 2521. These are overlapping categories (-1826 on both).
There are others, but they seem smaller, a few hundred at most.
I know DoE does "other things", but I don't expect these to be public anyway.
Could it have been a publishing mistake, or some order to undo recent publications for review by the new admin?
If the number of dataset saw a massive jump (50%), then back down a week later, that seems more like the correction of an error.
Comical to think the Trump admin would be the ones "scrubbing" data when they passed the OPEN Government Data Act as part of the Foundations for Evidence Based Policymaking Act at the beginning of 2019. The law that requires agencies to publish their data to the site!
Trump must really be playing 4d chess if one of the first actions in office is to "scrub" a few obscure and yet unidentified datasets from data.gov.
Also calling SVI obscure is quite hilarious.