U.S. Census director resigns amid turmoil over funding of 2020 count | Hacker News Reader

U.S. Census director resigns amid turmoil over funding of 2020 count

washingtonpost.com

201 pointspetethomas137 comments

It's funny how easy people think this is.

Yes it's "just a data collection app".

You're talking about an org that, if your address is in the middle of a swamp, they will send someone by boat to find you.

They take collection and processing very seriously.

If you gave a startup 200 million or whatever, you'd have a pretty accurate census of internet connected people in the top 20 cities in the US. Oh, and a declaration of victory from the startup, plus 300 billion in market cap in the hope that they may actually be able to count everyone someday!

Somehow, i doubt you'd come up with something that can collect data well, know where to be focusing field representatives, etc with 600k+ field representatives in a reasonable and efficient manner.

Has anyone here tried to organize people,targets, and data in say a company twice the size of all of IBM? How did that go? :)

(the census is honestly relatively cheap. it costs about 50 bucks per person, total. Obviously, counting rural, etc areas is the majority of the cost)

The 2020 Census is going to be weird, too. Disadvantaged minorities, etc are statistically less likely to be counted[1], and especially given the climate in the US, counting immigrants is going to be especially hard.

[1] Which is why in the US, they democrats often try to pass rules around using statistical methods, and the republicans claim it requires actual enumeration

The difficulties you mention are entirely related to the operational costs of carrying out the census. I see no justification for the data collection system itself to cost over $900 million.

Err, no.

You understand this isn't a dumb mobile app someone types something into with a simple server backend right?

It's the system that manages and handles the operational data collection and statistical analysis for the census.

It's not a CRUD app bob types something into at the end of the day. It manages the whole effort.

And the cost likely includes wages/costs of people for things like training users on the system, setting up and running a help desk for census staff to call when they're completing the census, costs for managing and running procurement for the different components, etc.

You've told us what it isn't, why not tell us what it is. "manages the whole effort" is pretty meaningless, a time sheet application can make that claim.

Imagine building software to handle all the HR needs, IT needs, accounting needs, CRM needs, etc, for a 600,000 person company. Assume that this is also going to be the mother of legacy-tech-debt projects, too. Then, add on the feature of ingesting and actually processing and storing the raw census data, too. Then imagine that it's not a company, but a government agency doing that.

>Then imagine that it's not a company, but a government agency doing that.

I think that is the key takeaway here.

I think we all agree that's it's not trivial. What is far less convincing is your case for it costing $900 million. Dispatching people to addresses is not a novel problem, package carriers do it every day. There isn't some massive R&D effort needed here. The tech already exists, the task at hand is merely a matter of application.

I know several people who worked for the last census. The issue you run in to is that people don't only live at addresses. For instance, they attempt to count all homeless people. They attempt to count everyone in prison. They try to count every single person in the United States.

This is not as easy as you make it sound. For instance, how would you make sure you counted every transient effectively?

The way the census solves that problem is to go out and count all of them in a single night.

That's a level of accuracy mot people couldn't even begin to plan for. If you tasked FedEx with that problem, they'd probably find all the easy people and then hand off the rest to the USPS like they do for excessively rural customers.

Now you're back to harping on operational costs. It doesn't take a $900 million application to assign people areas of a city to canvas.

Tracking that you've performed the operations correctly is extremely complex, necessitating an extremely complex application.

What are you talking about? How is it extremely complex?

So far this is a simple "business" program (there's nothing that requires inventing new Computer Science things, or new hardware, or writing a kernel module, or designing for ultra large storage, or ultra low latency, or global consistency), the requirements should be listed by order of importance by the Census stats and ops team, field teams, administrators; then ask a few teams to come up with a high-level sketch; select 3 of the designs, let 3-5 teams prototype and then assign the remaining of the work to the team that was the most competent.

It's not a 900 000 000 dollars job. That means you can employ thousands of Google engineers for a year to build it. That's insanely expensive.

As someone pointed out, here's the doc and scope:

  - Research and Data Analytics
  - Strategic Planning, Program Development, and Integration
  - Communications Support for Decennial Census Operations and Other Programs
  - Field Recruitment Advertising and Communications
  - Traditional Advertising and Media Buying
  - Digital Advertising and Other Communications Technologies
  - Social Media
  - Public Relations
  - Communications Planning and Materials for the Partnership Program
  - Statistics in Schools Program
  - Website Development and Digital Engagement
  - Rapid Response Activities
  - Project and Financial Management
  - Stakeholders Relations
  - Communications Support for the 2020 Census Data Dissemination

ftp://ftp.census.gov/about/business-opportunities/2020-comm-final-rfp-1-21-16.pdf

I don't think your simple plan will ever come up with something that covers all the requirements in finite time.

How long do you think it takes "3-5 teams" to prototype something covering even a miniscule amount of the above?

This is not surprising though, SV always thinks it has some idea how to solve government scale problems, but most "problem solving" in SV is done by eliminating the customers you don't want to care about.

All I claimed was that the "product development" should not cost 900M USD.

But this is a communications campaign, a PR/PSA whatever, and recruitment and so on.

The actual IT systems for the census seem to be in-house developed by the Census Bureau. (Or at least the prototype was, and it's already in place.)

I don't think people are saying it's easy. People are saying it's not $1.5b with an additional $1b for IT hard.

based on what experience managing what large scale efforts?

Just because you're not a quarterback doesn't mean you can't know when the team sucks.

"God, those Sportsballers are terrible!" "Shut up, John! Like you would know, do you play football? You didn't even make tryouts!"

"Just because you're not a quarterback doesn't mean you can't know when the team sucks."

This is a great, meaningless, soundbite :) I was serious. People just dismiss the complexity out of hand, without even bothering to think that it might be harder than they think.

Heck, if you want to go with your analogy: It's like saying "this team is too expensive to run" with not even the slightest idea of what actually goes into running a team.

That isn't a thing you can do with any real sanity. you can't just stare at the small little piece you think you see and say the whole thing is a waste of time and money without any idea what is actually involved.

> The 2020 Census is designed to cost less per housing unit than the 2010 Census (when adjusted for inflation), while continuing to maintain the highest standards of accuracy amongst all population groups [0]

If it was obvious to the census this would be so expensive they shouldn't of advertised the system to congress as a cost saving measure. I value the census (probably more than most), but I won't cry foul about budget cuts until I'm convinced the per person cost of data collection is actually rising, as Thompson and Serrano seem to think.

[0] https://www2.census.gov/about/budget/FY2017-census-budget.pd...

> They take collection and processing very seriously.

Well, it is a constitutional requirement. https://en.wikipedia.org/wiki/United_States_Census

But no problem, if they don't have enough money, they can just ... not do it. Or they can just count the ones that volunteer to be counted. That'll save gerrymandering having to be redrawn.

The view of how much money this actually is is seriously skewed in these comments. US GDP is around $19 trillion. The cost of this program is 0.1% of GDP. For comparison, the B2 bomber cost $2.4B itself, and the Iraq was cost upwards of $12B a month. One can argue "we shouldn't spend on those either" (and I agree), but it's impossible to cogently argue that this is a lot of money. Like the budgets of the NEA and many other very cheap programs, it's an absolute pittance. A pittance.

0.1% of GDP is actually fairly large. In isolation it's fine, but government probably have dozens or hundreds of similar "small" programs we never think about. So sure, wars cost a lot more money, but running a hundred programs that each cost you 2 billion dollars will also be very expensive.

Not fundamentally disagreeing, I just think it's easy to miss the full picture if you conclude that 0.1% of GDP is negligible.

I saw 0.1% and freaked out.

Your parent was off by an order of magnitude: $2B / $19,000B == 0.01%

Agree with both points, 0.1% is not nothing - that's what NASA scrapes by on. FWIW, the article mentions a ramp up in spending going up to the census year - so I take it that this is not an annual cost.

Troubled funding for the census leads to easier gerrymandering.

Today had been full of surprises.

That was my immediate thought as well.

That's the Trump MO. They drop something big like the FBI director that they have in the bullpen to overshadow another turd like this.

Underfunding the census directly hurts the democrats, as urban areas always need more human outreach than the burbs, whose residents dutifully fill out their surveys.

It's not just gerrymandering, its outright warfare. Census numbers drive things like sales tax distribution, school aid, and local government aid too. The strategy here is to starve out democratic strongholds.

There have been confessions dating back to the Nixon administration of war like tactics to directly harm the democratic base (and plenty more before him). It is not new, they can just be more blatant the more drawn the partisan lines are into independent isolated silos.

After all, what are republicans distraught by this behavior going to do? Side with the enemy, whose ideals their culture have defined as not just wrong but evil? Or in the inverse, is a liberal dissatisfied with the democrats behavior going to side with those whom the left deem imbeciles and tyrants?

We saw in this last election what voter disdain in the major parties causes. A bunch of third party votes, some minor protests that the media systemically silenced or condemned, that still pale next to the structural rigidity of first past the post and a firmly established ideological divide meant to keep the poor fighting amongst themselves.

as urban areas always need more human outreach than the burbs

I would disagree as others have pointed out. It's the rural folks who couldn't care less that need to be tracked down. A single person could easily canvass an entire neighborhood in a large city.

And I think the comment about dropping the big news at a convenient time is something every politician does. How many times have you seen announcements by the gov't go out on a Friday afternoon? Avoid media attention!!

> urban areas always need more human outreach

Um, what?

It's the rural areas that are a nightmare.

I lived in a rural area during the last census, and they had to go all out if they wanted to count every cabin and cottage. I'd much prefer to be assigned to 100 people in a city. They'd probably all be the same apartment complex.

It does? How?

Indeed I don't think these two issues are related. The data used to gerrymander is results of previous elections and party registration information. The ability to gerrymander is based on party control of state governments. Neither of these things have anything to do with the census.

Population distribution is discovered through census data. That is what is used for redistricting. Not the previous election.

If all you have to go on is population distribution you wouldn't be able to effectively gerrymander. You don't need to just know how people are distributed, you need to know who they are likely to vote for in an election.

> You don't need to just know how people are distributed, you need to know who they are likely to vote for in an election.

That's not how gerrymandering is used.

Gerrymandering as a practice targets minorities. It's based on (highly) educated guesses. You take a minority population center and draw the district lines through it, bisecting the voting power of the population. What would be one whole voting block in one district is not split over N districts (N districts with large Republican populations).

Minorities are more likely to vote (D)emocrat. Republicans get so few minority votes that it doesn't matter where a republican minority votes. Their vote is good in any district.

Going back to the census, if you don' have adequate funding for the census, then some districts will be underfunded. Underfunding generally means an undercount, not an over count. If you underfund (and undercount) minority districts it's easier to redraw the districting lines to break up the minority district and include them in the surrounding districts.

That was maybe gerrymandering before computerization. Today we have software with good block by block data.

From http://www.salon.com/2016/06/13/this_is_how_the_gop_rigged_c...

"The smart set that denies this also doesn’t understand how profoundly the technology changed between 2000 and 2010, let alone between 1990 and 2010. In 1990 and even in 2000, the computers and the data sets were primitive. In some cases, they were still laying out actual parchment maps and using markers. By 2010, programs like Maptitude–and all of the public data sets that are available, as well as private data sets that the parties can purchase and add on to Maptitude–made it as easy as clicking a mouse to shift a line one block in any direction. And the data was so good, and our partisanship had hardened in such a way as to make it pretty clear how individual blocks vote, that you could see how shifting the lines would likely shift the results. As a result, you can draw districts so precisely that they might even look like competitive 51-49 districts, but they can still be reliable partisan performers. It is a completely different world. The Republicans know this. The Democrats and the media are still catching up."

I'm mostly surprised that it took this long. Crippling the census should be a high priority in information warfare.

A bit unrelated but the US Census should really work with the US Postal Service when performing the census. It would save the Census bureau some money and would provide the Postal Service with an additional source of funding.

I'm sure both departments might like that but unfortunately they're currently in the grip of ideologues who would like to defund both departments and shut them down permanently.

US Census can't really work with anyone when it comes to their data. They take their responsibilities under Title 13 [0] (which prohibits them from sharing information they collect) very seriously.

They're trying various options to share some limited data (like address data) [1], but haven't gotten very far. They've done some work on "Community TIGER" [2], which aims to give validation information back to local governments for geographic data, but not the improvements that Census generates as part of the decennial census.

The US Postal Service has no motive to share with the Census Bureau. For one thing, the USPS makes all of its money selling limited access to its address list to advertisers. Additionally, the USPS's address list (or delivery points) doesn't necessarily correspond with people and where they live.

PS: I'm interested in this sort of thing because I help run OpenAddresses [3], a community-built list of authoritative address data sources from around the world. There's a lot of data out there!

[0] https://www.law.cornell.edu/uscode/text/13/9 [1] https://fcw.com/articles/2011/09/14/census-bureau-title-13.a... [2] https://www2.census.gov/geo/pdfs/gssi/Community_TIGER.pdf [3] https://openaddresses.io/

I used to be GIS Coordinator for a medium size municipality. The Census Bureau sent out DVD's of address points for the 2010 count for cities to validate. There was some collaboration between cities, USPS, and Census on it but as you likely know address standards are a mess in the US. Many states have efforts to standardize but it's often difficult because addressing is left to the municipality and often falls to a non GIS person (city planner, building inspector etc) who has little concept of normalization or why it's a bad idea to create addresses that are difficult for a computer to understand.

Openaddresses.io is an awesome project, as a GIS professional thank you! This type of data can be immensely useful!

Our of curiosity, what sorts of addresses are ones that are difficult for a computer to understand?

All addresses are pretty hard for computers to understand, so the people building the software to make them understand (geocoders) usually make them work for the most 'in-demand' address formats. Just like every other bit of software, finding the special cases and figuring out how to understand them is tricky.

http://www.columbia.edu/~fdc/postal/ is a pretty good overview of most of the postal addressing systems around the world.

It's almost always just a geocoding/data format issue. It's machine readable but you can get quirks when an address is something like "1 1/2 Hacker Street" - is that "1.5 Hacker Street", "Unit 1 1/2 Hacker Street", "1 Hacker Street, Unit 1 of 2" etc. It might not be a problem if you are dealing with a small discrete area but when you are trying to merge multiple areas it can be a challenge.

Then you have situations where a street name is the same as the region/state (is the source data just missing attributes?), confusion about the direction prefix/suffix (1 W Hacker Street, 1 Hacker Street W - can vary within the same city on the same street), poorly named streets like "1 W Hacker Street East or "1 Hacker Street East W", or street name types that are not commonly used (1 Hacker Launch Pad etc).

It's even more fun when you start talking about countries that either use older addressing formats (I've seen addresses in Ireland that are just like (Old Blue Cottage, Some Town) or are not in english.

There are many more situations, these are just what immediately come to mind.

I'm curious what caused the cost of the new electronic system to increase so much. I understand there could be a lot to it on the back end to ensure privacy and anonymity of the data collected, but it doesn't seem like it should be a huge deal technically. We're talking about ~325 million people and collecting demographic info and address info [1]. The IRS has far far more variables to collect info on, however they have fewer people. 325 million people is nothing compared to scaled companies like Facebook, Amazon, or Google (far more data points per person, far more people).

Any speculation on cause or other considerations I'm missing? Did a quick search on Google News and didn't find anything. All the companies I listed have huge teams, but I am still not seeing how the cost has exploded.

https://www.census.gov/history/pdf/2010questionnaire.pdf

Pointing to the highest performing companies that have been selected by the market is not really a fair comparison for a government agency.

It's more relevant to look at what a median tech company would end up doing, if they had to launch at scale without organic growth but had unlimited funding.

Totally fair, but to play devil's advocate for a split second, let's not forget that the Census Dept has to make people aware of itself and get people to give their info willingly and within timelines, given no real incentive to do so.

Facebook, Amazon, and Google are collecting info that is willingly passed their way.

We are probably talking about buying data entry and management bloatware from a big mediocre enterprise contractor and then integrating the monstrosity.

Totally agree... Private sector just isn't going to serve people with a negative ROI and stay in business. Facebook's costs would skyrocket if they had to chase down the last 10-20% of the population necessary for comprehensive coverage.

These are all structural issues with solving the problem, before we even get to a procurement process that picks a mediocre LPTA contractor instead of Google.

> Pointing to the highest performing companies that have been selected by the market is not really a fair comparison for a government agency.

Fair point. All the companies I pointed out are flush with cash, making the annual budget of the census comically small.

So if we want large publicly visible government software projects to start going well consistently, what would it take? Recent poorly run projects have included the failed launch of healthcare.gov which was "rescued" by SV techies and this project which is way over budget. There are many more which are smaller, less visible to the public, and I'm sure there are a set of projects which get done on time and on or below budget.

The government contracting system is fubar.

Everything is done by contractors and has layers and layers of nonsense that has to be tracked and complied with. For a program like the census that includes everything from development to Helpdesk to field service nationwide. I bet the end user support alone is $30-50M through a web of contractors.

Work like this also gets deliberately broken up in chunks to lock in the votes of influential congressmen. More chunks are less efficient and more expensive.

When I worked for a non-federal US government, the process for a small business to get "on contract" to actually bid on many projects cost anywhere from $20-50k in legal, insurance and compliance costs.

>I'm curious what caused the cost of the new electronic system to nearly double

Nitpick: the article mentions that the cost estimates "had ballooned by nearly 50%", which is not the same as doubling (which would have involved the estimates increasing by nearly 100 percent)

Thanks, fixed that. Got mixed up between reading the article and typing the comment.

My guess is that the contract under scrutiny to the RFP COMM-16-BC-2020. Here are the services to be provided:

  - Research and Data Analytics
  - Strategic Planning, Program Development, and Integration
  - Communications Support for Decennial Census Operations and Other Programs
  - Field Recruitment Advertising and Communications
  - Traditional Advertising and Media Buying
  - Digital Advertising and Other Communications Technologies
  - Social Media
  - Public Relations
  - Communications Planning and Materials for the Partnership Program
  - Statistics in Schools Program
  - Website Development and Digital Engagement
  - Rapid Response Activities
  - Project and Financial Management
  - Stakeholders Relations
  - Communications Support for the 2020 Census Data Dissemination

The scope here is pretty large and it includes advertising, etc., so one can see how it might cost hundreds of millions. It isn't just about setting up a database and running a web site.

Full document is here:

ftp://ftp.census.gov/about/business-opportunities/2020-comm-final-rfp-1-21-16.pdf

That sounds like it covers just about everything you would reasonably expect to be part of the preparations for the census. Which raises the question, what's the other $2 billion of the bureau's budget over the next two years being spent on?

Continuing to run the other ongoing projects that the census does, like the American Community Survey.

Maybe they could just eliminate the census entirely and just have Congress pick whatever numbers they want. That would save a lot of money. \s

I know you're joking, but for those who may not be familiar, the Census is mandated explicitly by the Constitution. So that would require a constitutional amendment.

This is something to keep an eye on, given the fact that Congressional district borders will be drawn using 2020 data.

Seats in the house of representatives are allocated based on census data. District borders can be drawn based on whatever data state governments want.

650 million dollars to tally some simple demographic information? I understand there is always more to the story - but this doesn't pass the smell test. Additionally, what does the Census Bureau need 1.5 billion for on a non-census year?

I don't mean that rhetorically - 1.5 billion a year pays 15,000 people an annual salary of 100k. Where is that money going?

AWS bills

"Most recently, Census 2010 employed 635,000."

https://www.census.gov/history/www/faqs/agency_history_faqs/...

That's during the census itself, which is still three years away. The funding in question is just to develop the infrastructure to record the data.

Per the article, spending typically jumps in the years leading up to a decennial count.

And it is pretty important, as I'm assuming it gets fed in as base data to a lot of models (and our representative democracy).

Ever done a census?

Several hundred million dollars would equate to a few dollars a head in the US, which my gut feel says is not bad value (and may even be an unrealistic underestimate)

That doesn't mean there aren't cheaper ways to generate specific figures/estimates than a census, but an important point is that those aren't censuses.

Tabulation of the data is not the most expensive part, although even that is far more complicated than most realise. The data is literally not there to tally: you have to collect it.

Now keep in mind you have to target and include, in no particular order:

People with no homes People with no internet People with no smartphones Old people Young people People with lots of money People with no money Newborns The dead Non-English speakers The illiterate Etc.

You need interviewers, transport, pre-testing and callibration, actual implementation, post-testing and calibration.

This is all without even considering finding the people with the right skills to design, implement and coordinate this, and without accounting for any software or capital costs.

Additionally, censuses ramp up and ramp down, you can't just put aside funding for the census year: systems of that size cannot be hired and deployed in that amount of time. It's a bigger operation with more bodies than many military exercises, and I'm guessing actually they do it N times cheaper.

I'm assuming that the US census is much like ours: in the pre/post years you're planning the next roll out, analyzing the things that went wrong last year, experimenting with improvements, all with a possible backup-back out case in case those experiments go wrong and a hard deadline that cannot be missed.

I don't know, it starts to seem more reasonable if you consider that they're collecting simple demographic information from everyone, including people without computers or permanent addresses. My guesses, from most to least plausible:

—Personnel. As skybrian mentioned, the last census employed half a million people [0], so I assume this system has to keep track of all of them and what places they're assigned to canvas.

—Hardware. The 2010 census used 500,000 handheld computers [1]. This cost probably doesn't include any handhelds, but it does mean that it has to receive data from devices.

—Mapping and GIS. The census releases data for tons of geographic levels, so this system needs to either interface with whatever existing system they have or implement all of that.

—Data formats. This information is collected in multiple formats (phone, paper, in-person survey), possibly including scanned images.

—Related information. The census has to count everyone, so they also have to track a huge number of addresses.

—Other surveys. The Census Bureau does more than just the census itself, so that cost might include other surveys.

Even if it's more expensive than it should be, it's still a pretty good deal. Wikipedia says the census determines how $400 billion a year in government spending is allocated. It's not sourced, but that seems plausible. The 2010 census cost $15 billion dollars [2] in total, so that's less than 1 percent.

[0]: https://www.census.gov/history/www/faqs/agency_history_faqs/...

[1]: https://en.wikipedia.org/wiki/United_States_Census_Bureau#Ha...

[2]: http://www.thefiscaltimes.com/Articles/2011/01/13/Complying-...

Edit: fix formatting

> And it comes less than a week after a prickly hearing at which Thompson told lawmakers that cost estimates for a new electronic data collection system had ballooned by nearly 50 percent.

I'd love to know the details of that project overrun.

Honestly, so would I. I took a project management class in college, and many of the examples were NASA projects because they were sexy. Sadly, I don't remember any case studies referring to project overruns. And, seriously, if you're looking at NASA projects, you shouldn't have trouble finding examples that went over budget or didn't make the deadline.

There are good lessons to learn from less-successful cases.

The latest PMR is online. https://census.gov/programs-surveys/decennial-census/2020-ce...

Should the new electronic system really cost that much? Do they work with the SV, or will they just give the project to a terrible contractor like the health insurance exchange market?

It's ridiculous this is the top trending submission when the president fired the FBI director that was investigating his ties to Russia. If political stories are fair game, this is important but a rounding error next to Comey.

@dang, the HN ranking system is so trivially hackable via downvotes and flags by a motivated minority.

(1) It's been known since forever that Trump and the FBI director didn't see eye to eye, and the former employs the latter. Still significant, just not surprising.

(2) This story is tech related and way more on topic for HN then pure politics (https://news.ycombinator.com/newswelcome.html). I'm sure Reddit is having kittens, if politics is your thing.

> It's been known for a long time that Trump and the FBI director didn't see eye to eye, and the former employs the latter. Still significant, just not surprising.

The president firing the FBI director is huge news. The story is currently at the top of every single message board and news outlet except this one.

Precisely why it doesn't need to be here.

https://news.ycombinator.com/newsguidelines.html

> Off-Topic: Most stories about politics, or crime, or sports, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's probably off-topic.

(emphasis mine)

Fact is that submission this is the sort of thing that could be interesting to the HN community and will probably get lost in the mainstream with news of Comey going. Comey going is huge news - there are other places to discuss it.

Disagree with this rule. This forum should not be a place for just showcasing freakin personal tech project only. Yesterday I read someone asking the community share how they spend their work day. One HNer said he works as a professional firefighter. Now I am not sure if he has done any programming or not but that's unexpected for a technical forum like this. I think technolgists like us have the same obligations as everyone else to particpate in civil matters professionally. We don't want to yell out the f word here but marking politics as off-topic is obsurd given privacy is a hot topic we almost never escape from politics. If neutrality has been covered on the news, dont't allow it. But hey I don't run HN so someone is going to tell me read ToS.

HN should embrace submissions either directly or indirectly related to politics. I feel our tech leaders are so silent today to the White House so silent on Trumpcare. Where the heck are they?

I come here specifically because it's not one of the million billion other places where all other stories get steamrolled by alt.gov.trumpdrama. I can think of two dozen places in less than 10 seconds where political news, analysis, and all of the related stone-throwing get top billing, and I utilize many of them regularly... my 1337 dev skills don't somehow make those political resources less useful to me. If there are government related stories with a technical bent, they do just fine here. Just because tech people have a civic duty to be aware of what's going on in the government doesn't mean that all tech communities should be political.

It's called finding people with similar interest. Knowing other technologists are interested in solving social issues isn't something we easy can get on othet sites which are overpopulated. HN is in a good size and the folks are mostly friendly and can the capacity to discuss professionally and openly.

I am disappoited that many of the tech leaders don't speak out hard enough. If I were as powerful as they, I wouldn't stand back. The fact our Congress is a disgrace when it comes to passing the new health care bill, we shouldn't hold back. There is no reason to. Because while that bill may not hurt thousands of Google/facebook employees, it will have impacts on their users and many of their own employees' families.

"It's called finding people with similar interest." That's what we're doing here. It's Hacker News. Those similar interests are technology and innovation. If politics are involved in tech stories, then the stories are here (hence the story we're commenting on.)

When you say your website is 'for' something, the implication is that you're 'not for' something else. This website is for technical news.

The fact that these "leaders" are involved in technology is mostly incidental, which is why some random tech CEO's opinion on health care doesn't get a whole lot of traction here. Unless there is a technological bent to their take on something, it's not really a technical story– it's yet another rich person that has something to say about politics. Even the ones who became notable for technical reasons, when speaking on non-technical political topics, are no more notable to me than Hollywood personalities doing the same.

I'm not saying you're wrong for caring about what they think or thinking that they should speak up more– I'm saying that the story would be newsfeed fodder at every site from Breitbart to Dissent Magazine, not to mention a million specialized sub-reddits. But how many places can I go to see a neatly ranked list of quality techy stories, such as the latest git release, some neat electronic music composition teaching demo, and news about modules in the JDK? Very, very few. The fact that this is a narrow niche makes it a resource for many people. We 'need' this space to discuss this tech stuff which is important to us as technologists, but given equal footing, would get crushed by stories about world-leadership-level politics.

So I fail to see any reason behind your implication that this group of people interacts with political news so differently that we need our own space to discuss them; we're not even unified in having a particular interest in politics, let alone solidarity on specific political viewpoints.

If you're really sure that I'm wrong, then why not leave Hacker News to start your own political-tech website? The hacker news source is available on Sourceforge. Make a fork and make a poli-hacker news. If people are interested, they'll show up. Even just starting a "poli-tech" subreddit to see if anyone bites.

My theory? We're just not that special.

(As an aside, I also think you're greatly overestimating the likelihood that tech CEOs opinions would align with your own, and it's quite likely that's why they're keeping their mouths shut to begin with. I know if I worked for some startup with a CEO who was fine at running a business but REALLY loved Milo Yiannopolis, I'd be pretty happy that he kept his mouth shut so our company would stay afloat and I could keep paying my rent.)

Who cares if they didn't see eye to eye. The FBI director was investigating his ties to Russia which could have been criminal. Close to a dozen FSB agents or people with ties to the agency died in suspicious circumstances after Steele's Trump dossier surfaced. Saying "the former employs the latter" grossly understates this - they serve 10 year as a compromise to prevent the accumulation of excess power (Hoover) but to insulate them from exactly this type of thing.

The director of the FBI serves at the pleasure of the President, as evidenced by the fact the latter just fired the former. If you don't like that, then I suggest you work on passing a law preventing it.

Obviously an employee of the President isn't going to police the President very effectively. Historically, that's what special prosecutors have been used for.

Yup. From a different branch of government.

How many months or years must he remain employed for a witch hunt that likely will never end?

The dossier had been unsubstantiated for months when it was first reported, and it's been months and it's still unsubstantiated. This despite the media being decisively against Trump and it being the juiciest imaginable story. It's time to move on.

Why?

This is a far more important story than Comey.

Why do you feel this story is more important?

Appointees serve at the pleasure of the appointing authority. Firing one is a story, but ultimately not earth shattering.

The Census is something that is critically important to all sorts of "backend" processes in government. For example, your county probably apportions sales tax based on census figures.

It's sort of like the criticality of the employment and inflation statistics from the Department of Labor. Many processes assume that these tasks are completed correctly and with integrity.

When senior people like his who are cogs in the wheels of government resign at the prime of their careers, that means that something is broken or is about to be broken.

And let's not forget that the Census is the Constitutionally-authorized function for collecting the population data needed to enumerate representation in the House of Representatives: https://www.census.gov/programs-surveys/decennial-census/abo...

It should be noted that the Director of the FBI serves a 10 year term, specifically to avoid the "serving at the pleasure of the President" item.

The census happens rarely and impacts politically important concepts like congressional districts. By undercutting the ability to have an accurate census, people can gain political advantage. Right now the Republican Party is basically propped up by older white men and women, but that is a rapidly diminishing demographic in the US. Pretty soon it will be dwarfed by other groups of people. By skewing the census, and the rarity at which they happen, Republicans can gain another 10 years or so of influence beyond what they should have (going by the numbers).

Yeah, this seems pretty serious to me. It's not a drop in the ocean as the op makes it sound .

I'm not trying to minimize this seriously bad Census Bureau news, but I am trying to contextualize it.

You can't downvote submissions.

Some of the things that affect submission ranking is number of points, (increases) age, (decreases) and number of comments. Somewhat surprisingly, having more comments will penalize the ranking of a story, to push flamewars off the front page. This is to prevent HN from being Reddit, where r/all is a constant rolling flamewar. (The FBI submission has 5 times as many comments as this story, and has already fallen off the front page)

1) It happened 3 hours ago, 2-3 hours after the story in this submission was filed; 2) You can flag submissions, though, and you can downvote comments - and both lower the story's rank; 3) You can also spam a story with critical/spammy comments to provoke downvotes, also pushing down its rank.

Easy peasy.

1) Okay, so the comments penalty is especially strong. Sorry.

2) I don't think that downvoting a comment on a submission will affect that submission's ranking. If you leave a bunch of spurious flags the moderators will disable the flag feature on your account, though. (Happened to me)

3) Assuming a pool of colluding users, this would actually be a plausible attack-- except that HN's userbase is tiny, and the moderators are active. Voting rings show up easily, but I don't know if they're looking for "comment rings". An easy technical solution would be to ban users with "throwaway" in their name from commenting on submissions.

Downvotes of comments don't lower stories.

I really strongly had the impression that it was a combination of flags, some sort of metric of polarity (lots of upvotes/downvotes among the comments, and then moderator actions. I don't recall why I have that recollection, but apologies for any mistaken impression. It's probably better that it's not entirely transparent, either.

I managed to push a post off of the first page the other day, because my finger is too big for my phone - I clicked the Flag button instead of the actual link. I immediately clicked Unflag, but it took a while for the post to recover.

Just an FYI - this story is as much, if not more, an IT story, than it is a political one. HN rightly favors discussing this.

As I keep saying, "politics" is a subjective description. When people see something as personally relevant/interesting to them or affecting them or people like them, then it's fair game for HN. When people see something as personally not relevant to them or not affecting them or people like them, it's "politics".

Which tells you some things about HN: the only way you get widespread agreement on what's "politics" and what isn't is by having a monoculture.

It seems to be that the criteria for HN posts is not "personally relevant/interesting to smbdy or affecting smbdy".

The criteria is IT related or scientifically interesting.

Not true. Plently of submissions about books and arts.

Are you people _still_ driving the Russia narrative despite there being ZERO actual evidence? All of the news I've seen posted online are just opinion puff pieces and people having gut feelings. No tangibles.

Yes, because there are red flags and smoke. Not to mention known economic ties. We aren't driving a narrative, we are concerned for the state of the US.

So "you people" who drove the Obama birth certificate narrative for ten years (not only with no evidence for, but concrete evidence against) are criticising a different set of "you people" for driving a narrative?

And a narrative with a lot of clear evidence of Russians in the same place at the same time as Trump team members, and probably a lot of evidence that the FBI, CIA and NSA have but can't release... hardly ZERO (in caps!) evidence. I'm from New Zealand so I don't have a dog in this fight, but it stinks to high heaven.

But both sets of "you people" should read this: http://theoatmeal.com/comics/believe

Neither story is a good fit for HN. HN is for the gratification of intellectual curiosity, not the other more intense forms of curiosity (or gratification). Like most political stories, these fall out of HN's scope.

> so trivially hackable

That's a misunderstanding of how HN's system works. Hot political stories typically get tons of upvotes. That happens first. Then they get flagged and/or moderators penalize them. That's routine, and it's what happened to both of these submissions—the only difference was the several hours' gap between them.

It's going to be a really hard summer trying to thread that needle. Having seen politics ruin plenty of good forums, though, I sympathize. Appreciate the consistency.

There seems to be a wide variance in the way political stories are handled; for example, political issues not involving Trump seem more accepted. I suppose consistency in this matter isn't that important, except that it's a bit frustrating to spend time on comments or submissions that are (effectively) solicited, and then have them (effectively) discarded. It would be helpful if the standards were explained in a little more detail, perhaps in the Guidelines (so people can find them), so we know what to expect. As I said in another comment, I feel like I'm trying to reverse engineer your policies and intentions.

While personally I wish there were more political issues discussed intelligently on HN - we shouldn't imagine that we live in an enclave safe from the very serious political situation - I will say that the recent discussions have been the most uninformative, valueless, wastes of my time that I've seen on HN in a long while. It's not hopeless; around the time of Trump's travel orders, for example, they seemed much better (and were allowed on the front page).

A lot of this is probably random. The system has a wide range of fluctuation, and trying to pin that down, clean it up, or make it consistent would likely just generate more noise.

The should do the census and register Americans to vote in the same process.

1bn is a bit much for a CRUD app... I'm exaggerating of course, but I believe not as much as they are on the costs..

what? the census requires massive amounts of people to do all sorts of stuff including large numbers of people to actually conduct the census.

$10k for the app, $250m for the IBM hardware, $350m for the IBM support contract, and $399,990,000 set aside for legal defense when the government decides what they spec'd wasn't what they needed and sues the contractor.

Odds that Facebook ends up doing a privatized 2020 census?

Making all US Citizens legally obligated to have Facebook account

What would happen if Twitter banned Donald Trump for a tweet? Could he compel them to reinstate him?

No. 10th Amendment.

Ugh, lets hope not.

0. An opt-in Census is useless.

By 2020 President Zuckerberg will mandate all citizens have Facebook accounts.

Prompting the few remaining Google+ loyalists to flee to the mountains, Red Dawn style.

I bet they could get it done for us for half of the estimated 1bn cost of the "new electronic system" alone.

Ugh, so far I've been able to avoid the Facebook bandwagon (and according to me teenagers there will never be a reason to use Facebook - apparently Snapchat is the new Facebook). That would suck if I had to get an account just to participate in the census.

The above comment is an excellent example of political gaslighting, there is ample, undeniable evidence of some sort of GOP collision with Russia. No sane person looking at the evidence could come to any other conclusion, but by confidently stating the exact opposite of the truth the above commenter seeks to sow doubt in the mind of a potentially disinterested or confused audience. This is a an increasingly common tactic on these boards.

"Gaslighting" is certainly going both ways, but it's important to me that one side is horrifyingly eager for war with Russia and the other... isn't. I don't care what "the truth" is; I just don't want the current generation of humans to be the last. And for what? Politics? Truly, it's the mind-killer.

Please don't post like this to HN. It falls into the category of insinuating astroturfing/shillage by other users, which (unless you have evidence) is a breach of civility that is not allowed on HN. Someone holding an opposing opinion isn't evidence.

The "you must be a shill/troll/spy because of what you say" argument is common, toxic, and false. People need to abstain from it in this corner of the internet.

We detached this subthread from https://news.ycombinator.com/item?id=14305376 and marked it off-topic.

Can we use Machine learning to predict 2020 count? Do we really need to manually count?

Why hire nearly a million temporary census takers[1] every ten years? Even private companies have databases with far more detailed and more frequent demographic and psychographic information on every person in the United States.

[1]: https://www.census.gov/history/www/faqs/agency_history_faqs/...

Well we need some independent way to regularly reapportion representatives, if we're going to stick with a system of geographically-based representation.

Also, if you're suggesting that we don't need to literally go door-to-door counting people, you're not alone. Many people argue that statistics will give us a better measure than one that has the flaws of in-person data collection.

The problem is that the U.S. Constitution says the following:

> which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons. The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.

The key word is Enumeration. That arguably means counting, one-by-one, rather than making a statistical estimation.

It's not entirely "the same old way." There have been some improvements since 1790.

Honestly, why is it so expensive? If you created a startup with $50 Million, and gave them 4 years to implement this system, I'm sure it would be done much more efficiently. Then you "buy" the startup for $200 Million at the end of it, and all the employees get a nice payout.

It's equally likely that the startup would spend the $50 million with nothing to show for it.

I'm really struggling with Poe's Law with this comment.

What's more likely is that the founders and investors would take the payout, and the workers would get nothing.

And I extremely doubt the ability of a startup to do any kind of worthwhile count of people outside of major metropolitan areas.

"We counted the Bay Area and NYC, that's enough, right?"

Reply on news.ycombinator.com