The Shadow Inc. app that failed in Iowa last night
vice.com
vice.com
This app emerged from a mandate to make the caucus more accessible and transparent. It was well-intentioned but underfunded and lacked comprehensive organizational buy-in. Introducing tech can help but you have to spend tons of money to make it reliable and usable, then you have to spend more to train everyone in using it. This is a problem organizations of all sizes and shapes face when making massive IT changes.
Shadow is a firm that makes custom software for Democrats and progressives. It has an unnecessarily sinister name. There are not a lot of companies that make software for Democrats because it’s an awful job. You make very little money. Everyone hates you when things go wrong, which they will, because the product testing cycle and margins are nonexistent. Then everyone will assume things went wrong because you are some combination– you choose– of secretly evil, secretly working for Bernie, secretly working for The Establishment/Hillary (per someone's unpersuasive Imgur post below), or secretly working for Buttigieg.
Others have noted that Shadow also made software for the Buttigieg campaign. If you take my claims above as true, this should be unsurprising to you: a hard market where everyone hates you and no one has money to pay you is not attractive to enterprising software engineers, so there are few firms available to choose.
If I can ask, why the specific nature for their clientele? Isolating your solutions to a specific political party seems strange to me especially if the emphasis on making money. If anything, money normally supersedes political leanings at the end of the day which makes the whole fiasco even stranger to me. Wouldn't you want to provide a solution for both parties by default? Not doing so seems nefarious on it's own.
It sounds like pairing products with political leanings is not a good business decision? Which may explain where all the 'hate' is coming from - it's just bad business and sows the seeds of doubt in their credibility in the minds of many.
Even if you wanted to cross the aisle, it would be hard. If you work for Democrats, Republicans will not work with you. If you work for Republicans, Democrats will not work with you.
Campaigns aren't wrong to be concerned about who has their data. There is good historical reason for them to be skeptical that security on your system will be strong enough to prevent customers from accessing other customer's data. https://time.com/4155185/bernie-sanders-hillary-clinton-data...
Now imagine getting buy-in for using customer data to improve your models.
I can't blame campaigns for being paranoid about vendors potentially going across the aisle. NationBuilder is one of the only big names I know that does it. They are in a unique and unenviable position given the death grip NGP VAN has, and they pay a big price for it on nearly every front.
If you take your example of NationBuilder, a company I know well as I led their European expansion, it only sold software not data. Unlike most American campaign software, NationBuilder does not sell data as part of their package. NationBuilder does not believe that you can create a sustainable and powerful community by using data that you have purchased, and thus only sells the platform.
So yes I agree with you that while campaigns can (and should) be paranoid about who has access to their data, and even how it is being used, this data brokerage model isn't the model that all companies adopt and it's a really important distinction to make.
While I am here, I will add the following, tech companies who want to democratise democracy (and I'm not saying that's the mission for everyone) and help lower the barrier to entry, cannot claim to do so if they alone decide who has access to these tools. Crucial decisions, such as which parties have access to the latest technology, should not be in the hands of a few tech titans of Silicon Valley - where the power they already wield is already unmeasurable.
All of the above is a fascinating and important debate and also a very American one. Campaigning and political technology is incredibly partisan in America, compared to Europe where what we fear most are monopolies.
While highly political people I know don't seem to work in tech.
I mean, we’re doing anecdata here, but that doesn’t match my experience at all
So assuming it's possible in principle to be "not very political," the real question is how disengaged, uninformed, or lacking in understanding do you have to be for your statement to be true in itself, rather than a "quite political" statement. For myself, I think if someone is disengaged enough that they don't care who will become the next president, and there are no local political issues they will argue with someone about, they are "not very political."
> I think if someone is disengaged enough that they don't care who will become the next president, and there are no local political issues they will argue with someone about
What this is saying is that they are happy with the status quo, and they see no room for improvement, or they're not bothered if things are improved one way or another. You can be not interested in politics, but I think doing that is just supporting the current politics.
For many people, just existing is pretty political, like when an entire country has a debate and a vote on whether you should be afforded basic human rights.
I think there's something political about it, but I also believe that it could be done for non-political reasons (spiritual reasons, practical reasons, mental health reasons). As a pure narrative, it would most likely be politically motivated, but in hypothetical-land it wouldn't necessarily be. Even if there is something "political" about it, it doesn't really map to what people mean when the use the term "political." It may map to opinions about how the state should be governed, the broadest definition of political, but it's much more frequently used to refer to a particular subset of debates which are at least minimally polarizing in some way.
> What this is saying is that they are happy with the status quo, and they see no room for improvement, or they're not bothered if things are improved one way or another.
All I'm trying to point out is that it's a legitimate spectrum. When I say "someone disengaged enough that they don't care who will become the next president" it's very easy for the reader to imagine someone in shoes very like theirs and making that choice somewhat actively to not be engaged.
But in the wide world, you can be simply unable (or unwilling) to devote any energy to understanding how the current political environment relates to your life, and thereby form no opinions one way or the other. Perhaps you work 16 hours a day to support your family, or you live in a country only temporarily and don't even understand the language. This would imply the fact that you lack the resources to form a political opinion is "supporting the status quo."
I have no idea how to find out whether those tech stacks are indeed different. But if someone knows the answer I'd definitely find it instructive.
Whenever I've done calling or canvassing for progressive causes or democratic candidates we've used some version of NGP VAN (https://www.ngpvan.com/about).
I wouldn't be surprised if there are some cross-party solutions (especially the closer you get to core infrastructure—I'm sure both parties use AWS).
That sounds like a lot of money to a lay-person, but assuming $100/hr, which seems like at least a reasonable ballpark rate, that breaks down to 600 man-hours, or 15 man-weeks.
That is a very tight deadline to turn out a critical app like this. Especially since I assume it wasn't just a single person doing the coding.
People don't realize that software is expensive and a custom built application is probably not the correct choice for a single-use product. I'm sure whoever was in charge felt that $60k was a quite generous offer to build this. In reality it's basically nothing.
I used to have acquaintances approach me and say, "Hey I've got this great app idea you could help me with."
The response I found shut them down real quick was, "Great. Do you have $100k lying around to get the first beta out the door?"
Why an App? We all know there's nothing in it but boilerplate.
And while an app not hard, a https website with login and post capability is on the _extremely simple_ territory. As long as precinct official are trustable, I don't see any reason why they have to use an app.
I guess for the wow factor? As the entire political process is essentially becoming.
Yes, a webapp will cost less. But if the vendor did this bad a job on an app they probably could do just as bad on a webapp.
Changing tech wouldn't have solved the problem for this vendor.
It makes me think of all that uproar for some iPad app for the TSA that just randomly showed a left or right arrow to send tell people which line to get in. Surely easy to code. But I bet it was months or years of interviews to determine that's all they actually needed.
Making sure that average Joe preloads a webapp and to make sure it will be configured a day later isn't easy.
Properly syncing from the web app - as discussed in these threads - adds complexity though.
It's also fair to ask them to keep checking the native or web app to make sure the data was sent, so one doesn't have to rely on web workers staying alive in the background. Just save to localStorage, catch network errors and keep retrying.
Several US territories and other states[1] hold democratic caucuses of one form or another, but are less notable due to timing/delegate count.
[1] https://www.nytimes.com/2020/02/04/us/politics/what-states-c...
No. The plan was that it would be used in the IA and NV caucuses, which is why the IA and NV Democratic Parties jointly purchased it.
The other states and territories using caucuses were never planning on using it. (And mostly, the other caucuses are structured differently with more primary-like features, so it probably wouldn't make sense to use shared software tailored to the IA and NV caucuses for them.)
It should have been enough to fund a business analyst and an app developer for a few months. Plus Shadow Inc's overhead + profit.
It's absolutely bonkers how shitty the industry is. A kid can throw together an app in a few days, but somehow 60K USD is not enough.
That said I work in "consulting", I know how fast time, money, billable hours can be burned through, when there's no real drive, motivation, vision and delegation of responsibilities and control.
BLS, which while not perfect has far better data than most other sources trying to do this independently, says $84,280 in 2018.
https://www.bls.gov/ooh/computer-and-information-technology/...
> It should have been enough to fund a business analyst and an app developer for a few months
Probably. But that doesn't matter if no one advised the client that they needed a realistic, production-like field test (most of the cost for which wouldn't have been part of the contract, since it would have been client side; the developer-side support for that is a small fraction of the cost.) Errors in report definitions that exclude some data in circumstances that don't come up in developer-constructed testing scenarios is, like, not at all uncommon.
[0] https://www.bls.gov/ooh/computer-and-information-technology/...
To be honest, I'm not sure if that a few people for a few months is even enough to avoid something like this from happening: this is crazy stress launch in gymnasiums with very high device variety. It probably demands a couple months just of serious dedicated QA and dry runs if you really wanted to avoid this outcome.
No. The median software developer makes ~$50/hr. Overhead is 2-3x (usually higher for a smaller company but we'll be generous). So company is charging $100/hr. At 60k that's a single developer working 15 weeks (10 if we're being realistic). But we know that it is at least two developers so that's 7 weeks (2 on our realistic scale).
> A kid can throw together an app in a few days,
Sure, but it doesn't matter if flappy bird crashes occasionally. Flappy bird doesn't need to communicate with other users. I know this app they built wasn't that complicated, but the kids that are pumping out apps in a few days are above average talent and probably working more than 8hrs in a day. You're comparing above average to what is already suspected to be below average (considering they are recent bootcamp graduates and have little experience).
https://www.latimes.com/business/technology/story/2020-02-04...
Edit: Nevada has since said they will not use the app and are looking at backup options
A venture with a long development tail after release is absurdly different from this. They had to get it right the first (and only) time, and no one is anticipating even a one in a million chance of getting rich off it.
Working in the depths of google and running a company are two very different things.
My personal experience with google engineers is 20% code and 80% complaining that our issue tracking system isn't as perfect as googles was.
I lean towards agreeing with you that a simple website plus cheap laptop would the real world best case, but I can understand why you would want a native app.
I agree a web app isn't great for low friction data entry, but they only enter the totals for each candidate x each round, so that'd be a dozen or two dozen short numbers for the whole night per user.
I found this which has some specifics: https://www.cnn.com/2020/02/04/politics/iowa-caucus-voting-a...
The only extra thing I see is a photo upload. Which is not rocket science from a web app either. At least on iOS you can take people right to the photo gallery or the camera.
I would really like to believe this isn't a flask app I couldn't whip up in an afternoon and polish over a week in my free time... but I'm having a hard time.
You only had 1000 precincts, all you had to deal with is 1000 calls - and if one of them went wrong, you'll let that person know through some kind of reviewing mechanism. I understand that $60K is nothing but this kind of failure is absurd.
Looks like someone put wrong test headers in there, for what it does, the App is 26MB.
But regardless of the architectural nature of their (Shadow Inc’s) product it’s worth noting that high performing, mission critical software is challenging to do correctly. That is to say doing it with feasible, adequate solutions for security, availability, performance, testing, and usability from the end users and administrators perspectives is a whole world of responsibility for an engineering team, even if the objective seems trivial in casual conversation.
People like to characterize software projects as “easy” and almost never know the scope of what they’re talking about. Idk anything about Shadow. But judging by their lack of effective consulting on this project, and consequently allowing such a terrible product to be released, they are bad at what they do. This whole incident should go down as a lesson to folks to not screw around with silly budgets, inexperienced teams, and unrealistic timelines when looking for quality software. That the DNC or whoever doesn’t know any better is a sign of the complete cultural ignorance that otherwise functional adults in this world have about the seriousness of software as any solution to any real-world problem. Everyone is a failure in this case: Shadow, the DNC, even the commenters still misunderstanding things on this thread.
Most of the time here (and hence money) goes in gathering requirements and stakeholder engagement.
I'd imagine there is probably at least 200 hours of meetings, travel and phone calls just getting to the point of working out what they need. And yes, a lot of that is going to be physically sitting down with volunteers in some town and seeing how things are done.
https://www.nytimes.com/2020/02/04/us/politics/iowa-caucus-s...
The story when it fully comes out will be quite interesting. The assumptions that we all are making are almost certainly to turn out wrong in some way - I'm looking forward to trying to compare the ways in which I was wrong on my thinking today given the relatively sparse reporting thus far.
One screenshot showed an SQL error.
I think if you're doing an one man job, under pressure, making an app that has limited scalability, the last thing you should be doing is not using some kind of ORM or something similar (especially at a login page)
So it looks like they made it unnecessarily hard on themselves.
For software, dunno
Anecdotally, nations doing it that way seem to report faster than when software and electronics become involved.
First of all (and it seems I've been using this word a lot lately) this turns software development into a religion rather than a business with solid business strategy. The equivalent in other businesses, like the entertainment industry, are people who do work for stars for nothing or nearly nothing because they think it will lead somewhere. All they get is a weird and abusive form of servitude (I've seen this first hand) and no future.
Why would someone not write software for politics instead and make part of their value proposition that they are not biased (and, of course, take the steps necessary to actually deliver that).
We have a close relative in South America who owns a company that makes and markets voting hardware and software. They are not aligned with any specific political party, movement or cult. They sell to everyone, in and outside South America and have a very nice business.
As far as why people don't sell to everyone, the above explains a lot. Another explanation is that political sales (which is different from true election tech, which is purchased by governments) are driven by trust and personal relationships. Once you work for one party in the US, the other will not trust you.
I am not trying to diminish anyone at all, but I would think these kinds of party line attachments mean that not everyone ends-up with the most capable developers, particularly so if what drives this is passion rather than a skilled software shop that with proven capabilities and experience across a range of relevant domains with an equally proven track record of delivering quality, well tested bug-free product.
Of course, the parties would have to we willing to pay more than hobby money for what they need.
This is a real phenomenon, but not in the way you think. Democrats have historically had a tech edge, because the real talent leans left politically. Obama kind of killed that, but for a good cause - he diverted a lot of those politically-driven types into working directly for the federal government under the umbrella of USDS and 18F.
It’s quite the leap you make from there to religion.
Perhaps an example might serve as a better explanation of what I am thinking:
Trello is a real company with a product that is agnostic. Politicians of any school of though can use their products to great effect. They benefit from everyone using this product, including, perhaps, their competitors and opposition.
If, on the other hand, each political party hires a passionate coder to create their own kanban board software they are very likely to end-up with a bunch of suboptimal solutions.
In other words, if they can, if the nature of the software allows, it would benefit them to look for universal providers rather than party-aligned providers. If you can trust Trello with your political party data you can then trust a similar company with other data you might produce with their software.
I guess it comes down to the software. What kinds of things are they doing that requires custom tools that have to be written specifically for one party?
* There are types of software that are politics-specific, and therefore do not exist until a party pays for it to be made.
* Politics is a zero-sum game: if you're funding software (either institutionally, or by being a tech worker who works below market wage For The Cause), you don't want it to be used by your political opponents.
So commodity stuff that already exists is bipartisan. Any application that is specific to political campaigning is highly partisan.
[1] There's also a secondary issue of data security for cloud products; parties maintain extensive voter files with things like past voting history, up-to-date contact information, and past contacts [1a]. They do not want this information falling into the hands of the opposing party, and so don't want their data touching any organization that might potentially have cross-partisan ties, or godforbid might have cotenancy for D and R data.
[1a] This is basically the king of all CRMs, and these days they might be able to reimplement it as a bunch of plugins on top of a commercial provider like Salesforce, but those only became available relatively recently, data migration would be nontrivial, and there's a substantial amount of politics-specific functionality that would need to be rewritten as plugins.
As an aside - these instances of underfunded and underdeveloped services, specifically in the political sphere, are really painful reminders of the flip side to the idealistic technocratic future that I think a lot of folks around here assume to be an inevitability.
You may find this discussion enlightening. In short, trust is at a premium and there are few white-label providers.
Um, isn't this app allowing people to enter data, and sending that to an API? How could that be out of anyone's core skillset? Almost every single app must do that.
In other words it's exactly the kind of business that attracts activist types who have non-monetary motivations and it could therefore present an opportunity for tampering, particularly since any discrepancies can be dismissed as bugs from underfunding and rushed development.
Whatever is actually going on, the system is clearly insanely broken.
Also Shadow was openly hostile to the Sanders campaign in numerous encounters.
It’s not as bad as the CEO of a major voting software company saying he’ll “bring home the votes for Republicans” (Diebold, 2004) a but this is a clear case of conflict of interest.
Since that time he's probably had his own epiphanies from working in industry but one of mine was that we did shockingly little QA and everything was built as fast as humanly possible. Everything was building the airplane in the air and it was largely due to constraints of the political or legal kind. Timeline management and allocating resources to testing was always an afterthought.
Testing usually comes from a feedback loop of launching broken features, incident response, manager gets in hot water, engineer gets in hot water, engineer proposes testing plan, manager uses incident as primary source reference to secure additional budget, testing implemented.
In politics no service, company, initiative, or team lasts long enough to complete the cycle.
This is systemic corruption. A billionaire can dump as much money into a for-profit company to build a platform for a specific campaign and then when the campaign is over then can claim that there isn't enough business to keep the platform solvent for the next 2 to 4 years and do it all again with the benefit of the previous code base. It's better than a Super PAC because the for-profit company can work directly with the campaign and no one will bat an eye. The company can even have foreign financing. All the while, their losses amount to a huge tax write-off and the public pays the bill.
In the case of The Groundwork, it likely cost upwards of $10 million for the people, building lease, AWS costs, etc. But where did all of the money come from? Certainly Hillary didn't pay for this. I'd love to see Eric Schmidt's 2016 Tax Returns.
...Yikes.
Lmao.
CEO, COO, CTO...
Here's the thing I don't understand about them, I thought startups were averse to this? Am I wrong about startups or am I wrong about them being a startup?
If this incident hadn't happened, these folks could have proudly claimed that they were the CTO/COO/CPO etc of Shadow Inc on their LinkedIns and whatever article BuzzFeed/Forbes wrote about them.
If I'm on the hiring end, this is not something that will help. Instead, if someone's going to sell me the importance of their title at their 6 people startup, it's going to negatively affect their chances.
Basically schools give you assignments or projects that are well defined and have clearly defined goals/results and cover solved problems. While that's true in the generic sense of business problems it's not how we work, we have to be able to take initiative to define and adjust those goals as reality shifts and goals are more loosely defined. You really have to put it the work to get new folks up to speed with defining their tasks to reach the goals.
Not for nothing but it's tough when you don't have a senior or two on hand to bring people up to speed and mentor them for a month or two before you can really get a new grad productive. And it's difficult with time constraints to do that well, I have to define my own tasks and can't spend as much time defining things for a junior as well as I'd like or train them as much as I should.
College grads may be more blank a slate but I haven't noticed a big difference in terms of actual onboarding. For what it's worth I think boot camps are the next trade schools and we're going to see a division in labor between software techs and engineers just like we have for electrical engineers and electricians or MechEs and mechanics. Tale as old as time, all the work is valuable, just more nuanced.
It's a reporting tool. How the heck did you get to this position?
Given the numerous safety standards and qualifications we require of engineers in the most mundane jobs I'm not sure why interns get to write software that underpins the electoral process.
State party officials who made the plans to use the app without clearly any vetting, the management of Shadow who must have been making insane promises on a shoestring budget, the DNC who allowed the app anywhere near an election, and even pushed the state party officials in the direction of using it.
In big tech areas, there's usually a firm assumption of liberal-leaning employees and technologists. Maybe all the democrats who can code are too busy in the advertising industry.
At least now I don't feel bad about when I test in production. (just kidding, I didn't feel bad before).
https://www.theverge.com/2020/2/4/21122737/iowa-democractic-...
But looking at the TestFairy pricing page, the free version is actually limited to 200 sessions per month. I would read this as every time a user logs on - definitely not going to work for 400 users logging on multiple times in one night.
The company who produced this app didn't just fail on development but on any procurement/contractual sense. How could they have imagined that relying on a free tier of a beta testing platform could be acceptable in a real life production environment?
Yikes?
Hasn't logging in been one of the things we perfected over 30 years?
For app development, even if for two platforms (Android and iOS) it is a lot simpler these days, than desktop.
Even if it is, say, 500TB of data, in 300 different formats usually those formats aren't drastically "different." Maybe I'm not understanding what the application was supposed to do to not understand why this wasn't solved every quickly. Or maybe given the timeline it was solved quickly once the right people got involved and figured out what needed to be done.
I would have written an app that doesn't work for only $25000
At $50k they certainly weren't going to get what they wanted delivered, and they have no-one to blame but themselves.
Why Shadow?
When a light is shining, Shadows are a constant companion. We see ourselves as building a long-term, side-by-side “Shadow” of tech infrastructure to the Democratic Party and the progressive community at large.
In retrospect, it sounds bad but really, it's just a nothingburger.
Remember "deplorables"?
Many people at this company were directly involved with the HRC '16 campaign.
Everything about this reeks of a publicity stunt to "ruin" electronic voting in the public eye.
Going back to voting exclusively by ink on paper and hand counting is imperative.
[1]: https://www.theatlantic.com/politics/archive/2017/05/north-c... [2]: https://www.theguardian.com/us-news/2016/jul/20/texas-voter-...
Watch Broward County FL this November. Whole boxes of filled out ballots will be "found" mysteriously. Just like every election.
All four voted for Trump, but that's not super relevant. What's relevant is that it was four out of 127,000,000 votes. Even if the problem were 1,000 times worse then we know it still would have had no effect on the election, even if they were all in Wisconsin.
yes, the general public now understands how bobo this operation is!
I'm pretty comfortable with that.
The challenge is: is the person submitting the results who they say they are, and are they authorized to submit results for said precinct, did their data submission transform on transit. The misconception seems to be that any of the data is private.
Issuing 1700 instagram accounts and having each precinct post and tag a photograph would have been both more secure, and instantly verifiable. Anybody could go back later and go "yep thats the photo I posted." "Yep that is the account that we expected the photo to come from." "Yep, the account wasn't improperly accessed."
Everyone in the public can see the live results real time, and hand verify the summarization. A worst case scenario would be someone doctoring one of the Math Sheets, printing out a near perfect duplicate, and creating confusion over which document is the "real" one. Breaking into a physically secure area with a fake piece of paper may cause a situation with an unverifiable truth. Having a public, timestamped photograph minimizes counterfeiting and forgery.
Providing a csv/xlsx file, and having a whitelisted list of senders would have worked, along with a bot ingesting the files as they came. A onedrive with 1700 shared spreadsheets would have worked, each precinct manager would have just needed a copy of mobile excel. The technology that generates my fantasy football cheat sheets seems more advanced.
The goal here should be to get a definitive answer posted into the public record, timestamped and proven to be the appropriate sender.
The "pin" nonsense is the scary part of all this. Did anybody who shouldnt have download the app, and submit the wrong results?
>he had to sign in with an email and password, provide a two-factor authentication, and enter a one-time password generated by the Google Authenticator app.
What is the point of 2fa/authenticator, if the person is registering for the first time the night of the primary? Wouldnt 2fa be for preventing access to the account youve already made, not to prevent someone from creating your account? Unless somehow the phone numbers were already on file, and a sanity check took place before the 2fa step, which I doubt. Is 2fa anything more than security theater in this application?
This is how it's supposed to work: https://youtu.be/ViK9pe9Pv_s?t=27
[0] https://www.cbsnews.com/news/mayor-pete-buttigieg-gay-iowa-c...
Good old progressive enhancement. Not sure why people want to rip out the old functionality (paper) until the new functionality (e-voting) has a solid track record on it's own.
The Republicans do ballots - on paper, but the democrats don't have a ballot at all.
Folks can take home the stub to ensure non-repudiation etc..
They can look up the serial number on the stub they take home to see for sure it was recorded.
FYI we should also have basic ID requirements for voting. The arguments against it are kind of ridiculous, and federal or state ID should be free.
There should be no 'voter registration' required either: the state knows your address and that's that.
For lighter elections like city alderman, you can use your ID + passcode to SMS or email your vote.
Maybe make a law requiring that any tech used for legal voting has to get the thumbs up from the NSA or some super amazing agency that is also secular in nature. None of this 'quickie app for voting' rubbish.
At this point, I hate politics so much, I think every nation needs a 'caretaker' Pres/PM from the bureaucracy - someone who can clean out the cobwebs, get rid of a bunch of old laws, clean up the tax code, take on the public unions (I mean reform not destroy, I mean teachers probably need higher salaries) and basically do a big, operational house cleaning that has nothing to do with politics.
Theoretically a group can win delegates in a caucus with just an issue not a candidate - for example you can all agree that the most important issue is Oranges (a fruit that doesn't grow in Iowa...) and get a delegate without committing to a president. Of course in practice most people show up only for the president poll and leave as soon as that is over.
You realize that every single thing you said there is political, right?
> "clean out the cobwebs"
This isn't specific enough to respond to.
> "get rid of a bunch of old laws"
Which ones, and why? Sure there are probably some like where people can tie up horses or something that aren't especially relevant, but having them there causes no harm because they aren't used.
> "clean up the tax code"
There is no way to do this without creating winners and losers. Choosing who wins and who loses is a political act.
> "take on the public unions"
Taking on unions is one of the most political acts you can do. Whether or not it is "right", unions are a powerbase of some political groups.
> "operational house cleaning that has nothing to do with politics"
What - specifically - do you mean? The two examples you give here are highly political.
'Simplifying' the tax code could be ideological, but it's only overtly political given the interest groups of those who profit from complicated taxes.
But by that virtue - every little policy of every government agency implies 'politics' and so we'll get nowhere.
There's quite a lot of things that Americans widely support, even on some of the heavy 'litmus' issues, it's surprising how much Americans have in common.
So yes, you're right, everything would be pulled apart by Fox and MSNBC, but there's a lot that would be less divisive.
Either way in a caucus you are NOT electing a president, you are just choosing a delegate who goes on to the national convention where the president is chosen. There are rules so that the caucus results feed who is supported at the national convention, but there are ways around those rules if you want to go to the national convention and then support someone who wasn't even running in the first place - though you pretty well better be assured of a win before you think about trying those tricks as the media will go nuts. (the same rules exist in primary states, technically there is a caucus somewhere that chooses the delegate to the national convention who can then vote for someone else)
My experience is that customers hate to pay for testing once they see the product running. They assume that it's done. I had a customer tell me that if I did the programming right it should always work and testing should be minimal. I had to explain to him that that's not the case with software and testing is one of the most important parts of the software development cycle. He felt it was a waste and that I was looking to add extra costs for no reason.
I bet there was not enough money. People seem to feel that a few grand will cover the costs. They figure 10k is an outrageous amount. What they don't seem to understand is that it will barely cover the costs of planning the app.
Both the developers and the people that approved the app for use need to take responsibility. Too bad since it could have saved a lot of money and time in the long run.
BTW, this could have been a Google Form with a spreadsheet as a back end. But user training would have still been an issue. You can't get around that.
The reason this happens is because hundreds of millions of dollars are lit on fire during election season and all the sharks, including former Google employees, come out to swim. Even well intentioned projects get slammed by the crunch of the election season (seriously try shipping a well scaled app in < 2 months with terrible product direction) and ultimately fail - failing the needs of the entire citizenship of the country.
After the success of Obama's 2008 and 2012 campaigns even more money was funneled into IT as a sort of perceived silver bullet. But in 2016 it wasn't, and yet no analysis was done to correct the problems for the 2020 cycle - because the decision makers (all these "CTOs") are clueless fucks who are just there for the money and could care less about the integrity of our democratic system.
- in 2016 I worked for one of the companies in this niche and saw the bidding/sales/engineering processes first hand. FWIW I am a life long democratic voter and this makes me sick to my stomach.
The DNC and the state parties are often at odds with each other. There's a good series of "Reply All" episodes about this, focusing on the Alabama Democratic Party.
https://mobile.twitter.com/taraemcg/status/12239909781277245...
https://theconservativetreehouse.com/2020/02/05/more-manipul...
"Shadow Inc.’s website has scant information about the firm and its employees. But a review of social media accounts shows that three of the Washington, D.C.-based firm’s top executives worked on the 2016 Hillary Clinton campaign.
The CEO of Shadow Inc., Gerard Niemira, was the director of product on Clinton’s campaign, his LinkedIn profile says."
https://nationalinterest.org/blog/buzz/shadow-inc-mysterious...
https://mobile.twitter.com/kendallkarson/status/122481061735...
Edit: Seeing downvotes on this, but it's in support of tptacek's comment. The DNC and State democratic parties are separate and the latter run the elections. You might believe that the DNC is fixing the process, but the reality is that they have little influence in how the States choose to run their election/nomination processes.
In all honesty, the Iowa caucases are likely a huge pain in the ass for them and this failure is probably driving a lot of DNC folks crazy right now.
Well, you aren't technically wrong here, but I disagree with the sentiment. The RNC leveraged a the same kind of data used by the Obama team in '12 to squeak out an Electoral College win in `16.
I think it's easy to get overly ambitious with civic tech and deliver expensive garbage. However, when people stay focused and deliver then I think the results speak for themselves. While the '12 and '16 elections didn't turn on candidates tech strategies alone, it's clear that they contributed.
304 to 227 wasn’t a “squeak”
There is some analysis that boils down to the below:
“He concluded, with help from The Cook Political Report, that the election hinged not on Clinton's large 2.8 million overall vote margin over Trump, but rather on about 78,000 votes from only three counties in Wisconsin, Pennsylvania, and Michigan (by the same logic, Obama won in 2012 due to three counties in Florida, Ohio, and Pennsylvania)
That’s definitely a squeak.
I personally worked on digital marketing campaigns for years as a data scientist in a previous job, and have become entirely disillusioned with the entire notion (and I am not alone[0]). Its largely snake oil, and its effectiveness is incredibly overblown. But literally billions of dollars, large marketing teams embedded in every company, and some of the largest tech giants on the planet all have an incentive to drink the Kool-Aid.
[0] https://thecorrespondent.com/100/the-new-dot-com-bubble-is-h...
https://thelastpsychiatrist.com/2010/04/im_not_the_one_you_s...
(funny substitution tho)
The only thing that could conceivably provide a systematic opening to fix things (ie, not just fixing specific things, after they've become apparent, because they've already catastrophically failed), in the realm of badly-written science fiction and fantasy, would be:
Bernie Sanders (or another equally popular candidate, in the near-future) is ratfucked so badly and obviously (which this could be the start of the perception of, even if it is just sufficiently advanced incompetence to be indistiguishable from malice), then the nominee loses to Trump so convincingly, that it completely shatters the party -- like it is no longer able to function at a basic level, and will clearly, even to the most delusional, never be able to win another election again -- and then... something, I don't even know how this would work. I am not creative enough to write this laughable science fiction any further.
There might also be a route involving Bernie Sanders winning both the primary and the general by a landslide, but that comes with its own and different set of systematic problems that I don't think would make the party's competence at dealing with technical problems, at least, any better.
But in either case, the deepest underlying rot is still the toxic interests of its financial backers, which are reproduced in the very bones of the party, and occasionally pop up in especially ridiculous and very indirect ways like this (and many more that don't get frontpaged as a problem on HN, or the New York Times for that matter). There's no fixing anything without somehow getting rid of them. And there is no realistically traversable path to that.
If they ratfuck him this time anywhere near as bad as they did last time, he'll run independent. In that case, Trump will win even more easily than I've been expecting since February 2017. Of course, Trump has a sort of death wish, so he could invade Canada or something and then Bernie would win without owing anything to the other face of the status quo party.
The most recent results (at 62% reporting, somehow not including urban areas, for many hours now) show Bernie winning the popular vote but trailing in convention votes. I hope I'll hear Democrats complaining about the Electoral College again soon...
Besides, Bloomberg is more of a Republican than Trump is. Why would Bernie care to see him elected?
This is not even close to paranoid. Conflicts of interest are an important issue for nonprofits because it can cause them to lose tax exempt status, and the IRS makes a whole todo about it, and therefore so do competent boards. Trying to brush this under the rug because of politics is disgusting and would not be tolerated at ethical and competent organizations.
Sounds like they know exactly what they are doing.
Why ship a competently built project when you can ship a broken POS and scoop up even more money?
However, if you are only a campaign manager for six months or whatever and have to prove your worth or else move back in with your parents and spend the rest of your working life fetching coffee and slowly paying for your $200k georgetown political science degree, then yeah that app better be out in 2 damn months.
Rat races don't lead to quality, they lead to desperate people taking shortcuts and making mistakes, and fodder for managers to point to on a slide when they have to appear in front of superiors. Things that woudn't otherwise happen if pressure was lowered and we thought about things like burnout and well being rather than whatever meaningless work-related metric is hot right now. This sort of failure is inevitable in fields operating like this.
It seems pretty hard to screw that up so badly but clearly it’s quite easy to make a complete dumpster fire from those requirements.
https://www.theverge.com/2020/2/4/21122737/iowa-democractic-...
they used the a free tier of TestFairy that limited the number of test users to 200 and on iOS used TestFlight and did not use either the app store for the final version or make an enterprise version, and possibly could not distribute the app to the approximate 1700 users. They still would have needed to get the app through the app store approval process for general release to fix the app at least on the iOS side as far as I understand it. To fix it quickly would have just required them to go back to calling in results by voice as in past years. Don't know why they did not just do that.
They did that - but apparently the phone number for calling in the results was the same as for "help my app doesn't work", so the phone line was flooded. Unfortunately I don't have the source link to hand, so feel free to treat that as hearsay.
However, CNN had a live cross to someone trying to call in their results, and after an hour on hold, when they finally got through, the other side hung up on them while they were broadcasting live on air:
https://www.washingtonpost.com/nation/2020/02/04/iowa-caucus...
"It took an hour and a half for Shawn Sebastian to get hung up on in 10 seconds. And it unfolded on national television Monday, making the moment, in the words of some, “one of the most ridiculous things I’ve seen on cable TV.”"
Ok, did anyone actually design the app?
Maybe it's too hard to move to electronic voting nationwide. But every organization has governance and could use an electronic voting system based on BFT consensus of mutually distrusting parties. Vote using an app, it gets stored "on-chain", then you can check it on another app.
My favorite screenshot is the last one, it looks like a generic mobile Firefox error for a misspecified URL.
I don't want to speculate too much on that screenshot but it looks like they have an error in their Auth0 integration. When you use a service like Auth0 you give it a redirect url to send authenticated users to. It looks like they passed in an invalid url for that redirect. Maybe they only tested this 2fa step locally and something was different when they deployed to users' phones, I can't say for sure, but it doesn't look good.
One possible issue is that Fennec/Fenix doesn't open http URLs in external apps by default (unsure about custom protocols), whereas Chrome does.
The jest of it is one of the most important institutions in the United States (the Democratic National Committee) uses a highly nepotistic and incompetent system for managing IT which leads to colossal failures in marketing, canvassing, and security. Not to mention massive PII violates as millions of emails, phone numbers and SSNs, are passed around in plain-text via CSV files.
The reason this happens is because hundreds of millions of dollars are lit on fire during election season and all the sharks, including former Google employees, come out to swim. Even well intentioned projects get slammed by the crunch of the election season (seriously trying shipping a well scaled app in < 2 months with terrible product direction) and ultimately fail - failing the needs of the entire citizenship of the country.
After the success of Obama's 2008 and 2012 campaigns even more money was funneled into IT as a sort of perceived silver bullet. But in 2016 it wasn't, and yet no analysis was done to correct the problems for the 2020 cycle - because the decision makers (all these "CTOs") are clueless fucks who are just there for the money and could care less about the integrity of our democratic system.
- in 2016 I worked for one of the companies in this niche and saw the bidding/sales/engineering processes first hand. FWIW I am a life long democratic voter and this makes me sick to my stomach.
If you want to refer to something you said elsewhere, please use a link.
Other suspicious dealings are a premature victory announcement by Buttigeig's campaign and a leak of a picture of paper tallies which included a PIN allegedly used to login to the tally app. Looking for the tweet now...
>https://townhall.com/tipsheet/leahbarkoukis/2020/02/04/shado...
Edit: I'm not necessarily trying to suggest anything, but I'd like to point out that this post went from +10 to +2 in a matter of minutes.
There doesn’t seem to be any indication this was anything but a regular screwup. Throwing around suggestions of a vague conspiracy looks unwarranted.
You’re suggesting that to avoid suspicion companies providing services to political campaigns must be apolitical and to have never previously provided services to a candidate in the race or any previous opponent of a candidate in the race. How could that even be possible?
Is it really so difficult to keep your public persona neutral and vanilla if you're a damn CEO? Wasn't this standard practice until recently? What, because Trump vomits on Twitter 24/7 everyone else needs to as well?
I always try to give people the benefit of the doubt, so here’s your chance: any particular reason we shouldn’t dismiss you as not being serious?
No, I think it's questionable when a CEO publicly discloses a strong political bias, receives private funds from the personally favored candidate, has cordial photos with the candidate at social events, and then is hired by the DNC and charged with being unbiased in a secret, electronically controlled ballot.
There are tens of millions of dollars changing hands in the ramp up to the election. This is not a complex scheme involving millions of mouths to keep shut and olympic mental gymnastics to justify. A handful of politically leaning and/or paid off devs add a package to the code and you have whatever results you want.
Ask yourself, do you really think corruption only exists outside of the United States? The bottom line is there are many discrepancies which happen to align toward a possibly calculated malicious intent. This feels like a "conspiracy theory" because there are so many tenuously linked components and no hard proof - but that's how any real life unfolding event looks until people have poured over it for months post facto and sorted through everything.
In any case what I'm saying is that there's plenty of incompetence and bias here to at least be prudently suspicious.
The Iowa Caucuses are neither secret (like absurdly not) nor electronic?
The idea that the Democratic nominating process would be rigged is not ipso facto crazy given that it is publicly acknowledged to have actually happened last election cycle.
Maybe you can provide more details of your hypothesis, but what you have so far doesn’t make sense.
The count has been frozen at "62%" for six hours now. That just happens to be the first/only level at which "Mayor Pete" had a delegate lead over Bernie, even though Bernie still leads the popular vote. Several days from now, when the count ticks up to 100%, Bernie will lead both measures by large margins. Still, in the meantime, the corporate media can pretend like the Mayor is ahead. They know he's going to bomb in SC worse than Biden bombed in Iowa, but they're hoping that somehow they can get him to "Super Tuesday" when the other Mayor can take over for him due to his 9-figure ad spend.
Of course, the idea that the perfect candidate to beat a vain old NY rich guy is another vain old NY rich guy who also banned soft drinks the last time he held elected office is risible, but they don't care about that. The point is to ratfuck any candidate who might plausibly do fewer stupid wars.
Some people like to bore us with the "once is a coincidence" maxim. Some of them might like to pretend that this caucus is "once" and we'll have to wait a week or two to get upset about the ratfucking. Normal voters can remember four years ago, however, and this isn't even just the fourth time the national "Democratic" apparatus has ratfucked Bernie.
This doesn't necessarily mean there is a huge conspiracy, but the optics -and the reality- of the party machine functioning properly is terrible, and the choice is between incompetence or malfeasance. Either way, they shot themselves in the foot, and it's not a good look.
Aren't you though? I mean, your post is essentially "CLINTON" "ALLEGED" "SEVERE" "LEAK" . Lot's of words meant to imply some grand conspiracy. The reality is almost certainly that the developer (who has an unfortunate name) probably just didn't have enough time/budget to put out a decent product. Not everything is an actual conspiracy.
Calling it a "conspiracy" theory is just dismissive slander at this point. People have conspired successfully to achieve far greater and more malicious goals.
I think Georgia might be the most notorious for the behavior you describe.
I don't know if you're baiting me but personally I don't think corruption is limited to Democrats. Particularly if both sides start to believe the other is secretly cheating.
It feels like the American system works when a critical majority are playing with good faith - and that good faith is increasingly lacking.
Aren't you literally proposing that there is a conspiracy? And since there is no evidence then isn't that a theory?
What do you think it should be called? "conspiracy speculation"/"conspiracy hypothesis"/"possibility of a conspiracy"?
Intentionally releasing a broken app is an idiotic excuse for a conspiracy.
"The app was not deployed through traditional app stores or even sideloaded using an enterprise certificate. Instead, it was deployed through the TestFairy testing platform, which is similar to Apple’s TestFlight and used predominantly for Android and iOS apps that are not yet finalized."
https://www.theverge.com/2020/2/4/21122737/iowa-democractic-...
4 years ago I had the Republican app on my phone. I was the backup reporter just in case the main reporter had problems. Since the main reporter used the app correctly I didn't use it, but I had access to it.
Yeah, democracy is fucked.
0. https://theintercept.com/2020/02/04/iowa-caucus-app-shadow-a...
For the kind of malfeasance that concatenates an SQL query with user input and then shows the whole thing to the user in an error message, you don't need that.
If you want a tl;dr, though, you might consider that (a) there's a paper trail, (b) the app was always intended to be optional and some precincts weren't using it in the first place, and especially (c) caucuses are not secret votes. That last one is how different campaigns had their own estimates before the official counts started coming out in haphazard fashion. Whatever bad things one can say about the idea of caucusing in general -- and we're hearing an awful lot of bad things -- they're actually pretty damn difficult to rig.
Buttigieg overperformed polling because polling only captures people's first choices. A lot of voters whose first choice candidates didn't meet the 15% qualifying point in the first round switched allegiance to him in the second, and the net effect was that a lot of moderates ended up in his camp rather than Biden's. This end result is probably an "only in a caucus" thing, but it doesn't require nefarious intent.
Eventually you have to step back, take stock of your own experience with people, software, and their interactions, and make your own judgement. If you have no experiecne with software and people, then ask someone you trust who does. I don't see a whole lot of people who write software who are that surprised at what happened given the makeup of both the timing and the resources put into this app.
You can't really rig votes even with an app. Hanlon's razor probably applies here; never attribute to malice that which is adequately explained by stupidity
That being said, it's important to grasp what actually happened with the app rather than give in to misinformation and political apathy surrounding the news. Iowa voted by paper, so we are all just sitting tight while the hand count occurs as is traditionally done.
Do the paper vote - as it was always done. Let our technology analyze it afterwords. Why are we all trying to put our technology in front of this simple device. of paper.
You all know this rule = "make a single application do one thing well."
Again, this is ridiculous from a Credibility sense. Paper + networks, Paper + mobile, forget the paper, Mobile only, Mobile + scale. Where are we at? No where and much less. Just take a paper vote and tally it. Is that so hard?
My son was there working as volunteer, and so was his ex-girlfriend, who's still a close family friend. I was chatting with them both throughout the evening and into the wee hours this morning.
They worked different locations in the Des Moines metro area and the results they reported to me were pretty close to the same in both.
I'll say this about it, it's hard to imagine that less than 4500 calls to a server over the course of an hour or two would "crash" it, or even ten times that number. And that I think an SQL db is a poor choice for an app like that.
CouchDB, or most any open source "nosql" db, would've been a better choice and, really, so would a dead simple flat file db to store the data for each precinct in and those are both very easy to build for something as simple as this.
And they didn't even need an "app" per se. All they needed was a password protected web page with a dead simple form.
What happened makes it a bit difficult to avoid pondering if the delays weren't deliberate. Whatever the case may be, the DNC is who ended up taking it on the chin.
The data being submitted is so simple that using an SQL db adds a layer of unnecessary complexity.
So did their decisions for authentication, which appears to be where things got stuck.
Or 1678 password protected databases with one or more human readable files in them and let CouchDB handle user auth, which CouchDB makes really easy.
You could do the same thing with a simple flat file database using directories and files and system level user access, and it'd be very fast too.
This is pretty much the kind of thing CouchDB is designed for though and has tools built-in to make it easy.
This is exactly the kind of thing that SQL database is shine at and noSQL databases do poorly.
No, I've not. I just never liked the way it worked so I have to honest and open about my prejudice.
> This is exactly the kind of thing that [...] noSQL databases do poorly.
That's really not true. I've built web apps that do pretty much exactly what that voting apps does and I use CouchDB to do it because it was built specifically for this kind of thing, and it makes it very easy.
That's not the same as saying SQL won't work. Or even that it's a bad choice by design if it's what you're familiar with and good at using it.
How much time have you spent with CouchDB?
I do know that those coming from a long history of using SQL dbs often have a hard time using CouchDB. I had a hard time with SQL, so maybe it's a right brain/left brain kind of thing. Or maybe it's just not really wanting to learn a different way to do what you already know how to do with a different tool.
I chose to use a hand rolled flat file database early on (around 2000) as a backend for my web apps. When CouchDB hit v1.4 I switched to it because it worked very much the same as my db and offered a lot of advantages. It's gotten a lot better over the years since too.
So yeah, my view comes from a different perspective.
But in a voting application structuring the data is actually very very helpful, it keeps bad data out, and helps with auditing.
Validating data is something you can and should do before saving it in the DB and with CouchDB that would be done with a "design document" using javascript.
In this case validating that user input is a positive number or a zero should be done on the client side before even submitting the data as well.
I've used CouchDB on and off for some years[1], although I've used SQL databases for longer. I've also used NoSQL databases ranging from MongoDB to Cassandra, Hadoop, BDB (and variants), DynomoDB, etc.
CouchDB (or any other NoSQL database) just isn't the best choice here. The application is literally tables of relational data with multiple simultaneous users which is pretty much the perfect use case for a SQL Database Server.
[1] Since 2010! Wow that's a lot longer ago than it seems. https://mail-archives.apache.org/mod_mbox/couchdb-user/20101...
How is this data "relational"?
Each precinct has 3 numbers to submit for each candidate, that's not a big task for one person, and none of those precinct's candidates numbers are related to anything else in the db except the grand total of all precincts numbers and that's a one way relationship. The grand totals never change any of the data they're derived from.
If you want to give each candidate's precinct representative access to the precinct's db in CouchDB than you just limit their access to one file for their specific candidate in a db for their specific precinct, so by design it's purposely not related to any other candidate or precinct.
That's why CouchDB is a good choice for this kind of app.
The numbers relate to each candidate. It's that simple!
What happens when two people share authentication (which will happen!) and access the app at the same time. Maybe you are going to try and implement file-locking, yourself, badly?
SQL has literally been used for this kind of app since the 1980s. It's simple to use, and easy to find developers who use it.
If they both come from a different district and log into CouchDB they'll either only write to their file (one db, multi-user) or write to a file (or files) in their db (db per user).
If two users write to same file in same the db at the same time the DB will note the conflict and decide which data gets stored in the file.
https://docs.couchdb.org/en/stable/replication/conflicts.htm...
Yes, this is how I remember CouchDB working. It leaves the hard parts of conflic resolution up to the client app:
Once you have retrieved all the conflicting revisions, your application can then choose to display them all to the user. Or it could attempt to merge them, write back the merged version, and delete the conflicting versions - that is, to resolve the conflict permanently.
You do realize that SQL databases handle this automatically via locking schemes and transaction, right?
In this particular use case there really should only be one authorized user who can edit the data file for each precinct so race conditions like you describe just don't exist.
You can also use CouchDB's live sync feature to push the latest revision to everyone viewing the file in close to real time.
So conflicts are really a non-issue here. By design they just don't happen.
This should fall on the IDP (Iowa Democratic Party) as they commissioned the app...
The Iowa caucus is not a paper ballot. People stand in groups in a large multipurpose room and raise their hands for who they vote for. Depending on the viability threshold, people move to new groups and are recounted.
The exact rules are complicated and the process is run by unpaid humans. Moreover, in 2020 everyone has smartphones to post embarrassing mistakes on social media. And this year is a very crowded primary for the democrats, raising the chance for error.
Under this theory, the app is a convenient scapegoat to hide the fact that the process is inaccurate and bad. For politicians this seems pretty convenient. Who would you rather blame, and decades old tradition or an app contractor?
As anybody can tell you, nothing sinks projects faster than changing requirements late in the development cycle.
The app also sucked for another reason. Lots of Iowa is rural and cell coverage can be spotty. The app was designed with the all too common assumption in this day and age that the Internet works everywhere.
The process has never worked. I attended a couple caucuses myself between becoming 17 [0] and leaving Iowa. I still hear about them from family. They have various flaws. The most severe one IMHO is that due to the time commitment, they only reflect the voice of die-hards and/or those with few responsibilities. That's true for choosing the candidates and even more so for choosing the platform.
However, I imagine you mean this specific failure mode (unable to produce an authoritative count within X hours for some smallish X) is new. AFAIK, that's correct.
I would love it if this failure inspires them to change to a standard primary process. Probably won't, though.
[0] iirc you can participate if you will be 18 by the general election.
Example of rule confusion: "No new voters are permitted to join the caucus after the first alignment. But in at least 70 precincts, more than 4 percent of the total, there are more tabulated total votes on final alignment than on first alignment."
Picture it, there's 100 people in a room who all have to move themselves around to their respective candidates and people count them and shout out the numbers.
All of the candidates are represented (they're reporting the numbers) and can hear the same numbers, all of the voters can hear them too.
It's messy, but it's probably one of the most difficult systems to "hack" in terms of real numbers since so many people know the results immediately.
Random example: if candidate is below the viability threshold, that group is supposed to disband and join another group. You could have a mistake where, on the 2nd round, two non-viable groups merge and form a viable group.
For example, maybe Biden is at 14% and Klobuchar is at 2%. Under rules, those groups are both non-viable. But you could easily see a mistake where the caucus operators allow the 2% Klobuchar group to "merge" with the Biden group.