The TSA Randomizer App Cost $336k
kev.inburke.com
kev.inburke.com
Since this deals with national security, could probably justify higher rate resources staffing (security clearance required?). Ballpark $250/hr @ 8 hr/day so $200k buys you 100 days of capacity.
Project breakdown:
* UI/UX Design: 10 days (lots of stakeholders+approvals needed, maybe field testing with TSA agents, accessibility audit)
* Development: 15 days (this is very conservative but might be some business rules that aren't being shared -- remote monitoring, analytics, auditing, static analysis, etc)
* Project Management: 25 days
* QA: 15 days (multiple devices, high perceived security exposure)
* Third-party security audit/pen testing: 5 days
* Technical writer for documentation and training materials: 5 days
Total: 75 days of capacity spent.
Math doesn't work out exactly and some of these numbers are pulled from thin air, but it seems like it's at least in the right ballpark.
Do you folks really know people that are doing government contract work (for the TSA no less) for less than $100k?
And the security, analytics, and monitoring are probably all smaller specifics days for individual experts.
Point is though, you're thinking the right way on it. Very reasonable.
Also I agree the overhead of current T&M contracts likely doesn't make contracts under 100K worth pursuing especially given the huge amount of competition for some of this work and (unless an incumbent) a pretty low win probability.
Fingers crossed the TechFAR and some of the micro auction steps being taken make this a more dynamic market.
EDIT2: Here's the spending on that Contract by task order (as up to date as USASpending can be anyway) https://www.usaspending.gov/Pages/AdvancedSearch.aspx?k=CIO5...
EDIT: If the re-compete was done under OASIS then maybe we can get a lot of information out of this line by line transaction from USASpending: https://www.usaspending.gov/Transparency/Pages/AwardSummary....
It's government, so you'll likely have a 508 compliance audit (accessibility) as part of the project as well. Figure a work-week of that.
Security clearances are almost a definite "yes". Extensive background checks at a minimum -- and we're talking three days of paperwork, fingerprinting, calls to your third grade teacher, etc.
Why would this require more than a few days? Simple UI, then just generate a secure random number and draw an arrow. Bam. Give me 200k.
I mean the largest expense would almost certainly be third-party pen testing and QA. And why would it take a technical writer 5 days to document this? It has 1 function.
Sometimes bureaucratic inefficiency just really gets under my skin.
Judging the final output without any consideration for the process is akin to asking why it took a week to fix that hairy bug. It was only one line of code that changed, how could it possible take you all week to fix it?
There are processes that have to be followed, no matter how ridiculous they are, because they exist.
The point about field testing is fair, but shouldn't account for a huge chunk of the price. The invisible-to-the-public features point is also fair, but there's no way to estimate that cost of those.
The rest of the points, though, while fairly reflecting the way enterprise works, just show how utterly dysfunctional the process can be, and imo bolster, rather than refute, the position in the original article.
"That is literally a hello world ipad app, how could it possible cost $300k?!?" -- I attempted to provide an answer that goes a bit beyond "shrug govt" or "shrug consultants".
With the hammer, government apologists claimed it was to "secretly move money for DoD"... which really means it was the DoD and military just giving themselves a shit ton of money.
The US is very corrupt. The DoD dodged a bullet when 9/11 happened because that was the day they were going to be put under investigation.
Sadly enough, nothing happened at all, and this type of political graft is very common among the military and their politicians.
A recent case for example is the head of University of California who does not even belong there and is completely ill qualified to lead a college. Her only real experience is as a _spy chief_ for the military and DHS.
This type of political graft resulting from military connections is going to be far more common and invade further into civilian infrastructures. It's not like the NSA's complete treasonous behavior in spying on the US citizens they're supposed to protect isn't bad enough... that isn't even the tip of the iceberg, I'd wager. Before that of course was the CIA funneling drugs into cities to fund their own pockets and programs. Or even the joke of the "war on drugs" which all it was meant to do was control the profit flow of illegal street drugs.
These styles of military programs, also known as soft power projection, directed at the US population and citizens is getting far more common and invasive. These days, it is almost considered the norm.
> With the hammer, government apologists claimed it was to "secretly move money for DoD"... which really means it was the DoD and military just giving themselves a shit ton of money.
No, it was a $15 dollar hammer with $420 of project R&D costs averaged out and added to it: http://www.govexec.com/federal-news/1998/12/the-myth-of-the-...
The $435 price was then inflated to $600 by the media and finally $10,000 by you.
We have done recently. (Not TSA - not even US govt...)
And, in spite of our "mid 5 figures" price for the iOS+Android apps for a fairly simple app - the whole program of work the government department included this in, and hence the media reporting of "what the app cost" came to 1.3million dollars.
Now I get people asking me "How on earth did it cost over a million dollars for that app? Surely that's practically fraud!"
If the only requirement is "an arrow that randomly points left or right when you tap the screen" - I suspect _anybody_ who reads HN could whip up a provably-correct solution in under 30 mins just using html and javascript. I _strongly_ suspect there's a +lot_ of other requirements and costs we don't understand here. (But cynically, a lot of those costs could well be high paid consultants writing Powerpoint decks justifying why the only requirement should be "an arrow that randomly points left or right when you tap the screen". That'll be $325k thanks - here's our recommended outsourcing partner who can deliver the app for $11k.)
To your other point, we have in the past seen the media reporting these types of things in incomplete ways for fear of irritating their sources and losing reporting access.
If its true. In contemporary journalism the truth doesn't matter. Things can have perfectly reasonable explanations and be working as intended, and the only thing required to turn them into scandal is to omit those explanations and add a clickbait headline instead.
There's plenty of examples for that in Europe in the way journalists paint perfectly reasonable EU directives as utter nonsense handed down by stupid bureaucrats.
I doubt any company in America can tell you exactly what an app cost them to build. They can tell you what the product cost to develop, but the app itself is kind of beside the point: you don't get an app without the up-front work of determining the requirements, so why bother to separate the costs? Some projects have complex requirements and simple implementations, others have simple requirements and very complex implementations. The project cost is what everyone is concerned with anyway; and while the project manager should have the line-item information for the project, it's often not rolled up higher than that because the reconciliation would be nearly impossible.
You're absolutely right. They didn't buy an app; they bought a solution (of which the app was a small part).
Because it's not just the iPads; it's setting up the management structure for the iPads. How to procure them, locking them down so agents can't install Candy Crush, modifying existing IT processes to be able to manage device breakage/reloads/provisioning, etc...
None of this stuff is all that difficult, but nothing in business is free. So while any moron could write the app itself in half an hour, designing an end-to-end system that serves the needs of the customer (can be operated by someone with a GED; doesn't break down often; has simple workarounds for confusing scenarios; etc.) can cost a lot more than that. The software is just a piece of it.
Or did they have a problem they wanted solved, and specing out a way to do it, was just one part of the job? Because often, the output of the contracts I work on, is the just the final step of testing out many possible solutions or version, and actually shipping the one the client finally agreed to.
If I had a friend who needed that functionality and had a decent reason but no budget, say for an art project or a non profit event, I'd certainly offer to build them an HTML/JS thing for nothing. I'd think twice about starting an XCode project, dealing with App Store submission requirements, needing to deal with people asking how they install it on their Android tablets, etc.
At least for me - sometimes the stupidly simple HTML version of a thing is a perfectly good solution. (For the TSA, there are without doubt more requirements than the stupidly oversimplified one proposed here...)
You're inventing requirements which, while no doubt genuinely part of the TSA's requirements, go way beyond my single simple requirement as discussed.
Your friend's art project is not what we're discussing.
"You're inventing requirements which, while no doubt genuinely part of the TSA's requirements, go way beyond my single simple requirement as discussed."
I am not; as you yourself admitted, these are likely part of the TSA requirements, which is what we're discussing.
'If the only requirement is "an arrow that randomly points left or right when you tap the screen" - I suspect _anybody_ who reads HN could whip up a provably-correct solution in under 30 mins just using html and javascript.'
In the context of the proposal to do this in HTML/Javascript, I've been explicitly discussing a obviously oversimplified single requirement - not the "likely" requirements for the TSA project about which we can only make assumptions, since we've not seen them.
Sorry if we've been talking at cross purposes here.
I think those who don't work at a tiny four-person startup but don't generally touch the money should spend a bit of time at this site: https://tobytripp.github.io/meeting-ticker/ But rather than using it as a meeting ticker, use it to show how quick a team of 5 or 10 reasonably-well-paid people burn through the money over the course of even a day or three. Business is generally a lot more expensive than I think people often realize. I've sat through "all hands" meetings that clock in distressingly close to $350K by that calculator's reckoning (though rather a lot of engineering is really only half-listening and still working...). If you think about the expense all the time you can end up paralyzed but if you never acknowledge it you can end up with very skewed priorities.
To make this line of thought appealing to those who are probably the ones reading this, when your manager asks you for that stupid-ass feature that's going to take two or three days to implement, you've actually got a really big stick if you work the math and give them the numeric figure on what that's actually going to cost. Managers themselves often don't really think about this properly either. You can help them. If you happen to do it, shall we say, a bit "selectively", well that's their own darned fault for not thinking that way themselves more often....
What I found bizarre was that each time I've had to explain why I needed the numbers, and people were surprised at receiving a report of what the staff budget went. You'd think that would be expected, not something unusual. But permanent staff is not treated as a cost during day to day work, even though their time is a limited resource.
It's quite fascinating to see how quickly priorities change when the cost breakdowns make the rounds and someone higher up starts asking critical questions about why we spent $50k of staff time on feature X that no customer had asked for.
A big problem I've always had is getting hardware to improve productivity. Most of the time the management chain can't grasp why it's worth a $3k machine and maintenance costs to save each dev an hour a week in development time.
I think this goes to just how often people I management like the title and the power but don't understand that they're supposed to be in charge of strategy. I don't think most of them would know strategy if it walked up and bit them.
For example, if I can spend a day tweaking my project's build scripts and CI workflows to get it from taking 30 minutes to build/test/deploy down to 30 seconds, I've dramatically altered my team's workflow.
Also, there are very few reasons to buy your own hardware anymore -- cloud services are better at almost everything unless it's a low-latency operation (or you're in Australia - the speed of light is still too slow, sorry mate). Obviously there are exceptions, but with the scale of GPGPU build-out all the big public clouds are deploying, the benefits of anything else are rapidly disappearing.
Most managers aren't swayed by the argument that 1 hour = ($annual_salary / 2000). They know that if you give people an extra 5 minutes a day, they'll probably just spend it drinking coffee or socializing. Those certainly aren't bad things for employees to do, but unless your team is already running well beyond capacity, giving them more time to do things doesn't really improve productivity. Improving process (by, say, removing a manual step through automation) really gives you more bang for your buck.
Cost. I've yet to see a cloud service that get anywhere near as cheap as owning your own hardware (fully loaded with financing costs, full staff costs etc.) for anything that runs more than ~8 hours a day.
Cloud services have plenty of uses, especially for brief capacity spikes. Need to run an experiment for a few days that requires 20 extra servers? Go nuts. But even with reserved instances, if most of your servers run 24/7, you're throwing money out the window.
(And you don't need to own hardware - you can go for rented dedicated managed servers and still get ~80% of the savings)
And keep in mind salaries are just a fraction of the costs of staff time. Payroll taxes plus benefits can add up to almost 20% of base salary and that's not counting occupancy and equipment costs.
Bottom line: people cost a lot. It's smart to think carefully through the ramifications of planning decisions.
It's pretty universally known that engineers are horrible at estimating their tasks but this particular app is so simple as to be impossible to overestimate. It literally would take less than a day for a (half)-competent engineer (and maybe 1 designer/artist) to do the work for this (I won't give you the task breakdown but I'll say the task list is short).
So the answer is probably more in line with the other associated costs as you mentioned. Another just as likely (and as simple) answer; they charged some know nothing bureaucrat a ton because that's how it works in DC.
Yeah right.. The recoloring of the arrow itself I'm sure took 10 days...
I'm working with a bank right now trying to just get them a UI for an authentication page. We've struggled 2 weeks trying to figure out what the hell they want for the FAQ page... Between the design-spec they provided and where we are now, I'm not sure why they have designers in the first place..
Our first product was according to their spec, which they scrapped instantly and stated they expected X,Y,Z instead, which weren't in the spec (Page should have an accordion, the accordion should have buttons, all buttons should have certain looks and sizes. The spec didn't even have borders, much less an accordion and no buttons in sight..) and when we produced whatever they stated they wanted they threw out half of it and flipped things around (now the buttons shouldn't be arrows, they should be + signs that become - signs when clicked on...)
I'd hate to think what the UI/UX process is trying to get through multiple layers of governmental bodies....
Or like making a mission patch that's an octopus with its arms around the earth, with the tagline "nothing is beyond our reach"?
It's all about audience, right? Palantir used to have t-shirts that said "save the shire" on the back -- dunno if they still make those.
These requirements, sometimes they're mandatory and are called set-asides, sometimes they're not mandatory and are called aspirational targets, or they might have any number of other names. They all serve a worthwhile purpose; they help combat years or decades of preferential treatment by procurement officers to pick people of "the correct ethnicity".
Like, by now it's been pretty well-established that people tend to hire people that look like them, i.e. one of the factors that contributes into the disproportionate representation of various groups in the tech industry we often talk about.
Same thing has been happening in purchasing for a long, long time, and what you end up with is a situation where minority-owned firms will have been largely excluded from government contracting. When this happens for very long, it becomes self-reinforcing and self-perpetuating; there's no institutional knowledge in ethnic community A that passes from one generation to the next about large-scale project bidding or management, and you end up with this sort of disproportionate representation and the ensuing economic effects.
The set-aside or aspirational target is at least a start at countering that. Like everything, it gets abused by the people with the power and the money (which is where you get the minority contractor who acts as a pass-through to give the real work to the majority-owned firm; that minority contractor isn't building any business of their own, certainly nothing they could pass onto their kids or whatever), but it's better than what we had before, which was a system that perpetuated an even more unfair system for a lot of people.
EDIT: #yolo
As you say, this doesn't even counter the problem. The "minority" is just skimming the "take" not building a business or anything else useful.
In same cases? Maybe. In many cases (including my own MBE/DBE-certified business) we are using those projects to build a business (2 bootstrapped partners to 8 employees and national recognition in 4 years) and to use some of that revenue & expertise to help folks that normally couldn't afford our services. And there are any number of businesses I can point to that are achieving similar success (of course there are also failures, but that's true of businesses in general).
With regard to Frondo's claim "where you get the minority contractor who acts as a pass-through to give the real work to the majority-owned firm" - at this point I'm pretty sure that's the exception rather than the rule. The various state MBE/DBE certifying agencies are extremely diligent at policing this behavior and will slap down firms found to engage in it. Primes and subs are audited, contacts are reviewed, MBE/DBE subs are interviewed (included onsite visits) etc. MBE/DBEs must submit annual sworn financial statements, along with examples of contracts, invoices and evidence that firm can conduct the work it's contacted for (resumes of principals, references, list of equipment, and so on).
Also, congratulations on using the MBE/DBE certification to build your business! That's excellent, and I'm quite happy for you.
Either nobody was willing to bid lower: the system works.
Or someone would be willing to bid lower but its a team filled with white guys: the system works. Maybe these other companies should think about their hiring practices considering that they exist in an area with a large minority that doesn't seem present in the team.
I don't know the details though, maybe there's some capital requirements or something that's pretty hard to match
Some people have commented in other threads that just subcontracting can get you into a lot of trouble if caught though.
The legit business people who aren't just patsies often get screwed by these arrangements. They end up holding a massive short term liability that the big contractor (IBM, Oracle, EDS, etc) wants paid NOW, but the receivable is mired in some crazy 120 day late pay hell with the government.
If we want to hire minorities to improve their lot, just hire them. Or require actual bids on contracts that a small business person can grok instead of blanket purchase arrangements that require Fortune 50 sized legal/contract teams to bid on.
Of course, if the point is to generate extra income for the spouses of corporate execs and campaign contributions, than sure, these programs are uber successful.
At the local level, at least in the Pacific Northwest, this is exactly what happens. Local agencies have targets of anywhere from 15-30%, for how much they'd like to spend directly on contract dollars awarded to minority-owned/woman-owned firms.
Most are hitting between 5-15% now, but I'd say probably 4 out of 5 purchasing managers wants to do better and actively encourages their staff to do outreach to the minority/woman-owned small business communities.
Buyers still have a challenge in finding qualified minority/woman-owned firms, but there are pipelines in place to help build those firms up. The set-aside/aspirational target system can support that, too, by favoring prime contractors that work with minority/woman-owned subcontractors. A lot of primes have mentorship programs that target minority/woman-owned companies to help them grow their businesses.
In the government purchasing world, this stuff is all pretty non-controversial because it generally achieves its stated goal. I was just trying to say that abuse happens here too, but abuse of any system happens wherever there is a system.
Of course, in the real world, what happens is that the company owner/CEO gives his wife or another female relative 51% of the company, and/or the title of CEO or managing director.
This dubious practice obviously doesn't scale up to large/publicly-held organizations, but it's widespread in the "small/disadvantaged business" community.
It turns out that it's awfully hard to do the right thing by doing two wrong things.
The purchasing managers I talk to all seem pretty keen on doing right by their aspirational goals, and the firms that try and cheat the system that way, well, it's known that they're trying to cheat the system.
Furthermore, all this stuff is public record--all the purchasing decisions, awarded contracts, etc., and I know that the local papers here in the Pacific Northwest periodically sift through that data looking for signs of abuse. They do report it when it comes up, too, but it simply isn't prevalent, and I'd be quite surprised to see any interpretation of the data that would let you characterize it as "widespread".
I would like to mention, though, that I've literally never seen a contract issued by a state agency or local unit of government to purchase electronic component testing services or equipment (either directly or a larger contract with a subcontracted piece for that).
That's not to say it never happens, just that most of the contracting dollars I see spent are on construction and personal services (counseling and social work, IT services, and the like).
I would suggest that your experience probably isn't reflective of the disadvantaged business community as a whole, and I suspect any empirical evidence we could unearth (i.e. the public purchasing records, and so on) would support that claim pretty unambiguously.
There are some controls in place. For example, my understanding is that in the federal government, contractors that are preferentially selected for various reasons must perform a majority (at least a technical majority, e.g. 51%) of the work in-house, although I don't know a lot about how that is measured or supervised so I'm not sure how effective it is. I have certainly seen cases that looked like abuse.
These days it wouldn't be a stretch to add "better for the environment" along the lines of "what is the best deal," since that is also a decision that affects everyone in the community, whether the company you hire pollutes more or less. Favoring contractors that pollute less seems uncontroversial.
Adding "addresses systemic racism that we have in the past committed and that has hurt a part of our community (who are also taxpayers, just like everyone else)" also seems pretty uncontroversial--unless you don't think racism exists or should be addressed.
When do we start addressing the systemic racism which systematically discriminates against business owned by people of the wrong colour — when that colour is white? I.e., addressing systemic racism by applying systemic racism seems pretty foolish.
In a perfect world, minority-owned and woman-owned businesses would never have faced any discrimination at all. In a perfect world, everyone would have competed on an even playing field since the beginning of government buying. That is not this world.
And the real-world consequence of those decades of discrimination is that minority communities had another barrier placed in front of their ability to build wealth in their communities--not the only one, of course, but still a real one and one that's definitely done lasting damage to those minority communities.
After all this time, and all that damage done, saying "well, we won't discriminate now but you still need to somehow catch up from decades of discrimination, and we won't do anything to undo the damage we did," if you think that's appropriate, then that's fine, you can think that. But it's only foolish if you don't really think at all about the history of the situation and the accumulated effects of that long history of discrimination.
You'd even be in good company if you wanted to take that shallow view--a lot of people I talk to don't seem to want to acknowledge that racism ever existed, or if it did it isn't a problem now, or if it is a problem then someone else should do something about it, and on and on and on.
(And, you know, all these aspirational targets, like spending 20% of our dollars on minority-owned firms, they're not hard-and-fast rules. If there are no qualified minority firms, then the work still goes to a white-owned firm, and after decades of discriminatory procurement practices there aren't necessarily a lot of minority-owned firms with the capacity to manage government contracting projects, so it's not like white folks are suffering in any way in the government procurement space.)
Except, y'know, for being ineligible for certain government contracts.
What's worse is that those people are being penalised without having been guilty of anything. This is utterly inimical to a free and health society.
Yes, there are some lingering negative effects from past racist discrimination: but there are also ongoing negative effects from current racist discrimination. You don't fix racism with more racism: you fix it by not being racist.
And that has done long-lasting damage to their communities. And they weren't guilty of anything, either! You're absolutely right, decades of systemic racism were utterly inimical to a free and healthy society, and minorities bore the brunt of that for a very, very long time.
Systemic racism created a very un-free and un-healthy situation. A lot of blameless people suffered through no fault of their own, generation upon generation.
Couldn't agree more, racism is bad, &c &c.
This small attempt to undo that--preferring minority-owned businesses for a certain amount of government dollars spent--doesn't really seem to be affecting white-owned government contractors, and it does seem to have a pretty positive effect at starting to undo the effects of all that (utterly inimical to a free and healthy society) discrimination.
Again, you can cling to that flag of "all racism is bad!!!" and that's fine. I'm at least somewhat satisfied that you went so far as to acknowledge that there has been discrimination in the past--a lot of people who sound like you are reluctant to even admit that there's ever been a problem. So, well done you.
In terms of outcomes and fairness and justice, however, there are probably better principled stands to take than the one against minority-owned/woman-owned contracting preference programs.
And there's this often-explicit assumption that the biggest thing holding back minorities is the white man. Nothing could be further from the truth today. The biggest problems faced by poor minorities are outrageous levels of violent crime among themselves, widespread illegitimacy, family collapse, and broken cultures that value the wrong things. Many groups have thrived against all odds throughout history, and they sure didn't get ahead with victimhood politics. The story of the Jews always comes to mind. Persecuted for over two thousand years, they not only survived but thrived in some of the most hostile environments possible, like repeated mass expulsions from different European countries, culminating but not ending with the Holocaust. When they got Israel, they quickly made it the richest, freest, and most powerful country in the region, by far.
I also want to reiterate my point from my earlier comment, which I don't think you responded to, that this kind of feel-good policy-making comes back to bite us. It hurts the taxpayer today by using up more of his money to provide the same service, and it hurts him tomorrow by obscuring what the real costs are and perpetuating bad decision making. The creeping inefficiencies become a permanent dead weight on our ability to invest elsewhere or weather a debt crunch.
If we want to have social programs that give minorities a leg up, then let's vote on those and forthrightly decide what portion of our national income we are willing to put that end. Hidden welfare for well-off minorities isn't the right way to do it.
I have relatives like you, too, who come out with something ridiculous like "the moon landings were faked!!" or "cancer's all a scam by Big Pharma to keep us doped up!!" and we used to go back and forth for hours about it. I eventually stopped engaging because these were people who were just carrying around a worldview in their heads that wouldn't allow them to have an honest discussion, and obviously bore no relation to reality.
So you go on thinking all the things you do. I don't think I or anyone would ever persuade you that you had some information wrong or some history wrong or some principles in your thinking wrong. I'm not even going to try, because I know what it's like arguing with folks like you.
Best of luck, and carry on!
Coming from tech startups this was completely shocking, I was so used to an engineer driven culture. That developers couldn't write their own tests, or manage their own infra, or deploy multiple times a day was super frustrating.
There sounds like there are some great initiatives to change this old approach, but until then I can't imagine many talented devs would put up with the bureaucratic bullshit.
Honestly, most of the wasted time wasn't the engineering teams being slow, but anytime something had to be run by the government, it halted. It was utterly depressing.
I can see advantages to having someone else write tests for the developers.
And I don't see how they stop you from writing tests for personal use, if you want to.
I have no problem with other people testing my code, but that there wasn't a culture of engineers writing their own tests at all was surprising and troubling.
Tests you write for your own benefit are not a deliverable. They're part of your work process.
As I said before, it's not ridiculous at all to require that deliverable tests be written by some other party than the developer whose code needs to pass them.
I'l roughly segment projects into those that can fail a bit and those that can't.
Now obviously, for things that can't fail even a little you can't just take someone's word. But why can you take the word of two people, or ten? The failures (in programming and code review) are absolutely correlated. If one person fails at something another likely will fail in the same place or when reviewing it. When people die if there's a bug, or you're even losing a lot of money, you should not be trusting best-effort human anything. This is where imperative programming against a test suite breaks down and just can't cope. You need to switch out your internals for something provable in its domain. (eg, the math to land a space shuttle in a fixed-time infrastructure (ie before it lands itself)).
And for everything else, it's a $/$ calculation. How much do you want to spend to have some unknowably smaller amount of risk? And usually the best value for the dollar is having the original engineering team work on the tests, with outside oversight and good metrics. If an integrated team is having problems getting full branch coverage (the only worthwhile coverage metric...) in a given method, they rewrite the method. External teams have to test what's given (or waste a ton of time in communication delays) and that usually ends up with suboptimal tests and suboptimal coverage. What you can't do though is simply ask a developer if their work is properly tested and trust their answer. You need real metrics and to know what they mean for you. (Like benchmarks.)
Spec verification is (should be...) black-box integration testing. And yes, this is a good chance to get a second set of eyes on the spec to make sure nothing was missed in the implementation. (unit-tests can't catch missing code!)
Code tests should be more white-box and should be measured against line/code coverage instead of spec coverage. These are what I think the original devs should write, and should be 95%+ of the total test volume.
The problem with expecting testing to pick up spec errors is that it takes comes at the end of the design/build phase instead of the beginning when you can make changes easily.
Nobody's stopping them.
No regrets leaving the agency world.
Yeah... the clue is that an entire bureaucracy is required before even the first line of code was written. This thing probably took months and months of meetings with tons of people before the prototype was written.
A beginner dev could build many simple web applications in a day. However, it will probably have security holes and spaghetti code.
Even though it may be cheaper to just do it 10 times over with cheap one person shops and throw out the worst 9.
I get the feeling this "knowing who to sue" saying is a bit of a myth.
It's an arrow that points in one of two directions. I can not think of a single way this could be made insecure. Even if someone was able to influence the direction the arrow was pointing, everyone is still going through the same ineffective screening process.
High chance somebody gets shot.
2. Everything required to make a bomb can be bought past the TSA checkpoint.
3. It would stand to reason that if a terrorist organization can get a conspirator to infiltrate the TSA, they could get, I don't know, two?
4. There's more than one person involved in the screening process.
It's not like TSA agents haven't already been caught letting people smuggle contraband past them:
http://www.foxnews.com/us/2015/12/19/tsa-agent-accused-smugg...
Not even a newbie programmer is going to do this. Every language out there has a Random() function.
Seriously, all these worst case scenarios and we're still well within the land of what even a middling iOS developer could crank out in under an hour.
I wouldn't bet on that. I've seen all sort of horrendous monstrosities that students and new grads have come up with.
For example, even a newbie wouldn't trust a remote client to tell you the size of a string they are sending you so you can echo it back to them, yet we still had Heartbleed and it sat there for a long time, undiscovered.
That quickly takes you down a rabbit hole, which may be why the contract was so expensive.
http://stackoverflow.com/questions/9234686/generating-random...
I don't actually much care about the security of TSA hardware (my view is that it is all expensive security theater in any case). However, if you told me you spent 300K designing a very simple control interface for say, a critical component of the electrical grid, and the argument you gave me for the cost is proper security engineering, I would buy that. I certainly would prefer it to a $30 USD solution developed as a HackerRank project, even if the nominal functionality is the same.
If they sit on it and whip it out the moment the green light is given, then what?
If they whip it out at the next red-tape meeting, then what?
But if you actually did that, you'd be told off by your manager for wasting resources.
You either understand and enable the other people to accomplish what they want to accomplish, or you're not being invited ever again.
If you did that, your contract would be terminated and the PMs would find someone who knows how to turn that 1 hour prototype implementation into 3000 hours of work.
We went through months of daily meeting with 10+ very expensive execs, PMs and Business Primes from all over the business.
One day the only other coder in the room leaned over to me and said "Are we still talking about <x,y,z> that you or I could code in 10 minutes?"
Yes, we were.
I coded it and showed my managers and so forth. I was completely ignored, and the meetings rolled on. In a place like that, there are so many people that have job titles that have nothing to do with "getting it done", which in a funny way makes them less interested in getting it done. Their job is to analyze, plan, document, process map, etc. etc. so that's what they're going to do, no matter how trivial the thing is.
The manager who was in charge of the application thought that a quick solution would make him look bad. I was explicitly ordered to sit on my thumbs for six weeks.
The thing is, once you know the solution every other action is senseless. There are people who are perfectly happy being a billable body for a living. I'm not one of them.
The larger issue is that the government is not setup to move quickly, and for the majority of things the government does oversight is a good thing. That level of process does make it hard to do things like this efficiently, but it's all about trade-offs.
They will be told to shut up, because they haven't listened to the requirements. No one will want to hear it on that call. They will be told it's the wrong audience.
If they push forward, they'll have a conversation with a lot of people after the meeting about how they aren't being helpful.
They'll be told to have a separate meeting with others to validate it. Then in that meeting, those people will rip it to shreds, and indicate that it needs to conform to X Y Z... The PM will then work with you to go through when what actions will be done...
Then they'll schedule new meetings. On and on.. Nothing will change.
I've tried it, and seen it tried. It doesn't work.
You go through the process, or you create even more bullshit red-tape for yourself, and you're seen as a loose cannon no one trusts.
Then you have all the daily meetings to communicate to all of those people the messaging. Then you have the meetings to clarify all the wrong messaging that went in some of those meetings.
Then you have the meetings for the customer facing communication. Then you have the customer communication.
I'm not even close to done...
You can argue that you don't need that many people, and certainly for a start up that's often true.
But in enterprise software, you actually do need a ton of people because of the expectation.
I'm in the Business and came from start ups. I'd VASTLY prefer to do things in small groups to just get things done.
But even though I want that, no one else does, including the customer. They want all the bells and whistles of a ton of people. It makes no sense at all.
Hell, one of my current projects has about 20 people on a phone call, and 4 that actually understand anything going on. Think about all the money wasted in all of that, and yet... It's the way business "works".
TSA spends most of it's budget through a provision called "Other Transaction Authority" which is essentially a vehicle to make purchases with barely any oversight from congress. http://time.com/4134368/tsa-price-of-security/ (Paywall)
Why? Because writing and signing a contract is expensive, for both the contractor and the government. So contracts are typically written in a way to make them very easy to extend, and existing contracts are often used as vehicles to tack on additional funding for new contracts. The original contracting agency would also usually charge a fee to the other agencies for use of their contract in this way.
So, this kind of glance at individual awards under a single contract is really too simple a view. A lot of these funds could be (and from my guess, probably are) going to completely independent projects from the randomizer.
TL;DR: Having multiple awards under a single contract is typically a sign of the government working around bureaucracy and attempting to save costs.
TSA later reported that the actual cost of the randomizer was around $47k. The total figure I had mentioned earlier was part of a larger contract with IBM. I saw the generic "IGF::CT::IGF MOBILE APPLICATION DEVELOPMENT" note on the award and assumed it was all for the same project.
It's also weird that TSA's response to OP's FOIA request cited a different and higher figure (~$340k) for the randomizer app -- I'm guessing they were grouping in other projects under the same award here too?
Besides, $47k seems like a reasonable amount if that figure includes training and deployment costs.
Note, I do understand that sometimes new ways of doing things allow one to solve a problem such it would have previously been near impossible. However, most of the problems that need solving aren't these situations. I.e. if you think you have a big data problem, you probably don't, etc.
I think this especially detrimental in the government sector because the folks buying the services don't always understand the technology or the maintenance requirements that come with it. Then we all foot the bill (through taxes) for a lot of OJT for said new tech.
1. Kevin you may want to block out your address in the attached letter.
2. I'm missing the part where this amount is tied directly to the creation of this application and this application only. From my read this is a T&M contract for mobile development there's nothing specific to this application. It could span multiple engagements and apps. Considering this contract went into place over 2 years ago this could likely be the case.
3. This is a cap not an invoice. They could have billed 60k.
This may be the preceding contract but very similar scope and line items if you want more details, no FOIA required.
The passenger can't complain about being targeted or profile or singled out, because, hey, it's random.
(The actual rationale of the randomizer might have been to avoid accusations of profiling, while still being able to do profiling. Leaving the bad guys off balance is simply a benefit.)
(Contract negotiation, etc is not explicitly built into these, they're reflected in a higher billing rate.)
While a dungeon master presiding over a hopeful throw for critical hits would be a solution, I don't think the TSA wants to make security theater entertaining - it makes the accidental nature of the whole endeavor obvious.
Anyway, with the level of uncertainty already high in queue fill rates - I don't think anybody is going to feel confident enough that they'd alter their diabolical plans. Oh and I just remembered something that renders all this pointless, the functionality for randomly alerting is already built into the metal detectors... undoubtedly a contract requirement for a feature that is never used.
Perhaps I'm exaggerating. It would certainly raise costs to the attacker to recruit an additional collaborator, and to wait for a serendipitous scheduling. But I'm not convinced it raises the difficulty by $300k.
Additionally, by observing several passengers being directed this way or that, you could brute force the seed and the system becomes deterministic.
https://play.google.com/store/apps/details?id=tobi.wuerfel&h... [1]
[1] Not affiliated or tested. It was the first I found without inapp purchases or ads.
Not that I really think that matters at all, but that would be shot down in the first meeting it was discussed immediately, and you'd be considering insane if you kept pushing it.
https://kev.inburke.com/kevin/tsa-randomizer-app-cost-336000...
It's a time and materials contract which means the contract award is the "ceiling", but you have to perform work billed against it to get paid any of that.
Plus, the second page details several extension options that could increase the ceiling to $1,176,280. Were those options exercised?
An excellent use of secrecy technologies would be government contracts: partition problems into pieces that don’t necessarily reveal the final purpose or customer (e.g. “choose uniformly between A and B” is too vague to be guessed as a requirement from only a government agency). Then, anonymously ask for bids. That way, you might actually get the developer who offers a measly price for a trivial task, and only finds out later that his check is from the government.
Z-effective scores...findable.
Really, what we need is IBM's invoices and TSA's pay statements.
Also, you shouldn't need to file an FOIA request to get this information. Shouldn't all bids and contracts be public?
[0] Exemption (b)(4): Records that contain trade secrets and commercial or financial information obtained from a person that is privileged or confidential.
The app was probably $10k, rest was the compliance cost.
The real distribution of where the money goes on projects is certainly funny from a sane persons perspective, but in a different way than this :)
I don't know how anyone can look at this situation and not see the government is wasting a lot of money in their pursuit of not understand technology.
Can we expect to see the source code for this application pretty soon?
This is a completely reasonable price. Honestly, it seems cheap.
int tsaNumber = rand.nextInt();