18F micro-purchase experiment bid down to $1 | Hacker News Reader

18F micro-purchase experiment bid down to $1

github.com

101 pointsxn58 comments

I started off my career at 19 years old building custom 468x80 pixel web banners back in 2000.

I found customers on EBAY (yes, literally EBAY's web section) and charged $9.95 a pop which was a fraction of the big agency rate.

I would pour hours into designing and tweaking the banners to fit into the 15k size allotment of the day while also being aesthetically beautiful.

After hundreds of iterations I learned how to not only make beautiful banners, but banners that drove CLICKS & conversions which was all the customer ultimately cared about.

As the free market took hold in my own life the customers that were showing up for the cheap rate were pushed aside from the customers that wanted to harness the value I was delivering and were happy to pay increasing fees.

On the back of those customer relationships I've spent 15 years building amazing businesses and exceeding my highest expectations for where a lowly photoshop jockey could go in the world.

My point is while this 18F project is fascinating as a thought experiment the publicity alone for the winning bidder @ $1.00 disqualifies that from being the actual "rate".

This isn't an experiment in "I'll deliver quality, high-skill work for $1.00" this is an experiment in "Who wants a SHOT" at making a name, an intro, or have a permanent stand-out line on their resume.

If you want to see what kind of work really gets accomplished for $1.00 at scale, go poke around Mechanical Turk or Fivver.

Sorry, off-topic, but if you have those banner ads from 2000 sitting around, you should look into throwing them on archive.org if you haven't already. Nice little piece of Internet history you may have there.

Charging $1 is less ridiculous when you consider how many people also donate software design and development work for free to causes they believe in all the time. How many developers do you think Bernie Sanders has who have effectively bid $0 to create projects for his campaign?

Back in 2004, I worked for a SaaS startup that sold reverse auctioning services for low-cost suppliers (e.g. China) to huge US manufacturers. The biggest problem that we faced was that the final auction price would end up ridiculously low. The manufacturers were super happy until they found out much later that the supplier couldn't deliver.

Then, we came up with a new scheme to do certification and auditing on low-cost suppliers. We even had a large team of scouts combing through these countries and conducting reviews. That way the manufacturers could limit their auctions to only suppliers that passed certain filters.

Again, the manufacturers were happy. The lowest bid price would always go really low. They were happy, that is, until the suppliers started to quickly game the system. They'd pass the review criteria but they'd use other tricks - outsourcing to other companies, degrading other parts of their service offering, etc.

Then, it just ended up being an 'arms race': we'd enhance our criteria and they'd find somewhere else to cut corners.

Needless to say, our company was sold for a loss (mainly for our patent portfolio) and absorbed into our investor's other companies)

The bidding structure, as specified is:

1. You can only bid lower than the previous lowest bid. 2. At the end of the auction, the lowest bid will have 10 days to complete the project. If they fail, the next lowest bid will.

This appears to be a simple attempt to game the system. A low bid of $1 prevents any other bids from being registered, which guarantees that whatever the previously lowest bid was will not have to compete on price with other good-faith bidders.

This would work if the lowest bidder failed to perform, they would have to pay back the second lowest bidder. For example, if they bid $1, but the second lowest bid was $1000, they would have to pay back $1000 for not performing. They should probably also establish a floor to eliminate this $1 gaming mechanics.

What if the lowest bidder is nowhere to be found?

They could require a deposit to bid, I suppose.

For larger projects, the contractor is required to be bonded to cover things like this.

Is $1 a large project?

Since the amount at risk is the amount of the bid prior to yours, that should determine both the amount of any bond requirements and, if there is a sharp cutoff, the cutoff for the requirement for a bond.

the point of this process was procurement in increments less than $3,500 to avoid many regulations.

If lowest bidder is nowhere to be found, then the next lowest bidder gets the work.

You chose way of punishment. It is better to split work into 3 parts: 2/3, 2/9, and 1/9, and assign 2/3 to winner, 2/9 to second winner, and 1/9 to third winner. In this case whole ecosystem will grow around these auctions, which will produce into better market and lower prices automagically.

So you get two friends and they bid $2 and $1.

Simplest way to make this only weakly gameable is to allow bids to be entered that are higher than the current minimum so that you can bid to be the 2nd person to do work. It's still gamble because you can have people bid artificially low in order to intentionally delay work or give someone more time to do things, but it's likely a better system than failure penalties.

It seems like the "strategy" here would be 1.) submit a bid of $3000, and 2.) have a friend submit a bid at $1. The bidding closes, your friend intentionally sandbags the project, then you complete the project at $3k.

For this to work, your friend would need to have a company registered on SAM. Although, I can confirm that this was not the case in this bidding. I am the bidder who bid $499 - which is the next lowest bid after $1. And no, I did not ask a friend to bid $1.

Why did you bid $499 for what is clearly more than $499 of work?

I don't think that's obvious. There aren't necessarily easily interchangeable contacts available for more than that, and for many people $500 in under ten days is more than they would earn in their job. Students for example could easily be qualified to do this, perhaps be on a break and have time, and would otherwise be earning nothing. Not to mention the resume padding and connection building opportunities.

I understand the work is more, however, I recently setup my consulting business and this seemed like a great opportunity to get the name out there and expand the network.

They could've avoided this at least partly with a sealed bid process and a (secret) minimum reasonable bid to disqualify ridiculous low bids. Advertising the fact there is a floor but not its value.

At that point, rational bidders would have had no choice but to bid something they are willing to work for. Collusion between bidders is possible (as routinely happen for things like road paving...) but if there are enough bidders then it becomes impossible to collude with reasonable success expectations.

Further a registry is needed to allow black listing of trolls and penalizing of folks who routinely fail to meet terms. It would also be possible to give slight weighted edge to bidders that have a history of delivering conformant implementations.

The micro-purchase experiment is .. an experiment!

My guess is that bidders are interested strengthening the mechanism, so they've demonstrated exactly how this early version is broken.

Why would bidders want a stronger mechanism? So they can have a reliable income stream going forward. The broken mechanism serves no one well.

Update: https://news.ycombinator.com/item?id=10522022

Their bidding rules encourage participants to behave strategically by bidding more than their reservation price, frequently rebidding and waiting until just before the auction ends to place bids.

Here's a suggested bidding format that would avoid these problems:

(1) Everyone bids the lowest price that they would be willing to accept for completing the task

(2) If the lowest bid is unique, the task is assigned to the lowest bidder and they are offered the rate offered by the next lowest bidder

(3) If many people bid the same price, one is randomly chosen and they are offered the rate that they bid.

(4) If they fail to complete the task according the requirements, the next lowest bidder is offered a quote according to the criteria above.

This is just standard auction design theory.

Second price auctions are pretty great [1]. I'm sure 18F will figure things out the next time around.

The interesting 'broken' mechanic in this system is the fact that you can't bid higher than the current lowest bid, which isn't usually a problem. Is fixing that equivalent to having a second price auction (in terms of incentives)?

[1] https://en.wikipedia.org/wiki/Generalized_second-price_aucti...

As you say, in a standard auction you can expect the same equilibrium using second price and a traditional bidding system. The difference is that participants need to waste time rebidding after being outbid and can face a technical challenge of submitting bids in time in the scrambles that inevitably occur just before auctions close.

However, as you point out, the situation gets worse because we want to create a queue of low-bidders willing to step in should the original winner fail to deliver. Allowing multiple bids and/or bids higher than the current lowest would go some way to addressing this but participants may bid strategically.

Eg the current lowest is $50, I'm willing to do it for $70 but don't expect any more bids under $100 so bid $95, failing to get the job when someone else bids $80. Also it's unclear whether I get 2 x 10 days to complete the task if I bid both the lowest and second lowest bid, which is clearly suboptimal and could even lead to people placing a chain of lower and lower bids, completing a functional solution but then deliberately waiting for their lower bids to time out before submitting the solution.

A second price auction would be better in this case for both these reasons.

Good point. The additional problem here seems to be people underbidding significantly (obviously $1 is an extreme), and there's no way for 18F to determine quality. While waiting 10 days doesn't seem like a huge problem, the overhead of iterating through contractors could become huge, especially on time-sensitive tasks.

One solution would be to charge each bidder (at the time of the contract, if they're the lowest) a 'down payment' to be involved - where the down payment might be some function of their bid and the next highest bid.

Interesting, but what am I looking at? Someone is willing to do a job for a dollar?

"what am I looking at?"

An utterly terrible auction system.

What's terrible about it?

They didn't think that they would be attacked.

The system specifies that the lowest bidder gets the job, and has 10 days to fulfill it. If they don't, the second-lowest bidder has 10 days.

The lowest bid possible is $1. Once that bid has been made, nobody else can bid.

Strategy: create 2 companies. Company A bids the maximum price as soon as the auction is created. Company B bids $1 immediately thereafter.

The underlying entity now gets 20 days to complete the work for the maximum price. During the first 10 days they need to be careful not to accidentally send in the final project.

"During the first 10 days they need to be careful not to accidentally send in the final project."

This deserves a special mention as a risk factor for this sort of business model :)

It's trivially broken if you can get in the first two bids:

(1) Bid high, profitable amount, then:

(2) Bid the lowest possible amount.

(3) Do nothing for 10 days, meaning the result of the auction reverts to the 2nd highest bid (your initial high bid).

Would it help if we triggered a new round of bidding after every failure, disqualifying anyone who won previous bids?

Seems like you should just allow anyone to bid anything at any time. There's no reason to disallow someone from bidding higher than the current lowest bid.

Sorry, I thought I edited quickly enough but I clearly didn't. Readers, I had previously omitted the wording winner so it read like I wanted entirely new bidders, disqualifying all previous bidders.

That would solve the immediate problem, but make it easy to DoS the system with $1 bids

You can submit low bids, not do the work, and delay projects. The rules will obviously need to change for the market to be functional.

A job for the government, specifically. 18F is the (outgrowth of the) team that overhauled healthcare.gov. Presumably it's an altruistic open source programmer willing the help out the government without pay.

https://18f.gsa.gov/

I think it might be a programmer trying to troll the bid system, and willing to code (or "lose their bid on acceptance criteria").

A company doesn't need to be purely altruistic to 1) publicly contribute to an open source project and 2) effectively 'win' a government contract. Given the simplicity of the task in question, one could choose to take it on as a reasonable business development expense.

I suppose it could be someone who wants to work on the project for kudos...

Or, more likely, there is someone trying to game the system as one of the comments suggests.

I would turn an investigative eye towards the second lowest bid.

[edit: cwarden's comment in the thread explains the problem perfectly - rule's lawyers are quite useful]

How it is best to cook kudos? I have some spare rice and tomatoes...

If you have a day job, the scale of these tasks are (potentially) implementable by a 1-3 person LLC. A $1 bid could be for resume fodder, a more serious bid is bonus money.

So technically someone could have bid $1, $2 .... $x, then not do anything and delay the project by 10*x days?

You can only bid lower than the previous bid. By placing a bid of $1, the auction effectively ends. Now only people who bid previously have a chance of getting awarded the work, but only after everyone who bid less fails to deliver.

GP is saying I could have (with the help of 2999 shell companies) bid $3000, $2999, $2998, ..., $1, and I would effectively have 30000 days to do the work for $3000.

The value of the contract is $1 + $X dollars in expected future revenue from having 18F on your resume, which is probably higher than the starting bid of $3499. $1 is win-win, because reputation is really valuable in our world.

Maybe someone with low expenses gets to put "government contractor" on their resume/portfolio and bill better next time.

The bidding structure was set up to have provoked a bidding war. 18f tried a bold experiment with unconventional ways. It might be good to take a leaf out of the conventional system...at least for things that work well. There isn't much broken in the way the govt takes bids for work: They release an RFP, vendors review it, ask questions, govt. answers the questions, vendors send their bids for the work. All that works well. What doesn't work well is when the govt. teams assigned to review the bids from the vendors don't have the engineering know-how to assess which vendor can actually deliver on what they want. I've seen a CMS system being proposed and go on to win the work to solve the need for keeping scattered databases of submarines in-sync.

If a vendor performs part of the service to the government for this micro purchase bid and does not end up meeting the full criteria, are they able to sue for partial recoupment of costs of delivery of the software and/or entitled to it? I can imagine this has already been sorted for commodity delivery but I'm not sure what happens when that sort of thing is applied to services. Firm Fixed Price contracts aren't usually an all-or-nothing deal, but I guess in this case the case could be made for all the risk being on the vendor side given the scale and intent.

So is this just a miniature version of how most government contract bids work? Somebody bids something stupidly low, and then tries to deal with that problem later?

Not at all. Most govt. contracts work like this: Govt. opens a Request For Proposal on FedBizOps [0], vendors review the proposal, ask questions, govt. tries to answer all questions from all vendors and releases all answers again on [0]. Then vendors submit their proposals with written explanation of how they'll do the work, also, they are required to submit how much it will cost the govt. When the proposal date closes, the govt. sets up a review board that goes through all the proposals by vendors and reviews them based on competency and price. This review panel picks one vendor to win the proposal.

[0] - https://www.fbo.gov/

That's how it used to work, but most places have much more nuanced processes in place now. Which makes it especially laughable how the guys in the GP apparently didn't even do a basic search on the state of the art in procurement and have to learn all the mistakes of the last 20 years on their own :)

was there any part of the bidding where they verified the bidders were registered with SAM.gov?

As part of placing the bid the vendors were asked to provide their D&B number. Whether 18f was actively verifying whether the D&B exists in SAM....not sure.

Reply on news.ycombinator.com