Rent-a-coder hilarity (2008)
blog.willbenton.com
blog.willbenton.com
He wanted sample vulnerable Java code to use for secure development training. None of us enjoyed writing Java web apps, so this seemed like a cunning plan to get sample (yet custom) code with minimal effort. Hopefully, full of bugs.
He asked for a "task tracker" (IIRC, might have been phone book), and specified Java with "no framework" as the implementation language. He chose the dodgiest bidder he could who looked like they might finish the job. I think he put a few gotchas in the specifications.
Actually, as delivered, the app was solid, and based on JSF2 ;)
What I learned about rent-a-coder:
* There are corporate refugees who do this on the side to make extra money.
* CRUD apps appear to be a sweet spot for rent-a-codering.
* Code quality is hit-and-miss if you're looking for disasters ;)
Well, maybe he should have specified he wanted an insecure application (or an "extra secure" application - expected result is having several layers of snake oil and "proprietary solutions" applied)
Didn't ask for "extra buggy" or "extra secure" because the ideal was the combination of honest mistakes and laziness that turn up in real code all time. I like the idea though ;)
I mean, you'd think that for $100 or whatever all up, they would have half-assed it a bit ;)
Probably the mistake was to stop at n=2 and not choosing a harder problem domain.
But more to the point, if your friend didn't have a lot of experience with this sort of thing, he probably wasn't that good at picking a bad apple. Or he just got unlucky, maybe.
Poor communication? Probably just ESL. Low bid? In a low GDP country that low bid goes a longer way.
If you grabbed an american with the same conditions you're more likely to get some illiterate kid instead of some Czech doctor moonlighting.
The first thing that strikes me is that people posting projects don't really understand the value of software engineering. They want someone to do crazy ambitious projects for pretty much no money. As if to say, "I have this idea which is really awesome and valuable, now I just need some monkey to do all the work." $500 should be enough for that, right? Then I'll cash in on my awesome idea...
Maybe some projects honestly work out like that. But a lot of the projects I see on these sites fall outside that scope. For big ambitious projects, we all know programmers are not fungible resources, and competent execution matters.
But the worst part is that people humor these projects, or more likely take advantage of the gullibility of the people asking for them... "Yes sir, I can solve that impossible problem promptly and under your budget of $500." It's really sad.
Not everyone lives in San Francisco or New York where they feel they must bill $100 p/h just to make rent. A smart person in an emerging economy where $500 per month puts them on a higher part of the local income curve could certainly realize that they can compete in a global marketplace on price. This does not necessarily make them incompetent, in fact it's not a bad way to "hack" oneself out of poverty.
Besides, many software applications are alternatives to excel spreadsheets or to handwritten HTML websites and don't necessarily require the hippest programmers who are well studied on the latest technologies. A customized rails scaffold can go a long way to solve a lot of real business problems, and doesn't HN often talk about the value of producing an "MVP"?
There's a Hollywood movie trope rolled out every 2.8433 years where a rich guy goes back to university and tries to buy his way past projects and tests. I always kind of imagine some rich kid in "Web Programing 317 section 7" contracting out his final project. Now that tuition is so expensive, $5K for a final project sounds more or less reasonable for a kid who just wants the credential and not the education/training. There should be some kind of ethical filter where if you're obviously trying to contract for a stereotypical final project around the end of the school year...
While this seemingly has been "popular" in Europe for PhDs / doctoral theses for quite some time (in Germany e.g. some ministers recently lost their posts for that or similar activities), it has become an even larger issue in countries like India across all levels of academic studies - see https://news.ycombinator.com/item?id=4610739 for more info on that.
I always found that oddly specific - they clearly acknowledge that people will use the site to pay someone to do class projects, and get better grades by doing so, and they very much don't care so long as it doesn't determine whether the buyer graduates.
Since anyone can bid, naturally you'll see bids on these impossible jobs, too.
But, underneath this superficial layer of incompetence lies a good selection of developers and clients in each of the freelance sites. Having done some work on more than one of them, I can say that as a developer, the key to finding actual work is to take the tests, have a portfolio, and wait for clients to contact you. If you must bid on the jobs by searching, search hard for the realistic ones and bid appropriately. The clients that understand their work will not usually go for the lowest bidder just because. They'll go for the one they have the most confidence in.
Rates are going to be driven by cost of living as much as expertise, so it's not unfeasible to imagine skilled developers in Malaysia / China / Russia / India who didn't get a green card (or didn't want one) being able to comfortably sell their services for < $10 per hour.
tbh, there should be a price picker. you specify a timeline. you specify quality, and then price is calculated for you.
this communicates that your want mvp, or real development. And for each the site can provide heuristics. that's where the site's secret sauce can come from.
Genius!
However I think BUG count is not the issue. It's code quality. What if I told you:
"I need a POS implementation of Foo, it doesn't have to be maintainable but has to work with a taaaaad bit of ability to patch it up. I am trying to prove a concept is viable and show it off to like 50 people. I will rewrite this code if I decide to implement the concept. Also I want this done in ~ 1 week"
Should that not drive down the price and time required?
That means probably minimal tests, minimal abstractions, lots of copy-pasta, but still within decent coding guidelines and meets the needs
create a few small tasks to get started. rent out a few coders for a week. make all of them code the given tasks. pick the best quality and continue.
you will get good quality out bad, whatever you prefer... And waste little money.
the only flaw is if all the coders who are participating are not the quality you wanted.
most people don't understand the costs and what you get for $100...
I'm not quite sure how to work that - maybe sue them as some sort of accessory to fraud, though that would probably first require the university charging cheating students with fraud.
Dear Mr T.,
Your problem is very much like I have been considering. Assume that we have a consistent and complete axiomatization of all true first-order logic statements about natural numbers. Then we can build an algorithm that enumerates all these statements. This means that there is an algorithm N(n) that, given a natural number n, computes a true first-order logic statement about natural numbers such that, for all the true statements, there is at least one n such that N(n) yields that statement. Now suppose we want to decide if the algorithm with representation a halts on input i. We know that this statement can be expressed with a first-order logic statement, say H(a, i). Since the axiomatization is complete it follows that either there is an n such that N(n) = H(a, i) or there is an n' such that N(n') = ¬ H(a, i). So if we iterate over all n until we either find H(a, i) or its negation, we will always halt. This means that this gives us an algorithm to decide the halting problem. Since we know that there cannot be such an algorithm, it follows that the assumption that there is a consistent and complete axiomatization of all true first-order logic statements about natural numbers must be false. There is no way your application can be built, and for this advice I am willing to waive any fee. Sincerely, K. Gödel.
Of course, a tool that could automatically prove useful properties about programs with no guidance, which would be significantly superior than all existing practical program analysis tools which require quite a lot of guidance, might be beyond the capabilities of the people that post bids for $300. Unless it used Mechanical Turk. :)
I wish this were a more popular thing. When I write a function, I can probably enumerate in my head all the things I want it to do, and not halting usually isn't one of those things. But the instinctive reaction is to worry about Turing-completeness, which is puzzling. When's the last time you wanted a program with fundementally unprovable behavior?
I'm working on one today. I'm analyzing a data set for tens of millions of possible statistical anomalies, and I'm putting my "investigate further" threshold to statistically one in billions. Each thing added increases the size of the search space.
Knowing that it is extremely unlikely to run forever really is good enough. Proving that it won't is a non-issue for me. (Right now it takes several hours to run.)
You don't usually want fundamentally unprovable behavior. However, you also don't want to sacrifice the ability to compute anything that is computable. And you can't throw out the bathwater of the halting problem without throwing out the baby of computing everything computable.
So, in practice, you want to use provably-halting-on-all-valid-inputs algorithms for the problems where you have available (either known, or that you can find with reasonable effort) such an algorithm, but retain the option of using other techniques for other problems.
Bah humbug. You can be a totally competent engineer and not immediately recognise the halting problem, just like you can be a totally competent auto-mechanical engineer and not recognise a oxidation-reduction reaction.
You can solve high-value real business problems every day for the rest of your life and never encounter a single decision that will be meaningfully informed by knowing about the halting problem.
Computer science != software engineering.
I dunno; I find that plenty of times dealing with fairly pedestrian business-problem requests its not uncommon to run into something that is desired that is equivalent to solving the halting problem; recognizing the difference between problems that are merely difficult and potentially expensive and risky investments of time and ones that are knowably impossible is pretty useful in terms of choosing where to allocate time and effort.
> Computer science != software engineering.
True, but software engineering depends on computer science, and, as in any form of engineering, understanding the well-known outer theoretical limits of what can be done is pretty important to engineering.
> True, but software engineering depends on computer science, and, as in any form of engineering, understanding the well-known outer theoretical limits of what can be done is pretty important to engineering.
I disagree - you can get by very well by understanding the inner, practical limits. Again, just like an auto-mechanical engineer: He doesn't need to understand the details of combustion or the process of refining oil, just enough to understand it works in the context of his domain: engineering cars. On the other hand, he also needs to know a little about suspension (but not the calculus of the harmonic oscillator) and enough metallurgy to weld the chassis safely.
And don't get me started on computer scientists that can't code.
In practice, many of the most useful things to invest time and energy in are difficult, potentially expensive, and risky (often, because of the risk, you want to dual track this with a less-risky, lower-payoff approach) things that have high payoffs if successful.
OTOH, the knowably impossible things are dead ends. So, no, they aren't the same thing in practice.
I think the real crux of the problem is that you can't determine in any of those cases whether the program is going to terminate. Consider bogosort or a brute-force algorithm for solving travelling salesman. These can take millions of years to complete even for small input sizes. Do we consider that an "infinite" runtime?
That's true, external inputs and true randomness is not accounted for. (However pseudorandom generators are included!)
I'm very proud of what we achieved so far, because we eliminated the biggest "pains" of online outsourcing; There is no bidding, our (invited and reviewed) contractors form a single estimate which is then presented to the client. And we focus on sub-$500 tasks, because the margin for error is significantly smaller, so if a client comes in with a huge job, we help him break it down into smaller pieces. We also maintain client/contractor ratio, so there is no need for contractors to compete with one another, anyone who wants to work, can work.
Why would buyers desire this?
Outsourcing is supposed to save you time, bidding wastes your valuable time.
Why does it waste more of my time in this bidding process than any other aspect of the marketplace? It's how you minimize costs, and any smart seller that realizes you're not going through the bidding process jacks up the price to milk you for all you're worth.
Honestly, I rarely get more than one or two bids when I'm looking for someone to work for me. I've been around long enough to have built relationships with vendors for various tasks. I know about how much I expect something to cost and if the bid comes in significantly higher or lower then the vendor and I talk it over and negotiate either a lower price or a changed scope. They don't jack up the price because the cost to acquire customers is very high. I'm guessing I saved 100 hours last year by not putting every last task out for bid.
So in my experience, bidding doesn't do any good, to either party.
On one of the other websites, they seem to want to install a spyware on your computer so the employers can see what you are doing at any given moment. I doubt a great coder would put up with that (I'm merely good and I wouldn't) and the odds of finding one there are lower. On the other hand, I think a great coder would be more likely to freelance on a site that has consistent work which they don't need to fight over.
So that's my speculation.
It's obviously of benefit to the contractors, I meant the other side of the transaction.
I'll be curious to see which client gives you a testimonial that thinks non-competing contractors is a great idea.
I notice the word "price" is not present in that testimonial.
It can be very valuable to have static analysis tools in an organization, I know my own company has them, and we don't share them. IBM as well puts tons of work into it. Motorola has shared one of their tools. Every bug and issue your tool can find is one much more cheaply solved.
Similarly, the client may be doing coding education, or interview, or other software where just a few limited lines and limited operations need to be checked, or it is OK to say if you hit n^4 loops you lose, etc..
http://www.getacoder.com/projects/solve%20p%20vs%20np_132036...
I used to work in a startup where the cubicals had sequential phone numbers. From time to time I'd get a call from one of the sales reps in the field.
"I'm talking to a customer," the rep would say, "Can our product do X?" Where 'X' was something like, oh I don't know, 'Computational Sushi' or 'License plate recognition,' or anything unrelated to what our real-time messaging middleware did.
"No," I would tell them (but using more words).
Then I would move to the next cube and instruct the engineer sitting there that the phone was about to ring, and that they should pick it up, listen and say "No" quite firmly. Repeat for the remaining cubicals.
Salesmen will say anything to make a sale. It's what they _do_.
The result? I was downvoted by some less than informed individuals who thought there was some deep fundamental difference between curly braces languages and whitespace significant ones. What The!?... On HN!? This was before Coffeescript was a widely known thing as it is today. Think what you'd like about Zed Shaw, but his "DB Invasion" idea has some merit.
me: What information do you have and what do you want to look up?
client: I have the folder and file names and there are 5 webpages like google where you can search for music. I want the programm to get the right song info from this. BUT IT MUST WORK 100% right no error ever! I had 6 other programmers try it but no one can make it right!
(the real clients english was even worse)
One need only specify some reasonable length either equivalent to infinity for such a program, use a heuristic of checking whether the program is in the same loop and if so, whether any of the variables involved in determining loop termination are changing. Of course, proper security precautions are required.
Of course, the real question anyone asked to develop such a program would be what runtime is equivalent to an infinite loop. Any real-world program must have a runtime bound that is predictable on invocation or it's useless.
For any finite number of clock cycles N, in a Turing-complete language, I can write you a program which terminates but takes longer than N cycles. There is therefore no value of N which will consistently give the correct results in your program.
Turing machines have an infinite tape, so under a Turing machine model, there are infinitely many loop termination conditions, and so it is possible to craft a program which never repeats the same state within N cycles for any N but still terminates.
If your machine is finite state (with no inputs) and s possible states, rather than a Turing machine, you can check if the program halts:
halts s program initialState = do
seenState <- newBooleanArrayOfSize s
setBooleanEntry seenState initialState
halts' initialState
where
halts' state0 =
if state0 == HaltingState
then return True
else
do
seenAlready <- getBooleanEntry seenState state0
if seenAlready
then return False
else
state1 <- nextState program state0
setBooleanEntry seenState state1
halts' state1
I see some have commented on the economic desperation or the salesman attitude of the coders bidding for this impossible job, but when someone asks a large population of salesmen that they have a serious intention to procure a bridge ... well ... there will be a few who'd definitely show up wanting to sell one.
Don't see the point of this whole thing really.
No, considering the poster's name is "AlanT"!
I kindly propose a two-phase project based on Test-Driven Development methodology, of which we are expert.
1. Specify and develop test harness.
2. Develop program that satisfy tests.
Any finite approximation would modify the task from an impossibly infinite one to something possible.
But I stand by adaption of the old sales technique "What does ____ mean to YOU? ... Oh, that won't be a problem."
- Party A contracts with Party B on a big software contract. - Party B sub-contracts on some of it to Party C, D, & E. - Party E is a bit dodgy and does a bunch of rent-a-coder. - Hey it compiles! It gets passed up the line. - Party B is late and just passes it on. - Party A is late and, "Hey, it works!" and delivers. - Hilarity does not ensue.
My personal experience with this kind of thing is that the further away from the actual concerned party you are (in other words, the people who are going to use the software), the more likely it is that the software is going to be wrong.
You have people responding that are assuming that they're going to test if a process takes too long and bail if it does. Pretty simple, actually, since for all practical purposes, when somebody describes a program as "taking forever," they rarely mean "forever" literally.
If you want to ask it, go ahead, but understanding practical limitations and engineering to those is a perfectly acceptable answer for the context of the rent-a-coder site (and an interview for a coding job). The only time it's not is if you're interviewing for some sort of academic position.
Is that advice online anywhere?
One of the biggest gripes from top tier engineers is that they hate being treated like animals by jerky business guys, and here we have a group of people mocking these engineers and the platform. It all seems disrespectful to me.
describe 'My experience, -> _.map ['amazing','awful'], (quality) -> _.map ['engineers','business guys'], (role) -> _.map ['Brooklyn','online platforms'], (place) -> assert "There are #{quality} #{role} in #{place}."
For giving a percentage, you need to specify a distribution of programmes first. E.g. programmes created reading from /dev/random, or programmes downloaded at random from the internet, or something more sensible.
1) Assuming rand(x,y) gives a uniform distribution between x and y:
for (int a = 0; a < INT_MAX; a += rand(-1,1));
3) Anything that makes a system call. You might never get scheduled again!
"The proof itself is over 100 pages long and consumed seven yearsof Wiles's research time. For solving Fermat's Last Theorem, he was knighted, and received other honors."
http://en.wikipedia.org/wiki/Wiles%27s_proof_of_Fermat%27s_L...
i = 4; while(there exists such p and q that prime(p) and prime(q) and p+q == i)) i+=2;
keyword: Goldbach conjecture. The Goldbach conjecture states that this program will never halt. No one could prove this yet; seems to be really hard.
"Every even integer greater than 2 can be expressed as the sum of two primes."
My program is simply constructed in such a way that it terminates if and only if the Goldbach conjecture is not true. Maybe my pseudocode is not very precise (and also english is not my mother language), but given the Goldbach conjecture anyone can construct the program more precisely if needed.
int i = 4;
bool sumFound = true;
while(sumFound)
{
sumFound = false;
for(int p = 0; p < MAX_INT && !sumFound; p++)
{
if (isPrime(p))
{
for (int q = 0; q < MAX_INT && !sumFound; q++)
{
if (isPrime(q))
{
if (p + q == i)
{
i += 2;
sumFound = true;
}
}
}
}
}
}
I guess what I'm saying is, isn't the implementation of the program the test in itself?
http://www.getacoder.com/projects/simple_meteorjs_app_157993...
If the customer is not happy, there are some sorts of resolution processes that I don't know much about.
What surprises me is that the clueless developers also had high 'reputation' (positive customer feedback), which should be at least some indication about their competence.
foreach(newProject as project){
new Bid( projeckt.averagePrice * 0.8 ), "We are the best nd have done this many times");
}
Some clients include a "password" in the project description and ignore all messages without the password.
Perfect.