Marketing Lessons from A/B Testing a Homeless Man's Sign
dailyconversions.com
dailyconversions.com
I was expecting something where he made iterative tweaks to the creative (i.e. the sign) and did continual comparisons of the success rate to see how different changes affected the result.
All this guy did was take a homeless guy's sign and add a really cool marketing idea to it, which increased the homeless guy's take. Yes, the results of his changes were awesome, but it really had nothing to do with "A/B testing" per se.
I guess what I'm asking is this: Is A/B testing an evolutionary approach to developing a product? Are "revolutionary" leaps possible if you strictly follow the A/B testing methodology? (My guess: probably not, which implies that most people don't follow A/B exclusively, which sounds like common sense).
Of course, in theory, A/B as the name implies is about testing two different approaches, so again, in theory, any stuff completely different can be seen as an experiment for A/B. The problem is that humans have been trying different ways to do things for a loooooooooong time, and the whole point of trying to teach and learn A/B is to test things, measure, and walk towards your goal.
So yes, you can "A/B test" two completely different things, but it will be so. much. harder to actually know what it worked. For example, you're A/B testing a bicycle and a Ferrari Testarossa. D'uh... obviously they'll work differently, but what caused? Price, color, speed, brand?
That's why when A/B testing is mentioned it's about small, isolable, repeatable, incremental and evolutionary approach. Because it's a technique with this purpose in mind.
But the best I've seen was "I bet you can't hit me with a quarter".
Focus was changed from "This is about me" to "This is about you." No need for A/B testing - this will almost always result in significant improvement.
But the underlying thesis is still valid. Testing multiple scenarios returns positive changes, and further showing an interesting application that I have never thought about -- helping the homeless with it.
(yes, i understand that in some sense he's helping the homeless guy; no i cannot yet fully articulate why this bothers me).
What works for me is to just chat with them. They don't all beg, but even if they do, one thing they all have in common is the need for ordinary conversation.
The fact that someone posted a story about A/B testing a homeless person's sign sends me two messages. a) They are thinking about homelessness and want to do something about it. b) They are trying to relate it to another problem that many other (non-homeless) people experience. I'm not sure what the author's motivations are, but they definitely got their message across, and they will definitely get people talking about it.
rather, it seems like it's connected to relative levels of power / wealth. i am wary of doing anything that "uses" someone when there's a large difference (in power / wealth) because it would be so easy to coerce them.
one way in which i can draw a difference: i really don't think this involved "ordinary conversation" (and if i am wrong, and it was - obviously there is no way for me to know for certain - then that would make me much more comfortable; it's also true that i am a poor communicator and i may be projecting my lack of ability...).
I got the same feeling. It's because he doesn't treat the guy like a person; he never calls the homeless man by his name, Keith, but keeps referring to him as a bum.
in general, saying "it's not [some extreme caricature of a position]" isn't much use in arguing against anything, and doesn't really help anyone much...
Giving this man more money, but without coupling that with professional assistance, may do him a lot more harm than good.
There's a reason why real academic studies have to pass through an ethics review, and I take issue with how cavalier this guy is about his study.
If the author of the blog has experience working with the homeless, then ignore all I've said above. But otherwise, he's operating way outside his field of expertise involving people and situations he doesn't understand, and complexities he does not seem to be aware of. This is dangerous.
Is it really so preposterous to assume that this guy's medical expenses caused his financial difficulties instead of assuming that he wasted everything he had on drugs? Sounds believable to me.
It'd be quite difficult to answer this question in a scientific manner, but I bet that large investments kill more businesses than they help (vs. bootstrapping)
And you can't seem to make your point without name-calling. Kudos.
This is where even a cursory familiarity with the issues surrounding homelessness in your area would be greatly helpful. A huge portion of the homeless population has severe substance abuse problems and/or mental health issues. I highly doubt a bum is going to be spending his panhandling dollars on hookers, but alcohol and blow is more than likely. Given what I know of the proportions through volunteer work, it is entirely unsafe to by default assume a homeless person does not have significant substance abuse problems. Sadly, this is one part of the homeless stereotype that is rooted in reality.
This isn't to pass judgment, but rather to point out that if you really want to help people get off the streets for good, and bootstrap themselves out of poverty, you need to do a lot more than just give money. For a great many homeless people the cause of their homelessness is a lot more than just being down on their luck, it has to do with long-standing problems - substance abuse, toxic environment, and other negative influences. Giving money without resolving these other external issues will just exacerbate the problem.
And note that we're not talking about poor people, we're talking about the homeless. These are people who can't show up to an interview showered with a clean t-shirt - they are not at all like the "merely" poor. Do these guys make enough from panhandling to afford even a shitty room somewhere? Odds are, no, and that's pretty much a bare minimum requirement if your goal is to get off the streets, clean up, and get employed. I contribute to organizations that do this for this very reason - my money is doing much more good (heck, it's at least not doing bad) in the hands of professionals who are able to offer an environment where people have access to the help they need, and the stability to make lasting changes in their life.
And FYI, I'm contributing significant amounts of my money and time to this particular issue, a lot more than change most people give to the homeless in any given year. What are you doing about it?
Note: Helping those who wish to not be homeless I support; but I don't think that simply giving them money will achieve that.
Also, since he apparently has the data, I'm dieing to know how much one of these guys does make per day.
rather contrasts with
"The next big difference is that we changed colors and went from cardboard to white to spark the interest of people walking by instead of automatically having negative associations that they have with cardboard and homeless people."
The second can (and in the context of the article maybe should) be tested. The small change is to make exactly the same text on the different material.
That said, I am now researching a way to add hand sanitizer to my website.
- always be testing, even (especially?) offline
- repackaging your offer can = drastic result
- incentives work
Agree that lack of full data was disappointing.
A) Homeless man begging
B) Useful service with a variable price
On another note: Hilarious idea to try and A/B test on a sign of a homeless man ;-).