Sure you can live in Harlem for cheap, but you're a bit removed and it doesn't exactly feel safe. These days it's gotten better, but I still wouldn't recommend a woman walk alone there at night. Living in what feels like a giant housing project isn't exactly what I consider the desirable "Manhattan" experience.
Mind linking me some of those good listings? I spent the last 3 years in the East Village, most apartments there in that price range are subpar by modern day standards. They're going to be obviously converted apartments with absurdly small bedrooms, no living room area, odd layouts, broker's fees (10-15% of the first year's rent), 5th floor walkups, Avenue C, etc. I'm not saying that deals don't exist, but they are very hard to come by.
Here's a StreetEasy search for 2BRs under $3,000 without broker's fees - I'm not tempted to snag any of these: http://streeteasy.com/for-rent/nyc/price:-3000%7Cbeds%3E=2%7...
I'm not too familiar with deep Brooklyn, but I know you can find alright studios there for $1,200/person. I'm just not sure if the tradeoff of increased commute and reduced safety due to living amongst poverty is worth it. But I'd love to be shown otherwise.