Hacker News Reader

Comment by snird | Hacker News Reader

It's probably not what you meant, but I recently created a weekend project bot that scans facebook for apartments rental ads. Thanks to this bot, I landed an opportunity I wouldn't have catch otherwise, saving me 6,000$ a year. Assuming I will hold my new apartment for 3 years, this is a net savings of 18,000$. I haven't sold anything, but yet I created something that "generated money" for me.

My code is open source by the way, and I wrote about it here: https://snir.dev/blog/apartments-bot/

View on HN

Interesting. I've never heard of apartment posts in facebook groups but I'm in the bay area.

Do other people here in the US have this experience as well? I was thinking it'd be Facebook marketplace, craigslist, and apartment sites (like apartments.com)

I can say that people in Israel prefer facebook groups for many reasons:

1. It allows for "team search". My friends knew I'm looking, so they tagged me in relevant posts they saw. Can't do it anywhere else (including marketplace by facebook themselves)

2. It allows for group communication with the publisher of the post. One asks - everybody sees.

3. It gives a sense of power to those who seek to rent. Many posts with high prices have comments like "You are crazy", "You are a pig", "whoever pays this much is as sucker". It almost warms the heart, we fight the greedy landlords. It almost make it fun, like a witch hunt.

As a landlord though, what’s the reason to use Facebook then? Is there no alternative like craigslist in Israel?

I’ve only seen posts of apartments for rent in Facebook groups that are aimed at college students. (In the US)

Otherwise, Craigslist is basically king.

Young people use facebook to find apartments.

So if your apartments target young couples or room mates, facebook is your place. For families we have craiglist-like solutions.

A Facebook group is the primary source for advertising rental properties in Juneau, AK. Just a normal group, not the marketplace.

Ditto Helsinki, Finland:

https://www.facebook.com/groups/235368102105

I listed my rental flat a bunch of places before using the group, since I started using it I'd not look elsewhere.

FB buying/selling groups (particularly for housing) are quite common I'd say. In my experience, you will generally find them categorized as part of a college or university (e.g. "San Jose State University (SJSU) Housing"), but a quick search for "Bay Area housing" also results in numerous such groups.

typical at college campuses in the us

Thanks for sharing this, I tried to write a bot to crawl a few music related Facebook groups a while ago and it was really difficult. I'm going to give it another shot following a similar method to this. Cheers!

This is great, congrats on the savings. Do you have any opinions on the ethics of web scraping? I have some projects I want to pursue that involve web scraping but I'm worried about

1) any illegal technicalities 2) is it a dick move to the website owner

I was a CTO of a company that did scraping to gather data and sell the analysis. We did this for sellers on Amazon.

In short about legality - it depends on the country. But most will define user content as owned by the user, not the platform, therefore legal to scrape and use, as the user intents for the data to be public.

But sites will fight you on this. Amazon specifically give up a really good fight.

Ofcourse, this talks about user content in an "open" platform. If the content is owned by a business that has no intentions for it to be open for everyone in every medium, its probably illegal.

In regard to being a dick move, it depends. If someone posted an ad in facebook, he intended for it to be seen. He doesn't care how. I just tunnel it to myself in easier way for me to consume.

I think there was also case law about this recently[1] regarding LinkedIn, where if the content isn't behind a login wall, it can't legally be considered anything but public. The UGC aspect is an interesting angle I haven't thought of, but that's a fantastic point too. Would I be correct in saying that the same mechanism companies set up to evade responsibility for illegal UGC also sets it up to be legal to scrape? And if the user owns the material, what happens if you've already made money selling that data and they ask for their cut of the proceeds?

Not OP, but I find that the ethical questions tend to not be about web scraping itself and more what you are using it to accomplish. If you want to scrape and rehost some data then your ethical violation is in the copying and rehosting, not that you automated the downloading of data. It would be the same if you copied all the data by hand and then rehosted it.

Then, of course, the way you go about it is important, too. Sapping the resources and bandwidth of a small site by sending it thousands of requests a minute is very different from automatically checking a public index once an hour.

1) No idea... probably location-specific too.

2) I ask the owner first (perhaps they even want to send me the data directly). If they don't respond, I go ahead and scrape, spreading the requests over a long time so as not hit their website too hard.

> is it a dick move to the website owner

FB gets more user engagement out of this, which is their goal. No harm, no foul.

Reply on news.ycombinator.com