I need to find an apartment
mattrighetti.com
mattrighetti.com
Conclusion: it hasn't changed much at all.
It's still necessary to search on a myriad of sites. Most of the time, by the time you inquire, a place is gone. Or the ad was misleading and that "$2000 2BR" is actually "$2000 for the cheapest Studio we have, but we also have 2BR for $3.5K".
Finding places that match my criteria still requires careful inspection of the description. Checking a filter box for "Parking" excludes places that only mention parking in the description.
With all of the modernization and streamlining that has happened in other parts of our lives, apartment hunting remains unstructured, messy, frustrating, stressful.
Aggregators like Padmapper helped for awhile, but these days, it seems like most listings are posted by scammy middlemen just trying to make a commission on the listing, and now the site actively makes the process even harder.
Is hunting for an apartment doomed to be this messy forever? It seems like an area that is ripe for disruption or a better option, but what would it take to actually solve this?
---
With all of that said, this post is exactly what I needed right now. I've been toying with a very similar idea to improve the efficiency of my search process and this is good inspiration.
From the agent's perspective, it's good to drive people to their company's website rather than one of the big aggregate ones. Having more people visit it means they're more likely to do business with that company specifically. Getting people to visit your company's site is pretty straightforward: they put up ads there earlier, often so much earlier that by the time you see an ad on an aggregate site, it's already fully booked for viewings.
And that's how we end up with dozens of different company sites, each crappy in their own way, and in the middle of it all, people trying to find a home pulling their hair out, refreshing a bunch of different websites every few hours.
https://www.statista.com/statistics/257344/top-lobbying-spen...
Think it’s all related apartments and houses not really being fungible.
ESIT: Also, more directly to GP’s experience, when I was in the market in NL I found the biggest aggregator to be very useful, which is apparently different from their experience: funda.nl
This is how capitalism works when there is no government helping monopolies to form in the name of security / justice / anti-money laundering / insert-politician-buzzword-of-the-day.
The natural end condition of an unregulated market is a monopoly. That is why we need regulation.
So, something is operating to provide implicit regulation. In this case it seems like chaos serves that role. Monopoly relies on a self-ordering property common to most market conditions.
I find it much preferable to North American sites like https://realtor.ca that think that number of baths is more important than floorspace (which to my utter amazement is often not even available).
Even houses can be difficult in many cases.
Selling though is a completely different story it's almost impossible without an agent. And they do provide a valuable service; like photo session, publishing on funda and others, taking care of house-viewing etc.,
There are sometimes rare jewels published directly by individuals on immoweb.be but the bulk of the offers is trash.
This is something I noticed last time I was apartment hunting in Montréal. You'd see a sign or listing, call, and the number would be out of service. As if landlords were using virtual numbers/call forwarding services and just shutting it off once the unit was rented.
I don't call phone numbers on listing sites anymore, even if there's a face and an agency next to it. Better off looking up the agency, going to their website an calling that number (which in all three cases of the above, were different than the one on Zillow).
---
* by "interesting set of experiences" I mean calling the number on a listing took me to an automated menu and a robotic voice where I had to enter the street number, and then first five letters of a street name so it could "look up" the property I was inquiring about, and connect me to the right seller.
Hanging up on that nonsense immediately and calling the leasing agency's number via their website got me a human being in two rings, and the leasing agent for the unit I was interested in after about 45 seconds of hold music.
This is strange, wasn't my experience at all. Just about every house I viewed through funda was advertised by agents. And when I put up my house for sale the first thing the agent did was to publish on funda. All the agents care about is selling a house they get a hefty sum (0.3% if I remember correctly) and are totally fine advertising on funda.
As a house seeker I dealt with a bunch of agents and the experience wasn't all that bad to be honest. Mind you all this was in early 2019; from what I hear late 2020 onwards the market has gone bananas in the Netherlands and Amsterdam in particular.
- paid searches (even at other agents)
- unpaid searches at their own agency
- social media
- agency website
- funda
each step would be on the next day, so if you're only watching funda, you're already four days behind
The market for rentals is never efficient and will never be! People who post to Zillow with a perfect description know they can extract more money for it.
Today, there would be a lot more information available. However, as you suggest, I assume all that available information probably increases the efficiency of connecting a property with potential buyers/renters so you're less likely to luck into something that others just haven't found yet. (The funny thing is that the house I ended up buying turned out be being sold by someone I knew quite well at work.)
Various things cost me a fair bit of money, effort, and angst that I'm glad I didn't know about at the time. But, at the end of the day, I got a ridiculously good deal--especially by today's standards--for a semi-rural place only about an hour out of Boston.
There were a few outright lying apartment management companies, but I managed to look at some nice places being rented out directly by their owners and settled on a good one
Tried the same thing with NYC and things went nowhere fast.
Everyone that attempts to address the housing market just winds up realizing how lucrative it is to not fix it
Once they get a little data and hire some agents its just a reversion to the mean
Would a pad mapper intentionally make some listings bad in order to reduce effective supply?
"Hooray, sounds great!"
"Ok, which element of that list would you like?"
I hesitate to say it, but there is already a solution to this, and it has been the one that New York has had for years: you hire an agent. A good agent will sort through all of that for you, and streamline the process to the point where you can go from search to signing in under a week (if not quicker). Of course, there's the problem of bad agents, but that's true of anything. As you note, online services are often bad, as well.
In other words, I don't see this as a problem without a solution; I see it as a problem without a software automated solution, which is true for a lot of problems in life, particularly ones that involve huge numbers of messy, opinionated people with different motivations and interests.
Just like when buying a house, an apartment broker works on behalf of the landlord or the tenant. They charge a fee which is a percentage of a year’s rent. It is negotiable.
There are agencies that make most of their business in this area, and generally have better rate structures (typically a month or two of rent, flat rate). Others want to charge you on a percentage basis, which you can negotiate, and obviously changes the incentive structure. All of these are "expensive" (relative to free), but worth it depending on how you value your time.
Other people are (rightly) noting that there is risk of getting a scammy real estate agent, but that's true of any service. You have to look at reviews and trust your gut. Good real estate agents are motivated to do the right thing for you, because they see it as a marketing channel into larger sales in the future. Bad real estate agents are in it only for the immediate cash. These people behave in such obviously different ways that they're pretty easy to discriminate, in practice.
The only way they might be working for you is if you are paying them an hourly rate or per showing or something like that.
If you're saying that someone doesn't "work for you" unless you pay them a flat or hourly rate, then I disagree completely. All incentive structures have problems. Someone who works on contingency is incentivized to get you to sign. If they take a percentage cut, of course, they want you to sign the most expensive thing they can. Someone who works on a flat-rate basis is incentivized to do as little custom work as possible. Someone who works on an hourly basis is incentivized to drag out the proceedings. There is no perfect system.
Most of these agreements are structured where the fee is paid by you, after signing a lease, as a function of the monthly/annual rental price. The agent works for you, you just don't pay up front or a fixed cost or on an hourly basis. Does this change the incentive structure? Yes. Does it mean that they don't provide you a benefit? No. You just have to be aware of the incentives.
You know how we know this is a solvable problem? Listings for places to buy don’t have nearly as many problems.
> You know how we know this is a solvable problem? Listings for places to buy don’t have nearly as many problems.
Just as OP said, there are tons of sites out there that claim to "solve" this, but don't. So no, I don't know that this is a computer-solvable problem.
It seems it's in their job description to try to convince me to pay more for less. They listen carefully to my requirements and then take me to places that don't match at all.
I guess they figure since I've taken my entire afternoon with them, and since they've been friendly, I might give up on what I'm looking for and rent whatever it is they decided to shill that day.
Of all the countries I've rented in and had occasion to interact with agents, ones in Turkey behaved the most like cartoon caricatures: they use every sitcom used car salesman trick in the book. It would be funny to watch, except that they're perpetually wasting their time and mine.
In NYC specifically, there are agencies that make most of their business from renter-side representation. You can find the good ones easily on the usual review sites.
As someone else in this thread noted, the landlord often hires an agent because they don't want to deal with the tidal wave of crap that comes from dealing with applicants. That person gets paid, obviously, and the landlord isn't the one paying them. That's the rate you're talking about here. These people are not incentivized to provide good service to the renter, and typically don't. They suck.
Higher-end buildings often have in-house agents (again: you're paying for this, whether you realize it or not), and these are called "no fee" buildings. But an increasing number of places don't have either, and just provide access to professional renter's agents instead of listing publicly. Or they do list publicly, and barely respond to the tidal wave of yahoos, knowing that motivated renters hire an agent. YMMV.
She was pretty much useless. Most of the places she sent us didn't match our criteria very well, and quite a few were above where we wanted to spend. As each day went by and we nixxed place after place she clearly de-prioritized us in her work (I'm assuming she was working with multiple other people who were looking for places) and became less responsive.
At the end of the day I had to do my own hunt and fly up a second time to look at places over a weekend to find something.
And that's a paid agent from a large moving corp. I do wonder if a (more expensive) boutique agent would have been more effective but honestly having worked in real estate I consider them all at the same level as used car salesmen anyway.
In Melbourne Australia, what I have seen is that rental agents are different and the real estate agents are different although they work for the same company.
As an aside I also tried to automate my apartment hunt. The main thing that matters in NYC is time to respond. Unfortunately Zillow, StreetEasy, etc are not very easy to automate on the messaging side due to bot countermeasures. It was an incredibly time consuming, manual process. Happily found a great place though.
If multiple candidates compete over the same apt, no searching method in the world will resolve the issue. Either you have high supply of apts, or a super intelligent city software that matches people to available apartments based on their socioeconomic and professional profiles.
aka it is decentralized
It was hard. There are so many scams out there. If you are not hand curating listings, you have to rely on somewhat novel approaches to filter out all the bad data. For example, any mention of 'Jesus' or 'God' automatically blocked the listing. Sure there might have been legitamate listings, but anytime I would go in and check 99% of the time it was a scam. You also have scams of people listing units they don't own or have any relation too. They ask for a security deposit up front and just pocket it, leaving the new tenant to have an awkward conversation with the real resident.
Data is often unformatted too. Scraping out bedrooms and bathrooms can even be a challenge on websites like craigslist where the listing is just one big paragraph. (Not anymore, craigslist has come a long way, but that's how it was 10 years ago). Often times you had to just search for the closest number around the word "bed" or "bath". Don't even think about getting features like "driveway" or "laundry" out of them.
In the end, we ended up utilizing some pretty intense ETL pipelines to collate historical data, census information, property assessment data, and other things to try to get a more accurate picture on our listings.
But that didn't win out. What won out are the sites like Apartments.com or Zillow, where legitimate property owners can post their listings in a formatted searchable way. We could scrape them and post the same listings on our site, but at that point we were just pushing our customers to another platform that honestly worked better than our own.
We couldn't have the most up to date data, that was determined by how fast we could go back to scrape a listing. And often times we were knee deep in a battle to avoid being blocked by these companies. Often times, after we had exhausted our proxies, the only thing left to use was Tor.
[...]
> We couldn't have the most up to date data, that was determined by how fast we could go back to scrape a listing. And often times we were knee deep in a battle to avoid being blocked by these companies. Often times, after we had exhausted our proxies, the only thing left to use was Tor.
The flip side is if you are looking at listing from small property management companies, they are probably low on resources. The websites aren't very well optimized to serve thousands of concurrent users. Every request to search inventory hits SQL Server which is probably one small box with no automatic failover. I don't like it but unless we somehow help these people better optimize their website (how?),they will continue using heavy handed tactics like blocking scrapers.
Source: got to the "send me a viewing fee in itunes gift cards" stage of one of these scams, once. Googled the initial email and it's a common scan template.
The places you get are amazing. Clear views of the SF harbor, tucked away in PacHeights, full parking, cheaper rent etc. After doing this for 15 years I knew all the managers of those 10 buildings by name. My friends that "wing it" get sooo screwed. They go through listings, trying to piece if it is a nice place, always getting their initial deposit stolen, application fees swiped, going to open apartments 20 people deep, dealing with large corporate landlords, on and on. Make friends!
In the ancient "before times" there were also actual rental agencies where a human would look for an appropriate apartment for you. I didn't know about them, but a local NorCal person turned me on and it was the only way to find things. They followed all of the different local listings daily and knew the area fairly well (e.g. how parking, transport, different commutes would be). It was $25/mo and ~$100/yr for them to go through the listings call the renter, if necessary. They usually also collected $100 fee, if they hooked you up (though I'm sure you could work around it) with one of their (usually several times a week email of 5-10) available recommendations and usually a $50+ fee for running a "credit check". The reason it seemed to work was that the renters trusted the rental agency to only send them real referrals, and the agency knew all about scammers and fake listings, and daily follow-up. It only worked due to the relatively small area and and high volume. Each agent was looking for ~50-100 people during a month and making several $k/mo themselves, filling them typically in a few months.
Like travel agents used to be an actual thing... but now you either do it yourself on-line, or you're going on some giant family cruise thing.
Dwellsy.com has the largest inventory of any rental property site out there, is aiming for zero fraud (and has quite a few features implemented to prevent against it,) parses descriptions to more accurately represent the actual amenities (it's decent, but not perfect) and doesn't clump units in apartment buildings into one ranged price listing, as we prefer to have the units represented as individual listings.
Hopefully it'll help, but feel free to reach out directly if you run into issues.
call them with the bdrm count you're looking for, and if they know exactly what building you're talking about and units they have available, sometimes they'll give you the code to walk right in and check the place out.
you can use this convo to push the rate down a bit or ask for a rent concession, and since you're likely talking to the landlord/agency it'll be no-fee.
that's how i've found my last 3 rentals :) gl!
I was going to say the same:
> walk around the neighborhood you're thinking of renting in...call them...
It works far better in Belltown (Seattle) than the free "apartment shopper weekly" printed ones (which have glossy pictures and outrageous prices).
Your entire post is spot-on. Thank You.
I think there was a fee in the $50 to $75 range, but it included the credit check and access to their listings.
"With more than 1200 prospective tenants, it was really difficult for the landlord to make a decision.
Unfortunately, we have to inform you that the landlord has decided on other prospective tenants.
Thank you for your understanding! "
If they'd max out, they'd still get multiple applications, but maybe 10-30 of those, half of them from people with bad credit rating.
Still bad, but not as extreme as it sounds like.
I know it is not apples to apples comparison but it’s the first thing that came to my mind.
By two orders of magnitude.
Ivies range from about 1:20 to 1:10 in non-covid times.
Probably a lot less self-selecting out than you think.
I guess I would have expected a few top Ivies to have significantly lower acceptance rates on the grounds that one could imagine a lot of average-ish students putting one of those schools down just in case they hit the lottery.
But actually, the top Ivies, MIT, West Point, top music schools, and a couple of schools I've never even heard of (!) are all in the 5 to 10% admissions range.
There are many, many of these applicants who have roughly a 0% chance of admission.
I am not saying that elite schools are easy to get in (they are not), but someone who is a strong all-around applicant will most likely get into at least one elite school.
Most applicants are challenged by the fact that they only have one or two strong points (like grades and scores), and those “strong points” often aren’t even that strong. Athletics, community involvement, leadership, above-and-beyond academics, etc. go a long way.
[In 1917] the government spearheaded great changes in people’s lives, starting with housing. The Soviets nationalized private property and started the so-called “уплотнение” (housing compaction), giving rooms to working-class families in large central apartments that used to belong to [previous owners]. Note that renters were “compacted” too. This project officially laid the groundwork for the often bizarre kommunalka housing - a resilient symbol of the Soviet past. https://www.rbth.com/lifestyle/331621-lenin-putin-soviet-rus...
So you are having problems looking for a 2bd apartment for a family of 3? What about one room in an apartment with a bath shared by 4 other families?
FTFY.
There is plenty of basic housing. It just happens not to be where people are looking most of the time.
Are you kidding ?
It's the best piece of advice you'll read all year.
Go and do likewise!
That worked in the 1990s. Not sure it's viable today.
You might still be able to find something decent for around 100k euro, I'm not sure.
That's the same thing as 5% yield right? (Just that that's the way around I usually hear it, might be a regional thing.)
That sounds like perhaps the high end of normal range, but not extraordinary?
No reasonable investor finds it attractive to build new rentals in German cities. Result is a shortage. And because of the shortage, rents skyrocket. And because of that, you have "Mietpreisbremse". And because of that, there is even less rentals and even more shortage.
If only anybody would understand that you can't fight the market like that.
[1] https://www.npr.org/2021/12/03/1061333538/berlin-voted-for-t...
Berlin citizens. It's a public referendum.
> seizing landlords' properties
with 3000 or more units.
> defending against an illegal law
It's not a law and it will very likely never be one as it's pretty much against the constitution.
The problem with property taxes of course are the people paying them are also the people living there. So good luck running on that platform heh.
First of all: the problem at the core is that Berlin is pretty much built out, there is not much space any more to expand to. Yes, skyscrapers could be a solution, but everyone seriously thinking about this gets invited to look at Frankfurt am Main and then forget about the idea because that city is simply hideous. Skyscrapers are very unwelcome in Germany because of that example, not to mention they need some seriously solid ground to make a proper foundation... and Berlin doesn't have that.
Second: that law was enacted because people were fed up with rents going up all the time - including absolutely ridiculous practices like "kalte Entmietung" (aka having construction crews in allll the time to drive out tenants) or charging ridiculous prices and raising rents while the property devolves ever more because the profits from the raised rents are clearly being siphoned off instead of being used to keep up maintenance.
Third: People were especially pissed off about the large landlords (Vonovia etc.) which bought their stock at fire-sale prices (literally - the 90s CDU government under Diepgen had left a veritable banking scandal and the Senate was forced to sell everything it had in the early 2000s for cheap) and now raised rent like there was no tomorrow. It was naked and utter greed that turned the public opinion so bad.
> Once a tenant lives in your place, it's really hard to get rid of them. Even if they act like total scumbags, trash the place, disturb the neighbors, pay their rent late.
Paying rent persistently late is grounds for termination, it's easy to grab an eviction order at a court. Disturbing neighbors... well let's keep it at "having illegal hotels aka AirBnBs as neighbors is way worse".
> No reasonable investor finds it attractive to build new rentals in German cities. Result is a shortage.
People build like mad wherever they still can and that is the problem. Land values in urban areas exploded - Munich for example had land values double in four years [1] - and the result is that because the land is so expensive, the resulting rental price are going to be exorbitantly expensive as well.
In the end the solution is not the market, as the market will over time only serve to evict the weakest of society - and as a result, as we have seen in Berlin and with the Yellow Vests in France, this has the potential to erode trust in democracy itself.
The solution is to revitalize the rural areas. Public transit and high-speed Internet access are the key points where the suburban and rural areas across the West have the worst issues - fix that and people will actively move towards these areas instead of all being forced to compete for urban quarters!
[1] https://www.sueddeutsche.de/muenchen/muenchen-grundstueckspr...
But yes I agree, make living outside cities worthwhile and attractive.
Please don't. That's not how a civilized debate works.
> First of all: the problem at the core is that Berlin is pretty much built out
Sorry, when comparing to other cities world-wide, that's hard to buy. Places like San Francisco, ok, maybe. Ocean in 3 of 4 directions. Vancouver, ok, maybe. Mountains on 2 sides, ocean on one side, border to US on one side. But Berlin? It's all flat, fields and tree plantations ("forest") all around. Even just adding a floor or two here and there would be fine. There are many areas that could get quite some more density with 5-6 floor houses. But then NIMBY hits. Easier to be angry at the "rich landlords".
> Second: that law was enacted because people were fed up with rents going up all the time
I'm trying to tell you that that's not solving the problem. High rents are just a symptom. The root cause is a mismatch of supply and demand. It's as if you are trying to legislate away gravity. That won't work in the long run. Either you need to increase supply or decrease demand. I can see that you argue that rural areas should be strengthened, but I fail to see how that would suffice. People want to live in Berlin.
> Third: People were especially pissed off about the large landlords (Vonovia etc.) which bought their stock at fire-sale prices
The people should be pissed off at the politicians that sold them the stock. It was rational to buy it. Selling out was not. And that wasn't just CDU. Even the SPD+Linke senate did similar things later on. Yet, everybody gets reelected all the time. (Not defending CDU here, they are just as guilty.)
Tempelhofer Feld is needed as greenfield in the central city for climate reasons.
Ideally it's a compromise, everything gives a little. A few more areas for housing, a little higher buildings, a bit higher prices, a bit more attractive rural areas.
Manhatten is one of the densest populated urban centers in the world. Nearly no other green areas, even street trees are a rare sight. Everything is concrete. Berlin on the other hand is very green. Parks, trees on the street, Tiergarten, Grunewald, and yes, Tempelhofer Feld. You could build housing for tens of thousands of people while still keeping the area green with lots of trees and parks for people to relax in.
Just keeping it a huge empty space and at the same time complain about high rents is ridiculous. This is not solved by complaining about money hungry investors. It's the supply-vs-demand ratio that's the problem. Not greed. (It may play a role, but would not have much of a chance without the underlying issue.) Of course you can keep it "for climate reasons". But that comes with a price and you need to live with those consequences. "Mietpreisbremse" is not going to solve any of those.
The idea that the Tempelhofer Feld would have become anything but a mall with adjacent luxury apartments is bonkers.
The German constitutional court had just forced Berlin to sell of most of its housing infrastructure and the mayor and his confidants clearly were either unwilling or incapable of enforcing laws that would have ensured some form of public control and affordable housing options.
This was the time when people were offering tours putting all these vacant buildings on display that were left there waiting for the rent moratorium to end. When corporations freely redesignated large and central areas to the next biggest mall and luxury apartment project.
Edit: wording
Couldn't be happier, but I wouldn't recommend it to someone who is single.
There's culture and museums and lots of different food in cities, but all I really need is a small smattering of decent establishments in exactly the vein you just mentioned. And of course a good environment for working from home.
I think there is something to be said for New England towns where the design of the town center predates the automobile and is therefore pleasantly dense and walkable. They just don't seem to make towns like that any more.
If you can be 100% remote and don't need the transit options to the big city then just a little further out you can find charming towns for even less.
We survived in little pedestrian-only villages. That thing that now is considered luxury or even a tourist destination.
Culture is also concentrated.
Because in most of Europe (Austria for me) it didn't flourish and not every job can be done remotely as not everything IT related is web development and in many companies management did not give ownership and resources to maintain remote work IT infrastructure to anyone, so everything is chaotic and crumbling to bits now that things are opening up and most people are back in the offices.
Few companies here are keeping 100% remote once the vaccine mandate came through and things started opening up this spring, and now the new standard is various hybrid work variants where you get 1-3 days per week remote and the rest in the office, or one week remote, one in the office, without any logic or arguments behind these choices, just "because management said so" and "this is how we do it here".
>Why are people still crowding the cities?
Because that's where the amenities and the networking opportunities still are. Universities, clubs, bars, music halls, swimming pools, great restaurants, bouldering halls and gyms, music and jazz bars and basically anything to do with art.
Oh, and most importantly, dating. It's way more difficult to meet someone when you live alone in the sticks VS being surrounded by people your own age sharing similar interests.
Because the top employers by number of people are all about schlepping shit from A (where population isn't) to B (where population is), and that can't be done remotely. Then you have the manufacturers, and that can't be done remotely either.
Most jobs cannot be done solely using a computer keyboard.
Edit: There were 3.5 million residents in 1993. It is projected to reach 3.6 million residents in 2029 [1]
It's got to be something to do with financial markets and foreign investment
1 - https://www.macrotrends.net/cities/204296/berlin/population
And btw: new built appartments have almost free price setting.
[1] https://www.npr.org/2021/12/03/1061333538/berlin-voted-for-t...
It reflects a political environment that makes renting risky. And illegal laws can still cause hardship while they're being stayed and struck down. The corporate landlords can afford to fight. Single-home landlords cannot. This time the referendum constrained itself to corporate landlords; next time it may not.
And btw: it's stated in the constitution, that ownership introduces also responsibility to the society.
Eminent domain exists everywhere. Berlin’s politics are more extreme than the norm. Most homeowners would pay a low single-digit percent of their home value to insure against expropriation in that environment. Taking a property off the rental market is analogous; sacrificing a cash flow to safeguard the investment.
We can go back and forth on theory all day. I’m m reflecting what people I know are doing. Being flippant with (or even demonising) owners’ interests circles back into affordability crises worldwide, from Berlin to San Francisco.
People forget that you can always “price in” regulatory changes. Then you just see which path maximizes profit. What may seem ridiculous (leaving a unit empty) actually makes a lot of sense based on the rules.
Doppelte Haushaltsführung is a tax relief for people needing a 2nd place close to work. Also the work of your gf needs to be at least 1 hour away from the main apartment [1], she has to pay at least 10% of the cost of the main apartment to be allowed to deduct it (and maybe more requirements that I'm forgetting right now). She also has to pay 15% additional Zweitwohnungssteuer on the rent of the 2nd apartment.
[1] https://www.haufe.de/personal/entgelt/verwaltungsregeln-zur-...
I don’t see rent control as very useful, but I doubt it is very relevant to the current housing shortages throughout the cities that have it either (given that the same problems apply to cites that don’t have it).
[1] https://gitlab.com/magnolia1234/bypass-paywalls-chrome-clean
Population can stay the same, but number of single people wanting to live alone means less people per dwelling, which means more demand.
Also, Berlin was an odd place compared to other cities in Europe, when it comes to housing. Looking from the outside it seems like it was due for an upwards correction after 3 decades of pretty low rents since the fall of The Wall.
I don't understand what you mean or how it relates to supply and demand.
Say there's a supply on 100 apartments and a demand from 120 artists. 100 will pay €1k per month, and 20 will have to leave the city. Landlords can't increase to €1050 a month because nobody can afford that.
If demand is higher than supply, rents will increase, but if the demand is entirely from people earning €1000 a month those rents won't increase past €1000 a month.
However you have a demographic on €5000 a month move in, they can afford higher rents. The same 100 apartments exist, and there's a demand from 120 people, but now those people are earning €5k a month. Rents thus shoot up to take all the money that can be spent on rent, because the alternative is moving out of the city and thus not being able to earn the €5k/month.
The people owning the property do well in this situation, and thus don't want supply to increase to the points that there is more supply than demand
A few years back, I was looking for a rental in a German city with a similarly hot market. One place I looked at was not very modern but had quite low rent, so there were 300 applicants and I didn't even try. Another place was put on the market 3 days prior, there were 5 people looking at it with me, 3 of them applied (including me), and I got it. It was more expensive though (not insane, mind you) and very modern, recently renovated.
So, of course there will be crazy places, but if you only shoot for the ones with top demand, then of course you can come with anecdotes that sound insane.
I'm not sure if this is what gp meant, but:
In some cases the current tenant (who is looking to move out) offers you the landlord's contact details if you agree to buy their kitchen for way too much money. This works because for a landlord looking for new tenants is a lot of hassle so they're likely to take you if you contact them.
I did follow a different approach though, and reverse engineered the APIs of the most common rental listing websites here.
I didn't publish the code (or better said, I didn't make it public), but it's similar to yours. Instead of SQLite I used a PostGis database where I stored the apartments with their point to point public transport distance from my office (pre-covid search). I did it in Rust.
Whilst the application is not available to the public (yet), all of the libraries I've created are now available.
I should have both a Rust and a Golang version for most / all of them.
On a side note, I did a similar project (scraper + visualizer / search) for finding a job for my girlfriend by scraping LinkedIn, Xing and a couple of other local job posting websites.
[0]: https://github.com/denysvitali/sbb-api-rs
[1]: https://github.com/denysvitali/homegate-rs
How did you convert an address to long/lat for postgis?
If I would have been confronted with such a situation though, I would have probably used some Geocoding API like Google Maps's geocoding [0] or OSM's Nominatim [1].
[0]: https://developers.google.com/maps/documentation/geocoding/o...
The apartment was also new. I'd say a very good price/value ratio.
I now moved away from Zurich, but that apartment was very convenient. When I organized the viewing, ~100 people were interested and ~30 visited the apartment.
Obviously in the end only one got it (:
In Internet olden times, we dreamed about automated agents that would run searches on your behalf, scour the web to compare prices and hunt down things you were interested in. But user-centric fantasies like this don't make fortunes, so walled gardens is what we ended up with.
The ones that do are either very expensive or will have 20+ people showing up the next day (sometimes the same day) for a visit. It's crazy and while this article is very neat I'm not sure a bot that scrapes listing in batch is the answer.
I decided that their parents must have tipped/bribed the original agent to get at that listing (since none of these ever made it to any search site, not that there were many at that time).
0: https://www.self.inc/blog/cities-biggest-increase-rental-pri...
A week later she phoned us before she listed a new one, and we went to see it 3 hours later and took it.
It goes to show the human aspect still matters, a lot. That one email thanking her was probably worth way way more than eg hours spent on a bot to automate applications.
Many apartments also don’t even make it to an agent because people have friends they promise it to and refer them to the landlord.
Here in Austria, most leeches, sorry, realtors, avoid putting the street in the ad to prevent you from trying to contact the property directly and bypassing their fees, so apartment hunting is even more of a headache.
Still, amazing work from OP.
I guess the variety of concerns will vary from country to country depending on local laws / existing open data though.
The owner doesn't really have a reason to care, I don't think, and may even be incentivized in some situations to reveal information the tenants would rather they not.
Goodness. I should give the First Amendment a hug.
In most places renters are only granted 24 hours notice to have complete strangers come walk through their living spaces while the space is shown off. They have no real right to refuse to allow that.
If people start knocking on their door asking to come look at the place they are free to refuse, or ignore the doorbell or anything else. Of course they have a freedom against being harassed, so if people persist they can get police involved. That seems like a separate issue to me from having their address posted online in a rental listing.
In some EU countries they do. The landlord need to ask permission to have visitors at your place and the tenant has the right to refuse.
Are you sure the tenant isn't just allowed to refuse "a time slot" but also must offer alternatives?
Story time: A couple of years ago, I was looking to buy an apartment and saw one spacious apartment centrally located that was way below market price but had no pictures in the ad. I called the agent and asked when can I visit as soon as possible since there's no pictures. Agent said the old lady currently renting it does not allow visitors inside, that's why there's no pictures. I asked how in the world can you buy an apartment without seeing it? Agent said that's why the owner is selling it below market price. And the charry on top, if you do buy it, you'll be forced by law to keep the current tenant who's paying frozen rent, not adjusted for the past 20 years, until she either decides to levees voluntarily or dies. That's why the owner had to sell below market.
someone being this hostile about it is rare but it sometimes happening is a necessary consequence of these rights.
anyway though in most places our protections are not this extreme. you can refuse a specific visit but have to provide a time when they can come later.
Similarly, in the city where I studied for university, I immediately knew where the house was located just from outside pictures of it in the ad, at least for the districts of the city I was familiar with.
In places like The Netherlands, a seller generally works with one seller's agent only, and there's no incentive to hide full addresses. Funda.nl has a huge amount of detail, from high quality photos, to floor plans.
On the contrary, in Singapore, a seller often has multiple agents. The agents compete with each other and would rather not share any information at all publicly to avoid other agents using their information/photos, or buyer's bypassing them. It leads to propertyguru.com.sg with no addresses, bad photos, all photos watermarked with the agent pictures. As little information as possible, just enough to draw you into an in-person visit.
I tried a few different ways of finding an apartment, but in the end it was one case where the old phrase "it's not what you know, it's who you know" worked in my favor.
During my freshman year I had found and been working a job on campus in one of the department offices (side note: if you're a student, you're best off trying to find a student assistant opening in a department office; at least at SJSU, most paid above minimum wage, were pretty lax and could be used for some study time, and you have so many opportunities to make connections with professionals).
One of my coworkers there had a family and they were living in one of these small ~8 unit apartment buildings downtown. It was a privately-owned building, not one of these newer, fancy conglomerate ones.
She connected me with the management/owner and long story short, I got a decently livable place with rent control for about 1/3 of market rate due to being a lower income student and taking a unit that wasn't recently refurbished (old carpet, old paint, etc.).
In my case, the tool was ultimately useless because I discovered the way SF housing worked for nice apartments was 1) Openings are posted on Friday 2) Saturday, open house collects applications 3) Deal is signed on Sunday and post taken down. Anything my crawler picked up was stuff that was overpriced or in bad condition. For all my sophistication, the dataset was just the bad stuff.
Apartment hunting is a great example of needing to know the underlying behaviors of your dataset before you use it to make decisions.
It was a pain after for bureaucracy but that was after landing an apartment.
In Japan they're often missing other things like lots of pictures but at least the number 1 requirement to list an apartment is a floor plan.
https://suumo.jp/jj/chintai/ichiran/FR301FC001/?ar=030&bs=04...
I wish Craigslist would require one or somehow strongly suggest that the person offering the apartment would get better matches and have less of their own time wasted if they'd add a floor plan.
I also wish they'd add some examples of good and bad pictures. So many people take pictures with their phone and the field of view is so small and lighting so bad the pictures are basically useless. I'm not saying they need pro-equipment but my guess is there are at least some guidelines that would be helpful. How about no portrait mode pics as just one example that would likely lead to more useful images.
We were promised jetpacks!
If anything, with all the ads and SEO games taking place, searching for the relevant information has become _more_ difficult than in the early days of Google.
It is 2022 and we still do not have a universal search system which would allow everyone to quickly and efficiently perform more sophisticated searches. The most skilled and determined people are capable of doing so by creating various single-purpose data collection and data mining tools themselves. But even for them it takes a non-trivial time and effort to produce something usable. There is nothing out there (that I know of) which would be easily applicable to everyday needs.
Many of the public search boxes are backed by elasticsearch or other similar “recommendation” tools which are optimized for speed and point-someone-in-the-right-direction accuracy.
Since we no longer have exact text matching or many of the advanced operators (such as `not`), I’d argue that for sophisticated searches search boxes have actually gotten significantly worse.
However, my approach didn't include these map visuals; that's a great idea and a nice execution!
Something tells me you're joining a big online book store ;-).
Tell me when you're in town for a beer, would love to hear how this unfolds.
Once you are in Luxembourg, feel free to reach out, I’ll buy you a beer to console you for the expensive apartment prices ;)
Always interested in what people are up to (both as an entrepreneur as well as a politican)
1. go to craigslist
2. play with the filter
3. add keywords in the search bar like "rent control" and "-studio"
4. go to the maps page
5. bookmark the page
6. check every day
In the US/France, there's too much fragmentation and real estate agents
In China I would go to the lady selling phone cards in the street and managed to visit a lot of apartments like that.
(since the current title just conveys some general topic of apartment need, with no indication if it's about "my partner died and it's interesting to notice how large a whole house is" or something)
I just wonder if OP ever thought of contacting a broker? I would have done that before I wrote a single line of code.
Scraping all this data is really great, but a human broker is going to know the local area and the other human nuances of apartment searching in the local area.
Plus, brokers already have access to their own software that’s different than what’s on the public Internet, along with human connections. Why reinvent the wheel?
My last apartment came up as an option before it even hit any sort of public property advertisement platform, and that was thanks to our broker working directly with the landlord.
A lot of brokers specialize on ex-pats, too!
To expand on my point about knowing the local area, brokers know things like “what kind of person likes to live here?” Is it a young and trendy area or a quiet family neighborhood?
(Admittedly, that information can be based on biases rather than data.)
It’s something to think about. I know you might be looking to save money, but where you live is going to be a major purchase decision. Plus, spending time on your own apartment hunting platform isn’t technically free, your time is valuable.
In any event, I’m sure with the kind of skills and dedication you have you’ll find a place in no time.
It depends on the city. In NYC, the broker is pretty much always paid for by the person looking for an apartment, and it's quite a bit of money. I haven't looked for an apartment for 10 years, but I think it was 5-10% of the annual rent. (I imagine this gravy train is coming to its end, though. 10% was fine when your employer was paying for relocation, but in the All Remote future, this free money goes away.)
Honestly here in Austin, unless you're walking into an apartment leasing office -- you're going to waste so much on rejected application fees by not utilizing an agent to rent a single family home.
A new law a couple years ago mandated that, if the landlord is not a person (a corp / company / org...) they must be the ones paying the fee.
But I've been "extorted" twice in the last year by companies moving the payment to my person. And I say "extorted" because they defer details or information about the rental until the last minute, when you are days away from having to leave your old place and you have no other option.
My wife is a realtor here in TX and when she is in-between home sales she helps people find rent houses and apartments. The seller pays a pre-agreed amount of 30-40% of the first months rent as a type of compensation. The buyer doesn't pay her anything at all. She is incentivized to find the right property for her client so she can get paid, and she also is better positioned to talk to the listing agent/landlord about you and the current competition among applicants. She ends up being able to save people money on application fees that would never get approved.
Half the houses we looked at fit the profile we wanted but we found ourselves.
Absolutely stellar form of data science!
Keep up the great work!
So much sophistication based on at least partly incomplete, filtered/distorted, and inaccurate data. So much important aspects that cannot be in an ad or intentionally left out and can only be extracted by painstaking efforts. Even the initial filtering out could be unreliable then.
It's all Google Maps with arguments so i should probably open source it, someone might find it useful.
Some people take protecting their data seriously
I'm really curious to know the breakdown of home/building ownership in the world, and how much the person makes in that city. Curious how much property is owned by individuals vs companies.
If you find the seller, you usually get a better deal. If the seller finds you, …
There’s great value in building scrapers specialized to a particular city, country-wide aggregators are not enough.
Also, I would recommend closing each page when you are done with it, using page.close().
> Only thing that concerns me is that it will spawn n parallel requests to the website which could consume a lot of memory since we're talking about spawning a new tab with Puppeteer (Chrome :sigh:) for each link.
Your scraping being slow and using Chrome might be a blessing in disguise though. If you aren't careful you can get detected as a bot and banned from the site.
https://github.com/PuerkitoBio/goquery https://github.com/robertkrimen/otto
Cheap, Area code, Palatial. Pick two.
Nevermind, someone beat us to it.
Obligatory xkcd: https://xkcd.com/974/
And you are in the beginning of your career, can just as well post it to make yourself attractive for future employers. Better solve an actual problem that you had than to post some made-up nonsense-projects on Github just because people think you gotta have "a Github" when applying somewhere.
You submit precious personal info over to a random stranger along with payment to check your credit. Last time I checked, this fee is $25-50 USD. You've now given them enough information to steal your identity. You also may need to do this multiple times if you're rejected. The person receiving this information rarely has identity protection measures in place and you receive no guarantees that they've destroyed this information.
A much better solution would be to develop an escrow service which acts as a middleman between renters and landlords. Aspiring renters could sign up with the escrow and submit their information once. Their one-time fee would cover a fixed number of queries by potential landlords. The queries could include such information as income validation, criminal history, job history, rental history, etc. But these queries could be approved by the renter in advance. This sort of query process allows the renter to control the types of information given out. The queries could be structured to allow pass/fail results only, so that landlords do not receive intimate details but only know whether or not a renter is acceptable based on their criteria.
Once you have a system like that, you can open it up to all sorts of validation queries. Think of it as a general personal information escrow service.
This idea has been bouncing around in my head for over a decade now but I'm more of a low-level systems engineer so I never got around to trying to implement it. Seems like a decent startup idea though.
Grow up, learn to take on boring tasks without playing with toys
But even in Finland, housing is only sometimes free and then on a net basis (rent can be canceled out by a means-tested housing benefit).
During COVID lockdowns I watched lots of international TV drama series on Netflix.
One of those "random things I learnt on Netflix" was that they (sort of) offer this in the UAE if you've lived & worked there for 10+ years.[1]
Its basically means-tested housing. So they offer anything from interest-free 25 year loans to fully subsidised housing.
[1] https://www.tamm.abudhabi/en/aspects-of-life/benefits-for-em...
That being said, there's subsidised, including to 100%, housing in France too. And cities are legally required to build affordable housing for those schemes, so it's not "it exists on paper, but nothing has been built in decades so nobody can get in".