Hacker News Reader

I appreciate you bringing this up, because I think this plays out all the time. I live in a small isolated town, and there wasn't a lot of turnover in houses even before the pandemic. I wrote a short script that scrapes all the listings, and alerts me to any new local listing. It was a fun little project to learn about generating text messages, and to see how effective a cron job on a sleeping MacBook can be. We ended up getting a house through a friend who sold their home without listing it, but those alerts were really helpful, and almost got us one house a little earlier.

The bigger story here is what this means for the divide between tech-literate people and the rest of the population. I don't think the parent commenter was wrong to write their script, and I don't think I was wrong to write mine, and if you wrote something similar I think it was a reasonable move as well. But we have to recognize the advantage this gives us, as people who are able to spot these opportunities and act on them.

I don't have any answers, but I think this story brings up a good discussion. I'm curious to hear other people's thoughts on this.

> The bigger story here is what this means for the divide between tech-literate people and the rest of the population.

Where I live the good deals from private sellers (mostly older people) pretty much never get posted online, thus you would have to socially integrate somehow offline in the region you would like to buy a house :). It looks like I still have to wait for some cultural development to benefit from your proposed divide.

Especially in smaller markets, there aren't very many agents. You can get a lot out of inviting realtors to coffee from time to time. When somebody is thinking about selling, they often talk to realtors early on, so you can learn about them before they hit the MLS.

Regarding tech literacy, I participated in a project in early 2021 for this reason.

COVID vaccination slots were put online and would disappear the second they went up. Vulnerable people, like the elderly, had trouble booking the appointments they needed.

What ended up happening is a few developers got together and put together an alert system that scraped for appointment slots and automated alerts about when they were available publicly on social media. Quite a few people were able to get slots they wouldn't have got by just manually trying to get a slot.

>> Quite a few people were able to get slots they wouldn't have got by just manually trying to get a slot.

And the people who played fair didn't get those slots. Obviously there's nothing illegal about this (or the articles similar solution) but it rubs me the wrong way morally. Similar to people getting appointments because they knew someone working in a vaccine centre. If there is a clear order and booking system (e.g. oldest + most vulnerable to youngest/national booking system) and you skip the line or 'hack' the system to give yourself a better chance at getting a slot I'd find it hard not to think you're a dick. NB: this depends a lot on how things work in your specific country so I'm not judging your specific situation but just generally, 'hacking' systems to give yourself an unfair chance at getting something in high demand feels wrong to me.

I think the answer here is to have a system that can't be "gamed" by more tech-literate people. Have a time period where people can throw in their names for consideration, and a randomly-ordered list is generated. You go down the list until you reach a person who can put down the deposit. A lottery, essentially.

You could probably poke holes in that system I just described, but it seems more fair than one that is prone to someone writing up a script to scoop it up. I don't want to live in a world of script wars where every desirable entity is quickly snatched by one of several competing computer programs.

I’m hard pressed to think of any first-come, first-served system for allocating scarce goods that wouldn’t be better run as a lottery.

The problem is that typically the person selling the item doesn’t care who gets it, so there’s literally zero incentive to spend any effort making a fairer distribution system.

What about an auction? That’s how the housing market works in e.g. Norway.

That's definitely not fair either, as it disproportionally benefits those with more funds. Which only leads to wider inequality.

I didn’t mean to imply that lottery was the best system in all cases, just that it’s pretty much always better than first one wins. Though thinking about it more a lottery probably results in more total entries, and you might end up giving the thing to someone who is closer to ambivalent than the folks willing to stand in line at 5AM.

Auctions are a great way to see that the item ends up in the hands of the person who “values” it most, where value is measured in currency. It does have the problem that in a system of high inequality the people at the bottom have essentially zero chance of winning.

> Have a time period where people can throw in their names for consideration, and a randomly-ordered list is generated. You go down the list until you reach a person who can put down the deposit. A lottery, essentially.

Stocks should be traded this way, too. Having an arms race for fiber latency makes a mockery of the idea that they are for efficient allocation of capital. Queue up all the trades requested in each second/minute, then execute them in random order.

> I think the answer here is to have a system that can't be "gamed" by more tech-literate people

The person selling the house wants to see offers as soon as possible and can accept one at any time--the sooner the better--or hold out for a better one if they're not satisfied. Any system that changes that dynamic to the seller's detriment is not "more fair" to the seller.

we expect that an auto mechanic gets a good deal getting his own car fixed, that real estate brokers have an edge getting their own properties, doctors with medical care, etc.

You spent years becoming tech savvy to save you a little time scanning real estate ads... I'm not sure what problem you're trying to fix?

What are you using to scrape listing data? afaik mls doesn't expose an api.

they don't, but Trulia/zillow/etc are getting it somehow, and there's clearly a database on the backend, so

Yeah and they probably pay out the nose for it. How do us plebes get these data?

Great story though I initially thought you meant you wrote a script that paid off your house :).

Myself as well!

I had a feeling his script somehow made him skip the queue.

When I was looking for a new apartment one of my friends told me about his startup that was basically scraping databases of realestate companies and I signed up with them and to my surprise I was alerted for an apartment in the location I wanted and it was up just a few minutes so I was able to contact the agent before they even started their advertisements.

I got the apartment but sadly the startup was bought and shut down operations soon aferwards

I had a similar experience, though not with a house. My sons school was organizing a couple of sport activities. We could sign him up for the different activities, but it was first come first serve. We were early, so our son ended up doing all the activities. This meant some kids ended up getting nothing.

I found this very unfair. Better would have been to let all the kids list there preferences and then fairly distribute the kids over the activities. However, I could see why the school wouldn't want to bother with this.

I ended up creating a tool (1) for them to do this easily. It's quite ugly, but it's open source (2) and it works. The tool is in dutch because it's a dutch school.

(1) https://eerlijke-indeling.nl/ (2) https://github.com/deliriouspenguin/eerlijke-indeling

A similar story happened to me in the past with appointment slots at an embassy, and - shameless plug, I built a tool[0] to monitor websites on my behalf, and send custom alerts to Telegram, Discord, Slack, or archive data in Airtable, Sheets etc. No coding or devops is required, you can get it done in 2 min [1].

We're in Beta so stability is still a goal we're pursuing, but for many happy paths Monitoro is a huge timesaver.

[0]: https://monitoro.co

[1]: https://www.youtube.com/watch?v=gZkJDk1lzrY

This is pretty cool.

Thank you! :)

I did basically exactly this but for reserving a spot at the visa centre for a residence permit appointment.

Subverting a small aspect of the Hostile Environment[1] took a little bit of the edge of the general bastardry of the whole process. Still a bloody expensive and emotionally draining experience though: if it has to be done again, I won't be putting a loved one though that again and we'll both be leaving.

[1] https://en.wikipedia.org/wiki/Home_Office_hostile_environmen...

>I own the house I currently live in because of a script I wrote.

Somewhat similar story for me. I live in a big European city where young people/students predominantly live in flatshares. One website which pretty much has a monopoly on room ads here also allows room seekers to post an inquiry themselves. So many people do it that after less than half an hour, your ad will already be somewhere at the end of page 2. I wrote a script to automatically update my ad every few minutes to push it back to the top. I ended up getting a lot of responses I probably wouldn't have gotten otherwise.

I think there are a large percentage of programmers at Swedish national soccer games, as they release tickets online at a first come, first served basis. (At least 10 years ago) This is also true for fancy restaurant bookings.

I have similar stories for launch day Nvidia cards, concert tickets, festival tickets, and original artwork from creators on Etsy. Any time I catch myself wasting time refreshing a page for availability, I try to replace myself with a script.

I also own the house I currently live in because of a script I wrote!

I wrote it in 2012 to automatically buy bitcoin once a week.

One of the things that always frustrates me (despite being very much in the "first world problems" category) is when queueing or reservation systems are unclear or chaotic or otherwise unfair, although I generally only encounter this at stores, never for something as significant as a house! The most ubiquitous one is the practice of totally separate parallel queues where you just have to guess which queue will be fastest. Another lovely related one is when you're waiting in your queue for quite some time, then a new queue opens up right next to you and quickly fills up with newcomers who haven't been waiting at all! Come on store owners, just have a single queue with multiple consumers popping off the same queue! Again, I'm not going to pretend this isn't totally a first world problem!

Another variation I have been on both sides of: stores favoring certain channels over others. For example, a local fast-food restaurant might prioritize answering the phone for carry-out orders over in-person carry out orders.

Is a jealousy issue? Without the second line you were going to take as long as you were. Opening a second line helped people further back catch up and in some cases surpass your speed. Someone else having a quicker experience shouldn't have changed your experience.

maybe it's more of a fairness issue. in theory, the proper way should be to have a single line that funnels to any available lane, so any newcomer would still have to join the same line and not get any advantage over someone who has been waiting.

I wouldn’t call it jealousy to want everyone to be served in the order they arrived, since that’s obviously the intention of flat unprioritized queues. You can certainly make arguments that FIFO isn’t the best or right way to process a grocery store checkout queue, but for now I’d rather it be implemented well rather than poorly.

Wouldn't it average out over time anyway? I.e. sometimes you're unlucky and have to wait twice as long as if you'd arrived to the queue a bit later, sometimes you're lucky and have your wait time cut in half. Over many store trips it's halfway between the one-queue and two-queue processing time.

Berlin's international film festival has this horrendous system where they'll release a small batch of tickets online every morning on certain dates, and you just have to sit there, refresh refresh and hope you clicked fast enough.

Serious major events use a lottery system, which is vastly more fair and means you don't need the server infrastructure to deal with this huge flood of requests. I'm not really sure why everything doesn't work like that: let people submit their requests/applications/whatever, and if you truly have zero other criteria, just process them in random order after a cutoff.

Something went wrong here. The price of the house should have been raised until only 1 person wanted to buy it. If you're first in line, the incentive is to sell it at a higher price to the person who's second in line, right?

I have to imagine some sort of regulatory thing was going on here, like they have to provide X% of units at Y% of market rate in order to get some tax concession or something.

I know there's not going to be a lot of sympathy for higher housing prices here, but it does feel like something went wrong in this specific case.

For the people selling, it is often more profitable (or at least less hassle) to sell quickly with minimal fuss and move onto the next one.

It's like how event tickets are routinely scalped. The thing that is going wrong is that the additional bid isn't worth the reputational drama.

Homo economicus would understand that the seller is simply being efficient. Homo sapiens gets all emotional about it.

In a big development I imagine clearing inventory in a straightforward way at prices that'll net you a good profit margin anyway is more appealing than trying to run bidding wars for each unit individually. Depending on the market it's also possible that the gap between "price four people are willing to pay" and "price nobody is willing to pay" isn't very significant.

There's also a goodwill aspect in other cases: Sony kept selling PS5s at MSRP even when scalpers were getting $800+, because better for them that people are pissed at the scalpers than dealing with the bad PR of such a huge price hike over previous generations.

Yeah, for established brands like Sony who expect to have long recurring relationships with customers, it wouldn't make sense to run auctions on products because you're just going to upset people and no one will think you're making mass-market products any more. The same goes for some popular musicians: they could make more at a particular show by auctioning off tickets, but they often go out of their way to have a lottery and prevent people from re-selling tickets at higher prices.

It seems possible that a large housing developer has similar incentives.

Camping reservations for Ontario Parks are a big mess like this— so much so that people do ridiculous hacks like booking the full two weeks that lead into their desired weekend and later paying a fee to cancel just the first part of it. By the time the actual popular weekend would have come available for online booking, it was already taken weeks earlier.

I wrote a script to poll the Parks Canada reservation system for availability for a camping trip I wanted to do, so that I'd get notified the instant someone cancelled and space became available.

Sadly after two weeks they banned my IP for the rest of the year and I was no longer able to make reservations without going through a proxy.

I'm pretty sure they do this on their own now. I set up "notify me" for a site and got tons of emails as sites became available.

A friend and I made a website based around this called campalert.live We've had to stop doing BC and National Parks as the API changed and we haven't had time to update our end. But Alberta still works!

Wow, thank you for explaining this!! I was wondering how weekends that were "not yet available for online booking" were already nearly fully booked when I looked on the booking site. Very irritating that people do this... With that out of the way, I'll go ahead and start doing this myself, if that's the price of admission

It's definitely a prisoner's dilemma type of situation— no one wants it to be like that, but once there's a critical mass of people doing it, then you kind of have to. Once everyone does, you're back to square one, except with a system based on unwritten rules that are confusing and exclude the uninitiated.

And it's not obvious how to fix it in a way that doesn't screw up the system for someone else:

- exorbitant cancellation fees would punish people whose plans change, and a lot of people would just swallow the extra cost anyway.

- insisting that the entire timeframe be in scope at time of booking would make it hard or impossible to book longer vacations at popular spots (which at this point is basically anything within 4h of Toronto).

- making the whole season available at once would create an ultra high stakes feeding frenzy, with the servers probably ending up crashing.

- some kind of bidding or lottery setup would be extra complexity, and would end up rewarding people with the time, money, and schedule flexibility to "play" whatever the system is.

I suppose the Ontario Parks mandate is to just maximize utilization, so from their point of view it doesn't really matter as long as weekends are booked solid and at least some proportion of the canceled midweek days end up being scooped up later by retirees or whoever.

I definitely appreciate a single queue. It doesn’t seem to be the norm though.

If it’s a place you frequent, pay close attention. The checker is usually a stronger indicator of line speed than the number of items in peoples’ baskets.

Whole Foods near me does this awful thing where the lines are in the aisles. You can’t even see how long the line is without walking the entire length of the store.

This was tried in a number of stores in Sweden. It lasted a couple of months after which they reverted to one queue per cashier. Turns out it wasn’t popular, either with the vendor, the customers, or both.

On a personal note I found the practice irritating for not really clear reasons. One might be that it’s a chore to traverse the whole queue cordoning when it happens to be empty. In any case, over time being the one skipping the line when a new register opens averages out. Also, there’s the lucky draw feeling when you get to skip that otherwise gets lost.

A number of stores here in Kitchener (Canada) use the single queue approach, particularly to feed a bank of self checkout stations. But I've also seen it at Old Navy (clothing), Winners (discount clothing), Indigo (bookstore), and others.

I think the main thing that's maybe a bit annoying is just that it's easy for the single line to stall if the people at the front of it aren't paying attention or don't know what to look for. So the store often ends up having to have an extra attendant who just stands there and pokes people for when a till comes available.

id love to get their honest opinion on the 'no hard feelings' part

lol don't feel bad; you took advantage of a broken system and automated your way through it. The first person in line would've done the same if they knew how!

Did the same for my previous apartment. I wrote a script scraping various “self-sale” sites (sales without real estate agents), the script would notify me when new apartments living up to my criteria was posted online.

When I got such a notification I would quickly check photos of the place etc. and then call the seller. The guy selling the place I ended buying, said it felt like I called him up instantly after he posted it online.

I used to watch the craigslist free section with a script, and I got that response a lot.

I was calling within 1 minute of post time sometimes.

Being first pretty much gets you anything you want in that section.

I did the same with the apartment I currently live in. Although it's a rental, places here are taken within minutes of being posted online, and my bot gave me the edge to put in a reservation before anyone else was

Let me guess, Berlin?

Your generosity gave you the best possible outcome in this deal. Impressive.

Did you end up running the scraper longer for your new neighbour?

I did, yes. I set it up with some specific conditions that sent him an alert whenever a house similar to ours became available. In the end, he got the house directly next to ours and very similar in design.

Our garages actually share a wall.

That's really kind of you to run that scraper for him as well. Glad it all worked out~

Very cool! Great story.

Reminds me of "Project Thor" from my university days. Course registrations were online, but in phases, e.g. Seniors for a week, then Juniors for a week, and so down the line.

However, many students would snag a spot in a course only to drop it as they shuffled courses around. There existed a "queue" for the overflow, but it appeared to work as a CRON job or otherwise involved a human in the loop as one would routinely see:

Spanish 102 (Registered: (29/30), Queued:(60))

While in this state, one could simply click on the course and register, completely bypassing this poorly designed queue. So, naturally, I wrote a script to scrape the site on once a minute (hammering it, and so Project Thor) and alert me once a slot had opened up. Eventually provided this service to my friends as well.

I did something similar at UIUC very many years ago. I was called to the office of head of Department of Computer Science and kindly told while he thinks it is cool that I am applying my skills relevant to my field of study, I should stop what I am doing as even simply scraping could be construed malicious if it got the attention of the wrong people in the school.

I've got something similar for appointments at citizens' bureaus in some European cities. The waiting time is usually around a month, which is insane imo.

My script constantly looks for open/canceled spots and reserves them (some need email/phone confirmation, which makes it a bit harder). Pretty standard stuff.

That's how I got my appointment in 3 days instead of a month.

So, I'm thinking of making money from this, but I'm not sure about the legality of it all.

What is a citizens’ bureau?

Not OP, but sounds like the places in Germany where you go to get or extend your passport, register after moving to a new place etc. Notoriously hard to get a timely appointment for things that should not require an in person interaction in the first place.

As konha said, it's the government offices, providing services for registration, driver's licence, taxes, etc. It works the same way in several EU countries.

Sometimes you can just show up (and wait in a queue) but most often you make an appointment beforehand. In many states/cities, the waiting times are weeks and months long (even in my home city everything's booked a month ahead and you can see the big queues if you go in person).

People can indeed just refresh the page themselves and hope a spot opens up, but that's pretty annoying.

My university had a queuing system used for presenting assignments. You could always access the page, but the queue wasn't open unless the TAs were there ready to take assignments. When the queue opened it would send a websocket message, so I made a browser extension that connected and waited for the queue open message and then immediately entered the queue. The effect wasn't huge (even in larger classes you would only have to wait 20-30 minutes even if you entered late) but it was nice to be able to sit back and not worry about getting a good spot.

My university actually had a bidding system. You would start the year with 10,000 points, and you could bid any number on electives. So you could spend a lot of points on one particular popular elective, and be guaranteed to get it, but then you would not be able to bid as high on other electives for the rest of the year.

The system was pretty smart, except for the part that you could bid negative amount of points on an elective that wasn't particularly popular and increase your total. That was a pretty major flaw.

[edit: I should say that I didn't use my extra points for anything. I wasn't particularly interested in the popular electives anyway. I did make sure to end the year with 10,001 points, hoping they would fix the bug, but I don't think they did.]