Scraped 300k Japan real estate listings and created map-based metrics

What an insane ride it's been building this thing.... After coming back from a 2 week trip to Japan to visit my old study abroad friends and professor this past May, I decided to build https://nipponhomes.com then https://nipponhomes.com/analytics . Out of transparency, I'm losing like $2k/year in raw cloud costs (Vercel, Supabase, Zyte, OpenRouter, AWS, etc.) but I am so humbled in how much I've learned just by building this. Tbh, I realize that I am so fortunate to have the time and money to experiment with something like this, as I know many people out there are really struggling with the current market (heck I am getting rejected left and right too by my dream companies). Documenting here some of my learnings to share:

1. I learned how to create a lightweight, custom multi-modal recommendation system; I also ended up getting 2nd place for in Liquid AI's hackathon with this. (https://github.com/angelotc/lfm2-vl-embeddings) . Turns out you just need an MLP or attention layer to fuse two sets of dense embeddings.

2. Queues + async workers are a must for processing things at scale (listings in my case). Kinda go into it more in this video: https://youtu.be/qXOk7_3vZgQ?si=Mk1l3dYhzdQuvFe3&t=360

3. You need proxies (Zyte, BrightData, OxyLabs, etc.) to scrape at scale if you don't want to build your own proxy rotation system.

4. Wasn't getting sign ups until I added this feature where after a person views 3 listings, they have to sign up. That like 10x'd my signups (#growthhackingiguess)

Ps. I kinda built this out of depression tbh lol as I got rejected to Meta for the 2nd year in a row and Open AI for the 3rd time. The site currently has 8k monthly users, which is super cool, but tbh I don't know if I want to keep working on it anymore as I'm not really learning anymore, and just adding shit here and there. I know the site isn't perfect yet, and I'm getting some interests from major banks, japan real estate consultants ( the folks that help you buy the houses), and competitors (they want the data) in case you folks were interested on who is reaching out.