senecaso

Maybe we could just publish a dump, in a standard format (WARC?), at a well-known address, and have the crawlers check there? The content could be regularly updated, and use an etag/etc so that crawlers know when its been updated.

I suspect that even some dynamic sites could essentially snapshot themselves periodically, maybe once every few hours, and put it up for download to satiate these crawlers while keeping the bulk of the serving capacity for actual humans.

Best of luck to you and your team on user acquisition!

pricerunner is another site which operates in a similar space. We had plans to build out the price tracking and a number of other features, so that we would appeal more to users who had your use cases. Sadly, we weren't getting enough traction. We did have regular users from the EU, but we simply couldn't seem to get in front of enough eyeballs for it to matter. At least at first, I expect that a large amount of your traffic to a new site like this has to be driven by Google, and we failed on that front as well. I'm not an SEO expert, so there were likely many things we did wrong or didn't even do which lead to this situation.

re: a sales platform, that's a pretty big challenge to take on, which would require massive investment up front. Not sure thats a viable route for most. We did have plans to address the "without hoping to various websites" problem, as we identified that as problematic for users very early on. The solution was relatively simple, but required more money to build out. We simply ran out of funds before we could get there.

Still, there were problems with our solution that if addressed may have provided a better market fit. If we had had more runway, we would have worked to address them, but that simply wasn't in the cards.

I remember in once case, I found what appeared to be an escort service listing "models" on Shopify. It was super creepy. I needed to get in front of that one pretty quick as well, as it was turning up in results.

Seems he is indexing nearly 650k shops.

We spent time posting in forums helping people find items they were looking for, and we had a few posts here on HN that generated short-lived, explosive traffic bursts. I remember those days we had posts get picked up on HN, it was always an exciting night!

We were looking at influencers and getting our name getting bloggers to talk about us, but, again, without capital, our options were very limited here. I'm sure someone with more of a marketing background would have found a bunch of ways we could have generated organic user growth, but neither me or my business partner had that skill set.

If I were to do it again, I think I would try to get someone with a marketing background involved to help gain traction. Without that, even the best product in the world will die of starvation if no one finds it.

A few years ago, my partner and I built vendazzo.com (now defunct). It was an e-commerce search engine on products listed on Shopify shops (sound familiar? :)). At the time, we had > 100m products listed, and I don't remember how many shops we were indexing.. over 100k I think, but we had access to over a million. Overall, I think your approach is very similar to ours, but we managed to keep our costs lower. At the time, we were spending ~$550/mo, and our search times were under 300ms. We had established partnerships with a number of shops, and we had a few users, but not nearly enough. That's where the wheels came off. The site operated for over a year, but the monthly costs wore us down until we finally decided to pull the plug.

I still maintain that this is a good idea, and constantly have to fight off the urge to "try again", however, to do it properly, I think funding would be necessary, or finding some way to organically gain a lot of users.

Looking back, there are things I could have done to reduce my opex further, but in the end, it still wouldn't have mattered if I couldn't figure out how to acquire users.

Getting access to people's contact information would be useful at a later point in the process, but early on I think immediate feedback would be the most beneficial. Even something along the lines of a HN thread like this one would be enough engagement at first. In fact, perhaps that is even sufficient :)

You can find a rough breakdown of our product counts by category in our "Why Shopify is the best (only?) alternative to Amazon" post: https://blog.vendazzo.com/why-shopify-is-the-best-alternativ...

Of the 140k shops we currently have, the vast majority are clothing (shoes, shirts, jewelry, etc), home & garden (furniture, kitchen, cups, glasses, etc), and vehicle parts. The other categories still have a significant number of products in them, but we're light on the computer accessories, and much of what you find will be either older generation or enterprise. Its possible that the 140k shops that we selected just happen to not be electronics/computer shops, but until we're able to index the remaining 1.2M+ shops (that we're aware of) we won't know for sure.

One other thing to point out is that the filters (after you search), can be very helpful We're still not at a point where we can build out semantic search, so using the filters to select a category, colour, etc, can drastically improve the relevance of your results. Hope that helps!

BTW, I have made changes to remove the "Route Protection Package" from our index, and it should start disappearing over the next week or so. Thanks for pointing that out!

We don't have a lot of shops with electronics at the moment, so a query for "router" would be light anyway, but these particular results are not useful at all.

https://vendazzo.com/

We still have a long ways to go, but you can search for items and filter based on things like category and distance (if you are trying to shop locally). We're doing things on a very tight budget at the moment, so sometimes the site goes down, but we're working on it :)

If you have any feedback, we would love to hear it.

https://vendazzo.com/

Our product is very much a WIP, and will likely fall over with even modest load, but feel free to kick the tires. We have a lot of work we need to do, but if anyone (buyer or seller) has feedback, we would love to hear it. We're currently only indexing ~140k shops, and ~90m items, mostly from shops in the US, Canada, and Europe, but its a start.

That said, depending on your age, with a max RRSP contribution of 27k, if you can contribute the max for 20yrs, you should have enough to retire on RRSP alone, assuming reasonable retirement living.

Assuming you contribute $27230/yr for 20yrs, and manage to get 5% growth on that account each year, you would end up with about $1m in the RRSP. It all depends on how much time you have to grow the account I guess.