Show HN: Data on Shopify Stores | Hacker News Reader

Show HN: Data on Shopify Stores

shopgram.io

51 pointsrajool30 comments

Hi Everyone! I’m Ali, co-founder of Shopgram. We recently launched Shopgram (https://shopgram.io) which gathers, analyses and publishes data about Shopify stores to help merchants having a successful business.

My team and I released a Shopify app to help merchants boost their sales. Upon launch, we realized we need more data about Shopify stores. We crawled more than 1 million shops and assumed it might be useful for merchants too. With online shopping growing massively in the last few years and the additional acceleration due to the pandemic, more users have headed towards online shopping. As a result, Shopify, a platform for building online stores expedited its growth. There are many merchants who would benefit from the data about trending Shopify stores and products, so we published the gathered data for free and it was well received by Product Hunters. Learning more about Shopify stores made us wonder what challenges shop owners deal with and what their major problems are. Throughout dealing with the challenges, we took a step back and put ourselves in a new merchant or to-be merchant position, searching for the most popular niche markets or products to sell. Price, margin, quality, delivery time, etc., can affect the product merchants are trying to sell on their website.

For this matter, first we identified Shopify-made stores, gathered their data then cleaned them. Same products with different names and vast merchandise made the process hard. We solved this by applying AI tools and Machine Learning to classify the products. e crawled and categorized stores based on products they sell so that one can explore stores or products in a category. At the end we also added a search ability on the processed data. You can see the results at https://shopgram.io

Shopgram has three main parts for addressing the challenges merchants face. Stores - https://shopgram.io/stores Best Shopify stores created by Shopify per product categories based on geographic information. Products: https://shopgram.io/products Popular products sold in Shopify are shown here. We’ve also added features to provide more metadata on them, how to procure such products and finally what are their similar products. Insights: https://insights.shopgram.io Statistics which helps merchants choose the niche market to increase their sales. We’ll be pleased if you can take a look at Shopgram and share your thoughts on how we can improve the site and make it more useful.

Seems interesting. So I suppose your focus long term is on the products side of thing only, to allow merchants to work out which products they should sell? Do you plan on adding any other insights for data such as shipping etc?

That's right. We are going to help them with choosing best vendors and channel, to supply high quality products. That's an interesting idea to add insights about shipping too. Thanks for your comment.

It's a nitpick but to me the title on the page reads like shoporam, "g" looks more like "o".

Ooh I see :) Thanks for sharing it with me.

Hi Ali! I'm a software engineering lead working in the Shopify ecosystem as well.

This is an awesome resource, I just shared it around at my company. It's a lot of data to crawl and no joke!

Thank you so much!

Hi Madelyn, Thanks for your comment I would be gald if we could have a talk. My email is myname at shopgram dot io

how'd you get the list of shopify merchants to start with?

Crawling all subdomain.myshopify.com I’d expect

Reverse IP Lookup

Isn't this limited to the guestimated subdomain ?

We've extracted a list of IP addresses for shopify address (by checking whois on a relatively small number of stores we already know they're on Shopify) Then we've used some reverse whois service (like https://myip.ms) to find other domains that refer to the same IP. Those stores are definately on Shopify as their domain points to an IP address that belongs to Shopify.

This is really interesting. Thank you for the explanation.

I'd love to hear about how you crawl and store the data. Doesn't shopify block you as you are hitting there servers soo much for all the site?

Till now, No :)

Would love to see a list of all the stores in a particular category. Would pay for that. I know there are lists you can buy but your data seems more in-depth.

Glad to see you liked it. We'll add more info to it in near future too.

I thought you could search by store category already!

Cool tool, I'd be interested to see more categories, specifically for vintage and antique items (please note the distinction as well). These sellers are mostly on shopify these days and the community around them is growing fast as it bridges costumers, collectors, cosplayers, and many others

Glad to see you liked it.

You could search for any categories already. Wasn't it useful?

Very interesting thanks. For future developments enhancing the search to allow exclusions would be good. I'm currently searching for "cotton sun hat" but I get masses of caps (which I don't want) so being able to do this "cotton sun hat brim -cap" would be great.

That's great.

Thanks for your feedback.

I'm curious how you're establishing the location of the stores.. There seem to be a lot from JP that are surprising.

Are the locations simply the DNS location?

Also, I guess what's the aim ?

We are using the data crawled from the homepage together with currency information and estimate the location based on them.

Could be you looked like JP to the store because your crawler was in JP and shopify is locale aware.

:-?

I'll check.

this is actually a cool way to find online stores. I would recommend a link to the store right from the list, instead of having to go to the details page.

Thanks for your recommendation, got it.

how do you learn what popular products are?

also what happens after sign in? Is there any need to sign in?

Reply on news.ycombinator.com