Show HN: Daily price tracking for Trader Joe's
traderjoesprices.com
traderjoesprices.com
[0]: https://github.com/cmoog/traderjoes/blob/ea2da58a84d3a04e28f...
And referrer is set twice!
I think it is funny that this misspelling hasn't been fixed after all of these years. It was typo'ed in the original http spec in 1996: https://en.wikipedia.org/wiki/HTTP_referer
But you're technically right, the best kind of right.
(btw its absolutely lovely that you put hanging out with your kids in profile
((Gush much, who?? me!???)))
/need to close my lisp
(Yes it's my April Fool's RFC.)
UK: travelling, signalling
USA: traveling, signaling
The double l serves no purpose in these.
However, there is no such UK/USA split in "referring": both use double "r".
There is a phonetic reason for that which is that the second syllable has the emphasis in "refer", unlike "traveling" and "signaling" where the leading syllable has it.
The double "rr" indicates the stressed syllable, without which "refering" and "referer" could be misread with emphasis on the first syllable, like "buffering" or "suffering"/"sufferer".
A native speaker with good intuition for spelling would never write "refering" or "referrer", even if they find "signaling" acceptable.
I’ve always held on to my grocery receipts for the last 8 years. I took pictures of them but when I moved a box got water damaged so now I only have like the last 3ish years.
Is there any open source software that I can use to transfer these receipts into a useful csv?
I have an idea for a few interesting data visualizations as I’d often buy the same things every week. Grocery bill went from like $70 to $150 with not much changes from what I can tell.
Would be cool to put it out in the public.
Nextcloud also has OCR. You can use a scanner with either.
Avoid touching the receipts. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5453537/
My only complaint is that it lacks a good organization workflow. I have a shared network folder and any file (image, pdf, etc) that you put on that folder gets immediately consumed into Paperless. This happens almost immediately. I have a printer/scanner that allows me to scan to a SMB network drive. So I configured any scans from it to go to that shared folder, which makes integration really nice. I also use GeniusScan on my phone to scan to the same network drive (which requires pro, which is ~$16 a year I think). Genius Scan can save locally to your phone and upload later when you get home, which makes for a good workflow. The problem is that once it gets into Paperless, there's not a good workflow for reviewing and labeling the file. I have been meaning to sit down and provide a contribution to the open source project to improve this, but haven't found the time to do it yet. This is the biggest weakness of the project imo.
For those that have never used paperless. The naming may confuse you. It started off as an open source project named paperless. Then it got abandoned and a team picked it up to update it and make it more modern, and they renamed it paperless-ng (for angular I assume, the new frontend). Then that project lost momentum, so it was forked again and is now paperless-ngx which is the current iteration of it. It currently has a very strong community and gets good updates.
And thanks for the heads up about the toxicity, I use to save them all but after the move I simply take a picture with my phone and throw them out.
paperless doesn't seem to be my exact use case but hopefully after it does the OCR transformation it can allow you to make a csv file.
I'd even pay a decent amount to do it. After doing some more research it seems like MS Office might handle this workflow too in Excel (convert receipt picture to csv data).
Huge user base right off the bat.
Pie in the sky, alas.
Maybe they could grab some preventative lobby money with the threat of it. /s
Of course, I can only imagine this coming to fruition if it is packed to the brim with tracking and 3rd party dissemination.
Probably one day we'll have a "citizen preferences file" where the confidentiality of our interactions with various entities can be granularly set.
* May also enroll you in “helpful” store newsletters and sell your email to every company on your grocery store receipt.
If you're lucky, you won't need to go to a grocery store and determine what a code means, you will only need to map the code to an actual item you bought.
Is it possible you can discuss more what you did?
Or use JSON mode with the API.
1. Scan to FTP dir to TIFF
2. Nightly job submits image(s) from the dir to Veryfi (their free tier is enough and they looks like the best for receipts OCR)
3. Save that raw JSON, enhanced JSON (fix occasional mis-attribution for discounts, calculate unit price), and CSV. Filename is a purchase date - correctly extracted by Veryfi.
4. Render with bash + gnuplot.
5. TODO: store into some DB and render with Grafana or something.
Edit: formatting
FWIW cmoog, I think it is pretty cool. I've always felt that good surveillance on retail prices with specific product information (like weight/servings) would be a solid data set for economists and people like me to understand "real" inflation, "greed" inflation, and general product pricing trends.
I love the idea of pricing over time. It seems like things jump around in 50 cent increments quite a bit. Would be interesting to graph it. I wonder if you'd be able to gather enough data before TJ's decides to implement better endpoint security, though. Having tried to do something similar with airline website data, they sure can make it challenging.
The most appealing part of this hack is that the advertisers weren't the customer, they were the product. :-)
(I shop their stores, but I don't use their website at all.)
Retailers that utilize price zones typically have the baseline price that drives the prices for each price zone (e.g. set prices in California to be 10% more than baseline). Getting the baseline price in an automated way is the ideal solution.
Simple supply/demand.
In simple terms on Valentine's Day they can charge whatever the hell they want and people still want to buy it.
At other times of the year people have lots of alternatives to roses and are more price sensitive.
Economics is in a dire state of proving theories right. It only accounts for perfectly rational actors. A famous nobel prize winner was proven wrong in the same year or so and he said his graphs didn't account for it.
They would if rice prices went through the roof, which they won't because people who sell rice are not stupid.
Econ 101 certainly explains why you can actually buy gluten free products now (supply for demand) and why gluten free products are usually more expensive (niche market).
My ECON 101 prof talked about the limitations of models and theories based on assumption of rational actors in the 1980s ... I have no idea why everyone thinks this is a gotcha.
Throw in subsidies like how Clinton ruined Jamaican rice with protectionism and your supply demand and price are no longer rational. https://www.texasgateway.org/resource/201-protectionism-indi...
Econ 101 ignores subsidies which account for 100% of the big 3 crops and more like rice, and ignores any form of economics.
Yes, Economics can't predict recessions or the next Beanie Baby fad, that doesn't mean supply and demand is nonsense.
Sometimes increases in demand can both increase supply and decrease cost.
In fact, this actually demonstrates another effect: as something becomes cheaper, people will consume more of it. Strictly speaking, demand is a curve where quantity demanded is inversely proportional to price; if “demand goes up” we usually mean the entire curve shifts to the right. There’s also a supply curve: quantity supplied is directly proportional to price. The intersection of these curves is the market price; nobody “decides the prices” unless you have a monopoly or cartel or government interference.
Housing, incidentally, is a classic example of government interference.
and https://theconversation.com/why-does-the-price-of-turkeys-fa...
The products often become loss-leaders (or close to it) to draw people into stores, so perhaps the interesting thing here is that valentines roses have a different pricing pattern -- probably because, unlike turkey, they don't freeze well, so supply is much less elastic.
one of the few notable production gql users?
I wonder how the author even found the endpoint! Is this at risk of Trader Joe's noticing and moving to require an auth token? I can't imagine the openness being intentional, at least outside that brief period of techno-optimism in the late 2000s where it seemed like everyone was offering data sources to build into your Twitter bot.
Possible secondary application: could this GraphQL endpoint potentially be used to determine when a product is being discontinued?
But if you compare it to stores that actually compete on price, it's overall much more expensive even if some items or categories are well priced. I.e. Grocery Outlet or Winco or Aldi (TJ's parent company?) or any number of ethnic grocery stores like Zion or Ranch 99 are much cheaper. YMMV but here in Southern California, there are probably a dozen different chains that compete on price that blow it out of the water.
Also won’t prices differ by location? So many questions.
Discussion of regional price differences in other comments.
url <- HTTP.parseRequest "https://www.traderjoes.com/api/graphql"
https://github.com/cmoog/traderjoes/blob/master/Prices.hs#L8...https://www.julyp.com/shared-widget/018d8a17-fc96-72b3-806f-...
Looking through your code, I see that the default store is Chicago South Loop (701). This would be helpful information to include on the website displaying the results.
So that shouldn't be an issue.
I live in Los Angeles. Many times I've shopped at the TJ's in Silver Lake and one of the TJ's in Pasadena on the same day. Most prices are the same, but on many items the Silver Lake store is consistently 5-10% higher.
I've also shopped in midwestern TJ's and noted that the prices were generally lower than LA.
Products differ significantly, too. Items with the exact same name and packaging can be totally different regionally. For example, "Sonoma Chicken Salad" used to be a favorite of mine here in California. The Iowa version was disgusting, with roughly twice the mayonnaise, fewer nuts and grapes, and 3x the sugar.
Part of it is that TJ's used to be much more about the 'one-off buy of a weird but tasty product'. They would find a product they could sell, buy as much of it as was possible, and sell through it, never to order and sell it again. Over time though, it grew to be the store where people went for basic staples, and so the way they sourced products probably changed to a more traditional model that grocery stores use, where many of the more perishable products are regionally sourced.
So what you experienced with the Sonoma Chicken Salad (which, I commend your appreciation, that used to be a favourite of mine to get for lunch) is likely a result of them just being completely different products made in different places by different companies.
Trader Joes in the 90's and early 2000's was a cool quirky grocery store to pick up some fun stuff and good wine to round out the weekly grocery shopping. TJ's in 2024 feels like Kroger standing on Whole Foods shoulders wearing a trench coat.
They list prices on their website even without a local store selected. And then picking a local store in different locations, I can't find any prices that change.
I mean, obviously there might be exceptions. And I assume local produce varies, the same way it varies in every supermarket not just by the season but by the week. Most fresh produce isn't even listed on their site, and things like fresh salads are going to be based on local produce prices. (E.g. this daily price tracker doesn't have any entries for apples of any kind, for instance.)
But I can't find any evidence of any Trader Joe's products (whether frozen or snacks or jarred or bakery) having different prices between stores. Which is what I meant -- the stuff on their website. But it's good to clarify the difference between that and fresh produce.
(I could always be wrong, but you can find it repeated all over forums that Trader Joe's prices are the same everywhere, and they are in my experience as well -- it seems to be "common knowledge".)
It would be neat to be able to click on a product and see a price history graph as well (since it seems like you should have this data your db).
But where are the cornichons?
https://www.harpercollinsleadership.com/9781400225422/becomi...
Beer and wine can be bought at grocery stores, though on Sunday you cannot buy until at least noon. At 11:45 on a Sunday morning you will often see a few people waiting near the checkout with a cart full of groceries and a 12-pack.
It's stupid.
Really cool, and gives you POV like you’d get with Uber Surge.
Is this how retailers hedge and make up for losses elsewhere?
I wish we could also know tracked wholesale price.
Open nodes at stores could help BLS & commercial orgs doing regular price data gathering. I work on supplier pricing dynamics and for commoditized products freeing your pricing data is powerful.
https://www.foodandwine.com/news/trader-joes-two-buck-chuck-...
Yes, please, continue to shop at Trade Joe's and subscribe to Spotify. Please.
You are suggesting these are stores people should avoid for labor-unfriendly practices. Assuming that people are aligned with you on that value set, I don't think a sarcastic comment without actual recommendations, on a post about a tangentially related project, is going to move any needles.
And so, assuming the mods let us live: What are your recommended alternatives to TJs? TJs is, for better a worse, one of the few grocery store options with both decent quality and not entirely ludicrous prices. This project makes it even more enticing, because you can finally look up prices directly and see price history.
If there's another store that offers all that, I think a lot of people would be interested, independent of "why".
Besides that, we'd get updates from corporate with a list of new price tags we'd print out any time they changed something (100% with regional fluctuation baked in, but not at a store level).
I regularly use the Ralphs app to cross shop while I'm at Trader Joes. They are only 2 minutes walking apart, so I normally start at TJs. However, sometimes I end up at Ralphs first and now having this data it could lead to an unplanned trip to TJs to save a few bucks.
Now do all the grocery stores!