Comment by evil-olive | Hacker News Reader

Comment by evil-olive | Hacker News Reader

Parent Full thread

this part of the response doesn't pass the smell test for me:

> Accusation 4: ‘[Eric] scraped our app store, in violation of the agreement that we reached with him previously’

> Here’s what happened. I wanted to highlight some of my favourite watchfaces on the Pebble Appstore. Last Monday Nov 10, after I put my kids to sleep and between long calls with factories in Asia, I started building a webapp to help me quickly go through Pebble Appstore and decide which were my top picks.

> Let me be crystal clear - my little webapp did not download apps or ‘scrape’ anything from Rebble. The webapp displayed the name of each watchface and screenshots and let me click on my favs. I used it to manually look through 6000 watchfaces with my own eyes. I still have 7,000 to go. Post your server logs, they will match up identically to the app I (well…Claude) wrote (source code here)

so it wasn't "scraping"...it was just a vibe-coded webapp that made at least 6,000 requests to Rebble's servers in a short period of time? possibly more, depending on how many intermediate versions of the app he tested, and possibly many more, if one of those intermediate versions had a vibe-coded "feature" like prefetching a bunch of data for performance reasons?

he agreed not to scrape their services. and then scraped their services. and his excuse seems to boil down to "but I was doing it for a cool reason"

and he tosses in completely unrelated details about putting his kids to bed and having long calls with factories in Asia. those seem calculated to make him sound more relatable - an honest, hardworking, humble family man.

this seems like a relatively minor point in the overall dispute, but if he's unwilling or unable to take any responsibility there, it doesn't boost my confidence that he's being honest about the rest of it.

Scraping has a very clear meaning here, that of exfiltrating data to store. If he just loaded some images to memory so he could pick favorites, that doesn't fit any definition of scraping I'm aware of.

I never heard someone say bulk downloading and data extraction wasn't scraping if it used volatile storage.

I suppose I could describe my use of Hacker News as scraping but perhaps addiction is a better term.

When have you seen someone scrape data for volatile storage?

I've got several bots that scrape various places for transient information which isn't stored anywhere but merely transformed and then posted to the Fediverse and/or notified to my phone.

(I suppose you could argue that the information ending up in a post or notification is "non-volatile" storage but honestly? People will laugh at you.)

Price comparison. Alternative front ends.

It's irrelevant to the definition of "scraping." Scraping from a website is grabbing data, in bulk, not through the website's own interface. It doesn't matter if you save the data, use it in RAM, or just download directly into `/dev/null`. Just like scraping paint off of a wall is still "scraping" whether you're letting it fall onto the floor, collecting it into a trashcan, or sweeping it out the door afterwards.

I think the key question is whether the automated actions resulted in information being retained by Pebble. If it was just going through a motion and pulling some data (or pulling all data but only keeping some of it), then that would be consistent with Eric's story and not be the kind of scraping that Rebble is worried about. They're worried about the content being archived somewhere else, and they seem to think that happened. But did it?

One thing I'm confused about in this whole thing is what makes Rebble think they have a right to the data in the first place? They scraped it! "We don't like you scraping the data we scraped" doesn't hold water for me, whether Eric retained it or not.

Yeah, they definitely started by scraping. Apparently 500 of the 13,500 apps were submitted post-Pebble, and Rebble also apparently did a bunch of other upgrades over time.

But you're right that there's some hypocrisy here, given their roots, and they don't really acknowledge that.

I think the whole conversation shows how ridiculous it is to be worried so much about who's "scraping" what. The open web is designed to be public and permissive. If you don't want someone accessing "your" content, then don't serve it to the public. And if you do decide to serve to the public, don't complain when someone accesses that data in a way you don't like. The Internet would be so much better without all these people obsessed about how their bits were being accessed and about whether X counts as "scraping" or Y counts as "scraping." Good grief, people! Find something else to worry about.

Perhaps in general, but in this case it seems like they did have an agreement not to scrape, which overrides the general scrape-at-will ethos that you're describing.

Pebble threw it away, Rebble did not, and Core is a newcomer whith no right to anything.

Core is making new, compatible hardware, at scale, not as a hobby.

We can buy that hardware from Core, today.

That gives them quite a few rights.

What? Absurd. It gives them the right to nothing at all. They can make an app store themselves any time they want.

Well... I have a watchface on the old store. It is non functional because external APIs changed. I just recently decided to update it, and there is now a much improved version in my github account.

I asked Rebble weeks (!!) ago to give me back access to my own account and binaries on their store and to this day, I heard nothing from them. Nada.

If Core start a new store, I will immediately put the new, much improved version of my old watch face on their store. Rebble can keep the old, non-functional one in their archive if they want.

Do you mean that you uploaded it to Pebble back in the day before Rebble? Have you gone through these steps? https://help.rebble.io/recover-developer-account/?viewall=tr...

Yes. I did all that. Sent the email (many times). Got no reply. Ever.

It might not be the kind of scraping rebble is worried about, but a bunch of requests to extract data into another form is very plainly scraping and the contract doesn't differentiate based on intent or whether the process is entirely automated. The entire contract is similarly loose and informal, which contributes to these sorts of misunderstandings.

The most reasonable solution would have been for Eric to send an email first, but few contract disputes start with everyone doing the most reasonable thing.

From the post on rebble.io

> We made it absolutely clear to Eric that scraping for commercial purposes was not an authorized use of the Rebble Web Services.

So, another point of consideration is whether looking at names and pictures so you can personally favorite them constitutes as commercial use. Based on what Eric said, I don't really think so.

> I wanted to highlight some of my favourite watchfaces on the Pebble Appstore.

It was for a commercial purpose. Not a personal one.

But to be clear, the agreement does allow him API access to view apps and display metadata. Presumably, to build App Store experiences on top of the data. Which could easily include something like stack ranking your favorite apps as a review system, or displaying favorites.

Saying this is scraping is so pedantic, and given that Eric’s company is paying for access to the API, they should kick rocks.

Scraping is about harvesting data. Just using the API like any other user is clearly not scraping.

Is browsing linkedin scraping? Is browsing hacker news through an alternate client scraping?

No, scraping is rehosting hacker news.

I do not believe that's the proper definition of "scraping"

Reply on news.ycombinator.com