Hacker News Reader

Little crafty trick: grab the flickr dataset (yfcc100m) and snag photos tagged as whatever you want. It's like a janky google image search. I've put together datasets of airplanes, bicycles, dogs, etc this way.

It's not entirely accurate, but it's good enough. Within a few hours you can have a pretty large dataset of whatever you want, really. (Yay for massive dataset plus user tags.)

You can snag a copy from my server, if you want. (Warning: It's a direct link to a 54GB json file.) https://battle.shawwn.com/sdc/f100m/yfcc100m_dataset.json

I have a script called "janky-image-search" that returns random results from searching this dataset. Here are a few random hits for `janky-image-search ceiling`:

https://www.flickr.com/photos/35981213@N00/9374577304

https://farm3.staticflickr.com/2084/2513539756_7a768de44c.jp...

http://www.flickr.com/photos/96453841@N00/7077625757/

https://www.flickr.com/photos/28742299@N04/4683245369/

etc.

The quality is hit or miss, but it seems better than 70%.

EDIT: Here's a gallery of the first 100 hits for "ceiling": https://cdn.gather.town/storage.googleapis.com/gather-town.a...

But as you can see, it's not effortless. Most of those ceilings are old. So it depends what you want. It's why labeled data is worth billions of dollars (scale.ai et al).

I found that people don't upload misdirected ceiling photos and videos to YouTube and Flickr. How unfortunate. Yes, it's better than nothing.

Now you've made me want to create a random-ceilings dataset. I spent about an hour trying to think of some way to get you a bunch of un-curated ceilings, to no avail.

If you ended up using the ceilings uploaded to your app directly from your users, it might be possible that there wasn't any better solution than the one you came up with.

Perhaps if there's a database of those 360 photo globes you could crop out the upward-pointing part?

Super neat!

(Small detail: the first 100 hits for ceilings are all _interesting_ ceilings? Nothing like the kind of ceilings usually encountered on a video chat app?)

Why would anyone post something boring on a photography site? There's a bar for quality on flickr; it's not twitter.

Of course, the point is that it may not be good data for training a model.

I was clarifying, since the above poster was using question marks.

Yeah, flickr might not be a good data set to train for boring images. Maybe we need a high quality repository of bad photos.

That dataset will work well for detecting nudity in Zoom calls from the Sistine Chapel :)

Before I download 54gb from your server, can you tell me the structure of the json?

What is a good tool to search it without 64gb of free memory?

My janky-image-search script looks like this:

  tag="${1}"
  shift 1
  curl -fsSL https://battle.shawwn.com/sdc/f100m/yfcc100m_dataset.json | jq '{user_tags, machine_tags, description, title, item_download_url, item_url}' -c | egrep -i "\"($tag)\"" "$@"

Basically, it trades bandwidth for memory. And hetzner has free bandwidth.

Here's curl -fsSL https://battle.shawwn.com/sdc/f100m/yfcc100m_dataset.json | head -n 1, which should give you all the info about the structure. Or most of it.

  {"date_taken": "2013-08-10 11:05:54.0", "item_farm_identifier": 3, "item_url": "http://www.flickr.com/photos/9315487@N04/9497940823/", "user_tags": [], "title": "IMG_7216", "item_extension_original": "jpg", "latitude": null, "user_nickname": "meowelk", "date_uploaded": 1376370010, "accuracy": null, "machine_tags": [], "item_server_identifier": 2856, "description": null, "item_secret_original": "15f4257122", "user_nsid": "9315487@N04", "license_name": "Attribution-NonCommercial-ShareAlike License", "capture_device": "Canon EOS 7D", "item_marker": false, "item_id": 9497940823, "longitude": null, "item_download_url": "http://farm3.staticflickr.com/2856/9497940823_0c0d854111.jpg", "license_url": "http://creativecommons.org/licenses/by-nc-sa/2.0/", "item_secret": "0c0d854111"}

I just dump those to disk and then use `jq` to extract the item_download_url.

FWIW, the full ceiling search finished with 9,325 results. Here's the first 1k.

https://cdn.gather.town/storage.googleapis.com/gather-town.a...

Thanks for the information. So the 54 gigabytes is just plain text JSON? How big is the full image dataset?

Good question. It's 100m images, but I haven't tried to find how large all of it is.

We downloaded a subset of 3 million images, which is apparently 379.77 GiB. So a linear extrapolation would be (100/3*379.77) = ~12,659 GiB for the full 100m images.

12TB really isn't too bad. It's massive, yes, but imagenet 21k is 1.2TB.

I don’t need it, but that’s really awesome of you!

Wow I immediately recognized that first one as the ceiling of a Washington DC Metro station.

Because fewer PhD are minted by "Improved on SotA ceiling detection rates by 50%."

Maybe not PhD, but as a research topic, I think doing this well is an interesting task. I learned that ceilings are not featureless. (I know, it's a kind of ridiculous to say I "learned" this, since I could have learned this from, oh well, looking at my ceiling just now, but it really was a surprise when I encountered this in machine learning data context.)

Ceilings often have lightings! Lightings (either on and off) vary between geography and culture! (They do! Extreme variations!) Lightings, if on, will cause flares! We just sampled from actual distribution (privacy policy included we can monitor connection for anti-abuse and we can use data for research and development of anti-abuse), but dataset collected with care will be valuable. I am sure actually doing this will reveal other considerations, and even just with points I mentioned, it should be good enough for a paper, if not for PhD.

That sounds like the problem in a nutshell. "X is simple, so just gin me up a data set of it." (Four months later) "X is not simple, in all these ways..." Which is another way of saying that discovery and communication with SMEs is an important side skill in project success.

What would be the novel scientific contribution over any other image classifier? What's the storyline? How do I get excited? You collected a dataset for some boring class and trained a standard sota model on it. This is an undergrad project report, not a research paper.

The main question people ask when considering reading a paper is "So what??"

Not an ML person, but I generally understand the concept. Why do you care about ceilings here? I get that the camera is pointed weird ways, but if you're trying to detect nudity presumably if trained on a bunch of images of that subject, it would just call ceilings False.

No, because training set does not include ceilings.

I think I understand, so you basically need in your training set anything that the production model would presumably see for it to work well? You can't just say here are a bunch of positives and a bunch of negatives, the negatives have to be actual things the model will see.

Yes, with emphasis on "well". It does work without ceilings. But your training set should include everything you will frequently encounter in test set. Ceilings are too frequent to miss.

If you think about it, this can't be otherwise. Positive/negative is misleading, since that distinction is the entire thing we are trying to teach. Let's call two labels A and B. You tell the model:

This long penis is A. This short penis is A. This cat is B. This dog is B. Now, what is this ceiling?

The model, looking at the ceiling, discovers a long fluorescent tube. It is long! Neither cat nor dog is long (The model is yet to discover longcat), and while penis comes in long and short variety, all penises seem long-ish. The ceiling is A.

In the typical ML task formulation, closed-set classification, the model is forced to make a choice between the provided categories, even if the output is not really either. And neural networks tend to perform erratically outside of the training data distribution.

Adding common inputs to the training (or at least validation and test) sets is a good solution. Its hard data work, but will pay off. There are some techniques outside of closed-set classification that can help reduce the problems, or make the process of improving it more effective:

- Couple the classifier with a out-of-distribution (novelty/anomaly) detector. Samples that score high are considered "Unknown" and can be flagged for review. - Learn a distance metric for "nudity" instead of a classifier, potentially with unsupervised or self-supervised learning (no labels needed). This has higher chance of doing well on novel examples, but it still needs to be validated/monitored. - Use one-class classifier, trained only on positive samples of nudity. This has the disadvantage that novel nudity is very likely to be classified as "not nudity", which could be an issue.

If your model supports Out of distribution detection, you are very correct.

Why didn't you train on random images from the web? If you carefully select your sources, most of them will be non-nudes.

I'm sure there's ways to extract loads of e.g. tiktok videos, especially the non trending / highly produced ones (I'm making assumptions about the use case here).

If you "carefully select your sources" then you aren't training on "random images from the web".

If you are using random images from the web, there is a decent chance that you will get a result like this.

https://www.theguardian.com/commentisfree/2021/feb/03/what-a...

The Guardian labels it sexist. I assumed that it was because the majority of young attractive women on the internet are posting bikini pictures.

> The Guardian labels it sexist. I assumed that it was because the majority of young attractive women on the internet are posting bikini pictures.

They're both true at the same time, which is exactly the point. The people in the article in the paper weren't trying to teach their neural net that men usually wear suits and women usually wear bikinis, but they did. They only noticed because the failure mode was really obvious. In some other situation, the exact same kind of sexist bias might exist in the data -- and important decisions, affecting people's lives, may be made -- without anybody noticing first.

I'm wondering if you could just take some 3d scenes of say flat interiors and then render them in a good quality with camera looking in random directions to use that as training data. You get actually correct labels because you can calculate it when rendering and label accordingly.