The complexity of downloading favicons | Hacker News Reader

The complexity of downloading favicons

simplecto.com

71 pointssimplecto37 comments

17: sometimes there are multiple sizes, according to the spec (https://html.spec.whatwg.org/multipage/links.html#rel-icon)

18: sometimes favicons are not square

19: sometimes favicons have transparent pixels (GIF) or an alpha channel (PNG)

Good ones! I forgot about the transparent pixels.

May I add those with credit to you?

My pleasure, go ahead :)

ICO files can have an alpha channel or a bit mask.

This article on how ridiculous it is trying to sort favicons out from the end user perspective reminded me of how difficult it was to get right as a creator.

I use this tool: https://realfavicongenerator.net/

Rather than make everything simple it makes everything easy.

Thanks to the Sam for immersing in that subject so others can reap the benefit.

I've used https://www.ionos.ca/tools/favicon-generator in the past. It has the image upload ability as well as the ability to quickly draw something on a 16x16 pad for when you just want an absurdly simply icon. (in no way affiliated with the site)

I remember having similar issues a few years ago, I think I ended up using a google service to capture favicons (can't seem to find it anymore)

EDIT: Found it! https://www.google.com/s2/favicons?domain=

(simply add a domain name after '=' and it will return the website favicon)

I came here to post this same tip.

"<img src="https://www.google.com/s2/favicons?domain=news.ycombinator.c... />"

Be advised... while this img tag is ok for embedding, clicking on the link in HN reveals an extra quote character.

Funny how google keeps this service around indefinitely but will kill off any number of other services :-) Though, I'm sure this is much cheaper to keep the lights on vs. something like Wave (RIP) or Plus.

Is this a tool that Google made public to developers use or people just found it?

I've used this in a side project and it's still very hit or miss. Especially depending on how you format your domain URL.

Chromium favicon service for comparison: https://github.com/chromium/chromium/tree/master/components/...

So much code for what ideally ought to be a simple task!

I glanced over it quickly. A lot of the code files are really small, a couple of lines each. The rest is a mixture of actual logic, boilerplate and integration overhead (mostly due to a lot of event based asynchronous task handling). The overhead comes with the territory when integrating something into a big codebase such as a browser. The business rules are still a few 100 lines, though.

BTW they recently added SVG support.

Another recent complication is serving different favicons for browsers running a dark mode. For example, somehow nytimes.com inverts theirs on my browser and it actually looks great. Sites need to be aware that a dark icon over a transparent background will be invisible in dark mode.

This is an annoying downside of dark mode, which I tend to have enabled on everything all the time: Favicons often don't look good (github, circleci...). Also many slack emoji don't show up.

I don't know if there's a good solution to this. It's unfeasible to require developers to provide two icons, particularly in places that are mothballed. Should browsers always render favicons against a light/white background?

It's not really a problem of dark vs. light, it's the fact that transparency in a generic logo is a pretty bad idea. I'm not sure what transparency is supposed to solve that an explicitly designed color wouldn't.

It's actually relatively easy using a media-query in CSS: https://css-tricks.com/dark-mode-favicons/

Favicons are also not always defined in <head>, they are automatically requested by the browser whether you have the <link> tag or not. The <link> tag mainly allows you to specify locations and file types other than the default /favicon.ico.

16: Sometimes it is an animated gif.

That's so funny, I've never seen one moving. Do browsers just convert the first frame into a static icon on the fly when they're downloaded or have I just never noticed one moving?

I've seen some moving favicons. Even JavaScript that could animate depending on what you did on the site.

You can use an animated png too! A fun edge I found back in the day was apng allows you to have a static frame for viewers that don't understand apngs. Firefox understood apng, but if you bookmarked the page it would show the first frame of the animation rather than the static frame. The bug I opened for that is now 10 years old. :)

You can even control them with JavaScript which... well, see the game for yourself:

http://www.p01.org/defender_of_the_favicon/

(Press N to start, the arrow keys to move.)

This is today's best thing. Thanks. Made in 2008 - everything still working in a modern browser.

They’re just not common, that’s all. Most browsers support them (not sure about Chrome, but I believe all the rest do), and even if the browser nominally doesn’t, you can change the favicon dynamically in JavaScript to achieve the same effect (think how some services change their favicon to show an unread count or “new message” indicator, and then start doing that repeatedly—though when the tab is in the background, that’ll probably only net you 1fps).

A lot of these problem apply to crawling in general. I run adstxt.com which crawls ads.txt files.

Some issues we have seen:

- Pages that say they are one encoding but are actually another

- Redirects to other sites

- invalid contents or malformed content. How hard should we try to interpret what they meant? For example, an html page that looks like a valid CSV.

- Page too long or extremely slow

- Invalid certs

20. Sometimes the favicon is dynamically generated by the page JS and is actually a playable arcade game: http://www.p01.org/defender_of_the_favicon/

I always try to squash as much of the complexity as I can with some well tested tools or libraries. In this case I'd use imagemagick's convert to resize and convert any of the possible file formats to the type of image I need. Solves a lot of points on this list. For the download part, curl would also solve a few points (ssl, redirects). Of course it depends if you're writing this for a SaaS product, a desktop or mobile Application.

This list fits into https://github.com/kdeldycke/awesome-falsehood

I wrote https://github.com/scottwernervt/favicon in Python to get a company's logo. Looks like I need to handle more edge cases.

Downloading that little icon you see in you browser tabs should be a simple exercise. It turned out to be a lot more complicated than I thought. Be vigilant that you are not shaving a Yak. This is my story...

Favicons can be TIFF? Browsers don't have support for TIFF—for one, libtiff has historically been a huge source of security vulnerabilities. What pages have TIFF favicons?

I had a list of 500 domains that I used for testing when originally developing. To be clear I don't recall seeing tiff files loaded in browsers, but I definitely remember seeing them downloaded and crashing my code.

I wrote favicon-getter just as a means to populate an icon next to a list of links in another project. Hence, all the references to Yak-shaving.

In the end I just assumed that I would "almost" never see a tiff used as favicon and move on.

favicons? Try scraping the page title of a random link

The consistent problem for me is that cloudflare doesnt like (some/certain/it seems random) ip ranges or user agents and pops up a captcha immediately

Have you tried proxy-rotation services like Crawlera? This is on my radar if I encounter too many issues.

https://scrapinghub.com/crawlera

Reply on news.ycombinator.com