Hacker News Reader

Comment by achille | Hacker News Reader

This is a standard feature of many browsers, including Safari.

E.g: On OSX you can download a .dmg file or .zip file, and when opening the OS will warn: "XYZ is an application downloaded from the internet. Are you sure you want to open it?". The information about the origin of the file comes from extended attributes.

See: https://www.idownloadblog.com/2017/04/20/fix-application-fro...

View on HN

Windows has the functionality you describe too, but it works only by storing a flag specifying what kind of origin the file has, not specifically what the origin was. Your article seems to indicate that mac OS uses basically the same system as Windows.

EDIT: According to some other comments in this thread, I'm wrong. Mac OS does store the whole origin.

EDIT 2: Looks like I'm wrong about Windows too, which also stores the whole origin. This actually disagrees with what is written in the bug report, so perhaps it needs to be updated.

No, MacOS stores the origin of the file. Downloaded with Safari:

    $ xattr -l Downloads/Ethiopian_Airlines_ET-AVJ_takeoff_from_TLV_\(46461974574\).jpg
    com.apple.lastuseddate#PS:
    00000000  D9 76 8A 5C 00 00 00 00 9F E5 89 0D 00 00 00 00  |.v..............|
    00000010
    com.apple.metadata:kMDItemDownloadedDate:
    00000000  62 70 6C 69 73 74 30 30 A1 01 33 41 C1 1D 57 2B  |bplist00..3A..W+|
    00000010  FF B1 3A 08 0A 00 00 00 00 00 00 01 01 00 00 00  |..:.............|
    00000020  00 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00  |................|
    00000030  00 00 00 00 13                                   |.....|
    00000035
    com.apple.metadata:kMDItemWhereFroms:
    00000000  62 70 6C 69 73 74 30 30 A2 01 02 5F 10 7D 68 74  |bplist00..._.}ht|
    00000010  74 70 73 3A 2F 2F 75 70 6C 6F 61 64 2E 77 69 6B  |tps://upload.wik|
    00000020  69 6D 65 64 69 61 2E 6F 72 67 2F 77 69 6B 69 70  |imedia.org/wikip|
    00000030  65 64 69 61 2F 63 6F 6D 6D 6F 6E 73 2F 64 2F 64  |edia/commons/d/d|
    00000040  32 2F 45 74 68 69 6F 70 69 61 6E 5F 41 69 72 6C  |2/Ethiopian_Airl|
    00000050  69 6E 65 73 5F 45 54 2D 41 56 4A 5F 74 61 6B 65  |ines_ET-AVJ_take|
    00000060  6F 66 66 5F 66 72 6F 6D 5F 54 4C 56 5F 25 32 38  |off_from_TLV_%28|
    00000070  34 36 34 36 31 39 37 34 35 37 34 25 32 39 2E 6A  |46461974574%29.j|
    00000080  70 67 3F 64 6F 77 6E 6C 6F 61 64 5F 10 19 68 74  |pg?download_..ht|
    00000090  74 70 73 3A 2F 2F 65 6E 2E 77 69 6B 69 70 65 64  |tps://en.wikiped|
    000000A0  69 61 2E 6F 72 67 2F 08 0B 8B 00 00 00 00 00 00  |ia.org/.........|
    000000B0  01 01 00 00 00 00 00 00 00 03 00 00 00 00 00 00  |................|
    000000C0  00 00 00 00 00 00 00 00 00 A7                    |..........|
    000000ca
    com.apple.quarantine: 0083;5c8a76d8;Safari;ADF309D2-762B-4FE2-AEC6-104E019BDBF9

Additionally, the same information is stored in an SQLite database in your home directory. In fact, the ID at the very end of the output is the primary key to the table:

    sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2 "select * from LSQuarantineEvent where LSQuarantineEventIdentifier = 'ADF309D2-762B-4FE2-AEC6-104E019BDBF9'"

The entry in the table also doesn't seem to be deleted when you delete the downloaded file. That is, you can get a list of all files you've ever downloaded:

    sqlite3 ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV2 "select * from LSQuarantineEvent"

Hmmm, I'm not seeing any entries in that table from Safari (my primary browser). I am seeing all the Hombrew Cask downloads, iChat (messages) and a couple Firefox and Brave based downloads.

    > select LSQuarantineDataURLString from LSQuarantineEvent where LSQuarantineAgentName = 'Safari';
    https://upload.wikimedia.org/wikipedia/commons/d/d2/Ethiopian_Airlines_ET-AVJ_takeoff_from_TLV_%2846461974574%29.jpg?download

Maybe you've got Safari configured in some way that prevents it, but with the default configuration, Safari definitely gets entries in that db.

No, macOS stores the whole thing in the "com.apple.metadata:kMDItemWhereFroms" xattr as a plist. e.g.,

    [
      0 => "https://raw.githubusercontent.com/marco-c/code-coverage-reports/master/web-platform-tests.tar.xz"
      1 => "https://github.com/marco-c/code-coverage-reports/blob/master/web-platform-tests.tar.xz"
    ]

Pretty sure [0] is the downloaded file, and [1] is the page from which the download was initiated.

Windows doesn't just have a flag.

For example:

- If I download this using Chrome: https://aka.ms/getvsdbgps1

- Open a command prompt, cd to my Downloads directory

- Execute this:

    notepad GetVsDbg.ps1:Zone.Identifier

I get:

    [ZoneTransfer]
    ZoneId=3
    HostUrl=https://vsdebugger.blob.core.windows.net/vsdbg-16-0-11220-2/GetVsDbg.ps1

Some files I see also have a ReferrerUrl

I've always thought NTFS streams was a really cool idea- wish MS had kept them in ReFS

Apparently they added it back after the initial release.

Support for alternate data streams was initially not implemented in ReFS. In Windows 8.1 64-bit and Server 2012 R2 the file system reacquired support for alternate data streams, with lengths of up to 128K

https://en.wikipedia.org/wiki/ReFS#Removed_features

Safari only records that file should be quarantined, not the URL.

Here are the extended attributes for an image downloaded by Safari in Sierra:

  ecthelion ~>xattr -l Downloads/fm_800-2.jpg
  com.apple.quarantine: 0083;5c8aa472;Safari;1021CF85-4F78-492B-A8E3-766C44A3A671

Here is the equivalent in Chrome Canary:

  ecthelion ~>xattr -l Downloads/fm_480.jpg
  com.apple.metadata:_kMDItemUserTags:
  00000000  62 70 6C 69 73 74 30 30 A0 08 00 00 00 00 00 00  |bplist00........|
  00000010  01 01 00 00 00 00 00 00 00 01 00 00 00 00 00 00  |................|
  00000020  00 00 00 00 00 00 00 00 00 09                    |..........|
  0000002a
  com.apple.metadata:kMDItemWhereFroms:
  00000000  62 70 6C 69 73 74 30 30 A2 01 02 5F 10 29 68 74  |bplist00..._.)ht|
  00000010  74 70 73 3A 2F 2F 62 6C 6F 67 2E 6D 61 6A 69 64  |tps://blog.majid|
  00000020  2E 69 6E 66 6F 2F 69 6D 61 67 65 73 2F 66 6D 5F  |.info/images/fm_|
  00000030  34 38 30 2E 6A 70 67 5F 10 18 68 74 74 70 73 3A  |480.jpg_..https:|
  00000040  2F 2F 62 6C 6F 67 2E 6D 61 6A 69 64 2E 69 6E 66  |//blog.majid.inf|
  00000050  6F 2F 08 0B 37 00 00 00 00 00 00 01 01 00 00 00  |o/..7...........|
  00000060  00 00 00 00 03 00 00 00 00 00 00 00 00 00 00 00  |................|
  00000070  00 00 00 00 52                                   |....R|
  00000075
  com.apple.quarantine: 0081;5c8aa4ab;Google Chrome Canary;B718AF12-557C-47AD-840A-0CA98281F256

That is contrary to what I'm seeing. The extended attributes for an image downloaded by Safari (in Mojave) do contain the kMDItemWhereFroms attribute.

    com.apple.metadata:kMDItemWhereFroms:
    00000000  62 70 6C 69 73 74 30 30 A2 01 02 5F 10 7D 68 74  |bplist00..._.}ht|
    00000010  74 70 73 3A 2F 2F 75 70 6C 6F 61 64 2E 77 69 6B  |tps://upload.wik|
    00000020  69 6D 65 64 69 61 2E 6F 72 67 2F 77 69 6B 69 70  |imedia.org/wikip|
    [snipped for brevity]
    com.apple.quarantine: 0083;5c8a76d8;Safari;ADF309D2-762B-4FE2-AEC6-104E019BDBF9

Have you modified the configuration of Safari in any way? I never use Safari, except this once to download an image to test with. It seems possible to me that Safari might disable it if you flip some privacy or security switches, which I haven't done.

Reply on news.ycombinator.com