Hacker News Reader

Comment by arp242 | Hacker News Reader

This is on Void. It doesn't have --with-openssl configure args in the package, although the binary also doesn't link to lib{ssl,crypto}. It probably gets auto-detected to "no"(?)

View on HN

Yep, from a Void Linux container it appears that they do not link to libcrypto:

  $ ldd /usr/bin/cksum
   linux-vdso.so.1 (0x00007fb354763000)
   libc.so.6 => /usr/lib64/libc.so.6 (0x00007fb354549000)
   /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb354765000)

For context, I am a committer to GNU Coreutils. We have used OpenSSL by default for a few years now [1]. Previously it was disabled by default because the OpenSSL license is not compatible with the GPL license [2]. This was resolved when they switched to the Apache License, Version 2.0 in OpenSSL 3.0.0.

If the Void people wanted to enable OpenSSL, they would probably just need to add openssl (or whatever they package it as) to the package dependencies.

[1] https://github.com/coreutils/coreutils/commit/0d77e1b7ea2840... [2] https://www.gnu.org/licenses/license-list.html#OpenSSL

Cheers; I guess I should have checked the coreutils implementation; I kind of just assumed it has one implementation instead of being a compile option :embarrassed-emoji:

I also have an Arch machine where it does link to libcrypto, and it seems roughly identical (or close enough that I don't care, this is a live server doing tons of $stuff so has big error bars):

  md5sum              1.58s user 0.31s system 98% cpu 1.908 total
  ~/verify -a md5     1.59s user 0.13s system 99% cpu 1.719 total
  
  sha256sum           0.71s user 0.12s system 99% cpu 0.840 total
  ~/verify -a sha256  0.74s user 0.12s system 99% cpu 0.862 total

Still wish it could do multi-core though; one reason I looked in to this is because I wanted to check 400G of files and had 15 cores doing nothing (I know GNU parallel exists, but I find it hard to use and am never quite sure I'm using it correctly, so it's faster to write my own little Go program – especially for verifying files).

Interesting, there must be something wrong here. Here is a benchmark using the same commit and default options other than adjusting '--with-openssl=[yes|no]':

  $ dd if=/dev/random of=input bs=1000 count=$(($(echo 10G | numfmt --from=iec) / 1000))
  10737418+0 records in
  10737418+0 records out
  10737418000 bytes (11 GB, 10 GiB) copied, 86.3693 s, 124 MB/s
  $ time ./src/sha256sum-libcrypto input 
  b3e702bb55a109bc73d7ce03c6b4d260c8f2b7f404c8979480c68bc704b64255  input

  real 0m16.022s
  $ time ./src/sha256sum-nolibcrypto input 
  b3e702bb55a109bc73d7ce03c6b4d260c8f2b7f404c8979480c68bc704b64255  input

  real 0m39.339s

Perhaps there is something wrong with the detection on your system? As in, you do not have this at the end of './configure':

  $ grep -F 'HAVE_OPENSSL_' lib/config.h
  #define HAVE_OPENSSL_MD5 1
  #define HAVE_OPENSSL_MD5_H 1
  #define HAVE_OPENSSL_SHA1 1
  #define HAVE_OPENSSL_SHA256 1
  #define HAVE_OPENSSL_SHA3 1
  #define HAVE_OPENSSL_SHA512 1
  #define HAVE_OPENSSL_SHA_H 1

Sorry, I meant "roughly identical [to my Go program]", not "roughly identical [to the version without OpenSSL]". The ~/verify binary is my little Go program that is ~4 times faster on my Void system, but is of roughly equal performance on the Arch system, to check that coreutils is not slower than Go (when using OpenSSL). Sorry, I probably didn't make that too clear.

Ah, I should have guessed from the program name. :)

I thought I remember Go having pretty optimized assembly for crypto routines. But I have not used the language much. If you have your source code uploaded somewhere I'd be interested to have a look to see what I am missing out on.

Reply on news.ycombinator.com