Comment by jagrsw | Hacker News Reader

Comment by jagrsw | Hacker News Reader

Parent Full thread

The ISA is really beautiful (maybe with the exception of BCD additions). I have a m68k disk image/kernel which works with qemu-system-m68k, out of historical sentiment (used to have A500, now I have A4000 - sadly no time to play with it, so in my basement).

As for abi/syscalls. I noticed that strace reports hundreds of calls to get_thread_area() under Linux/m68k. I presume under x86, arm etc. some CPU register is used to store pointer to the TLS, but under m68k "they" decided to invoke syscall every single time. I don't think it'd be very efficient, and also it pollutes strace output, unfortunately, given how frequently TLS is accessed these days.

  $ uname -a
  Linux ds-m68k 5.16.0-rc1-g02e0dceb9e3b #7 Fri Jan 14 01:27:22 CET 2022 m68k GNU/Linux
  $ strace -c /bin/ls
  % time     seconds  usecs/call     calls    errors syscall
  ---- ----------- ----------- --------- --------- ----------------
  0.00    0.000000           0         6           read
  0.00    0.000000           0         1           write
  0.00    0.000000           0        10           close
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         2         2 access
  0.00    0.000000           0         3           brk
  0.00    0.000000           0         3           ioctl
  0.00    0.000000           0         1           munmap
  0.00    0.000000           0         2         2 statfs
  0.00    0.000000           0         1           uname
  0.00    0.000000           0        10           mprotect
  0.00    0.000000           0        14           mmap2
  0.00    0.000000           0         2           getdents64
  0.00    0.000000           0         8           openat
  0.00    0.000000           0       285           get_thread_area
  0.00    0.000000           0         1           set_thread_area
  0.00    0.000000           0         9           statx
  ---- ----------- ----------- --------- --------- ----------------
  100.00    0.000000           0       359         4 total

I have been playing with defining my own toy ABI, and I came up with a trick for thread-local storage:

Allocate the segment for TLS and the thread stack together so that both start (stack grows down, TLS up) at a 2*n - aligned address, with n large enough so that the stack size never exceeds 2*n. Then, you could always get TLS-1 from SP by setting the lower bits to 1.

To avoid an odd size for the segment, you could put the TLS block at negative offset from the end of the segment.

I had intended mine for machines with 64-bit pointers, but data structures tend to be smaller on 32-bit machines, so they can cope with smaller stacks. On the M68K Amiga the stack size was fixed and often relatively small: recommended minimum was 16000 bytes, and for large programs recommended size was 80000. Modern operating systems for CPUs with MMUs grow the stack dynamically but still have to set aside a part of the address space for the maximum it could grow into.

IIRC that's how TLS used to work in LinuxThreads.

I wouldn't say anything with separate register files for addresses and data is "beautiful", but it's certainly miles better than other microprocessors at the same time like the 65816.

I still remember the time on a BBS that someone talked me out of getting an Apple IIgs and getting an Amiga 500 instead. I wish I remembered his handle.

Could you post a bit more info about that binary? (eg: run ldd to dump the used libs, etc)

Could you get some kind of stack trace at the time of the get_thread_area call?

I'd like to be able to pinpoint the call(s) somewhere in

https://codesearch.debian.net/search?q=get_thread_area&perpk...

Maybe that is a form of exploit mitigation? Maybe they can't cache the TLS pointer because the kernel may change it out from under you without warning? I do agree that it smells like a possible performance issue and it seems like there must be a better way to do it.

From https://lists.debian.org/debian-68k/2007/11/msg00071.html (Draft TLS/NPTL ABI for m68k and ColdFire, version 0.2)

  There are no spare registers available to designate as the thread
  register.  Therefore, kernel magic is needed to obtain the thread
  pointer from userspace.

I guess there was unused 'fs' in intel. There was nothing left in m68k.

This doesn't really answer the question. Even if there isn't a free register the value could be stored on the stack and avoid the syscall overhead. Or maybe I'm misunderstanding what that variable is doing. Or at the very least the value could be calculated once via the kernel and then cached in main memory/a general purpose register.

I'm speculating heavily here, but it seems TLS seems to be userland thing, but with OS support (e.g. CLONE_SETTLS flags, *_thread_area()).

Maybe it was thought more like a thread-level thing, and not runtime thing. E.g. if we had some competing runtimes in one process space (e.g. go, jvm, libc), then they can all obtain TLS pointer via some unified method (via register or syscall). Otherwise there would be need for some chosen runtime, which would keep the pointer and distribute it around - which would create another set of problems.

Putting it on the top of initial process stack would work I guess, though.

The value is per thread, so if you're caching it to main memory (including in the stack) then you need one copy per thread and you need to somehow know which one to use based on what thread is running. You'd need to store that index somewhere so there'd be an infinite recursion. Even if it's stored at the base of the current stack you don't want to have to walk all the stack frames to find it.

You can store it in a GPR instead (that's what some modern architectures do) but then it needs to be part of the ABI. Otherwise at the beginning of a function you can't assume any register contains the correct value.

Reply on news.ycombinator.com