If we can guarantee that the pawn stays in its own file then I see a path for improvement (by having the actual position vs back row usage as a 1-bit toggle), but this is not broadly the case due to movement across files on pawn captures.
Using own king's position loses the info of the file. Iff you guarantee that the pawns are listed in order, this is not a problem. But later in TFA a permutation or sorting of pawns is suggested, which would then mess things up.
Please note that a slightly modified version of my method, where a pawn that can be captured e.p. is swapped (not merely moved) with whatever is in its corresponding back rank before any other encoding takes place, is compatible with all the other clever techniques suggested in the comments.
This is detailed in the Python code. https://github.com/savarin/bitpacker/blob/239d68dcd3ec5db67e...
Yes using the back row works too! I was trying to see if we can get an improvement on the 18 additional bits needed from the post (or 14 additional bits by taking advantage of knight and bishop ordering).
These discussions have been great, very much enjoying seeing the incremental improvements!
Edit: I did another pass, item 4 in the notes did mention this.
> [4] For en passant we need the pawn to remain on its home file. Hence we exclude the pawn from this step if it can be captured en passant. Captures can appear on any file.