Yes, sorry I was unclear about that. I knew multiple parts of it, but not what position the segments would go into. I had also mixed the passphrase up with another old obsolete one, so I had too many of these segments (back then I broke up the passphrases into blocks of 4 chars in my head) to fit in.
There was also a couple of chars that I was uncertain about so those got "fully" brute forced.
The worst part about the process was that it was hard to create rules for Hashcat to generate all different possibilities for the passphrase with all the information I knew about it baked in.