Yeah, maybe I can help here as I faced the same stark realisation some years back.
Generally speaking, higher testosterone levels are correlated with higher protein and fat intakes, mainly because cholesterol is used by the Leydig cells in your balls to produce testosterone. I am not saying you should start adding butter to your coffee or whatever other dumb things the keto people are saying, but going very low-fat and/or low protein is not good for your test levels.
I greatly increased the amount of animal protein I eat to roughly 2g/kg (roughly 1g/lb, if that's easier). I ignored fat, just consuming it with my protein sources. To make room for the added protein, which is now the centre of my diet, I eliminated most starchy carbs (pasta, rice, etc) as they are low-value foods. I still eat bread because, well, freshly baked bread is delicious and I am not a low-carb zealot. My main carb sources are now vegetables, lentils and beans, the latter two of which have some protein as well. But the best sources are always animal-based and this will no doubt irritate many people who read this.
I had my T levels taken before and after and I saw a pretty big increase in free test. But to be honest, test levels vary a lot even over the course of a single day so the best thing is to simply go by the old standby: are you consistently getting wood in the morning? If yes, you are probably okay.
Another bonus to the high-protein life is increased muscle protein synthesis. When you are young, this is largely hormonally driven but as you get older, falling test levels make it more difficult to build and keep muscle. This leads to age-related muscle wasting, also called sarcopenia. You can delay this process simply by eating a lot more protein to drive an anabolic response.