but then you dont have visibility of what changes you have made. having config management improves visibility of how something is configured
The visibility of the changes aren't a virtue in themselves, it is only when you do something with them that they become useful.
Spending hours documenting my build system for my pet laptop to save myself 60 seconds a few times a year isn't a good payoff for me. If the payoff is that you also learn configuration management along the way, then that's fine.
(I also don't quite understand how you're testing and validating your build system for your laptop without periodically wiping it and reprovisioning it -- and once you start considering things like that I definitely don't have enough time for all that)