Ask HN: How the sysadmin can catch some penetrator?

What log files or what systemd log binaries can have the history of attacks such as trying to exploit some service or proof of some happened penetration via known or unknown bug? What intelligence/logging/anomaly-detecting software the serious guys use for the sake of equipping some online computer with a honeypot-class analisys tool?

I want to have safe online activity for some always-on always-online multi-purpose machine with static IP, no router and a lot of ports opened, having some GNU/Linux OS and not always free and open-source running software.