Yes, Alt+F4 does close the application (no more process). Clicking on the X closes the window and moves the icon to the system tray, effectively hiding the icon and making the icon on the task bar disappear (or if pinned seem like the application isn't running).
Anecdotally, I rarely use alt+f4 to close a window.