For those wondering, head into Setting > Privacy and Security > Last Seen to change the setting. Available options are Everybody, My Contacts, and Nobody. Manual overrides for specific contacts can be toggled with the Always Share With setting.
There is one exception to this: people will see you online for a brief period of time right after you send THEM a message (otherwise it would feel like talking to a wall).
See: https://telegram.org/faq#q-can-i-hide-my-last-seen-time And: https://telegram.org/faq#q-who-can-see-me-online