Threaded conversations are better handled by email or newsgroups as opposed to chat in my opinion.
Edit: Removed extraneous word.
I think I saw it because irccloud was using it with their slack integration?
Let a user click the timestamp to the left of the messages (or a "#" or other symbol), which inserts something like this into the reply box:
> Replying to username:
Then your message you send ends up looking like:
> Reply to username: I agree!
Clicking on "Reply to username" or just the username could scroll back the chat and highlight the message you replied to.
Of course, the context here is a client that does indeed implement extra features, and you're right that a text format per message would be an interesting way to implement this: the client could simply hide those messages from the normal view and slot them into the thread system. People using other clients would just see some extraneous line noise like "Replying to <username>|<timestamp>: hello world".