HTTP2 (if used) already makes the metadata parts of the protocol pretty concise and low-overhead; it's just the messages themselves that are large. So it'd be nice if—at least for the message bodies—there was an optional binary layer-6 encoding (e.g. CBOR, BERT-RPC) that clients and servers could try to negotiate with their Accept headers, with an expectation that many servers would implement it, and an allowance for constrained clients that implement that encoding but not JSON.
$ telnet irc.freenode.net 6667
...
NICK Ara4n
USER Ara4n Ara4n irc.freenode.net :Matthew Hodgson
...
JOIN #matrix
PRIVMSG #matrix :test
curl "https://matrix.org/_matrix/client/r0/rooms/!cURbafjkfsMDVwdRDQ%3Amatrix.org/send/m.room.message/uuid?access_token=$access_token" -X PUT --data '{"msgtype":"m.text","body":"test"}'
Meanwhile definitely are some lightweight C matrix clients out there - e.g. https://github.com/matt-williams/matrix-esp8266/blob/master/... is a proof of concept client in C that runs on an ESP8266(!), or the libpurple plugin: https://github.com/matrix-org/purple-matrix.
In the end, if you don't like HTTP or web technologies, there is NOTHING stopping people from contributing alternative transports - COAP, MQTT, capnproto, QUIC, XMPP, WebSockets or whatever floats their boat. But it seems that the simple HTTP+JSON API works well enough that nobody has bothered yet (other than https://github.com/matrix-org/matrix-doc/blob/master/drafts/...).
* No standard maximum message length. If you input a line which is more than 510 characters after being prefixed with the channel name, server name, and other metadata, it may be truncated for some clients. If you're on a network which includes multiple servers, some clients may actually see your message truncated in different locations, based on how long their local server's name is.
* No standard text encoding. Some clients assume ISO8859-1, some assume UTF-8, and some will mix and match (ew).
* No way to determine when certain commands have succeeded. In particular, commands which can generate multiple lines of output (like WHOIS) don't have a standardized termination. Command-line clients sidestep the issue by just showing all the output in one place; higher-level clients have to guess.
* No standard identity services. NickServ is a common bolt-on, but it's a total hack. (It works by pretending it's another IRC server!)
* Bold, colors, and emotes (/me) are all client-side hacks.
* DCC file transfer is a complete and utter mess. It's like TFTP, but poorly implemented, over a TCP transport.
If they would tolerate a little breakage of backwards compatibility, the big networks could easily move to utf8 with standardized identity and colours, etc.
Unfortunately there is a small but very vocal faction of IRC users that are highly entrenched and resistant to change.
> * No standard identity services. NickServ is a common bolt-on, but it's a total hack. (It works by pretending it's another IRC server!)
SASL and account-notify.
> * No way to determine when certain commands have succeeded. In particular, commands which can generate multiple lines of output (like WHOIS) don't have a standardized termination. Command-line clients sidestep the issue by just showing all the output in one place; higher-level clients have to guess.
Batch messages.
You can find more at http://ircv3.net/, there’s a lot of specs on modern IRC features. IRC has changed a lot, you might want to take a look at it again ;)
Well, doesn't that mean the limit is 510 characters including the command? I never as an user had issue with this. This is also implementation issue, if you use one server such as you do with slack, then you know its capabilities.
> * No standard text encoding. Some clients assume ISO8859-1, some assume UTF-8, and some will mix and match (ew).
Well, there's a CHARSET value in 005 numeric from the server which declares the official encoding.
> No way to determine when certain commands have succeeded. In particular, commands which can generate multiple lines of output (like WHOIS) don't have a standardized termination. Command-line clients sidestep the issue by just showing all the output in one place; higher-level clients have to guess.
I'm not sure what you're talking about, there's 318 RPL_ENDOFWHOIS numeric: https://tools.ietf.org/html/rfc1459#section-4.5.2
> No standard identity services. NickServ is a common bolt-on, but it's a total hack. (It works by pretending it's another IRC server!)
You mean it works by pretending that it is another user. The reason for it is an optional component. Some networks (especially the largest one) are running successfully without such services. I kind of like this approach because it leaves the server protocol simple (networks often provide multiple services NickServ, ChanServ, MemoServ etc) and makes the component optional.
> Bold, colors, and emotes (/me) are all client-side hacks.
If you think about it, does it have to be server side? Server is just to relay messages, what they mean is up to a client. In fact pretty much any chat network that allows multiple clients has different looking emoticons. So making it part of the network doesn't help at all with unification.
> DCC file transfer is a complete and utter mess. It's like TFTP, but poorly implemented, over a TCP transport.
HTTP is also over TCP so not sure what you are trying to say here (BTW: TFTP works over UDP).
But ignoring the TCP part, yes DCC feels like it has problem, but that's mainly because of NAT becoming popular. Hopefully as IPv6 is adopted we get back to previous times where ach user had own IP address.
Anyway, comparing DCC to slack etc. Is a bit unfair, because their service is nothing like DCC (Direct-Client-to-Client i.e. P2P) when you upload a file to slack you upload it to their server, but with DCC you're sending it directly to the user without any 3rd party involvement.
Looking at the issues you listed, it feels to me that the problem is not as much with IRC servers, but with IRC clients, in particularly lack of standardization. You could solve most of the issue by defining client protocol on top of IRC, for example:
- handling encoding - abstracting communication with services - colors/emoticons - file transfer (instead of using DCC which sucks with NAT) a client could upload file to a server and provide link to others, emulating behavior of other chat networks
The nice thing about it is that most of this could be implemented in a way that standard clients would still be usable, but if a standard would be created and other IRC clients would respect it, IRC could easily compete with existing solutions.
> Well, doesn't that mean the limit is 510 characters including the command?
No. The limit is on what the server sends to other clients when a message is delivered. For instance, if you send:
PRIVMSG #channel :some long message
:your@hostmask PRIVMSG #channel :some long message
(I incorrectly said earlier that the prefix is based on the server -- it's actually your client hostmask.)
> HTTP is also over TCP so not sure what you are trying to say here (BTW: TFTP works over UDP).
DCC file transfers work on a chunk-by-chunk basis. The sending client sends one block of data at a time, then waits for an acknowledgement before sending the next one. (This is why I describe it as being like TFTP.)
> DCC file transfers work on a chunk-by-chunk basis. The sending client sends one block of data at a time, then waits for an acknowledgement before sending the next one. (This is why I describe it as being like TFTP.)
You're right, I was not aware of the acknowledgments. But this in previous response I said that a lot of issues could be fixed on a client side. DCC is also completely implemented with clients and similarly can be fixed this way. There apparently is already DCC XMIT standard which fixes latency caused by the ACKs (assuming our client wouldn't do same thing as Slack does which is upload files to S3 and share the URL)
Here's the spec if you're interested[1].
There's also a weechat script if you want a command line client[2]
Too bad much of the web stinks, with terrible responsiveness and usability and impose those results through bloated protocols and spaghetti architecture on anything that tries to interface with them.
Multiple times on caltrain I've had people ask me how I'm on the internet as I have meetings on IRC without even noticing the connectivity has gone poor -- while their webchat stuff with multiple megabytes of ajaxy whatever won't even load.
I agree that the web is slow and crap a lot of the time, but chat using gzipped JSON over HTTP/2 would be just as fast as IRC as long as it's only sending the same level of info as IRC.
Because otherwise you'll get to the same place as we were with HTTP/1.1: your nice 'simple' protocol will need to be replaced with one that can support multiple streams, etc.
They are, for what matrix intends to do.
Before, you could only use a programming language if had a production-ready implementation of the protocol you wanted to speak. Otherwise you would spend the next weeks or months tediously writing one.
This was so crippling that XML actually seemed like a good idea. Just imagine!
No, it's not, it's a huge pile of crap, because HTTP is so complicated that you essentially have to distrust any implementation of it that you haven't audited, and even then, it's questionable (plus, most implementations turn out to be buggy if you look closely, and serious vulnerabilities aren't exactly uncommon), and it doesn't even fit many common scenarios without additional hacks like long polling.
Also, "any programming language" is a huge exaggeration. Ever used HTTP on a microcontroller?
> Before, you could only use a programming language if had a production-ready implementation of the protocol you wanted to speak. Otherwise you would spend the next weeks or months tediously writing one.
Now, that's just bullshit. If you have an HTTP implementation, you necessarily also have a TCP implementation. You don't need to write anything in order to use TCP, you just open a socket and use it and forget about HTTP. HTTP serves no purpose in most cases.
Oh, and maybe more importantly: Ever used HTTP for peer-to-peer communication? No, you need a server! Because it's so much more convenient to add a server "in the cloud" than to simply use a TCP connection between two devices that need to communicate. (And no, of course, you technically don't need a server, but in practice that's how people deal with the pointless complexity of HTTP.)
The critical difference? Speed of development. Websockets was so blindingly quick to get working. Haven't hit the edge cases yet though. If I were doing it with raw sockets I'd have to reimplement many of those layers and write a mobile app as opposed to a mobile webpage with only 300 lines of JS.
> Ever used HTTP for peer-to-peer communication? Sort of. They're peers, but one always acts as the server, I just haven't needed to make them true peers, but I could if I needed to. One of the nodes connects to a public server and has other private nodes connect to it. (300 lines of Python/Twisted)
Now, is that websockets or is that "not C"?
I mean, speed of development certainly is valuable, but I would think that same speed of development only needs little more than an 8 bit µC (or, for that matter, is even possible on an 8 bit µC if the memory size is at the upper end of the range).
> Sort of. They're peers, but one always acts as the server, I just haven't needed to make them true peers, but I could if I needed to.
But that's the thing: You could if you needed to. Instead of it just being the natural thing to do. Though I guess NAT also is part of the problem here, which is another of those idiocies of the "modern internet".
The second point answers the first. It is websockets rather than C, because the client is a random user on a mobile web browser.
Now if I could control the other endpoint and were it not for those modern idiocies, P2P could easily be done in C. The problem then becomes discoverability (if you're after true distributed P2P) rather than the actual communications.
(I did port the lightweight lwIP TCP/IP stack to a 16-bit microcontroller and custom RTOS many years ago...)
On the other hand, it's hard to impress with blinkenlights and basic motor driving.
I'm currently going through a phase of thinking on how I could apply functional programming theory and techniques to the next iteration of this embedded system...
And I still prefer doing that over writing a matrix client with HTTP and JSON.
Specifically because my solution works via throttled internet with dialup speeds, while matrix doesn’t.
I’m not saying Matrix is bad, Matrix has some awesome properties (especially regarding encryption). And maybe someday it can become awesome, once it gains a proper binary transport system, which is more efficient and keeps a single socket open. And maybe then, we can finally replace IRC, XMPP, etc. But until then, I’ll keep working on improving IRC.
In terms of the line protocol - we all agree that HTTP+JSON is an inefficient baseline for Matrix. It's just a matter of time for someone to propose a proper binary transport (although HTTP2+gzipped JSON really isn't that bad, and works fine on dialup speeds). Luckily this thread has prompted a few people to offer to contribute one though, so perhaps we'll see some progress there at last :)
What is large, you might ask? 18'000 users in a channel.
And still it should feel and be as easily usable as a channel with 10 users, even via dialup.
That's the point where every byte that isn't used optimally becomes a problem. And where HTTP2 eith gzip or even brotli just isn't enough.
Where Java's IO is too slow, and you need to use the new native noncopying IO subsystem.
But I'll also be happy if, someday, we'll have a truly universal protocol for this.
That's too bad - mostly for the people that won't experience it or see it as viable.
In 2017 I use (al)pine as my sole mailtool. It's many, many times faster and more efficient than web based email - and much more private and secure.