Skype's Principal Architect explains why peer-to-peer was eliminated
markmail.org
markmail.org
By the way, use WebRTC or some completely peer-based video conferencing (WebRTC still uses central servers to establish connections). Don't use Skype. Doesn't make any sense from a technology standpoint or from a privacy standpoint or anything.
I am actually amazed by how little discussion there is on Hacker News and reddit about peer-to-peer technologies for things like video conferencing, publishing and realtime data collaboration.
This whole NSA thing should be enough to push things like named data networking/content centric networking, WebRTC and meshnets (http://www.reddit.com/r/darknetplan) into the mainstream.
I believe that we are going to move away from a lot of traditional ways of doing things pretty soon. For one, obviously centralized servers have to go. Replace that with encrypted, privacy-focused data oriented networking.
Replace centralized internet backbones with wifi meshnets, private laser systems, even private fiber networks, etc.
Its very nice to have a simple programming language and API available with things like Webkit and Mozilla browsers, but that still puts too much power in the hands of a few companies, and locks us into particular implementations, and also traps us within the browser.
Programmers and businesses do need easy-to-use, cross-platform and cross-device APIs and programming languages. I think we can port a lot of the Web APIs to a language-neutral, binary-neutral, implementation-neutral semantic representation. Dump as much of the unnecessary complexity from Web APIs as possible.
Take a look at what HTML/SVG etc. provide and re-engineer a more general purpose solution for representing all kinds of things _including_ software source code that is based on truly semantic representation inspired by description logics which separates presentation from display/editing.
WebRTC without a signalling server: http://blog.printf.net/articles/2013/05/17/webrtc-without-a-...
Not to mention that a STUN server is still needed for NAT traversal, as the author acknowledges.
You're damn right. It wasn't even a few days after this whole thing broke that I started working on a simple, distributed and automatic file synchronization program. No third parties and open source.
I was absolutely stunned to find out that nothing out there meets those criteria (despite my best efforts). The only thing that was close was git-annex assistant, but I don't want to use git for simple file synchronization (particularly for large files).
There's also Bittorrent Sync, but it's completely locked down. :-/
Everyone can spend their time working on their own solution but no one would use it as everyone's work will be incompatible with each other. This means no one (or rather, not enough people) will use any solutions that are developed and revert back to something like Skype, FB message, etc.
I think what we need is to establish a foundation that coordinates the development efforts for these secure and reliable software packages with a few centralized projects that become a standard (here's hope for WebRTC).
[EDIT] I forgot one of my most frustrating nits with unison: you need to have the same version on both ends. It's solvable by building the right version yourself, but it's damn annoying.
Plus, I am not fond at all of its interface/configuration. Very complex.
I solved that problem, using unison, I guess near 15 years ago the unix way, not the windows way.
Its been awhile but with a little symlink fun you can make a little shell script that builds a partial mesh and syncs both the script itself, its list of possible mesh members (as opposed to the 3 or so I used) and all the other stuff, more or less silently run out of .bash_profile or .bashrc or whatever it was. There's already excellent tools in unix for scripting, automation, batch, etc, so no need to include them all into one tool.
The windows way, would be to put all those tools into one (buggy) executable with a gui and an installer that also installs browser toolbars or whatever.
I switched to git away from unison and never looked back. Eventually it got to be too big of a PITA to install 2 or 3 debian packages of different versions of unison all of which can only talk to certain other versions. So then my shell script had to detect which unison versions are installed on this node, and match up to the master list of other nodes and their versions, ... oh forget all this garbage just try this new subversion svn thing. And later, forget this whole script, just put everything important thats small in git and everything thats big in multiple AFS servers running on RAID.
This really surprised me; for some reason, I assumed it was open source. Since the storage is entirely peer-to-peer, I wonder what their business model is?
[EDIT] This was all I could find.[1] Which isn't much.
[1] - http://forum.bittorrent.com/topic/8816-will-syncapp-be-open-...
Regarding specifically BitTorrent protocol specification[0], it's free to use. Anyone can build their open or closed source BitTorrent client without reverse engineering.
Then any changes you make to its folder will
automatically be committed to git, and synced to
repositories on other computers.
I guess that means I'm out of legitimate excuses? :P
The _changes_ are tracked (addition, removals, etc.) not the content.
Years ago I worked at a startup which made a distributed, p2p automatic file synchronization system. The startup didn't work out, but the system (mostly) worked. We spent a lot of time trying to solve things that are not largely solved, so if I were to start over today, a lot of the hard work would be done.
To be more specific, if I were you, I would start by using the p2p connectivity and data channel code built into WebRTC (at webrtc.org) or libjingle (at https://code.google.com/p/libjingle/). This is what AeroFS (at https://aerofs.com/) is doing.
By the way, writing a distributed file sync system is not that simple, even though it might seem simple at first. Things get complicated really fast. I think that ones reason one of these hasn't taken off is because it's really hard to get right. But, I still hope it will happen one day.
Oh, and there is http://camlistore.org/, which is an open source project run by a really smart Googler. It's not specific to file syncing, but it has a powerful foundation. A combination of the p2p bits from WebRTC and the distributed data bits from camlistore would be pretty powerful. It makes me wish I could find some more 20% time :).
(Full disclosure: I work at Google, often on the WebRTC code base, and am the maintainer of libjingle)
I've looked into camlistore before (particularly since I'm doing this project in Go), but I've been overwhelmed fairly quickly.[1] From the description on the web site though, I do get the sense that it would be incredibly useful---but it feels so heavyweight. My problem is that I think I can get file synchronization working in just a few thousand lines, which is a considerable impetus to chug along without resorting to such a huge dependency.
Re libjingle: it looks like an amazing project. But to use it, I would have to violate my so-far-lifelong-commitment not to use C++ (I'm at 11 years now). I will readily admit that whatever I come up with won't even be a fraction as capable as libjingle. And if that doesn't end up being enough, my project will fail.
Thanks for your comments though. At the very least, it may provoke me to switch tactics once I become a little less ignorant.
By the way, what do you mean by "not to use C++"? Do you mean write it in it, or do you mean use libraries written in it? I can understand the former, but the latter seems a bit extreme. Go makes it pretty easy to link to C libraries. Why not use libjingle from Go? That sounds fun and a lot less work.
Those are exactly the things I intend to leave out in my initial work. If it happens to catch on, hopefully I'll get some help. If not, it will still be very useful to me without those features.
> Finally, you'll want to add encryption in there; luckily, Go has good crypto libraries, but getting crypto right is still really hard and it's easy to mess up and have an insecure system. At the very least, you could use libjingle as your guide.
Yup. I think this is one of the things that Bittorrent Sync gets right: strong symmetric AES encrpytion.
> By the way, what do you mean by "not to use C++"? Do you mean write it in it, or do you mean use libraries written in it? I can understand the former, but the latter seems a bit extreme. Go makes it pretty easy to link to C libraries. Why not use libjingle from Go? That sounds fun and a lot less work.
Not to write in it. In the Go community, there's a heavy bias to avoid linking to external libraries if it can be avoided. There is a certain amount of cachet that comes from being able to do `go get ...` and have it work without having to make sure another library is installed.
I'm at the infancy stages of my understanding, so it's certainly possible that I'll come to my senses and build on top of something else. But for now, I'm going to blaze my own trail. At least partially to learn. (Two of the most important pieces---encryption and p2p---are things I have very little experience with.)
C: http://nice.freedesktop.org/wiki/ (glib-based so wrapping friendly to other gnome-supported languages)
This is a pretty bold statement, and I'd like to hear your reasoning for it.
I've always seen that centralized is better than decenteralized, just because its easier. Which is why when we see github go down, some teams can't continue to work even though git itself is decentralized. One click solutions, and decentralized applications tend to not go hand in hand.
I'll agree, if we want to maintain privacy these things will have to be decentralized. However I have a hard time making the case that decentralization will become the defacto standard. Even the web has a psuedo-central hub - DNS.
> some teams can't continue to work even though git is decentralized.
I think you mean some teams won't continue to work because they don't know how to use git. If the de facto center goes down, a team is still able to use git to share changes with each other.
And IPv4/6 is decentralised, DNS makes it convenient for regular people to use. I would say DNS is decentralised as well, just the allocation of domain names comes from a few sources.
To take down a decentralized network with a bug you'd have to have every user on the exact same version and no way for them to simply role back to the previous version. I.e. nearly impossible.
Combine this with auto-updating clients and you've got a crashing software bug that brings down the whole network.
Agreed however that the centralisation point effectively still exists thanks to automated updates.
On a fully distributed network, that's pretty hard to do. DDOS works by having tons of nodes to attack with. That won't work very well when your target has as many or more nodes than you do.
Taking down a centralized target is vastly easier.
>Combine this with auto-updating clients and you've got a crashing software bug that brings down the whole network.
Only if I can't role my client back nor turn off auto-update.
I've tried multiple times to engage on this topic. From my experience, the people set of people who are interested in these topics for "freedom" reasons and the set of people who are technically informed enough to contribute even minimally to technical discussions are non-overlapping.
Certainly not speaking about myself, but that's the sense I get from talking to people whose interests overlap like you'd hope.
The people who hide away are the same as the people who want you to sign an NDA before they talk about their startup because it's just too revolutionary and cranks who invent new crypto algorithms in their basement.
DS design is extremely complex and requires the coordinated effort of a lot of really smart people to get right. Anyone building these systems in secret just has no idea how terrible they are at it.
The response on HN has been an order of magnitude larger than the response from any person I know. Of everyone I know, I'm more concerned about the NSA spying than anyone else, and I'm less concerned than most commenters. I begin to wonder if this isn't some kind of nerd-sniping paranoia-groupthink.
If all of your friends were jumping off of a bridge, why wouldn't you? If you don't trust them in what they do, why are they your friends?
The minority of people who litter make the experience of walking/living/sitting in that area less pleasant for everyone else, and they increase the likelihood that others will litter ("this park is already full of trash; should I really go chasing after that wrapper that fell?"). Alternatively, you could not make it worse, or you could even make it better by picking up a few things and throwing them out as you pass through.
The minority who don't vaccinate encourage others to avoid vaccinations, and it doesn't take very many of them to undercut or even nullify the benefits of population vaccination (like protecting newborns who can't yet be vaccinated).
Etc. etc..
Do you want to be worse, meaner, less moral, etc. than the average? Or do you want to strive for a little more than that in life?
Realistically, most of us aren't going to have a huge positive or negative impact on the world; but thinking locally, if you are going to seek a mate, friendships, etc., you'll probably do better (and be happier) in life if you at least try to do better than the average.
You may even get an opportunity to make a seriously good decision somewhere along the way, if you have good moral habits and keep your wits about you.
But if people do care about something, they are motivated to act, and may significantly alter the course of what happens -- not always, and not always as they wanted, but even a small number of motivated people can make serious waves, depending on their skill, charisma, cleverness, opportunity, etc..
Hence, what we feel matters.
But that's a very wrong way to approach things. You have to connect the dots and see where this is going, before it's too late to do anything to stop it.
Plus, I bet most don't even realize that even if authorities don't come knocking at their door because through their spying they saw they downloaded that last Game of Thrones episode (how long until that happens, anyway?), they could still use the spying to influence judges, politicians, mayors, and so on - which will affect their day to day lives, depending on the decisions those people take through blackmail.
But as I said, most people can't connect the dots like that. Ignorance is a bliss, as they say.
The first and easiest argument to make about this is the chilling effect it will have on journalism -- even some things that are unclassified and should be public knowledge can be prosecuted as whistle-blowing.
Or, decentralization in the face of a very specific threat (e.g. MAFIAA and piracy-optimized things like BitTorrent, and FinCEN and things like Bitcoin).
Until recently, there weren't widely-believed threats to general Internet use.
WebRTC depends on things non-home internet filters out or prevents, so it is useless. The way to get around that? Use central servers and make your traffic look like HTTP. Heck, even STUN protocols admit there are cases where it won't work, but WebRTC fans seem to think they'll always be able to get a P2P connection. The technology is just naive as far as I'm concerned. Something that only works on home internet and a few open networks is worthless.
Mozilla is looking at making a foray into this with its "Firecloud" initiative[1]. While Firecloud itself appears to be a relatively innocuous endeavour, it would lay the infrastructure for P2P RTC-based meshnets, which are something I've been hoping to see for a while now.
Well, this is exactly why I am a huge fan of technologies like MOAI (http://getmoai.com/) where the VM is under the control of the implementor, whose efforts to put the working VM on Platform-[X] results in a single codebase that will run on all Platform[s]. In my opinion, it pretty much matches your requirements, from the simple language, easy to use API, and more importantly: control over the host environment.
The only issue is that competence in MOAI in developing a cross-platform truly 100% compatible code base means you have to scale your attention from the up-front App, down to the lower-level host code. Or else. But thats 100% of the fun, imho ..
Note that this is just one of many such solutions to the issues you describe, and it is indeed a fertile playground for anyone with as much affinity for Lua - and similar tools - as one must have, to suffer the platform wars pleasantly, these days ..
This isn't so much of a technical problem as a political one. Although, in principle I agree- I would much rather have p2p so I could at least have a fighting chance via encryption.
The implementation at webrtc.org gives you cross-platform, cross-device source code in C++. You can build any API on top of that, in any language you want. It's already really easy to use if you an to use the HTML5 WebRTC API built into Chrome (and the same API in Firefox).
It seems to me that you have everything you want, or at least all the pieces of the puzzle. Someone just needs to do the work of assembling them the way you want them, and then of convincing the people with whom you want to talk (your friends, family, or business associates) to use your securely built communication system. Technology-wise it's possible, even fairly easy if you know what you want. But the social aspect (convincing others to use it) is more difficult.
(Full Disclosure: I work at Google, and I have worked both on Hangouts and WebRTC)
I was talking more generally about Web browsers and web APIs, not specifically WebRTC. I still see the web platform as currently the most open, practical solution we have.
However, the reality is that Google IS still a business. And so quite a lot of this key infrastructure, although it is open source, is still being packaged up by a giant private company.
Yes, we can use the web platform including things like WebRTC to do what we want, but I believe the the medium and long-term future will move away from specific implementations by a few companies and towards open machine-processable specifications.
I believe that the way forward will be to move away from textual programming languages and into truly semantic representations of programs, protocols, and implementations. Obviously it will be many years before this is mainstream.
"P2P is harder to build" is an equally loaded statement, though you've caught my intrigue. I wrote a WebRTC signaling server in a weekend that is capable of letting peers establish N:N connections with each other for video conferencing.
Outside of cases where TURN would have been necessary, the clients were all connected, P2P with encrypted connections. (And even with TURN, I believe end-to-end encryption could be ensured, but the traffic would go through a third party)
WebRTC makes P2P a lot easier, especially since it doesn't require installing software. I think getting peer connectivity is 10% of the problem. Video conferencing is pretty much a best case for P2P; general apps are much harder. Things like data sync and handing churn are very complex and not needed in client-server architectures.
* Avoiding things like the supernode crash from December 2012.[1]
* Supporting the growing mobile share, which can't contribute as supernodes.
[1] http://blogs.voxeo.com/voxeotalks/2010/12/23/skype-outage-i-...
>
> these devices are a lot different: they're running on battery, sometimes on WiFi
> but often on expensive (both in money and battery) 2G or 3G data networks,
> and essentially "off" most of the time.
> On iOS devices, applications are killed and evicted from memory when they attempt
> to do too much background processing or use too much memory.
> On Windows RT and Windows 8 Modern applications, when the application is not in
> the foreground we only get a few seconds of CPU execution time every 15 minutes
> and again, strict memory limitations if we want to stay loaded.
> And when the Skype application is unloaded, it can no longer receive incoming
> calls or IMs, rendering it a lot less useful.
>
( i work on verbs.im, we're a team of 4 now, here's a link to our privacy policy http://includetech.co/privacy/verbs)
The server becomes a proxy for your account, and being able to send push notifications.
Your apps become thinner clients needing lesser battery, bandwidth.
Infact we've got a step further, and replaced XMPP/HTTP with MQTT to save even more battery.
( here's a nice article profiling power consumption of MQTT vs HTTP here across wifi/3g: http://stephendnicholas.com/archives/1217 )
That being said, the recent iOS 7 have introduced API's for longer running background apps
~B
But when one computer calls another, and neither are behind NAT/firewall/etc., is the actual audio stream going directly between computers, or are they both sending the actual audio to/from the Microsoft servers?
Obviously Microsoft could choose to redirect it through their servers whenever desired, but I assume that audio communication is still direct, whenever possible?
I can see a feasible argument for this when you're talking about routing it through your WAN, but if you're that concerned about internal security then you should roll your own voicechat solution that you can keep on your own self-hosted (non-cloud) internal servers.
I cannot understand why none of the alarmist articles about Skype (some even by so-called "cypherpunks") has ever even bothered to run such a simple check.
FWIW, I ran my own simple test using netstat when Skype first centralized the supernodes. I walked away satisfied when I still saw primarily P2P activity during voice/video calls to an IP that belonged to the remote party. (It used UPnP port forwarding to traverse the NAT/firewall, if anybody is curious.) But then, I'm the trusting kind and dug no further. And it's been a couple years since. If there truly are "hackers" on here, I encourage them to try their own experiments and share them.
It was bought by MS in 2011. It's literally the last couple of years that people are concerned about. It's recent changes that have moved from distributed to central routing.
From the OP article (apparently by Skype [past] principal architect):
>"[...] people who were concerned about privacy -- including many of us in the security industry -- used Skype for secure communications.
>Both eBay and Silverlake maintained this architecture, as well as the Luxembourg HQ.
>Since being acquired by Microsoft, however, the service has been re-architected to run through MSFT-owned servers, rendering encryption functionally meaningless and making it just as easy as POTS to monitor.
>None of this -- neither the $7.5B acquisition itself, nor the decision to move to a datacenter model -- make strategic or business sense for Microsoft as far as I can tell." //
Back then my conclusion was that actual voice / video traffic was still direct P2P, although it kept connections open to other IPs, probably for signaling/stun/relaying. I haven't had a chance to run the test again because, well, I'm on a mobile device :-)
But my bigger point was that on a forum ostensibly for hackers, there should not be so much speculation and innuendo about something that can be pretty easily tested.
Prism slide: Skype: 2/6/11
http://en.wikipedia.org/wiki/File:Prism_slide_5.jpg
But "Microsoft to acquire Skype": May 10, 2011
The only thing that seems related is "and other nice features like spam filtering and malicious URL removal" which he briefly mentions, but I don't find it a good enough reason to drop encryption and make it possible to intercept communication. Seems like an awful tradeoff.
Basically, the perception is "if you don't send the message when all of you are online, really weird things happen".
White paper: https://bitmessage.org/bitmessage.pdf [pdf link]
https://www.listbox.com/member/archive/247/2013/06/sort/time...
This is not some revilutionary idea. It's getting back to the original design.
Small scale peer to peer networks can still be achieved, easily, even in the presence of "firewalls", "ISP's" and "NAT".
You need a least one reachable IP address to be able to pierce NAT and "bootstrap" the network. Easy.
You need to do some simple encapsulation. Again, easy.
You need an OS with a TAP interface. Easy? iPhone denies programmers the TAP device. And any other company controlling a mobile OS could easily block your peer to peer networking capability by interfering with your access to the TAP device.
Maybe it makes more sense to distribute your peer to peer solution as part of an open source OS that will run on many hardware architectures than to distribute a solution that purports to run on all the popular OS's.
Maybe "apps" don't have to run on the OS that does the peer to peer connection. Maybe they can run "through" it instead: point your router/gateway settings at this open source OS, running on some small form factor pocket-sized computer like the Pi, and voila, you have a decent simulation of the "original" end-to-end Internet that you can use with people you know (the original Internet was among people who knew each other).
The signaling is open, so it is mainly up to the developer to decide the topology they want to enable: p2p, mesh, centralized MCU, hub-and-spoke, overlays, etc.
However, in many cases you'd have to use a TURN server to communicate from within very restrictive firewalls. This turn server could possibly run by anyone, including you. Just need to call the appropriate API with your own TURN server.
I run restund (a open source turn server) on an amazon instance and route my packets via that (I also remove all other ICE candidates and just keep the TURN).
They asked "how do we fix this" and sadly the answer was generally (if you were on a mac), you don't, just shutdown your client when you're not using it and it won't become a supernode. It wouldn't really have been a big deal if it was 5-10 connections, but it wasn't so it became a problem.
I assume eliminating the crypto has to do with eliminating overhead, but it could have to do with DPI and making it more firewall friendly.
People with a clue should no longer be using Skype.
http://arstechnica.com/security/2013/05/think-your-skype-mes...
In the case of
instant messaging, we have merged the Skype and Windows Messenger message
delivery backend services, and this now gets you delivery of messages even
when the recipient is offline, and other nice features like spam filtering
and malicious URL removal.
Even comments here indicate people clearly didn't read this email or fail to understand it at all.
I am unaware of a way to ensure privacy but still have a third party have the ability to read your messages. However, I'm open to ideas.
The location of the "supernodes" (on client machines, or in a datacenter) is separate from whether encrypted traffic can be read by a third party. Using a totally decentralized system makes it a little harder to eavesdrop, but it's entirely possible to develop a completely secure system in the centralized model.
That said, I've never been terribly impressed with Skype's security model. Even before Microsoft bought them. It might have been decentralized, and the traffic might be encrypted, but the encryption lacks a strong trust model. There's never been any way to make sure that your connection hasn't been MITM'ed, since the system doesn't provide any way to verify the identity (key fingerprint) of the person you're talking to.
The current encryption is fine if you're worried about rogue network administrators, but has never been sufficient if you're worried about being targeted by state actors.
For Skype's security model, I thought that before MS bought them they had E2E encryption (sure, it can be MITM'ed, but the conversation's that's not MITM'ed cannot be decoded easily, whereas they could be now).
(Is it fair to say that I verify _some_, not all, of my friends' identity via the way the talk? I feel like if I have E2E encryption, I would be fine as a MITM would have a pretty difficult time faking how some of my friends talk.)
E2E encryption is only as secure as the key exchange, so if the central Skype server gives you and your friend the keys you need to encrypt/decrypt then just passes along your E2E-encrypted packets... anyone who was able to snag the keys and has access to the encrypted packets can decrypt everything just as you can.
Is there any evidence that they removed E2E crypto for voice calls? People keep saying this but their reason seems to be "because MS".
You and I have a secure P2P connection (aka Skype). But my client and your client check hashes of URLs against a third party service before transmitting the message to the other person.
There, you can blacklist URLs and it doesn't send any message contents to anyone (only hashes of ONLY urls) and then sends the actual full message, encrypted on the wire.
It's not hard to conceive how this system would work.
Like I said, people still don't get it.
Supernodes are a directory service. "Traffic" as in text and video traffic rarely, if ever, is transmitted through Supernodes.
"In the case of instant messaging, we have merged the Skype and Windows Messenger message delivery backend services, and this now gets you delivery of messages even when the recipient is offline, and other nice features like spam filtering and malicious URL removal."
And wish to add to that, other nice features like "censorship and loss of privacy."
I remember when I ditched MSN in favor of a what I thought was a secure communications solution, I waved goodbye to Microsoft. Here I am again, my messages routed yet again through their backend.