Revised and much faster, run your own high-end cloud gaming service on EC2
lg.io
lg.io
It was a fun little project to get it all working and it was playable but barely.
The added latency makes you feel a little floaty (like a bad VR experience - the kind that make you sick). The witcher looked good when you stood still but not while moving. Ark's performance was abysmal on the aws hardware such that I had to turn down the settings.
Part of the bad experience is likely my ISP's fault - I had spikes of packet loss - though I think this is fairly typical.
The cost is mostly not in the spot instance (that /is/ cheap) but in the storage (EBS) that you need to keep around to mount a spot instance whenever you want to play. The bandwidth cost is non-trivial for the amount of data you're streaming. I had about a 40 euro bill for the month under somewhat light use.
In the end, I can't say I recommend it. Onlive and Gaikai didn't do so well. That might have been their business models but I believe the tech is equally to blame. It's a ridiculously hard thing to get right consistently.
It's probably better, and not more work, to create a security group that only allows:
* UDP on port 1194 (Openvpn server) * TCP on port 3389 (Remote Desktop) * ICMP (for ping)
you're using steam in-home streaming through the vpn.
These rules restrict the ports listening directly on the internet. The average gaming PC is behind a home router, has no ports reachable from the internet, and network games still work fine.
If your college was like mine, what they did was filtering outbound connections, which is different. Outbound traffic is unfiltered by default on EC2, and it's fine to leave it that way.
There's one problem though, and it's latency, even 50ms will feel very laggy. We need more decentralized data centers! With a data-center in each city you could get latency down to less then a millisecond.
I think the next digital revolution will be low latency, and a flora of new services coming with it.
If your mouse cursor or terminal was continually an eighth of a second behind your input, you'd get pissed fast.
Anyone remember this?
e: Found it!
http://www.pcmag.com/article2/0,2817,2464341,00.asp
https://news.ycombinator.com/item?id=8210957 (comments on actual paper)
This is the relevant technical details from the pcmag article:
Microsoft's DeLorean system takes a look at what a player is doing at any given point and extrapolates all the possible movements. It streams a rendering of these from a server to a player's console. Thus, when a player decides what he or she plans to do, that scene—for a lack of a better way to phrase it—is already ready to go.
> ping -c 4 sdb.amazonaws.com
PING sdb.amazonaws.com (176.32.102.211) 56(84) bytes of data.
64 bytes from 176.32.102.211: icmp_seq=1 ttl=239 time=7.35 ms
64 bytes from 176.32.102.211: icmp_seq=2 ttl=239 time=7.51 ms
64 bytes from 176.32.102.211: icmp_seq=3 ttl=239 time=7.11 ms
64 bytes from 176.32.102.211: icmp_seq=4 ttl=239 time=7.15 ms
--- sdb.amazonaws.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 7.119/7.285/7.512/0.180 msInput > PC > amazon > PC > monitor
Maybe there's something in your PC that is adding to the lag, like a slow software render.
It could also be that the machine answering to the ping is closer due to anycast routing.
Can anyone come up with a practical way to measure the actual lag from input to screen render!? For example using a high speed camera.
The decoding was in hardware, and I have dual amd 7850s, so I don't think that was the issue (and I wasn't trying to run at the full 4k either)
There was a very noticeable input -> display lag, according to telemetry on steam it was ~40ms total, which is fine for a lot of games, but really noticeable and annoying for something like gta. I mean I've played civ5 over vnc before, and 40ms would be a godsend compared to that, but it was still more than playable.
On my home internet connection, pinging, for example, google.dk gets me a response time of a few milliseconds, and a HTTP GET request for the root URL has the same latency. But if I do a HTTP GET request as part of the search, the latency is much higher.
I think the google.dk/com main page (and probably other heavily visited sites) are cached by ISPs, so that you don't necessarily reach Google when you ping or HTTP GET the root domain, but rather some network cache device between you and your ISP.
So be careful trusting that ping latency necessarily equals HTTP GET latency, or latency for some other request, to a server.
While I would hate to be locked into an ecosystem that was pay-per-usage, this article is just amortizing the cost of hardware over time.
The PC gaming community is (I hope) fairly intolerant of any sort of lock-in. If Steam went the way of pay-per-hour there's enough competition that we'd see a transition to other services.
Unless Valve goes through some other route to give themselves an advantage (e.g., buy Xfinity Valve package and get unlimited access to Valve Streaming services, the rest of the internet is low-bandwidth/high-latency), there's not much to worry about.
> Steam
I've got no problem with Steam, but it's a pretty locked in platform already
"Why on earth do you need more data then this!? You know downloading movies is illegal, right!? Ohh, and you will also get a free Ipad."
The problem is that people refuse new technology like fiber networks, because they do not see any use-cases beyond e-mail and Facebook.
My depressing thoughts about ISP's is that the only reason they use fiber instead of copper is that fiber is much cheaper over long distances.
It failed spectacularly.
For strategy games and such, I'm sure streaming will always be viable. As we move into the VR realm, even a tiny amount of latency won't be acceptable. So we're always going to need local rendering for action games.
A solution like this is a clever hack, because it's taking advantage of an immense economy of scale provided by a company with really deep pockets operating on a multinational pool of servers that still on balance see a wide variety of different use cases, allowing for a lot of distributed load.
Imagine if every user was demanding that same level. As I write this, Steam is counting over 8 million players logged on. Now imagine trying to imagine trying to guarantee them all gaming-level real time performance (fun note here: that GRID card used for the AMI costs $2,000 alone), and doing it all on a reasonable price.
Even our Amazon hack isn't doing that. 50 cents an hour doesn't sound like much, but the average gamer is putting in 22 hours a week. Some five million enthusiasts are regularly pulling 40. That's anywhere from $40-80 a month. Not to mention the cost of the games themselves.
OnLive was trying to offer this for $15 a month. And originally, they were even footing the cost of the games, aiming for a Netflix/Gametap approach (the latter of which also failed, I might add).
Unsurprisingly, they went spectacularly broke.
Something like this will only be economical if you really plan on only using it for the occasional single player game a few times a year.
If you use it to play games regularly at even modest frequencies (20+ hours per week), the 50c per hour will quickly start to accumulate and become a recurring monthly bill of $40+.
At that point you might as well invest in a decent gaming desktop and stream from it instead. A i3 + 750 TI based system can match the performance of this setup and would cost less than $500 all-in, which is about the cost of 1 year of streaming from this setup at 20 hrs per week. You'll get a much better experience due to the much lower latency and not having to worry about penny pinching on every session.
Are you sure about that?
However, the overall gaming experience you get with the local streaming setup will probably be similar to, if not superior to, the remote streaming setup regardless of absolute hardware power, because i3 + GTX 750 TI can definitely handle most games at 60fps in 720p (and a lot of them at 1080p in my experience). So the comparison ends up between streaming 720p with 50ms latency due to bandwidth constraints over WAN vs streaming 720p (and some 1080p) with <5ms latency within a LAN.
I had always assumed that their utilization rates would be much higher than 2 users per physical CPU, even for higher end instances like this one.
The Nvidia GRID GPU almost definitely is shared as well, since that's the entire purpose of the GRID SKU:
http://www.nvidia.ca/object/grid-technology.html
According to this, the K520 supports up to 16 concurrent users while sporting 2x GK104 based GPUs of power. When fully utilized I figured each user won't be getting much more power than a single midrange GPU, and I can't imagine why Amazon would choose to not keep them fully utilized either. If you have any sources otherwise, I'd love to see them as well.
Regarding the CPU, when creating an instance you can see this: "G2 instances are backed by 1 x NVDIA GRID GPU (Kepler GK104) and 8 x hardware hyperthreads form an Intel Xeon E5-2670". According to [Intel's product page](http://ark.intel.com/products/64595), the processor only has 16 hyperthreads, so 2 users per CPU. My reasoning may be wrong though, I'm not a virtualization expert at all.
IIRC Carmack had written several posts on how they are chasing and killing latency in everything from USB input to LCD front buffering.
Think about getting a VR headset like the Oculus and the only thing you needed was to plug it into the network, and then have access to virtually all games available at a hourly fee!?
I was a little surprised by the cost as well. At the rate that I'm gaming these days it would be like $10-20 per month, that's pretty damn good (price of games not included obviously).
Say you allocate another 50 cents an hour for software then you are getting to a $20 or $40 a month spend which is getting in the range of a good video game habit. This leg could support sales of an AAA product and also could probably fund AA games.
Get your configuration right and you have a system for multiplayer games with pretty low latency behind the servers -- even if the video gets screwed up, the games will always stay in sync.
It is no way an accident that the costs wind up like that because everybody else is thinking about "Can I replace $X spent on hardware with $Y a hour spend in EC2."
I wrote a simple script (https://github.com/zachlatta/dotfiles/blob/master/local/bin/...) to really easily spin up and down the machine I set up for game streaming.
But don't they pull the instance out from under you if someone outbids you? Does anyone have experience with that?
And one more question - how is the performance? The OP shows screenshot of game running in 1280*800, but that might be because of the macbook resolution. Can it do fullhd or 4k?
Does anybody know much about the EC2 hyper-visor schedulers or in the case of large instances, does it even run with a serious hypervisor?
However, one thing I would be extremely wary of is running your Steam account from AWS or any other server environment. The last thing you want is to get flagged as a bot or VPN abuser and banned; Valve customer support isn't exactly known for being particularly understanding or responsive. Personally I would just load up a throwaway Steam account with a few games and use that.
Home Streaming also works really surprisingly well, although it feels weird to play GPU-heavy games like Shadow of Mordor on my dinky little Macbook. With an XBox controller, no less. (I have the "unofficial" Mac drivers installed, but I'm not sure if Steam uses that or has its own XB360 controller support built-in.)
Steam's inbuilt streaming is fantastic for spectator use, it's just not interactive.
I wonder if you could set up the tunnel going the other way so that the EC2 instance goes through your home network to access the internet?
Sony et al probably have more specific patents for their own setups, but to the extent they cover this setup they'd either be obvious, or they'd be trying to sue you for doing stuff covered under nVidia or Valve's patents. This, alas, doesn't necessarily protect you, but would certainly raise some PR issues. Plus Amazon might have some questions as well, since they don't want people getting hit for using cloud services to do things; anything that smells like special cloud licensing just because you're doing X "but in the cloud!" is going to hurt their business model.
I can't guarantee they have nothing to sue over, but the costs/benefits would not seem to argue in favor of Sony suing.
On top of that hardware costs vs performance along with hardware h264 encoding is getting cheaper.
Almost no one will pay for laggy games at last gen resolution.
The thing that I would worry more about when running a service like this would be liability for what users of your service do / say.
If anybody out there seriously starts working on such a service, let me know!
My rig is about 7/8 years old and I've only upgraded the video card when I had to. I'm looking at building a new rig just to keep up and a "cheap" one comes out to $800 for me. Something like this would be very desirable. I would love to be able to shoot this to my main big TV and only using my Chromebook to access it.
I've been using it to set up a site-to-site VPN between my home network and my Azure VMs and in my experiences the performance has been quite good. They also claim to have higher max throughput than OpenVPN but I haven't yet verified those claims myself.
While I respect and find the technology fascinating and cool, it feels like leasing a car I'll never own versus owning and evening buying a new one every few years at a much lower price. For those who game a few hours a week however, I can see this being a cheap alternative to a gaming rig.
I would have to think most people with a job and family might struggle to find 3-4 hours a week to game. No offense intended by the above. I'm quite envious :).
Sometimes, I think about that quote in that movie Office Space, when Peter Gibbons said, "You know, I've never really liked paying bills."
I'm not sure that I understand the economics of the spot option from Amazon's POV since it seems that a person who needed persistent servers could set their spot max price at the same price as an on-demand instance and always be ahead by doing it that way (with the caveat that there would be a short interruption if forced to switch to on-demand).
For someone who needs cheapest possible compute power with a flexible schedule, spot arbitrage makes sense of course.
Not if you need persistence: the spot price can go above the on-demand price. So if you set your max spot price to the on-demand price your instance might get terminated.
People have played that game before and lost. Spot pricing can and has gone above the on-demand prices. It is normally a great way to save money, but there's no way to avoid the possibility of having your instance terminated. If that's not something you've built around then you have to stick to on-demand (or reserved).
eu-central-1b: $0.1053
In Australia the speed of light actually matters for the distances light needs to travel - this is probably the same for mid-west towns not near an AWS centre too (not sure of their presence there, but I expect they are predominately on the coasts).
The speed of light is 300,000 kms per second.
Distance wise: Perth to Sydney approx 4,000kms or 13ms travel time, and Brisbane to Sydney approx 1,000kms or 3ms travel time.
I'd say it is highly doubtful to get sub 10ms from Brisbane to Sydney if you're going via even just a handful of routers. And Perth to Sydney? Forget it!
The standard deviation of any ping test I do is at least 3ms due to how DOCSIS works, and I can't ping my next door neighbor (same ISP) through the internet in under 16ms. I've done traceroutes against several servers that are in the same city as I am in North Carolina and the only one where my traffic didn't first go to Atlanta or DC (or both) was to a server hosted by an ISP that has no physical presence outside of North Carolina. My cable modem and my ISP's CMTS each have out of control bufferbloat that adds hundreds of milliseconds of latency under load in each direction, which can't be entirely mitigated by my router's traffic shaping and AQM. There's little reason to believe that they've got got any AQM further upstream given the large latency spikes I see even when my last-mile link is quiet. Disregard for latency is pervasive in the design of the ISP's network.
Else I would say that it runs super well! The input lat is ~15ms, ping is ~30ms and display lat is ~30ms, reported from the steam client. I have no trouble whatsoever playing FPS games at all, or any other game for that matter.
Admittedly, it's not exactly the same as the setup in this article since it's not streaming over the internet, but I would assume that as long as your latency is low and bandwidth is high, you'll have no problem with this setup.
Game server <--> Amazon <--> Home
From a pure latency point of view, this is probably not an issue. However, whatever events you see on screen, the game server sees you reacting slower, because your keyboard basically has a delay of 20-30ms (Home --> Amazon).
Maybe, it depends a bit on the game you play, if this has any negative consequences or not.
Edit: This could be somewhat compensated by the faster connection of Amazon <--> Game server (just assuming that it's faster than a connection from home).
Anecdotal evidence: historically I have a 60% winrate on US West servers (~40-70 ping) and only 50% winrate on US East servers (~110 ping). I've played on EU servers and it is absolutely miserable at 200+ ping. If I had to play regularly on anything above 110 and I'd probably drop to 45% or 40% winrate.
After a certain number of games you get used to it and start anticipating. It's not perfect, that's for sure, but it's definitely doable. Or are you talking about 5k+ MMR? In that case it is trickier, of course, but I know even pros managing to do it :)
It uses the same in home streaming feature of steam.
For me (London) it looks like I could get a latency of 28ms if I hosted my rig in their Dublin datacenter, which sounds OK. (But obviously this would be on top of the ping to the multiplayer host, if you're playing online.)
This site has a much larger number of providers:
http://cloudharmony.com/speedtest-latency-for-compute-limit-...
But of course most of them don't offer GPU servers.
The post suggest ~60FPS limit, which isn't competitive really at all. 120-144FPS is the standard for competitive play currently. If you encode at 144FPS, and decode at 144FPS then you already have a GPU that supports competitive play, so cloud platforms become moot.
Previous holder of the title was spending terawatts of power bitcoin mining.
DO / Linode will mostly be cheaper and proooobably offer a better price/performance ratio. You will however get a lot less options (especially GPU wise)
Let’s take your average (okay, this is really beefy, but I couldn't find something lower at most dedicated hosters) gaming system: 8 Cores (16 Threads), 32GB RAM, 2TB HDD. At Digital Ocean, you pay approx. 320$/month for 12 Cores, 32GB RAM, and 320GB SSD
Let’s take your average small hoster, in this case Hetzner, as comparison: You pay 55$/month (plus 55$ setup) for 8 cores (16 Threads), 32GB RAM and 2TB HDD.
I’m just showing this as example: Using Cloud hosters for stuff that is running 24/7 is not cost effective, especially if you always need the same performance – use a dedicated server instead, hosters that specialize on that can provide far better offers.
At those prices, it would be a better deal than hosting after around a year given it would continue working and have resale value.
Jackel Gaming Rig http://elitegamingcomputers.com/gaming-computers/#2
http://shield.nvidia.com/grid-game-streaming
But maybe it's not quite what you want, since it's limited to their games. Never personally tried it though.
http://arstechnica.com/gaming/2015/01/playstation-now-review...
http://arstechnica.com/gaming/2015/07/hands-on-with-playstat...
Frame has a platform offering if you want to setup some other virtual world or game.
I guess I should do a Show HN, but it just hasn't been a priority with working out the kinks.
Has anyone managed to script something where you just press a button/run a local script and it does all the work, including saving your image to EBS before you turn the thing off and stop paying for the instance?
This is such a cool idea, makes me realize what other creative solutions are just lurking, ready to slap me across the face.