Instant Cloud – SSD Bare Metal Servers
instantcloud.io
instantcloud.io
ubuntu@instantcloud:~$ dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 11.0101 s, 97.5 MB/s
ubuntu@instantcloud:~$ sudo hdparm -tT /dev/nbd0
/dev/nbd0:
Timing cached reads: 2180 MB in 2.00 seconds = 1090.44 MB/sec
Timing buffered disk reads: 268 MB in 3.02 seconds = 88.85 MB/sec root@instantcloud:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=16 --size=4G --readwrite=randwrite
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.11
Starting 1 process
Jobs: 1 (f=1): [w(1)] [100.0% done] [0KB/18714KB/0KB /s] [0/4678/0 iops] [eta 00m:00s]
root@instantcloud:~# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=16 --size=4G --readwrite=randread
test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16
fio-2.1.11
Starting 1 process
Jobs: 1 (f=1): [r(1)] [100.0% done] [22307KB/0KB/0KB /s] [5576/0/0 iops] [eta 00m:00s]VULTR 2 GB INSTANCE dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 8.11451 s, 132 MB/s
hdparm -tT /dev/vda
/dev/vda:
Timing cached reads: 25106 MB in 2.00 seconds = 12568.51 MB/sec
Timing buffered disk reads: 496 MB in 3.02 seconds = 164.16 MB/sec
DIGITALOCEAN 2GB INSTANCE hdparm -tT /dev/vda
Timing cached reads: 8926 MB in 2.00 seconds = 4468.52 MB/sec
Timing buffered disk reads: 542 MB in 3.00 seconds = 180.43 MB/sec
dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 32.0005 s, 33.6 MB/s~# dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 2.35872 s, 455 MB/s
(yes: it's just markdown here).
This is a 1GB Debian Wheezy instance with regular HDD in Memset [1]:
dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 17.7142 s, 60.6 MB/s
I checked the host and its load is average. DO's 33.6 MB/s sounds poor.[1] disclaimer: Memset hosting is my employer, I built some of this; but this is an honest test (personal VM running several services).
EDIT: formatting
EDIT 2: same test on a VM with SSD disk:
dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.48953 s, 308 MB/s
Again, the host has average load.I'd recommend using fio to do this using libaio and direct disk reads/writes, and IOPing for basic latency tests:
* Random 4k read test for flash storage:
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
--filename=test --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
--readwrite=randread
* And writes: fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
--filename=test --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
--readwrite=randwrite
---Here's an example here is a test storage unit I'm logged into at work right now (NOTE: THIS IS NOT ON INSTANTCLOUD!):
root@s1-san5:~ # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test \
--filename=/dev/md200 --bs=4k --iodepth=128 --size=5G --numjobs=12 --norandommap \
--readwrite=randread
test: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=128...
Run status group 0 (all jobs):
READ: io=61440MB, aggrb=2339.7MB/s, minb=199652KB/s, maxb=200362KB/s, mint=26167msec, maxt=26260msec
Disk stats (read/write):
md100: ios=15636294/0, merge=0/0, ticks=0/0, in_queue=1581403800, util=100.00%, aggrios=7864320/0, aggrmerge=0/0, aggrticks=128880/0, aggrin_queue=131372, aggrutil=100.00%
nvme0n1: ios=7576253/0, merge=0/0, ticks=123812/0, in_queue=125748, util=100.00%
nvme1n1: ios=8152387/0, merge=0/0, ticks=133948/0, in_queue=136996, util=100.
While the test is running is you'll see the storage performance (Note the MB/s in the third [] and iops in the fourth []): Jobs: 12 (f=12): [r(12)] [12.3% done] [2537MB/0KB/0KB /s] [650K/0/0 iops] [eta 00m:50s]
And when it's completed:* Throughput: aggrb=2339.7MB/s
* IOs (Read in this case): ios=15636294/0
And then with IOPing to test latency:
* Here's an example of really bad storage latency on my crappy old rotational RAID array at home:
root@nas:/mnt/raid# ioping /dev/md0
4 KiB from /dev/md0 (block device 7.28 TiB): request=1 time=27.2 ms
4 KiB from /dev/md0 (block device 7.28 TiB): request=2 time=15.7 ms
* Here's an example of pretty good storage latency on my new storage at work: root@s1-san5:~ # ioping /dev/md200
4 KiB from /dev/md200 (block device 1.09 TiB): request=1 time=136 us
4 KiB from /dev/md200 (block device 1.09 TiB): request=2 time=124 us
4 KiB from /dev/md200 (block device 1.09 TiB): request=3 time=112 us
* And one more on a VM with storage provisioned over iSCSI to a very slow rotational storage array that's quite busy: root@nagios:~ # ioping /dev/xvda
4096 bytes from /dev/xvda (device 15.0 Gb): request=1 time=11.6 ms
4096 bytes from /dev/xvda (device 15.0 Gb): request=2 time=0.2 ms
4096 bytes from /dev/xvda (device 15.0 Gb): request=3 time=7.1 ms
4096 bytes from /dev/xvda (device 15.0 Gb): request=4 time=1.2 ms
---OK So let's try this on instantcloud.io / scaleway:
* IOP/s - Random 4k reads:
bs: 12 (f=12): [r(12)] [0.5% done] [31050KB/0KB/0KB /s] [7762/0/0 iops] [eta 37m:32s]
* IOP/s - Random 4k writes: Jobs: 12 (f=12): [w(12)] [0.2% done] [0KB/7848KB/0KB /s] [0/1962/0 iops] [eta 02h:29m:10s]
* Latency: root@instantcloud:~# ioping /dev/nbd0
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=1 time=1.4 ms
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=2 time=1.4 ms
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=3 time=1.4 ms
4.0 KiB from /dev/nbd0 (device 46.6 GiB): request=4 time=2.8 ms
Conclusion:Good performance for small arm devices but not even close to even a single entry-level consumer grade SATA SSD.
▶ dd bs=1M count=1024 if=/dev/zero of=test conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 7.32253 s, 147 MB/sOne of the engineers tried running our server stack on a raspberry for a laugh.. I was gobsmacked to hear that the whole thing just worked (it's a custom networking protocol stack running in userspace) if just a bit slower than usual. I can imagine making use of loads of ultra-cheap servers distributed all around the world... IF the networking issue can be solved.
Perhaps the time is right for a more compact and cheaper wired networking solution... maybe limit the bandwidth to 1Gbps but make it ultra compact with much cheaper electronics. Sigh... a man can dream.
I guess it's just a novelty. Might as well go virtual.
In terms of space/power/reliability/scalability large systems win. Sure a single raspberry doesn't draw much power. But it doesn't provide much computing power either. Throw in a few hundred of those systems and you feel the heat. Still the computing power is comparable to a single rack server. You want to use a RAID6 on the raspberry, sure, can be done but throw in 4 times the number storage devices. Compare that to the rack server with 16 SSDs configured as RAID6 where the data is shared by hundreds of virtual machines. If you compare the "energy per bit" or "energy per operation" the high-powered server CPUs win versus most anything out there.
So I'd say:
* If you can justify a real server, do use one instead of dozens of "simple machines".
* If you can use the cloud do it instead of providing your own hardware.
* Exceptions may apply where security or reliability is concerned. (You wouldn't run your heart monitor in the cloud when a small dedicated system does the job.)
For test purposes, a local Pi cloud (or something similarly-priced) is a decent deal.
One guy's working theory is that they overheat, so last I heard he was attaching heatsinks to the major chips, but I haven't heard if that helped the reliability much.
I can't imagine going to top-of-rack directly from (say) 512 or 1024 Pis in a rack. So you need intermediate switches, probably a small one every couple of U. From there to top of rack at 10G (could get away with 1G if you know your network bandwidth over your couple U of Pis won't get saturated). Top of rack switch will need to be optical, probably redundant, at 40G aggregate or better. Did I mention that those first-layer switches probably take up a U themselves?
Per Pi, we might be spending as much on each switch port as we are on the Pi itself, maybe more (a 48-port switch that we use is about $2k delivered, or about $40/port, and that's just the first switching layer). You can probably buy cheaper switches than the ones we use; I don't know if there is drop in reliability. Haven't figured the cost of the optical links, either, but they can get spendy as well.
I think that box of Pis needs its own switching fabric, so that 1G link never leaves the chassis the Pi is in. The switch doesn't need to be fancy, but it looks pretty custom and you'll have to amortize its cost over a big build.
I really hate wires :-)
But doing all that custom electronics does take the fun out of the idea of "just a bunch of cheap raspberry pi's doing their thing". So maybe not.
Just imagine racking in machines like that which take N USB connections, where that's 1, 2, 4, 8, 16 or whatever is necessary.
You can even add redundancy by connecting each Pi server to more than one root hub.
Writing the USB-based switch for this would be fun (probably someone has done this, though).
You can buy a single off the shelf Proliant microserver which is the size of a shoebox, put Vmware on there (or your virtuaization product of choice) and have god knows how many VM's running with a very modest power load that will blow away any sort of jerry-rigged Pi's shoved into a 1U solution. And be reliable and have raid and have ecc memory. Hardware is kinda a solved problem now.
Shame the pi isn't more powerful. I was looking at deploying Freepbx in my home and the performance of the web interace of the Pi is terrible. I'm not sure what people actually use them for. I'm probably going to just get a beaglebone that's 2-3x as powerful for a measly $15 more.
Unfortunately the latency to the servers from Australia is so poor and makes it practically unusable.
Here is a trace from my 100Mbit fibre:
samm ~ % mtr -n --report 212.47.250.196
Start: Fri Apr 10 19:36:45 2015
HOST: samm-mbp Loss% Snt Last Avg Best Wrst StDev
1.|-- 192.168.0.1 0.0% 10 1.7 2.0 1.6 4.7 0.7
2.|-- 150.101.212.44 0.0% 10 2.7 3.1 2.5 4.2 0.0
3.|-- 150.101.208.65 0.0% 10 5.3 6.9 2.7 38.1 11.0
4.|-- 150.101.33.28 0.0% 10 28.3 20.8 14.6 28.3 5.6
5.|-- 150.101.33.149 0.0% 10 170.6 170.4 170.1 170.8 0.0
6.|-- 62.115.33.97 0.0% 10 170.8 170.8 170.1 171.4 0.0
7.|-- 213.155.135.156 0.0% 10 240.9 240.7 240.2 241.2 0.0
8.|-- 80.91.251.103 0.0% 10 322.3 335.0 321.4 413.3 30.2
9.|-- 213.155.136.209 20.0% 10 319.7 329.8 317.9 355.6 14.4
10.|-- 62.115.40.86 0.0% 10 319.1 319.3 318.7 320.0 0.0
11.|-- 195.154.1.41 0.0% 10 332.5 333.0 332.3 333.9 0.0
12.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
13.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
14.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
15.|-- 212.47.250.196 0.0% 10 395.0 334.8 319.4 395.0 23.5
Edit: Spelling (Typo)I'm told that traceroute isn't a reliable way to determine end point latency. I can ping 212.47.250.196 and get a round-trip time of ~365ms. That the intermediate hops each take 170 - 333ms to response in the trouceroute is meaningless. Or so I thought? Maybe I'm not sure what you're getting at?
(Edit: I mean iiNet. I mean TPG.)
Here's an example from PIPE networks (TPG):
root@dev-samm:~ # mtr -n --report 212.47.250.196
HOST: dev-samm Loss% Snt Last Avg Best Wrst StDev
1.|-- <removed> 0.0% 10 0.9 1.1 0.9 2.0 0.3
2.|-- <removed> 0.0% 10 0.5 0.5 0.4 0.6 0.1
3.|-- <removed> 0.0% 10 1.1 2.4 1.0 9.1 2.8
4.|-- <removed> 0.0% 10 1.1 1.1 1.0 1.3 0.1
5.|-- 203.219.106.21 0.0% 10 2.5 2.8 1.1 4.7 1.1
6.|-- 202.7.171.25 0.0% 10 11.2 13.1 11.2 14.6 1.3
7.|-- 203.29.129.195 0.0% 10 11.1 12.9 11.1 14.7 1.2
8.|-- 64.86.21.53 0.0% 10 210.2 212.2 210.0 229.9 6.2
9.|-- 64.86.21.2 0.0% 10 373.1 373.2 372.9 373.9 0.3
10.|-- 66.198.127.1 0.0% 10 382.3 382.4 382.2 383.1 0.3
11.|-- 66.198.127.6 0.0% 10 382.1 382.0 381.8 382.1 0.1
12.|-- 66.198.70.21 0.0% 10 378.7 379.3 378.5 381.6 1.2
13.|-- 80.231.130.33 0.0% 10 380.6 381.0 380.6 383.0 0.8
14.|-- 80.231.130.86 20.0% 10 373.9 374.1 373.9 374.7 0.2
15.|-- 80.231.154.17 10.0% 10 371.3 371.5 371.3 371.9 0.2
16.|-- 80.231.153.58 10.0% 10 379.4 379.1 378.9 379.4 0.2
17.|-- 5.23.24.6 10.0% 10 354.1 354.4 353.9 355.6 0.5
18.|-- 195.154.1.39 10.0% 10 355.2 355.2 354.9 355.5 0.2
19.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
20.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
21.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
22.|-- 212.47.250.196 10.0% 10 354.5 354.3 354.0 354.9 0.3
A throughput test struggles to obtain 1Mbit: samm ~ % iperf -c 212.47.248.211
------------------------------------------------------------
Client connecting to 212.47.248.211, TCP port 5001
TCP window size: 129 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.0.22 port 56958 connected with 212.47.248.211 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-11.1 sec 1.38 MBytes 1.04 Mbits/secAnyway, I think this is a super cool and creative service, kudos to the author.
Edit: Possible bug, I get nothing on Safari after 1 minute of waiting: http://i.imgur.com/pKoNPrV.jpg
Edit 2: Ah, on trying again, I get an error that all servers are busy. Maybe we killed it again, HN :)
Anyway with a service like this it would be far easier for anyone to make his hacking attempts more untraceable.
* CAPTCHA
* Require single-sign on via Facebook or Google+
* SMS token
* IP throttle
No, none of these are remotely watertight and no, not every developer is inclined to connect to their VPS via Facebook (or even has a FB account). Just saying, these are ways other sites try to add a bit of friction to what could otherwise be a runaway script spinning up thousands of servers. And without requiring a credit card.
But wait...
You can also use SSH as a dynamic SOCKS proxy quick for an ad-hoc VPN. I bounced SSH over to port 443, because why not. Fired up SSH on my local machine with a local dynamic proxy (ssh -D 4444 -p443 ubuntu@212.47.231.xxx), set my proxy on Firefox to localhost:4444 and voila. Free VPN through them for 30 minutes. Not uber fast (server is in France) but usable.
After the 30min session the window closed and it told me that session expired. Although the actual server was still up at the ip. About 5 minutes later apache shut down, so I guess the server was destroyed.
This is great for experimenting things. You could quickly for example test "sudo rm -rf" at the root and see what it does. Nice one!
I mean, for example total beginners may find this truly helpful.
Do you? JSLinux (by Fabrice Bellard) disagrees :) http://bellard.org/jslinux/
€0.02 GB/month, with unlimited requests and transfer https://www.scaleway.com/pricing
Definitely not ideal if I've wasted 5 minutes of my server's runtime just guessing the password.
Edit: Got it finally, 7 minutes in - a line right under the mystery letter made it look like an 'L' when it was actually an 'I'.
Do you mean to say "Get a free server for 30 minutes"?
http://www.kimsufi.com/ca/en/index.xml
Meanwhile for $42 you can get dedicated SSD
http://www.soyoustart.com/us/essential-servers/
The only problem with all of these is no ECC
All C1 servers are busy, please retry in few minutes :)"
The gaping problems with such paradigms (chiefly survivability/evolvability over time) were well highlighted for large scale, general systems by the internet itself, RFC3439 (2002) puts it thus: Upgrade cost of network complexity: The Internet has smart edges ... and a simple core. Adding an new Internet service is just a matter of distributing an application ... Compare this to voice, where one has to upgrade the entire core.
My take: cloud computing is about to get smart edges; cloud providers are about to be commodified; and we are about to effect an appropriately flexible layer of additional abstraction to the entire field of computing that will further push us towards a position in which we treat computation as any other service and networked communication itself as a means of economic exchange.
In general, don't use unconditional in-page redirects, whether they be javascript or meta refresh. If you want to do a redirect like that, you can have your server serve a 301 and the browser will collapse history appropriately, but if you must do it in JS then use history.replaceState.
https://devlearnings.wordpress.com/2014/08/22/limiting-fork-...