devicenull

> We want to dual bound the network connections to increase performance and reliability. This will allow us to take routers out of service during low traffic times, for example to restart them after a software upgrade.

does not really agree with

> Each of the two physical network connections will connect to a different top of rack router.

Sure, you can do it with something like MLAG, but that's really just moving your SPOF to somewhere else (the router software running MLAG). Router software being super buggy, I wouldn't rely on MLAG being up at all times.

> N1 Which router should we purchase?

Pick your favorite. For what you're looking for here, everything is largely using the same silicon (broadcom chipsets).

> N2 How do we interconnect the routers while keeping the network simple and fast?

Don't fall into the trap of extending vlans everywhere. You should definitely be routing (not switching) between different routers. You can read through http://blog.ipspace.net/ for some info on layer 3 only datacenter networks.

You'd want to use something like OSPF or BGP between routers.

> N3 Should we have a separate network for Ceph traffic?

Yes, if you want your Ceph cluster to remain usable during rebuilds. Ceph will peg the internal network during any sort of rebuild event.

> N4 Do we need an SDN compatible router or can we purchase something more affordable?

You probably don't need SDN unless you actually have a SDN use case in mind. I'd bet you can get away with simpler gear.

> N5 What router should we use for the management network?

Doesn't really matter, gigabit routers are pretty robust/cheap/similar. I'd suggest same vendor as you go for whatever your public network routers.

Also, consider another standalone network for IPMI. I can tell you that the Supermicro IPMI controllers are significantly more reliable if you use the dedicated IPMI ports and isolate them. You can use a shitty 100mbit switches for this, the IPMI controllers don't support anything higher.

> D5 Is it a good idea to have a boot drive or should we use PXE boot every time it starts?

PXE booting at every boot is cool, but can end up sucking up a lot of time. If you have not already designed your systems to do this, and have experience with PXE, then don't.

> The default rack height seems to be 45U nowadays (42U used to be the standard).

You may not have accounted for PDUs here. Some racks will support 'zero-U' PDUs, but you'd need to confirm this before moving on.

> H3 How can we minimize installation costs? Should we ask to configure the servers to PXE boot?

Assume remote hands is dumb. Provide stupidly detailed instructions for them. Server hardware will PXE by default, so that's not really a concern. IPMI controllers come up via DHCP too, so once you've got access to those you shouldn't need remote hands anymore.

> D2 Should we use Bcache to improve latency on on the Ceph OSD servers with SSD?

Did you consider just putting your Ceph journals on the SSD? That's a lot more standard config then somehow using bcache with OSD drives.

> Also stop letting marketplace sellers email me begging for feedback after every marketplace item I accidentally order. I try my best to not order marketplace seller items anymore but when I accidentally do (or buy a gift for someone that is only offered this way) I always end up getting emails from these guys. Are you sharing my email address with them? Does unsubscribing or responding to them share my email address with them? I have no idea. There is never anything useful and it's impossible to unsubscribe from all past and future marketplace emails which is really annoying. Come on, amazon, I really want to love you and continue shopping there but it's getting to the point that I'd rather go to wal-mart! (ok not really)

I've begun adding 1-star reviews when I get requests begging for feedback. Seems like the only thing I can do to discourage the behavior

No, phone/dsl lines have very low latency. The latency is purely a factor of time to get to the satellite.

Uplink bandwidth has been an issue in the past (phone/dsl is fairly slow speeds, at least in the places you'd want satellite internet)

From the article:

> As of September 1, 2016, there were 1061 CCS fast-charging connectors in the United States, versus 2010 Tesla Supercharger hookups.

And, depending on your transit agreements you may have to pay for all that incoming bandwidth.

That's bizzare, the page loads, then renders a big purple box saying

Page not found We couldn’t find anything at this address. Please check the URL or go to the homepage.

404?

Vultr: Lets you upload your own ISO, and run whatever OS you want

Depends on your merchant account I believe. We tokenize them somehow, and can do further transactions by referencing the first transaction.

There is no reason to make them tiny: http://solairegeneration.com/project/rutgers-university/

You just build the solar panel mounts like you would usually build lighting mounts (big concrete bases).

Height is less of a concern in some parking lots (where it's virtually all cars)

Do you have the fun issues where the power button sticks, resulting in an endless boot loop? I've resorted to percussive maintenance, which oddly fixes the problem for months at a time.

So basically GameRail was before it's time.

They'd have to buy transit from a provider. Peering is connectivity between two ISPs. Transit is connectivity between you and the rest of the internet.

Peering doesn't get you to the internet at large though, it just gets you to the people connected to the particular exchange (and they're not usually willing to relay your traffic to other networks)

> Or better yet, ZeroMQ-style sockets with proper security and encryption?

ZeroMQ supports CURVE encryption + authentication as of 4.0

http://hintjens.com/blog:49

You're describing how most IPMI controllers are implemented. This sounds great and all, until you realize the vendors don't bother to keep things up to date, and run all sorts of extra shit so they have a bigger feature list.

Line rate for 1gbps ethernet is over 1 million PPS, so you can definitely send the packets that fast. No promises the target will be able to respond that fast.

See https://github.com/robertdavidgraham/masscan

> But Oculus will have room scale soon too: Oculus Touch is coming soon.

They haven't even shipped all their headset preorders yet, I doubt "soon" is the word you want there.

US outlets don't have any sort of fuse/breaker built in.

> Nearly everything sold on Amazon these days is 3rd parties. I'd wager somewhere in the 95%+ range. Same with "Prime" items (these are just 3rd parties that use the FBA option).

This matches what I see. What's even more annoying is that it seems like every third party seller also has decided it's a good idea to spam me with emails asking for reviews after every purchase. Amazon provides no way of unsubscribing from there (they block one particular seller if you contact them, which is utterly useless)

You check the card number via the Luhn algorithm, and tell them about it? That's not giving any data to fraudsters.

My reading was that their C&C monitoring stuff was polling once an hour, not the bots themselves

Sounds like the Cumulus Linux approach (they use some daemon called switchd iirc)

It's also worse, because if you were to deploy BCP38, you don't really get a whole lot from it. Sure, it's a good thing to do, but at the end of the day people aren't going to pick your service because you have BCP38 setup properly.

And Amazon's answer to all this seller spam is "oh, just unsubscribe from it". Completely missing the point that I'd have to do this once per seller.

AFAIK, it depends on TDP. So, yea, you won't get all of them running at max turbo. (Even if you did, they could still be running at a lower p-state and just reporting a higher frequency. Frequency and performance aren't really the same thing anymore)

They provide white-labeled protection for DNS too, if you're a big service provider.

> If someone is harvesting IPs from NTP queries sent to Debian infrastructure for intelligence gathering, that in itself is a big deal.

Debian doesn't run the NTP pool.

Yep, that's exactly what it's saying.

If you have a machine behind a firewall, why do you care that it's IP is secret? Do you also worry about people finding out your IPv4 IP?