Rackspace lays off 200 locals in company-wide cuts
therivardreport.com
therivardreport.com
They've reimagined the company as providing support on third-party platforms, namely AWS and Azure, instead of focusing on building, selling and supporting their own offerings. That model requires significantly fewer employees. That change, coupled with the buyout, means that there's no surprise in seeing layoffs at Rackspace, and there are likely more to come.
I'm a former Racker.
They usually borrow money at really high leverage to buy the company so they risk very little of their own capital. Then they cut anything and everything to minimize costs (and choke off the ability to compete in the future) then dump the carcass(es) on whatever suckers they can find because they made the balance sheet look better temporarily.
Banks line up to lend them money because there is a never-ending supply of suckers and these firms usually manage to find one before the inevitable crash and burn.
Why is it so obvious to you, but not these other people? How does this charade perpetuate? Maybe there is value in there somewhere?
Buyers always think that they have some insight about the business that can materialise some yet-unrealised potential. You've probably done it yourself a few times, "if only Company X did this and that, instead of what they're doing now -- they'd make loads!". Everyone knows PE don't know how to run businesses beyond the basics, so buyers will always see untapped potential. That makes for high prices.
To be fair, Rackspace's market cap was only about 4 billion at time of acquisition. They don't really have the capital to get into a war with Amazon, Google, or Microsoft, when data centers cost hundreds of millions/billions to build.
The economies of scale they're fighting against are gigantic, so it makes sense to go towards supporting the clouds of other companies rather than building out their own bare metal.
The sales engineer / presales architect working to design a cross-cloud solution for a customer is a very very different kind of "smart" from the Linux geeks that keep your systems running once they're deployed
Neither is better than the other, mind you - they're just different
We told them the strength of the DDOS attacks and they said they could mitigate it. We signed a contract, got DDOS and we were null routed by them. I got to talk to one of their techs and they said they couldn't actually mitigate DDOS attacks of that strength. He told us to use DDOS Arrest which worked. So they lied about their ability to mitigate DDOS and were stuck with this expensive 2 year contract.
Then a month later our site went down for 48 hours, and it was because they had a dead router in their internal network, but they insisted nothing was wrong and I had to debug the issue myself to get them to find the issue.
They got into doing AWS consulting and sold the startup I was working for an AWS setup that was about 100x too large and overengineered for their static site.
Prior, I had used them for a number of projects and always found them to be professional and helpful. I think the old Rackspace is gone.
I know this because I worked at Rackspace, and one of the larger means of income for the city of Windcrest is semi-bogus traffic tickets given to people coming and going from Rackspace headquarters.
I didn't ever live in or near San Antonio (or Windcrest!), but I did live in Austin and Houston so I am familiar with the Texas phenomenon of cities entirely or almost entirely surrounded by much larger cities. For instance, near Houston there's Bellaire and West University Place.
The comment being replied to said "Well, Windcrest is a subset of San Antonio, Texas." Not "the city of San Antonio, Texas." Both officially and unofficially, San Antonio is the name of not just an incorporated city, but of the metropolitan area of which it's the largest city. When you say that the metro area, as opposed to the city, was "not what was being discussed," that was an assumption the comment I replied to was making. But nowhere is that said in the comment that started this thread.
The principle of charity (https://en.wikipedia.org/wiki/Principle_of_charity) "requires interpreting a speaker's statements to be rational and, in the case of any argument, considering its best, strongest possible interpretation." The interpretation of the statement being replied to that makes it true is "Well, Windcrest is a subset of the San Antonio, Texas metro area." The "pedantic" comment I was replying to ignored that possible interpretation in favor of an interpretation that made the comment false. But why not interpret the comment in such a way that it's true, given that there's a very common way of reading that (San Antonio the metro area) where it's so?
https://en.wikipedia.org/wiki/India%E2%80%93Bangladesh_encla...
RS will do really well as a services business, lots of fortune business that need to migrate to software-defined infrastructure.
Which, while not officially supported by Rackspace, was started there, is sponsored by, and is currently maintained by them.
We're moving a lot of files over to Google Cloud Storage (Nearline) as well as full backups to Coldline storage.
Nearline storage is 1/10 the cost of cloud files. Lifecycle policy is very handy for automatic file rotation. Command line is slick.
Rsync was a total lifesaver in the migration. Moved 4TB of files over 26 ish hours.
We actually gave up managed bare metal hosting with another provider for AWS a few years ago, when we found that AWS's automated infrastructure had reached a point wherein self-management was a viable option at a much lower cost.
Granted, our installation is straightforward, but overall, it seems AWS would reduce the need for managed services.
Speaking from an enterprise point of view, that makes all the difference. A lot of companies will have mishmashes of different systems with one-off setups that are not worth the effort of automating.
The main selling point of AWS, from an enterprise perspective, is to shift costs in the right balance-sheet column, lower build times, and lower headcount because you can cut the hardware boys. Everything else (automation etc) is entirely optional and only worth the effort in some particular circumstances.
i.e. Full-time work is only full-time until it ends, but the ending is inadequately spelled out on the side of management, so it becomes an ongoing expense drag, leading to diminishing returns.
Multiply this by the number of employees, layoffs seem inevitable.
The danger of this (to the employees) is a recession creating another 2009-like economy, where the best one could hope for is a contract gig of more than a couple of months. People tend to forget how disposably they can be treated during hard times.
What might work is automated Just-in-Time contracting, with minimal agency cost and lead-times, and with automation providing the stability of full-time employment. LinkedIn 2.0 essentially (fyi: this doesn't exist, I just made it up)
Expect them to close/sell their cloud hosting division.
Rackspace and HP needed OpenStack to take off in order to generate demand for their public clouds. When that didn't happen HP shut down its cloud. Rackspace couldn't do that, their cloud accounted for too much of their revenue, so what we're seeing is their fallback plan.
Interesting anecdote: yesterday the OpenStack Foundation sent out an email announcing the speakers for the upcoming OpenStack summit. The subject line: "Hear from Google's VP of Cloud Platforms at OpenStack Summit Boston".
The idead of a bunch of giant openstack public clouds is dying, but openstack as the solution to build private clouds still seems to be going strong.
Lanham never would have missed the opportunity to drop "fanatical" in.
"The company is cutting jobs in its corporate administration and management teams."
Sounds like they are not releasing technical and customer facing talent?
As a true Austinite, I never really considered San Antonio a "tech hub" or indicator of anything in tech.
Err, I don't think so. Many good companies have an Austin office though, in addition to companies hq-ed there.
customer support
Maybe there was a way to escalate it, but it wasn't readily evident. Which tells you something. So, I just moved on to AWS. More transparency on the platform and more predictable support.
Bad news. AWS is no different. EC2, IAM, whatever will have issues for hours, AWS won't update their status board (to the point someone wrote an extension to show the "real" status [1]), etc. And this is with business support we pay for. You needed to autoscale? Better go make yourself a mojito and wait for the dust to settle.
"There is no cloud, its just someone else's computer" [2]
[1] https://chrome.google.com/webstore/detail/real-aws-status/ka...
In the end someone has a pager, a machine and an editor. In ye-old-days this problem was in-house with a COLO machine. That's the reasoning behind uptime. 99%/year is 361 and just over half a day. Can you loose two and a half days at christmas? (web commerce) or the same time on a new product release? (SaaS) or 2.5 days development time at a critical bug fix with your customers?
The bad stuff always happens at the worst time. The market is saying, ^cloud^ but the tradeoff is ^service^. I don't know the answer(s).
but such a product would be expensive
90% seems way off. AWS is known for not being cheap (neither is Rackspace), but I highly doubt that Rackspace was ten times the price of Amazon.
There are MANY applications that simply are too complex to handle such an arch... so, - do so, when you can, but know when that's just stupid.... stacks are like snowflakes, we LOVE them to death, or we love them to DEATH...
Not if you need always-on instances.
Their support used to be the best in the business and we were willing to pay a premium for that but even their support started to suffer starting early 2015.
It is unfortunate because Rackspace was an amazing partner to our business for a long-time, which made it a very hard choice when we decided to move on.
We were very expensive but the companies must have saved money in hiring their own Engineers as were always growing.
AWS/GCP/Azure have made doing a lot of what we used to do incredibly more accessible to more people.
I don't have services like Pingdom or Pager Duty for my sites. I have Rackspace. If a site goes down, they immediately attempt to recover it, acting within the account rules I set up with them. If that fails, I get a phone call from an expert and we troubleshoot together.
I once got a call that one of my servers had started sending out a high volume of email. When I took the call, Rackspace folks had already found the nasty script doing it and just needed my ok to take the site down for a few minutes to boot it.
The service is really great. But, it ends just below the application layer, and these days, that's where most of the problems show up. So I am looking at moving off of Rackspace, but I'm looking up the service ladder at application-aware hosting services like WP Engine or Pantheon. As opposed to down the service ladder, to a self-service "don't call us" system like AWS or Google.
Anecdote: A couple years ago, I had one explain to me (in a way that made sense) how the battery on the raid array was probably the cause of some problems with https. And _he was right_.
Maybe not everyone there is a certified genius, but that really blew my mind. They really know hardware. I haven't talked to one who couldn't save my tail in a pinch. Rackspace might seem a bit on the expensive side, but their support is absurdly good.
Working there was a great learning experience, and I met a ton of super intelligent, motivated people from the Austin office.
But, all good things come to an end. Early 2015 I noticed talent being pushed towards newer offerings (e.g. Azure support), which made the typical, aging, "Linux support for small to medium business on bare metal machines" not so 'fanatical' anymore. People started to jump ship because the force behind the changes. It was a tough transition and probably still is; so I'm not really surprised that the layoffs are continuing.
In any case, I appreciate your compliment and I'm glad you have had a good time with the company.
Unfortunately stating in early 2015 the support started to deteriorate. That coupled with the cost we were paying for their dedicated servers/fanatical support stopped making since.
But yup, around the first part of 2015 what used to be a one ring call to connect with an expert became 5 minutes, then 15, and a couple weeks back I waited 45 minutes to talk to a Linux tech. Unfuckingacceptable, not as a "managed" customer paying an extra $100/mo and $0.12. We've slowly but surely moved clients to other hosts and I can count the servers still at Rackspace on one hand and still have fingers.
What is the learn cycle?
"The purpose of the learn cycle is to determine the condition of the battery. The learn cycle charges, fully discharges, and then recharges the battery in order to determine the condition and health of the battery. The battery full charge capacity degrades over time and a battery is deemed completely degraded when it can no longer hold a charge for 24 hours and must be replaced."
Source: https://discuss.pivotal.io/hc/en-us/articles/204955498-FAQ-G...
And if you disagree, please explain...so that we can all learn and understand.
Also the server time doesn't matter for TLS. It's the client that has to verify the certificate validity. (https://tools.ietf.org/html/rfc5246.html#section-7.4.1.2 "Clocks are not required to be set correctly by the basic TLS protocol")
So no, raid battery should have nothing to do with the system clock.
Thanks for taking the time to link the relevant section of the RFC. I'll remember this next time (or at least know where to check quickly ;) )
2. The clock on the server wouldn't have anything to do with the RAID array. If I disconnect a hard drive, my motherboard still gets along fine.
When a program writes bytes to the file system, those bytes aren't guaranteed to arrive on the underlying, permanent storage. They're cached for some variable amount of time in the memory of the host operating system.
In UNIX, calling fsync will force the OS to flush any outstanding writes to the underlying storage, which should mean they're safe. So far so good.
Nicer RAID controllers have a lot of high speed memory memory, and they are basically their own operating system. When they receive a write from the host OS, and their battery is good, they lie to the host OS and say that the write went through all the way to disk/SSD, even though it probably didn't. This has the effect of making the fsync call return a lot more quickly.
The bytes are still safe in this case because if there's a power loss of the whole server, there's enough battery power left to allow the controller to write through any data it lied about writing.
When the battery goes bad, by default the RAID controller will cease its lying about when the bytes it receives are written all the way through, which has the effect of slowing things down.
I can't say how lower IO performance caused the https problem in question without more details, but there are a lot of possible correlations.
NB: For brevity, I simplified a lot of how this stuff works, so some of what I said above isn't 100% accurate as stated.
This guy is super talented and worked for NASA, listen to him, he will not steer you wrong.
What a bizarre comment! 'Fanatical'?
At their HQ, they gave out a 'fanatical support' award which was a straightjacket, with some tribal backstory related to being 'so fanatical that you're insane'.
- Former Racker
Branding your support 'excessive' or 'single-minded' would also be bizarre.
Fanaticism is typically taken with religious or political connotations, and derogatorily.
Cambridge [0] says:
> [informal] extremely interested in something, to a degree that someone [sic] people find unreasonable
> [disapproving] holding extreme beliefs that may lead to unreasonable or violent behaviour: a fanatical group that has threatened to assassinate doctors
Oxford [1] says:
> A person filled with excessive and single-minded zeal, especially for an extreme religious or political cause: 'religious fanatics'
> ... originally described behaviour that might result from possession by a god or demon ... (Oxford [1])
(Italics mine.)
[0] - http://dictionary.cambridge.org/dictionary/english/fanatical
Add '~atic' to any of those and it's sounds much more extreme; something to disapprove of.
I quoted Cambridge online dictionary above as defining it as disapproving, the same dictionary defines 'fan' exactly as you use it, and does not give 'fanatic' as a synonym.
Of course it originates as an abbreviation, but the usual usage is very different.
I have at least backed up my surprise at the use with two different dictionary citations, so it can't be that strange of a view.