Serving 250k developers with one support engineer
blog.railway.app
blog.railway.app
I asked this because the interview process has this step where you ask the candidates to do a project and than go over it with someone. One of these little projects is design our api or build a github issue viewer frontend/backend. You still have 4 more rounds after someone invested all that time.
The amount of effort to work here seems high and the chances of the position existing and you filling it is so low. I feel bad for the many looking for work.. do you invest 10+ hours in a process where a job may not exist in the first place and the company brags about the goal of one person servicing a million
Then there’s also the reality at this stage you need people who can wear multiple hats. And there’s a bunch of roles where your ideal candidate doesn’t have a neat pre-established label. So sometimes we’ll post the exact same role with different titles to try and make sure it gets the attention of someone who most strongly aligns with one of those job titles.
At a previous place we worked remotely, and so the same job would be posted as both remote and then also as a dozen different specific city locations too. But if we were hiring multiple of any roles you wouldn’t multiply that, you’d just post the one listing(s) and keep it open until all the positions were filled.
A jobs page is a marketing artefact for potential hires, it’s not a financial reporting/forecasting tool.
TLDR: don’t make any assumptions from a job board about how many roles a company is realistically hiring for.
> do you invest 10+ hours in a process where a job may not exist in the first place
The company isn’t going to spend the time walking someone through the hiring process if they don’t have any intention of hiring someone.
10 hours of interviewing is barely more than a single workday. It wasn’t long ago that interview processes required several rounds of phone screenings, interviews, and maybe testing before you were expected to come on-site for an entire day of in-person interviews - or more! I don’t think it’s unreasonable for a company to expect candidates to invest some time into going through an interview process. The phenomenon of hiring people into high paying jobs after a couple hours of casual interviews was largely an artifact of the recent tech bubble. It’s also part of the reason we’re seeing mass layoffs, IMO, as a lot of these companies hired so fast that they couldn’t reasonable screen everyone. It turned it “hire now, fire later if it doesn’t work out” when cash was plentiful.
>not necessarily nefarious
It might not be nefarious, but if companies are putting out "feelers", and I'm investing ~30 hours per company, and half the companies I'm applying to are like this, then there's a huge waste of time (mine and theirs). I've started calling out this bullshit: "is there a job here, or is this a position to gauge the market?"
>10 hours of interviewing is barely more than a single workday.
It's never 10 hours. It's 5+ interviews, plus a take-home test (that can take ~10-20 hours), plus scheduling, prep, etc. It's never 10 hours. Stop validating shitty behaviour.
>I don’t think it’s unreasonable for a company to expect candidates to invest some time into going through an interview process.
I wouldn't mind, if there was a job on the other end. I can't even list the number of times I've gone through the whole interview process (at big companies!) and have been ghosted at the very end. I wish they'd stop wasting my time.
>The phenomenon of hiring people into high paying jobs after a couple hours of casual interviews was largely an artifact of the recent tech bubble.
That's how it's been most of history. 10-30 hours of interviews for a single job is a relatively new phenomenon.
>It’s also part of the reason we’re seeing mass layoffs
This is so false. We're in layoffs because we overhired, not because the interview process is too easy.
I just want to say that this is a much higher number than necessary. If you've got skills there's just no reason to ever accept more than 90-120 minutes total before the on-site. That's plenty to receive multiple top-of-market offers (or not top-of-market) even in the current market. Any company pushing trying to make an interview take this long on your side is not worth working at.
Hiring is a matching process. If a company with an annoying hiring process is a bad match for you, that's fine.
Remember, the company has more information than the applicant.
(This question is asked in good faith).
The problem is the sheer number of steps required, of which the project is just one step. For example for a recent position I looked at:
1. Initial interview with some HR person: 1 hour
2. Interview with a tech person and review some code sample (spot security issues, bugs etc): 1.5 hours
3. Take home project (supposed to take 2 hours, but you'll want to make it look good so you'll probably spend twice that): 4 hours
4. Another in-depth interview with a couple other tech people: 2 hours
5. Another interview with the CTO: 30 mins
6. References & background check.
That's not atypical. Round after round of interviews. It becomes an endurance test, where only the most dedicated will stay the distance. While the 8-10 hours might not seem much stretched out over several days, you want to add some prep, keep your calendar clear so it does eat up your spare time, especially if you are already working.
And that's just one job. What if you are interviewing at multiple companies?
Note the take-home is at the start of the process, not the end: so you can end up putting a lot of effort in before you have even cleared four or five other hurdles. Oh, and I don't even have an offer yet. It could well be the offer on the table isn't worth my time.
Now you could say "well, we only want dedicated people". Fine, but I'm dedicated in so far as I get paid to be working for you. I don't get paid to run your interview gauntlet. Maybe you want people willing to do free overtime?
And this is way more than it used to be maybe 10 years ago, and it's for small to medium sized companies, not FAANG or other big corps. Nor is it a feature of recent layoffs and resulting increase in the talent pool: this has been the case for a few years now.
I think there's a few factors at work:
- Endurance test
- Risk aversion culture in management
- Copypasta whatever Google or Apple do
FWIW, last time I interviewed at Google, there was no take home test, so I don't know where people are copying that from but I don't think it's Google.
This very much depends on what type of IP agreement you have with your current employer.
It doesn’t make sense for a company to open up their code base and infrastructure to random candidates and ask them to work on it. That’s an IP and leak nightmare.
OTOH, if you are taking a paid contract job to work with a company on real work as a trial, you could definitely be violating contractual agreements with an employer. This type of interviewing is extremely rare, though. Few full time job holders would consider it, so it would be largely limited to unemployed people who have the time to do it.
Please be civil.
Posting multiple job listings targeting different salary ranges isn’t a “hack response”. It’s literally how you hire properly without a crystal ball to predict exactly who will apply.
Again, no company is going to drag candidates through interviews for jobs that don’t exist. I don’t understand why anything thinks that’s the case. Interviewing is work for the interviewers, too.
They didn't cut down on the process, or the five stage interview processes during the last two years of over hiring, they doubled down on them.
There are very few careers where entire industries like leetcode, hackerrank, coderpad etc have been founded to promote the practice of making every Dev slog through a mass of made up crap intent on wasting the time of everyone involved to make HR feel like they've achieved something.
This happens all the time. The team will have an internal candidate they want, but HR dept is worried about optics of fairness, so the team is forced to put on a show by interviewing people who have no chance of being hired.
Oh, the logical fallacy of appealing to logic.
I have personally seen companies post, interview, with no intention of hiring. No guff!
Case 1:
* "Just in case" they get a contract to provide thing X, they "want to be ready" to go.
Case 2:
* Forced to look at internal staff first, knowing full well they would not hire external, but going through the motions for appearance sake
Beyond the above two, I have heard:
* A government department interviewing, offering, and then the union hears of it, and challenges the new hire
* A company interviewing a dozen people, ready to hire, and a stakeholder suddenly thinking "let's contract this out instead"
* A company doing 3rd round, time consuming (for them and the candidates) interviews, after an offer, while waiting for the response. And I mean, scheduling and doing, not "already scheduled and doing".
I usually spend around 20hr on the test task they give me. On top of this there are at least 3/4 rounds of interview (spanning from half an hour to two hours).
I tend to make sure that the job actually exists though and I'm wary of small companies. I'm particularly avoiding the ones where I would be the only developer working on my domain/stack.
Automated triage is great for digital companies that want to scale cheaply, but not so great for the people who need to figure out how to usefully navigate them, with the brick wall support experience of Google/Facebook/etc being the already vivid example. It helped those companies grow and court techno-optimist investors, but nobody outside of their enterprise customers is happy about the customer service experience. Eventually, that becomes dirt on their reputation and works against their continued growth and success.
When I first joined- this is why I would take great pains to respond to everyone extremely quickly. When I realized that was not sustainable, the focus then shifted to really making sure that our users are heard and their concerns are acted upon. Sure first contact SLA of 30 mins is great but- doesn't mean shit if we don't solve your problem.
So I think yes, tooling should be first. I think there is just a way for that tooling to serve the developer, not some OKR.
If I can solve my problem though an automated support channel that’s actually better! Because I don’t need to wait for a human to do something. Computers don’t sleep, take lunch breaks, etc.
The problem with google/Facebook/etc isn’t that they don’t have human support. It’s that when your account gets locked out there’s no way of getting your problem solved. If a working useful automated support system existed few would complain.
For better or worse, slow, biologically constrained humans can often understand and solve problems when given the authority to do so. (I'm well aware there are horrible human support options.)
Is it weird then that they have an open req for a support engineer: https://railway.app/careers/support-engineer
This is actually the original blogpost that we used to make our first hire!
> You will be working with our existing Support Engineer, Angelo - to build out processes, formalize policies, and build out integrations between systems to make it so that we can track and record issues.
I accidentally left that one open... but the Ashby interface is so confusing I don't know where to toggle the visibility. I will remove that and then wait for our jobs page to rebuild.
> At Railway, we provide best in class benefits. Great salary, full health benefits including dependents, strong equity grants, equipment stipend, and much more. For more details, check back on the main careers page.
I don't think this is true.
At scale, or with a seasoned ops team/team member, the scenario is a bit different, but if you're asking "is it more secure for a few full-stack developers hacking at a startup to roll their own infra or use something like fly.io" I feel preeetty comfortable saying the latter is going to be more secure.
On the other hand, many PaaS providers obfuscate their security implementation, and ultimately your data could be compromised by their mistakes. Things you should look for when evaluating PaaS providers:
- How are resources, networks, etc. separated/isolated per customer?
- What are YOUR security responsibilities on the platform?
- How transparent is the provider about their security controls? Do they have security whitepapers, SOC 2 reports, etc. that are transparent and legit? Better yet, can they prove to you in the product how security controls are being implemented?
Disclaimer: I'm the CEO and founder of Aptible [1], a PaaS specifically built to meet and prove security requirements for companies in regulated/high-compliance environments.
[0] https://aws.amazon.com/compliance/shared-responsibility-mode...
I have a question about endpoints. It seems like you guys charge per endpoint. I don't quite understand this. So if I'm developing an api only application, every api endpoint I develop in my application will be charged? And for Aptible to keep track, would I have to register each endpoint I develop?
If my application was just serving dynamic html pages, I wouldn't be charged per url of my application right? So why would I be charged per api endpoint?
EDIT:
Another question. Do you guys offer any SSO solutions? If not, if I used say Auth0 for authentication, are there any issues with integrating with Aptible?
We don't provide a solution for implementing SSO in your own application, but many of our customers do integrate with Auth0 without issue. For your own team's access _to Aptible_, we offer SSO through SAML integration with any provider (Google, Okta): https://deploy-docs.aptible.com/docs/sso
Another question. I tried looking for the answer on the website, but couldn't find it. Is it possible to use my own AWS account and integrate it with Aptible or does Aptible provide their AWS assets for my use? The former would be ideal for us as we would like to own (more accurately, rent them ourselves) all of our AWS assets and just have someone like Aptible help us to manage them.
Alternatively, have you checked out other PaaS-in-your-own-IaaS solutions like:
- https://www.flightcontrol.dev/
- https://coolify.io/ (OSS, not managed)
These might not meet all your needs, and I think they're all relatively new.
I work at Amazon and most teams do not have a single support engineer despite serving hundreds of millions of customers. The number of employees in the position X is absolutely meaningless.
As a hobby dev alternative to Heroku this might be fine. For a serious business I don’t see this as an alternative.
- Serving >100k users
- Helping you to achieve SOC 2
In my capacity as one of the leaders at a Heroku alternative PaaS, I've studied the PaaS market a bit to understand the space and available alternatives to Heroku. Here's what I've found about some of the most popular:
- Fly is managing its own infrastructure allowing it to be extremely competitive on cost. But on the flip side, its heavy focus on infrastructure is missing the “managed” options that make PaaS so valuable, such as a true Managed Database offering. This IMO makes it less of a viable alternative to Heroku.
- Render is offering a more truly “managed” alternative, and is innovating on cost as well. But it’s early and is still missing some table stakes reliability features that you'd probably expect from a Heroku alternative.
- Railway has a blockbuster FTUX, I love deploying and using PostgreSQL databases in the UI without even signing up. Coincidentally probably how it got to its count of 250k developers. But it’s own docs caution that it’s not exactly production ready, especially its databases.
- Platform.sh has grown really well by focusing on enterprise marketing teams and their use cases. I think this is a great niche for them and has paid off well. In their capacity of working with enterprises I'm sure they could handle SOC 2, etc.
- Aptible has run critical web apps and APIs dealing with sensitive data for hundreds of companies and has helped a few go public or get to billion dollar acquisitions. I am certainly biased, but Aptible seems to be the only non-Heroku PaaS focused on product/engineering teams that has repeatedly handled true "production" requirements, like your larger user base (many of our customers fit this description) and SOC 2 (most of our customers use our security & compliance dashboard for this). But that comes at a cost: Aptible is typically more expensive than the others, perhaps save for Platform.sh.
Disclaimer: I'm one of the leaders at Aptible, which in my (admittedly biased) view is the best positioned alternative to Heroku for product/engineering teams that have any sort of scale or production use case.
You get a good level of abstraction for infrastructure (github.com/platformsh-templates for examples) that gets provisioned automatically, plus a built-in relationship between branches and envs that results in true staging environments for every pull request.
docs.platform.sh
1. Get a support tool (or in this case, roll your own, then buy one when you outgrow that)
2. Build/buy tooling to direct users to existing documentation that might answer their questions
3. Move any routine tasks that require a human to automated self-serve tools.
4. Rope your community into answering eachother's questions.
And these days, good documentation is 1st criteria to trust a platform / framework / library.
So there is one but, if you can't find it, it's as good as there not being one!
It's currently located at https://nixpacks.com/docs/guides/configuring-builds#change-w...
Is there some place you'd expect it to be instead? All ears on feedback!
Back in 2010, Monash University’s Bachelor of Software Engineering had a unit ENG1061 Engineering Profession mandatory for the degree (for a terrible reason: Engineers Australia accreditation of the degree, which no one that I ever talked to cared about in the slightest). The first lecture listed The 17 Branches of Engineering. Guess what wasn’t one of them. It went downhill from there.
Going by the top of their pricing page (https://railway.app/pricing): $10/GB per month RAM & $20/vCPU per month is actually pretty expensive when compared to services like fly.io and even EC2/Azure.
Apart from that, what vCPUs are these exactly? What are the bandwidth costs? Custom domains? Certificates? Load balancing? Disks? Databases?
Clicking "Choose Plan" takes you to a login page, but then if you log in the pricing page disappears altogether.
At the moment we don't but I spoke to the fmr. Head of Support at Stripe and he told me we should think about our product coverage like an API, some parts of the product (builds for example) should be more critical on response times than others. My main goal this year is to publish an SLA table so that we can work with ever larger customers setting the right expectation.
That growth curve is amazing.
(disclosure: am small angel)