Run Gitlab pipelines 2x faster and 5x cheaper without changes in .gitlab-ci.yml | Hacker News Reader

Run Gitlab pipelines 2x faster and 5x cheaper without changes in .gitlab-ci.yml

gitlab-pipelines.puzl.cloud

33 pointsNikPuashkin45 comments

ProTip: Don't price shop for CI runners and give some random company access to your company's source code and secret keys/tokens, I heard those are usually a pretty big deal for companies, something about IP and competitive advantage, compliance, legal liability, attack surface, supply chain poisoning, etc. Figured I'd pass the info along.

Self-hosted is always good, if you can afford hosting own hardware.

I wonder what's cheaper, self hosting my own ci runner, or the risk of my company's source code being given to a random third party and trusting they won't do anything with it and that they will secure it to make sure nobody else gains access to it.

It's a tough one, I'll let you know what I land on

Good for you. Don't you think it's good to have a choice?

I sure do. There are countless open source CI systems I trust that I can choose from, and countless paid offerings with a very long track record of being stable and secure, with a papertrail of security audits and pen testing to back them up, some of them are even open source too. None of them are trying to compel me to act due to the very advertising $0.99 cent tactick. Price is the last thing I care about when it comes to my code and who has it, where it runs, and what I'm opening myself up to. Security and peace of mind are worth a price imo so I'll happily pay my SCM for runners, or pay to host my own, even if it's a bit extra.

When your pitch doesn't hinge off of your price being way lower than everyone else, then you'll be worth glancing at. Racing to the bottom isn't something I'd expect to be around long, esp since others in your problem domain are commenting here saying it's not possible to not be running at a loss

It isn't about price, it's about effectiveness, so both speed and price.

The solution is architecturally different from anything on the market, including the GitHub controller you've posted here. I guess, you didn't even open the link of the post, because there is a scroll #3, which gives you quite a detailed explanation of the service tech advantage.

Name three that you like?

Opensource ones are Jenkins/Jenkins X, Drone, Travis-CI, Argo, GitLab CI, Dager, etc

Paid offerings I've personally used and liked, GitHub Actions, CircleCI (their docker layer caching was a game changer back in the day), and CodeShip

> ProTip: Don't price shop for CI runners and give some random company access to your company's source code and secret keys/tokens

> I've personally used and liked, GitHub Actions, CircleCI

https://circleci.com/blog/january-4-2023-security-alert/

Yeah. Random company.

Even if I felt inclined to give your company my business, your attitude towards any valid criticism has made it so I would never go near your company or anything that you do in the future.

You're expecting companies to trust you with the ability to literally destroy them. If you can't handle security skeptical people you picked the wrong fucking niche, buddy, you're handling their source code and secrets/tokens. Your response to me saying I trust companies that have been around for a decade and have a history of being secure was to share a post about one of them having security incidents. Got em, you dunked on me! I never said they never had any incidents, because I never would. No software is 100% secure. You missed the key part, which was the longstanding existence and the paper trail, which the thing you linked to is actually a part of, which is important. They get my business over a fly by night .cloud company being pitched by a person being defensive when people have security concerns, for a new saas, in another country, who has yet to link to any third party security audits, compliance certifications, disclosure policies, SLAs, protections, vetting/audit process, etc.

I'm always open to discuss any security concerns, which are based on technology. However, you started here from the toxic biased comments. If my company is 3yo, I can't make it 10yo in one day, obviously, so there's no point to discuss it seriously. The age of the company is not an objective security criteria for an engineer, that's why I referred you to the CircleCI security breche.

Regarding the paper work, it's to be done yet. New company can't start from applying for ISO 9002. If you would like to discuss the technical security aspects of the solution (there are many), you're welcome to my DM or to the channel in Discord.

Yep. No chance in hell I'd trust a random pipeline runner. We'll gladly continue running our own.

For those running GitHub actions and k8s, https://actions-runner-controller.github.io/actions-runner-c... is pretty slick. I deployed this for my last job and set it up to autoscale based on inbound webhooks so at peak commit times we'd not waste engineer's time and during off times use nothing extra. You're in control of the nodes, and their performance, can use cheap spot instances and have both security and performance without handing your source code over to any more third parties.

I’ve been using this in production for about 6 months. It’s pretty good but has some warts around runner scheduling that need to be ironed out. It took a fair amount of tweaking with configuration to get something that worked for our setup.

Thanks for sharing! Several questions:

1. Could you address concerns regarding the data exposure and privacy? According to the registry, company owners are:

Nikita Paushkin 38912300100 (Russian Federation) Direct ownership 25.01.2020

Ekaterina Marova 49003260123 Direct ownership 04.09.2021

2. Reading through the Customer Agreement, the list of banned countries includes Iceland... Why so?

3. Is there a looking glass to assess the latency? Who owns the hardware?

Thank you for your interest!

1. The company is Estonian and regulated by the EU GDPR. Founders and all employees are the residents of the EU.

2. Was some stupid bug during the latest merge, already fixed, thanks a lot for finding this!

3.1. If we talk about the latency of cluster API, such a board is in our backlog atm. However, GitLab runners are clients by its nature, so it's hard to say what the API latency gives you here. Could you please elaborate?

3.2. The hardware is owned by GPU Computing OU and is placed in Tallinn, Estonia. We're working on adding US region as well.

1. Russian citizens who reside in Estonia or Estonian citizens who have renounced their Russian citizenship?

I apologize if this comes across as overly intrusive, however the point is important given the current geopolitical context.

2. All good, happens to everyone!

3.1. I was referring to Customer <-> GitLab latency, we've seen a fair bit of time wasted on roundtrips :(

3.2. Noted. APAC?

1. I don't think that the color of the passport is important and I, obviously, can't list all the employees here and disclose their citizenship. I'm, personally, a Russian citizen relocated to the EU far ago.

3.1. We don't provide the GitLab hosting services, only runners + pipeline jobs. You can use self-hosted GitLab or GitLab.com with our service.

3.2. We have a few customers from this region currently, so if you're from APAC and you're interested in the service, please tell us more about your requirements:

- https://discord.com/invite/wt8GzF8aG4

- support@puzl.cloud

And one more question, could you provide an insight into how you guys handle isolation between customers? Same physical cores, etc.

Each job runs in the isolated KVM, so you have the same level of isolation like with any traditional cloud instances on AWS, GCP, etc.

No offense, but is this a sock puppet account?

No, but I am clearly biased /s

Runners are for sure slow and expensive.

My solution: buy a $5k server with 128 cores and lots of memory. Sits in my basement. Runs our CI/CD for the cost of electricity. Think it got our full test suite down from 20 minutes on circle to 5 minutes.

This is the way.

Can't get large company's to do it but it makes so much more financial sense to lower the cost of ci so you can use as much of it as you want.

Can't disagree here. If you have time and a member in your team who can cook it, own hardware is worth a shot. But should get prepared for a lot of things to cook. You still have to maximize your server utilization together with security: running containers on a single host with 128 cores works seamless only if your workloads are trusted and the amount of processes is dozens, not hundreds or thousands.

We provide CI CD services from a small data center we run, but the problem that we typically have is that cloud runners are:

1. Too cost effective to sustain a business competing against it (cloud) in terms of price

2. Too critical to business for us to cut costs by simplifying hardware requirements/cutting down redundancies (power and network)

I find the direction of your business fascinating because not only are you promising higher performance service (our main selling point is that we buy HW that cloud providers won’t buy.. like low core count high clock speed[1]) but you are promising to run the business on a fraction of the revenue.

We encourage our customers with pricing perks and deals to buy more than they need, because we need the consistently high revenue even if they don’t use their service to keep the doors open and the hardware up to date. Also there is not much difference in the power usage between idle and 100% usage, so our overheads aren’t reduced when they aren’t using. Surely you would need a HUGE scale to run your company profitably.

Are you hosting your runners in the cloud at a loss ATM? We should connect.

[1]: we can build SW in 6 minutes vs 20 minutes on an EC2 large with no IOPS bottleneck. Cloud is actually generally pretty slow.

It's very insightful! Please knock me in DM in Discord, if you use it https://discord.com/invite/wt8GzF8aG4

Re: clouds: We're fully independent team and do host everything ourselves. We started from cheap k8s-based cloud GPU platform back in 2020 and now decided to go to SaaS field with all the research and experience we've collected.

I don't think you are doing yourself a favor by referring potentially interested customers to discord. It makes the product seem amateurish. CI is critical and often sensitive.

Not a Discord fan either but whole companies are now using Discord like if it is Slack. I don't think the product looks amateur by having a Discord.

Well, we're in the public comments section and here is no DM, so I'm referring to the other public channel where I can be found in person ¯\_(ツ)_/¯

We have all the contacts on the main page, but if using Discord makes it amateurish, here is the good old email: support@puzl.cloud

"all your base are belong to us"

Why should I trust you with my valuable application and secrets?

That's a good question for any cloud platform. I can only say, that to make it as trustful as possible:

- we encrypt all your environment variables on-flight before they're inside the isolated KVM

- we do a network isolation of customer runners and pipeline jobs in the local network

- we don't store the stdout of your pipeline jobs

- we use ephemeral filesystem in a runtime to avoid storing any data even accidentally

Very interesting, I wonder how they are doing it. The FAQ mentions that the CPU cores are shared.

My guesses:

- They are doing something like Netflix's internal spot market https://netflixtechblog.com/creating-your-own-ec2-spot-marke....

- Or, they reserve a bunch of super large burstable instances at a discount, then run many jobs on there each in a firecracker vm, then do some sort of quota/reservation per job to allow it to use some of the burstable credits when it needs to.

None of the above :) We're fully independent and host everything ourselves.

We're happy to announce a new SaaS on puzl.cloud. It lets you focus on your CI/CD workflows while we handle your GitLab runners and pipeline jobs. We call this approach Zero Idle because you pay only for actual CPU and memory consumption, not for allocation.

This is the #1 way in the world of running pipeline jobs in terms of cost/performance. It beats everything from AWS, GCP, and Azure to GitLab.com SaaS runners, since they all are built on traditional cloud instances. If you don't use GitLab yet, you should consider it just to be able to boost your operations with puzl.cloud :)

Forgot to say: everyone's welcome to our community Discord server, which we opened a few days ago. There's anything about devops, Kubernetes, GPUs and cloud computing in general.

https://discord.com/invite/wt8GzF8aG4

Check out https://dime.run/ if you need something for Github actions

> Join the waitlist today

If you're not ready for the general public, what makes you think posting on HN will get you even _less_ traffic?

We are slowly onboarding customers in batch. The waitlist is for batching, not because the service is not ready. We do need more traffic to get better efficiency and predictability in batching.

I actually thought about building the same thing today, iops is slow & the CPU is outdate in GitHub actions & scaling it up is quit expensive.

Yep, GitHub is the next in our roadmap. A bit more tricky to scale it, so we decided to start with GitLab, first.

You can use namespace.so today which provides the best performance for Github actions in the market.

With the added capability of cache volumes: high-performance zero-cost cross invocation caching.

Built by a team of ex-Googlers and ex-Digital Ocean.

Disclaimer: i’m founder and ceo (happy to answer any questions!)

Looked into your product today. Only found a way to make docker build commands faster.. while that's a big part for sure. It has nothing to do with our needs right now.

This is fantastic - we do something similar for Github actions with WarpBuild.

There's just so much time and compute wasted otherwise.

Thank you for the feedback! Yeah, we're in the same niche: slow and costly pipelines are annoying.

Reply on news.ycombinator.com