Fizzadar

Extremely aware of this (see pyinstaller attempt): https://github.com/pyinfra-dev/pyinfra/pull/768)

I chose Python because it’s what I was writing all day back in 2015. Which makes me realise pyinfra is almost 10!

Edit: I mostly write Go or YAML (k8s) these days but Python still makes an appearance from time to time (outside of pyinfra dev).

Conceptually I think it’s much nicer to define the state of the system rather than the steps to get there, and tool of choice figures it out.

But there’s always edge cases and situations that doesn’t work which is why pyinfra supports both and they can combine any way you like.

Interesting, not heard of CDK before! Kind of similar? As long as the language is Python I suppose! Would be possible to integrate with other languages too I guess, not something I’ve ever looked into though.

Totally agree on templating which is why inventories have always been python code just as operations, giving maximum flexibility (with some complexity/type confusion drawbacks).

Yes... and no. It depends on the operation (the docs explicitly state if an operation is _not_ idempotent "stateless operation"). Operations are either:

- state definitions, "ensure this apt package is installed" (apt.packages: https://docs.pyinfra.com/en/next/operations/apt.html#operati...) - stateless, "run these shell commands" (server.shell: https://docs.pyinfra.com/en/next/operations/server.html#oper...)

Most operations are state definitions and much preferred, the stateless ones exist to satisfy edge cases where either the state-ful version isn't implemented or simply isn't possible.

I think I tried to shy away from specifically being "Ansible does this bad so pyinfra does this" and instead focus on the features that differentiate like "Instant debugging with realtime stdin/stdout/stderr output (-vvv).". But it seems like that isn't enough and the landing page needs to be more explicit in comparison. Ty for the feedback!

> Agree with those saying the landing page needs work.

Any & all feedback much appreciated! It's basically just a very rough copy of the README at the moment.

This alone is the entire reason I started working on pyinfra, loops in YAML is just evil.

You'd need to create the EC2 instance outside of pyinfra (ie in Terraform). This could be done as part of the inventory itself, but wouldn't self-delete afterwards. If using Terraform there's a connector that allows you to plug Terraform output as a pyinfra inventory: https://docs.pyinfra.com/en/2.x/connectors/terraform.html

> It is a configuration management tool, like Ansible?

Yes

> Is it meant for running one-off commands across the infrastructure, like Salt?

Also yes.

> It says it integrates with Terraform, so it's not a provisioning tool...

The TF integration is specifically to use TF as an inventory source - ie TF to create resources and pyinfra to then configure them.

> What does it do different (and presumably better) than other tools?

The homepage covers the highlights, I originally created pyinfra because debugging Ansible was complicated (no plain stderr as not "just" commands on the remote side) and slow, but things have evolved significantly since then.

> The Getting Started guide doesn't cover this. The FAQ doesn't cover this, and the Docs doesn't have an Introductory section to cover this.

Hugely appreciate this feedback, this is super helpful and something I will attempt to make clearer.

---

Quick attempt at a better explanation: You write Python code that defines operations (either state "this apt package should be installed" or stateless "run this command"), provide an inventory of targets (SSH, local machine) and pyinfra executes it.

Roughly sits where Ansible does for configuring servers, but also solves the case of "how do I run this command across my server fleet" (which I believe Ansible can also do).

Hey all, I'm the creator/primary maintainer of pyinfra! Super excited (a little terrified) to see this on the frontpage, happy to answer any questions :)

I also hang out on the Matrix room: https://matrix.to/#/#pyinfra:matrix.org

Another thing: the GH repo points at currently in beta v3 and the docs for this are here: https://docs.pyinfra.com/en/next (highly recommend starting with v3, I just haven't had any time recently to wrap up the release, but it's stable).

Heh, saw the UK in the headline and expected another leap towards our 1984 inspired future. Nice to see a change that actually benefits us that live here! Small step in the right direction.

Both Terraform and Pulumi differ slightly I think provisioning cloud resources - pyinfra can be used alongside to setup instances/etc, I’ve used the pyinfra/Terraform combo with great success.

For ansible/chef, etc the main reasons/benefits boil down to:

- instant feedback esp on errors, get the stdout/stderr of whatever command pyinfra was executing, there’s no agent or abstraction to hide it

- configure in python rather than yaml+jinja2 mess

- integrate with the whole python package ecosystem

- speed and small overhead as inventories scale

Missed this comment apologies!

- ops are (mostly) declarative, but some (server.shell) will always execute the command given

- inventory is just that, basically a list of hosts to target plus associated data, docs page: https://docs.pyinfra.com/en/2.x/inventory-data.html

- absolutely for syncing files, check out the files.put and files.template operations (and the files ops in general): https://docs.pyinfra.com/en/2.x/operations/files.html

Thank you for posting this! Happy to answer any questions anyone has :)

I say all this as a MacBook, iPhone and AirPods user (:

Honestly Apple’s response to the DMA changes is pathetic, they’re acting like a petulant child. I really hope the EU throws the book at them. Will believe it when I see it, fingers crossed.

Really good write up. Been using gevent in pyinfra[1] for years and swear by it. Had some pains with setup, and am usually very wary of such magic, but it’s just really solid. Mostly write go these days though which has taken the shine off for sure!

Twisted, however, is a different beast. Have spent s decent chunk of time working on Matrix synapse homeserver[2], written in twisted, and oh my it just sucks.

[1] https://github.com/Fizzadar/pyinfra

[2] https://github.com/matrix-org/synapse

Another huge Tinc fan here. Used it in prod for 5 or so years before switching to zerotier for easier management as we grew. Tinc is rock solid and dead easy to configure.

Used to live a 15 minute walk away with my (now) wife and we would spend every Saturday walking through different routes ending up at The Stag (great pub). Such a wonderful place, really felt like the countryside yet in the city… fond memories.

Yes! I know of people that have connected to systems without any traditional shell/unix stuff. You can change the shell (or have none) used if needed using the global `_shell_executable` global arg (https://docs.pyinfra.com/en/2.x/arguments.html#shell-control...).

Most operations rely on various Linux/similar tools but the `server.shell` operation plus shell flag above should get you connecting and executing commands. Please do reach out if this doesn’t work for your setup!

I wrote pyinfra so am biased but we used it at my previous company on a fleet of roughly 1.5k physical boxes (including deploys spanning all of them at once) extensively, so it scales to that level well. We had probably 30 groups within that each with their own deployments as well and all tied together using a single Jenkins box.

I would love to test beyond that but haven’t had the opportunity sadly! Performance and scale is a headline feature so will always look to push this further.

> Only downside is I couldn't make it work with my SSH agent, but that might be a problem with Paramiko and not Pyinfra.

Interested to hear more about this, might be something that can be added to paramiko (or on top of).

Way back when I started pyinfra my only goal was performance I recall turning a 40 minute ansible run on 500ish hosts into 5 minutes. Big caveat: anecdotal and pretty old information there.

It’s been too long since I did perf testing but I still believe pyinfra Will significantly outperform ansible for the same tasks, you’ll have to try it ;) (also if perf is not good please raise an issue perf is absolutely a feature I wish to maintain)

Hey all, I (& many amazing contributors) built pyinfra! Crazy to see it on the homepage I’ll do my best to answer any questions!

Not sure where this is from but there's a critical part of the quote missing here:

> Every app that requests access to restricted scope Google user’s data and has the ability to access data from or through a third party server is required to go through a security assessment

An email client that only transmits data to/from Google's own IMAP/SMTP servers does not have the ability to access data through any third party server, and thus does not require the audit.

Source: https://support.google.com/cloud/answer/9110914?hl=en#zippy=...

The OAuth verification FAQs state:

> Ensure your app complies with the Google APIs Terms of Service, Google's API Services User Data Policy, and the Additional Requirements for Specific Scopes, which includes undergoing an annual security assessment if your app accesses restricted scope Google users data from or through a third-party server.

In the case of an email client data is transmitted directly from/to Google's own IMAP/SMTP servers and not a third party, and is thus exempt from the assessment.

I’ve gone through this process for my email client Kanmail [1]. The third party audit is not required for email clients that run on end users computers and store credentials locally.

By the looks of it Pegasus falls into this category and should not have any issues getting approved (still need the YT video and such but the Google team are surprisingly responsive and helpful in my experience).

[1] https://kanmail.io

Author here happy to answer any questions about pyinfra!

I actually find it better in recent times! Gmails spam filter has become noticeably worse for me (false positives and false negatives) whilst Fastmails is OK. Doesn’t catch everything but it never puts legit stuff into spam either.

Thank you!

So far the project hasn't cost anything directly financial, just my time which I intend to keep putting into the project because I enjoy it :). I use pyinfra day to day and it's great to see others getting use out of it!

There's a small but growing list of excellent contributors who really help drive things along and I'd love to grow that. A lot of time I put into the project recently and going onward is more around documentation and community to help with this.

I suppose another thing to note here is at it's core pyinfra is a pretty small codebase that doesn't require massive (time/money) effort to work on. The majority of code is in the operations themselves which I don't foresee expanding much (in favor of 3rd party packages).

Hope this provides some detail!