Pyinfra automates infrastructure super fast at scale
pyinfra.com
pyinfra.com
Show HN: pyinfra v2 - https://news.ycombinator.com/item?id=30999030 - April 2022 (2 comments)
Pyinfra v2.0 Released - https://news.ycombinator.com/item?id=30973976 - April 2022 (3 comments)
Show HN: Pyinfra v1.4 - https://news.ycombinator.com/item?id=26983266 - April 2021 (3 comments)
Pyinfra – automate infrastructure super fast at scale - https://news.ycombinator.com/item?id=23487178 - June 2020 (64 comments)
Pyinfra v0.2 - https://news.ycombinator.com/item?id=12956784 - Nov 2016 (2 comments)
I use Ansible currently to manage the config on a Ubiquiti EdgeRouter. Using actual Python would be even better.
Most operations rely on various Linux/similar tools but the `server.shell` operation plus shell flag above should get you connecting and executing commands. Please do reach out if this doesn’t work for your setup!
I'm guessing the former since it's agentless.
Only downside is I couldn't make it work with my SSH agent, but that might be a problem with Paramiko and not Pyinfra.
Interested to hear more about this, might be something that can be added to paramiko (or on top of).
So this is the real thing. I am going to use it for my DevOps life.
Keep it up!!
- you use an actual programming language (which you're likely to already know) instead of desperately figuring out how to simulate a for loop in yaml or to replace a part of a string, or whatnot
- you have all Python static analysis tools at your disposal. mypy/pylint etc can make your deploys less error prone
- you can easily implement a custom DSL in python, so deploys ends up more declarative and with less copy pasting
- it's easy to implement custom arbitrary operations (since you're working in python) -- and again you can reuse variables, etc instead of having to use some weird templating or pass them in command line
I'm honestly trying to understand whether this hating on Ansible is more of a bandwagon thing (similar to how people shit on the Python dependency management ecosystem) and stems from having bad experiences that have completely turned them off of it.
Honest questions in response to your points:
1. What are you deploying where you would need static analysis?
2. If you did end up writing your own DSL on top of Python at what point would you concede that it's probably better to use someone's else creation as opposed to forever iterating on your own thing for the sake of using your own thing?
3. Why would you want "custom arbitrary operations" and how custom are they that Ansible and Jinja2 can't do that for you in usage patterns that are far more common than your own creation would be?
---
[1]: https://mitogen.networkgenomics.com/ansible_detailed.html
So I ended up doing some crazy hacks passing around state instead of just writing a function.
https://docs.ansible.com/ansible/latest/dev_guide/developing...
Then just writing a function.
https://github.com/willhallonline/docker-ansible
Still multiple major releases behind but still
Also I wonder how often the new releases are used (market share?) since so many people moved to cloud/k8s
For one, YAML can only emulate one kind of DSL: a declarative language. You have no advanced operations on top of YAML without using modern YAMLv3 syntax which is honestly mind boggling to read at times. Python is far more expressive and has tooling that can strengthen your operational resilience.
> I'm honestly trying to understand whether this hating on Ansible is more of a bandwagon thing
I've heard this expression echoed commonly when ones favorite tooling is under critique. Nobody hates Ansible, people do have experience that's taught them not to use something. Whether you feel your case overlaps with theirs is an exercise that pithy internet opinions can't sort out for you.
> What are you deploying where you would need static analysis?
Anything sufficiently dynamic. Static analysis is for making things less error prone. There's still all the same runtime problems, but at least there's an expressive path to fixing the runtime problems.
> 2. If you did end up writing your own DSL on top of Python at what point would you concede that it's probably better to use someone's else creation as opposed to forever iterating on your own thing for the sake of using your own thing?
That's a value judgement as to whether the person you're speaking to feels comfortable writing and maintaining a DSL. In my own experience, I've written things on my own that I had full control over that were sufficiently narrowly defined and I was able to move fast because of those things. I've also sat around waiting for features I'll never get or waiting for someone else's use case to become mine without avail because someone refused to "reinvent the wheel" without realizing the wheel they think exists isn't real.
> Why would you want "custom arbitrary operations" and how custom are they that Ansible and Jinja2 can't do that for you in usage patterns that are far more common than your own creation would be?
The custom orchestrators I've written were compensating for software that is deeply entrenched in it's infra and vice versa. Jinja buys me nothing that writing in Python or Go can, and Ansible is providing facilities the languages already give me. Ansible, in this case, is very much there to serve Jinja.
Of course, that's not to say that either way is wrong. Sometimes it's appropriate to use Ansible. Other times it's appropriate to write your own. Even in the latter decision, by doing so, it affords learnings about what you'll need beneath the hood so you can make educated decisions if you roll back to the former.
It’s been too long since I did perf testing but I still believe pyinfra Will significantly outperform ansible for the same tasks, you’ll have to try it ;) (also if perf is not good please raise an issue perf is absolutely a feature I wish to maintain)
Anyone want to post their experience using it here?
I would love to test beyond that but haven’t had the opportunity sadly! Performance and scale is a headline feature so will always look to push this further.