OpsMop – Config management and app deployment from the creator of Ansible | Hacker News Reader

OpsMop – Config management and app deployment from the creator of Ansible

github.com

128 pointsyankcrime58 comments

DSLs always seem like terrible relationships. Fun and exciting at first but soon she's telling you what to wear, what to say, who you can hang out with, etc. Oh sure, you can "Do what you want", and "It's fine." But you know she will punish you for it later when you least expect it. You would break it off but now you have so much invested she is integrated into your life. Maybe she will mellow out if you "put a ring on it" and buy a support package...

I may be wrong since I just skimmed the docs, but this looks like an attempt to allow general purpose python when needed, rather than being fully locked into a DSL.

Righto (author here)! It's a lot more relaxed (IMHO) than some. Most methods are raw python. The "set_foo" methods are just methods that get called when the objects are constructed to populate certain fields. You can just pass those in and leave them off if you want.

The question I don't see answered anywhere on their website or repo is "why?". Why in 2018 with

a) many mature configuration management systems like puppet, chef, ansible, and salt

b) lots of architectures that no longer require traditional configuration management (e.g. "serverless", immutable deployments of VM images, docker images on hosted kubernetes)

is there a need for a new CM tool? What is OpsMop doing different or better that gives it a hope of gaining traction?

From an initial glance it seems OpsMop is more of an extension of Python than a stand-alone tool built using it.

I think OpsMop allows you to create Python scripts that include deployment tasks along with "standard" code.

An interesting approach in my opinion.

I think Fabric http://www.fabfile.org/ does some of that already.

Ok, so Fabric did exist even way back in 2012 when I did ansible.

The common pattern of the day was to use Puppet to do OS config management and then Fabric to push out the apps.

Fabric lacked declarative resources to describe the desired state of the end system, which is where Ansible came in.

Ansible differed because it still allowed direct control over multi-machine orderings, which is (IMHO) easiest in push modes, so it was kind of (IMHO) the best of both worlds access models.

OpsMop has the same kind of push model, still keeping the declarative resources, it's just all Python.

However, on the remote end, it uses mitogen, so very efficient RPC ops (1 per role + files) - https://sweetness.hmmz.org/2018-12-18-mitogen-opsmop.html - versus the multiple of SSH operations ansible did per task.

And not to mention being able to utilize Ansible playbooks to build those Docker images too.

https://www.ansible.com/integrations/containers/ansible-cont...

Michael DeHaan, creator of Ansible, wrote an intro here:

OpsMop: Building the Next Generation of Configuration Management Tooling ==> https://medium.com/@michaeldehaan/opsmop-building-the-next-g...

Asked Michael DeHaan on Twitter today about the "Why?" https://twitter.com/ElijahLynn/status/1075446656533811200:

"Efficiency in both operation (speed) and development of plugins is a big one."

Tools which are basically dumbing down a generation in exchange for efficiency at scale. The day will come when provided with three whiteboxes, install media, switch + cabling -- there will be a lot of nothing going on.

Or - better yet: the blue collar guys do that.

on b -- if you're building containers and other "immutable infrastructure" correctly, you're still using CM. That so many think a Dockerfile accomplishes the same things is a testament to how poorly most of us understand what we're doing.

As a "devops engineer" cough sysadmin cough, configuration management is still a freaking nightmare.

I write too much YAML, I do not know why everyone all at once decided they were done building abstractions and instead wanted to write a ton of boilerplate.

I use ansible currently for our SMALL business. I have thousands of lines of ansible to do relatively simple tasks. YAML is not a programming language, you have to repeat your self over and over again. If you get tired of doing that, you are out of luck. You are not writing in a programming language so you cannot develop an abstraction.

That being said, the main advantage of Ansible is that someone has already gone in and written all of the boilerplate code for provisioning various resources in my cloud provider. Unless OpsMop has this, I don't see the value-add.

I switched from chef to ansible because in my opinion a DSL is the wrong way to go for this stuff. Granted I've only ever managed fairly small deployments. But so much more can go wrong when you are writing code vs writing yaml. If your yaml parses it will probably work. I write ruby application code everyday and I love ruby. But I hated writing chef scripts. I also ended up with many fewer lines of yaml than ruby.

Requiring you to be a developer to do automation is not a sustainable strategy. Ansible is a better fit for non-developers to do automation, which helps drive adoption

And since it's agentless you can automate almost anything... As mentioned above - the playbooks created by the community avoids you from reinventing the wheel. Companies like Cisco are writing playbook to automate their switches

Shameless plug, you might find judo interesting: https://github.com/rollcat/judo

This is ansible for people who like Python.

It's kind of sad because I feel I've invested 5 years into Ansible but now Michael releases something equally impressive seemingly based on experiences and lessons learned while building ansible.

So it's very hard not to just abandon Ansible and go with this new shiny tool.

They are just tools. People will come up with better ones every other year, and we'll abandon the old stuff. That's ok, components should be replaceable without having to reinvent the whole thing.

That's ok, components should be replaceable without having to reinvent the whole thing.

A fine, admirable and accurate (IMO) mindset. Things get weird when recruiters find out about these new tools and suddenly we're having that time honored discussion about how 'tech hiring is broken' because the people screening resumes don't realize OpsMop hasn't been out for 4 years and no one has 4 years experience with OpsMop.

Everyone scrambles to learn OpsMops-so they can get a job where OpsMop ends up being abandoned and the DevOps team goes back to ansible, other teams start adding it to their tool chains because someone got hired who fiddled with it once and becomes an OpsMop house, a few engineers get very good at it, realize they can probably write a better version, three years later someone's posted "Show HN: OpsBroom" to the front page.

Repeat.

I suspect people will probably adopt it if they like it mostly for new projects.

We still do have a lot of modules to write yet, so if you are interested in that, stop by talk.msphere.io - I'm open to a lot of enhancements there, but also want to keep it somewhat controlled.

A lot of the early adoption is no doubt going to be folks doing CM-based builds for images (EC2, docker, whatever), where everything remains pretty disconnected.

A mid-way adoption stage might be using opsmop to run existing ansible content, though ... that's really pointing towards wanting to add something like rsync, which is easy, but... I need to think about that a little bit.

> So it's very hard not to just abandon Ansible and go with this new shiny tool.

Huh? No it's not, not yet. It's clearly marked as alpha software.

I'd say it's already ready to be adopted by some personal or PoC project.

And I'm finding the Python DSL very attractive in theory. At least I'm assuming I'll be able to use it just like I use Python. So I'm not restricted by opsmop if I want to lookup secrets in secrets management API of my choice for example.

Or connect the inventory methods to whatever asset mgmt API I might be using. Possible with Ansible too of course but using Python as DSL gets rid of the entire necessity of ansiblelib and following along with its API.

Although I mostly loathe DSLs and have been wanting something almost precisely like this for quite some time looking at the examples I think I might prefer the Ansible YAML approach better.

Ansible YAML is definitely more tidy and concise despite Python's awesome readability. And even though you get all the benefits of the full Python language is a definite advantage (try turning a list ["www1", "www2", "www3"] into the string "www1:443,www2:443,www3:443" without plugins or some serious ugliness) I wonder if having Ansible/Jinja2 preclude most of those abilities is a feature and not a disadvantage.

Either way as someone who has just found himself in an environment that uses Chef/Ruby for Ops stuff and Python for everything else I'll be watching this closely.

Based on Ansible and other systems going all the way back to GNU make, I believe that the programming language version wins every time.

We're programmers, and sometimes we're going to need to program to solve this problem. Taking away that option is always a pain. Even if what you wanted was possible with Jinja2, it wasn't necessarily very readable anyway.

Proponents of it will point at that this is the sort of thing that Lisp is good (and they'd be right), but there are other functional languages that are good at it too - the simpler syntax is a bit of a headstart. Elixir's macros system enables a number of lovely little DSLs (e.g. Phoenix router or Ecto DB schemas).

I agree that your problem is much harder than it should be and I think it is due to the format() filter not being fit for use in map(). Nevertheless here are 3 versions how you could solve this problem in Jinja2:

    {{ list | map('regex_replace', '(.*)', '\\1:443') | join(',') }}
    {{ list | zip_longest([], fillvalue='443') | map('join', ':') | join(',') }}
    {% for item in list %}{{item}}:443{%- if not loop.last %},{%- endif %}{%- endfor %}

I wish the following would be possible:

    {{ list | map('format', '%s:443') | join(',') }}

I ended up using a custom plugin to make it easier. The actual real world attempt was more complicated and had a crappier version of Ansible. I also wish the map/format one worked.

My historical argument was always that if a playbook looks ugly, it is wrong.

Over the years I haven't been around to keep saying that, but I'd be inclined to have to find ways to open up the programming model while still keeping the mostly organized guide-rails in place.

Out of curiosity I checked it out. This feels a lot like Chef or Puppet DSLs. On the pro side, this gives you flexibility to write whatever convoluted logic you want in code. But that also means you need to know the quirks of each DSL once you start to deal with modules.

And despite them calling the language a DSL, it really is a subset of a general purpose programming language, so there's if readability is a goal, well there's going to be an unavoidable amount of line noise.

As a matter of personal taste, I find this to be less readable than Ansible's YAML. Though it's not the most elegant way to express things, it's still a lot cleaner and predictable than code.

In all fairness, the author addresses this in his documentation:

> You might like the language of one thing more than the other. Pick that thing.

This looks promising. We still use ansible because we have a lot of time invested in the roles we've built, but I _so_ hate the yaml DSL of it and all I ever want when writing a new role is the ability to debug it properly with a decent toolset and logging.

I was worried this would be proprietary commons clause licensed like the creators vespen project... but I was pleasantly surprised that is is open source, Apache (real Apache) licensed! Further, it appears that Vespene has been open sourced under the Apache license as well!

Thank you Michael for creating these projects and releasing them as Open Source! I am a big user of Ansible, and have often wished there was a good way to write plays in Python. I will be keeping a close on on both of these projects!

You are welcome! Yeah the thing I was worried about with that kinda went away.

Feedback would be welcome, stop by the forum sometime!

So. With Ansible's YAML it was always a pain when your configuration grew beyond what's envisioned and you needed some more conditions and loops―because the programming functionality was always incomplete and needed to be implemented in every new case (e.g. you couldn't do loops of roles when I last checked).

I guess Mr DeHaan decided that this was a mistake and returned to the beaten path of full-featured programming. In which, of course, you can't spill config data into your files as freely as you would with YAML, since it has to be distinguished from language constructs and thus encrusted in syntactical markings.

Now, it seems to me that Lisp better marries data structures to occasional programming functionality, especially for Ansible where every part of a config is a declarative value first, and an instruction secondly. More so compared to Python, which uses statements instead of expressions in plenty of notorious cases.

Of course, you'd still have to mark values with more syntax than in YAML's free-flow, and I guess Lisp isn't good at embedding expressions in strings. In this regard, I'm very curious to see one day a lightweight version of Lisp syntax reminiscent of YAML. Notably, Ansible's configs essentially use the syntax of "function: argument -keyword: value -keyword: value..."

I'm not sure if I've seen a Lisp-based Ansible clone. There's Vektra's Tachyon in Go, but apparently it only uses Lisp for embedded condition snippets: https://github.com/vektra/tachyon

"I guess Mr DeHaan decided that this was a mistake and returned to the beaten path of full-featured programming."

Nope, and I've written on this a bit -- I think Python is a better fit for those that want to program in Python.

For those that want to build their own CMS in something else, go for it!

If anyone modifies this thing to use real YAML (not whatever Ansible was passing off as YAML) or HCL as the configuration file format, please post it to HN? I'm never going to use a programming language DSL.

What I really want is a real document standard that describes common configuration management functions, and then implement a parser in every CM tool, so all of them can do the same basic functions with one standard file. Like a k8s file that can describe Terraform-like modules/providers/resources, and each CM can have it's own implementation to build, test, and apply them.

As an aside, we really aren't using YAML to its full potential. We could benefit from an implementation of YPath so we can work with YAML files the way we use XPath with XML, and tags are greatly underutilized.

Whereas I am only willing to use a programming language DSL. In any interesting real world case, you are going to run into cases requiring some form of arbitrary logic. Some DSLs will provide you functionality to do loops, ifs etc., but you can't beat a real, actual, programming language.

So very true. More than once I created scripts that executed Ansible commands (with POPEN, exec and whatnot)..

That's assuming you should be adding logic. The whole idea of CM is to get away from writing programs. Logic adds complexity, is subject to bugs, and is hard to maintain. Most logic I've seen is merely adding procedural code rather than declarative, which is a terrible pattern for a CM.

I have worked with nightmarishly complicated Ansible playbooks and roles designed to provide declarative infrastructure when it didn't exist in a module. If all you had to change was a group_vars setting, you could be confident in your change. But if you had to edit a role, playbook or task, nobody was sure if it would break something in the future or not. It was much less robust than a real module because it was just the DSL.

With a Python DSL you could write code that was declarative, but because the language is complex, lexing is more difficult, and simple changes to configuration become more error-prone.

By moving to images and containers, the majority of configuration management is now largely unnecessary. We will always need a CM tool, but they should give users less rope to hang themselves with, not more.

First off, it sounds like your playbooks sucked - sorry about that, with great power comes great responsibility. Even as I wanted to make things simple, all things could be abused, and in adding more flexibility I've seen some evil things done in my name :)

So as immutable systems grow - that's a damn good thing. I personally think the docker ecosystem is a trainwreck currently, but that's only my opinion, and doesn't really affect anyone else. But the same concepts have been around for ages in EC2 in immutable systems.

What typically happens is people use the CM tool to provide a higher level abstraction than Bash for describing the image build, so they get features that should have always been in docker files but never were (templates are the biggest one for me).

Further, there is always a need to deploy the "undercloud". Even in heavily containerized microservices setups, there's a need to handle all the magic upgrade fun of upgrading a Mongo cluster, a Cassandra cluster, or a Kafka setup.

Whether we call it configuration management, app deployment, or just a structured way to push scripts around and see what failed, we're going to need those tools.

I believe immutable systems is a fantastic philosophy where it works. It doesn't work everywhere. And CM tools still have a niche for those that want to use them in the build process.

If folks want straight docker files instead, I'm also cool with that.

"(not whatever Ansible was passing off as YAML"

It at least was parseable.

The trick was, it was always supposed to be at the level of complexity of "grocery lists, but..." ... but frequently I think people tried to program in it a little hard.

OpsMop is based on the idea of a cyborg system - writeable for humans, but easy to interface with machines.

I take some alternate views, and believe YAML is best kept streamlined, which is why I also did not use YAML anchors in ansible.

This is an early stage in the project but it looks like its intending to be a Chef or Puppet in python with a terraform-esque plan and apply stage.

Isn’t Ansible a “Chef or Puppet in Python“?

Yeah, but yaml.

Ansible is a DSL in yaml. Its written in python but youre really only exposed to the jinja templating. This is way more pure to the language.

Very different task and process model by default (ssh is a first-class citizen in Ansible). There’s also differences in the default mode of push or pull from a source of control (ansible pull mode is sorta similar to Salt’s minions and Chef / Puppet master servers except with a git repo typically). Chef and Puppet are much more traditional

Ansible is an imperative language while Puppet is declarative.

Ansible is push mode (ssh connection) while Puppet and Chef have an agent that pulls and enforces configuration.

Ansible is great for quick hacky or one time solutions like building a server. It's very newb friendly and easy to use out of the box. Puppet is great for enforcing a configuration every 30 minutes and scales across Data Centers. In my opinion Chef is more robust and is easier to use and share cookbooks on Supermarket.

Ansible Galaxy isn't as great for sharing work, but I find Ansible is so easy to use I don't need anyone else's work.

Once this has a pull mode (agent) it seems it will be like a very lightweight Puppet like declarative tool. Both Puppet and Chef have very mature products for serving and managing infrastructure. They can scale.

This tool might be awesome years from now, but I have to use Ansible everyday at work. From 2.0 to 2.8 it has become a solid and fairly mature product. I also use Puppet and have for 10 years. It feels old and is a pain to use across 1000s of machines in an Enterprise. There is a learning curve and training to be done at all times. At my last job I used Chef and I really love it for being powerful, easy to use, and just having a great design.

We currently install Puppet with Ansible once initial config is done to enforce policies. I could see this potentially replacing the Puppet side of that equation some day.

I'll for sure watch this tool and experiment with it in my lab as it grows. I do love Python and Puppet is far to long an overhaul. It will be interesting to see what directions are taken as this new product matures.

I do not think this is about using python instead of yaml. It is super easy to write Python modules for Ansible and use them. We have to use them for the things that aren't included in the base such as managing Infoblox etc.

There is ansible-pull. It's not as useful as one might initially think, as there's lots of ansible code written assuming push mode.

Upon looking around in the demo repo, this seems like so close to pure Python that I almost don't see the point. A Terraform for config management sounds grand, but this just seems like a collection of Python classes.

The lesson learned from ansible probably is: if they still don't use any kind of ansible, chef or whatever. They never will. (my current job for example) but they might accept something like a "base framework"

It's ok if you think of it as such, the veneer over that actually isn't that deep, and that's ok.

The trick is there are some norms you want to express in the language, things like handlers and such, and if you write directly at low level python every time that can be sometimes difficult to maintain a level of consistency (IMHO).

Here's where it's NOT just a collection of python classes - https://sweetness.hmmz.org/2018-12-18-mitogen-opsmop.html

Check out habitat if you like the sound of this: https://www.habitat.sh/

I'm a big fan of the Chef people and they are good folks. Habitat is about a way to distribute applications to remote nodes and is somewhat an alternative to docker. It's kind of apples and carbeutors. Just like Habitat does not compete with Chef, Habitat and OpsMop are also very different things.

Chef is a closer thing to compare it to.

I just want to say the name OpsMop is awesome.

I may be late to realize this but Python is becoming a mandatory language to master for DevOps people.

I guess it's time to learn ...

uhm. no unittests for the project ? :((

Valid point. I started this about a month ago, and as APIs were evolving I did not want to slow my velocity by having the tests fight against shifting it around.

I suspect the first release will be in about Feburary, but the time to collaborate on working on it is now.

It is designed to be fairly testable, but if you've ever written a similar system, you know that distributed testing of modules on a matrix of 12 platforms IS a trainwreck.

Any units would mostly be proving out the language.

The best way to do that is with a specialized callback, that shares the event stack, and we can then introspect that the events happened in the correct order.

As the callbacks have been changing too much now, writing a boatloat of tests would have been a mistake.

At this point, I'm most interested in feedback on language features, and these will come.

It was an absolute mistake that ansible itself was not more testable at a mock level, a problem made a lot better now by everything being done in one process.

Ansible had the ability to write modules in any language, but this was never really used in practice.

I'm all for choice in configuration management. That said, I find this quote on their page amusing:

> The purpose of the Opsmop community is to come together to build the ultimate configuration tool we all want to use.

Kinda reminds me of XCKD: https://xkcd.com/927/

I'm all for more standards! LOL.

Actually this was meant to describe what we do when we work on OpsMop. It's like "hey, let's all figure out what we want in an ideal universe and build it together". Versus like "hey, you're all my user peeps"

It's interesting, we have developed a very similar framework internally for our appliance and use it in our product for 4 years. We "present" files from templates filled with variables coming from the user interface and those presenters call "notifiers" (e.g. restarting a systemd service when a file changed on disk).

Reply on news.ycombinator.com