mpdehaan2

Ok, so Fabric did exist even way back in 2012 when I did ansible.

The common pattern of the day was to use Puppet to do OS config management and then Fabric to push out the apps.

Fabric lacked declarative resources to describe the desired state of the end system, which is where Ansible came in.

Ansible differed because it still allowed direct control over multi-machine orderings, which is (IMHO) easiest in push modes, so it was kind of (IMHO) the best of both worlds access models.

OpsMop has the same kind of push model, still keeping the declarative resources, it's just all Python.

However, on the remote end, it uses mitogen, so very efficient RPC ops (1 per role + files) - https://sweetness.hmmz.org/2018-12-18-mitogen-opsmop.html - versus the multiple of SSH operations ansible did per task.

Righto (author here)! It's a lot more relaxed (IMHO) than some. Most methods are raw python. The "set_foo" methods are just methods that get called when the objects are constructed to populate certain fields. You can just pass those in and leave them off if you want.

ha, totally not going to write another CM layer. That is one of the fatal blunders, up there with fighting a land war in Asia and all that.

There's a fair amount of that available in http://docs.vespene.io/importing.html ... if you have more ideas can you stop by https://talk.vespene.io/c/ideas ? ... I don't want to make the syntax more complex than the YAML that is there now, but I think we can add other fields if there is something you want or a capability that might be missing.

Thanks so much. I've had this conversation with a lot of founders and we all feel we are in a race to the bottom of sorts where the power dynamic is all with the big corps.

We don't like open core but proprietary software is the best way to make money, so I'm trying to push that just a tiny amount.

Indeed!

I wanted another Science Fiction reference to something I loved.

Vespene gas is very important in build order.

My probable worst name was Cobbler (the PXE server) for those that were into cockney, but it was originally named "Shoe", and the idea was a Cobbler makes boots.

Lo siento!

While you're welcome to that opinion, Django has been freaking awesome. This whole thing got written in 3 months, not even working full time. I wrote pipelines in like a day.

It seems like you're saying I should have used a many-to-many there rather than 7 FKs. Probably, but shrug... it can be fixed later if it ever becomes a problem.

See https://medium.com/@michaeldehaan/going-with-the-commons-cla...

No discontent. Quite the opposite - providing some options for less things being open core while trying to earn a small amount on my work.

If people want to call that not meeting some definition, I'm ok with that, but ascribing malice to it seems a little harsh.

There really aren't any liabilities any more so than in violating the GPL, and people have been over that for eons. Don't sell it without joining the partner program, more or less. Rates are posted, it's cheap, and helps support the upstream.

At least, I don't think I'm evil. Some people might not know me, but I don't feel evil :)

Compiling C code with puppet or whatever seems a little weird to me, to be honest :)

If you want to keep your data externally, there are some good options for this. I'd read the plugin docs about variable plugins, that can easily source variables for things like etcd or Consul.

A script for a given repo could be nothing more than "make go", and that actually is a pretty good practice, to keep that in source control.

Still, you need something to build your code, and to have a good place to see the status.

Ultimately, code still needs to be built, and orgs do want to avoid images they cannot easily recreate. That's the role of a build system and making sure the process to create those artifacts is in source control.

I should also mention that your build script CAN be sourced from source control.

In your .vespene file, just say "script: <path>.

However, the variables in that file can be evaluated by Jinja2 variables, so for instance your security team could set up a snippet everybody could use or your feature flags could be defined from there.

Another cool feature of Vespene is in each project build root all those variables appear as vespene.json files, so it can be a good tool to use to launch all kinds of automation from a web console where you want to record results.

Hey!

Any node typically would run the webserver because it's not that heavy, but doesn't technically have to. It could just run one worker process. Right now the workers do need database access and I suspect that will stay that way for the short term.

Windows needs to happen, but I'm not sure when it happens. (Also a good list discussion).

I'm a bit curious about your raspberry pi use case - maybe a good topic for the forum? http://talk.vespene.io/

I'd argue the definition of what is open source is up for debate, but I had some reasons for this and thought about it for weeks.

I hope people know my history and how I've run projects in the past.

But what this does for me is keep me from releasing anything open core. With ansible, ansible was open source but the GUI was not, and we had to hold something back.

The traditional alternatives to OSS are to make a support business, which somewhat encourages making buggy or hard to install software, or a consulting business (IMHO, same) and all of those detract from building a product, collectively, with some of your favorite people on the internet.

So basically Vespene is going to be like a pseudo-foundation.

The structure for this is described here - http://docs.vespene.io/partnership.html - which keeps consulting and support totally free for small shops, and encouraging larger ones that could potentially make millions form Vespene to give back a very small amount.

What normally happens is large companies close six figure contracts supporting open source, and they don't give back a dime, and my goal here is to essentially make a developer's salary off of this project by having those who make more give back a very small amount.

This seems pretty fair to me, but I also recognize that some people don't agree with it.

That all being said, it's not a "no-commercial" clause in any way, and still free for small consultancies, so I'm not anticipating any major problems.

Most of the time, software gets installed in a place of business for that business.

For some of my previous thoughts on this, I'd refer folks to https://medium.com/@michaeldehaan/why-open-source-needs-new-... and later my current thinking https://medium.com/@michaeldehaan/going-with-the-commons-cla...

It's not something I've considered lightly, but this keeps MORE software open, and for a small one-person shop, I think that's more noble than trying to hold some code back and not release it at all.

Some people just want 100% free, gimme gimme, etc - and fine - you're entitled to pick one of those things. That's ok.

I personally view this a little more pragmatically, under the view of fairness, and also have taken steps (in the CLA, etc), to guarantee that trust is never going to be abused.

For instance, the license reverts to pure Apache if Vespene were ever to change hands.

On the downside, I can't get screwed over by IBM or Amazon while I'm trying to crank out free software for everyone. That seems fair too, seeing the time investment running an open source community and working on the code takes.

It's fine if you don't like it - there are things I don't like about it, but ultimately single packages can't express multi-node configurations very easily. We need something.

While I "grew up" as it were, believing in RPM, we live in a very post-distro, multi-language kind of world, and there's a need for things to glue that together. How many times have I tried to campaign against "wget tarball" as a deployment mechanism, I don't know. It's rough and yes, there's a lack of discipline in ops that needs to improve.

Immutable systems is a VERY interesting way to solve that, but it doesn't work for certain stateful things and you always need something to deploy the undercloud.

I'd encourage you to try to build your own experimental project to try to find different ways to do it, as this is the only real way that technology ever gets ahead.

https://github.com/vespene-io/vespene is the one!

I think we owe Jenkins a TON of credit for being a free solution that has a lot of great traction.

I'm used to seeing pages of checkboxes and can't stand how obscure some of the configuration is, but mostly I wanted to write something that could handle configuration differences between hundreds of projects, so that is why there are things like Snippets in Vespene.

Plus, I wanted to make something that was a little better for ops use cases (so people don't also have to go pick up something like Rundeck), so that's why there are things like the SSH integration and http://docs.vespene.io/launch_questions.html

It's more about capabilities and future capabilities than the frustrations IMHO.

Though I do have a bunch of friends who are frustrated with plugin compatibility, plugin hunting (we're doing more of a "batteries included" approach like I ran with ansible - just with much less modules), and stuff like that.

I was also able to add in some stuff like container build isolation really easily, and that's all included stock.

I don't know, I'm really not trying to make one of those cliched checkbox X vs Y type grids, but when I started building Vespene I was most concerned about being able to make builds more consistent when you had hundreds of microservices developed by a lot of different teams. This is why I did http://docs.vespene.io/variables.html#snippets

I also saw a lot of people trying to do ops style stuff from tools like Jenkins, which is why Ansible has built-in SSH automation so it can hold on to SSH keys and use them in your behalf, and has some really cool built in RBAC so you can decide who can run what and it can be different than who can edit what.

Mostly I just want to build an architecture that can go interesting places - what is here today I think is usuable, but it's just a starting point. My thinking is if you make an architecture that is really pluggable, and the code is easy to read and add to, it can go to some really neat places.

I'd just recommend taking a look at the various chapters of http://docs.vespene.io/ to get a feel for features, and if you like it, spin up a copy using the setup instructions and see what you think.

Maybe it's just me, but I always have trouble figuring out what apps are by reading the documentation. I'd usually understand more if there was an architecture diagram, which also clues me in that I need to do one for Vespene :)

One other reason I didn't build something is I don't have a current need to use something like that, so I'm probably not the one that should be developing it. There's a lot of datacenter use cases I'm ultimately not super familiar with.

Sorry about that if I still have a typo somewhere - let me know where you found it if I do.

Still hard to keep myself from typing that :)

I'm pretty sure it's going to happen, or at least having Docker build files where you can run your own (on the workers, you'd want to install what you wanted anyway). I've had a couple of conversations from folks wanting to do that and it seems like it would be pretty easy, just making the entry point supervisor vs systemd.

See my earlier reply about http://docs.vespene.io/importing.html - though I'm pretty darn open to making this do a lot more in the .vespene file.

Vespene is pretty new so a lot of things are up for grabs. Pipelines were implemented in about a day, so the potential to make rapid upgrades is pretty big.

That all being said, I think you have 95% of that right there today with http://docs.vespene.io/importing.html

This is a JSON file, and it can define what pipeline a project is part of, and what stage it is in that pipeline.

So it's like "this project is part of the analytics pipeline, and it is part of the deployment stage".

But you still have to make the analytics pipeline in the GUI right now. This means that you have to say (and this is all) the analytics pipeline has the following stages in order: A, B, C, and then D.

I think it's quite possible to have the definition OF the analytics pipeline in the vespene.json too, but it's a bit of a question of which one wins if there are conflicting definitions.

If you want to stop by talk.vespene.io this would be a great thing to bounce ideas around on!

So back around 2003 I was working at Adaptec (an IBM spinoff) and we were trying to put together a build system, I think I ended up front-ending it around DamageControl, which was a Ruby port of CruiseControl, except I had to hand-code a fake "SCM" module to run timed (versus commit-based builds). Our builds took like 8 hours! At that time, I really wanted to help people with build environments but never really did it.

Then, what sort of happened is I kept running into overly complex Jenkins installs. The typical miles of config checkboxes was overwhelming for me, but a larger problem was that organizations would have like 200 microservices projects, and all the build scripts would be slightly different.

The idea was then to take the templating system from Cobbler - which allows lots of variables merged in at various levels, and Jinja2, and allow reuse of build scripts through variables and snippets.

There was some other frustration with Jenkins - the idea that it was a ginormous codebase and still not database based, and you had to hunt for good plugins.

Ansible was pretty successful for getting everybody together to contribute to common modules (it maybe took on too many though) and make sure you didn't have to go on too much of a plugin hunt. So could possibly we start a "batteries included" build system?

Finally, I wanted to try out some opportunity to merge the functionality of a build system with something like Rundeck, so you didn't need two tools. I also found Rundeck complicated to set up, but if you want to run some self-service automation, most people do that within their build system. So I figured I could add in some SSH and Q&A interactivity in there to easily get that going.

My other idea was to write a VM/container controller, and while I think there still needs to be a simple one as Kubernetes is getting to be a beast, that's too much work to bite off without a lot of help :)

Honestly the main thing is I just wanted to work with a lot of the same open source folks again (and a bunch of new ones), so I'm really looking forward to that!

Hey folks, this is my new project. You might possibly remember me from creating Cobbler and Ansible.

I thought I'd share and if you have any questions, let me know!

This seems like FUD to me.

While OSI may have coined the term "open source" as a reaction to the word "free software" in the past, it did not invent the idea of free software. Rather, the term open source was a reaction to the desire for commerical enterprises to avoid saying software was free.

Now it rejects the same argument from developers of software to make a profit, which I find ironic relative to the founding mission.

Previously, Commons Clause was called out in aggressive terms in twitter, rather than seeking to understand the underlying rationale.

Some of my thoughts:

https://medium.com/@michaeldehaan/why-open-source-needs-new-...

I am left to believe OSI views this as a useful political time to self-market, or otherwise sees licenses like this - which intend to fairly compensate software developers - as something that does not promote the interests of those that primarily fund it.

I'm sorry, but we don't need a gatekeeper anymore.

(In case the handle is missed, I created Ansible and I'm a little opinionated on this)

DevOps isn't really a thing, but an amalgam of things. Some people think it's about culture or something, which I think is too obvious to be a thing. In the beginning when many people were using the word lightly, most people really just used it to mean automated systems administration, which is more likely called "Operations" now. That's fine. Some people use it to mean groups of people who make ops tools to allow developers to self-deploy their own stuff. That's also fine.

Most likely what you are looking for is to learn how to do IT Operations stuff they way people are currently doing it.

Reading a lot of articles is fine, trying lots of tools is fine. Talking to people at your company that DO ops is huge. Make friends with the guys who run the build systems, do security, or anything like that for starters and they can show you lots. Plus I strangely find that ops guys are much better to go to lunch with than developers. Don't know why :)

You should read up on AWS lots, as it will likely be across your career path at some time. Try a configuration management tool (or two). Learn about monitoring systems and logfile collection/analysis systems. Do a little bit of reading on computer security. Vagrant is probably useful, but optional, though you should at least get going on a virtualized Linux box. Reading up on Immutable Systems is worthwhile. Pick up either CloudFormation or Terraform, or both if on AWS. I don't know Google as well, but it has a lot of similar things.

DevOps Days conferences can be good sometimes but often they are too cultural to get down into technical bits. But they are cheap and usually close by, so they are things.

If you have a local meetup group that can be absolutely great.

The really nice thing about AWS now is there are tons of parts and it is pretty cheap to try things out, where before you probably couldn't get your IT guy to let you play with a load balancer or get you your own database instance. Now you can, so that makes it a lot easier to learn than it was before.

IMHO I don't like books because they are often written by people who don't DO things (DevOps has an unfortunate "thought leader" problem, which impacts conferences and tries to get everyone to believe the same things), and podcasts/videos are too slow for me, and my brain is a lot more random access.

Don't get caught up in assuming you must do any one particular thing. For instance, Continuous Deployment is a spectrum, it's not appropriate for everyone.

And at most people's scale, you have no need for something like Docker or Kubernetes when basic AWS instances require a lot less to keep going in your head.

Developing on master is something I favor, with tagging for releases, though it makes sense to create release branches if you are delivering software packages (that others will install) and not just deploying it to .com websites. Hotfixes can then be applied to both the maintaince branch and master.

Don't appreciate being called delusional. That being said, you underscore the point that many CEOs are not great -- which is exactly the point about overcompensation. Also, see my profile.

It's not wrong to tell everyone they may have the skills to match their current mediocre CEO that is pulling down the megabucks today. And believe me, a lot of mediocre CEOs are pulling down megabucks.

Being you are in the VC game, you are dealing with people in startups, where the compensation hasn't gone crazy away from things yet.

Finally, some of these views may be seen as socialistic. It matters what you value. In general, I have seen a lot of VC decisions that value capital ahead of ethics, and this is unfortunate.

I'd suggest rather than look at it as you being overpaid, look at it as everyone else being underpaid - in other words, corporate profit sharing is not what it should be.

Sure, that's what share holders do. It's however not EQUITABLE.

To slowly change this over time, we have to work at it, by ensuring fair compensation at companies we start and for our employees.

This can change - it's not a statement of the present - the status quo is pretty bad - but a desire for the future.

To get there, we have to start expressing value for where creation comes from.

Again, see profile :)