GitHub Actions is my new favorite free programming tool [video]
bytesized.xyz
bytesized.xyz
Github documentation is a disaster. They leave out critical parts. They don't provide examples. Everything they write is terse, confusing, and incomplete.
They have short little articles on how to do things, and for each sub-task they have a link to docs somewhere else. This would be fine, except the links don't point to anything useful.
To give an example: they say you can use the Github API to talk to Github Packages, but the link goes to their generic GraphQL documentation. They don't point to any reference material on the actual calls to the packages service. If it exists, I can't find it.
If you go to the main page in your account for Github Packages, it says that all you have to do is this:
mvn deploy -Dregistry=https://maven.pkg.github.com/mycompany -Dtoken=GH_TOKEN
That is just straight-out wrong. It does not work.
Seriously, Github, you need to fire your documentation team and hire some people who know how to write. Perhaps you should hire people who have actually used your tools to write the docs. Or just provide some freakin' working examples.
It’s not just GitHub. There’s a language I (and the rest of HN) love that has adopted “story format” for documentation but is missing real, hard technical documentation apart from method-level codedocs. It’s nice for building momentum but it leaves you unable to find a central location with an architectural documentation of the system.
Writing documentation also seems to be a task you kick out to your junior developers that you'd rather not give more critical assignments to, which is backwards as can be.
They have quick starts, tutorials and how-to-guides. How are even those things conceptually different at all?. And their reference is just a list of their methods like if I knew out of the box how to use your API. Like, what the heck. Just tell me how to talk to your API in simple terms. How hard is it?
"To use the CGP vision API there are two ways to authenticate and connect. 1) REST 2) SDKS" and then just explain to me how to do it simply.
Don't ramble like if this was the fucking Iliad. Go straight to the point and organize your information better.
The API reference in particular you pointed to has a top-level description + linkable reference to each individual method / resource along with brief description. AND, for older versions, as well as a description of "objects" used in the API. I guess it would have been nice to also have a sandbox playground, like some of their other APIs do, but this one is fairly good?
Just curious, what's an example of great API in your mind?
*(possibly not in-depth enough, but I wouldn't be able to tell as a newbie).
I have to often disregard the documentation and get into the gcloud (their Python-based CLI and API) to understand what the heck is going on and their CLI/API code is no good either. Layers and layers of abstraction in their code only to make some REST API calls. The amount of over-engineering and abstraction that is present in their Python code would put even Java SimpleBeanFactoryAwareAspectInstanceFactory developers to shame.
For example integrating support for Google Drive took months, where I ended up using a third party solution, and users are met with a big red warning about my app now being unknown. And the file system is laggy. Compared to integrating Dropbox which took three days and just works. Google Drive probably have thousands of pages of documentation. While Dropbox have one or two.
Maybe Rust or Clojure...?
The TS docs basically just say "use declare to declare [thing]" over and over, and it's only mentioned in the handbook on a page that doesn't seem to be listed in the handbook menu anywhere.[0] You have to search for it and wing it, because the search result for this just says "By Example." Notice that there is no article highlighted in the handbook menu on the left like there would be on any of the pages listed there.
[0]: https://www.typescriptlang.org/docs/handbook/declaration-fil...
"Documentation needs to include and be structured around its four different functions: tutorials, how-to guides, explanation and technical reference. Each of them requires a distinct mode of writing. People working with software need these four different kinds of documentation at different times, in different circumstances - so software usually needs them all."
I agreed with its premise — I've found things that only have "story format" docs (i.e. explanation / how-to guides) to be really frustrating. On the other hand, if all that's available is a deep and highly particular technical reference, it's really difficult to get started sometimes.
but to summarize...
A tutorial: - is learning-oriented - allows the newcomer to get started - is a lesson
A how-to guide: - is goal-oriented - shows how to solve a specific problem - is a series of steps
"This package is really good. Here is an example. This feature is cool! This other feature solves a common problem. Here is a list of some of the functions."
I haven't done any substantial Java work in probably 8 years and I still miss the JDK docs.
Concourse CI is the one I'm running up against this today - I'm taking over a partially built installation which has all sorts of hard-coded config environment variables that I'm trying to figure out their purpose.
Their documentation has basic getting-started stuff[1], but doesn't list all the config variables and their use. It's open source, so I cloned the repo, but they don't directly reference the environment variables there, either.
Compare that to, say, Ansible's documentation - it exhaustively lists all the options[2] and what they're used for, with links to other related documentation.
[1] https://concourse-ci.org/concourse-web.html [2] https://docs.ansible.com/ansible/latest/reference_appendices...
Our CLI hasn't really aged well and that's something I'd like to address soon - probably going more in the direction of config files instead of flags/env vars. It'd be a lot easier to document with a proper schema. Thanks for the feedback and sorry the experience is still pretty rocky!
Honestly the whole application seems pretty good - but everything is focussed on more hello-world stuff, which is great for getting going on my machine - not so great for running it for real.
Whether you use environment vars or a config file, I don't mind - one benefit environment vars gives is that setting CONCOURSE_BIND_IP to the instance's IP is relatively simple, eg on AWS:
CONCOURSE_BIND_IP=$(curl http://169.254.169.254/latest/meta-data/local-ipv4)
If it's a config file then I need to update that before the application starts, which is more fiddling.
For a long time now we've been primarily focusing on the core design, concepts, and architecture at the expense of documentation and introductory material and outreach - we've never had a dedicated technical writer, so documentation is really best-effort. Now that the dust is settling on our roadmap I feel a lot more confident in increasing our focus on onboarding, operability, and developer/user experience in this coming year. :)
Introspection is not valid in place of documentation.
When it comes to APIs if you're eschewing documentation it's because (I hope) you are trying to minimize support costs and thus devs using the API won't have someone to ask questions of so those edge cases become mysterious and assumptions are made that may be wrong.
If you're designing an API that serves up items that may be on sale how would you name fields so that the price and possibly sale discount are unambiguous to read?
You can probably use another scripting language to use the github interface.
https://github.com/bbs-io/syncterm-windows
Aside, you need to pass your github token in as part of your workflow config.
https://github.com/bbs-io/syncterm-windows/blob/master/.gith...
For my Java project, I set up GitHub Packages as a server in my settings.xml, and then use:
mvn deploy -Dgithub.username=${{github.actor}} -Dgithub.password=${{github.token}} -s settings.xml
as a step in my GitHub Actions workflow.
I'm not super familiar with maven, so I'll talk with the Packages team about how we can improve the documentation here.
Agreed. A lot of times they seemed to use the ole "make a blog post about X and we'll call it documentation" strategy that a lot of others seem to be employing as well. The net result is useless documentation that is often outdated by the time a product is live. However, that doesn't change the fact that it's still the top search engine result for the topic or the de facto page many link to within their own actual 'documentation'.
But I do agree with you that it's very from from being an alternative, and instead should be used to supplement healthy documentation.
And that’s to watch a coworker who hasn’t used it before try to follow your docs. Try again, and then pick another coworker. By then it should be good enough for the majority to follow, and you take the rest of the questions as they come. Put the first draft out and you’ll be swamped with complaints.
Those one line snippets are likely going to be taken down, most systems are too complicated for those to work. It looks nice for a couple package systems but that Maven one is obviously flawed.
I'd be happy to help you dig through whatever you're trying to do; I'm @clarkbw on twitter. I'd also love another set of eyes on the changes to the setup-java action, specifically getting Maven authentication to work well.
https://github.com/actions/setup-java/pull/29
(I'm the PM heading up the Packages project at GitHub)
https://github.community/t5/How-to-use-Git-and-GitHub/Move-N...
I haven't really tried to do anything with Github Actions, I start looking at the docs and think, nah, not ready yet.
It does seem like a powerful tool.
It turns out that good docs is hard and "expensive", can't be handled as an afterthought without actually investing time/resources. A powerful complex tool that took a while to develop is gonna need a similar significant investment in docs.
There's also a lot of things you should be able to do but are not apparent or clear - and yeah the documentation is a disaster
Given that you are using maven, which can be a bit fiddly, might it be the case that some of your issues are with your build? E.g. the way maven deploy works is very dependent on how you set up the pom file.
We self host nearly everything at work except for github. (And it's VERY difficult for me to get anything that costs money approved regardless of price)
For me, this is a big red flag.
The response I get is we don't _NEED_ a CI/CD (and haven't had one for years), so they would rather I find a solution that can run on our hardware for free.
I'll add that I enjoy my work environment and they treat me very well, I'm young and I'm known to "like shiny new things". I also have to work on how I pitch why we need certain things, again I'm pretty young so I'm not well practiced in that just yet.
You are not making this sound like less of a red flag!
But yeah, if you are both new and young there, you can't just push hard for all the things you want. But in this case... you're right, heh.
If I were you, I'd be keeping my eyes out for job opportunities in environments where you can actually learn something about good modern software engineering practices form their example. (Easier said then done, don't I know it).
I was in your boots some time ago, managing CI before I convinced the company that paying for a hosted CI is cheaper than me maintaining our self-hosted whatever CI we were trying at that point :) Jenkins was atrocious and I spent ungodly amount of time dealing with all kinds of issues, while TeamCity was mostly configure-and-forget. Initial setup is pretty straightforward, so is later migration to a real database for example. Updates are seamless and we had some projects on it even after we started using paid CI (Bitrise), because there was no motivation to migrate as everything was running smoothly.
You still own enough of it that if the service is down, you can just execute the scripts manually to build, test and deploy
[0] https://buildkite.com [1] https://github.com/buildkite/elastic-ci-stack-for-aws
Recently at work we've done an analysis of a of CI/CD tools. Many of them are self-hosted, free (and open source) and support a variety of workflows - including pull request related checks (either by polling of via webhooks). A cursory search will yield you a lot to play with, so... go ahead and do that.
That said, if you don't want to think about it too much, you can't go wrong with Jenkins.
Some people will suggest GitLab. I'd steer clear of GitLab, though, because of their operational incompetence [1] and their funny ideas about mandatory corporate espionage [2] - I mean, telemetry - which they only backed away from because people yelled at them. That second one was enough to disqualify them in our analysis (the first one just cemented the idea).
[1] https://about.gitlab.com/blog/2017/02/10/postmortem-of-datab...
[2] https://www.theregister.co.uk/2019/10/30/gitlab_backtracks_o...
As soon as you need multiple pipelines per branch it doesn’t work well.
Most small projects need only one pipeline and are well suited. Other things like terraform needing multiple pipelines for multiple environments are better suited for a CI platform that handles multiple pipelines.
Really. Having one repo checked out into a docker container with a specific node version toolchain and running your test suite with code coverage + publishing html reporting is a matter of a .gitlab_ci.yml config file with 6 to 10 lines .
The other is that there's currently no way to label specific workers. Because of how our network/firewall is segmented we would like to be able to specify a worker for Staging/Dev/Production separately. The closest thing I found was this [0] pull request from October, but it doesn't add the ability to add custom labels.
Is there any timeline for either of these features (especially the labels)?
I don't know why they didn't allow us to use any docker image we want so we don't have to waste time to use actions to install dependencies...Eg, if your app depend on both Go, Ruby, Node at build time you will need:
https://github.com/actions/setup-go https://github.com/actions/setup-node https://github.com/actions/setup-ruby
I much prefer CircleCI way
Can you imagine allowing anyone on the internet to run an arbitrary container on your server for free?
I built an old-style Docker container (i.e., one that runs code like "apt-get install foo") on Github Actions successfully, so I assume Docker works fine. I haven't tried getting root on a build worker, but I imagine they mitigate that in some way. (Perhaps by having a pool of VMs and blowing it up after your build is done.)
I think what the OP is talking about is CI systems whose pipelines are declared by a series of "run this command in this container" instructions. Github Actions doesn't work that way, but you can still run containers.
Why be specific? Why not include a shell-script in your repository, and allow that to run the tests? That way you just need one "Run test-script" action:
https://github.com/bbs-io/syncterm-windows/blob/master/.gith...
Each step of a pipeline is just a docker container of your choice and running whatever commands you want with a persistent workspace volume throughout the pipeline.
There's a few others that work this way like Semaphore, Drone CI, GoCD, CircleCI, Azure Devops but none are as fast, seamless and easy as Buddy.
- name: build
run: |
docker run --rm -v \
"$PWD":/usr/src/my-project \
-w /usr/src/my-project \
-e GOOS -e GOARCH \
golang:1.13 go build -v -o my-project-$GOOS-$GOARCH
env:
GOOS: ${{ matrix.os }}
GOARCH: ${{ matrix.arch }}
(GitHub Actions's build environment already has Docker installed/configured, so there's no setup necessary for "run: docker run" to work.)I did this explicitly because I didn't want to use that "setup-go" action you linked, which at the time (I haven't doubled checked it recently) didn't support Go 1.13. Far better to me to use the Docker image, which has explicit instructions even for cross-platform compilation in its README, than rely on some weird action setup, in my opinion.
- name: Build
run: docker build -t me/image:latest .
env:
DOCKER_BUILDKIT: 1
GOOS: ${{ matrix.os }}
GOARCH: ${{ matrix.arch }}...unless your Dockerfile ends up doing that too? If so I'd be interested to see it if you're willing to share!
Without that, they'll run directly on the virtual machine that you specify.
I don't know what this means. Are there any examples?
Running in a container is the one big thing I'm missing and couldn't find any documentation about.
There are a lot of cool examples demo'ing some nifty stuff, but I'd like to see a lot of very basic examples first. The sort of thing anyone could/would copy into their project and use, then later learn how to make it fancy.
E.g:
YOU WILL NOTICE THIS ONE
in a list of
other normal text
HAVE TO WAIT UNTIL
EVERY ONE LOOKS LIKE THIS
so then this one
WILL STAND OUT AGAIN?
The process was mostly just changing out a Jenkins file to the GitHub yml and sorting a few issues that cropped up.
This happened already with VS Code vs. Github's Atom editor whose development has ceased earlier this year. (Not that I ever was a huge fan of Atom, but its discontinuation is a direct result of the Microsoft acquisition.)
If I had to bet, I think “azure devops” will go away and GitHub will eat it.
Currently the Actions free tier is cheaper than using azure build pipelines which is kind of weird, but nice for GitHub users. I think it’s because GitHub is a limited subset (ie, doesn’t need to support Windows IaaS build types) so it may be cheaper to run.
I bet we'll see a click to deploy button on github for most common web frameworks soon.
Having your git history hooked up to your deployed infrastructure will allow them to do some amazing things around live debugging. (similar to what google is doing with cloud source repositories https://cloud.google.com/debugger/).
Also, GitHub Actions is based on Azure Pipelines already.
It doesn't seem like they've ceased development on Atom - the latest version release was just 3 days ago? https://github.com/atom/atom/releases
---
I've started learning GitHub Actions, at first just for simple build/deploy script, but hope to make more extensive use of the feature.
I did consider how it overlaps with Microsoft's other business areas, and I came to the conclusion that they probably wouldn't be investing resources into this, just to sunset it anytime soon. (I'm sure there are historical examples though, like some Google products..)
In the end, I do somewhat share your hesitation, and believe it's a good idea to always be ready to migrate the essentials. For GitHub Actions, I suppose that means having most of the functionality written in generic scripts.
name: master-pull-request
on:
pull_request:
branches:
- master
jobs:
test:
name: run tests
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v1
- name: Run gradle test
run: |
./gradlew test
I much prefer Drone CI's YAML: kind: pipeline
name: default
steps:
- name: run tests
image: openjdk:8-jdk-slim # Docker images!
commands:
- ./gradlew testIt's just an issue of preference, I suppose, but I prefer GitHub's way of doing things. I find it easier to read and understand.
Most of the actions were pre-existing Makefile targets used during the bootstrap process anyway, so the YAML config was fairly lightweight.
Echoing the other comments in the thread, the docs did feel a bit sparse when I last poked around (this was back when Actions was still in beta).
That being said, there was only Ubuntu images for the Linux builds, and I'm not aware of a way to run custom containers at the moment.
Actions themselves are meant to support the workflow and distill a set of complex steps into something that can be done in a single workflow command. (For example, download and set up a new version of a Java JDK.) They're meant to be used as steps in many peoples workflows, instead of executing a single workflow.
The actions themselves can be built as either a container, which is self-contained, or as JavaScript. I often recommend the latter, since that will work cross platform (containers only work for people who are using workflows running on Linux).
We've been experimenting with this and are using this to add some CI to our react native frontend.
Otherwise it could be a good product, and I'd really like if I could manage the repos and the CI from one unified place