Hacker News Reader

Comment by duped | Hacker News Reader

I just want to be able to write all my workflow code as typescript (including the config - no YAML, for the love of god, no more YAML!) and run it locally with a debugger attached.

It's cost me hundreds to thousands of dollars to implement nontrivial workflows because of how the YAML is parsed (for example, empty strings when using a secret that has been renamed or removed) and the lack of introspection or debuggability when something goes wrong.

It's gotten to the point where new any new workflows I write are thin wrappers around a single script and I don't import any actions besides actions/checkout (even that has been bug prone, historically).

All that said, it's not like other platforms are better. But they certainly are cheaper and don't have dumb breakages when you need cross platform builds (has upload-artifact been fixed for executables on MacOS yet?)

View on HN

This is exactly what I'm working on at https://www.inngest.com. You can write statically typed workflows, run any code as an action (if it runs in a container, or use lambda otherwise), have complex conditionals, and coordinate between events (eg. pause until an event comes in that matches a condition, with TTL paths). There's a debugger to step-over workflows, and see the input/output as it happens. Coming soon, the ability to run actions locally during debug mode.

Workflows are statically typed in that they have event triggers - ideally with an event schema (which we generate for you) - and each "action" has static typing of inputs/outputs (plus additional workflow configuration for reusability). They're defined as code, and can be viewed or edited visually.

It also bundles an event hub, so you can automatically run workflows when events happen in real time. For example, if you want to run a churn flow on signup, create a workflow with a `signup.new` event trigger. The workflows can also coordinate between events, too, so in the churn workflow you can wait for an "interactivity" event from the user for up to 1 day, then time out and run some other flow/logic.

It's workflows, generalised. As if you put Github Workflows, Lambda, Segment, and Zapier in a blender.

If you want early access, you can always reach me at tony [at] inngest.com. I'm rolling out invites every week.

Do you have clients using this for CI/CD? It looks very interesting, if a bit more general-purpose.

CD (including ourselves), operational pipelines, sales pipelines, internal events/logic, scheduled flows, and machine learning are a few of the areas I'm seeing.

CI isn't _necessarily_ our current target. As of yet, there's no concept of "failed runs" in the classic CI sense. We highlight failed workflows but will retry actions by default. If things fail, we allow you to retry/edit the data, debug in line, etc.

It's definitely possible to use the failed workflow runs as a CI pipeline, but the UX would need an overhaul. Right now, we do user-event attribution to show you which workflows users are running in your system, their version, and the steps - which is needed for actual, live system workflows vs CI workflows.

You should post a Show HN when it's open for people try out. See https://news.ycombinator.com/showhn.html and https://news.ycombinator.com/item?id=22336638. Email me at hn@ycombinator.com with a draft text and I'll look it over for you beforehand. Same offer goes for anyone, just please don't expect a quick reply- that depends on the wild swings of the HN inbox. You can also see how we advise YC startups to do this at https://news.ycombinator.com/yli.html - the logistical parts are YC-only but the communication parts apply to everybody.

Ah, thank you Dang, I'll reach out when it's ready!! Very much appreciate you letting me know. It'll likely be several weeks before then - I'm super product focused and want to make sure the UX is down before a general release.

Having dealt with a fair bit of CI on a decent number of services (AppVeyor, Travis, CircleCI and GitHub Actions), I’ve come to the exact same conclusion.

CI is a script, and the YAML configs for those various services configure the machine type, OS and toolchain. Everything else is contained within the script. Sometimes even toolchain setup is handled by the script.

Not following this model has wasted so much time when migrating services or trying to tweak what CI does.

With a script you can run it locally to ensure it performs the steps desired, leaving the CI “setup” to minimal environment/toolchain debugging.

Yea, making it so statements are composing via YAML was a very poor choice. One case I had was needing to change my actions so that steps D, E, and F could be retried together. What is the github YAML specification for decrementing a 'tries_remaining' variable, then going back to a previous step? I couldn't find it, so ended up just having to rewrite the action so that most of the steps are now in one bash script.

Not being able to execute it locally has also wasted a lot of time, and people needing to make 50+ changes to the master branch until they get it right.

This might be an unpopular opinion but bundling CI/CD logic with the application code has been a huge blunder IME. If you're adding CI/CD for the first time to a brownfield project you're looking at dozens of minor YAML tweaks to master directly (or a bunch of rubber-stamped PRs).

Why can't I have a separate interface where I just say "build this Github project, and put the content on this on-prem server/kube cluster/VM/whatever."

If I'm doing something complex with GitHub Actions I like to work in a branch, iterate with a bunch of messy tweak commits and then squash-merge into main once I've got it working exactly right.

Another trick that works well is putting GitHub Actions in an entirely separate repository. There's nothing to stop actions in one repo from checking out code from another - I use that trick quite frequently.

You do have to jump through a few extra hoops to set it up so that code in your actions repo starts running automatically on commits to your main repo, but you can do that with a small action in the main repo that triggers a build in the actions repo.

Interesting approach. Do you mind sharing link with repos?

I don't have any at the moment where a commit to one triggers a build in another, but I have a bunch that run on a schedule, pull code from other repos and do stuff with it.

Best example of that is here: https://github.com/simonw/covid-19-datasette/blob/main/.gith...

These are both great tips, thank you!

Who CICD's the CICD? It's turtles all the way up.

Testing a pipeline that depends on a merge to a branch, or a specific tag, is troublesome. Easier to just iterate in the mainline until you're ready.

When you edit an action on GitHub, the web editor has support for workflow syntax and will determine if your action is viable before committing. This is even easier if you use the new codespace shortcut (press .) on GitHub.

You might also want to take a look at act which allows you to run github actions locally, this is typically how I do actions development:

https://github.com/nektos/act

I'm curious what prevents you from writing your own actions in typescript now?

> the web editor has support for workflow syntax and will determine if your action is viable before committing.

I don't use the web editor, but more importantly it can't catch logical errors (missing required with: arguments, secrets that don't exist, environment variable names, etc).

> I'm curious what prevents you from writing your own actions in typescript now?

When I say "I want to write actions in typescript" I mean that I want to specify the entirety of my CI using a typescript program, without any YAML configuration. In particular, the jobs of a workflow themselves.

I have many jobs shared between build/test/release with slightly different triggers and configurations, but the only way to handle this in actions (especially when using imported actions) is by copying/pasting YAML. That wound up being untenable, and it's why I stripped out all action dependencies and wrote the automation to not use them such that all workflows

I've also had use cases for recursive workflows.

Act doesn't cover any of my use cases.

Have you considered writing a code generator that lets you define actions in TypeScript and have it generate the YAML for you?

This is absurd

We built this for my company. It's called YATM. Yet Another Thing to Maintain. ;)

I've seen both sides of this. Heavily scripted Jenkins groovy files and weird YAML Gitlab pipe lines who admittedly are hard to deal with and understand especially for someone new.

IME I prefer the YAML files because it forces the "DevOps" guy to put most of the stuff in outside scripts, which can than be tested and run on a local dev machine.

Nothing worse than having to debug a freaking CI runner. I never got why a job should be more complicated than "fetch -> build -> test -> deploy" with maybe a bit of intelligence to handle build triggers and artifact management.

I don't want to deal with a web of groovy scripts with sed hacks everywhere and which involves 10 other jobs who does the same kind of unholy blasphemous hacks and who always break one another.

I totally agree, and I’ve seen some completely nuts GH actions setups already where people seem to want to use every single possible action. What happens when the charging model no longer suits you? I’ve done self hosted Jenkins -> TravisCI -> CircleCI -> Self hosted GitLab on one of my projects in the past few years… I want to do as little CI provider specific stuff as possible.

For CI specifically, you should IMO check out https://dagger.io. It abstracts and generalizes CI in exactly the way you need: repeatable, locally runnable, and also cacheable on remote servers. Write once, run anywhere CI - what you have locally will be _exactly_ what runs remotely.

Do you have a picture of what typescript code for workflow would look like? Are you thinking of something that calls APIs directly or runs and spits out some kind of JSON instructions in place of yaml?

It would call APIs directly. For example, I have a lot of release automation that integrates with AWS. Rather than requiring AWS specific actions, I could import the AWS sdk via node and make the calls myself.

A good exercise is to look at a simple but nontrivial build/test/release automation:

- on pull requests build & test, on tags build/test and release, and once a day release a nightly build.

- cache dependencies globally and build artifacts on each branch for incremental builds

- once daily, clear out artifacts on deleted branches or merged PRs

A script to do this would be around 100 lines and be readable/maintainable (if build/test or release change, it is shared by the different cases!). The script can have a main entrypoint that will dispatch to the runners as needed.

I know it sounds a lot like jenkins, but node has a much better ecosystem than groovy.

Did anybody experiment with cuelang as source for GitHub actions yaml?

You could use TypeScript IaC like AWS CDK or Pulumi with FaaS like AWS Lambda or Cloudflare Workers.

End to end TypeScript from IaC to FaaS.

There is a serious market niche for that. There is SharePoint/Flow for Office, but it is overall dreadful software.

I think platforms like nextcloud try to build some workflow engines, but I don't know how far that is and in which format they are implemented. I don't like Zapier, but many people seem to like it.

Yep. I am currently needing to try and set up code coverage reporting in GitHub (can't use Codecov as the company does not want to pay for anything) and it would be so much more trivial if we could just write regular code rather than futzing around with Bash and yaml.

You can write custom actions in JavaScript, using the toolkit: https://docs.github.com/en/actions/creating-actions/creating...

However, the workflow referencing the custom action would still be the Actions YAML syntax.

(disclaimer: I work at GH, but not on Actions)

Disclaimer? This is the good part of HN, being told of fun bits like this.

Thank you.

It's still polite to disclose as a habit.

Hi Matt! Sabiha from Codecov here - curious how many seats you're looking to add as Codecov is free for up to 5 users. From there we are currently priced at $10/seat. We've ensured that our pricing is viable for all kinds of businesses and that it outweighs the costs of building this in house :). If it makes sense, feel free to reach out to sabiha@codecov.io if I can help.

Use Teamcity. There you can use Kotlin.

Jenkins have Jenkinsfile in Groovy, but it's still kinda hard to debug it. Never tried

The Visual Studio team totally needs to steal your ideas and deliver implementations for sharing your workflow.

A graphical development environment and TS library set for something like UML wanted to be for OOP only for everything we need to do and want to do with our codes in any development environment from the enterprise VS group wallet grab setup we still run, to a "redmode" for working from Emacs.

It's not all as bad as XKCD says about creating new standards : the world of code writers is big enough now for many equivalent standards to thrive in coexistence. We're probably still too emotionally scarred from the very recent years when anything that smacks of making standards and arguing betweent them had much more serious consequences than possible today.

Reply on news.ycombinator.com