Testing Github actions locally

Within Red Hat, we previously had been working on a complete refactor of a custom GitHub action called build-chain. During the course of this project, however, we realized that the only way to perform any kind of end-to-end testing for this custom action was by running it manually on GitHub. We would have had to make multiple test repositories and fake pull requests (PRs), and then manually verify the output upon running the action. Too much manual work!

We noticed that this issue wasn’t just limited to custom actions but also affected any kind of workflow. We ran into this annoying problem of having to test it by pushing it to GitHub, triggering the workflow, manually checking the output, making fixes and repeating this process until we got it right. At the end of this vicious cycle, we ended up with a polluted git history consisting of multiple useless commits and workflow runs.

Wouldn’t nektos/act cover this case?

The command line interface (CLI) tool nektos/act is used to run GitHub actions locally. So it might seem that this tool resolves our issues, but it lacks certain important features:

• There is no API for it (i.e. it is difficult to interact with this tool programmatically)
• There is no option to mock APIs during a workflow run
• There is no option to mock an entire step during a workflow run
• It runs the workflow using the actual repository, risking making actual changes to it (for example, running a workflow which pushes to a branch)

This led to the creation of mock-github and act-js—a one-stop solution for creating a local git environment to run our GitHub actions locally and programmatically.

Mock-github

Mock-github is a Node.JS library that allows you to configure and make completely local and functioning git repositories. In these repositories, you can add, commit and push files, create branches, merge branches, etc. Moreover, it provides an Octokit-like interface called Moctokit for mocking your GitHub API calls. For the sake of consistency, it also provides an interface called Mockapi to mock any APIs using the same interface that Moctokit uses (a use case for this will become more apparent later in this post).

Act-js

Act-js is a Node.JS wrapper for the nektos/act CLI tool, and provides a way to programmatically run your GitHub actions locally and verify their output. Additionally, you can mock any API call that’s made during the workflow run using Moctokit and Mockapi. As an alternative, you can also mock the entire step itself.

An example

Let’s see an example of how we can use these two libraries to test a workflow file programmatically.

Consider the following simple workflow file:

name: Test
on: push
jobs:
 test:
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v3
     - name: github api call
       run: |
         result=$(curl -s http://api.github.com/repos/owner/project)
         echo "$result"
     - name: custom api call
       run: |
         result=$(curl -s http://google.com)
         echo "$result"

It checks out the repository and makes two API calls—one to GitHub and one to Google.

Thanks to mock-github and Act-js, it’s now possible to test this workflow using Jest in a TypeScript environment. The first thing to do is create a local git repository in which we can run our GitHub action. Think of this as creating a clean standard environment to test your code in: 

 const github = new MockGithub({
   repo: {
     foo: {
       files: [
         {
           src: path.resolve(__dirname, "..", ".github"),
           dest: ".github",
         }
       ],
     },
   },
 });
 await github.setup();

This creates a local repository called “foo”. Our workflow files are placed into the .github directory of the foo repository. In this case, only the workflow file is needed in our local repository, but you can copy over whichever file/directory you want to the local repository.

Now, we should also prepare to mock the API calls during our test workflow run. In this case, it might seem a bit silly to do so, but imagine a more complex workflow where it depends on the data these APIs sent back. If we do not mock these APIs, then the results of running the same test might differ from one run to another (for example,  if the repository is deleted then the GitHub API will fail). Moreover, what if we have a strict API rate limit? We don’t want to waste them on testing. So, we will use Moctokit and Mockapi.

const moctokit = new Moctokit("http://api.github.com");
const mockapi = new Mockapi({
   google: {
     baseUrl: "http://google.com",
     endpoints: {
       root: {
         get: {
           path: "/",
           method: "get",
           parameters: {
             query: [],
             path: [],
             body: [],
           },
         },
       },
     },
   },
 });

Here we have initialized Moctokit to use “http://api.github.com” as our base URL, and we have initialized Mockapi with the schema of our custom API. We are now ready to execute our workflow locally using Act from the act-js library:

const act = new Act(github.repo.getPath("foo"));
const result = await act.runEvent("push", {
   mockApi: [
     moctokit.rest.repos
       .get({ owner: "owner", repo: "project" })
       .setResponse({ status: 200, data: { full_name: "owner/project" } }),
     mockapi.mock.google.root
       .get()
       .setResponse({ status: 200, data: "mock response" }),
   ],
 });

First, we initialize Act to run only in the foo local repository. Next, we trigger the workflow using the push event and define two mocks for this workflow execution. The first one is used to mock the GitHub API. Whenever an API call is made to GitHub to get the details for the repository owner/project in the workflow, it will receive our mocked response back rather than actual data. The next mock is for the Google API, which also behaves the same way.

Finally, let's compare the result of the run with the expected output:

expect(result).toStrictEqual([
   {
     name: "Main actions/checkout@v3",
     status: 0,
     output: "",
   },
   {
     name: "Main github api call",
     status: 0,
     output: '{"full_name":"owner/project"}',
   },
   {
     name: "Main custom api call",
     status: 0,
     output: "mock response",
   },
]);

Our expected output consists of three steps (as seen in the workflow file). For the checkout step, we don’t expect any output. It should just successfully execute as indicated by status 0. The next two steps should print the data from our mocked response without failing. Notice that “Main” is added before each step. This indicates that these steps are not part of the pre or post section of the workflow.

Now, put all of this together in ci.test.ts file:

import { Act } from "@kie/act-js";
import { Mockapi, MockGithub, Moctokit } from "@kie/mock-github";
import path from "path";
let github: MockGithub;
beforeEach(async () => {
 github = new MockGithub({
   repo: {
     foo: {
       files: [
         {
           src: path.resolve(__dirname, "..", ".github"),
           dest: ".github",
         },
       ],
     },
   },
 });
 await github.setup();
});
afterEach(async () => {
 await github.teardown();
});
test("api workflow", async () => {
 const moctokit = new Moctokit("http://api.github.com");
 const mockapi = new Mockapi({
   google: {
     baseUrl: "http://google.com",
     endpoints: {
       root: {
         get: {
           path: "/",
           method: "get",
           parameters: {
             query: [],
             path: [],
             body: [],
           },
         },
       },
     },
   },
 });
 const act = new Act(github.repo.getPath("foo"));
 const result = await act.runEvent("push", {
   mockApi: [
     moctokit.rest.repos
       .get({ owner: "owner", repo: "project" })
       .setResponse({ status: 200, data: { full_name: "owner/project" } }),
     mockapi.mock.google.root
       .get()
       .setResponse({ status: 200, data: "mock response" }),
   ],
 });
 expect(result).toStrictEqual([
   {
     name: "Main actions/checkout@v3",
     status: 0,
     output: "",
   },
   {
     name: "Main github api call",
     status: 0,
     output: '{"full_name":"owner/project"}',
   },
   {
     name: "Main custom api call",
     status: 0,
     output: "mock response",
   },
 ]);
});

Then run it:

$ npm test
> non-custom-actions@1.0.0 test
> jest
PASS  test/ci.test.ts
 ✓ api workflow (2365 ms)
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        3.24 s, estimated 5 s
Ran all test suites.

Success! We can now more easily test any changes to our workflow file locally.

Conclusion

Mock-github and act-js enable us to take a test-driven approach to writing github actions while making sure that our git history remains clean. 

Make sure to check out the repositories for these libraries along with some more complex examples:

• mock-github
• act-js
• Examples