Implement a GitHub Actions Testing Suite

This tutorial will show you how to create a test suite for a codebase hosted in a GitHub repository. Using GitHub actions, you'll be able to -

• Run the tests automatically when pull requests (PRs) are opened
• Ensure that the tests still pass after merging
• Put checks in place to ensure successful deployments

I've created an example repository on GitHub that you can clone and reference while going through the tutorial, linked here.

GitHub Actions are a way to execute software workflows which is deeply integrated with GitHub repositories. Using YAML, you can specify workflows, each of which can have multiple steps, that will run based on specific triggers, such as on pull requests, on pushes, and more.

There is also a marketplace for GitHub Actions where you can find and use Actions created by other users within your own workflows; many of these Actions are free to use.

For our example project, let's create a simple web service that we can make API calls to using TypeScript and Fastify.

When our service receives a POST request, it will respond with what day of the week that date is on. For testing, we'll use Vitest; however, Jest and Mocha are popular alternatives you can consider. If you wanted to test a user interface (UI), Playwright and Cypress are options you could use for end-to-end and UI tests.

Railway has a template for deploying a Fastify application, so, to get started, all you have to do is:

1. Go to the template's GitHub repository and click "Deploy on Railway"
2. Login or sign up using your GitHub account or your email
3. Eject from the template's GitHub repository so that you have your own copy to build off of and edit

Ejecting from template in service settings

Now, you have your own Fastify application that deploys from GitHub to Railway automatically! You can clone the new repository and start working on it from our machine.

Let's create an API route for getting the weekday of a date. In the src/routes/root.ts file, you can add a Fastify route with the following code:

// src/routes/root.ts
import { FastifyInstance, FastifyPluginAsync } from "fastify";

const root: FastifyPluginAsync = async (fastify: FastifyInstance) => {
  // other template code...

  // new code
  fastify.get("/weekday", async (request, reply) => {
    const dateQuery: string | undefined = (request.query as any).date;
    if (!dateQuery) {
      return reply
        .code(400)
        .send({ error: "Date query parameter is required" });
    }
    const date = new Date(dateQuery);
    if (date.toString() === "Invalid Date") {
      return reply.code(400).send({ error: "Invalid date format" });
    }
    console.log(date);
    const weekdayIndex = date.getDay();
    const weekdays = [
      "Sunday",
      "Monday",
      "Tuesday",
      "Wednesday",
      "Thursday",
      "Friday",
      "Saturday",
    ];
    const weekday = weekdays[weekdayIndex];
    return reply.code(200).send({ weekday });
  });
};

export default root;

When you run our application locally with npm run dev, you'll be able to send requests to the API with this format:

curl "localhost:3000/weekday?date=$DATE"

The route will expect the date string in a query parameter. If you omit the query parameter or send an unparseable string, you'll get a 400 error code back. Otherwise, the JSON response body will contain the weekday of the date.

To set up testing, you can follow the steps from Vitest's getting started guide (https://vitest.dev/guide/) to initialize and configure your tests.

First, add Vitest as a depedency by running:

npm install -D vitest

Next, add an entry to the scripts section of your package.json file which will use Vitest when your run "test":

{
  "scripts": {
    "test": "vitest"
  }
}

Next, create a file for your tests called integration.test.ts. You can add a few tests for our API inside:

// integration.test.ts
import { expect, test } from "vitest";

test("returns correct weekday", async () => {
  const res = await fetch("http://localhost:3000/weekday?date=03/01/2023");
  const data = await res.json();
  expect(data.weekday).toBe("Wednesday");
});

test("rejects missing query parameter", async () => {
  const res = await fetch("http://localhost:3000/weekday");
  expect(res.status).toBe(400);
});

test("rejects unparseable date string", async () => {
  const res = await fetch("http://localhost:3000/weekday?date=foobar");
  expect(res.status).toBe(400);
});

Now, if you have the Fastify application running locally, you can run the tests with:

npm run test

Now that you've set up testing and written our first tests, we can create the workflows for GitHub Actions to run. GitHub will look at the root of your repository for a .github/workflows/ folder. In that folder, you can create YAML files outlining the steps to your workflows. Each file can specify different triggers which would cause it to run.

First, create a YAML file at .github/workflows/tests.yaml.

Then, add the following to the top of the file:

name: tests

on:
  push:
    branches:
      - main
  pull_request:

This defines a workflow that will run whenever a pull request is opened and whenever a commit is pushed to main.

Next, you can specify the jobs that the workflow will run. You'll have one for running the tests with Vitest; it first checks out the repository, then sets up node so that you have access to npm, and then it installs, builds, and runs the app before running the tests:

name: tests

on:
  push:
    branches:
      - main
  pull_request:

jobs:
  vitest:
    name: API Tests
    runs-on: ubuntu-latest
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: "22"
    - name: Install deps
      run: npm install
    - name: Build app
      run: npm run build
    - name: Run app
      run: RUNNER_TRACKING_ID="" && (nohup npm run start &) # this starts the app in the background
    - name: Run tests
      run: npm run test

If we want to make sure that the tests pass before being able to merge pull requests, you can create a branch ruleset on the repository:

Creating a new ruleset in GitHub

You turn on enforcement and specify who can bypass the rules:

Configuring the branch ruleset in GitHub

And you can also specify which branches you want this rule to apply to:

Adding a branch target in GitHub

There are a number of rules that you can read about below. The one we're interested in is "Require status checks to pass":

Setting require status checks to pass

To add the tests as a requirement before merging, just type in their name (API Tests):

Adding the API Tests as required checks

Now, pull requests cannot be merged unless the tests are passing (unless you are an admin or on the bypass list, in which case you can bypass the checks and merge PRs anyways).

Now that we're finished with the workflow, let's try making a change to the app. Create a new branch by running:

git checkout -b $BRANCH_NAME

and:

git push -u origin $BRANCH_NAME

In src/routes/root.ts, let's imagine that while we're making changes, we make a typo and accidentally change the route path to "wekday":

// src/routes/root.ts
// ...
const root: FastifyPluginAsync = async (fastify: FastifyInstance) => {
  // ...
  fastify.get("/wekday", async (request, reply) => {
    // ...
  });
};

export default root;

Let's open a pull request from the new branch to main. At the bottom of the pull request's page, you'll see that merging is blocked due to the tests failing:

Demonstrating tests failing in GitHub

GitHub will even send us an email to let us know that our workflow run has failed:

Email notification about failed test from GitHub

It's great that the tests are catching our mistakes on pull requests, but it's also wise to make sure bad merges don't get deployed. Since the tests run when pull requests are also merged to main, you can configure Railway to wait until tests are passing before auto-deploying. In the Railway project that's been deployed, you can go to the settings for your "server" service and enable that setting:

Configuring Railway to wait for ci

We now have a test suite set up via GitHub Actions that will let us know if tests are failing before we merge pull requests or deploy our code!

This example project is fairly simple, but GitHub Actions has many advanced features that come in handy as your project grows. For example:

• You can speed up your continuous integration pipeline by running tests concurrently
• You can use a matrix of different configurations to ensure your tests pass under various different environments
• You can cache artifacts from your workflow runs to speed up future runs