Running Cypress E2E-tests in GitHub Actions

Recently I have been working on a React app and have started adding end-to-end (E2E) tests using Cypress.io.

Cypress is super powerful and really nice for writing automated tests. I was able to get it up and running pretty quickly and write several different test specs for various user scenarios.

Once I was happy with the tests locally, I wanted to integrate them into our CI/CD pipeline so that every time someone on our team created a PR, pushed a commit to a PR, or merged code into the main branch, all of the tests would run automatically. This would help us identify any side effects that a feature change might have caused throughout the application.

For my CI/CD pipeline, I am using GitHub actions. However, when I updated my action to run Cypress, I ran into a few errors. If you are reading this blog you may be facing the same issues. If so, let’s debug together…

TL;DR: If you are under time pressure, my working config is at the end of the blog with an explanation on how it works. I hope it helps!

My starting config (Which does not work)

name: Front-end CI
on:
    push:
        branches: [main]
    pull_request:
        branches: [main]

jobs:
    cypress-run:
        steps:
            - name: Checkout
              uses: actions/checkout@v2
            - run: npm ci
            - name: Cypress run
              uses: cypress-io/github-action@v2
              with:
                  build: npm run build
                  start: npm start

You can see my first attempt in the code snippet above. I have a Cypress job set up to run on pushes to the main branch and also on pull requests. Within the job, I am starting with the GitHub Checkout action to checkout the code from our GitHub repository. Then, I use the Cypress-io GitHub action to start the project.

Cypress GitHub Actions Version

The first issue I came across was with the versions I was using for the Checkout and Cypress actions.

Below is the exact error message:

Error: You are attempting to use Cypress with an older config file: cypress.json. When you upgraded to Cypress v10.0 the config file was updated and moved to a new location: cypress.config.ts. You may need to update any CLI scripts to ensure that they are referring the new version. This would typically look something like: "cypress open — config-file=cypress.config.ts”

Resolution — Updating both action versions resolved these error for me.

I would recommend, if you are using these two actions, to look for the most up-to-date versions:

Latest Checkout Action version can be found here.

Latest Cypress-io Action version can be found here.

More Version Errors

With the actions versions updated, it did resolve the above error messages, but now I was getting a new error:

Error:0308010C:digital envelope routines::unsupported

This unfortunately was also a version issue for me, this time I needed to update my version of react-scripts.

With react scripts updated, it fixed the above error message and I could see the server and tests starting on GitHub.

However, all tests were failing:

Cypress tests failing to load

Cypress was giving me the following error:

Your page did not fire its load event within 60000ms.You can try increasing the pageLoadTimeout value in home/runner/work/portal/portal/cypress.config.ts to wait longer.Browsers will not fire the load event until all stylesheets and scripts are done downloading.

My initial thought was that the tests could be starting before my server was built and running, and so I added this following wait-on clause to my action deceleration:

jobs:
    cypress-test:
        needs: test
        runs-on: ubuntu-latest
        steps:
            - name: Check out repository code
              uses: actions/checkout@v3
            - run: npm ci

            - name: run cypress tests in chrome
              uses: cypress-io/github-action@v5
              timeout-minutes: 10
              with:
                  browser: chrome
                  record: true
                  build: npm run build
                  start: npm start
                  wait-on: 'http://localhost:3000'

This tells the test to wait until http://localhost:3000 can respond.

With this in place, it was time to run the pipeline again!

Unfortunately it was still giving me this same error:

What stood out to me particularly about why the tests were failing was that they were timing out waiting for the first url to load. This was weird, as I could see the server was starting, but none of the tests were starting.

This had me confused because the tests were all working locally for me. I needed a way to see the test running in my GitHub box.

Enter Cypress Cloud

Cypress cloud is a web application that allows you to view recordings of your Cypress tests. It also allows you to view screenshots of test failures and offers a number of different analytical insights.

I set up a free account and it gave me a record key for recording tests. To verify that the record key definitely worked, I tested it out locally first by running the following command:

npx cypress run --record --key <<your_record_key>>

After running the command I was able to see my local tests passing within the Cypress Cloud application and could view the recording of them:

Since the free tier has limited amounts of monthly recordings, I removed the key from my local run as we can visually view the test locally, so there is no need to record them. However, in cases where they are failing on GitHub and working fine locally, I needed to record what was happening there as I don’t have the same visibility.

To allow Cypress to record the tests on GitHub, all I had to do was add the record ID to my action secrets and updated my actions environment to reference it:

I could see from the actions terminal that my tests were being recorded. Now time to check out what was happening in Cypress Cloud:

It looked like nothing was loading on the app, and so the tests were timing out — This is when it hit me, I’ve only ever seen this happen locally when I haven’t got my env vars present.

To test my theory, I removed my env file and ran the project/tests locally, and I could see the exact same issue. Now I knew my environment variables were not getting passed into the Cypress build.

Adding the variables to the Cypress job was actually easier than expected. You can pass in and reference the secret values from GitHub action like this:

With the env vars added, my pipeline was finally passing and so was my Cypress tests.

Here is the finished working configuration for my Cypress tests:

name: Front-end CI

on:
    push:
        branches: [main]
    pull_request:
        branches: [main]

jobs:
    cypress-test:
        needs: test
        runs-on: ubuntu-latest
        steps:
            - name: Check out repository code
              uses: actions/checkout@v3
            - run: npm i

            - name: run cypress tests in chrome
              uses: cypress-io/github-action@v5
              timeout-minutes: 10
              with:
                  browser: chrome
                  record: true
                  build: npm run build
                  start: npm start
                  wait-on: 'http://localhost:3000'
              env:
                  CYPRESS_RECORD_KEY: ${{ secrets.CYPRESS_RECORD_KEY }}
                  <Add in all your project specific env vars here>

If this blog helped and you would like to see more blogs on GitHub Actions or automated tested be sure to clap or comment!