Acceptance Testing with CodeceptJS

Over the past few months I've been getting up to speed with automated testing and I'm still seeing only the tip of the iceberg. For one, there are different types of automated tests, each with many synonyms. Here's a quick summary:

Unit Tests - tests the most basic components of your app/site. These are the simplest and fastest tests to create and run.

Integration Tests - tests the connections between the basic components and possibly the backend.

UI Tests - tests the application as a user would, ensuring the app behaves as expected on the front-end. Also known as functional tests, acceptance tests, or end-to-end tests.

Performance Tests - tests the performance of the application/site.

The Issue

A few days ago I was helping one of my Thinkful mentees search for jobs and came across a posting at Greenhouse.io. As I usually do on new sites, I clicked around the page to see how it worked and explored the UI/X. One interesting feature was the option to upload a CV/resume via attachment, Google Drive, Dropbox or good 'ole paste. Here's the posting I'm referring to.

I clicked the attachment option, which opened my file browser so I could select a file. I closed it, the 4 upload options returned. Then clicked the remaining options to see how they worked. Dropbox opened a modal to allow me to login to Dropbox to select a file. Closing it revealed the 4 upload options again. Google Drive did the same, except when I exited the modal the upload options were gone. That is, I exited the modal without selecting a document and now the 4 upload options were nowhere to be seen.

The only way to make the 4 options reappear was to refresh the page, causing the entered form data to be erased.

A perfect chance to get some experience with a new testing framework I figured...

CodeceptJS

It seems each day brings a new testing framework. A quick search will return dozens, possibly hundreds. Some are made for specific frameworks (like Angular or React), others more general. I've been diving into [InternJS])(https://theintern.github.io/) for my primary work at Citrusbyte. While not super experienced at this point, Intern seems very robust and works for a wide range of tests, and could be used across several ongoing projects.

For this limited test on Greenhouse.io I decided to try a new framework - CodeceptJS. The main selling point for me was it was primary geared toward acceptance testing, which is exactly what I needed in this case.

Also, since I'm not a developer for Greenhouse.io I can't easily do unit/integration tests. Acceptance tests were the only option, and addressed the issue I came across.

Getting Started

Having no experience with CodeceptJS I started with their Quickstart guide.

Install CodeceptJS

npm install -g codeceptjs

Initialize Codecept

codeceptjs init

This started a terminal prompt with some questions about the project. For the most part I accepted the defaults.

Install Webdriver and Selenium

npm install -g webdriverio
npm install -g selenium-webdriver

With everything setup you can generate a test file via the command line with:

codeceptjs gt

then follow the prompts and name the test file.

Tests are pretty straightforward. Here's what a simple test looks like:

Feature('My First Test');

Scenario('test something', (I) => {
  I.amOnPage('https://www.google.com');
  I.seeElement('#hplogo');
});

This simple test would go to the Google homepage and look for an element with an id of hplogo. Strangely, this is the id on Google's logo on their home page. I assume hp stands for 'homepage'.

Greenhouse Upload Tests

For the Greenhouse job posting page tests I first took a look at the expected behavior. It looked like the 4 upload buttons were supposed to remain visible while performing the upload action. Here's my first test:

Scenario('Test the "Attach" option', (I) => {
  I.amOnPage('/improbable/jobs/69982#.V3xgbpMrLUI');
  I.click('div[data-field=resume] a[data-source=attach]');
  I.seeElement('div[data-field=resume] a[data-source=attach]');
});

You'll notice the URL is just a relative path, that's because the base URL is stored in a codecept.json config file. First I navigate to the job posting page, then click the 'Attach' option, then test if the 'Attach' option is still visible.

The tests for Dropbox and Paste were similar, with Dropbox performing slightly differently:

Scenario('Test the "Dropbox" option', (I) => {
  I.amOnPage('/improbable/jobs/69982#.V3xgbpMrLUI');
  I.click('div[data-field=resume] a[data-source=dropbox]');
  I.see('Loading attachment from Dropbox...', 'div[data-messages=uploaders]');
});

Scenario('Test the "Paste" option', (I) => {
  I.amOnPage('/improbable/jobs/69982#.V3xgbpMrLUI');
  I.click('div[data-field=resume] a[data-source=paste]');
  I.seeElement('div[data-field=resume] a[data-source=paste]');
});

For Google I did a test similar to the Attach and Paste options:

Scenario('Test the "Google Drive" option', (I) => {
  I.amOnPage('/improbable/jobs/69982#.V3xgbpMrLUI');
  I.click('div[data-field=resume] a[data-source=google-drive]');
  I.seeElement('div[data-field=resume] a[data-source=google-drive]');
});

To run the tests:

codeceptjs run --steps

The Output

Once the tests are run here's the output showing a failure on the Google Drive test.

Test output

Here's the full repo if you want to run it yourself.

Summary

CodeceptJS seems like a promising acceptance testing tool. It's relatively new and appears to be in active development. I'll definitely keep it in mind for future acceptance testing.

Comments