Introducing Pleasantest

At Cloud Four, automated tests save us time by automatically checking for regressions in interactivity, accessibility, and appearance. We’ve used several different tools in the past, each with its own set of trade-offs, to help us ship quality interfaces. We created a new testing tool, Pleasantest, to make UI testing easier, more realistic, and more reliable.

One tool we’ve used to test our UI components is jsdom, the DOM implementation that is included with Jest. This setup was great because it allowed us to use Testing Library, which helped us write tests that were resilient to changes and that tested the accessibility of our components. But while working with this setup, we ran into several issues related to the fact that jsdom doesn’t have a rendering engine so it is missing many browser features. When we write tests that use an emulated DOM without a rendering engine, the tests cannot give us the confidence that a real browser would. Also, polyfilling and stubbing out browser features missing from jsdom is time-consuming and tedious.

Another tool we’ve used is Cypress, which avoids many of jsdom’s problems. Cypress lets you write tests that run in real browsers, which helps improve confidence compared to tests that run in jsdom. Cypress is great at testing entire applications, where you point your Cypress tests to the URL of your app server. They recently added support for testing individual components. However, one of our main gripes with Cypress is that it is a separate test runner from what we use for our unit tests. Each time developers switch between writing a unit test for some logic and writing a UI test, they have to make the mental jump to remember how to use a separate test runner, different assertion syntax, and different conventions. Because of its design, Cypress implements its own functionality (command chains, aliases, custom commands) rather than supporting language features that developers are often familiar with (await, variables, functions). This leads to an increased barrier to entry for people who are already familiar with JavaScript.

We began making Pleasantest as an experiment to see if we could create a testing tool that took what we liked from both kinds of tests. Pleasantest integrates with all of our favorite testing tools from the Testing Library ecosystem. It uses Puppeteer to avoid the problems associated with using an emulated DOM. You can render and test individual components, or point Pleasantest to a URL to load to test entire applications.

By writing tests using Pleasantest, we can maintain the quality of the work we ship, and we can ensure reliability and consistency in functionality and features as time goes on. We’ll walk through an example of how to test a component using Pleasantest.

For this example, we’ll write tests for a React component. We can start by installing Jest and Pleasantest, and the types for Jest for editor autocompletion:

npm i -D jest @types/jest pleasantest
Code language: CSS (css)

The component we’ll test is an example modal from @reach/dialog. You can see the code for the demo on GitHub and you can preview it on Netlify.

We’ll start by creating a new test file, index.test.js, with an empty test:

test('Shows modal when button is pressed', async () => {

})
Code language: JavaScript (javascript)

We can run the test by running npx jest --watch. It will rerun whenever we change the test file, or we can manually rerun it by pressing enter in the terminal.

To mark the test as a Pleasantest test, we’ll wrap the test function in withBrowser:

const { withBrowser } = require('pleasantest')

test(
  'Shows modal when button is pressed',
  withBrowser(async () => {

  })
)
Code language: JavaScript (javascript)

In our example, there is an index.js file in the same folder as the test, which renders the button that opens the modal. We can tell Pleasantest to run that index.js file by using the utils.loadJS function. Since the index.js file renders the app into a <div> with an id of root, we’ll make sure that exists too:

const { withBrowser } = require('pleasantest')

test(
  'Shows modal when button is pressed',
  withBrowser(async ({ utils }) => {
    await utils.injectHTML('<div id="root"></div>')
    await utils.loadJS('./index.js')
  })
)
Code language: JavaScript (javascript)

Next we’ll find the “Open Dialog” button using the getByRole query from Testing Library. In Pleasantest, all queries, matchers, and actions need to be awaited, because the communication with the browser is asynchronous. Note that the screen object needs to be added to the test function parameters. We’ll also use the queryByText query from Testing Library, and the expect(...).not.toBeInTheDocument() matcher from jest-dom to make sure that the modal contents are not present before the button is pressed.

test(
  'Shows modal when button is pressed',
  withBrowser(async ({ utils, screen }) => {
    await utils.injectHTML('<div id="root"></div>')
    await utils.loadJS('./index.js')

    const button = await screen.getByRole('button', { name: /open dialog/i });
    await expect(
      await screen.queryByText(/I am a dialog/i),
    ).not.toBeInTheDocument();
  })
);
Code language: JavaScript (javascript)

Then we can click the button and make sure that the modal appears. Adding the user parameter to our test function gives us access to interaction methods like user.click():

test(
  'Shows modal when button is pressed',
  withBrowser(async ({ utils, screen, user }) => {
    await utils.injectHTML('<div id="root"></div>')
    await utils.loadJS('./index.js')

    const button = await screen.getByRole('button', { name: /open dialog/i });
    await expect(
      await screen.queryByText(/I am a dialog/i)
    ).not.toBeInTheDocument();

    await user.click(button);

    await expect(await screen.queryByText(/I am a dialog/i)).toBeVisible();
  })
);
Code language: JavaScript (javascript)

This test covers the basic functionality of making sure the modal opens correctly. Next, we can add tests for the various ways to close the modal.

There are three ways to close the modal: The close button, clicking on the overlay outside the modal, and pressing the escape key. We’ll start with the escape key since it is the easiest.

Since the logic for rendering the component and opening the modal is the same for all the tests, we can create reusable render and openDialog functions (outside of the test call):

const render = async (utils) => {
  await utils.injectHTML('<div id="root"></div>');
  await utils.loadJS('./index.js');
};

const openDialog = async (screen, user) => {
  const button = await screen.getByRole('button', { name: /open dialog/i });
  await user.click(button);
};
Code language: JavaScript (javascript)

Then we can create a new test and use the functions. We’ll use the page.keyboard.press method from Puppeteer to press the escape key.

test(
  'Escape key closes modal',
  withBrowser(async ({ utils, screen, user, page }) => {
    await render(utils);
    await openDialog(screen, user);

    await expect(await screen.queryByText(/I am a dialog/i)).toBeVisible();

    await page.keyboard.press('Escape');

    await expect(
      await screen.queryByText(/I am a dialog/i)
    ).not.toBeInTheDocument();
  })
);
Code language: JavaScript (javascript)

One thing to keep in mind is that we ran the same queryByText query twice without assigning the result to a variable because we specifically want the test to re-query the DOM after the dialog is closed, rather than reusing the same result.

Testing to make sure the close button works correctly is nearly the same, but we’ll use the user.click method to click the button:

test(
  'Close button closes modal',
  withBrowser(async ({ utils, screen, user }) => {
    await render(utils);
    await openDialog(screen, user);

    await expect(await screen.queryByText(/I am a dialog/i)).toBeVisible();

    const closeButton = await screen.getByRole('button', { name: /close/i });
    await user.click(closeButton);

    await expect(
      await screen.queryByText(/I am a dialog/i)
    ).not.toBeInTheDocument();
  })
);
Code language: JavaScript (javascript)

Lastly, for testing clicking the overlay, we can use Puppeteer’s page.mouse.click to trigger a click at a specific x and y position:

test(
  'Clicking outside modal closes modal',
  withBrowser(async ({ utils, screen, user, page }) => {
    await render(utils);
    await openDialog(screen, user);

    await expect(await screen.queryByText(/I am a dialog/i)).toBeVisible();

    // (10px, 10px) should be outside the modal
    await page.mouse.click(10, 10);

    await expect(
      await screen.queryByText(/I am a dialog/i)
    ).not.toBeInTheDocument();
  })
);
Code language: JavaScript (javascript)

Those tests cover the most obvious behaviors of the modal, but there is still more functionality to test. One important aspect of the implementation is the accessibility of the component. Pleasantest lets us use Testing Library queries and jest-dom matchers that help us make sure the elements have the right accessible roles and labels. We’ll use getByRole and expect(...).toHaveAccessibleName() to check the role and label of the modal:

test(
  'Accessibility structure of modal',
  withBrowser(async ({ utils, screen, user }) => {
    await render(utils);
    await openDialog(screen, user);
    const modal = await screen.getByRole('dialog');
    await expect(modal).toHaveAccessibleName('example dialog');
  })
);
Code language: JavaScript (javascript)

We can also test that the focus is trapped within the modal. This means that when you press tab and shift-tab to navigate through the focusable elements, it should only cycle through elements inside the modal, and skip anything behind the modal. Puppeteer’s page.keyboard.press method, and jest-dom’s expect(...).toHaveFocus() let us test cycling through the focusable button and links inside the dialog.

test(
  'Focus is trapped in the modal when it opens',
  withBrowser(async ({ utils, screen, user, page }) => {
    await render(utils);
    await openDialog(screen, user);

    // When the modal is opened, the close button should be focused automatically
    const closeButton = await screen.getByRole('button', { name: /close/i });
    await expect(closeButton).toHaveFocus();

    await page.keyboard.press('Tab');
    const firstLink = await screen.getByRole('link', { name: /here/i });
    await expect(firstLink).toHaveFocus();

    await page.keyboard.press('Tab');
    const secondLink = await screen.getByRole('link', { name: /focusable/i });
    await expect(secondLink).toHaveFocus();

    // After pressing tab the third time it should cycle back to the close button,
    // instead of focusing on content that is hidden behind the modal
    await page.keyboard.press('Tab');
    await expect(closeButton).toHaveFocus();

    // Shift-tab should cycle back to the last focusable element within the modal
    await page.keyboard.down('Shift');
    await page.keyboard.press('Tab');
    await page.keyboard.up('Shift');

    await expect(secondLink).toHaveFocus();
  })
);
Code language: JavaScript (javascript)

This test suite now tests the user-facing functionality of the component and will catch regressions before users do. You can see the full code for this example and download Pleasantest on GitHub.

We’re excited to use Pleasantest on our projects. Going forward, we’ll continue to add features to Pleasantest that will make testing easier and more comprehensive. We encourage you to give it a try and let us know how it works!