Test helpers: The next generation

In a previous blog post I talked about the new testing API that has just been published a few days ago and will change how we write tests for our Ember apps, but I ended it with a negative note on the creation of custom test helpers: They won’t work anymore unless you make some changes.

In this post I’ll dive on the reason why they don’t work, how to refactor them, and specially for addon authors, show how to refactor them in a way that keeps backwards compatibility.

Why they don’t work anymore?

First let’s clarify a couple points.

When I said that test helpers won’t work anymore with the new testing APIs, this only affects global acceptance test helpers. Those are the helpers that are magically available on the global scope during acceptance tests, and are registered using Ember.Test.registerHelper or Ember.Test.registerAsyncHelper.

This does not affect integration or unit tests because that kind of helper never worked with them to begin with.

It is also important to highlight that if you update ember-cli-qunit to 4.2, absolutely nothing will break. In the new version both the old and the new APIs coexist, and that will remain true for a while. You can opt-in to refactor your tests to the new API at your own pace. Manually or using the awesome ember-qunit-codemod to do the heavy lifting.

When you transform a moduleForAcceptance test to the new API that uses setupApplicationTest then, and only then, those helpers will stop working for that module.

There are several reasons why they don’t work:

Those global helpers rely on the Ember.Application being passed in as first argument, which makes the newer (and much faster) system that reuses the same application and only creates a new ApplicationInstance per test impossible.
Ember.Test.registerAsyncHelper in particular implements a very complicated promise chain that doesn’t play nicely with some of the improvements of the new system.
Using globals that happen to be available on window is bonkers, and very very hard to teach. - @rwjblue

But worry not, it’s not like you’ve lost them forever. You can refactor them in a few minutes.

Let’s see how.

Newer and simpler

If the bad news is that you have to refactor, the good news is that the refactored version will be easier to understand and will gain a very important ability: It will also work in “integration” tests now! (Now known as Rendering tests)

Let’s see some concrete example from ember-i18n:

import { registerHelper } from '@ember/test';

registerHelper('t', function(app, key, interpolations) { // requires 3 arguments
  const i18n = app.__container__.lookup('service:i18n'); // uses private API
  return i18n.t(key, interpolations);
});

This was registering a t helper that allowed you to get translations out of the i18n service on your tests.

Step 1: Convert it to a good old function

The first simplification step is that you no longer need to “register” anything. Since in the new tests you are going to directly type import { t } from 'path-to-helpers', you can just export a function.

export function t(app, key, interpolations){
  const i18n = app.__container__.lookup('service:i18n');
  return i18n.t(key, interpolations);
})

The inconvenient bit about the code above is that you want to invoke this helper with t('home.header.login-btn'), without passing the app as first argument as seen above.

Furthermore, your test helpers shouldn’t even want to depend on the application at all, as there is no app in the context of integration tests. The only thing you want is a way of accessing the i18n service from the function. And one that doesn’t require the private and ugly __container__ API.

That is what the getContext utility from @ember/test-helpers is for.

Step 2: Stop using `app.container`, use `getContext().owner`

import { getContext } from '@ember/test-helpers';

export function t(key, interpolations) { // Only two args
  let { owner } = getContext();          // getContext().owner - the universal way to access the container/registry
  let i18n = owner.lookup('service:i18n');
  return i18n.t(key, interpolations);
});

Step 3: There is no step 3, just use it.

You’re done! Now you can use it in any application, rendering or non-rendering test.

import { module, test } from 'qunit';
import { setupTest } from 'ember-qunit';
import { t } from '../my-helpers';

module('Component | user-avatar', function(hooks) {
  setupTest(hooks);

  test('It renders', function(assert) {
    let comp = this.owner.factoryFor('component:user-avatar').create({ username: 'cobra' });

    assert.equal(comp.get('footer'), t('avatar.footer', { name: 'cobra' }));
  });
});

What about asynchronous helpers registered with Ember.Test.registerAsyncHelper?

There is no difference except that async helpers are async functions:

import { fillIn, click } from '@ember/test-helpers';

export async function fillLogin(username, password) {
  await fillIn('#username-field', username);
  await fillIn('#password-field', password);
  return click('#submit-button');
});

import { module, test } from 'qunit';
import { setupRenderingTest } from 'ember-qunit';
import hbs from 'htmlbars-inline-precompile';
import { render } from '@ember/test-helpers';
import { fillLoginForm } from '../my-helpers';

module('Component | login-form', function(hooks) {
  setupRenderingTest(hooks);

  test('overly short passwords are invalid', async function(assert) {
    await render(hbs`{{login-form}}`);
    await fillLoginForm('admin', 'a');

    assert.dom('#errors').containsText('Password is too short');
  });
});

In the above example note that you can use helpers inside other helpers very easily, even if those helpers themselves are asynchronous.

Also this example does not use getContext because it doesn’t require accessing anything inside the container. Not all helpers do.

Let’s recap:

Helpers are now just regular exported functions.
Async helpers are just … [drumroll] … exported async functions.
With async/await, you can compose other async helpers to create new ones easily.
These helpers work in any kind of test using the same getContext public API.

This looks like a great refactor to me!

Maintaining backwards compatibility is good (specially for addon authors)

If you are migrating to the new API all at once, the above is enough.

However if you are an addon author that wants to make your work available to a wide range of projects using a wide range of Ember versions, you can’t expect your users to be fully updated. People will migrate their tests to the new API bit by bit.

Even if you are not an addon author, perhaps your team will migrate the test suite to the new API over the course of a 3 weeks sprint and you want to support both styles simultaneously.

In those cases, you’ll need to do some extra work.

Let’s start with the simplest option, that works for apps but not for addons:

// in tests/helpers/custom-helpers.js
import { getContext } from '@ember/test-helpers';
import { registerHelper } from '@ember/test';

function _t(owner, key, interpolations) {
  let i18n = owner.lookup('service:i18n');
  return i18n.t(key, interpolations);
}

export function t(key, interpolations){
  let { owner } = getContext();
  return _t(owner, key, interpolations)
});

registerHelper('t', function(app, key, interpolations) {
  return _t(app.__deprecatedInstance__, key, interpolations);
});

This will work for apps that are in the process of migrating to the new testing API because you know they will ember-cli-qunit 4.2 installed and therefore import { getContext } from '@ember/test-helpers'; will not fail.

Tests using the new API can explicitly do import { t } from '../helpers/custom-helpers' and those in the old style will still have the global t available to them.

However in addons you don’t know if the consumer app has the new ember-qunit installed, so import { getContext } from '@ember/test-helpers' may be an invalid import and thrown an error.

In that case, I suggest to put the new and the old helpers in different files and extract the helper’s shared logic to a private file that is imported from both.

// /addon-test-support/-private/t.js
export function _t(owner, key, interpolations) {
  let i18n = owner.lookup('service:i18n');
  return i18n.t(key, interpolations);
}

// /addon-test-support/new-helpers.js
import { getContext } from '@ember/test-helpers';
import _t from './-private/t';

export function t(key, interpolations) {
  let { owner } = getContext();
  return _t(owner, key, interpolations)
}

// /addon-test-support/index.js
import { registerHelper } from '@ember/test';
import _t from './-private/t';

registerHelper('t', function(app, key, interpolations) {
  let owner = app.__deprecatedInstance__;
  return _t(owner, key, interpolations);
});

Tests using the new testing API can import exactly the helpers they need while those using the old style can continue to use the registered globals.

Bonus: Where should addons put their test helpers?

This is an aside that has nothing to do with the new testing API. The following advice is valid since 2015 and I bring it up because it’s something that I’ve done wrong more than once and I’ve seen other addons making my same mistake.

If you are the author of an addon and you want to ship some test helpers to make life easier for its users, where should you put those helpers?

Many addons authors used to put those helpers in a directory named /test-support. This “magical” directory is automatically merged with the /tests directory of the consuming app. If you create a helper and you put it in /test-support/helpers/my-helper.js, developers working on apps that use your addon can import that helpers as if it was in /tests/helpers/my-helper.js.

This has two problems:

If you have the pretty reasonable idea of putting your helpers in /test-support/helpers/custom-helpers.js, and the application has the also pretty reasonable idea of creating their own helpers and putting them in /tests/helpers/custom-helpers.js, then you have a naming problem and only one can win. I’m not even sure which one will.
Import paths are relative to the test you are importing from. Due to that, sometimes the path will be ../helpers/custom-helpers but other will be ../../../helpers/custom-helpers. I hate having to navigate the directory structure to import stuff.

Instead, addons that want to expose helpers should put them inside /addon-test-support. And even better, export everything from /addon-test-support/index.js. This directory is also magically merged into the test tree by ember-cli, but it has the name of your addon as its namespace.

You can type: import { doSomething } from 'cool-addon/test-support';. And this import path will remain stable no matter how nested your test’s file is.

If you are not already doing this, the necessary refactor of your test helpers to work with the new tests is also a fantastic occasion to kill two birds with a stone and put the new importable helpers in the /addon-test-support directory and leave the legacy global helpers in the /test-support.

Happy testing!