Fraser Xu

by Fraser Xu

Running Headless JavaScript Testing with Electron On Any CI Server

Background

Since the end of 2015, the Envato Front End team has been working on bringing a modern development workflow to our stack. Our main project repo powers sites like themeforest.net and serves around 150 million Page Views a month, so it is quite a challenge to re-architect our front end while maintaining a stable site. In addition, the codebase in 9 years old, so it contains the code from many developers and multiple approaches.

We recently introduced our first React based component into the code base when we developed an autosuggest search feature on the homepage of themeforest.net and videohive.net. The React component was written with ES6, and uses Webpack to bundle the JavaScript code.

As I mentioned above, it’s a 9 year old code base and nobody can guarantee that introducing something new won’t break the code, so we began all the work with tests in mind. This post documents our experiences developing the framework for testing the React based autosuggestion component.

videohive

One issue I had while writing unit test code is that some of the code depends on a browser based environment because they need to access to some browser only object or APIs.

The first solution

Most of the unit tests nowadays are running with Nodejs, so in order to emulate a browser environment, jsdom showed up.

A JavaScript implementation of the WHATWG DOM and HTML standards, for use with Node.js.

Here’s a handy snippet that you could use before your testing code to prepare a DOM environment:

1
2
3
4
5
6
7
8
9
10
11
12
13

import jsdom from 'jsdom'

// This part inject document and window variable for the DOM mount test
export const prepareDOMEnv = (html = '<!doctype html><html><body></body></html>') => {
  if (typeof document !== 'undefined') {
    return
  }
  global.document = jsdom.jsdom(html)
  global.window = global.document.defaultView
  global.navigator = {
    userAgent: 'JSDOM'
  }
}

And in your test code, you could just import it and use it by calling the function.

1
2
3

import { prepareDOMEnv } from 'jsdomHelper'

prepareDOMEnv()

If your code depends on some DOM helper function like jQuery, you may also need to include the source code of jQuery into the prepared environment, you could do:

1
2
3
4
5
6
7
8
9
10
11
12

import fs from 'fs'
import jsdom from 'jsdom'
import resolve from 'resolve'

const jQuery = fs.readFileSync(resolve.sync('jquery'), 'utf-8')

jsdom.env('<!doctype html><html><body></body></html>', {
  src: [jQuery]
}, (err, window) => {
  console.log('Voilà!', window.$('body'))
  // your actual test code here.
})

Notes: in the official jsdom github repo, they give an example of loading jQuery from the CDN which needs an additional network request and can be unreliable and not work if without network. They also have an example loading jQuery source code with nodejs fs module but it’s not clean as you have to tell the path to jQuery.

Everthing looks OK so far, but why do we bother to having a real browser environment?

The reason is that once things get compliated, your code may depend on more browser based APIs. Of course you could fix your code but what if you are using 3rd party moudles from npm, and one of them happen to depends on XMLHttpRequest, it’s nearly impossible to “mock” everything, and to be honest, I feel uncomfortable doing it this way as it’s really tricky and kinda dirty.

Let’s run it in a browser

Why not Phantomjs

From the problem we saw above, it’s pretty straight forward to think about just running all the tests in a real browser. If you search “headless browser testing” on Google, the first result will be PhantomJS.

I haven’t used phantomjs a lot and I’m not familar with how it works, but I’ve been heard bad things about it, “lagging behind more and more from what actual web browser do today”, “have 1500+ opened issues on Github”, “unicode encode issue for different language”.

The last concern is actually from my own experince and I mentioned it in my another blog post PDF generation on the web.

Last but not least, I’m not quite confident about how Phantomjs deals with Nodejs code. As my testing code is actually not browser only code, it needs to access to nodjes fs module as well.

Let’s talk about Electron

What is Electron?

Build cross platform desktop apps with web technologies. Formerly known as Atom Shell. Made with <3 by GitHub.

It would take another blog post to explain what Electron is and what it does, I have built a few projects with it and also have written a few blog posts about it. The short version, and what really matters to me, is A Nodejs + Chromium Runtime, actively maintained by fine folks from Github and used by Atom editor, Slack etc. To conclude I’ll quote from one of my favourite JavaScript developer dominictarr

Electron is the best thing to happen to javascript this year. Now we get root access to the browser!

Let’s run our code in browser

Please read the quick start guide and make sure you know how to write your first Electron App.

Since we are not building a real Electron app here but only want to run our JavaScript code in it, there’s a project called browser-run.

You can install it with npm install browser-run and use it like this:

1
2

$ echo "console.log('Hey from ' + location); window.close()" | browser-run
Hey from https://localhost:53227/

Run test in Electron

And if you are writting your test with tape, you could even pipe your testing result to a test reporter like faucet

1

browserify -t babelify test.js | browser-run -p 2222 | faucet

There are also a tool specific designed for tape named tape-run

A tape test runner that runs your tests in a (headless) browser and returns 0/1 as exit code, so you can use it as your npm test script.

With this tool, it is even easier to run your test.

1

browserify -t babelify test.js | tape-run | faucet

Tip: There’s also one module to run mocha test named electron-mocha.

Important notes

As the title indicate, this post is about running tests on any CI server. The reason is that most of the CI servers are neither Mac or Windows, and there’s a known issue with running Electron on Linux, you need a few setup steps to get it running.

Here’s a few notes copied from the repo and thanks to juliangruber for including my section on running it on gnu/linux there.

To use the default electron browser on travis, add this to your travis.yml:

1
2
3
4
5
6
7
8

addons:
  apt:
    packages:
      - xvfb
install:
  - export DISPLAY=':99.0'
  - Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &
  - npm install

Source.

For Gnu/Linux installations without a graphical environment:

1
2
3
4

$ sudo apt-get install xvfb # or equivalent
$ export DISPLAY=':99.0'
$ Xvfb :99 -screen 0 1024x768x24 > /dev/null 2>&1 &
$ browser-run ...

There is also an example docker machine here.

Final step

Once we have all setups ready, our test will be much simpler without the need to “hack” a browser like environment:

1
2
3
4
5
6
7
8
9
10
11
12
13

import test from 'tape'
import React from 'react'
import jQuery from 'jquery'
import { render } from 'react-dom'

test('should have a proper testing environment', assert => {
  jQuery('body').append('<input>')
  const $searchInput = jQuery('input')

  assert.true($searchInput instanceof jQuery, '$searchInput is an instanceof jQuery')

  assert.end()
})

And you can put the test code in npm script and call it on your CI

1
2
3
4
5
6
7

{
  // ...
  "scripts": {
    "test": "browserify -t babelify test.js | tape-run | faucet"
  }
   // ...
}

Conclusion

Voilà! That’s all we needed to get headless JavaScript test running on any CI server. Of course your testing environment may different from mine but the idea is there.

As front-end development is changing rapidly recently with things like single page application, isomorphic universal apps, also front-end tooling system like npm, Browserify, Babel, Webpack, testing will become more complex. I hope this setup will make your life suck less and be significantly eaiser.

Last but not least, if you have any questions or better way for testing setups, let us know!