Isomorphic (ES6 + Node.JS) Module Boilerplate W/ Universal Testing

Congratulations! If you are still JavaScripting in 2019, you made it to the end of dependency deployment hell. This post introduces the isomorphic module boilerplate. Features:

• Modules written in ES6, but also available in Node.JS.
• Install and build source code from git, not [npm, bower, centralized package manager].
• Support a flat directory structure.
• Also support node_modules folders for backwards compatability.
• Run the same unit tests in NodeJS as well as browsers
• Run unit tests against every browser and OS (on travis-ci.org)

History

ECMAScript

First a little history of our problem. Originally, JavaScript (ECMAScript) had no concept of a module. Everything was just scripts on top of scripts. Example ES5 "module":

(
  // My module
  function helloWorld () {
    console.log("hello world")
  }
)()

CommonJS and AMD

After some time, the community came up with a few competing standards, including CommonJS (used by Node.JS on servers) and Asynchronous Module Definition aka AMD (used by RequireJS in browsers).

CommonJS Example:

// My Module
require('other-module')
module.exports = {
  helloWorld: function helloWorld () {
    console.log("hello world")
  }
}

AMD Example:

define(['other-module'], function(othermodule) {
  return function helloWorld() {
    console.log("hello world")
  }
})

For a long time, this division was a hard one, and modules did not cross the line without transpilers such as gulp, babel, webpack, rollup. While there's nothing inherently wrong with transpiling JavaScript, it is an expensive and potentially unnecessary step. Why can't your module just run everywhere, as is?

UMD

Enter Universal Module Definitions aka UMD, a set of patterns that are both CommonJS and AMD compatible. UMD is not a single fixed pattern, but rather a complex set of choices, depending on exact deployment environment. The code looks quite complex, and it is indeed a headache to implement.

UMD Example:

(function(root, factory) {
  if (typeof define === 'function' && define.amd) {
    define(['other-module'], function(othermodule) {
      return (root.returnExportsGlobal = factory(othermodule));
    });
  } else if (typeof module === 'object' && module.exports) {
    module.exports = factory(require('other-module'));
  } else {
    root.returnExportsGlobal = factory(root.othermodule);
  }
}(typeof self !== 'undefined' ? self : this, function(othermodule) {
  return function helloWorld() {
    console.log("hello world")
  }
}))

Phew, obviously that is not an ideal developer experience. But hey, it worked, and your code would run in the browser the same as it would run in on the server.

ES6

Finally, in 2015, ECMAScript 2015 aka ES6 was published, which included a standardized module format. This format was simple, clean, and easy to use.

ES6 Module Example:

import { otherfn } from 'other-module'
export function helloWorld() {
  console.log("hello world")
} 

Problem solved, right? Not so fast!

ES6 modules are, unfortunately, not compatible with the others, including UMD. This is mainly due to it's asyncronous nature, requiring all import and export statements to be at the top level of the module. (i.e. not inside any sort of conditional block or function)

Furthermore, as of 2019, parts of the module ecosystem such as the dynamic import statement are still in the draft status. Firefox only this year started supporting dynamic imports, and Node.JS still puts them behind an --experimental-modules flag.

esm Pattern

Thankfully, some nice community members created esm, a convenient tool for using ES6 modules in Node.JS today.

esm Example:

// content of index.node.js
require = require("esm")(module/*, options*/)
module.exports = require("./index.js")

// content of index.js
import { otherfn } from 'other-module'
export function helloWorld() {
  console.log("hello world")
}

This is a strong and future-proof module pattern, and is easy to use thanks to the npm init esm and yarn init esm commands. Still, it doesn't solve every problem.

One big problem between the different module systems are dependencies. Node.JS looks for dependencies by name in a special node_modules folder, while typical browser deployments have a flat file structure, or even a single bundled script. What type is our other-module dependency? Should we look for it in ./node_modules, or where?

Furthermore, how is the other-module distributed? npm? A CDN? How can you trust the code hasn't been tampered with by middle men?

Future

Isomorphic Module Boilerplate

Welcome to the future! The isomorphic module boilerplate uses esm and gpm (git+npm) to publish modules compatible with both ES6 and Node.JS, as well as flat and node_modules directory structures.

Git Distribution

Isomorphic module boilerplate uses gpm to install packages from git source code instead of a centralized package manager. This eliminates middle men from the code distribution channel, and ensures the latest code is available.

Note that gpm is a peer dependency of Isomorphic module boilerplate, and must be installed globally. The following command will do the trick.

npm i -g https://github.com/isysd-mirror/gpm#isysd

After this, npm can be used as normal, since gpm is set in the preinstall hook in package.json.

gpm -n .. -t .. -u https -e -i .

This preinstall hook will install dependencies and devDendencies to the parent directory (..), preferring https to ssh as a git protocol.

A postinstall hook is currently required to ensure that the esm package is built, since the git branch does not include the build directory. The script is prettified here for your convenience.

try {
  require('../esm/esm.js')(module);
} catch (e) {
  require('child_process').execSync('npm i',
  {
    cwd: require('path').join('..', 'esm')
  })
}

This script will check if esm is built, and run npm i in ../esm if it is not.

Flat + node_modules

Isomorphic module boilerplate is compatible with node_modules folders, as well as flat, deployable folders (i.e. every dependency in a single folder, side by side). The goal is for you to be able to deploy your isomorphic module to a browser environment as-is, without any bundling or mapping of package names.

It's easiest to understand by following an example installation.

Step 1 create JS source directory

mkdir js
cd js

Step 2 clone this module (and/or fork your own)

git clone https://github.com/isysd-mirror/iso-module-boilerplate.git
cd iso-module-boilerplate
$ ls
index.js  index.node.js  package.json  package-lock.json  README.md  test.js
$ ls ..
iso-module-boilerplate

Step 3 npm install

$ npm install

# List of files in parent (/js) directory no includes esm and iso-test repositories from git, as well as their dependencies
$ ls ..
esm  iso-module-boilerplate  iso-test  is-wsl  open  tree-kill

# Node_modules contains symbolic links to modules in parent directory
$ ls -l node_modules/
total 0
lrwxrwxrwx 1 isysd isysd  9 Apr 14 00:34 esm -> ../../esm
lrwxrwxrwx 1 isysd isysd 14 Apr 14 00:34 iso-test -> ../../iso-test

Test Everywhere

Isomorphic module boilerplate imports iso-test to run the same test code in both NodeJS as well as the browser of your choice.

There are no complex test drivers, extra browser builds, or complex APIs to learn.

Write your unit test in test.js and call finishTest with the result. Anything beginning with "pass" will pass, everything else will fail, including uncaught errors.

Since iso-test is a devDependency, gpm does not install it automatically. Before testing, install it with:

gpm -n .. -t .. -u https -i iso-test

Finally, Travis CI is integrated to test your code using chromium, chrome, firefox, and safari, on linux, osx, and windows.

That's a lot of test coverage.

To get started, fork the boilerplate! It is licensed MIT and contributions are welcome.