01 Oct 2018, 20:37

A Quick Look at WorkerDOM

In the browser, many operations occur on a single main thread. Due to the number of things we need to handle as web developers such as styling, DOM updates, fetches, data transforms, timers and so forth we can end up with a lot going on that thread. Unfortunately, this thread is also responsible for handling user inputs and rendering to the screen, which are user critical requirements and have large impacts on their experience. Here if we have tasks that run for long periods this can block responding to user input and rendering new frames.

In the browser we have access to threading via Web Workers. Web Workers allow us to run JavaScript in a separate thread whilst also being able to message back to the main thread. The limitations of Web Workers are that they have transfer times for data between the main and worker thread, limited transfer types (i.e. Functions and Errors aren’t allowed) and also perhaps most importantly no DOM access (i.e. document.getElementByID for example). I have experimented and written about transfer times for workers, showing that for small loads the fear is probably over played. However a core aspect of frontend development is, maybe somewhat obviously, trying to update what is happening on the screen (render new content). This makes the DOM limitation pretty painful for those using Web Workers, as it mostly limits it to computational logic which we then proxy back to the main thread.

On mobile devices, where performance perhaps matters the most, we’re now seeing low-end devices with multiple cores (for example a Nokia 1 and the Micromax Bharat Go both have quadcore processors). Arguably this is where multiple threaded code would probably have the largest benefit. However, we still have a particularly single-threaded approach to writing browser code. This is why it’s interesting to see the emergence of the recently announced WorkerDOM library from the AMP team, which aims to let developers leverage workers more easily, by removing the aforementioned limitation of lack of DOM access (or at least perceived lack of DOM access).

What is WorkerDOM?

WorkerDOM is a library which you can include in your applications and pages. It provides much of the regular DOM API and in turn proxied access to the real DOM on the main thread, passing mutations to the DOM over to the main-thread to be handled. This encourages a lot of logic that was previously happening on the main thread to be handled by the Web Worker, leaving the main thread to handle user input and DOM changes. The library itself is written in TypeScript and is compiled down to both global variable and module formats.

Setting up WorkerDOM

You can include WorkerDOM by installing it via NPM like so:

npm install @ampproject/worker-dom

or if you use Yarn then:

yarn add @ampproject/worker-dom

Alternatively you could use a script tag directly, using a CDN, in the following fashion:

<script src="https://unpkg.com/@ampproject/worker-dom@0.1.2/dist/index.js"></script>

We can actually take this a step further as WorkerDOM is distributed as both a global (aliased as MainThread) and an ES Module, allowing for the import syntax. This means you can do the following;

<script src="https://unpkg.com/@ampproject/worker-dom@0.1.2/dist/index.mjs" type="module"></script>
<script src="https://unpkg.com/@ampproject/worker-dom@0.1.2/dist/index.js" nomodule defer></script>

Here only browsers that understand modules (all modern browsers, except Samsung Internet now support modules) will receive the module script.

First Steps

WorkerDOM allows us to expose a specific part of the DOM to be upgraded, we can do this by doing something we would probably not normally do; we set the src attribute on the containing element we want to upgraded to work with WorkerDOM. The src attribute is updated to be the name of the script we’re interested in running as our worker script. For the purposes of this blog, we are going to generate some prime numbers and render them into the DOM. For our index.html file, we might start with something like this in our case:

<body>
    <div src="primes.js" id="primes">
    </div>
</body>

So now we have an element that we can upgrade in our body. We need to declare explicitly that we want to upgrade the element using the upgradeElement method. We can do that like so:


<script type="module">
    import {upgradeElement} from '/dist/index.mjs';
    upgradeElement(document.getElementById('primes'), '/dist/primes.mjs');
</script>

<script nomodule async=false defer>
    document.addEventListener('DOMContentLoaded', function() {
        MainThread.upgradeElement(document.getElementById('primes'), '/dist/primes.js');
    }, false);
</script>

You can see we take both the module and none module code paths allowing us to handle both scenarios. This means we can now begin looking at our actual worker logic!

The Worker File

The coolest part about WorkerDOM is that it allows us to behave as if we have DOM access in the worker thread. That means we have access to properties like document.createElement for example. Continuing on with the concept that we want to generate primes and add them to the DOM, we could do something like this:


const startNumber = 1;

function generatePrimes() {

    while (document.body.firstChild) {
        document.body.removeChild(document.body.firstChild);
    }

    const numDivs = 1000;
    const limit = startNumber + numDivs;
    const primes = sieveOfEratosthenes(limit); // An algorithm for generating primes up to 'limit'

    const div = document.createElement('div');
    div.className = 'parent';

    for (let i = startNumber; i < limit; i++) {
        const numberDiv = document.createElement('div');
        numberDiv.className = "number";
        const numberText = document.createTextNode(i);
        if (primes.has(i)) {
        numberDiv.style.fontWeight = 'bold';
        numberDiv.style.color = '#240098';
        }
        numberDiv.appendChild(numberText);
        div.appendChild(numberDiv);
    }

    document.body.appendChild(div);
    startNumber += numDivs;

}

setTimeout(generatePrimes, 0); // Not sure why we need this in a timeout?
document.body.addEventListener('click', generatePrimes);

This code will create divs with numbers in them, with prime numbers being highlighted and put in bold. The numbers will update on click on the document. Notice how we can behave as if this worker code is on the main thread, with access to DOM APIs. It’s worth pointing out the main difference here is that the "primes" div from the index.html is considered our document.body here.

As it stands, WorkerDOM is currently in alpha, and as such is still being worked on. Hopefully, this post has given you a reasonable overview of the library, and if you are interested in learning more about WorkerDOM I would recommend these resources:

24 Dec 2017, 14:53

Getting going with ES6 Maps

As we move into a world where modern browsers are supporting the nearly the entire ES6 specification, we have a variety of data structures to use for storing and accessing our data as native built-in primitives. In ES6 we specifically gain access to ‘keyed collections’. Keyed collections encompasses four built-ins, these are namely Set, Map, WeakSet and WeakMap. Here we’ll be talking about the Map in particular. Let’s get stuck in!

What is a Map?

A Map is a collection of key value pairs, where both key and values can be of any type. This differs from how one might have traditionally used a JavaScript Object as a map, as historically keys have been strings only (this has changed with the introduction of Symbols but that’s for a different blog post to come!).


const obj = { 1 : "one" }
typeof Object.keys(obj)[0] // Evaluates to string

So with Maps, we have the additional power of using anything we like as our key, even a HTMLElement, which is powerful because it makes it easy to associate data with a DOM node. To illustrate the use of none String keys, we can see this example using booleans:


const map = new Map([
    [true, "Things that are true"],
    [false, "Things that are false"]
]);

Here true and false are the respective keys in our collection. So how does JavaScript know how to evaluate something as matching a Map key? Well the approach is very similar to the === (identity) operator, with the minor caveat that NaN (Not a Number) equates to NaN even though NaN !== NaN.

How do we use it?

Maps provide a set of methods for interfacing with them such as get, set and delete. These methods provide a clean inteface for interacting with our data. Let’s see how that works in practice:


const one = "one";
const map = new Map([ ["one", 1] ])
map.get("one"); // Returns 1
map.get(one); // Returns 1

map.set("two", 2);
// Returns the Map with "two" set to 2

map.delete("one");
// Returns true if successful or false if unsuccessful

map.clear();
// Returns undefined

As you can see there’s a little gotcha here, get, set and delete all have different return behaviours. Get will return the associated value, set will return the Map itself, and delete will return a boolean, and clear returns undefined. Something to keep in mind!

We also have some really nice convenience methods to help us interface. For example, has which will tell us if a key exists in a Map:


const map = new Map([ ["one", 1], ["two", 2] ]);
map.has("two") // true

This is very useful for quickly checking the existence of key in a Map. What about if we want to iterate over a Maps contents? Firstly there are a few convenient methods in this regard: values, entries and forEach. Let’s look at values and keys. Both these methods return an iterable. Iterables are a little outside the scope of this post, but think of them as JavaScript variables that have defined behaviours for what happens when you iterate over them (for example using for…of):


// Entries
const map = new Map([ ["one", 1], ["two", 2] ]);

for (const num of map.entries()) {
    console.log(num); 
    // Logs ["one", 1]
    // Then ["two", 2]
}

// Values
for (const key of map.values()) {
    console.log(key);
    // Logs 1
    // Then 2
}

Alongside this we have the forEach method that some of you may be familiar with from the Array method of the same name. It behaves like so:


const map = new Map([ ["one", 1], ["two", 2] ]);
map.forEach((key, value) => console.log(key, value));
// Logs ["one", 1]
// Then ["two", 2]

Lastly we can quickly and conveniently get the size of a map with the size method. It is fairly straight forward and operates like so:

const pets = Map([ ["dog", "woof!"], ["cat", "meow"], ["goldfish", "bop bop bop"] ]);
animals.size(); // Evaluates to 3

Which would have been less elegant using an Object:

var pets = {"dog", "woof!", "cat": "meow", "goldfish": "bop bop bop"};
Object.keys(pets).length;

Why use a Map?

Simply put, the Map is more idiomatic and effective primitive than an Object for use as a mutable key-value pair data structure. We have the added flexibility of using anything we please as our keys, alongside a clear set of a powerful methods that we have shown above. These methods allow us to be more semantic and straightforward about interacting and manipulating our data. It also avoids some of the potential pitfalls of the Object, for example, because of the interface of a Map, we can avoid the possibility of colliding with default Object properties such as toString or hasOwnProperty. This is obviously an edge case but provides us with extra reassurance:


// With a traditional Object
const obj = {};
obj["toString"] = "woops";
console.log(obj.toString);
// Evaluates to woops

// With a Map
const map = Map();
map.get("toString") // undefined
map.set("toString", "This is fine");

Overall it expresses a lot of cool properties that suite it better to behaviours we wanted from using an Object as a Map previously. Happy Mapping!

23 Aug 2017, 19:28

Low Hanging PWA Fruit: Manifest Files and Service Worker Precache

It would be a fair assessment to state that Progressive Web Apps (often abbreviated to PWA) are a popular buzzword in the web development and JavaScript communities at the moment. We have also seen further excitement recently, generated by Apple’s move to start supporting development for PWA features within Safari.

Many great posts and documentation exist around explaining the full case for PWAs and how they benefit users. For a solid introduction to Progressive Web Apps check out Google’s page on the matter. In addition, if you’re looking for something tangible there’s also lots of great examples of PWAs that you can explore in more detail at pwa.rocks.

Perhaps now more than ever it’s time to start exploring how you can start looking into aligning your app or site with PWA functionality. This post will be more practical than theoretical, but to give some context I will list out some of the core concepts of what makes a PWA for those of you unfamiliar:

  • Pages are responsive: The app’s content fits in a sensible way on a tablets, mobiles etc
  • Site is served over HTTPS: assets and resources are served using encryption over the wire
  • The entry page loads offline for users
  • The app uses a manifest.json file to allow adding the app to a users homescreen
  • Load times are reasonable even on slower connections like 3G
  • Each page has its own unique URL
  • The site works across modern browsers

Here we will just focus in on two of these concepts and how to integrate them, namely adding a manifest.json file and adding a Service Worker. Subsequently, we can also use the Service Worker to allow the initial page to work offline by using a precache.

Adding a Manifest File

A manifest file provides a few additional weapons to the web app arsenal for developers. The file itself is fairly plain; a JSON file that contains configuration and metadata for various aspects of the application.

However what the manifest file allows for is a whole host of interesting benefits for those looking to give a more ‘app like’ experience:

  • Allowing users to add a web app to their home screen
  • Giving that home screen button a unique name and icon
  • Giving a splash screen to the app when the user loads it
  • Being able to add a theme colour for the web app
  • Define if the app should load ‘standalone’ or in a web browser

Now we’ve gone through why you would add a manifest.json file, let’s take a look at an example config:


{
  "name": "SomeCoolApp",
  "short_name": "SomeCoolApp",
  "start_url": ".",
  "display": "standalone",
  "background_color": "#111111",
  "description": "An app for doing cool things",
   "icons": [{
    "src": "images/manifest/48.png",
    "sizes": "48x48",
    "type": "image/png"
  }, {
    "src": "images/manifest/72.png",
    "sizes": "72x72",
    "type": "image/png"
  }, {
    "src": "images/manifest/96.png",
    "sizes": "96x96",
    "type": "image/png"
  }, {
    "src": "images/manifest/144.png",
    "sizes": "144x144",
    "type": "image/png"
  }, {
    "src": "images/manifest/168.png",
    "sizes": "168x168",
    "type": "image/png"
  }, {
    "src": "images/manifest/homescreen192.png",
    "sizes": "192x192",
    "type": "image/png"
  }],
  "related_applications": [{
    "platform": "web"
  }, {
    "platform": "play",
    "url": "https://play.google.com/store/apps/details?id=SomeCoolApp"
  }]
}

You can see that we provide a name and short_name which give a name and an abbreviated name for the app when space is a premium. start_url expresses where the kick off URL for the application when the user starts the app. For example, it could be their profile page or the home page.

As mentioned, we can decide if the app should load in a web browser or have a more native experience. This is realised through the display property. We can provide standalone to give a more native feel as it hides away browser features like the URL bar (this may be a negative feature depending on your feelings!). There are other options for this such as fullscreen which hides OS UI features, and also minimal-ui which take a middle ground on the matter.

The background_color property allows for us to express a colour that will occur before the app has fully loaded into its container. icons provide a way to give icons for our applications at different device sizes. For a full list of options check out MDN’s page on the matter.

You could write the manifest file manually (see Google’s instructions here). However, if you want a more streamlined experience you could use a generator instead. One generator I found user-friendly and straight forward is the app-manifest app on Firebase, which you can use to generate a zip file which can be unloaded into your web app directory.

Lastly we can include it in our home page’s HTML as so:

<link rel="manifest" href="/manifest.json">

Adding a Service Worker

The Service Worker is a slightly more complex concept than the manifest file. In essence Service Workers are web workers that allow code to run outside of the web page context within the browser. The Service Worker also in some ways acts as a network proxy, allowing the developer to change the behaviour of network requests (i.e. to read from a cache rather than a remote resource). In principle, these two concepts allow for a host of new opportunities for web apps, including push notifications, offline interactivity and background sync (sending server requests upon resumption of connectivity).

Before we dive in any deeper, it’s important to note that Service Workers require HTTPS to operate, so you will need to have that setup on your site before continuing (checkout Let’s Encrypt if you’re in need of an SSL certificate).

Although we could dig in deep into the Service Worker internals and life cycle, I would rather focus on getting us up and running with a Service Worker. Others have done a great job explaining Service Workers in depth, for example see the Google primer on Service Worker or check out Phil Nash’s great introductory video here. Instead we will focus on one example Service Worker template that allows us to get a registered Service Worker into our page quickly.

An easy to leverage Service Worker template is provided by sw-precache. Using a precache allows you to serve resources from the Service Worker cache, rather than having to wait for it to come over the wire which improves the performance of the app. It also allows us to get resources whilst offline. sw-precache itself is a tool that allows for the automatic code generation of a Service Worker for caching assets that we would normally receive over the network (images, css etc). As sw-precache requires a preprocessing step to generate the Service Worker we need to leverage a task runner. Here we can use gulp, a JavaScript task runner to create a task to generate the list of resources we want to precache. Here’s some example code with comments:


    // Register the service-worker task with Gulp
    gulp.task('service-worker', function(callback) {

        // Get hold of the sw-precache module
        var swPrecache = require('sw-precache');
        var rootDir = '../public/';
        var serviceWorkerName = "precache"; 
        var fileName = `${rootDir}${serviceWorkerName}.js`;

        // Write the precache Service Worker
        swPrecache.write(fileName, {

            // An array of file paths to precache (we use globbing)
            staticFileGlobs: [
                rootDir + 'index.html',
                rootDir + 'templates/.{html}',
                rootDir + 'js/**/*.{js}',
                rootDir + 'css/**/*.{css}',
                rootDir + 'fonts/**/*.{svg,eot,ttf,woff,woff2}',
                rootDir + 'icons/**/*.{svg}',
                rootDir + 'images/**/*.{png,jpg}',
            ],
            // We remove the prefix and can replace it with another
            // Here `../public/` becomes `/` for example
            stripPrexix: rootDir,
            replacePrefix: "/"

        }, callback);

    });

Running gulp service-worker will generate our Service Worker precache file called precache.js. We can now register this into our app using the code expressed below:

    // Check to make sure the Service Worker feature exists
    if ("serviceWorker" in navigator) {
        // On page load, call the Service Worker registration
        window.addEventListener('load', registerSW);
    }

    // The actual registration code
    var registerSW = function() {

        // The location of our generated pre-cache Service Worker
        var swPath = "/precache.js";
        navigator.serviceWorker.register(swPath)
            .then(registrationSuccess, registrationFailed);

    }

    var registrationFailed = function(error) {
        // Do something if there is an error
    }

    var registrationSuccess = function(registration) {
        // Do something if everything is successful
    }

Once the Service Worker is registered the user will make use of the cache when making network requests for cached assets.

Conclusion

Here we have shown how you could add a manifest and a Service Worker to your web app without complicating your application code too much. If you already have gulp in your build process integrating sw-precache shouldn’t be too much extra effort to get up and running. Similarly using the manifest.json app shown above we can simply generate a working manifest file for our application.

Hopefully, this article has begun to pave the way for further PWA features in your applications. Support is gradually getting there so the reasons to avoid the potential benefits are decreasing. If you are interested in auditing your app to see how much it adheres to Google’s PWA checklist, check out a Chrome extension called Lighthouse. Lighthouse will score your app depending on how well it compares a set of pre-defined criteria.