Developer Guide

Middleware

Middleware is custom Node.js code you, or your users, write that executes right before their extension code is run. It allows you to augment the default extension execution logic supported by Extend without requiring any changes in the code of your users.

Most common situations where middleware is useful include:

  1. Authorization. Middleware allows you to implement authorization and access control logic for calls to extension endpoints. For example, the Creating Extensions section recommends using a pattern of securing extension endpoints that builds on middleware.

  2. Logging. Middleware allows you to build a custom logging solution. For example, you can intercept and capture all calls to console.log made by extension code to filter or export it to an external system.

  3. Context enhancement. You can use middleware to provide extension code with additional data or functionality beyond what is provided by default. For example, you can pre-fetch data from a database or make a function that fetches data available for calling from the extension code.

Middleware for an extension is determined via the wt-middleware metadata property. The value will be the URL where the middleware is hosted. It is important to note that this URL should return the source of the middleware and not actually execute it. In other words, if you open the URL specified here in your browser you should see the JavaScript code for the middleware. This is different from a regular extension URL where opening that URL in your browser would actually execute the code versus returning it.

Another important facet of middleware is that the function you write must return a function. In other words, you’re going to write a function that will return the middleware that will then be used by your extensions.

Let’s look at a middleware example that adds a simple authentication check to an extension. Consider the code below:

'use strict';

const AUTH_SECRET_NAME = 'wt-auth-secret';

module.exports = createAuthenticationMiddleware;

function createAuthenticationMiddleware() {
    return function authMiddleware(req, res, next) {
        const ctx = req.webtaskContext;

        if (ctx.secrets && ctx.secrets[AUTH_SECRET_NAME]) {
            const match = (ctx.headers['authorization'] || '')
                .trim()
                .match(/^bearer\s+(.+[^\s-])\s*$/i);

            if (match && match[1] === ctx.secrets[AUTH_SECRET_NAME]) {
                return next();
            }

            const error = new Error('Unauthenticated extensibility point');
            error.statusCode = 401;

            return next(error);
        }

        return next();
    };
}

First make note of how the function createAuthenticationMiddleware is returning a function (authMiddleware). As described above, the code needs to return the middleware function that will be used.

The middleware looks for a secret named by the constant AUTH_SECRET_NAME. If it exists, then the value of that secret must be passed in via the Authorization header. So for example, an extension would add a secret named wt-auth-secret and give it a value like catsAreBetterThanDogs. When this middleware is applied to an extension, the caller will need to pass in an Authorization header that matches that value.

In order to use this middleware, it must be placed online and available over HTTPS. You can find this particular example on GitHub here: https://github.com/auth0/webtask-userspace/blob/master/packages/bearer-auth-middleware/index.js. The corresponding raw URL may be found here: https://raw.githubusercontent.com/auth0/webtask-userspace/master/packages/bearer-auth-middleware/index.js. This is the URL you would use as your middleware.

Compilers

While middleware act as a basic “precursor” to your extensions, compilers have the ability to completely transform how your users create and use extensions. By default, extensions follow one of three programming models:

module.exports = function (cb) {...}
module.exports = function (ctx, cb) {...}
module.exports = function (ctx, req, res) {...}

In order to offer a different programming model, a layer must be built that will adapt between the custom programming model and one of the above. Extend compilers enable such an adaptation layer to exist outside of the extension script itself.

The Extend compiler concept

Extend compilers enable webtasks to be written in domain specific languages, such as:

  • Node.js script with custom programming model
  • Express, T-SQL, C#
  • CoffeeScript, TypeScript
  • Pug, EJS
  • or anything else that can be transpiled to one of the three basic programming models

The compiler concept introduces a first class “compilation” step for an extension script. Compilers are executed by the Extend runtime and run on the server-side when a request to execute an extension is made. Results of the compilation are subject to the same caching mechanisms as if one of the built-in programming models were used. A compiler is declaratively associated with an extension when it is created which enables the code to focus exclusively on the domain-specific programming model.

A webtask compiler is a Node.js function that takes a string literal representing the extension script and is responsible for returning a JavaScript function that matches one of the three basic programming models that the Extend runtime understands. If no compiler is specified for an extension, the script must use one of the three supported programming models directly.

Associating an extension script with a compiler

A webtask compiler is associated with a webtask script at the time of webtask creation using the metadata property wt-compiler. This can be accomplished through the HTTP APIs when creating a webtask, or through the wt-cli command, for example:

wt create hello.js --meta wt-compiler={value}

The value of the wt-compiler property identifies the compiler using either of two mechanisms:

  • Node.js module: The value must be {module_name}[/{function_name}].
  • HTTP[S] URL: The value must be an http[s]://... URL that responds to GET request with the source code of the compiler.

When the extension is called, the runtime will pass the script for compilation to the specified compiler function. The result is then cached and re-used on subsequent calls.

Creating a compiler

In order to create a compiler you must implement a Node.js function that accepts a script in your domain specific language and transforms it into a JavaScript function in one of the three signatures supported by the runtime:

module.exports = function (options, cb) {
  return cb(null, function (cb) {
    cb(null, options.script);
  });
};

The compiler in the example above returns a JavaScript function that matches the simplest programming model of function (cb) {...} and simply returns the script itself as a result of its execution.

The compiler function accepts an options object and a callback:

  • options.script is the script in domain specific language.
  • options.secrets is a hash of all secret parameters the extension has been created with.
  • options.nodejsCompiler is a function with a function (script, callback) signature that implements the default compilation logic for Node.js extensions. It is used if a custom compiler is not specified. It is provided as an optional facility to simplify implementation of compilers that simply transform one JavaScript programming model into another.
  • callback is a function with a function (error, func) signature that must be called when compilation is finished. func must be a JavaScript function in one of the three signatures supported natively by extensions.

Once the compiler is implemented, you must host it somewhere where it can be referenced by an HTTPS URL, for example in S3, on a CDN, or on Github. We have uploaded our example compiler here.

Create an extension using wt create and use the meta tag to refer to the webtask compiler via HTTPS. For our example, we want to create a webtask based on the contents of a foo.js file, and we want to compile it using the compiler we have created at the previous steps:

wt create foo.js --meta wt-compiler=https://gist.githubusercontent.com/tjanczuk/14f67be8bb73f58a5d3c371605558379/raw/a8d0b2539f2904ae22d857ea8c0c0bca4ff8f7ac/reflector_compiler.js

Meta values can also be set using the Extend editor:

The Extend editor supports setting meta values

Compiler versioning

There are two mechanisms that can be used to version compilers depending on the scenario:

  • Non-breaking: New compiler code can be slipstreamed into an existing Node.js module or HTTP[S] URL. This is suitable for progressive, backwards-compatible enhancements. The key benefit of this approach is that existing extensions do not have to be modified and they remain associated with the wt-compiler specified at the time of their creation.
  • Breaking/New: Brand new compiler code can support completely a different domain specific language (e.g. C# instead of Node.js). This is particularly suitable for breaking changes that require explicit opt-in by the author of the extension code. In order to use a new compiler, a new extension must be created or the old one re-created with the new value of wt-compiler and appropriately modified extension script.

Compiler examples

You can review more compiler examples here.