Building WASM modules for DCP jobs

A work function can call into a compiled C/C++/Rust/etc. library via WebAssembly, the same way any other JavaScript can. Getting a WASM module to actually run cleanly inside a DCP worker, though, depends on how it was compiled – a handful of choices at build time determine whether the module needs any special handling at all once it reaches a work function.

This page covers producing a DCP-ready module: the Emscripten flags and build shape that avoid extra work downstream. Getting a module you already have into a work function – as a job argument, or via job.requires() – is covered in the Node.js and Web job tutorials; this page stops once you have a single, working file.

The running example throughout is a real WASM build of FFTW, the C library for computing discrete Fourier transforms.

The constraint that shapes everything: No unrestricted `fetch()`

A DCP worker sandbox restricts fetch() to an explicit origin allowlist. This is a deliberate security boundary, not an oversight: a work function is untrusted code that can end up running on a machine you don’t control, so it shouldn’t be able to make arbitrary outbound network calls on that machine’s behalf – reaching internal-only services, or exfiltrating data to an attacker-controlled endpoint. The allowlist is what stops a work function from doing either.

That’s a problem for WASM specifically because Emscripten’s default output tries to fetch() its own .wasm binary by URL as part of module initialization – and a worker sandbox will reject that fetch outright, regardless of whether the URL is real or a same-page data: URI.

This single fact is why most of the friction in shipping WASM to DCP exists at all, and it’s entirely avoidable at compile time.

Compile with `-sSINGLE_FILE=1 -sMODULARIZE=1`

Emscripten can embed the compiled .wasm bytes directly inside the generated JS file instead of emitting them as a separate .wasm asset. With that flag set, the default module-loading code never calls fetch() at all – it decodes the embedded bytes from a string literal already present in the file and instantiates directly from them. There’s no URL involved, so there’s nothing for the sandbox’s origin restriction to block.

The practical result: a module built this way needs no special-case code to run inside a work function. No custom instantiateWasm hook, no manually fetching and re-encoding a .wasm file, no base64 juggling. You get one self-contained .js file, and calling its factory function with zero options just works.

FFTW’s build, reduced to the relevant emcc invocation:

# Build FFTW itself with no SIMD, no threads (see the next section for why),
# then compile a small C driver against it:
emcc -O2 fftw-driver.c libfftw3.a \
  -I. \
  -s MODULARIZE=1 -s EXPORT_NAME=FftwModule \
  -s EXPORTED_FUNCTIONS='["_fftw_dft_1d_wasm","_malloc","_free"]' \
  -s EXPORTED_RUNTIME_METHODS='["ccall","cwrap","HEAPF64"]' \
  -s ALLOW_MEMORY_GROWTH=1 \
  -s SINGLE_FILE=1 \
  -o fftw-module.js

fftw-driver.c is a thin wrapper exposing a couple of plain C functions around FFTW’s own API, operating on caller-allocated buffers:

#include <fftw3.h>
#include <emscripten.h>

/* Complex-to-complex 1D DFT. re_in/im_in/re_out/im_out are each n-element
 * double arrays already in wasm memory (caller allocates via
 * Module._malloc, frees the same way). sign is FFTW_FORWARD (-1) or
 * FFTW_BACKWARD (+1). */
EMSCRIPTEN_KEEPALIVE
int fftw_dft_1d_wasm(double *re_in, double *im_in, double *re_out, double *im_out, int n, int sign) {
  fftw_complex *in = (fftw_complex *) fftw_malloc(sizeof(fftw_complex) * (size_t) n);
  fftw_complex *out = (fftw_complex *) fftw_malloc(sizeof(fftw_complex) * (size_t) n);
  if (!in || !out) { fftw_free(in); fftw_free(out); return 2; }

  for (int i = 0; i < n; i++) { in[i][0] = re_in[i]; in[i][1] = im_in[i]; }

  fftw_plan p = fftw_plan_dft_1d(n, in, out, sign, FFTW_ESTIMATE);
  fftw_execute(p);

  for (int i = 0; i < n; i++) { re_out[i] = out[i][0]; im_out[i] = out[i][1]; }

  fftw_destroy_plan(p);
  fftw_free(in);
  fftw_free(out);
  return 0;
}

Once built, fftw-module.js is a single file with everything embedded. Loading it needs nothing beyond what Emscripten’s own MODULARIZE output already gives you:

const FftwModule = require('./fftw-module.js'); // or eval'd from fetched text, in the browser - see the job tutorials
const Module = await FftwModule(); // no options - no instantiateWasm needed

Note

The size cost of SINGLE_FILE=1 isn’t new cost – it’s the same bytes you’d otherwise base64-encode at runtime to ship the module as a job argument, just paid once at build time instead. For a small library like FFTW the difference is negligible; for a large one (tens of MB), the resulting file is simply a large text file, which has no functional downside inside a work function.

Build with no threads, no SIMD that requires threads

DCP work functions run single-threaded, with no SharedArrayBuffer. A library compiled with pthreads, OpenMP, or SIMD codelets that assume threaded execution won’t run correctly in that environment.

Most native libraries only enable those at build time if you explicitly ask for them. FFTW’s ./configure, for example, only turns on SIMD (SSE/AVX/NEON) or threading if you pass --enable-sse2, --enable-threads, --enable-openmp, etc. – all default off:

emconfigure ./configure --disable-shared --enable-static --disable-fortran --disable-doc
emmake make

Leaving those flags off gives portable, generic-C codelets with no threading assumptions baked in – exactly what a single-threaded sandbox needs. Check the library you’re porting for the equivalent switches before you build.

A patch you’ll likely need: `node:`-scheme requires

Even with SINGLE_FILE=1, Emscripten’s generated glue includes a dead code path for ENVIRONMENT_IS_NODE that falls back to require("node:fs") / require("node:crypto"). That branch never executes inside a sandboxed worker (there’s no process global there to satisfy the environment check) – but if you plan to load the module via job.requires(), DCP’s local-dependency bundler still has to statically resolve every require() call it can see in the source, including ones on branches that are unreachable at runtime, and the node: URI scheme isn’t something it resolves. The fix is a one-line patch after linking:

node -e '
const fs = require("fs");
const path = process.argv[1];
let data = fs.readFileSync(path, "utf8");
data = data.replaceAll("require(\"node:fs\")", "require(\"fs\")")
           .replaceAll("require(\"node:crypto\")", "require(\"crypto\")");
fs.writeFileSync(path, data);
' fftw-module.js

This is worth doing as a standard last build step regardless of whether you’re using job.requires() for this particular module – it’s cheap, and it means the same build works whichever way you end up loading it later.

Wrap the raw `ccall`/`cwrap` interface in a friendly API

The fftw_dft_1d_wasm driver function above deals in raw heap pointers – callers have to allocate, copy data in, call, copy data out, and free, in the right order, every time. That’s fine for the module’s own internals, but it’s an unpleasant API to actually consume. Worth writing a small wrapper module that hides the bookkeeping behind a plain-values-in, plain-values-out function:

const FftwModule = require('./fftw-module.js');

let modulePromise = null;
function getModule() {
  if (!modulePromise) modulePromise = FftwModule();
  return modulePromise;
}

function allocDoubles(Module, arr) {
  const p = Module._malloc(arr.length * 8);
  Module.HEAPF64.set(arr, p / 8);
  return p;
}
function readDoubles(Module, p, n) {
  return Array.from(Module.HEAPF64.subarray(p / 8, p / 8 + n));
}

async function dft1d(reIn, imIn) {
  const Module = await getModule();
  const n = reIn.length;
  const fn = Module.cwrap('fftw_dft_1d_wasm', 'number', [
    'number',
    'number',
    'number',
    'number',
    'number',
    'number',
  ]);

  const reInP = allocDoubles(Module, reIn);
  const imInP = allocDoubles(Module, imIn);
  const reOutP = Module._malloc(n * 8);
  const imOutP = Module._malloc(n * 8);
  try {
    const rc = fn(reInP, imInP, reOutP, imOutP, n, -1 /* FFTW_FORWARD */);
    if (rc !== 0) throw new Error(`fftw_dft_1d_wasm failed, code ${rc}`);
    return {
      re: readDoubles(Module, reOutP, n),
      im: readDoubles(Module, imOutP, n),
    };
  } finally {
    [reInP, imInP, reOutP, imOutP].forEach((p) => Module._free(p));
  }
}

module.exports = { dft1d };

Now a caller just does const { re, im } = await fftw.dft1d(reArray, imArray); – no Emscripten knowledge, no manual memory management, no ccall/cwrap in sight. This step is optional (nothing about DCP requires it), but it’s the difference between a module only its author can use comfortably and one anybody on the team can pick up.

Verify it before you ever involve DCP

Before wiring a new module into a work function at all, load it under plain node and call it directly:

const fftw = require('./fftw-wasm.js');
const { re, im } = await fftw.dft1d([1, 0, -1, 0], [0, 0, 0, 0]);
console.log(re, im);

This catches build-format problems – a bad EXPORTED_FUNCTIONS list, a missing symbol, a SINGLE_FILE embedding bug – independently of DCP’s own machinery, which makes them much faster to diagnose than debugging the same failure after a real job dispatch.

What you have at this point

A single .js file that instantiates with zero special-case code, doesn’t touch fetch(), doesn’t assume threads, and (if you added one) a friendly API on top of it. That’s the DCP-ready artifact. Getting it into a work function from there – fetching it as a job argument in the browser, or job.requires()-ing it directly in Node.js – is a small, mechanical step covered in the job tutorials for each platform.

Building WASM modules for DCP jobs

The constraint that shapes everything: No unrestricted fetch()

Compile with -sSINGLE_FILE=1 -sMODULARIZE=1