DCP with Node.js

These tutorials walk through creating and running distributed DCP jobs using Node.js, covering setup, job definition, execution, and result processing.

Environment

Requirements:

Sample Code

This is a minimal code example for a DCP job in Node.js. Full and more advanced configuration options are available in the tutorials, examples, and the specification.

async function main() {
    const compute = require('dcp/compute');

    const inputSet = [1, 2, 3, 4, 5, 6, 7, 8, 9];

    async function workFunction(input, arg1, arg2) {
        progress();
        return input * arg1 * arg2;
    };

    let job = compute.for(inputSet, workFunction, [25, 11]);

    job.on('accepted', () => console.log(`Job id: ${job.id}\nAwaiting results...`));
    job.on('error', (error) => console.error('  Job error:', error));

    let results = await job.exec();

    console.log(results);
};

require('dcp-client').init().then(main);

Job Tutorials

Basic DCP Job Tutorial (Node.js)

Learn the basics of creating and running your first DCP Job in Node.js—this tutorial is essential before moving on to advanced job features.

Distributed ETL with DuckDB-WASM (Node.js)

Run real SQL, joins and aggregations included, inside DCP workers using DuckDB compiled to WebAssembly: the same class of query-driven ETL most teams run today on a managed Spark cluster, distributed on DCP instead. Also covers static job arguments, the arg1, arg2 values passed in the sample code above.

Examples

Batch inference

Distributed batch AI inference with ONNX Wasm and WebGPU backends, and using python pre- and post-processing scripts.

Numerical modelling

Distributed numberical integration for a computational electrodynamics problem, using local modules.