Data privacy and security in DCP

DCP lets you harness a global network of workers, coordinated through a central Scheduler, without handing that Scheduler custody of your data or unrestricted access to your account. Two mechanisms make this possible: you control where each piece of a job’s data physically travels, and you control exactly what any given credential is allowed to do. This page covers both.

The Scheduler coordinates; it doesn’t have to hold your data

A DCP job has four components: the input set (data to process), the work function (code that processes each input), the arguments (extra parameters shared by every invocation of the work function), and the results (the output). By default, all four flow through the Scheduler – simplest to set up, appropriate for non-sensitive workloads. But each of these four can instead be routed directly between your own infrastructure and the workers, bypassing the Scheduler for that piece of the job.

Remote input data. With RemoteDataSet, workers fetch input data straight from URLs you host, instead of receiving it from the Scheduler:

const { RemoteDataSet } = require('dcp/compute');

const inputSet = new RemoteDataSet(
  'https://your-server.example/data/0.json',
  'https://your-server.example/data/1.json',
);
const job = compute.for(inputSet, workFunction);

RemoteDataSet also works for the arguments array, and individual elements of an input set or argument list can mix inline values with remote URLs – you don’t have to route everything the same way. See Deploying jobs with remote data for the full guide, including RemoteDataPattern for URL-templated input sets.

Remote results. job.setResultStorage(locationUrl, postParams) has workers deliver each slice’s result directly to a URL you specify, instead of to the Scheduler:

job.setResultStorage(
  new URL('https://your-server.example/results'),
  { apiKey: 'your-storage-provider-key' }, // e.g. credentials for S3, Dropbox, etc.
);

postParams is sent as application/x-www-form-urlencoded alongside the result, so it can carry whatever your storage endpoint needs to authenticate the upload. The real value never reaches the Scheduler, or anywhere else in DCP, at all – what flows back through the normal job-completion channel is whatever your own endpoint’s response to that POST happens to be, not the data itself. See Getting results back once storage is remote for what this means in practice.

Remote work function. The work function can be a URL, fetched by workers at execution time instead of being embedded in the job:

const job = compute.for(
  inputSet,
  new URL('https://your-server.example/work-function.js'),
  args,
);

This keeps proprietary logic off the Scheduler entirely – useful for protecting algorithms, and for building catalogues of vetted, pre-approved work functions that third parties can run without ever seeing (or being able to alter) the code.

Note

Whatever isn’t routed remotely still has to cross the wire to a Worker, which means it goes through KVIN serialization – see that page for what kinds of values do and don’t survive the trip.

Put together: a job where the input set and work function are both remote, and results are written to your own storage, never exposes its data or its logic to the Scheduler at all – the Scheduler only ever sees that a job exists, which workers are assigned to it, and that everyone involved has authenticated. That’s enough for it to do its job (coordination and payment) without needing to know what the job actually computes.

Fine-grained access with proxy keys

The API keys you create in the Portal (see Getting set up) aren’t bearer tokens with all-or-nothing access – each one is a cryptographically signed, capability-scoped credential. Under the hood, every operation in DCP – job.exec, job.cancel, job.results.fetch, worker.register, and many others – is named by an operation key, and a key can be issued for a narrow slice of the operation-key tree rather than the whole thing. Restrictions can be layered further: a key can be limited to a specific job or resource, capped to a number of uses, or given an expiry date.

This is what the Portal’s key Purpose picker is choosing between under the hood – “Job Creation” and “Universal Access” are two friendly presets over the same fine-grained mechanism; nothing stops a more specialized key from being cut for a narrower purpose than either preset covers.

Some concrete shapes this takes today:

  • A view-only dashboard. A key scoped to job.results/job.results.fetch for one job ID, with no job.exec, job.cancel, or payment access, can be handed to a stakeholder who needs to watch progress and results but should never be able to spend funds or alter the job.

  • A third-party work-function provider. A key scoped to registering vetted work functions, with no visibility into any job’s data or results, lets you build a catalogue of trusted, auditable computations without extending trust any further than that.

  • A worker fleet operator. Registering and monitoring workers (worker, worker.admin) is its own separate capability from anything job- or bank-related.

Note

Fine-grained scoping over bank accounts and financial operations specifically – for example, a key that can only view or spend from one account – is on the roadmap but not yet implemented; today’s operation keys cover job lifecycle, results, compute groups, and worker management.

Keys can be revoked at any time, immediately cutting off whatever access they held – see Getting set up for how to create and manage them from the Portal.

Summary

DCP’s approach to data privacy is architectural: routing controls mean sensitive data and proprietary logic only travel where you send them, and capability-scoped keys mean every credential does exactly what it was issued for and nothing more. Together, they let distributed computation happen at scale without asking you to trade away data governance to get it.