Serializing job data with KVIN

Whenever DCP moves a value across a process boundary – an input set element or argument going out to a worker, a result coming back – it needs to turn that value into a string first. Plain JSON.stringify() can’t represent everything JavaScript can, so DCP falls back to KVIN, a serialization library that’s a functional superset of JSON: anything JSON can encode, KVIN can encode, plus a number of things JSON can’t.

Note

DCP doesn’t always use KVIN. Plain strings, booleans, and ordinary finite numbers are sent as JSON, since it’s cheaper; KVIN is only used for a value once its type or shape needs it. This choice happens automatically – your job doesn’t need to specify which encoding to use.

What KVIN adds over JSON

Compared to plain JSON, KVIN can round-trip:

  • Typed Arrays (Uint8Array, Float64Array, etc.), with a compact representation for large ones

  • Map and Set

  • Sparse arrays, and arrays with extra enumerable properties attached

  • Object graphs with cycles – an object that (directly or indirectly) contains a reference to itself

  • Boxed primitives (new Number(1), new String('x'), new Boolean(true))

  • Regular expressions

  • undefined, NaN, Infinity/-Infinity, and -0 – none of which survive a round trip through plain JSON

  • BigInt

  • Two DCP-specific types: wallet.Address and DcpURL – put an Address or DcpURL instance directly into your input set or arguments, and it comes back out the other side as a real Address/DcpURL instance, not a plain object

What doesn’t survive the trip

These are real constraints on what you can put into a job’s input set, arguments, or results – not theoretical edge cases:

  • Custom class instances, in general, don’t round-trip as that class. KVIN only reconstructs instances of a small, opt-in allow-list of constructors (Address and DcpURL, for DCP job values). An instance of any other custom class either comes back as a plain object (losing its prototype and methods) or, if it has more than one level of custom-class inheritance, fails outright with Cannot serialize property ... - multiple inheritance is not supported. If your work function needs an instance’s behavior, not just its data, reconstruct the instance inside the work function from plain data instead of relying on it surviving the trip intact.

  • Functions aren’t reconstructed as callable functions by default. KVIN can represent a function’s source and own properties, but by default the deserializer turns that back into a boxed string containing the source, not an executable function – deserializing arbitrary code into something callable is, from a security standpoint, equivalent to eval(). This applies to functions used as values inside your data (for example, a callback tucked into an input datum) – it’s separate from the work function itself, which DCP handles through its own, deliberate mechanism.

  • WeakMaps come back empty. Their keys can’t be enumerated, so there’s nothing to serialize; a WeakMap you send is a WeakMap you get back with nothing in it.

  • Symbols lose their identity. Only the description string round-trips; the deserialized Symbol is a new one, not === to the original.

  • Promises can’t be serialized. Resolve a Promise and send its value, not the Promise itself – attempting to serialize one synchronously throws.

  • DOM nodes and other host objects generally aren’t supported. Anything KVIN doesn’t recognize and can’t reduce to a primitive throws Cannot serialize property ... which is a ... rather than silently dropping data.

KVIN’s own documentation additionally flags a few edge cases as known-flaky rather than fully supported: arrays that reference the same object more than once, arrays that mix numeric and non-numeric properties, and sparse arrays in general. These aren’t outright unsupported, but they’re worth avoiding if you have a choice.

Practical guidance

Most job data – numbers, strings, booleans, plain objects and arrays, null – serializes with no special handling and no need to think about any of this. Reach for the richer types above (Typed Arrays, Map/Set, circular references, BigInt) freely; DCP handles them transparently. The thing to actually design around is custom class instances and functions-as-values: if a work function needs to end up with a real, callable object rather than inert data, build it from plain data on the receiving side instead of depending on it surviving serialization as-is.

If the data itself is too large, or too sensitive, to serialize and send through the scheduler at all, see Deploying jobs with remote data – workers can fetch input data, arguments, and even the work function directly from a location you control, bypassing this serialization step (and the scheduler) entirely.