Protocol API
DCP uses cryptographic message-signing techniques to provide message non-repudiation and protection against request forgery and replay attacks.
The Protocol API represents the low-level entity connection and message-passing interfaces in DCP. The Protocol itself can sit on top of HTTP, HTTP/2, WebSockets, TCP, or men in Jeeps with pockets full of USB keys: the actual transport is irrelevant at the API level, except for the protocol field of the URL object used to establish a connection.
The Protocol API is used to establish secureable communications between two entities, implement remote procedure call semantics, and provide the necessary tools for a developer to implement access controls and describe authorizations. Additionally, the API allows developers to create messages bearing secure authorization and access memos which can be transferred between entities via untrusted third parties.
Record of Issue
Date |
Author(s) |
Ver |
Change |
|---|---|---|---|
Jul 29 2026 |
(docs) |
3.0 |
Documents |
Mar 9 2021 |
KC Erb |
2.0 |
Remove |
Oct 19 2020 |
KC Erb |
1.4 |
Update definition of |
Feb 10 2020 |
KC Erb |
1.3 |
|
Jan 28 2020 |
Ryan Rossiter |
1.2 |
|
Jan 14 2020 |
Wes Garland |
1.1 |
|
Jan 08 2020 |
Wes Garland |
1.0 |
Initial release to sprint-planning team |
Intended Audience
This document has been prepared for public dissemination.
Note
dcp-client does not load dcp/protocol-v4. Despite the name, protocol-v4 is a frozen, no-longer-referenced module – dcp-client’s bundle loader registers dcp/protocol (dcp/src/protocol/) as the protocol official-API module, and that is the module this document describes. It self-reports Connection.versionInfo.implementation 5.3.0, compatible (Connection.versionInfo.compatibility) with ^5.2.0.
Glossary
entity: any component of a Distributed Computer system (e.g. Scheduler, Bank, Client) which communicate via DCP.
protected resource: A unit of data associated with an Ethereum key-pair (private key, address) which may only be accessed by entities that know the private key.
resource address: the address of a protected resource, such as a Bank Account.
resource owner: any bearer of a resource’s private key.
guardian: An entity mediating access to a protected resource; e.g. the Bank acts as a guardian for bank accounts.
authorize: A resource owner authorizes a resource guardian to use the resource in conjunction with an operation.
initiator: the entity which initiates the connection (“client” in traditional client-server topology).
target: the entity to which the initiator connected.
peer: the initiator’s target, or the target’s initiator.
message: An instance of
Connection.Message(or one of its subclasses: Request, Response, Notification, Batch).message originator: the bearer of the private key that was used to sign the message.
request message: A message, generally sent from initiator to target, containing instructions (e.g. withdraw $7 from bank account 123456).
response message: A message, generally sent from target to initiator, containing the response to, or result of, a request message.
notification message: A fire-and-forget message that, unlike a request, does not receive a response – only an ack.
ack: A packet (not a
Message) which acknowledges receipt of a request, response, notification, or batch by either peer, and carries the next nonce the sender should use.authenticate: Messages are authenticated by connections to ensure they came from the peer. Payloads that authorize the use of protected resources are authenticated to ensure they were issued by a resource owner.
Overview
This high-level protocol is designed to operate at OSI Level 5 (Session Layer) or higher. It enables bi-directional communication between peers with stateful sessions, with a client/server-flavoured startup mechanism. The current implementation direction is to implement the protocol on top of socket.io, however this is not a strict requirement and should be treated as an implementation detail.
Connections are established by having the initiator contact the target. This allows us, in particular, to traverse NAT without worrying about STUN, and also allows us to operate atop Level 7 protocols such as HTTP.
Protocol connections are stateful, but not necessarily connected. Each Connection has a session id which is used to identify the connection at the DCP level. It is possible that the underlying protocol or connection could change during a DCP session.
Future directions which should be possible with this message exchange format and API:
Improvements from v4 in this spec:
connections now have control over transmission verification instead of relying on underlying transport
users must create transport and connection pools on a target to decouple the two and allow the underlying transport to disconnect without affecting the dcp session
the addition of
stateto connections allows finer control over startup, shutdown, and expected behavior when an established connection loses its transport
Improvements from v3 in this spec:
better future-proofing
data layer encapsulation separate from payload
can now have both bank account and identity keys in one message
can specify messages which can be handed off securely to a third-party
with the entire payload in the signed portion of the message (old version did not sign against URL), we close a certain class of potential security vulnerability.
uniform message format allows tighter security controls and error management
nonce is no longer optional, closing possible CSRF and replay attack surface
all messages have identity
DCP Sessions are identified by the dcpsid property which is present in every message (except the initial connection messages where it is established). There is no requirement to send all messages for a given DCP Session on the same underlying transport.
Messages are exchanged in the form of requests and responses. Either peer can send a request; all requests require a response. Each peer can have, at most, one open request at a time. A peer may also send a notification, which (like a request) carries a payload but (unlike a request) is not answered with a response – only with an ack.
Security protocols are consistent and invariant across message types:
All message transmissions (requests, responses, and acks) are signed with the sending peer’s identity key.
Authorization is part of the payload; the resource owner identifies which peer (or peers) may act on the request.
No message-related information is contained outside of the (signed) payload; specifically, when authorizing a guardian to use a protected resource, the guardian’s address is only in the payload.
All response and request messages have a nonce and all acks have an ack-token, which is used to protect against cross-site request forgeries and replay attacks.
Initial request id is specified by initiator
Initial nonce is specified by target in an ack to the initial request
Initial response has the same id as the request it is responding to
Initial response is ack’d as well, providing a nonce to the target
All requests and responses include the nonce most recently received on that connection
All requests and responses include an ack token which must be present on the ack for that message
Request ids may only be used once
DCP Session ID is specified during the response to the initial request and never changes for a given session
The identity key for a given session never changes
Multiple messages can be sent in a single ‘batch’ message; this is supported intrinsically in the protocol, and message batching is handled automatically by virtue of the JavaScript event loop.
Requests can be created for secure transmission through a third party; for example, a Client can send a message to a Scheduler which gives said Scheduler (and only that Scheduler) permission to access a particular account on a specific Bank.
Data Representation
Network Traffic
All data transmitted by the DCP protocol ‘on the wire’ has been serialized with the JavaScript-native JSON code. There is no requirement that objects and values sent on the wire have have a 1:1 correspondence with the API layer types.
Hexadecimal Values
Hexadecimal values (such as Ethereum addresses) sent over the network as strings should have the 0x prefix removed. If it is present upon receipt, it should be ignored.
Ethereum Addresses
Ethereum addresses should be sent over the network in checksum format. Addresses which appear to be in checksum format, but are not valid addresses, should trigger rejections at the point where the address is passed to the wallet.Address constructor (the constructor will throw).
Time
All time values in the DCP protocol are represented as seconds since the epoch; in most cases, fractional seconds are supported as floating-point numbers. When converting between fractional and whole seconds, values should be truncated and not rounded.
Classes
Message
A Message object represents a message which can be sent between DCP entities. There are four types of Connection.Message:
Request
Response
Notification
Batch
Acks are a related but distinct kind of packet – see Connection.Ack, below – and are not instances of Connection.Message.
Connection
A Connection object represents a connection to another DCP entity. A DCP connection may ‘live’ longer than the underlying transport’s connection, and the underlying transport connection (or, indeed, transport) may change throughout the life of the DCP connection.
DCP connections are uniquely identified by the DCP Session ID, specified by the dcpsid property, present in every message body. This session id is negotiated during connection, with the initiator and target each providing half of the string.
Connection objects inherit from EventEmitter.
new Connection (targetDescriptor, optional options)
This constructor returns an object which represents a connection between DCP entities. note - the entities will not actually be connected until a call to this.connect() or this.send().
targetDescriptor is mandatory, and must be an object with a location property; the constructor throws if either is missing. A bare string/URL/DcpURL is not accepted – pass { location: target }. The identity is supplied as options.identity (below), not as a separate positional argument.
targetDescriptor: {object} - describes the entity to connect to (commonly adcpConfigfragment). Its properties are:location: {URLorDcpURL} - mandatory - the target’s address, valid from the internet.friendLocation: {DcpURL} - optional - an address for the target that is only valid from an intranet; when bothlocationandfriendLocationare given, the best one is chosen by examining IP addresses.identity: {object} - optional - an object with anaddressproperty that is a Promise resolving to the target’swallet.Address; when the resolved address matches what’s actually seen on connect, it overrides the initiator’s identity cache (seeclearIdentityCache, below) unlessoptions.strictis truthy.
options: An object specifying arbitrary options for configuring a connection. All are optional; shown values are the hardcoded defaults, which are overridden (in order of increasing precedence) bydcpConfig.dcp.connectionOptions.default, any origin/hostname-specific entries indcpConfig.dcp.connectionOptions, and finally this argument itself.identity: {instance ofwallet.Keystore,wallet.Address, or a Promise resolving to either} - the identity keystore used to sign messages; used for non-repudiation. When omitted, defaults to a promise forwallet.getId()(see: Wallet API). AnAddressis used to construct a passphrase-lessKeystorefor identification without signing capability.connectTimeout(default90): seconds an initiator will try to connect to a target before giving up.reconnectTimeout(default600): seconds an initiator will try to re-establish a disconnected session before giving up.closeTimeout(default7): seconds to wait for aclosemessage to be delivered before forcing the underlying connection closed.maxRTT(default120): seconds to wait for an ack before assuming the transport is dead and forcing a reconnect.validitySlopTime(default600): seconds of tolerance given to a received message’s validity window (see Validity, below) to account for clock skew between peers.identityUnlockTimeout(default300): number of (floating-point) seconds to leave the identity keystore unlocked between invocations ofConnection.send.allowBatch(defaulttrue): {boolean} - if false, limits each transmission to one message.maxMessagesPerBatch(default20): tuning parameter for batch size. If less than 1, equivalent toallowBatch: false.targetBatchSize(default100e6): once a batch’s accumulated payload size (in JSON-stringified characters) reaches this figure, no further messages are added to it.maxBatchSize(default200e6): a hard ceiling on a batch’s accumulated payload size; a message that would push the batch over this limit is left for the next batch instead.batchWaitTime(default0.03): seconds to wait, Nagle’s-algorithm style, for other messages to arrive and join the current batch before transmitting.connectBackoff(default{ mult: 0.5, base: 2, max: 30 }): backoff curve (seedcp/utils’sBackoffclass) used to space out repeated connection attempts to the same target.transports(default['socketio']): array of transport module names to try, in order, when establishing a connection.ttl: A number or an object describing the time-to-live for thevalidityproperty of message payloads. If a number N is specified, it will be treated as{default: N}. The units are floating-point seconds.min(default15): the minimum ttl allowable (request receiver only)max(default600): the maximum ttl allowable (request receiver only)default(default120): the ttl to use when not specified (request receiver or sender)
Note
The option that is documented (and named in the hardcoded defaults) as maxTransmitAttempts (default 5) is intended to cap the number of times an unacknowledged on-deck message is retransmitted before the connection gives up and shuts down with EINVAL. The retry-counting code, however, reads this.options.maxTransmissionAttempts – a different property name – which is always undefined, so the tries++ > undefined comparison is always false and this safety limit never actually fires. In practice a message that can never be acknowledged will be retried indefinitely (subject to reconnectTimeout/closeTimeout elsewhere) rather than failing fast after 5 attempts.
Note
There is no ntp connection option. Whether an entity trusts its own clock (Connection.hasNtp) is instead determined automatically by role: targets are assumed to have NTP available and use their system clock directly; initiators calculate the current time from the most recently received Response’s timestamp instead (see Connection.currentTime(), below).
Connection State Machine
Connection.state reflects the connection’s lifecycle, and can be one of the following values:
state |
|
|---|---|
initial |
the state a connection starts in, before any connection attempt has been made. |
connecting |
negotiating a |
established |
fired immediately after connection establishment ( |
disconnected |
the session is still valid, but the underlying transport is not currently connected. Initiators try to reconnect automatically; targets wait for the initiator to reach out again on a new transport. |
closing |
|
closed |
the connection is finished and cannot be used again. |
Connection.identity
The identity keystore or undefined. This property is only guaranteed to be defined after the connection is established.
Connection.peerAddress
Undefined until connection; then it becomes an instance of wallet.Address representing the public address of the connected peer.
Connection.dcpsid
Undefined until connection; then it becomes a string representing a unique DCP session.
Connection.role
One of 'unspecified', 'initiator', or 'target'. 'unspecified' until the first call to connect() (which sets it to 'initiator') or accept() (which sets it to 'target').
Connection.versionInfo (static)
{ implementation, compatibility } – the protocol version this build of the module implements ('5.3.0'), and the semver range of peer versions it is willing to talk to ('^5.2.0'). There is no Connection.VERSION string.
Connection.Message
A constructor with Protocol.Message on its prototype chain; used to construct batch, request, response, and ack messages for transmission on this connection.
Connection.close()
This method is synchronous and does not return a value; there is no promise to await. It sends a close message to the peer, on this pass of the event loop or later, and immediately transitions the connection to the closing state (see Connection State Machine, above). Once the underlying connection has been confirmed closed, the connection transitions to closed and an end event is emitted, followed by a close event once everything has been finalized.
Any messages that were queued before calling close() will be delivered before sending the close operation. If the close message is not sent in a timely manner, the connection will be forcefully closed by rejecting all pending message promises and then closing the underlying connection. (Timeout is configured by connectionOptions.closeTimeout.)
Any subsequent calls to Connection.send() on a closed connection will result in an error due to this invalidated session.
See: Reserved Operations section, close; Events section, end and close
async Connection.connect()
This method, when invoked by an initiator,
establishes the connection between the two entities. Connection establishment means:
Establish underlying transport protocol connection (when applicable, e.g. an HTTP or web socket connection)
Establish version compatibility (
body.payload.data.version)Exchange initial nonces (
body.nonce,body.id)Establish
dcpsid(DCP Session ID)set
this.peerAddressto the remote peer’s public address
resolves after sending
operation: 'connect'message and receiving the responserejects with Error if the connection cannot be established, or if connection was already established
If target determines that the connection cannot be established due to a protocol version mismatch, the target will respond with a message whose body has the following properties:
success = ‘false’
type = ‘protocol’
code = ‘EVERSION’
message = < semver expression of acceptable version >
If the initiator determines that the target’s protocol version is incompatible with its own, it will also reject with Error.code = 'EVERSION' – both directions of this check use the same error code.
Implementations reject with Error.code = 'EADDRCHANGE' if the connection address has changed for that URL since the last time we connected to that URL. (Analogue: ssh fingerprint change) This check is performed against the process-wide identity cache (see clearIdentityCache, below), not on a per-Connection basis.
See: Reserved Operations section, connect
async Connection.keepalive()
This method sends a keepalive to the peer, and resolves when the response has been received.
See: Reserved Operations section, keepalive
Connection.request(operation, data)
Sugar for connection.send(new connection.Request({ operation, data })). Returns the same promise Connection.send() would.
Connection.notify(data)
Sugar for connection.send(new connection.Notification(data)). Returns the same promise Connection.send() would – see Connection.Notification, below, for what that promise resolves to.
Connection.hasPendingTx()
Returns true if there is a message queued for transmission or awaiting acknowledgement (an on-deck message), false otherwise.
Connection.isQuiescent()
Returns true if there are no dialogues (sent messages awaiting a Response or Ack) outstanding on this connection.
async Connection.send(message)
This method sends a message to the connection peer. It takes exactly one argument – calling it with zero or more than one argument throws an assertion error. If the connection has not yet been established, this routine will first invoke (but, notably, does not await) this.connect().
resolves with the Response if
messageis a Request, resolves once the peer’s ack formessagehas arrived for any other message type, or rejects with Errordoes not mutate passed message, except for
message.idif
messageis not already an instance ofthis.Request,this.Response, orthis.Notification,we construct a new
this.Requestusing passed object as the constructor argument
assign message to this new Request
generates unique
message.idto associate this transmission of the message with its response.while the connection is
closing, only Response messages (and a Request whose operation isclose) may be sent; anything else throws immediately.
Return value
Connection.send() always returns a Promise.
Response Messages
The promise is resolved or rejected as soon as the response message has been delivered to the peer via the underlying transport as indicated by an ack.
When the promise is resolved, its value is true. If the promise is rejected, it will be rejected with an instance of Error.
Request Messages
This promise is resolved with a response message when the peer sends a Response with the same id as this Request. The success property of this response will be true and the payload property will hold the corresponding data (if any).
If the peer responds with success false, the payload property will instead be an instance of connection.ErrorPayload.
If the API consumer needs to differentiate between error payloads which were instances of Error and/or its superclasses at the peer end, the API consumer will need to inspect type
nameproperty.
The promise will be rejected if there is some underlying problem with the local machine, software bugs, network, etc. so that the connection is unable to send messages (for example when the closeTimeout is reached and there are still unsent messages in the queue).
The promise is rejected with a rejection object that is an instance of Error.
Batch Messages
Batch messages are used internally when more then one message is queued to be sent. It carries with it one nonce for the whole batch, and each message is parsed and handled normally on the receiving end.
Upon receipt of a batch, the receiver immediately sends an ack to give the sender a new nonce.
connection.ErrorPayload
This class is used to create and represent payloads which indicate unexpected errors (such as a version error or a file that does not exist), and not application level errors (such as a bank account which does not have enough money to deploy a job).
There are two related but distinct things named ErrorPayload:
protocol.ErrorPayload– a module-level export, a bare marker constructor with an empty body, used only so thatinstanceof protocol.ErrorPayloadworks without reference to any particular Connection.connection.ErrorPayload– a per-connection constructor (documented below) produced by a factory function, whose instances inherit fromprotocol.ErrorPayload.
There is one constructor signature, plus a static .from() for rehydrating a received payload.
new connection.ErrorPayload(error, optional code, optional Ctor)
error: {string|Error} - if a string, it is wrapped as
new Ctor(error)(with.codeset tocode, if given) before proceeding; otherwise, used directly.code: {string} - optional - assigned to
this.code, overriding anycodealready present onerror.Ctor: {function} - optional, default
Error- the constructor used to wraperrorwhen it is a string. Ignored whenerroris not a string.
Every own (and inherited-but-own-enumerable-after-copy) property of error – name, message, stack, and anything else present, including getters and bound methods – is copied onto the new ErrorPayload instance as an enumerable, writable property, making the result reliably serializable via JSON.stringify(). The instance also gets an origin property: the wallet.Address of the identity keystore associated with the Connection that created it.
Note
There is no special handling for non-Error arguments (e.g. numbers, plain objects passed as error) – passing something other than a string or an Error-like object copies whatever own properties it happens to have, rather than raising a TypeError.
connection.ErrorPayload.from(errorLikeObject)
Static method. Turns a plain object (i.e. the result of JSON.parse() on a received payload) into an instance of ErrorPayload, additionally rehydrating errorLikeObject.origin (if present) into a wallet.Address instance rather than leaving it as a string. This is what Connection.Response.from() uses internally to reconstitute the payload of an unsuccessful response received from the peer.
Connection.Message
A Connection.Message object represents a message which can be sent between DCP entities on a given connection. Internally, all message types (Request, Response, Notification, Batch) share a common DcpMessage base class (exposed for instanceof checks only – it is never constructed directly and is not itself part of Connection). It is not constructed directly; use Connection.Request, Connection.Response, Connection.Notification, or Connection.Batch instead.
Connection.Message.connection
This property is a reference to the connection instance of which this constructor is a property.
async Connection.Message.sign(ackToken)
Signs a message using the identity keystore supplied during Connection instantiation, using its makeSignedMessage() method (see: Wallet API).
ackToken: the token the peer must include on its ack for this message, to be included in the signed payload.
The nonce included in the signed payload is not a parameter – it is taken automatically from connection.lastRxNonce (the last nonce received from the peer) and consumed (set to false) as soon as it’s read, which is how the protocol enforces that nonces are used at most once.
This function returns a promise which resolves to a string which is an Ethereum signed message.
Connection.Message.toNetwork()
Returns the plain-object “network form” of the message ({ id, type, success, payload, auth, dcpsid, time }) that sign() seals for transmission. Not normally called directly by API consumers.
async Connection.Message.send()
equivalent to this.connection.send(this)
Connection.Request
This class, which inherits from Connection.Message, represents a request message that may be sent to the connection peer.
Request Messages have the following properties:
id: unique string. The API will provide one immediately before transmission. A given entity will never process two messages with the same id and overlapping validity time.
payload: An object which represents the payload which is transmitted to the connection peer. If specified in the constructor, its properties are used to initialize the message payload.
operation: string describing the operation; has meaning to the peer.
data - undefined or an arbitrary value which can be serialized to JSON which represents the arguments to the operation.
validity: The
validityproperty of a Request payload is an object which can be fully (or partially) populated by the API consumer; they will be fully populated byRequest.send()as needed.stamp: A string which is unique enough to prevent us from accidentally creating indifferentiable unique messages, possibly on different connections, even if they are otherwise identical and were created at exactly the same time.
Suggested algorithm:
md5sum(request.id + (request.dcpsid || Date.now() + Math.random()))
time: the current time, according to the target’s clock (or NTP), expressed an integer number of seconds which have elapsed since the epoch (C
time_t)ttl: optional - the number of (floating point) seconds after which the message expires. If this is not specified, the guardian (and potentially any intermediary machines) will use their own default value.
allow: an array identifying the resource guardian allowed to perform the operation on a resource when the message is received from a given accessor. Each element in the array has the shape
{ resource: address, guardian: address, accessor: address }(See Connection.Request.authorize).
auth: This property is an object that relates to payload.allow. It authorizes a guardian to perform the operation using one or more protected resources (See Connection.Request.authorize). It contains key-value pairs of
<resource address>: <payload signature>.
new Connection.Request()
form 1: new Connection.Request() : A new Request Message is constructed
form 2: new Connection.Request(payload {object}, optional resourceKeystore) : A new Request Message is constructed; the passed object is used to specify the message payload.
form 3: new Connection.Request(operation {string}, data) : A new Request Message is constructed; the passed string is used to specify the message payload operation property; the (optional) second argument is used as the payload data property.
None of these forms take a resourceKeystore argument – authorize() (below) must always be called explicitly.
async Connection.Request.respond(…)
This method is a convenience method which is equivalent to
(new Connection.Response(this, ...)).send()
async Connection.Request.send()
Connection.Request does not override send() – this is the plain Connection.Message.send() described above, equivalent to this.connection.send(this). There is no resourceKeystore argument to send() – call and await this.authorize(resourceKeystore) (below) before send(); see Send Message using protected resource, under Sample Code.
Before the message is actually sealed for transmission, any authorizations memoized by prior calls to authorize() are applied: the signature for this.payload is calculated via resourceKeystore.makeSignature() (see: Wallet API) for each memoized keystore, updating the auth property to have a (key, value) pair of (resourceKeystore.address, signature). Redundant memos for the same resource are collapsed into a single signing operation.
Connection.Response
This class represents Response messages on this connection, and inherits from Connection.Message.
id: same id as Request message that precipitated this response
success: true | false (boolean)
if success is false, this means we could not perform the request for whatever reason, with more details in the payload property.
payload: when success is true, this property can carry arbitrary information, and need not be specified at all. When success is
false, this property will be an ErrorPayload object.
new Connection.Response()
form 1: new Connection.Response() : A new Response Message is constructed.
form 2: new Connection.Response(request, error {instance of Error | connection.ErrorPayload}) : A new Response Message is constructed;
the passed request is used to determine the request id
this.successis falsethis.payloadbecomesnew connection.ErrorPayload(error)
form 3: new Connection.Response(request, payload) : A new Response Message is constructed;
the passed request is used to determine the request id
this.successis truethe passed data is used to specify
this.payload
Connection.Notification
This class, which inherits from Connection.Message, represents a fire-and-forget notification sent to the connection peer. It shares its authorization machinery (authorize, doesAuthorize, validityError) with Connection.Request, but is never answered with a Response – only with an ack.
payload:
{ data, allow, validity }– same shape as a Request’s payload, minusoperation.
new Connection.Notification(data)
Constructs a new Notification whose payload.data is the passed value.
async Connection.Notification.send()
Equivalent to this.connection.send(this) (or, more conveniently, connection.notify(data)). The returned promise resolves once the peer has ack’d the notification – there is no response payload to resolve with.
Connection.Batch
This class represents Batch messages on this connection, and inherits from Connection.Message.
Future versions of this protocol will also have Batch messages that contain Batch messages. The current intention is that Batch messages will only be used internally by the protocol itself (indeed, attempting to receive a Batch containing another Batch is treated as a protocol error and closes the connection).
Connection.Ack
An Ack is not a Connection.Message – it does not inherit from the DcpMessage base class, has no id, and is never delivered to the request/notification events. It is constructed and transmitted entirely inside Connection’s own bookkeeping (Connection.prototype.sendAck), not by API consumers.
new Ack(messageBody, nextNonce)–messageBodyis the network-form body of theMessagebeing acknowledged (from which.messageIdand.tokenare copied);nextNonceis the nonce the peer must use on its next transmission..toNetwork()returns{ type: 'ack', dcpsid, token, nonce, messageId }.
Because acks are responsible for carrying the next nonce to a peer, they use a different unique identifier, the ackToken, to prove their validity. Thus requests/responses/notifications carry ack tokens and acks carry nonces.
Note
The messageId property is called out in the source as “not part of the 5.3 spec; here for 5.2 compat and DCP-3957” – it exists to work around a backward-compatibility bug (DCP-3957) when talking to older (5.2) peers during reconnect, and may be removed in a future revision.
Connection.currentTime()
This routine returns the current time for the purposes of populating the Request message payload.validity.time property.
If the Connection is a target, or no responses have ever been received (Connection.clockInfo unset), the local clock is used. Otherwise, the time is calculated based on the most-recently-received Response.time and a delta between “now” and when that message was received. This delta should not be calculated based on the system clock, as this could jump mid-session if the system administrator adjusts the system clock. Instead, the calculation should be based on something like performance.now() on the browser or require('perf_hooks').performance.nodeTiming.duration on NodeJS.
This routine returns the integer number of seconds which have elapsed since the epoch (C time_t).
Transport
There is no base Transport class or module. Transports are plain, independently-implemented modules, one per name in connectionOptions.transports (currently only 'socketio' ships), loaded on demand with a plain require('./transports/' + transportName) and duck-typed against the contract below.
A transport module is required to export:
TransportClass: {function} - constructor, called asnew TransportClass(targetDescriptor, options)on the initiator side (optionsbeing the named sub-object ofconnectionOptionsfor this transport, e.g.connectionOptions.socketio). Instances extendEventEmitterand must provide:.name: {string} - the transport’s name (matches its module name, e.g.'socketio')..reliable: {boolean} - when true,Connectionassumes the transport itself guarantees in-order, at-most-once delivery, and skips its own retransmission of an unacknowledged on-deck message over the same transport instance..send(sealedMessage): transmits a sealed (signed, JSON-stringified) message string to the peer..close(): tears down the underlying transport-level connection.events:
connect(transport-level connection established),connect-failed(argument: Error; connection attempt failed, but other transports or later attempts may still succeed),error(argument: Error; treated as fatal for this connection attempt),end/close(transport-level connection lost),message(argument: sealed message string received from the peer).
Listener: {function} - constructor used only on the target side, for accepting incoming transport-level connections (seeTarget, referenced under Sample Code, below); not relevant to initiator-only code.
Static Methods
clearIdentityCache(identity | true)
This method clears the identity cache that is used by Connection.connect() to track (URL, identity) pairs.
form 1: argument is instance of wallet.Keystore : cache entry corresponding to argument.address is cleared
form 2: argument is instance of wallet.Address : cache entry corresponding to argument is cleared
form 3: argument is boolean value true
: entire cache is cleared
Other exports
A few additional properties live on the protocol module itself, mostly for dcp-client’s own bootstrap process rather than everyday API consumers:
protocol.ErrorPayload– the connection-independent base class described underconnection.ErrorPayload, above; useful for aninstanceofcheck when you don’t have a specificConnectionhandy.protocol.version–{ provides: '5.0.0', api: '5.0.0' }; the scheduler-compatibility version this build advertises. Distinct from (and, confusingly, numerically lower than)Connection.versionInfo, above, which describes the wire protocol itself.protocol.getGlobalIdentityCache()– returns the singletonIdentityCacheinstance that backsclearIdentityCache(). Its instance method.forgetIdentity(baseHref)removes a single cached(baseHref, address)pair without clearing the whole cache; there is no top-levelprotocol.forgetIdentity()wrapper for it.protocol.fetchSchedulerConfig(location),protocol.setSchedulerConfigLocation_fromScript(scriptElement),protocol.getSchedulerConfigLocation()– used once, internally, duringdcp-clientstartup to locate and fetchdcpConfig; not intended for use by application code.
Events
Connection
Connection objects inherit from EventEmitter; all of the events below fire with this set to the Connection instance.
ready
Fired once, the first time the identity keystore supplied to (or generated by) the constructor has resolved and unlocked. No argument.
session
Fired once a dcpsid has been negotiated with the peer, whether during initial connection or a reconnect. Receives the new dcpsid as its argument.
connect
A UI hint meaning “internet available”; fired when a transport becomes ready to carry traffic. Receives the connection’s URL as its argument.
disconnect
A UI hint meaning “internet not available”; fired when the underlying transport disconnects. Receives the connection’s URL as its argument.
connectionProgress
A UI hint describing the state of an in-progress transport-level connection attempt. Receives an object as its argument, one of:
{ begin: transportName }– tryingtransportNamenext{ fail: transportName, code, httpStatus? }–transportNamefailed, with a reason code (and HTTP status, if applicable){ fail: transportName, error }–transportNamefailed with an unexpected error
request
The request event is emitted by Connection objects when the connected peer sends a Request message (other than one of the reserved operations, which are intercepted beforehand – see Reserved Operations, below), or when the local entity extracts such a Request message that was encapsulated in a Batch message.
The event handler will receive as its argument the Request object, if and only if the Request passes the steps outlined in Message Authorization.
notification
The notification event is emitted by Connection objects when the connected peer sends a Notification message, or when the local entity extracts one that was encapsulated in a Batch message, mirroring request above (including passing Message Authorization first).
beforeSend
Fired immediately before an on-deck message is turned into its network form for signing/transmission – i.e. after batching decisions have been made, but before Message.sign() is called. Receives the Message object as its argument.
send
Fired every time a message is hand off to the transport for transmission; this does not include the contents of Batch messages. (Specifically, a batch message with 10 requests in it would trigger send once but request ten times, once each peer processes it.) This does not indicate that the peer has received the message – see sent, below.
The event handler is invoked with the Message object as its only argument.
sent
Fired once the peer has acknowledged receipt of a previously-sent message (i.e. once its ack has arrived). For a Batch message, this fires once for the Batch itself and once more for each message it contained.
The event handler is invoked with the Message object as its only argument.
ack
Fired whenever this connection transmits an Ack (i.e. immediately after receiving and acknowledging any incoming Message). Receives the Ack object – see Connection.Ack, above – as its argument. Note the naming symmetry with sent: sent fires for messages we sent that were acknowledged by the peer; ack fires for acknowledgements we send for messages the peer sent us.
recvResponse
Fired when a Response message is received from the peer, before it is matched up with its originating Request. Receives the Response object as its argument.
error
Fired when an error happens that would otherwise go uncaught, most commonly when a request event handler throws. Receives the Error as its argument.
end
Fired once, as soon as the connection begins closing (whether via close(), a close request from the peer, or an unrecoverable error) and transitions to the closing state. No argument. At this point Connection.send() will only accept Response messages, not new Requests.
close
Fired once, after the connection has been fully finalized: the underlying transport is closed, and the dcpsid is invalidated and will never be valid again. Receives the Error that precipitated the close as its argument (this is a generic 'connection closed' Error for a normal, API-directed close).
Message Transmission & Receipt
DCP Messages are encapsulated within Ethereum messages for wireline transmission; these are signed with the originator’s identity key for non-repudiation.
Ethereum messages
Every Ethereum message is a JSON-stringified JavaScript object with the following properties:
owner: the public address of the message sender (i.e. identity address)
signature: a checksum of the message body, generated using the message sender’s private key (identity key).
body: an object containing DCP-related properties, such as
type,payload,id,dcpsid,auth, etc.
Message types are differentiated during transmission with a type property in the message body, however at the API level, this property is not exposed and the instanceof operator should be used to determine message types if the need arises.
Ethereum messages are created by the Connection.Message.sign() method, which is invoked by Connection.send().
Message Grammar
This grammar describes JavaScript objects which are serialized with the usual JSON semantics for transmission.
Grammar Syntax
Syntax |
meaning |
|
|---|---|---|
A → B | C |
“A is a B or a C” |
|
{} |
Object containing properties as defined by this syntax between braces: |
|
a, b, c |
properties a, b, c |
|
a: ‘abc’ |
property a has string value ‘abc’ |
|
b* |
property b is optional |
|
… |
any number of arbitrary properties |
|
[ things ] |
an array of things |
|
thing+ |
One or more things |
|
thing* |
Zero or N things, where N is positive, whole, and finite. |
|
‘abc’ |
the string literal, |
|
integer |
the set of all integers in the range |
|
string |
any Unicode String representable by the current engine; a minimum of 128 × 1024 × 1024 code points must be supported by a supported implementation. |
DCP Message Exchange Grammar
DCP Messages are exchanged as Ethereum signed messages, which are objects serialized with JSON before transmission.
signed-message → { owner, signature, body }
body → request
| response
| notification
| batch
request → { type: 'request', id, payload, auth*, dcpsid, nonce<last>, ackToken }
response → { type: 'response', id<request>, time, success: boolean, payload, dcpsid, nonce, ackToken }
notification → { type: 'notification', id, payload, auth*, dcpsid, nonce<last>, ackToken }
batch → { type: 'batch', id, payload: [ (request | response | notification)+ ], dcpsid, nonce<last>, ackToken }
payload → { operation, validity, allow*, ... } /* request */
| { data, validity, allow* } /* notification -- no operation */
| anything /* response */
allow → [ { accessor: address, guardian: address, resource: address }* ]
auth → { <resource-address>: signature }
time → integer
ttl → integer
stamp → string
validity → { time, ttl, stamp }
boolean → true
| false
ack → { type: 'ack', dcpsid, token, nonce<next>, messageId }
token → string
owneris the identity address (public key) of the entity sending the messagesignatureis a checksum of the message body or payload, calculated with the identity private keybodyis the body of the messageidis a unique per-transmission message iddcpsidis the session id established during connection startupid<request>is theidof the request to which we’re respondingnonce<last>is thenoncesupplied by the most recently received Messagenonce<next>is thenoncethe peer must use on its next transmission (only meaningful on an ack, which is what carries it)messageIdis the id of the message being ack’dtokenis like a nonce but for acksackTokenis a fresh, per-transmission token; the peer must echo it back astokenin its Ack for this messageoperationis the operation to perform
Message Transmission Implementation Details
Message signing has significant overhead, as does establishing connections in the underlying protocol. For this reason, we employ transparent opportunistic batching. The actual mechanism, in the current implementation, is a single FIFO (Connection.txQueue) plus a one-message-at-a-time “on deck” slot, not a separate pending/transmit split:
Connection.send(message)pushesmessageontotxQueue, registers it in the dialogue registry (which is whatsend()’s returned Promise resolves/rejects through), and callsrequestTxQueueService(). If the connection hasn’t been contacted yet,connect()is triggered but not awaited here – ordering is preserved because nothing else can jump the queue beforeconnect()’s ownconnectRequest is enqueued.requestTxQueueService()waits for whatever preconditions currently block progress (identity resolved, the previous on-deck message’s ack received, and – before a session exists – the top-of-queue message being a permitted pre-session operation) and then callsserviceTxQueue().serviceTxQueue()is the pump:If something is already on deck, it’s busy; service is rescheduled shortly.
Otherwise, it shifts one message off
txQueueintoonDeck.If batching is allowed and the connection is established (or closing),
combineToBatch()waits briefly (batchWaitTime, or a single tick if the on-deck message is a Response) for more messages to arrive ontxQueue, then folds as many of them as fit (subject tomaxMessagesPerBatch/targetBatchSize/maxBatchSize) into a singleBatchthat replaces the on-deck message.The on-deck message is signed (
Message.sign(ackToken), which is also where the nonce is consumed and any deferredauthorize()placeholders are resolved) and, if a transport is currently attached, transmitted.
A
maxRTTtimer starts on transmission; if no matching Ack arrives before it fires, the transport is treated as dead and closed, which triggers reconnect logic; on reconnect, the same on-deck message (or the last unacknowledged Ack) is retransmitted rather than re-queued.Once the matching Ack arrives (see Message Receipt, below), the on-deck slot clears, the
sentevent fires, and the queue is serviced again.
Message Receipt Implementation Details
Connection.prototype.receive handles every incoming signed message, in this order:
Parse the sealed message if not already parsed; determine the peer’s address from
owner(the first address seen on a connection becomes authoritative – a later message claiming a different owner is rejected).Verify the signature against that peer address. If it cannot be verified, the connection is shut down (it cannot be resurrected) and the error propagates via the
errorevent / a rejected dialogue promise – there is no separateconsole.warn-then-throw step; the shutdown path handles logging.If the message type is
ack, it’s handled separately (seeConnection.Ack, above) and receipt processing stops here.If this message’s
idexactly matches the last message received, it’s a duplicate (the peer’s Ack for it was probably lost); the last Ack is retransmitted and the message is not reprocessed.Otherwise, an Ack is sent immediately, before any further processing of the message’s contents.
Before a session exists, only
requestmessages with operationconnect/close, andresponsemessages, are accepted; anotificationorbatchat this point is a protocol error. Once a session exists,dcpsidandnonceare checked (see Message Authentication Algorithm, above), andbatchmessages are unwrapped one level (a batch containing a batch is rejected) with each inner message re-dispatched by type.Dispatch by type:
request→ validity-checked, then therequestevent (or a reserved-operation handler);notification→ validity-checked, then thenotificationevent (no response is ever sent for a notification, only the Ack already sent above);response→ matched against an outstanding Request in the dialogue registry (a Response that matches nothing shuts the connection down withEINVAL), then therecvResponseevent, then the original Request’s promise is resolved.
Message Authentication Algorithm
Message authentication happens transparently and automatically at the protocol. No unauthenticated requests will ever be presented to the application layer under any circumstances.
In the current implementation, the checks run in this order (Connection.prototype.receive):
check the signature against the peer’s address (the first peer address seen on a connection is authoritative; a later message from a different address is rejected)
check the message isn’t an exact-duplicate retransmission (by
id) of the last message received – if it is, the last ack is simply retransmitted, without re-processingonce a session exists (
dcpsidset): check the message’sdcpsidmatches, and itsnoncematches the last one this connection transmitted
This applies uniformly to every message type (request, response, notification, and each message inside a batch) – there is no message type that skips the dcpsid/nonce check.
Used predominately for:
non-repudiation
to prevent cross-site request forgeries (CSRF)
to prevent replay attacks
Request Authentication Algorithm
Request authentication is used to prevent unauthorized access to protected resources. While it happens strictly at the application layer, the Protocol API provides the mechanisms for making this consistent and easy.
This authentication is based on the following principles:
every protected resource has a unique Ethereum address
only the entities that are authorized to use the resources know the corresponding private keys
the guardian knows the public addresses of all protects resources that it protects
the guardian can remember all payload validity stamps that it receives from anyone, for their entire validity period.
100% of the information required to grant access to the resource is contained within the
payloadproperty of a Request
Allow and Auth
The allow and auth fields work together to document what entity is allowed to make use of the protected resource (as described by the payload) and to document that authorization with a signature generated for the payload and with the private key corresponding to the public address identiying the protected resource.
The
allowproperty of the Requestpayloadcontains an array with entries in the form{ resource, guardian, accessor }which describes whichresourceon whichguardianis allowed to be modified by whichaccessor.The
authproperty of the Request contains a key/value pair lookup table of resource addresses and signaturesThese signatures were generated with the resource addresses’ corresponding private keys
This gives the resource guardian the confidence that the entity with control of the protected resource authorized the entity making the request to make it, even if those two entities are not the same entity.
Cheque Scenario
For example, Dan might write Wes a cheque for $1,000,000, which Wes would present to the Royal Bank, asking for permission to withdraw the money from Dan’s account, and Dan could hand this cheque to Jack to give to Wes.
In this scenario,
the Royal Bank is the guardian
Dan’s bank account is the protected resource
Wes is the entity making the request
Dan’s signature has authorized the request, which specifies that
Wes is allowed to make it
It is drawn on Dan’s bank account
Wes is only authorized to withdraw $1,000,000
The cheque is the Request
Jack is an intermediary who has no part in the transaction other than to pass around the request
There is no special relationship between Jack and Dan except that
Dan trusts Jack to deliver the cheque to Wes
Wes trusts that Jack isn’t going to give him a fake cheque
Validity
When a Request message is sent, the sender stamps the payload with the transmission time, according to either NTP or the clock on the target.
Every entity has both minimum and maximum TTL values. If the ttl property of the validity property of the Request’s payload is specified and between the minimum and maximum value, that value is used for the Request’s TTL. Otherwise,
if that
ttlis specified but too short, the minimum value is usedif that
ttlis specified but too long, the maximum value is usedif that
ttlis not specified, the default value is usedif the default value is was not specified on the Connection, then the minimum value is used
The receiving entity then examines the time property of the validity object.
if the
operationis not'connect':if the
operation.validity.timeproperty of the Request message is not defined, an error.code=EINVAL ErrorPayload response is sentif the
timeis in the future, an error.code=ETIMETRAVEL ErrorPayload response is sentif the
time+ the TTL is in the past, an error.code=EEXPIRED ErrorPayload response is sentif the receiver has ever seen a request with the same
stampon any connection from any source, an error.code=EDUP response is sentIf no error response was sent, the ‘
request’ event is fired.
Cheque Scenario (cont’d)
Revisiting the cheque scenario above, the bank also needs to ensure that the cheque is being presented to the bank for the first time, and is not a digital or photo copy.
Every cheque has a cheque number (validity.stamp) that accompanies the bank transit number (guardian address) and account number (resource address). The bank keeps a list of all the cheque numbers that have been drawn on that account, but keeping a list of all cheques forever would be burdensome. The bank’s solution is to look at the date on the cheque (validity.time) and adds one year (validity.ttl) to that period. If that date is in the past, the cheque is more than a year old and will not be honoured.
Target Startup Notes
In order to be fully secure against replay attacks, targets operating the DCP protocol must employ one of the two following algorithms to prevent message replays (purposeful or otherwise) from triggering a given behaviour more than once, including after a maintenance cycle or crash-recovery:
store all validity-checking information (eg.
payload.validity.stampand its expiry time) in an ACID-compliant storage system
do not acknowledge message receipt until the backing store confirms the data has been permanently recorded
Wait for at least the maximal maximum TTL associated with any connection which the target may receive Requests on before accepting any new messages.
Care must be taken by system administrators operating guardian entities when adjusting the system clock or extending validity times on established guardians, as this could open the system up to a replay attack. For this reason, it is highly recommended that all time changes on systems hosting guardians be made via NTP.
Response Authentication Algorithm
same checks as Message Authentication above – including the nonce check; responses are not exempt from it
additionally,
idmust match an outstanding Request on that connection, or the connection is shut down withEINVAL(a Response that doesn’t correlate to anything we sent is treated as a hostile or badly-broken peer)
Reserved Operations
Certain payload.operation values in Request messages are reserved for use by the protocol itself;
connect
close
keepalive
The messages are sent by same-named methods of Connection instances, and automatically responded at the protocol level without triggering the request event.
connect
The connect operation is the only Request which must be sent from an initiator to a target; all other Request/Response pairs can be exchanged between peers.
This operation establishes the initial DCP connection, ensuring version compatibility, providing initial nonce/id values, and creates the DCP Session Identifier (dcpsid). Both the initiator and the target provide half of dcpsid, by direct concatenation (no separator) of a string each side generates independently. Both halves must be absolutely unique in their environments.
Request connect (initiator -> target) Body
{
id: 'f123',
nonce: 'aaabbbcccddd',
payload:
{
operation: 'connect',
data: { version: '5.3.0', sid: 'jrK3xz9mQeF2' },
validity: { time: '117082920', stamp: '77d25ca91196ceb1c0b851660989b51a', ttl: 15 },
}
}
Response connect (target -> initiator) Body
{
id: 'f123',
time: '1578683050',
success: 'true',
dcpsid: 'jrK3xz9mQeF2Vd8qLp2Yt0Wn',
nonce: '1337c0ded00d',
payload: {
version: '5.3.0',
operation: 'connect'
}
}
or, in the case of a protocol version mismatch (checked in both directions, with the same error code either way):
{
id: 'f123',
time: '1578683050',
success: 'false',
payload:
{
message: "version mismatch; peer is version 3.9.0; we support ^5.2.0",
code: 'EVERSION'
}
}
A sample Request as sent ‘on the wire’, after being signed, including Ethereum envelope:
{
owner: <initiator's identity.address>,
signature: <signature(body)>
body:
{
id: <whatever but unique>,
payload:
{
operation: <name of operation>,
data: /* JSONable whatever or undefined */,
allow: [ { resource: <resource address>, guardian: <guardian address>, accessor: <accessor address> }, ... ],
validity: { time, stamp, ttl }
}
auth: { <resource address>: signature(payload) },
dcpsid: /* unique per "connection", assigned during connect */
nonce: /* last nonce received */
}
}
close
Request: { payload: { operation: 'close' } }
Response: { success: true }
keepalive
Request: { payload: { operation: 'keepalive', data: <sender's Date.now()> } }
Response: { success: true, payload: { success: true } }
Error Codes
The following error codes are defined by this specification for protocol-level errors:
error.code |
Meaning |
|---|---|
EVERSION |
the initiator’s and target’s protocol versions are not compatible with each other (probably one is too old); used symmetrically by both sides of the |
EADDRCHANGE |
the peer’s address is not the same as it was the last time we connected to that URL |
EDCPSID |
the initiator proposed an invalid half of a |
EEXPIRED |
the request has expired |
ETIMETRAVEL |
the request’s timestamp is in the future |
EDUP |
the request is a duplicate |
EINVAL |
the message is invalid |
EPIPE |
the connection was closed locally (the default |
ECONNRESET |
the connection was closed because the peer sent a |
ETIMEDOUT |
a connection or reconnection attempt exceeded |
Note
Beyond the protocol-level codes above, the implementation also raises a number of internal, implementation-specific codes (DCPC-1014, DCPC-1018, DCPC-1024 through DCPC-1027, DCPC-1321, DCPC-1322, DCP-3957) on assorted internal-consistency failures; these are not part of the wire protocol and are not enumerated here.
Sample Code
Send Simple Message
let conn = new protocol.Connection({ location: peerURL });
let response = await conn.send({ operation: 'add', data: [1, 2, 3, 4, 5] });
console.log('The answer is', response.payload);
Receive Simple Message
Setting up a target is much simpler than managing a pool of connections and transports by hand: the protocol.Target class does that for you, creating a Connection internally the first time it sees a given peer, and emitting a connection event (which does not, itself, imply that a session has been established yet – listen for session or request on the Connection for that).
const identity = await wallet.getId();
const target = new protocol.Target(
['socketio'],
{ identity, location: new DcpURL('https://localhost:9000') },
(transportName, server) => console.log(`listening for ${transportName} on`, server.address()),
);
target.on('connection', (conn) => {
conn.on('request', (request) => registerRequestHandler(request));
});
Send Message via Third-Party
A message which is handed to the scheduler to allow the scheduler to escrow some funds from a bank account without multiple round trips might look like this:
let idKS = await wallet.getId();
let schedulerConn = new protocol.Connection({ location: schedulerLocation }, { identity: idKS });
let bankAccountKS = await wallet.get();
let request = new schedulerConn.Request('escrow', { source: bankAccountKS.address, amount: 100 });
await request.authorize(bankAccountKS); // must be awaited, and done, before send()
let response = await request.send();
Note that the bank/scheduler message payload details are beyond the scope of this specification.
Send Message using protected resource
In this example, the bank might understand an operation named ‘transfer’, which transfers funds from one account to another. The source account would require the message sender to have the authority to do this.
let bank = new protocol.Connection({ location: bankURL });
let fundsSourceKS = await wallet.get();
let fundsTarget = new wallet.Address('0xc0ffee');
let request = new bank.Request('transfer', {
amount: 123.45,
source: fundsSourceKS.address,
target: fundsTarget
});
await request.authorize(fundsSourceKS); // must authorize explicitly before send()
let result = await request.send();
console.log('Funds were successfully transferred; receipt:', result)
Receive Message requesting use of protected resource
Using the example from before, let’s look at what registerRequestHandler might do.
async function registerRequestHandler(request) {
let response;
switch (request.payload.operation) {
case 'transfer':
response = await oper_transfer(request);
break;
default:
response = new Error('Invalid Request');
break;
}
request.respond(response);
}
async function oper_transfer(request) {
let {amount, source, target} = request.payload.data;
if (!request.doesAuthorize(source))
return { status: 'not authorized' };
let result = await require('./bank_guts').transfer(amount, source, target);
if (result !== true)
return { status: 'fail', error: result };
return { status: 'ok' };
}