bsips/bsip-1203.md

    BSIP: 1203 (unassigned)
    Title: Blockchain scanning for inbound Stealth transactions
    Authors: Christopher J. Sanborn
    Status: Draft
    Type: Protocol
    Created: 2018-01-29
    Discussion: https://github.com/bitshares/bsips/issues/91


## Abstract

The existing Stealth implementation ([BSIP-0008](bsip-0008.md)) requires the sender to manually communicate *transaction receipts* to the recipients of each transaction to alert them to the presence of an inbound balance transfer, creating a danger of lost funds due to miscommunicated or lost receipts. This BSIP explores options for automated discovery of inbound transactions while still preserving fundamental privacy features of unlinkability and anonymity.

## Motivation

"Stealth addresses" are a method of providing _unlinkability_ to blockchain transactions.  Unlinkability is a major component of the Privacy Triad: _unlinkability_, _confidentiality_, and _untraceability_.  Using a stealth address, a sending wallet is able to compute a child public key that derives from a public key encoded in the address, but which cannot be correlated, or "linked", to the address public key, unless you are either the sender or the receiver.  This child key becomes the authorization key for transaction outputs (TXOs) intended for the receiver.  As such, third party observers cannot link TXOs to addresses, nor even link together independent TXOs which are destined to the same address.

Although this is a great benefit to privacy, it complicates the matter of detecting inbound transactions, since a wallet cannot simply scan for transactions which explicitly identify the destination address.

Existing [Stealth Phase I](bsip-0008.md) functionality already includes the use of stealth addresses, but does not include a solution for detection of inbound transactions.  As a result of which, user adoption of the Stealth feature has been very minimal.  We propose below a solution to inbound transaction detection as well as some additional enhancements to the stealth addressing scheme, including a proposed new address format that allows for watch-only wallets.

## Rationale

A confidential transaction (cTX) does not identify the recipient.  As such, there is no direct way for a wallet to use only its Stealth address to query the p2p network for inbound transactions.  In the current "phase one" implementation of Stealth ([BSIP-0008](bsip-0008.md)), inbound discovery is a manual process requiring the sender to communicate "transaction receipts" to the intended recipients of each transaction output in order to alert each recipient of their incoming balance.  Transaction receipts are encrypted data structures that embed the Pedersen commitment of the transaction output and the value and blinding factor that the recipient needs to "open" the commitment.  Additionally, the receipt records the one-time public key which the recipient must use to derive the private key offset needed to spend the incoming coin, via a shared-secret procedure between the one-time key and the recipient's address key.  The need to communicate transaction receipts is burdensome and introduces substantial risk of lost funds due to failure to communicate or retain receipts.

_Keys involved in a cTX output (cTXO):_
* **One-Time Key (OTK)** — The sender generates this key (public and private) from randomness and uses it to generate a shared-secret between the OTK and the recipient's Address ViewKey.  The OTK PubKey will be clear-text encoded in the Tx receipt, and optionally also recorded in the transaction output to enable automated discovery.
* **Address Keys (ViewKey and SpendKey)** — These are public keys encoded in the recipient's stealth address.  The goal of a stealth address scheme is to _not_ identify these public keys in any transaction output.  A long-form address encodes _two_ public keys, referred to as ViewKey and SpendKey. The SpendKey serves as a base point from which individual Tx output AuthKeys are computed as an offset, and the ViewKey is used with the OTK to compute that offset.  A short-form address encodes only a single public key, which serves as both the ViewKey and SpendKey.
* **Tx Output Authorization Key (AuthKey)** — This public key will be recorded in the confidential transaction output (cTXO) as the key which is authorized to spend the commitment.  This key is offset from the address SpendKey by a secret offset that only the sender and recipient can calculate (from the shared secret between OTK and ViewKey).  The sender can only know the offset, but not the full secret key to the AuthKey.  The recipient, knowing the private key behind the SpendKey, can compute the private key to AuthKey and therefore can spend the commitment.

Automated discovery could be enabled if the receipt were embedded within the transaction data structure and if an aspect of that data structure supported a challenge condition which the recipient could recognize.

Current network rules allow for a receipt to be embedded in each Tx output via a `stealth_memo` field which is formatted in a similar way as the encrypted memos that may accompany regular (non-Stealth) transfer operations. These memos are composed of a header specifying the OTK PubKey and the "message PubKey" for which the recipient holds the corresponding private key, followed by cipher text which is AES encrypted with a shared-secret key between the OTK and the message PubKey.  For this `stealth_memo` field, the current behavior of the CLI reference wallet is to use the recipient's Address PubKey as the message PubKey.  Although this is a reasonable choice for encrypting message text generally, it has the severe downside of identifying the recipient's Address PubKey in the memo header, and therefore breaks anonymity and negates the unlinkability provided by using a stealth address scheme.  For this reason, the CLI reference wallet currently does _NOT_ actually embed the memo in the Tx ouput but instead Base58 encodes it and prints it to the screen, calling it a "transaction receipt." The sender must manually, and secretly, transmit this to the recipient via a side channel.

**Stealth Memo structure: (Stealth I)**

<span></span> | <span></span>
-----: | :---
**One-time PubKey:** | Chosen from randomness by sender &nbsp; **_(33 bytes)_**
**Message PubKey:** |  Public key controlled by recipient. &nbsp; **_(33 bytes)_**<br>
**Cipher Text:** | AES encrypted message, using _key &leftarrow; Shared(OTK,MPK)_

_Current Stealth I behavior is to use the Address PubKey as the message PubKey, which reveals intended recipient!!_

A very simple solution would be to change the behavior of embedding the Address PubKey in the message PubKey field, and to instead record the Tx output AuthKey in this slot.  Because the recipient is also able to derive this AuthKey through knowledge of her own address private keys (in combination with the OTK recorded in the header), the recipient would simply need to test the OTK against each of their own Address Keys to see if the resulting AuthKey matches the one in the header.  If it does, then the output is recognized as destined to the recipient, even though the recipient's Address PubKeys are not identified in the memo header.  The computational cost of this is one Diffie Hellman round, a hash operation, and a child key derivation.  It should be noted that the AES encryption key should still be computed from the shared secret between the OTK and the address ViewKey, however, as this will allow view-only wallets which cannot compute the private key behind the AuthKey to decrypt the memo and tally the incoming transaction.

**Stealth Memo structure: (Proposed: Stealth II)**

<span></span> | <span></span>
-----: | :---
**One-time PubKey:** | Chosen from randomness by sender &nbsp; **_(33 bytes)_**
**cTXO AuthKey:** |  Child public key of the stealth address and the OTK. &nbsp; **_(33 bytes)_**<br>
**Cipher Text:** | AES encrypted message, using _key &leftarrow; Shared(OTK,ViewKey)_

_Proposed Stealth II behavior is to embed the AuthKey in the second slot, while still encrypting the message data with a shared key between the OTK and the Address key (specifically, the ViewKey so that watch-only wallets can read the commitment data)._

To support this strategy, a wallet will need to inspect all cTX activity on the network and test the challenge conditions on each transaction.  This could be achieved if API nodes are extended to provide an API call to retrieve `stealth_memo` fields from all cTXOs appearing in a specified block range.  The wallet could simply download the memos, test the challenge on each one, and identify and decrypt the ones that are destined to the wallet.  No need would remain to manually transmit transaction receipts.  The receipts would be embedded, compactly and unlinkably, in the Tx outputs.

## Specifications

We specify two protocols. In the first subsection, [_Wallet procedure..._](#wallet-procedure-for-recognizing-own-commitments), we specify the recognition protocol by detailing wallet behaviors for:
* Creating transaction outputs that can be recognized by their recipients, and,
* Recognizing transaction outputs that are destined to the wallet.

And in the second subsection, [_API requirements..._](#api-requirements-to-allow-detection-of-inbound-commitments), we propose a new API call for querying nodes for transaction outputs to be scanned for recognizable markers. This is an added feature for API nodes and does not involve any consensus changes.

### Wallet procedure for recognizing own commitments

Assumptions:

1. Wallet has access to a set of private keys corresponding to stealth addresses which may own commitments on the blockchain.  These private keys are needed to "recognize" incoming transactions.  If the wallet is a watch-only wallet for a particular address, then it is assumed to have the private and public ViewKey, but only the public SpendKey.
2. Wallet can query an API node for commitments occurring between specified block heights, to obtain sets of embedded receipts to scan for owned commitments. ([See below](#api-requirements-to-allow-detection-of-inbound-commitments) for this process.)

We detail procedures for stealth address formats which encode either a single public key, or two distinct public keys in which one key is the ViewKey and the other the SpendKey. The single-key format is already in use on the BitShares network and is borrowed from the original Confidential Transactions specification. The dual-key format allows for additional wallet features and is borrowed from CryptoNote-based coins such as Monero.  No changes to the network nodes are required for wallets to support dual-key address formats.  In fact, the single-key format can be thought of as a special case of the dual-key format in which the same key is used as the ViewKey and the SpendKey.

**Address Formats:**

&nbsp; | Format:
:------:|--------
**CT-style:** | Single public key and checksum. Public key _A_ serves both viewing and spending roles.<br><br> Format: `BTSaaaaaaaaaaaaaaaaaaaacccc`
**CryptoNote-style:** | Two public keys plus a checksum. Public key _A_ serves the viewing role and public key _B_ serves the spending role.<br><br> Format: `BTSaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbcccc`
**Invoice Nonce:** | This one encodes a single PubKey serving both the viewing and spending role, but also includes a 64-bit "nonce" or "tag" that the spending wallet is to include in the encrypted memo part of the cTXO, allowing the receiver to interpret payment as being applied to a specific invoice. <br><br> Format: `BTSaaaaaaaaaaaaaaaaaaaannnnnnnncccc`

_(In the address formats above we consider the part following the "BTS" identifier to be Base58 encodings of the concatenated byte buffer representations of public keys and checksum bytes. C.f. [Base58Check](https://en.bitcoin.it/wiki/Base58Check_encoding) encoding.)_


The dual-key format separates the duties of spending a commitment from those of reading the commitment, such that a person in possession of only the "viewing key" (the private key corresponding to the additional pubkey in the address) can discover, interpret, and tally incoming transactions, but _cannot_ spend them.  The "spending key" (private key corresponding to the primary pubkey in the address) is needed to authorize the spending of a commitment. The dual-key address format and signing procedures are described in detail in [[vS13]](#references) and reviewed below.

#### Procedure for single-key stealth addresses (CT-style)

Recognizability depends on there being a deterministic relationship between the AuthKey that authorizes expenditure of a particular cTXO, the one-time key (OTK) that the sender generated randomly for the cTXO, and the recipient's Address key (or keys).

We assume that the stealth address encodes public keys corresponding to two purposes: discovery, and expenditure.  When an address encodes only one public key, that key is used for both purposes. We refer to the key for discovery as the "view" key, and denote the private, public pair as _(v, ViewKey)_.  For spending, we denote the key pair as _(s, SpendKey)_.

The AuthKey for a cTXO is an EC point summation of the address's SpendKey and an EC point "offset," which, for present purposes we will denote by the private, public pair _(o, Offset)_, with _Offset = o*G_, where _G_ is the generator point for the curve.

&nbsp; _AuthKey &nbsp;=&nbsp; SpendKey &nbsp;+&nbsp; Offset_

Anonymity is preserved so long as only the sender and the receiver are able to compute _o_ and _Offset_.  The algorithm for computing _Offset_ is a deterministic function of the OTK and ViewKey only, (and not the SpendKey).  This allows the the recipient to recover the SpendKey by simple subtraction of _Offset_ from the AuthKey.  The recipient's wallet then compares the computed SpendKey against the address SpendKey.  The wallet may even compare against a whole list of SpendKeys that the wallet may have used to generate an address family with a common ViewKey, allowing for differentiable invoices, without sacrificing efficiency of scanning. (See _[address-per-invoice](#utility-of-dual-key-addresses)_ below.)

Algorithm | Description / Specification
:---:|:---------------------------
_Shared(a,B)&nbsp;&rarr;&nbsp;secret <br> Shared(A,b) &rarr; secret_ | This yields a "shared secret" between public keys _A_ and _B_ computable only by parties possessing at least one of the private keys _a_ and _b_.<br><br>&nbsp; _secret = SHA512(P<sub>X</sub>)_; &nbsp; &nbsp; _P = aB = Ab_<br><br>For BitShares Stealth, the secret is a byte buffer (64 bytes) computed from the SHA512 hash of the encoded _X_ coordinate (32 bytes) of EC point _P_. (c.f. [EC Diffie-Hellman](https://en.wikipedia.org/wiki/Elliptic-curve_Diffie–Hellman).)
_ChildOffset(B,index)&nbsp;&rarr;&nbsp;offset_ | This yields an integer-valued private key _offset_ that generates the keypair _(offset, Offset = offset*G)_.  The offset is considered to be a "child" of key _B_, and the parameter _index_ is a byte buffer.<br><br>&nbsp; _child = BigInteger(SHA256(Compressed(B)_ &vert;&vert; _index))_<br><br>_Compressed(B)_ refers to the SEC1 "compressed" representation of public key _B_.  The &vert;&vert; symbol refers to concatenation.

**Sending:**

The sender's procedure for computing the offset and generating the AuthKey for the cTXO is detailed as follows:

<ol>
<li> Compute a "shared secret" between between the sender and receiver:

_secret = Shared(otk, ViewKey)_

The sender must be careful to leak neither the shared shared secret nor the private _otk_ key.
</li>
<li> Compute the OffsetKey as a child of the ViewKey, indexed by the shared secret:

_offset = ChildOffset(ViewKey,secret)_<br>
_OffsetKey = offset*G_
</li>
<li> Compute the AuthKey by summing SpendKey and OffsetKey:

_AuthKey = SpendKey + OffsetKey_
</li>
</ol>

**Receiving:**

The receiver, having acquired a list of cTXO metadata that includes _OTK_ and _AuthKey_, goes through the following process to test for ownership:

<ol>
<li> Compute a "shared secret" between between the sender and receiver:

_secret = Shared(OTK, viewpriv)_
</li>
<li> Compute the OffsetKey as a child of the ViewKey, indexed by the shared secret:

_offset = ChildOffset(ViewKey,secret)_<br>
_OffsetKey = offset*G_
</li>
<li> Recover the public SpendKey by subtracting OffsetKey from AuthKey:

_SpendKey = AuthKey - OffsetKey_

</li>
<li> Compare the recovered SpendKey against all wallet SpendKeys that may have been used with the ViewKey to generate an address.  If one matches, then the cTXO is "recognized."  To later spend the cTXO, the wallet computes the private authorization key as:

_authpriv = spendpriv + offset_
</li>
</ol>

Thus, a wallet may undertake to periodically download and scan the metadata for Stealth transactions and test for outputs that can recover a wallet's public SpendKey from knowledge of the private ViewKey.

##### Embedding recognizability data in the transaction

For the recipient to have the practical ability to recognize a cTXO as their own, the cTXO, as recorded on the blockchain, must include the following two items: 1.) The one-time PubKey (OTK) that the sender generated for shared-secret generation, and, 2.) the authorization PubKey (AuthKey) of the cTXO.  Because the AuthKey is computed deterministically from _Shared(OTK,AddrKey)_, it stands that if the recipient can generate the same AuthKey, then the cTXO belongs to them.

The structure of a cTXO is as follows:

_Field_ | _Purpose_
-------:|:-------
**`commitment`:** | Blinded value commitment &nbsp; _(EC curve point, 33 bytes)_
**`range_proof`:** | Proof data supporting transaction validity &nbsp; _(0 to ~5 KiB)_
**`owner`:** | BitShares owner structure specifying weighted list of keys or accounts that may spend this commitment. (Typically lists just one public key, the "AuthKey" for the cTXO.)
**`stealth_memo`:** | Also known as the "transaction receipt" when transmitted off-chain.<br><br> **`one_time_key`:** &nbsp; _(EC curve point, 33 bytes)_<br> **`to`:** &nbsp; Use the AuthKey here! _(EC curve point, 33 bytes)_<br> **`encrypted_memo`:** &nbsp; Data that recipient needs to "open" the commitment.<br><br> _The stealth memo is optional as far as the network is concerned, but essential if you want the recipient to be able to detect the incoming transaction without sending a "receipt."_

_(An example transaction showing all these fields can be seen in [block 22157273](https://cryptofresh.com/tx/8182e9d78cbce7df281bc252a9e6d87566ca0622). In this Tx, the stealth_memo '`to`' field unwisely names the recipient's address key, rather than the cTXO Auth key, and thus breaks unlinkability.)_

Since the `stealth_memo` field can be used to record both the OTK and the AuthKey, all the wallet needs to do to scan for incoming transactions is to download batches of stealth memos and, for each one, test whether the combination of the OTK and the wallet's Address key yields the AuthKey.  If it does, then derive the AES decryption key from _Shared(OTK,ViewKey)_ and use that to read the additional data in `encrypted_memo`. 

Structure of `encrypted_memo`:

_Field_ | _Purpose_
-------:|:-------
**`from_key`:** | Original use:<br><br><ul>Identifies address key of sender **(optional)**. &nbsp; _(EC curve point, 33 bytes)_</ul>Alternate possible uses:<br><br> <ul><li>Instead of ID'ing the 'from' address, could use this field to embed an _invoice nonce_ to allow receiver to correlate payment to an invoice.</li></ul>
**`amount`:** | Value of commitment. &nbsp; _(Integer, 32 bytes)_
**`blinding_factor`:** | Blinding factor. &nbsp; _(Integer, 32 bytes)_<br><br>_Note: Except when a blind_sum is needed, the blinding factor is deterministic from a hash of the shared secret, meaning this field can potentially be repurposed or omitted. To guarantee that the blinding factor can always be deterministic, transactions can be padded with a commitment to zero to absorb the blind_sum._
**`commitment`:** | The Pedersen commitment. &nbsp; _(EC curve point, 33 bytes)_<br><br> _Note: This field is redundant, since the commitment is determined by_ C = amnt * H + blind * G, _and could potentially be omitted._
**`check`:** | Checksum to confirm correct decryption. &nbsp; _(4 bytes)_

_(TODO: How is this serialized? Do omitted fields "take up space"?  Can fields be chosen a la carte? How hard will it be to extend this memo format, for, say, multiple assets in the case of CA? See fc::raw::pack.)_

 
### API Requirements to Allow Detection of Inbound Commitments

(Discuss here a proposed API call for retrieving commitments by block height range)

(Note: the data returned by this call should also include (or, perhaps this should be a separate call) a list of all commitments that were SPENT in the same block height range.  In the case of RingCT this would be a list of key-images used, since with RingCT the network doesn't know which commitments are spent or not.)

(Discuss also possible optimizations such as batching historical commitment and spend data into ranges of fixed size (say intervals of 2^17 blocks, representing a little over four days) so that these data-balls can be cached and returned efficiently without the node needing to satisfy a unique query every time.)

## Discussion

### Utility of dual-key addresses

Utilization of the dual-key address format has numerous interesting use cases, including:

* The ability to maintain **watch-only wallets**. By entrusting only the View Key to a view-only node, it is possible for this node to monitor for activity to an address without granting spending access to the same node.  This allows for such things as: opt-in transparency; cash register monitoring; organizational internal auditing, etc.

* The ability to use **address-per-invoice** without introducing substantial additional scanning overhead.  To use this, one keeps the same viewing PubKey and iterates the Spending PubKey part of the address, generating a distinct address per invoice.  When scanning the blockchain, the _Child Offset_ is a function of the shared-secret between OTK and ViewKey, such that _AuthKey = SpendKey + Offset_.  To obtain the public SpendKey, a watching wallet can subtract the Offset from the AuthKey to obtain the SpendKey, and simply compare against a list of per-invoice SpendKeys that were used to generate addresses.  Adding additional SpendKeys to scan for does not incur any additional EC group operations, merely additional byte-wise comparisons, which are trivial. _(Note that while the invoice addresses generated in this manner are distinct, they are not unlinkably distinct, since they share the viewing component.  If a business wants individual invoices to be mutually unlinkable, then this scheme will not be sufficient.  However, this is a consideration for a business's off-chain security practices, as the addresses themselves never appear on-chain or in a transaction.)_

### Possible future extensions

#### Additional address formats

The two stealth address formats described above provide for single key and dual key addresses, where the latter allows for separation of transaction monitoring from the ability to spend, allowing for view-only wallets.

There may be use cases for additional address formats allowing for more complex situations.

One would be a multi-sig situation in which the address format encodes multiple spending keys and a weighting requirement.  Although, this would make the resulting address very lengthy, it would also add an interesting use-case.  And, since BitShares authority structures already allow for a vector of authorizing keys and weights, it should be possible to implement the feature on the wallet side, without needing any changes to consensus or API.  This idea is not explored further here but merely suggested for future exploration if there is a desire for the feature.

Another use case for an extended address format would be... (TODO: Discuss including an "invoice nonce" in the address format for correlating incoming transactions to a particular invoice.  C.f. Bitcoin where using an address-per-transaction serves both unlinkability as well as invoicing.  With Stealth addresses, there is no need to increment addresses for unlinkability, and doing so to facilitate invoicing only increases the scanning overhead by introducing the need to test against additional private keys.  But by including an invoice nonce in the address format, which the spending wallet would carry over into the encrypted part of the `stealth_memo`, the recipient can correlate payments to invoices while using only a single address key (or a single pair of spending/viewing keys in the dual-key case).)

### Pitfalls and Cautions

#### Specific risks and remedies

##### Sender leaks private _otk_ or private _shared secret_:

TODO: analyze

#### An attack on address key from leak of a transaction private key

A confidential output will have associated with it an "Output PubKey." He who can provide a signature from that PubKey is authorized to spend the commitment.  Automated detection of inbound commitments depends on the deterministic computation of an offset between the One-time PubKey and the Address PubKey, which is computed from the shared secret between the sender and receiver.  Because only the offset is deterministic, the sender cannot compute the private key to the Output PubKey.  Only the receiver can do this (by knowing both the offset and the Address private key).

Because Output PubKeys are only used once, wallet software designers may be led to believe that the security of the Output PrivKeys are only important up until the commitment is spent.  (What would it matter, to leak that private key, when the commitment it authorizes is no longer spendable?)  This would be a mistake, however, because anyone who can compute the additive offset can subtract it from the Output PrivKey to reveal the Address PrivKey.  Although the general public is not expected to be privy to that offset, the _sender_ of the output is in posession of the offset (and the ability to compute it due to knowing the random _k_ behind the One-time PubKey).  This means the sender can compute the Address PrivKey, in the event that the recipient leaks the Output PrivKey.

Thus, wallet designers should be advised to treat the Output PrivKeys handled by their wallets with at least as much care as the Address PrivKeys, even long after the commitments they authorize have been spent.  A leak of a single commitment's PrivKey is tantamount to a leak of the PrivKey for the entire wallet.

(A similar risk of revealing a parent PrivKey from leak of a child PrivKey and parent XPUB when using non-hardened derivation is noted in the Bitcoin BIP-32 protocol for Hierarchical Deterministic Wallets.) 

## Summary for Shareholders

Although the goal of this BSIP is to support the long-range vision of [Stealth Phase II](bsip-1200.md), the implementation of this BSIP would provide value _right now_, as it would enable the utilization of even the Phase I _Confidential Transactions_ operations without the reliance on burdensome transaction receipts, which are the primary end-user stumbling block to routine Stealth use.

We have detailed in this document procedures for producing recipient-recognizable transaction outputs from stealth addresses.  Specifically, we have detailed the procedure for _two_ distinct stealth address formats: a single-key address format which is identical to that which is already in use, and a new dual-key address format which separates the duties of monitoring from spending, and thus allows for watch-only Stealth wallets.  Support for the dual-key address format would require no network or consensus changes.  It requires only wallet support.

Most of the work needed to implement this BSIP is in the wallet, namely the correct production of recognizable cTXOs and the process of scanning for owned cTXOs.  The only network-level change is the addition of two API calls: one to return batches of `stealth_memo` fields included in a block range, and one to return batches of consumed commitments (in the case of CT transactions) or used key-images (in the case of RingCT transactions).  These API calls would need to be available on wallet-serving API nodes, but would not be needed on block-producing witness nodes.  There are no consensus changes proposed in this document.

## Copyright

This document is placed in the public domain.

## References

[vS13] - Nicolas van Saberhagen, _Cryptonote v 2.0_, 2013 - https://cryptonote.org/whitepaper.pdf

## See Also