Merge pull request #59 from bitshares/bsip35

Add BSIP 35: Mitigate Rounding Issue On Order Matching
2018-02-27 14:08:58 +01:00 · 2018-02-27 14:08:58 +01:00 · 5bf3ddc5f6
commit 5bf3ddc5f6
parent e3e8bd79b3 3831794d4e
2 changed files with 489 additions and 2 deletions
--- a/README.md
+++ b/README.md
@ -42,7 +42,6 @@ Number             | Title                                                    |
 [32](bsip-0032.md) | Always Match Orders At Maker Price                                          | Abit More | Protocol | Draft
 [33](bsip-0033.md) | Maker Orders With Better Prices Take Precedence                             | Abit More | Protocol | Draft
 [34](bsip-0034.md) | Always Trigger Margin Call When Call Price Above Or At Price Feed           | Abit More | Protocol | Draft
 [35](bsip-0035.md) | Mitigate Rounding Issue On Order Matching                                   | Abit More | Protocol | Draft
 [36](bsip-0036.md) | Remove expired price feeds on maintenance interval                          | oxarbitrage | Protocol | Draft
 [37](bsip-0037.md) | Allow new asset name to end with a number                                   | oxarbitrage | Protocol | Draft
--- a/bsip-0035.md
+++ b/bsip-0035.md
@ -0,0 +1,488 @@
    BSIP: 0035
    Title: Mitigate Rounding Issue On Order Matching
    Author: Abit More <https://github.com/abitmore>
    Status: Draft
    Type: Protocol
    Created: 2018-02-19
    Discussion: https://github.com/bitshares/bitshares-core/issues/132,
                https://github.com/bitshares/bitshares-core/issues/184,
                https://github.com/bitshares/bitshares-core/issues/342
    Replaces: -
    Worker: To be done
 # Abstract
 Under some circumstances, when two orders get matched, due to rounding,
 one order may be paying more than enough, even paying something but receiving
 nothing. This looks unfair.
 This BSIP proposes an overall mechanism to mitigate rounding issue when matching
 orders and avoid something-for-nothing issue completely.
 This BSIP also sets two principles for order matching:
 * never pay more than enough, and
 * something-for-nothing shouldn't happen.
 # Motivation
 There are mechanisms in the system to try to avoid something-for-nothing issue,
 however, not all scenarios are well-handled, see [bitshares-core
 issue #184](https://github.com/bitshares/bitshares-core/issues/184) for example.
 Other than that, rounding issue occurs frequently and has led to a lot of
 confusion among market participants, see [bitshares-core
 issue #342](https://github.com/bitshares/bitshares-core/issues/342) for example.
 # Rationale
 ## Amounts, Prices and Rounding
 Amounts in the system are integers with per-asset fixed precisions.
 The minimum positive amount of an asset is called one Satoshi.
 Prices in the system are rational numbers, which are expressed as
 `base_amount / quote_amount` (precisions are omitted here).
 To calculate how much amount of asset B is equivalent to some amount of
 asset A, need to calculate `amount_of_a * a_to_b_price` which is
 `amount_of_a * b_amount_in_price / a_amount_in_price`. The accurate result
 of this formula is a rational number. To convert it to the final result which
 is an amount, which is an integer, may need to round.
 ## Order Matching
 An order means someone is willing to give out some amount of asset X expecting
 to get some amount of asset Y. The ratio between the two assets is the price of
 the order. The price can be expressed as either `x_amount / y_amount` or
 `y_amount / x_amount`, when we know which amount in the price is of which asset,
 the two expressions are equivalent. The amount of asset X is known and fixed.
 In a market, E.G. the X:Y market, some people are selling X for Y, some people
 are selling Y for X (or say buying X with Y). Orders are classified by type
 (buy or sell), then ordered by price. For each type, the order offering the
 best price is on the top. So, in every market there may be a top buy order and
 a top sell order, name them highest bid and lowest ask, so there is
 a highest bid price (in terms of `asset X amount / asset Y amount`),
 and a lowest ask price (in terms of `asset X amount / asset Y amount` as well).
 When the highest bid price is higher or equal to the lowest ask price, the two
 top orders can be matched with each other.
 ## The Match Price
 In a continuous trading market, orders are placed one by one, when comparing
 every two orders, it's deterministic that one order is placed earlier than the
 other.
 In BitShares, it doesn't mean that the transaction that contains the first
 order is signed before the transaction contains the second, but means that
 the first order is processed earlier than the second in the witness node that
 produced the block that contains the second order.
 When two orders get matched, the one placed earlier is maker, the other one
 is taker. Say, the maker provides an offer, the taker accept the offer.
 So, when calculating who will get how much, we use the maker order's price,
 aka maker price, as the match price.
 ## The Need for Compromise
 When matching two orders, due to rounding, usually we're unable to completely
 satisfy both parties.
 Here is an example mentioned in the 4th comment of [bitshares-core
 issue #132](https://github.com/bitshares/bitshares-core/issues/132):
 Alice's order: Sell CORE at $3 / 8, balance 1000000 CORE
 Bob's order: Buy CORE at $19 / 50, balance $10
 Both assets have precision of 1, i.e. the order balances are
 1000000 CORE-satoshis and 10 USD-satoshis repectively.
 Alice is selling at $3/8 CORE = $0.375 / CORE and Bob is buying at
 $19 / 50 CORE = $0.38, so based on the price, Alice and Bob should match.
 Bob's $10 / $0.38 ~ 26.3. So 26.3 is the fewest CORE he is willing to accept
 (assuming that the meaning of "price" is "the least favorable exchange rate
 a party is willing to accept in trade"). Combined with the design restriction
 that satoshis are indivisible, in practice this means Bob will only accept 27
 or more CORE for his $10.
 But $10 / 27 gives a price smaller than $0.370 and $0.371, which is smaller
 than Alice's sale price of $0.375. So neither party can fill this offer.
 We need to come to a compromise.
 ## The Possible Solutions
 There are some possible solutions listed in the 5th comment of [bitshares-core
 issue #132](https://github.com/bitshares/bitshares-core/issues/132):
 - (a) Fill someone at a less favorable exchange rate than the price they
 specified in their order.  Downside:  This violates the above definition of
 price; i.e. if a user enters a price intending the system to never sell below
 that price in any circumstance, the system will not always behave in a way
 which fulfills that user intent.
 - (b) Keep both orders on the books.  Downside:  This complicates the matching
 algorithm, as now Alice might be able to match an order behind Bob's order.
 Naive implementation would have potentially unbounded matching complexity;
 a more clever implementation might be possible but would require substantial
 design and testing effort.
 - (c) Cancel an order.  This is complicated by the fact that an order such as
 a margin call cannot be cancelled.  Downside:  When there are margin calls
 happening, it seems perverse to delete a large order that's willing to fill
 them just because the lead margin call happens to fall in a narrow window
 which causes a rounding issue.  Also, orders cancelled by this mechanism
 cannot be refunded.  Otherwise an attacker who wants to consume
 a lot of memory on all nodes could create a large number of orders, then
 trigger this case to cancel them all, getting their investment in deferred
 cancellation fees back without paying the cancel op's per-order fee as
 intended.
 - (d) Require all orders to use the same denominator.  Altcoin exchanges and
 many real-world markets like the stock market solve this problem by specifying
 one asset as the denominator asset, specifying a "tick" which is the smallest
 unit of price precision, and requiring all prices to conform.
 Downside:  Complicates the implementation of flipped market UI, may require
 re-working part of market GUI, reduces user flexibility, new asset fields
 required to specify precision, if `n` assets exist then `O(n^2)` markets
 could exist and we need to figure out how to determine the precision
 requirement for all of them.
 ## The Chosen Solution
 Current code actually implemented (a) in the first place: when matching two
 orders, if there is a rounding issue, the order with smaller volume will be
 filled at a less favorable price. It's the least bad compromise since it has
 the most efficiency (highest traded volume while not hard to implement) among
 the solutions.
 The algorithm can be described as follows (sample code is
 [here](https://github.com/bitshares/bitshares-core/blob/2.0.171105a/libraries/chain/db_market.cpp#L311-L324)):
 Assuming the maker order is selling amount `X` of asset A, with price
 `maker_price = maker_b_amount / maker_a_amount`; assuming the taker is buying
 asset A with amount `Y` of asset B, with price
 `taker_price = taker_b_amount / taker_a_amount`. Anyway, since the two orders
 will be matched at maker price, the taker price doesn't matter here as long
 as it's higher than or equal to maker price. Note: currently all limit orders
 are implemented as sell limit orders, so in the example, the taker order can
 only specify amount of asset B but not amount of asset A.
 Now compare `X * maker_price` with `Y`. To be accurate (avoid rounding),
 compare `X' = X * maker_b_amount` with `Y' = Y * maker_a_amount`.
 * The best scenario is when `X' == Y'`, which means both orders can be
  completely filled at `maker_price`.
 * If `X' < Y'`, it means the maker order can be completely filled but the
  taker order can't, aka the maker order is smaller.
  In this case, maker pay amount `X` of asset A to taker, taker pay amount
  `Y" = round_down( X' / maker_a_amount )` of asset B to maker.
  Note: due to rounded down, it's possible that `Y"` is smaller than the
  rational number
  `X * maker_price`, which means `Y" / X` may be lower than `maker_price`,
  that said, the maker order may has been filled at a less favorable price.
 * If `X' > Y'`, it means the taker order can be completely filled but the
  maker order can't, aka the taker order is smaller.
  In this case, taker pay amount `Y` of asset B to maker, maker pay amount
  `X" = round_down( Y' / maker_b_amount )` of asset A to taker.
  Note: due to rounded down, it's possible that `X"` is smaller than the
  rational number
  `Y / taker_price`, which means `Y / X"` may be higher than `taker_price`,
  that said, the taker order may has been filled at a less favorable price.
 ## Issues With The Chosen Solution
 ### The Something-for-nothing Issue
 When filling a small order at a less favorable price, the receiving
 amount is often rounded down to zero, thus causes the something-for-nothing
 issue. Current code tried to solve the issue by cancelling the smaller order
 when it would receive nothing, but only applied this rule in a few senarios
 (the processed parties won't be paying something for nothing):
 * when matching two limit orders, processed the maker
 * when matching a limit order with a call order, processed the call order
 * when matching a settle order with a call order, processed the call order
 * when globally settling, processed the call order
 Other senarios that need to be processed as well (these to-be-processed parties
 may be paying something for nothing in current system):
 * when matching two limit orders, process the taker
 * when matching a limit order with a call order, process the limit order
 * when matching a force settle order with a call order, process the settle order
 * when globally settling, process the settlement fund
 * when force settling after an asset has been globally settled, paying the force
  settle order from global settlement fund, process the settle order
 ### The Broader Rounding Issue
 Something-for-nothing is only a subset of rounding issues, it's the most extreme
 one. There are much more scenarios that one of the matched parties would be
 paying more than enough, although they're not paying something for nothing
 overall. Some scenarios are discussed in [bitshares-core
 issue #342](https://github.com/bitshares/bitshares-core/issues/342).
 Take a scenario similar to the one described in the 4th comment of
 [bitshare-core
 issue #132](https://github.com/bitshares/bitshares-core/issues/132) as an
 example:
 * Alice's order: Sell CORE at `$3 / 80 = $0.0375`, balance `50 CORE`
 * Bob's order: Buy CORE at `$19 / 500 = $0.038`, balance `$100`
 Current system would process them as follows:
 * If Alice's order is maker, use `$3 / 80` as match price; since Alice's order
  is smaller, round in favor of Bob's order, so Alice will pay the whole `50
  CORE` and get `round_down(50 CORE * $3 / 80 CORE) = round_down($1.6) = $1`,
  the effective price would be `$1 / 50 = $0.02`;
 * If Bob's order is maker, use `$19 / 500` as match price; since Alice's order
  is smaller, round in favor of Bob's order, so Alice will pay the whole `50
  CORE` and get `round_down(50 CORE * $19 / 500 CORE = round_down($1.9) = $1`,
  the effective price would still be `$1 / 50 = $0.02`.
 Both results are far from Alice's desired price `$0.0375`. Actually, according
 to Bob's desired price, paying `round_up($1 * 500 CORE / $19) = 27 CORE` would
 be enough, then the effective price would be `$1 / 27 = $0.037`, which is
 still below Alice's desired price `$0.0375`, but much closer than `$0.02`.
 ## The Improved Solution Proposed By This BSIP
 The detailed rules proposed by this BSIP with new rules highlighted:
 * match in favor of taker, or say, match at maker price;
 * round the receiving amounts according to rules below.
  * When matching two limit orders, round down the receiving amount of the
    smaller order,
    * **if the smaller order would get nothing, cancel it;**
    * **otherwise, calculate the amount that the smaller order would pay as
   `round_up(receiving_amount * match_price)`.**
    * **After filled both orders, for each remaining order (with a positive
   amount remaining), check the remaining amount, if the amount is too small
   so the order would receive nothing on next match, cancel the order.**
  * When matching a limit order with a call order,
    * **if the call order is receiving the whole debt amount, which means it's
   smaller and the short position will be closed after the match, round up its
   paying amount; otherwise,** round down its paying amount.
    * **In the latter case,**
      * **if the limit order would receive nothing, cancel it (it's smaller,
     so safe to cancel);**
      * **otherwise, calculate the amount that the limit order would pay as
     `round_up(receiving_amount * match_price)`. After filled both orders,
     if the limit order still exists, the remaining amount might be too small,
     so cancel it.**
  * When matching a settle order with a call order,
    * **if the call order is receiving the whole debt amount, which means it's
   smaller and the short position will be closed after the match, round up its
   paying amount; otherwise,** round down its paying amount.
    * **In the latter case,**
      * **if the settle order would receive nothing,**
        * **if the settle order would be completely filled, cancel it;**
        * **otherwise, it means both orders won't be completely filled, which
       may due to hitting `maximum_force_settlement_volume`, in this case,
       don't fill any of the two orders, and stop matching for this asset at
       this block;**
      * **otherwise (if the settle order would not receive nothing), calculate
     the amount that the settle order would pay as
     `round_up(receiving_amount * match_price)`. After filled both orders,
     if the settle order still exists,
     match the settle order with the call order again. In the new match, either
     the settle order will be cancelled due to too small, or we will stop
     matching due to hitting `maximum_force_settlement_volume`.**
    * **That said, only round up the collateral amount paid by the call order
   when it is completely filled, so if the call order still exist after the
   match, its collateral ratio won't be lower than before, which means we won't
   trigger a black swan event, nor need to check whether a black swan event
   would be triggered.**
  * When globally settling, **in favor of global settlement fund, round up
   collateral amount.**
  * When paying a settle order from global settlement fund, for predition
   markets, there would be no rounding issue, also no need to deal with
   something-for-nothing issue; for other assets, apply rules below:
    * if the settling amount is equal to total supply of that asset, pay the
   whole remaining settlement fund to the settle order;
    * otherwise, in favor of global settlement fund since its volume is bigger,
   round down collateral amount. **If the settle order would receive nothing,
   raise an exception (aka let the operation fail). Otherwise, calculate the
   amount that the settle order would pay as
   `round_up(receiving_amount * match_price)`; after filled the order, if there
   is still some amount remaining in the order, return it to the owner.**
 ## Examples Of The Improved Solution
 ### Example 1
 Take the example mentioned in the 4th comment of [bitshares-core
 issue #132](https://github.com/bitshares/bitshares-core/issues/132):
 * Alice's order: Sell CORE at `$3 / 8 = $0.375`, balance `1000000 CORE`
 * Bob's order: Buy CORE at `$19 / 50 = $0.38`, balance `$10`
 Process:
 * If both orders are limit orders
  * If Alice's order is maker, use `$3 / 8` as match price;
    since Bob's order is smaller, round in favor of Alice's order,
    so Bob will get
    `round_down($10 * 8 CORE / $3) = round_down(26.67 CORE) = 26 CORE`,
    and Alice will get
    `round_up(26 CORE * $3 / 8 CORE) = round_up($9.75) = $10`,
    the effective price would be `$10 / 26 CORE = $0.3846`.
  * If Bob's order is maker, use `$19 / 50` as match price; since Bob's
    order is smaller, round in favor of Alice's order, so Bob will get
    `round_down($10 * 50 CORE / $19 = round_down(26.32 CORE) = 26 CORE`,
    and Alice will get
    `round_up(26 CORE * $19 / 50 CORE) = round_up($9.88) = $10`,
    the effective price would still be `$10 / 26 CORE = $0.3846`.
 * If Alice's order is a call order, since it's bigger, round in favor of it,
  we will get same results.
 ### Example 2
 If we change the example to this:
 * Alice's order: Buy CORE at `3 CORE / $8 = 0.375`, balance `$1000000`
 * Bob's order: Sell CORE at `19 CORE / $50 = 0.38`, balance `10 CORE`
 Process:
 * If both orders are limit orders, we get similar results as above.
 * If Bob's order is a call order, it should have a debt amount which is an
  integer, for example `$26`, then
  * Alice would get
    * `round_up(26 * 3 / 8) = round_up(9.75) = 10 CORE` as a maker, or
    * `round_up(26 * 19 / 50) = round_up(9.88) = 10 CORE` as a taker.
  * Bob would get the full debt amount which is `$26`.
 * If Bob's order is a call order, but the debt amount is a bit high,
  for example `$27`, then Alice would get
  * `round_up(27 * 3 / 8) = round_up(10.125) = 11 CORE` as a maker, or
  * `round_up(27 * 19 / 50) = round_up(10.26) = 11 CORE` as a taker.
  However, since the collateral is only `10 CORE`, this match will fail and
  trigger a black swan event.
 ### Example 3
 If we change the example to that one used above:
 * Alice's order: Sell CORE at `$3 / 80 = $0.0375`, balance `50 CORE`
 * Bob's order: Buy CORE at `$19 / 500 = $0.038`, balance `$100`
 Assuming both orders are limit orders, they'll be processed as follows:
 * If Alice's order is maker, use `$3 / 80` as match price; since Alice's order
  is smaller, round in favor of Bob's order, so Alice will get
  `round_down(50 CORE * $3 / 80 CORE) = round_down($1.6) = $1`,
  and Bob will get `round_up($1 * 80 CORE / $3) = round_up($26.67) = $27`,
  the effective price would be `$1 / 27 = $0.037`;
 * If Bob's order is maker, use `$19 / 500` as match price; since Alice's order
  is smaller, round in favor of Bob's order, so Alice will get
  `round_down(50 CORE * $19 / 500 CORE = round_down($1.9) = $1`,
  and Bob will get `round_up($1 * 500 CORE / $19) = round_up($26.3) = $27`,
  the effective price would also be `$1 / 27 = $0.037`.
 # Specifications
 ## When Matching Two Limit Orders
 ### Handling Something-For-Nothing Issue
 In `match( const limit_order_object&, OrderType ... )` function of `database`
 class, after calculated `usd_receives` which is for the taker,
 check if it is zero.
 If the answer is `true`, skip filling and see the order is filled, return `1`,
 so the order will be cancelled later.
 ### Handling Rounding Issue
 In `match( const limit_order_object&, OrderType ... )` function of `database`
 class, after calculated `receives` for the smaller order, if it isn't zero,
 calculate `pays` for it as `round_up(receives * match_price)`.
 If the smaller order is taker, after filled, even if there is still some amount
 remaining in the order, see it as completely filled and set the lowest bit of
 return value to `1`.
 If the smaller order is maker, since it will be culled when filling,
 no need to change the logic.
 ## When Matching A Limit Order With A Call Order
 In `check_call_orders(...)` function of `database` class,
 if the call order is smaller, round up `order_receives`,
 otherwise round down `order_receives`.
 In the latter case,
 * if `order_receives` is zero, skip filling and cancel the limit order.
 * otherwise, calculate `order_pays` as
  `round_up(order_receives * match_price)`, then the limit order will be
  either completely filled, or culled due to too small after partially filled.
 ## When Matching A Settle Order With A Call Order
 In `match( const call_order_object&, ... )` function of `database` class,
 if the call order is smaller, round up `call_pays`,
 otherwise round down `call_pays`.
 In the latter case, check if `call_pays` is zero.
 * If the answer is `true`,
  * if `call_receives` is equal to `settle.balance`,
    call `cancel_order(...)` with parameter set to `settle`,
    then return a zero-amount collateral asset object;
  * otherwise, return a zero-amount collateral asset object directly.
 * Otherwise, calculate `call_receives` as `round_up(call_pays * match_price)`,
  then fill both orders normally. If the settle order still exists after the
  match, it will be processed again later but with different condition.
 After returned, need to check the amount of returned asset at where calling the
 `match(...)` function, specifically, `clear_expired_orders()` function of
 `database` class. If the returned amount is `0`, break out of the `while` loop.
 If the settle order is still there and the returned amount is `0`,
 label that processing of this asset has completed. Also, in the outer loop,
 need to check the label, if found it's completed, process next asset.
 ## When Globally Settling
 In `global_settle_asset(...)` function of `database` class, round up `pays`.
 ## When Paying A Settle Order From Global Settlement Fund
 In `do_apply(...)` function of `asset_settle_evaluator` class,
 after calculated `settled_amount` and adjusted it according to the "total
 supply" rule, check if it's zero.
 If the answer is `true`, and the asset is not a prediction market,
 throw a `fc::exception`.
 If the answer is `false`, and the asset is not a prediction market,
 and `op.amount.amount` is not equal to `mia_dyn.current_supply`,
 calculate `pays` as `round_up(settled_amount * bitasset.settlement_price)`,
 then, only deduct `pays` from total supply, and refund
 `op.amount.amount - pays` to the user.
 # Discussion
 There is an argument suggests when matching call orders, we should always
 round in favour of the call. If a settlement receives 0 collateral as a result,
 that's acceptable, because the settlement price is unknown at the time when
 settlement is requested, so no guarantee is violated (within the range of
 rounding errors). This should keep the collateral > 0 as long as there is
 outstanding debt. A counter-argument supports rounding up to 1 Satoshi since
 rounding down to zero may break the promise of "every smart coin is backed
 by something".
 There is an argument says breaking the `min_to_receive` limit is a no-go,
 because that's why it's called a "limit order". A counter-argument says
 slightly breaking the limit is the least bad compromise.
 # Summary for Shareholders
 [to be added if any]
 # Copyright
 This document is placed in the public domain.
 # See Also
 * https://github.com/bitshares/bitshares-core/issues/132
 * https://github.com/bitshares/bitshares-core/issues/184
 * https://github.com/bitshares/bitshares-core/issues/342