Merge pull request #61 from bitshares/oxarbitrage-patch-1

Add BSIP 36: Remove expired feeds

bsip-0036.md

@@ -0,0 +1,73 @@
+    BSIP: 0036
+    Title: Remove expired feeds
+    Author: oxarbitrage <https://github.com/oxarbitrage>
+    Status: Draft
+    Type: Protocol
+    Created: 2018-02-22
+    Discussion: https://bitsharestalk.org/index.php?topic=25996.0
+    Replaces: -
+    Worker: To be done
+
+# Abstract
+
+Currently, expired bitasset feeds remain in the blockchain forever. The only way to get them erased is if operation `update_feed_producers` is called with new publishers on the asset. At this call, and only at this case, old publisher feeds will be deleted from the blockchain. If this operation never occurs or if it occurs maintaining some or all of the same publishers feeds will not be removed.
+
+Some examples of this are the followings, where we can see feeds from 2015:
+
+- http://open-explorer.io/#/objects/2.4.9 - HKD
+- http://open-explorer.io/#/objects/2.4.21 - USD
+- http://open-explorer.io/#/objects/2.4.10 - RUB
+- etc.
+
+This BSIP proposes a protocol change to remove expired feeds at maintenance time.
+
+# Motivation
+
+Improve performance of the bitshares-core.
+
+
+# Rationale
+
+The proposed change only affects the feed price when the feed expiration time is increased. This rarely happens, if ever, so in practice the effect on pricing is negligible. The savings in performance however, is not.
+
+# Specifications
+
+As mentioned in the abstract feeds stay in the blockchain forever unless `asset_update_feed_producers_evaluator::do_apply` is called.
+
+The BSIP proposes to remove expired feeds at maintenance time by adding feed cleanup code at `db_main.cpp`.
+
+Solution is not that simple, `asset_update_bitasset_operation` can change `feed_expiration_time` and `minimum_feeds` required for the asset.
+
+If a feed is expired and the `feed_expiration_time` changes, expired feeds may became valid, similar if `minimum_feeds` (required to calculate the median) is increased, older feeds may became valid and used by the core market engine if available.
+
+This scenario actually happened in the past and that is the reason the change will fail unless a hardfork protection is added.
+
+## The proposed solution
+
+Goal is to remove expired feeds in a safe manner. To do this we need to keep current behavior before hardfork date and only remove feeds at maintenance intervals after the hardfork. This will allow the chain to synchronize and start the cleanup at a point where all the feeds are valid, safely being able to remove them an continue clean from that time on.
+
+This a permanent hardfork, code before hardfork date will work one way(current behavior) and after date new code will work different making the feed cleanup at maintenance times.
+
+Feed cleanup code will only be executed after proposed hardfork date, at the first maintenance interval after date old feeds will be removed, node will not be able to use them anymore as they will not be available in the database. 
+
+If `asset_update_bitasset_operation` is called before hardfork operation will be normal as it is currently is now.
+
+If `asset_update_bitasset_operation` is called after hardfork and increases the `minimum_feeds` while they are not enough feeds `update_median_feeds()` will set a null feed. This is current `asset_update_bitasset_operation`, no need to make any change in the operation.
+
+
+# Discussion
+
+https://bitsharestalk.org/index.php?topic=25996.0
+
+# Summary for Shareholders
+
+[to be added if any]
+
+# Copyright
+
+This document is placed in the public domain.
+
+# See Also
+
+* https://github.com/bitshares/bitshares-core/issues/518
+* https://github.com/bitshares/bitshares-core/pull/598