Robin's Blog

This is the Rotki blog at which Robin, the accounting bird, is writing about finance, accounting, analytics, DeFi, cryptoeconomics and more!

© 2019-2023. All rights reserved.

Why creating a new asset in an immutable ledger should not reuse an old name

19 Nov 2019

Introduction

Yesterday MakerDAO reached a very important milestone. It introduced a new type of DAI stablecoin that can have multiple underlying assets used for collateral as opposed to the old DAI which could only be supported by collateralizing ETH.

Unfortunately at the same time they decided that they were so attached to the name DAI that they wanted to give the same name to the new token. And they could only do this by renaming the old token. Which is exactly what MakerDAO went with as you can see here.

This post is going to explain in detail why from a user’s or an application’s perspective the decision to change the name is quite bluntly put idiotic. Renaming DAI to SAI has zero advantages and introduces a long list of problems as seen below.

Problems introduced by renaming

In this section we will see a writeup of all the problems that the renaming is introducing along with a short explanation for each one.

Token Symbol

A contract deployed in the Ethereum blockchain is immutable. The DAI contract is here. If you try to read the symbol attribute it will be DAI. The symbol attribute is part of the ERC20 interface and ignoring it breaks that interface. You can not arbitrarily decide to now call the token by a different name (SAI) while the symbol of the token is DAI.

Centralized Exchanges

Right now centralized exchanges are in a tough spot. They have to perform DAI to MCDAI upgrades for their users and then decide on how to name the new assets. If an exchange intends to keep both tokens listed then the naming becomes a problem.

In the end the problem is pushed to the users as can be seen from above. For users SAI is DAI but if as a user you attempt to deposit what you think is DAI to Kraken you will lose it. If that does not constitute a UX nightmare I honestly do not know what does.

Decentralized Exchanges

Decentralized exchanges do not have much of a choice. They do not hold the tokens for their users so they will not be able to automatically upgrade DAI for them. They need to offer trades in both tokens and then the naming does indeed become a problem.

Software libraries

Software needs to be making assumptions for some basic things. One of these assumptions that software in our field make is on the immutability of contracts and by extension the token contract symbols.

Software libraries that made correct assumptions in the past will now break. As an example, one of the most used javascript libraries in crypto UI applications is cryptocurrency-icons. It pulls icons for crypto assets based on the symbol name.

At the moment, the DAI symbol returns the old logo of DAI and the SAI symbol will return an error. All projects using this library (and it’s a lot) will have to update whenever a new version comes out. At the moment there is no version supporting the new naming so all projects using it are displaying the wrong icons.

External APIS

A lot of APIs, for example price feeds such as cryptocompare have had DAI listed. Now they need to update their APIs in order to point DAI to the price of the new multicollateral DAI and create a new entry for the SAI token which points to the old DAI price. And that is hoping that there is no other cryptocurrency with the SAI symbol. I am pretty sure not every one of these price feeds will follow the new naming and then end-user applications will have to maintain converters between different price feeds using different symbols. It’s a mess.

End user applications

Every single end-user application that deals with crypto and DAI has to adjust. As an example we can take Rotki. If we go with renaming DAI to SAI we have to create a new release that will upgrade the user’s databases renaming the assets and rewritting history. If we do not we will confuse our users.

What is the solution?

At this point nothing can be done. The renaming has to go through since it has already started. But due to all the reasons outlined above it was a terrible decision.

The proper solution would have been a very simple one. DAI stays as DAI and is the symbol of the single collateral DAI. And then the new multi-collateral token is called MCDAI or well … anything other than the same name as the previous token.

Conclusion

Despite all the problems with the renaming the achievement of having a multicollateral DAI is something to be proud of.

All of the above problems in isolation do not sound really bad but if you sum them all up it ends up as a nightmare for all actors involved. It’s a mess for application developers, a mess for users and a mess for service providers. And all of these could have been avoided by simply doing the obvious thing and providing a new name for a new token!

For Rotki a blog post will follow explaining how we will handle the switch from SAI to DAI and how it will be reflected in the databases of our users. If you liked this post and have not tried Rotki yet you can get the latest release from here. By using Rotki you can take ownership of your financial data! Please try it out and provide us with feedback so that the software can improve.

Rebranding to Rotki

14 Nov 2019

Why rebrand?

Since launching v1.0 of the Rotkehlchen application a common theme on feedback was that it’s a nice app with a definitive use case but people simply can not spell and much less pronounce the name.

No matter the attachment to the name rebranding becomes very important when people can not write down the name of the project to recommend it to their friends or can not even google it successfuly. Projects like Rotki live and die from word of mouth so everything that can be done to facilitate it should be done.

To that end as of right now the project name has been rebranded to Rotki.

What has changed?

  • The github repo has been moved from rotkehlchenio/rotkehlchen to https://github.com/rotki/rotki.
  • We got a new domain name and the entire website can now be found at rotki.com. Same applies for the blog and the api under the equivalent subdomains.
  • All @rotkehlchen.io emails now have @rotki.com equivalents.
  • The twitter handle was renamed from rotkehlchenio to rotkiapp
  • The documentation was moved to https://rotki.readthedocs.io/en/latest/.
  • We now own the rotki.eth ethereum ENS name and it points to the ethereum donation address for the project.

The old rotkehlchen.io domain name is still ours and will keep working for at least 1 more year.

Thank you

Thank you for your continued support! It is only through feedback from our users that Rotki can become a better tool that will enable you to take ownership of your financial data.

If you see any remaining places where the name needs to be changed just let us know via Twitter and we will get to it.

If you haven’t done so yet please download the latest Release for your OS, try Rotki out and give us feedback on what you would like to see improved.

Dealing with ambiguity in token symbols

01 Sep 2019

Introduction

This post tries to explain why Rotki’s approach to dealing with ambiguity in crypto assets is needed and why other approaches are error prone and most probably lead to broken results.

We will see what Rotki does in detail and what challenges exist when interfacing with multiple exchanges and other external services.

Understanding The Problem

The best way to understand the problem is by example. Take the cryptoasset with the symbol KEY. Most of our competitors would simply see you have a KEY balance and query for its price at any given moment from a website such as cryptocompare.com.

The problem is that as of the writing of this post three different tokens exist that use that symbol.

To make matters worse, price aggregator websites such as cryptocompare and coinpaprika have different representations.

For example:

  • SelfKey is known as KEY in cryptocompare
  • Bihu Key is known as BIHU cryptocompare

  • KeyCoin is known as KEYC in cryptocompare

  • SelfKey is known as key-selfkey in coinpaprika
  • BihuKey is known as key-key in coinpaprika
  • KeyCoin is not known in coinpaprika

Add different representations for each symbol in different exchanges in the mix and then the whole situations gets even more entangled as we can see with ETHOS and BQX below.

Do you believe that competitor webapps take all these different situations into account? Want a bet? Let’s go and check their implementation … oh wait … we can’t. Not only do they need you to upload all your data to them but their sourcecode is not open and thus you can’t audit their calculations.

Your SelfKey could be priced as Keycoin or viceversa, or worse bail out since it can’t find a price and not consider it in any profit/loss calculations.

Solving the Problem

The only way to handle this problem is by maintaing a database of all assets that are supported. Rotki does this by maintaining the all_assets.json file.

What are the rules for an asset to get listed? Rather simple actually:

  1. If a user asks for an asset it will be added as long as a price for it exists in a supported price aggregator website (for the moment either in coinpaprika or cryptocompare).
  2. If it’s listed in any of the supported exchanges.
  3. In both of the above cases historical prices need to also be discoverable in a price aggregator website. The lack of price discovery is for example why the STL token PR is blocked as of the time of writing this post.

For assets with identical symbols like KEY we end up with the following in assets.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

"KEY": {
    "ethereum_address": "0x4CC19356f2D37338b9802aa8E8fc58B0373296E7",
    "ethereum_token_decimals": 18,
    "name": "Selfkey",
    "started": 1508803200,
    "symbol": "KEY",
    "type": "ethereum token"
},
"KEY-2": {
    "ethereum_address": "0x4Cd988AfBad37289BAAf53C13e98E2BD46aAEa8c",
    "ethereum_token_decimals": 18,
    "name": "Bihu KEY",
    "started": 1507822985,
    "symbol": "KEY",
    "type": "ethereum token"
},
"KEY-3": {
    "active": false,
    "ended": 1452038400,
    "name": "KeyCoin",
    "started": 1405382400,
    "symbol": "KEY",
    "type": "own chain"
},

Asset Entry

All entries are comprised of a unique asset identifier, which is the key in the above JSON object. The identifier is always the symbol of the asset, and if an asset with the same symbol already exists then the identifier get a number prefix as seen above.

The rest of the attributes will be explored below. The attributes of an asset may change in future iterations of Rotki but as of v1.0.3 it’s the following:

name

This is a required attribute. It’s the name by which the asset is commonly known.

symbol

This is a required attribute. It’s the symbol the asset has. This is not guaranteed to be unique across all the supported assets as is also made clear by the KEY token example.

type

This is a required attribute. It’s the type of asset this is. Determines if it’s a blockchain asset and if yes on which chain it is. Valid values as of writing this post can be seen here:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

asset_type_mapping = {
    'fiat': AssetType.FIAT,
    'own chain': AssetType.OWN_CHAIN,
    'ethereum token and own chain': AssetType.OWN_CHAIN,
    'ethereum token and more': AssetType.ETH_TOKEN_AND_MORE,
    'ethereum token': AssetType.ETH_TOKEN,
    'omni token': AssetType.OMNI_TOKEN,
    'neo token': AssetType.NEO_TOKEN,
    'counterparty token': AssetType.XCP_TOKEN,
    'bitshares token': AssetType.BTS_TOKEN,
    'ardor token': AssetType.ARDOR_TOKEN,
    'nxt token': AssetType.NXT_TOKEN,
    'Ubiq token': AssetType.UBIQ_TOKEN,
    'Nubits token': AssetType.NUBITS_TOKEN,
    'Burst token': AssetType.BURST_TOKEN,
    'waves token': AssetType.WAVES_TOKEN,
    'qtum token': AssetType.QTUM_TOKEN,
    'stellar token': AssetType.STELLAR_TOKEN,
    'tron token': AssetType.TRON_TOKEN,
    'ontology token': AssetType.ONTOLOGY_TOKEN,
    'exchange specific': AssetType.EXCHANGE_SPECIFIC,
    'vechain token': AssetType.VECHAIN_TOKEN,
    'binance token': AssetType.BINANCE_TOKEN,
}

active

The active attribute is optional. If missing, a true value is implied. It signifies if the asset is actively traded in any exchange and has a price.

ended

This is an optional attribute but is required if an asset is not active. If an asset is not active this attribute signifies the timestamp at which all trading (and thus price) ceased for the asset.

ethereum_address

If the type of the asset is ethereum_token or related then it should also contain this entry. This entry contains the EIP55 encoded address of the token’s contract address in the main ethereum chain.

ethereum_token_decimals

Just like the previous entry if the asset is an ethereum token we also need to know its decimals in order to know how to display it to the user.

forked

This is an optional attribute. If the asset is a fork of another asset then the originating asset before the fork should be shown here.

For example BCH has the BTC forked attribute since it’s a fork off Bitcoin.

swapped_for

This is an optional attribute. If the asset ceased to exist but was swapped for another asset then this attribute points to the new asset.

For example SCJX was a counterparty token which got swapped for the STORJ ethereum token.

Conversions

With our list of assets at hand we need to be able to interact with other websites such as price aggregators or exchanges. The idea is that we maintain the Rotki database of assets which is our local “truth”. And when serializing our assets to communicate with another website or deserializing the assets we read from a website we need converters.

This is where the assets/converters.py module comes in.

For all incoming assets we convert to the Rotki format when necessary:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

def asset_from_kraken(kraken_name: str) -> Asset:
    if not isinstance(kraken_name, str):
        raise DeserializationError(f'Got non-string type {type(kraken_name)} for kraken asset')
    name = KRAKEN_TO_WORLD.get(kraken_name, kraken_name)
    return Asset(name)


def asset_from_cryptocompare(cc_name: str) -> Asset:
    return Asset(CRYPTOCOMPARE_TO_WORLD[cc_name])


def asset_from_poloniex(poloniex_name: str) -> Asset:
    if not isinstance(poloniex_name, str):
        raise DeserializationError(f'Got non-string type {type(poloniex_name)} for poloniex asset')

    if poloniex_name in UNSUPPORTED_POLONIEX_ASSETS:
        raise UnsupportedAsset(poloniex_name)

    our_name = POLONIEX_TO_WORLD.get(poloniex_name, poloniex_name)
    return Asset(our_name)


def asset_from_bittrex(bittrex_name: str) -> Asset:
    if not isinstance(bittrex_name, str):
        raise DeserializationError(f'Got non-string type {type(bittrex_name)} for bittrex asset')

    if bittrex_name in UNSUPPORTED_BITTREX_ASSETS:
        raise UnsupportedAsset(bittrex_name)

    name = BITTREX_TO_WORLD.get(bittrex_name, bittrex_name)
    return Asset(name)


def asset_from_binance(binance_name: str) -> Asset:
    if not isinstance(binance_name, str):
        raise DeserializationError(f'Got non-string type {type(binance_name)} for binance asset')

    if binance_name in UNSUPPORTED_BINANCE_ASSETS:
        raise UnsupportedAsset(binance_name)

    if binance_name in RENAMED_BINANCE_ASSETS:
        return Asset(RENAMED_BINANCE_ASSETS[binance_name])

    name = BINANCE_TO_WORLD.get(binance_name, binance_name)
    return Asset(name)

and the asset.py module itself has code to export from the Rotki format to all websites that need conversions:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

def to_kraken(self) -> str:
    return WORLD_TO_KRAKEN[self.identifier]

def to_bittrex(self) -> str:
    return WORLD_TO_BITTREX.get(self.identifier, self.identifier)

def to_binance(self) -> str:
    return WORLD_TO_BINANCE.get(self.identifier, self.identifier)

def to_cryptocompare(self) -> str:
    cryptocompare_str = WORLD_TO_CRYPTOCOMPARE.get(self.identifier, self.identifier)
    # There is an asset which should not be queried in cryptocompare
    if cryptocompare_str is None:
        if self.identifier == 'MRS':
            raise UnsupportedAsset(
                'Marginless is not in cryptocompare. Asking for MRS '
                'will return MARScoin',
            )
        else:
            raise RuntimeError(
                f'Got {self.identifier} as a cryptocompare query but it is '
                f'documented as returning None and is not handled',
            )

    return cryptocompare_str

Whenever a new asset is added and a conversion needs to be included then it is appropriately plugged into any of the above modules.

How to keep this all up to date?

There are two ways to keep all these mappings and the asset database up to date:

  1. Automated CI testing. We have tests for every exchange (for example here is binance) which will warn us when an exchange adds or changes something.

  2. Our users will be warned when they are trying to interact with something that is not supported or when something breaks and as such are incentivized to open issues at the Rotki repo.

Why not use other asset databases?

The reason is simple. That would be adding yet another conversion for us to maintain. For a project like Rotki, where certainty for what each symbol means needs to exist the only way to go is to have our own database as most of the currently known token databases out there are either ethereum specific or incomplete.

The only thing that could work is a standardized database of all crypto assets maintained by multiple different entities.

ITSA is trying to do something similar, but the entire dataset seems to be for “members only” and the way they operate seems to be non-transparent. We believe that any such standardization effort should be open-source and use collaborative tools such as Github.

It would be really neat to see people collaborate on Github in the Rotki database of assets and conversions. If that happens, perhaps it can become a standard that other projects can also use.

Conclusions

Working with multiple tokens across different websites is a hard problem. The Rotki approach took a lot of work to build but now that it’s there maintaining it is not that hard. Our database is open for everyone to use and contribute. If you want to edit information on a token or add a new one simply open a pull request.

Finally remember that Rotki is opensource and self-funded software. If you appreciate what we are doing please consider purchasing a premium subscription in order to help us keep developing and also enjoy premium only features such as analytics and priority support and feature requests.