5

I have noticed many transactions on Etherscan that indicate one of the parties is a certain type of wallet (e.g. Poloniex or Shapeshift). Here is an example.

How does it identify the source? Do these exchanges publicly register their addresses (e.g. on IPFS or with an API)?

Edit: Clarification of my question with an example: Let's say Shapeshift does 10,000 transactions today. That would result in 10,000 addresses being generated. I would hope there is no way to control the format of these addresses (since they are SHA3 hashes), so we are left with 10,000 random address strings. How do these 10,000 individual addresses get registered with Etherscan? I have a hard time believing it is a manual process (this would constitute a full time job for multiple people) and I would guess there's some kind of public lookup table or that they are calling an Etherscan API, but I've never seen any documentation either way.

  • Regarding your edit, could you confirm that you are talking about address strings in the form of "0x1234abcd...fedc" and not a name like "joesaddress"? – The Officious BokkyPooBah Apr 9 '16 at 14:05
  • By "address" I am referring to the ethereum address (a hash string). By "register" I mean the process of mapping an ethereum address to a name (e.g. "Poloniex wallet"). – ethereum_alex Apr 9 '16 at 16:00
4

When you are seeing the name next to the address, it is a manual process of associating a text string with that address. It's that simple. They have a metadata field that they can add to any address. This process is not via an API or across the entire Ethereum network. Each blockchain explorer (etherscan, etherchain, and live.ether.camp) do it differently and have different accounts labeled.

As you don't seem to take my answer as correct, here is EtherScan's answer:

Submit links to your official site under the comments section and Verify the Source code of your contract.. and then wait :-)

In fact, after answering that question I wanted to make sure it was correct, so I commented on MyEtherWallet's donation account address. A couple days later, Matt @ EtherScan upvoted my comment and now we have a tag.

Furthermore, not all addresses on Shapeshift are listed. I just looked at one of my accounts and the unique address I sent the ETH to is not labeled as Shapeshift.

However, the address that ETH was sent from (in this case, I didn't the correct amount of ETH so it was returned to me) is labeled: http://etherscan.io/address/0x120a270bbc009644e35f0bb6ab13f95b8199c4ad

This seems to be their Ethereum hot wallet (although this is most likely changing due to the security breach on 4/7) and so it can be easily labeled. I looked around and couldn't find any other addresses labeled as Shapeshift, so I think that your assumption that every address ShapeShift generates in labeled Shapeshift is wrong.

  • Okay manual entry is definitely more reasonable if ShapeShift reuses addresses and Etherscan doesn't tag all the one time throwaways. Sorry to be a doubter; just wanted to see some sort of official word on this because it all seemed very mysterious to me. – ethereum_alex Apr 9 '16 at 21:17
7

How does EtherScan know I am sending to a Shapeshift/Poloniex wallet?

The short answer is that the From: and To: wallet (also known as account or address) details are sent along with the transaction throughout the Ethereum network and EtherScan has a tap (node) into the network.

The long answer follows:

  • Any computer that wants to receive or send transactions from / to the Ethereum network would either run an Ethereum node or have a connection an Ethereum node.

  • The main Ethereum node software is geth, the official Go language implementation of the Ethereum protocol. Further information can be found on ethereum.github.io and the source code on github.

  • EtherScan would be running an Ethereum node.

  • A transaction is sent to the Ethereum network using the sendTransaction call as documented in Contracts and Transactions. The following command is run directly in geth or through an Application Programming Interface (API):

    eth.sendTransaction({from: '0x036a03fc47084741f83938296a1c8ef67f6e34fa', to: '0xa8ade7feab1ece71446bed25fa0cf6745c19c3d5', value: web3.toWei(1, "ether")})
    
  • The node software then broadcasts this unconfirmed transaction to the other nodes it is connected to. These nodes then broadcasts this same transaction to the nodes they are connected to. The transaction eventually reaches most, if not all, of the nodes in the Ethereum network.

  • As EtherScan is running an Ethereum node, it will receive a copy of the unconfirmed transaction. You can view the list of transactions on the right hand side of the EtherScan main page, and a link to PendingTxns.

  • Here is an example of an unconfirmed transaction from the EtherScan website - the Block Height field is marked (Pending). Note that the From: and To: addresses are specified in the transaction: Pending Transaction 0xb68c6c0c1af99b0e08e05e768523256bbc2e6c2896f6cb7eb9ed5042dc2232c3

  • Some of the nodes on the Ethereum network are mining nodes. Each of these mining nodes receive the unconfirmed transactions being propagated across the Ethereum network and package them into block. The mining nodes then try to solve a computationally intensive computation to solve a mathematical problem.

  • The first mining node that solves the mathematical problem packages the solved block including the now confirmed transaction to the rest of the nodes in the Ethereum network. The miner will receive a block reward for "winning" this block.

  • EtherScan being one of the nodes on the network will receive this block including the now confirmed transaction. You will find the list of blocks on the left side of the EtherScan website. Here is the same unconfirmed transaction now showing up as a confirmed transaction on EtherScan: Confirmed Transaction 0xb68c6c0c1af99b0e08e05e768523256bbc2e6c2896f6cb7eb9ed5042dc2232c3



How does it identify the source? The source address is in the From: field in the sent transaction.



Do these exchanges publicly register their addresses (e.g. on IPFS or with an API)?

No. But for you to see the name Poloniex next to the Poloniex's address, they have had to register their name with EtherScan - see How Can I Add My Name Next To Address On Etherscan? .



But doesn't Shapeshift generally create a new address for every transaction?

If you are sending ShapeShift bitcoins to convert to ethers, SS would create a bitcoin address for you to send your bitcoins to. The ethers would be sent to your specified Ethereum address.

If you are sending SS ethers to convert to bitcoins, SS would create a new Ethereum address for you to send your ethers to. The bitcoins would then be sent to your specified bitcoin address.



Is there some sort of API these exchanges are calling for every new address that isn't listed in etherscan docs?

How Accounts Are Created

The accounts were all created at one point in time using one of the Ethereum node software, most likely geth. The command in geth to create an account (see the answer from Deploying the Greeter contract via the geth CLI is not registering in my private blockchain) follows:

user@Kumquat:~/ESE/Deploy$ geth account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase: 
Repeat Passphrase: 
Address: {730b87e78b07fb2bf1b1e8b127b3353d08d72706}

Programmatically the account creation can be run within the geth using the command:

personal.newAccount("passphrase")

This command can be wrapped in an API like the standard Ethereum JSON RPC API, but it is not due to security considerations.

More details on account creation can be found at Managing your accounts .


The geth Code To Create Accounts

The geth code to create new accounts can be found in accounts/account_manager.go :

func (am *Manager) NewAccount(auth string) (Account, error) {
    key, err := am.keyStore.GenerateNewKey(crand.Reader, auth)
    if err != nil {
            return Account{}, err
    }
    return Account{Address: key.Address}, nil
}

Which in turn calls crypto/key_store_passphrase.go that does the account creation magic:

func (ks keyStorePassphrase) GenerateNewKey(rand io.Reader, auth string) (key *Key, err error) {
    return GenerateNewKeyDefault(ks, rand, auth)
}



How do these 10,000 individual addresses get registered with Etherscan?

Edit: Clarification of my question with an example: Let's say Shapeshift does 10,000 transactions today. That would result in 10,000 addresses being generated. I would hope there is no way to control the format of these addresses (since they are SHA3 hashes), so we are left with 10,000 random address strings. How do these 10,000 individual addresses get registered with Etherscan? I have a hard time believing it is a manual process (this would constitute a full time job for multiple people) and I would guess there's some kind of public lookup table or that they are calling an Etherscan API, but I've never seen any documentation either way.

I'll demonstrate the process of getting the data with the Ethereum geth node client running on from the command line. There are equivalent API calls to run the same commands I type into the console and retrieve the returned data.

beefee@Kumquat:~$ geth console
I0410 00:59:38.739148   18803 database.go:71] Alloted 16MB cache to /home/beefee/.ethereum/chaindata
I0410 00:59:38.926341   18803 database.go:71] Alloted 16MB cache to /home/beefee/.ethereum/dapp
I0410 00:59:38.929816   18803 backend.go:314] Protocol Versions: [63 62 61], Network Id: 1
I0410 00:59:38.930737   18803 backend.go:362] Blockchain DB Version: 3
I0410 00:59:38.936326   18803 blockchain.go:214] Last header: #1303921 [4410d5b0…] TD=12764539073939571484
I0410 00:59:38.936411   18803 blockchain.go:215] Last block: #1303921 [4410d5b0…] TD=12764539073939571484
I0410 00:59:38.936444   18803 blockchain.go:216] Fast block: #1303921 [4410d5b0…] TD=12764539073939571484
I0410 00:59:38.948725   18803 cmd.go:115] Starting Geth/v1.3.6/linux/go1.5.1
I0410 00:59:38.948873   18803 server.go:311] Starting Server
I0410 00:59:40.853966   18803 udp.go:212] Listening, enode://d8a2002783851b7bd966a4d6cff2d91919a81bb42c2d2164c4a1a3c7a75473fba87c62d7190ad3e81bb9d83d5f5c314e9249ef2ce28fcbd9ea146b241d8c512b@[::]:30303
I0410 00:59:40.854203   18803 backend.go:526] Server started
I0410 00:59:40.856840   18803 server.go:552] Listening on [::]:30303
I0410 00:59:40.865236   18803 ipc.go:112] IPC service started (/home/beefee/.ethereum/geth.ipc)
instance: Geth/v1.3.6/linux/go1.5.1
 datadir: /home/beefee/.ethereum
coinbase: 0xbeefeebeefeebeefeebeefeebeefeebeefeebeef
at block: 1303921 (Sat, 09 Apr 2016 18:01:23 AEST)
modules: admin:1.0 db:1.0 debug:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 shh:1.0 txpool:1.0 web3:1.0

The second last line above shows me that my blockchain data was last synchronised to block 1303921. I now type eth.blockNumber in the geth console and confirm that the last block is 1303921:

> eth.blockNumber
1303921

I type admin.peers and geth tells me that I am connected to one other Ethereum node. This number will increase as my geth instance discovers more peers on the network:

> admin.peers
[{
    caps: ["eth/61", "eth/62", "eth/63"],
    id: "dbc0024aa19bccca07371ae80efd0ca9fb3393a645aae532039edb43703a8a0a6ceaf00d445dbdbed6f9a6bffdc7a9526bc9f2a45ffbc88dfb65d29359ecd587",
    name: "Geth/v1.4.0-unstable/linux/go1.5.1",
    network: {
      localAddress: "192.168.1.14:54363",
      remoteAddress: "13.73.0.80:30303"
    },
    protocols: {
      eth: {
        difficulty: 12804217404531728000,
        head: "b76e33014471d9929b624a1160a539d76a1397894c9a4aba2ed79c39fe8349f2",
        version: 63
      }
    }
}]

Then the synchronisation starts and my geth instance receives new blockchain blocks since I last synchronised my chain. The first message shows that 4 blocks were imported with 367 transactions, the second 2 blocks with 21 transactions.

> I0410 01:02:23.660101   18824 blockchain.go:1251] imported 4 block(s) (0 queued 0 ignored) including 367 txs in 3.7787239s. #1303925 [9b90c2a9 / 9a9c4463]
I0410 01:02:23.699594   18824 blockchain.go:1251] imported 2 block(s) (0 queued 0 ignored) including 21 txs in 39.366618ms. #1303927 [51c73110 / 6ad7e8ad]
I0410 01:02:23.929353   18824 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 14 txs in 57.625409ms. #1303928 [2472bfe0 / 2472bfe0]

My blockchain data is synchronised up to 1305673 now. I retrieve the block information and this block has no transactions as shown by the message transactions: [] below:

> eth.blockNumber
1305673
> eth.getBlock(1305673, true)
{
  difficulty: 26912099163184,
  extraData: "0xd783010306844765746887676f312e352e31856c696e7578",
  gasLimit: 4712388,
  gasUsed: 0,
  hash: "0x760c0430bb58ad52184e065ebcfc8033a56a9d733c3d95218c5bea080f4b668c",
  logsBloom: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  miner: "0x2a65aca4d5fc5b5c859090a6c34d164135398226",
  nonce: "0x241f99fc5fe741b5",
  number: 1305673,
  parentHash: "0x3397535d554535738d9116721661c16a40d050811971303e31b10f1c8f661998",
  receiptRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  sha3Uncles: "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
  size: 541,
  stateRoot: "0x7d45816464427590a2f47a3f142f6b629bab6516869bf1be3e42dbb2eaba0e50",
  timestamp: 1460214123,
  totalDifficulty: 12812372432726769049,
  transactions: [],
  transactionsRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
  uncles: []
}

I run the command eth.getBlock("latest", true) and we are now up to block 1305695 where you can see two transactions, each with their from: and to: addresses:

> eth.getBlock("latest", true)
{
  ...
  miner: "0x61c808d82a3ac53231750dadc13c777b59310bd9",
  nonce: "0xc0e616c43374dfac",
  number: 1305695,
  parentHash: "0xa8a5dee1f42ba4d3e211006960dfa554a37a8bc35f83cb4b45f3b7a46f587d4c",
  ...
  timestamp: 1460214363,
  totalDifficulty: 12812965182286929610,
  transactions: [{
      blockHash: "0xf326691704f297404207e476bd076f2cbbefc3247afc5cecae3e47ca4adbdcc8",
      blockNumber: 1305695,
      from: "0x63a9975ba31b0b9626b34300f7f627147df1f526",
      gas: 90000,
      gasPrice: 20000000000,
      hash: "0xeed4dc863c788bc182cc6c4b83c55904487876a7e25eeca02d8510dd61449e65",
      input: "0x",
      nonce: 179074,
      to: "0x3ed74b5067bc1020bdbeb25fd8628baa29573a55",
      transactionIndex: 0,
      value: 15044289999999998
  }, {
      blockHash: "0xf326691704f297404207e476bd076f2cbbefc3247afc5cecae3e47ca4adbdcc8",
      blockNumber: 1305695,
      from: "0xb1fca483324dbeddbf5862559988d33dc0d6495e",
      gas: 53000,
      gasPrice: 20000000000,
      hash: "0x36a02cfcdc3f1c8dfa8e43e38150fc49e1d8ab7af84a559ce31882070efd0d19",
      input: "0x",
      nonce: 190,
      to: "0xb02a824df54dfb0fa36ca2cb263419b8ff840c79",
      transactionIndex: 1,
      value: 11933101527912825
  }],
  transactionsRoot: "0x7f53b96130949ded6a218c74e64982b7024d3ec68f6a9547c0ea49f5bccb03ec",
  uncles: []
}

And here is what block 1305695 looks like on EtherScan:

EtherScan Block 1305695

And here is what the two transactions look like on EtherScan:

EtherScan Transactions For Block 1305695

Getting the data from the geth Ethereum node client is how EtherScan gets data on the the new addresses.

  • But doesn't Shapeshift generally create a new address for every transaction? Is there some sort of API these exchanges are calling for every new address that isn't listed in etherscan docs? – ethereum_alex Apr 9 '16 at 0:11
  • I've update the answer with further information. If you have more questions you may want to consider creating new questions as the moderators of this site prefer that individual questions are asked separately so they are searchable. – The Officious BokkyPooBah Apr 9 '16 at 1:05
  • @BokkyPooBah imho a lot of this answer seems to miss the question :) Question wants to know did polo tell each of the explorers to register their address? Or does polo register somewhere, then the explorers manually pick up and add to their sites? If polo/shift uses new address each time, how can etherscan know this and keep up? – eth Apr 9 '16 at 1:46
  • Improved now. Please review. Thx. – The Officious BokkyPooBah Apr 9 '16 at 4:03
  • This is a good overview of how a third party can index the blockchain, but there's still a "magic" layer that I'm not seeing any talk of. I have updated the question. – ethereum_alex Apr 9 '16 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.