How does EtherScan know I am sending to a Shapeshift/Poloniex wallet?
The short answer is that the From: and To: wallet (also known as account or address) details are sent along with the transaction throughout the Ethereum network and EtherScan has a tap (node) into the network.
The long answer follows:
Any computer that wants to receive or send transactions from / to the Ethereum network would either run an Ethereum node or have a connection an Ethereum node.
The main Ethereum node software is geth, the official Go language implementation of the Ethereum protocol. Further information can be found on ethereum.github.io and the source code on github.
EtherScan would be running an Ethereum node.
A transaction is sent to the Ethereum network using the sendTransaction call as documented in Contracts and Transactions. The following command is run directly in geth or through an Application Programming Interface (API):
eth.sendTransaction({from: '0x036a03fc47084741f83938296a1c8ef67f6e34fa', to: '0xa8ade7feab1ece71446bed25fa0cf6745c19c3d5', value: web3.toWei(1, "ether")})
The node software then broadcasts this unconfirmed transaction to the other nodes it is connected to. These nodes then broadcasts this same transaction to the nodes they are connected to. The transaction eventually reaches most, if not all, of the nodes in the Ethereum network.
As EtherScan is running an Ethereum node, it will receive a copy of the unconfirmed transaction. You can view the list of transactions on the right hand side of the EtherScan main page, and a link to PendingTxns.
Here is an example of an unconfirmed transaction from the EtherScan website - the Block Height field is marked (Pending). Note that the From: and To: addresses are specified in the transaction:
Some of the nodes on the Ethereum network are mining nodes. Each of these mining nodes receive the unconfirmed transactions being propagated across the Ethereum network and package them into block. The mining nodes then try to solve a computationally intensive computation to solve a mathematical problem.
The first mining node that solves the mathematical problem packages the solved block including the now confirmed transaction to the rest of the nodes in the Ethereum network. The miner will receive a block reward for "winning" this block.
EtherScan being one of the nodes on the network will receive this block including the now confirmed transaction. You will find the list of blocks on the left side of the EtherScan website. Here is the same unconfirmed transaction now showing up as a confirmed transaction on EtherScan:
How does it identify the source?
The source address is in the From: field in the sent transaction.
Do these exchanges publicly register their addresses (e.g. on IPFS or with an API)?
No. But for you to see the name Poloniex next to the Poloniex's address, they have had to register their name with EtherScan - see How Can I Add My Name Next To Address On Etherscan? .
But doesn't Shapeshift generally create a new address for every transaction?
If you are sending ShapeShift bitcoins to convert to ethers, SS would create a bitcoin address for you to send your bitcoins to. The ethers would be sent to your specified Ethereum address.
If you are sending SS ethers to convert to bitcoins, SS would create a new Ethereum address for you to send your ethers to. The bitcoins would then be sent to your specified bitcoin address.
Is there some sort of API these exchanges are calling for every new address that isn't listed in etherscan docs?
How Accounts Are Created
The accounts were all created at one point in time using one of the Ethereum node software, most likely geth. The command in geth to create an account (see the answer from Deploying the Greeter contract via the geth CLI is not registering in my private blockchain) follows:
user@Kumquat:~/ESE/Deploy$ geth account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase:
Repeat Passphrase:
Address: {730b87e78b07fb2bf1b1e8b127b3353d08d72706}
Programmatically the account creation can be run within the geth using the command:
personal.newAccount("passphrase")
This command can be wrapped in an API like the standard Ethereum JSON RPC API, but it is not due to security considerations.
More details on account creation can be found at Managing your accounts .
The geth Code To Create Accounts
The geth code to create new accounts can be found in accounts/account_manager.go :
func (am *Manager) NewAccount(auth string) (Account, error) {
key, err := am.keyStore.GenerateNewKey(crand.Reader, auth)
if err != nil {
return Account{}, err
}
return Account{Address: key.Address}, nil
}
Which in turn calls crypto/key_store_passphrase.go that does the account creation magic:
func (ks keyStorePassphrase) GenerateNewKey(rand io.Reader, auth string) (key *Key, err error) {
return GenerateNewKeyDefault(ks, rand, auth)
}
How do these 10,000 individual addresses get registered with Etherscan?
Edit: Clarification of my question with an example: Let's say Shapeshift does 10,000 transactions today. That would result in 10,000 addresses being generated. I would hope there is no way to control the format of these addresses (since they are SHA3 hashes), so we are left with 10,000 random address strings. How do these 10,000 individual addresses get registered with Etherscan? I have a hard time believing it is a manual process (this would constitute a full time job for multiple people) and I would guess there's some kind of public lookup table or that they are calling an Etherscan API, but I've never seen any documentation either way.
I'll demonstrate the process of getting the data with the Ethereum geth node client running on from the command line. There are equivalent API calls to run the same commands I type into the console and retrieve the returned data.
beefee@Kumquat:~$ geth console
I0410 00:59:38.739148 18803 database.go:71] Alloted 16MB cache to /home/beefee/.ethereum/chaindata
I0410 00:59:38.926341 18803 database.go:71] Alloted 16MB cache to /home/beefee/.ethereum/dapp
I0410 00:59:38.929816 18803 backend.go:314] Protocol Versions: [63 62 61], Network Id: 1
I0410 00:59:38.930737 18803 backend.go:362] Blockchain DB Version: 3
I0410 00:59:38.936326 18803 blockchain.go:214] Last header: #1303921 [4410d5b0…] TD=12764539073939571484
I0410 00:59:38.936411 18803 blockchain.go:215] Last block: #1303921 [4410d5b0…] TD=12764539073939571484
I0410 00:59:38.936444 18803 blockchain.go:216] Fast block: #1303921 [4410d5b0…] TD=12764539073939571484
I0410 00:59:38.948725 18803 cmd.go:115] Starting Geth/v1.3.6/linux/go1.5.1
I0410 00:59:38.948873 18803 server.go:311] Starting Server
I0410 00:59:40.853966 18803 udp.go:212] Listening, enode://d8a2002783851b7bd966a4d6cff2d91919a81bb42c2d2164c4a1a3c7a75473fba87c62d7190ad3e81bb9d83d5f5c314e9249ef2ce28fcbd9ea146b241d8c512b@[::]:30303
I0410 00:59:40.854203 18803 backend.go:526] Server started
I0410 00:59:40.856840 18803 server.go:552] Listening on [::]:30303
I0410 00:59:40.865236 18803 ipc.go:112] IPC service started (/home/beefee/.ethereum/geth.ipc)
instance: Geth/v1.3.6/linux/go1.5.1
datadir: /home/beefee/.ethereum
coinbase: 0xbeefeebeefeebeefeebeefeebeefeebeefeebeef
at block: 1303921 (Sat, 09 Apr 2016 18:01:23 AEST)
modules: admin:1.0 db:1.0 debug:1.0 eth:1.0 miner:1.0 net:1.0 personal:1.0 shh:1.0 txpool:1.0 web3:1.0
The second last line above shows me that my blockchain data was last synchronised to block 1303921. I now type eth.blockNumber in the geth console and confirm that the last block is 1303921:
> eth.blockNumber
1303921
I type admin.peers and geth tells me that I am connected to one other Ethereum node. This number will increase as my geth instance discovers more peers on the network:
> admin.peers
[{
caps: ["eth/61", "eth/62", "eth/63"],
id: "dbc0024aa19bccca07371ae80efd0ca9fb3393a645aae532039edb43703a8a0a6ceaf00d445dbdbed6f9a6bffdc7a9526bc9f2a45ffbc88dfb65d29359ecd587",
name: "Geth/v1.4.0-unstable/linux/go1.5.1",
network: {
localAddress: "192.168.1.14:54363",
remoteAddress: "13.73.0.80:30303"
},
protocols: {
eth: {
difficulty: 12804217404531728000,
head: "b76e33014471d9929b624a1160a539d76a1397894c9a4aba2ed79c39fe8349f2",
version: 63
}
}
}]
Then the synchronisation starts and my geth instance receives new blockchain blocks since I last synchronised my chain. The first message shows that 4 blocks were imported with 367 transactions, the second 2 blocks with 21 transactions.
> I0410 01:02:23.660101 18824 blockchain.go:1251] imported 4 block(s) (0 queued 0 ignored) including 367 txs in 3.7787239s. #1303925 [9b90c2a9 / 9a9c4463]
I0410 01:02:23.699594 18824 blockchain.go:1251] imported 2 block(s) (0 queued 0 ignored) including 21 txs in 39.366618ms. #1303927 [51c73110 / 6ad7e8ad]
I0410 01:02:23.929353 18824 blockchain.go:1251] imported 1 block(s) (0 queued 0 ignored) including 14 txs in 57.625409ms. #1303928 [2472bfe0 / 2472bfe0]
My blockchain data is synchronised up to 1305673 now. I retrieve the block information and this block has no transactions as shown by the message transactions: [] below:
> eth.blockNumber
1305673
> eth.getBlock(1305673, true)
{
difficulty: 26912099163184,
extraData: "0xd783010306844765746887676f312e352e31856c696e7578",
gasLimit: 4712388,
gasUsed: 0,
hash: "0x760c0430bb58ad52184e065ebcfc8033a56a9d733c3d95218c5bea080f4b668c",
logsBloom: "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
miner: "0x2a65aca4d5fc5b5c859090a6c34d164135398226",
nonce: "0x241f99fc5fe741b5",
number: 1305673,
parentHash: "0x3397535d554535738d9116721661c16a40d050811971303e31b10f1c8f661998",
receiptRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
sha3Uncles: "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
size: 541,
stateRoot: "0x7d45816464427590a2f47a3f142f6b629bab6516869bf1be3e42dbb2eaba0e50",
timestamp: 1460214123,
totalDifficulty: 12812372432726769049,
transactions: [],
transactionsRoot: "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
uncles: []
}
I run the command eth.getBlock("latest", true) and we are now up to block
1305695 where you can see two transactions, each with their from: and to: addresses:
> eth.getBlock("latest", true)
{
...
miner: "0x61c808d82a3ac53231750dadc13c777b59310bd9",
nonce: "0xc0e616c43374dfac",
number: 1305695,
parentHash: "0xa8a5dee1f42ba4d3e211006960dfa554a37a8bc35f83cb4b45f3b7a46f587d4c",
...
timestamp: 1460214363,
totalDifficulty: 12812965182286929610,
transactions: [{
blockHash: "0xf326691704f297404207e476bd076f2cbbefc3247afc5cecae3e47ca4adbdcc8",
blockNumber: 1305695,
from: "0x63a9975ba31b0b9626b34300f7f627147df1f526",
gas: 90000,
gasPrice: 20000000000,
hash: "0xeed4dc863c788bc182cc6c4b83c55904487876a7e25eeca02d8510dd61449e65",
input: "0x",
nonce: 179074,
to: "0x3ed74b5067bc1020bdbeb25fd8628baa29573a55",
transactionIndex: 0,
value: 15044289999999998
}, {
blockHash: "0xf326691704f297404207e476bd076f2cbbefc3247afc5cecae3e47ca4adbdcc8",
blockNumber: 1305695,
from: "0xb1fca483324dbeddbf5862559988d33dc0d6495e",
gas: 53000,
gasPrice: 20000000000,
hash: "0x36a02cfcdc3f1c8dfa8e43e38150fc49e1d8ab7af84a559ce31882070efd0d19",
input: "0x",
nonce: 190,
to: "0xb02a824df54dfb0fa36ca2cb263419b8ff840c79",
transactionIndex: 1,
value: 11933101527912825
}],
transactionsRoot: "0x7f53b96130949ded6a218c74e64982b7024d3ec68f6a9547c0ea49f5bccb03ec",
uncles: []
}
And here is what block 1305695 looks like on EtherScan:
And here is what the two transactions look like on EtherScan:
Getting the data from the geth Ethereum node client is how EtherScan gets data on the the new addresses.
