Summary

The hack that occurred on May 12 2016 to steal 7,218 ethers from Patrick only affects miners where:

  • Incoming RPC connections from the Internet on TCP port 8545 are forwarded to the machine running the Ethereum node software geth;
  • geth is running with the parameters --rpc enabled; and
  • The Ethereum Wallet (Mist) is used on the same geth machine and is used to send a transaction, requiring the geth account to be unlocked for a 2 second period.

The hacker sent a continuous stream (every 2 seconds) of sendTransaction(...) request to Patrick's IP address on port 8545, and this request was forwarded to the geth instance. When Patrick used the Ethereum Wallet to send a transaction, the Ethereum Wallet unlocked Patrick's account for 2 seconds and the hacker's transaction succeeded in moving Patrick's ethers to the hacker's account.



The Details

User Patrick on the DAOHub forum posted the message yesterday with the subject [Urgent]Buying DAO using Mist has hacked.

Patrick used the Ethereum Wallet to transfer 1 ether to test out buying TheDAO tokens on his mining computer. When (so far it seems that) Ethereum wallet unlocked his geth wallet for 2 seconds (via the IPC API, not exposed to the Internet), a bot that was watching his geth wallet activity swooped in to send a transfer instruction to geth (via JSON-RPC, exposed to the Internet) and transferred his remaining 7218 ethers into the hacker's account.


Here is Patrick's account showing the 1 ether transfer before the removal of his remaining balance ~ 17 seconds later (0x95a3ba0aabc6296cbbe3862a889ecb37979da493): enter image description here


Here is the hacker's account showing the 7,218 ethers being split into many fragments of 97.54489533580829 ether transactions (0xc5d431ee2470484b94ce5660aa6ae835346abb19): enter image description here


Here is the bit of code from the Ethereum Wallet mist/interface/client/templates/popupWindows/sendTransactionConfirmation.js #228 that unlocks the geth account for 2 seconds:

    web3.personal.unlockAccount(Session.get('data').from, pw || '', 2, function(e, res){

The 1 ETH transaction occurred at 2016-05-12 03:12:57. The 7218.368874849814 ETH transaction occurred at 2016-05-12 03:29:03, ~ 17 seconds after the 1 ETH transaction, which is the 2 second window + the average ~ 15 second Ethereum blocktime.


After the hack, Patrick ran some analysis on his computer and found network communications every few seconds over the geth RPC port 8545 executing the following commands:

  • eth_accounts - to list the accounts on the geth instance
  • eth_mining - to find out the mining activity on the node
  • miner_setEtherBase - to try to set geth etherbase account but this failed
  • eth_getBalance - to find the balance of Patrick's accounts


There is a question of why the RPC port 8545 was enabled so that RPC communications from the Internet would be forwarded to the geth computer.

Patrick runs GPU miners outside his network and enabled the port forwarding for port 8545 for his GPU miners outside his network, understanding that his account would remain locked for 99.99% of the time. All it took was a 2 second window.



As @tayvano has listed below, Holy shit my eth accounts been hacked documents another exploit via the RPC protocol.

Also note that a hacker with access to the local machine will also be able to access geth's IPC communication during this 2 second unlock period to send rogue transactions.


What are some steps that can be taken to reduce the chances of this occurring?

  • 2
    Another one for a case study when you have time. Rpc related as well. Follow up thread too. m.reddit.com/r/ethereum/comments/3ird55/… – tayvano May 13 '16 at 3:21
  • So is this predominantly caused by running Mist on your Mining rig? Is the layman's takeaway here to never do that? – HodlDwon May 13 '16 at 3:57
  • I heard about it as it was trending hot on reddit, can't remember which subreddit now as it has been removed, or I can't find it anymore. If there's censorship there this is not good, anyone can find me the reddit link? Will repost the news anyway – euri10 May 13 '16 at 4:24
  • Sad story, but this is just the beginning. What's strange in this case is that Patrick looks to be an advanced computer user. But isn't that the matter? Basic users would probably only have mist on a PC, not mining, no open node,... I hope using the basic wallet and browser only will keep a lot of users from being hacked. Anyway ETH haters will probably jump on this news. Sad day... – Nicolas Massart May 13 '16 at 7:26
  • Where were the network communications coming from? It sounds like Patrick's computer was compromised. – Nick Johnson May 13 '16 at 15:01
up vote 30 down vote accepted

(Please edit me. I'm a community wiki)

Summary

  1. Don't allow TCP request on port 8545 from the Internet to be forwarded to your geth machine
  2. Don't enable --rpc without checking that only your GPU mining computers can access TCP port 8545 on your Ethereum node geth computer
  3. Don't run the Ethereum Wallet on the same machine as your geth computer.
  4. If you need a "solo mining pool", consider using the pool backend software that will restrict the instructions that will be forwarded to your geth instance.

EDIT 31/05/2016 - SECURITY ALERT!

From blog.ethereum.org - Security Alert – cpp-ethereum keeps accounts unlocked:

Affected configurations: cpp-ethereum (eth, AlethZero, …) version 1.2.0 up to 1.2.5 (fixed in 1.2.6) Note: Neither “geth” nor “Mist” nor the “Ethereum Wallet” (unless explicitly used together with cpp-ethereum) are affected by this, they lock accounts correctly again.

Severity: High

Possible Attacks: Attackers can spend funds from previously used accounts if they have access to the local machine or to an exposed json-rpc interface.

Details: Due to a bug in cpp-ethereum, accounts stay unlocked once their password has been entered until cpp-ethereum is closed. This includes accounts encrypted with the “master password” entered upon startup or any password entered through Mist. This means that an attacker can spend funds from the account as soon as they have access to the RPC interface. For that to happen, they either need access to the local filesystem or the exposed http-json-rpc interface (not the default setting). Using Mist in “Mist mode” (not the default setting) and navigating to a malicious website also provides that website access to the RPC interface.

Proposed temporary workaround: Restart eth after each transaction, do not expose the json-rpc interface via http and upgrade to version 1.2.6 as soon as binaries are released. A fix has already been merged to the develop branch.



Details

Here are some steps to reduce the chances of getting your Ethereum wallet hacked.


1. Restrict Access To RPC On Your Mining Computers

For most Ethereum miners, you will be using geth to running the blockchain node and ethminer to perform the mining calculations.

If you are running only one geth instance and the ethminers on just one computer, restrict your RPC port to only serve on the local computer. The option --rpcaddr 127.0.0.1 (which is the default setting) will restrict your RPC communications to just that computer.

If you are running only one geth instance on one computer and ethminer on the same computer plus ethminer on other computers that will need to communicate with the one geth instance, you will have to use your computer's local network IP, for example --rpcaddr 192.168.1.123. In this case, it is best set up a firewall to restrict communications via RPC to geth to your mining rigs only.


2. Don't Run The Ethereum Wallet (Mist) On Your Mining geth Node Computer

If you run Mist on your mining node, when you send a transaction with Mist, it will communicate via the IPC socket to instruct geth to unlock your account for 2 seconds.

If you have RPC turned on and this port is accessible from the Internet, scripts and bots will be able to send transactions on your geth node to snatch your coins in the short period of time that Mist unlocks your accounts, as happened to Patrick.

If you have RPC turned off and your mining computer has been compromised, any locally installed malware will be able to use the IPC socket to snatch your coins.

Don't run Mist on your mining computer. Use it on a separate secure computer instead.


3. Check You $HOME/.ethereum/history File And Erase If Necessary

The history of the commands you type into geth ... console is stored in $HOME/.ethereum/history. The following shows what I have just typed into the console:

> eth.mining
true
> personal.unlockAccount(eth.accounts[0], "invalidpassword", 2)
Decryption failed: MAC mismatch
    at InvalidResponse (<anonymous>:-81662:-62)
    at send (<anonymous>:-156322:-62)
    at unlockAccount (<anonymous>:-133322:-62)
    at <anonymous>:1:1

And here are the last two lines of my $HOME/.ethereum/history file:

eth.mining
personal.unlockAccount(eth.accounts[0], "invalidpassword", 2)

Check the contents of your history file using the command:

user@Kumquat:~$ grep unlock $HOME/.ethereum/history
personal.unlockAccount(eth.accounts[0], "invalidpassword", 2)

If you have ever used the command personal.unlockAccount(...), erase your history file:

rm $HOME/.ethereum/history

By default the directory in which your history file is located is only readable by your own user. But if your account is compromised, your password is in your history file:

user@Kumquat:~$ ls -al | grep .ethereum
drwx------   9 user user       4096 May 14 00:28 .ethereum


4. Move Your Mining Rewards If They Are Too Much To Lose

Your mining computer is continually communication with other programs on the Internet and is at a higher-than-usual risk of getting compromised.

Hackers can easily find Ethereum nodes on the Internet and your computer may be targeted. Move your mining rewards to a more secure Ethereum account to reduce the size of your losses if you do get hacked.


5. Reduce Your Attack Surface If Possible

Run the server version of Ubuntu Linux if possible. It is a little bit harder to install, especially the GPU drivers, but you have a lower chance of getting hacked as fewer services run in this version compared to the desktop version.

  • Ubuntu Desktop has many more bits of software running continually with exposed ports. If you do have to use Ubuntu desktop, turn off unnecessary services, don't install untrusted software. You may want to use the privacy settings to reduce any leakage of information:

    • Settings -> Security & Privacy -> Files and Applications OFF
    • Settings -> Security & Privacy -> Search OFF
    • Settings -> Security & Privacy -> Diagnostics ALL OFF
    • /etc/default/appport. Set enabled=0
  • Macs and Windows continually phone home and very likely have backdoors for Apple and Microsoft respectively. Even if Apple and Microsoft have little incentive to expose themselves by utilising the backdoors, the continuous chatter makes it hard to detect abnormal communications between your server and other computers.


6. Secure Your Network

  • Check you have a relatively secure ADSL modem/router or networking device. Backdoors are sometimes installed by the manufacturer and your Internet Service Provider (if you got your modem from them as part of the service). While they may not want to cause any malicious damage by utilising the backdoors, hacker have exploited some of the poorly secured networking devices. For example, see Cisco, Linksys, Netgear backdoors.
  • Run your mining computers in the DMZ (demilitarised zone) to separate your Internet exposed mining computers from your other networked devices. You should test your networking device's DMZ functionality if you use this as I have come across some networking devices that don't partition the DMZ properly.
  • Firewall your mining computers so that the only incoming port from the Internet is TCP and UDP 30303, and if you have RPC switched on for your geth instance, only allow communications on the RPC port from your mining computers.


7. Don't Use The User root (Or User With Administrator Privileges) To Run Your Miners

If your Internet facing programs (geth) has a vulnerability, a hacker can cause more damage if the program is running with administrator privileges.


8. Check Your Port Forwarding Rules On Your Router Or Firewall

Only the port 30303 for TCP and UDP traffic should be forwarded by your modem or firewall to your geth node computer. Port 8545 (or whatever you configured for your RPC port) should NOT be forwarded to your geth node computer.

My firewall rules are for my 2 mining rigs (.120 has the geth instance, .121 is just a GPU miner, .111.* is my DMZ):

* Allow Internet to DMZ geth machine TCP 30303
    Protocol TCP, Source Any, Destination 192.168.111.120:30303
* Allow Internet to DMZ geth machine UDP 30303
    Protocol UDP, Source Any, Destination 192.168.111.120:30303
* Allow Ethereum RPC 8545 traffic from GPU miner to geth machine
    Protocol TCP, Source 192.168.111.121, Destination 192.168.111.120:8545
* Allow SSH traffic to mining rigs
    Protocol TCP, Source 192.168.1.*, Destination 192.168.111.*:22
* Block everything else
    Protocol *, Source Any, Destination 192.168.111.*:*


9. Test For Open Ports On Your Internet Connection

Use a service like GRC to probe for the open ports on your Internet connection. Click on grc.com, click on Shields UP!!, then click on ShieldsUP! again. Read the text and click Proceed.

On the web page shown below, I've typed in ports 8545, 30303 and clicked User Specified Custom Port Probe:

enter image description here

The probe results are shown below - Port 8545 (geth RPC) is not open to the Internet while port 30303 (Ethereum P2P port) is: enter image description here

It's probably a good idea to test All Service Ports (Ports 0 to 1055) at the same time to confirm which ports you have open. The following results show that I only have port 443 (HTTPS) open: enter image description here


10. Opening RPC Ports For Miners Outside Your Network (Your Own Solo Mining Pool With Your Own Distributed Miners)

Don't open up your RPC port 8545 if you have GPU miners outside your network.

Your alternatives if you have GPU miners outside your network with your geth instance:

  1. Run geth locally within each of your GPU miner's networks pointing to the same coinbase account if necessary; or
  2. Consider running the ether-pool component of open-ethereum-pool. You don't have to run the web frontend, the unlocking or the payout modules. Only safe instruction from/to the GPU miners to/from geth passes through the mining pool software. One advantage of running this software is that you get a better performance from the stratum mining proxy protocol. The ability to mine with a particular coinbase (payout address) is not contingent on the mining node having access to the coinbase's private keys; you can generate the address and private key on another machine and set the mining node's coinbase to the generated address. This reduces damage from a compromised node: past payouts cannot be stolen by merely compromising the node.
  • Please don't post work in progress. Use a markdown sandbox somewhere or create a sandbox on meta. – Afri May 13 '16 at 7:15
  • 1
    Nice list, could you add some more details on port 8545 exposure? Thanks. – Nicolas Massart May 13 '16 at 7:20

0. Read the official security advisory.

ethereum blog: [security alert] insecurely configured geth can make funds remotely accessible

1. Never turn on RPC on hot wallets. Even with IP and CORS limited to localhost. Just don't.

geth                # rpc is disabled by default
eth                 # rpc is disabled by default
parity              # rpc is disabled by default

2. Never turn on IPC on a machine you don't feel you have complete control about the security. Or use parity.

geth --ipcdisable   # ipc is enabled by default, turn it off
geth --no-ipc       # ipc is enabled by default, turn it off
parity              # parity has not ipc (yet)

3. Don't use accounts to store huge amounts of ether. Use multisignature contracts.

4. Don't use online computers to store huge amounts of ether. User air gapped machines or paper wallets.

  • The problem about multisig wallet contract is that it was advertised a long long time ago to be experimental. So probably some users still rely on basic account to store ether because they're feared of using multisig wallets and face a bug. – Nicolas Massart May 13 '16 at 7:31
  • Note that air gapped pc with a copy of myetherwallet JS site on it looks to be the most secure way to generate offline transaction while being so simple that it reduces the fear to make a mistake (even if there is still some anxiety when moving large amounts, but same as when you go to your bank) – Nicolas Massart May 13 '16 at 7:43
  • 1
    Parity has ipc :) – Marius Darila May 13 '16 at 9:24
  • IPC in parity must be new / in development. How to disable it? – Afri May 13 '16 at 9:26
  • it's disabled by default, rpc is enabled by default. --rpc Does nothing; JSON-RPC is on by default now. – Marius Darila May 13 '16 at 9:31

I imagine perhaps that Mist does not use signed transactions but rather unlocks the an account to send unsigned transactions. So perhaps sending signed transactions using web3 on a NodeJS command line console or by using a script would allow one to send a transaction without the risk of attack. This assumes that accounts are never unlocked when sending signed transactions but I am not sure that is the case.

Comments would be appreciated

protected by Afri May 22 '17 at 10:07

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.