I am running 2 miners and 1 non-mining node using docker. Just can't see the 2 miners work in tandem. Each can assemble its DAG. Only seems to work with a single miner.
I am using Geth/peer1/v1.8.16-stable-477eb093/linux-amd64/go1.9.4 from ethereum/go-client:alpine.
Nodes are configured through static-nodes.json and I can reach each node on port 30303 (tested with nc -zv)
[
"enode://b27ff3191f16c7de4c9037eaf419e8d3f25c847b551dd15100c5a375f02032ca5275af29cb29c64258e99c450e0d8825010a13d18c0d404b4c95fb8a1ee9e699@172.17.0.3:30303",
"enode://29bd9b51c2b7f031bf5be15d4cb6028300d4d254ace507bfb5ce283121566f3cd3d96ba303c3ce5c857a7daa29522999cbe545ea64f66c639f4ec526ae3f7cb8@172.17.0.4:30303",
"enode://dc014606c0ee47f9ce917c9497b1a1b2819dfdc4f2d574965b6aa38f447b3c5721f6a63e8942f97cef41f656cce2429a6[email protected]:30303"
]
Genesis has low difficulty
{
"config": {
"chainId" : 11,
"homesteadBlock" : 0,
"eip150Block" : 0,
"eip150Hash" : "0x0000000000000000000000000000000000000000000000000000000000000000",
"eip155Block" : 0,
"eip158Block" : 0,
"byzantiumBlock" : 0,
"ethash" : {}
},
"coinbase" : "0x0000000000000000000000000000000000000000",
"difficulty" : "0x100",
"extraData" : "0x0000000000000000000000000000000000000000000000000000000000000000",
"gasLimit" : "0xffffffff",
"nonce" : "0x0000000000000042",
"mixhash" : "0x0000000000000000000000000000000000000000000000000000000000000000",
"parentHash" : "0x0000000000000000000000000000000000000000000000000000000000000000",
"timestamp" : "0x00",
"alloc" : {}
}
When I start 2 miners, both mining nodes seem to hang after DAG is built. NOTE: see edits below. I can trigger block creation if I stop one miner. Below is peer1 's output (after a restart and killing peer2 to get some 11 blocks created) It just hangs there and nothing happens.
WARN [09-29|16:21:19.909] Sanitizing cache to Go's GC limits provided=1024 updated=660
INFO [09-29|16:21:19.922] Maximum peer count ETH=25 LES=0 total=25
INFO [09-29|16:21:19.948] Starting peer-to-peer node instance=Geth/peer1/v1.8.16-stable-477eb093/linux-amd64/go1.9.4
INFO [09-29|16:21:19.949] Allocated cache and file handles database=/root/.ethereum/geth/chaindata cache=495 handles=1024
INFO [09-29|16:21:20.485] Initialised chain configuration config="{ChainID: 11 Homestead: 0 DAO: <nil> DAOSupport: false EIP150: 0 EIP155: 0 EIP158: 0 Byzantium: 0 Constantinople: <nil> Engine: ethash}"
INFO [09-29|16:21:20.485] Disk storage enabled for ethash caches dir=/root/.ethereum/geth/ethash count=3
INFO [09-29|16:21:20.486] Disk storage enabled for ethash DAGs dir=/root/.ethash count=2
INFO [09-29|16:21:20.486] Initialising Ethereum protocol versions="[63 62]" network=11
INFO [09-29|16:21:20.497] Loaded most recent local header number=11 hash=f506b2…e1a16f td=1444928 age=1h6m40s
INFO [09-29|16:21:20.498] Loaded most recent local full block number=11 hash=f506b2…e1a16f td=1444928 age=1h6m40s
INFO [09-29|16:21:20.498] Loaded most recent local fast block number=11 hash=f506b2…e1a16f td=1444928 age=1h6m40s
INFO [09-29|16:21:20.501] Loaded local transaction journal transactions=0 dropped=0
INFO [09-29|16:21:20.513] Regenerated local transaction journal transactions=0 accounts=0
INFO [09-29|16:21:20.513] Starting P2P networking
INFO [09-29|16:21:20.517] Stats daemon started
INFO [09-29|16:21:20.517] RLPx listener up self="enode://b27ff3191f16c7de4c9037eaf419e8d3f25c847b551dd15100c5a375f02032ca5275af29cb29c64258e99c450e0d8825010a13d18c0d404b4c95fb8a1ee9e699@[::]:30303?discport=0"
INFO [09-29|16:21:20.530] IPC endpoint opened url=/var/run/geth.ipc
INFO [09-29|16:21:20.530] Transaction pool price threshold updated price=1000000000
INFO [09-29|16:21:20.531] Updated mining threads threads=1
INFO [09-29|16:21:20.531] Transaction pool price threshold updated price=1000000000
INFO [09-29|16:21:20.531] Etherbase automatically configured address=0xFDb60C5297aE226dd8B8813cd94995813eEaDd22
INFO [09-29|16:21:20.531] Commit new mining work number=12 sealhash=aeaead…9c6d63 uncles=0 txs=0 gas=0 fees=0 elapsed=176.2µs
Below are the commands I use from a batch file (including the netstats container) Each container has its separate folder mapped into its container.
docker run -d --rm -v /c/Users/P33/playground/p2p/shared/peer1:/root --name peer1 --link netstats ethereum/client-go:alpine --nodiscover --networkid 11 --mine --minerthreads 1 --ipcpath /var/run/geth.ipc --identity peer1 --syncmode "full" --ethstats "peer1:password@ws://netstats:3000"
docker run -d --rm -v /c/Users/P33/playground/p2p/shared/peer2:/root --name peer2 --link netstats ethereum/client-go:alpine --nodiscover --networkid 11 --mine --minerthreads 1 --ipcpath /var/run/geth.ipc --identity peer2 --syncmode "full" --ethstats "peer2:password@ws://netstats:3000"
docker run -d --rm -v /c/Users/P33/playground/p2p/shared/peer3:/root --name peer3 --link netstats -p 8545:8545 ethereum/client-go:alpine --nodiscover --networkid 11 --rpc --rpcapi eth,net,web3,personal --rpcaddr "0.0.0.0" --rpccorsdomain "*" --ipcpath /var/run/geth.ipc --identity peer3 --syncmode "full" --ethstats "peer3:password@ws://netstats:3000"
Any thoughts? TIA!
EDIT 1
When adding a 60s delay between the launch of miner1 and miner2, I observe the DAG process completes with miner1 in the lead; start of mining; then both hang. I then stop and re-launch with the same 60s delay. First miner stays in the lead then reaches a "Mining too far in the future" (at about 80 blocks) and again both miners hang there.
I was hoping to see both miners compete and randomly "win" the next block as they are sharing identical resources on the same laptop...
EDIT 2
I managed to get this small network of 2 miners and one peer going a little longer by doing the following:
- start first miner and pause for 60s
- edit static-nodes.json on first miner to point to other miner and peer
- second miner only points to peer and leaves first miner alone
- peer points to both miners
I had previously each member pointing to the other two (NOK). Having miner 2 point to miner 1 (even after delay) would eventually halt the block generation after 20-30 blocks on this laptop.
With this setup I can reach 190+ blocks... still not sure why...
vmodule=fetcher=6,downloader=6 logs below
miner 1
DEBUG[10-05|08:00:20.959] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:00:40.959] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:01:00.960] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:01:20.962] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:01:40.963] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:02:00.964] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:02:20.965] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:02:40.965] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:03:00.966] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|08:03:20.966] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
miner 2
DEBUG[10-05|07:56:36.651] Queued propagated block peer=36d0dbecc9bbf66d number=192 hash=4fda3f…90ff75 queued=1
DEBUG[10-05|07:56:36.651] Importing propagated block peer=36d0dbecc9bbf66d number=192 hash=4fda3f…90ff75
DEBUG[10-05|07:56:36.758] Queued propagated block peer=36d0dbecc9bbf66d number=193 hash=3c3c32…c9e9b3 queued=1
DEBUG[10-05|07:56:36.758] Importing propagated block peer=36d0dbecc9bbf66d number=193 hash=3c3c32…c9e9b3
DEBUG[10-05|07:56:44.029] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|07:57:04.029] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|07:57:24.030] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|07:57:44.030] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|07:58:04.030] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
DEBUG[10-05|07:58:24.031] Recalculated downloader QoS values rtt=20s confidence=1.000 ttl=1m0s
peer 3 (not mining)
DEBUG[10-05|07:56:33.362] Inserted new block number=189 hash=b0209e…4edc82 uncles=0 txs=0 gas=0 elapsed=140.556ms
INFO [10-05|07:56:33.363] Imported new chain segment blocks=1 txs=0 mgas=0.000 elapsed=143.084ms mgasps=0.000 number=189 hash=b0209e…4edc82 cache=27.26kB
DEBUG[10-05|07:56:33.371] Reinjecting stale transactions count=0
DEBUG[10-05|07:56:35.048] Inserted new block number=190 hash=aecf0c…285ae1 uncles=0 txs=0 gas=0 elapsed=54.696ms
INFO [10-05|07:56:35.048] Imported new chain segment blocks=1 txs=0 mgas=0.000 elapsed=54.858ms mgasps=0.000 number=190 hash=aecf0c…285ae1 cache=27.26kB
DEBUG[10-05|07:56:35.048] Reinjecting stale transactions count=0
DEBUG[10-05|07:56:36.434] Inserted new block number=191 hash=f33bf0…b82196 uncles=0 txs=0 gas=0 elapsed=54.571ms
INFO [10-05|07:56:36.434] Imported new chain segment blocks=1 txs=0 mgas=0.000 elapsed=68.644ms mgasps=0.000 number=191 hash=f33bf0…b82196 cache=27.26kB
DEBUG[10-05|07:56:36.435] Reinjecting stale transactions count=0
DEBUG[10-05|07:56:36.703] Inserted new block number=192 hash=4fda3f…90ff75 uncles=0 txs=0 gas=0 elapsed=12.576ms
INFO [10-05|07:56:36.703] Imported new chain segment blocks=1 txs=0 mgas=0.000 elapsed=12.688ms mgasps=0.000 number=192 hash=4fda3f…90ff75 cache=27.26kB
DEBUG[10-05|07:56:36.703] Reinjecting stale transactions count=0
DEBUG[10-05|07:56:36.802] Inserted new block number=193 hash=3c3c32…c9e9b3 uncles=0 txs=0 gas=0 elapsed=15.051ms
INFO [10-05|07:56:36.802] Imported new chain segment blocks=1 txs=0 mgas=0.000 elapsed=16.326ms mgasps=0.000 number=193 hash=3c3c32…c9e9b3 cache=27.26kB
DEBUG[10-05|07:56:36.802] Reinjecting stale transactions count=0
EDIT 3
I tried different topologies, reducing the connections between peers. Even with { Miner 1 -> Miner 2 -> Peer 3 }, the setup hangs after a while (with delay after starting first miner) or immediately. Not sure at this point where else to look...
EDIT 4
So what I can observe is that miner 1 starts mining blocks as it gets a headstart. Miner 2 starts syncing. Once the situation occurs of "mining too far in the future" both hang at message "wait 2s" - as when I start immediately both miners without delay. I have limited each miner to 1 thread. When I stop miner 2 while it's generating blocks, miner 2 picks up. But when miner 2 is stopped, the first does not pick up again....
EDIT 5
I changed the title from stalling to slow sync... I left the 3 nodes running and one of the miners produced a new block after about 30 minutes. In the screen capture below, 2 miners are at the bottom and a non-mining node at the top. Each miner points to the regular peer. I am on SSD 2ith 16GB RAM - each container gets 2GB and I set the Geth cache to 512KB.
