Geth Version: 1.9.12-stable keeps getting killed...Out of memory: Kill process (geth) score 901 or sacrifice child

Question

I am experiencing issues with my Geth (Version: 1.9.12-stable) nodes. Very often Geth exhausts all memory, which causes a sudden drop in blocks, resulting in the node falling behind. I am running Ubuntu 16.04.6 LTS with CPU-Optimized / 16 GB RAM / 8 vCPUs

The behavior I'm seeing is that Geth starts eating up all the RAM, gets killed off, and is automatically restarted as per my systemd flags. During this process, a bunch of the blocks that were previously in memory get trashed, resulting in my node falling behind.

error message:

[1134203.601725] Out of memory: Kill process 8605 (geth) score 901 or sacrifice child
[1134203.603676] Killed process 8605 (geth) total-vm:10724244kB, anon-rss:7352756kB, file-rss:0kB

/etc/systemd/system/geth.service:

[Unit]
Description=geth

[Service]

ExecStart=/usr/bin/geth --rpcapi eth,web3,debug,txpool,net,shh,db,admin,debug --rpc --ws --wsapi eth,web3,debug,txpool,net,shh,db,admin,debug  --wsorigins localhost --gcmode full --rpcport=8547 --maxpeers 250

User=geth
Restart=always
LimitNOFILE=1048576

[Install]
WantedBy=multi-user.target

Would tweaking the --cache flag help with this issue? In the meanwhile, I have a simple bash script in place which checks for current memory usage and restarts Geth via systemd if the memory usage exceeds 80%. That seems to help.

Thanks in advance!

Answer 1

Your spec is more than enough for a node, maybe try a different version of Geth or try Nethermind - I am running the latter on much lower spec and with your spec you could be synced quite fast on 1.8.10.