I am experiencing issues with my Geth (Version: 1.9.12-stable) nodes. Very often Geth exhausts all memory, which causes a sudden drop in blocks, resulting in the node falling behind. I am running Ubuntu 16.04.6 LTS with CPU-Optimized / 16 GB RAM / 8 vCPUs
The behavior I'm seeing is that Geth starts eating up all the RAM, gets killed off, and is automatically restarted as per my systemd flags. During this process, a bunch of the blocks that were previously in memory get trashed, resulting in my node falling behind.
[1134203.601725] Out of memory: Kill process 8605 (geth) score 901 or sacrifice child [1134203.603676] Killed process 8605 (geth) total-vm:10724244kB, anon-rss:7352756kB, file-rss:0kB
[Unit] Description=geth [Service] ExecStart=/usr/bin/geth --rpcapi eth,web3,debug,txpool,net,shh,db,admin,debug --rpc --ws --wsapi eth,web3,debug,txpool,net,shh,db,admin,debug --wsorigins localhost --gcmode full --rpcport=8547 --maxpeers 250 User=geth Restart=always LimitNOFILE=1048576 [Install] WantedBy=multi-user.target
Would tweaking the --cache flag help with this issue? In the meanwhile, I have a simple bash script in place which checks for current memory usage and restarts Geth via systemd if the memory usage exceeds 80%. That seems to help.
Thanks in advance!