I'm trying to run a geth full node with archiving for development\learning and custom analytics. I have a physical Windows server running HyperV where I carved out the following VM specs:
- 24 vCPUs
- 128 GB DDR3 RAM
- 80GB OS drive on local NVME storage
- 8TB Tiered Storage Drive (see below)*
- Ubuntu 20.04
- Geth 1.10.2-stable
According to etherscan stats a full node with archive today is about 7.2TB of disk.
*Unfortunately I don't have 8TB worth of SSD, but I have a NAS with 24TB of HDD in a raid which can throughput 116MB\s (~about max speed of 1GB nic), and another locally attach 1TB nvme. Previous posts indicate that running a full node on HDD is not sufficient or possible to keep up, but what about tiered storage?
Since my host is windows, I'm using storage spaces to combine the local 1TB nvme with 8TB iscsi LUN with SSD,HDD tiering. I then attached the tiered disk to VM as /eth1 to host the chaindata.
I'm running geth via following command:
./geth --datadir /eth1/data/ --datadir.ancient /eth1/data/geth/chaindata/ancient --mainnet --syncmode full --gcmode archive --txlookuplimit=0 --cache.preimages --cache=8192
The tiering functionality is a bit of a black box to me, and i'm not sure when certain chaindata is considered hot or cold to storage spaces. I think all initial writes go to LUN. I'd be nice if I could just configure geth to say use 950GB of recent data on drive X, and store\move all other chain data to drive Y, which I assume is kind of how the freezer works? I saw there is a
datadir.ancient parameter but not sure when and how that data is moved there.