Congestion tx not found in mempool

Question

I have a parity node running and am watching the txpool for incoming tx's to a specific address. I need to catch the tx in the mempool. The node is in France and the tx's are sent from Ireland. All is working as expected, until there's congestion.

Issue

As soon as Gas price on eth-netstats goes up to 10 Gwei, then the tx isn't found even if it gets confirmed in less than 1 minute. In my last test it was 20Gwei, I sent tx and it was confirmed in 30secs but wasn't in mempool on my node.

Stack

There are 36 peers on the node and cache size is 20GB ram. Here are the parity flags:

--base-path /home/parity/.local/share/io.parity.ethereum/
--cache-size 20480
--jsonrpc-interface all
--jsonrpc-hosts all
--jsonrpc-cors '*'
--ws-interface all
--ws-apis web3,eth,pubsub,net,parity,parity_pubsub,private,traces,rpc,shh,shh_pubsub
--ws-origins all
--ws-hosts all

Solutions

Here are some ideas I've had. I'm presuming under congestion, when my node and its peers's mempools are full, the tx is sent outside of range eg a mempool in China, gets confirmed there, added to block, thus skipping my & peers mempools. But could also be ram issue, a limit on mempool size, parity config etc.

1. Increase Peer List
2. Strip some flags on node to increase ram
3. Static list of peers for western europe and surrounding regions
4. Just give up (eth 2.0 may go smoothly but nothing ever does)

Answer 1

I would suggest first exploring the tx-queue related configuration options. From the Parity documentation:

--tx-queue-mem-limit=[MB] Maximum amount of memory that can be used by the transaction queue. Setting this parameter to 0 disables limiting. (default: 4)

--tx-queue-size=[LIMIT] Maximum amount of transactions in the queue (waiting to be included in next block). (default: 8192)

--tx-queue-per-sender=[LIMIT] Maximum number of transactions per sender in the queue. By default it's 1% of the entire queue, but not less than 16.

--tx-queue-strategy=[S] Prioritization strategy used to order transactions in the queue. S may be: gas_price - Prioritize txs with high gas price (default: gas_price)

If you have not increased your default queue size, your local mempool will max out long before you're hitting limitations such that you need to optimize your node's configuration or the hardware it runs on.

As for limiting peers to western Europe: you don't specify if the transactions you wish to monitor are entering the mempool via a node in Ireland or simply being broadcasted from Ireland. If the latter, it's quite possible they're first sent to a node outside the region you're watching. Anyway most nodes aren't running geo-based peer lists so unless the transaction is being directly broadcasted to the miner, it will be seen by peers across the globe very quickly.

More effective would be a cluster of nodes owned by a single controller that monitors their peer lists and prevents any 2 nodes from sharing a peer. This should ensure you have a much larger sampling of the mempool and so a greater chance of spotting these transactions while they are still pending. Building a setup like this takes some work and can be expensive to run, though.

Also some info on tweeking the peer count: Optimizing Parity Peer Count