How to reduce the number of eth_chainId calls when using web3 Python?

Question

I'm using the web3 Python module. Every time I call a function on the blockchain the module seems to call the eth_chainId method first, apparently for validation reasons. This is very annoying due to log being full of these calls and also will be expensive due to the number of extra calls made.

Is there a way to disable these calls?

Answer 1

Before fix this issue

To understand why chain_id requested each time please read about the web3.py middleware and middleware onion. https://web3py.readthedocs.io/en/stable/middleware.html

Root cause (each time you execute a call or send a transaction, chain_id checked to be equal in your connection vs in your call): https://github.com/ethereum/web3.py/blob/68cc94769b531149ad33f6c0e5818a1880fc8486/web3/middleware/validation.py#L60

Using simple cache mechanism (more web3.py way)

Read more about the web3.py cache: https://web3py.readthedocs.io/en/stable/middleware.html#cache

from typing import Optional, Set, cast
from web3.types import RPCEndpoint
from web3.middleware.cache import construct_simple_cache_middleware
from web3.middleware import simple_cache_middleware

# from web3 import Web3
# from web3.providers.rpc import HTTPProvider
# w3 = Web3(HTTPProvider(...))

SIMPLE_CACHE_RPC_CHAIN_ID = cast(
    Set[RPCEndpoint],
    (
        "web3_clientVersion",
        "net_version",
        "eth_chainId",
    ),
)

cache_chain_id_middleware = construct_simple_cache_middleware(
   rpc_whitelist=SIMPLE_CACHE_RPC_CHAIN_ID
)
w3.middleware_onion.add(
   cache_chain_id_middleware,
   name="Cache chain_id"
)

# w3.middleware_onion.add(simple_cache_middleware)

You can use simple_cache_middleware to cache everything out-of-box. Or use more sophisticated time_based_cache_middleware and latest_block_based_cache_middleware.

More gentle version of @kfx solution

from web3.middleware import validation
from web3._utils.rpc_abi import RPC

skip_validation = True
# Store original version of METHODS_TO_VALIDATE
_METHODS_TO_VALIDATE = validation.METHODS_TO_VALIDATE

if skip_validation:
    # whatever methods you want to not validate
    validation.METHODS_TO_VALIDATE = [
         i for i in _METHODS_TO_VALIDATE if i not in [RPC.eth_call, RPC.eth_estimateGas]
    ]
else:
    # restore orignal
    validation.METHODS_TO_VALIDATE = _METHODS_TO_VALIDATE

Note: please be aware that this method will affect on library level. If you have several instances of Web3 (i.e. to with different accounts or to different blockchain ids), this change will affect all connections in current Python process (theoretically threads will be affected as well, but better to double check).

Performance

I was testing with scripts that included many .call() calls and several build_transaction.

"W/o optimisation (11:32)", "With skip validation (11:35)", "Cache chain_id (11:37)" test results is on graphic below.

Comments:

• W/o optimisation - 155 eth_chainId
• With skip validation - 15 calls eth_chainId calls
• Cache chain_id - 4 eth_chainId calls

Answer 2

I found how to disable the validations, although not in a elegant way:

from web3.middleware import validation
validation.METHODS_TO_VALIDATE = []

Answer 3

Simplified and elegant version of @kfx and @valentinmk solitions:

from web3.middleware import ExtraDataToPOAMiddleware, validation
from web3._utils.rpc_abi import RPC

validation.METHODS_TO_VALIDATE = set(validation.METHODS_TO_VALIDATE) - {RPC.eth_call, RPC.eth_estimateGas}