I have an answer for this.
NodeJS Web3 compiles the Solidity contract into an artifact (a JSON file), and then loads it into the testing environment.
With Python Web3, I am loading the binary itself (which I have compiled myself via
solc) into the testing environment.
But while NodeJS Web3 compiles with no optimization whatsoever, when I compile with
solc, I do apply optimization (using
--optimize --optimize-runs 5000000).
Hence the huge difference in gas consumption.
I do specify optimization in my Truffle configuration file:
So the difference is not purely because of "with optimization" vs "without optimization", but perhaps because of the fact that nevertheless, two different compilers (and two different compilation outputs - artifacts vs binaries) are being tested.
The explanation above is wrong.
I have conducted a thorough comparison, and both utilities give identical gas estimations.
The problem was in my own testing method:
In the NodeJS test I estimated gas before I commenced with the actual transaction, while in the Python test I did so afterwards.
Of course, the correct way is to do it beforehand, because otherwise, you are measuring the gas consumption of an essentially different state (i.e., not of the state in which the transaction takes place).