How to extract or create a message with web3.js to sign with an external hardware device?

Question

I'm using the Web3 library. I want to extract a signature hash from a Web3 transaction to send to an external hardware device, sign it there, then use the signature I get from there to broadcast my transaction on Ropsten.

On their GitHub page I found a test example:

https://github.com/ethereum/web3.js/blob/2.x/packages/web3-eth/tests/src/signers/TransactionSignerTest.js

On lines 26-37 I checked how it's signed:

    const tx = {
        chainId: '0x11',
        data: '0x',
        gas: '0x7530',
        gasPrice: '0x3b9aca00',
        nonce: '0x3',
        to: '0xb414031Aa4838A69e27Cb2AE31E709Bcd674F0Cb',
        value: '0x64'
    };

    await expect(
        transactionSigner.sign(tx, '3a0ce9a362c73439adb38c595e739539be1e34d19c5e9f04962c101c86bd7616')

Apparently transactionSigner does the signing:

https://github.com/ethereum/web3.js/blob/aaf26c8806bc9fb60cf6dcb6658104963c6c7fc7/packages/web3-eth/src/signers/TransactionSigner.js

In that class, at lines 62-63:

    const ethTx = new EthereumTx(transaction);
    ethTx.sign(Buffer.from(privateKey, 'hex'));

So they're actually using another library, ehtereumjs-tx:

https://github.com/ethereumjs/ethereumjs-tx/blob/master/src/transaction.ts

At lines 269-270 we find:

const msgHash = this.hash(false)
const sig = ecsign(msgHash, privateKey)

This is what I want, the "msgHash" variable. How can I get to it without needing to butcher half the library and rewrite it manually? Or is there a simple way for me to construct it myself, and if so, how exactly?

For example, in bitcoinlib-js, there is a SignerAsync object, which you can construct yourself and simply assign its "sign(hash: Buffer)" property your own custom signing function. Bitcoinlib automatically throws in this "hash" thing that I need into the sign(hash: Buffer) and I simply pass it to my external Hardware device. Is there anything similar in Web3?

Answer 1

Turns out the trick is in not calling Web3's transactionSigner(), it is too high level. Call the ethereumjs-tx library directly:

// Regular ethereum tx object
const txData = {
    chainId: '0x11',
    data: '0x',
    gas: '0x7530',
    gasPrice: '0x3b9aca00',
    nonce: '0x3',
    to: '0xb414031Aa4838A69e27Cb2AE31E709Bcd674F0Cb',
    value: '0x64'
};

const ethTX = new Transaction(txData, {
    chain: ETH_ROPSTEN_CHAINID // Ropsten testnet chainID = 3
});

// Hash the ethereumjs-tx Transaction object
const transactionHashToSign = ethTX.hash(false);

// Now send this to your external hardware device
const signature: string = hardwareDevice.sign(transactionHashToSign);

Then you must extract the R, S and V hexstrings from this signature yourself and set them to the ethTX object, as now you haven't used Web3's or ethereumjs-tx's "sign" functions that handle this for you.

For me, I produce a DER-signature from the hardware device. Extracting R and S for that is well-documented, but depending on your hardware device, V is not always part of the signature. You have to compute it yourself (I used C++'s botan library). You can do that by extracting the public key from the signature and comparing that with the public key you signed it with (see https://crypto.stackexchange.com/questions/98316/possible-to-directly-calculate-the-recovery-id-from-a-msg-signature-and-public). Basically the only way is to try public key extraction for different v values and see which result matches your public key, then that's the correct v value.

Once you've done that, set them to your "ethTX" object:

  ethTX.r = Buffer.from(r, 'hex');
  ethTX.s = Buffer.from(s, 'hex');
  ethTX.v = Buffer.from(v, 'hex');

Then you can do

const rawTx: string = '0x' + ethTX.serialize().toString('hex');

and broadcast this raw transaction.