3

I am attempting to verify a signature returned by the eth_sign JSON-RPC method.

https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_sign

The verification is done using the following python code and test vectors. geth is running behind the scenes, being accessed over RPC with all API's enabled.

  • Private Key being used for testing: 0x5e95384d8050109aab08c1922d3c230739bc16976553c317e5d0b87b59371f2a
  • Message being signed: 1234567890abcdefghijklmnopqrstuvwxyz
  • Hash of message: 0x089c33d56ed10bd8b571a0f493cedb28db1ae0f40c6cd266243001528c06eab3
  • Signature Returned: 0xcb8a42e69ff562a4391ef5bb6346bdb3dd144e96c6462ea950a71fb707d2816b1b636277fcef51ead6139376cef5d5d244d677da06d655c42727de3e340a5bb701
  • Public Key extracted from returned signature: 0x15f8c20c46c734b12527925392937890e7eba985
from sha3 import sha3_256
from secp256k1 import PrivateKey, PublicKey, ALL_FLAGS

from bitcoin import encode_pubkey, ecdsa_raw_sign

from ethereum.utils import privtoaddr

from eth_tester_client.utils import (
    mk_random_privkey,  # generates a random byte-string of length 32
    encode_address,  # encodes an ethereum address into it's hex representation
    encode_data,  # encodes a string into it's hex representation
    encode_number,  # encodes an integer into it's big_endian hex representation
    coerce_return_to_bytes,  # decorates a function, forcing it's return value to bytes.
)

from web3.web3.rpcprovider import TestRPCProvider
from web3.utils.encoding import (
    force_bytes,  # forces a string to bytes
    encode_hex,  # converts a string into `hex`
    decode_hex,  # converts a `hex` encoded value to `bytes`
    add_0x_prefix,  # adds a `0x` prefix if not present
    remove_0x_prefix,  # removes a `0x` prefix if present
)


@coerce_return_to_bytes
def sha3(s):
    return add_0x_prefix(sha3_256(s).hexdigest())

# sanity check that the correct sha3 (keccak) is being used.
assert sha3(b'') == b'0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470'


def test_eth_sign(web3):
    private_key_hex = b'0x5e95384d8050109aab08c1922d3c230739bc16976553c317e5d0b87b59371f2a'
    private_key = decode_hex(private_key_hex)

    # This imports the private key into the running geth instance and unlocks
    # the account so that it can sign things.
    # `0xa5df35f30ba0ce878b1061ae086289adff3ba1e0`
    address = web3.personal.importRawKey(private_key, "password")
    web3.personal.unlockAccount(address, "password")

    assert add_0x_prefix(encode_hex(privtoaddr(private_key))) == add_0x_prefix(address)

    # the data to be signed
    data = b'1234567890abcdefghijklmnopqrstuvwxyz'
    # the hash of the data `0x089c33d56ed10bd8b571a0f493cedb28db1ae0f40c6cd266243001528c06eab3`
    data_hash = web3.sha3(data)

    assert force_bytes(data_hash) == sha3(data)

    priv_key = PrivateKey(flags=ALL_FLAGS)
    priv_key.set_raw_privkey(private_key)
    pub_key = priv_key.pubkey
    pub_key_serialized = pub_key.serialize(compressed=False)
    pub_key_address = sha3(encode_pubkey(pub_key_serialized, 'bin')[1:])[-40:]

    # This verifies that the address recovery code is working as expected since
    # the roundtrip operation matches the address being used to sign.
    assert add_0x_prefix(pub_key_address) == add_0x_prefix(address)

    # This is a sanity check using the pattern found internally in `pyethereum`
    v, r, s = ecdsa_raw_sign(decode_hex(data_hash), private_key)
    verify_pub_key = PublicKey(flags=ALL_FLAGS)
    verify_pub_key.public_key = verify_pub_key.ecdsa_recover(
        decode_hex(data_hash),
        verify_pub_key.ecdsa_recoverable_deserialize(
            decode_hex(encode_number(r, 32)) + decode_hex(encode_number(s, 32)), v - 27,
        ),
        raw=True,
    )
    verify_pub_key_serialized = verify_pub_key.serialize(compressed=False)
    verify_pub_key_address = sha3(encode_pubkey(verify_pub_key_serialized, 'bin')[1:])[-40:]

    assert add_0x_prefix(verify_pub_key_address) == add_0x_prefix(address)

    # Now have geth sign some data.
    # `0xcb8a42e69ff562a4391ef5bb6346bdb3dd144e96c6462ea950a71fb707d2816b1b636277fcef51ead6139376cef5d5d244d677da06d655c42727de3e340a5bb701`
    signature_hex = web3.eth.sign(remove_0x_prefix(address), data)
    signature_bytes = decode_hex(signature_hex)


    # Now lets try to recover the public key from the signature returned from
    # the `geth` JSON-RPC `eth_sign` endpoint.
    rec_pub_key = PublicKey(flags=ALL_FLAGS)
    rec_pub_key.public_key = rec_pub_key.ecdsa_recover(
        decode_hex(data_hash),
        rec_pub_key.ecdsa_recoverable_deserialize(
            signature_bytes[:64], signature_bytes[64] - 1,
        ),
        raw=True,
    )
    rec_pub_key_serialized = rec_pub_key.serialize(compressed=False)

    # This address is expected to be the same one used to sign.  Instead I get
    # `0x15f8c20c46c734b12527925392937890e7eba985` which is wrong
    rec_pub_key_address = sha3(encode_pubkey(rec_pub_key_serialized, 'bin')[1:])[-40:]

    # this fails because the addresses don't match
    assert add_0x_prefix(rec_pub_key_address) == add_0x_prefix(address)

    # try to verify the signature against the public key derived from the
    # original private key.  It fails.
    #
    # It of course passes if I use the recovered public key
    recoverable_signature = pub_key.ecdsa_recoverable_deserialize(signature_bytes[:64], signature_bytes[64])
    signature = pub_key.ecdsa_recoverable_convert(recoverable_signature)
    is_valid = pub_key.ecdsa_verify(
        msg=data,
        raw_sig=signature,
        digest=sha3_256,
    )
    assert is_valid

I have verified the following things to try and debug this.

  • I am using the correct sha3 implementation (keccak)
  • I can produce signatures using the given private key that recover the correct address as well as verify against the recovered and original public key.
  • The address returned by web3 is the expected address that would be created by the private_key bytes.
  • The returned signature also does not match the coinbase account or any of the other accounts present in the running geth instance.

I've mucked around with the following.

  • Changing the v value passed into ecdsa_recoverable_deserialize
  • Calling ecdsa_verify with raw=True and msg=sha3_256(data).digest().

In addition to all of this, I'm confused why the eth_sign calls return different signatures for the same inputs. I've verified that I see this same behavior from the javascript console.

1
  • 1
    Seemed like a well thought through question and I was planning on offering a bounty on it... Seems you judge the answer isn't worthwhile to keep here?
    – eth
    Jul 6, 2016 at 0:30

1 Answer 1

3

I have had a similar issue and discovered that the eth_sign RPC call does not sign the passed message, but a transformation of it.

Quote from https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_sign

The sign method calculates an Ethereum specific signature with: sign(keccak256("\x19Ethereum Signed Message:\n" + len(message) + message))).

(Where len(message) is the ASCII decimal representation of the length)

One option is to pass keccak256("\x19Ethereum Signed Message:\n" + len(message) + message) instead of message as first argument to ecrecover(). Another is to use a more low-level version of eth_sign.

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.