While using populus on pytest how could I transfer byte array as an input to the smart-contract like in solidity browser?

Question

I have following contract; I can call set()with following byte array on solidity browser. The smart contract accepts the input as bytes32. I assume if geth and solidity browser sees a string starting with 0x, it converts it to byte format.

Following transaction call works without having any problem.

myContract.set("0x15CEF23823A9410D60CB6E6CC56046EC6035A9353100476EA28C94752AF104CD")

---

myContract.get() Returns: "0x15CEF23823A9410D60CB6E6CC56046EC6035A9353100476EA28C94752AF104CD

My contract:

contract Contract {
  bytes32 public hash;
  function set(bytes32 hash_) returns (bool success)
  {
        hash = hash_;
  }
  function get() constant returns bytes32 {
     return hash;
   }
}

---

I want to do same operation on Populus. When I try to call it as on test_greeter.py:

def test_greeter(web3, accounts, chain):
    myContract, _ = chain.provider.get_or_deploy_contract('Lottery')
    set_txn_hash = myContract.transact().set("0x15CEF23823A9410D60CB6E6CC56046EC6035A9353100476EA28C94752AF104CD")

It gives following error, I assume python sees the input given to set() as string instead of bytes. And the string version of the input exceeds the byte32.

   @classmethod
    def _encode_abi(cls, abi, arguments, data=None):
        argument_types = get_abi_input_types(abi)

        if not check_if_arguments_can_be_encoded(abi, arguments, {}):
            raise TypeError(
                "One or more arguments could not be encoded to the necessary "
                "ABI type.  Expected types are: {0}".format(
>                   ', '.join(argument_types),
                )
            )
E           TypeError: One or more arguments could not be encoded to the necessary ABI type.  Expected types are: bytes32

[Q] While using Populus on pytest how could I transfer byte array as an input to the contract like in solidity browser?

Thank you for your valuable time and help.

Answer 1

Where is the problem?

In the broader stack trace, you'll see that _encode_abi is actually in web3.py, so it's more of a web3 issue than a populus one.

This has been a long-standing issue in web3.py v3, and v4-beta works as expected; you can pass in a hex string to an ABI bytes* argument. See https://github.com/ethereum/web3.py/issues/329 where this issue was discussed. Unfortunately, I don't believe there is a v4-compatible version of populus, yet.

What is the best solution?

Alper's answer is valid. In web3 v3, the argument must be passed in as python bytes type.

The "Web3 way" to convert to bytes would use Web3.toBytes(), like:

def test_greeter(web3, accounts, chain):

   hex_str = "0xadc4f61a8a9995668f01ef1e2a0977be34c1d74d925a9a33d7f0b8fb4972a1ce";
   data    = web3.toBytes(hexstr= hex_str);

   set_txn_hash = myContract.transact().set(data);

   ret = myContract.call().get()
   ret = web3.toHex(ret.encode('latin-1'))[2:]    

Why must the argument be passed as bytes?

This behavior is partially as a side-effect of needing to support python 2, which cannot tell the difference between bytes and str. So web3 had to choose whether to support the native python bytes type, or hex-encoded strings. It could not reliably detect the difference. Since web3 v4 requires python3, it can tell the difference, and accepts bytes or hex str arguments for an ABI bytes* type.

How can I verify that this is the behavior?

Unfortunately, there isn't much in the way of documentation about this behavior, so we have to look at the source. The code that decides if an argument maps to an ABI type is in web3.utils.abi.is_encodable():

def is_encodable(_type, value):
    ...
    elif base == 'bytes':
        if not is_string(value):
            return False

        if not sub:
            return True

        max_length = int(sub)
        return len(value) <= max_length

Because the hex-encoded string is 64 or 66 characters long, and the length of the bytes32 argument is 32, this is_encodable() test will return False. Since this function returns False, web3 will reject the call saying that the argument could not be encoded to bytes32.

Answer 2

Instead of passing bytes value as string, we have to pass the argument as hex-data, by removing 0x at the beginning..

test.py:

def test_greeter(web3, accounts, chain):
   hex_string = "adc4f61a8a9995668f01ef1e2a0977be34c1d74d925a9a33d7f0b8fb4972a1ce";
   hexData    = hex_string.decode("hex");

   set_txn_hash = myContract.transact().set(hexData); //now works
   chain.wait.for_receipt(set_txn_hash);

   ret = myContract.call().get()
   ret = ret.encode('latin-1')
   ret = ret.encode("hex")

   print(ret); //now prints "adc4f61a8a9995668f01ef1e2a0977be34c1d74d925a9a33d7f0b8fb4972a1ce"

---

Additional solution:

import binascii, base58

Qm = b'\x12 '

def convertBytes32Ipfs(bytes_array):
    bytes_init = base58.b58decode("Qm")
    merge = Qm + bytes_array
    return base58.b58encode(merge).decode("utf-8")


def convertIpfsBytes32(hash_string):
    bytes_array = base58.b58decode(hash_string)
    b = bytes_array[2:]
    return binascii.hexlify(b).decode("utf-8")

def test_custom_greeting(web3, chain):
    print('\n')
    greeter, _ = chain.provider.get_or_deploy_contract('Contract')    

    ipfsHash = "QmWmyoMoctfbAaiEs2G46gpeUmhqFRDW6KWo64y5r581Vz"
    hex_str = convertIpfsBytes32(ipfsHash) # "7d5a99f603f231d53a4f39d1521f98d2e8bb279cf29bebfd0687dc98458e7f89";
    data    = web3.toBytes(hexstr= hex_str);
    set_txn_hash = greeter.transact().setGreeting(data)
    chain.wait.for_receipt(set_txn_hash)

    ret = greeter.call().greet()
    ret = ret.encode('latin-1')
    hex_str = web3.toHex(ret)[2:]
    ipfsHashRet = convertBytes32Ipfs(ret)
    assert ipfsHash == ipfsHashRet