How to convert/decode Solidity bytes32 to Python string via web3.py

Question

Inside a Solidity contract, I have this variable: zBytes32 = "HelloBytes32";

after Python web3.py retrieving that variable,

zbytes32 = contractInstance.functions.getzBytes32().call()

print(zbytes32) shows what zbytes32's value is:

# b'HelloBytes32\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

zbytes32 = zBytes32.decode('utf8')# shows HelloBytes32
print(zbytes32) 

decode('utf8') removes b in the front, print() will hide ALL the \x00, but they are STILL THERE!!! To prove it:

zBytes32str= str(zbytes32)
print('zbytes32 = '+ zBytes32str)
if zBytes32str == 'HelloBytes32':
    print('zbytes32str == "HelloBytes32"')
else:
    print('zbytes32str != "HelloBytes32"')

zbytes32str != "HelloBytes32" is shown!!! SO HOW CAN I properly decode byte32 in Python???

print('\ngetBytes32 raw: {}'.format(zbytes32))# shows HelloBytes32
print('zbytes32.decode("utf8") as string = '+ zbytes32)# shows HelloBytes32

ALTHOUGH they look the same as HelloBytes32, actually they are NOT!

Perhaps that explains the following situation:

arr = contractInstance.functions.getVariables().call()
print('arr[4] in str()= '+ str(arr[4]))#must be converted to string
b2= arr[4].decode('utf8')
print('b2.decode("utf8")= '+ b2)
arr[4] = ''
print('after arr[4] = "", arr[4]= '+ arr[4])
arr[4] = b2
print('after arr[4] = b2, arr[4]= '+ arr[4])
arr[0] = 'What man?'
print('after arr[0] = "What man?", arr[0]= '+ arr[0])
print('\nget many variables: {}'.format(arr))

it shows in the terminal:

arr[4] in str()= b'HelloBytes32\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
b2.decode("utf8")= HelloBytes32
after arr[4] = "", arr[4]= 
after arr[4] = b2, arr[4]= HelloBytes32
after arr[0] = "What man?", arr[0]= What man?

get many variables: ['What man?', '0x583031D11...', 111111, False, 'HelloBytes32\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', ['0xCA35b7d...', '0x14723A...'], [9999999, 555555]]

See the above array. When I set a new value to arr[0], it works.

But when I set a new value to arr[4], it kind of works ...

WHY is it when I show the whole array, arr[4] still shows those \x00 ???

I need to pass those values on to another script so I have to make sure they are clean from those \x00. Please help. Thank you

Answer 1

The issue here is that your variable has a fixed length of 32 bytes, which explains the zeroes in the right. You can strip the zeros in python:

zbytes32 = contractInstance.functions.getzBytes32().call()
zbytes32 = zbytes32.hex().rstrip("0")
if len(zbytes32) % 2 != 0:
    zbytes32 = zbytes32 + '0'
zbytes32 = bytes.fromhex(zbytes32).decode('utf8')

Printing the variable zbytes32 gives: 'HelloBytes32' and: zbytes32=='HelloBytes32' is True

The conditional part if len(zbytes32) % 2 != 0 is there because it may be that the last character is indeed zero and we are removing it, the way to know this is that every byte is represented by two hexadecimal characters so if the length of the variable zbytes32 after removing the zeros is odd, one zero must be added to the right.

Hope this helps

Answer 2

Since you seem to be assuming that the bytes are decodable with UTF-8, you might actually want a string type in Solidity instead of bytes32.

If you change the variable type to String and redeploy, then the web3.py contract with the new ABI would return to you an already decoded string as "HelloBytes32".