web3.sha3 dealing with hex in both geth and web3.js

Question

in both geth and web3.js, when we call

> web3.sha3("0x536f6d6520415343494920737472696e6720746f20626520686173686564")

we get:

requirement of using web3.fromAscii before sha3 is deprecated
new usage: 'web3.sha3("hello")'    
see https://github.com/ethereum/web3.js/pull/205
if you need to hash hex value, you can do 'sha3("0xfff", true)'
"b21dbc7a5eb6042d91f8f584af266f1a512ac89520f43562c6c1e37eab6eb0c4"

So I proceed to do this:

> web3.sha3("0x536f6d6520415343494920737472696e6720746f20626520686173686564",true)

upon which I get:

"bf328f7c064d78daf8fa3e2ffe36c1c3cb22ae02832d4ead2995130ca21a8524"

This does not match the original value that I intended to get, namely

"b21dbc7a5eb6042d91f8f584af266f1a512ac89520f43562c6c1e37eab6eb0c4"

I was wondering if someone could explain to me why the two values are different. The reason I'm asking is that I'd like to sha3 this:

sha3("0xfff")

which gives me a invalid byte index error, but I know that

sha3("0xfff",true)

is not going to get me what I want, based on the above observations.

Answer 1

Please use a newer version of Geth and web3.js, because the error message you displayed indicates they are old: the error has been removed.

The API is to use {encoding: 'hex'} as the 2nd argument. (true is not supported anymore and ignored.)

With a newer version, you will get the Keccak-256 hash you are expecting:

> web3.sha3("0x536f6d6520415343494920737472696e6720746f20626520686173686564",{encoding:'hex'})
"b21dbc7a5eb6042d91f8f584af266f1a512ac89520f43562c6c1e37eab6eb0c4"

For 0xfff:

> web3.sha3('0xfff', {encoding:'hex'})
"004ef40e5e8e6b2638aa35f3fa52d70fb12b25cc067f3476fca2b102ce79f91e"

Here is an updated version of Geth which behaves as described above:

> web3.version
{
  api: "0.15.1",

Answer 2

You get different output because you're asking web3.sha3 to interpret your input differently when you pass in true.

web3.sha3("0x536f6d6520415343494920737472696e6720746f20626520686173686564")

Without the true, web3.sha3 runs toAscii on your input before running SHA3 on it. Basically, it tries interpreting the hex pairs in your hex string into ASCII characters and then runs SHA3 on the resulting string. This is also why you get that error when trying to run web3.sha3('0xfff'): the hex string isn't composed of hex pairs and so can't be interpreted as representative of an ASCII string.

web3.sha3("0x536f6d6520415343494920737472696e6720746f20626520686173686564",true)

With the true, it runs SHA3 on your string as-is. Whatever you pass in is exactly what gets passed on to CryptoJS's SHA3 function.