geth vs ganache (testrpc): failed transactions produce different results

Question

It seems that geth and ganache-cli (testrpc) have differing behaviour on failed transactions. Suppose

(1) I have a trivial smart contract with a variable, a getter&setter, and an event. The setter takes a boolean argument, and if that argument is false, it fails. Otherwise, it sets the variable and fires and event.

(2) I have a test that does get()/set(true,42)/get()/set(false,84)/get()

(3) when I run with ganache-cli as a backend, the second set() produces a VM revert exception

(4) when I run with geth, I get a transaction hash, and looking it up, yields a receipt with empty log (so the tran didn't run). Further, the subsequent get() shows that the variable has not been changed.

(5) by contrast, the first set() yields a tranhash/receipt from both backends, and of course, the log is there and I can find the events.

To sum up: running against both backends, the state-behaviour of ethereum is the same. BUT against geth, we get no exception on the second set(), whereas with ganache-cli, we DO get an exception.

I don't know whether this is expected behaviour. But I do know this: I checked both openzeppelin-solidity and gnosis/MultiSigWallet, and both of them have unit-tests that expect the ganache-cli behaviour, and fail when run against geth.

One of the two is wrong here. If I'd make a guess, it's ganache-cli. But OTOH, two well-respected projects both count on this behaviour.

Not sure what to think. Anybody got a clue about this?

contract Bar
{
  uint n ;
  event Argle(uint arg);
  function argle_int(uint arg) internal {
    n = arg ;
  }

  function argle(bool pass, uint arg) public {
    Argle(arg) ;
    require(pass) ;
    argle_int(arg);
  }

  function get() public view returns (uint) {
    return n ;
  }
}

Answer 1

sendTransaction to ganache-cli will synchronously return the revert or the return value of the function. This is not how it works in a real network which is asynchronous in nature. AFAIK, there is no way to configure this in ganache-cli.

As stated by benjamincburns in https://github.com/trufflesuite/ganache-cli/issues/412, the focus seems to be in implementing the JSON RPC pedantically, but not necessarily in behaving identically to geth or partiy.

As I said over there, our goal isn't to behave identically to geth or parity, but rather to somewhat pedantically implement the JSON RPC spec."

If you want that your tests work for both, you'll have to check for both cases:

• catch the throw (ganache-cli)
• wait for transaction receipt and check for status 0x00/0x01 (geth / partiy / ...).

IMHO, this is rather a workaround and ganache-cli should behave in the same way like geth/parity do.

Btw. you should always test your code at least once with a real node.