are truffle tests inconsistent?

Question

Im currently testing my contract and I've written truffle tests in js for the first time. Rarely (1 out of 10 times?) I get unexpected results:

This is the code structure I use:

var contract = artifacts.require("./Contract.sol");
var otherContract = artifacts.require("./otherContract.sol");
var instance;
contract("DevToken", accounts => {

    before(async() => {
        instance = await contract.deployed();
    });

    it("init test", async() => {

        var contractAddress = await instance.getAddress.call();

        assert.equal(contractAddress, otherContract.address,
        "address: " + contractAddress + " should be equal to " + otherContract.address);
    });

    // e.g. balance in contract is 0.5 ETH
    // function toWei() is defined as web3.toWei(x,"ether")
    it("should test balance", async() => {

        await instance.withdraw(toWei(0.5),{from:accounts[0]});

        var allowanceBalance = await instance.allowanceBalance.call();

        assert.equal(toWei(0.5), allowanceBalance.toNumber(),
        "should have allowanceBalance of " + 0.5 + " ETH, actual allowanceBalance: " + fromWei(allowanceBalance) + " ETH");

    });

});

basically what happens is that when testing initial variables set in the constructor or testing variables after a contract call, sometimes the test fails and shows me the previous state before the call (or 0 in case a constructor variable fails). Then I change nothing and run the test again and everything is fine.

Example of an assertion error output where I set an address in the constructor:

AssertionError: address: 0x0000000000000000000000000000000000000000 should be equal to 0xb5ed2d3af7d0efa5ba6f805ae633f6516003f5e2: expected '0x0000000000000000000000000000000000000000' to equal '0xb5ed2d3af7d0efa5ba6f805ae633f6516003f5e2'

Example of an assertion error when I withdraw from a balance

AssertionError: should have allowanceBalance of 0 ETH, actual allowanceBalance: 0.5 ETH: expected 0 to equal 500000000000000000

in the second example the withdrawal goes through and 0.5 eth get sent to my wallet, so everything works as expected, only the test doesn't. It seems as if sometimes, the previous contract call isn't finished calculating and updating stuff yet and the second call to check the values is already fetching the variable states. It kinda ignores the await purpose.

To clarify again: The tests work 90% of the time (without changing stuff), so there should be no error with my contracts and unless I am calling stuff wrongly, there should be no error with my tests either.)

I'm using ganache as my testrpc, btw.

Is there some inconsistency with truffle js tests? Why does this happen? Is there a way to fix this?

Answer 1

I ran into a similar problem, only using the then notation with Promises. In my case, the cause of my problem was functions being executed before others finished. My solution was to use more thens to assign more of a structure to my test file. Before doing that, I had erratic & changing results; after it, I achieved consistent results from Ganache.