Private transaction from node A to node B is visible by node C

Question

I am testing the GoQuorum private transactions using the simpleStorage smart contract. The blockchain has been created in a Ubuntu Linux 20.04.2 LTS using quorum-wizard custom network-bash options with 3 nodes and privacy enabled using Tessera. I sent a private transaction from node A to node B, but noticed that the transaction is visible by node C. I tested using both web3j/web3j-quorum and web3.js/quorum-js, and deploying the contract with Truffle or with web3j/web3.js. I need help to understand why this behavior is happening and how to fix it.

simpleStorage.sol:

pragma solidity ^0.5.0;

contract SimpleStorage {
    uint private storedData;

    function set(uint x) public {
        storedData = x;
    }

    function get() public view returns (uint) {
        return storedData;
    }
}

ABI:

[
    {
      "constant": false,
      "inputs": [
        {
          "name": "x",
          "type": "uint256"
        }
      ],
      "name": "set",
      "outputs": [],
      "payable": false,
      "stateMutability": "nonpayable",
      "type": "function"
    },
    {
      "constant": true,
      "inputs": [],
      "name": "get",
      "outputs": [
        {
          "name": "",
          "type": "uint256"
        }
      ],
      "payable": false,
      "stateMutability": "view",
      "type": "function"
    }
  ]

truffle migration:

const SimpleStorage = artifacts.require("SimpleStorage");

module.exports = function(deployer) {
  deployer.deploy(SimpleStorage, { privateFor: ["<tessera-node2-public-key>"] });
};

Java code:

package com.example.demo;

import okhttp3.OkHttpClient;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.web3j.abi.FunctionEncoder;
import org.web3j.abi.datatypes.Function;
import org.web3j.abi.datatypes.generated.Uint256;
import org.web3j.crypto.CipherException;
import org.web3j.crypto.Credentials;
import org.web3j.crypto.RawTransaction;
import org.web3j.crypto.WalletUtils;
import org.web3j.protocol.core.DefaultBlockParameterName;
import org.web3j.protocol.core.methods.response.EthGetTransactionCount;
import org.web3j.protocol.core.methods.response.EthSendTransaction;
import org.web3j.protocol.core.methods.response.TransactionReceipt;
import org.web3j.protocol.exceptions.TransactionException;
import org.web3j.protocol.http.HttpService;
import org.web3j.quorum.Quorum;
import org.web3j.quorum.enclave.Enclave;
import org.web3j.quorum.enclave.SendResponse;
import org.web3j.quorum.enclave.Tessera;
import org.web3j.quorum.enclave.protocol.EnclaveService;
import org.web3j.quorum.tx.QuorumTransactionManager;
import org.web3j.tx.response.PollingTransactionReceiptProcessor;
import org.web3j.utils.Numeric;

import java.io.File;
import java.io.IOException;
import java.math.BigInteger;
import java.util.*;

@SpringBootApplication
public class DemoApplication implements CommandLineRunner {
    private static final String TESSERA1_PUBLIC_KEY = "<tessera-node1-public-key>";
    private static final String TESSERA2_PUBLIC_KEY = "<tessera-node2-public-key>";

    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
    }

    @Override
    public void run(String... strings) throws CipherException, IOException, TransactionException {
        // initialize web3j with the quorum RPC address
        Quorum quorum = Quorum.build(new HttpService("http://127.0.0.1:22000"));

        // initialize the enclave service using the tessera ThirdParty app URL
        EnclaveService enclaveService = new EnclaveService("http://127.0.0.1", 9081, new OkHttpClient());

        // initialize the tessera enclave
        Enclave enclave = new Tessera(enclaveService, quorum);

        File source = new File(Objects.requireNonNull(DemoApplication.class.getClassLoader().getResource(
            "accounts/" + "<node-1-public-key>"))
            .getFile());

        // load the account from the filesystem
        Credentials credentials = WalletUtils.loadCredentials("", source);

        // create a quorum transaction manager
        // This object (used by the generated code) does the following:
        // 1. sends the raw payload to tessera and retrieves the txHash
        // 2. replace the transaction payload with the received txHash
        // 3. create and sign a raw transaction using the provided credentials
        // 4. invoke the eth_SendRawPrivateTransaction API to send the transaction to quorum
        QuorumTransactionManager qtm = new QuorumTransactionManager(quorum,
            credentials,
            TESSERA1_PUBLIC_KEY,
            Collections.singletonList(TESSERA2_PUBLIC_KEY),
            enclave,
            30,
            1000);

        Function function = new Function(
            "set",
            Collections.singletonList(new Uint256(55)),
            Collections.emptyList()
        );

        String encodedFunction = FunctionEncoder.encode(function);
        PollingTransactionReceiptProcessor pollingTransactionReceiptProcessor = new PollingTransactionReceiptProcessor(quorum, 1000, 10);

        /* deploy contract again using a single QuorumTransactionManager methods */
        EthGetTransactionCount txCount1 = quorum.ethGetTransactionCount(credentials.getAddress(), DefaultBlockParameterName.LATEST).send();

        RawTransaction rawTx1 = RawTransaction.createTransaction(BigInteger.valueOf(txCount1.getTransactionCount().intValue()),
            BigInteger.ZERO, BigInteger.valueOf(8000000), "<contract-address>", BigInteger.ZERO,
            encodedFunction);

        // send the signed transaction to quorum
        EthSendTransaction sentTx1 = qtm.signAndSend(rawTx1);
        String txHash1 = sentTx1.getTransactionHash();
        System.out.println("Transaction hash: " + txHash1);

        // poll for the transaction receipt
        TransactionReceipt transactionReceipt1 = pollingTransactionReceiptProcessor.waitForTransactionReceipt(txHash1);
        System.out.println("Transaction receipt: " + transactionReceipt1);

        /* deploy contract again using exposed QuorumTranasctionManager methods */

        // store the raw transaction payload in tessera
        SendResponse storeRawResponse = qtm.storeRawRequest(encodedFunction, TESSERA1_PUBLIC_KEY,
            Collections.singletonList(TESSERA2_PUBLIC_KEY));

        System.out.println("Raw transaction hash from tessera:" + storeRawResponse.getKey());

        String tesseraTxHash = Numeric.toHexString(Base64.getDecoder().decode(storeRawResponse.getKey()));

        // find the current nonce for the account (for use in the next transaction)
        EthGetTransactionCount txCount2 = quorum.ethGetTransactionCount(credentials.getAddress(), DefaultBlockParameterName.LATEST).send();

        // create raw transaction with tessera tx hash
        RawTransaction rawTx2 = RawTransaction.createTransaction(BigInteger.valueOf(txCount2.getTransactionCount().intValue()),
            BigInteger.ZERO, BigInteger.valueOf(8000000), "<contract-address>", BigInteger.ZERO, tesseraTxHash);

        // build and sign private transaction
        String signedTxHex = qtm.sign(rawTx2);

        // send the signed transaction to quorum
        EthSendTransaction ethSendTransaction = qtm.sendRaw(signedTxHex, Collections.singletonList(TESSERA2_PUBLIC_KEY));
        String txHash2 = ethSendTransaction.getTransactionHash();
        System.out.println("Transaction hash: " + txHash2);

        // poll for the transaction receipt
        TransactionReceipt transactionReceipt2 = pollingTransactionReceiptProcessor.waitForTransactionReceipt(txHash2);
        System.out.println("Transaction receipt: " + transactionReceipt2);
    }
}

Node.js code:

const path = require("path");
const Web3 = require("web3");
const quorumjs = require("quorum-js");

async function simpleStoragePrivate() {
  web3 = new Web3(httpEndpoint);
  quorumjs.extend(web3);
  
  const TM1_PUBLIC_KEY = "<tessera-node1-public-key>";
  const TM2_PUBLIC_KEY = "<tessera-node2-public-key>";

  const simpleStorageAddress = "<contract-address>";
  const simpleStorage_abi = require(path.resolve("abi/SimpleStorage.json"));
  const simpleStorage = new web3.eth.Contract(simpleStorage_abi.abi, simpleStorageAddress);

  let encodedABI = simpleStorage.methods.set(55).encodeABI();

  const rawTransactionManager = quorumjs.RawTransactionManager(web3, {
    privateUrl: "http://127.0.0.1:9081"
  });

  const encryptedPrivateKey = JSON.stringify(require(path.resolve("accounts/<node-1-public-key>.json")));
  const decryptedAccountObject = web3.eth.accounts.decrypt(encryptedPrivateKey, "");

  let nonce = await web3.eth.getTransactionCount("<node-1-public-key>");

  const txnParams = {
    nonce: nonce,
    gasPrice: 0,
    gasLimit: 8000000,
    value: 0,
    data: encodedABI,
    from: decryptedAccountObject,
    to: simpleStorageAddress,
    privateFrom: TM1_PUBLIC_KEY,
    privateFor: [TM2_PUBLIC_KEY],
    isPrivate: true,
  };

  rawTransactionManager.sendRawTransaction(txnParams)
    .then(function (result, error) {
      if (error)
        console.log(error);
      else
        console.log(result);
    });
}

async function getSimpleStorage() {
  const simpleStorage_abi = require(path.resolve("abi/SimpleStorage.json"));
  const simpleStorage = new web3.eth.Contract(simpleStorage_abi.abi, "<contract-address>");
  let result = await simpleStorage.methods.get().call({ gas: 800000, from: "<node-3-public-key>" });
  console.log(result);
}

async function main() {
  web3 = new Web3("http://127.0.0.1:22000");
  quorumjs.extend(web3);

  var program = require("commander");

  program
    .version("0.0.1")
    .description("simple storage") 

  program
    .command("simpleStoragePrivate")
    .description("Simple storage private")
    .action(simpleStoragePrivate);

  program
    .command("getSimpleStorage")
    .description("Get simple storage current value")
    .action(getSimpleStorage);

  program.parse(process.argv);
}

process.on("unhandledRejection", e => {
  console.log(e)
});

main();

Output of simpleStoragePrivate:

{
  blockHash: '0x733a8d4e42e452a7b8e1ebf180f2a12e3cfae842ffc1c886b042dca202630014',
  blockNumber: 227662,
  contractAddress: null,
  cumulativeGasUsed: 0,
  from: '0x0b0475f8b0daa3fed541b01929cf1c00fafe0ed4',
  gasUsed: 0,
  logs: [],
  logsBloom: '0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
  status: true,
  to: '0x1c699a3e9aad17d6ac8f6ecb9296ac7fbed3a27a',
  transactionHash: '0xd5559efb80ae67d87c3c296c7cb54146a11097d7902087bc2640ab54d3ba5a66',
  transactionIndex: 0
}

web3.eth.getTransaction("0xd5559efb80ae67d87c3c296c7cb54146a11097d7902087bc2640ab54d3ba5a66"):

{
  blockHash: '0x733a8d4e42e452a7b8e1ebf180f2a12e3cfae842ffc1c886b042dca202630014',
  blockNumber: 227662,
  from: '0x0b0475F8b0daa3FED541B01929CF1C00FaFe0eD4',
  gas: 8000000,
  gasPrice: '0',
  hash: '0xd5559efb80ae67d87c3c296c7cb54146a11097d7902087bc2640ab54d3ba5a66',
  input: '0xaf78054945b1f125097adac218f9a5fb1cf75213eb17fa8a6a9cbaad14632fb8b94dd15236bb6904c806bcef749960701ccac7ff433ba448291d74e55fa56aa9',
  nonce: 100,
  to: '0x1C699a3e9AaD17d6Ac8f6EcB9296aC7fBED3a27a',
  transactionIndex: 0,
  value: '0',
  v: '0x26',
  r: '0xfe035fcb803d6303eba215bcb0e366071e4554fac1beade560131ad220570ca7',
  s: '0x7156811467834bb44de8b827ab8cb1d64b730fec2625c84be3e5b98ff77b2793'
}

Outputs of getSimpleStorage for nodes A, B and C:

$ node src/simpleStorage.js getSimpleStorage
55

$ node src/simpleStorage.js getSimpleStorage
55

$ node src/simpleStorage.js getSimpleStorage
55

Versions:

quorum-wizard 1.3.3

GoQuorum 21.4

Tessera 21.4

Truffle v5.4.0

Solidity 0.5.0

Java openjdk 11.0.11

web3j-quorum 4.8.4 / web3j 4.8.4

quorum-js 0.3.6

web3.js 1.4.0

Answer 1

I forgot to change the port of the endpoint to 22002 (node C) instead of 22000 (node A). This way via geth attach in node C returns 0 instead of 55. Attaching A or B returns 55. Calling via web3.js returns an error: Returned values aren't valid, did it run Out of Gas? You might also see this error if you are not using the correct ABI for the contract you are retrieving data from, requesting data from a block number that does not exist, or querying a node which is not fully synced.