What Continuous Integration services support running Truffle Solidity tests?

Question

What are the best practices for running a CI server that runs Truffle Solidity tests?

Answer 1

Honestly, any CI service will do the only requirements are a test blockchain (easy to install ganache-cli ) or use the docker image and other packages of interest including solidity-coverage, eth-gas-reporter, solhint and possibly automatically solidity documentation through solidity-docgen. Although, it took a while to figure out how to use solidity-docgen in travisCI.

Notes

If you use the scripts below, make sure to properly define your ganache command in the package.json. It's highly important to do ganache-cli > /dev/null &, not just ganache-cli, otherwise Travis will hang on listening to the port you defined until it's shut down automatically. You can read more about > /dev/null & and why it's required here.

Also, a simple stop command would like this: kill $(lsof -t -i:7545).

Sample CI Files

These are basic CI files for travis, gitlab and circle, and as you can see, solidity tests can be a single job in a larger pipeline.

TravisCI

sudo: required
language: node_js
node_js:
  - "9"
cache:
  directories:
  - node_modules
install:
  - npm install -g truffle
  - npm install -g ganache-cli
  - npm install
script:
  - npm run lint
  - npm run solium
  - npm run ganache
  - sleep 5
  - truffle migrate
  - truffle test
  - npm run stop
after_script:
  - npm run coverage && cat coverage/lcov.info | coveralls

GITLAB CI

image: node:8.1.1

cache:
  paths:
  - node_modules/

stages:
  - truffle-tests
  - coverage

smart_contracts:
  stage: truffle-tests
  services:
   - trufflesuite/ganache-cli
  script:
   - npm install truffle
   - ./node_modules/truffle/build/cli.bundled.js compile
   - ./node_modules/truffle/build/cli.bundled.js migrate --network gitlab
   - ./node_modules/truffle/build/cli.bundled.js test --network gitlab

solidity-coverage:
  stage: coverage
  script:
    - npm install solidity-coverage truffle
    - npm run coverage
  allow_failure: true
  artifacts:
  paths:
    - coverage
  expire_in: 10 days
  allow_failure: true
  coverage: /All files[^|]*\|[^|]*\s+([\d\.]+)/

CircleCI

version: 2
jobs:
  build:
    working_directory: ~/smart-contracts
    # The primary container is an instance of the first image listed. The job's commands run in this container.
    docker:
      - image: circleci/node:8-browsers
    # The secondary container is an instance of the second listed image which is run in a common network where ports exposed on the primary container are available on localhost.
      - image: trufflesuite/ganache-cli
    steps:
      - checkout
      - run:
          name: Update npm
          command: 'sudo npm install -g npm@latest'
      - restore_cache:
          key: dependency-cache-{{ checksum "package.json" }}
      - run:
          name: Install npm wee
          command: npm install
      - run:
          name: Migrate Contracts
          command: ./node_modules/truffle/build/cli.bundled.js migrate --network development
      - run:
          name: Test Contracts
          command: ./node_modules/truffle/build/cli.bundled.js test --network development
      - save_cache:
          key: dependency-cache-{{ checksum "package.json" }}
          paths:
            - node_modules

Answer 2

GitHub Action

truffle:
  runs-on: ubuntu-latest
  defaults:
    run:
      working-directory: ./contracts

  steps:
    - uses: actions/checkout@v2
    - name: Set up NodeJS 12
      uses: actions/setup-node@v2-beta
      with:
        node-version: '12'
    - name: Cache npm packages
      uses: actions/cache@v2
      with:
        path: ./node_modules
        key: ${{ runner.os }}-node-${{ hashFiles('**/package.json') }}
        restore-keys: |
          ${{ runner.os }}-node-
    - name: Install npm packages
      run: npm install
    - name: Start ganache
      run: |
        ./node_modules/ganache-cli/cli.js -p 8545 --gasLimit=0x1fffffffffffff --allowUnlimitedContractSize -e 1000000000 > /dev/null &
        while netstat -lnt | awk '$4 ~ /:8545$/ {exit 1}'; do sleep 5; done
    - name: Migrate Contracts
      run: ./node_modules/truffle/build/cli.bundled.js migrate --network development
    - name: Test Contracts
      run: ./node_modules/truffle/build/cli.bundled.js test --network development
    - name: Stop ganache
      run: kill $(lsof -t -i:8545)