How to create types for structs that have contract types instead of address?

Question

The struct I want to sign using ether's v5.7 (or 6) is as following,

struct Order {
  ...
  IERC20 token,
  uint256 tokenAmount,
  ...
}

Now if I change IERC20 to address in the struct and the typehash on the contract side ether's seems to generate correct signatures, but with IERC20 it generates incorrect signatures.

(cross checked it with foundry vm.sign produces the correct signature which ercrecover accepts but the library does not)

I am more inclined to find out a way to somehow make it work without changing the contract, so is there a way it will work with IERC20 instead of address?

Basic implementation here -> https://github.com/HareemSaad/Hashv4-Typed-Singing-Discussion