Notice how ret is the location in memory of the length of the bytes array ret. It's the same with word, so when you do mstore(add(ret, 32), word), you're actually storing the location in memory of the length of array word at the start of the data in the ret array. Here is the correct version of what you're trying to do:

    function debug() view returns (bytes) {
        bytes memory ret = new bytes(6);
        bytes memory word = new bytes(6);
        word = hex"61626f757400";
        assembly {
            mstore(add(ret, 32), mload(add(word, 32)))
        }
        return ret;
    }

Notice how ret is the location in memory of the length of the bytes array ret. It's the same with word, so when you do mstore(add(ret, 32), word), you're actually storing the location in memory of the length of array word at the start of the data in the ret array. Here is the correct version of what you're trying to do:

function debug() view returns (bytes) {
    bytes memory ret = new bytes(6);
    bytes memory word = new bytes(6);
    word = hex"61626f757400";
    assembly {
        mstore(add(ret, 32), mload(add(word, 32)))
    }
    return ret;
}