I ran into the eip-170 contract code size limitation. I need to trim it down. Do comments affect the size of the contract as I have a lot of them and don't mind getting rid of them.
No, they affect only the contents of the hashed-metadata part in the bytecode, which is the 32 bytes (64 hexadecimal characters) which appear right before the last 2 bytes (4 hexadecimal characters).
The compiler appends a hash of the file's metadata (which includes your comments) at the end of the bytecode, so that you can retrieve this file in an authenticated manner.
As of solc 0.5.x, those last 2 bytes have changed to 11 bytes (22 hexadecimal characters).