When Ethereum decides to do an upgrade after the merge, do validators have to pick which chain they're going to support i.e. if they sign blocks / attest on both chains after a hardfork has occurred will they get slashed or can they choose to validate both? I'm assuming they could choose to validate both either way, the question is do they have to exit and then stake again after the fork to avoid getting slashed or are different chain IDs already accounted for?
do validators have to pick which chain they're going to support
Validators do not have to pick which chain they're going to support: correctly programmed clients will be in consensus with each other and follow the same chain (automatically).
if they sign blocks / attest on both chains after a hardfork has occurred will they get slashed or can they choose to validate both?
Validators will get slashed.
"Double voting is when a validator votes for two different blocks during the same epoch, which means they are signalling support for two different versions of reality. The simplest example of why this is forbidden is a validator sending transaction $a$ in block $A$ and $b$ in block $B$ where $a$ and $b$ spend the same ETH. This is the Proof of Stake version of the classic double-spend attack."
The same blog post that describes above, also describes surround votes.
"Slashing of surround votes also prevents two versions of the chain from becoming finalised by punishing validators who create votes which present multiple different versions of reality which they claim to be true at the same time. More specifically, attestations (votes for blocks) are surround votes when a validator attests to one version of reality and later attests to another version, but in a way that doesn't make clear that they no longer believe in the first."
Note: for an October 2022 validator to follow a hardfork (such as the planned Shanghai upgrade) it would have to additionally run an upgraded client. Running more than 1 active client for a validator has anecdotally been the most common way validators have been slashed (until Distributed Validators (DV) are operational.) It is important to stop the validator client software before upgrading or installing a new version (to prevent slashing accidents).