"Bitcoins are only divisible down to 8 decimal places If there is a need for them, additional decimal places can be added with hard fork."
The number of decimal places for an ERC20 token is specified when the token contract is deployed, in the (optional)
decimals parameter. Eighteen decimal places is a common choice (the same as Ether), but it can be more or less than this. Once it has been set it cannot be changed without re-deploying the contract, which would be majorly disruptive.
Note that the number of decimal places is just a display/UI issue. All tokens are transacted in integer numbers of units, from
E.g. If I sent you one token and
decimals is three for that token, then it would be displayed that I had sent you
0.001 tokens, but under the hood it's just a transfer of one unit.
The ERC20 standard is here, but it's fairly unspecific on this topic.
In answer to your direct question, ERC20 tokens are obviously not infinitely divisible.
The Ethereum network handles Wei. There are 10^18 wei in an ether, but technically there is no such thing as an ether. Deep inside the code, you won't see any calculations in ether. All the calculations are in Wei. Ether is just a output option.
An ERC20 token, on the other hand, is a piece of software written on top of the Ethereum network. Because the code is turing complete, you can do anything you like, including programming some crazy-ass token that might appear to be infinitely divisible, but it would be limited by available resources--especially 'gas usage.' In other words, if the calculation you would need to do to support what appeared to be an infinitely divisible token used too much gas, it wouldn't work.
As noted above, many ERC20 tokens use 18 decimals places (i.e., one token == one ether). I would guess this is for ease of understanding. There's nothing in the spec that forces an ERC20 token to be a particular number of decimals, so you have to understand that for each token.
While ERC20 tokens are not infinitely divisible, as their quantity is represented by an integer, it is possible, if necessary, to issue another ERC20 token representing smaller quantities.
See for example the Great Unicorn Meat Grinder.