The network itself won't have problems with the data and gas usage won't be a big problem if you're only storing the hashes.
All data on the network is readable for free (without gas) from a full node. Read operations are not sent to the network at all but they are all completed on the node you use. Therefore performing lots of read operations will not be a problem for you. You can read more about this here: If view or pure function doesn't cost any gas, would they be abused/free ride?
Also contract storage "limits" won't be an issue as contracts have basically unlimited storage size. More about this here: Is there a (theoretical) limit for amount of data that a contract can store?
Therefore I don't think you will have any issues with your approach.
For practical purposes, you should of course think in advance about how you want to access your data. If you store data for example in a mapping, you won't be able to iterate over the mapping data. If you need to iterate over the data, you need to store data in a different format, such as arrays.