I want to make a Dapp using ethereum. So, should I use a Node.js server and a database to store some user's data and store rest of the data on the smart contract that needs to be decentralized or should I keep all the data on the smart contract?
It very unlikely to store all the data in an Ethereum smart contract because:
i. Ethereum is a public blockchain and all the data stored on smart contract are public. Except if you don't have user private data in your app, you shouldn't store everything on a smart contract but only data that can be disclosed.
(PS: a simple email address is considered as private data).
ii. Ethereum transaction are expensive. Each transaction has a cost (gas fee) for the user and more data you store, more expensive it is.
You should reduce the smart contract data to the minimum you need to make it run.
In general, when building a decentralized application, the architecture is kind of triangular:
Frontend / \ / \ Smart Contract --- Backend
And you need to be very careful with boundaries between on-chain and off-chain data.