I have never used truffle, but my view (and the general rule as I understand) when it comes to source control is that you commit any files and folders that you want to track that can not be recreated.
You should not commit the build directory because the built files can be.. rebuilt. Storing that data would be an unnecessary use of space/bandwidth.
It also necessitates good practices. For example you can build your code, and then modify the source (from which those files are built). If you submitted everything to source control (without rebuilding), another team member might check it out and work with the built files which in this case are not fully up to date.
A brief look at their website implies that this build folder holds compiled contracts. I would not call that a build, but hey-ho :)
In my opinion you would be best served submitting everything to source control except the build folder.