To do this, you need to define a VM log collector, which implements StructLogCollector. This function gets called on every step of the VM, and is provided with copies of the memory, stack, and modified parts of the storage, along with the program counter, current opcode, gas left, and other data. It's also called when an error occurs that causes a transaction to fail.
To extract the data you want, you need to watch for several things: 'CREATE', 'CALL', 'CALLCODE' and 'DELEGATECALL' opcodes, which invoke new contracts or accounts and potentially transfer value, and 'SUICIDE' which returns value to the caller. You also need to watch for anything that causes a call to return, and for the aforementioned errors.
While tracing, you need to maintain a stack that represents the stack of calls made by the current transaction, with each stack frame containing a list of value transfers that have been made so far. A non-error return pops a stack frame, adding all its value transfers to the frame below. An error return pops the frame, discarding all the transfers. When you pop the final frame, the set of transfers on it are the ones that were finalized as part of the transaction.
An implementation of all of this can be seen as part of my Etherquery code.