Let's imagine we have a 30kk gas limit block in ETH. 25kk of gas are already taken, and it's time for my transaction, which consumes 10kk gas. Will it be pushed to this block, with kind of "Out of gas" exception and 5kk consumed gas, or it gonna be pushed to the first block where it could consume full 10kk of gas?
It will stay in the mempool for possible inclusion in a future block.
Generally, miners are incentivized to pack as much gas as possible into blocks, at the highest offered gasPrice. It's like trying to pack a container with the highest possible mass - constrained by space but aiming for the highest possible weight.
So, a viable algorithm would be:
- Order mempool transactions by gasPrice, highest to lowest, regardless of size (gas)
- Insert txns in descending gasPrice order, skipping any that don't fit in the available space (gas consumption).
- Stop when gasLimit is reached or no mempool transactions that will fit in the remaining free space exist.
Hope it helps.