A study on fast rate-distortion optimized coding mode decision for H.264

The H.264 video coding standard can achieve considerably higher coding efficiency than any other existing standards by deciding the best mode among many prediction modes and various sixes of prediction blocks. Although the coding efficiency is improved by using Lagrange optimization for the mode decision, computational complexity increases significantly at the encoder. In this paper, we propose the fast rate-distortion optimization method for the hierarchical and adaptive coding mode decision in order to reduce the number of candidates for the best coding mode.