Shape adaptive integer transform for coding arbitrarily shaped objects in H.264/AVC

The use of shape-adaptive transforms is a popular approach for coding arbitrarily shaped objects in image/video coding due to their adaptability at object edges and low complexity. In this respect shape adaptive DCT (SA-DCT) and shape adaptive DWT (SA-DWT) have been proposed in previous literature. The Integer Transform (IT), a derivative of the 4x4 DCT, has been adopted in the latest H.264/AVC standard for coding image blocks in residual data (texture). The associated integer arithmetic guarantees fast and accurate coding/decoding. In this paper, we propose a novel Shape Adaptive Integer Transform (SA-IT) which can be effectively used in future for enabling arbitrary shaped object coding in H.264. Though Integer Transforms are a derivative of 4x4 DCTs, in H.264, to maintain integer arithmetic capability, the post-and pre-scaling factors of transform process are integrated into the forward and inverse quantiser stages respectively for reducing the total number of multiplications and avoiding the loss of accuracy. Thus SA-IT considerably differs from SA-DCT and calls for novel design and implementation considerations based on combining those merits of both SA-DCT and IT. We provide theoretical proofs and support them with experimental justifications.