H.264/MPEG-4 AVC

From Wikipedia, the free encyclopedia

H.264, MPEG-4 Part 10, or AVC (for Advanced Video Coding), is a digital video codec standard that is noted for achieving very high data compression. It was written by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG) as the product of a collective partnership effort known as the Joint Video Team (JVT). The ITU-T H.264 standard and the ISO/IEC MPEG-4 Part 10 standard (formally, ISO/IEC 14496-10) are jointly maintained so that they have identical technical content. The final drafting work on the first version of the standard was completed in May 2003.

Contents 1 Overview 2 Features 3 Profiles 4 Levels 5 Standardization Committee and History 6 Versions 7 Patent licensing 8 Open Source/Free Software licensing 9 Applications 10 Products and Implementations 10.1 Prominent software implementations 10.2 Software encoder feature comparison 10.3 Prominent hardware implementations 10.3.1 Decoding 10.3.2 Encoding 11 References 12 See also 13 External links

[edit] Overview

The H.264 name follows the ITU-T naming convention (where the standard is a member of the H.26x line of VCEG video coding standards), while the MPEG-4 AVC name relates to the naming convention in ISO/IEC MPEG (where the standard is part 10 of ISO/IEC 14496, which is the suite of standards known as MPEG-4). The standard was developed jointly in a partnership of VCEG and MPEG, after earlier development work in the ITU-T as a VCEG project called H.26L. It is thus common to refer to the standard as H.264/AVC (or AVC/H.264 or H.264/MPEG-4 AVC or MPEG-4/H.264 AVC) to emphasize the common heritage. The name H.26L, referring to its ITU-T history, is less common, but still used. Occasionally, it is also referred to as "the JVT codec", in reference to the Joint Video Team (JVT) organization that developed it. (Such partnership and multiple naming is not uncommon – for example, the video codec standard known as MPEG-2 also arose from the partnership between MPEG and the ITU-T, where MPEG-2 video is known to the ITU-T community as H.262.)

The intent of the H.264/AVC project was to create a standard that would be capable of providing good video quality at substantially lower bit rates (e.g., half or less) than previous standards would require (e.g., relative to MPEG-2, H.263, or MPEG-4 Part 2), and to do so without increasing the complexity of design so much that it would be impractical (excessively expensive) to implement. An additional goal was to provide enough flexiblity to allow the standard to be applied to a wide variety of applications (e.g., for both low and high bit rates, and for low and high resolution video) and to make the design work effectively on a wide variety of networks and systems (e.g., for broadcast, DVD storage, RTP/IP packet networks, and ITU-T multimedia telephony systems).

The standardization of the first version of H.264/AVC was completed in May of 2003. The JVT then developed extensions to the original standard that are known as the Fidelity Range Extensions (FRExt). These extensions enable higher quality video coding by supporting increased sample bit depth precision and higher-resolution color information (including sampling structures known as YUV 4:2:2 and YUV 4:4:4). Several other features are also included in the Fidelity Range Extensions project (such as adaptive switching between 4×4 and 8×8 integer transforms, encoder-specified perceptual-based quantization weighting matrices, efficient inter-picture lossless coding, and support of additional color spaces). The design work on the Fidelity Range Extensions was completed in July of 2004, and the drafting work on them was completed in September of 2004.

Further recent extensions of the standard have included adding five new profiles intended primarily for professional applications (and deprecating one of the prior FRExt profiles that industry feedback indicated should have been designed differently), adding extended-gamut color space support, defining additional aspect ratio indicators, and defining two additional types of "supplemental enhancement information" (post-filter hint and tone mapping).

[edit] Features

H.264/AVC/MPEG-4 Part 10 contains a number of new features that allow it to compress video much more effectively than older standards and to provide more flexibility for application to a wide variety of network environments. In particular, some such key features include:

• Multi-picture inter-picture prediction including the following features:
  • Using previously-encoded pictures as references in a much more flexible way than in past standards, allowing up to 32 reference pictures to be used in some cases (unlike in prior standards, where the limit was typically one or, in the case of conventional "B pictures", two). This particular feature usually allows modest improvements in bit rate and quality in most scenes. But in certain types of scenes, for example scenes with rapid repetitive flashing or back-and-forth scene cuts or uncovered background areas, it allows a very significant reduction in bit rate.
  • Variable block-size motion compensation (VBSMC) with block sizes as large as 16×16 and as small as 4×4, enabling very precise segmentation of moving regions.
  • Six-tap filtering for derivation of half-pel luma sample predictions, in order to lessen the aliasing and eventually provide sharper images.
  • Quarter-pixel precision for motion compensation, enabling very precise description of the displacements of moving areas. For chroma the resolution is typically halved both vertically and horizontally (see 4:2:0) therefore the motion compensation precision is down to one-eighth pixel.
  • Weighted prediction, allowing an encoder to specify the use of a scaling and offset when performing motion compensation, and providing a significant benefit in performance in special cases—such as fade-to-black, fade-in, and cross-fade transitions.
• Spatial prediction from the edges of neighboring blocks for "intra"coding (rather than the "DC"-only prediction found in MPEG-2 Part 2 and the transform coefficient prediction found in H.263+ and MPEG-4 Part 2).
• Lossless macroblock coding features including:
  • A lossless PCM macroblock representation mode in which video data samples are represented directly, allowing perfect representation of specific regions and allowing a strict limit to be placed on the quantity of coded data for each macroblock.
  • An enhanced lossless macroblock representation mode allowing perfect representation of specific regions while ordinarily using substantially fewer bits than the PCM mode (not supported in all profiles).
• Flexible interlaced-scan video coding features (not supported in all profiles), including:
  • Macroblock-adaptive frame-field (MBAFF) coding, using a macroblock pair structure for pictures coded as frames, allowing 16x16 macroblocks in field mode (vs. 16x8 half-macroblocks in MPEG-2).
  • Picture-adaptive frame-field coding (PAFF or PicAFF) allowing a freely-selected mixture of pictures coded as MBAFF frames with pictures coded as individual single fields (half frames) of interlaced video.
• New transform design features, including:
  • An exact-match integer 4×4 spatial block transform (conceptually similar to the well-known DCT design, but simplified and made to provide exactly-specified decoding), allowing precise placement of residual signals with little of the "ringing" often found with prior codec designs.
  • An exact-match integer 8×8 spatial block transform (conceptually similar to the well-known DCT design, but simplified and made to provide exactly-specified decoding, not supported in all profiles), allowing highly correlated regions to be compressed more efficiently than with the 4×4 transform.
  • Adaptive encoder selection between the 4×4 and 8×8 transform block sizes for the integer transform operation (not supported in all profiles).
  • A secondary Hadamard transform performed on "DC" coefficients of the primary spatial transform (for chroma DC coefficients and also luma in one special case) to obtain even more compression in smooth regions.
• A quantization design including:
  • Logarithmic step size control for easier bit rate management by encoders and simplified inverse-quantization scaling.
  • Frequency-customized quantization scaling matrices selected by the encoder for perceptual-based quantization optimization (not supported in all profiles).
• An in-loop deblocking filter which helps prevent the blocking artifacts common to other DCT-based image compression techniques.
• An entropy coding design including:
  • Context-adaptive binary arithmetic coding (CABAC), which is a clever technique to losslessly compress syntax elements in the video stream knowing the probabilities of syntax elements in a given context (not supported in all profiles).
  • Context-adaptive variable-length coding (CAVLC), which is a lower-complexity alternative to CABAC for the coding of quantized transform coefficient values. Although lower complexity than CABAC, CAVLC is more elaborate and more efficient than the methods typically used to code coefficients in other prior designs.
  • A common simple and highly structured variable length coding (VLC) technique for many of the syntax elements not coded by CABAC or CAVLC, referred to as Exponential-Golomb coding (or just Exp-Golomb).
• Loss resilience features including:
  • A network abstraction layer (NAL) definition allowing the same video syntax to be used in many network environments, including features such as sequence parameter sets (SPSs) and picture parameter sets (PPSs) that provide more robustness and flexibility than provided in prior designs.
  • Flexible macroblock ordering (FMO, also known as slice groups and not supported in all profiles) and arbitrary slice ordering (ASO), which are techniques for restructuring the ordering of the representation of the fundamental regions (called macroblocks) in pictures. Typically considered an error/loss robustness feature, FMO and ASO can also be used for other purposes.
  • Data partitioning (DP), a feature providing the ability to separate more important and less important syntax elements into different packets of data, enabling the application of unequal error protection (UEP) and other types of improvement of error/loss robustness (not supported in all profiles).
  • Redundant slices (RS), an error/loss robustness feature allowing an encoder to send an extra representation of a picture region (typically at lower fidelity) that can be used if the primary representation is corrupted or lost (not supported in all profiles).
  • Frame numbering, a feature that allows the creation of "sub-sequences" (enabling temporal scalability by optional inclusion of extra pictures between other pictures), and the detection and concealment of losses of entire pictures (which can occur due to network packet losses or channel errors).
• Switching slices (called SP and SI slices and not supported in all profiles), features that allow an encoder to direct a decoder to jump into an ongoing video stream for such purposes as video streaming bit rate switching and "trick mode" operation. When a decoder jumps into the middle of a video stream using the SP/SI feature, it can get an exact match to the decoded pictures at that location in the video stream despite using different pictures (or no pictures at all) as references prior to the switch.
• A simple automatic process for preventing the accidental emulation of start codes, which are special sequences of bits in the coded data that allow random access into the bitstream and recovery of byte alignment in systems that can lose byte synchronization.
• Supplemental enhancement information (SEI) and video usability information (VUI), which are extra information that can be inserted into the bitstream to enhance the use of the video for a wide variety of purposes.
• Auxiliary pictures, which can be used for such purposes as alpha compositing.
• Support of Monochrome, 4:2:0, 4:2:2, and 4:4:4 color sampling structures (depending on the selected profile).
• Support of sample bit depth precision ranging from 8 to 14 bits per sample (depending on the selected profile).
• The ability to encode individual color planes as distinct pictures with their own slice structures, macroblock modes, motion vectors, etc., allowing encoders to be designed with a simple parallelization structure (supported only in the three 4:4:4-capable profiles)
• Picture order count, a feature that serves to keep the ordering of the pictures and the values of samples in the decoded pictures isolated from timing information (allowing timing information to be carried and controlled/changed separately by a system without affecting decoded picture content).

These techniques, along with several others, help H.264 to perform significantly better than any prior standard can, under a wide variety of circumstances in a wide variety of application environments. H.264 can often perform radically better than MPEG-2 video—typically obtaining the same quality at half of the bit rate or less.

Like other ISO/IEC MPEG video standards, H.264/AVC has a reference software implementation that can be freely downloaded. Its main purpose is to give examples of H.264/AVC features, rather than being a useful application per se. (See the links section for a pointer to that software.) Some reference hardware design work is also under way in MPEG.

[edit] Profiles

The standard includes the following seven sets of capabilities, which are referred to as profiles, targeting specific classes of applications:

• Baseline Profile (BP): Primarily for lower-cost applications with limited computing resources, this profile is used widely in videoconferencing and mobile applications.
• Main Profile (MP): Originally intended as the mainstream consumer profile for broadcast and storage applications, the importance of this profile faded when the High profile was developed for those applications.
• Extended Profile (XP): Intended as the streaming video profile, this profile has relatively high compression capability and some extra tricks for robustness to data losses and server stream switching.
• High Profile (HiP): The primary profile for broadcast and disc storage applications, particularly for high-definition television applications (this is the profile adopted into HD DVD and Blu-ray Disc, for example).
• High 10 Profile (Hi10P): Going beyond today's mainstream consumer product capabilities, this profile builds on top of the High Profile — adding support for up to 10 bits per sample of decoded picture precision.
• High 4:2:2 Profile (Hi422P): Primarily targeting professional applications that use interlaced video, this profile builds on top of the High 10 Profile — adding support for the 4:2:2 chroma sampling format while using up to 10 bits per sample of decoded picture precision.
• High 4:4:4 Predictive Profile (Hi444PP): This profile builds on top of the High 4:2:2 Profile — supporting up to 4:4:4 chroma sampling, up to 14 bits per sample, and additionally supporting efficient lossless region coding and the coding of each picture as three separate color planes.

In addition, the standard now contains four additional all-Intra profiles, which are defined as simple subsets of other corresponding profiles. These are mostly for professional (e.g., camera and editing system) applications:

• High 10 Intra Profile: The High 10 Profile constrained to all-Intra use.
• High 4:2:2 Intra Profile: The High 4:2:2 Profile constrained to all-Intra use.
• High 4:4:4 Intra Profile: The High 4:4:4 Profile constrained to all-Intra use.
• CAVLC 4:4:4 Intra Profile: The High 4:4:4 Profile constrained to all-Intra use and to CAVLC entropy coding (i.e., not supporting CABAC).

	Baseline	Extended	Main	High	High 10	High 4:2:2	High 4:4:4 Predictive
I and P Slices	Yes	Yes	Yes	Yes	Yes	Yes	Yes
B Slices	No	Yes	Yes	Yes	Yes	Yes	Yes
SI and SP Slices	No	Yes	No	No	No	No	No
Multiple Reference Frames	Yes	Yes	Yes	Yes	Yes	Yes	Yes
In-Loop Deblocking Filter	Yes	Yes	Yes	Yes	Yes	Yes	Yes
CAVLC Entropy Coding	Yes	Yes	Yes	Yes	Yes	Yes	Yes
CABAC Entropy Coding	No	No	Yes	Yes	Yes	Yes	Yes
Flexible Macroblock Ordering (FMO)	Yes	Yes	No	No	No	No	No
Arbitrary Slice Ordering (ASO)	Yes	Yes	No	No	No	No	No
Redundant Slices (RS)	Yes	Yes	No	No	No	No	No
Data Partitioning	No	Yes	No	No	No	No	No
Interlaced Coding (PicAFF, MBAFF)	No	Yes	Yes	Yes	Yes	Yes	Yes
4:2:0 Chroma Format	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Monochrome Video Format (4:0:0)	No	No	No	Yes	Yes	Yes	Yes
4:2:2 Chroma Format	No	No	No	No	No	Yes	Yes
4:4:4 Chroma Format	No	No	No	No	No	No	Yes
8 Bit Sample Depth	Yes	Yes	Yes	Yes	Yes	Yes	Yes
9 and 10 Bit Sample Depth	No	No	No	No	Yes	Yes	Yes
11 to 14 Bit Sample Depth	No	No	No	No	No	No	Yes
8x8 vs. 4x4 Transform Adaptivity	No	No	No	Yes	Yes	Yes	Yes
Quantization Scaling Matrices	No	No	No	Yes	Yes	Yes	Yes
Separate Cb and Cr QP control	No	No	No	Yes	Yes	Yes	Yes
Separate Color Plane Coding	No	No	No	No	No	No	Yes
Predictive Lossless Coding	No	No	No	No	No	No	Yes
	Baseline	Extended	Main	High	High 10	High 4:2:2	High 4:4:4 Predictive

[edit] Levels

Level number	Max macroblocks per second	Max frame size (macroblocks)	Max video bit rate (VCL) for Baseline, Extended and Main Profiles	Max video bit rate (VCL) for High Profile	Max video bit rate (VCL) for High 10 Profile	Max video bit rate (VCL) for High 4:2:2 and High 4:4:4 Predictive Profiles	Examples for high resolution @ frame rate (max stored frames) in Level
1	1485	99	64 kbit/s	80 kbit/s	192 kbit/s	256 kbit/s	128x96@30.9 (8) 176x144@15.0 (4)
1b	1485	99	128 kbit/s	160 kbit/s	384 kbit/s	512 kbit/s	128x96@30.9 (8) 176x144@15.0 (4)
1.1	3000	396	192 kbit/s	240 kbit/s	576 kbit/s	768 kbit/s	176x144@30.3 (9) 320x240@10.0 (3) 352x288@7.5 (2)
1.2	6000	396	384 kbit/s	480 kbit/s	1152 kbit/s	1536 kbit/s	320x240@20.0 (7) 352x288@15.2 (6)
1.3	11880	396	768 kbit/s	960 kbit/s	2304 kbit/s	3072 kbit/s	320x240@36.0 (7) 352x288@30.0 (6)
2	11880	396	2 Mbit/s	2.5 Mbit/s	6 Mbit/s	8 Mbit/s	320x240@36.0 (7) 352x288@30.0 (6)
2.1	19800	792	4 Mbit/s	5 Mbit/s	12 Mbit/s	16 Mbit/s	352x480@30.0 (7) 352x576@25.0 (6)
2.2	20250	1620	4 Mbit/s	5 Mbit/s	12 Mbit/s	16 Mbit/s	352x480@30.7(10) 352x576@25.6 (7) 720x480@15.0 (6) 720x576@12.5 (5)
3	40500	1620	10 Mbit/s	12.5 Mbit/s	30 Mbit/s	40 Mbit/s	352x480@61.4 (12) 352x576@51.1 (10) 720x480@30.0 (6) 720x576@25.0 (5)
3.1	108000	3600	14 Mbit/s	17.5 Mbit/s	42 Mbit/s	56 Mbit/s	720x480@80.0 (13) 720x576@66.7 (11) 1280x720@30.0 (5)
3.2	216000	5120	20 Mbit/s	25 Mbit/s	60 Mbit/s	80 Mbit/s	1280x720@60.0 (5) 1280x1024@42.2 (4)
4	245760	8192	20 Mbit/s	25 Mbit/s	60 Mbit/s	80 Mbit/s	1280x720@68.3 (9) 1920x1088@30.1 (4) 2048x1024@30.0 (4)
4.1	245760	8192	50 Mbit/s	62.5 Mbit/s	150 Mbit/s	200 Mbit/s	1280x720@68.3 (9) 1920x1088@30.1 (4) 2048x1024@30.0 (4)
4.2	522240	8704	50 Mbit/s	62.5 Mbit/s	150 Mbit/s	200 Mbit/s	1920x1088@64.0 (4) 2048x1088@60.0 (4)
5	589824	22080	135 Mbit/s	168.75 Mbit/s	405 Mbit/s	540 Mbit/s	1920x1088@72.3 (13) 2048x1024@72.0 (13) 2048x1088@67.8 (12) 2560x1920@30.7 (5) 3680x1536/26.7 (5)
5.1	983040	36864	240 Mbit/s	300 Mbit/s	720 Mbit/s	960 Mbit/s	1920x1088@120.5 (16) 4096x2048@30.0 (5) 4096x2304@26.7 (5)
Level number	Max macroblocks per second	Max frame size (macroblocks)	Max video bit rate (VCL) for Baseline, Extended and Main Profiles	Max video bit rate (VCL) for High Profile	Max video bit rate (VCL) for High 10 Profile	Max video bit rate (VCL) for High 4:2:2 and High 4:4:4 Predictive Profiles	Examples for high resolution @ frame rate (max stored frames) in Level

[edit] Standardization Committee and History

In early 1998 the Video Coding Experts Group (VCEG – ITU-T SG16 Q.6) issued a call for proposals on a project called H.26L, with the target to double the coding efficiency (which means halving the bit rate necessary for a given level of fidelity) in comparison to any other existing video coding standards for a broad variety of applications. VCEG was chaired by Gary Sullivan (Microsoft (formerly PictureTel), USA). The first draft design for that new standard was adopted in August 1999. In 2000, Thomas Wiegand (Heinrich Hertz Institute, Germany) became VCEG co-chair. In December of 2001, VCEG and the Moving Picture Experts Group (MPEG – ISO/IEC JTC 1/SC 29/WG 11) formed a Joint Video Team (JVT), with the charter to finalize the draft new video coding standard for formal approval submission as H.264/AVC in March 2003. The JVT was (is) chaired by Gary Sullivan, Thomas Wiegand, and Ajay Luthra (Motorola, USA). In June 2004, the Fidelity range extensions (FRExt) project was finalized. Since January 2005, the JVT has been working on an extension of H.264/AVC towards scalability by an Annex called Scalable Video Coding (SVC). The JVT management team was extended by Jens-Reiner Ohm (Aachen University, Germany). Since July 2006, the JVT works on an extension of H.264/AVC towards multi-view video coding (MVC).

[edit] Versions

Versions of the H.264/AVC standard include the following completed revisions (dates are final approval dates in ITU-T, while final "International Standard" approval dates in ISO/IEC are somewhat different and later in most cases):

• First version containing Baseline, Extended, and Main profiles (May 2003).
• Corrigendum containing various minor corrections (May 2004).
• Second version containing Fidelity Range Extensions (FRExt) containing High, High 10, High 4:2:2, and High 4:4:4 profiles (March 2005).
• Corrigendum containing various minor corrections and adding three aspect ratio indicators (September 2005).
• Amendment containing various minor changes (June 2006):
  • Removal of prior High 4:4:4 profile (processed as a corrigendum in ISO/IEC).
  • Minor extension adding extended-gamut color space support (bundled with above-mentioned aspect ratio indicators in ISO/IEC).

Planned additions:

• Addition of High 4:4:4 Predictive and four Intra-only profiles (High 10 Intra, High 4:2:2 Intra, High 4:4:4 Intra, and CAVLC 4:4:4 Intra) — not yet completed.
• Scalable video coding (SVC) — not yet completed.
• Corrigendum containing various minor corrections — not yet completed.
• Multi-view coding (MVC) — not yet completed.

[edit] Patent licensing

As with MPEG-2 Parts 1 and 2 and MPEG-4 Part 2 among others, the vendors of H.264/AVC products and services are expected to pay patent licensing royalties for the patented technology that their products use (where software patent regulations are upheld). The primary source of licenses for patents applying to this standard is a private organization known as MPEG LA (which is not affiliated in any way with the MPEG standardization organization, but which also administers patent pools for MPEG-2 Part 1 Systems, MPEG-2 Part 2 Video, MPEG-4 Part 2 Video, and other technologies). In January 2007, a U.S. District court jury gave an advisory opinion that one patent owned by Qualcomm should be invalidated.^[1] Qualcomm had claimed that the patent had been incorporated in H.264 in violation of its patent.^[2][3] The U.S. District Court judge has yet to rule on the verdict.^[4]

[edit] Open Source/Free Software licensing

Discussions are often held regarding the legality of free software implementations of codecs like H.264, especially concerning its legal use GNU LGPL and GPL implementations of H.264 and other patented codecs. Consensus in discussions, and the allowable use depend on the laws of local jurisdictions. If operating and/or shipping a product in a country or group of countries where none of the patents covering H.264 apply, then using, for example, an LGPL implementation of the codec is not a problem: There is no conflict between the software license and the (non-existent) patent license.

Conversely, shipping a product in the US which includes an LGPL H.264 decoder/encoder would be in violation of the software license of the codec implementation. In simple terms, the LGPL and GPL licenses require that any rights held in conjunction with distributing and using the code also apply to anyone receiving the code, and no further restrictions are put on distribution or use. If there is a requirement for a patent license to be sought, this is a clear violation of both the GPL and LGPL terms. Thus, the right to distribute patent-encumbered code under those licenses as part of the product is revoked per the terms of the GPL and LGPL.

There have been no known court cases testing this legal interpretation to be correct, however its interpretation fits best with statements regarding the topic made by the Free Software Foundation on this patent rights issue, in cases likely to use an expert/authoritative source on interpretation of the GPL and LGPL in a possible lawsuit.^{[citation needed]}

[edit] Applications

H.264/AVC experienced widespread adoption within a few years of the completion of the standard. It is employed widely in applications ranging from television broadcast to video for mobile devices. In order to ensure compatibility and problem-free adoption of H.264/AVC, many standards bodies have amended or added to video standards so that users of these standards can employ H.264/AVC.

Both of the major candidate next-generation DVD rival formats deployed in 2006 include the H.264/AVC High Profile as a mandatory player feature — specifically:

• The HD DVD format of the DVD Forum
• The Blu-ray Disc format of the Blu-ray Disc Association (BDA)

The Digital Video Broadcast (DVB) standards body in Europe approved the use of H.264/AVC for broadcast television in Europe in late 2004. The Advanced Television Systems Committee (ATSC) standards body in the United States is considering the possibility of specifying one or two advanced video codecs for its optional Enhanced-VSB (E-VSB) transmission mode for use in U.S. broadcast television. It has included H.264/AVC and VC-1 into Candidate Standards as CS/TSG-659r2^[5] and CS/TSG-658r1^[6] respectively for this purpose. The status of terrestrial broadcast adoption in some specific countries is as follows:

• The prime minister of France announced the selection of H.264/AVC as a requirement for receivers of HDTV and pay TV channels for digital terrestrial broadcast television services (referred to as "TNT") in France in late 2004.
• The terrestrial broadcast systems in Brazil, Estonia and Slovenia are expected to use H.264/AVC for all digital television services.
• Its use has begun in Lithuania.
• The Digital Multimedia Broadcast (DMB) service in the Republic of Korea will use H.264/AVC.
• Mobile-segment terrestrial broadcast services of ISDB-T in Japan will use the H.264/AVC codec, including major broadcasters such as NHK and Fuji Television.

Direct broadcast satellite TV services will use the new standard, including:

• BBC HD is using H.264 for its terrestrial DVB-T service, available as test in the London area, also via satellite on Astra 2D @28E
• DirecTV (in the United States)
• Dish Network (in the United States)
• Euro1080 (in Europe)
• Premiere (pay television network) (in Germany)
• ProSieben HD & Sat1 HD (in Germany, ProSiebenSat.1 Media AG^[7])
• SkyHD (in the United Kingdom and Ireland) — the company's standard definition service continues to use MPEG-2
• Sky Italia (in Italy)
• SVT HD (Sveriges Television in Sweden)

USDTV announced plans to use H.264 for its pay-for-premium ATSC channels, which can only be decrypted by USDTV's set top boxes.

The 3rd Generation Partnership Project (3GPP) has approved the inclusion of H.264/AVC as an optional feature in release 6 of its mobile multimedia telephony services specifications.

The North Atlantic Treaty Organisation (NATO) and the Motion Imagery Standards Board (MISB) of the United States Department of Defense (DoD) have adopted H.264/AVC as their preferred video codec for a broad variety of military applications.

The Internet Engineering Task Force (IETF) has completed a payload packetization format (RFC 3984) for carrying H.264/AVC video using its Real-time Transport Protocol (RTP).

The Internet Streaming Media Alliance (ISMA) has adopted H.264/AVC for its new ISMA 2.0 specifications.

Based on ITU-T H.32x standards, H.264/AVC is widely used for videoconferencing. Essentially all new videoconferencing products now support it.

The International Telecommunications Union-Radiocom. Sector (ITU-R) has adopted H.264/AVC in

• ITU-R BT.1687 "Video bit-rate reduction for real-time distribution of large-screen digital imagery applications for presentation in a theatrical environment" and
• ITU-R BT.1737 "Use of the ITU-T Recommendation H.264 (MPEG-4/AVC) video source-coding method to transport high definition TV programme material" for HDTV contribution, distribution, and satellite news gathering.

In October 2005, Apple began selling H.264-encoded videos over the Internet through their iTunes Music Store.^[8] Initially selling just television series and music videos, they expanded in September 2006 to sell films.

Panasonic is planning to implement the AVC-Intra codec to introduce MPEG-4 AVC to the Broadcast community through its P2HD products.

AVCHD is an implementation of H.264 by Sony and Panasonic.

[edit] Products and Implementations

[edit] Prominent software implementations

• x264 is a GPL-licensed H.264 encoder that is used in the free VideoLAN and MEncoder transcoding applications and, as of December 2005, remains the only reasonably complete open source and free software implementation of the standard, with support for Main Profile and High Profile.^[9] A Video for Windows was available in previous revisions of x264, but it has been discontinued. x264 won an independent video codec comparison organized by Doom9.org in December 2005.^[10] Program-pack called Gordian Knot uses x264 to encode ripped DVD video material.

• The LGPL-licensed libavcodec includes a H.264 decoder. It can decode Baseline Profile, Main Profile and High Profile video, except PAFF interlaced video. It is used in many programs like in the free VLC media player and MPlayer multimedia players, and in ffdshow and FFmpeg decoders projects.

• CoreAVC by CoreCodec is a highly optimized commercial H.264 decoder. According to independent tests by people on the Doom9.org forums, it is the fastest software decoder as of June 2006. The standard version supports Baseline Profile, Main Profile and High profile, except interlaced video. The professional edition supports both PAFF and MBAFF interlaced video beginning from version 1.1. The professional edition also supports speedups on SMP capable systems.

• Nero Digital, co-developed by Nero AG and Ateme, includes an H.264 encoder and decoder (as of September 2005, corresponding to Main Profile, except interlaced video support), along with other MPEG-4 compatible technologies. It was updated in 2006 to support High Profile.

• Apple Computer has integrated H.264 into Mac OS X version 10.4 (Tiger), as well as QuickTime version 7. The encoder conforms to Main Profile. The decoder supports Baseline, Extended, and most of Main Profile.^[11] QuickTime 7 is also now available for Microsoft's Windows operating system. Apple's iChat video conferencing software uses H.264, as does its latest version of DVD Studio Pro. DVD Studio Pro allows for the burning of HD-DVD content to both standard DVDs and HD-DVD media. Apple's Compressor can also encode in H.264 format.

• BT Group offers a modular implementation of H.264. Written in C/C++, it has been ported to various platforms from PCs to mobile phones. All 4:2:0 profiles (Baseline/Main/Extended/High) are supported.^[12]

• MainConcept H.264/AVC SDK offers encoding and decoding in all profiles and levels supported by the standard. MainConcept also offers a stand alone encoding app.

• Sorenson offers an implementation of H.264. The Sorenson AVC Pro codec is available in Sorenson Squeeze 4.1 for MPEG-4.

• Vanguard Software Solutions provides H.264/AVC Encoder and Decoder SDKs for most of the commonly used platforms as well as DirectShow solutions for Windows. Live video streaming is the main target.

[edit] Software encoder feature comparison

	QuickTime	Nero Digital	x264	Mainconcept	Elecard	Telestream
I and P Slices	Yes	Yes	Yes	Yes	Yes	Yes
B Slices	Yes	Yes	Yes	Yes	Yes	Yes
SI and SP Slices	No	No	No	No	No	No
Multiple Reference Frames	Yes	Yes	Yes	Yes	Yes	Yes
In-Loop Deblocking Filter	Yes	Yes	Yes	Yes	Yes	Yes
CAVLC Entropy Coding	Yes	Yes	Yes	Yes	Yes	Yes
CABAC Entropy Coding	No	Yes	Yes	Yes	Yes	Yes
Flexible Macroblock Ordering (FMO)	No	No	No	No	No	No
Arbitrary Slice Ordering (ASO)	No	No	No	No	No	No
Redundant Slices (RS)	No	No	No	No	No	No
Data Partitioning	No	No	No	No	No	No
Interlaced Coding (PicAFF, MBAFF)	No	No	Yes	Yes	Yes	No
4:2:0 Chroma Format	Yes	Yes	Yes	Yes	Yes	Yes
Monochrome Video Format (4:0:0)	No	No	No	No	Yes	No
4:2:2 Chroma Format	No	No	No	No	No	Yes
4:4:4 Chroma Format	No	No	No	No	No	No
8 Bit Sample Depth	Yes	Yes	Yes	Yes	Yes	Yes
9 and 10 Bit Sample Depth	No	No	No	No	No	No
11 to 14 Bit Sample Depth	No	No	No	No	No	No
8x8 vs. 4x4 Transform Adaptivity	No	Yes	Yes	Yes	Yes	Yes
Quantization Scaling Matrices	No	No	Yes	No	No	No
Separate Cb and Cr QP control	No	No	Yes	Yes	Yes	No
Separate Color Plane Coding	No	No	No	No	No	No
Predictive Lossless Coding	No	No	Yes	No	Yes	No
Film Grain Modelling	No	No	No	No	No	No
	QuickTime	Nero Digital	x264	Mainconcept	Elecard	Telestream

[edit] Prominent hardware implementations

[edit] Decoding

Several companies are mass-producing custom chips capable of decoding H.264/AVC video. Chips capable of real-time decoding at HDTV picture resolutions include these:

• Broadcom BCM7411, BCM7401
• Conexant CX2418X
• Sigma Designs SMP8630, EM8622L, and EM8624L
• STMicroelectronics STB7100, STB7109, NOMADIK (STn 8800/8810/8815/8820 series)
• Micronas DeCypher 8100^[13]
• Texas Instruments TMS320DM642, TMS320DM643x, and TMS320DM644x DSPs based on DaVinci Technology ( except for 1080i/p )

Such chips will allow widespread deployment of low-cost devices capable of playing H.264/AVC video at standard-definition and high-definition television resolutions.

Many other hardware implementations are deployed in various markets, ranging from inexpensive consumer electronics to real-time FPGA-based encoders for broadcast. A few of the more familiar hardware product offerings for H.264/AVC include these:

• Imagination Technologies Ltd. licensable IP cores for SoC development. VXD-370 HD Decoder H.264 with Baseline, Main and High Profile support up to Level 4.1 (50Mbps). Also decodes VC-1 (WMV9), MPEG-4, MPEG-2, JPEG. www.imgtec.com.

• ATI Technologies' newest graphics processing unit (GPU), the Radeon X1000-series, features hardware acceleration of H.264 decoding starting in the Catalyst 5.13 drivers.

• NVIDIA has released drivers for hardware H.264 decoding on its GeForce 7 Series and some GeForce 6 Series GPUs.^[14]

• Micronas anounced MicRacer a hardware H.264 decoding PCIe add-in card reference design.^[15]

• Apple's 5th Generation iPod can play H.264 Baseline Profile up to Level 3 with support for bit rates up to 1.5 Mbit/s, image resolutions up to 640x480, and frame rates up to 30 frames per second. This device also plays MPEG-4 Part 2 Simple Profile video, up to 2.5 Mbps, 640x480 pixels, 30 frames per sec. Additionally, video of up to 720x480 (NTSC DVD) encoded in the iPod compliant H.264 profile may be viewed on the device; if transferred with an iTunes alternative. Playback at full DVD resolution does not require any firmware modification to the iPod.

• The Sony Playstation Portable features hardware decoding of H.264 video from UMD disks and Memory Stick Pro Duo flash cards. The device supports Main Profile up to Level 2.1 with bit rates up to 768 kbit/s from the Memory Stick. Video files on the Memory Stick are still limited to resolutions up to 320x240, while UMD-Video and PMF file resolutions can be up to 480x272 with bitrates of 1+ Mbit/s.

• The Microsoft Xbox 360 features a separate HD-DVD drive that plugs into the console via USB that can play back HD-DVD discs, which includes HD-DVD discs using the H.264 codec.

• The Sony Playstation 3, which can work as a full-blown Blu-ray movie player, is able to play Blu-ray movies encoded with the standard H.264 codec. It can also play H.264 MP4 files at 720p60, as well as AVC-HD format DVD (An implementation of H.264 for Sony/Panasonic/JVC DVD camcoder).

• 4i2i Communications has implemented a complete HDTV capable High Profile decoder IP core that will fit onto an inexpensive FPGA device.^[16]

• Certain models of Nokia and SonyEricsson mobile phone can playback H.264

[edit] Encoding

Fujitsu has announced a 1080i encoding/decoding IC that will be introduced in March 2007, priced at 120 USD. The chip will be produced in a 90 nm process and will support High Profile Level 4 (up to 25 Mbit/s).^[17]

The DMS-02 media processor from 3DLabs promises to encode D1 video at 30 fps^[18] (equivalent to High 4:2:2 Profile, Level 3).

Ambarella claims to be able to encode 1080i at below one watt of power consumption in their A1 SoC platform.^[19]

[edit] References

[edit] See also

• Video
• Codec
• H.263
• x264
• MPEG-2
• MPEG-4
• IPTV
• List of devices that support H.264/MPEG-4 AVC
• ITU-T Video Coding Experts Group (VCEG)
• ISO/IEC Moving Picture Experts Group (MPEG)

[edit] External links

• H.264/AVC overview paper including new FRExt enhancements (Sullivan, Topiwala, and Luthra)
• Various papers on H.264/AVC and related topics (Wiegand)
• More papers on H.264/AVC and related topics (Marpe)
• H.264/AVC Software Coordination (Suehring)
• H.264/MPEG-4 Part 10 Tutorials (Richardson)
• Book: H.264 and MPEG-4 Video Compression (Richardson)
• H.264/AVC Textbook (in Japanese: Okubo, Kadono, Kikuchi, and Suzuki)
• JVT Experts Group document archive
• MPEG LA Terms of H.264/MPEG-4 AVC Patent License
• MPEG Industry Forum
• AVC Alliance
• ITU-T official publication page
• ISO official publication page
• W&W Communications H.264 Overview and IEEE Paper
• NVIDIA PureVideo - High-definition H.264
• Apple HD Gallery Using H.264 (requires QuickTime 7)
• Apple's HD Trailers Using H.264 (requires QuickTime 7)
• Annual MPEG-4 AVC/H.264 video codecs comparisons by Moscow State University:
  • 1-st annual MPEG-4 AVC/H.264 codecs comparison (2004)
  • 2-nd annual MPEG-4 AVC/H.264 codecs comparison (2005)
  • 3-rd annual MPEG-4 AVC/H.264 codecs comparison with cross-years report (2006)
• Overview and links on doom9
• List of H.264/AVC resources and codecs
• Video tutorial on how to encode h.264 with Megui

v • d • e Multimedia compression formats
Video compression formats	ISO/IEC MPEG-1 · MPEG-2 · MPEG-4 · MPEG-4/AVC ITU-T H.261 · H.262 · H.263 · H.264 Others AVS · Bink · Dirac · Indeo · MJPEG · RealVideo · Theora · VC-1 · VP6 · VP7 · WMV
Audio compression formats	ISO/IEC MPEG MPEG-1 Layer III (MP3) · MPEG-1 Layer II · AAC · HE-AAC ITU-T G.711 · G.722 · G.722.1 · G.722.2 · G.723 · G.723.1 · G.726 · G.728 · G.729 · G.729.1 · G.729a Others AC3 · Apple Lossless · ATRAC · FLAC · iLBC · Monkey's Audio · μ-law · Musepack · Nellymoser · RealAudio · SHN · Speex · Vorbis · WavPack · WMA
Image compression formats	ISO/IEC/ITU-T JPEG · JPEG 2000 · lossless JPEG · JBIG · JBIG2 · PNG  · WBMP Others APNG  · BMP · GIF · ILBM · PCX · TGA · TIFF · HD Photo
Media container formats	General 3GP · ASF · AVI · DMF · DPX · FLV · Matroska · MP4 · MXF · NUT · Ogg · Ogg Media · QuickTime · RealMedia · VOB Audio only AIFF · AU · WAV