Jan van der Meer, Independent Consultant, The Netherlands
Jan van der Meer retired from Philips as Director of Standardization in July 2008; he is currently an independent consultant. Jan worked for Philips from 1979 until 2008 and is now an independent consultant. He has played a leading role in the MPEG standards committee almost from its very beginning, contributing not only to the developments of the MPEG-1, MPEG-2 and MPEG-4 standards, but also to their usage in specific application areas, such as specified by 3GPP, DAVIC, DVB, IETF and ISMA. Jan is worldwide acknowledged for his contributions to MPEG and other standard bodies and is (co-)inventor of 12 patents. He has chaired the MPEG Systems Group and discussed and promoted the use of MPEG technologies in a large variety of standardization bodies.

Preface xiii

About the Author xvii

Acknowledgements xxi

Part One BACKGROUNDS OF MPEG-2 SYSTEMS 1

1 Introduction 3

1.1 The Scope of This Book 7

1.2 Some Definitions 7

References 8

2 Technology Developments Around 1990 9

References 11

3 Developments in Audio and Video Coding in MPEG 13

3.1 The Need for Compression 13

3.1.1 Compression Factors for Audio 14

3.1.2 Compression Factors for Video 14

3.2 MPEG Video 19

3.2.1 Introduction 19

3.2.2 MPEG-1 and MPEG-2 Video Essentials 20

3.2.3 Evolution of MPEG Video 39

3.3 MPEG Audio 47

3.3.1 MPEG-1 and MPEG-2 Audio Essentials 47

3.3.2 Evolution of MPEG Audio 53

References 59

4 Other Important Content Formats 61

4.1 Metadata 61

4.2 Timed Text 64

4.3 Lossless and Scalable Lossless Audio 69

4.4 Multiview Video 69

4.5 3D Video 70

4.5.1 Left and Right Views in a Single Video Stream 73

4.5.2 Depth Information Associated to 2D Video 75

4.5.3 Use of MVC to Convey Left and Right Views 78

4.5.4 Further 3D Video Evolution 79

References 80

5 Motivation for a Systems Standard 83

6 Principles Underlying the MPEG-2 Systems Design 87

6.1 Building an End-to-End System 87

6.1.1 Constant End-to-End Delay 87

6.1.2 Video Coding Delay 88

6.1.3 Audio Coding Delay 94

6.1.4 Delay Compensation 95

6.2 The Multiplex and Demultiplex Operation 97

6.3 Delivery Schedule of MPEG System Streams 106

6.4 Synchronization of Audio and Video 108

6.5 MPEG-2 System Streams and the STD Model 113

6.6 Timing Issues 118

6.6.1 Frequency and Tolerance of the STC in MPEG-1 Systems 119

6.6.2 Regeneration of the STC in System Decoders 121

6.6.3 Frequency and Tolerance of the STC in MPEG-2 Systems 125

6.7 Quality of Service Issues 127

6.8 Transport Layer Independence 131

References 132

7 MPEG-1 Systems: Laying the MPEG-2 Foundation 133

7.1 Driving Forces 133

7.2 Objectives and Requirements 136

7.3 Structure of MPEG-1 System Streams 138

7.4 The MPEG-1 System Target Decoder 143

7.5 The MPEG-1 System Stream 155

7.5.1 Data Structure and Design Considerations 155

7.5.2 Constrained System Parameter Streams 161

7.5.3 Compliancy Requirements of MPEG-1 System Streams 166

7.6 MPEG-1 Applications 168

7.6.1 Compact Disc 168

7.6.2 Computers 169

7.7 Conclusions on MPEG-1 169

References 170

Part Two THE MPEG-2 SYSTEMS STANDARD 171

8 The Development of MPEG-2 Systems 173

8.1 Driving Forces 173

8.2 Objectives and Requirements 176

8.3 The Evolution of MPEG-2 Systems 178

References 185

9 Layering in MPEG-2 Systems 187

9.1 Need for Program Streams and Transport Streams 187

9.2 PES Packets as a Common Layer 188

9.3 Program Streams 189

9.4 Transport Streams 193

9.4.1 Transport Packets 193

9.4.2 Conveying PES Packets in Transport Packets 195

9.4.3 The Size of Transport Packets 196

9.4.4 Multiple Programs, PSI, Descriptors and Sections 199

9.4.5 Conveying Sections in Transport Packets 213

References 214

10 Conditional Access and Scrambling 217

10.1 Support of Conditional Access Systems 217

10.2 Scrambling in Transport Streams 219

10.3 Improving the Interoperability between CA Systems 224

10.4 Scrambling in Program Streams 225

Reference 226

11 Other Features of MPEG-2 Systems 227

11.1 Error Resiliency 227

11.2 Re-Multiplexing of Transport Streams 230

11.3 Local Program Insertion in Transport Streams 234

11.3.1 Usage of Local Program Insertions 234

11.3.2 Associated PSI Issues 235

11.3.3 Time Base Discontinuities 236

11.4 Splicing in Transport Streams 239

11.5 Variable Bitrate and Statistical Multiplexing 245

11.6 Padding and Stuffing 245

11.7 Random Access and Parsing Convenience 248

11.8 Carriage of Private Data 250

11.9 Copyright and Copy Control Support 254

11.10 Playback Trick Modes 255

11.11 Single Program and Partial Transport Streams 255

11.12 Program Stream Carriage within a Transport Stream 258

11.13 PES Streams 260

11.14 Room for Future Extensions 260

References 261

12 The MPEG-2 System Target Decoder Model 263

12.1 Introduction to the MPEG-2 STD 263

12.2 The Program Stream STD: P-STD 264

12.2.1 Description of P-STD 264

12.2.2 Buffer Management in the P-STD 267

12.2.3 CSPS: Constrained System Parameter Program Stream 268

12.2.4 Usage of P-STD for PES-STD 270

12.3 Transport Stream STD: T-STD 275

12.3.1 Description of T-STD 275

12.3.2 The Use of Transport Buffers 279

12.3.3 System Data Processing and Buffer Management 281

12.3.4 Processing of Elementary Stream Data 284

12.3.5 T-STD Buffers for Elementary Stream Decoding 288

12.3.6 Buffer Management for Elementary Stream Data 290

12.4 General STD Constraints and Requirements 290

12.5 Content Format Specific STD Issues 292

12.5.1 Decoding of MPEG Audio Streams in STD Model 292

12.5.2 Decoding of MPEG Video Streams in STD Model 295

13 Data Structure and Design Considerations 299

13.1 System Time Clock Samples and Time Stamps 299

13.2 PES Packets 301

13.3 Descriptors of Programs and Program Elements 309

13.3.1 General Format of Descriptors 309

13.3.2 Types of Descriptors 311

13.3.3 System Orientated Descriptors 311

13.3.4 General Content Descriptors 315

13.4 Program Streams 319

13.5 Sections 326

13.6 Transport Streams and Transport Packets 329

Reference 331

14 Content Support in MPEG-2 Systems 333

14.1 Introduction 333

14.2 MPEG-1 334

14.2.1 MPEG-1 Video 334

14.2.2 MPEG-1 Audio 334

14.2.3 MPEG-1 System Stream 334

14.3 MPEG-2 336

14.3.1 MPEG-2 Video 336

14.3.2 MPEG-2 (BC) Audio 338

14.3.3 MPEG-2 AAC 340

14.3.4 MPEG-2 DSM-CC 341

14.3.5 MPEG-2 System Stream 342

14.3.6 MPEG-2 IPMP 343

14.4 (ITU-T Rec.) H.222.1 343

14.5 MHEG 344

14.6 MPEG-4 345

14.6.1 MPEG-4 Visual 345

14.6.2 MPEG-4 Audio 346

14.6.3 MPEG-4 Timed Text 349

14.6.4 MPEG-4 Systems 350

14.7 AVC 354

14.8 SVC 360

14.9 3D Video 366

14.9.1 Service Compatible and Frame Compatible 3D Video 366

14.9.2 Depth or Parallax Map as Auxiliary Video Stream 369

14.9.3 MVC 370

14.10 JPEG 2000 Video 376

14.11 Metadata 377

14.12 Overview of Assigned Stream-type Values 387

References 389

15 The Real-Time Interface for Transport Streams 391

Reference 396

16 Relationship to Download and Streaming Over IP 397

16.1 IP Networks and MPEG-2 Systems 397

16.2 Streaming Over IP 397

16.3 Download 400

16.4 Carriage of MPEG-2 Systems Across IP Networks 400

16.5 Adaptive HTTP Streaming 401

References 401

17 MPEG-2 System Applications 403

18 The Future of MPEG-2 Systems 407

Reference 412

Epilogue 413

Annexes 423

Index 427

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.