High-Quality Visual Experience (eBook)

Title Page 1
Preface 4
Contents 6
Part I Quality of Visual Information 10
Automatic Prediction of Perceptual Video Quality: Recent Trends and Research Directions 11
Introduction 12
Performance Evaluation of Algorithms and Databases 13
A Brief Foray into the Human Visual System 15
Reduced-Reference Algorithms 17
Techniques Based on Watermarking 17
Other Techniques 18
No-Reference Algorithms 21
Blockiness-Based Techniques 21
Multiple Artifact Measurement Based Techniques 22
Other Techniques 24
Conclusion 26
References 27
Quality of Experience for High Definition Presentations – Case: Digital Cinema 32
Introduction 32
Role of Human Visual System in the Perception of Visual Quality 35
Cortical Processing of Stimuli 36
Perception and Attention 36
Subjective Image Quality Assessment in D-Cinema 38
Laboratory Set Up 38
Test Materials 41
Test Methods and Conditions 41
Subjects 43
Subjective Data Analysis 43
Objective Image Metrics in D-Cinema 44
Visual Characteristics in D-Cinema and Its Application to Image Quality Metrics 45
Performance Evaluation of Image Quality Metrics in D-Cinema 49
Quality as Part of the Business Plan 51
Case: D-Cinema 51
Does Quality Matter? 53
Discussions and Conclusions 54
References 56
Quality of Visual Experience for 3D Presentation – Stereoscopic Image 58
Introduction 59
Subjective Stereoscopic Image Quality Assessment 63
Experimental Materials and Methodology 63
Experimental Results and Analysis 65
Perceptual Stereoscopic Image Quality Metric Based on 2D Image Quality Metrics and Disparity Analysis 69
Introduction to 2D Image Quality Metrics 70
Performance Analysis of IQMs on 2D and Stereoscopic Image Quality Assessment 72
Perceptual Stereoscopic Quality Assessment Based on Disparity Information 75
Conclusions 81
References 82
Part II Video Coding for High Resolutions 85
The Development and Standardization of Ultra High Definition Video Technology 86
Introduction 86
Progress of Digital Video Technique 87
History of Video Coding 87
Technical Standardization for Video Coding 91
Video Coding Technology 94
The Shannon Theory and Video Coding Technology 94
Main Components of Video Coding 95
MPEG Coding Methods 106
Requirement for Quality of UHD Video System 118
Required Specifications for UHD Video Service 118
Expectation for New Video Coding Technology 123
Progress of Device Technologies Supporting UHD Video 123
Video Capture Device 123
Display 124
Storage 127
Digital Network for Communication and Broadcasting 127
Standardization of UHD Video Coding Technology 130
Realization of UHD Video Application 130
Next Generation Video Coding Standard 130
Challenge toward UHD Video Coding 131
Realization of UHD Service and System 137
Realization of UHD Service 137
Another Possibility of Video Coding Technology in Future 137
Summary 139
References 139
Compression Formats for HD Recording and Production 141
Introduction 141
720p vs. 1080i 143
H.264/AVC 145
Motion JPEG 2000 148
Experiments 151
Conclusions 158
References 159
Super Hi-Vision and Its Encoding System 161
Introduction 161
Super Hi-Vision Systems 162
Video System of SHV 162
Audio System of SHV 163
Roadmap of SHV 165
Codec Systems 165
Video Format Converter 166
MPEG-2-Based Codec 167
AVC/H.264-Based Codec 169
Demonstrations and Experiments 171
IP Transmission 171
Satellite Transmission 174
Conclusion 177
References 178
A Flexible Super High Resolution Video CODEC and Its Trial Experiments 180
Introduction 180
Basic Concept 181
Basic Concept of SHR CODEC 181
Spatially Parallel SHR CODEC System 181
Video Shift and Padding Function 183
Switchable Cascade Multiplexing Function 185
Two Synchronization Schemes 188
Rate Control in Multiplexing Mode 189
Evaluation 192
Experimental System 194
Trial 195
Conclusion 197
References 198
Mathematical Modeling for High Frame-Rate Video Signal 199
Introduction 199
Relationship between Frame-Rate and Inter-frame Prediction Error 202
Mathematical Model for Temporal Sub-sampling by Frame Skip 202
Mathematical Model of Temporal Down-Sampling by Mean Filter 206
Evaluation of Proposed Models 210
Captured High Frame-Rate Video Signal 210
Regression Analyses of the Proposed Models 210
Conclusion 216
References 216
Part III Visual Content Upscaling 218
Next Generation Frame Rate Conversion Algorithms 219
Introduction 220
Related Work 220
Region-Based FRUC by Homography Model Parameter Interpolation (FRUC-RH) 222
Motion Layer Correspondence Establishment 223
Motion Model Parameter Interpolation 224
Layer Map Interpolation 227
Middle Frame Interpolation 228
Results 229
Region-Based Multi-view FRUC by Rigid Body Motion Model Parameter Interpolation (FRUC-RM) 231
Depth-Based Moving Object Segmentation 232
3D Motion Estimation 235
Middle Frame Interpolation 236
Summary 242
References 242
Spatiotemporal Video Upscaling Using Motion-Assisted Steering Kernel (MASK) Regression 245
Introduction 245
Review of Steering Kernel Regression 248
Kernel Regression in 2-D 248
Steering Kernel Function 251
Motion Assisted Steering Kernel Regression 254
Spatiotemporal Kernel Regression 254
Motion Assisted Steering Kernel Function 255
Spatial Upscaling and Temporal Frame Interpolation 256
A Practical Video Upscaling Algorithm Based on MASK 258
Block-by-Block Processing 260
Motion Estimation and Adaptive Temporal Penalization 261
Quantization of Orientation Map 262
Adaptive Regression Order 265
Example Video Upscaling and Denoising Results 267
Conclusion 273
References 273
Temporal Super Resolution Using Variational Methods 275
Background 275
Frame Rate Requirements 277
Blur Acceptance in Human Vision 277
Related Work 278
Motion Compensated Frame Rate Conversion with Simultaneous Flow and Intensity Calculations 279
Benchmarking in Testing 280
Outline 280
Energy Minimization Formulation 280
Variational Temporal Super Resolution 280
Implementation of TSR: Frame Doubling 283
Experiments 285
Parameters 285
Evaluation Methodology 286
Frame Doubling Results 287
Discussion: Improving Variational Optic Flow for Motion Compensated Methods 294
Conclusion 294
References 295
Synthesizing Natural Images Using Spatial Layout Information 297
Introduction 297
Image Quilting 298
Discussion 300
Texture Synthesis for Natural Images 303
Texture Synthesis Based on Segmentation 305
Conclusion 308
References 308
Part IV 3D Visual Content Processing and Displaying 309
The Use of Color Information in Stereo Vision Processing 310
Introduction 310
Stereo Vision Basics 312
The Pinhole Camera Model 312
The Stereo Camera Setup 313
The Stereo Correspondence Problem 315
Stereo Vision Basics 312
The Pinhole Camera Model 312
The Stereo Camera Setup 313
The Stereo Correspondence Problem 315
The Use of Color Information in Stereo Vision 319
Color Spaces 319
A Survey of Color Based Stereo Methods 322
Comparisons and Discussion 325
Conclusion 327
References 327
3D Object Classification and Segmentation Methods 330
Introduction 330
Shape Estimation 331
3D Edge Detection 331
3D Corner Detection 334
Surface Extraction 335
3 D Image Segmentation and Classification 336
Thresholding 337
Context Modelling and Classification of 3D Objects 339
Neural Network Based Segmentation 340
Results 341
Conclusion 345
References 346
Three-Dimensional Video Contents Exploitation in Depth Camera-Based Hybrid Camera System 347
Introduction 347
Video-Plus-Depth Generation 349
Hybrid Camera System 349
Depth Calibration 351
Depth Image Generation 353
Hierarchical Decomposition of Depth Images 355
MPEG-4-Based 3D Video Contents Exploitation 358
Experimental Analysis 360
Evaluation of Depth Accuracy 360
3D Video Contents Generation 363
Conclusions 365
References 366
Improving 3D Visual Experience by Controlling the Perceived Depth Distortion 368
Introduction 369
Previous Related Knowledge 371
Viewing 371
Shooting 372
Distortion Analysis and Model 375
Shooting Design Scheme for Chosen Distortion 377
Controlled Depth Distortion 378
Perfect Depth Effect 379
Derived Shooting Technologies 380
3D Computer Graphics Software 380
3D Devices 382
Combination of Real and Virtual 3D Scenes 383
Conclusion 385
References 385
3D Visual Experience 388
Introduction 388
Stereoscopic Displays 388
Time Sequential Separation 389
Wavelength Based Separation 389
Polarization Based Separation 391
Discussion of Stereoscopic Systems 393
Stereoscopic 3D Uncompressed Image Formats 393
Multi-user Stereo and CAVE Systems 395
Head Mounted Displays 396
Autostereoscopic Displays 396
Parallax Barrier 397
Lenticular Lens 398
Wavelength Selective Filters 399
Multiview 3D Uncompressed Image Formats 399
Volumetric Displays 400
Light Field Displays 402
Integral Imaging 402
Holographic Displays 402
HoloVizio Type Light-Field Displays 403
Conclusion 404
References 404
3D Holoscopic Imaging Technology for Real- Time Volume Processing and Display 408
Background 408
3D Holoscopic Content Generation 409
Computer Generation of 3D Holoscopic Images 413
3D Object Segmentation 415
3D Holoscopic Image Compression 417
Preprocessing of 3D Holoscopic Images 418
2D WDT Based Compression Algorithm 419
Proposed Compression Algorithm 420
Simulation Results and Discussions 422
Conclusions 423
References 423
Part V Accessing Technologies for Visual Content 426
Video Streaming with Interactive Pan/Tilt/Zoom 427
Introduction 427
Related Work 431
Coding for Random Access 431
Navigation Path Prediction 433
Multicasting 434
Spatial-Random-Access-Enabled Video Coding 435
Pre-fetching Based on RoI Prediction 439
P2P Multicasting for Interactive Region-of-Interest 441
System Architecture 441
P2P Protocol 442
Protocol Performance 444
Server Bandwidth Allocation 445
Conclusions 445
References 447
End-to-End Management of Heterogeneous Environments Enabling Quality of Experience 452
Introduction 453
Architecture and Overall Concept 454
End-to-End Management Enabling QoS/QoE 454
Cross-Layer QoS/QoE Adaptation Concept 455
ENTHRONE Integrated Management Supervisor (EIMS) 456
Architecture Overview 456
End-to-End QoS Manager 457
Service Manager 458
Adaptation Manager 458
Metadata Manager 459
Search Manager 459
Multicast Manager 459
Caching and CDN Manager 460
Summary 460
Measuring Quality of Experience for MPEG-21-Based Cross-Layer Multimedia Content Adaptation 461
Introduction 461
Probing Quality of Service 461
An Interoperable QoS Model for Video Transmission Exploiting Cross-Layer Interactions 462
Conclusions 469
References 469
Quality-Driven Coding and Prioritization of 3D Video over Wireless Networks 472
Introduction 472
3D Video Transmission 474
3D Video Fundamentals 474
Perceptually Optimized Coding 476
Quality of Service for Wireless Local Area Networks 476
Robust Transmission 477
Cross-Layer Design for Optimized 3D Video Transmission 478
Design of Cross Layer Video Communications System 478
3D Video Object Segmentation for Packet Prioritization 480
Quality-Driven Coding 480
Optimized Transmission and Decoding 481
Experimental Results 483
Experimental Setup 483
Performance Evaluation 485
Subjective Quality Results 488
Conclusions 488
References 489
Scalable Indexing of HD Video 491
Introduction 491
HD Content Indexing in the Compressed Domain 492
Scalable Compression Standards 492
Scalable Extraction of Low and Mid-level Features from Compressed Streams 496
On Scalable Content-Based Queries 504
HD Content Indexing Using Patch Descriptors in the Wavelet Domain 508
Sparse Multiscale Patches and Motion Patches Descriptors 508
Using the Kullback-Leibler Divergence as a Similarity Measure 510
Scalable Content-Based Queries with Patches Descriptors 512
Conclusion and Perspectives 516
References 516
Stereo Correspondence in Information Retrieval 519
Introduction 519
Maximum Likelihood Estimation for Flat Surface Detection 521
Estimation of a Depth Map 521
Iterative RANSAC Planar Surface Detection 522
Case Study 525
Segment Based Stereo Matching 527
Colour Segmentation 528
Estimation of Disparity Planes 529
Connectivity Based Stereo Correspondence 531
Wide Baseline Stereo Correspondence Using Local Descriptors 533
Evaluation of Different Methods 535
Conclusions and Future Work 539
References 539
Author Index 542