Abstract
Scalable video quality enhancement refers to the process of enhancing low quality frames using high quality ones in scalable video bitstreams with time-varying qualities. A key problem in the enhancement is how to search for correspondence between high quality and low quality frames. Previous algorithms usually use block-based motion estimation to search for correspondences. Such an approach can hardly estimate scale and rotation transforms and always introduces outliers to the motion estimation results. In this paper, we propose a pixel-based outlier-free motion estimation algorithm to solve this problem. In our algorithm, the motion vector for each pixel is calculated with respect to estimate translation, scale, and rotation transforms. The motion relationships between neighboring pixels are considered via the Markov random field model to improve the motion estimation accuracy. Outliers are detected and avoided by taking both blocking effects and matching percentage in scale-invariant feature transform field into consideration. Experiments are conducted in two scenarios that exhibit spatial scalability and quality scalability, respectively. Experimental results demonstrate that, in comparison with previous algorithms, the proposed algorithm achieves better correspondence and avoids the simultaneous introduction of outliers, especially for videos with scale and rotation transforms.
Similar content being viewed by others
References
Sodagar I. The MPEG-DASH standard for multimedia Streaming Over the Internet. IEEE Multimedia, 2011, 18(4): 62–67
Schwarz H, Marpe D, Wiegand T. Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17(9): 1103–1120
Song B C, Jeong S C, Choi Y. Video super-resolution algorithm using bi-directional overlapped block motion compensation and onthefly dictionary training. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(3): 274–285
Hung E M, de Queiroz R L, Brandi F, de Oliveira K F, Mukherjee D. Video super-resolution using codebooks derived from keyframes. IEEE Transactions on Circuits and Systems for Video Technology, 2012, 22(9): 1321–1331
Ferreira R U, Hung EM, de Queiroz R L. Video super resolution based on local invariant features matching. In: Proceedings of 19th IEEE International Conference on Image Processing. 2012, 877–880
Lowe D G. Object recognition from local scale-invariant features. In: Proceedings of the 17th IEEE International Conference on Computer Vision. 1999, 1150–1157
Freeman WT, Jones T R, Pasztor E C. Example-based superresolution. IEEE Computer Graphics and Applications, 2002, 22(2): 56–65
Brandi F, de Queiroz R, Mukherjee D. Super resolution of video using key-frames. In: Proceedings of the IEEE International Symposium on Circuits Systems. 2008, 1608–1611
Brandi F, de Queiroz R L, Mukherjee D. Super-resolution of video using key-frames and motion estimation. In: Proceedings of the 15th IEEE International Conference on Image Processing. 2008, 321–324
Oliveira K F, Brandi F, Hung E M, de Queiroz R L, Mukherjee D. Bipredictive video super-resolution using key-frames. In: Proceedings of SPIE Symposium on Electronic Image, Visual Information Processing and Communication. 2010, 1–5
Hung E M, de Queiroz R L, Mukherjee D. Inter-frame postprocessing for intra-coded video. Journal of Communication and Information Systems, 2013, 28(1): 1–7
Wen J, Li S, Lu Y, Fang M, Dong X, Chang H, Tao P. Cross segment decoding for improved quality of experience for video applications. In: Proceedings of the 2013 IEEE Data Compression Conference. 2013, 231–240
Wang Q, Tang X, Shum H. Patch based blind image super resolution. In: Proceedings of the 10th IEEE International Conference on Computer Vision. 2005, 709–716
Stephenson T A, Chen T. Adaptive Markov random fields for example-based super-resolution of faces. Journal on Applied Signal Processing, 2006, 2006: 1–11
Qiu G. Interresolution look-up table for improved spatial magnification of image. Journal of Visual Communication and Image Representation. 2000, 11: 360–373
Elad M, Datsenko D. Example-based regularization deployed to superresolution reconstruction of single image. The Computer Journal Advance Access, 2007, 20: 15–30
Besag J. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B, 1974, 36: 192–293
Sun D, Roth S, Lewis J, Black M J. Learning optical flow. Lecture Notes in Computer Science, 2008, 5304: 83–97
Liu C, Yuen J, Torralba A, Sivic J, Freeman W T. SIFT flow: dense correspondence across different scenes. Lecture Notes in Computer Science, 2008, 5304: 28–42
Pan F, Lin X, Rahardja S, Lin W, Ong E, Yao S, Lu Z, Yang X. A locally adaptive algorithm for measuring blocking artifacts in images and videos. Signal Processing: Image Communication, 2004, 19(6): 499–506
Brown M, Lowe D G. Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 2007, 74(1): 59–73
Horn B, Schunck B. Determining optical flow. Artificial Intelligence, 1981, 16: 185–203
Wang S, Uchida S, Liwicki M, Feng Y K. Part-based methods for handwritten digit recognition. Frontiers of Computer Science, 2013, 7(4): 514–525
Mehrotra H, Majhi B. Local feature based retrieval approach for iris biometrics. Frontiers of Computer Science, 2013, 7(5): 767–781
PRIYA R, Shanmugama T H. Comprehensive review of significant researches on content based indexing and retrieval of visual information. Frontiers of Computer Science, 2013, 7(5): 782–799
Wang Y W, Zhou Y C, Liu Y, Luo Z, Guo D H, Shao J, Tan F, Wu L, Li J H, Yan B P. A grid-based clustering algorithm for wild bird distribution. Frontiers of Computer Science, 2013, 7(4): 475–485
Kang L, Wu L D, Yang Y H. A Novel Unsupervised Approach for Multilevel Image Clustering from Unordered Image Collection. Frontiers of Computer Science, 2013, 7(1): 69–82
Author information
Authors and Affiliations
Corresponding author
Additional information
Xuan Dong received his BS in computer science and technology from Beihang University, China, in 2010. He is a PhD candidate in the Department of Computer Science and Technology, Tsinghua University. His current research interests include computational photography, video processing, video coding, and image segmentation.
Jiangtao Wen received his BS, MS, and PhD all in electrical engineering from Tsinghua University, China in 1992, 1994, and 1996, respectively. From 1996 to 1998, he was a staff research fellow at the University of California, Los Angeles (UCLA). After UCLA, he served as the principal scientist at PacketVideo Corp., chief technical officer at Morphbius Technology Inc., director of Video Codec Technologies at Mobilygen Corp., and as a technology advisor at Ortiva Wireless and Stretch, Inc. Since 2009, he has been a professor in the Department of Computer Science and Technology, Tsinghua University. His research focuses on multimedia communication over challenging networks and computational photography.
Rights and permissions
About this article
Cite this article
Dong, X., Wen, J. A pixel-based outlier-free motion estimation algorithm for scalable video quality enhancement. Front. Comput. Sci. 9, 729–740 (2015). https://doi.org/10.1007/s11704-015-4184-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11704-015-4184-0