Current Location:homepage  Research
Research Center of Intelligent Interface & Human-Computer Interaction
Editor:贾岩  Updated:2016-01-07  Views:217

1 Mission

The main Missions of Human-Computer Interaction and Visual Intelligence Center include: audio video coding technologies, content-based information retrieval from mass multimedia data, intelligent human-computer interaction, and applied algorithms. Currently, several projects from the National 973 Program, the National Fund of Sciences, the National Hi-Tech R&D Program of China(863 Program) are being studied in the lab.

2. Overview

With the growing popularity of digital imaging systems, the use of images and videos has become the most common approach to record our life. On the one hand, millions of images and videos are captured and viewedon social networks or onphoto and video sharing sites daily. On the other hand, image and video signals usually suffer from quality degradation due to the limitation of camera lens, noises in transmission channels, and compression techniques. In the era of big data, as the sheer volume of these images/videos can easily overwhelm the communication bandwidth and in-device storage, the loss image/video compression is indispensable in almost all visual communication and computing systems. At the same time, it is necessary to restore and improve the objective and subjective qualities of image/video by facilitating the image/video analysis so that it improves the pleasure of human eyes. Moreover, the field of computer vision has received an increasing attention to expand the understanding of image/video to determine what meaningful information it can extract and apply to other research domains, such as intelligent surveillance, robotics, and human-computer interaction.

The target research work in this field of the school is to develop effective algorithms to address the challenges in the era of big data, to promote theoretical progress, and to further on rate-distortion theory, sparse representation and machine learning. The milestone works in this field include the following three aspects: (1) the development of effective methods for image/video compression, some of which have become the part of international video coding standards, such as H.264/AVC, HEVC and AVS; (2) the development of several novel models and optimization strategies for image/video restoration, and achieved state-of-the-art performance; (3) the development of efficient and robust algorithms for recognition and classification of low-quality face and palm print images.

There are four professors and four associate professors working in this field, of whom five are doctoral supervisors and all eight are master supervisors. Currently, there are more than 10 doctoral students and 30 master students studying in this field. Leading professors include: Prof. Debin Zhao, Prof. Hongxun Yao, Prof. Wangmeng Zuo, Prof. Xiaopeng Fan and Associate Prof. Xianming Liu.

In the past four years (2011-2015), the faculty members in this field have been granted over 20 research projects, among which 10 are funded by National Natural Science Foundation of China (NSFC) and 1 by National Program on Key Basic Research Project (973 Program). Also they have won 1 Second Prizes of Science-Technology Progress of Ministerial or Provincial Level, 1 Best Paper Award at an international conference. More than 500 research papers have been published in international and domestic academic journals and conferences, including over 70 papers published in top journals and conferences.

70 Ph.D. students and more than 300 master students have graduated from this research field of the school. The outstanding alumni include Prof. Wen Gao (ACM Fellow and IEEE Fellow), Prof. David Zhang (IEEE Fellow and IAPR Fellow), Prof. Feng Wu (IEEE Fellow), and Prof. Xilin Chen (IEEE Fellow).

The detailed information of the research work in this field can be found in

3. Research Topics

  • Image/video compression: Video coding standardization; video content soft transmission; low complexity and low bit-rates image coding for resource-deficient wireless visual communication.

  • Image/video quality assessment: No-reference and full-reference image/video subjective quality assessment.

  • Low level vision and 3D reconstruction: Data-driven models in image denoising, super-resolution and blind deconvolution, multi-view stereo and SLAM; image generation and compositing.

  • Object detection and tracking: CNN and RNN models for object detection, new correlation filter-based models for improving the object tracking robustness to scale variations, occlusion, and distracters.

  • Image classification: Metric learning, deep learning, domain adaptation, and their applications to general image classifications tasks and several specific ones like face recognition, person re-identification.

  • Multimedia retrieval: Deep learning for multimedia representation; content based multimedia retrieval; large scale similarity search; video content analysis and understanding.

4. The Faculty

Prof. Debin Zhao

He is a professor of the School of Computer Science and Technology and the director of Research Center of Intelligent Interface & Human-Computer Interaction. He was conferred the B.S., M.S., and Ph.D. degrees in computer science from HIT in 1985, 1988, and 1998, respectively.

He has published over 200 technical articles in refereed journals and conference proceedings, such as IEEE Transon CSVT, IEEE Trans on IP, IEEE Trans on Multimedia, DCC, and CVPR. His papers have been cited more than 7000 times on Google scholar.

He has presided and accomplished more than 20 research projects. Three of these projects were awarded the Second Prize of National Science and Technology Progress, and then ten of them the Prize of Science and Technology Progress of Heilongjiang Province, respectively. He was awarded the IEEEVCIP2011and PCM2008 best paper awards. 

Prof. Hongxun Yao

Personal website:

She is a professor of the School of Computer Science and Technology, HIT. She received her B.S. and M.S. degrees in computer science from Harbin Shipbuilding Engineering Institute, Harbin, in 1987 and 1990, respectively, and the Ph.D. degree in computer science from HIT in 2003. She has authored six books and has published over 200 research papers.

Her research interests include computer vision, pattern recognition, multimedia computing, and human–computer interaction technology. Prof. Yao has received both the Honor Title of the New Century Excellent Talent in China and the Enjoy Special Government Allowances Expert in Heilongjiang, China. She is the general co-chair of ACII2005, the PC program chair of ACM ICIMCS2010, and a guest editor of IEEE Trans. on CSVT.

Her professional activities include service on various program committees for many conferences, such as World Multimedia Conference on Systemics, Cybernetics and Informatics, International Conference on Machine Learning and Cybernetics, International Conference for Young Computer Scientists.

Other researchers

  • Prof. Xiaopeng Fan, working on video coding and transmission.

  • Associate Prof. Xianming Liu, working on low-level computer vision.

5. Selected Publications

The faculty members of this field in the school publish their innovative findings, especially on top journals such as IJCV, JMLR, IEEE Trans. Image Processing (TIP), IEEE Trans. Multimedia (TMM), IEEE Trans. on Inf. Forensics and Security (TIFS), and top conferences such as CVPR, ICCV, ECCV, NIPS, ICML, ACM MM and IJCAI.

5.1 Selected Journal Papers

  1. Xianming Liu, Xiaolin Wu, Jiantao Zhou, Debin Zhao. Data-driven Soft Decoding of Compressed Images in Dual Transform-Pixel Domain. IEEE Transactions on Image Processing (TIP), 2016, 25(4): 1649-1659. (IF: 3.625)

  2. Lei Zhang, Wangmeng Zuo, David Zhang. LSDT: Latent Sparse Domain Transfer Learning for Visual Adaptation. IEEE Transactions on Image Processing (TIP), 2016, 25(3): 1177-1191. (IF: 3.625)

  3. Zhaoxin Li, Kuanquan Wang, Wangmeng Zuo, Deyu Meng, Lei Zhang. Detail-preserving Content-aware Variational multi-view stereo reconstruction. IEEE Transactions on Image Processing (TIP), 2016, 25(2): 864-877. (IF: 3.625)

  4. Xianming Liu, Deming Zhai, Debin Zhao, Guangtao Zhai, Wen Gao. Progressive Image Denoising Through Hybrid Graph Laplacian Regularization: A Unified Framework. IEEE Transactions on Image Processing (TIP), 2014, 23(4): 1491-1503. (IF: 3.625)

  5. Xianming Liu, Debin Zhao, Jiantao Zhou, Wen Gao, Huifang Sun. ImageInterpolation via Graph-Based Bayesian Label Propagation. IEEE Transactions on Image Processing (TIP),2014, 23(3): 1084-1096. (IF: 3.625)

  6. Jian Zhang, Debin Zhao, Wen Gao.Group-Based Sparse Representation for Image Restoration. IEEE Transactions on ImageProcessing (TIP), 2014, 23(8):3336-3351. (IF: 3.625)

  7. Wangmeng Zuo, Lei Zhang, Chunwei Song, David Zhang, Huijun Gao. Gradient Histogram Estimation and Preservation for Texture Enhanced Image Denoising. IEEE Transactions on Image Processing (TIP), 2014, 23(6): 2459-2472. (IF: 3.625)

  8. Rongrong Ji, Ling-Yu Duan, Jie Chen, Hongxun Yao, Junsong Yuan, Yong Rui, Wen Gao. Location Discriminative Vocabulary Coding for Mobile Landmark Search. International Journal of Computer Vision (IJCV), 2012, 96(3): 290-314. (IF: 3.810)

  9. Rongrong Ji, Hongxun Yao, Wei Liu, Xiaoshuai Sun, Qi Tian. Task-Dependent Visual-Codebook Compression. IEEE Transactions on Image Processing (TIP), 2012, 21(4):2282-2293. (IF: 3.625)

  10. Xianming Liu, Debin Zhao, Ruiqin Xiong, Siwei Ma, Wen Gao, Huifang Sun. Image Interpolation Via Regularized Local Linear Regression. IEEE Transactions on Image Processing (TIP), 2011, 20(12): 3455-3469. (IF: 3.625)

5.2 Selected Top Conference Papers

  1. Xianming Liu, Xiaolin Wu, JiantaoZhou, Debin Zhao. Data-driven Sparsity-based Restoration of JPEG-compressed Images in Dual Transform-pixel Domain. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), 2015.6.7-2015.6.12, Boston MA USA, 5171-5178.

  2. Sicheng Zhao, Yue Gao, Xiaolei Jiang, Hongxun Yao, Tat-Seng Chua, Xiaoshuai Sun. Exploring Principles-of-Art Features for Image Emotion Recognition.  Proceedings of ACM Conference on Multimedia (ACM MM 2014), 2014.113-2014.11.7, Orlando FL USA, 47-56. (Full paper).

  3. Xiaoshuai Sun, Xin-Jing Wang, Hongxun Yao, Lei Zhang. Exploring Implicit Image Statistics for Visual Representativeness Modeling. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013), 2013.6.23-2013.6.28, Portland OR USA, 516-523.

  4. Deming Zhai, Hong Chang, Yi Zhen, Xianming Liu, Xilin Chen, Wen Gao. Parametric Local Multimodal Hashing for Cross-View Similarity Search. Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI 2013), 2013.8.3-2013.8.9, Beijing China, 2754-2760.

  5. Xianming Liu, Changhu Wang, Hongxun Yao, Lei Zhang. The scale of edges.Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012), 2012.6.16-2012.6.21, Providence RI USA, 462-469.

6. Selected Research Projects

  1. National Program on Key Basic Research Project (973 Program),Research on Sparse Representation based Video Coding (Grant No. 2015CB351804). $0.7M, 2015-2015.

    This project aims to develop an effective image/video coding methods based on sparse representation and compressive sensing. The local compressive sampling strategy is investigated in order to design a low complexity and standard-complaint image/video coding scheme. Other related research topics include group dictionary learning and efficient optimization.

  2. National Program on Key Basic Research Project (973 Program), Distributed Multi-View Video Coding (Grant No. 2009CB320905). $0.55M, 2009.1-2013.12.

This project aims to study the Wyner-Ziv theory and develop effective methods for distributed multi-view video coding. The research topics include distributed video coding (DVC) scheme based on the human visual system, Witsenhausen–Wyner video coding, and joint video/depth ate allocation for multiview Video Coding.

    3.  National Natural Science Foundation of China (NSFC), Statistical Modeling based Image/Video Restoration and Compression (Grant No. 61272386). $0.12M, 2013.1-2016.12.

This project is to develop restoration models for recovering high-quality images in the problems of image blind deconvolution, non-uniform image deblurring, image super-resolution etc. Targets of investigation in this project include the auto-regressive model and sparse representation model and their applications in image restoration.

   4. The Program of Ministry of Education for New Century Excellent Talents, Image Processing and Pattern Recognition. $0.09M, 2013.1-2015.12.

This project is to develop efficient and robust algorithms for recognition and classification of low-quality face and palmprint images. A kernel classification framework for metric learning is put under investigation, which can not only generalize many popular metric learning methods such as large margin nearest neighbor (LMNN) and information theoretic metric learning (ITML), but also suggest new metric learning methods.

7 .Selected Awards

  1. Heilongjiang Province Natural Science Award, second-class prize. Theory and Algorithms of Mass Data Management. Prof. Hongxun Yao. 2011. This award is for the study of the fundamental scientific problems in the computer vision, linking the machine learning methodologies with the visual feature representation towards a very strong and systematic understanding of various general or specific computer vision systems.

  2. IEEE Visual Communication and Image Processing 2011 Best Paper Award. High-Quality Image Restoration from Partial Random Samples in Spatial Domain. Dr. Jian Zhang, Prof. Ruiqin Xiong, Prof. Siwei Ma, Prof. Debin Zhao. 2011. This paper won the only one best paper award from more than 300 submitted papers.

8. Social Contribution

After years of efforts, many original research fruits are achieved in our center. More than 400 academic papers are published on the domestic and/or international journals or conferences. Currently, our center’s multi-functional perception technology, digital audio and video encoding and decoding technology, voice recognition and synthesis and brain-computer interaction, etc, have won 4 National Scientific and Technological Advancement Awards (second class),oneNational Natural Science Award (second class), one second prize for National Technological InventionAward, and more than 10 provincial and ministerial awards; a large number of original work and technical achievements have been completed, extensively serving the national economy and social construction.

  • Multi-functional perception theory and technology
     Weestablished theeffective interactive mechanism model based on multi-functional perception, and solved the problem of large vocabulary Chinese sign language recognition and synthesis. This work has promoted the application and intensification of intelligent human-computer interaction and multi-functional perception technologies in the construction of a harmonious society, and has produced important influences at home and abroad. The research achievement Chinese Sign Language Recognition and Synthesis Based on Multi-Function Perception Theory won the second prize of National Science and Technology Progress Award.

  • Efficient image and video coding theory and key technologies

Combining the country's major needs, it has gained influence and discourse power for our country in the international standardization organization of data compression for interactive media. Many technologies of our center are adopted by standards ISO/IEC 14496-5, 14496-7, 14496-10 and ITU-T H.264, providing a complete solution for their application. At the same time, as the main contribution team of the AVS video standard, wemade a major contribution to the formulation of China's national standardsand brought profound influence on the information industry in China. The research achievement Efficient Digital Video Coding and Decoding Technology and its application in international standards and national standards” won the second prize for national technology invention award.