凌风探梅

Research URL

http://cg.cs.tsinghua.edu.cn/research_main.htm

Research Projects

Intelligent processing of internet visual media, Innovative Research Groups, National Natural Science Foundationh, PI: SHi-Min Hu, Project Number: 61521002, 2016-2021.

2016

Multiphase SPH Simulation for Interactive Fluids and Solids
ACM Transactions on Graphics, Vol. 35, No. 4. ACM SIGGRAPH 2016

Xiao Yan, Yun-Tao Jiang, Chen-Feng Li, Ralph R. Martin, and Shi-Min Hu

This work extends existing multiphase-fluid SPH frameworks to cover solid phases, including deformable bodies and granular materials.In our extended multiphase SPH framework, the distribution and shapes of all phases, both fluids and solids, are uniformly representedby their volume fraction functions. The dynamics of the multiphase system is governed by conservation of mass and momentumwithin different phases. The behavior of individual phases and the interactions between them are represented by correspondingconstitutive laws, which are functions of the volume fraction fields and the velocity fields. Our generalized multiphase SPH frameworkdoes not require separate equations for specific phases or tedious interface tracking. As the distribution, shape and motion of eachphase is represented and resolved in the same way, the proposed approach is robust, efficient and easy to implement. Various simulationresults are presented to demonstrate the capabilities of our new multiphase SPH framework, including deformable bodies, granularmaterials, interaction between multiple fluids and deformable solids, flow in porous media, and dissolution of deformable solids.

PlenoPatch: Patch-based Plenoptic Image Manipulation
IEEE Transactions on Visualization and Computer Graphics, 2016, to appear.

Fang-Lue Zhang, Jue Wang, Eli Shechtman, Zi-Ye Zhou, Jia-Xin Shi, and Shi-Min Hu

Patch-based image synthesis methods have been successfully applied for various editing tasks on still images, videos and stereopairs. In this work we extend patch-based synthesis to plenoptic images captured by consumer-level lenselet-based devices for interactive,efficient light field editing. In our method the light field is represented as a set of images captured from different viewpoints. We decomposethe central view into different depth layers, and present it to the user for specifying the editing goals. Given an editing task, our methodperforms patch-based image synthesis on all affected layers of the central view, and then propagates the edits to all other views. Interactionis done through a conventional 2D image editing user interface that is familiar to novice users. Our method correctly handles object boundaryocclusion with semi-transparency, thus can generate more realistic results than previous methods. We demonstrate compelling results ona wide range of applications such as hole-filling, object reshuffling and resizing, changing object depth, light field upscaling and parallaxmagnification.

Faithful Completion of Images of Scenic Landmarks using Internet Images
IEEE Transactions on Visualization and Computer Graphics, 2016, to appear.

Zhe Zhu, Hao-Zhi Huang, Zhi-Peng Tan, Kun Xu, and Shi-Min Hu

Previous works on image completion typically aim to produce visually plausible results rather than factually correct ones. Inthis paper, we propose an approach to faithfully complete the missing regions of an image. We assume that the input image is taken at awell-known landmark, so similar images taken at the same location can be easily found on the Internet. We first download thousands ofimages from the Internet using a text label provided by the user. Next, we apply two-step filtering to reduce them to a small set of candidateimages for use as source images for completion. For each candidate image, a co-matching algorithm is used to find correspondences ofboth points and lines between the candidate image and the input image. These are used to find an optimal warp relating the two images.A completion result is obtained by blending the warped candidate image into the missing region of the input image. The completion resultsare ranked according to combination score, which considers both warping and blending energy, and the highest ranked ones are shown tothe user. Experiments and results demonstrate that our method can faithfully complete images.

Support Substructures: Support-Induced Part-Level Structural Representation
IEEE Transactions on Visualization and Computer Graphics, 2016, to appear.

Shi-sheng Huang, Hongbo Fu, Lingyu Wei, Shi-Min Hu

In this work we explore a support-induced structural organization of object parts. We introduce the concept of support substructures, which are special subsets of object parts with support and stability. A bottom-up approach is proposed to identify such substructures in a support relation graph. We apply the derived high-level substructures to part-based shape reshuffling between models, resulting in nontrivial functionally plausible model variations that are difficult to achieve with symmetry-induced substructures by the state-of-the-art methods. We also show how to automatically or interactively turn a single input model to new functionally plausible shapes by structure rearrangement and synthesis, enabled by support substructures. To the best of our knowledge no single existing method has been designed for all these applications.

Efficient, Edge-Aware, Combined Color Quantization and Dithering
IEEE Transactions on Image Processing, 2016, Vol. 26, No. 3, 1152 - 1162.

Hao-Zhi Huang, Kun Xu, Ralph R. Martin, Fei-Yue Huang, and Shi-Min Hu

In this paper we present a novel algorithm to simultaneously accomplish color quantization and dithering of images. This is achieved by minimizing a perception-based cost function which considers pixel-wise differences between filtered versions of the quantized image and the input image. We use edge aware filters in defining the cost function to avoid mixing colors on opposite sides of an edge. The importance of each pixel is weighted according to its saliency. To rapidly minimize the cost function, we use a modified multi-scale iterative conditional mode (ICM) algorithm which updates one pixel a time while keeping other pixels unchanged. As ICM is a local method, careful initialization is required to prevent termination at a local minimum far from the global one. To address this problem, we initialize ICM with a palette generated by a modified mediancut method. Compared to previous approaches, our method can produce high quality results with fewer visual artifacts but also requires significantly less computational effort.

Comfort-driven disparity adjustment for stereoscopic video
Computational Visual Media, 2016, Vol.2, No. 1, 3-17

Miao Wang, Xi-Jin Zhang, Jun-Bang Liang, Song-Hai Zhang, and Ralph R. Martin

Pixel disparity¡ªthe offset of corresponding pixels between left and right views¡ªis a crucial parameter in stereoscopic three-dimensional (S3D) video, as it determines the depth perceived by thehuman visual system (HVS). Unsuitable pixel disparity distribution throughout an S3D video may lead tovisual discomfort. We present a unified and extensible stereoscopic video disparity adjustment frameworkwhich improves the viewing experience for an S3D video by keeping the perceived 3D appearance asunchanged as possible while minimizing discomfort. We first analyse disparity and motion attributes of S3Dvideo in general, then derive a wide-ranging visual discomfort metric from existing perceptual comfortmodels. An objective function based on this metric is used as the basis of a hierarchical optimisation methodto find a disparity mapping function for each input video frame. Warping-based disparity manipulationis then applied to the input video to generate the output video, using the desired disparity mappings asconstraints. Our comfort metric takes into account disparity range, motion, and stereoscopic windowviolation; the framework could easily be extended to use further visual comfort models. We demonstrate thepower of our approach using both animated cartoons and real S3D videos.

2015

3D indoor scene modeling from RGB-D data: a survey
Computational Visual Media, Vol. 1, No. 4, 267-278

Kang Chen, Yu-Kun Lai, Shi-Min Hu

3D scene modeling has long been a fundamental problem in computer graphics andcomputer vision. With the popularity of consumer-level RGB-D cameras, there is a growing interest in digitizingreal-world indoor 3D scenes. However, modeling indoor 3D scenes remains a challenging problem because of thecomplex structure of interior objects and poor quality of RGB-D data acquired by consumer-level sensors.Various methods have been proposed to tackle these challenges. In this survey, we provide an overview ofrecent advances in indoor scene modeling techniques, as well as public datasets and code libraries which canfacilitate experiments and evaluation.

Simultaneous Camera Path Optimization and Distraction Removal for Improving Amateur Video
IEEE Transactions on Image Processing, 2015, Vol.24, No.12, 5982 - 5994.

Fang-Lue Zhang, Jue Wang, Han Zhao, Ralph R. Martin, Shi-Min Hu

A major difference between amateur and professional video lies in the quality of camera paths. Previous work on video stabilization hasconsidered how to improve amateur video by smoothing the camera path. In this paper, we show that additional changes to the camera path canfurther improve video aesthetics. Our new optimization method achieves multiple simultaneous goals: (i) stabilizing video content over short timescales, (ii) ensuring simple and consistent camera paths over longer time scales, and (iii) improving scene composition by automatically removingdistractions, a common occurrence in amateur video. Our approach uses an L1 camera path optimization framework, extended to handle multipleconstraints. Two-passes of optimization are used to address both low-level and high-level constraints on the camera path. Experimental and userstudy results show that our approach outputs video which is perceptually better than the input, or the results of using stabilization only.

Magic Decorator: Automatic Material Suggestion for Indoor Digital Scenes
ACM Transactions on Graphics, Vol. 34, No. 6, Article No. 232, SIGGRAPH ASIA 2015.

Kang Chen, Kun Xu, Yizhou Yu, Tian-Yi Wang, Shi-Min Hu

Assigning textures and materials within 3D scenes is a tedious andlabor-intensive task. In this paper, we present Magic Decorator,a system that automatically generates material suggestions for 3Dindoor scenes. To achieve this goal, we introduce local materialrules, which describe typical material patterns for a small group ofobjects or parts, and global aesthetic rules, which account for theharmony among the entire set of colors in a specific scene. Bothrules are obtained from collections of indoor scene images. We castthe problem of material suggestion as a combinatorial optimizationconsidering both local material and global aesthetic rules. We havetested our system on various complex indoor scenes. A user studyindicates that our system can automatically and efficiently producea series of visually plausible material suggestions which are comparableto those produced by artists.

Fast Multiple-fluid Simulation Using Helmholtz Free Energy
ACM Transactions on Graphics, Vol. 34, No. 6, Article No. 201, SIGGRAPH ASIA 2015.

Tao Yang, Jian Chang, Bo Ren, Ming C. Lin, Jian Jun Zhang, and Shi-Min Hu

Multiple-fluid interaction is an interesting and common visual phenomenonwe often observe. In this paper we present an energybasedLagrangian method that expands the capability of existingmultiple-fluid methods to handle various phenomena, includingextraction, partial dissolution, etc. Based on our user-adjustedHelmholtz free energy functions, the simulated fluid evolves fromhigh-energy states to low-energy states, allowing flexible capture ofvarious mixing and unmixing processes. We also extend the originalCahn-Hilliard equation to gain abilities of simulating complexfluid-fluid interaction and rich visual phenomena such as motionrelatedmixing and position based pattern. Our approach is easy tobe integrated with existing state-of-the-art smooth particle hydrodynamic(SPH) solvers and can be further implemented on top of theposition based dynamics (PBD) method, improving the stability andincompressibility of the fluid during Lagrangian simulation underlarge time steps. Performance analysis shows that our method is atleast 4 times faster than the state-of-the-art multiple-fluid method.Examples are provided to demonstrate the new capability and effectivenessof our approach.

Efficient Construction and Simplification of Delaunay Meshes
ACM Transactions on Graphics, Vol. 34, No.6, Article No.174, SIGGRAPH ASIA 2015.

Yong-Jin Liu, Chun-Xu Xu, Dian Fan, Ying He

Delaunay meshes (DM) are a special type of triangle mesh wherethe local Delaunay condition holds everywhere. We present anefficient algorithm to convert an arbitrary manifold triangle meshM into a Delaunay mesh. We show that the constructed DM hasO(Kn) vertices, where n is the number of vertices in M and Kis a model-dependent constant. We also develop a novel algorithmto simplify Delaunay meshes, allowing a smooth choice of detaillevels. Our methods are conceptually simple, theoretically soundand easy to implement. The DM construction algorithm also scaleswell due to its O(nK logK) time complexity.Delaunay meshes have many favorable geometric and numericalproperties. For example, a DM has exactly the same geometryas the input mesh, and it can be encoded by any mesh data structure.Moreover, the empty geodesic circumcircle property impliesthat the commonly used cotangent Laplace-Beltrami operator hasnon-negative weights. Therefore, the existing digital geometry processingalgorithms can benefit the numerical stability of DM withoutchanging any codes. We observe that DMs can improve theaccuracy of the heat method for computing geodesic distances.Also, popular parameterization techniques, such as discrete harmonicmapping, produce more stable results on the DMs than onthe input meshes.

Active Exploration of Large 3D Model Repositories
IEEE Transactions on Visualization and Computer Graphics, Vol. 21, No.12, 1390-1402.

Lin Gao, Yan-Pei Cao, Yu-Kun Lai, Hao-Zhi Huang, Leif Kobbelt, Shi-Min Hu

With broader availability of large-scale 3D model repositories, the need for efficient and effective exploration becomes more and more urgent. Existing model retrieval techniques do not scale well with the size of the database since often a large number of very similar objects are returned for a query, and the possibilities to refine the search are quite limited. We propose an interactive approach where the user feeds an active learning procedure by labeling either entire models or parts of them as ¡°like¡± or ¡°dislike¡± such that the system can automatically update an active set of recommended models. To provide an intuitive user interface, candidate models are presented based on their estimated relevance for the current query. From the methodological point of view, our main contribution is to exploit not only the similarity between a query and the database models but also the similarities among the database models themselves. We achieve this by an offline pre-processing stage, where global and local shape descriptors are computed for each model and a sparse distance metric is derived that can be evaluated efficiently even for very large databases. We demonstrate the effectiveness of our method by interactively exploring a repository containing over 100K models.

Anisotropic density estimation for photon mapping
Computational Visual Media, Vol. 1, No. 3, 221-228

Fu-Jun Luan, Li-Fan Wu, Kun Xu

Photon mapping is a widely used technique for global illumination rendering. In the densityestimation step of photon mapping, the indirect radiance at a shading point is estimated througha ltering process using nearby stored photons; an isotropic ltering kernel is usually used. However,using an isotropic kernel is not always the optimal choice, especially for cases when eye paths intersectwith surfaces with anisotropic BRDFs. In this paper, we propose an anisotropic ltering kernel for densityestimation to handle such anisotropic eye paths. The anisotropic ltering kernel is derived from therecently introduced anisotropic spherical Gaussian representation of BRDFs. Compared to conventionalphoton mapping, our method is able to reduce rendering errors with negligible additional cost whenrendering scenes containing anisotropic BRDFs.

Semi-Continuity of Skeletons in 2-Manifold and Discrete Voronoi Approximation
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, No. 9, 1938 - 1944.

Yong-Jin Liu

The skeleton of a 2D shape is an important geometric structure in pattern analysis and computer vision. In this paper we study the skeleton of a 2D shape in a 2-manifold $\mathcal{M}$, based on a geodesic metric. We present a formal definition of the skeleton $S(\Omega)$ for a shape $\Omega$ in $\mathcal{M}$ and show several properties that make $S(\Omega)$ distinct from its Euclidean counterpart in $\mathbb{R}^2$. We further prove that for a shape sequence $\{\Omega_i\}$ that converge to a shape $\Omega$ in $\mathcal{M}$, the mapping $\Omega\rightarrow\overline{S}(\Omega)$ is lower semi-continuous. A direct application of this result is that we can use a set $P$ of sample points to approximate the boundary of a 2D shape $\Omega$ in $\mathcal{M}$, and the Voronoi diagram of $P$ inside $\Omega\subset\mathcal{M}$ gives a good approximation to the skeleton $S(\Omega)$. Examples of skeleton computation in topography and brain morphometry are illustrated.

A simple approach for bubble modelling from multiphase fluid simulation
Computational Visual Media, Vol. 1, No. 2, 171-181

Bo Ren, Yuntao Jiang, Chenfeng Li, Ming C. Lin

This article presents a novel and flexible bubble modelling technique for multi-fluid simulations using a volume fraction representation. By combining the volume fraction data obtained from a primary multi-fluid simulation with simple and efficient secondary bubble simulation, a range of real-world bubble phenomena are captured with a high degree of physical realism, including large bubble deformation, sub-cell bubble motion, bubble stacking over the liquid surface, bubble volume change, dissolving of bubbles, etc. Without any change in the primary multi-fluid simulator, our bubble modelling approach is applicable to any multi-fluid simulator based on the volume fraction representation.

PatchTable: Efficient Patch Queries for Large Datasets and Applications
ACM Transactions on Graphics, Vol. 34, No. 4, Article No. 97, SIGGRAPH 2015.

Connelly Barnes, Fang-Lue Zhang, Liming Lou, Xian Wu, Shi-Min Hu

This paper presents a data structure that reduces approximate nearestneighbor query times for image patches in large datasets. Previouswork in texture synthesis has demonstrated real-time synthesisfrom small exemplar textures. However, high performancehas proved elusive for modern patch-based optimization techniqueswhich frequently use many exemplar images in the tens of megapixelsor above. Our new algorithm, PatchTable, offloads as muchof the computation as possible to a pre-computation stage thattakes modest time, so patch queries can be as efficient as possible.There are three key insights behind our algorithm: (1) a lookuptable similar to locality sensitive hashing can be precomputed, andused to seed sufficiently good initial patch correspondences duringquerying, (2) missing entries in the table can be filled during precomputationwith our fast Voronoi transform, and (3) the initiallyseeded correspondences can be improved with a precomputed knearestneighbors mapping. We show experimentally that this acceleratesthe patch query operation by up to 9 over k-coherence,up to 12 over TreeCANN, and up to 200 over PatchMatch. Ourfast algorithm allows us to explore efficient and practical imagingand computational photography applications. We show resultsfor artistic video stylization, light field super-resolution, and multiimageediting.

Panorama completion for street views
Computational Visual Media, Vol. 1, No. 1, 49-57

Zhe Zhu, Ralph R. Martin, Shi-Min Hu

This paper considers panorama images used for street views. Their viewing angle of 360 degree causes pixels at the top and bottom to appear stretched and warped. Although current image completion algorithms work well, they cannot be directly used in the presence of such distortions found in panoramas of street views. We thus propose a novel approach to complete such 360 degree panoramas using optimization-based projection to deal with distortions. Experimental results show that our approach is efficient and provides an improvement over standard image completion algorithms.

Fast Wavefront Propagation (FWP)for Computing Exact Geodesic Distances on Meshes
IEEE Transactions on Visualization and Computer Graphics, 2015, Vol 21, No. 7, 822-834.

Chunxu Xu, Tuanfeng Y. Wang, Yong-Jin Liu, Ligang Liu, Ying He

Computing geodesic distances on triangle meshes is a fundamental problem in computational geometry and computergraphics. To date, two notable classes of algorithms, the Mitchell-Mount-Papadimitriou (MMP) algorithm and the Chen-Han (CH)algorithm, have been proposed. Although these algorithms can compute exact geodesic distances if numerical computation is exact,they are computationally expensive, which diminishes their usefulness for large-scale models and/or time-critical applications. In thispaper, we propose the fast wavefront propagation (FWP) framework for improving the performance of both the MMP and CHalgorithms. Unlike the original algorithms that propagate only a single window (a data structure locally encodes geodesic information) ateach iteration, our method organizes windows with a bucket data structure so that it can process a large number of windowssimultaneously without compromising wavefront quality. Thanks to its macro nature, the FWP method is less sensitive to meshtriangulation than the MMP and CH algorithms. We evaluate our FWP-based MMP and CH algorithms on a wide range of large-scalereal-world models. Computational results show that our method can improve the speed by a factor of 3-10.

A Response Time Model for Abrupt Changes in Binocular Disparity
The Visual Computer, 2015, Vol. 31, N0. 5, 675-687.

Tai-Jiang Mu, Jia-Jia Sun, Ralph Martin, Shi-Min Hu

We propose a novel depth perception modelto determine the time taken by the human visual system (HVS) to adapt to an abrupt change in stereoscopic disparity, such as can occur in a scene cut. A seriesof carefully designed perceptual experiments on successive disparity contrast were used to build our model.Factors such as disparity, changes in disparity, and the spatial frequency of luminance contrast were taken intoaccount. We further give a computational method to predict the response time during scene cuts in stereoscopic cinematography, which has been validated in user studies. We also consider various applications of ourmodel.

Global Contrast based Salient Region Detection
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,Vol. 37, No. 3, 569 - 582.

Ming-Ming Cheng, Niloy J. Mitra, Xiaolei Huang, Philip H. S. Torr, and Shi-Min Hu
(Earlier version was presented in IEEE CVPR 2011)

Automatic estimation of salient object regions across images, without any prior assumption or knowledge of the contentsof the corresponding scenes, enhances many computer vision and computer graphics applications. We introduce a regional contrastbased salient object detection algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherencescores. The proposed algorithm is simple, efficient, naturally multi-scale, and produces full-resolution, high-quality saliency maps.These saliency maps are further used to initialize a novel iterative version of GrabCut, namely SaliencyCut, for high quality unsupervisedsalient object segmentation. We extensively evaluated our algorithm using traditional salient object detection datasets, as well as amore challenging Internet image dataset. Our experimental results demonstrate that our algorithm consistently outperforms 15 existingsalient object detection and segmentation methods, yielding higher precision and better recall rates. We also show that our algorithmcan be used to efficiently extract salient object masks from Internet images, enabling effective sketch-based image retrieval (SBIR) viasimple shape comparisons. Despite such noisy internet images, where the saliency regions are ambiguous, our saliency guided imageretrieval achieves a superior retrieval rate compared with state-of-the-art SBIR methods, and additionally provides important targetobject region information.

2014

BiggerPicture: Data-Driven Image Extrapolation Using Graph Matching
ACM Transactions on Graphics, 2014, Vol. 33, No. 6, Article No. 173 (ACM SIGGRAPH ASIA 2014).

Miao Wang, Yu-Kun Lai, Yuan Liang, Ralph R. Martin, Shi-Min Hu

Filling a small hole in an image with plausible content is wellstudied. Extrapolating an image to give a distinctly larger one ismuch more challenging¡ªa significant amount of additional contentis needed which matches the original image, especially nearits boundaries. We propose a data-driven approach to this problem.Given a source image, and the amount and direction(s) in which it isto be extrapolated, our system determines visually consistent contentfor the extrapolated regions using library images. As well asconsidering low-level matching, we achieve consistency at a higherlevel by using graph proxies for regions of source and library images.Treating images as graphs allows us to find candidates forimage extrapolation in a feasible time. Consistency of subgraphsin source and library images is used to find good candidates for theadditional content; these are then further filtered. Region boundarycurves are aligned to ensure consistency where image parts arejoined using a photomontage method. We demonstrate the powerof our method in image editing applications.

Improving Visual Quality of View Transitions in Automultiscopic Displays
ACM Transactions on Graphics, 2014, Vol. 33, No. 6, Article No. 192(ACM SIGGRAPH ASIA 2014).

Song-Pei Du, Piotr Didyk, Fredo Durand, Shi-Min Hu, Wojciech Matusik

Automultiscopic screens present different images depending on theviewing direction. This enables glasses-free 3D and provides motion parallax effect. However, due to the limited angular resolutionof such displays, they suffer from hot-spotting, i. e., image quality is highly affected by the viewing position. In this paper, weanalyze light fields produced by lenticular and parallax-barrier displays, and show that, unlike in real world, the light fields producedby such screens have a repetitive structure. This induces visual artifacts in the form of view discontinuities, depth reversals, and excessive disparities when viewing position is not optimal. Although the problem has been always considered as inherent to the technology,we demonstrate that light fields reproduced on automultiscopic displays have enough degrees of freedom to improve the visual quality. We propose a new technique that modifies light fields usingglobal and local shears followed by stitching to improve their continuity when displayed on a screen. We show that this enhances visual quality significantly, which is demonstrated in a series of userexperiments with an automultiscopic display as well as lenticular prints.

Automatic Semantic Modeling of Indoor Scenes from Low-quality RGB-D Data using Contextual Information
ACM Transactions on Graphics, 2014, Vol. 33, No. 6, Article 208(ACM SIGGRAPH ASIA 2014).

Kang Chen, Yu-Kun Lai, Yu-Xin Wu, Ralph Martin, Shi-Min Hu

We present a novel solution to automatic semantic modeling of indoorscenes from a sparse set of low-quality RGB-D images. Suchdata presents challenges due to noise, low resolution, occlusion andmissing depth information. We exploit the knowledge in a scenedatabase containing 100s of indoor scenes with over 10,000 manuallysegmented and labeled mesh models of objects. In seconds,we output a visually plausible 3D scene, adapting these models andtheir parts to fit the input scans. Contextual relationships learnedfrom the database are used to constrain reconstruction, ensuring semanticcompatibility between both object models and parts. Smallobjects and objects with incomplete depth information which aredifficult to recover reliably are processed with a two-stage approach.Major objects are recognized first, providing a known scenestructure. 2D contour-based model retrieval is then used to recoversmaller objects. Evaluations using our own data and two publicdatasets show that our approach can model typical real-world indoorscenes efficiently and robustly.

Multiple-fluid SPH Simulation Using a Mixture Model
ACM Transactions on Graphics, 2014, Vol. 33, No. 5, article 171.

Bo Ren, Chen-Feng Li, Xiao Yan, Ming C. Lin, Javier Bonet, and Shi-Min Hu

This paper presents a versatile and robust SPH simulation approach formultiple-fluid flows. The spatial distribution of different phases or componentsis modeled using the volume fraction representation, the dynamicsof multiple-fluid flows is captured by using an improved mixture model,and a stable and accurate SPH formulation is rigorously derived to resolvethe complex transport and transformation processes encountered inmultiple-fluid flows. The new approach can capture a wide range of realworldmultiple-fluid phenomena, including mixing/unmixing of miscibleand immiscible fluids, diffusion effect and chemical reaction etc. Moreover, the new multiple-fluid SPH scheme can be readily integrated into existingstate-of-the-art SPH simulators, and the multiple-fluid simulation is easy toset up. Various examples are presented to demonstrate the effectiveness ofour approach.

Interactive Image-Guided Modeling of Extruded Shapes
Computer Graphics Forum, 2014, Vol. 33, No. 7, 101-110 (Pacific Graphics 2014).

Yan-Pei Cao, Tao Ju, Zhao Fu, Shi-Min Hu
(This paper is one of the two Best student papers in Pacific Graphics 2014)

A recent trend in interactive modeling of 3D shapes from a single image is designing minimal interfaces, andaccompanying algorithms, for modeling a specific class of objects. Expanding upon the range of shapes thatexisting minimal interfaces can model, we present an interactive image-guided tool for modeling shapes made upof extruded parts. An extruded part is represented by extruding a closed planar curve, called base, in the directionorthogonal to the base. To model each extruded part, the user only needs to sketch the projected base shape in theimage. The main technical contribution is a novel optimization-based approach for recovering the 3D normal ofthe base of an extruded object by exploring both geometric regularity of the sketched curve and image contents.We developed a convenient interface for modeling multi-part shapes and a method for optimizing the relativeplacement of the parts. Our tool is validated using synthetic data and tested on real-world images.

Learning Natural Colors for Image Recoloring
Computer Graphics Forum, 2014, Vol. 33, No. 7, 299-308 (Pacific Graphics 2014).

Hao-Zhi Huang, Song-Hai Zhang, Ralph R. Martin, Shi-Min Hu

We present a data-driven method for automatically recoloring a photo to enhance its appearance or change aviewer¡¯s emotional response to it. A compact representation called a RegionNet summarizes color and geometricfeatures of image regions, and geometric relationships between them. Correlations between color property distributionsand geometric features of regions are learned from a database of well-colored photos. A probabilisticfactor graph model is used to summarize distributions of color properties and generate an overall probability distributionfor color suggestions. Given a new input image, we can generate multiple recolored results which unlikeprevious automatic results, are both natural and artistic, and compatible with their spatial arrangements.

Polyline-sourced geodesic Voronoi diagrams on triangle meshes
Computer Graphics Forum, 2014, Vol. 33, No. 7, 161-170 (Pacific Graphics 2014).

Chunxu Xu, Yong-Jin Liu, Qian Sun, Jinyan Li and Ying He

This paper studies the Voronoi diagrams on 2-manifold meshes based on geodesic metric (a.k.a. geodesic Voronoidiagrams or GVDs), which have polyline generators. We show that our general setting leads to situations morecomplicated than conventional 2D Euclidean Voronoi diagrams as well as point-source based GVDs, since atypical bisector contains line segments, hyperbolic segments and parabolic segments. To tackle this challenge,we introduce a new concept, called local Voronoi diagram (LVD), which is a combination of additively weightedVoronoi diagram and line-segment Voronoi diagram on a mesh triangle. We show that when restricting on a singlemesh triangle, the GVD is a subset of the LVD and only two types of mesh triangles can contain GVD edges.Based on these results, we propose an efficient algorithm for constructing the GVD with polyline generators.Our algorithm runs in O(nNlogN) time and takes O(nN) space on an n-face mesh with m generators, whereN = max{m;n}. Computational results on real-world models demonstrate the efficiency of our algorithm.

Parametric meta-filter modeling from a single example pair
The Visual Computer, 2014, Vol. 30, No.6-8, 673-684.

Shi-Sheng Huang, Guo-Xin Zhang, Yu-Kun Lai, Johannes Kopf, Daniel Cohen-Or, Shi-Min Hu

We present a method for learning a meta-filter from an example pair comprising an original image A and its filtered version A' using an unknown image filter. A metafilter is a parametric model, consisting of a spatially varying linear combination of simple basis filters. We introduce a technique for learning the parameters of the meta-filter f such that it approximates the effects of the unknownfilter, i.e., f(A) approximates A'. The meta-filter can be transferred to novel input images, and its parametric representation enables intuitive tuning of its parameters to achieve controlled variations. We show that our technique successfully learns and models meta-filters that approximate a large variety of common image filters with high accuracy both visually and quantitatively.

SalientShape: group saliency in image collections
The Visual Computer, 2014, Vol. 30, No.4, 443-453.

Ming-Ming Cheng, Niloy J. Mitra,Xiaolei Huang, Shi-Min Hu

Efficiently identifying salient objects in large image collections is essential for many applications including image retrieval, surveillance, image annotation, and object recognition. We propose a simple, fast, and effective algorithm for locating and segmenting salient objects by analysing image collections. As a key novelty, we introduce group saliency to achieve superior unsupervised salient object segmentation by extracting salient objects (in collections of pre-filtered images) that maximize between-image similarities and within-image distinctness. To evaluate our method, we construct a large benchmark dataset consisting of 15 K images across multiple categories with 6000+ pixel-accurate ground truth annotations for salient object regions where applicable. In all our tests, group saliency consistently outperforms state-of-the-art single-image saliency algorithms, resulting in both higher precision and better recall. Our algorithm successfully handles image collections, of an order larger than any existing benchmark datasets, consisting of diverse and heterogeneous images from various internet sources.

A practical algorithm for rendering interreflections with all-frequency BRDFs
ACM Transactions on Graphics, 2014, Vol. 33, No.1, Article No. 10.

Kun Xu, Yan-Pei Cao, Li-Qian Ma,Zhao Dong, Rui Wang, Shi-Min Hu

Algorithms for rendering interreflection (or indirect illumination) effects often make assumptions about the frequency range of the materials' reflectance properties. For example, methods based on Virtual Point Lights (VPLs) perform well for diffuse and semi-glossy materials but not so for highly glossy or specular materials; the situation is reversed for methods based on ray tracing. In this article, we present a practical algorithm for rendering interreflection effects with all-frequency BRDFs. Our method builds upon a spherical Gaussian representation of the BRDF, based on which a novel mathematical development of the interreflection equation is made. This allows us to efficiently compute one-bounce interreflection from a triangle to a shading point, by using an analytic formula combined with a piecewise linear approximation. We show through evaluation that this method is accurate for a wide range of BRDFs. We further introduce a hierarchical integration method to handle complex scenes (i.e., many triangles) with bounded errors. Finally, we have implemented the present algorithm on the GPU, achieving rendering performance ranging from near interactive to a few seconds per frame for various scenes with different complexity.

A Sketch-Based Approach for Interactive Organization of Video Clips
ACM Transactions on Multimedia Computing, Communications, and Applications, 2014, Vol. 11, No.1, Article No. 2.

Yong-Jin Liu, Cui-Xia Ma, Qiufang Fu, Xiaolan Fu, Sheng-Feng Qin, and Lexing Xie

With the rapid growth of video resources, techniques for efficient organization of video clips are becomingappealing in the multimedia domain. In this article, a sketch-based approach is proposed to intuitivelyorganize video clips by: (1) enhancing their narrations using sketch annotations and (2) structurizing the organizationprocess by gesture-based free-form sketching on touch devices. There are two main contributionsof this work. The first is a sketch graph, a novel representation for the narrative structure of video clips tofacilitate content organization. The second is a method to perform context-aware sketch recommendationscalable to large video collections, enabling common users to easily organize sketch annotations. A prototypesystem integrating the proposed approach was evaluated on the basis of five different aspects concerningits performance and usability. Two sketch searching experiments showed that the proposed context-awaresketch recommendation outperforms, in terms of accuracy and scalability, two state-of-the-art sketch searchingmethods. Moreover, a user study showed that the sketch graph is consistently preferred over traditionalrepresentations such as keywords and keyframes. The second user study showed that the proposed approachis applicable in those scenarios where the video annotator and organizer were the same person. The thirduser study showed that, for video content organization, using sketch graph users took on average 1/3 lesstime than using a mass-market tool MovieMaker and took on average 1/4 less time than using a state-of-theartsketch alternative. These results demonstrated that the proposed sketch graph approach is a promisingvideo organization tool.

Other publications in 2014

1. Bin Liu, Ralph Martin, Ji-Wu Huang, Shi-Min Hu, Structure Aware Visual Cryptography, Computer Graphics Forum, 2014, Vol. 33, No. 7, 141-150 (Pacific Graphics 2014).

2. Cheng-Chi Yu, Yong-Jin Liu, Tianfu Wu, Kai-Yun Li, Xiaolan Fu, A global energy optimization framework for 2.1D sketch extraction from monocular images, Graphical Models, 2014, Vol. 76, No.5, 507-521.

3. Tai-Jiang Mu, Ju-Hong Wang, Song-Pei Du, Shi-Min Hu, Stereoscopic image completion and depth recovery, The Visual Computer, 2014, Vol. 30, No.6-8, 833-843.

4. Long Zeng, Yong-Jin Liu, Jin Wang, Dong-Liang Zhang, Ming-Fai Yuen, Sketch2Jewelry: Semantic feature modeling for sketch-based jewelry design, Computers & Graphics, 2014, Vol. 38, No.1, 69-77 (Presented in CAD/Graphics 2013).

2013

Recovering a Semantic Editing History from a Before-and-After Image Pair
ACM Transactions on Graphics, Vol. 32, No.6, Article No. 194, 2013 (SIGGRAPH ASIA 2013).

Shi-Min Hu, Kun Xu, Li-Qian Ma, Bin Liu, Bi-Ye Jiang and Jue Wang

We study the problem of inverse image editing, which recovers a semantically-meaningful editing history from a source image and an edited copy. Our approach supports a wide range of commonlyused editing operations such as cropping, object insertion and removal, linear and non-linear color transformations, and spatiallyvarying adjustment brushes. Given an input image pair, we first apply a dense correspondence method between them to match edited image regions with their sources. For each edited region, we determine geometric and semantic appearance operations that have been applied. Finally, we compute an optimal editing path from the region-level editing operations, based on predefined semantic constraints. The recovered history can be used in various applications such as image re-editing, edit transfer, and image revision control.

PatchNet: A Patch-based Image Representation for Interactive Library-driven Image Editing
ACM Transactions on Graphics, Vol. 32, No.6, Article No. 196, 2013 (SIGGRAPH ASIA 2013).

Shi-Min Hu, Fang-Lue Zhang, Miao Wang, Ralph R. Martin, Jue Wang

We introduce PatchNets, a compact, hierarchical representation describingstructural and appearance characteristics of image regions,for use in image editing. In a PatchNet, an image region withcoherent appearance is summarized by a graph node, associatedwith a single representative patch, while geometric relationships betweendifferent regions are encoded by labelled graph edges givingcontextual information. The hierarchical structure of a PatchNetallows a coarse-to-fine description of the image. We show howthis PatchNet representation can be used as a basis for interactive,library-driven, image editing. The user draws rough sketches toquickly specify editing constraints for the target image. The systemthen automatically queries an image library to find semanticallycompatiblecandidate regions to meet the editing goal. Contextualimage matching is performed using the PatchNet representation, allowingsuitable regions to be found and applied in a few seconds,even from a library containing thousands of images.

3-Sweep: Extracting Editable Objects from a Single Photo
ACM Transactions on Graphics, Vol. 32, No.6, Article No. 195, 2013 (SIGGRAPH ASIA 2013).

Tao Chen, Zhe Zhu, Ariel Shamir, Shi-Min Hu, Daniel Cohen-Or

We introduce an interactive technique for manipulating simple 3Dshapes based on extracting them from a single photograph. Suchextraction requires understanding of the components of the shape,their projections, and relations. These simple cognitive tasks forhumans are particularly difficult for automatic algorithms. Thus,our approach combines the cognitive abilities of humans with thecomputational accuracy of the machine to solve this problem. Ourtechnique provides the user the means to quickly create editable 3Dparts¡ª human assistance implicitly segments a complex object intoits components, and positions them in space. In our interface,three strokes are used to generate a 3D component that snaps to theshape¡¯s outline in the photograph, where each stroke defines onedimension of the component. The computer reshapes the componentto fit the image of the object in the photograph as well as tosatisfy various inferred geometric constraints imposed by its global3D structure. We show that with this intelligent interactive modelingtool, the daunting task of object extraction is made simple.Once the 3D object has been extracted, it can be quickly edited andplaced back into photos or 3D scenes, permitting object-driven photoediting tasks which are impossible to perform in image-space.

Anisotropic Spherical Gaussians
ACM Transactions on Graphics, Vol. 32, No.6, Article No. 209, 2013 (SIGGRAPH ASIA 2013).

Kun Xu, Wei-Lun Sun, Zhao Dong, Dan-Yong Zhao, Run-Dong Wu, Shi-Min Hu

We present a novel anisotropic Spherical Gaussian (ASG) function, built upon the Bingham distribution [Bingham 1974], which is much more effective and efficient in representing anisotropic spherical functions than Spherical Gaussians (SGs). In addition to retaining many desired properties of SGs, ASGs are also rotationally invariant and capable of representing all-frequency signals. To further strengthen the properties of ASGs, we have derived approximate closed-form solutions for their integral, product and convolution operators, whose errors are nearly negligible, as validated by quantitative analysis. Supported by all these operators, ASGs can be adapted in existing SG-based applications to enhance their scalability in handling anisotropic effects. To demonstrate the accuracy and efficiency of ASGs in practice, we have applied ASGs in two important SG-based rendering applications and the experimental results clearly reveal the merits of ASGs.

A Metric of Visual Comfort for Stereoscopic Motion
ACM Transactions on Graphics, Vol. 32, No.6, Article No. 222, 2013 (SIGGRAPH ASIA 2013).

Song-Pei Du, Belen Masia, Shi-Min Hu and Diego Gutierrez

We propose a novel metric of visual comfort for stereoscopic motion, based on a series of systematic perceptual experiments. We take into account disparity, motion in depth, motion on the screen plane, and the spatial frequency of luminance contrast. We further derive a comfort metric to predict the comfort of short stereoscopic videos. We validate it on both controlled scenes and real videos available on the internet, and show how all the factors we take into account, as well as their interactions, affect viewing comfort. Last, we propose various applications that can benefit from our comfort measurements and metric.

Change Blindness Images (Spotlight paper)
IEEE Transactions on Visualization and Computer Graphics, Vol. 19, No.11, 1808-1819, 2013.

Li-Qian Ma, Kun Xu, Tien-Tsin Wong, Bi-Ye Jiang and Shi-Min Hu

Change blindness refers to human inability to recognize large visual changes between images. In this paper, we present the first computational model of change blindness to quantify the degree of blindness between an image pair. It comprises a novel context-dependent saliency model and a measure of change, the former dependent on the site of the change, and the latter describing the amount of change. This saliency model in particular addresses the influence of background complexity, which plays an important role in the phenomenon of change blindness. Using the proposed computational model, we are able to synthesize changed images with desired degrees of blindness. User studies and comparisons to state-of-the-art saliency models demonstrate the effectiveness of our model.

Flow Field Modulation
IEEE Transactions on Visualization and Computer Graphics, Vol. 19, No.10, 1708-1719, 2013.

Bo Ren, Chen-Feng Li, Ming C. Lin, Theodore Kim, and Shi-Min Hu

The nonlinear and non-stationary nature of Navier-Stokes equations produces fluid flows that can be noticeably different inappearance with subtle changes. In this paper we introduce a method that can analyze the intrinsic multiscale features of flow fieldsfrom a decomposition point of view, by using the Hilbert-Huang transform method on 3D fluid simulation. We show how this methodcan provide insights to flow styles and help modulate the fluid simulation with its internal physical information. We provide easy-toimplementalgorithms that can be integrated with standard grid-based fluid simulation methods, and demonstrate how this approachcan modulate the flow field and guide the simulation with different flow styles. The modulation is straightforward and relates directly tothe flow¡¯s visual effect, with moderate computational overhead.

Sketch2Scene: Sketch-based Co-retrieval and Co-placement of 3D Models
ACM Transactions on Graphics,Vol. 32, No. 4, Article No. 123, SIGGRAPH 2013.

(click for project webpage)
Kun Xu, Kang Chen, Hongbo Fu, Wei-Lun Sun, Shi-Min Hu

This work presents Sketch2Scene, a framework that automatically turns a freehand sketch drawing inferring multiple scene objectsto semantically valid, well arranged scenes of 3D models. Unlike the existing works on sketch-based search and composition of 3Dmodels, which typically process individual sketched objects one by one, our technique performs co-retrieval and co-placement of 3Drelevant models by jointly processing the sketched objects. This is enabled by summarizing functional and spatial relationships amongmodels in a large collection of 3D scenes as structural groups. Our technique greatly reduces the amount of user intervention neededfor sketch-based modeling of 3D scenes and fits well into the traditional production pipeline involving concept design followed by 3Dmodeling. A pilot study indicates that the 3D scenes automatically synthesized by our technique in seconds are comparable to thosemanually created by an artist in hours in terms of visual aesthetics.

Cubic Mean Value Coordinates
ACM Transactions on Graphics,Vol. 32, No. 4, Article No. 126, SIGGRAPH 2013.

(click for project webpage)
Xian-Ying Li, Tao Ju and Shi-Min Hu

We present a new method for interpolating both boundary values and gradients over a 2D polygonal domain. Despite various previousefforts, it remains challenging to define a closed-form interpolant that produces natural-looking functions while allowing flexiblecontrol of boundary constraints. Our method builds on an existing transfinite interpolant over a continuous domain, which in turnextends the classical mean value interpolant. We re-derive the interpolant from the mean value property of biharmonic functions, andprove that the interpolant indeed matches the gradient constraints when the boundary is piece-wise linear. We then give closed-formformula (as generalized barycentric coordinates) for boundary constraints represented as polynomials up to degree 3 (for values) and1 (for normal derivatives) over each polygon edge. We demonstrate the flexibility and efficiency of our coordinates in two novel applications,smooth image deformation using curved cage networks and adaptive simplification of gradient meshes.

Qualitative Organization of Collections of Shapes via Quartet Analysis
ACM Transactions on Graphics,Vol. 32, No. 4, Article No. 71, SIGGRAPH 2013.

(click for project webpage)
Shi-Sheng Huang, Ariel Shamir, Chao-Hui Shen, Hao Zhang, Alla Sheffer, Shi-Min Hu, Daniel Cohen-Or

We present a method for organizing a heterogeneous collection of 3D shapes for overview and exploration. Instead of relying onquantitative distances, which may become unreliable between dissimilar shapes, we introduce a qualitative analysis which utilizesmultiple distance measures but only in cases where the measures can be reliably compared. Our analysis is based on the notion ofquartets, each defined by two pairs of shapes, where the shapes in each pair are close to each other, but far apart from the shapes in theother pair. Combining the information from many quartets computed across a shape collection using several distance measures,we create a hierarchical structure we call categorization tree of the shape collection. This tree satisfies the topological (qualitative)constraints imposed by the quartets creating an effective organization of the shapes. We present categorization trees computed onvarious collections of shapes and compare them to ground truth data from human categorization. We further introduce the concept of degreeof separation chart for every shape in the collection and show the effectiveness of using it for interactive shapes exploration.

Manipulating Perspective in Stereoscopic Images
IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19, No. 8, 1288-1297.

Song-Pei Du, Shi-Min Hu and Ralph R Martin

Stereoscopic ("3D") devices and content relying on stereopsis are now widely available. However, traditional imageediting techniques cannot be directly used to edit stereoscopic media, as extra constraints are needed to ensure consistentchanges are made to both left and right images. This paper addresses the problem of manipulating perspective in stereoscopicpairs. We note that a straightforward approach based on depth recovery is unsatisfactory. Instead, our method relies on featurecorrespondences between stereoscopic image pairs. Given a new, user-specified perspective, we determine correspondenceconstraints under this perspective, and optimize a 2D warp for each image which preserves straight lines, and guarantees properstereopsis relative to the new camera. Experiments demonstrate that our method generates new views with suitable stereoscopicoutput which correspond well to expected projections, for a wide range of specified perspective. Various advanced camera effects,such as dolly zoom and wide angle effects, can also be readily generated for stereoscopic image pairs using our method.

Aesthetic Image Enhancement by Dependence-Aware Object Re-Composition
IEEE Transactions on Multimedia, Vol. 15, No. 7, 1480-1490, 2013.

Fang-Lue Zhang, Miao Wang, Shi-Min Hu

This paper proposes an image enhancement method to optimize photo composition, by rearranging foreground objects in the photo. To adjust objects¡¯ positions while keeping the original scene content, we first perform a novel structure dependence analysis on the image to obtain the dependencies between all background regions. To determine the optimal positions for foreground objects, we formulate an optimization problem based on widely used heuristics for aesthetically pleasing pictures. Semantic relations between foreground objects are also taken into account during optimization. The final output is produced by moving foreground objects, together with their dependent regions, to optimal positions. The results show that our approach can effectively optimize photos with single or multiple foreground objects without compromising the original photo content.

Time-Line Editing of Objects in Video
IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19, No.7, 1218-1227.

Shao-Ping Lu, Song-Hai Zhang, Jin Wei, Shi-Min Hu and Ralph R Martin

We present a video editing technique based on changing the time-lines of individual objects in video, which leavesthem in their original places but puts them at different times. This allows the production of object-level slow motion effects, fastmotion effects, or even time reversal. This is more flexible than simply applying such effects to whole frames, as new relationshipsbetween objects can be created. As we restrict object interactions to the same spatial locations as in the original video, ourapproach can produce high-quality results using only coarse matting of video objects. Coarse matting can be done efficientlyusing automatic video object segmentation, avoiding tedious manual matting. To design the output, the user interactively indicatesthe desired new life-spans of objects, and may also change the overall running time of the video. Our method rearranges thetime-lines of objects in the video whilst applying appropriate object interaction constraints. We demonstrate that, while this editingtechnique is somewhat restrictive, it still allows many interesting results.

Motion-Aware Gradient Domain Video Composition
IEEE Transactions on Image Processing, 2013, Vol. 22, No.7, 2532 - 2544.

Tao Chen, Jun-Yan Zhu, Ariel Shamir, and Shi-Min Hu

For images, gradient domain composition methods like Poisson blending offer practical solutions for uncertain object boundaries and differences in illumination conditions. However, adapting Poisson image blending to video faces new challenges due to the added temporal dimension. In video, the human eye is sensitive to small changes in blending boundaries across frames, and slight differences in motions of the source patch and target video. We present a novel video blending approach that tackles these problems by merging the gradient of source and target videos and optimizing a consistent blending boundary based on a user provided blending trimap for the source video. Our approach extends mean-value coordinates interpolation to support hybrid blending with a dynamic boundary while maintaining interactive performance. We also provide a user interface and source object positioning method that can efficiently deal with complex video sequences beyond the capabilities of alpha blending.

Internet visual media processing: a survey with graphics and vision applications
The Visual Computer, 2013, Vol. 29, No.5, 393-405.

Shi-Min Hu, Tao Chen, Kun Xu, Ming-Ming Cheng, Ralph R. Martin

In recent years, the computer graphics and computer vision communities have devoted significant attention to research based on Internet visual media resources. The huge number of images and videos continually being uploaded by millions of people have stimulated a variety of visual media creation and editing applications, while also posing serious challenges of retrieval, organization, and utilization. This article surveys recent research as regards processing of large collections of images and video, including work on analysis, manipulation, and synthesis. It discusses the problems involved, and suggests possible future directions in this emerging research area.

Mixed-Domain Edge-Aware Image Manipulation
IEEE Transactions on Image Processing, 2013, Vol. 22, No. 5, 1915 - 1925.

Xian-Ying Li, Yan Gu, Shi-Min Hu, and Ralph R. Martin

This paper gives a novel approach to edge-aware image manipulation. Our method processes a Gaussian pyramid from coarse to fine, and at each level, we apply a nonlinear filter bank to the neighborhood of each pixel. Outputs of these spatially-varying filters are merged using global optimization, and this optimization problem is solved using an explicit mixeddomain (real space and DCT transform space) solution, which is efficient, accurate, and easy-to-implement. We demonstrate applications of our method to a set of problems including detail and contrast manipulation, HDR compression, non-photorealistic rendering, and haze removal.

PoseShop: A Human Image Database and Personalized Content Synthesis
IEEE Transactions on Visualization and Computer Graphics, 2013, Vol.19, No. 5, 824-837.

Tao Chen, Ping Tan, Li-Qian Ma, Ming-Ming Cheng, Ariel Shamir and Shi-Min Hu

We present a human image database collected from online images where human figures are segmented out of their background. Theimages are organized based on action semantic, clothes attributes and indexed by the shape of their poses. The database is built by downloading,analyzing, and filtering over 3 million human images from the Internet and can be queried using either silhouette sketch or a skeleton to find agiven pose. We demonstrate the application of this database for multi-frame personalized content synthesis in the form of comic-strips, where themain character is the user or his/her friends. We address the two challenges of such synthesis, namely personalization and consistency over aset of frames, by introducing head swapping and clothes swapping techniques. We also demonstrate an action correlation analysis application toshow the usefulness of the database for vision application.

A Data-Driven Approach to Realistic Shape Morphing
Computer Graphics Forum, (Eurographics 2013), Vol. 32, No. 2, 449-457, 2013

Lin Gao, Yu-Kun Lai, Qixing Huang and Shi-Min Hu

This paper proposes a novel data-driven approach for shape morphing. Given a database with various models belonging to the same category, we treat them as data samples in the plausible deformation space. These models are then clustered to form local shape spaces of plausible deformations. We use a simple metric to reasonably represent the closeness between pairs of models. Given source and target models, the morphing problem is casted as a global optimization problem of finding a minimal distance path within the local shape spaces connecting these models. Under the guidance of intermediate models in the path, an extended as-rigid-as-possible interpolation is used to produce the final morphing. By exploiting the knowledge of plausible models, our approach produces realistic morphing for challenging cases as demonstrated by various examples in the paper.

Efficient Synthesis of Gradient Solid Textures
Graphical Models, Vol. 75, No. 3, 104-117, 2013

(An earlier version has been presented in Computaional Visual Media 2013, Beijing, and received Best paper Award)
Guo-Xin Zhang, Yu-Kun Lai and Shi-Min Hu

Solid textures require large storage and are computationally expensive to synthesize. In this paper, we propose a novel solid representation called gradient solids to compactly represent solid textures, including a tricubic interpolation scheme of colors and gradients for smooth variation and a region-based approach for representing sharp boundaries. We further propose a novel approach based on this to directly synthesize gradient solid textures from exemplars. Compared to existing methods, our approach avoids the expensive step of synthesizing the complete solid textures at voxel level and produces optimized solid textures using our representation. This avoids significant amount of unnecessary computation and storage involved in the voxel-level synthesis while producing solid textures with comparable quality to the state of the art. The algorithm is much faster than existing approaches for solid texture synthesis and makes it feasible to synthesize high-resolution solid textures in full. Our compact representation also supports efficient novel applications such as instant editing propagation on full solids.

Semi-Regular Solid Texturing from 2D Image Exemplars
IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19, No. 3, 460-469.

Song-Pei Du, Shi-Min Hu and Ralph R.Martin

Solid textures, comprising 3D particles embedded in a matrix in a regular or semi-regular pattern, are common innatural and man-made materials, such as brickwork, stone walls, plant cells in a leaf, etc. We present a novel technique forsynthesizing such textures, starting from 2D image exemplars which provide cross-sections of the desired volume texture. Theshapes and colors of typical particles embedded in the structure are estimated from their 2D cross-sections. Particle positionsin the texture images are also used to guide spatial placement of the 3D particles during synthesis of the 3D texture. Ourexperiments demonstrate that our algorithm can produce higher-quality structures than previous approaches; they are bothcompatible with the input images, and have a plausible 3D nature.

Poisson Coordinates
IEEE Transactions on Visualization and Computer Graphics, 2013, Vol.19, No. 2, 344-352.

Xian-Ying Li and Shi-Min Hu,

Harmonic functions are the critical points of a Dirichlet energy functional, the linear projections of conformal maps. Theyplay an important role in computer graphics, particularly for gradient-domain image processing and shape-preserving geometriccomputation. We propose Poisson coordinates, a novel transfinite interpolation scheme based on the Poisson integral formula, asa rapid way to estimate a harmonic function on a certain domain with desired boundary values. Poisson coordinates are an extensionof the Mean Value coordinates (MVCs) which inherit their linear precision, smoothness, and kernel positivity. We give explicit formulaefor Poisson coordinates in both continuous and 2D discrete forms. Superior to MVCs, Poisson coordinates are proved to be pseudoharmonic(i.e., they reproduce harmonic functions on n-dimensional balls). Our experimental results show that Poisson coordinateshave lower Dirichlet energies than MVCs on a number of typical 2D domains (particularly convex domains). As well as presenting aformula, our approach provides useful insights for further studies on coordinates-based interpolation and fast estimation of harmonicfunctions.

View-Dependent Multiscale Fluid Simulation
IEEE Transactions on Visualization and Computer Graphics, 2013, Vol. 19, No. 2, 178-188.

Yue Gao, Chen-Feng Li, Bo Ren and Shi-Min Hu

Fluid motions are highly nonlinear and non-stationary, with turbulence occurring and developing at different length and timescales. In real-life observations, the multiscale flow generates different visual impacts depending on the distance to the viewer. Wepropose a new fluid simulation framework that adaptively allocates computational resources according to the human visual perception.First, a 3D empirical model decomposition scheme is developed to obtain the velocity spectrum of the turbulent flow. Then, dependingon the distance to the viewer, the fluid domain is divided into a sequence of nested simulation partitions. Finally, the multiscale fluidmotions revealed in the velocity spectrum are distributed non-uniformly to these view-dependent partitions, and the mixed velocityfields defined on different partitions are solved separately using different grid sizes and time steps. The fluid flow is solved at differentspatial-temporal resolutions, such that higher-frequency motions closer to the viewer are solved at higher resolutions and vice versa.The new simulator better utilizes the computing power, producing visually plausible results with realistic fine-scale details in a moreefficient way. It is particularly suitable for large scenes with the viewer inside the fluid domain. Also, as high-frequency fluid motions aredistinguished from low-frequency motions in the simulation, the numerical dissipation is effectively reduced.

2012

Structure Recovery by Part Assembly
ACM Transactions on Graphics, Vol. 31, No. 6, Article No. 180, ACM SIGGRAPH ASIA 2012.

(click for project webpage, data set is available)
Chao-Hui Shen, Hongbo Fu, Kang Chen and Shi-Min Hu

This work presents a technique that allows quick conversion of acquired low-quality data from consumer-level scanning devices to high-quality 3D models with labeled semantic parts and meanwhile their assembly reasonably close to the underlying geometry. This is achieved by a novel structure recovery approach that is essentially local to global and bottom up, enabling the creation of new structures by assembling existing labeled parts with respect to the acquired data. We demonstrate that using only a small-scale shape repository, our part assembly approach is able to faithfully recover a variety of high-level structures from only a single-view scan of man-made objects acquired by the Kinect system, containing a highly noisy, incomplete 3D point cloud and a corresponding RGB image.

An Optimization Approach for Extracting and Encoding Consistent Maps in aShape Collection
ACM Transactions on Graphics, Vol. 31, No. 6, Article No. 167, ACM SIGGRAPH ASIA 2012.

Qi-xing Huang, Guoxin Zhang, Lin Gao, Shi-Min Hu, Adrian Butscher and Leonidas Guibas

We introduce a novel diffusion-based approach for computing high quality point-to-point maps among a collection of shapes so that several desirable properties are satisfied. The proposed approach takes as input a sparse set of initial maps between pairs of shapes (sufficient to connect the model graph) and implicitly builds a new set of pointwise maps between all pairs of shapes which aim to (1) align with the initial maps, (2) map neighboring points to neighboring points, and (3) provide cycle-consistency, so that map compositions along cycles approximate the identity map. Maps among subsets of the shapes that admit nearly perfect loop closure are highly redundant and can be compactly represented by maps from a single base shape to other shapes. Our algorithm extracts such a set of base shapes so that every other shape is ¡°covered¡± by at least one of the base shapes.

ImageAdmixture: Putting Together Dissimilar Objects from Groups
IEEE Transactions on Visualization and Computer Graphics, 2012, Vol. 18, No.11, 1849-1857.

Fang-Lue Zhang, Ming-Ming Cheng, Jiaya Jia, Shi-Min Hu

We present a semi-automatic image editing framework dedicated to individual structured object replacement from groups. Themajor technical difficulty is element separation with irregular spatial distribution, hampering previous texture and image synthesis methodsfrom easily producing visually compelling results. Our method uses the object-level operations and finds grouped elements based onappearance similarity and curvilinear features. This framework enables a number of image editing applications, including natural imagemixing, structure preserving appearance transfer, and texture mixing.

Fisheye Video Correction
IEEE Transactions on Visualization and Computer Graphics, 2012, Vol. 18, No.10, 1771-1783.

Jin Wei, Chen-Feng Li, Shi-Min Hu, Ralph Martin, and Chiew-Lan Tai

Various types of video are captured with fisheye lenses, particularly surveillance video, due to their ability to capture a widefield of view. However, distortion changes as objects in the scene move, making fisheye video difficult to interpret and uncomfortableto watch. Current still fisheye image correction methods are either limited to small angles of view, or are strongly content-dependent,and therefore not suitable for processing video streams. We present a novel scheme for fisheye video correction, which minimizestime-varying distortions and preserves salient content features in a coherent manner. Our optimization process is controlled by userannotation, and includes a comprehensive set of measures addressing different aspects of natural scene appearance. These termsare all formulated in quadratic form, leading to a quadratic programming problem which can be solved in a closed form using a sparselinear system. We illustrate our method with a range of examples, demonstrating coherent natural-looking video output in which thevisual quality of individual frames is comparable to state-of-the-art methods for still fisheye photograph correction.

Interactive Images: Proxy-based Scene Understanding for Smart Manipulation
ACM Transactions on Graphics (ACM SIGGRAPH),2012,Vol. 31, No. 4, article number 99,

Youyi Zheng, Xiang Chen, Ming-Ming Cheng, Kun Zhou, Shi-Min Hu, Niloy J. Mitra

Images are static and lack important depth information of underlying 3D scenes. We introduce interactive images in the context of man-made environments wherein objects are simple, regular, share various non-local relations (e.g., coplanarity, repetitions, etc.), and are often repeated. We present an interactive framework to create a partial scene reconstruction based on cuboid-proxies using minimal user interaction. This enables a range of intuitive image edits mimicking real-world behavior, which are otherwise difficult to achieve. Effectively, the user simply provides high-level semantic hints, while our system ensures plausible operations by conforming to the extracted non-local relations. We demonstrate our system on a range of real-world images and validate the plausibility of the results using a user study.

Other publications in 2012

1. Li-Qian Ma and Kun XU, Efficient antialiased edit propagation for images and videos, Computer & Graphics, Vol. 36, No. 8, 1005-1012.

2. Yong-Liang Yang and Chao-Hui Shen, Multi-Scale Salient Features for Analyzing 3D Shapes, Journal of Computer Science and technology, Vol. 27, No. 6, 1092-1099, 2012.

3. Long Zeng, Yong-Jin Liu, Sang-Hun Lee, Ming-Fai Yuen, Q-Complex: efficient non-manifold boundary representation with inclusion topology, Computer-Aided Design, Vol. 44, No. 11, 1115-1126, 2012.

4. Ling-Qi Yan, Yahan Zhou, Kun Xu and Rui Wang,Accurate Translucent Material Rendering under Spherical Gaussian Lights, Computer Graphics Forum, Vol. 31, No. 7, 2267-2276, 2012.

5. Long Zeng, Yong-Jin Liu, Ming Chen, Ming-Fai Yuen, Least squares quasi-developable mesh approximation, Computer Aided Geometric Design, Vol. 29, No. 7, 565-578, 2012.

6. Chen Goldberg, Tao Chen, Fang-Lue Zhang, Ariel Shamir, Shi-Min Hu, Data-Driven Object Manipulation in Images, Computer Graphics Forum, Vol. 31, No. 2, 265-274, 2012 (Eurographics 2012).

7. Tao Chen, Aidong Lu and Shi-Min Hu, Visual storylines:Semanticvisualizationofmoviesequence, Computer & Graphics, Vol. 36, No. 4, 2012.

8. Cui-Xia Ma, Yong-Jin Liu, Hong-An Wang, Dong-Xing Teng, Guo-Zhong Dai, Sketch-based Annotation and Visualization in Video Authoring, IEEE Transactions on Multimedia, Vol. 14, No. 4, pp. 1153-1165, 2012.

9. Yong-Jin Liu, Yi-Fu Zheng, Lu Lv, Yu-Ming Xuan, Xiao-Lan Fu, 3D Model Retrieval based on Color+Geometry Signatures, The Visual Computer, Vol. 28, No. 1, pp.75-86, 2012.

2011

Interactive Hair Rendering and Appearance Editing under Environment Lighting
ACM Transactions on Graphics, Vol. 30, No. 6, ACM SIGGRAPH ASIA 2011.

Kun Xu, Li-Qian Ma, Bo Ren, Rui Wang, Shi-Min Hu

We present an interactive algorithm for hair rendering and appearance editing under complex environment lighting represented as spherical radial basis functions (SRBFs). Our main contribution is to derive a compact 1D circular Gaussian representation that can accurately model the hair scattering function introducedby [Marschner et al. 2003]. The primary benefit of this representation is that it enables us to evaluate, at run-time, closed-form integrals of the scattering function with each SRBF light, resultingin efficient computation of both single and multiple scatterings. In contrast to previous work, our algorithm computes the rendering integrals entirely on the fly and does not depend on expensive precomputation. Thus we allow the user to dynamically change the hair scattering parameters, which can vary spatially. Analyses show that our 1D circular Gaussian representation is both accurate and concise. In addition, our algorithm incorporates the eccentricity of the hair. We implement our algorithm on the GPU, achieving interactive hair rendering and simultaneous appearance editing under complex environment maps for the first time.

Adaptive Partitioning of Urban Facades
ACM Transactions on Graphics, Vol. 30, No. 6, ACM SIGGRAPH ASIA 2011.

Chao-Hui Shen, Shi-Sheng Huang, Hongbo Fu, Shi-Min Hu

Automatically discovering high-level facade structures in unorganized 3D point clouds of urban scenes is crucial for applications like digitalization of real cities. However, this problem is challenging due to poor-quality input data, contaminated with severemissing areas, noise and outliers. This work introduces the conceptof adaptive partitioning to automatically derive a flexible andhierarchical representation of 3D urban facades. Our key observationis that urban facades are largely governed by concatenatedand/or interlaced grids. Hence, unlike previous automatic facadeanalysis works which are typically restricted to globally rectilineargrids, we propose to automatically partition the facade in anadaptive manner, in which the splitting direction, the number andlocation of splitting planes are all adaptively determined. Such anadaptive partition operation is performed recursively to generate ahierarchical representation of the facade. We show that the conceptof adaptive partitioning is also applicable to flexible and robustanalysis of image facades. We evaluate our method on a dozenof LiDAR scans of various complexity and styles, and the imagefacades from the eTRIMS database and the Ecole Centrale Parisdatabase. A series of applications that benefit from our approachare also demonstrated.

Online Video Stream Abstraction and Stylization
IEEE Transactions on Multimedia, vol.13, no.6, pp.1286-1294, Dec. 2011

Song-Hai Zhang, Xian-Ying Li, Shi-Min Hu, and Ralph R. Martin

This paper gives an automatic method for onlinevideo stream abstraction, producing a temporally coherent outputvideo stream, in the style with large regions of constant colorand highlighted bold edges. Our system includes two novel components.Firstly, to provide coherent and simplified output, wesegment frames, and use optical flow to propagate segmentationinformation from frame to frame; an error control strategy isused to help ensure that the propagated information is reliable.Secondly, to achieve coherent and attractive coloring of theoutput, we use a color scheme replacement algorithm specificallydesigned for an online video stream. We demonstrate real-timeperformance for CIF videos, allowing our approach to be used forlive communication and other related applications.Index Terms¡ªAbstraction, color scheme replacement, opticalflow, segmentation, temporal coherence, video stream.

A Geometric Study of V-style Pop-ups: Theories and Algorithms
ACM Transactions on Graphics 2011, Vol. 30, No. 4, ACM SIGGRAPH 2011

Xian-Ying Li, Tao Ju, Yan Gu, Shi-Min Hu

Pop-up books are a fascinating form of paper art with intriguing geometricproperties. In this paper, we present a systematic study ofa simple but common class of pop-ups consisting of patches fallinginto four parallel groups, which we call v-style pop-ups. We givesufficient conditions for a v-style paper structure to be pop-uppable.That is, it can be closed flat while maintaining the rigidity of thepatches, the closing and opening do not need extra force besidesholding two patches and are free of intersections, and the closedpaper is contained within the page border. These conditions allowus to identify novel mechanisms for making pop-ups. Basedon the theory and mechanisms, we developed an interactive toolfor designing v-style pop-ups and an automated construction algorithmfrom a given geometry, both of which guaranteeing the popuppabilityof the results.

Global Contrast based Salient Region Detection
IEEE CVPR, p. 409-416, 2011,

[bib]
Ming-Ming Cheng, Guo-Xin Zhang, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu

Reliable estimation of visual saliency allows appropriate processing of images without prior knowledge of their content, and thus remains an important step in many computer vision tasks including image segmentation, object recognition, and adaptive compression. We propose a regional contrast based saliency extraction algorithm, which simultaneously evaluates global contrast differences and spatial coherence. The proposed algorithm is simple, efficient, and yields full resolution saliency maps. Our algorithm consistently outperformed existing saliency detection methods, yielding higher precision and better recall rates, when evaluated using one of the largest publicly available data sets. We also demonstrate how the extracted saliency map can be used to create high quality segmentation masks for subsequent image processing.

Construction of Iso-contours, Bisectors and Voronoi Diagrams on Triangulated Surfaces
IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 8, 1502-1517, 2011

Yong-Jin Liu, Zhan-Qing Chen, Kai Tang

In the research of computer vision and machine perception, three-dimensional objects are usually represented by 2-manifold triangular meshes M. In this paper, we present practical and efficient algorithms to construct iso-contours, bisectors and Voronoi diagrams of point sites onM, based on an exact geodesic metric. Compared to Euclidean metric spaces, the Voronoi diagrams on M exhibit many special properties that fail all the existing Euclidean Voronoi algorithms. To provide practical algorithms for constructing geodesic-metric-based Voronoi diagrams on M, this paper studies the analytic structure of iso-contours, bisectors and Voronoi diagrams on M. After a necessary preprocessing of model M, practical algorithms are proposed for quickly obtaining full information about iso-contours, bisectors and Voronoi diagrams on M. The complexity of the construction algorithms is also analyzed. Finally three interesting applications, surface sampling and reconstruction, 3D skeleton extraction and point pattern analysis are presented that show the potential power of the proposed algorithms in pattern analysis.

Image Retargeting Quality Assessment
Computer Graphics Forum, 2011, Vol. 30, No. 2, Eurographics 2011,

Yong-Jin Liu, Xi Luo, Yu-Ming Xuan, Wen-Feng Chen, Xiao-Lan Fu

Content-aware image retargeting is a technique that can flexibly display images with different aspect ratios and simultaneously preserve salient regions in images. Recently many image retargeting techniques have been proposed. To compare image quality by different retargeting methods fast and reliably, an objective metric simulating the human vision system (HVS) is presented in this paper. Different from traditional objective assessment methods that work in bottom-up manner (i.e., assembling pixel-level features in a local-to-global way), in this paper we propose to use a reverse order (top-down manner) that organizes image features from global to local viewpoints, leading to a new objective assessment metric for retargeted images. A scale-space matching method is designed to facilitate extraction of global geometric structures from retargeted images. By traversing the scale space from coarse to fine levels, local pixel correspondence is also established. The objective assessment metric is then based on both global geometric structures and local pixel correspondence. To evaluate color images, CIE Lab color space is utilized. Experimental results are obtained to measure the performance of objective assessments with the proposed metric. The results show good consistency between the proposed objective metric and subjective assessment by human observers.

Connectedness of Random Walk Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011. 33(1): p. 200 -202..
Ming-Ming Cheng, Guo-Xin Zhang

Connectedness of random walk segmentation is examined, and novel properties are discovered, by considering electrical circuits equivalent to random walks. A theoretical analysis shows that earlier conclusions concerning connectedness of random walk segmentation results are incorrect, and counterexamples are demonstrated.

Other publications in 2011

1. Wen-Qi Zhang, Yong-Jin Liu, Approximating the Longest Paths in Grid Graphs, Theoretical Computer Science, 2011, Vol. 412, No. 39, 5340-5350.

2. Yong-Jin Liu, Kai Tang, Wen-Yong Gong, Tie-Ru Wu, Industrial Design using Interpolatory Discrete Developable Surfaces, Computer-Aided Design, 2011, Vol. 43, No. 9, 1089-1098, 2011.

3. Guo-Xin Zhang, Song-Pei Du, Yu-Kun Lai, Tianyun Ni, Shi-Min Hu, Sketch Guided Solid Texturing, Graphics Models, 2011, Vol. 73, No.3, 59-73.

4. Cui-Xia Ma, Yong-Jin Liu, Hai-Yan Yang, Dong-Xing Teng, Hong-An Wang, Guo-Zhong Dai, KnitSketch: A Sketch Pad for Conceptual Design of 2D Garment Patterns, IEEE Transactions on Automation Science and Engineering, 2011, Vol. 8, No. 2,

5. Yong-Jin Liu, Cui-Xia Ma, Dong-Liang Zhang, Easytoy: a plush toy design system using editable sketch curves, IEEE Computer Graphics & Applications, 2011, Vol. 31, No. 2,

6. Shao-Ping Lu and Song-Hai Zhang, Saliency-Based Fidelity Adaptation Preprocessing for Video Coding, Journal of Computer Science and Technology, 2011, Vol. 26, No. 1, 195-202

2010

Instant Propagation of Sparse Edits on Images and Videos
Computer Graphics Forum, Special issue of Pacific Graphics 2010, Vol. 29, No. 7, 2049-2054
Yong Li, Tao Ju, Shi-Min Hu

The ability to quickly and intuitively edit digital contents has become increasingly important in our everyday life.We propose a novel method for propagating a sparse set of user edits (e.g., changes in color, brightness, contrast,etc.) expressed as casual strokes to nearby regions in an image or video with similar appearances. Existingmethods for edit propagation are typically based on optimization, whose computational cost can be prohibitivefor large inputs. We re-formulate propagation as a function interpolation problem in a high-dimensional space,which we solve very efficiently using radial basis functions. While simple to implement, our method significantlyimproves the speed and space cost of existing methods, and provides instant feedback of propagation results evenon large images and videos.

Popup: Automatic Paper Architectures from 3D Models
ACM Transactions on Graphics 2010, Vol. 29, No. 4, ACM SIGGRAPH 2010
Xian-Ying Li, Chao-Hui Shen, Shi-Sheng Huang, Tao Ju, Shi-Min Hu

Paper architectures are 3D paper buildings created by folding and cutting. The creation process of paper architecture is often labor intensive and highly skill-demanding, even with the aid of existing computer-aided design tools. We propose an automatic algorithm for generating paper architectures given a user-specified 3D model. The algorithm is grounded on geometric formulation of planar layout for paper architectures that can be popped-up in a rigid and stable manner, and sufficient conditions for a 3D surface to be popped up from such a planar layout. Based on these conditions, our algorithm computes a class of paper architectures containing two sets of parallel patches that approximate the input geometry while guaranteed to be physically realizable. The method is demonstrated on a number of architectural examples, and physically engineered results are presented.

RepFinder: Finding Approximately Repeated Scene Elements for Image Editing
ACM Transactions on Graphics 2010, Vol. 29, No. 4, ACM SIGGRAPH 2010

Ming-Ming Cheng, Fang-Lue Zhang, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu

Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation between instances, illumination variation, etc. Manually enforcing such relations is laborious and error prone. We propose a novel framework where simple user input in the form of scribbles are used to guide detection and extraction of such repeated elements. Our detection process is based on a novel boundary band method, and robustly extracts the repetitions along with their mutual depth relations. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.

Metric-Driven RoSy Fields Design
IEEE Transactions on Visualization and Computer Graphics, 2010, Vol. 16, No. 1, 95-108.

Yu-Kun Lai, Miao Jin, Xuexiang Xie, Ying He, Jonathan Palacios, Eugene Zhang, Shi-Min Hu and Xianfeng David Gu

This work introduces a rigorous and practical approach for automatic N-RoSy field design on arbitrary surfaces with user defined field topologies.The user has full control of the number, positions and indices of the singularities, the turning numbers of the loops, and is able to edit the field interactively. We formulate N-RoSy field construction asdesigning a Riemannian metric, such that the holonomy along any loop is compatible with the local symmetry of N-RoSy fields. We prove the compatibility condition using discrete parallel transport. The complexity of N-RoSy field design is caused by curvatures. Inour work, we propose to simplify the Riemannian metric to make it flat almost everywhere. This approach greatly simplifies the process and improves the flexibility, such that, it can design N-RoSy fields with single singularity, and mixed-RoSy fields. To demonstrate the effectiveness of our approach, we apply our designsystem to pen-and-ink sketching and geometry remeshing.

Other publications in 2010

1. Yong-Jin Liu, Dong-Liang Zhang, Matthew Ming-Fai Yuen, A survey on CAD methods in garment design, Computers in Industry, 2010, Vol. 61, No. 6, 576-593

2. Yong-Jin Liu, Kam-Lung Lai, Gang Dai, Ming-Fai Yuen, A semantic feature model in concurrent engineering, IEEE Transactions on Automation Science and Engineering, 2010, Vol. 7, No. 3, 659-665

3. Yu-Ping Wang, Shi-Min Hu, Optimization approach for 3D model watermarking by linear binary programming, Computer Aided Design, 2010, Vol. 27, No. 5, 395-404

4. Yong-Jin Liu, Wen-Qi Zhang, Kai Tang, Some notes on maximal arc intersection of spherical polygons: its NP-hardness and approximation algorithms, The Visual Computer, 2010, Vol. 26, No. 4, 287-292

5. Chao-Hui Shen, Guo-Xin Zhang, Yu-Kun Lai, Shi-Min Hu, Harmonic Field Based Volume Model Construction from Triangle Soup, Journal of Computer Science and Technology, 2010, Vol. 25, No. 3, 562-571

6. Jin Wei and Yu Lou, Feature Preserving Mesh Simplification Using Feature Sensitive Metric, Journal of Computer Science and Technology, 2010, Vol. 25, No. 3, 595-605

7. Yu-Kun Lai, Leif Kobbelt and Shi-Min Hu, Feature aligned quad dominant remeshing using iterative local updates, Computer Aided Design, 2010, Vol. 42, No. 2, 109-117

(An earlier version has been presented in ACM Symosium on Solid and Physical Modeling, June 2-4, 2008)

2009

Sketch2Photo: Internet Image Montage
ACM Transactions on Graphics, Vol. 28, No. 5, Article No. 124, ACM SIGGRAPH ASIA 2009

Tao Chen, Ming-Ming Cheng, Ping Tan, Ariel Shamir, Shi-Min Hu
The paper was selected as one of the top 10 most innovative and promising worldwide initiatives of 2009 by the Netexplorateur jury.

We present a system that composes a realistic picture from a user provided sketch with text labels. The composed picture is generated by seamlessly stitching several photographs automatically searched from internet according to the sketch and its text labels. While online image search generates noisy results, our system can automatically select suitable photographs to generate a high quality composition. To achieve this, we first design a filtering scheme to exclude undesirable images from searched results. Then we propose a novel image blending algorithm for seamless image composition. Our blending algorithm returns a numeric score for each blending, which is used to optimize the combination of searched images. Several vivid results are generated in the experiments. We also perform a user study to demonstrate the advantages of our system.

Efficient Affinity-based Edit Propagation using K-D Tree
ACM Transactions on Graphics, Vol. 28, No. 5, Article No. 118, ACM SIGGRAPH ASIA 2009

Kun Xu, Yong Li, Tao Ju, Shi-Min Hu, Tian-Qiang Liu

Image/video editing by strokes has become increasingly populardue to the ease of interaction. Propagating the user inputs to the restof the image/video, however, is often time and memory consumingespecially for large data. We propose here an efficient scheme thatallows affinity-based edit propagation to be computed on data containingtens of millions of pixels at interactive rate (in matter of seconds).The key in our scheme is a novel means for approximatelysolving the optimization problem involved in edit propagation, usingadaptive clustering in a high-dimensional, affinity space. Ourapproximation significantly reduces the cost of existing affinitybasedpropagation methods while maintaining visual fidelity, andenables interactive stroke-based editing even on high resolution imagesand long video sequences using commodity computers.

Simulating Gaseous Fluids with Low and High Speeds
Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 1845-1852

Yue Gao, Chen-Feng Li, Shi-Min Hu, Brian A. Barsky

Gaseous fluids may move slowly, as smoke does, or at high speed, such as occurs with explosions. High-speedgas flow is always accompanied by low-speed gas flow, which produces rich visual details in the fluid motion.Realistic visualization involves a complex dynamic flow field with both low and high speed fluid behavior. Incomputer graphics, algorithms to simulate gaseous fluids address either the low speed case or the high speedcase, but no algorithm handles both efficiently. With the aim of providing visually pleasing results, we present ahybrid algorithm that efficiently captures the essential physics of both low- and high-speed gaseous fluids. Wemodel the low speed gaseous fluids by a grid approach and use a particle approach for the high speed gaseousfluids. In addition, we propose a physically sound method to connect the particle model to the grid model. Byexploiting complementary strengths and avoiding weaknesses of the grid and particle approaches, we producesome animation examples and analyze their computational performance to demonstrate the effectiveness of thenew hybrid method.

Edit Propagation on Bidirectional Texture Functions
Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 1871-1877

Kun Xu, Jiaping Wang, Xin Tong, Shi-Min Hu, Baining Guo

We propose an efficient method for editing bidirectional texture functions (BTFs) based on edit propagationscheme. In our approach, users specify sparse edits on a certain slice of BTF. An edit propagation scheme isthen applied to propagate edits to the whole BTF data. The consistency of the BTF data is maintained by propagatingsimilar edits to points with similar underlying geometry/reflectance. For this purpose, we propose to useview independent features including normals and reflectance features reconstructed from each view to guide thepropagation process. We also propose an adaptive sampling scheme for speeding up the propagation process.Since our method needn't any accurate geometry and reflectance information, it allows users to edit complexBTFs with interactive feedback.

A Shape-Preserving Approach to Image Resizing
Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 1897-1906

Guo-Xin Zhang, Ming-Ming Cheng, Shi-Min Hu, Ralph R. Martin

We present a novel image resizing method which attempts to ensure that important local regions undergo a geometricsimilarity transformation, and at the same time, to preserve image edge structure. To accomplish this, wedefine handles to describe both local regions and image edges, and assign a weight for each handle based on animportance map for the source image. Inspired by conformal energy, which is widely used in geometry processing,we construct a novel quadratic distortion energy to measure the shape distortion for each handle. The resizingresult is obtained by minimizing the weighted sum of the quadratic distortion energies of all handles. Compared toprevious methods, our method allows distortion to be diffused better in all directions, and important image edgesare well-preserved. The method is efficient, and offers a closed form solution.

Generalized Discrete Ricci Flow
Computer Graphics Forum, Special issue of Pacific Graphics 2009, Vol. 28, No. 7, 2005-2014

Yong-Liang Yang, Ren Guo, Feng Luo, Shi-Min Hu, Xianfeng Gu

Surface Ricci flow is a powerful tool to design Riemannian metrics by user defined curvatures. Discrete surfaceRicci flow has been broadly applied for surface parameterization, shape analysis, and computational topology.Conventional discrete Ricci flow has limitations. For meshes with low quality triangulations, if high conformalityis required, the flow may get stuck at the local optimum of the Ricci energy. If convergence to the global optimumis enforced, the conformality may be sacrificed.This work introduces a novel method to generalize the traditional discrete Ricci flow. The generalized Ricci flowis more flexible, more robust and conformal for meshes with low quality triangulations. Conventional method isbased on circle packing, which requires two circles on an edge intersect each other at an acute angle. Generalizedmethod allows the two circles either intersect or separate from each other. This greatly improves the flexibility androbustness of the method. Furthermore, the generalized Ricci flow preserves the convexity of the Ricci energy, thisensures the uniqueness of the global optimum. Therefore the algorithm won't get stuck at the local optimum.Generalized discrete Ricci flow algorithms are explained in details for triangle meshes with both Euclidean andhyperbolic background geometries. Its advantages are demonstrated by theoretic proofs and practical applicationsin graphics, especially surface parameterization.

Automatic and Topology-Preserving Gradient Mesh Generation for Image Vectorization
ACM Transactions on Graphics, Vol. 28, No. 3, article 85, ACM SIGGRAPH 2009

Yu-Kun Lai, Shi-Min Hu, Ralph R. Martin

Gradient mesh vector graphics representation, used in commercialsoftware, is a regular grid with specified position and color,and their gradients, at each grid point. Gradient meshes can compactlyrepresent smoothly changing data, and are typically used forsingle objects. This paper advances the state of the art for gradientmeshes in several significant ways. Firstly, we introduce atopology-preserving gradient mesh representation which allows anarbitrary number of holes. This is important, as objects in imagesoften have holes, either due to occlusion, or their 3D structure. Secondly,our algorithm uses the concept of image manifolds, adaptingsurface parameterization and fitting techniques to generate the gradientmesh in a fully automatic manner. Existing gradient-meshalgorithms require manual interaction to guide grid construction,and to cut objects with holes into disk-like regions. Our new algorithmis empirically at least 10 times faster than previous approaches.Furthermore, image segmentation can be used with ournew algorithm to provide automatic gradient mesh generation fora whole image. Finally, fitting errors can be simply controlled tobalance quality with storage.

Vectorizing Cartoon Animations
IEEE Transactions on Visualization and Computer Graphics, 2009, Vol. 15, No. 4, May/June, 618-629

Song-hai Zhang, Tao Chen, Yi-Fei Zhang, Shi-Min Hu, Ralph R. Martin

We present a system for vectorizing 2D raster format carton animations. The output animations are visually flicker free, smaller in file size, and easy to edit. We identify decorative lines separately from coloured regions. We use an accurate and semantically meaningful image decomposition algorithm which supports an arbitrary color model for each region. To ensure temporal coherence in the output cartoon, we reconstruct a universal background for all frames, and separately extract foreground regions. Simple user-assistance is required to complete the background. Each region and decorative line is vectorized and stored together with their motions from frame to frame.

A new watermarking method for 3D model based on integral invariant
IEEE Transactions on Visualization and Computer Graphics, 2009, Vol. 15, No. 2, March/April, 285-294

Yu-Ping Wang and Shi-Min Hu

In this report, we propose a new semi-fragile watermarking algorithm for the authentication of 3D models based on integral invariants. To do so, we embed a watermark image by modifying the integral invariants of some of the vertices. Basically, we shift a vertex and its neighbors in order to change the integral invariants. To extract the watermark, test all the vertices for the embedded information, and combine them to recover the watermark image. How many parts can
the watermark image be recovered would help us to make the authentication decision. Experimental test shows that this method is robust against rigid transform and noise attack, and useful to test purposely attack besides transferring noise and geometrical transforming noise. An additional contribution of this paper is a new algorithm for computing two kinds of integral invariants.

Other publications in 2009

1. Yong-Jin Liu, Yu-Kun Lai and Shi-Min Hu, Stripification of Free-Form Surfaces with Global Error Bounds for Developable Approximation, IEEE Transactions on Automation Science and Engineering, 2009, Vol. 6, No. 4, 700-709

2. Yu-Kun Lai, Shi-Min Hu, Ralph R. Martin and Paul L. Rosin,apid and Effective Segmentation of 3D Models using Random Walks, Computer Aided Geometric Design, 2009, Vol. 26, No. 6, 665-679.

(An earlier version has been presented in ACM Symosium on Solid and Physical Modeling, June 2-4, 2008)
3. Song-hai Zhang, Tao Chen, Yi-Fei Zhang, Shi-Min Hu, Ralph R. Martin, Video-Based Running Water Animation in Chinese Painting Style, Science in China Series F: Information Sciences, 2009, Vol. 52, No. 2, 162-171

4. Zhe Bian, Shi-Min Hu and Ralph R Martin, Evaluation for Small Visual Difference Between Conforming Meshes on Strain Field, Journal of Computer Science and Technology, 2009, Vol. 24, No. 1, 65-75

The preliminary version of this work has been presented on GMP2008.

2008

Optimal Surface Parameterization Using Inverse Curvature Map
IEEE Transactions on Visualization and Computer Graphics, 2008, Vol. 14, No. 5, Septmber/Octber, 1054-1066.

Yong-Liang Yang, Junho Kim, Feng Luo, Shi-Min Hu, and Xianfeng Gu

Mesh parameterization is a fundamental technique in computer graphics. The major goals during mesh parameterization are to minimize both the angle distortion and the area distortion. Angle distortion can be eliminated by use of conformal mapping, in principle. Our paper focuses on solving the problem of nding the best discrete conformal mapping that also minimizes area distortion. Major theoretical results and practical algorithms are presented for optimal parameterization based on the inverse curvature map. Comparisons are conducted with existing methods and using different energies. Novel parameterization applications are also introduced. The theoretical framework of the inverse curvature map can be applied to further study discrete conformal mappings.

Shrinkability Maps for Content-Aware Video Resizing
Computer Graphics Forum, Special issue of Pacific Graphics 2008 , Vol. 27, No. 7, 1797-1804 .

Yi-Fei Zhang,Shi-Min Hu, Ralph R. Martin

A novel method is given for content-aware video resizing, i.e. targeting video to a new resolution (which may involve aspect ratio change) from the original. We precompute a per-pixel cumulative shrinkability map which takes into account both the importance of eachpixel and the need for continuity in the resized result. (If both x and y resizing are required, two separate shrinkability maps are used, otherwise one suffices). A random walk model is used for efficient offline computation of the shrinkability maps. The latter are stored with the video to create a multi-sized video, which permits arbitrarysized new versions of the video to be later very efficiently created in real-time, e.g. by a video-on-demand server supplying video streams to multiple devices with different resolutions. These shrinkability maps are highly compressible, so the resulting multi-sized videos are typically less than three times the size of the original compressed video. A scaling function operates on the multi-sized video, to give the new pixel locations in the result, giving a high-quality content-aware resized video.

Shape Deformation using a Skeleton to Drive Simplex Transformations
IEEE Transactions on Visualization and Computer Graphics, 2008, Vol. 14, No. 3, May/June, 693-706

Han-Bing Yan, Shi-Min Hu, Ralph R Martin, and Yong-Liang Yang
The preliminary version of this work has been presented on CGI 2006

This paper presents a skeleton-based method for deforming meshes (the skeleton need not be the medial axis). The significant difference from previous skeleton-based methods is that the latter use the skeleton to control movement of vertices whereas we use it to control the simplices defining the model. By doing so, errors that occur near joints in other methods can be spread over the whole mesh, via an optimization process, resulting in smooth transitions near joints of the skeleton. By controlling simplices, our method has the additional advantage that no vertex weights need be defined on the bones, which is a tedious requirement in previous skeleton-based methods. Furthermore, by incorporating the translation vector in our optimisation, unlike other methods, we do not need to fix an arbitrary vertex, and the deformed mesh moves with the deformed skeleton. Our method can also easily be used to control deformation by moving a few chosen line segments, rather than a skeleton.

Spherical Piecewise Constant Basis Functions for All-Frequency Precomputed Radiance Transfer
IEEE Transactions on Visualization and Computer Graphics, 2008, Vol. 14, No. 2, March/April, 454-467

Kun Xu, Yun-Tao Jia, Hongbo Fu, Shi-Min Hu and Chiew-Lan Tai

This paper presents a novel basis function, called spherical piecewise constant basis function (SPCBF), for precomputed radiance transfer. SPCBFs have several desirable properties: rotatability, ability to represent all-frequency signals, and support for efficient multiple product. By partitioning the illumination sphere into a set of subregions, and associating each subregion with an SPCBF valued 1 inside the region and 0 elsewhere, we precompute the light coefficients using the resulting SPCBFs. We run-time approximate BRDF and visibility coefficients with the same set of SPCBFs through fast lookup of summed-area-table (SAT) and visibility distance table (VDT), respectively. SPCBFs enable new effects such as object rotation in all-frequency rendering of dynamic scenes and onthe-fly BRDF editing under rotating environment lighting. With graphics hardware acceleration, our method achieves real-time frame rates.

Video: Download video here (13.0MB).

Other publications in 2008

1. Yu-Kun Lai, Yong-Jin Liu, Yu Zang and Shi-Min Hu, Fairing Wireframes in Industrial Design, IEEE International Conference on Shape Modeling and Applications, June 4-6, 2008, 29-35.

2. Yong-Jin Liu, Matthew Ming-Fai Yuen, Geometry-optimized virtual human head and its applications, Computer & Graphics, 2008, Vol. 32, No. 6, 624-631

2007

Editing The Topology of 3D Models by Sketching
ACM Transactions on Graphics, Vol. 26, No. 3, Article 42, ACM SIGGRAPH 2007

Tao Ju, Qian-Yi Zhou and Shi-Min Hu

We present a method for modifying the topology of a 3D model with user control. The heart of our method is a guided topology editing algorithm. Given a source model and a user-provided target shape, the algorithm modifies the source so that the resulting model is topologically consistent with the target. Our algorithm permits removing or adding various topological features (e.g., handles, cavities and islands) in a common framework and ensures that each topological change is made by minimal modification to the source model. To create the target shape, we have also designed a convenient 2D sketching interface for drawing 3D line skeletons. As demonstrated in a suite of examples, the use of sketching allows more accurate removal of topological artifacts than previous methods, and enables creative designs with specific topological goals.

Video: Download video here (31.8MB). (Cannot open the video? Cannot hear the audio? Get latest QuickTime player.)

Software: A software MendIT based on this paper will come soon, please refer to webpage: http://graphics.usc.edu/~qianyizh/software.html

Real-time homogeneous translucent material editing
EuroGraphics 2007, Computer Graphics Forum, Vol. 26, No. 3, 545–552.

Kun Xu, Yue Gao, Yong Li, Tao Ju and Shi-Min Hu

This paper presents a novel method for real-time homogeneous translucent material editing under fixed illumination. We consider the complete analytic BSSRDF model proposed by Jensen et al.[JMLH01], including both multiple scattering and single scattering. Our method allows the user to adjust the analytic parameters of BSSRDF and provides high-quality, real-time rendering feedback. Inspired by recently developed Precomputed Radiance Transfer (PRT) techniques, we approximate both the multiple scattering diffuse reflectance function and the single scattering exponential attenuation function in the analytic model using basis functions, so that re-computing the outgoing radiance at each vertex as parameters change reduces to simple dot products. In addition, using a non-uniform piecewise polynomial basis, we are able to achieve smaller approximation error than using bases adopted in previous PRT-based works, such as spherical harmonics and wavelets. Using hardware acceleration, we demonstrate that our system generates images comparable to [JMLH01] at real-time frame-rates.

Video: Download video here (17.3MB).

Topology Repair of Solid Models Using Skeletons
IEEE Transactions on Visualization and Computer Graphics, 2007, Vol. 13, No. 4, 675-685.

Qian-Yi Zhou, Tao Ju and Shi-Min Hu

We present a method for repairing topological errors on solid models in the form of small surface handles, which often arise from surface reconstruction algorithms. We utilize a skeleton representation that offers a new mechanism for identifying and measuring handles. Our method presents two unique advantages over previous approaches. First, handle removal is guaranteed not to introduce invalid geometry or additional handles. Second, by using an adaptive grid structure, our method is capable of processing huge models efficiently at high resolutions.

Slides: Download slides here (24.8MB). A poster for this paper is also available, download poster here (7.5MB).

Software: A software TopoMender based on this paper is now available, please refer to webpage: http://graphics.usc.edu/~qianyizh/software.html

Robust Feature Classification and Editing
IEEE Transactions on Visualization and Computer Graphics, 2007, Vol. 13, No.1, January/Feburary, 34-45.

Yu-Kun Lai, Qian-Yi Zhou, Shi-Min Hu, Johannes Wallner and Helmut Pottmann

Sharp edges, ridges, valleys and prongs are critical for the appearance and an accurate representation of a 3D model. In this paper, we propose a novel approach that deals with the global shape of features in a robust way. Based on a remeshing algorithm which delivers an isotropic mesh in a feature sensitive metric, features are recognized on multiple scales via integral invariants of local neighborhoods. Morphological and smoothing operations are then used for feature region extraction and classification into basic types such as ridges, valleys and prongs. The resulting representation of feature regions is further used for feature-specific editing operations.

Other publications in 2007

1. Jean-Baptiste Debard(Yang Fei), Romain Balp (Bai Luomin) , Raphaelle Chaine, Dynamic Delaunay tetrahedralisation of a deforming surface, The Visual Computers, 2007, Vol. 23, No. 12, 975 - 986

2. Yong-Jin Liu, Qian-Yi Zhou and Shi-Min Hu, Handling Degenerate Cases in Exact Geodesic Computation on Triangle Meshes, The Visual Computers, 2007, Vol. 23, No. 9-11, 661-668.

3. Yong-Jin Liu, Kai Tang, Ajay Joneja, Modeling dynamic developable meshes by the Hamilton principle, Computer-Aided Design, 2007, Vol. 39, No. 9, 719-731.

4. Han-Bing Yan, Shi-Min Hu, Ralph R Martin, 3D morphing using strain field interpolation, Journal of Computer Science and Technology, 2007, Vol. 22, No. 1, 147-155.

2006

Geometry and Convergence Analysis of Algorithms for Registration of 3D Shapes
Geometry and convergence analysis of algorithms for registration of 3D shapes, International Journal of Computer Vision, 2006, Vol. 67, No. 3, 277-296.

Helmut Pottmann, Qi-Xing Huang, Yong-Liang Yang and Shi-Min Hu

The computation of a rigid body transformation which optimally aligns a set ofmeasurement points with a surface and related registration problems are studiedfrom the viewpoint of geometry and optimization.We provide a convergence analysisfor widely used registration algorithms such as ICP, using either closest points (Besland McKay [2]) or tangent planes at closest points (Chen and Medioni [4]), and fora recently developed approach based on quadratic approximants of the squareddistance function [24]. ICP based on closest points exhibits local linear convergenceonly. Its counterpart which minimizes squared distances to the tangent planes atclosest points is a Gauss-Newton iteration; it achieves local quadratic convergencefor a zero residual problem and { if enhanced by regularization and step size control{ comes close to quadratic convergence in many realistic scenarios. Quadraticallyconvergent algorithms are based on the approach in [24]. The theoretical results aresupported by a number of experiments; there, we also compare the algorithms withrespect to global convergence behavior, stability and running time.

Robust Principal Curvatures on Multiple Scales
Proceedings of 4th Eurographics Symposium on Geometry Processing (2006). Eurographics Association, 223-226.

Yong-Liang Yang, Yu-Kun Lai, Shi-Min Hu and Helmut Pottmann

Geometry processing algorithms often require the robust extractionof curvature information. Wepropose to achieve this with principal component analysis (PCA) oflocal neighborhoods, defined via spherical kernels centered on thegiven surface $\Phi$. Intersection of a kernel ball $B_r$ or itsboundary sphere $S_r$ with the volume bounded by $\Phi$ leads tothe so-called ball and sphere neighborhoods. Information obtainedby PCA of these neighborhoods turns out to be more robust thanPCA of the patch neighborhood $B_r \cap \Phi$ previously used.The relation of the quantities computed by PCA with the principalcurvatures of $\Phi$ is revealed by an asymptotic analysis as thekernel radius $r$ tends to zero. This also allows us to define principalcurvatures ''at scale $r$'' in a way which is consistent with the classicalsetting. The advantages of the new approach are discussed in a comparisonwith results obtained by normal cycles and local fitting; whereas the formermethod somewhat lacks in robustness, the latter does not achieve a consistentbehavior at features on coarse scales. As to applications,we address computing principal curves and feature extractionon multiple scales.

Other publications in 2006

1. Qi-Xing Huang, Simon Flory, Natasha Gelfand, Michael Hofer and Helmut Pottmann, Reassembling Fractured Objects by Geometric Matching, ACM Transactions on Graphics, Vol. 25, No. 3, 569-578, ACM SIGGRAPH 2006

2. Yang Liu, Helmut Pottmann, Johannes Wallner, Yong-Liang Yang and Wenping Wang, Geometric Modeling with Conical Meshes and Developable Surfaces, ACM Transactions on Graphics, Vol. 25 , No. 3, 681-689, ACM SIGGRAPH 2006

3. Yu-Kun Lai, Shi-Min Hu and Helmut Pottmann, Surface Fitting Based on a Feature Sensitive Parameterization, Computer-Aided Design, 2006, Vol. 38, No. 7, 800--807.

4. Li Jin, Donguk Kim, Lisen Mu, Deok-Soo Kim and Shi-Min Hu, A Sweepline Algorithm for Euclidean Voronoi Diagram of Circles, Computer-Aided Design, 2006, Vol. 38, No. 3, 260-278.

5. Yu-Kun Lai, Shi-Min Hu and Ralph R. Martin, Surface Mosaics, The Visual Computer, 2006, Vol. 22, No. 9-10, 604-611 (Pacific Graphics 2006).

6. Jiaping Wang, Kun Xu, Kun Zhou, Stephen Lin, Shi-Min Hu and Baining Guo, Spherical Harmonics Scaling, The Visual Computer, 2006, Vol. 22, No. 9-10, 713-720 (Pacific Graphics 2006).

7. Yu-Kun Lai, Qian-Yi Zhou, Shi-Min Hu and Ralph R. Martin, Feature Sensitive Mesh Segmentation, ACM Symp. Solid and Physical Modeling, 7-16, 2006.

8. Xiao-Hua Cai, Yun-Tao Jia, Xi Wang, Shi-Min Hu and Ralph R. Martin, Rendering Soft Shadows using Multilayered Shadow Fins, Computer Graphics Forum, 2006, Vol.25, No.1, 1-14.

2005

Video Completion using Tracking and Fragment Merging
The Visual Computer, 2005, Vol. 21, No. 8-10, 601-601. (Pacific Graphics 2005)

Yun-Tao Jia, Shi-Min Hu and Ralph R. Martin

Video completion is the problem of automatically filling space-time holes in video sequences left by the removal of unwantedobjects in a scene. We solve it using texture synthesis, filling a hole inwards using three steps iteratively:we select the most promising target pixel at the edge of the hole, we findthe source fragment most similar to the known part of the target neighborhood, and we merge sourceand target fragments to complete the target neighborhood, reducing thesize of the hole. Earlier methods were slow, due to searching the whole video data forsource fragments or completing holes pixel by pixel; they also producedblurred results due to sampling and smoothing. For speed, we trackmoving objects, allowing us to use a much smaller search space whenseeking source fragments; we also complete holes fragment by fragmentinstead of pixelwise. Fine details are maintained by use of a graph cutalgorithm when merging source and target fragments. Further techniquesensure temporal consistency of hole filling over successive frames.Examples demonstrate the effectiveness of our method.

Other publications in 2005

1. Xu-Ping Zhu, Shi-Min Hu, Chiew-Lan Tai, and Ralph R Martin, A Marching Method for Computing Intersection Curves of Two Subdivision Solids, in Mathematics of Srufaces XI, Eds. R. R. Martin, H. Bez, M. A. Sabin, 458-471, 2005.

2. Johannes Wallner, Hans-Peter Schrocker, Shi-min Hu, Tolerance in geometric constraints solving, Reliable Computing, 2005, Vol. 11, No. 3, 235-251.

3. Yu-Kun Lai, Shi-Min Hu, Xianfeng Gu, Ralph R. Martin, Geometric texture synthesis and transfer via geometry images, ACM Solid and Physical Modeling, MIT, USA, June 13-15, 2005, 15-26.

4. Shi-min Hu, Johannes Wallner, A second order algorithm for orthogonal projection onto curves and surfaces, Computer Aided Geometric Design, 2004, Vol.22, No. 3, 251-260.

5. Qi-Xing Huang, Shi-Min Hu and Ralph R. Martin, Fast degree elevation and knot insertion for B-spline curves, Computer Aided Geometric Design, 2005, Vol 22, No. 2, 183-197.

2004

Generalized Displacement Maps
Proceedings of Eurographics Symposium on Rendering, 2004

Xi Wang, Xin Tong, Stephen Lin, Shimin Hu, Baining Guo and Heung-Yeung Shum

In this paper, we introduce a real-time algorithm to render the rich visual effects of general non-height-field geometricdetails, known as mesostructure. Our method is based on a five-dimensional generalized displacement map(GDM) that represents the distance of solid mesostructure along any ray cast from any point within a volumetricsample. With this GDM information, we propose a technique that computes mesostructure visibility jointly in objectspace and texture space which enables both control of texture distortion and efficient computation of texturecoordinates and shadowing. GDM can be rendered with either local or global illumination as a per-pixel processin graphics hardware to achieve real-time rendering of general mesostructure.

Other publications in 2004

1. Han-Bing Yan, Shi-Min Hu, Ralph R. Martin, Morphing Based on Strain Field Interpolation, Journal of Computer Animation and Virtual Worlds (CAVW), 2004, Vol.15, No.3-4, 443-452.

2. Shi-min Hu, Johannes Wallner, Error Propagation through Geometric Transformations, Journal for Geometry and Graphics, 2004, Vol.8, No.2, 171-183.

3. Shi-Min Hu, Chen-feng Li, Hui Zhang, Actual Morphing: A phsical-based approach for blending two 2D/3D shapes, ACM Symposium on Solid Modeling and Applications, Genova, Italy, June 9-11, 2004.

2003

View-Dependent Displacement Mapping
ACM Transactions on Graphics, Vol. 22, No. 3. 334-339, ACM SIGGRAPH 2003

Lifeng Wang, Xi Wang, Xin Tong, Steve Lin, Shimin Hu, Baining Guo and Heung-Yeung Shum

Significant visual effects arise from surface mesostructure, such asfine-scale shadowing, occlusion and silhouettes. To efficiently renderits detailed appearance, we introduce a technique called viewdependentdisplacement mapping (VDM) that models surface displacementsalong the viewing direction. Unlike traditional displacementmapping, VDM allows for efficient rendering of selfshadows,occlusions and silhouettes without increasing the complexityof the underlying surface mesh. VDM is based on per-pixelprocessing, and with hardware acceleration it can render mesostructurewith rich visual appearance in real time.

Other publications in 2003

1. Xu-Ping Zhu, Shi-Min Hu and Martin Ralph, Skeleton-Based Seam Computation for Triangulated Surface Parameterization, In Proceedings of Mathematics in Surfaces X, Sept 2003, Leeds, UK; Lecture Notes in Computer Science, 2003. [PS]
2. Chiew-Lan Tai, Hu Shi-Min and Qixing Huang, Approximate merging of B-Spline curves via knot adjustment and constrained optimization, Computer Aided Design, 2003, Vol. 35, No. 10, 893 - 899.

3. Xi Wang, Lifeng Wang, Ligang Liu, Shi-Min Hu and Baining Guo, Interactive Modeling of Tree Bark, In: Proceedings of Pacific Graphics 2003, IEEE CS Press, Oct 8-10, 2003

4. Tao Wang, Yong Rui, Shi-Min Hu and Jia-guang Sun, Adaptive tree similarity learning for image retrieval, Multimedia Systems, 2003, Vol. 9, 131-143.

2002

1. Shi-Min Hu, Chiew-Lan Tai, Song-Hai Zhang, An Extension algorithm for B-spline curves by curve unclamping, Computer Aided Design, 2002, Vol. 34, No. 5, 415-4191.

2. Yan-Tao Li, Shi-Min Hu and Jia-Guang Sun, A Constructive Approach to Solving 3-D Geometric Constraint Systems Using Dependence Analysis, Computer Aided Design, 2002, Vol. 34, No. 2, 97-108.

3. Liu Shi-Xia, Hu Shi-Min, Sun Jiaguang, Two accelerating techniques for 3D reconstruction, Journal of Computer Science and Technology, 17(3), 362-368, 2002.

2001

1. Shi-Xia Liu, Shi-Min Hu, Yu-Jian Chen, Jia-Guang Sun, Reconstruction of curved solids from engineering drawings, Computer Aided Design, 2001, Vol. 33, No. 14, 1059-1072.

2. Shi-Min Hu, Youfu Li, Tao JU, Xiang Zhu, Modifying the shape of NURBS surfaces with geometric constraints, Computer Aided Design, 2001, Vol. 33, No. 12, 903-912.

3. Shi-Min, Hu Conversion between triangular and rectangular Bezier patches, Computer Aided Geometric Design, 2001, Vol.18, No. 7, 667-671. (In Special issue of memory of P. Bezier).

4. Shi-Min Hu, Hui Zhang, Chiew-Lan Tai, Jia-Guang Sun, Direct Manipulation of FFD: Efficient Explicit Solutions and Decomposible Multiple Point Constraints, The Visual Computers, 2001, Vol. 17, No. 6, 370-379.

5. Jun-Hai Yong, Shi-Min Hu, Jia-Guang Sun, Degree reduction of B-spline curves, Computer Aided Geometric Design, 2001, Vol. 13, NO. 2, 2001, 117-127.

6. Shi-Min Hu, Ruofeng Tong, Tao JU, Jia-Guang Sun, Approximate merging of a pair of Bezier curves, Computer Aided Design, Vol 33, No. 2, 125-136, 2001.

7. Jun-Hai Yong, Shi-Min Hu, JIa-Guang Sun, CIM Algorithm for Approximating Three Dimensional Polygonal Curves, Journal of Computer Science and Technology, 16(6), 489-497,2001.

8. Tao Wang, Yong Rui and Shi-Min Hu, Optimal Adaptive Learning for Image Retrieval, Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR 2001), I-1140 to 1147, Kauai, Hawaii, December 11-13, 2001.

9. Yan-Tao Li, Shi-Min Hu, Jia-Guang Sun, On the numerical redundancies of geometric constraint systems, Proceedings of Pacific Graphics 2001, 118-123, IEEE Computre Society Press, 2001, Tokyo.

10. Jian-Hua Wu, Shi-Min Hu, Chiew-Lan Tai and Jia-Guang Sun, An effective feature-preserving mesh simplification scheme based on face constriction, Proceedings of Pacific Graphics 2001, 12-21, IEEE Computre Society Press, 2001, Tokyo.

Structure analysis and interactive synthesis of Internet visual media, Major international (regional) joint research program, National Natural Science Foundation, PI: Shi-Min Hu, Project number: 61120106007, 2012-2016.

Discrete representation and analysis methods for digital media, National Natural Science Foundation, PI: Yong-Jin Liu, Project number: 61272228, 2013-2016.

Computational geometry and graphics, Excellent Youth project, National Natural Science Foundation, PI: Yong-Jin Liu, Project number: 61322206, 2014-2016.

Image & Video editing based on object analysis, National Natural Science Foundation, PI: Song-Hai Zhang, Project number: 61373069, 2014-2017.

Research URL

你可能感兴趣的:(URL)