home projects vita group demo
Featured Projects
Real-time Video-based Facial Performance Capture and ManipulationAutomatic algorithms to real-time capture 3D high-quality facial mesh sequences as well as video-based facial expression manipulation and face swapping
Analysis and Modeling Human Behaviors in Multiparty ConversationsQuantifying, understanding, and modeling human behaviors and gestures when they are involved with multiparty conversations
Papers:
CHI'17,
SCA'19,
PRESENCE'22,
MIG'22
[Video (SCA'19)], [Video (PRESENCE'22)], [Video (MIG'22)] 
Skeletal Rigging from Mesh/Point SequencesUsing the input of an animated mesh sequence, this approach can automatically extract its optimal skeleton structure, joint positions, linear blend skinning weights, and the corresponding bone transformations.
Visual Traffic Simulation, Modeling, and EvaluationDesign of novel and efficient algorithms and systems to simulate large-scale realistic traffic in various road and urban settings, including mixed traffic, lane-changing, simulated traffic evaluation, and intersectional traffic simulation.
Papers:
CAVW'14a,
CAVW'14b,
CAVW'15a,
CAVW'15b,
SCA'16,
TVCG'17,
TVCG'18a,
TVCG'18b,
CGF'19,
ICCV'19,
ECCV'20,
CAVW'22
[Video TVCG'17)], [Video (SCA'16)], [Video (TVCG'18b)] 
3D Face Modeling from In-the-wild Images or VideoAutomatic methods to reconstruct 3D face models with fine-scale details from a single in-the-wild image or video, including automatically augment coarse-scale 3D faces with synthesized fine-scale geometric wrinkles.
Interactive Complex Object Modeling from Multi-view Imagesdevelopment of a novel interactive modeling approach based on multi-view images including interactive 3D modeling and stochastic motion parameter estimation.
Hexahedral Mesh Generation, Optimization, and EvaluationA first and complete pipeline to optimize the global structure of a hex-mesh, by identifying and removing those valid removal base-complex sheets that contain misaligned singularities, and a volumetric partitioning strategy based on a generalized sweeping framework to seamlessly partition the volume of an input triangle mesh into a collection of deformed cuboids.
Papers:
SIGGRAPH'15,
TVCG'16,
SGP'17,
SIGGRAPH Asia'17
[Video (SIG'15)], [Video (TVCG'16)], [ Source Code (SGP'17)], [ Dataset (SGP'17)], [ Source Code (SIGGRAPH Asia'17)] 
Crowd Simulation and EvacuationNew algorithms and systems to simulate crowds or group behavior in unexpected hazard situations, computational model of crowd emotion evolution, or simulated crowds by integrating physical strength consumption
Mesh Animation CompressionEfficient algorithms to compress mesh animation sequences through adaptive spatio-temporal segmentations in order to support progressive transmission over networks
Two-layer Sparse Compression of Dense-Weight Blend SkinningIt introduces an efficient two-layer sparse compression technique to substantially reduce the computational cost of a dense-weight skinning model, with insignificant loss of its visual quality.
Smooth Skinning Decomposition with Rigid BonesThe SSDR model can effectively approximate the skin deformation of nearly articulated models as well as highly deformable models by a low number of rigid bones and a sparse, convex bone-vertex weight map.
Eye and Head AnimationsData-driven synthesis algorithms for realistic and coordinated eye and head movements based on (live or pre-recorded) speech input
Papers:
CAVW'05
CG&A'05,
TASLP'07,
VR'09,
CHI'11,
TVCG'12 ,
SCA'19
[Video (VR'09)], [ Video (TVCG'12)], [Video (SCA'19)] 
Visual Speech Animation SynthesisAutomated algorithms and systems to generate expressive visual speech animations based on novel typed or spoken input
Live Speech Driven Facial AnimationFully-automated algorithms to generate realistic lip-sync and head-and-eye animations based on live speech input alone, which can run in real-time on an off-the-shelf computer.
Human Motion Compression and RetrievalEffective algorithms for compressing and retrieving character motions by hierarchically extracting meaningful motion patterns from dataset, or by performing sparse dictionary learning from dataset.
Insect Swarm SimulationA highly efficient field-based, GPU friendly approach to realistically simulate behavior patterns of a variety of insect swarms (including flying insects, ants, etc.) under various scenarios.
Creative Virtual Tree Modeling and MorphingA new method to efficiently generate a set of morphologically diverse and inspiring virtual trees through hierarchical topology-preserving blending, aiming to facilitate designers' creativity production, and a new way to morph between two distinct trees.
Feature-preserving Mesh Denoisinga novel scheme for robust feature-preserving denoising for 3D mesh models in order to remove noise while maximally preserving geometric features
Sketch-based 3D ModelingData-driven 3D modeling algorithms and systems to rapidly create 3D objects or 3D realistic textured faces based on freeform 2D occluding contours or freeform 2D sketch input
Perceptually Guided Facial AnimationA perceptually guided computational framework is introduced to generate expressive facial animation, and quantitatively and automatically evaluate visual speech animation.
Visual-Haptic Interfacing for Robot-Assisted Minimally Invasive Cardiac SurgeryA novel human-in-the-loop, visual haptic interfacing system for planning and performing real-time MRI guided, robot-assisted, minimally invasive surgical interventions on the beating heart, with particular focus on aortic valve implantations. |
Other Projects
Data-driven Guzheng Playing AnimationA fully automatic, deep learning based framework to synthesize realistic upper body animations based on novel Guzheng music input. We design a generative adversarial network based approach to capture the temporal relationship between Guzheng music and human motion.
Marker Optimization for Facial Motion Acquisition and DeformationAn approach to compute optimized marker layouts for facial motion acquisition as optimization of characteristic control points from a set of high-resolution, ground-truth facial mesh sequences
Blendshape Facial AnimationAlgorithms for automating or reducing time-consuming tuning efforts of the blendshape animation approach, including target interference reducing, cross-model mapping, and model construction.
Interactive Cage Generation for DeformationAn efficient complete pipeline to generate high quality cages for 3D models with arbitrary topological complexities, including high genus or complex models.
Indoor Layout Modeling and GenerationAutomated methods to synthesize plausible indoor scene layouts, including motion planning for convertible indoor scene layout design and layout programming via virtual navigation detectors
Total Human Modeling based on Single RGB-D ShotAn efficient algorithm and pipeline to fully recover the 3D human body pose and shape including head and hand positions from a single RGB-D shot, based on machine learning algorithms.
Facial Animation Editing and TransferringEfficient and automated algorithms and systems for editing and transferring facial motions across different models or modalities, with intuitive and little user interaction.
Freestyle Group Formation Generation for Agent-based Crowd SimulationAn intuitive and efficient approach to generate arbitrary and precise group formations by sketching 2D formation boundaries in a freeform manner.
Context-Aware Motion Diversification and Perception for CrowdsA novel scheme for dynamically controlling motion styles of agents to increase the motion variety of a crowd.
Online Mocap Marker Labeling for Interacting TargetsAn online, automated mocap marker labeling approach for multiple interacting articulated targets by fitting rigid bodies and exploiting trained structure and motion models
3D Geometric Model WatermarkingA novel, robust, and high-capacity watermarking method for 3D Meshes with arbitrary connectivities in spatial domain based on affine invariants, by embedded the watermark as affine-invariant length ratios of one diagonal segment to the residing diagonal intersected by the other one in a coplanar convex quadrilateral.
Papers:
TOMM'12
Energy-efficient GPU and Mobile Graphics ComputingDesign of novel algorithms to statistically model and predict the power consumption of GPU-based computing, and perform in-depth quantitative analysis on the power consumption and runtime performance aspects of graphics computing and applications on mainstream smartphone devices.
Image-based Face Illumination TransferAn image-based technique that transfers illumination from a source face image to a target face image, without any prior information regarding the lighting conditions or the 3D geometries of the underlying faces.
Papers:
TVC'10
Virtual Garment Simulation and FittingNovel algorithms and system to simulate virtual garments and fitting for virtual humans
User-centric Interfaces for Mobile ComputingDesign various user-centric user interfaces for mobile computing, including an automated avatar interface is driven by the text messages exchanged between chatting participants, as well as single-handed mobile keyboard design.
Improving Gaming Experience by Incorporating Physiological Feedback and Gamers' ProfilesNovel methodology to improve users' gaming experience by real-time monitoring their physiological signals or their individualized gamer profile, and thus automatically adjusting the difficulty level of computer games.
Papers:
CHI'09,
SandBox'10
Perceptual Understanding of Synthetic Facial Expressions, Animations, and Head MotionInvestigate the region-significance and culture-dependence of facial expressions and movements, in terms of subject identification, emotion perception of 3D facial expression, and visual perception of avatar head movements.
Analysis of Mutimodal Emotion RecognitionComprehensive analysis and understanding of the strengths, limitations, and combinational aspects of mutimodal emotion recognition algorithms and systems, based on multi-channles of humans including visual facial expression channel and acoustic speech channel.
Spine Tracking and Segmentation in Neuron ImagesNovel automated algorithms to accurately track, detect, segment a large number of dentrite spines from time-lapse confocal microscopy neuron images, in both 2D image level and reconstructed 3D geometric level.
Self-represented Avatars for Teen Physical ActivitiesStudy the feasibility and effectiveness of using self-represented avatars for increasing teen physical activities.
Visualization and Planning of Neurosurgical InterventionsA highly efficient, GPU-accelerated visualization and planning technique to allow interactive, quantitative or semi-quantitative comparison among various points of entrance for neurosurgical procedures that require straight access.
Interactive Teeth SegmentationInteractive geometric algorithm to accurately segment the upper and lower teeth from 3D dental models that are directly reconstructed from 2D CT dental images, with intuitive and little user intervention. |
Our Generous Sponsors
|