Sketch and Sculpt Interfaces

utahteapotespresso1espresso3espresso2esp

...evolution of the Espressoman!

Sketch and sculpt interfaces have often been touted as “natural” approaches to interactive design. While these metaphors are indeed a promising medium of visual communication, there are a number of inherent limitations in the motor control of the human hand, drawing or gesturing skill,  perception and  the ambiguities of inference,  that make the leap from  2D input to 3D shape a challenging task. Over the past five years our ongoing research has produced a number of successful systems (www.ilovesketch.com, www.meshmixer.com, www.crossshade.com) that facilitate the leap from 2D input to 3D models despite these limitations.

Keynote: SmartGraphics 2014

Publications

     
spacetime 50 Juxtaform: interactive visual summarization for exploratory shape design
Karran Pandey, Fanny Chevalier, Karan Singh
ACM SIGGRAPH 2023
Paper Project
spacetime 49 Face Extrusion Quad Meshes
Karran Pandey, J. Andreas Bærentzen, Karan Singh
ACM SIGGRAPH 2022
Paper Project
spacetime 48 Piecewise-Smooth Surface Fitting onto Unstructured 3D Sketches
Emilie Yu, Rahul Arora, J. Andreas Bærentzen, Karan Singh, and Adrien Bousseau
ACM SIGGRAPH 2022
Paper Project
muscles 47 Mid-Air Drawing of Curves on 3D Surfaces in Virtual Reality
Rahul Arora and Karan Singh
ACM Transactions on Graphics (TOG), presented at SIGGRAPH 2021
Abstract    BibTeX DOI   Paper   Video   arXiv   Project page (code and more)
muscles 46 Interactive Modelling of Volumetric Musculoskeletal Anatomy
Rinat Abdrashitov, Seungbae Bang, David I.W. Levin, Karan Singh, Alec Jacobson
ACM SIGGRAPH 2021
Project Page
colorbuilder 45 CASSIE: Curve and Surface Sketching in Immersive Environments
Emilie Yu, Rahul Arora, Tibor Stanko, J. Andreas Bærentzen, Karan Singh, and Adrien Bousseau
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) 2021
AbstractBibTeXDOIPaperVideoProject Page (Code and more)
colorbuilder 44 Color by Numbers: Interactive Structuring and Vectorization of Sketch Imagery
Amal Dev Parakkat, Marie-Paule Cani, Karan Singh
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) 2021 
Paper
colorbuilder 43 Color Builder: a Direct Manipulation Interface for Versatile Color Theme Authoring
M. Shugrina, W. Zhang, F. Chevalier, S. Fidler and K. Singh
ACM CHI 2019
Live Demo | Paper | Video
vr hands 42 MagicalHands: Mid-Air Hand Gestures for Animating in VR
Rahul Arora, Rubaiat Habib Kazi, Danny Kaufman, Wilmot Li, and Karan Singh
Proceedings of the 32nd Annual ACM Symposium on User Interface and Software Technology (UIST) 2019
AbstractBibTeXPaperVideoProject Page (code and more)
chi_vr_draw 41 Model-Guided 3D Sketching
Pengfei Xu, Hongbo Fu, Karan Singh, Hui Huang and Chiew-Lan Tai.
IEEE Transactions on Visualization and Computer Graphics 2018.
[PDF] [Project Page]
chi_vr_draw 40 SymbiosisSketch: Combining 2D and 3D Sketching for Designing Detailed 3D Objects in Situ
Rahul Arora, Rubaiat Habib, Tovi Grossman, George Fitzmaurice and Karan Singh. 
Proceedings of the ACM SIGCHI Comference on Human Factors in Computing Systems 2018
AbstractBibTeXDOI[PDF][Video]Talk at CHI
chi_vr_draw 39 Experimental Evaluation of Sketching on Surfaces in VR
Rahul Arora, Rubaiat Habib, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice 
Proceedings of the ACM SIGCHI Comference on Human Factors in Computing Systems 2017
AbstractBibTeXDOI[PDF][Video]
sketchsoup 38 SketchSoup: Exploratory Ideation using Design Sketches
R. Arora, I. Darolia, V. P. Namboodiri, K. Singh and A. Bousseau
Computer Graphics Forum, CGF 2016
[PDF]
vr storyboard 37 Multi-Device Storyboards for Cinematic Narratives in VR
Rorik Henrikson, Bruno De Araujo, Fanny Chevalier, Karan Singh, Ravin Balakrishnan
User Interface Software and Technology, UIST 2016
[PDF] [Details]
storeoboard 36 Storeoboard: Sketching Stereoscopic Storyboards
Rorik Henrikson, Bruno De Araujo, Fanny Chevalier, Karan Singh, Ravin Balakrishnan
Proceedings of the SIGCHI conference on Human Factors in computing systems (CHI 2016).
[PDF][Video][Details]
inv_toon_shading 35 Using Isophotes and Shadows to Interactively Model Normal and Height Fields
Qiuying Xu, Songrun Liu, Yotam Gingold, Karan Singh
Computers & Graphics 2016
[PDF][Details]
cartoon 34

Modeling Character Canvases from Cartoon Drawings
Mikhail Bessmeltsev, Will Chang, Nicholas Vining, Alla Sheffer, Karan Singh
ACM Transactions on Graphics, Volume 34 Issue 5, October 2015.
[PDF][Video][Details]

secondskin 33 SecondSkin: Sketch-based Construction of Layered 3D Models
Chris de Paoli, Karan Singh
ACM Transactions on Graphics, SIGGRAPH 2015
[PDF][Details]
invtoonshading 32

Inverse Toon Shading: Interactive Normal Field Modeling with Isophotes
Qiuying Xu, Yotam Gingold, Karan Singh
Sketch-Based Interfaces and Modeling (SBIM), 2015.
[PDF][Video]


ff 31 FlatFitFab: Interactive Modeling with Planar Sections
James McCrae, Nobuyuki Umetani, Karan Singh
User Interface Software and Technology, 2014
[PDF] [Details][Dinsosaur][Dragon][Bee]
t2f 30

True2Form: 3D Curve Networks from 2D Sketches via Selective Regularization Baoxuan Xu William Chang Alla Sheffer Adrien Bousseau James McCrae Karan Singh. (SIGGRAPH 2014) [PDF][Details]

 

pam 29

Interactive Shape Modeling using a Skeleton-Mesh Co-Representation J. Andreas Bærentzen, Rinat Abdrashitov, Karan Singh. (SIGGRAPH 2014) [PDF] [Details]

fc 28 Flow complex based shape reconstruction from 3D curves.
Bardia Sadri, Karan Singh. ACM Transactions on Graphics 2014 (presented at SIGGRAPH 2014). [PDF]
mosaic 27 Mosaic: Sketch-Based Interface for Creating Digital Decorative Mosaics. R. Abdrashitov, E. Guy, J. Yao, K. Singh. Sketch-Based Interfaces and Modeling, 2014.[PDF] Best Paper Award
multidim 26 Sculpting multi-dimensional nested structures. L. Stanculescu, R. Chaine, M.P. Cani, K. Singh. Shape Modeling International 2013. [PDF] [Video]
dp 25 Direct Space-Time Trajectory Control for Visual Media Editing.
S. Santosa, F. Chevalier, R. Balakrishnan and K. Singh.In Proceedings of the SIGCHI conference on Human Factors in computing systems (CHI '13), pages 1149-1158. May 2013. Best Paper Honorable Mention [PDF] [Video]
spaceship 24 Design-Driven Quadrangulation of Closed 3D Curves. Mikhail Bessmeltsev, Caoyu Wang, Alla Sheffer, Karan Singh. ACM Transactions on Graphics (Proc. SIGGRAPH ASIA 2012), Volume 31, Issue 5, December 2012 [PDF] [Details]
crossshade 23 CrossShade: Shading Concept Sketches Using Cross-Section Curves. Cloud Shao, Adrien Bousseau, Alla Sheffer, Karan Singh. ACM Transactions on Graphics, SIGGRAPH, 2012. [PDF] [Details]
concepture 22 Concepture: A Framework for Recognizing Gestures with Repetitive Patterns. Nilgun Donmez, Karan Singh. Eurographics Sketch Based Interfaces and Modeling, SBIM 2012. [PDF] (Best Paper Award)
mickey 21 Elasticurves: Exploiting Stroke Dynamics and Inertia for the Real-time Neatening of Sketched 2D Curves. Yannick Thiel, Karan Singh, Ravin Balakrishnan. ACM UIST 2011 [PDF] [Slides] [Video] [Details]
slices 20 Slices: A Shape-proxy Based on Planar Sections
James McCrae, Karan Singh, Niloy Mitra
SIGGRAPH Asia, 2011 [PDF] [Details]
french 19 Neatening sketched strokes using piecewise French Curves
James McCrae, Karan Singh
Sketch-Based Interfaces and Modeling, 2011
[PDF] [Project]
geob 18 GeoBrush: Interactive Mesh Geometry Cloning (2011). 
Kenshi Takayama, Ryan Schmidt, Karan SinghTakeo IgarashiTamy BoubekeurOlga Sorkine.
Eurographics 2011 / Computer Graphics Forum.
[PDF] [Video] [Details, Slides, Demo, Source Code
17 meshmixer: an interface for rapid mesh composition (2010). 
Ryan Schmidt, Karan Singh.
ACM SIGGRAPH 2010 - Talks Program
[PDF] [Video] [Project] [Program] [ACM DL]
16 Analytic Drawing of 3D Scaffolds (2009)
Ryan Schmidt, Azam Khan, Karan Singh, Gord Kurtenbach. 
ACM TOG, 28(5) (Proc. SIGGRAPH ASIA 2009)
.
[PDF] [Slides] [Video] [Details]
Dog Image 15 On Expert Performance in 3D Curve-Drawing Tasks (2009)
Ryan Schmidt, Azam Khan, Gord Kurtenbach, Karan Singh. 
Eurographics Symposium on Sketch-Based Interfaces and Modeling 2009. 
[PDF] [Slides] [Details]
14 Sketch-Based Path Design James McCrae, Karan Singh. Graphics Interface, 2009.

 

13 EverybodyLovesSketch: 3D Sketching for a Broader Audience. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. Proceedings of ACM Symposium on User Interface Software and Technology 2009 (Victoria, BC, Canada, October 4-7, 2009), pp. 59-68 [PDF][video][Details]
12 ILoveSketch: As-natural-as-possible sketching system for creating 3D curve models. Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh,  ACM Symposium on User Interface Software and Technology 2008 (Monterey, CA, USA, October 19-22, 2008). [PDF][video][Details]
11 Sketching Piecewise Clothoid Curves. James McCrae, Karan Singh. ( EG Sketch-Based Interfaces and Modeling, 2008). [PDF][video][Details] Best paper award.

 

10 Sketch-Based Procedural Surface Modeling and Compositing with Surface Trees (2008)
Ryan Schmidt, Karan Singh.
Computer Graphics Forum, 27(2), pp. 321-330. (Proceedings of Eurographics 2008).
[PDF] [Video] [Details]
Dog Image 9 Sketching and Composing Widgets for 3D Manipulation (2008)
Ryan Schmidt, Karan Singh, Ravin Balakrishnan.
Computer Graphics Forum, 27(2), pp. 301-310. (Proceedings of Eurographics 2008).
[PDF] [Video] [Details]
Dog Image 8 Sketching, Scaffolding, and Inking: A Visual History for Interactive 3D Modeling (2007)
Ryan Schmidt, Tobias Isenberg, Pauline Jepp, Karan Singh, Brian Wyvill.
Proceedings of NPAR 2007, pp. 23-32.
[PDF] [PDF Slides] [Details]
7

Industrial motivation for interactive shape modeling: a case study in conceptual automotive design.

Karan Singh (ACM SIGGRAPH 2006, Course Notes).

6

Space Deformations and their Application To Shape Modeling.

Alexis Angelidis, Karan Singh (ACM SIGGRAPH 2006, Course Notes).

5 An interface for 3D sculpting via physical proxy.

Jia Sheng, Ravin Balakrishnan, Karan Singh. (2006).  Proceedings of GRAPHITE 2006 – International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia. p. 213-220. [video]

4

A suggestive interface for image guided 3D sketching.

Steven Tsang, Ravin Balakrishnan, Karan Singh, Abhishek Ranjan. (2004).  Proceedings of CHI 2004 the ACM Conference on Human Factors in Computing Systems. p. 591-598.

[video]

3

An interface for creating and manipulating curves using a high degree-of-freedom input device.

Tovi Grossman, Ravin Balakrishnan, Karan Singh. (2003). Proceedings of CHI 2003 p. 185-192.

[video]

interview/demo on Discovery Channel's Daily Planet

interview/demo on CityPulse24

2

Interactive curve and surface design using digital french curves

Karan Singh (ACM I3DG 1999).

1

Wires: A geometric deformation technique Karan Singh, Eugene Fiume (SIGGRAPH 1998)