This document describes a technique called shading-based surface editing that allows users to intuitively modify 3D surfaces by directly sketching changes to the shading or lighting of the surface. The key ideas are to relate small changes in shading to small, smooth changes in surface shape using shape-from-shading reconstruction, and to formulate the problem as an optimization that preserves surface detail outside of edited regions. Stroke attributes and constraints are used to stably modify shapes and handle features like highlights. Results demonstrated real-time editing of surfaces on a laptop. Future work includes improving techniques for blurring strokes.

1. Shading-Based Surface Editing YotamGingold and Denis Zorin New York University SIGGRAPH08

2. Abstract A free-form surface modeling based on shading

3. Outline Introduction Related Work Shading Changed to Shape Changes Overview of the System Problem Formulation Results Conclusions and Future Work

4. Introduction

5. Surface Editing 2D UI  3D model & motion Shape-from-shading (SfS) reconstruction Sketch-basedmodeling

6. Motivation Indirection User action vs. appearance change Hard to deform Smooth outline Remove shadow Reshape a highlight

7. Purpose A directly sketched-based surface modeling Principle of continuity if a user makes a small change in surface appearance, the resulting shape change should be small

8. Challenges standard formulation(Lambertian surface, orthographic projection, directional light) is known to be ill-posed.

9. Challenges Avoid small shading modification leading to large and unintuitive model changes Preserve existing surface detail during editing Region of interesting (ROI)during modifying Realtime surface update

10. Our major techniques Design stroke-based 2D UI SfS by solving a quadratic optimization

11. Related Work

12. Related Work Shape-Preserving [Sorkine et al. 04] [Yu et al. 04] [Wardetzky et al. 07] Shape-from-Shading [Rushmeier et al. 03] [Prados 04] Sketch-based modeling [Igarashi et al. 99] [Cheutet et al. 04] [Lawrence and Funkhouser 04] [Kara et al. 06] [Karpenko and Huges 06] [Nealen et al. 07] Silhouette Editing [DeCarloet al. 03]. Suggestive contour [Nealen et al.05], [Zimmermann et al. 07]

13. Shape-from-Shading Reconstruction The recover of shape form a gradual variation of shading in the image z x y Z(x,y)

14. Shading Changes to Shape Changes Guarantee the stability of surface changes and satisfying boundary constraints

15. a continuous solution an approximate solution all solutions are discontinuous (with either one or two sides fixed)

16. Instability near highlights Conclusion Smooth deformation can’t erase highlight A large change in the surface shape Strategy Terminate erasing strokes at highlight Highlight removal

17. Slope ambiguity Convex-concave ambiguity slope ambiguity Strategy- choose the slope to change the surface the least

18. Overview of the System

19. I(q) = ρ(n(p)) Only one light source pl pv Iimage q q n v L Lambertian & glossy reflection model , n(p) I p Msurface User modified ~Msurface ~I

20. Lambertianand Glossy Reflection Model β is the degree of glossiness pis the Phong exponent h = (v+l) / |v+l|

21. Iimage I(q) = ρ(n(p)) Only one light source C pl q q v n L q = P(p) ~I Lambertian & glossy reflection model , n(p) p P-1(C) Msurface

23. Smoothness

25. Stroke attributes – f (softness)

26. Stroke attributes – w (width)

27. Problem Formulation

28. Surface Optimization Function Detail-preserving Preserving appearance outside strokes Stroke constrain Match the modified surface under the stroke Detail-preserving Stroke constrain

29. Detail-preserving [Yu et al. 04]

30. The vector Laplacianis the normal scaled by the mean curvature [Sorkine et al. 04] If the surface changes remain close to isometric, the Laplacian operator does not change [Wardetzky et al. 07]. The Laplacian difference ΔM : Laplace-Beltrami operator H: mean curvature

31. Small triangle distortion ? = isometric deformations

32. Hypothesis If the triangle distortion stays small, one can view the Laplacian difference energy as a weighted normal change penalty (detail-preserving)

33. Hypothesis Want the normals to retain their spatial direction with respect to the viewing direction and the light source Strokes constrain the rotation of normals Find min. αs.t. ρ(n(α)) = Itrg

34. Stroke smoothness and thick strokes Weaken the link between stroke and the rest of surface (detail-preserving) C P(x0) h(r) x0 r w/2 -w/2 (1-c)/d = f

35. Detail-preserving (detail-preserving)

36. Stroke constrain

37. xj xi Constrain the new tangent

38. Constraint the projected position of P(p) = P(ap1 + (1-a) p2) C P(p1) xj P(p) P(p2) xi p p1 p2

39. Realization of Stroke Attributes Stroke smoothness and thick strokes Silhouette strokes Interaction with highlights Highlight motion strokes

40. Silhouette strokes No opacity α, no value Iv Smoothness

41. Interaction with highlights Cross a highlight Local max of ρ(n)  Large changed range, discontinuity  Terminate stroke at the highlight

42. Highlight motion strokes Xnewtrg of after before

44. Result MacBook Pro + 2GHx Intel Core Duo processor Performance issues Stroke size, ROI setting, mesh size, degree of adaptive refinement

45. Conclusions and Future Work Shading-based surface editing A direct and intuitive UI to modify surface Intuitive shading strokes Future work Blur stroke

46. END

47. END