Mais conteúdo relacionado Semelhante a S. Pena Serna - Enriching (20) S. Pena Serna - Enriching1. Enriching 3D Collections
Sebastian Pena Serna
Fraunhofer-Institut für Graphische
Datenverarbeitung IGD
Fraunhoferstraße 5
64283 Darmstadt
Tel +49 6151 155 – 468
sebastian.pena.serna@igd.fraunhofer.de
www.igd.fraunhofer.de
© Fraunhofer IGD
2. Definitions
3D Collection
Digital archive with multimedia material and 3D artifacts, which is
associated with semantic information
Building
Acquisition and ingestion of digital assets and their corresponding
provenance information
Accessing
Browsing and exploration of digital assets in the 3D collection
Enriching
Increasing the associations within the semantic network
2
© Fraunhofer IGD
9. Digitization
3D geometry
Material properties
Digital provenance
9
© Fraunhofer IGD
10. Processing
Improve the quality of 3D artifacts
Process 3D artifacts for different purposes (e.g. research,
presentation)
10
© Fraunhofer IGD
11. Provenance
Legacy and rich processing metadata
used_as_derivation_source
A.15-1955-dome-out.zip used_as_derivation_source
IvoryPanel
3IvoPan_LegacyData.rdf 2009CA5307v
Coloured.ply
4Ivory_Arc3DPro Arc3D-A.15-1955_dmy.v3d 5Ivory_MeshLa
cEvent.rdf bProcEvent.rdf
has_created created_derivative
digitized created_derivative
Legend
A.15-1955-dome-
forms_part_o out.rdf
f 2009CR4851_0.rdf has_created Digitization_Process
1IvoryPanel_O
bjAcqEvent.rdf forms_part_o … 2009CR4851_0.tif Formal_Derivation
f 2009CA5306_0.rdf
…
forms_part_o 2IvoryPanel_ forms_part_o Sub-events
f DocEvent.rdf f has_created 2009CA5306_0.tif
Data_Object
Man_Made_Object
11
© Fraunhofer IGD
12. Ingestion
Individual objects with high- Large acquisition campaigns
quality metadata with similar structures
12
© Fraunhofer IGD
13. Accessing a 3D collection
Accessing:
search and browse
13
© Fraunhofer IGD
14. Metadata Accessing
Stanford Repository
3D artifacts without
searchable metadata
http://www-graphics.stanford.edu/data/3Dscanrep/
14
© Fraunhofer IGD
15. Metadata Accessing
AIM@SHAPE
3D artifacts with basic
searchable metadata,
e.g. categories, keywords
http://shapes.aim-at-shape.net/
15
© Fraunhofer IGD
16. Metadata Accessing
3D-COFORM
3D artifacts with rich
metadata
Fundamental
categories and
relationships
Searchable material
and shape properties
16
© Fraunhofer IGD
21. 3D Shape Annotation
Aim: associate digital 3D shapes with related information and
knowledge on the represented object
Annotation: mechanism for enriching digital 3D shapes with
semantics
Result: annotated shape or a semantically enriched shape,
combining:
the geometric description
contextual information
knowledge of the represented object
the created relationships
© Fraunhofer IGD
22. Sponsors
Projects:
AIM@SHAPE (http://www.aimatshape.net/)
Focus K3D (http://www.focusk3d.eu/)
3D-COFORM (www.3d-coform.eu)
V-MusT (http://www.v-must.net/)
Enhancing Engagement with 3D Heritage Data through Semantic Annotation
(http://www.ddsgsa.net/projects/empire/Empire/Home.html)
Semantic Annotations for 3D Artefacts
(http://itee.uq.edu.au/~eresearch/projects/3dsa)
Technologies:
Linking Open Data
(http://esw.w3.org/SweoIG/TaskForces/CommunityProjects/LinkingOpenData)
3D Internet (Alpcan et al. 2007 [33])
© Fraunhofer IGD
25. Geometric Definition
Aim:
Understand the intrinsic
structure of the digital 3D
shape (Attene et al. 2006
[1], De Floriani et al. 2010
[2])
Associate semantics with
relevant part(s) of the
digital 3D shape
(Spagnuolo and Felcidieno
2009 [3])
© Fraunhofer IGD
27. Geometric Definition
Principles:
RANSAC (Schnabel et al. 2007 [7])
Curvature analysis (Madeira et al. 2007 [8])
Contour analysis (Liu and Zhang 2007 [9])
Discrete operators (Reuter et al. 2009 [10])
Physics (Fang et al. 2011 [11])
Concavity (Au et al. 2011 [12])
© Fraunhofer IGD
30. Geometric Definition
Strategies:
Skeletons to identify the structure of the digital 3D (Tierny et al.
2007 [19], Shapira et al. 2008 [20]) and/or by means of fitting
primitives (Attene et al. 2006 [21]).
© Fraunhofer IGD
31. Geometric Definition
Strategies:
User assisted segmentation
for complex digital 3D
shapes or for additional
requirements, e.g. functions
or styles (De Floriani et al.
2008 [22], Miao et al. 2009
[23], Bergamasco et al. 2011
[24]).
© Fraunhofer IGD
35. Geometric Definition
Challenges:
Difficult to generate a plausible and context-aware
geometric definition for different classes of objects.
The current strategies cannot easily be mapped to the
different applications’ requirements within a given domain.
There are few approaches trying to map principles to specific
applications’ requirements.
A combination of principles, strategies and user guidance
could generate the expected results.
© Fraunhofer IGD
37. Structured Information and
Knowledge
There is a vast amount of existent information and
knowledge related to any digital 3D shape:
Information related to the intrinsic structure of the 3D
shape
Information related to the meaning of the represented
object
Information related to the digital provenance
Knowledge related to the application domain
© Fraunhofer IGD
39. Structured Information and
Knowledge
Structured Information for describing digital 3D shapes
using concepts within a particular domain (Catalano et
al. 2009 [34], De Luca et al. 2011 [35], Mortara et al.
2006 [36]).
© Fraunhofer IGD
40. Structured Information and
Knowledge
Structured Information in the engineering
domain
Product and Manufacturing Information (PMI)
Geometric Dimensions and Tolerances (GD&T)
Functional Tolerancing and Annotation
(FT&A).
Standard ASME Y14.41-2003 Digital Product
Data Definition Practices
ISO 1101:2004 Geometrical Product
Specifications (GPS) - Geometrical tolerancing.
(Spatial Corp.)
© Fraunhofer IGD
43. Mechanisms for Annotating
Different mechanisms have been proposed, which vary
depending on:
application domain
degree of user intervention that they require
technology supporting them
degree of structured information which they involve.
© Fraunhofer IGD
44. Mechanisms for Annotating
Application domain
Product design (Andre and Sorito
2002 [39])
Architecture (Pittarello and Gatto
2011 [40])
Cultural Heritage (Hunter and
Gerber 2010 [41])
Chemistry (Gawronski and
Dumontier 2011 [42])
Medicine (Trzupek et al. 2011 [43])
© Fraunhofer IGD
45. Mechanisms for Annotating
User intervention
Semi-automatic mechanisms normally require of a
degree of user intervention to define an annotation
(Shapira et al. 2010 [13], Kalogerakis et al. 2010 [18]).
© Fraunhofer IGD
46. Mechanisms for Annotating
Supporting technology:
stand-alone modeling
systems
stand-alone 3D viewers Siemens NX
(Pena Serna et al. 2011
[27])
web based viewers
(Hunter et al. 2010 [44])
© Fraunhofer IGD
48. Representation of the Annotation
Approach to structure, store and transmit the
annotating process output
Important for the annotation’s indexing, retrieval and
reutilization.
There is no agreed format for this.
© Fraunhofer IGD
49. Representation of the Annotation
Strategies: Persistent annotations
Store the annotation in a database based on a semantic
model.
The model describes the associations or relations
between different media ([16], [27], Hunter et al. 2010
[45]).
© Fraunhofer IGD
50. Representation of the Annotation
Strategies: Transient annotations
Store and transmit annotations in a data file.
MPEG-7 (Bilasco et al. 2006 [46])
VRML / X3D (Pittarello and Faveri 2006 [47], [40], [26])
Jupiter (JT) Data Format
Product Representation Compact (PRC) Data Format
COLLADA ([37], [38])
Universal 3D Data Format
ASME Y14.41 Digital Product Definition Data Practices
© Fraunhofer IGD
51. Representation of the Annotation
Issues:
Stability, flexibility and easy of use
There is no notion of annotation representation.
It is considered as a piece of text, which is stored in a database or
as a tag on a digital 3D shape.
Annotations’ interoperability
Degree of independency from transient digital 3D shapes.
© Fraunhofer IGD
52. Enriching a 3D collection
Challenges and Opportunities
This remains an active area of research. Different challenges need to
be solved to fully support a semantic enrichment pipeline:
Automatically extracting information from a digital 3D shape
Modeling semantic information
Automatically linking it to the digital 3D shape
Using standards to store, interoperate, and preserve annotations in
the long term
© Fraunhofer IGD
53. Enriching a 3D collection
Challenges and Opportunities
Opportunities of using semantically aware 3D shapes:
searching 3D shapes
intelligently interacting with semantically aware 3D shapes
shape matching or deriving meaning of new shapes
high-level editing
goal oriented 3D synthesizing
knowledge management
semantic visualization and interaction
© Fraunhofer IGD
54. Workflow with 3D collections
Accessing:
search and browse
54
© Fraunhofer IGD
55. Enabling Technologies
Cloud Computing
Storage and computation Cloud
capacity online
Computing
3D Internet
Visualization of 3D artifacts on
standard web browsers
Mobile devices Mobile 3D
Access and visualization on the devices Internet
move
55
© Fraunhofer IGD
56. Emerging Challenges
Define workflows
Create services
Enable intuitive access
Provide contextualized interfaces
User involvement and engagement
56
© Fraunhofer IGD
57. References
[1] ATTENE M., BIASOTTI S., MORTARA M., PATANÉ G., SPAGNUOLO M., FALCIDIENO B.: Computational methods for understanding 3D shapes. Computers & Graphics
30, 3 (June 2006), 323–333.
[2] DE FLORIANI L., MAGILLO P., PAPALEO L., PUPPO E.: Shape modeling and understanding: Research trends and results of the G3 group at DISI.
[3] SPAGNUOLO M., FALCIDIENO B.: 3D media and the semantic web. IEEE Intelligent Systems (March/April 2009), 90–96.
[4] ATTENE M., KATZ S., MORTARA M., PATANÉ G., SPAGNUOLO M., TAL A.: Mesh segmentation - a comparative study. In Shape Modeling International (2006).
[5] SHAMIR A.: A survey on mesh segmentation techniques. Computer Graphics Forum 27, 6 (2008), 1539–1556.
[6] CHEN X., GOLOVINSKIY A., FUNKHOUSER T.: A benchmark for 3D mesh segmentation. In ACM SIGGRAPH 2009 papers (New Orleans, Louisiana, 2009), ACM, pp. 1–
12.
[7] SCHNABEL R., WAHL R., KLEIN R.: Efficient RANSAC for Point-Cloud shape detection. Computer Graphics forum 26, Number 2 (June 2007), 214–226.
[8] MADEIRA J., SILVA S., STORK A., PENA SERNA S.: Principal Curvature-Driven segmentation of mesh models: A preliminary assessment. In 15 EPCG - Encontro
Português de Computação Gráfica. (2007).
[9] LIU R., ZHANG H.: Mesh segmentation via spectral embedding and contour analysis. Volume 26 (2007), Number 3.
[10] REUTER M., BIASOTTI S., GIORGI D., PATANÉ G., SPAGNUOLO M.: Discrete Laplace-Beltrami operators for shape analysis and segmentation. Computers & Graphics
33, 3 (June 2009), 381–390.
[11] FANG Y., SUN M., KIM M.: Heat-Mapping: a robust approach toward perceptually consistent mesh segmentation. IEEE Computer Vision and Pattern Recognition
(CVPR) 2011 (2011), pp 2145–2152.
[12] AU O. K., ZHENG Y., CHEN M., XU P., TAI C.: Mesh segmentation with concavity-aware fields. IEEE Trans. Vis. Comp. Graphics (2011).
[13] SHAPIRA L., SHALOM S., SHAMIR A., COHEN-OR D., ZHANG H.: Contextual part analogies in 3D objects. Int. J. Comput. Vision 89, 2-3 (2010), 309–326.
[14] WANG Y., XU K., LI J., ZHANG H., SHAMIR A., LIU L., CHENG Z., XIONG Y.: Symmetry hierarchy of Man-Made objects. Computer Graphics Forum 30, 2 (2011), 287–
296.
[15] HO T., CHUANG J.: Volume based mesh segmentation. Journal of Information Science and Engineering 27 (2011).
[16] ATTENE M., ROBBIANO F., SPAGNUOLO M., FALCIDIENO B.: Characterization of 3D shape parts for semantic annotation. Computer-Aided Design 41, 10 (Oct. 2009),
756–763.
[17] GOLOVINSKIY A., FUNKHOUSER T.: Consistent segmentation of 3D models. Computers & Graphics 33, 3 (June 2009), 262–269.
[18] KALOGERAKIS E., HERTZMANN A., SINGH K.: Learning 3D Mesh Segmentation and Labeling. ACM Transactions on Graphics 29, 3 (2010).
© Fraunhofer IGD
58. References
[19] TIERNY J., VANDEBORRE J.-P., DAOUDI M.: Topology driven 3d mesh hierarchical segmentation. In Proceedings of the IEEE International Conference on Shape
Modeling and Applications 2007 (Washington, DC, USA, 2007), IEEE Computer Society, pp. 215–220.
[20] SHAPIRA L., SHAMIR A., COHEN-OR D.: Consistent mesh partitioning and skeletonisation using the shape diameter function. The Visual Computer: International
Journal of Computer Graphics 24, 4 (Mar. 2008).
[21] ATTENE M., FALCIDIENO B., SPAGNUOLO M.: Hierarchical mesh segmentation based on fitting primitives. The Visual Computer: International Journal of Computer
Graphics 22 (2006), 181–193.
[22] DE FLORIANI L., PAPALEO L., CARISSIMI N.: A Java3D framework for inspecting and segmenting 3D models. In Proceedings of the 13th international symposium on
3D web technology (Los Angeles, California, 2008), ACM, pp. 67–74.
[23] MIAO Y., FENG J., WANG J., JIN X.: User-controllable mesh segmentation using shape harmonic signature. Progress in Natural Science 19, 4 (Apr. 2009), 471–478.
[24] BERGAMASCO F., ALBARELLI A., TORSELLO A.: Semi-supervised segmentation of 3D surfaces using a weighted graph representation. In Proceedings of the 8th
international conference on Graph-based representations in pattern recognition (GbRPR’11) (2011).
[25] JI Z., LIU L., CHEN Z., WANG G.: Easy mesh cutting. Computer Graphics Forum 25, 3 (2006), 283–291.
[26] PAPALEO L., DE FLORIANI L.: Manual segmentation and semantic-based hierarchical tagging of 3D models. (2010) pp. 25–32.
[27] PENA SERNA S., SCOPIGNO R., DOERR M., THEODORIDOU M., GEORGIS C., PONCHIO F., STORK A.: 3D-centered media linking and semantic enrichment through
integrated searching, browsing, viewing and annotating. In VAST11: The 12th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural
Heritage (Prato, Italy, 2011).
[28] KAPLANSKY L., TAL A.: Mesh segmentation refinement. In Computer Graphics Forum (Pacific Graphics), 28(7) (Oct. 2009), pp. 1995–2003.
[29] KNOPP J., PRASAD M. , VAN GOOL L. : Scene Cut: Class-specific Object Detection and Segmentation in 3D Scenes. In 3DIMPVT, Hangzhou, 2011
[30] JULIUS D., KRAEVOY V., SHEFFER A.: D-charts: Quasi-developable mesh segmentation. In Computer Graphics Forum, Proceedings of Eurographics 2005 (Dublin,
Ireland, 2005), vol. 24, Eurographics, Blackwell, pp. 581–590.
[31] MORTARA M., SPAGNUOLO M.: Semantics-driven best view of 3D shapes. Computers & Graphics 33, 3 (June 2009), 280–290.
[32] SIMARI P., NOWROUZEZAHRAI D., KALOGERAKIS E., SINGH K.: Multi-objective shape segmentation and labeling. In Proceedings of the Symposium on Geometry
Processing (Berlin, Germany, 2009), Eurographics Association, pp. 1415–1425.
[33] ALPCAN T., BAUCKHAGE C., KOTSOVINOS E.: Towards 3d internet: Why, what, and how? In Proceedings of the International Conference on Cyberworlds CW ’07
(October 2007), pp. 95 – 99.
[34] CATALANO C., CAMOSSI E., FERRANDES R., CHEUTET V., SEVILMIS N.: A product design ontology for enhancing shape processing in design workflows. Journal of
Intelligent Manufacturing 20, 5 (Oct. 2009), 553–567. 3
© Fraunhofer IGD
59. References
[35] LUCA L. D., BUSAYARAT C., STEFANI C., VÉRON P., FLORENZANO M.: A semantic-based platform for the digital analysis of architectural heritage. Computers &
Graphics 35, 2 (Apr. 2011), 227–241.
[36] MORTARA M., PATANÉ G., SPAGNUOLO M.: From geometric to semantic human body models. Computers&Graphics 30, 2 (Apr. 2006), 185–196.
[37] RODRIGUEZ ECHAVARRIA K., MORRIS D., ARNOLD D.: Web based presentation of semantically tagged 3D content for public sculptures and monuments in the UK.
In Proceedings of the 14th International Conference on 3D Web Technology (Darmstadt, Germany, 2009), ACM, pp. 119–126.
[38] HAVEMANN S., SETTGAST V., BERNDT R., EIDE., FELLNER D. W.: The Arrigo showcase reloaded - towards a sustainable link between 3D and semantics. J. Comput.
Cult. Herit. 2, 1 (2009), 1–13.
[39] ANDRE P., SORITO R.: Product manufacturing information (PMI) in 3D models: a basis for collaborative engineering in product creation process (PCP). In 14th
European Simulation Symposium and Exhibition (2002).
[40] PITTARELLO F., GATTO I.: ToBoA-3D: an architecture for managing top-down and bottom-up annotated 3D objects and spaces on the web. In Web3D ’11
Proceedings of the 16th International Conference on 3D Web Technology (2011).
[41] HUNTER J., GERBER A.: Harvesting community annotations on 3D models of museum artefacts to enhance knowledge, discovery and re-use. Journal of Cultural
Heritage 11, 1 (2010), 81–90.
[42] GAWRONSKI A., DUMONTIER M.: MoSuMo: a semantic web service to generate electrostatic potentials across solvent excluded protein surfaces and binding
pockets. Computers & Graphics 35, 4 (Aug. 2011), 823–830.
[43] TRZUPEK M., OGIELA M. R., TADEUSIEWICZ R.: Intelligent image content semantic description for cardiac 3D visualisations. Engineering Applications of Artificial
Intelligence In Press, Corrected Proof (2011).
[44] HUNTER J., YU C.-H., NAKATSU R., TOSA N., NAGHDY F., WONG K., CODOGNET P.: Supporting multiple perspectives on 3D museum artefacts through interoperable
annotations. Vol. 333 of IFIP Advances in Information and Communication Technology. Springer Boston, 2010, pp. 149–159.
[45] HUNTER J., COLE T., SANDERSON R., VAN DE SOMPEL H.: The open annotation collaboration: A data model to support sharing and interoperability of scholarly
annotations. (2010)
[46] BILASCO I. M., GENSEL J., VILLANOVA-OLIVER M., MARTIN H.: An MPEG-7 framework enhancing the reuse of 3D models. In Proceedings of the eleventh
international conference on 3D web technology (Columbia, Maryland, 2006), ACM, pp. 65–74.
[47] PITTARELLO F., FAVERI A. D.: Semantic description of 3D environments: a proposal based on web standards. In Proceedings of the eleventh international
conference on 3D web technology (Columbia, Maryland, 2006), ACM, pp. 85–95.
© Fraunhofer IGD
60. Thank You!
Sebastian Pena Serna
Fraunhofer-Institut für Graphische
Datenverarbeitung IGD
Fraunhoferstraße 5
64283 Darmstadt
Tel +49 6151 155 – 468
sebastian.pena.serna@igd.fraunhofer.de
www.igd.fraunhofer.de
60
© Fraunhofer IGD
61. IVB: Integrated Viewer / Browser
Access and enrichment of 3D collections
Searching and browsing
Searching: flexible formulation of queries
Browsing: exploration of multiple results and query
refinement
Viewing and Annotating
Viewing: inspection and analysis of multimedia objects
Annotating: building and enrichment of semantic
relationships
61
© Fraunhofer IGD
62. IVB: Searching and Browsing
Interface
multimedia results
Querying 3D
collections
Exploring
62
© Fraunhofer IGD
63. 63
Inspecting and tagging
3D models
© Fraunhofer IGD
Interface
IVB: Viewing and Annotating
Enriching 3D with
multimedia objects