SlideShare uma empresa Scribd logo

Model-driven engineering of multimodal user interfaces

•
•
Jean Vanderdonckt
Jean Vanderdonckt

This presentation explains to use model-driven engineering techniques for producing multimodal user interfaces for the Web.

NegóciosTecnologia
1 de 42
Baixar para ler offline
Model-Driven Engineering of User Interfaces for Multimodal Web Applications Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi
What is a multimodal user interface? ,[object Object],[object Object],[object Object]
What is a multimodal user interface? ,[object Object],[Tison,2003]
What is a multimodal user interface? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[Bodnar,2004]
What is a multimodal user interface? ,[object Object],[object Object],[object Object],[object Object],[Bellik,2002]
Why are multimodal user interfaces important? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[IntuiLab,2006]
Why are multimodal user interfaces challenging? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Concepts: The CARE Properties ,[object Object],[object Object],[object Object],[object Object],[object Object],Put that there [Bolt,1979]
What are the appropriate abstractions? ,[object Object],[object Object],[object Object],[object Object]
Why a design space? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Multimodal Web Applications ,[object Object],[object Object],[object Object],[object Object],[W3C,2006]
Solution space ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[Stanciulescu et al.,2005]
Design options for graphical web UIs Design options for graphical UIs Sub-task presentation Sub-task navigation Concretization of navigation & control Navigation concretization Control concretization
Design options for graphical web UIs ,[object Object],Sub-task presentation separated combined one by one combined box tabbed dialog box float window many at once float window all in one areas with separators group boxes bulleted list numbered list Abstract level Concrete level
Sub-task presentation – example ,[object Object],[object Object],T11 T12 T13 T1 AC11 AC12 AC13 AC1 T11 T12 T13 T1 AC11 AC12 AC13 NAC LHS RHS NAC LHS RHS
Design options for graphical web UIs ,[object Object],Navigation concretization [Stanciulescu,2006] placement cardinality local global simple multiple
Sub-task navigation - example ,[object Object],[object Object],AC1 T11 T12 T13 T1 AC11 AC12 AC13 AIC111 AIC121 AIC131 AC1 T11 T12 T13 T1 AC11 AC12 AC13 AIC11 NAC LHS RHS NAC LHS RHS
Design options for graphical web UIs ,[object Object],Sub-task navigation asynchronous sequential (synchronous)
Design options for graphical web UIs ,[object Object],Concretization of navigation & control separated combined
Design options for graphical web UIs ,[object Object],Control concretizations placement cardinality local global multiple simple
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],Design options for multimodal UIs [Scapin & Bastien,1997] [Vanderdonckt,1995] Prompt Grouping / Distinction of items Immediate feedback Guidance
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object]
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object]
Design options for multimodal web UIs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Multimodal example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[Stanciulescu,2006] Prompt: multimodal Please say your name (vocal + graphical) 1 Input: multimodal Juan Gonzalez 2 (vocal + graphical) Juan Gonzalez Guidance for input: iconic Guidance for feedback: iconic Feedback:multimodal Your answer was: Juan Gonzalez 3 (graphical + vocal)
Vocal Concrete Interaction Objects ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Vocal Concrete Interaction Objects ,[object Object]
Multimodal Concrete Interaction Objects ,[object Object],outputText + inputText vocalPrompt + vocalInput outputText + inputText + vocalInput + vocalFeedback + guidance groupBox + radioButtons vocalMenu + vocalItems + vocalInput groupBox + radioButtons + vocalMenu + vocalItems + vocalInput + guidance Graphical Vocal Multimodal
Concrete UI Relationships ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Tool support ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Multimodal Concrete Interaction Objects ,[object Object],[object Object]
Specification of transformation ,[object Object],[object Object],[object Object]
How transformation rules work… ,[object Object],[object Object],[object Object],[Limbourg,2004] Transformation rule Initial Model Resulting Model
UsiXML syntax of transformation rules NAC NAC RHS mappings
Transformation rule NAC: LHS: RHS: Multimodal prompt Guidance for input Multimodal input Guidance for output Vocal feedback
Case studies ,[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Conclusion ,[object Object],[object Object],[object Object],[object Object],[object Object],[Stanciulescu & Vanderdonckt,2006] [Beaudouin-Lafon,2004]
Future work ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
References ,[object Object],[object Object],[object Object]
Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs Special thanks to all members of the team!

Recomendados

Model-driven engineering of user interfaces
Model-driven engineering of user interfacesModel-driven engineering of user interfaces
Model-driven engineering of user interfacesJean Vanderdonckt
 
User Interface Design
User Interface DesignUser Interface Design
User Interface DesignGil Pasiona
 
SECh1516
SECh1516SECh1516
SECh1516Joe Christensen
 
User interface-design
User interface-designUser interface-design
User interface-designDarkHorse Technologies Pvt Ltd
 
Nota sendiri hci-HCI
Nota sendiri hci-HCINota sendiri hci-HCI
Nota sendiri hci-HCIShafy Fify
 
User Interface Design in Software Engineering SE15
User Interface Design in Software Engineering SE15User Interface Design in Software Engineering SE15
User Interface Design in Software Engineering SE15koolkampus
 
HCI 3e - Ch 11: User support
HCI 3e - Ch 11: User supportHCI 3e - Ch 11: User support
HCI 3e - Ch 11: User supportAlan Dix
 
Unit iii(part c - user interface design)
Unit iii(part c - user interface design)Unit iii(part c - user interface design)
Unit iii(part c - user interface design)BALAJI A
 

Mais conteúdo relacionado

Mais procurados

USER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPTUSER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPTvicci4041
 
user support system in HCI
user support system in HCIuser support system in HCI
user support system in HCIUsman Mukhtar
 
HCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rulesHCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rulesAlan Dix
 
Ch09-User Interface Design
Ch09-User Interface DesignCh09-User Interface Design
Ch09-User Interface DesignFajar Baskoro
 
Unit 7 performing user interface design
Unit 7 performing user interface designUnit 7 performing user interface design
Unit 7 performing user interface designPreeti Mishra
 
User Interface Design
User Interface DesignUser Interface Design
User Interface DesignJReifman
 
16 user interfacedesign
16 user interfacedesign16 user interfacedesign
16 user interfacedesignrandhirlpu
 
Mobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best PracticesMobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best PracticesOXD
 
User interface design(sommerville) bangalore university
User interface design(sommerville) bangalore universityUser interface design(sommerville) bangalore university
User interface design(sommerville) bangalore universityJaisha Shankar
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationSalocin Dot TEN
 
Graphical user interface of web form
Graphical user interface of web formGraphical user interface of web form
Graphical user interface of web formmentorrbuddy
 
golden rules of user interface design
golden rules of user interface designgolden rules of user interface design
golden rules of user interface designgadige harshini
 
HCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software processHCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software processAlan Dix
 
Software Usability Implications in Requirements and Design
Software Usability Implications in Requirements and DesignSoftware Usability Implications in Requirements and Design
Software Usability Implications in Requirements and DesignNatalia Juristo
 
Pressman ch-11-component-level-design
Pressman ch-11-component-level-designPressman ch-11-component-level-design
Pressman ch-11-component-level-designOliver Cheng
 
User interface design
User interface designUser interface design
User interface designSlideshare
 
Android design patterns in mobile application development presentation
Android design patterns in mobile application development presentationAndroid design patterns in mobile application development presentation
Android design patterns in mobile application development presentationMichail Grigoropoulos
 

Mais procurados (19)

Chapter 12 user interface design
Chapter 12 user interface designChapter 12 user interface design
Chapter 12 user interface design
 
USER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPTUSER INTERFACE DESIGN PPT
USER INTERFACE DESIGN PPT
 
user support system in HCI
user support system in HCIuser support system in HCI
user support system in HCI
 
Seii unit5 ui_design
Seii unit5 ui_designSeii unit5 ui_design
Seii unit5 ui_design
 
HCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rulesHCI 3e - Ch 7: Design rules
HCI 3e - Ch 7: Design rules
 
Ch09-User Interface Design
Ch09-User Interface DesignCh09-User Interface Design
Ch09-User Interface Design
 
Unit 7 performing user interface design
Unit 7 performing user interface designUnit 7 performing user interface design
Unit 7 performing user interface design
 
User Interface Design
User Interface DesignUser Interface Design
User Interface Design
 
16 user interfacedesign
16 user interfacedesign16 user interfacedesign
16 user interfacedesign
 
Mobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best PracticesMobile UI Design – User Centered Design and UI Best Practices
Mobile UI Design – User Centered Design and UI Best Practices
 
User interface design(sommerville) bangalore university
User interface design(sommerville) bangalore universityUser interface design(sommerville) bangalore university
User interface design(sommerville) bangalore university
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 Documentation
 
Graphical user interface of web form
Graphical user interface of web formGraphical user interface of web form
Graphical user interface of web form
 
golden rules of user interface design
golden rules of user interface designgolden rules of user interface design
golden rules of user interface design
 
HCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software processHCI 3e - Ch 6: HCI in the software process
HCI 3e - Ch 6: HCI in the software process
 
Software Usability Implications in Requirements and Design
Software Usability Implications in Requirements and DesignSoftware Usability Implications in Requirements and Design
Software Usability Implications in Requirements and Design
 
Pressman ch-11-component-level-design
Pressman ch-11-component-level-designPressman ch-11-component-level-design
Pressman ch-11-component-level-design
 
User interface design
User interface designUser interface design
User interface design
 
Android design patterns in mobile application development presentation
Android design patterns in mobile application development presentationAndroid design patterns in mobile application development presentation
Android design patterns in mobile application development presentation
 

Semelhante a Model-driven engineering of multimodal user interfaces

Methodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User InterfacesMethodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User InterfacesJean Vanderdonckt
 
2015-04-29 research seminar
2015-04-29 research seminar2015-04-29 research seminar
2015-04-29 research seminarifi8106tlu
 
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...Jean Vanderdonckt
 
Ni week 2018_ux_lab_viewcandothat
Ni week 2018_ux_lab_viewcandothatNi week 2018_ux_lab_viewcandothat
Ni week 2018_ux_lab_viewcandothatDMC, Inc.
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Jean Vanderdonckt
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Serenoa Project
 
Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)Howard Kramer
 
Device Independence
Device IndependenceDevice Independence
Device Independencebjornh
 
A Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User InterfacesA Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User InterfacesDavid Céspedes-Hernández
 
What Is Interaction Design
What Is Interaction DesignWhat Is Interaction Design
What Is Interaction DesignGraeme Smith
 
Designidm C L P Revised
Designidm C L P RevisedDesignidm C L P Revised
Designidm C L P Revisedguest7ff51a
 
Human-Computer Interaction: Adaptation, Distribution, Evaluation
Human-Computer Interaction: Adaptation, Distribution, EvaluationHuman-Computer Interaction: Adaptation, Distribution, Evaluation
Human-Computer Interaction: Adaptation, Distribution, EvaluationJean Vanderdonckt
 
Model-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web PagesModel-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web PagesJean Vanderdonckt
 
User Interface Composition with UsiXML
User Interface Composition with UsiXMLUser Interface Composition with UsiXML
User Interface Composition with UsiXMLJean Vanderdonckt
 
When User Interface Patterns Become Mobile
When User Interface Patterns Become MobileWhen User Interface Patterns Become Mobile
When User Interface Patterns Become MobileJean Vanderdonckt
 
UX Practices for dApps on Tezos
UX Practices for dApps on TezosUX Practices for dApps on Tezos
UX Practices for dApps on TezosNeven6
 
Information and Instructional Technologies
Information and Instructional TechnologiesInformation and Instructional Technologies
Information and Instructional TechnologiesAlaa Sadik
 

Semelhante a Model-driven engineering of multimodal user interfaces (20)

Methodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User InterfacesMethodology for the Development of Vocal User Interfaces
Methodology for the Development of Vocal User Interfaces
 
2015-04-29 research seminar
2015-04-29 research seminar2015-04-29 research seminar
2015-04-29 research seminar
 
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, a...
 
Ni week 2018_ux_lab_viewcandothat
Ni week 2018_ux_lab_viewcandothatNi week 2018_ux_lab_viewcandothat
Ni week 2018_ux_lab_viewcandothat
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...
 
Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...Distributed User Interfaces: How to Distribute User Interface Elements across...
Distributed User Interfaces: How to Distribute User Interface Elements across...
 
Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)Designing and evaluating web sites using universal design principles (hands on)
Designing and evaluating web sites using universal design principles (hands on)
 
Device Independence
Device IndependenceDevice Independence
Device Independence
 
A Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User InterfacesA Methodology for the Development of Vocal User Interfaces
A Methodology for the Development of Vocal User Interfaces
 
What Is Interaction Design
What Is Interaction DesignWhat Is Interaction Design
What Is Interaction Design
 
Designidm C L P Revised
Designidm C L P RevisedDesignidm C L P Revised
Designidm C L P Revised
 
Ch11
Ch11Ch11
Ch11
 
Human-Computer Interaction: Adaptation, Distribution, Evaluation
Human-Computer Interaction: Adaptation, Distribution, EvaluationHuman-Computer Interaction: Adaptation, Distribution, Evaluation
Human-Computer Interaction: Adaptation, Distribution, Evaluation
 
Model-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web PagesModel-Based Approaches to Reengineering Web Pages
Model-Based Approaches to Reengineering Web Pages
 
User Interface Composition with UsiXML
User Interface Composition with UsiXMLUser Interface Composition with UsiXML
User Interface Composition with UsiXML
 
When User Interface Patterns Become Mobile
When User Interface Patterns Become MobileWhen User Interface Patterns Become Mobile
When User Interface Patterns Become Mobile
 
UX Practices for dApps on Tezos
UX Practices for dApps on TezosUX Practices for dApps on Tezos
UX Practices for dApps on Tezos
 
ACCESSIBLE project concept and achievements
ACCESSIBLE project concept and achievementsACCESSIBLE project concept and achievements
ACCESSIBLE project concept and achievements
 
Information and Instructional Technologies
Information and Instructional TechnologiesInformation and Instructional Technologies
Information and Instructional Technologies
 
Summ11 useinterx
Summ11 useinterxSumm11 useinterx
Summ11 useinterx
 

Mais de Jean Vanderdonckt

To the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User InterfacesTo the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User InterfacesJean Vanderdonckt
 
Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...Jean Vanderdonckt
 
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...Jean Vanderdonckt
 
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...Jean Vanderdonckt
 
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...Jean Vanderdonckt
 
Gesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOpsGesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOpsJean Vanderdonckt
 
Engineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with SlimeEngineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with SlimeJean Vanderdonckt
 
Evaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and MeasuresEvaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and MeasuresJean Vanderdonckt
 
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...Jean Vanderdonckt
 
Designing Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and PitfallsDesigning Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and PitfallsJean Vanderdonckt
 
Fundamentals of Gestural Interaction
Fundamentals of Gestural InteractionFundamentals of Gestural Interaction
Fundamentals of Gestural InteractionJean Vanderdonckt
 
Gestural Interaction, Is it Really Natural?
Gestural Interaction, Is it Really Natural?Gestural Interaction, Is it Really Natural?
Gestural Interaction, Is it Really Natural?Jean Vanderdonckt
 
User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...Jean Vanderdonckt
 
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...Jean Vanderdonckt
 
Attach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you WorkAttach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you WorkJean Vanderdonckt
 
The Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV GesturesThe Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV GesturesJean Vanderdonckt
 
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper BodyHead and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper BodyJean Vanderdonckt
 
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for SmartphonesG-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for SmartphonesJean Vanderdonckt
 
Vector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture RecognitionVector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture RecognitionJean Vanderdonckt
 
An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gesturesJean Vanderdonckt
 

Mais de Jean Vanderdonckt (20)

To the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User InterfacesTo the end of our possibilities with Adaptive User Interfaces
To the end of our possibilities with Adaptive User Interfaces
 
Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...Engineering the Transition of Interactive Collaborative Software from Cloud C...
Engineering the Transition of Interactive Collaborative Software from Cloud C...
 
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
UsyBus: A Communication Framework among Reusable Agents integrating Eye-Track...
 
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
µV: An Articulation, Rotation, Scaling, and Translation Invariant (ARST) Mult...
 
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
RepliGES and GEStory: Visual Tools for Systematizing and Consolidating Knowle...
 
Gesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOpsGesture-based information systems: from DesignOps to DevOps
Gesture-based information systems: from DesignOps to DevOps
 
Engineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with SlimeEngineering Slidable User Interfaces with Slime
Engineering Slidable User Interfaces with Slime
 
Evaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and MeasuresEvaluating Gestural Interaction: Models, Methods, and Measures
Evaluating Gestural Interaction: Models, Methods, and Measures
 
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
Conducting a Gesture Elicitation Study: How to Get the Best Gestures From Peo...
 
Designing Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and PitfallsDesigning Gestural Interaction: Challenges and Pitfalls
Designing Gestural Interaction: Challenges and Pitfalls
 
Fundamentals of Gestural Interaction
Fundamentals of Gestural InteractionFundamentals of Gestural Interaction
Fundamentals of Gestural Interaction
 
Gestural Interaction, Is it Really Natural?
Gestural Interaction, Is it Really Natural?Gestural Interaction, Is it Really Natural?
Gestural Interaction, Is it Really Natural?
 
User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...User-centred Development of a Clinical Decision-support System for Breast Can...
User-centred Development of a Clinical Decision-support System for Breast Can...
 
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
Simplifying the Development of Cross-Platform Web User Interfaces by Collabo...
 
Attach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you WorkAttach Me, Detach Me, Assemble Me like you Work
Attach Me, Detach Me, Assemble Me like you Work
 
The Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV GesturesThe Impact of Comfortable Viewing Positions on Smart TV Gestures
The Impact of Comfortable Viewing Positions on Smart TV Gestures
 
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper BodyHead and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
Head and Shoulders Gestures: Exploring User-De fined Gestures with Upper Body
 
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for SmartphonesG-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
G-Menu: A Keyword-by-Gesture based Dynamic Menu Interface for Smartphones
 
Vector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture RecognitionVector-based, Structure Preserving Stroke Gesture Recognition
Vector-based, Structure Preserving Stroke Gesture Recognition
 
An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures An ontology for reasoning on body-based gestures
An ontology for reasoning on body-based gestures
 

Último

Pitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deckPitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deckHajeJanKamps
 
EUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exportersEUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exportersPeter Horsten
 
Types of Cyberattacks - ASG I.T. Consulting.pdf
Types of Cyberattacks - ASG I.T. Consulting.pdfTypes of Cyberattacks - ASG I.T. Consulting.pdf
Types of Cyberattacks - ASG I.T. Consulting.pdfASGITConsulting
 
Simplify Your Funding: Quick and Easy Business Loans
Simplify Your Funding: Quick and Easy Business LoansSimplify Your Funding: Quick and Easy Business Loans
Simplify Your Funding: Quick and Easy Business LoansNugget Global
 
Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...
Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...
Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...Aggregage
 
14680-51-4.pdf Good quality CAS Good quality CAS
14680-51-4.pdf Good quality CAS Good quality CAS14680-51-4.pdf Good quality CAS Good quality CAS
14680-51-4.pdf Good quality CAS Good quality CAScathy664059
 
Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...
Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...
Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...ssuserf63bd7
 
Paul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate ProfessionalPaul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate ProfessionalPaul Turovsky
 
Excvation Safety for safety officers reference
Excvation Safety for safety officers referenceExcvation Safety for safety officers reference
Excvation Safety for safety officers referencessuser2c065e
 
Customizable Contents Restoration Training
Customizable Contents Restoration TrainingCustomizable Contents Restoration Training
Customizable Contents Restoration TrainingCalvinarnold843
 
1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdfShaun Heinrichs
 
Rakhi sets symbolizing the bond of love.pptx
Rakhi sets symbolizing the bond of love.pptxRakhi sets symbolizing the bond of love.pptx
Rakhi sets symbolizing the bond of love.pptxRakhi Bazaar
 
MEP Plans in Construction of Building and Industrial Projects 2024
MEP Plans in Construction of Building and Industrial Projects 2024MEP Plans in Construction of Building and Industrial Projects 2024
MEP Plans in Construction of Building and Industrial Projects 2024Chandresh Chudasama
 
Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...
Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...
Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...PRnews2
 
Onemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring CapabilitiesOnemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring CapabilitiesOne Monitar
 
Unveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic ExperiencesUnveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic ExperiencesDoe Paoro
 
Technical Leaders - Working with the Management Team
Technical Leaders - Working with the Management TeamTechnical Leaders - Working with the Management Team
Technical Leaders - Working with the Management TeamArik Fletcher
 
Driving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon HarmerDriving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon HarmerAggregage
 
Fundamentals Welcome and Inclusive DEIB
Fundamentals Welcome and Inclusive DEIBFundamentals Welcome and Inclusive DEIB
Fundamentals Welcome and Inclusive DEIBGregory DeShields
 

Último (20)

Pitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deckPitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deck
 
EUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exportersEUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exporters
 
Types of Cyberattacks - ASG I.T. Consulting.pdf
Types of Cyberattacks - ASG I.T. Consulting.pdfTypes of Cyberattacks - ASG I.T. Consulting.pdf
Types of Cyberattacks - ASG I.T. Consulting.pdf
 
Simplify Your Funding: Quick and Easy Business Loans
Simplify Your Funding: Quick and Easy Business LoansSimplify Your Funding: Quick and Easy Business Loans
Simplify Your Funding: Quick and Easy Business Loans
 
Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...
Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...
Strategic Project Finance Essentials: A Project Manager’s Guide to Financial ...
 
14680-51-4.pdf Good quality CAS Good quality CAS
14680-51-4.pdf Good quality CAS Good quality CAS14680-51-4.pdf Good quality CAS Good quality CAS
14680-51-4.pdf Good quality CAS Good quality CAS
 
Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...
Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...
Intermediate Accounting, Volume 2, 13th Canadian Edition by Donald E. Kieso t...
 
Paul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate ProfessionalPaul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate Professional
 
Excvation Safety for safety officers reference
Excvation Safety for safety officers referenceExcvation Safety for safety officers reference
Excvation Safety for safety officers reference
 
Customizable Contents Restoration Training
Customizable Contents Restoration TrainingCustomizable Contents Restoration Training
Customizable Contents Restoration Training
 
1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf
 
Rakhi sets symbolizing the bond of love.pptx
Rakhi sets symbolizing the bond of love.pptxRakhi sets symbolizing the bond of love.pptx
Rakhi sets symbolizing the bond of love.pptx
 
MEP Plans in Construction of Building and Industrial Projects 2024
MEP Plans in Construction of Building and Industrial Projects 2024MEP Plans in Construction of Building and Industrial Projects 2024
MEP Plans in Construction of Building and Industrial Projects 2024
 
Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...
Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...
Introducing the AI ShillText Generator A New Era for Cryptocurrency Marketing...
 
Onemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring CapabilitiesOnemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
 
Unveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic ExperiencesUnveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic Experiences
 
Technical Leaders - Working with the Management Team
Technical Leaders - Working with the Management TeamTechnical Leaders - Working with the Management Team
Technical Leaders - Working with the Management Team
 
Driving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon HarmerDriving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon Harmer
 
The Bizz Quiz-E-Summit-E-Cell-IITPatna.pptx
The Bizz Quiz-E-Summit-E-Cell-IITPatna.pptxThe Bizz Quiz-E-Summit-E-Cell-IITPatna.pptx
The Bizz Quiz-E-Summit-E-Cell-IITPatna.pptx
 
Fundamentals Welcome and Inclusive DEIB
Fundamentals Welcome and Inclusive DEIBFundamentals Welcome and Inclusive DEIB
Fundamentals Welcome and Inclusive DEIB
 

Model-driven engineering of multimodal user interfaces

  • 1. Model-Driven Engineering of User Interfaces for Multimodal Web Applications Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13. Design options for graphical web UIs Design options for graphical UIs Sub-task presentation Sub-task navigation Concretization of navigation & control Navigation concretization Control concretization
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35. UsiXML syntax of transformation rules NAC NAC RHS mappings
  • 36. Transformation rule NAC: LHS: RHS: Multimodal prompt Guidance for input Multimodal input Guidance for output Vocal feedback
  • 37.
  • 38.
  • 39.
  • 40.
  • 41.
  • 42. Thank you very much for your attention For more information and downloading, http://www.isys.ucl.ac.be/bchi http://www.usixml.org User Interface eXtensible Markup Language http://www.similar.cc European network on Multimodal UIs Special thanks to all members of the team!