Remzi Ateş Gürşimşek
Public defense of research for the degree of Doctor of Philosophy
Department of Communication, Business and Information Technologies
Roskilde University, Denmark 2012

1. Introduction
2. Theoretical framework
Virtual worlds, social semiotics, design research
3. Methodology and Analytical Framework
Case-studies, data production, analysis
4. The Analysis
Social contexts and multimodal semiotics
5. Conclusions and Discussions
Implications, limitations, further directions

Second Life provides its users with
virtual places and artifacts as a virtual world.

BUT..
How can the world’s residents communicate by designing?
What do these
(Mediated) (Representation of)
• 3D Presence Space
(Avatar-based) (Sense of)
• Social Co-presence Place
(Content) (Design)
• User-generated Co-production Tools
virtual worlds mean to their makers?

BUT..
How can the world’s residents communicate by designing?
What do these
(Mediated) (Representation of)
• 3D Presence Space
(Avatar-based) (Sense of)
• Social Co-presence Place
(Content) (Design)
• User-generated Co-production
SO BEGAN MY QUEST FORTools
virtual worlds mean to their makers?

VIRTUAL
WORLDS
SOCIAL DESIGN
SEMIOTICS RESEARCH

Virtual Worlds as
Places
Avatars, Presence & Co-design and
Co-presence VIRTUAL Co-creation in VWs
WORLDS
SOCIAL DESIGN
SEMIOTICS RESEARCH

VIRTUAL
Semiotic resources WORLDS
Context of Situation
Meaning Potentials
SOCIAL DESIGN
SEMIOTICS RESEARCH
Multimodality and Mediated Discourse
Multimodal Discourse Analysis

VIRTUAL
WORLDS
Affordances and
constraints in design
SOCIAL DESIGN
SEMIOTICS RESEARCH
The (collaborative) Designerly ways of
design process knowing and doing

How do the VW users co-produce
multimodal meaning potentials in
virtual places and artifacts through
collaborative design, as exemplified by
the social semiotic analysis of the three
case studies in SL?

 Avatars, presence and co-presence
(Bartle 2004, Gordon 2008, Damer 1998, 2009,
Schroeder 1995, 2006, 2011)
 Spatial practices and space/place models in
design of collaborative virtual systems
(Dourish 2006, Schroeder 2011).
 Spatial technologies of representation,
verbal and non-verbal communication
(Ondrejka 2005, Boellstoff 2008, Damer 1998, 2009)
 Place metaphors, and representation of
sense of place in VWs
(Bartle 2004, Prasolova-Førland 2008, Taylor 2009).

 Phenomenology and the experiential perspective:
Place as ”organized world of meaning”
(Tuan 1977, Turner and Turner 2004)
 Space and place are built on social relations which
frame interactive behavior in collaborative virtual
environments
(Harrison and Dourish 1996)
 Design of a virtual place as architectural problem
(Bridges and Charitos 1997)
 Architectural space and cognitive/perceptual and
physical characteristics of sense of place in built
space
(Rasmussen 1959, O’Toole 1994, 2004, Najafi et al 2011)

 Social semiotics focuses on sign systems and semiotic
resources in particular contexts of situation, through
which they present various semiotic potentials for social
interaction (Hodge & Kress 1988, van Leeuwen 2005).
 Semiotic Resources:
systems of meaning at people’s disposal
 Context of Situation:
social interactions and relations in specific contexts
in which modes are shaped
 Meaning Potentials:
resourcefulness and aptness of the modes in
relation to the semiotic requirements of the
context of situation.

 Mode: socially and culturally shaped semiotic
resources for meaning-making
 Multimodal cohesion: orchestration of sign
systems as multimodal arrangements to present
meaning potentials
 The critique of multimodal approach says:
• Modes are neither pre-determined set of rules nor they can be
framed by the 5 senses (Pink 2011)
• Nature of the social is dynamic and problematic (Latour 2005)
• Power, social roles and agency defines the post-modernist view
of signs (Poynton 1993)

 Design translates rhetor’s intentions into
semiotically shaped material
 Discourse shapes the choice of design, but
choice of design in turn shapes the discourse
(Kress 2010)
 Design = de + signare
(Krippendorff 1998)
 Design as a basic human activity to shape and
bring relational order to the lived environment
(Papanek 1984, Buchanan 2001a, Heskett 2002)
 Affordances as meaning and action potentials
that are co-contructed through interactions of
people and objects
(McGenere and Ho 2000, Chemero 2003, Norman 2008)

 Design as a ”a reflective conversation with the situation”
(Cross 2007 [1990])
 Role of intuition, emotion and aesthetic judgement in design decisions
(Simonsen et al 2010)
 ”lll-defined” or ”wicked” problems of design thinking
(Cross 2007 [1982], Buchanan 1992)
 The problem and the solution in design as ”mutually constructive”,
”emergent” notions
(Wisser 2006)

 ”Individual insights” and ”momentary collective
processes”
(Hargadon and Becky 2006)
 Emergence of innovative solutions or unexpected
outcomes as dialogic process
(Grossen 2008, 2010)
 A ”problem-processing activity” with complex
problems and distributed knowledge
(Arias et al. 2000, Julier 2008)
 ”Thinking together apart” and”objects to think
with” in distributed design environments
(Larsson 2003)

 Co-presence and connected presence in computer-supported
collaborative design
(Schroeder 2011, Koutsabatsis et al. 2012)
 Physically remote but virtually co-located avatars in virtual design
studios turn space to inhabitated place
(Maher and Simoff 2000, Reffat et al. 2008, Weiley and Pisan 2008, Gu et al. 2011)
 The social creativity of residents as user-generated content , co-creation ,
and user-driven innovation and as exploitation of voluntary labor
(Ondrejka 2005, Pearce 2006, Fischer and Giaccardi 2007, Bonsu and Darmody 2008,
Kohler et al. 2009)

Virtual words
Avatars, co-presence and collaborative place-making in a 3D virtual studio
environment with distributed actors
Social semiotics and multimodality
Social construction of meaning potentials by using modes as semiotic
resources in a particular context
Design research
Designing affordances for both meaningful and functional places as a
communicative practice

 Virtual places and artifacts in SL as multimodal data
 Participatory design observation
 Ethnographic observations with co-designers
 Interviews and focus groups
Images of 3D environments and video captures of designer interactions in SL, interview
transcripts , field notes and online archives

• The three meta-functions in multimodal theory of
space and place (Stenglin 2008, 2009, O’Toole 1994,
2004) describe the:
 Experiential
 Interpersonal
 Textual
meaning potentials..
Nathan Winters, Architecture Is Elementary, Salt Lake City, UT: Peregrine Smith Books, 1986

The “nexus analysis” or MDA perspective to social semiotics considers:
The social actors and relations
The mediational means and affordances
The collaborative design processes
Multimodal semiotics in places and artifacts
As analytical units..
Multimodal Analysis and Mediated Discourse Analysis
Analyzing both the products and the social contexts of co-design

 Capturing power and agency with multimodal analysis
 Interpreting the analysis of contexts and contents of
design through multiple viewpoints on meaning
 Generalizations from case study findings
 Questions about identity, truthfulness and participation

 A design team and a  An organic ally formed  Four design teams, 15
research team group as design team students
 Professional and  Team and hierarchy  Educational context and
amateur participants based on inworld student responsibilities
 A professional contract friendship  Amateur and first-timer
 Roles and relations  Professional and  Roles and relations
defined amateur participants evolved by learning and
 Decision –making based  Roles and relations interest
on research priorities evolved over time

 Backgrounds, skills and purposes of the co-
designers (i.e. Experience, motivation)
 Professionalism, the amateur spirit and
being a ”builder” in SL (i.e. Outsourcing)
 Individual participations and the social
context (i.e. Access to resources)
 Power relations, organization, decision-
making and hierarchy in group formation
(i.e. Being the ”chief” of design)

 Movements of humans and avatars in
real and virtual places, as well as online
and offline media (i.e. Google, Blender,
Paper)
 Affordances and contraints of Second
Life determine interactions with both:
 The user-interface and design tools
 Social affordances of the world
 The socio-technical environment of
semiotic resources shapes not only the
processes but also the products of design
(i.e. Prim-based thinking)

Second Life Collaboration
Tools
(Co-)design
External
MEDIATIONAL Avatars 3D Objects
MEANS
Material Scripts, textures
Resources
Inspiration
Form
Visual language

 Division of tasks and collaborative
place-making in groups (i.e. Seats)
 Heterochronous, real-time or
asynchronous, collaboration (i.e. The
workshop groups)
 Collective practices or working
alone (i.e. Costume bazaar and the
‘pixel landscape’)

 Both inworld and offline  Mostly inworld  Both inworld and offline
processes processes processes
 Both real-time and  Both real-time and  Both real-time and
asynchronous working asynchronous working asynchronous working
 Tasks are divided or  Tasks are divided as  Tasks are divided by
outsourced parts of design interest and skill
 In another sim for  Metrotopia’s Sandbox
 On a workplace 900mt
building
above Metrotopia  Via various other online
 Via e-mail and Google’s media, including e-mail
 Via e-mail and file collaboration platform and MSN Messenger
transfer

Rank-scale/ Function EXPERIENTIAL INTERPERSONAL TEXTUAL
Practical Functions:
Genre and Style
(research/education/content-
Metaphoricity Modes as textual elements
sharing/community-building)
Social Presence Textures and colors
Imagined use-contexts
SIM AS Size and Ratio (Avatars in Virtual Space) Light and shadows
Public/Private Spaces
Direction: Verticality Opacity/Visibility
VIRTUAL PLACE Entry Point
Mood Rhythm and contrast in shapes and forms –
Map/Layout
(Building) Modality Division/organization of space
Flow of Movement
Color Open/closed spaces
(Open or Closed spaces)
Cues for Navigation/Orientation Field of Vision / Relation to environment Relation to neighbors
Functional and non-functional areas
Visibility Intertextuality (reference/mimicry/contrast)
Orientation
Height
ELEVATIONS & Sub-functions Elevations in relation to each other
Spaciousness
Access Elevations in relation to ground-plane
DIVISIONS Materials/Texture
Functional elevations Color and texture
Separation of Groups (Privacy) Visibility / Accessibility
(Floor) Movements between levels Degree and permanence of spatial Organization
Visual Links between floors/areas
Setting/layout
Functions of specific places
INTERACTION Modality Scale
Types of interaction (user-object, user-
Color Open/closed spaces
SPACES user)
Foregrounding of functions Orientation Relation to outside and other spaces
Type of places (activity, community or
(Room) Social Presence Spatial connectors
content oriented)
Focus of attention
Style
Interactivity
Modality Structural elements (prims)
Functions of virtual objects:
ARTIFACTS Color Textures/maps
Providing information
(Element) Relevance to environment Scripts and Animations
Social interaction
Affordances Positioning
Decoration
Stylistic coherence
INFORMATION Interactivity Modality
Functions of surfaces Relevance Information as texture
SURFACES
Types of information: (still image, Affordances Placement of surfaces Interaction with surfaces

 The analysis of Experiential meaning
potentials focuses on presence, movement
and practical functionality in virtual places.
METROTOPIA PAL WORKSHOP
Research and A social news Narrative
teaching purposes channel for music construction
Design hasn’t
Design concept and Design has changed
changed after
function defined throught the years
submission
Sim designed as a Sim designed as Small-scale objects
city with interactive open space with and interactive
buildings music zones areas
Visitors are guided
Sim divided into
by the orange ---
regions by streets
pathway

 The analysis of Interpersonal meaning
potentials focuses on the visual language
and representation of social presence.
METROTOPIA PAL WORKSHOP
Superhero genre Based on
”SL-like”
conventions superheroines
Tropes in an urban
metropolitan ”Pixel-landscape” Mixture of narrative
landscape
Consistency with Visualizing music Mythologies and
movies and games and rhythm popular culture
Sandbox and Fight Seating areas, the
The dance floor
Club Dojo club and live stage

 The analysis of Textual meaning potentials
focuses on structure, organization and
multimodal orchestration in design.
METROTOPIA PAL WORKSHOP
Divided into streets, Divided by zones
Elevated design
elevated by and pathways,
elements
buildings elevated ”Sl-like”
Prims and texture
Prims and textures Prims and textures
based modular
to signify buildings for signification
concept
Logo, notecards, Logo, notecards,
posters, maps, posters, maps, Logo and posters
teleportation teleportation
Interactivity Sound Metaphor

3D representations Users as co-designers of SL
Design and
of space, and build
experiential together
semiotics of place
How do the VW users co-produce
In particular
multimodal meaning potentials in contexts, by
using the
virtual places and artifacts through semiotic
resources
collaborative design, as exemplified by
the social semiotic analysis of the three
case studies in SL? As design
In a virtual elements,
design including
studio, As a user-driven platform for objects,
by using co-design of interconnected textures and
avatars virtual places. scripts.

 Analyzing virtual worlds as places
Meaning potentials in virtual places are defined by not only visual
characteristics, but also experiential potentials and affordances for avatar
interaction.

 Analyzing virtual worlds as places
 Analysis of multimodal design with nexus analysis framework
Meaning as multimodal texts places are defined by not onlyaction
Places potentials in virtual + place-making as mediated visual
characteristics, but also experiential potentials and affordances for avatar
interaction.
Content Context + relations

 Analyzing virtual worlds as places
 Analysis of multimodal design with nexus analysis framework
 Design as construction of a sense of place
Meaning as multimodal texts places are defined by not onlyaction
Places potentials in virtual + place-making as mediated visual
characteristics, but also experiential potentials and affordances for avatar
Designers use the available material and semiotic resources in SL to signify
interaction.
mediated presence in three dimensional space and co-presence in virtual places.
Content Context + relations

 Analyzing virtual worlds as places
 Analysis of multimodal design with nexus analysis framework
 Design as construction of a sense of place
 Affordances in virtual places and artifacts
Meaning as multimodal texts places are defined by not onlyaction
Places potentials in virtual + place-making as mediated visual
characteristics, but also experiential potentials and affordances for avatar
Designers use the available materialform semiotic resources in SL experience.
Design in SL orchestrates meaning, and and function for avatar to signify
interaction.
mediated presence the socio-technical environmentco-presence in virtual places.
in in three dimensional space and facilitates co-production of
AffordancesContent Context + relations
virtual places as multimodal discourses.

 Implications for platform and content developers
 Affordances for multimodal design
 Collaboration is not always “working together”
 Allow movement between media
 Expect various actors and contexts
 Balance ‘complexity of tools’ vs. ‘freedom to create’
 Know that limitations are open-ended, give people some space.

 Implications for platform and content developers
 Implications for designers
 Affordances for multimodal design
 Collaboration is not always “working together”
 Real-time testing and evaluation of ideas
 Allow movement between media
 Design by walking around
 Expect various actors and contexts
 Social affordances
 Balance ‘complexity of tools’ vs. ‘freedom to create’
 Potentials for participatory design and hands-on learning
 Know that limitations are open-ended, give people some space.
 Professional limitations and the learning curve

 New technologies
 Emerging virtual world platforms
 VR and mobile augmented reality
 3D content on the Web
 New social contexts
 Semiotic analysis of visitor experience
 Specific user groups, i.e. professional designers/architects
 New methods and perspectives
 Participatory design and user-driven innovation