2. Content
General overview
The project in one sentence
Overview of all research activities
Common user scenario
Other application domains
Technical demo set-up
2
3. General overview
Virtual Individual Networks (VIN)
Interdisciplinary Strategic Basic Research (ISBR),
Long-term interdisciplinary pre-competitive research
100% funded by IBBT.
Project duration:
01.09.2005 - 31. 08.2008
3
4. Multidisciplinary team
Different research groups are involved:
CUO - K.U.Leuven
SMIT - VUB
MICT - UGent
ICRI - K.U.Leuven
COSIC - K.U.Leuven
UGent - IBCN
EDM - UHasselt
ETRO - VUB
NES - IMEC
4
5. Project summary
Research into the use of computer-mediated
audiovisual communication, in order to
enrich the participation within open and
closed communities.
5
6. Research overview
COMPUTER-MEDIATED
AUDIOVISUAL COMMUNICATION
3D viewpoint
Video compression
AV emotion AV encryption
analysis
VIRTUAL
INDIVIDUAL
NETWORK
Content regulation Key management
Sociability guidelines Automatic data capturing
Secure network overlays
Online/offline
communities
Social networks
Social translucence
Social capital
Privacy
PARTICIPATION OPEN AND CLOSED
ENRICHMENT COMMUNITIES 6
7. Approach and collaboration within VIN
Individual research goals to allow basic research in
different domains
User
Legal
Technological
One common user scenario in which all research groups
participate:
to assure the interaction between different research
goals,
to attain common demonstrators and research results
that can be further investigated in GBO projects.
7
8. Common user scenario 1/4
PARTICIPATION
ENRICHMENT
OPEN AND
Erica, a passionate dancer of salsa and tango, is not
CLOSED only a member of a dance-school, but also active in
NETWORKS a number of online communities on Latin American Content regulation
dances. Online / offline communities
COMPUTER-
MEDIATED
AUDIOVISUAL
COMMUNICATION
Tonight Erica’s daughter is feeling ill and therefore
she decides not to go to the dancing lesson tonight
but to stay home in case her daughter needs her.
She puts her daughter early to bed and contacts her AV emotion analysis
dancing teacher in the dancing school via interactive
television. The dancing teacher receives an
indication on his screen that the person that is calling
is Erica and that she’s feeling sad.
Erica tells her teacher that she won’t be able to make
it to the lesson and that she will try to attend the Sociability through IDTV
class virtually by means of interactive television.
Viewpoint interpolation between different camera-
inputs makes this conversation with her teacher 3D viewpoint
pleasant since they can keep eye contact. Video compression
8
9. Common user scenario 2/4
PARTICIPATION
ENRICHMENT
OPEN AND
She also informed her best friend with her mobile
CLOSED device that she will not be at the dance lesson and
NETWORKS asks her if they could meet afterwards in the online
community to exchange experiences. Online/offline communities
COMPUTER- Online/offline sociability
MEDIATED
AUDIOVISUAL
COMMUNICATION
The dancing teacher agrees and tells Erica to
configure her webcams. Erica tells the system that
only her appearance may be filmed and that the Object tracking
system should block out the background.
This means that her classmates will see her dance
and her teacher will be able to give feedback on her Social networks, social
dance moves, but they will not see the private capital and social
environment in which Erica is practising. translucence
On the spur of the moment, Erica decides to allow
her husband John, who is working late, full access
to her camera-feed. Security network overlays
9
10. Common user scenario 3/4
PARTICIPATION
ENRICHMENT
OPEN AND
During class time, Erica changes viewpoints several
CLOSED times in order to experience the dancing lesson
NETWORKS and choreography in the best way possible. UI usability & sociability
Erica’s teacher who can see only Erica and has no
COMPUTER- authority to see the rest of the picture (i.e. the room
MEDIATED
AUDIOVISUAL in which Erica is dancing) congratulates Erica on her
COMMUNICATION progress.
John’s meeting is finished and takes the bus back Automatic data capturing
home. The network automatically detects that John
is on the move.
While he commutes between office and home he
decides to see how his wife is doing in the dancing
lesson. While he is on the move, a ‘universal Personal networks
communication layer’ provides automatic network
connection detection for his PDA.
Riding the bus, John watches his wife dancing in Distributed source coding
their messy living room and makes a mental note Interface adaptation
that they really should start looking for a cleaning
woman.
10
11. Common user scenario 4/4
PARTICIPATION
ENRICHMENT
OPEN AND
When John arrives home, he tells Erica that he
CLOSED really liked one of the songs she was dancing to.
NETWORKS So, after the lesson Erica asks her teacher if she
could have this particular piece of music. Erica’s Secure networks
COMPUTER- teacher, who believes that she has the right to
MEDIATED
AUDIOVISUAL share this material because it serves educational
COMMUNICATION purposes, replies that she will put the data on the
social software service or the private peer-to-
peer network used by the dancing school for
educational or at the online community of Offline/online communities
which she’s a member. Online/offline sociability
The teacher also invites Erica to check her blog
on which she will post a few tips and tricks Content regulation and legal
regarding the dance that they have been status of virtual individuals
practising.
11
12. Other application domains
Improve safety and security monitoring
in freely accessible locations like train and metro stations
Video Coding, Secure Networking and Encryption
safely send surveillance images over a network to the central monitoring location (not
necessarily close to the monitored building).
Key Management
temporary direct access to images can be given to the police or fire department in the
case of an emergency.
3D viewpoint interpolation
gives access to a virtual “flying camera” that can be navigated freely around the scene,
and allows a better assessment of the given situation by the security personnel
Microphone Array Technology
automatic spatial localization of sounds that may indicate a potentially dangerous
situation.
pick up sounds that come from these locations for further (automatic) analysis and
evaluation.
Audiovisual Emotion Analysis
automatic assessment of the emotional state of a single person (is (s)he angry or afraid)
or even a group of persons (e.g. a happy group of partygoers, or an angry group of
hooligans). Based on this assessment, the system can alert the security personnel to
potentially dangerous situations.
12
13. Other application domains
Improve efficiency and communication at work
A construction supervisor on the road would like to confer with the architects at the office
Video Coding, Secure Networking and Encryption
Safely send confidential data over a network to the central offices
Key Management
Only the construction supervisor and the architects at the office can get access to
the images
3D viewpoint interpolation
Allows the architects to have a excellent view of the situation in the field, looking
at the location from different angles
The supervisor and the architects can look each other directly in the eyes,
improving communication
Sociability
Offering optimized interaction functionalities like voice communication and virtual
drawing boards that enhance communication between the supervisor and the
architects
13
14. Erica
Technical set-up @ Home
ROI of Erica
Husband
Teacher
@ Mobile
@ PC (On the move)
Bluetooth
Eye gazing solved
WLAN
- Eye gazing (viewpoint interpolation) http://www.oxygen.lcs.mit.edu/H21.html
- Display/Interface adaptation
- Virtual Private Ad Hoc Network
- DSC on mobile (complexity issue)
- Transcoding from DSC to MPEG-4
3D display
- Region of Interest (ROI) tracking
- Security
Husband sees Erica dancing in living room,
Teacher sees Erica dancing without background
Stereo cameras
14
