Axa Assurance Maroc - Insurer Innovation Award 2024
Bio-inspired robotic touch
1. Artificial
Touch
L. Ascari
Artificial Touch
Introduction
Towards a new approach in prosthetics? The tactile
system
Modelling
L. Ascari Validation
Conclusions
and Future
HENESIS S.R.L. Options
References
Parma - February 22nd, 2012
—
All the activity described in the presentation has been
carried on while post-doc at
Scuola Superiore Sant’Anna, Pisa (I)
2. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Modelling
Bioinspiration Validation
Conclusions
2 The tactile system and Future
Options
Hardware
References
Software
3 Modelling
4 Validation
5 Conclusions and Future Options
3. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Modelling
Bioinspiration Validation
Conclusions
2 The tactile system and Future
Options
Hardware
References
Software
3 Modelling
4 Validation
5 Conclusions and Future Options
4. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Modelling
Bioinspiration Validation
Conclusions
2 The tactile system and Future
Options
Hardware
References
Software
3 Modelling
4 Validation
5 Conclusions and Future Options
5. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Modelling
Bioinspiration Validation
Conclusions
2 The tactile system and Future
Options
Hardware
References
Software
3 Modelling
4 Validation
5 Conclusions and Future Options
6. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Modelling
Bioinspiration Validation
Conclusions
2 The tactile system and Future
Options
Hardware
References
Software
3 Modelling
4 Validation
5 Conclusions and Future Options
7. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Touch in Robotics
Approach Prosthetics SoA
Approach
The pick and lift task The pick and lift task
Bioinspiration
Bioinspiration
The tactile
system
2 The tactile system Modelling
Hardware Validation
Software Conclusions
and Future
3 Modelling Options
References
4 Validation
5 Conclusions and Future Options
8. Touch in Robotics I
Artificial
Touch
L. Ascari
Introduction
Robots are now very complex and sophisticated systems. Touch in Robotics
Higher computational requirements. Prosthetics SoA
Approach
The pick and lift task
Automation robots: very high performing and reliable Bioinspiration
machines. The tactile
system
Outside the factory floor: limited interaction with humans, Modelling
specially in terms of autonomous behavior and of friendly Validation
HMIs1 , Conclusions
and Future
despite a huge market is expected to develop rapidly2 . Options
References
9. Touch in Robotics II
Artificial
Touch
L. Ascari
Tactile sensing can provide information about mechanical
properties such as compliance, friction, and mass. Introduction
Touch in Robotics
Knowledge of these parameters is essential if robots are to Prosthetics SoA
Approach
reliably handle unknown objects in unstructured The pick and lift task
Bioinspiration
environments. For interaction, localization of the stimulus
The tactile
is essential3 . system
Modelling
Validation
Conclusions
1 and Future
J. Ayers et al. Neurotechnology for biomimetic robots. MIT Press, Options
2002. References
2
WorldRobotics. World Robotics 2006. International Federation of
Robotics, Statistical Department, 2006. url:
http://www.worldrobotics-online.org/.
3
R. D. Howe. “Tactile sensing and control of robotic manipulation”. In:
Journal of Advanced Robotics 8 (1994), pp. 245–261.
10. For what?
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
Interaction The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Autonomy
Conclusions
and Future
Options
References
Locomotion
11. For what?
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
Interaction The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Autonomy
Conclusions
and Future
Options
References
Locomotion
12. For what?
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
Interaction The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Autonomy
Conclusions
and Future
Options
References
Locomotion
13. For what?
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
Interaction The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Autonomy
Conclusions
and Future
Options
References
Locomotion
14. SOA in robotic skins?
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
15. Open Issues
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
wiring Approach
The pick and lift task
robustness Bioinspiration
The tactile
stretchability system
Modelling
bandwidth
Validation
processing Conclusions
and Future
Options
References
16. Open Issues
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
wiring Approach
The pick and lift task
robustness Bioinspiration
The tactile
stretchability system
Modelling
bandwidth
Validation
processing Conclusions
and Future
Options
References
17. Open Issues
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
wiring Approach
The pick and lift task
robustness Bioinspiration
The tactile
stretchability system
Modelling
bandwidth
Validation
processing Conclusions
and Future
Options
References
18. Open Issues
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
wiring Approach
The pick and lift task
robustness Bioinspiration
The tactile
stretchability system
Modelling
bandwidth
Validation
processing Conclusions
and Future
Options
References
19. Open Issues
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
wiring Approach
The pick and lift task
robustness Bioinspiration
The tactile
stretchability system
Modelling
bandwidth
Validation
processing Conclusions
and Future
Options
References
20. Touch in Prosthetics - Commercial SoA
Artificial
Touch
More advanced: myoelectric control
L. Ascari
I-Limb Ultra from Touch Bionics
Introduction
Ultra from BeBionics Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
21. Touch in Prosthetics - Commercial SoA
Artificial
Touch
More advanced: myoelectric control
L. Ascari
I-Limb Ultra from Touch Bionics
Introduction
Ultra from BeBionics Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
Often refused by patients!
22. Touch in Prosthetics - Commercial SoA
Artificial
Touch
Classical prosthesis, cable actuated
L. Ascari
Otto bock grippers
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
23. Touch in Prosthetics - Commercial SoA
Artificial
Touch
Classical prosthesis, cable actuated
L. Ascari
Otto bock grippers
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
Not sensorized. Higher user acceptance. Why?
24. Contemporary prosthetics: directions and open
issues4
Artificial
Directions Touch
autonomous control of low level tasks L. Ascari
higher spatial resolution of the sensing system Introduction
Touch in Robotics
neural control (prototypes exist) Prosthetics SoA
Approach
feedback to the patient (preliminary results) The pick and lift task
Bioinspiration
The tactile
system
Open issues
Modelling
connection with tactile nerves Validation
dexterity Conclusions
and Future
sensitivity Options
References
CONTROL (myo-electrical vs neural)
feedback to the patient
4
R.G.E. Clement et al. “Bionic prosthetic hands: A review of present
technology and future aspirations”. In: The Surgeon 9.6 (12/2011),
25. Contemporary prosthetics: directions and open
issues4
Artificial
Directions Touch
autonomous control of low level tasks L. Ascari
higher spatial resolution of the sensing system Introduction
Touch in Robotics
neural control (prototypes exist) Prosthetics SoA
Approach
feedback to the patient (preliminary results) The pick and lift task
Bioinspiration
The tactile
system
Open issues
Modelling
connection with tactile nerves Validation
dexterity Conclusions
and Future
sensitivity Options
References
CONTROL (myo-electrical vs neural)
feedback to the patient
4
R.G.E. Clement et al. “Bionic prosthetic hands: A review of present
technology and future aspirations”. In: The Surgeon 9.6 (12/2011),
26. Basic questions
Artificial
Touch
Some fundamental questions
L. Ascari
What is the main issue with advanced prosthesis?
Introduction
Touch in Robotics
Prosthetics SoA
Is feedback to the user essential for this? Approach
The pick and lift task
Bioinspiration
“Solved” Issues The tactile
system
low level control with many signals (here) Modelling
parallel but portable processing (here) Validation
Conclusions
mechanics (single fingers, underactuation, . . . ) and Future
Options
References
27. Basic questions
Artificial
Touch
Some fundamental questions
L. Ascari
What is the main issue with advanced prosthesis? Object
Introduction
Slippage and Grasp force control Touch in Robotics
Prosthetics SoA
Is feedback to the user essential for this? Approach
The pick and lift task
Bioinspiration
“Solved” Issues The tactile
system
low level control with many signals (here) Modelling
parallel but portable processing (here) Validation
Conclusions
mechanics (single fingers, underactuation, . . . ) and Future
Options
References
28. Basic questions
Artificial
Touch
Some fundamental questions
L. Ascari
What is the main issue with advanced prosthesis? Object
Introduction
Slippage and Grasp force control Touch in Robotics
Prosthetics SoA
Is feedback to the user essential for this? No! Approach
The pick and lift task
Bioinspiration
“Solved” Issues The tactile
system
low level control with many signals (here) Modelling
parallel but portable processing (here) Validation
Conclusions
mechanics (single fingers, underactuation, . . . ) and Future
Options
References
29. Basic questions
Artificial
Touch
Some fundamental questions
L. Ascari
What is the main issue with advanced prosthesis? Object
Introduction
Slippage and Grasp force control Touch in Robotics
Prosthetics SoA
Is feedback to the user essential for this? No! Approach
The pick and lift task
Bioinspiration
“Solved” Issues The tactile
system
low level control with many signals (here) Modelling
parallel but portable processing (here) Validation
Conclusions
mechanics (single fingers, underactuation, . . . ) and Future
Options
References
30. Bio-inspired approach
Artificial
Touch
Why and to what extent?
L. Ascari
Ultimate model: man
Man Larger dimensions,
Introduction
Touch in Robotics
Infinite Complexity: higher densities Prosthetics SoA
sensors and processing Approach
The pick and lift task
Technological, Bioinspiration
wiring, processing The tactile
limitations system
Modelling
Model and
Simplification Principle Validation
Lower complexity validation Conclusions
Innovative approach
sensory systems and Future
Options
•Technology
References
•Processing
•Scalability
Star-nosed mole
31. Bio-inspired approach
Artificial
Touch
Why and to what extent?
L. Ascari
Ultimate model: man
Man Larger dimensions,
Introduction
Touch in Robotics
Infinite Complexity: higher densities Prosthetics SoA
sensors and processing Approach
The pick and lift task
Technological,
Bioinspiration
wiring, processing The tactile
limitations system
Touch sense Modelling
Model and
Simplification Principle Validation
Lower complexity validation Conclusions
Innovative approach and Future
sensory systems Options
•Technology
References
•Processing
•Scalability
Star-nosed mole
32. The human hand: tactile structure
Artificial
Touch
L. Ascari
Human hand touch Structure of the skin
Introduction
3 major groups of afferent Touch in Robotics
(tactile afferents, joint Prosthetics SoA
Approach
mechanoreceptors, spindles) The pick and lift task
Bioinspiration
The glabrous skin has 17.000 The tactile
system
tactile units
Modelling
4 main types of Validation
mechanoreceptors (Ruffini, Conclusions
and Future
Pacini, Merkel, Meissner) for Options
intensity, pressure, acceleration References
stimuli
33. The human hand: tactile structure
Artificial
Touch
L. Ascari
Human hand touch Structure of the skin
Introduction
3 major groups of afferent Touch in Robotics
(tactile afferents, joint Prosthetics SoA
Approach
mechanoreceptors, spindles) The pick and lift task
Bioinspiration
The glabrous skin has 17.000 The tactile
system
tactile units
Modelling
4 main types of Validation
mechanoreceptors (Ruffini, Conclusions
and Future
Pacini, Merkel, Meissner) for Options
intensity, pressure, acceleration References
stimuli
from Johansson and Westling (“Roles of glabrous skin receptors and
sensorimotor memory in automatic control of precision grip when
lifting rougher or more slippery objects”)
34. Sensors performance...
Artificial
Touch
... in engineering terms
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
35. The pick and lift task
Artificial
Two aspects are crucial for a stable grasp: Touch
L. Ascari
the ability of the HW/SW system to avoid object slip
Introduction
to control in real-time the grasping force. Touch in Robotics
Prosthetics SoA
Approach
Human physiology of the task The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
36. The pick and lift task
Artificial
Two aspects are crucial for a stable grasp: Touch
L. Ascari
the ability of the HW/SW system to avoid object slip
Introduction
to control in real-time the grasping force. Touch in Robotics
Prosthetics SoA
Approach
Human physiology of the task The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
37. On the need for feedback
Artificial
Touch
L. Ascari
Evidence Where?
Introduction
Johansson measured Touch in Robotics
50-60ms of reaction Prosthetics SoA
Approach
time The pick and lift task
Bioinspiration
incompatible with The tactile
system
propagation time to the
Modelling
motor cortex Validation
evidence of circuit Conclusions
and Future
closed at subcortical Options
level (olivo-cerebellar References
system and thalamus).
from Johansson and Westling (“Roles of glabrous skin receptors and
sensorimotor memory in automatic control of precision grip when
lifting rougher or more slippery objects”)
38. On the need for feedback
Artificial
Touch
L. Ascari
Evidence Where?
Introduction
Johansson measured Touch in Robotics
50-60ms of reaction Prosthetics SoA
Approach
time The pick and lift task
Bioinspiration
incompatible with The tactile
system
propagation time to the
Modelling
motor cortex Validation
evidence of circuit Conclusions
and Future
closed at subcortical Options
level (olivo-cerebellar References
system and thalamus).
from Johansson and Westling (“Roles of glabrous skin receptors and
sensorimotor memory in automatic control of precision grip when
lifting rougher or more slippery objects”)
39. On the need for feedback
Artificial
Touch
L. Ascari
Evidence Where?
Introduction
Johansson measured Touch in Robotics
50-60ms of reaction Prosthetics SoA
Approach
time The pick and lift task
Bioinspiration
incompatible with The tactile
system
propagation time to the
Modelling
motor cortex Validation
evidence of circuit Conclusions
and Future
closed at subcortical Options
level (olivo-cerebellar References
system and thalamus).
from Johansson and Westling (“Roles of glabrous skin receptors and
sensorimotor memory in automatic control of precision grip when
lifting rougher or more slippery objects”)
40. Biological vs Robotic worlds
Artificial
Touch
Do we have these limitations (signaling speed) in robots? L. Ascari
Man Introduction
Biological models for the Touch in Robotics
design of biomimetic robots Prosthetics SoA
Approach
The pick and lift task
Nerves
Brain Limbs Bioinspiration
The tactile
system
Interfacing
Bio and Modelling
Robotics
Validation
Robot
Conclusions
and Future
Options
• Robots as physical platforms
for validating biological models References
Artificial Electric Artificial
Brain wires limbs
3
41. Biological vs Robotic worlds
Artificial
Touch
Do we have these limitations (signaling speed) in robots? L. Ascari
Man Introduction
Biological models for the Touch in Robotics
design of biomimetic robots Prosthetics SoA
Approach
The pick and lift task
Nerves
Brain Limbs Bioinspiration
The tactile
system
Interfacing
Bio and Modelling
Robotics
Validation
Robot
Conclusions
and Future
Options
• Robots as physical platforms
for validating biological models References
Artificial Electric Artificial
Brain wires limbs
3
No, but other constraints exist. Ex: computational power
42. Biological vs Robotic worlds
Artificial
Touch
Do we have these limitations (signaling speed) in robots?
L. Ascari
Ultimate model: man Introduction
Man Larger dimensions, Touch in Robotics
Infinite Complexity: higher densities Prosthetics SoA
sensors and processing Approach
The pick and lift task
Technological, Bioinspiration
wiring, processing
The tactile
limitations
Touch sense system
Model and Modelling
Simplification Principle
Lower complexity validation Validation
Innovative approach
sensory systems Conclusions
•Technology and Future
Options
•Processing
References
•Scalability
Star-nosed mole
No, but other constraints exist. Ex: computational power
43. Biological vs Robotic worlds
Artificial
Touch
Do we have these limitations (signaling speed) in robots?
L. Ascari
Man Ultimate model: man
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Touch sense
Modelling
Validation
Lower complexity
Innovative approach sensory systems Conclusions
and Future
•Technological Options
•Processing
References
•Scalability
Star-nosed mole
No, but other constraints exist. Ex: computational power
44. Multidisciplinarity — The animal model (touch)
Artificial
Touch
L. Ascari
Condylura Cristata A nose to see / Eimer
Introduction
12 mobile appendages Touch in Robotics
covered with more than Prosthetics SoA
Approach
25.000 tactile receptors The pick and lift task
Bioinspiration
(Eimer organs) The tactile
system
Structure of the Eimer Modelling
organ: a sort of pillar with Validation
3 nervous terminations Conclusions
(for constant pressures, and Future
Options
vibrations, fine surface References
details);
foveated tactile vision.
from Catania and Kaas (“Somatosensory Fovea in the Star-Nosed
Mole: Behavioral Use of the Star in Relation to Innervation Patterns
45. Multidisciplinarity — The animal model (touch)
Artificial
Touch
L. Ascari
Condylura Cristata A nose to see / Eimer
Introduction
12 mobile appendages Touch in Robotics
covered with more than Prosthetics SoA
Approach
25.000 tactile receptors The pick and lift task
Bioinspiration
(Eimer organs) The tactile
system
Structure of the Eimer Modelling
organ: a sort of pillar with Validation
3 nervous terminations Conclusions
(for constant pressures, and Future
Options
vibrations, fine surface References
details);
foveated tactile vision.
from Catania and Kaas (“Somatosensory Fovea in the Star-Nosed
Mole: Behavioral Use of the Star in Relation to Innervation Patterns
46. Multidisciplinarity — The animal model (touch)
Artificial
Touch
L. Ascari
Condylura Cristata A nose to see / Eimer
Introduction
12 mobile appendages Touch in Robotics
covered with more than Prosthetics SoA
Approach
25.000 tactile receptors The pick and lift task
Bioinspiration
(Eimer organs) The tactile
system
Structure of the Eimer Modelling
organ: a sort of pillar with Validation
3 nervous terminations Conclusions
(for constant pressures, and Future
Options
vibrations, fine surface References
details);
foveated tactile vision.
from Catania and Kaas (“Somatosensory Fovea in the Star-Nosed
Mole: Behavioral Use of the Star in Relation to Innervation Patterns
47. Multidisciplinarity — The animal model (vision)
Artificial
Touch
L. Ascari
Honeybee Fixed yet good eye
Introduction
Non-mobile compound Touch in Robotics
eyes (ommatidia); Prosthetics SoA
Approach
The pick and lift task
3000-4000 facets each eye Bioinspiration
( = 64x64 pixel array); The tactile
system
spatial resolution = 1/60 Modelling
of the human eye; Validation
No distance information Conclusions
and Future
from stereo vision; Options
References
Center facets larger than
the peripheral sensors.
yet: high performance
48. Multidisciplinarity — The animal model (vision)
Artificial
Touch
L. Ascari
Honeybee Fixed yet good eye
Introduction
Non-mobile compound Touch in Robotics
eyes (ommatidia); Prosthetics SoA
Approach
The pick and lift task
3000-4000 facets each eye Bioinspiration
( = 64x64 pixel array); The tactile
system
spatial resolution = 1/60 Modelling
of the human eye; Validation
No distance information Conclusions
and Future
from stereo vision; Options
References
Center facets larger than
the peripheral sensors.
yet: high performance
49. Multidisciplinarity — The animal model (vision)
Artificial
Touch
L. Ascari
Honeybee Fixed yet good eye
Introduction
Non-mobile compound Touch in Robotics
eyes (ommatidia); Prosthetics SoA
Approach
The pick and lift task
3000-4000 facets each eye Bioinspiration
( = 64x64 pixel array); The tactile
system
spatial resolution = 1/60 Modelling
of the human eye; Validation
No distance information Conclusions
and Future
from stereo vision; Options
References
Center facets larger than
the peripheral sensors.
yet: high performance
50. Multidisciplinarity — The animal model (vision)
Artificial
Touch
L. Ascari
Honeybee Fixed yet good eye
Introduction
Non-mobile compound Touch in Robotics
eyes (ommatidia); Prosthetics SoA
Approach
The pick and lift task
3000-4000 facets each eye Bioinspiration
( = 64x64 pixel array); The tactile
system
spatial resolution = 1/60 Modelling
of the human eye; Validation
No distance information Conclusions
and Future
from stereo vision; Options
References
Center facets larger than
the peripheral sensors.
yet: high performance
51. Multidisciplinarity — The animal model (vision)
Artificial
Touch
L. Ascari
Honeybee Fixed yet good eye
Introduction
Non-mobile compound Touch in Robotics
eyes (ommatidia); Prosthetics SoA
Approach
The pick and lift task
3000-4000 facets each eye Bioinspiration
( = 64x64 pixel array); The tactile
system
spatial resolution = 1/60 Modelling
of the human eye; Validation
No distance information Conclusions
and Future
from stereo vision; Options
References
Center facets larger than
the peripheral sensors.
yet: high performance
52. Multidisciplinarity — The animal model (vision)
Artificial
Touch
L. Ascari
Honeybee Fixed yet good eye
Introduction
Non-mobile compound Touch in Robotics
eyes (ommatidia); Prosthetics SoA
Approach
The pick and lift task
3000-4000 facets each eye Bioinspiration
( = 64x64 pixel array); The tactile
system
spatial resolution = 1/60 Modelling
of the human eye; Validation
No distance information Conclusions
and Future
from stereo vision; Options
References
Center facets larger than
the peripheral sensors. optical flow balance
yet: high performance motion detection
(Flicker effect)
53. Multidisciplinarity — The computational model
Artificial
Touch
L. Ascari
Cellular non linear networks Parallel topological
Introduction
CNN is a massive parallel architecture Touch in Robotics
computing paradigm defined Prosthetics SoA
Approach
in discrete N-dimensional The pick and lift task
Bioinspiration
spaces. The tactile
system
A CNN is an N-dimensional
Modelling
regular array of elements Validation
(cells); Conclusions
and Future
Cells are multiple input-single Options
output analog processors, all References
described by one or just some
few parametric functionals.
from Chua and Roska (Cellular Neural Networks and Visual
Computing: Foundations and Applications)
54. Multidisciplinarity — The computational model
Artificial
Touch
L. Ascari
Cellular non linear networks Parallel topological
Introduction
CNN is a massive parallel architecture Touch in Robotics
computing paradigm defined Prosthetics SoA
Approach
in discrete N-dimensional The pick and lift task
Bioinspiration
spaces. The tactile
system
A CNN is an N-dimensional
Modelling
regular array of elements Validation
(cells); Conclusions
and Future
Cells are multiple input-single Options
output analog processors, all References
described by one or just some
few parametric functionals.
from Chua and Roska (Cellular Neural Networks and Visual
Computing: Foundations and Applications)
55. CNN characteristics I
Artificial
Touch
L. Ascari
Locality of the connections between the units: in fact the Introduction
Touch in Robotics
main difference between CNN and other Neural Networks Prosthetics SoA
Approach
paradigms is the fact that information are directly The pick and lift task
exchanged just between neighbouring units. Of course this Bioinspiration
The tactile
characteristic allows also to obtain global parallel system
processing. Modelling
Validation
A cell is characterized by an internal state variable,
Conclusions
sometimes not directly observable from outside the cell and Future
Options
itself;
References
More than one connection network can be present;
56. CNN characteristics II
Artificial
Touch
L. Ascari
A CNN dynamical system can operate both in continuous
Introduction
(CT-CNN) or discrete time (DT-CNN), with analogical Touch in Robotics
Prosthetics SoA
signals from different sources; Approach
The pick and lift task
CNN data and parameters are typically real values; Bioinspiration
The tactile
CNN operate typically with more than one iteration, i.e. system
they are recurrent networks; It is a Universal Machine Modelling
(CNN-UM); Validation
Conclusions
It offers Stored programmability; and Future
Options
a Hardware implementation exists. References
57. CNN core: the template
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
58. Template meaning
Artificial
Touch
L. Ascari
Introduction
Touch in Robotics
Prosthetics SoA
Approach
The pick and lift task
Bioinspiration
The tactile
system
Modelling
Validation
State-out
Conclusions
and Future
Options
References
in
59. Features of the ACE4K (16K) chip — 3TOps
Artificial
Touch
System Desktop PC, PC-104 industrial PC, Windows NT, 2000
L. Ascari
Bus PCI, 33 MHz, 32 bit data width;
Visual Microprocessor type ACE4k, 64x64 processor array Introduction
Grayscale image download (64x64) 2688 frame/sec 372 !s Touch in Robotics
Prosthetics SoA
Grayscale image readback (64x64) 3536 frame/sec (compensated through look-up table); 283!s
Approach
Binary image download (64x64) 44014 frame/sec; 22.72 !s The pick and lift task
Bioinspiration
Binary image readback (64x64) 23937frame/sec; 41.78 !s
Array operation (64x64) 9 !s + N*100ns The tactile
Logical operation (64x64) 3.8 !s
system
DSP type Texas TMS320C6202; 250MHz, 1600 MIPS operation Modelling
Memory 16MB, SDRAM 125 MHz; 2Mbyte FLASH (bootable)
Validation
Serial Ports 3
Other features Watch Dog, Timer
Conclusions
and Future
Options
Programmability C language, native languages References
Image processing library Several image processing functions optimized for CVM
Application Program Interface (API) Integrate the Aladdin systerm into different environments
60. Recall
Artificial
Touch
FINAL GOAL COMPUTATIONAL PLATFORM
L. Ascari
Introduction
ROBOTIC Touch in Robotics
PLATFORM Prosthetics SoA
Approach
The pick and lift task
Tactile Bioinspiration
system SW
The tactile
system
Modelling
TASK
CONTROLLER Validation
Tactile
system HW Conclusions
and Future
Options
References
TASK,
PHYSIOLOGICAL
STRATEGY
61. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Hardware
Software
Bioinspiration Modelling
Validation
2 The tactile system
Conclusions
Hardware and Future
Options
Software
References
3 Modelling
4 Validation
5 Conclusions and Future Options
62. The MEMS mechanoreceptor
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Rpu
Vc
Modelling
R1 R2 Validation
V13 V24
Conclusions
R3 R4 and Future
Options
References
0
63. The array — Fabrication steps
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Modelling
Validation
Conclusions
and Future
Options
References
64. The whole system — HW
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Modelling
Validation
Conclusions
and Future
Options
References
From L Ascari et al. “A miniaturized and flexible optoelectronic
sensing system for tactile skin”. In: Journal of Micromechanics
and Microengineering 17.11 (11/2007), pp. 2288–2298. issn:
0960-1317. doi: 10.1088/0960-1317/17/11/016. url:
http://ejournals.ebsco.com/direct.asp?ArticleID=
4A9A98E0B7D16F0C429C
65. The whole system — from HW to SW
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Modelling
Validation
Conclusions
and Future
Options
References
From L. Ascari et al. “Bio-inspired grasp control in a robotic
hand with massive sensorial input”. In: Biological Cybernetics
100.2 (2009), p. 109. doi: 10.1007/s00422-008-0279-0
66. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
67. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
68. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
69. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
70. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
71. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
Where is information? What kind of spatial and temporal
patterns? How to recognize and prevent slippage?
72. Recap
Artificial
Touch
L. Ascari
We have an array of analog multidirectional tactile signals
Introduction
The load cell were NOT calibrated: qualitative and only
The tactile
loose orthogonality system
Hardware
we can load and process analog tactile images on the CNN Software
Modelling
chip at 400 Hz
Validation
54 sensors wrapped around the thumb and index fingers of Conclusions
a robotic underactuated hand and Future
Options
robotic arm controlled by DSP References
Where is information? What kind of spatial and temporal
patterns? How to recognize and prevent slippage?
We need to learn the tactile “alphabet”
73. The task controller — FSM
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Modelling
Validation
Conclusions
and Future
Options
References
74. The task controller — FSM
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Modelling
Validation
Conclusions
and Future
Options
References
75. The task controller — Features
Artificial
Touch
L. Ascari
Introduction
The tactile
system
Hardware
Software
Modelling
Validation
Conclusions
and Future
Options
References
76. Outline
Artificial
Touch
1 Introduction
L. Ascari
Touch in Robotics
Touch in Prosthetics - Commercial SoA Introduction
Approach The tactile
system
The pick and lift task Modelling
Bioinspiration Validation
Conclusions
2 The tactile system and Future
Options
Hardware
References
Software
3 Modelling
4 Validation
5 Conclusions and Future Options
77. The slip effect in robotic grasp
Artificial
Touch
Slip as vibrations. “Catch and snap” effect on the rubber
L. Ascari
(60Hz stable + initial 10Hz component). Recall FAII human
mechanoreceptors. Introduction
The tactile
system
Modelling
Validation
Conclusions
and Future
Options
References
Holweg et al., “Slip detection by tactile sensors: algorithms and
experimental results”
78. Definition of Tactile Events of interest
Artificial
Touch
L. Ascari
Variations, oscillations, vibrations
Introduction
Time is divided in periods of
The tactile
duration T ∗ s system
Modelling
Variation change in signal
Validation
larger than σ in
Conclusions
same period and Future
Options
Oscillation seq. of 2 subsequent References
variations of opposite
sign in same T ∗ .
(m,n)
Vibration seq. of 2 oscillations
in 2 adjacent periods σ = 2% dynamic range
79. Definition of Tactile Events of interest
Artificial
Touch
L. Ascari
Variations, oscillations, vibrations
Introduction
Time is divided in periods of
The tactile
duration T ∗ s system
Modelling
Variation change in signal
Validation
larger than σ in
Conclusions
same period and Future
Options
Oscillation seq. of 2 subsequent References
variations of opposite
sign in same T ∗ .
(m,n)
Vibration seq. of 2 oscillations
in 2 adjacent periods σ = 2% dynamic range
80. Definition of Tactile Events of interest
Artificial
Touch
L. Ascari
Variations, oscillations, vibrations
Introduction
Time is divided in periods of
The tactile
duration T ∗ s system
Modelling
Variation change in signal
Validation
larger than σ in
Conclusions
same period and Future
Options
Oscillation seq. of 2 subsequent References
variations of opposite
sign in same T ∗ .
(m,n)
Vibration seq. of 2 oscillations
in 2 adjacent periods σ = 2% dynamic range