In this deck from the Stanford HPC Conference, Nicole Xu from Stanford University describes how she transformed a common jellyfish into a bionic creature that is part animal and part machine.
"Animal locomotion and bioinspiration have the potential to expand the performance capabilities of robots, but current implementations are limited. Mechanical soft robots leverage engineered materials and are highly controllable, but these biomimetic robots consume more power than corresponding animal counterparts. Biological soft robots from a bottom-up approach offer advantages such as speed and controllability but are limited to survival in cell media. Instead, biohybrid robots that comprise live animals and self- contained microelectronic systems leverage the animals’ own metabolism to reduce power constraints and body as an natural scaffold with damage tolerance. We demonstrate that by integrating onboard microelectronics into live jellyfish, we can enhance propulsion up to threefold, using only 10 mW of external power input to the microelectronics and at only a twofold increase in cost of transport to the animal. This robotic system uses 10 to 1000 times less external power per mass than existing swimming robots in literature and can be used in future applications for ocean monitoring to track environmental changes."
Watch the video: https://youtu.be/HrmJFyvInj8
Learn more: https://sanfrancisco.cbslocal.com/2020/02/05/stanford-research-project-common-jellyfish-bionic-sea-creatures/
and
http://www.hpcadvisorycouncil.com/events/2020/stanford-workshop/
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Biohybrid Robotic Jellyfish for Future Applications in Ocean Monitoring
1. Biohybrid robotic jellyfish for future
applications in ocean monitoring
Nicole W. Xu, James Townsend, Jack Costello,
Sean Colin, John O. Dabiri
Stanford University
HPC-AI Stanford Conference
April 21, 2020
2. 2
Nicole W. Xu
Stanford University
April 21, 2020
How much do we know about the ocean?
More than 80% of our ocean is unmapped,
unobserved, and unexplored.
3. 3
Importance of ocean monitoring
Track changes in temperature, acidity, concentrations of nutrients
Discovery of new species or behaviors
National GeographicMBARI
Toby Hudson Greg Torda
Nudibranch
Coral Bleaching
Algal Blooms
Nicole W. Xu
Stanford University
April 21, 2020
4. 4
Current methods of ocean monitoring
Remotely operated vehicles (ROVs) Autonomous underwater vehicles (AUVs)
Slocum Glider, AUVACRJE International
Challenges:
Costly
Bulky
High power consumption
Vehicle wakes
Yoda Productions
Nicole W. Xu
Stanford University
April 21, 2020
5. 5
How can we expand ocean monitoring tools?
Nicole W. Xu
Stanford University
April 21, 2020
Goal: Use technology to collect data about ocean environments
Needs
Able to explore new locations
Minimal power
Minimal cost
Minimal disturbance to aquatic life
How we can achieve this: Combine technological advances,
experimental work, and computational tools
National Geographic
6. 6
How can we expand ocean monitoring tools?
Nicole W. Xu
Stanford University
April 21, 2020
Goal: Use technology to collect data about ocean environments
Needs
Able to explore new locations
Minimal power
Minimal cost
Minimal disturbance to aquatic life
How we can achieve this: Combine technological advances,
experimental work, and computational tools
National Geographic
7. 7
Bioinspired robots offer advantages
Energy-efficient, bioinspired aquatic vehicles that leave natural
wakes to minimize environmental disturbances
Collect images, data, samples from new locations
Tadesse et al., Smart Materials and Structures, 2012Katzschmann et al., Science Robotics, 2018 MantaDroid
Fish Manta ray Jellyfish
Nicole W. Xu
Stanford University
April 21, 2020
Which animal model do we choose?
8. 8
Jellyfish as a model organism
Nicole W. Xu
Stanford University
Despite 500 million years of evolutionary pressure, the body
structure of moon jellyfish has remained largely unchanged.
Aurelia aurita
• Simple body structure
• Swimming is tied
into feeding, escape
behaviors, etc.
• Ubiquitous
5 cm
April 21, 2020
9. Nicole W. Xu
Stanford University
9
COT is a metric of energy efficiency
Opposite of mpg
COT =
Energy
Mass x Distance
April 21, 2020
Tesla
10. Nicole W. Xu
Stanford University
10
What can we learn from jellyfish?
Jellyfish have a low
cost of transport
compared to other
animals.
Jellyfish are energy
efficient.
Gemmell et al., PNAS, 2013
Flying
Running
Swimming
COT =
Energy
Mass x Distance
April 21, 2020
11. Spectrum of bioinspired jellyfish constructs
Nicole W. Xu
Stanford University
11
Nawroth et al, Nature Biotech, 2012
Biological
Construct
Limitation:
Survival in a
few specific
environments
Villanueva et al, PLoS ONE, 2014
Robotic
Construct
Limitation:
High power
consumption
April 21, 2020
13. 13
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
14. 14
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
15. Nicole W. Xu
Stanford University
Pacemakers: sensory organs that innervate the swim muscle
exumbrellar surface
(top)
subumbrellar surface
(underside)
5 cm
5 cm
15
Swim anatomy
April 21, 2020
16. Swim motion
Power stroke (active
contraction and thrust)
Recovery stroke
(passive relaxation and
feeding current)
All-or-nothing muscle
activation
Nicole W. Xu
Stanford University
16
5 cm
April 21, 2020
17. 17
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
18. External control of swimming
Electrodes act as “artificial pacemakers” to activate the muscles.
18
V
t
Input(s):
Signal voltage
Signal pulse width
Signal period (inverse of frequency)
Electrode location on the bell
Output:
Pulses (tag displacement)
Nicole W. Xu
Stanford University
April 21, 2020
21. Natural vs. stimulated pulses
21
Nicole W. Xu
Stanford University
April 21, 2020
Natural
TagDisplacement(pixels)
TagDisplacement(pixels)
0.25 Hz
TagDisplacement(pixels)
2 Hz
TagDisplacement(pixels)
1 Hz
22. Frequency spectrum of natural pulses
22
Unstimulated cases, N = 12 animals
Natural motion includes a bout frequency
and successive pulses per bout (a spread between 0.4-1 Hz).
0.16 Hz
0.4-1 Hz
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
Amplitude
TagDisplacement(pixels)
Example displacement, N = 1
23. Frequency spectrum of stimulated pulses
23
Stimulated at 1 Hz, N = 10 animals
1.02 Hz
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
Amplitude
TagDisplacement(pixels)
Example displacement, N = 1
24. N = 10 N = 10 N = 10 N = 10 N = 10 N = 10 N = 9
Stimulated frequency response (0.25 – 2 Hz)
24
N = 10 animals
PowerSpectralDensity
Frequency Output (Hz)
Nicole W. Xu
Stanford University
Successful one-to-one response
Max pulse frequency according to a proposed bio limit (Bullock, J Cell Comp Physio, 1943)
Peak frequency of unstimulated animals (natural behavior)
Xu and Dabiri, Science Advances, 2020
April 21, 2020
Successful frequency responses between 0.25 to 1 Hz
25. Electrode placement (1 Hz)
Where is the optimal location
for electrode placement?
Stomach
Midway
Pacemaker
Margin
Sensor placement will also
be relevant when we place
temperature/salinity/pH probes
on jellyfish in the ocean
Nicole W. Xu
Stanford University
25April 21, 2020
26. Electrode placement (1 Hz, N = 10)
Nicole W. Xu
Stanford University
26
Stomach Midway
Margin Rhopalia
.
.
.
.
.
.
.
.
.
.
.
.
x x
x
x
Electrodes can be placed anywhere for consistent
muscle responses
April 21, 2020
27. Using CFD to determine optimal sensor placement
Nicole W. Xu
Stanford University
27
Stomach Midway
Margin Rhopalia
.
.
.
.
.
.
.
.
.
.
.
.
x x
x
x
Future work: computational fluid dynamics (CFD) to
determine optimal sensor placement for untethered
experiments
April 21, 2020
Hoover et al., JFM, 2017
Gemmell et al., J. Royal Soc. Interface, 2015
28. Using CFD to determine optimal sensor placement
Nicole W. Xu
Stanford University
28
Stomach Midway
Margin Rhopalia
.
.
.
.
.
.
.
.
.
.
.
.
x x
x
x
Future work: computational fluid dynamics (CFD) to
determine optimal sensor placement for untethered
experiments
April 21, 2020
Hoover et al., JFM, 2017
29. 29
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
30. Swim controller design
1 cm 1 cm
1 cm
30
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
Design choices:
Small
Neutrally buoyant
Low power
Inexpensive
Housing
Cap
Microcontroller
Battery
Electrodes
April 21, 2020
32. 32
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
33. Frequency vs. swimming speed
0.9 m
1.8 m
swim controller
jellyfish
camera
Two electrodes
(straight swimming)
Frequency (Hz)
0.25-1.00 Hz
33
Nicole W. Xu
Stanford University
April 21, 2020
34. Frequency vs. swimming speed
34
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
35. Frequency vs. swimming speed
35
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
36. Robotic control can increase speeds up to 3x
Swim controller frequency (Hz)
Speed(bodydiameterss-1)
36
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
AR (aspect ratio) = height
diameter
AR = 0.3
Enhancementfactor
=stimulatedspeed/unstimulatedspeed
April 21, 2020
37. Robotic control can increase speeds up to 3x
Swim controller frequency (Hz)
Enhancementfactor
37
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
AR = 0.2
AR = 0.3
Speed(bodydiameterss-1)
AR (aspect ratio) = height
diameter
Enhancementfactor
=stimulatedspeed/unstimulatedspeed
April 21, 2020
38. Robotic control can increase speeds up to 3x
Swim controller frequency (Hz)
38
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
AR = 0.2
AR = 0.3
Speed(bodydiameterss-1)
AR (aspect ratio) = height
diameter
Enhancementfactor
=stimulatedspeed/unstimulatedspeed
April 21, 2020
We can increase jellyfish swimming up to 2.8 times.
Enhancement depends on body morphology/shape.
39. Mechanistic model
39
Nicole W. Xu
Stanford University
November 26, 2019
åF = mj
du
dt
Daniel, Canadian Journal of Zoology, 1983
Xu and Dabiri, Science Advances, 2020
Theoretical model that incorporates body
kinematics in thrust, drag, and acceleration
reaction forces
T - D - AR = mj
du
dt
T =
rw
Asub
æ
è
çç
ö
ø
÷÷
dVsub
dt
æ
è
ç
ö
ø
÷
2
D =
1
2
Cd
rw
Aj
u2
AR =arj
Vj
du
dt
a =
2ht
dt
æ
è
çç
ö
ø
÷÷
1.4
Thrust Drag Acceleration Reaction
mj mass of the jellyfish
u speed
⍴w density of saltwater
Asub area of the jellyfish subumbrella
Vsub volume of the jellyfish subumbrella
Cd drag coefficient
Aj area of the jellyfish
ht height of the jellyfish
dt diameter of the jellyfish
⍴w density of the jellyfish
Vj volume of the jellyfish
40. Mechanistic model
40
Nicole W. Xu
Stanford University
November 26, 2019
åF = mj
du
dt
Frequency (Hz)Frequency (Hz)
Speed(bodydiameterss-1)
Daniel, Canadian Journal of Zoology, 1983
Xu and Dabiri, Science Advances, 2020
Theoretical model that incorporates body
kinematics in thrust, drag, and acceleration
reaction forces
Speed(bodydiameterss-1)
41. 41
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
We can increase speeds, but at what cost to the animal?
42. Measuring frequency vs. respiratory rate
25 cm
10 cm
oxygen
probe
jellyfish
2 electrodes
power
supply
sealed top
O2Concentration(μmolO2L-1)
Time (min)
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
42April 21, 2020
43. Measuring frequency vs. respiratory rate
O2Consumption(μmolO2L-1hr-1g-1)
0 Hz 0.25 Hz 0.50 Hz 0.88 Hz
43
Nicole W. Xu
Stanford University
April 21, 2020
25 cm
10 cm
oxygen
probe
jellyfish
2 electrodes
power
supply
sealed top
47. Metabolic costs
47
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
åF = mj
du
dt
48. Metabolic costs: efficient swimming
48
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
Robotic control can enhance swimming speed up to 3x, with only
a 2x increase in cost of transport.
April 21, 2020
49. 49
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
April 21, 2020
But the electronic part ALSO consumes power.
50. Low power compared to other swimming robots
Biohybrid
(this work)
50
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
51. Low power compared to other swimming robots
Biohybrid
(this work)
51
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
Frame et al, Bioinsp Biomim, 2019
52. Low power compared to other swimming robots
Biohybrid
(this work)
52
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
Park et al, Science, 2016
53. Low power compared to other swimming robots
Biohybrid
(this work)
53
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
Hydroid
54. Low power compared to other swimming robots
Biohybrid
(this work)
This biohybrid robotic jellyfish uses 10-1000x less external power
per mass compared to existing swimming robots.
54
Nicole W. Xu
Stanford University
Xu and Dabiri, Science Advances, 2020
April 21, 2020
55. Spectrum of bioinspired jellyfish constructs
Nicole W. Xu
Stanford University
55
Biological
Construct
Limitation:
Survival in a
few specific
environments
Robotic
Construct
Limitation:
High power
consumption
Biohybrid
Robot
Advantages:
Low power & cost
Adaptable to many
environments
April 21, 2020
Nawroth et al, Nature Biotech, 2012Villanueva et al, PLoS ONE, 2014
56. 56
Building a biohybrid robotic jellyfish
Nicole W. Xu
Stanford University
1. Understand how jellyfish naturally swim.
2. Characterize how to control jellyfish muscle contraction
(tethered in-dish experiments).
3. Build a portable swim controller.
4. Characterize the artificial control of jellyfish swimming
(free-swimming experiments).
5. Improve robotic controllability toward ocean monitoring
(ongoing work).
Robustness and maneuverability in real environments
April 21, 2020
57. Robustness: robot survivability in the ocean
Marine Biological Laboratory in Woods Hole, MA
Scuba divers: Jack Costello, Sean Colin, James Townsend
57
Nicole W. Xu
Stanford University
April 21, 2020
67. 67
How can we expand ocean monitoring tools?
Nicole W. Xu
Stanford University
April 21, 2020
Goal: Use technology to collect data about ocean environments
Needs
Able to explore new locations
Minimal power
Minimal cost
Minimal disturbance to aquatic life
How we can achieve this: Combine technological advances,
experimental work, and computational tools
National Geographic
68. Next steps toward using biohybrid jellyfish robots
for ocean monitoring
68
Nicole W. Xu
Stanford University
Technology
Sensors
Science
Lab experiments to
control more complex
maneuverability (turning)
Test various jellyfish
species
Field experiments and
ocean monitoring
April 21, 2020
Technology Science
Computation
69. Next steps toward using biohybrid jellyfish robots
for ocean monitoring
69
Nicole W. Xu
Stanford University
Computational tools
AI for image
segmentation and
object tracking in
various environments
CFD for optimal
sensor placement and
predicting animal
trajectories for closed
loop feedback
April 21, 2020
Technology Science
Computation
70. Acknowledgements
• Members of the Dabiri lab
• Cabrillo Marine Aquarium
• Marine Biological Laboratory
- Jack Costello
- Sean Colin
- James Townsend
- Brad Gemmell
• NSF Graduate Research Fellowship
Weiland Family Fellowship
Timothy G. Shi Graduate Fellowship
Contact info: nicolexu@stanford.edu web.stanford.edu/~nicolexu
70
Nicole W. Xu
Stanford University
April 21, 2020