Droidcon London 2013 presentation about car sensor access

1. Sensor fusion between car and
smartphone
Gabor Paller
gaborpaller@gmail.com
Sfonge Ltd.
http://www.sfonge.com

2. Thanks for the memory

3. Car offers
●
Power (no need to worry about battery drain)
●
Fixed position (if in car kit)
●
Sensors
●
More sensors → more exciting applications!
●
Sensor fusion: the use of multiple sensors so
that they compensate each other's weaknesses

4. Relevant phone sensors
●
●
●
●
GPS – if the phone is placed in the car so that it
“sees” the satellites
Accelerometer – shaking, vibration, violent
movements like aggressive braking or turns
Gyroscope – turn detection, support sensor for
the accelerometer
Compass (magnetometer) – direction (not
sensitive to drift), magnetic signature

5. Relevant car sensors
●
●
●
●
Speed – driving style, distance (if integrated by
time)
RPM – driving style
Built-in GPS – not subject to incorrect
placement within the car
Large number of car diagnostic indicators –
remote diagnostic → not the subject of this
presentation

6. Obtaining car sensor data
●
●
●
●
OBDII (like OBD-2, On-board diagnostic)
Widely deployed (mandatory in the EU since
2001/2004)
Developed for diagnostic, not for obtaining
traffic data
Nevertheless, used for a variety of applications
like fleet management

7. (MirrorLink)
●
●
Emerging standard for car infotainment
system-phone connection
Developed by Car Connectivity Consortium
(CCC), http://www.mirrorlink.com/
●
Supported by some recent phones and cars
●
Even the specification costs 150 USD
●
Not used in the experiments in this presentation

8. ELM327
●
●
●
●
●
ELM327 is a popular microcontroller for OBD2 access. Developed by
ELM Electronics cloned by many
On the car side, it talks OBD2 to the car's on-board computer
On the terminal side, it offers a friendly serial interface with AT
commands reminiscent of the Hayes modem command set
Actual devices often offer USB or Bluetooth interface (anything goes
that has serial line support)
Bluetooth is ideal for Android device connectivity
●
●
Basic Bluetooth example programs like BluetoothChat can be used to
communicate with the device almost without changes
Example program:
http://mylifewithandroid.blogspot.com/2013/03/data-capture-application-for-car-speed.html

9. ELM327 sequence
●
ATD OK
●
Set all to defaults
●
ATZ ELM327 v1.4
●
Reset all
●
ATE0 OK
●
Switch off echo
●
ATM0 OK
●
Don't store discovered protocols
●
ATL0 OK
●
Don't send LF after CR
●
ATH0 OK
●
Don't show OBD headers
●
ATS0 OK
●
Don't insert spaces into hex digits
●
ATSP0 OK
●
Discover OBD protocol automatically
●
Query current data 0x00 (register support)
●
Query current data 0x0D (speed, 0 km/h)
●
●
0100 SEARCHING..
4100BE3EB811
010D 410D00

10. Cutting edge technology!

11. OBDII dongle

12. Precise fixing is crucial

13. First use case
●
●
Dead
reckoning-type
navigation when
GPS is not
available
E.g. in tunnels,
parking houses,
dense urban
settings (urban
valleys) ...

14. Sensors used
●
●
●
Base coordinate p0 comes from the GPS or
from earlier measurement
Absolute value of velocity vector (v) comes
from the speed sensor of the car
Direction of velocity vector
●
Gyroscope
●
Compass

15. Gyroscope recap
●
●
Measures rotation around 3 axes
More exactly: measures rotation speed (angular
velocity) around the axes
Δφ
v x=
Δt
Δ φ =v x Δ t
φ ' =φ + Δ φ

16. The problem of devices
●
Newer phones are not always better
●
●
Particularly not their sensors
It happens that a new sensor type – like
gyroscope – is introduced in good quality
●
Then the cost-cutting starts in later, “more
advanced” device types

17. Nexus S gyro
Car making 90 degree turns, device horizontal.
Red: angular velocity, blue: aggregated rotation degree

18. Galaxy Nexus gyro
Car making 90 degree turns, device horizontal.
Red: angular velocity, blue: aggregated rotation degree
Origin of the spikes: unknown

19. Only Nexus S measurements will be used in this
presentation

20. The problem of drift

21. Compass
●
●
Measures the device orientation wrt.
the magnetic vector of the Earth
This vector points toward the
magnetic center of the Earth
●
●
It has a component that points to the
magnetic North pole – that's what we
use for orientation
Beware of the z component! (also
called magnetic inclination). If the
device is not held horizontally, the
downward vector element influences
the measurement

22. Car's magnetic field
Turning around, out of car
Phone is inside the car, ignition on, car makes full circle

23. Find the offset
●
●
●
Identify 3 points on
the circle (use the
gyroscope to figure
out whether there
was enough
rotation)
Calculate the center
of the circle
Subtract the offset

24. Move the circle to the center
Circle moved to the zero point of the coordinate system
(on the x-y plane)
Initial circle

25. Compass vs. gyro
Compass
Gyro
Compass is harder to calibrate and less exact but it has no drift

26. Tunnels

27. Tunnels

28. Effect of external magnetic fields
Not called Chain Bridge by chance ...

29. Parking houses

30. Parking houses

31. Urban valleys

32. Urban valleys
No GPS reading
in the circle

33. Results
●
●
●
Dead reckoning using speed (from car) and
direction (from smartphone) provides reasonable
results when GPS is not available
Compass and gyroscope need to support each
other to obtain sufficiently precise direction
vector
Compass is an extremely problematic sensor in
car/urban environment that needs constant
recalibration

34. Second use case: vibrations
●
●
Goal: use the phone's accelerometer for something
useful (car does not have any similar sensor)
Separating motion acceleration: already discussed
here:
http://mylifewithandroid.blogspot.com/2012/05/better-motion-control-using.html
(can be used to classify driving habits)
●
Vibration:
●
Damaged road
●
Vehicle faults (not discussed here)

35. bumpy_road

36. Raw vibration
Bumpy road
section

37. After high-pass filter
Effects of slow acceleration changes (turns) are removed

38. Power
Bumpy road
section

39. Luckily we have exact position ...

40. Speed bumps

41. Drivers slow down ...
Speed drops &
vibration spike
at the same time:
speed bump
Red: speed, blue: acceleration absolute value

42. Or they don't ...
Not possible to detect
Red: speed, blue: acceleration absolute value power

43. Results
●
●
●
Localized road damages cannot always be
recognized
Damages that span longer distances may be
detected more reliably
In any case, with a large number of
measurements, road damages can be mapped
efficiently

44. Idea of a platform
●
Available for Android applications
●
Discovers car connectivity options and sensors
●
●
●
Notifies applications about the connectivity
status and available sensors
Allows single or repeated sampling of sensor
data
Controlling car's systems?

45. Questions?