This document discusses addressing device dependencies in Linux. It proposes a lightweight "device links" data structure and API to represent and manage dependencies between devices. Device links would help with tasks like power management and runtime ordering by capturing dependency relationships. The API would allow adding, removing, and querying device links. Managed device links could also help with probe ordering and auto-removal of dependent devices.

1. What To Do When Your Device Depends
on Another One
Rafael J. Wysocki
Intel Linux Systems Engineering
September 25, 2019
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 1 / 20

2. Introduction Background
Devices Depend On One Another
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 2 / 20

3. Introduction Background
Simple Case: Tree Topology
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 3 / 20

4. Introduction Imperfections
Example: Driver Core and Power Management
Driver Core
Type
Middle Layer
Device Driver
Device
Action
PM Domain
Class
Bus
Device Driver
Device
Device
Device
DeviceDevice Device
DeviceDevice
Bus
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 4 / 20

5. Introduction Imperfections
Example: Driver Probe Ordering
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 5 / 20

6. Device Links Design
Idea: Light-weight Solution On Top Of What Is There
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 6 / 20

7. Device Links Design
Device Links Data Structure
Device
Device
Device
Device
DeviceDevice
Device
Device
Device
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 7 / 20

8. Device Links Post-probe Use Cases
System-wide PM And Shutdown Ordering
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 8 / 20

9. Device Links Post-probe Use Cases
Asynchronous Execution Of System-wide PM Callbacks
Call Notifiers
Freeze Tasks
Device Suspend Device Resume
Thaw Tasks
Call Notifiers
Wait For a Wakeup Interrupt
Call Notifiers
Freeze Tasks
Nonboot CPU Offline
System Core Offline
Device Suspend
Platform Offline
System Core Online
Nonboot CPU Online
Device Resume
Thaw Tasks
Call Notifiers
Platform Online
.prepare()
.suspend()
.suspend_late()
.suspend_noirq()
.prepare()
.suspend()
.suspend_late()
.suspend_noirq().resume_noirq()
.resume_early()
.resume()
.complete()
.resume_noirq()
.resume_early()
.resume()
.complete()
Wait For a Wakeup Event
Platform-based Suspend
Suspend to Idle
Working State
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 9 / 20

10. Device Links Post-probe Use Cases
PM-runtime Ordering
Suspended
Active
Is Idle?
YES
NO
SuspendResume
Trigger Event
DL_FLAG_PM_RUNTIME (+ DL_FLAG_RPM_ACTIVE)
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 10 / 20

11. Device Links Post-probe Use Cases
Device Links Usage In Power Management
Driver Core
Type
Middle Layer
Device Driver
Device
Action
PM Domain
Class
Bus
Device Driver
Device
Device
Device
DeviceDevice Device
DeviceDevice
Bus
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 11 / 20

12. Device Links Device Link Management
Managed Device Links
DL_FLAG_AUTOREMOVE_CONSUMER
DL_FLAG_AUTOREMOVE_SUPPLIER
DL_FLAG_AUTOPROBE_CONSUMER
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 12 / 20

13. Device Links Device Link Management
Stateless Device Links
DL_FLAG_STATELESS
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 13 / 20

14. API and State Machine Addition and Removal API
Adding Device Links
struct device_link *device_link_add(struct device *consumer,
struct device *supplier,
u32 flags);
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 14 / 20

15. API and State Machine Addition and Removal API
Removing Stateless Device Links
void device_link_del(struct device_link *link);
void device_link_remove(void *consumer,
struct device *supplier);
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 15 / 20

16. API and State Machine Device Link State Machine
Managed Device Link State Machine
Dormant
Available
Consumer
Probe
Supplier
Unbind
Active
Supplier
Probe
Consumer
Unbind
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 16 / 20

17. API and State Machine Device Link State Machine
Rules Regarding Device Link Flags
1 STATELESS bumps up the reference counter.
2 Passing STATELESS along with link management flags is invalid.
3 AUTOREMOVE_SUPPLIER overrides AUTOREMOVE_CONSUMER.
4 “No management flags” overrides both autoremove flags.
5 Passing AUTOPROBE_CONSUMER along with autoremove flags is invalid.
6 RPM_ACTIVE without PM_RUNTIME is ignored.
7 PM_RUNTIME can always be set and is never unset.
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 17 / 20

18. Wrap-Up Summary / Discussion
Questions? Comments? Concerns?
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 18 / 20

19. Resources
References
Greg Kroah-Hartman, Linux Driver Model (https://www.youtube.com/watch?v=AdPxeGHIZ74).
Rafael J. Wysocki, Power Management Challenges in Linux (https:
//www.linuxplumbersconf.org/2017/ocw//system/presentations/4652/original/linux_pm_challenges.pdf).
Rafael J. Wysocki, PM Infrastructure in the Linux Kernel – Current Status and Future
(https://events.linuxfoundation.org/sites/events/files/slides/kernel_PM_infra_0.pdf).
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 19 / 20

20. Disclaimer
Intel technologies’ features and benefits depend on system configuration
and may require enabled hardware, software or service activation.
Performance varies depending on system configuration. No computer
system can be absolutely secure. Check with your system manufacturer or
retailer or learn more at www.intel.com.
Intel and the Intel logo are trademarks of Intel Corporation in the U.S.
and/or other countries. *Other names and brands may be claimed as the
property of others.
c Intel Corporation
Rafael J. Wysocki Addressing Device Dependencies September 25, 2019 20 / 20