O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Why and how we proxy our IoT broker connections

302 visualizações

Publicada em

ScaleDay conference

Publicada em: Tecnologia
  • Entre para ver os comentários

Why and how we proxy our IoT broker connections

  1. 1. 1
  2. 2. Why and how we proxy our IoT brokerWhy and how we proxy our IoT broker connectionsconnections MQTT at scale 2
  3. 3. Clément DECOODT DevOps - IoT 3
  4. 4. IoT?IoT? 4
  5. 5. Hello, IoT world!Hello, IoT world! MQTT 101 IoT platform architecture More details about the reverse proxy 5
  6. 6. MQTTMQTT Message Queuing Telemetry Transport Not a message queue! Publish-Subscribe protocol 6 . 1
  7. 7. 25 BrokerClient A CONNECT SUBSCRIBE server/temperature CONNACK Client B PUBLISH server/temperature 25 °C PUBLISH server/temperature 25 °C 25 DISCONNECT
  8. 8. Simon A. Eugster [ ],CC BY-SA 4.0 via Wikimedia Commons 6 . 2
  9. 9. MQTTMQTT First specs: 1999 OASIS Standard since 2014 Ultra light protocol Multiple endpoints: TCP, TLS, WebSocket 6 . 3
  10. 10. CLICLI Subscribing Publishing mosquitto_sub -i $deviceid -h iot.fr-par.scw.cloud -t $topic 1 2 3 mosquitto_pub -i $deviceid -h iot.fr-par.scw.cloud -t $topic -m $message 1 2 3 4 6 . 4
  11. 11. from paho.mqtt import client as mqtt client = mqtt.Client("electric_meter") client.connect("iot.fr-par.scw.cloud", 1883) def on_message(client, userdata, message): print("Received: ", str(message.payload.decode("utf-8"))) client.on_message = on_message client.subscribe("switches") client.publish("info","Electric meter v0.0.1") client.loop_start() 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 6 . 5
  12. 12. IoT StationIoT Station Episode 1: the hubEpisode 1: the hub 7
  13. 13. IoT at ScalewayIoT at Scaleway Connecting devices together Give access to cloud resources to devices Deploy code to the devices 8 . 1
  14. 14. IoT at ScalewayIoT at Scaleway Connecting devices together 8 . 1
  15. 15. IoT at ScalewayIoT at Scaleway Product plans Shared broker Dedicated High availability 8 . 2
  16. 16. Platform architecturePlatform architecture 9 . 1
  17. 17. ConstraintsConstraints gRPC server for API Single MQTT endpoint for all customers 9 . 2
  18. 18. Control planeControl plane Server-Worker architecture Workers act as state synchronizers 9 . 3
  19. 19. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 9 . 4
  20. 20. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking 9 . 4
  21. 21. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking 2 - Wrapping2 - Wrapping 9 . 4
  22. 22. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain 2 - Wrapping2 - Wrapping 9 . 4
  23. 23. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain 2 - Wrapping2 - Wrapping 9 . 4
  24. 24. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent 2 - Wrapping2 - Wrapping 9 . 4
  25. 25. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent 2 - Wrapping2 - Wrapping 9 . 4
  26. 26. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping 9 . 4
  27. 27. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping 9 . 4
  28. 28. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping Separation of concerns 9 . 4
  29. 29. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping Separation of concerns 9 . 4
  30. 30. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping Separation of concerns Broker-independent 9 . 4
  31. 31. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping Separation of concerns Broker-independent 9 . 4
  32. 32. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping Separation of concerns Broker-independent No service interruption 9 . 4
  33. 33. MQTT Broker - 2 alternativesMQTT Broker - 2 alternatives Let's not reinvent the wheel 1 - Forking1 - Forking Hard to maintain Broker-dependent Better access to the internals 2 - Wrapping2 - Wrapping Separation of concerns Broker-independent No service interruption 9 . 4
  34. 34. API Gateway gRPC API K8S Worker Database BrokersKubernetes Broker 1 Broker N Brokers (mosquitto) Homemade Legend Open-Source software Managed product 9 . 5
  35. 35. API Gateway gRPC API K8S Worker Database BrokersKubernetes Broker 1 Broker N Brokers (mosquitto) Homemade Legend Open-Source software Managed product Load Balancer MQTT Reverse Proxy 9 . 6
  36. 36. API Gateway gRPC API K8S Worker Database BrokersKubernetes Broker 1 Broker N Brokers (mosquitto) Homemade Legend Open-Source software Managed product Load Balancer MQTT Reverse Proxy TSDB (InfluxDB) 9 . 7
  37. 37. MQTT Proxy internalsMQTT Proxy internals 10
  38. 38. Wanted featuresWanted features Horizontally scalable TCP and WebSocket endpoints Device-related metrics TLS Termination MQTT parser on-the-fly 11 . 1
  39. 39. Pipeline stages explainedPipeline stages explained 11 . 2
  40. 40. Pipeline stages explainedPipeline stages explained 1 - WebSocket/TCP endpoint 11 . 2
  41. 41. Pipeline stages explainedPipeline stages explained 2 - TLS termination 1 - WebSocket/TCP endpoint 11 . 2
  42. 42. Pipeline stages explainedPipeline stages explained 3 - MQTT Packet metadata parsing 1 - WebSocket/TCP endpoint 2 - TLS termination 11 . 2
  43. 43. Pipeline stages explainedPipeline stages explained 4 - Device/Hub-related metadata fetching 1 - WebSocket/TCP endpoint 2 - TLS termination 3 - MQTT Packet metadata parsing 11 . 2
  44. 44. Pipeline stages explainedPipeline stages explained 5 - Authentication/Authorization 1 - WebSocket/TCP endpoint 2 - TLS termination 3 - MQTT Packet metadata parsing 4 - Device/Hub-related metadata fetching 11 . 2
  45. 45. Pipeline stages explainedPipeline stages explained 6 - Optional topic rewriting 1 - WebSocket/TCP endpoint 2 - TLS termination 3 - MQTT Packet metadata parsing 4 - Device/Hub-related metadata fetching 5 - Authentication/Authorization 11 . 2
  46. 46. Pipeline stages explainedPipeline stages explained 7 - Billing and metrics reporting 1 - WebSocket/TCP endpoint 2 - TLS termination 3 - MQTT Packet metadata parsing 4 - Device/Hub-related metadata fetching 5 - Authentication/Authorization 6 - Optional topic rewriting 11 . 2
  47. 47. Pipeline stages explainedPipeline stages explained 8 - MQTT Packet serialization and transmission 1 - WebSocket/TCP endpoint 2 - TLS termination 3 - MQTT Packet metadata parsing 4 - Device/Hub-related metadata fetching 5 - Authentication/Authorization 6 - Optional topic rewriting 7 - Billing and metrics reporting 11 . 2
  48. 48. Wait, how?Wait, how? 11 . 3
  49. 49. Dynamic broker discoveryDynamic broker discovery Finding the right broker at the right timeFinding the right broker at the right time 12 . 1
  50. 50. Dynamic broker discoveryDynamic broker discovery Shared brokers are easy to find Clustered/Dedicated brokers needs to be contacted somehow 12 . 2
  51. 51. Kubernetes DNS service discoveryKubernetes DNS service discovery 1. Give each broker a deterministic DNS record 2. Find the broker ID from the device ID, resolve the record from it 3. Connect, and have fun! 12 . 3
  52. 52. Let's meet together!Let's meet together! From 15h30 - Workshop - Pilotez une ampoule connectée 16h30 - GANYMEDE - Demystifier le buzzword All day - IoT Stand 13
  53. 53. Thank you!Thank you! Follow us on Twitter and LinkedIn @Scaleway 14
  54. 54. 15
  55. 55. Advanced MQTTAdvanced MQTT https://www.hivemq.com/blog/mqtt-essentials-part-1-introducing-mqtt/ 16

×