Presentation of the Human Firewall project developed as part of the course of IoT at Sapienza University

1. Human Firewall
Andrea Fioraldi - Mattia Nicolella - Leonardo Sarra
Neighbors united against scammers!

2. Human Firewall in brief: a smart intercom
- Always connected to the internet
- Take photo and profile the person that rings
the bell
- Trigger a notification to the house/business
owner and provide an evaluation of the
subject (based on peer review)
- It doesn't replace the existing intercom, it is
an addon

3. Features
Architecture
Technologies

4. A Cloud-Free System
● All the system COMPLETELY lives in the boards with the
exception of the part in the Telegram servers that is not
managed by us
● All the computation is local
● Made for a condominium (need all the devices in the same
LAN) but can be easily scaled using the same code but a
different deploy setup using a VPN

5. A Distributed Cluster
● Deploy the system in a NATted WiFi LAN (no open ports to the internet are
needed)
● One Master, many slaves
● A Slave failure is completely tolerated
● Master failure needs a manual reboot but not data loss
● So people can deactivate they intercom (e.g. I’m in holiday at the Bahamas and I
don’t want Telegram messages with the face of the postman) and reactivate it
later without losing the data collected from the others

6. Easy to Deploy
● Just clone the master-scripts and/or slave-scripts repository
● Run install.py
● All dependencies will be installed and the service will be installed in
systemd
● You need only to specify your private Telegram API key
● Abstraction in the code allow to deploy a demo version on a laptop

7. Universal Interface
The face of the person at the doorbell is
sent to you via Telegram
You can have it on your SmartPhone, PC
or Tablet
You can manage multiple doorbells and
multiple Telegram accounts can handle
the same doorbell (for families)
Place your screenshot here

8. Private Cooperative Feedbacks
● Faces features vector are collected and stored only
in the private deployed system and not in the cloud
● Images are used to compute the vector and then
deleted
● The “vendor” cannot get images or even computed
vectors from customers (so this can’t be a Google
product :) )

9. Features
Architecture
Technologies

10. Feedbacks
DB Bot
Users DB
Features
Vectors
Subscriber
Telegram client
Camera
ServerQuery &
Send
Find Master
Local
Slave
Slave Master
Vectori
ze
MQTT
Broker
Publisher

11. Feedbacks
DB Bot
Users DB
Features
Vectors
Subscriber
Telegram client
Camera
ServerQuery &
Send
Find Master
Local
Slave
Slave Master
Vectori
ze
MQTT
Broker
Publisher
Vector, rate,
board ID
Vector, rate,
chat ID, time

12. Feedbacks
DB Bot
Users DB
Features
Vectors
Subscriber
Telegram client
Camera
ServerQuery &
Send
Local
Slave
Slave Master
Vectori
ze
MQTT
Broker
Publisher
Find Master
Last
timestamp

13. Features
Architecture
Technologies

14. Face Recognition Library
An interface to the dlib's state-of-the-art face recognition library that provides several face locations
algorithms and pre-trained vectorization models.
We use the HOG model (O(#pixels)) to locate faces instead that the default CNN model of dlib, a less
accurate model but also less CPU intensive [1].
To compute encodings, the pretrained model is based on ResNet, a Deep CNN [2].
We use a threshold of 0.6 in the euclidean product to detect if two encodings are referred to the same
person, a bit high to scale but it is ok for a condominium.
[1] https://towardsdatascience.com/cnn-based-face-detector-from-dlib-c3696195e01c
[2] http://blog.dlib.net/2017/02/high-quality-face-recognition-with-deep.html

15. Mosquitto MQTT and Paho
We choose Eclipse Mosquitto as our broker, an open source implementation of an MQTT
broker.
Mosquitto is lightweight and is suitable for use on devices with a low CPU clock.
We choose to not have a persistent storage of messages to not waste disk space.
The MQTT client we opted for is the Eclipse Paho project, which provides open-source
client implementations of MQTT and MQTT-SN messaging protocols for multiple
programming languages (Python included).

16. Flask
Flask is a microframework for Python based on Werkzeug and Jinja 2.
We use it to host the master board web server since it is an extensible framework
which can be easily tailored for several purposes.
For the basic need of the project we only choose the Flask core module and some
basic extensions which make our web server lightweight and fast.

17. python-telegram-bot
To interact with Telegram’s APIs (over HTTP) in the bot subsystem we
opted for a community made wrapper named python-telegram-bot.
The wrapper implements a wide range of functionalities from sending
messages to conversation handling, while, preventing synchronization
issues from appearing.
It has a small memory-footprint and low CPU utilization because it
uses webhooks to prevent polling Telegram’s servers.

18. Raspberry Pi and Pi Camera
We used a Raspberry Pi 3 to host host all the above services and we
make full use of its computational power when we use the face
recognition library in the slave module and when we host the Flask
server and the Telegram bot.
To take the photos we chose the Pi Camera module which is able to
take full HD photos.

19. User feedback
“This may be useful for my grandma because it could prevent her from opening the door at people
that wants to take advantage of her.”
Luca, 22
“With that I could know who rings at the doorbell when I’m not at home.”
“The telegram notification may be useful when I’m using the earphones and I can’t hear the
doorbell.”
Eleonora, 25
Matteo, 23

20. Detailed article: https://bit.ly/2J0VAXt
GitHub organization: https://github.com/humanfirewall-iot19
Andrea contacts:
● GitHub: https://github.com/andreafioraldi
● LinkedIn: https://www.linkedin.com/in/andrea-fioraldi/
Leonardo contacts:
● GitHub: https://github.com/LithiumSR/
● LinkedIn: https://www.linkedin.com/in/leonardo-sarra-b34691182/
Mattia contacts:
● GitHub: https://github.com/Nick1296
LinkedIn: https://www.linkedin.com/in/mattianicolella/
Info & Contacts

21. Thanks for the attention!