4. Positioning users in a modern
network
no radio-goniometer at scale
cell of attachment has position, beam characteristics
over history, best position ~200m
5. Positioning at specific locations
handovers at specific cell-to-cell location
phone needs to be active
6. Positioning with more precision
better positioning with excellent data sources:
3G : GPEH
4G: LTE-CTR
10. On (not) tracking (any users)
"Swisscom strictly complies with all applicable legislations, in
particular with the telecommunications law and the data
protection initiative."
Jürg Studerus, Swisscom Senior Manager, Corporate Responsibility
11. Smart Data : Big Data without Big Brother
Privacy preservation is an asset
It makes sense to care as much about your customer as they do about you.
We technically enforce this
answering only synoptic questions, no individual ones,
with data flow control : we neutralize quasi-identifiers at every stage
13. Our choices
public good applications: making Switzerland run better,
understanding places, not individuals,
all results presented aggregated, anonymized.
17. Usages
New roads to divert transit traffic out of downtown (informs a 50M$
project)
Parking lot expansion and transformation (informs a 10M$ project)
Electric car charging station deployment
20. Spark configuration essentials for enterprise
jobs
spark.executor.memory="not the default 1g"
spark.kryo.registrator="something custom"// and companions
spark.shuffle.service.enabled="true"
spark.dynamicAllocation.enabled="true"
spark.deploy.recoveryMode="ZOOKEEPER"
spark.deploy.recoveryDirectory="/path/to/state"
spark.deploy.zookeeper.url="quorumMachine1:2181, ..."
NOT the only valuable settings, see https://techsuppdiva.github.io
for more
26. Selecting users on a path of Interest
Massive discrepancy between # of users (2-3E6)
and # of interesting users (1.5E3 on test segments)
Filtering interesting time series.
28. Locality-sensitive hashing short histories
A family H of hashing functions is -sensitive if:(r, cr, , )p1 p2
if then
if then
p–q ≤ r P [h(q) = h(p)] ≥rH p1
p–q ≥ cr P [h(q) = h(p)] ≤rH p2
More :
Locality Sensitive Hashing By Spark, Uber, Spark Summit
A Gentle Introduction to Locality-Sensitive Hashing with Apache Spark,
Scala by The Bay
29. Computing speeds: Solving graph
constraints
a speed comes from a user well-positioned, twice
plus route knowledge
given a history of cells, where was the user, exactly ?
30. Solving graph constraints
just a few users left in computation at this stage
so a lot invested in > linear complexity algorithms
32. Crucial elements
Quality, reliability of data sources
Automated ground truth checking
sensors
TEMS fleet
What's the ground truth for mode of transport, domicile, etc ?
Colleagues and friends volunteers
33. In the works
Accuracy improvements
More features (see you Spark Summit EU!)
Streaming for city
Thank you