Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
A Study on Estimation of Household
Kerosene Consumption for
Optimization of Delivery Plan
北海道大学 大学院情報科学院
情報理工学専攻 調和系工学研究室
修士2 年 劉兆邦

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
2
Background
Kerosene consumption estimation
• Kerosene is main resource for heating and shower
• Efficient kerosene delivery plan
- Need to know the remaining kerosene
- Apply inventory routing algorithm
• Difficulties in kerosene consumption estimation
- Change in consumption pattern
- New customer
Problem in existing method
• Based on experience
- Frequently deliver or rarely deliver
• Based on sensor
- Installed on part of the customers
- Unable to install sensor on all customers for now
Refueling and kerosene tank under snow

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
3
Related Work
• Kerosene consumption estimation is treated as multivariate
time series regression problem
• Related work
- Electricity consumption estimation[1][2]
• One building or one country
• Focused on future consumption
- Time series classification[3]
• Sound and Medical area
• Kerosene delivery recording composed by many similar time
series(each customer)
• Classify simple consumption pattern and estimate consumption
[1]Nivethitha Somu, Gauthama Raman MR, and Krithi Ramamritham. A hybrid model for building energy consumption forecasting using long
short term memory networks. Applied Energy, 261:114131, 2020.
[2] Tao Liu, Zehan Tan, Chengliang Xu, Huanxin Chen, and Zhengfei Li. Study on deep reinforcement learning techniques for building energy
consumption forecasting. Energy and Buildings, 208:109675, 2020.
[3] Xuchao Zhang, Yifeng Gao, Jessica Lin, and Chang-Tien Lu. Tapnet: Multivariate time series classification with attentional prototypical
network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6845–6852, 2020.

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Target
- Improve kerosene consumption estimation precision
• One day consumption, Refueling quantity(Between two refuels)
- Support optimization of delivery plan
• No measurable estimation model for delivery plan creation
• Reduce delivery times(about 23 times delivery each day one
worker)
• Reduce customer running out of kerosene(about 4 customers
each month)
Table of contents
- Refueling recordings and influence factors in Sapporo
- Formulate kerosene consumption estimation problem
- Proposed method
- Experiments
- Conclusion
Target and Contents 4

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
5
Overview
• Use refuel recordings(different interval) and influence factors
(fixed interval)
- Days between refuels are called refueling interval
• Estimate mean consumption or refuel quantity
• Mean consumption is applied in delivery planning algorithm

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
interval(day) 13 14 15 18 21 25 34 40
number 4 3 1 1 1 1 1 1
Interval of customer A
6
Refuel recordings
• Refuel recording in Sapporo
- Refuel time and quantity are recorded
Interval distribution
Recordings of customer A
Recordings of customer B
Same customer may have
different interval
Refuel recording
history is different
Refuel centered on
winter(Nov-April)
Interval is different from
the whole dataset

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
7
Influence Factor
• Meteorological data from Japan Meteorological
Agency
– Recording method
• Average of 24 recordings in one day
– Area：Sapporo
– Period：2010~2022
• Kerosene price data
– Interval is one month
Japan Meteorological Agency(気象庁):https://www.data.jma.go.jp/obd/stats/etrn/index.php
北海道生活協同組合連合会: http://www.doren.coop/oil.html
Temperature data in Sapporo Kerosene price data in Sapporo

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Variable name variable Remark
Sum consumption
for Customer 𝑢𝑖 in
one season
How many kerosene
this customer use each
season
Mean consumption
for Customer 𝑢𝑖 in
one season
How many kerosene
this customer use each
day
8
Formulation
Variable name variable Remark
Customer ID 𝑢𝑖
Refuel Time 𝑡𝑢𝑖,𝑠𝑙,𝑗
Customer 𝑢𝑖 , 𝑗𝑡ℎ
refuel time in season 𝑠𝑙
Refuel quantity 𝑞𝑢𝑖,𝑠𝑙,𝑗
Refuel quantity for 𝑗𝑡ℎ
refuel
※Assume every time is fully refueled
Season: Nov~next year April
Refuel recording
• Use kerosene refuel recording and influence factors as input
• Estimate mean consumption and refuel quantity

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
9
Formulation
• Multivariate time series: Instance 𝑋 ∈ 𝑇, dimenson of 𝑋 is 𝑛, 𝑚
• Data augment:
• Time series regression: Input is time series、output is scalar
Variable name variable Remark
Instance 𝑋 input
Period 𝑇 Refuel history
Timestep 𝑛 interval between continuous refuel
Features 𝑚
date 𝑑𝑖
Refuel recording 𝑟𝑘 Refuel time, refuel quantity,
customer ID
Estimation value 𝑐 Output(scalar)

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
10
Model
Data Processing: data augmentation and match with external
feature
Attention: Search relation between each time step
Deep Learning: DNN, LSTM, Deep Sets, transformer

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
11
Data process
Refuel recording
Date between refuel
Meteorological data
price
Sum consumption
Mean consumption
date
• Refuel recording data only include contents in refuel day
– Refuel quantity is determined by all the days from last refuel
• After data augmentation, information is efficiently used
– 𝑑1 is the date for last refuel、𝑑𝑛 is this refuel date
– Between 𝑑1 and 𝑑𝑛 is continuous date
• Add External features to corresponding date 𝑑

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
12
Attention
• Input vector of each timestep(date)
• Output vector contains relation between each time step
– Same customer with similar meteorological will have similar consumption
– Use attention to catch this relation
Variable name Remark
𝑥 input (feature vector)
𝑞,𝑘,𝑣 Vector transformed from input
𝑠 Attention score
𝑦 Weighted 𝑣
𝑜 Output(contains each time steps relation）
Output vector contains relation
between each timestep
Attention
score for each
timestep

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
13
Deep Leaning-DNN, LSTM
• DNN
– A normal deep learning method, can be seen as a
multivariate function
• Base line deep learning model
• LSTM
– A sequence specified deep learning model, mainly
used in time series data
• Kerosene recording is time series data

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
14
Deep Leaning-Deep Sets
• For kerosene consumption, sum of cold day is important
• Deep Sets treats time series as set
– permutation invariance
• Embedding: Change to other space
– Best result may not in dataset’ local space
• Sum the time axis to capture the set feature
d1 d2 d3 d4 d1 d2 d4
d3 Warm
Cold

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
15
Deep Leaning-Transformer
[1] Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30
(2017).
[2] Zerveas, George, et al. "A transformer-based framework for multivariate time series representation learning."
Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021.
Encoder Decoder
Use encoder
only
• Regression and classification
• Reduce training time and parameter

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
16
Experiments
• Experimental setting
• Experiment 1
– Pre-experiment
• Verify assumption
• Experiment 2
– Compare with baseline
• Linear regression
– Error analysis
• Experiment 3
– Number of customer run out of kerosene
– Delivery time

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
17
Experimental setting
customer recording
ShortHRR-Highcc 1,329 11,098
ShortHRR-Lowcc 194 2,167
LongHRR-Highcc 84 4,233
LongHRR-Lowcc 14 836
Original Refuel Recording Datasets(two different companies)
• LongHRR: Long History Refuel Recordings(few customers)
– 2009 ~ 2019(10years)
• ShortHRR: Short History Refuel Recordings(more customers)
– 2020 ~2023 (3years)
customer recording
LongHRR 212 11,706
ShortHRR 1,722 26,512
Processed Dataset for Experiment 1
cc: correlation coefficient between
consumption and temperature
Highcc: cc above threshold
Lowcc: cc under threshold
threshold: 0.6
Processed Dataset for Experiment 2
Relation between mean consumption
and temperature

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
18
Experimental setting
Evaluation Metric for Estimated Consumption
• MAE use refuel quantity as label (MAEr)
• MAE use mean consumption between continuous refuel as
label(MAEd)
• Use refuel interval to calculate mean consumption
Variable
name
variable
𝑐 Estimated mean consumption,
refuel quantity
𝑞 Real refuel quantity
𝑛 Number of test data
Artificial consumption pattern
• Some customer only contain one season recording
• An average consumption is used as their last season consumption pattern
Hyperparameter
• Epoch:200
• Trial: 3
• Learning rate: 0.001~0.000001(Reduce learning rate when metric has
stopped improving.)
• Train:Test = 8:2

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
19
Experiment 1
Linear DNN LSTM DeepSets Transformer
shortHRR-
highcc
36.53 29.67 76.65 29.53 30.30
shortHRR-
lowcc
85.19 71.41 103.08 71.76 75.67
longHRR-
highcc
44.66 37.18 34.35 35.43 37.55
longHRR-
lowcc
49.63 44.99 50.70 46.05 50.98
• High correlation coefficient customer has high estimation precision
compared to low correlation coefficient customer
- Model can capture general consumption pattern
• All best results are Deep Learning model
- In kerosene consumption estimation, deep leaning model can capture
nonlinear pattern better
Estimation result for Different Group(MAEr)
Average of 3 trials MAEr: L/each refuel

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
20
Experiment 2-1
Estimation result for Different Model
• DeepSets and Transformer have high precision
• Transfomer is more complicated than DeepSets
- Time consuming
• Models with attention has better performance for MAEd
- Relation between each day captured by attention
• LSTM performs better on longHRR
• Interval in longHRR centered between 1 week and 2 weeks
– Real world delivery has long range interval(ShortHRR)
Dataset Metric
Model
Linear DNN LSTM DeepSets
DeepSets
+Attention
Transformer
longHRR MAEd 6.09 6.01 5.67 5.96 5.59 5.59
shortHRR MAEd 4.63 4.34 6.20 4.30 3.98 3.96
longHRR MAEr 65.24 62.74 57.70 61.76 60.41 58.20
shortHRR MAEr 60.58 54.25 52.69 53.92 52.51 52.74
DeepSets+attention is more suitable for real world use
Average of 3 trials
MAEr: L/each refuel
MAEd: L/each day

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
21
Experiment 2-2
Most of the error (72%) are less than 60L and less than 30% than that refuel
• Extremely large error exists
• Need to pay attention to those customers when apply in real world use
Recordings Mean
Refuel
MAEr MAEd
artificial 2,849 215.10 64.69 4.70
real 2,453 223.46 39.33 3.51
Model performs much better in customer with real past consumption pattern
• A better artificial past consumption pattern may help improve the model
Error distribution(MAEr)
Artificial and real past consumption pattern

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
22
Experiment 3-1
Period: 2022.01.01~ 2022.03.31, 2022.11.01~2022.12.31,
Area: Sapporo
Estimation Model: Deep Sets - Attention, shortHRR、MAEd
IRP Algorithm: tabu search heuristic with rolling horizon procedure is applied
Current threshold: Currently used refueling threshold (real world), refuel under this value (0.5)
Low threshold : Reduce delivery times、Increase possibility of running out of kerosene (0.3)
Pattern threshold: Artificial past consumption pattern use current, real use low
Number of Customer: 568
period
1 7
Final delivery order
Delivery order
Delivery order for sub problem
e.g. (𝑫 = 𝟕、𝑾 = 𝟑、𝑺 = 𝟐)
Rolling Horizon
Delivery times and Working hours
• To make sure the influence of estimation model on delivery plan
Customer running out of kerosene:
• To make sure how many customer will run out of kerosene compared to real recording
Delivery plan setting
Delivery plan metric

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
23
Experiment 3-2
Delivery times and Working
hours
• Using current threshold only several customers running out of kerosene
GTC: Ground truth consumption based delivery plan
EC: Estimated consumption based delivery plan
Refuel
Threshold
Delivery times(times/day)
-weather type
Working hours(hour/day)
-weather type
true forecast true forecast
Real GTC EC GTC EC GTC EC GTC EC
Current(50%)
23
52 64(+12) 52 64(+12) 8.5 9.9(+1.4) 8.5 9.8(+1.3)
Low(30%) 31 39(+8) 31 39(+8) 6.2 7.2(+1.0) 6.2 7.3(+1.1)
pattern 44 52(+8) 44 52(+8) 7.6 8.8(+1.2) 7.6 8.7(+1.1)
Customer running out of
kerosene
• 10 more customers need to be delivered about 1 hour working
Refuel Threshold Real(1month)
Estimated each month
Nov Dec Jan Feb Mar
Current(50%)
4
4 3 10 6 7
Low(30%) 23 52 40 63 58
pattern 20 52 30 36 34

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
24
Experiment 3-2
Delivery times and Working
hours
• Using current threshold only several customers running out of kerosene
GTC: Ground truth consumption based delivery plan
EC: Estimated consumption based delivery plan
Refuel
Threshold
Delivery times(times/day)
-weather type
Working hours(hour/day)
-weather type
true forecast true forecast
Real GTC EC GTC EC GTC EC GTC EC
Current(50%)
23
52 64(+12) 52 64(+12) 8.5 9.9(+1.4) 8.5 9.8(+1.3)
Low(30%) 31 39(+8) 31 39(+8) 6.2 7.2(+1.0) 6.2 7.3(+1.1)
pattern 44 52(+8) 44 52(+8) 7.6 8.8(+1.2) 7.6 8.7(+1.1)
Customer running out of
kerosene
• 10 more customers need to be delivered about 1 hour working
Refuel Threshold Real(1month)
Estimated each month
Nov Dec Jan Feb Mar
Current(50%)
4
4 3 10 6 7
Low(30%) 23 52 40 63 58
pattern 20 52 30 36 34
Automate delivery plan creation
that used to be made by delivery
worker and their experience

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Conclusion 25
Conclusion
• Use meteorological and price data to help consumption estimation
• Formulated kerosene consumption estimation problem
• Proposed Deep Learning based kerosene consumption estimation
model
• Deep Sets with Attention mechanism is suitable for real world use
• Be able to automate delivery plan creation using current threshold
without increase customers that running out of kerosene
Prospect
• Find better way to create artificial past consumption pattern
• Create robust model to lower the refuel threshold without increasing
customers that running out of kerosene

Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
Copyright © 2020 調和系工学研究室 - 北海道大学 大学院情報科学研究院 情報理工学部門 複合情報工学分野 – All rights reserved.
26
Research achievements
Domestic conference
• 劉 兆邦, 横山 想一郎, 山下 倫央, 川村 秀憲, 多田 満朗, Estimation of Household Kerosene
Consumption Using Multi-Layer Perceptron for Determining Kerosene Delivery Plan, 第21回
複雑系マイクロシンポジウム, 8, オンライン(2022)
• 劉兆邦, 横山 想一郎, 山下 倫央, 川村 秀憲, 多田 満朗, Estimation of Household Kerosene
Consumption Using DeepSets for Efficient Kerosene Delivery Plan, 第22回データ指向構成マ
イニングとシミュレーション研究会(SIG-DOCMAS), 5, ハイブリッド開催（オンライン／横
浜）(2022)
Exhibition
• 令和4年度北楡会・北海道大学情報系交流会，情報科学研究院棟，2022年9月．
• 2022北海道ビジネスEXPO，アクセスサッポロ，2022年11月．
To be announced
• 劉 兆邦，横山 想一郎，山下 倫央，川村 秀憲，多田 満朗，Prediction of Household
Kerosene Consumption Using Deep Learning for Kerosene Delivery Planning，社会システム
と情報技術研究ウィーク（WSSIT2023），2023