From Pandas to GeoPandas - 地理資料處理與分析

From Pandas to
GeoPandas
倪鈵斯@PYCON TAIWAN 2016
Painted by Shih Jiang-Zhu.

誤入叢林的⼩小⽩白兔
ABOUT ME
倪鈵斯（nispc）
National Taiwan University of  
Science and Technology
Major in Information Management

ABOUT ME
Development Intern
DSP.IM (DSP智庫驅動)
2015~

Development Intern
2015~
Hacking Skills
Development Intern
2015~
Computer Science
Pandas GIS
ABOUT ME
台科⼤大資管系
2014~
DSP智庫驅動
2015~

ABOUT ME
台科⼤大資管系
2014~
DSP智庫驅動
2015~
Hacking Skills
Pandas GIS
From These Placees,
I Learned

ABOUT ME
Pandas GIS
GeoPandas

台灣最常⽤用的座標參考系統
The most Common CRS in Taiwan
Geographic coordinate system
• WGS84（Lat/Lon） 
Projected Coordinate System
• TWD67 ( TM2 )、TWD97 ( TM2 ) 
The image above using Mercator Projection, but TM2 using Transverse Mercator ProjectionNote:

以直觀（但不太正確）的角度解釋TM2的台北座標
Using TM2 to Locate Taipei City
Taipei City
x = 304803.322190434 m
y = 2767518.970508755 m
X
Y
(0, 0)

分辨⼿手上的資料是TM⼆二度分帶（TWD67 or 97）或經緯度座標
Distinguish which CRS of your Data
Before processing your data, you should know the CRS
of your dara.  
You can do the following if you are not sure: 
 
TM2 or Lat/Lon？
• If the numbers is bigger your data must be TM2.
TWD97( TM2 ) or TWD67( TM2 )？
• Normally, It’s TWD97 data, unless you get that earlier than
1999 ( 921 Jiji earthquake ).

台灣常⽤用的座標參考系統
More About Common CRS Used in Taiwan
• http://www.sunriver.com.tw/grid_tm2.htm
A website explaining the details of common
CRS Used in Taiwan

Shapefile的結構
The Structure of Shapefile
• Shapefile save numerical data and text data in dBase(.dbf) file.
• Shapefile save shapes and positions in .shp file.
• and .shx save index, .prj save projection information.

讓我們快速的匯入台灣村⾥里界圖！
Let’s Quickly Import Our Data
%pylab inline
pylab.rcParams['figure.figsize'] = (20.0, 20.0)
import geopandas as gpd
import shapely
villages_shp = gpd.read_file('./台灣⾥里界圖_TWD97/
Village_NLSC_121_1050219.shp')
Import the geoPandas, and setting ipython notebook to
plot map more beautifully.
Import a shapefile.

See the data
COUNTY_
ID
C_Name
OBJECTI
D
PRO_ID Substitute TOWN_ID T_Name … geometry
0 64 ⾼高雄市 8573 64 中興⾥里 6401000 旗津區 …
POLYGON
((177607.7583971
694
1 10007 彰化縣 8578 10 頂潭⾥里 1000712 ⽥田中鎮 …
POLYGON
((207765.5412078
464
2641446.5807414
2 10007 彰化縣 8579 10 中潭⾥里 1000712 ⽥田中鎮 …
POLYGON
((207195.1317646
012
2641217.4845214
3 10007 彰化縣 8580 10 ⻯⿓龍潭⾥里 1000712 ⽥田中鎮 …
POLYGON
((206697.0792388
434
2641761.2836532
4 10007 彰化縣 8581 10 ⼤大社⾥里 1000712 ⽥田中鎮 …
POLYGON
((206697.0792388
434
2641761.2836532
villages_shp.head()

villages_shp.plot(colormap='OrRd')
… and plot it!

geometry
楠梓區 POLYGON ((180500.0468580014 2511840.500121991,...
左營區 POLYGON ((179578.8300897071 2507758.486656731,...
三⺠民區 POLYGON ((182669.6859151478 2504196.741750024,...
⿎鼓⼭山區 POLYGON ((175673.2319138724 2502706.179941533,...
新興區 POLYGON ((178357.0004510713 2502407.254186806,...
前⾦金區 POLYGON ((177708.4723173666 2502176.189247195,...
鹽埕區 POLYGON ((177056.8290152341 2502272.220627455,...
苓雅區 POLYGON ((180854.1866929225 2501717.513408468,...
前鎮區 POLYGON ((180493.3661862851 2496473.692563967,...
旗津區 POLYGON ((178725.3208101905 2495444.779886731,...
⼩小港區 POLYGON ((184614.1874942174 2490602.750132099,...
GeoPandas如何儲存形狀
How GeoPandas Storage The Shapes

geometry
楠梓區 POLYGON ((180500.0468580014 2511840.500121991,...
左營區 POLYGON ((179578.8300897071 2507758.486656731,...
三⺠民區 POLYGON ((182669.6859151478 2504196.741750024,...
⿎鼓⼭山區 POLYGON ((175673.2319138724 2502706.179941533,...
新興區 POLYGON ((178357.0004510713 2502407.254186806,...
前⾦金區 POLYGON ((177708.4723173666 2502176.189247195,...
鹽埕區 POLYGON ((177056.8290152341 2502272.220627455,...
苓雅區 POLYGON ((180854.1866929225 2501717.513408468,...
前鎮區 POLYGON ((180493.3661862851 2496473.692563967,...
旗津區 POLYGON ((178725.3208101905 2495444.779886731,...
⼩小港區 POLYGON ((184614.1874942174 2490602.750132099,...
How GeoPandas Storage The Shapes
GeoPandas如何儲存形狀
All of the shapes and positions are save
in shapely.geometry objects

However, GIS not Only Deal with
Spatial Data,
But Also Attribute Data.
」
「

結合數值資料並且做資料視覺化
Combine Numerical Data and Visualization
geometry
楠梓區 POLYGON ((180500.0468580014 25...
左營區 POLYGON ((179578.8300897071 25...
三⺠民區 POLYGON ((182669.6859151478 25...
⿎鼓⼭山區 POLYGON ((175673.2319138724 25...
新興區 POLYGON ((178357.0004510713 25...
前⾦金區 POLYGON ((177708.4723173666 25...
鹽埕區 POLYGON ((177056.8290152341 25...
苓雅區 POLYGON ((180854.1866929225 25...
前鎮區 POLYGON ((180493.3661862851 24...
旗津區 POLYGON ((178725.3208101905 24...
⼩小港區 POLYGON ((184614.1874942174 24...GeoSeries

結合數值資料並且做資料視覺化
Combine Numerical Data and Visualization
⼈人⼝口⼈人⼝口密度 geometry
楠梓區
左營區
三⺠民區
⿎鼓⼭山區
新興區
前⾦金區
鹽埕區
苓雅區
前鎮區
旗津區
⼩小港區
census density
Series GeoSeries
？

To Combine Numerical Data and
Visualization,
We Should Preprocess Our Data
」
「

使⽤用⼩小資料練習Pandas及GeoPandas
Practice Pandas and GeoPandas with Small Data!
villages_shp = gpd.read_file(
‘./台灣⾥里界圖_TWD97/Village_NLSC_121_1050219.shp
)
Import a shapefile:
census_villages = pd.read_csv(
'Data/104年12⽉月⼈人⼝口數.csv', skiprows=[0]
)
Load census data:

COUNTY_
ID
C_Name
OBJECTI
D
PRO_ID Substitute TOWN_ID T_Name … geometry
0 64 ⾼高雄市 8573 64 中興⾥里 6401000 旗津區 …
POLYGON
((177607.7583971
694
2497521.1105418
1 10007 彰化縣 8578 10 頂潭⾥里 1000712 ⽥田中鎮 …
POLYGON
((207765.5412078
464
2641446.5807414
villages_shp.head(2)
census_villages.head(2)
統計年
⽉月區域別村⾥里⼾戶數⼈人⼝口數
⼈人⼝口數-
男
⼈人⼝口數-
⼥女
0歲-男 0歲-⼥女 …
0 10412
新北市
板橋區
留侯⾥里 691 1649 786 863 5 8 …
1 10412
新北市
板橋區
流芳⾥里 638 1564 732 832 3 3 …
have some problems.

census_villages['city'] =
census_villages[‘區域別’].copy(deep=True)
census_villages['city'] =
census_villages['city'].apply(lambda t: t[:3])
census_villages['district'] =
census_villages['區域別'].copy(deep=True)
census_villages['district'] =
census_villages['district'].apply(lambda t: t[3:])
統計年
⽉月
區域別村⾥里⼾戶數⼈人⼝口數⼈人⼝口數-男 … city district
0 10412 新北市
板橋區
留侯⾥里 691 1649 786 … 新北市板橋區
1 10412 新北市
板橋區
流芳⾥里 638 1564 732 … 新北市板橋區
2 10412
新北市
板橋區
⾚赤松⾥里 318 838 410 … 新北市板橋區

demo_villages = ['惠楠⾥里', '惠⺠民⾥里', '惠豐⾥里', '莒光⾥里',
'合群⾥里', '永清⾥里', '橋頭⾥里', ‘橋南⾥里']
kaohsiung_villages_shp =
villages_shp[villages_shp['C_Name']=='⾼高雄市']
demo_villages_shp = kaohsiung_villages_shp[
kaohsiung_villages_shp['V_Name'].apply(
lambda t: True if t in demo_villages else False
)
]
Select Some Villages for Demo
, and Leave the Columns which We Needed.
kaohsiung_villages_shp = kaohsiung_villages_shp[
['C_Name', 'T_Name', 'V_Name', ‘geometry’]
]

C_Name T_Name V_Name geometry
16 ⾼高雄市左營區永清⾥里 POLYGON ((177943.1675527846 25...
17 ⾼高雄市左營區莒光⾥里 POLYGON ((177698.0080210482 25...
2675 ⾼高雄市左營區合群⾥里 POLYGON ((177309.4301372209 25...
2705 ⾼高雄市楠梓區惠楠⾥里 POLYGON ((181185.0309484294 25...
2706 ⾼高雄市楠梓區惠⺠民⾥里 POLYGON ((180715.7155030172 25...
2707 ⾼高雄市楠梓區惠豐⾥里 POLYGON ((179848.5000471332 25...
3208 ⾼高雄市橋頭區橋頭⾥里 POLYGON ((179476.0936987746 25...
3209 ⾼高雄市橋頭區橋南⾥里 POLYGON ((179887.6875365854 25...
demo_villages_shp

Select Some Villages for Demo
demo_villages = ['惠楠⾥里', '惠⺠民⾥里', '惠豐⾥里', '莒光⾥里',
'合群⾥里', '永清⾥里', '橋頭⾥里', ‘橋南⾥里']
kaohsiung_census_villages =
census_villages[census_villages['city'] == '⾼高雄市']
demo_census_villages = kaohsiung_census_villages[
kaohsiung_census_villages['村⾥里'].apply(
lambda t: True if t in demo_villages else False
)
]
, and Leave the Columns which We Needed.
demo_census_villages = demo_census_villages[
['city', 'district', '村⾥里', '⼈人⼝口數']
]

city district 村⾥里⼈人⼝口數
3430 ⾼高雄市左營區永清⾥里 1288
3431 ⾼高雄市左營區莒光⾥里 2024
3433 ⾼高雄市左營區合群⾥里 2656
3463 ⾼高雄市楠梓區惠楠⾥里 3533
3464 ⾼高雄市楠梓區惠⺠民⾥里 4514
3465 ⾼高雄市楠梓區惠豐⾥里 5407
4020 ⾼高雄市橋頭區橋頭⾥里 2550
4021 ⾼高雄市橋頭區橋南⾥里 1299
demo_census_villages

比較處理空間資料及屬性資料—Group By（1）
Dealing with Spatial and Attribute Data Simultaneously

⾼高雄市左營區
⾼高雄市楠梓區
⾼高雄市橋頭區
demo_villages_shp_group =
demo_villages_shp.groupby(by=['C_Name', 'T_Name'])

C_Name T_Name geometry
0 ⾼高雄市左營區
POLYGON ((177943.1675527846
2510441.206417892,...
1 ⾼高雄市楠梓區
POLYGON ((181185.0309484294
2514217.982513821,...
2 ⾼高雄市橋頭區
POLYGON ((179887.6875365854
2517777.250005289,...
demo_district_shp = gpd.GeoDataFrame(
demo_villages_shp_group[‘geometry’]
.agg(lambda t: shapely.ops.unary_union(t)))
aggregation function

Dealing with Spatial and Attribute Data Simultaneously
比較處理空間資料及屬性資料—Group By（2）

demo_census_district_group =
demo_census_villages.groupby(by=['city', 'district'])
⾼高雄市左營區
⾼高雄市楠梓區
⾼高雄市橋頭區

city district ⼈人⼝口數
1288
2024
2656
3533
4514
5407
2550
1299
demo_census_district_group.sum()

1288
2024
2656
3533
4514
5407
2550
1299
demo_census_district_group.sum()
SUM
( )
SUM
( )
SUM ( )
= 5968
= 13454
=3849

POLYGON ((177943.1675527846
2510441.206417892,...
POLYGON ((181185.0309484294
2514217.982513821,...
POLYGON ((179887.6875365854
2517777.250005289,...
0 ⾼高雄市左營區 5968
1 ⾼高雄市楠梓區 13454
2 ⾼高雄市橋頭區 3849
Dealing with Spatial and Attribute Data Simultaneously ( Join by Attribute)
比較處理空間資料及屬性資料—Join by Attribute

POLYGON
((177943.1675527846
2510441.206417892,...
POLYGON
((181185.0309484294
2514217.982513821,...
POLYGON
((179887.6875365854
2517777.250005289,...
0 ⾼高雄市左營區 5968
1 ⾼高雄市楠梓區 13454
2 ⾼高雄市橋頭區 3849
demo_census_district_shp = demo_district_shp.merge(
demo_census_district,
left_on=('C_Name', 'T_Name'),
right_on=('city', 'district')
)

製作區域密度圖
Making a Choropleth Map
demo_census_district_shp.plot(
column='⼈人⼝口數',
scheme='QUANTILES',
k=3,
cmap=‘OrRd'
)
… and plot it!
https://github.com/geopandas/geopandas/blob/master/examples/choropleths.ipynb

Fake Mayor
This is imagined
此為講解GIS應⽤用之虛構情境
市⻑⾧長的煩惱
柯P照⽚片： ETtoday
背景圖⽚片：聯合影⾳音 / 鄭清源

環域分析
Buffer Analysis
https://docs.qgis.org/2.8/en/docs/gentle_gis_introduction/
vector_spatial_analysis_buffers.html

疊圖分析
Overlay Analysis
Intersection Union
Difference
Symmetrical
Difference
https://docs.qgis.org/2.8/en/docs/gentle_gis_introduction/
vector_spatial_analysis_buffers.html

忠孝橋引道問題
Zhongxiao Bridge’s Problem
Approach road to Zhongxiao Bridge
Select the
Approach road
Calculate the zone of
influence
Find the Villages
which be influenced
Buffer IntersectionFine “忠孝重慶” int.
and “忠孝環河" int.

roads = gpd.read_file('./Taiwan Road’)
匯入台灣道路資料
Import Roads’ Data
roads = roads.to_crs(epsg=3826)
Import Taiwan’s Roads:
To Calculate, We Should Project Our GeoDataframe to
TWD97（Epsg code: 3826）

選取台北的道路
Select Roads in Taipei
taipei_shp =
villages_shp[villages_shp[‘C_Name’]=='臺北市']
taipei_roads =
roads[roads.intersects(taipei_shp.unary_union)]

選出忠孝橋引道
Select “Approach road to Zhongxiao Bridge”
重慶路 = taipei_roads[
taipei_roads['name'].str.contains('重慶')==True
]
忠孝路 = taipei_roads[
taipei_roads['name'].str.contains('忠孝')==True
]
環河路 = taipei_roads[
taipei_roads['name'].str.contains('環河')==True
]

ax = 重慶路.plot(linewidth=3, color="Orange")
忠孝路.plot(linewidth=3, ax=ax, color="Blue")
環河路.plot(linewidth=3, ax=ax, color="Green")
忠孝路
環河路
重慶路

Fine “忠孝重慶” int. and “忠孝環河" int.
忠孝重慶int = 忠孝路.unary_union.intersection(
重慶路.unary_union
).centroid
忠孝環河int = 忠孝路.unary_union.intersection(
環河路.unary_union
).centroid

忠孝重慶int.x , 忠孝重慶int.y
We Get the Easternmost Position of “Approach road to
Zhongxiao Bridge”
忠孝環河int.x , 忠孝環河int.y
… and the Easternmost Position of “Approach road to
Zhongxiao Bridge”
Ouput: (301790.13325580396, 2771103.9320118735)
Output: (301073.18135184597, 2771299.2108038645)

忠孝路_x = 忠孝路.geometry.centroid.apply(
lambda t: t.x
)
忠孝橋引道 = 忠孝路[
(忠孝路_x < 忠孝重慶int.x)&
(忠孝環河int.x < 忠孝路_x)
]

畫出影響範圍
Calculate the Zone of Influence
忠孝橋引道buffer =
gpd.GeoSeries(忠孝橋引道.unary_union.buffer(500))
ax = 忠孝橋引道buffer.plot()
忠孝橋引道.plot(ax=ax, linewidth=4)

找出被影響的⾥里
Find the Villages which be Influenced
influences_villages_shp = villages_shp[
villages_shp
.intersects(忠孝橋引道buffer.unary_union)==True
]
COUNTY_ID C_Name … V_Name geometry
2365 63 臺北市 … 光復⾥里
POLYGON ((301794.5933154966
2771355.22341227, ...
2366 63 臺北市 … 黎明⾥里
POLYGON ((302365.136612736
2771239.723670578, ...
2374 63 臺北市 … ⽟玉泉⾥里
POLYGON ((301272.2482771481
2771805.173890756,...
2375 63 臺北市 … 建明⾥里
POLYGON ((302012.1867108274
2771553.461402568,...
...共13⾥里

ax = influences_villages_shp.plot()
忠孝橋引道buffer.plot(ax=ax)

找出各⾥里實際於影響區內的區域
Find the Influenced Part in the Villages
influences_villages_shp_clip =
influences_villages_shp.copy()
influences_villages_shp_clip['geometry'] =
influences_villages_shp_clip.intersection(
忠孝橋引道buffer.unary_union
)

ax = influences_villages_shp.plot(color=‘White')
influences_villages_shp[(
influences_villages_shp_clip['geometry'].area /
influences_villages_shp['geometry'].area
) > 0.8
].plot(ax=ax, color=‘Orange')

ax = influences_villages_shp.plot(color=‘White')
influences_villages_shp[(
influences_villages_shp_clip['geometry'].area /
influences_villages_shp['geometry'].area
) > 0.5
].plot(ax=ax, color=‘Orange')

DONEC QUIS NUNC
`
台灣的⼯工程師真的很棒
圖⽚片來源：⾃自由時報 / 廖振輝此為講解GIS應⽤用之虛構情境
This is imagined

遺珠之憾
Pearls of regret
There are two subjects are interesting, too.
• Spatial join
• Folium

From Pandas to GeoPandas - 地理資料處理與分析

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

From Pandas to GeoPandas - 地理資料處理與分析