舒服二人組在JCConf 2015的Workshop講義～歡迎大家來學Google App Engine唷～

1. 輕鬆學Google的雲端開發 -
Google App Engine入門
Simon Su & Sunny Hu
JCConf 2015
http://goo.gl/X8YY73

2. Github
https://github.com/peihsinsu/JCConf2015-GAE-Workshop

4. We are QNAPer, Now!

5. https://www.facebook.com/groups/GCPUG.TW/
https://plus.google.com/u/0/communities/116100913832589966421
Google Cloud Platform User Group Taiwan
我們是Google Cloud Platform Taiwan User Group。在Google雲端服務在台灣地區展露頭角之後，
有許多新的服務、新的知識、新的創意，歡迎大家一起分享，一起了解 Google雲端服務...
GCPUG透過網際網路串聯喜好 Google Cloud的使用者，分享與交流使用 GCP的點滴鑑驗。如果您
是Google Cloud Platform的初學者，您應該來聽聽前輩們的使用經驗；如果您是 Google Cloud
Platform的Expert，您應該來分享一下寶貴的經驗，並與更多高手互相交流；如果您還沒開始用
Google Cloud Platform，那麼您應該馬上來聽聽我們是怎麼使用 Google Cloud的！

7. ●
●
●
●
●
●
●

8. Compute
Compute
Engine
App Engine
Container
Engine
Storage
Cloud
Storage
Cloud SQL Cloud
Datastore
App Services
Cloud EndpointsPrediction API Translate APIBigQuery
Big Data
Pub/Sub Dataflow Bigtable

9. https://console.developers.google.com/project?utm_referrer=blank
Create your own project

10. https://console.cloud.google.com/freetrial
Free USD$300 Trail

12. ●

13. Eclipse version Installation instructions Direct plugin link
Eclipse 4.4 (Luna) Plugin for Eclipse 4.4 (Luna) https://dl.google.com/eclipse/plugin/4.4
Eclipse 4.3 (Kepler) Plugin for Eclipse 4.3 (Kepler) https://dl.google.com/eclipse/plugin/4.3
Eclipse 3.8/4.2 (Juno) Plugin for Eclipse 3.8/4.2 (Juno) https://dl.google.com/eclipse/plugin/4.2

15. ●
●
●

17. Java source code
Web source code
GAE setting
Servlet setting

19. ●
●
●
●

22. …

23. ●
●
●
●
●

25. 基本設定

26. <async-session-persistence enabled="true" />

31. …

32. Callback
Page
Result
Page
extends:
AbstractAppEngineAuthor
izationCodeServlet
extends:
AbstractAppEngineAuthor
izationCodeCallbackServl
et
1
2
3
4

33. 整合

34. web.xml

36. 取得
/get-application-id.jsp

39. ●
●
●
●

40. ●
●
●

41. Googler’s way
AppEngine
Traditional Web
applications
Web application
framework
AppEngine
(Java, Python, Go, PHP)
Java, Perl/CGI, PHP, Ruby,
Python...
Persistent storage
NoSQL
● Datastore
● Cloud SQL
RDBMS
● MySQL
● PostgreSQL
● SQL Server
● Oracle

42. Datastore RDBMS
Query language
flexibility
SQL-like query language
● Limited to simple
filter and sort
Full support of SQL
● Table JOIN
● Flexible filtering
● Subquery
Reliability and
Scalability
Highly scalable and
reliable
Hard to scale

43. ●
○
○
○
●
○
○

44. Datastore RDBMS
Category of object Kind Table
One entry/object Entity Row
Unique identifier of data entry Key Primary Key
Individual data Property Field

45. Property
Property
Property
PostEntry User
Kinds
Key: blog-1234
user: simonsu@xxx.com
message: xxxxxx
date: 3/1/2014
Key: simonsu@xxx.com
email: simonsu@xxx.com
followees: [user2@xxx.com,
user3@xxx.com]
followers:
Key: user2@xxx.com
email: user2@xxx.com
followees:
followers: [simonsu@xxx.com]
Entities
Keys

46. DatastoreService datastore =
DatastoreServiceFactory.getDatastoreService();
Entity employee = new Entity("Employee");
employee.setProperty("name", "Simon Su");
employee.setProperty("hireDate", new Date());
Key empKey = datastore.put(employee);
# DatastoreExample.java

47. // Use email as key when creating entity
Entity employee = new Entity("Employee", "work-id-D001");
datastore.put(employee);
// Later, use the key to retrieve the entity
Key userKey = KeyFactory.createKey("Employee", "work-id-D001");
Entity user = datastore.get(userKey);
# DatastoreExample.java

48. DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Key key = KeyFactory.createKey("User", "simonsu.mail@gmail.com");
try {
Entity ent = ds.get(key);
out.print(ent);
Map m = ent.getProperties();
out.println(m.toString());
} catch (EntityNotFoundException e) {
e.printStackTrace();
}

49. Query query = new Query("Person");
Query.Filter nameFilter = new Query.FilterPredicate(
"name", FilterOperator.EQUAL, "John");
query.setFilter(nameFilter);
PreparedQuery results = datastore.prepare(query);
# DatastoreExample.java

50. Query q = new Query("Person");
Query.Filter filter1 = new FilterPredicate(...);
Query.Filter filter2 = new FilterPredicate(...);
Query.Filter comboFilter =
CompositeFilterOperator.and(filter1, filter2);
q.setFilter(comboFilter);
q.addSort("name");
# DatastoreExample.java

51. Query query = new Query("Kind");
query.setAncestor(parentKey);

52. ● Manual configure:
WEB-INF/datastore-indexes.xml
● System generated:
WEB-INF/appengine-generated/datastore-indexes-auto.xml

53. <?xml version="1.0" encoding="utf-8"?>
<datastore-indexes>
<datastore-index kind="Widget">
<property name="x" direction="asc" />
<property name="date" direction="asc" />
</datastore-index>
<datastore-index kind="Widget">
<property name="y" direction="asc" />
<property name="date" direction="asc" />
</datastore-index>
</datastore-indexes>
<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes autoGenerate="true">
</datastore-indexes>

56. ●
●

57. Query for:
first_name = Cathy
last_name > Able
last_name < Mooney
Query for:
first_name > Cathy
last_name > Able

58. Query for:
first_name = Cathy
last_name > Able
sort by last_name Query for:
last_name > Able
sort by first_name

59. DatastoreService datastore = DatastoreServiceFactory.getDatastoreService()
Transaction txn = datastore.beginTransaction();
try {
Key ekey = KeyFactory.createKey("Employee", "Joe");
Entity employee = datastore.get(ekey);
/*... reading and writing on employee ...*/
datastore.put(employee);
txn.commit();
} finally {
if (txn.isActive()) {
txn.rollback();
}
}
# DatastoreExample.java

60. ● Datastore basic
● Datastore operation
● Restrictions
● Transaction

61. DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Entity job = new Entity("User", "simonsu@mitac.com.tw");
job.setProperty("name", "Simon Su");
job.setProperty("start", "20140103");
job.setProperty("create", new Date());
try {
Key k = ds.put(job);
out.println("Done..." + k.getId());
} catch (Exception e) {
e.printStackTrace();
}
User
simonsu@...
File: DatastoreExample.java

62. DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Key userKey = KeyFactory.createKey("User", "simonsu.mail@gmail.com");
Entity job = new Entity("Jobs", userKey);
job.setProperty("name", "engineer");
job.setProperty("start", new Date());
try {
Key k = ds.put(job);
out.println("Done..." + k.getId());
} catch (Exception e) {
e.printStackTrace();
}
User
Jobs
File: DatastoreExample2.java

63. DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Query q = new Query("Jobs");
Key ancestor = KeyFactory.createKey("User", "simonsu.mail@gmail.com");
q.setAncestor(ancestor);
PreparedQuery results = ds.prepare(q);
Iterator iter = results.asIterable().iterator();
while(iter.hasNext()) {
Entity ent = (Entity) iter.next();
out.println(ent);
}
File: DatastoreExample2.java

66. ●
●
●
●

67. Memcache

68. ●
●
●
●
●

69. MemcacheService cache = MemcacheServiceFactory.getMemcacheService();
cache.setErrorHandler(
ErrorHandlers.getConsistentLogAndContinue(Level.INFO));
// read from cache
value = (byte[]) cache.get(key);
// save to cache
if (value == null) {
// ........
cache.put(key, value);
}
# DatastoreExample.java

70. // Using the asynchronous cache
AsyncMemcacheService asyncCache =
MemcacheServiceFactory.getAsyncMemcacheService();
asyncCache.setErrorHandler(
ErrorHandlers.getConsistentLogAndContinue(Level.INFO));
// read from cache
Future<Object> futureValue = asyncCache.get(key);
// ... do other work in parallel to cache retrieval
value = (byte[]) futureValue.get();

71. if (value == null) {
// get value from other source
// ........
// asynchronously populate the cache
// Returns a Future<Void> which can be used to block until completion
asyncCache.put(key, value);
}

72. import java.util.Collections;
import net.sf.jsr107cache.*;
Cache cache;
try {
CacheFactory cacheFactory =
CacheManager.getInstance().getCacheFactory();
cache = cacheFactory.createCache(Collections.emptyMap());
} catch (CacheException e) {
// ...
}

73. String key; // ...
byte[] value; // ...
// Put the value into the cache.
cache.put(key, value);
// Get the value from the cache.
value = (byte[]) cache.get(key);

75. ●
●
●

76. ●
●
●

77. Task Queue

78. ●
●
●
●
○
○
○
○

79. ●
○
○
●
○
■
■
○
■
■
■

80. <queue-entries>
<queue>
<name>Queue-Name</name>
<rate>1/s</rate>
...
</queue>
<queue>
<name>Queue-Name2</name>
<mode>pull</mode>
...
</queue>
</queue-entries>
Push Queue
Pull Queue

82. Queue queue = QueueFactory.getDefaultQueue();
//named queue:
//QueueFactory.getQueue("Qname");
TaskOptions task =
TaskOptions.Builder
.withUrl("/path-to-my-worker")
.param(key, value);
queue.add(task);
Reachable Servlet

83. ●

84. ●
●
●
●
○
○
○
○
○
●

85. DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Queue queue = QueueFactory.getDefaultQueue();
try {
Transaction txn = ds.beginTransaction();
// … other operations
queue.add(TaskOptions.Builder.withUrl("/path-to-my-worker"));
// … other operations
txn.commit();
} catch (DatastoreFailureException e) {
// … exception handle
}

86. Queue queue =
QueueFactory.getQueue("queue_name");
TaskOptions task =
TaskOptions.Builder
.withMethod(TaskOptions.Method.PULL)
.payload(payload);
queue.add(task);

87. Queue queue =
QueueFactory.getQueue("queue_name");
tasks = q.leaseTasks(
duration,
TimeUnit.SECONDS,
how-many);

88. //delete named task
queue.deleteTask("task_ name");
//purge all tasks from queue
QueueFactory.getQueue("queue_name").purge();

89. <queue>
<name>pull-queue1</name>
<mode>pull</mode>
<acl>
<user-email>me@gmail.com</user-email>
<writer-email>you@gmail.com</writer-email>
<writer-email>she@gmail.com</writer-email>
</acl>
</queue>

90. Cron

91. <cronentries>
<cron>
<url>/recache</url>
<description>
Put your description here
</description>
<schedule>every 5 minutes</schedule>
</cron>
</cronentries>

92. ●
●
●
●
●
●
●
●

94. ●
●

95. ●
●
●
●
●
●
●
●

97. ●
●
●
●

98. ●
●

99. $ gcloud auth login
$ gcloud config set project [my-project-id]
$ gsutil mb gs://[my-bucket]
$ gsutil ls gs://[my-bucket]

101. $ gcloud sql -h
$ gcloud sql instances create [inst-name]
$ gcloud sql instances list
$ gcloud sql instances import instance-name --uri
gs://your-bucket/sql-dump-file.gz
$ gcloud sql instances patch --assign-ip [inst-name]
$ gcloud sql instances get [inst-name]

102. $ gcloud sql instances set-root-password test-sql
-p [password]
$ gcloud sql instances patch test-sql
--authorized-networks=[your-ip-address]

103. http://www.mysql.com/products/workbench/

104. ●
●

105. Using Cloud Storage

107. BlobstoreService blobstoreService =
BlobstoreServiceFactory.getBlobstoreService();
UploadOptions uploadOptions =
UploadOptions.Builder.withGoogleStorageBucketName(
"BUCKET-NAME"); //Name without “gs://”
String uploadUrl = blobstoreService.createUploadUrl(
"/callback-url", uploadOptions);
resp.getWriter().println("<form action=" + uploadUrl
+ " method="post" enctype="multipart/form-data">");
resp.getWriter().println("<input type="file" name="field-name"");

108. public void doPost(HttpServletRequest req,
HttpServletResponse resp) throws IOException {
BlobstoreService blobstoreService =
BlobstoreServiceFactory.getBlobstoreService();
Map<String, List<FileInfo>> blobs = blobstoreService.getFileInfos(req);
List<FileInfo> keys = blobs.get("field-name"); //input file name on
form
if (keys != null && keys.size() > 0) {
FileInfo fileKey = keys.get(0);
String objectUploaded = fileKey.getGsObjectName();
…. (Save the objectUploaded mapping to datastore or database)

109. public void doGet(HttpServletRequest req, HttpServletResponse resp){
BlobstoreService blobstoreService =
BlobstoreServiceFactory.getBlobstoreService();
BlobKey blobKey =
blobstoreService.createGsBlobKey("/gs/<bucket>/<object>");
try {
blobstoreService.serve(blobKey, resp);
} catch (IOException e){
e.printStackTrace();
}
….

110. 補充

111. ●
●

112. Connect Cloud SQL
SystemProperty.Environment.Value.Production) {
Class.forName("com.mysql.jdbc.GoogleDriver");
String url= "jdbc:google:mysql://instance-name/";
conn = DriverManager.getConnection(url);
try {
conn.createPreparedStatement…
...

113. ●
●
○
○
○

114. ●
●
●

115. <?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>mitac-cp300</application>
<version>eord</version>
….
<use-google-connector-j>true</use-google-connector-j>
….
</appengine-web-app>

116. ●
●
●
●

117. Connection conn = null;
if (SystemProperty.environment.value() ==
SystemProperty.Environment.Value.Production) {
// app is running in production
Class.forName("com.mysql.jdbc.GoogleDriver");
String url= "jdbc:google:mysql://instance-name/";
conn = DriverManager.getConnection(url);
...
} else {
// app is running on localhost
Class.forName("com.mysql.jdbc.Driver");
String url= "jdbc:mysql://db-ip-address/";
conn = DriverManager.getConnection(url);
...
}
conn.~

118. DriverManager.registerDriver(new AppEngineDriver());
conn = DriverManager.getConnection(connUrl);
String sql = "SELECT count(*) as total FROM " + tableName + "";
PreparedStatement stmt = conn.prepareStatement(sql);
ResultSet results = stmt.executeQuery();
while (results.next()) {
int total = results.getInt("total");
…
}

119. ●
●
●
●

120. Q & A