O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

Exploring Google APIs with Python

Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Exploring Google APIs
with Python
Wesley Chun - @wescpy
Developer Advocate, Google
Adjunct CS Faculty, Foothill College
De...
Agenda
1
Introduction to
Google APIs
2
Google Cloud
APIs overview
3
Serverless tools +
code samples
4
Inspiration
5
Wrap-u...
Cloud/GCP console
console.cloud.google.com
● Hub of all developer activity
● Applications == projects
○ New project for ne...
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Carregando em…3
×

Confira estes a seguir

1 de 31 Anúncio

Exploring Google APIs with Python

Baixar para ler offline

Half-hour tech talk given at user groups or technical conferences to introducing developers to integrating with Google (Cloud) APIs from Python .

ABSTRACT
Want to integrate Google technologies into the web+mobile apps that you build? Google has various open source libraries & developer tools that help you do exactly that. Users who have run into roadblocks like authentication or found our APIs confusing/challenging, are welcome to come and make these non-issues moving forward. Learn how to leverage the power of Google technologies in the next apps you build!!

Half-hour tech talk given at user groups or technical conferences to introducing developers to integrating with Google (Cloud) APIs from Python .

ABSTRACT
Want to integrate Google technologies into the web+mobile apps that you build? Google has various open source libraries & developer tools that help you do exactly that. Users who have run into roadblocks like authentication or found our APIs confusing/challenging, are welcome to come and make these non-issues moving forward. Learn how to leverage the power of Google technologies in the next apps you build!!

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a Exploring Google APIs with Python (20)

Anúncio

Mais recentes (20)

Anúncio

Exploring Google APIs with Python

  1. 1. Exploring Google APIs with Python Wesley Chun - @wescpy Developer Advocate, Google Adjunct CS Faculty, Foothill College Developer Advocate, Google Cloud ● Mission: enable current and future developers everywhere to be successful using Google Cloud and other Google developer tools & APIs ● Focus: GCP serverless (App Engine, Cloud Functions, Cloud Run); higher education, Google Workspace, GCP AI/ML APIs; multi-product use cases ● Content: speak to developers globally; make videos, create code samples, produce codelabs (free, self-paced, hands-on tutorials), publish blog posts About the speaker Previous experience / background ● Software engineer & architect for 20+ years ○ Yahoo!, Sun, HP, Cisco, EMC, Xilinx ○ Original Yahoo!Mail engineer/SWE ● Technical trainer, teacher, instructor ○ Taught Math, Linux, Python since 1983 ○ Private corporate trainer ○ Adjunct CS Faculty at local SV college ● Python community member ○ Popular Core Python series author ○ Python Software Foundation Fellow ● AB (Math/CS) & CMP (Music/Piano), UC Berkeley and MSCS, UC Santa Barbara ● Adjunct Computer Science Faculty, Foothill College (Silicon Valley)
  2. 2. Agenda 1 Introduction to Google APIs 2 Google Cloud APIs overview 3 Serverless tools + code samples 4 Inspiration 5 Wrap-up 01 Introduction to Google APIs Why are you here?
  3. 3. Cloud/GCP console console.cloud.google.com ● Hub of all developer activity ● Applications == projects ○ New project for new apps ○ Projects have a billing acct ● Manage billing accounts ○ Financial instrument required ○ Personal or corporate credit cards, Free Trial, and education grants ● Access GCP product settings ● Manage users & security ● Manage APIs in devconsole ● View application statistics ● En-/disable Google APIs ● Obtain application credentials Using Google APIs goo.gl/RbyTFD API manager aka Developers Console (devconsole) console.developers.google.com
  4. 4. Two different client library "styles" ● "Platform-level" client libraries (lower level) ○ Supports multiple products as a "lowest-common denominator" ○ Manage API service endpoints (setup & use) ○ Manage authorization (API keys, OAuth client IDs, service accounts) ○ Google Workspace, Google Analytics, YouTube, Google Ads APIs, etc. ○ Install: developers.google.com/api-client-library ● "Product-level" client libraries (higher level) ○ Custom client libraries made specifically for each product ○ Managing API service endpoints & security mostly taken care of ○ Only need to create a "client" to use API services ○ Install (Cloud/GCP & Firebase): cloud.google.com/apis/docs/cloud-client-libraries ○ Install (Maps): developers.google.com/places/web-service/client-library ● Some Google APIs families support both, e.g., Cloud Google APIs client libraries for common languages; demos in developers.google.com/api- client-library cloud.google.com/apis/docs /cloud-client-libraries
  5. 5. Three different authz credential types ● Simple: API keys (to access public data) ○ Simplest form of authorization: an API key; tied to a project ○ Allows access to public data ○ Do not put in code, lose, or upload to GitHub! (can be restricted however) ○ Supported by: Google Maps, (some) YouTube, (some) GCP, etc. ● Authorized: OAuth client IDs (to access data owned by [human] user) ○ Provides additional layer of security via OAuth2 (RFC 6749) ○ Owner must grant permission for your app to access their data ○ Access granularity determined by requested permissions (user scopes) ○ Supported by: Google Workspace, (some) YouTube, (some) GCP, etc. ● Authorized: service accounts (to access data owned by an app/robot user) ○ Provides additional layer of security via OAuth2 or JWT (RFC 7519) ○ Project owning data grants permission implicitly; requires public-private key-pair ○ Access granularity determined by Cloud IAM permissions granted to service account key-pair ○ Supported by: GCP, (some) Google Workspace, etc. SIMPLE AUTHORIZED Which do you choose?
  6. 6. OAuth2 or API key HTTP-based REST APIs 1 HTTP 2 Google APIs request-response workflow ● Application makes request ● Request received by service ● Process data, return response ● Results sent to application (typical client-server model) 02 Google Cloud APIs overview
  7. 7. Running Code: Compute Engine > Google Compute Engine delivers configurable virtual machines of all shapes and sizes, from "micro" to 416 vCPUs, 11.776 TB RAM, 256 TB HDD or SSD disk; GPUs & TPUs (Debian, CentOS, CoreOS, SUSE, Red Hat Enterprise Linux, Ubuntu, FreeBSD; Windows Server 2008 R2, 2012 R2, 2016, 1803, 1809, 1903/2019, 1909) cloud.google.com/compute
  8. 8. Running Code: App Engine Got a great app idea? Now what? VMs? Operating systems? Big disk? Web servers? Load balancing? Database servers? Autoscaling? With Google App Engine, you don't think about those. Just upload your code; we do everything else. > cloud.google.com/appengine Running Code: Cloud Functions Don't have an entire app? Just want to deploy small microservices or "RPCs" online globally? That's what Google Cloud Functions are for! (+Firebase version for mobile apps) cloud.google.com/functions firebase.google.com/products/functions
  9. 9. Running Code: Cloud Run Got a containerized app? Want its flexibility along with the convenience of serverless that's fully-managed plus auto-scales? Google Cloud Run is exactly what you're looking for! Need custom HW? Cloud Run on GKE cloud.google.com/run Managed containers: Kubernetes Engine Got a containerized application? Google Kubernetes Engine is an enterprise-grade, fully-managed container orchestration service. cloud.google.com/kubernetes-engine
  10. 10. Storing Data: Cloud Storage & Cloud Filestore cloud.google.com/storage cloud.google.com/filestore Storing Data: Cloud SQL SQL servers in the cloud High-performance, fully-managed 600MB to 416GB RAM; up to 64 vCPUs Up to 10 TB storage; 40,000 IOPS Types: MySQL Postgres SQLServer (2019) cloud.google.com/sql
  11. 11. Storing Data: Cloud Firestore The best of both worlds: the next generation of Cloud Datastore (w/product rebrand) plus features from the Firebase realtime database (For choosing between Firebase & Cloud Firestore: see firebase.google.com/docs/firestore/rtdb-vs-firestore; for choosing between Cloud Datastore & Firestore: see cloud.google.com/datastore/docs/firestore-or-datastore) cloud.google.com/firestore Storing and Analyzing Data: BigQuery Google BigQuery is a fast, highly scalable, fully-managed data warehouse in the cloud for analytics with built-in machine learning (BQML); issue SQL queries across multi-terabytes of data cloud.google.com/bigquery
  12. 12. Passing Data & Events: Pub/Sub Google Pub/Sub: a fast, highly scalable, fully-managed multi fan-in/fan-out publisher-subscriber queuing system for messaging & event ingestion (and processing) cloud.google.com/pubsub Machine Learning: Cloud Translation Access Google Translate programmatically through this API; translate an arbitrary string into any supported language using state-of-the-art Neural Machine Translation cloud.google.com/translate
  13. 13. Machine Learning: Cloud Natural Language Google Cloud Natural Language API reveals the structure and meaning of text, performing sentiment analysis, content classification, entity extraction, and syntactical structure analysis; multi-lingual cloud.google.com/language Machine Learning: Cloud Vision & Video Intelligence Google Cloud Vision & Video Intelligence APIs enable developers to extract metadata & understand the content of images & videos, making them searchable & discoverable. cloud.google.com/vision cloud.google.com/video-intelligence
  14. 14. Google Workspace Top-level documentation and comprehensive developers overview video at developers.google.com/gsuite (formerly G Suite and Google Apps) APIs 03 Serverless tools + code samples Using Google APIs w/Python
  15. 15. > Google Compute Engine configurable VMs of all shapes & sizes, from "micro" to 416 vCPUs, 11.776 TB RAM, 256 TB HDD/SSD plus Google Cloud Storage for data lake "blobs" (Debian, CentOS, CoreOS, SUSE, Red Hat Enterprise Linux, Ubuntu, FreeBSD; Windows Server 2008 R2, 2012 R2, 2016, 1803, 1809, 1903/2019, 1909) cloud.google.com/compute cloud.google.com/storage Yeah, we got VMs & big disk… but why*? Serverless: what & why ● What is serverless? ○ Misnomer ○ "No worries" ○ Developers focus on writing code & solving business problems* ● Why serverless? ○ Fastest growing segment of cloud... per analyst research*: ■ $1.9B (2016) and $4.25B (2018) ⇒ $7.7B (2021) and $14.93B (2023) ○ What if you go viral? Autoscaling: your new best friend ○ What if you don't? Code not running? You're not paying. * in USD; source:Forbes (May 2018), MarketsandMarkets™ & CB Insights (Aug 2018)
  16. 16. Why does App Engine exist? ● Focus on app not DevOps ○ Web app ○ Mobile backend ○ Cloud service ● Enhance productivity ● Deploy globally ● Fully-managed ● Auto-scaling ● Pay-per-use ● Familiar languages ● Test w/local dev server Hello World (Python "MVP") app.yaml runtime: python38 main.py from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return 'Hello World!' requirements.txt Flask>=1.1.2 Deploy: $ gcloud app deploy Access globally: PROJECT_ID.appspot.com cloud.google.com/appengine/docs/standard/python3/quickstart
  17. 17. Why does Cloud Functions exist? ● Don't have entire app? ○ No framework "overhead" (LAMP, MEAN...) ○ Deploy microservices ● Event-driven ○ Triggered via HTTP or background events ■ Pub/Sub, Cloud Storage, Firebase, etc. ○ Auto-scaling & highly-available; pay per use ● Flexible development environment ○ Cmd-line or developer console (in-browser) ○ Develop/test locally with Functions Framework ● Cloud Functions for Firebase ○ Mobile app use-cases ● Available runtimes ○ JS/Node.js 8, 10, 12, 14 ○ Python 3.7, 3.8, 3.9 ○ Go 1.11, 1.13 ○ Java 11 ○ Ruby 2.6, 2.7 ○ .NET Core 3.1 main.py def hello_world(request): return 'Hello World!' Deploy: $ gcloud functions deploy hello --runtime python38 --trigger-http Access globally (curl): $ curl REGION-PROJECT_ID.cloudfunctions.net/hello Access globally (browser): https://REGION-PROJECT_ID.cloudfunctions.net/hello Hello World (Python "MVP") cloud.google.com/functions/docs/quickstart-python
  18. 18. The rise of containers... ● Any language ● Any library ● Any binary ● Ecosystem of base images ● Industry standard FLEXIBILITY “We can’t be locked in.” “How can we use existing binaries?” “Why do I have to choose between containers and serverless?” “Can you support language _______ ?” Serverless inaccessible for some... CONVENIENCE
  19. 19. Cloud Run: code, build, deploy .js .rb .go .sh .py ... ● Any language, library, binary ○ HTTP port, stateless ● Bundle into container ○ Build w/Docker OR ○ Google Cloud Build ○ Image ⇒ Container Registry ● Deploy to Cloud Run (managed or GKE) ● GitOps: (CI/)CD Push-to-deploy from Git State HTTP Hello World (Python "MVP") main.py import os from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello World!' if __name__ == '__main__': app.run(debug=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080))) cloud.google.com/run/docs/quickstarts/build-and-deploy requirements.txt Flask>=1.1.2
  20. 20. Hello World (Python "MVP") Dockerfile FROM python:3-slim WORKDIR /app COPY . . RUN pip install -r requirements.txt CMD ["python", "main.py"] .dockerignore Dockerfile README.md *.pyc *.pyo .git/ __pycache__ Build (think docker build and docker push) then deploy (think docker run): $ gcloud builds submit --tag gcr.io/PROJ_ID/IMG_NAME $ gcloud run deploy SVC_NAME --image gcr.io/PROJ_ID/IMG_NAME OR… Build and Deploy (1-line combo of above commands): $ gcloud run deploy SVC_NAME --source . Deploy (think docker push): $ gcloud run deploy --image gcr.io/PROJ_ID/IMG_NAME --platform managed Access globally: SVC_NAME-HASH-REG_ABBR.a.run.app Docker & Dockerfile OPTIONAL!! BigQuery: querying Shakespeare words from google.cloud import bigquery TITLE = "The most common words in all of Shakespeare's works" QUERY = ''' SELECT LOWER(word) AS word, sum(word_count) AS count FROM `bigquery-public-data.samples.shakespeare` GROUP BY word ORDER BY count DESC LIMIT 10 ''' rsp = bigquery.Client().query(QUERY).result() print('n*** Results for %r:n' % TITLE) print('t'.join(col.name.upper() for col in rsp.schema)) # HEADERS print('n'.join('t'.join(str(x) for x in row.values()) for row in rsp)) # DATA
  21. 21. Top 10 most common Shakespeare words $ python bq_shake.py *** Results for "The most common words in all of Shakespeare's works": WORD COUNT the 29801 and 27529 i 21029 to 20957 of 18514 a 15370 you 14010 my 12936 in 11722 that 11519 ● BigQuery public data sets: cloud.google.com/bigquery/public-data ● BQ sandbox (1TB/mo free): cloud.google.com/bigquery/docs/sandbox ● Other public data sets: cloud.google.com/public-datasets (Google Cloud), research.google/tools/datasets (Google Research), and Kaggle (kaggle.com) ● COVID-19 ○ How to use our data sets (see blog post) ○ JHU Coronavirus COVID-19 Global Cases data set ○ List of all COVID-19 data sets ● Cloud Life Sciences API: cloud.google.com/life-sciences (see blog post) ● Cloud Healthcare API: cloud.google.com/healthcare (see blog post) BigQuery & public data sets Spring 2020
  22. 22. from google.cloud import vision image_uri = 'gs://cloud-samples-data/vision/using_curl/shanghai.jpeg' client = vision.ImageAnnotatorClient() image = vision.types.Image() image.source.image_uri = image_uri response = client.label_detection(image=image) print('Labels (and confidence score):') print('=' * 30) for label in response.label_annotations: print(label.description, '(%.2f%%)' % (label.score*100.)) Vision: label annotation/object detection $ python3 label-detect.py Labels (and confidence score): ============================== People (95.05%) Street (89.12%) Mode of transport (89.09%) Transport (85.13%) Vehicle (84.69%) Snapshot (84.11%) Urban area (80.29%) Infrastructure (73.14%) Road (72.74%) Pedestrian (68.90%) Vision: label annotation/object detection g.co/codelabs/vision-python
  23. 23. ● Not just for conversations ● Create microservice utilities ● Build chat bots to... ○ Automate workflows ○ Query for information ○ Other heavy-lifting ● Plain text or rich UI "cards" ● Very flexible ("any") development environment ○ POST to HTTP port Hangouts Chat bots (bot framework & API) "Hello World" (echo bot) Python+Flask: GAE or other hosting from flask import Flask, request, json app = Flask(__name__) @app.route('/', methods=['POST']) def on_event(): event = request.get_json() msg = {} if event['type'] == 'MESSAGE': text = 'Hi {}. You sent: {}'.format( event['user']['displayName'], event['message']['text']) msg = {'text': text} return json.jsonify(msg) Hangouts Chat bots goo.gl/jt3FqK
  24. 24. Search YouTube for videos from __future__ import print_function from googleapiclient import discovery from settings import API_KEY QUERY = 'python -snake' trim = lambda x, ct: ('%s%s' % (x[:ct], '...' if len(x)>ct else '')).ljust(ct+3) print('n** Searching for %r videos...' % QUERY) YOUTUBE = discovery.build('youtube', 'v3', developerKey=API_KEY) res = YOUTUBE.search().list(q=QUERY, type='video', part='id,snippet').execute().get('items', []) for item in res: print('http://youtu.be/%st%s' % ( trim(item['id']['videoId'], 24), trim(item['snippet']['title'], 48))) Maps APIs geocoding & places queries import googlemaps from settings import API_KEY GMAPS = googlemaps.Client(key=API_KEY) print('n** Geocode address:') rsp = GMAPS.geocode('1600 Amphitheatre Pkwy 94043') latlong = rsp[0]['geometry']['location'] print('tGeocode:', latlong['lat'], ',', latlong['lng']) print('n** Address lookup:') rsp = GMAPS.reverse_geocode((37.4222934, -122.0841409)) print('tAddress:', rsp[0]['formatted_address']) print('n** Place query:') pl_id = GMAPS.find_place('Villa Tugendhat', input_type='textquery')['candidates'][0]['place_id'] place = GMAPS.place(pl_id)['result'] print('tPlace:t', place['name']) print('tAddress:t', place['formatted_address']) print('tWebsite:t', place['website']) print('tMaps URL:t', place['url']) $ python3 maps-demo-pub.py ** Geocode address: Geocode: 37.4222934 , -122.0841409 ** Address lookup: Address: 1600 Amphitheatre Pkwy, Mountain View, CA 94043, USA ** Place query: Place: Villa Tugendhat Address: Černopolní 45, 613 00 Brno, Czechia Website: tugendhat.eu Maps URL: maps.google.com/?cid=5889127146370224362 Google Maps APIs ● Directions API ● Distance Matrix API ● Elevation API ● Geocoding API ● Geolocation API ● Time Zone API ● Roads API ● Places API ● Maps Static API
  25. 25. Other Google APIs & platforms ● Firebase (mobile development platform + RT DB; ML Kit) ○ firebase.google.com & firebase.google.com/docs/ml-kit ● Google Data Studio (data visualization, dashboards, etc.) ○ datastudio.google.com/overview ○ goo.gle/datastudio-course ● Actions on Google/Assistant/DialogFlow (voice apps) ○ developers.google.com/actions ● YouTube (Data, Analytics, and Livestreaming APIs) ○ developers.google.com/youtube ● Google Maps (Maps, Routes, and Places APIs) ○ developers.google.com/maps ● Flutter (native apps [Android, iOS, web] w/1 code base[!]) ○ flutter.dev 04 All of Cloud (inspiration) Build powerful solutions with GCP and G Suite
  26. 26. Cloud image processing workflow Archive and analyze Google Workspace (formerly G Suite) images with GCP
  27. 27. Cloud Vision Google Workspac e (formerly G Suite) GCP Cloud image processing workflow Cloud Storage Drive Sheets Archive image Categorize image Record results
  28. 28. Cloud image processing workflow def drive_get_file(fname): rsp = DRIVE.files().list(q="name='%s'" % fname).execute().get['files'][0] fileId, fname, mtype = rsp['id'], rsp['name'], rsp['mimeType'] blob = DRIVE.files().get_media(fileId).execute() return fname, mtype, rsp['modifiedTime'], blob def gcs_blob_upload(fname, bucket, blob, mimetype): body = {'name': fname, 'uploadType': 'multipart', 'contentType': mimetype} return GCS.objects().insert(bucket, body, blob).execute() def vision_label_img(img, top): body = {'requests': [{'image': {'content': img}, 'features': [{'type': 'LABEL_DETECTION', 'maxResults': top}]}]} rsp = VISION.images().annotate( body=body).execute().get['responses'][0] return ', '.join('%s (%.2f%%)' % (label['description'], label['score']*100.) for label in rsp['labelAnnotations']) def sheet_append_rows(sheet, rows): rsp = SHEETS.spreadsheets().values().append( spreadsheetId=sheet, range='Sheet1', body={'values': rows}).execute() return rsp.get('updates').get('updatedCells') def main(fname, bucket, sheet_id, folder, top): fname, mtype, ftime, data = drive_get_img(fname) gcs_blob_upload(fname, bucket, data, mtype) vision_label_img(data, top) sheet_append_row([sheet_id, fname, mtype, ftime, len(data), rsp]) API method calls in Bold Driver calls in Bold Italics ● Project goal: Imagine enterprise use cases Workspace & GCP; got one! ● Specific goals: free-up highly-utilized resource, archive data to colder/cheaper storage, analyze images, generate report for mgmt ● Download image binary from Google Drive ● Upload object to Cloud Storage bucket ● Send payload for analysis by Cloud Vision ● Write back-up location & analysis results into Google Sheets ● Blog post: goo.gle/3nPxmlc (original post); Cloud X-post ● Codelab: self-paced (1+-hour) hands-on tutorial ● g.co/codelabs/drive-gcs-vision-sheets ● Application source code ● github.com/googlecodelabs/analyze_gsimg App summary
  29. 29. 05 Wrap-up Summary & resources Session Summary ● Google provides more than just apps ○ We're more than search, YouTube, Android, Chrome, and Gmail ○ Much of our tech available to developers through our APIs ● Tour of Google (Cloud) APIs & developer tools ○ Workspace: not just a set of productivity apps… you can code them too! ○ GCP: compute, storage, networking, security, data & machine learning tools ■ Serverless: frees developers from infrastructure ■ So you can focus on building solutions ● Interesting possibilities using ALL of Cloud (GCP + Workspace) ● Also explore other Google developer products & APIs
  30. 30. ● Documentation ○ GCP: cloud.google.com/{docs,appengine,functions,run,vision,automl,translate,language, speech,texttospeech,video-intelligence,firestore,bigquery,compute,storage,gpu,tpu} ○ Workspace: developers.google.com/{gsuite,drive,calendar,gmail,docs,sheets,slides,apps-script} ● Introductory "codelabs" ([free] self-paced, hands-on tutorials) ○ Workspace APIs: g.co/codelabs/gsuite-apis-intro (featuring Drive API) ○ Apps Script: g.co/codelabs/apps-script-intro ○ App Engine: codelabs.developers.google.com/codelabs/cloud-app-engine-python ○ Cloud Functions: codelabs.developers.google.com/codelabs/cloud-starting-cloudfunctions ○ Cloud Run: codelabs.developers.google.com/codelabs/cloud-run-hello-python3 ○ Others: g.co/codelabs (all Google codelabs) and gcplab.me (GCP-only) ● Videos: youtube.com/GoogleCloudPlatform (GCP) and goo.gl/JpBQ40 (Google Workspace) ● Code samples: github.com/GoogleCloudPlatform (GCP) and github.com/gsuitedevs (Google Workspace) ● Free trial (new users) and Always Free (daily/monthly tier) - cloud.google.com/free ● Know AWS/Azure? Compare w/GCP at cloud.google.com/docs/compare/{aws,azure} Resources (industry) Thank you! Questions? Wesley Chun @wescpy Progress bars: goo.gl/69EJVw Slides: bit.ly/2SHybQg

×