Amazon SageMaker 紹介 & ハンズオン（2018/07/25 実施）
•
16 gostaram•9,786 visualizações
2018/07/03 に実施した Amazon SageMaker ハンズオンの資料です． https://pages.awscloud.com/AmazonSageMaker20180725-jp.html
Recomendados
Recomendados
Mais conteúdo relacionado
Mais de Amazon Web Services Japan
Mais de Amazon Web Services Japan (20)
Último
Último (20)
Amazon SageMaker 紹介 & ハンズオン（2018/07/25 実施）
- 3. • • • ⎼ Amazon Athena ⎼ AWS Glue ⎼ Amazon SageMaker
- 11. SageMaker
- 12. SageMaker 5
- 14. Jupyter Notebook • • • 4 ml.t2, ml.m4, ml.p2, ml.p3 • VPC ENI VPC
- 15. CreateTrainingJob API Docker • • 2 • m4, m5, c4, c5, p2, p3 • S3 https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html
- 18. / : SageMaker Tensorflow / MXNet / Chainer github • SageMaker pull / • instance_type ’local’ https://github.com/aws/sagemaker-python-sdk#local-mode https://github.com/aws/sagemaker-tensorflow-containers https://github.com/aws/sagemaker-mxnet-containers https://github.com/aws/sagemaker-chainer-container
- 19. CreateEndpoint API Docker • AB • • https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateEndpointConfig.html https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateEndpoint.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/API_UpdateEndpoint.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/API_UpdateEndpointWeightsAndCapacities.html
- 20. : Batch Transform Job • CreateTransformJob API • • S3 S3
- 21. : AWS Greengrass SageMaker / . .- . / - ./ .- . • SageMaker S3 • DeepLens
- 22. SageMaker
- 23. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
- 24. SageMaker 2 AWS SDK • • SageMaker SDK • • AWS SDK SageMaker SDK AWS SDK scikit-learn • Python Spark • Jupyter Notebook https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark
- 25. create-endpoint create-notebook-instance create-training-job delete-endpoint delete-notebook-instance describe-endpoint describe-notebook-instance … estimator = TensorFlow(…) estimator.set_hyperparameters(…) estimator.fit(…) predictor = estimator.deploy(…) Predictor.predict(…)
- 28. SageMaker
- 29. • Linear Learner • Factorization Machines • XGBoost • Image Classification • seq2seq • K-means • k-NN • PCA • LDA • Neural Topic Model • DeepAR Forecasting • BlazingText (word2vec) • Random Cut Forest • Object Detection New! https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html New!
- 30. Image Classification ResNet • ResNet CNN • ILSVRC 2015 1 • ImageNet • use_pretrained_model 1 0 https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/imageclassification_caltech/Image-classification-transfer-learning.ipynb https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/He_Deep_Residual_Learning_CVPR_2016_paper.pdf https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/image-classification.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html
- 31. Object Detection SSD • • VGG or ResNet • ImageNet • use_pretrained_model 1 0 https://github.com/awslabs/amazon-sagemaker-examples/blob/master/introduction_to_amazon_algorithms/object_detection_pascalvoc_coco/object_detection_image_json_format.ipynb https://arxiv.org/pdf/1512.02325.pdf https://docs.aws.amazon.com/sagemaker/latest/dg/object-detection.html https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/IC-Hyperparameter.html
- 32. SageMaker S3
- 36. Tensorflow / PyTorch / Chainer / MXNet
- 39. Tensorflow • model_fn • estimator_fn tensorflow.estimator • keras_model_fn tf.keras • train_input_fn • eval_input_fn • serving_input_fn • input_fn • output_fn https://docs.aws.amazon.com/sagemaker/latest/dg/tf-training-inference-code-template.html
- 40. MXNet • train • save • model_fn: • transform_fn: • https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet-training-inference-code-template.html
- 51. SageMaker 3 1: • SageMaker 2: GPU • AWS
- 52. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK
- 53. AWS SDK SageMaker SDK • SageMaker SDK Jupyter Notebook • AWS SDK SageMaker SDK AWS SDK
- 54. ML • SageMaker 1 ML • • 0.14 USD/GB/ • • 0.016 USD/GB https://aws.amazon.com/jp/sagemaker/pricing/
- 55. 3 SageMaker Example Notebooks • https://github.com/awslabs/amazon-sagemaker-examples SageMaker SDK • https://github.com/aws/sagemaker-python-sdk (Doc : https://readthedocs.org/projects/sagemaker/) SageMaker • https://docs.aws.amazon.com/ja_jp/sagemaker/latest/dg/whatis.html
- 57. Tensorflow • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras/tensorflow_abalone_age_predictor_using_keras.ipynb • Tensorflow Tensorboard • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_resnet_cifar10_with_tensorboard • Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker-python- sdk/tensorflow_distributed_mnist/tensorflow_distributed_mnist.ipynb • Keras + Tensorflow • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python- sdk/tensorflow_abalone_age_predictor_using_keras • MXNet • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk
- 58. • scikit-learn SageMaker SageMaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.ipynb • SageMaker R • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/install_r_kernel/example_r_notebook.ipynb • SageMaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/handling_kms_encrypted_data/handling_kms_encrypted_data.ipynb • XGBoost SageMaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/xgboost_bring_your_own_model/xgboost_bring_your_own_model.ipynb • Tensorflow Sagemaker • https://github.com/awslabs/amazon-sagemaker- examples/blob/master/advanced_functionality/tensorflow_iris_byom/tensorflow_BYOM_iris.ipynb • Redshift MXNet R • https://github.com/awslabs/amazon-sagemaker-examples/tree/master/advanced_functionality
- 59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Web Services Japan, K. K. Amazon SageMaker
- 60. (SAMEJIMA, Masaki) SageMaker AWS Amazon SageMaker
- 61. SageMaker https://aws.amazon.com/jp/console/ (Chrome, Firefox IE, Safari ) SageMaker
- 63. SageMaker
- 65. IAM S3 S3
- 66. • VPC, •
- 67. Jupyter Notebook • InService Jupyter Notebook • New → Terminal
- 68. Terminal cd SageMaker/ wget http://bit.ly/sagemaker-notebooks -O sagemaker-notebooks.zip unzip sagemaker-notebooks.zip O( )
- 70. • • Random Cut Forest • XGBoost MNIST • Chainer on SageMaker • Chainer MLP MNIST
- 71. : Random Cut Forest Random Forest (1) (2) …… ≤ 1 > 1 ≤ 0.2 > 0.2
- 72. : XGboost …… ≤ 1 > 1 ≤ 0.2 > 0.2 ≤ 0.8 > 1 ≤ 0.6 > 0.6