Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)

1. Building Dynamic Pipelines in Azure Data Factory Cathrine Wilhelmsen SQLSatOslo · August 31st, 2019

4. @cathrinew cathrinew.net

9. Dynamic Data Pipelines

15. Parameters

27. Expressions and Functions

31. Combining Strings

36. DEMO Let's parameterize some pipelines!

37. ForEach Loops

40. Array Items

44. Object Items

49. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Limitations when using Foreach Loops Nested ForEach Loops are not supported Maximum number of items: 100 000 Maximum number of parallel executions: 50 Default number of parallel executions: 20

50. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Best Practices for using ForEach Loops Executing one activity? Add directly in ForEach Executing multiple activities? Create new pipeline and execute from ForEach

51. DEMO Let's create some loops!

52. Lookups

55. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Lookup Output { "firstRow" : { "Column1" : "Value", "Column2" : "Value" } } { "count" : "2", "value" : [ { "Column1" : "Value", "Column2" : "Value" }, { "Column1" : "Value", "Column2" : "Value" } ] }

58. © 2018 Cathrine Wilhelmsen (hi@cathrinew.net) Lookup Output: All Rows { "count" : "2", "value" : [ { "Column1" : "Value", "Column2" : "Value" }, { "Column1" : "Value", "Column2" : "Value" } ] } value

60. DEMO Let's add a lookup to control our loop!

61. Controls and Conditionals

66. DEMO Let's create some if statements!

71. DEMO Let's check for file existence!

Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)

Esté a ler uma amostra.

Confira estes a seguir

Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo)

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a Building Dynamic Pipelines in Azure Data Factory (SQLSaturday Oslo) (13)

Mais de Cathrine Wilhelmsen (15)

Mais recentes (20)