1.
@cathrinew | cathrinew.net

2.
Session Abstract

3.
@cathrinew
cathrinew.net

4.
You are an
Azure Data Factory
developer

5.
This is getting
pretty tedious…

6.
There has to be
a better way!

7.
Dynamic
Solutions

8.
Why would you use dynamic solutions?

9.
How dynamic should the solution be?

10.
How dynamic should the solution be?

11.
What can make a solution dynamic?
Parameters and Variables:
Pass input values and set or update values during runtime
Expressions and Functions:
Modify the content of values during runtime
Loops and Lookups:
Control logic and executions based on external configuration values

12.
Parameters
and Variables

13.
What are Parameters?

14.
How are Parameters passed?
trigger pipeline dataset
user
activity linked service
pipeline

15.
Parameters
@pipeline().parameters.
@dataset().
@linkedService().
ParameterName
ParameterName
ParameterName

16.
Parameters
@pipeline().parameters.
@dataset().
@linkedService().
ParameterName
ParameterName
ParameterName

17.
Parameters
@pipeline().parameters.
@pipeline().DataFactory
@pipeline().TriggerTime
ParameterName
and System Variables

18.
What are Variables?

19.
How are variables controlled?
Set Variable Append Variable

20.
Variables
@variables('VariableName')
@first(variables('VariableName'))
@last(variables('VariableName'))

21.
Variables
@variables('VariableName')
@first(variables('VariableName'))
@last(variables('VariableName'))

22.
Expressions
and Functions

23.
What are Expressions?
the @ symbol
"@toUpper(pipeline().parameters.FileName)"
"COLORS.CSV"

24.
What are Functions?
String:
Date:
Collection:
Logical:
Conversion:
Math:
concat, substring, startswith, endswith …
adddays, addhours, formatDateTime …
co s, f s , s , g h …
f, , o , qu s, ss, g …
createArray, y, jso , xm , …
add, sub, mul, v, m , m x, mo , …

25.
Combining Strings

26.
How to combine strings?
@concat()
@{…}

27.
What is String Concatenation?
@concat(
' TRUNCATE TABLE dbo. ',
pipeline().parameters.TableName
)

28.
What is String Interpolation?
TRUNCATE TABLE dbo.
@{pipeline().parameters.TableName}

29.
DEMO
Let's add some
parameters!

30.
Loops

31.
ForEach Loop Input
@item()

32.
Array Items

33.
Array Items

34.
Array Items
colors
@item()

35.
Array Items
o ch
o ch
o y
o y
o y
o y
o y
o y
@item()
parts
@item()
colors
@item()
sets

36.
Object Items

37.
Object Items

38.
Object Items
{ "Name": "colors", "Extension": "csv" }
@item()

39.
Object Items
{ "Name": "colors", "Extension": "csv" }
@item().Name

40.
Object Items
o ch
o ch
o y
o y
o y
o y
o y
o y
@item().Name
parts
@item().Name
colors
@item().Name
sets

41.
DEMO
Let's add a loop!

42.
Lookups

43.
Lookup

44.
Lookup Output

45.
Lookup Output

46.
Lookup Output: First Row Only
{
"Name" : "colors",
"Extension" : "csv"
}
@{activity('Lookup')
.output.firstRow}

47.
Lookup Output: First Row Only
{
"Name" : "colors",
"Extension" : "csv"
}
@{activity('Lookup')
.output.firstRow
.Name}

48.
Lookup Output: All Rows
@{activity('Lookup')
.output.value}
[
{
"Name" : "parts",
"Extension" : "csv"
},
{
"Name" : "sets",
"Extension" : "csv"
}
]

49.
Lookup Output
First Row Only:
Use to pass single values to another activity
@{activity('Lookup').output.firstRow.ColumnName}
All Rows:
Use to pass entire collection to another activity
@{activity('Lookup').output.value}

50.
DEMO
Let's add a lookup!

51.
How do you make dynamic solutions?
1.
2. with parameters
3. solution in a loop
4. from configuration table

52.
Good luck!

53.
@cathrinew
cathrinew.net
hi@cathrinew.net
thank you!