SlideShare uma empresa Scribd logo

Fundamentals of accuracy_assessment_session2_czaplewski

•
•
Raymond Czaplewski
Raymond Czaplewski

I presented an introductory workshop at the 9th international Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, also known as Accuracy2010 . This presentation is second of three. The first was conducted by Dr Steve Stehman, and the third by Dr Giles Foody

EducaçãoEsportesTecnologia
1 de 117
Baixar para ler offline
Session 2 Fundamentals of Accuracy Assessment Raymond L Czaplewski United States Forest Service Rocky Mountain Research Station Fort Collins, Colorado USA 1
Session 2 Topics • Different sample designs – Simple Random Sampling (Systematic Sampling) – Stratified Random Sampling • Different sample survey estimators • Different sample sizes, n=30, 60, 150 • How close are estimates to true value? • Example of a 30×30 = 900 pixel world 2
Hypothetical “real world” True (reference) population N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N N N ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Reference class N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 30×30 = 900 pixels ^ ^ Natural N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Urban Crop N N N N ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ N N N N N N ^ ^ ^ N ^ ^ ^ ^ N N N N N ^ N N ^ N N N N N N N N ^ ^ N N ^ ^ N ^ N ^ N N N N N ^ ^ N N N N N N N N N ^ N N N N ^ ^ N N N N N N ^ ^ N ^ ^ N N N N N N N ^ N N N N N ^ ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ N ^ ^ N N N N N N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ ^ ^ N ^ ^ N N N N ^ N ^ ^ ^ N N N N N N N N N N N N N ^ ^ ^ N N N ^ N N N N N N N ^ N ^ N N N N N N N N N N N N N ^ ^ ^ N ^ N ^ ^ N N N N N N N ^ N N N N N N N N N N N N N N ^ ^ ^ ^ N ^ N N N N N N N N ^ ^ ^ N N N N N N N N N N N N N ^ ^ ^ N ^ ^ ^ ^ 3
Hypothetical remotely sensed thematic map model for this “real world” Map #1 N N N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ N N ^ N N ^ N ^ N ^ ^ N ^ ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N ^ N ^ ^ N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ N ^ N N ^ N ^ ^ N N N ^ N ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N ^ ^ ^ N N N ^ N N N ^ ^ ^ ^ N ^ ^ N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ N N N N N ^ ^ N N ^ ^ ^ N ^ ^ ^ ^ N N N ^ ^ ^ N N ^ ^ N ^ N N ^ ^ ^ ^ ^ ^ N ^ ^ N ^ N N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ N ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N ^ N N N N ^ ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N ^ ^ ^ N ^ N ^ N ^ N N ^ ^ ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ N N ^ N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 30×30 = 900 pixels N ^ ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ N ^ N ^ N N N N ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ N ^ N ^ ^ N N N N ^ ^ ^ ^ N N N ^ N N ^ ^ ^ N N N ^ ^ ^ ^ ^ N ^ Map class N N N N N N ^ ^ N ^ N ^ N N ^ N ^ N ^ N ^ N ^ N ^ N N ^ N ^ N N N N N N N ^ ^ N N ^ N ^ ^ ^ N N N ^ Natural N N N N ^ ^ ^ ^ N N N N N N N ^ N ^ ^ N ^ ^ ^ Urban ^ N ^ ^ N ^ ^ ^ ^ N N N ^ ^ N N ^ N N N N N ^ ^ ^ N ^ ^ N ^ N ^ ^ N ^ N N N N N ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ ^ Crop ^ ^ ^ N N N N ^ N ^ N N N ^ N N N N N N N N N N ^ ^ ^ ^ N ^ N N ^ N ^ N N ^ N N N N N ^ N N ^ N ^ ^ N N ^ ^ N N N ^ N ^ N N N N N N N N N ^ N N ^ ^ N ^ N N ^ ^ ^ ^ N N N N N ^ ^ N N N N N ^ N N N N N N N ^ N N ^ N N ^ N N N N ^ N ^ N ^ N N N N ^ N N N N ^ ^ ^ ^ ^ ^ N N ^ N ^ 4
Remotely sensed thematic Map #1 True Error Matrix Error matrix presented by Steve Stehman Reference class Natural Urban Crop Total Traditional Analysis: Error Natural 226 27 74 327 Map class (Confusion) Matrix Urban 18 108 36 162 Reference Land Cover Crop 89 36 286 411 Mapped Natural Urban Crop Total Total 333 171 396 900 Natural 0.25 0.03 0.08 0.36 Overall accuracy 69% kappa 51% Urban 0.02 0.12 0.04 0.18 Reference class Crop 0.10 0.04 0.32 0.46 Natural Urban Crop Total Natural 25% 3% 8% 36% Map class Total 0.37 0.19 0.44 Urban 2% 12% 4% 18% Crop 10% 4% 32% 46% Total 37% 19% 44% 100% True (reference) population Map #1 N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N ^ N N ^ N ^ N ^ ^ N ^ ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N N ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N ^ N ^ 30×30 = 900 pixels N N N ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N ^ N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ N ^ N N ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N ^ N ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N ^ ^ ^ N N N ^ N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ N ^ ^ N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ ^ N N ^ ^ ^ N ^ ^ ^ ^ N N N ^ ^ ^ N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ N N ^ ^ N ^ N N ^ ^ ^ ^ ^ ^ N ^ ^ N ^ N N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ ^ ^ ^ N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ N N N ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N ^ ^ ^ N ^ N ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N ^ ^ ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ N N ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ ^ N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ N ^ N ^ N N N N ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ N ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ N N N ^ N N ^ ^ ^ N N N ^ ^ ^ ^ ^ N ^ N N N N N N ^ ^ ^ N N N N N N ^ ^ ^ N ^ ^ ^ ^ N N N N N N N ^ ^ N ^ N ^ N N ^ N ^ N ^ N ^ N ^ N ^ N N N N N ^ N N ^ N N N N N N N N ^ ^ N N ^ ^ N ^ N ^ N ^ N N N N N N N ^ ^ N N ^ N ^ ^ ^ N N N ^ N N N N N ^ ^ N N N N N N N N N ^ N N N N ^ ^ N N N N N ^ ^ ^ ^ N N N N N N N ^ N ^ ^ N ^ ^ ^ N N N N ^ ^ N ^ ^ N N N N N N N ^ N N N N N ^ ^ N ^ ^ ^ N ^ ^ N ^ ^ ^ ^ N N N ^ ^ N N ^ N N N N N ^ ^ ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ N ^ ^ N ^ N ^ ^ N ^ N N N N N ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ ^ N N N N N N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ ^ ^ N ^ ^ ^ ^ N N N N ^ N ^ N N N ^ N N N N N N N N N N ^ ^ ^ ^ N ^ N N N N N ^ N ^ ^ ^ N N N N N N N N N N N N N ^ ^ ^ N N N ^ N N ^ N ^ N N ^ N N N N N ^ N N ^ N ^ ^ N N ^ ^ N N N N N N N N N ^ N ^ N N N N N N N N N N N N N ^ ^ ^ N ^ N ^ ^ ^ N ^ N N N N N N N N N ^ N N ^ ^ N ^ N N ^ ^ ^ N N N N N N N N N N N N N ^ ^ N ^ ^ N N N N N N N N N N N N N N N N N N N N N N N N N N ^ ^ ^ ^ ^ ^ ^ N N ^ ^ ^ N N ^ ^ ^ N N N N N N N N ^ N ^ N ^ ^ N N N ^ N N N N N N ^ N ^ N N N N N N N N N ^ N N ^ N N ^ ^ ^ ^ ^ N ^ N N ^ N ^ N ^ 5
True error matrix parameters Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 25% 3% 8% 36% Map class Map class Urban 18 108 36 162 Urban 2% 12% 4% 18% Crop 89 36 286 411 Crop 10% 4% 32% 46% Total 333 171 396 900 Total 37% 19% 44% 100% Overall accuracy 69% kappa 51% 6
True error matrix parameters, graphical presentation True Map Land Cover area Natural 37% 19% 44% Urban True Reference Land Cover area Crop 0% 50% 100% Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 25% 3% 8% 36% Map class Map class Urban 18 108 36 162 Urban 2% 12% 4% 18% Crop 89 36 286 411 Crop 10% 4% 32% 46% Total 333 171 396 900 Total 37% 19% 44% 100% Overall accuracy 69% kappa 51% 7
True error matrix parameters, graphical presentation 36% 18% 46% True Map Land Cover area Natural Urban True Reference Land Cover area Crop 0% 50% 100% Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 25% 3% 8% 36% Map class Map class Urban 18 108 36 162 Urban 2% 12% 4% 18% Crop 89 36 286 411 Crop 10% 4% 32% 46% Total 333 171 396 900 Total 37% 19% 44% 100% Overall accuracy 69% kappa 51% 8
True error matrix parameters, graphical presentation User's Accuracy Producer's Accuracy Reference class Reference class Natural Urban Crop Total Natural Urban Crop Natural 69% 8% 23% 100% Natural 68% 16% 19% Map class Map class Urban 11% 67% 22% 100% Urban 5% 63% 9% Crop 22% 9% 70% 100% Crop 27% 21% 72% Total 100% 100% 100% Overall accuracy 69% kappa 51% User’s Accuracy Producer’s Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Overall accuracy True 0% 50% 100% 9
True error matrix parameters, graphical presentation User's Accuracy Producer's Accuracy Reference class Reference class Natural Urban Crop Total Natural Urban Crop Natural 69% 8% 23% 100% Natural 68% 16% 19% Map class Map class Urban 11% 67% 22% 100% Urban 5% 63% 9% Crop 22% 9% 70% 100% Crop 27% 21% 72% Total 100% 100% 100% Overall accuracy 69% kappa 51% User’s Accuracy Producer’s Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Overall accuracy True 0% 50% 100% 10
True error matrix parameters, graphical presentation User's Accuracy Producer's Accuracy Reference class Reference class Natural Urban Crop Total Natural Urban Crop Natural 69% 8% 23% 100% Natural 68% 16% 19% Map class Map class Urban 11% 67% 22% 100% Urban 5% 63% 9% Crop 22% 9% 70% 100% Crop 27% 21% 72% Total 100% 100% 100% Overall accuracy 69% kappa 51% User’s Accuracy Producer’s Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Overall accuracy True 0% 50% 100% 11
In the real world, we do not know the true classification for all 900 pixels ? True (reference) population True error matrix ? 900 Reference class Natural Urban Crop Map class Natural N ^ Urban Crop ^ 12
In the real world, we do know the true classification for 30 sampled pixels ? Sample of (reference) population True true (reference) population True error matrix 900 Reference class Natural Urban Crop Map class Natural N ^ Urban Crop ^ 13
In the real world, we do know the true classification for 30 sampled pixels ? Sample of true (reference) population True error matrix 900 Error matrix estimate from sample Reference class Natural Urban Crop Total Natural 8 0 2 10 Map class Urban 0 4 2 6 Crop 2 2 10 14 Total 10 6 14 30 Overall accuracy 73% kappa 58% Reference class Natural Urban Crop Total Map class Natural 27% 0% 7% 33% Urban 0% 13% 7% 20% Crop 7% 7% 33% 47% Total 33% 20% 47% 100% 14
In the real world, we do not know the true classification for all 900 pixels • Let us leave the real world for the next 30 minutes to compare – Known estimate of an error matrix with a sample of 30 pixels – Unknown true error matrix for all 900 pixels 15
Comparison of true (unknown) error matrix with (known) sample estimate True (unknown) error matrix Error matrix estimate from sample Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 8 0 2 10 Map class Map class Urban 18 108 36 162 Urban 0 4 2 6 Crop 89 36 286 411 Crop 2 2 10 14 Total 333 171 396 900 Total 10 6 14 30 Overall accuracy 69% kappa 51% Overall accuracy 73% kappa 58% 16
Examples of random sampling error, simple random sample #1, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 17
Examples of random sampling error, simple random sample #2, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 18
Examples of random sampling error, simple random sample #3, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 19
Examples of random sampling error, simple random sample #4, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 20
Examples of random sampling error, simple random sample #5, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 21
But how good is the sample estimate? Example, Producer’s Accuracy Urban Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 22
Example: Producers accuracy for urban Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% Estimated Producer's Accuracy = 80% Crop Urban ` Natural 0% 50% 100% 23
Sample #1, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 80% Crop Urban ` Natural 0% 50% 100% 24
Sample #2, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 71% Crop Urban ` Natural 0% 50% 100% 25
Sample #3, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 60% Crop Urban ` Natural 0% 50% 100% 26
Sample #4, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 53% Crop Urban ` Natural 0% 50% 100% 27
Sample #5, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 70% Crop Urban ` Natural 0% 50% 100% 28
Sample #6, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 73% Crop Urban ` Natural 0% 50% 100% 29
Sample #7, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 88% Crop Urban ` Natural 0% 50% 100% 30
Sample #8, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 60% Crop Urban ` Natural 0% 50% 100% 31
Sample #9, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 32
Sample #10, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 45% Crop Urban ` Natural 0% 50% 100% 33
Sample #11, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 34
Sample #12, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 86% Crop Urban ` Natural 0% 50% 100% 35
Sample #13, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 75% Crop Urban ` Natural 0% 50% 100% 36
Sample #14, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 73% Crop Urban ` Natural 0% 50% 100% 37
Sample #15, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 47% Crop Urban ` Natural 0% 50% 100% 38
Sample #16, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 57% Crop Urban ` Natural 0% 50% 100% 39
Sample #17, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 64% Crop Urban ` Natural 0% 50% 100% 40
Sample #18, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 70% Crop Urban ` Natural 0% 50% 100% 41
Sample #19, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 42
Sample #20, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 77% Crop Urban ` Natural 0% 50% 100% 43
Sample #21, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 44
Sample #22, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 76% Crop Urban ` Natural 0% 50% 100% 45
Sample #23, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 100% Crop Urban ` Natural 0% 50% 100% 46
Sample #24, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 90% Crop Urban ` Natural 0% 50% 100% 47
Sample #25, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 50% Crop Urban ` Natural 0% 50% 100% 48
Sample #26, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 77% Crop Urban ` Natural 0% 50% 100% 49
Sample #27, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 91% Crop Urban ` Natural 0% 50% 100% 50
Sample #28, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 51
Sample #29, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 27% Crop Urban ` Natural 0% 50% 100% 52
Sample #30, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 55% Crop Urban ` Natural 0% 50% 100% 53
Sample #31, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 56% Crop Urban ` Natural 0% 50% 100% 54
Sample #32, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 55
Sample #33, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 56
Sample #34, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 71% Crop Urban ` Natural 0% 50% 100% 57
Sample #35, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 83% Crop Urban ` Natural 0% 50% 100% 58
Sample #36, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 62% Crop Urban ` Natural 0% 50% 100% 59
Sample #37, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 64% Crop Urban ` Natural 0% 50% 100% 60
Sample #38, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 88% Crop Urban ` Natural 0% 50% 100% 61
Sample #39, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 57% Crop Urban ` Natural 0% 50% 100% 62
Sample #40, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 56% Crop Urban ` Natural 0% 50% 100% 63
Sample #41, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 57% Crop Urban ` Natural 0% 50% 100% 64
Sample #42, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 73% Crop Urban ` Natural 0% 50% 100% 65
Sample #43, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 66
Sample #44, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 55% Crop Urban ` Natural 0% 50% 100% 67
Sample #45, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 63% Crop Urban ` Natural 0% 50% 100% 68
Sample #46, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 71% Crop Urban ` Natural 0% 50% 100% 69
Sample #47, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 55% Crop Urban ` Natural 0% 50% 100% 70
Sample #48, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 71
Sample #49, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 77% Crop Urban ` Natural 0% 50% 100% 72
Sample #50, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 89% Crop Urban ` Natural 0% 50% 100% 73
Truth = 63% 300 Number of samples 200 100 0 0 20 40 60 80 100 True accuracy = 63% % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 74
In the real world, we do not know the true value 300 Number of samples 200 100 0 0 20 40 60 80 100 True accuracy = 63% % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 75
In the real world, we do not know the true value, and we have only 1 sample Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 76
Examples of random sampling error, simple random sample • Any single sample estimate can differ from true error matrix from random sampling error • Given our only sample with n=60, the estimated urban producers accuracy = 78% even though the true value is 63% • However, the sample estimate is expected to equal the true value over all possible samples 77
Examples of random sampling error, simple random sample • How can we improve reliability of estimate? • What if sample size increased from n=60 to n=150? 78
Examples of random sampling error, simple random sample #51, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 79
Examples of random sampling error, simple random sample #52, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 80
Examples of random sampling error, simple random sample #53, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 81
Examples of random sampling error, simple random sample #54, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 82
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski
Fundamentals of accuracy_assessment_session2_czaplewski

Recomendados

Dam phan bang thu trong xuat nhap khau
Dam phan bang thu trong xuat nhap khauDam phan bang thu trong xuat nhap khau
Dam phan bang thu trong xuat nhap khau
robodientu
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Chocolate Cupcake Recipe
Chocolate Cupcake RecipeChocolate Cupcake Recipe
Chocolate Cupcake Recipe
fwoohlaure
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Sbragi sara 2011-12_es3
Sbragi sara 2011-12_es3Sbragi sara 2011-12_es3
Sbragi sara 2011-12_es3
sarasbragi
 
Chapter 9
Chapter 9Chapter 9
Chapter 9
PhysicsJackson
 

Recomendados

Dam phan bang thu trong xuat nhap khau
Dam phan bang thu trong xuat nhap khauDam phan bang thu trong xuat nhap khau
Dam phan bang thu trong xuat nhap khau
robodientu
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Chocolate Cupcake Recipe
Chocolate Cupcake RecipeChocolate Cupcake Recipe
Chocolate Cupcake Recipe
fwoohlaure
 
Formula1
Formula1Formula1
Formula1
heduar98carlos
 
Sbragi sara 2011-12_es3
Sbragi sara 2011-12_es3Sbragi sara 2011-12_es3
Sbragi sara 2011-12_es3
sarasbragi
 
Chapter 9
Chapter 9Chapter 9
Chapter 9
PhysicsJackson
 
FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024
Elizabeth Walsh
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
RamjanShidvankar
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.
MaryamAhmad92
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentation
camerronhm
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
Nguyen Thanh Tu Collection
 
On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
Mebane Rash
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
Celine George
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
pradhanghanshyam7136
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
TechSoup
 
Unit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxUnit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptx
VishalSingh1417
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
agholdier
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
Dr. Ravikiran H M Gowda
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
christianmathematics
 
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Pooja Bhuva
 
Wellbeing inclusion and digital dystopias.pptx
Wellbeing inclusion and digital dystopias.pptxWellbeing inclusion and digital dystopias.pptx
Wellbeing inclusion and digital dystopias.pptx
Jisc
 
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptxSKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
Amanpreet Kaur
 
Fostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds in the ClassroomFostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds in the Classroom
Pooky Knightsmith
 
Sociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning ExhibitSociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning Exhibit
jbellavia9
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
Ramakrishna Reddy Bijjam
 
Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functions
KarakKing
 
2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
Expeed Software
 

Mais conteúdo relacionado

Último

Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functions
KarakKing
 

Último (20)

FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024FSB Advising Checklist - Orientation 2024
FSB Advising Checklist - Orientation 2024
 
Application orientated numerical on hev.ppt
Application orientated numerical on hev.pptApplication orientated numerical on hev.ppt
Application orientated numerical on hev.ppt
 
ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.ICT role in 21st century education and it's challenges.
ICT role in 21st century education and it's challenges.
 
SOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning PresentationSOC 101 Demonstration of Learning Presentation
SOC 101 Demonstration of Learning Presentation
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
 
On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
Unit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptxUnit-V; Pricing (Pharma Marketing Management).pptx
Unit-V; Pricing (Pharma Marketing Management).pptx
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.This PowerPoint helps students to consider the concept of infinity.
This PowerPoint helps students to consider the concept of infinity.
 
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
Beyond_Borders_Understanding_Anime_and_Manga_Fandom_A_Comprehensive_Audience_...
 
Wellbeing inclusion and digital dystopias.pptx
Wellbeing inclusion and digital dystopias.pptxWellbeing inclusion and digital dystopias.pptx
Wellbeing inclusion and digital dystopias.pptx
 
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptxSKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
SKILL OF INTRODUCING THE LESSON MICRO SKILLS.pptx
 
Fostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds in the ClassroomFostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds in the Classroom
 
Sociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning ExhibitSociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning Exhibit
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
 
Salient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functionsSalient Features of India constitution especially power and functions
Salient Features of India constitution especially power and functions
 

Destaque

How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 

Destaque (20)

2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot2024 State of Marketing Report – by Hubspot
2024 State of Marketing Report – by Hubspot
 
Everything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPTEverything You Need To Know About ChatGPT
Everything You Need To Know About ChatGPT
 
Product Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage EngineeringsProduct Design Trends in 2024 | Teenage Engineerings
Product Design Trends in 2024 | Teenage Engineerings
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental HealthHow Race, Age and Gender Shape Attitudes Towards Mental Health
How Race, Age and Gender Shape Attitudes Towards Mental Health
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdfAI Trends in Creative Operations 2024 by Artwork Flow.pdf
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
 
Skeleton Culture Code
Skeleton Culture CodeSkeleton Culture Code
Skeleton Culture Code
 
PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024PEPSICO Presentation to CAGNY Conference Feb 2024
PEPSICO Presentation to CAGNY Conference Feb 2024
 
Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)Content Methodology: A Best Practices Report (Webinar)
Content Methodology: A Best Practices Report (Webinar)
 
How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024How to Prepare For a Successful Job Search for 2024
How to Prepare For a Successful Job Search for 2024
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie InsightsSocial Media Marketing Trends 2024 // The Global Indie Insights
Social Media Marketing Trends 2024 // The Global Indie Insights
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024Trends In Paid Search: Navigating The Digital Landscape In 2024
Trends In Paid Search: Navigating The Digital Landscape In 2024
 
5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary5 Public speaking tips from TED - Visualized summary
5 Public speaking tips from TED - Visualized summary
 
ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd ChatGPT and the Future of Work - Clark Boyd
ChatGPT and the Future of Work - Clark Boyd
 
Getting into the tech field. what next
Getting into the tech field. what next Getting into the tech field. what next
Getting into the tech field. what next
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search IntentGoogle's Just Not That Into You: Understanding Core Updates & Search Intent
Google's Just Not That Into You: Understanding Core Updates & Search Intent
 
How to have difficult conversations
How to have difficult conversations How to have difficult conversations
How to have difficult conversations
 
Introduction to Data Science
Introduction to Data ScienceIntroduction to Data Science
Introduction to Data Science
 
Time Management & Productivity - Best Practices
Time Management & Productivity - Best PracticesTime Management & Productivity - Best Practices
Time Management & Productivity - Best Practices
 
The six step guide to practical project management
The six step guide to practical project managementThe six step guide to practical project management
The six step guide to practical project management
 
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
 

Fundamentals of accuracy_assessment_session2_czaplewski

  • 1. Session 2 Fundamentals of Accuracy Assessment Raymond L Czaplewski United States Forest Service Rocky Mountain Research Station Fort Collins, Colorado USA 1
  • 2. Session 2 Topics • Different sample designs – Simple Random Sampling (Systematic Sampling) – Stratified Random Sampling • Different sample survey estimators • Different sample sizes, n=30, 60, 150 • How close are estimates to true value? • Example of a 30×30 = 900 pixel world 2
  • 3. Hypothetical “real world” True (reference) population N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N N N ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Reference class N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 30×30 = 900 pixels ^ ^ Natural N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ Urban Crop N N N N ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ N N N N N N ^ ^ ^ N ^ ^ ^ ^ N N N N N ^ N N ^ N N N N N N N N ^ ^ N N ^ ^ N ^ N ^ N N N N N ^ ^ N N N N N N N N N ^ N N N N ^ ^ N N N N N N ^ ^ N ^ ^ N N N N N N N ^ N N N N N ^ ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ N ^ ^ N N N N N N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ ^ ^ N ^ ^ N N N N ^ N ^ ^ ^ N N N N N N N N N N N N N ^ ^ ^ N N N ^ N N N N N N N ^ N ^ N N N N N N N N N N N N N ^ ^ ^ N ^ N ^ ^ N N N N N N N ^ N N N N N N N N N N N N N N ^ ^ ^ ^ N ^ N N N N N N N N ^ ^ ^ N N N N N N N N N N N N N ^ ^ ^ N ^ ^ ^ ^ 3
  • 4. Hypothetical remotely sensed thematic map model for this “real world” Map #1 N N N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ N N ^ N N ^ N ^ N ^ ^ N ^ ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N ^ N ^ ^ N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ N ^ N N ^ N ^ ^ N N N ^ N ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N ^ ^ ^ N N N ^ N N N ^ ^ ^ ^ N ^ ^ N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ N N N N N ^ ^ N N ^ ^ ^ N ^ ^ ^ ^ N N N ^ ^ ^ N N ^ ^ N ^ N N ^ ^ ^ ^ ^ ^ N ^ ^ N ^ N N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ N ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N ^ N N N N ^ ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N ^ ^ ^ N ^ N ^ N ^ N N ^ ^ ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ N N ^ N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 30×30 = 900 pixels N ^ ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ N ^ N ^ N N N N ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ N ^ N ^ ^ N N N N ^ ^ ^ ^ N N N ^ N N ^ ^ ^ N N N ^ ^ ^ ^ ^ N ^ Map class N N N N N N ^ ^ N ^ N ^ N N ^ N ^ N ^ N ^ N ^ N ^ N N ^ N ^ N N N N N N N ^ ^ N N ^ N ^ ^ ^ N N N ^ Natural N N N N ^ ^ ^ ^ N N N N N N N ^ N ^ ^ N ^ ^ ^ Urban ^ N ^ ^ N ^ ^ ^ ^ N N N ^ ^ N N ^ N N N N N ^ ^ ^ N ^ ^ N ^ N ^ ^ N ^ N N N N N ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ ^ Crop ^ ^ ^ N N N N ^ N ^ N N N ^ N N N N N N N N N N ^ ^ ^ ^ N ^ N N ^ N ^ N N ^ N N N N N ^ N N ^ N ^ ^ N N ^ ^ N N N ^ N ^ N N N N N N N N N ^ N N ^ ^ N ^ N N ^ ^ ^ ^ N N N N N ^ ^ N N N N N ^ N N N N N N N ^ N N ^ N N ^ N N N N ^ N ^ N ^ N N N N ^ N N N N ^ ^ ^ ^ ^ ^ N N ^ N ^ 4
  • 5. Remotely sensed thematic Map #1 True Error Matrix Error matrix presented by Steve Stehman Reference class Natural Urban Crop Total Traditional Analysis: Error Natural 226 27 74 327 Map class (Confusion) Matrix Urban 18 108 36 162 Reference Land Cover Crop 89 36 286 411 Mapped Natural Urban Crop Total Total 333 171 396 900 Natural 0.25 0.03 0.08 0.36 Overall accuracy 69% kappa 51% Urban 0.02 0.12 0.04 0.18 Reference class Crop 0.10 0.04 0.32 0.46 Natural Urban Crop Total Natural 25% 3% 8% 36% Map class Total 0.37 0.19 0.44 Urban 2% 12% 4% 18% Crop 10% 4% 32% 46% Total 37% 19% 44% 100% True (reference) population Map #1 N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N ^ N N ^ N ^ N ^ ^ N ^ ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N N ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N ^ N ^ 30×30 = 900 pixels N N N ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N ^ N ^ N ^ N ^ ^ ^ ^ N ^ ^ ^ N ^ ^ N ^ N N ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N ^ N ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N ^ ^ ^ N N N ^ N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ N ^ ^ N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N N N ^ ^ N N ^ ^ ^ N ^ ^ ^ ^ N N N ^ ^ ^ N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ N N ^ ^ N ^ N N ^ ^ ^ ^ ^ ^ N ^ ^ N ^ N N N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ ^ N N N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N N ^ N ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ ^ ^ ^ N N N ^ N ^ ^ ^ ^ N ^ ^ N ^ ^ N N N ^ N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ N ^ ^ ^ N ^ N ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ N N ^ ^ ^ N ^ ^ ^ ^ ^ N ^ ^ ^ ^ N N ^ N N N N ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N N ^ N ^ ^ ^ N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N ^ ^ ^ N N N N N N N ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ N N ^ N ^ ^ ^ ^ N ^ ^ ^ ^ ^ ^ N ^ N ^ N N N N ^ ^ ^ ^ N N N N ^ ^ ^ ^ ^ ^ ^ N ^ ^ N N N N ^ ^ ^ ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ N ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N ^ ^ ^ ^ N ^ ^ ^ ^ ^ N N N N ^ ^ ^ ^ N N N ^ N N ^ ^ ^ N N N ^ ^ ^ ^ ^ N ^ N N N N N N ^ ^ ^ N N N N N N ^ ^ ^ N ^ ^ ^ ^ N N N N N N N ^ ^ N ^ N ^ N N ^ N ^ N ^ N ^ N ^ N ^ N N N N N ^ N N ^ N N N N N N N N ^ ^ N N ^ ^ N ^ N ^ N ^ N N N N N N N ^ ^ N N ^ N ^ ^ ^ N N N ^ N N N N N ^ ^ N N N N N N N N N ^ N N N N ^ ^ N N N N N ^ ^ ^ ^ N N N N N N N ^ N ^ ^ N ^ ^ ^ N N N N ^ ^ N ^ ^ N N N N N N N ^ N N N N N ^ ^ N ^ ^ ^ N ^ ^ N ^ ^ ^ ^ N N N ^ ^ N N ^ N N N N N ^ ^ ^ N ^ ^ N N N N ^ N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ N ^ ^ N ^ N ^ ^ N ^ N N N N N ^ N ^ ^ N N N N N ^ ^ ^ ^ ^ N ^ ^ N N N N N N ^ ^ N N N N N N N N N N N N N N ^ ^ ^ ^ ^ N ^ ^ ^ ^ N N N N ^ N ^ N N N ^ N N N N N N N N N N ^ ^ ^ ^ N ^ N N N N N ^ N ^ ^ ^ N N N N N N N N N N N N N ^ ^ ^ N N N ^ N N ^ N ^ N N ^ N N N N N ^ N N ^ N ^ ^ N N ^ ^ N N N N N N N N N ^ N ^ N N N N N N N N N N N N N ^ ^ ^ N ^ N ^ ^ ^ N ^ N N N N N N N N N ^ N N ^ ^ N ^ N N ^ ^ ^ N N N N N N N N N N N N N ^ ^ N ^ ^ N N N N N N N N N N N N N N N N N N N N N N N N N N ^ ^ ^ ^ ^ ^ ^ N N ^ ^ ^ N N ^ ^ ^ N N N N N N N N ^ N ^ N ^ ^ N N N ^ N N N N N N ^ N ^ N N N N N N N N N ^ N N ^ N N ^ ^ ^ ^ ^ N ^ N N ^ N ^ N ^ 5
  • 6. True error matrix parameters Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 25% 3% 8% 36% Map class Map class Urban 18 108 36 162 Urban 2% 12% 4% 18% Crop 89 36 286 411 Crop 10% 4% 32% 46% Total 333 171 396 900 Total 37% 19% 44% 100% Overall accuracy 69% kappa 51% 6
  • 7. True error matrix parameters, graphical presentation True Map Land Cover area Natural 37% 19% 44% Urban True Reference Land Cover area Crop 0% 50% 100% Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 25% 3% 8% 36% Map class Map class Urban 18 108 36 162 Urban 2% 12% 4% 18% Crop 89 36 286 411 Crop 10% 4% 32% 46% Total 333 171 396 900 Total 37% 19% 44% 100% Overall accuracy 69% kappa 51% 7
  • 8. True error matrix parameters, graphical presentation 36% 18% 46% True Map Land Cover area Natural Urban True Reference Land Cover area Crop 0% 50% 100% Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 25% 3% 8% 36% Map class Map class Urban 18 108 36 162 Urban 2% 12% 4% 18% Crop 89 36 286 411 Crop 10% 4% 32% 46% Total 333 171 396 900 Total 37% 19% 44% 100% Overall accuracy 69% kappa 51% 8
  • 9. True error matrix parameters, graphical presentation User's Accuracy Producer's Accuracy Reference class Reference class Natural Urban Crop Total Natural Urban Crop Natural 69% 8% 23% 100% Natural 68% 16% 19% Map class Map class Urban 11% 67% 22% 100% Urban 5% 63% 9% Crop 22% 9% 70% 100% Crop 27% 21% 72% Total 100% 100% 100% Overall accuracy 69% kappa 51% User’s Accuracy Producer’s Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Overall accuracy True 0% 50% 100% 9
  • 10. True error matrix parameters, graphical presentation User's Accuracy Producer's Accuracy Reference class Reference class Natural Urban Crop Total Natural Urban Crop Natural 69% 8% 23% 100% Natural 68% 16% 19% Map class Map class Urban 11% 67% 22% 100% Urban 5% 63% 9% Crop 22% 9% 70% 100% Crop 27% 21% 72% Total 100% 100% 100% Overall accuracy 69% kappa 51% User’s Accuracy Producer’s Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Overall accuracy True 0% 50% 100% 10
  • 11. True error matrix parameters, graphical presentation User's Accuracy Producer's Accuracy Reference class Reference class Natural Urban Crop Total Natural Urban Crop Natural 69% 8% 23% 100% Natural 68% 16% 19% Map class Map class Urban 11% 67% 22% 100% Urban 5% 63% 9% Crop 22% 9% 70% 100% Crop 27% 21% 72% Total 100% 100% 100% Overall accuracy 69% kappa 51% User’s Accuracy Producer’s Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Overall accuracy True 0% 50% 100% 11
  • 12. In the real world, we do not know the true classification for all 900 pixels ? True (reference) population True error matrix ? 900 Reference class Natural Urban Crop Map class Natural N ^ Urban Crop ^ 12
  • 13. In the real world, we do know the true classification for 30 sampled pixels ? Sample of (reference) population True true (reference) population True error matrix 900 Reference class Natural Urban Crop Map class Natural N ^ Urban Crop ^ 13
  • 14. In the real world, we do know the true classification for 30 sampled pixels ? Sample of true (reference) population True error matrix 900 Error matrix estimate from sample Reference class Natural Urban Crop Total Natural 8 0 2 10 Map class Urban 0 4 2 6 Crop 2 2 10 14 Total 10 6 14 30 Overall accuracy 73% kappa 58% Reference class Natural Urban Crop Total Map class Natural 27% 0% 7% 33% Urban 0% 13% 7% 20% Crop 7% 7% 33% 47% Total 33% 20% 47% 100% 14
  • 15. In the real world, we do not know the true classification for all 900 pixels • Let us leave the real world for the next 30 minutes to compare – Known estimate of an error matrix with a sample of 30 pixels – Unknown true error matrix for all 900 pixels 15
  • 16. Comparison of true (unknown) error matrix with (known) sample estimate True (unknown) error matrix Error matrix estimate from sample Reference class Reference class Natural Urban Crop Total Natural Urban Crop Total Natural 226 27 74 327 Natural 8 0 2 10 Map class Map class Urban 18 108 36 162 Urban 0 4 2 6 Crop 89 36 286 411 Crop 2 2 10 14 Total 333 171 396 900 Total 10 6 14 30 Overall accuracy 69% kappa 51% Overall accuracy 73% kappa 58% 16
  • 17. Examples of random sampling error, simple random sample #1, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 17
  • 18. Examples of random sampling error, simple random sample #2, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 18
  • 19. Examples of random sampling error, simple random sample #3, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 19
  • 20. Examples of random sampling error, simple random sample #4, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 20
  • 21. Examples of random sampling error, simple random sample #5, sample size n=30 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 21
  • 22. But how good is the sample estimate? Example, Producer’s Accuracy Urban Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 22
  • 23. Example: Producers accuracy for urban Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% Estimated Producer's Accuracy = 80% Crop Urban ` Natural 0% 50% 100% 23
  • 24. Sample #1, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 80% Crop Urban ` Natural 0% 50% 100% 24
  • 25. Sample #2, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 71% Crop Urban ` Natural 0% 50% 100% 25
  • 26. Sample #3, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 60% Crop Urban ` Natural 0% 50% 100% 26
  • 27. Sample #4, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 53% Crop Urban ` Natural 0% 50% 100% 27
  • 28. Sample #5, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 70% Crop Urban ` Natural 0% 50% 100% 28
  • 29. Sample #6, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 73% Crop Urban ` Natural 0% 50% 100% 29
  • 30. Sample #7, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 88% Crop Urban ` Natural 0% 50% 100% 30
  • 31. Sample #8, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 60% Crop Urban ` Natural 0% 50% 100% 31
  • 32. Sample #9, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 32
  • 33. Sample #10, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 45% Crop Urban ` Natural 0% 50% 100% 33
  • 34. Sample #11, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 34
  • 35. Sample #12, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 86% Crop Urban ` Natural 0% 50% 100% 35
  • 36. Sample #13, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 75% Crop Urban ` Natural 0% 50% 100% 36
  • 37. Sample #14, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 73% Crop Urban ` Natural 0% 50% 100% 37
  • 38. Sample #15, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 47% Crop Urban ` Natural 0% 50% 100% 38
  • 39. Sample #16, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 57% Crop Urban ` Natural 0% 50% 100% 39
  • 40. Sample #17, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 64% Crop Urban ` Natural 0% 50% 100% 40
  • 41. Sample #18, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 70% Crop Urban ` Natural 0% 50% 100% 41
  • 42. Sample #19, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 42
  • 43. Sample #20, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 77% Crop Urban ` Natural 0% 50% 100% 43
  • 44. Sample #21, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 44
  • 45. Sample #22, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 76% Crop Urban ` Natural 0% 50% 100% 45
  • 46. Sample #23, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 100% Crop Urban ` Natural 0% 50% 100% 46
  • 47. Sample #24, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 90% Crop Urban ` Natural 0% 50% 100% 47
  • 48. Sample #25, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 50% Crop Urban ` Natural 0% 50% 100% 48
  • 49. Sample #26, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 77% Crop Urban ` Natural 0% 50% 100% 49
  • 50. Sample #27, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 91% Crop Urban ` Natural 0% 50% 100% 50
  • 51. Sample #28, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 51
  • 52. Sample #29, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 27% Crop Urban ` Natural 0% 50% 100% 52
  • 53. Sample #30, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 55% Crop Urban ` Natural 0% 50% 100% 53
  • 54. Sample #31, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 56% Crop Urban ` Natural 0% 50% 100% 54
  • 55. Sample #32, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 55
  • 56. Sample #33, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 56
  • 57. Sample #34, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 71% Crop Urban ` Natural 0% 50% 100% 57
  • 58. Sample #35, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 83% Crop Urban ` Natural 0% 50% 100% 58
  • 59. Sample #36, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 62% Crop Urban ` Natural 0% 50% 100% 59
  • 60. Sample #37, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 64% Crop Urban ` Natural 0% 50% 100% 60
  • 61. Sample #38, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 88% Crop Urban ` Natural 0% 50% 100% 61
  • 62. Sample #39, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 57% Crop Urban ` Natural 0% 50% 100% 62
  • 63. Sample #40, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 56% Crop Urban ` Natural 0% 50% 100% 63
  • 64. Sample #41, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 57% Crop Urban ` Natural 0% 50% 100% 64
  • 65. Sample #42, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 73% Crop Urban ` Natural 0% 50% 100% 65
  • 66. Sample #43, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 40% Crop Urban ` Natural 0% 50% 100% 66
  • 67. Sample #44, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 55% Crop Urban ` Natural 0% 50% 100% 67
  • 68. Sample #45, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 63% Crop Urban ` Natural 0% 50% 100% 68
  • 69. Sample #46, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 71% Crop Urban ` Natural 0% 50% 100% 69
  • 70. Sample #47, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 55% Crop Urban ` Natural 0% 50% 100% 70
  • 71. Sample #48, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 67% Crop Urban ` Natural 0% 50% 100% 71
  • 72. Sample #49, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 77% Crop Urban ` Natural 0% 50% 100% 72
  • 73. Sample #50, n=60 Truth = 63% Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 89% Crop Urban ` Natural 0% 50% 100% 73
  • 74. Truth = 63% 300 Number of samples 200 100 0 0 20 40 60 80 100 True accuracy = 63% % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 74
  • 75. In the real world, we do not know the true value 300 Number of samples 200 100 0 0 20 40 60 80 100 True accuracy = 63% % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 75
  • 76. In the real world, we do not know the true value, and we have only 1 sample Number of samples 10 5 0 0 20 40 60 80 100 % Producers Accuracy Estimated Producer's Accuracy = 78% Crop Urban ` Natural 0% 50% 100% 76
  • 77. Examples of random sampling error, simple random sample • Any single sample estimate can differ from true error matrix from random sampling error • Given our only sample with n=60, the estimated urban producers accuracy = 78% even though the true value is 63% • However, the sample estimate is expected to equal the true value over all possible samples 77
  • 78. Examples of random sampling error, simple random sample • How can we improve reliability of estimate? • What if sample size increased from n=60 to n=150? 78
  • 79. Examples of random sampling error, simple random sample #51, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 79
  • 80. Examples of random sampling error, simple random sample #52, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 80
  • 81. Examples of random sampling error, simple random sample #53, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 81
  • 82. Examples of random sampling error, simple random sample #54, sample size n=150 Area of each Land Cover Type Sample Map Land Cover area True Map Land Cover area Natural Sample Reference Land Cover area Urban True Reference Land Cover area Crop 0% 50% 100% User's Accuracy Producer's Accuracy Crop Crop Urban Urban Natural Natural 0% 50% 100% 0% 50% 100% kappa Sample Overall accuracy True 0% 50% 100% 82