SlideShare uma empresa Scribd logo
1 de 25
Baixar para ler offline
What is the problem?                 How can we deal with concept drift?   Summary




         Extensional Mapping-Chains for studying Concept
                    Drift in Political Ontologies

                        Shenghui Wang1 Stefan Schlobach2
                       Janet Takens3 Wouter van Atteveldt3

                                   1
                                     The Network Institute
                             2
                               Department of Computer Science
                         3
                             Department of Communication Science
                               Vrije Universiteit Amsterdam



                                        ICA 2010
                                        Singapore
What is the problem?                      How can we deal with concept drift?   Summary



Content analysis in Communication Science

              Communication scientists study all sorts of media content
              related to human communication
              Content analysis based on the NET method
                       concepts: political actors and issues
                       relations: associations, opinions, or actions.

      Example
      Het Openbaar Ministerie (OM) wil de komende vier jaar mensen-
      handel uitroeien.
What is the problem?                      How can we deal with concept drift?   Summary



Content analysis in Communication Science

              Communication scientists study all sorts of media content
              related to human communication
              Content analysis based on the NET method
                       concepts: political actors and issues
                       relations: associations, opinions, or actions.

      Example
      Het Openbaar Ministerie (OM) wil de komende vier jaar mensen-
      handel uitroeien.
What is the problem?                      How can we deal with concept drift?   Summary



Content analysis in Communication Science

              Communication scientists study all sorts of media content
              related to human communication
              Content analysis based on the NET method
                       concepts: political actors and issues
                       relations: associations, opinions, or actions.

      Example
      Het Openbaar Ministerie (OM) wil de komende vier jaar mensen-
      handel uitroeien.

                                    -1
                          om                       human trafficking
What is the problem?                                                        How can we deal with concept drift?                                                                            Summary



Semantic network analysis
                                                                    2271                     1067                   1388                  740                   1373
                                              2127
                         2467                                                                                                                                                      1268




                                                                                  2351
                                                                                                         907
                                                            1223                                                                                  1739
                                       1708                                                                                   2516
                                                                                                                                                                      1706
                        2393
                                                                                               2438                                                                                 1721
                                                                            1077
                                                                                                                    1052
                                                 2323                                                                                           1234
                                                               1034                                   2394
                                                                                                                                                                      964
                                 1275                                              2120                                           1936
                       1059                                                                                    1045
                                                                                                2221                                               2124                             2753
                                                     2171            2653
                                                                                       752
                                                                                                                       2655          2054                              856
                                       693                                                            1608
                                                                           341                                                                     2647
                                                            1011
                        2806                                                            1145                   1625
                                                                                                                            2076                                                     648
                                                 1386                                               1233
                                                                       1545                                                               329
                                                                                                                                                          480                361
                                 475                                                    845                         1906
                                                             2259
                                                                                                       883                    1439
                       2090                                                1474                                                                 1332
                                               2002                                                                                                               2377
                                                                                         1306                   2077
                                                                                                                                                                                     548
                                                            654
                                2696                                                                   2606                              2492
                                                                       1635                                            2471                              484
                                                                                         956
                                                                                                                                                                            889
                                               2199

                                                              1198                               623            2614
                       2573                                                      545                                              1409            1259
                                                                                                                                                                                    2251

                                 1940
                                                                                                                                                                      2751
                                                      539                                                1827
                                                                      870                2186                              2151                 2423




                         464
                                                                                                                                                                                   1097
                                        1841                 2070                                            1073                                              1932
                                                                                   438                                              2403
What is the problem?                      How can we deal with concept drift?   Summary



Network-based communication science study



      What information can we extract from these networks?
              Politicians are networking
              Politics is perceived by citizens via media
              Media study by semantic network analysis
                       Who   is determining the subjects?
                       Who   is teaming up?
                       Who   is more credible?
                       Who   owns which topic?
What is the problem?                    How can we deal with concept drift?   Summary



Before network analysis




      We first need to build the networks!
              Requires: large corpora with annotated textual content
                       Manual coding against coding books (ontologies)
                       Automated content analysis in progress
What is the problem?                    How can we deal with concept drift?   Summary



Before network analysis




      We first need to build the networks!
              Requires: large corpora with annotated textual content
                       Manual coding against coding books (ontologies)
                       Automated content analysis in progress
What is the problem?                     How can we deal with concept drift?         Summary



What is the problem?




      Problems with constructing annotated content
              Data from different time periods or genres
              Coded by different teams at different moments
                       Manifesto Research Group: 25 countries, from 1945 to 2006
                       Comparative Policy Agendas project: media content,
                       manifestos, legislative texts, government press statements, etc.
                       Election campaign coverage from 1994 to 2006
What is the problem?                 How can we deal with concept drift?   Summary



What are the challenges?



      Interoperability problem while sharing information
              Different teams use different code books
              Example
                       illegal immigration
                       labour migrants
              Different coding books should be merged or at least connected
              Not the focus of this paper
What is the problem?                 How can we deal with concept drift?   Summary



What are the challenges?



      Interoperability problem while sharing information
              Different teams use different code books
              Example
                       illegal immigration
                       labour migrants
              Different coding books should be merged or at least connected
              Not the focus of this paper
What is the problem?                 How can we deal with concept drift?   Summary



What are the challenges?



      Interoperability problem while sharing information
              Different teams use different code books
              Example
                       illegal immigration
                       labour migrants
              Different coding books should be merged or at least connected
              Not the focus of this paper
What is the problem?   How can we deal with concept drift?   Summary



Follow the Fashion?
What is the problem?               How can we deal with concept drift?    Summary



Women’s role?




              Suffragettes said that women’s role in society is unacceptable
              Pope says that women’s role in society is unacceptable
What is the problem?                 How can we deal with concept drift?   Summary



Concept drift




      Our problem: Concept drift
              Meaning of concepts changes over time
              Analysis based on evolving concepts must consider temporal
              locality
              Study concept drift itself is useful
What is the problem?               How can we deal with concept drift?   Summary



Datasets




              Five political ontologies which were used to annotate
              newspaper articles
              23 639 manually annotated newspaper articles during five
              recent Dutch national election campaigns
              There even exist manual mappings but most of them are
              lexically very similar
What is the problem?                     How can we deal with concept drift?        Summary



Detecting concept drift




      We use extensional mapping techniques
              Consider concepts at different time to be different concepts
              Use extensional method to detect the links between concepts
              at different time
                       Assumption: similar sentences should be coded with similar
                       concepts, therefore, similar concepts should have similar
                       extension.
What is the problem?   How can we deal with concept drift?   Summary



Representing concept drift using mapping chains
What is the problem?                How can we deal with concept drift?   Summary



Evaluating concept drift




      What can we learn from those chains?
              Do they agree with the political reality?
              Do they tell us something we do not noticed before?
              Are some concepts more stable/unstable than others?
      Quantitative evaluation is interesting, but qualitative analysis
      seems to tell us something too.
What is the problem?                How can we deal with concept drift?   Summary



Qualitative analysis of mapping chains




              Association vs. similarity
              Early erroneous associations can turn large parts of the
              analysis practically useless.
What is the problem?                How can we deal with concept drift?   Summary



Qualitative analysis of mapping chains




              Association vs. similarity
              Early erroneous associations can turn large parts of the
              analysis practically useless.
What is the problem?                                                                How can we deal with concept drift?                                                                                                                Summary




              “productiviteit” (Productivity)
                                                                                                                                                                                                               06_economic growth
                                                                                                                                                                                      0.0880

                                                                                                                                                     03_economische groei             0.0315
                                                                                                                                         0.0569                                                                    06_begroting
                                                                                                                                                                                      0.0327
                                                                                                                                         0.0499
                                                                                    0.0657                02_economische groei                       03_financieringstekort           0.0336
                                                                                                                                                                                                                 06_bezuinigingen
                                          0.0387
                 94_productiviteit                        98_welvaart valence       0.0587
                                                                                                              02_welvaart                0.0361                                       0.1505
                                                                                                                                                         03_spaarloon                                              06_spaarloon
                                                                                                                                                                                      0.0518

                                                                                                                                                                                                               06_levensloopregeling




              “euthanasie” (Euthanasia)
                                                                                                                                                     0.1789                                06_gay marriage
                                                                                                                                  03_euthanasie      0.1704
                                                                                                                  0.2999                             0.3491
                                                                                                                                                                                               06_abortion
                                                                                                                  0.2519
                                                                                       02_euthanasie                             03_homohuwelijk     0.1883

                                                                           0.1057
                                                                                                                                                                              06_verbetering communicatie overheid burger
                                                                                                                                                     0.0165

                                                                                                                  0.2185         03_milieuactivist   0.0425
                                                                           0.0768                                                                                                           06_criminelen
                                                   98_oeuthanasie                     02_milieuactivist
                                                                                                                  0.0507                             0.0310
                                     0.2636                                                                                         03_justitie      0.0291
                                                                                                                                                                                            06_asielzoekers
                 94_euthanasie

                                     0.0457
                                                                                                                                                                                        06_gratis schoolboeken
                                                                                                                                                     0.0441
                                                                                                                  0.1117         03_referendum eu    0.0313
                                                                           0.1016
                                                   98_hreferendum                      02_referendum              0.0432                                                                    06_referendum
                                                                                                                                                     0.0571
                                                                                                                                  03_referendum      0.0454
                                                                           0.0882
                                                                                                                                                                                          06_burgerinitiatief


                                                                                                                                                                                            06_werknemers
                                                                                                                                                     0.0548
                                                                                                                  0.0293
                                                                                       02_cdavvdlpf                               03_zondagsrust     0.0398
                                                                                                                  0.0257                                                                    06_sunday rest


                                                                                                                                                     0.0511
                                                                                                                                  03_scholieren                                                06_leerlingen
                                                                                                                                                     0.0286

                                                                                                                                                                                               06_education
What is the problem?                                  How can we deal with concept drift?                                     Summary



If we know two end-point concepts have the same meaning


                 Kite-shaped chains
                                                               02_mensenrechten


                               98_avluchtelingen                  02_jusititie
                                                                                           03_politie

                                                                02_criminaliteit

                                                                                          03_justitie
                                98_rcriminaliteit               02_drugkoeriers


                                                                02_cellentekort
                                                                                        03_criminaliteit
           94_asielzoekers   98_kabinet kokmierlods            02_bedrijfsleven


                                                                02_democratie
                                98_asielzoekers
                                                               02_buitenlanders         03_asielzoekers     06_asielzoekers


                                  98_okerken                 02_instroom beperking    03_opvang illegalen


                                                                02_asielzoekers        03_vluchtelingen


                                                                                         03_illegalen
What is the problem?                                                How can we deal with concept drift?                                                         Summary




              “christelijken” (Christians)
                                          98_oabortus                           02_normen waarden
                 94_christelijken                                                                          03_multiculturele samenleving        06_christenen
                                    98_ochristelijk christenen             02_multiculturele samenleving




              “asielzoekers” (Asylum seeker)
                                                                                      02_mensenrechten


                                              98_avluchtelingen                           02_jusititie
                                                                                                                  03_politie

                                                                                       02_criminaliteit

                                                                                                                 03_justitie
                                               98_rcriminaliteit                       02_drugkoeriers


                                                                                       02_cellentekort
                                                                                                              03_criminaliteit
                  94_asielzoekers          98_kabinet kokmierlods                      02_bedrijfsleven


                                                                                        02_democratie
                                               98_asielzoekers
                                                                                      02_buitenlanders        03_asielzoekers              06_asielzoekers


                                                  98_okerken                       02_instroom beperking    03_opvang illegalen


                                                                                       02_asielzoekers        03_vluchtelingen


                                                                                                                03_illegalen
What is the problem?               How can we deal with concept drift?      Summary



Summary




              By looking at extensions of concepts, we can detect concept
              drift
              Domain experts found that the detected concept drift makes
              sense
              Automated matching techniques can help domain experts to
              find hidden links between concepts
              More work needs to be done

Mais conteúdo relacionado

Mais de Shenghui Wang

Non-parametric Subject Prediction
Non-parametric Subject PredictionNon-parametric Subject Prediction
Non-parametric Subject PredictionShenghui Wang
 
Our journey with semantic embedding
Our journey with semantic embeddingOur journey with semantic embedding
Our journey with semantic embeddingShenghui Wang
 
Linking entities via semantic indexing
Linking entities via semantic indexingLinking entities via semantic indexing
Linking entities via semantic indexingShenghui Wang
 
Semantic indexing for KOS
Semantic indexing for KOSSemantic indexing for KOS
Semantic indexing for KOSShenghui Wang
 
Contextualization of topics - browsing through terms, authors, journals and c...
Contextualization of topics - browsing through terms, authors, journals and c...Contextualization of topics - browsing through terms, authors, journals and c...
Contextualization of topics - browsing through terms, authors, journals and c...Shenghui Wang
 
Exploring a world of networked information built from free-text metadata
Exploring a world of networked information built from free-text metadataExploring a world of networked information built from free-text metadata
Exploring a world of networked information built from free-text metadataShenghui Wang
 
Ariadne's Thread -- Exploring a world of networked information built from fre...
Ariadne's Thread -- Exploring a world of networked information built from fre...Ariadne's Thread -- Exploring a world of networked information built from fre...
Ariadne's Thread -- Exploring a world of networked information built from fre...Shenghui Wang
 
Learning Concept Mappings from Instance Similarity
Learning Concept Mappings from Instance SimilarityLearning Concept Mappings from Instance Similarity
Learning Concept Mappings from Instance SimilarityShenghui Wang
 
Measuring the dynamic bi-directional influence between content and social ne...
Measuring the dynamic bi-directional influence between content and social ne...Measuring the dynamic bi-directional influence between content and social ne...
Measuring the dynamic bi-directional influence between content and social ne...Shenghui Wang
 
Similarity Features, and their Role in Concept Alignment Learning
Similarity Features, and their Role in Concept Alignment Learning Similarity Features, and their Role in Concept Alignment Learning
Similarity Features, and their Role in Concept Alignment Learning Shenghui Wang
 
What is concept dirft and how to measure it?
What is concept dirft and how to measure it?What is concept dirft and how to measure it?
What is concept dirft and how to measure it?Shenghui Wang
 
Study concept drift in political ontologies
Study concept drift in political ontologiesStudy concept drift in political ontologies
Study concept drift in political ontologiesShenghui Wang
 

Mais de Shenghui Wang (13)

Non-parametric Subject Prediction
Non-parametric Subject PredictionNon-parametric Subject Prediction
Non-parametric Subject Prediction
 
Our journey with semantic embedding
Our journey with semantic embeddingOur journey with semantic embedding
Our journey with semantic embedding
 
Linking entities via semantic indexing
Linking entities via semantic indexingLinking entities via semantic indexing
Linking entities via semantic indexing
 
Semantic indexing for KOS
Semantic indexing for KOSSemantic indexing for KOS
Semantic indexing for KOS
 
Contextualization of topics - browsing through terms, authors, journals and c...
Contextualization of topics - browsing through terms, authors, journals and c...Contextualization of topics - browsing through terms, authors, journals and c...
Contextualization of topics - browsing through terms, authors, journals and c...
 
Exploring a world of networked information built from free-text metadata
Exploring a world of networked information built from free-text metadataExploring a world of networked information built from free-text metadata
Exploring a world of networked information built from free-text metadata
 
Ariadne's Thread -- Exploring a world of networked information built from fre...
Ariadne's Thread -- Exploring a world of networked information built from fre...Ariadne's Thread -- Exploring a world of networked information built from fre...
Ariadne's Thread -- Exploring a world of networked information built from fre...
 
Learning Concept Mappings from Instance Similarity
Learning Concept Mappings from Instance SimilarityLearning Concept Mappings from Instance Similarity
Learning Concept Mappings from Instance Similarity
 
Measuring the dynamic bi-directional influence between content and social ne...
Measuring the dynamic bi-directional influence between content and social ne...Measuring the dynamic bi-directional influence between content and social ne...
Measuring the dynamic bi-directional influence between content and social ne...
 
Similarity Features, and their Role in Concept Alignment Learning
Similarity Features, and their Role in Concept Alignment Learning Similarity Features, and their Role in Concept Alignment Learning
Similarity Features, and their Role in Concept Alignment Learning
 
What is concept dirft and how to measure it?
What is concept dirft and how to measure it?What is concept dirft and how to measure it?
What is concept dirft and how to measure it?
 
ECCS 2010
ECCS 2010ECCS 2010
ECCS 2010
 
Study concept drift in political ontologies
Study concept drift in political ontologiesStudy concept drift in political ontologies
Study concept drift in political ontologies
 

ICA Slides

  • 1. What is the problem? How can we deal with concept drift? Summary Extensional Mapping-Chains for studying Concept Drift in Political Ontologies Shenghui Wang1 Stefan Schlobach2 Janet Takens3 Wouter van Atteveldt3 1 The Network Institute 2 Department of Computer Science 3 Department of Communication Science Vrije Universiteit Amsterdam ICA 2010 Singapore
  • 2. What is the problem? How can we deal with concept drift? Summary Content analysis in Communication Science Communication scientists study all sorts of media content related to human communication Content analysis based on the NET method concepts: political actors and issues relations: associations, opinions, or actions. Example Het Openbaar Ministerie (OM) wil de komende vier jaar mensen- handel uitroeien.
  • 3. What is the problem? How can we deal with concept drift? Summary Content analysis in Communication Science Communication scientists study all sorts of media content related to human communication Content analysis based on the NET method concepts: political actors and issues relations: associations, opinions, or actions. Example Het Openbaar Ministerie (OM) wil de komende vier jaar mensen- handel uitroeien.
  • 4. What is the problem? How can we deal with concept drift? Summary Content analysis in Communication Science Communication scientists study all sorts of media content related to human communication Content analysis based on the NET method concepts: political actors and issues relations: associations, opinions, or actions. Example Het Openbaar Ministerie (OM) wil de komende vier jaar mensen- handel uitroeien. -1 om human trafficking
  • 5. What is the problem? How can we deal with concept drift? Summary Semantic network analysis 2271 1067 1388 740 1373 2127 2467 1268 2351 907 1223 1739 1708 2516 1706 2393 2438 1721 1077 1052 2323 1234 1034 2394 964 1275 2120 1936 1059 1045 2221 2124 2753 2171 2653 752 2655 2054 856 693 1608 341 2647 1011 2806 1145 1625 2076 648 1386 1233 1545 329 480 361 475 845 1906 2259 883 1439 2090 1474 1332 2002 2377 1306 2077 548 654 2696 2606 2492 1635 2471 484 956 889 2199 1198 623 2614 2573 545 1409 1259 2251 1940 2751 539 1827 870 2186 2151 2423 464 1097 1841 2070 1073 1932 438 2403
  • 6. What is the problem? How can we deal with concept drift? Summary Network-based communication science study What information can we extract from these networks? Politicians are networking Politics is perceived by citizens via media Media study by semantic network analysis Who is determining the subjects? Who is teaming up? Who is more credible? Who owns which topic?
  • 7. What is the problem? How can we deal with concept drift? Summary Before network analysis We first need to build the networks! Requires: large corpora with annotated textual content Manual coding against coding books (ontologies) Automated content analysis in progress
  • 8. What is the problem? How can we deal with concept drift? Summary Before network analysis We first need to build the networks! Requires: large corpora with annotated textual content Manual coding against coding books (ontologies) Automated content analysis in progress
  • 9. What is the problem? How can we deal with concept drift? Summary What is the problem? Problems with constructing annotated content Data from different time periods or genres Coded by different teams at different moments Manifesto Research Group: 25 countries, from 1945 to 2006 Comparative Policy Agendas project: media content, manifestos, legislative texts, government press statements, etc. Election campaign coverage from 1994 to 2006
  • 10. What is the problem? How can we deal with concept drift? Summary What are the challenges? Interoperability problem while sharing information Different teams use different code books Example illegal immigration labour migrants Different coding books should be merged or at least connected Not the focus of this paper
  • 11. What is the problem? How can we deal with concept drift? Summary What are the challenges? Interoperability problem while sharing information Different teams use different code books Example illegal immigration labour migrants Different coding books should be merged or at least connected Not the focus of this paper
  • 12. What is the problem? How can we deal with concept drift? Summary What are the challenges? Interoperability problem while sharing information Different teams use different code books Example illegal immigration labour migrants Different coding books should be merged or at least connected Not the focus of this paper
  • 13. What is the problem? How can we deal with concept drift? Summary Follow the Fashion?
  • 14. What is the problem? How can we deal with concept drift? Summary Women’s role? Suffragettes said that women’s role in society is unacceptable Pope says that women’s role in society is unacceptable
  • 15. What is the problem? How can we deal with concept drift? Summary Concept drift Our problem: Concept drift Meaning of concepts changes over time Analysis based on evolving concepts must consider temporal locality Study concept drift itself is useful
  • 16. What is the problem? How can we deal with concept drift? Summary Datasets Five political ontologies which were used to annotate newspaper articles 23 639 manually annotated newspaper articles during five recent Dutch national election campaigns There even exist manual mappings but most of them are lexically very similar
  • 17. What is the problem? How can we deal with concept drift? Summary Detecting concept drift We use extensional mapping techniques Consider concepts at different time to be different concepts Use extensional method to detect the links between concepts at different time Assumption: similar sentences should be coded with similar concepts, therefore, similar concepts should have similar extension.
  • 18. What is the problem? How can we deal with concept drift? Summary Representing concept drift using mapping chains
  • 19. What is the problem? How can we deal with concept drift? Summary Evaluating concept drift What can we learn from those chains? Do they agree with the political reality? Do they tell us something we do not noticed before? Are some concepts more stable/unstable than others? Quantitative evaluation is interesting, but qualitative analysis seems to tell us something too.
  • 20. What is the problem? How can we deal with concept drift? Summary Qualitative analysis of mapping chains Association vs. similarity Early erroneous associations can turn large parts of the analysis practically useless.
  • 21. What is the problem? How can we deal with concept drift? Summary Qualitative analysis of mapping chains Association vs. similarity Early erroneous associations can turn large parts of the analysis practically useless.
  • 22. What is the problem? How can we deal with concept drift? Summary “productiviteit” (Productivity) 06_economic growth 0.0880 03_economische groei 0.0315 0.0569 06_begroting 0.0327 0.0499 0.0657 02_economische groei 03_financieringstekort 0.0336 06_bezuinigingen 0.0387 94_productiviteit 98_welvaart valence 0.0587 02_welvaart 0.0361 0.1505 03_spaarloon 06_spaarloon 0.0518 06_levensloopregeling “euthanasie” (Euthanasia) 0.1789 06_gay marriage 03_euthanasie 0.1704 0.2999 0.3491 06_abortion 0.2519 02_euthanasie 03_homohuwelijk 0.1883 0.1057 06_verbetering communicatie overheid burger 0.0165 0.2185 03_milieuactivist 0.0425 0.0768 06_criminelen 98_oeuthanasie 02_milieuactivist 0.0507 0.0310 0.2636 03_justitie 0.0291 06_asielzoekers 94_euthanasie 0.0457 06_gratis schoolboeken 0.0441 0.1117 03_referendum eu 0.0313 0.1016 98_hreferendum 02_referendum 0.0432 06_referendum 0.0571 03_referendum 0.0454 0.0882 06_burgerinitiatief 06_werknemers 0.0548 0.0293 02_cdavvdlpf 03_zondagsrust 0.0398 0.0257 06_sunday rest 0.0511 03_scholieren 06_leerlingen 0.0286 06_education
  • 23. What is the problem? How can we deal with concept drift? Summary If we know two end-point concepts have the same meaning Kite-shaped chains 02_mensenrechten 98_avluchtelingen 02_jusititie 03_politie 02_criminaliteit 03_justitie 98_rcriminaliteit 02_drugkoeriers 02_cellentekort 03_criminaliteit 94_asielzoekers 98_kabinet kokmierlods 02_bedrijfsleven 02_democratie 98_asielzoekers 02_buitenlanders 03_asielzoekers 06_asielzoekers 98_okerken 02_instroom beperking 03_opvang illegalen 02_asielzoekers 03_vluchtelingen 03_illegalen
  • 24. What is the problem? How can we deal with concept drift? Summary “christelijken” (Christians) 98_oabortus 02_normen waarden 94_christelijken 03_multiculturele samenleving 06_christenen 98_ochristelijk christenen 02_multiculturele samenleving “asielzoekers” (Asylum seeker) 02_mensenrechten 98_avluchtelingen 02_jusititie 03_politie 02_criminaliteit 03_justitie 98_rcriminaliteit 02_drugkoeriers 02_cellentekort 03_criminaliteit 94_asielzoekers 98_kabinet kokmierlods 02_bedrijfsleven 02_democratie 98_asielzoekers 02_buitenlanders 03_asielzoekers 06_asielzoekers 98_okerken 02_instroom beperking 03_opvang illegalen 02_asielzoekers 03_vluchtelingen 03_illegalen
  • 25. What is the problem? How can we deal with concept drift? Summary Summary By looking at extensions of concepts, we can detect concept drift Domain experts found that the detected concept drift makes sense Automated matching techniques can help domain experts to find hidden links between concepts More work needs to be done