SlideShare a Scribd company logo
1 of 84
Recommender Systems
            and Learning Analytics in TEL




Hendrik Drachsler
Centre for Learning Sciences and Technology
Open University of the Netherlands 1
Goals of the lecture
1. Crash course Recommender Systems (RecSys)

2. Overview of RecSys in TEL

3. Open research issues for RecSys in TEL

4.TEL RecSys and Learning Analytics




                        2
Introduction into
Recommender Systems
     Introduction       Objectives

                                 Technologies

                                     Evaluation




                    3
Introduction::Application areas
 Application areas
 • E-commerce websites (Amazon)
 • Video, Music websites (Netflix, last.fm)
 • Content websites (CNN, Google News)
 • Information Support Systems

Major claims
 • Highly application-oriented research area, every domain and
  task needs a specific RecSys
 • Always build around content or products they never
  exist as on their own


                                4
Introduction::Definition
Using the opinions of a community of users to
help individuals in that community to identify more
effectively content of interest from a potentially
overwhelming set of choices.
Resnick & Varian (1997). Recommender Systems, Communications of the ACM, 40(3).




                                       5
Introduction::Definition
Using the opinions of a community of users to
help individuals in that community to identify more
effectively content of interest from a potentially
overwhelming set of choices.
Resnick & Varian (1997). Recommender Systems, Communications of the ACM, 40(3).


Any system that produces personalized
recommendations as output or has the effect of
guiding the user in a personalized way to interesting
or useful objects in a large space of possible options.
Burke R. (2002). Hybrid Recommender Systems: Survey and Experiments,
User Modeling & User Adapted Interaction, 12, pp. 331-370.

                                       5
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example




           6
Introduction::Example

What did we learn from the small exercise?
  • There are different kinds of recommendations
  a. People who bought X also bought Y
  b. there are more advanced personalized recommendations

   • When registering, we have to tell the RecSys what we like
   (and what not). Thus, it requires information to offer suitable
   recommendations and it learns our preferences.




                                6
Introduction:: The Long Tail




Anderson, C., (2004). The Long Tail. Wired Magazine.
                                       7
Introduction:: The Long Tail



“We are leaving the age of information and
entering the age of recommendation”.
                              Anderson, C. (2004)




Anderson, C., (2004). The Long Tail. Wired Magazine.
                                       7
Introduction:: Age of RecSys?
      ...10 minutes on Google.




                   8
Introduction:: Age of RecSys?
      ...10 minutes on Google.




                   8
Introduction:: Age of RecSys?
... another 10 minutes, research on RecSys is
  becoming main stream.
Some examples:
– ACM RecSys conference
– ICWSM: Weblog and Social Media
– WebKDD: Web Knowledge Discovery and Data Mining
– WWW: The original WWW conference
– SIGIR: Information Retrieval
– ACM KDD: Knowledge Discovery and Data Mining
– LAK: Learning Analytics and Knowledge
– Educational data mining conference
– ICML: Machine Learning
– ...

... and various workshops, books, and journals.

                                9
Objectives
of RecSys       probabilistic combination of
                – Item-based method
                – User-based method
                – Matrix Factorization
                – (May be) content-based method



                The idea is to pick from my
                previous list 20-50 movies that
                share similar audience with
                “Taken”, then how much I will like
                depend on how much I liked those
                early movies
                – In short: I tend to watch this movie
                because I have watched those
                movies … or
             10
                – People who have watched those
                movies also liked this movie
                (Amazon style)
Objectives::Aims

• Converting Browsers into
    Buyers

• Increasing Cross-sales
• Building Loyalty
                                                          Foto by markhillary




Schafer, Konstan & Riedel, (1999). RecSys in e-commerce. Proc. of the 1st ACM on
electronic commerce, Denver, Colorado, pp. 158-169.
                                         11
Objectives::RecSys Tasks
Find good items
presenting a ranked list of
recommendendations.


                                               probabilistic combination of
                                               – Item-based method
                                               – User-based method
                                               – Matrix Factorization
                                               – (May be) content-based method
Find all good items
user wants to identify all
                                              The idea is to pick from my
items that might be                           previous list 20-50 movies that
                                              share similar audience with
interesting, e.g. medical                     “Taken”, then how much I will like
                                              depend on how much I liked those
or legal cases                                early movies
                                              – In short: I tend to watch this movie
                                              because I have watched those
Herlocker, Konstan, Borchers, & Riedl (2004). Evaluating Collaborative Filtering
                                              movies … or
Recommender Systems. ACM Transactions on–Informationhave watched those pp. 5-53.
                                           12
                                                People who Systems, 22(1),
                                              movies also liked this movie
                                              (Amazon style)
Objectives::RecSys Tasks
Find good items                             Receive sequence of items
presenting a ranked list of                 sequence of related items is
recommendendations.                         recommended to the user,
                                            e.g. music recommender
                                               probabilistic combination of
                                               – Item-based method
                                               – User-based method
                                               – Matrix Factorization
Find all good items                         Annotation in context
                                               – (May be) content-based method

user wants to identify all                  predicted usefulness of an
items that might be                         item that pick from mythatis currently
                                              The idea is to the user
                                              previous list 20-50 movies
interesting, e.g. medical                   viewing, e.g. linkslike
                                              share similar audience with within a
                                              “Taken”, then how much I will
or legal cases                              websitehow much I liked those
                                              depend on
                                              early movies
                                              – In short: I tend to watch this movie
                                              because I have watched those
Herlocker, Konstan, Borchers, & Riedl (2004). Evaluating Collaborative Filtering
                                              movies … or
Recommender Systems. ACM Transactions on–Informationhave watched those pp. 5-53.
                                           12
                                                People who Systems, 22(1),
                                              movies also liked this movie
                                              (Amazon style)
Objectives::RecSys Tasks
Find good items                             Receive sequence of items
presenting a ranked list of                 sequence of related items is
recommendendations.                         recommended to the user,
                                            e.g. music recommender

                 There are more tasks available... of
                                  probabilistic combination
                                  – Item-based method
                                               – User-based method
                                               – Matrix Factorization
Find all good items                         Annotation in context
                                               – (May be) content-based method

user wants to identify all                  predicted usefulness of an
items that might be                         item that pick from mythatis currently
                                              The idea is to the user
                                              previous list 20-50 movies
interesting, e.g. medical                   viewing, e.g. linkslike
                                              share similar audience with within a
                                              “Taken”, then how much I will
or legal cases                              websitehow much I liked those
                                              depend on
                                              early movies
                                              – In short: I tend to watch this movie
                                              because I have watched those
Herlocker, Konstan, Borchers, & Riedl (2004). Evaluating Collaborative Filtering
                                              movies … or
Recommender Systems. ACM Transactions on–Informationhave watched those pp. 5-53.
                                           12
                                                People who Systems, 22(1),
                                              movies also liked this movie
                                              (Amazon style)
RecSys Technologies
1. Predict how much a user
  may like a certain product

2. Create a list of Top-N
  best items

3. Adjust its prediction
  based on feedback of the
  target user and like-
  minded users
Hanani et al., (2001). Information Filtering: Overview of Issues, Research and Systems",
  User Modeling and User-Adapted Interaction, 11.
                                            13
RecSys Technologies
1. Predict how much a user
  may like a certain product

2. Create a list of Top-N
  best items

3. Adjust its prediction
  based on feedback of the                         Just some examples
  target user and like-                              there are more
  minded users                                    technologies available.
Hanani et al., (2001). Information Filtering: Overview of Issues, Research and Systems",
  User Modeling and User-Adapted Interaction, 11.
                                            13
Technologies::Collaborative filtering




  User-based filtering
  (Grouplens, 1994)

Take about 20-50 people who share
similar taste with you, afterwards
predict how much you might like an
item depended on how much the others
liked it.

You may like it because your
“friends” liked it.
                                       14
Technologies::Collaborative filtering




  User-based filtering                             Item-based filtering
  (Grouplens, 1994)                                 (Amazon, 2001)

Take about 20-50 people who share           Pick from your previous list 20-50 items
similar taste with you, afterwards          that share similar people with “the
predict how much you might like an          target item”, how much you will like the
item depended on how much the others        target item depends on how much the
liked it.                                   others liked those earlier items.

You may like it because your                You tend to like that item because
“friends” liked it.                         you have liked those items.
                                       14
Technologies::Content-based filtering




  Information needs of user and characteristics of items are
    represented in keywords, attributes, tags that describe
    past selections, e.g., TF-IDF.




                              15
Technologies::Hybrid RecSys
Combination of techniques to overcome
disadvantages and advantages of single techniques.

 Advantages                   Disadvantages
                             probabilistic combination of
                             – Item-based method
• No content analysis        • Cold-start problem
                             – User-based method
                             – Matrix Factorization

• Quality improves           • Over-fitting
                             – (May be) content-based method


• No cold-start problem      • New user / item problem
                            The idea is to pick from my
• No new user / item         • Sparsity
                            previous list 20-50 movies that
                            share similar audience with
  problem                    “Taken”, then how much I will like
                             depend on how much I liked those
                             early movies
                             – In short: I tend to watch this movie
                             because I have watched those
                             movies … or
                          16
                             – People who have watched those
                             movies also liked this movie
                             (Amazon style)
Technologies::Hybrid RecSys
Combination of techniques to overcome
disadvantages and advantages of single techniques.

 Advantages                     Disadvantages
                            probabilistic combination of
                            – Item-based method
• No content analysis          • Cold-start problem
                            – User-based method
                            – Matrix Factorization

• Quality improves             • Over-fitting
                            – (May be) content-based method


• No cold-start problem        • New user / item problem
                             The idea is to pick from my
• No new user / item           • Sparsity
                             previous list 20-50 movies that
                             share similar audience with
  problem                    “Taken”, then how much I will like
                              Just some examples there
                             depend on how much I liked those
                             early movies
                             –are more watch this movie
                               In short: I tend to (dis)advantages
                             because I have watched those

                          16
                             movies … or
                                               available.
                             – People who have watched those
                           movies also liked this movie
                           (Amazon style)
Evaluation
of RecSys
                probabilistic combination of
                – Item-based method
                – User-based method
                – Matrix Factorization
                – (May be) content-based method



                The idea is to pick from my
                previous list 20-50 movies that
                share similar audience with
                “Taken”, then how much I will like
                depend on how much I liked those
                early movies
                – In short: I tend to watch this movie
                because I have watched those
                movies … or
             17
                – People who have watched those
                movies also liked this movie
                (Amazon style)
Evaluation::General idea
    Most of the time based on performance measures
      (“How good are your recommendations?”)

For example:

•Predict what rating will a user give an item?
•Will the user select an item?
•What is the order of usefulness of items to a user?

Herlocker, Konstan, Riedl (2004). Evaluating Collaborative Filtering Recommender
Systems. ACM Transactions on Information Systems, 22(1), 5-53.
                                          18
Evaluation::Reference datasets




         ... and various commercial datasets.
                19
Evaluation::Approaches
                       1. Simulation
•User preference
•Prediction accuracy
•Coverage
•Confidence
•Trust
•Novelty               2. User study
•Serendipity
•Diversity
•Utility
•Risk
•Robustness                            +
•Privacy
•Adaptivity
•Scalability
                               20
Evaluation::Metrics
 Precision – The portion of
 recommendations that were
 successful. (Selected by the
 algorithm and by the user)

 Recall – The portion of relevant
 items selected by algorithm
 compared to a total number of
 relevant items available.

 F1 - Measure balances Precision
 and Recall into a single
 measurement.

Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of
Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962,
2009.                                     21
Evaluation::Metrics
 Precision – The portion of
 recommendations that were
 successful. (Selected by the
 algorithm and by the user)

 Recall – The portion of relevant
 items selected by algorithm
 compared to a total number of
 relevant items available.

 F1 - Measure balances Precision
 and Recall into a single
 measurement.

Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of
Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962,
2009.                                     21
Evaluation::Metrics
 Precision – The portion of
 recommendations that were
 successful. (Selected by the
 algorithm and by the user)

 Recall – The portion of relevant
 items selected by algorithm
 compared to a total number of
 relevant items available.

 F1 - Measure balances Precision
 and Recall into a single
 measurement.

Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of
Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962,
2009.                                     21
Evaluation::Metrics
 Precision – The portion of
 recommendations that were
 successful. (Selected by the
 algorithm and by the user)

 Recall – The portion of relevant
 items selected by algorithm
 compared to a total number of
 relevant items available.

 F1 - Measure balances Precision            Just some examples there
 and Recall into a single                   are more metrics available
 measurement.                                     like MAE, RSME.
Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of
Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962,
2009.                                     21
Evaluation::Metrics
                                       5
 Conclusion:
                                       4
 Pearson is better




                                RMSE
 than Cosine,                          3
                                                                         Pearson
 because less                          2
 errors in predicting                                                    Cosine
                                       1
 TOP-N items.                          0
                                           Netflix     BookCrossing




Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of
Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962,
2009.                                     22
Evaluation::Metrics
                                             5
 Conclusion:
                                             4
 Pearson is better




                                RMSE
 than Cosine,                                3
                                                                                                  Pearson
 because less                                2
 errors in predicting                                                                             Cosine
                                             1
 TOP-N items.                                0
                                                         Netflix          BookCrossing


                                                        News Story Clicks
 Conclusion:                                 80%

 Cosine better than              Precision
                                             60%
 Pearson, because
                                             40%
 of higher precision
                                             20%
 and recall value on
 TOP-N items.                                0%
                                                   5%   10%   15%   20%   25%   30%   35%   40%

                                                       Recall
Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of
Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962,
2009.                                     22
RecSys::TimeToThink
What do you expect that a RecSys in a
MUP/PLE should do with respect to ...

• Aims
• Tasks
• Technology
                        Blackmore’s custom-built LSD Drive

• Evaluation            http://www.flickr.com/photos/
                        rootoftwo/



                   23
Goals of the lecture
1. Crash course Recommender Systems (RecSys)

2. Overview of RecSys in TEL

3. Open research issues for RecSys in TEL

4.TEL RecSys and Learning Analytics




                       24
Recommender Systems
for TEL
    Introduction        Objectives

                                 Technologies

                                     Evaluation




                   25
TEL RecSys::Definition
     Using the experiences of a community of
     learners to help individual learners in that
     community to identify more effectively learning
     content from a potentially overwhelming set of
     choices.
Extended definition by Resnick & Varian (1997). Recommender Systems, Communications of the
  ACM, 40(3).




                                           26
Find

TEL RecSys::Learning spectrum                     M
                                                   fo
                                                   co


                                                 Maj
                                                  sy
                                                  fa
                                                 We
                                                  ap




Cross, J., Informal learning. Pfeifer. (2006).
                                         27
The Long Tail




Graphic: Wilkins, D., (2009).   28
The Long Tail of Learning




Graphic: Wilkins, D., (2009).   28
The Long Tail of Learning
          Formal

                                 Informal




Graphic: Wilkins, D., (2009).   28
TEL RecSys::Technologies




           29
TEL RecSys:: Technologies




            30
TEL RecSys:: Technologies




            31
TEL RecSys:: Technologies


          RecSys Task:
          Find good items

          Hybrid RecSys:
          •Content-based on
           interests
          •Collaborative filtering


            31
TEL RecSys::Tasks
  Find good items
  e.g. relevant items for a learning
     task or a learning goal




                                                 The idea is to pick from my
                                                 previous list 20-50 movies that
                                                 share similar audience with
                                                 “Taken”, then how much I will like
                                                 depend on how much I liked those
                                                 early movies
                                                 – In short: I tend to watch this movie
                                                 because I have watched those
Drachsler, H., Hummel, H., Koper, R., (2009). Identifying the goal, user model and conditions of
                                                 movies … or
     recommender systems for formal and informal–learning. Journal watched those
                                              32
                                                   People who have of Digital Information. 10(2).
                                                 movies also liked this movie
                                                 (Amazon style)
TEL RecSys::Tasks
  Find good items
  e.g. relevant items for a learning
     task or a learning goal


 Receive sequence of items
 e.g. recommend a learning path
     to achieve a certain
     competence

                                                 The idea is to pick from my
                                                 previous list 20-50 movies that
                                                 share similar audience with
                                                 “Taken”, then how much I will like
                                                 depend on how much I liked those
                                                 early movies
                                                 – In short: I tend to watch this movie
                                                 because I have watched those
Drachsler, H., Hummel, H., Koper, R., (2009). Identifying the goal, user model and conditions of
                                                 movies … or
     recommender systems for formal and informal–learning. Journal watched those
                                              32
                                                   People who have of Digital Information. 10(2).
                                                 movies also liked this movie
                                                 (Amazon style)
TEL RecSys::Tasks
  Find good items
  e.g. relevant items for a learning
     task or a learning goal


 Receive sequence of items
 e.g. recommend a learning path
     to achieve a certain
     competence

Annotation in context                            The idea is to pick from my
e.g. take into account location,                 previous list 20-50 movies that
                                                 share similar audience with
     time, noise level, prior                    “Taken”, then how much I will like
     knowledge, peers around                     depend on how much I liked those
                                                 early movies
                                                 – In short: I tend to watch this movie
                                                 because I have watched those
Drachsler, H., Hummel, H., Koper, R., (2009). Identifying the goal, user model and conditions of
                                                 movies … or
     recommender systems for formal and informal–learning. Journal watched those
                                              32
                                                   People who have of Digital Information. 10(2).
                                                 movies also liked this movie
                                                 (Amazon style)
Evaluation
 of TEL
 RecSys         probabilistic combination of
                – Item-based method
                – User-based method
                – Matrix Factorization
                – (May be) content-based method



                The idea is to pick from my
                previous list 20-50 movies that
                share similar audience with
                “Taken”, then how much I will like
                depend on how much I liked those
                early movies
                – In short: I tend to watch this movie
                because I have watched those
                movies … or
             33
                – People who have watched those
                movies also liked this movie
                (Amazon style)
TEL RecSys::Review study




             34
TEL RecSys::Review study




Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H. G. K., & Koper, R. (2011).
Recommender Systems in Technology Enhanced Learning. In P. B. Kantor, F. Ricci,
L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 387-415).
Berlin: Springer.                          34
TEL RecSys::Review study


      Conclusions:

      Half of the systems (11/20) still at design or prototyping stage
       only 8 systems evaluated through trials with human users.




Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H. G. K., & Koper, R. (2011).
Recommender Systems in Technology Enhanced Learning. In P. B. Kantor, F. Ricci,
L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 387-415).
Berlin: Springer.                          34
Thus...
“The performance results
of different research
efforts in recommender
systems are hardly
comparable.”

(Manouselis et al., 2010)
                                 Kaptain Kobold
                                 http://www.flickr.com/photos/
                                 kaptainkobold/3203311346/




                            35
Thus...
TEL recommender
 “The performance results
experiments lack
 of different research
 efforts in recommender
transparency. They need
 systems are hardly
to be repeatable to test:
 comparable.”
• Validity
•(Manouselis et al., 2010)
  Verification
• Compare results                 Kaptain Kobold
                                  http://www.flickr.com/photos/
                                  kaptainkobold/3203311346/




                             35
TEL RecSys::Evaluation/datasets




              36
TEL RecSys::Evaluation/datasets




Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G.,
Lindstaedt, S., Stern, H., Friedrich, M., & Wolpers, M. (2010). Issues and Considerations
regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning.
Presentation at the 1st Workshop Recommnder Systems in Technology Enhanced Learning
(RecSysTEL) in conjunction with 5th European Conference on Technology Enhanced
Learning (EC-TEL 2010): Sustaining TEL: From Innovation to Learning and Practice.
September, 28, 2010, Barcelona, Spain.         36
Evaluation::Metrics          MAE – Mean Absolute Error:
                                                   Deviation of recommendations
                                                   from the user-specified ratings.
                                                   The lower the MAE, the more
                                                   accurately the RecSys predicts user
                                                   ratings.




Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Beham, G., Duval, E.,
(2011). Dataset-driven Research for Improving Recommender Systems for Learning. Learning
Analytics & Knowledge: February 27-March 1,37  2011, Banff, Alberta, Canada
Evaluation::Metrics          MAE – Mean Absolute Error:
                                                   Deviation of recommendations
                                                   from the user-specified ratings.
                                                   The lower the MAE, the more
                                                   accurately the RecSys predicts user
                                                   ratings.




 Outcomes:
 Tanimoto similarity +
 item-based CF was
 the most accurate.


Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Beham, G., Duval, E.,
(2011). Dataset-driven Research for Improving Recommender Systems for Learning. Learning
Analytics & Knowledge: February 27-March 1,37  2011, Banff, Alberta, Canada
Evaluation::Metrics         MAE – Mean Absolute Error:
                                                   Deviation of recommendations
                                                   from the user-specified ratings.
                                                   The lower the MAE, the more
                                                   accurately the RecSys predicts user
                                                   ratings.




Outcomes:
•User-based CF Algorithm that
predicts the top 10 most relevant
 Outcomes:
items for a user has a F1 score
 Tanimoto similarity +
of almost 30%.
 item-based CF was
•the most accurate.
  Implicit ratings like download
 rates, bookmarks can
 successfully used in TEL.
Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Beham, G., Duval, E.,
(2011). Dataset-driven Research for Improving Recommender Systems for Learning. Learning
Analytics & Knowledge: February 27-March 1,37  2011, Banff, Alberta, Canada
TEL RecSys::Evaluation




Combined approach by          Kirkpatrick model by
 Drachsler et al. 2008        Manouselis et al. 2010
                         38
TEL RecSys::Evaluation
 1. Accuracy
 2. Coverage
 3. Precision




Combined approach by          Kirkpatrick model by
 Drachsler et al. 2008        Manouselis et al. 2010
                         38
TEL RecSys::Evaluation
 1. Accuracy
 2. Coverage
 3. Precision



 1. Effectiveness of learning
 2. Efficiency of learning
 3. Drop out rate
 4. Satisfaction


Combined approach by                 Kirkpatrick model by
 Drachsler et al. 2008               Manouselis et al. 2010
                                38
TEL RecSys::Evaluation
 1. Accuracy                           1. Reaction of learner
 2. Coverage                           2. Learning improved
 3. Precision                          3. Behaviour
                                       4. Results


 1. Effectiveness of learning
 2. Efficiency of learning
 3. Drop out rate
 4. Satisfaction


Combined approach by                 Kirkpatrick model by
 Drachsler et al. 2008               Manouselis et al. 2010
                                38
Goals of the lecture
1. Crash course Recommender Systems (RecSys)

2. Overview of RecSys in TEL

3. Open research issues for RecSys in TEL

4.TEL RecSys and Learning Analytics




                       39
TEL RecSys::Open issues

1. Evaluation of TEL RecSys
2. Publicly available datasets
3. Comparable experiments
4. Body of knowledge
5. Privacy and data protection
6. Design learning driven RecSys




                          40
Goals of the lecture
1. Crash course Recommender Systems (RecSys)

2. Overview of RecSys in TEL

3. Open research issues for RecSys in TEL

4.TEL RecSys and Learning Analytics




                       41
Greller, W., & Drachsler, H., 2011.
                          42
Greller, W., & Drachsler, H., 2011.
                          42
Greller, W., & Drachsler, H., 2011.
                          42
Greller, W., & Drachsler, H., 2011.
                          42
Greller, W., & Drachsler, H., 2011.
                          42
Greller, W., & Drachsler, H., 2011.
                          42
Greller, W., & Drachsler, H., 2011.
                          42
Learning Analytics::TimeToThink
 •   Consider the Learning Analytics
     framework and imagine some great TEL
     RecSys that could support you in your
     stakeholder role

     alternatively

 • Name a learning task where a TEL
     RecSys would be useful for.


                      43
Thank you for attending this lecture!
 This silde is available at:
 http://www.slideshare.com/Drachsler

 Email:       hendrik.drachsler@ou.nl
 Skype:       celstec-hendrik.drachsler
 Blogging at: http://www.drachsler.de
 Twittering at: http://twitter.com/HDrachsler


                      44

More Related Content

Viewers also liked

Phoenix Az Dept Of Commerce Tribal Energy And Renewable Presentation
Phoenix   Az Dept Of Commerce Tribal Energy And Renewable PresentationPhoenix   Az Dept Of Commerce Tribal Energy And Renewable Presentation
Phoenix Az Dept Of Commerce Tribal Energy And Renewable PresentationICF_HCD
 
Sharing Data Sets to Personalize Learning
Sharing Data Sets to Personalize LearningSharing Data Sets to Personalize Learning
Sharing Data Sets to Personalize LearningHendrik Drachsler
 
Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...
Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...
Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...Martin Rehm
 
So You Think You Can Speak: From Proposal to Podium
So You Think You Can Speak: From Proposal to PodiumSo You Think You Can Speak: From Proposal to Podium
So You Think You Can Speak: From Proposal to PodiumFootprint Consulting Group
 
A Clever Way to Scale-out a Web Application
A Clever Way to Scale-out a Web ApplicationA Clever Way to Scale-out a Web Application
A Clever Way to Scale-out a Web ApplicationKazuho Oku
 
Winners of HTML5 BYOG - InGDIn
Winners of HTML5 BYOG - InGDInWinners of HTML5 BYOG - InGDIn
Winners of HTML5 BYOG - InGDInKinshuk Sunil
 
Rexona City Tracks Mobile App Concept
Rexona City Tracks Mobile App ConceptRexona City Tracks Mobile App Concept
Rexona City Tracks Mobile App ConceptJorge Teixeira
 
Some emerging technologies for learning resource centres
Some emerging technologies for learning resource centresSome emerging technologies for learning resource centres
Some emerging technologies for learning resource centresLis Parcell
 
A Long Walk to Water - Lssn 12
A Long Walk to Water - Lssn 12A Long Walk to Water - Lssn 12
A Long Walk to Water - Lssn 12Terri Weiss
 
Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...
Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...
Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...valrit
 
Mis 2 Princesas Chico 2
Mis 2 Princesas Chico 2Mis 2 Princesas Chico 2
Mis 2 Princesas Chico 2Carlos Arrieta
 
Cijeli brojevi vježba
Cijeli brojevi vježbaCijeli brojevi vježba
Cijeli brojevi vježbaGavranica
 
Android Abc2009 Fall Shima091130 1
Android Abc2009 Fall Shima091130 1Android Abc2009 Fall Shima091130 1
Android Abc2009 Fall Shima091130 1shimay
 
Imd 100503 Herrie In De Keuken
Imd 100503 Herrie In De KeukenImd 100503 Herrie In De Keuken
Imd 100503 Herrie In De Keukenmr38schev
 
G1 clustering - content convergence v2 jve
G1 clustering - content convergence v2 jveG1 clustering - content convergence v2 jve
G1 clustering - content convergence v2 jveJuan V. Vidagany Espert
 

Viewers also liked (20)

Phoenix Az Dept Of Commerce Tribal Energy And Renewable Presentation
Phoenix   Az Dept Of Commerce Tribal Energy And Renewable PresentationPhoenix   Az Dept Of Commerce Tribal Energy And Renewable Presentation
Phoenix Az Dept Of Commerce Tribal Energy And Renewable Presentation
 
Happy Meals
Happy MealsHappy Meals
Happy Meals
 
Sharing Data Sets to Personalize Learning
Sharing Data Sets to Personalize LearningSharing Data Sets to Personalize Learning
Sharing Data Sets to Personalize Learning
 
Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...
Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...
Comparing Communities of Learning: Incoming Bachelor Students & Working Profe...
 
So You Think You Can Speak: From Proposal to Podium
So You Think You Can Speak: From Proposal to PodiumSo You Think You Can Speak: From Proposal to Podium
So You Think You Can Speak: From Proposal to Podium
 
Ethics privacy washington
Ethics privacy washingtonEthics privacy washington
Ethics privacy washington
 
A Clever Way to Scale-out a Web Application
A Clever Way to Scale-out a Web ApplicationA Clever Way to Scale-out a Web Application
A Clever Way to Scale-out a Web Application
 
Winners of HTML5 BYOG - InGDIn
Winners of HTML5 BYOG - InGDInWinners of HTML5 BYOG - InGDIn
Winners of HTML5 BYOG - InGDIn
 
Rexona City Tracks Mobile App Concept
Rexona City Tracks Mobile App ConceptRexona City Tracks Mobile App Concept
Rexona City Tracks Mobile App Concept
 
Iatefl 2014
Iatefl 2014Iatefl 2014
Iatefl 2014
 
SEO Explained
SEO ExplainedSEO Explained
SEO Explained
 
Selce 2013
Selce 2013Selce 2013
Selce 2013
 
Some emerging technologies for learning resource centres
Some emerging technologies for learning resource centresSome emerging technologies for learning resource centres
Some emerging technologies for learning resource centres
 
A Long Walk to Water - Lssn 12
A Long Walk to Water - Lssn 12A Long Walk to Water - Lssn 12
A Long Walk to Water - Lssn 12
 
Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...
Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...
Can Cities Learn from Each Other? The Hong Kong- Malmö Sustainability Collabo...
 
Mis 2 Princesas Chico 2
Mis 2 Princesas Chico 2Mis 2 Princesas Chico 2
Mis 2 Princesas Chico 2
 
Cijeli brojevi vježba
Cijeli brojevi vježbaCijeli brojevi vježba
Cijeli brojevi vježba
 
Android Abc2009 Fall Shima091130 1
Android Abc2009 Fall Shima091130 1Android Abc2009 Fall Shima091130 1
Android Abc2009 Fall Shima091130 1
 
Imd 100503 Herrie In De Keuken
Imd 100503 Herrie In De KeukenImd 100503 Herrie In De Keuken
Imd 100503 Herrie In De Keuken
 
G1 clustering - content convergence v2 jve
G1 clustering - content convergence v2 jveG1 clustering - content convergence v2 jve
G1 clustering - content convergence v2 jve
 

Similar to Guest lecture Recommender Systems in TEL at RWTH Aachen, Germany

Introduction to recommendation system
Introduction to recommendation systemIntroduction to recommendation system
Introduction to recommendation systemAravindharamanan S
 
OMRES-ProgressPresentation1.pptx
OMRES-ProgressPresentation1.pptxOMRES-ProgressPresentation1.pptx
OMRES-ProgressPresentation1.pptx045HridikGulatiT17
 
Toward the Next Generation of Recommender Systems:
Toward the Next Generation of Recommender Systems: Toward the Next Generation of Recommender Systems:
Toward the Next Generation of Recommender Systems: Vincent Chu
 
MLforIR.pps
MLforIR.ppsMLforIR.pps
MLforIR.ppsbutest
 
Social Recommender Systems Tutorial - WWW 2011
Social Recommender Systems Tutorial - WWW 2011Social Recommender Systems Tutorial - WWW 2011
Social Recommender Systems Tutorial - WWW 2011idoguy
 
A Survey Of Collaborative Filtering Techniques
A Survey Of Collaborative Filtering TechniquesA Survey Of Collaborative Filtering Techniques
A Survey Of Collaborative Filtering Techniquestengyue5i5j
 
MLforIR.pps
MLforIR.ppsMLforIR.pps
MLforIR.ppsbutest
 
Chapter 02 collaborative recommendation
Chapter 02   collaborative recommendationChapter 02   collaborative recommendation
Chapter 02 collaborative recommendationAravindharamanan S
 
Chapter 02 collaborative recommendation
Chapter 02   collaborative recommendationChapter 02   collaborative recommendation
Chapter 02 collaborative recommendationAravindharamanan S
 
Mendeley: Recommendation Systems for Academic Literature
Mendeley: Recommendation Systems for Academic LiteratureMendeley: Recommendation Systems for Academic Literature
Mendeley: Recommendation Systems for Academic LiteratureKris Jack
 
Mechanical Librarian
Mechanical LibrarianMechanical Librarian
Mechanical LibrarianAndre Vellino
 
Recommendation Systems Basics
Recommendation Systems BasicsRecommendation Systems Basics
Recommendation Systems BasicsJarin Tasnim Khan
 
Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17Longqi Yang
 
Construindo Sistemas de Recomendação com Python
Construindo Sistemas de Recomendação com PythonConstruindo Sistemas de Recomendação com Python
Construindo Sistemas de Recomendação com PythonMarcel Caraciolo
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringViet-Trung TRAN
 

Similar to Guest lecture Recommender Systems in TEL at RWTH Aachen, Germany (20)

Introduction to recommendation system
Introduction to recommendation systemIntroduction to recommendation system
Introduction to recommendation system
 
WORD
WORDWORD
WORD
 
OMRES-ProgressPresentation1.pptx
OMRES-ProgressPresentation1.pptxOMRES-ProgressPresentation1.pptx
OMRES-ProgressPresentation1.pptx
 
Toward the Next Generation of Recommender Systems:
Toward the Next Generation of Recommender Systems: Toward the Next Generation of Recommender Systems:
Toward the Next Generation of Recommender Systems:
 
MLforIR.pps
MLforIR.ppsMLforIR.pps
MLforIR.pps
 
Social Recommender Systems Tutorial - WWW 2011
Social Recommender Systems Tutorial - WWW 2011Social Recommender Systems Tutorial - WWW 2011
Social Recommender Systems Tutorial - WWW 2011
 
A Survey Of Collaborative Filtering Techniques
A Survey Of Collaborative Filtering TechniquesA Survey Of Collaborative Filtering Techniques
A Survey Of Collaborative Filtering Techniques
 
MLforIR.pps
MLforIR.ppsMLforIR.pps
MLforIR.pps
 
Chapter 02 collaborative recommendation
Chapter 02   collaborative recommendationChapter 02   collaborative recommendation
Chapter 02 collaborative recommendation
 
Chapter 02 collaborative recommendation
Chapter 02   collaborative recommendationChapter 02   collaborative recommendation
Chapter 02 collaborative recommendation
 
Mendeley: Recommendation Systems for Academic Literature
Mendeley: Recommendation Systems for Academic LiteratureMendeley: Recommendation Systems for Academic Literature
Mendeley: Recommendation Systems for Academic Literature
 
Mechanical Librarian
Mechanical LibrarianMechanical Librarian
Mechanical Librarian
 
Recommenders.ppt
Recommenders.pptRecommenders.ppt
Recommenders.ppt
 
Recommenders.ppt
Recommenders.pptRecommenders.ppt
Recommenders.ppt
 
Recommendation Systems Basics
Recommendation Systems BasicsRecommendation Systems Basics
Recommendation Systems Basics
 
Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17Immersive Recommendation Workshop, NYC Media Lab'17
Immersive Recommendation Workshop, NYC Media Lab'17
 
Construindo Sistemas de Recomendação com Python
Construindo Sistemas de Recomendação com PythonConstruindo Sistemas de Recomendação com Python
Construindo Sistemas de Recomendação com Python
 
Collab filtering-tutorial
Collab filtering-tutorialCollab filtering-tutorial
Collab filtering-tutorial
 
Culbert recommender systems
Culbert recommender systemsCulbert recommender systems
Culbert recommender systems
 
Recommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filteringRecommender systems: Content-based and collaborative filtering
Recommender systems: Content-based and collaborative filtering
 

More from Hendrik Drachsler

Trusted Learning Analytics Research Program
Trusted Learning Analytics Research ProgramTrusted Learning Analytics Research Program
Trusted Learning Analytics Research ProgramHendrik Drachsler
 
Smart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao ParganaSmart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao ParganaHendrik Drachsler
 
Verhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning AnalyticsVerhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning AnalyticsHendrik Drachsler
 
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...Hendrik Drachsler
 
E.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic datasetE.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic datasetHendrik Drachsler
 
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...Hendrik Drachsler
 
Towards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning AnalyticsTowards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning AnalyticsHendrik Drachsler
 
Fighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-levelFighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-levelHendrik Drachsler
 
LACE Project Overview and Exploitation
LACE Project Overview and ExploitationLACE Project Overview and Exploitation
LACE Project Overview and ExploitationHendrik Drachsler
 
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the ConsistentDutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the ConsistentHendrik Drachsler
 
Recommendations for Open Online Education: An Algorithmic Study
Recommendations for Open Online Education:  An Algorithmic StudyRecommendations for Open Online Education:  An Algorithmic Study
Recommendations for Open Online Education: An Algorithmic StudyHendrik Drachsler
 
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...Hendrik Drachsler
 
DELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning AnalyticsDELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning AnalyticsHendrik Drachsler
 
The Future of Big Data in Education
The Future of Big Data in EducationThe Future of Big Data in Education
The Future of Big Data in EducationHendrik Drachsler
 
The Future of Learning Analytics
The Future of Learning AnalyticsThe Future of Learning Analytics
The Future of Learning AnalyticsHendrik Drachsler
 
Six dimensions of Learning Analytics
Six dimensions of Learning AnalyticsSix dimensions of Learning Analytics
Six dimensions of Learning AnalyticsHendrik Drachsler
 
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store - Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store - Hendrik Drachsler
 
Ethics and Privacy in the Application of Learning Analytics (#EP4LA)
Ethics and Privacy in the Application of Learning Analytics (#EP4LA)Ethics and Privacy in the Application of Learning Analytics (#EP4LA)
Ethics and Privacy in the Application of Learning Analytics (#EP4LA)Hendrik Drachsler
 

More from Hendrik Drachsler (20)

Trusted Learning Analytics Research Program
Trusted Learning Analytics Research ProgramTrusted Learning Analytics Research Program
Trusted Learning Analytics Research Program
 
Smart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao ParganaSmart Speaker as Studying Assistant by Joao Pargana
Smart Speaker as Studying Assistant by Joao Pargana
 
Verhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning AnalyticsVerhaltenskodex Trusted Learning Analytics
Verhaltenskodex Trusted Learning Analytics
 
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
Rödling, S. (2019). Entwicklung einer Applikation zum assoziativen Medien Ler...
 
E.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic datasetE.Leute: Learning the impact of Learning Analytics with an authentic dataset
E.Leute: Learning the impact of Learning Analytics with an authentic dataset
 
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
Romano, G. (2019) Dancing Trainer: A System For Humans To Learn Dancing Using...
 
Towards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning AnalyticsTowards Tangible Trusted Learning Analytics
Towards Tangible Trusted Learning Analytics
 
Trusted Learning Analytics
Trusted Learning Analytics Trusted Learning Analytics
Trusted Learning Analytics
 
Fighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-levelFighting level 3: From the LA framework to LA practice on the micro-level
Fighting level 3: From the LA framework to LA practice on the micro-level
 
LACE Project Overview and Exploitation
LACE Project Overview and ExploitationLACE Project Overview and Exploitation
LACE Project Overview and Exploitation
 
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the ConsistentDutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
Dutch Cooking with xAPI Recipes, The Good, the Bad, and the Consistent
 
Recommendations for Open Online Education: An Algorithmic Study
Recommendations for Open Online Education:  An Algorithmic StudyRecommendations for Open Online Education:  An Algorithmic Study
Recommendations for Open Online Education: An Algorithmic Study
 
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
Privacy and Analytics – it’s a DELICATE Issue. A Checklist for Trusted Learni...
 
DELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning AnalyticsDELICATE checklist - to establish trusted Learning Analytics
DELICATE checklist - to establish trusted Learning Analytics
 
LACE Flyer 2016
LACE Flyer 2016 LACE Flyer 2016
LACE Flyer 2016
 
The Future of Big Data in Education
The Future of Big Data in EducationThe Future of Big Data in Education
The Future of Big Data in Education
 
The Future of Learning Analytics
The Future of Learning AnalyticsThe Future of Learning Analytics
The Future of Learning Analytics
 
Six dimensions of Learning Analytics
Six dimensions of Learning AnalyticsSix dimensions of Learning Analytics
Six dimensions of Learning Analytics
 
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store - Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
Learning Analytics Metadata Standards, xAPI recipes & Learning Record Store -
 
Ethics and Privacy in the Application of Learning Analytics (#EP4LA)
Ethics and Privacy in the Application of Learning Analytics (#EP4LA)Ethics and Privacy in the Application of Learning Analytics (#EP4LA)
Ethics and Privacy in the Application of Learning Analytics (#EP4LA)
 

Recently uploaded

COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 

Recently uploaded (20)

COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 

Guest lecture Recommender Systems in TEL at RWTH Aachen, Germany

  • 1. Recommender Systems and Learning Analytics in TEL Hendrik Drachsler Centre for Learning Sciences and Technology Open University of the Netherlands 1
  • 2. Goals of the lecture 1. Crash course Recommender Systems (RecSys) 2. Overview of RecSys in TEL 3. Open research issues for RecSys in TEL 4.TEL RecSys and Learning Analytics 2
  • 3. Introduction into Recommender Systems Introduction Objectives Technologies Evaluation 3
  • 4. Introduction::Application areas Application areas • E-commerce websites (Amazon) • Video, Music websites (Netflix, last.fm) • Content websites (CNN, Google News) • Information Support Systems Major claims • Highly application-oriented research area, every domain and task needs a specific RecSys • Always build around content or products they never exist as on their own 4
  • 5. Introduction::Definition Using the opinions of a community of users to help individuals in that community to identify more effectively content of interest from a potentially overwhelming set of choices. Resnick & Varian (1997). Recommender Systems, Communications of the ACM, 40(3). 5
  • 6. Introduction::Definition Using the opinions of a community of users to help individuals in that community to identify more effectively content of interest from a potentially overwhelming set of choices. Resnick & Varian (1997). Recommender Systems, Communications of the ACM, 40(3). Any system that produces personalized recommendations as output or has the effect of guiding the user in a personalized way to interesting or useful objects in a large space of possible options. Burke R. (2002). Hybrid Recommender Systems: Survey and Experiments, User Modeling & User Adapted Interaction, 12, pp. 331-370. 5
  • 15. Introduction::Example What did we learn from the small exercise? • There are different kinds of recommendations a. People who bought X also bought Y b. there are more advanced personalized recommendations • When registering, we have to tell the RecSys what we like (and what not). Thus, it requires information to offer suitable recommendations and it learns our preferences. 6
  • 16. Introduction:: The Long Tail Anderson, C., (2004). The Long Tail. Wired Magazine. 7
  • 17. Introduction:: The Long Tail “We are leaving the age of information and entering the age of recommendation”. Anderson, C. (2004) Anderson, C., (2004). The Long Tail. Wired Magazine. 7
  • 18. Introduction:: Age of RecSys? ...10 minutes on Google. 8
  • 19. Introduction:: Age of RecSys? ...10 minutes on Google. 8
  • 20. Introduction:: Age of RecSys? ... another 10 minutes, research on RecSys is becoming main stream. Some examples: – ACM RecSys conference – ICWSM: Weblog and Social Media – WebKDD: Web Knowledge Discovery and Data Mining – WWW: The original WWW conference – SIGIR: Information Retrieval – ACM KDD: Knowledge Discovery and Data Mining – LAK: Learning Analytics and Knowledge – Educational data mining conference – ICML: Machine Learning – ... ... and various workshops, books, and journals. 9
  • 21. Objectives of RecSys probabilistic combination of – Item-based method – User-based method – Matrix Factorization – (May be) content-based method The idea is to pick from my previous list 20-50 movies that share similar audience with “Taken”, then how much I will like depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those movies … or 10 – People who have watched those movies also liked this movie (Amazon style)
  • 22. Objectives::Aims • Converting Browsers into Buyers • Increasing Cross-sales • Building Loyalty Foto by markhillary Schafer, Konstan & Riedel, (1999). RecSys in e-commerce. Proc. of the 1st ACM on electronic commerce, Denver, Colorado, pp. 158-169. 11
  • 23. Objectives::RecSys Tasks Find good items presenting a ranked list of recommendendations. probabilistic combination of – Item-based method – User-based method – Matrix Factorization – (May be) content-based method Find all good items user wants to identify all The idea is to pick from my items that might be previous list 20-50 movies that share similar audience with interesting, e.g. medical “Taken”, then how much I will like depend on how much I liked those or legal cases early movies – In short: I tend to watch this movie because I have watched those Herlocker, Konstan, Borchers, & Riedl (2004). Evaluating Collaborative Filtering movies … or Recommender Systems. ACM Transactions on–Informationhave watched those pp. 5-53. 12 People who Systems, 22(1), movies also liked this movie (Amazon style)
  • 24. Objectives::RecSys Tasks Find good items Receive sequence of items presenting a ranked list of sequence of related items is recommendendations. recommended to the user, e.g. music recommender probabilistic combination of – Item-based method – User-based method – Matrix Factorization Find all good items Annotation in context – (May be) content-based method user wants to identify all predicted usefulness of an items that might be item that pick from mythatis currently The idea is to the user previous list 20-50 movies interesting, e.g. medical viewing, e.g. linkslike share similar audience with within a “Taken”, then how much I will or legal cases websitehow much I liked those depend on early movies – In short: I tend to watch this movie because I have watched those Herlocker, Konstan, Borchers, & Riedl (2004). Evaluating Collaborative Filtering movies … or Recommender Systems. ACM Transactions on–Informationhave watched those pp. 5-53. 12 People who Systems, 22(1), movies also liked this movie (Amazon style)
  • 25. Objectives::RecSys Tasks Find good items Receive sequence of items presenting a ranked list of sequence of related items is recommendendations. recommended to the user, e.g. music recommender There are more tasks available... of probabilistic combination – Item-based method – User-based method – Matrix Factorization Find all good items Annotation in context – (May be) content-based method user wants to identify all predicted usefulness of an items that might be item that pick from mythatis currently The idea is to the user previous list 20-50 movies interesting, e.g. medical viewing, e.g. linkslike share similar audience with within a “Taken”, then how much I will or legal cases websitehow much I liked those depend on early movies – In short: I tend to watch this movie because I have watched those Herlocker, Konstan, Borchers, & Riedl (2004). Evaluating Collaborative Filtering movies … or Recommender Systems. ACM Transactions on–Informationhave watched those pp. 5-53. 12 People who Systems, 22(1), movies also liked this movie (Amazon style)
  • 26. RecSys Technologies 1. Predict how much a user may like a certain product 2. Create a list of Top-N best items 3. Adjust its prediction based on feedback of the target user and like- minded users Hanani et al., (2001). Information Filtering: Overview of Issues, Research and Systems", User Modeling and User-Adapted Interaction, 11. 13
  • 27. RecSys Technologies 1. Predict how much a user may like a certain product 2. Create a list of Top-N best items 3. Adjust its prediction based on feedback of the Just some examples target user and like- there are more minded users technologies available. Hanani et al., (2001). Information Filtering: Overview of Issues, Research and Systems", User Modeling and User-Adapted Interaction, 11. 13
  • 28. Technologies::Collaborative filtering User-based filtering (Grouplens, 1994) Take about 20-50 people who share similar taste with you, afterwards predict how much you might like an item depended on how much the others liked it. You may like it because your “friends” liked it. 14
  • 29. Technologies::Collaborative filtering User-based filtering Item-based filtering (Grouplens, 1994) (Amazon, 2001) Take about 20-50 people who share Pick from your previous list 20-50 items similar taste with you, afterwards that share similar people with “the predict how much you might like an target item”, how much you will like the item depended on how much the others target item depends on how much the liked it. others liked those earlier items. You may like it because your You tend to like that item because “friends” liked it. you have liked those items. 14
  • 30. Technologies::Content-based filtering Information needs of user and characteristics of items are represented in keywords, attributes, tags that describe past selections, e.g., TF-IDF. 15
  • 31. Technologies::Hybrid RecSys Combination of techniques to overcome disadvantages and advantages of single techniques. Advantages Disadvantages probabilistic combination of – Item-based method • No content analysis • Cold-start problem – User-based method – Matrix Factorization • Quality improves • Over-fitting – (May be) content-based method • No cold-start problem • New user / item problem The idea is to pick from my • No new user / item • Sparsity previous list 20-50 movies that share similar audience with problem “Taken”, then how much I will like depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those movies … or 16 – People who have watched those movies also liked this movie (Amazon style)
  • 32. Technologies::Hybrid RecSys Combination of techniques to overcome disadvantages and advantages of single techniques. Advantages Disadvantages probabilistic combination of – Item-based method • No content analysis • Cold-start problem – User-based method – Matrix Factorization • Quality improves • Over-fitting – (May be) content-based method • No cold-start problem • New user / item problem The idea is to pick from my • No new user / item • Sparsity previous list 20-50 movies that share similar audience with problem “Taken”, then how much I will like Just some examples there depend on how much I liked those early movies –are more watch this movie In short: I tend to (dis)advantages because I have watched those 16 movies … or available. – People who have watched those movies also liked this movie (Amazon style)
  • 33. Evaluation of RecSys probabilistic combination of – Item-based method – User-based method – Matrix Factorization – (May be) content-based method The idea is to pick from my previous list 20-50 movies that share similar audience with “Taken”, then how much I will like depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those movies … or 17 – People who have watched those movies also liked this movie (Amazon style)
  • 34. Evaluation::General idea Most of the time based on performance measures (“How good are your recommendations?”) For example: •Predict what rating will a user give an item? •Will the user select an item? •What is the order of usefulness of items to a user? Herlocker, Konstan, Riedl (2004). Evaluating Collaborative Filtering Recommender Systems. ACM Transactions on Information Systems, 22(1), 5-53. 18
  • 35. Evaluation::Reference datasets ... and various commercial datasets. 19
  • 36. Evaluation::Approaches 1. Simulation •User preference •Prediction accuracy •Coverage •Confidence •Trust •Novelty 2. User study •Serendipity •Diversity •Utility •Risk •Robustness + •Privacy •Adaptivity •Scalability 20
  • 37. Evaluation::Metrics Precision – The portion of recommendations that were successful. (Selected by the algorithm and by the user) Recall – The portion of relevant items selected by algorithm compared to a total number of relevant items available. F1 - Measure balances Precision and Recall into a single measurement. Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962, 2009. 21
  • 38. Evaluation::Metrics Precision – The portion of recommendations that were successful. (Selected by the algorithm and by the user) Recall – The portion of relevant items selected by algorithm compared to a total number of relevant items available. F1 - Measure balances Precision and Recall into a single measurement. Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962, 2009. 21
  • 39. Evaluation::Metrics Precision – The portion of recommendations that were successful. (Selected by the algorithm and by the user) Recall – The portion of relevant items selected by algorithm compared to a total number of relevant items available. F1 - Measure balances Precision and Recall into a single measurement. Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962, 2009. 21
  • 40. Evaluation::Metrics Precision – The portion of recommendations that were successful. (Selected by the algorithm and by the user) Recall – The portion of relevant items selected by algorithm compared to a total number of relevant items available. F1 - Measure balances Precision Just some examples there and Recall into a single are more metrics available measurement. like MAE, RSME. Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962, 2009. 21
  • 41. Evaluation::Metrics 5 Conclusion: 4 Pearson is better RMSE than Cosine, 3 Pearson because less 2 errors in predicting Cosine 1 TOP-N items. 0 Netflix BookCrossing Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962, 2009. 22
  • 42. Evaluation::Metrics 5 Conclusion: 4 Pearson is better RMSE than Cosine, 3 Pearson because less 2 errors in predicting Cosine 1 TOP-N items. 0 Netflix BookCrossing News Story Clicks Conclusion: 80% Cosine better than Precision 60% Pearson, because 40% of higher precision 20% and recall value on TOP-N items. 0% 5% 10% 15% 20% 25% 30% 35% 40% Recall Gunawardana, A., Shani, G., (2009). A Survey of Accuracy Evaluation Metrics of Recommendation Tasks, Journal of Machine Learning Research, 10(Dec):2935−2962, 2009. 22
  • 43. RecSys::TimeToThink What do you expect that a RecSys in a MUP/PLE should do with respect to ... • Aims • Tasks • Technology Blackmore’s custom-built LSD Drive • Evaluation http://www.flickr.com/photos/ rootoftwo/ 23
  • 44. Goals of the lecture 1. Crash course Recommender Systems (RecSys) 2. Overview of RecSys in TEL 3. Open research issues for RecSys in TEL 4.TEL RecSys and Learning Analytics 24
  • 45. Recommender Systems for TEL Introduction Objectives Technologies Evaluation 25
  • 46. TEL RecSys::Definition Using the experiences of a community of learners to help individual learners in that community to identify more effectively learning content from a potentially overwhelming set of choices. Extended definition by Resnick & Varian (1997). Recommender Systems, Communications of the ACM, 40(3). 26
  • 47. Find TEL RecSys::Learning spectrum M fo co Maj sy fa We ap Cross, J., Informal learning. Pfeifer. (2006). 27
  • 48. The Long Tail Graphic: Wilkins, D., (2009). 28
  • 49. The Long Tail of Learning Graphic: Wilkins, D., (2009). 28
  • 50. The Long Tail of Learning Formal Informal Graphic: Wilkins, D., (2009). 28
  • 54. TEL RecSys:: Technologies RecSys Task: Find good items Hybrid RecSys: •Content-based on interests •Collaborative filtering 31
  • 55. TEL RecSys::Tasks Find good items e.g. relevant items for a learning task or a learning goal The idea is to pick from my previous list 20-50 movies that share similar audience with “Taken”, then how much I will like depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those Drachsler, H., Hummel, H., Koper, R., (2009). Identifying the goal, user model and conditions of movies … or recommender systems for formal and informal–learning. Journal watched those 32 People who have of Digital Information. 10(2). movies also liked this movie (Amazon style)
  • 56. TEL RecSys::Tasks Find good items e.g. relevant items for a learning task or a learning goal Receive sequence of items e.g. recommend a learning path to achieve a certain competence The idea is to pick from my previous list 20-50 movies that share similar audience with “Taken”, then how much I will like depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those Drachsler, H., Hummel, H., Koper, R., (2009). Identifying the goal, user model and conditions of movies … or recommender systems for formal and informal–learning. Journal watched those 32 People who have of Digital Information. 10(2). movies also liked this movie (Amazon style)
  • 57. TEL RecSys::Tasks Find good items e.g. relevant items for a learning task or a learning goal Receive sequence of items e.g. recommend a learning path to achieve a certain competence Annotation in context The idea is to pick from my e.g. take into account location, previous list 20-50 movies that share similar audience with time, noise level, prior “Taken”, then how much I will like knowledge, peers around depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those Drachsler, H., Hummel, H., Koper, R., (2009). Identifying the goal, user model and conditions of movies … or recommender systems for formal and informal–learning. Journal watched those 32 People who have of Digital Information. 10(2). movies also liked this movie (Amazon style)
  • 58. Evaluation of TEL RecSys probabilistic combination of – Item-based method – User-based method – Matrix Factorization – (May be) content-based method The idea is to pick from my previous list 20-50 movies that share similar audience with “Taken”, then how much I will like depend on how much I liked those early movies – In short: I tend to watch this movie because I have watched those movies … or 33 – People who have watched those movies also liked this movie (Amazon style)
  • 60. TEL RecSys::Review study Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H. G. K., & Koper, R. (2011). Recommender Systems in Technology Enhanced Learning. In P. B. Kantor, F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 387-415). Berlin: Springer. 34
  • 61. TEL RecSys::Review study Conclusions: Half of the systems (11/20) still at design or prototyping stage only 8 systems evaluated through trials with human users. Manouselis, N., Drachsler, H., Vuorikari, R., Hummel, H. G. K., & Koper, R. (2011). Recommender Systems in Technology Enhanced Learning. In P. B. Kantor, F. Ricci, L. Rokach, & B. Shapira (Eds.), Recommender Systems Handbook (pp. 387-415). Berlin: Springer. 34
  • 62. Thus... “The performance results of different research efforts in recommender systems are hardly comparable.” (Manouselis et al., 2010) Kaptain Kobold http://www.flickr.com/photos/ kaptainkobold/3203311346/ 35
  • 63. Thus... TEL recommender “The performance results experiments lack of different research efforts in recommender transparency. They need systems are hardly to be repeatable to test: comparable.” • Validity •(Manouselis et al., 2010) Verification • Compare results Kaptain Kobold http://www.flickr.com/photos/ kaptainkobold/3203311346/ 35
  • 65. TEL RecSys::Evaluation/datasets Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G., Lindstaedt, S., Stern, H., Friedrich, M., & Wolpers, M. (2010). Issues and Considerations regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning. Presentation at the 1st Workshop Recommnder Systems in Technology Enhanced Learning (RecSysTEL) in conjunction with 5th European Conference on Technology Enhanced Learning (EC-TEL 2010): Sustaining TEL: From Innovation to Learning and Practice. September, 28, 2010, Barcelona, Spain. 36
  • 66. Evaluation::Metrics MAE – Mean Absolute Error: Deviation of recommendations from the user-specified ratings. The lower the MAE, the more accurately the RecSys predicts user ratings. Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Beham, G., Duval, E., (2011). Dataset-driven Research for Improving Recommender Systems for Learning. Learning Analytics & Knowledge: February 27-March 1,37 2011, Banff, Alberta, Canada
  • 67. Evaluation::Metrics MAE – Mean Absolute Error: Deviation of recommendations from the user-specified ratings. The lower the MAE, the more accurately the RecSys predicts user ratings. Outcomes: Tanimoto similarity + item-based CF was the most accurate. Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Beham, G., Duval, E., (2011). Dataset-driven Research for Improving Recommender Systems for Learning. Learning Analytics & Knowledge: February 27-March 1,37 2011, Banff, Alberta, Canada
  • 68. Evaluation::Metrics MAE – Mean Absolute Error: Deviation of recommendations from the user-specified ratings. The lower the MAE, the more accurately the RecSys predicts user ratings. Outcomes: •User-based CF Algorithm that predicts the top 10 most relevant Outcomes: items for a user has a F1 score Tanimoto similarity + of almost 30%. item-based CF was •the most accurate. Implicit ratings like download rates, bookmarks can successfully used in TEL. Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., Beham, G., Duval, E., (2011). Dataset-driven Research for Improving Recommender Systems for Learning. Learning Analytics & Knowledge: February 27-March 1,37 2011, Banff, Alberta, Canada
  • 69. TEL RecSys::Evaluation Combined approach by Kirkpatrick model by Drachsler et al. 2008 Manouselis et al. 2010 38
  • 70. TEL RecSys::Evaluation 1. Accuracy 2. Coverage 3. Precision Combined approach by Kirkpatrick model by Drachsler et al. 2008 Manouselis et al. 2010 38
  • 71. TEL RecSys::Evaluation 1. Accuracy 2. Coverage 3. Precision 1. Effectiveness of learning 2. Efficiency of learning 3. Drop out rate 4. Satisfaction Combined approach by Kirkpatrick model by Drachsler et al. 2008 Manouselis et al. 2010 38
  • 72. TEL RecSys::Evaluation 1. Accuracy 1. Reaction of learner 2. Coverage 2. Learning improved 3. Precision 3. Behaviour 4. Results 1. Effectiveness of learning 2. Efficiency of learning 3. Drop out rate 4. Satisfaction Combined approach by Kirkpatrick model by Drachsler et al. 2008 Manouselis et al. 2010 38
  • 73. Goals of the lecture 1. Crash course Recommender Systems (RecSys) 2. Overview of RecSys in TEL 3. Open research issues for RecSys in TEL 4.TEL RecSys and Learning Analytics 39
  • 74. TEL RecSys::Open issues 1. Evaluation of TEL RecSys 2. Publicly available datasets 3. Comparable experiments 4. Body of knowledge 5. Privacy and data protection 6. Design learning driven RecSys 40
  • 75. Goals of the lecture 1. Crash course Recommender Systems (RecSys) 2. Overview of RecSys in TEL 3. Open research issues for RecSys in TEL 4.TEL RecSys and Learning Analytics 41
  • 76. Greller, W., & Drachsler, H., 2011. 42
  • 77. Greller, W., & Drachsler, H., 2011. 42
  • 78. Greller, W., & Drachsler, H., 2011. 42
  • 79. Greller, W., & Drachsler, H., 2011. 42
  • 80. Greller, W., & Drachsler, H., 2011. 42
  • 81. Greller, W., & Drachsler, H., 2011. 42
  • 82. Greller, W., & Drachsler, H., 2011. 42
  • 83. Learning Analytics::TimeToThink • Consider the Learning Analytics framework and imagine some great TEL RecSys that could support you in your stakeholder role alternatively • Name a learning task where a TEL RecSys would be useful for. 43
  • 84. Thank you for attending this lecture! This silde is available at: http://www.slideshare.com/Drachsler Email: hendrik.drachsler@ou.nl Skype: celstec-hendrik.drachsler Blogging at: http://www.drachsler.de Twittering at: http://twitter.com/HDrachsler 44