ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

1. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) Javier Sanz-Cruzado1, Craig Macdonald2, Iadh Ounis2 and Pablo Castells1 @JavierSanzCruza, @Craig_macdonald, @iadh, @pcastells 1 Universidad Autónoma de Madrid http://ir.ii.uam.es 2University of Glasgow http://terrierteam.dcs.gla.ac.uk/ April 15th 2020
2. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Contact recommendation ? - 1 1 - 2 1 ? - ? 1 3 - 1 4 - Users Users Rating matrix = adjacency matrix 1
3. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Motivation Search Contact recommendation Models (BM25, Query Likelihood…) IR axioms Evaluation (Metrics,…) 2
4. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Space mapping Previous work (Sanz-Cruzado & Castells, ECIR 2019): Search Contact recommendation ? Relevant result DocumentQuery Term Relevant link Candidate user Target user Neighbor user 3
8. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Model mapping: e.g. BM25  BM25 for search Search Contact recommendation – Query 𝑞 Target user neighborhood Γ 𝑢 – Document 𝑑 Candidate user neighborhood Γ 𝑣 – Term 𝑡 ∈ 𝑑 ∩ 𝑞 Neighbor 𝑡 ∈ Γ 𝑢 ∩ Γ 𝑣 – Term frequency in 𝑑, freq 𝑡, 𝑑 Weight of the 𝑡, 𝑣 edge, w 𝑡, 𝑑 – Document 𝑑 length, 𝑑 Candidate user 𝑣 length, len v = 𝑥∈Γ 𝑣 𝑤 𝑥, 𝑣 7
9. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Model mapping: e.g. BM25  BM25 for search  BM25 for contact recommendation 8
10. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019  Goal: Determine a set of heuristics (axioms) that a good IR system should satisfy  Many constraints have been proposed addressing many properties: – Term frequency – Term discrimination – Length normalization – …  Useful for: – Development of effective IR models – Diagnose of their performance Axiomatic thinking in IR Are IR axioms useful for contact recommendation? Fang et al. SIGIR 2004, Fang et al. ACM TOIS 2011 9
11. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Axiomatic thinking in IR Are IR axioms useful for contact recommendation? Adaptation Empirical observation 10
12. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Axioms in text search Fang et al. SIGIR 2004, Fang et al. ACM TOIS 2011 TFC1: 𝑓𝑞 𝑑 grows with freq 𝑡, 𝑑 TFC3: frequency distributed over many terms better than concentrated on few terms TFC2: dampened growth on freq 𝑡, 𝑑 Term frequency constraints (TFCs) 11
13. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Axioms in contact recommendation EWC1: 𝑓𝑢 𝑣 grows with w 𝑡, 𝑣 EWC3: weight distributed over many neighbors better than concentrated on few neighbors EWC2: dampened growth on w 𝑡, 𝑣 Edge weight constraints (EWCs) 12
14. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Other axioms in contact recommendation EWC1 EWC3 EWC2 Neighbor discrimination constraint (NDC) NDC: Popular common neighbors have less importance Candidate length normalization constraint (CLNC) CLNC: Penalize candidate users with many uncommon neighbors 13
15. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Applicability to contact recommendation approaches  IR axioms can also be analyzed on other contact recommendation algorithms  Only if the algorithm recommends people at distance 2! ? ? Queries Terms Documents Distance 2  Adapted IR models  Adamic-Adar algorithms  Similarity-based  ···  kNN  Matrix factorization  PageRank  ··· 14
16. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Experimental setup Facebook Twitter-1month Twitter 200-tweets Interaction Follows Interaction Follows # users 4,039 9,528 9,770 9,985 9,964 Directed ✘ ✔ ✔ ✔ ✔ Weighted ✘ ✔ ✘ ✔ ✘ Train/Test split Random Temporal Temporal Temporal Temporal 15
17. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Procedure 1. Take an IR model 2. Remove pieces related to the axiom 3. Compare EWC1 EWC3 EWC2 NDC CLNC 16
18. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Edge weight constraint EWC1 Are edge weights good? Conclusion NO But the best algorithm (BM25) uses weights… nDCG@10 nDCG@10 Point in the plot = recommendation algorithm 17
19. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Edge weight constraint EWC2 Conclusion nDCG@10 nDCG@10 YES Is it good to dampen growth with edge weight? Point in the plot = BM25 with different parameter setting 18
20. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Neighbor discrimination constraint NDC Neighbor discrimination isbetter Neighbor discrimination isworse Point in the plot = different parameter setting of algorithm Is good to discriminate popular common neighbors? Conclusion YES Δ nDCG@10 With vs. Without neighbor discrimination 19
21. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Candidate length normalization constraint CLNC Length normalization isbetter Length normalization isworse Point in the plot = different parameter setting of algorithm Is it bad to recommend popular people? Conclusion NoΔ nDCG@10 With vs. Without length normalization Because of preferential attachment behaviour in social networks 20
22. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Conclusions  In general, IR axioms correlate with higher accuracy in contact recommendation.  Exception: Length normalization. Collision with preferential attachment  Future work: – Extend this to algorithms recommending at distance greater than 2 – Analyze to what extent those algorithms satisfy the constraints 21
23. IRGIR Group @UAM Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective 42nd European Conference on Information Retrieval (ECIR 2020) 15 April 2019 Thank you for your attention! Questions?

Notas do Editor

IDEA: Highlight tha fact that, previously, we adapted the textual search task from IR for the task of recommending people in social networks. Now, we have gone further, and we adapt IR axioms to check whether they are useful or not. This has two steps: first, an adaptation of the IR models for the contact recommendation task, and second, an observation of them to determine if they are good or not for the contact recommendation task.

So, the presentation starts HERE:

Hello, my name is Javier Sanz-Cruzado, and I’m going to present this work, in which we explore the utility of adapting of information retrieval axioms for the contact recommendation task. The present work is the result of a collaboration between the University of Glasgow, represented by Craig Macdonald and Iadh Ounis, and Universidad Autónoma de Madrid, with Pablo Castells and myself.
Our work focuses on a particular perspective of recommender systems known as contact recommendation.
Given a social network representing the different relations between users in our system, contact recommendation’s goal is to find people in the network with whom the different users might be interested in connecting with.
Our work continues a research line that we iniciated in last year’s ECIR, in which we explore the relationship between textual search and contact recommendation.

Last year, we explored how to adapt information retrieval models such as BM25 or query likelihood for recommending people in social networks in a collaborative filtering manner. In addition, we also used the methodology and metrics applied for textual search to evaluate such models.

This year, we wanted to go a step further, by adapting the use of axiomatic thinking techniques from the textual search task to the contact recommendation field.
All these past and present adaptations from the search task to contact recommendation are built on top of a mapping of spaces, where we map the three spaces in the IR task (queries, documents and terms) to a single space in contact recommendation: the space of users in the social network.
In that mapping, we first match the documents and the candidate users, since they both represent the elements that we want to retrieve.
Then, we match the query and the target user, since they represent the input of our systems: the explicit information need in the case of the query, and the profile of the user representing his/her tastes.
And, finally, as we want to explore everything from a collaborative filtering perspective, we match the terms with the connections of the target and candidate users: their neighbors.
Using that map, we are ready to translate the different IR models for contact recommendation. Here, we show an example with one of the most well-known approaches: BM25. First, we have its formulation for the search case.

Then, for adapting it to a contact recommendation approach, we just have to change the different pieces in search for pieces in contact recommendation. For example, we take the query, and we change it by the neighborhood of the target user. We take the neighborhood of the candidate instead of the document, and the common neighbors between the target and candidate users instead of the common terms between the query and the document.

Other elements we change are the frequency of the terms in the documents, which is now taken as the weight of the edges between the candidate user and their neighbors, and the length of the document, now transformed in the length of the candidate user (or, in other words, the sum of the weights of the edges shared with his neighbors).
With all of this, we can easily adapt the model for the contact recommendation case.
As we explained earlier, beyond IR models, in this work we wanted to adapt the information retrieval axioms. But, what do we want to say when we talk about IR axioms?

In textual search, axiomatic thinking is a research line that tries to formalize the properties that a model should satisfy to provide good results. That formal properties are known as axioms, and they target things like how we should treat long documents or whether we should give more importance to query terms appearing in only a few documents.

These axioms or constraints are useful for the development of new and effective IR models, as well as for diagnosing the performance of the existing ones.

In this work, we wondered if, under the previous mapping, these axioms could play a similar role for the contact recommendation task, or, in other words, if they are useful for that. In order to check this, we did two things: first, a theoretical work, in which we explored the adaptation of the IR axioms for the task, and, second, an observation on whether satisfying them leads to better models.
In order to check this, we did two things: first, a theoretical work, in which we explored the adaptation of the IR axioms for the task, and, second, an observation on whether satisfying them leads to better models.
We selected for adaptation the basic and fundamental axioms in textual search, proposed by Fang and colleagues in 2004 and improved in 2011. They categorized them in several groups, according to the elements that the axioms where related to. We start by adapting the term frequency constraints, that explian how the overall score should vary when the frequency of the terms in documents varies.

There are three axioms within this category. The first one indicates that the score should increase as the frequency of the term does. In BM25, this is satisfied thanks to the element we highlight in yellow. The second tells that this growth should be done in a diminishing returns pattern, so the difference in score between increasing from 1 appearance of the term and 2 appearances should be greater than between 100 and 101. For satisfying this, BM25 has the small element in the denominator than we highlight.

And, finally, the third one says that distributing a large frequency value over many common terms should be better than concentrating it on a single term. And this is true thanks to the sum over the common terms.
Using the mapping that we explained earlier, adapting these axioms to contact recommendation is straightforward. For the term frequency constraints, for example, we just change the frequency of the terms in the documents by the weight of the edges between the candidate users and the neighbors shared with the target user.

As the original model for textual search already satisfies the axioms for such task, when we adapt them for the contact recommendation task, IR models still keep the properties and satisfy them.
This adaptation can be extended to other axioms. For example, other adapted contact recommendation axioms we use in our work are the neighbor discrimination constraint, that indicates that, the more popular the common terms between the target and candidate users are, the les discriminative power they have, and they should be given les importance, and the candidate length normalization constraint, that says that we should not favor popular users with many uncommon neighbors with the target user.
Then, our study can be straightforwardly applied to IR model adaptations. But, what about other people recommendation algorithms? We observed that, as IR axioms asume a structure like the one shown in the slide, where there are two steps between the queries and the documents, we could extend our analysis to other recommendation approaches that recommend people at distance 2 in the network. This, beyond IR models includes others such as Adamic-Adar.

However, other well-known and effective approaches, such as matrx factorization or PageRank, which are able to recommend people at distance further than 2 cannot be analyzed using these constraints.
Until now, we have just focused on the adaptation of the IR axioms. However, even when their translation is straightforward, there is something that changes: the semantics of those axioms for the new task. So, in order to see if they are really useful for the contact recommendation task, we had to carry several experiments to see if satisfying them is related an improvement on the accuracy of the algorithms. We carried them over data from different social networks: an undirected and unweighted Facebook graph, two directed and weighted interaction Twitter networks, and two directed and unweighted Twitter follow networks.
Since each axiom is related to a particular element in the model formulation, the procedure we followed was the following: for each of the axioms, we took IR models satisfying such axiom, and we removed the element that made them satisfy it. Then, we compared their accuracy, and studied whether keeping that piece was Good or not. We show the results for some of these axioms:
First, we started with the first edge weight axiom. For this, we wanted to check if the use of edge weights is good.

For this, we compared IR models using weights against IR models using binary weights. Each point in the plot represents a different recommendation algorithm, and the x axis indicates its nDCG@10 value for the unweighted models, whereas the y axis shows the same for the weighted models.

It can be observed that, in general, unweighted models achieve a better accuracy, so we might say that the first axiom is not useful. However, if we look it closely, we observe that the best model (the red point above) uses the weights, so we could not fully discard this constraint.
So, for the second constraint, we took this best model, BM25, and we wondered if dampening the growth of the edges was good. So, we removed the dampening term, and we compared several parameter settings of BM25 with and without that dampening. In this plot, the nDCG value for the original algorithm is shown in the X axis, revealing that moderating the growth of the score provides positive results.
Next, we explore the neighbor discrimination concern, asking if it is good giving more weight to unpopular common users or not. We took for that several IR models, and we removed their discrimination terms. We changed here the representation, so each point represents the difference in nDCG for a parameter setting of an algorithm. We sorted the settings by difference, and values greater than 0 represent settings for which applying neighborhood discrimination is better and values smaller than 0, settings for which removing it improved the accuracy.

As we could observe, it seems clear that applying the neighborhood discrimination provides much better results than removing it.
And, finally, we explored the effect of the length normalization, wondering if is it bad or not to recommend bad people. Following the same scheme as we did for the previous constraint, we show here that most values in the plot are below 0, thus showing that recommending popular people is not bad, and contradicting our axiom. We found that this is because the constraint contradicts the preferential attachment behaviour in social networks, where popular users are more likely to receive new connections.
In conclusion, we can say that, in general, satisfying IR axioms leads to improvements on the accuracy of the recommenders, with the exception of length normalization ones, since they collide with preferential attachment evolutionary patterns of social networks. As future work, we would like to extend this work for analyzing the rest of approaches.

Thank you for your attention and, if you have any questions….

ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

Esté a ler uma amostra.

Confira estes a seguir

ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

Mais Conteúdo rRelacionado

Semelhante a ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective (20)

Mais recentes (20)

ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

Notas do Editor

ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

Esté a ler uma amostra.

ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

Recomendadas

Mais Conteúdo rRelacionado

Semelhante a ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective (20)

Mais recentes (20)

ECIR 2020 - Axiomatic Analysis of Contact Recommendation Methods in Social Networks: An IR perspective

Notas do Editor