This paper presents the results of the rst participation of our multi-institutional team in the Retrieving Diverse Social Images Task at MediaEval 2014. In this task we were required to develop a summarization and diversification approach for social photo retrieval. Our approach is based on irrelevant image ltering, image re-ranking, and diversity promotion by clustering. We have used visual and textual features, including image metadata and user credibility information. http://ceur-ws.org/Vol-1263/mediaeval2014_submission_38.pdf

1. Recod @ MediaEval 2014:
Diverse Social Images Retrieval
Rodrigo T. Calumby, Vinícius P. Santana,
Felipe S. Cordeiro, Otávio A. B. Penatti,
Lin T. Li, Giovani Chiachia, Ricardo da S. Torres
o.penatti@samsung.com
[rtcalumby, vpsantana, fscordeiro]@ecomp.uefs.br
[lintzyli, chiachia, rtorres]@ic.unicamp.br
UEFS
Acknowledgments: UEFS/PROBIC, Samsung Research Institute Brazil, CNPq, and FAPESP (2013/11359-0).

2. PROPOSED APPROACH
Filtering
Geographic Face detection
Diversification Re-ranking
Visual
input
list
Clustering
output
list Selection
Credibility

3. input
list
output
list
Filtering
Re-ranking
Diversif
ication
Geo Filter
Face Filter
NumFacesFilter
> 1 face → non-relevant
(location: Christ the Redeemer, Rio de Janeiro)
1-NN Classifier
Features
1) number of faces
2) biggest face size
3) smallest face size
4) average face size
5) total face size
Test Image:
Validation (devset)
leave-one-location-out
Runs (testset)
target: full devset
kNN
10km radius
limit from
reference
lat/long of the
location
(location: Iguazu Falls, Brazil/Argentina)
FaceClassifierFilter

4. Filtering
Re-ranking
Diversif
ication
Visual Credibility
CredScore:
visualScore X faceProportion X tagSpecificity
Location representatives:
(location: Casa Batlló, Barcelona)
Original Re-ranked
1
4√n+1
RelScore (nth image):
Finalcore: CredScore x RelScore
Original Re-ranked

5. input
list
output
list
Filtering
Re-ranking
Diversif
ication
Clustering
Selection
- Descending cluster size
- Most relevant item from
each cluster
(location: Arc de Triomphe, Paris)
- kMedoids: 50 clusters
- Initial medoids: rank offset positions
Output list

6. RESULTS – OFFICIAL MEASURES
Run Filtering Re-ranking Diversification P@20 CR@20 F1@20
NumFacesFilter - kMedoids (BoVWsparse
1 GeoFilter and
max +
HOG) 0.7130 0.4030 0.5077
2 GeoFilter and
NumFacesFilter - kMedoids (Cosine) 0.6976 0.4139 0.5133
3 GeoFilter and
NumFacesFilter
Visual re-ranking (CM3x3 +
HOG + BIC)
kMedoids (BoVWsparse
max +
HOG + Cosine) 0.7016 0.4177 0.5168
4 GeoFilter and
NumFacesFilter
Visual re-ranking (CM3x3 +
HOG + BIC) and Credibility
re-ranking
kMedoids (CN3x3) 0.7598 0.4288 0.5423
5
GeoFilter and
FaceClassifierFilt
er
Visual re-ranking (CM3x3 +
HOG + BIC) and Credibility
re-ranking
kMedoids (CN3x3) 0.7407 0.4076 0.5206

7. CONCLUSIONS
- Geographic and face-based filtering improved precision.
- The simple NumFacesFilter outperformed the FaceClassifierFilter.
- Visual and credibility re-ranking improved precision.
- kMedoids outperfomed MMR on devset and was applied for the runs.
- Inter and inner cluster sorting were effective for improving
relevance.

8. Recod @ MediaEval 2014:
Diverse Social Images Retrieval
Thank you!
Rodrigo T. Calumby, Vinícius P. Santana, Felipe S. Cordeiro,
Otávio A. B. Penatti, Lin T. Li, Giovani Chiachia, Ricardo da S. Torres
o.penatti@samsung.com
[rtcalumby, vpsantana, fscordeiro]@ecomp.uefs.br
[lintzyli, chiachia, rtorres]@ic.unicamp.br
UEFS
Acknowledgments: UEFS/PROBIC, Samsung Research Institute Brazil, CNPq, and FAPESP (2013/11359-0).