Recommender systems evaluation

A 3D benchmark

Alan Said, D. Tikk, Yue Shi, Martha Larson, Klara Stumpf, Paolo Cremonesi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Citations (Scopus)

Abstract

Recommender systems add value to vast content resources by matching users with items of interest. In recent years, immense progress has been made in recommendation tech- niques. The evaluation of these has however not been matched and is threatening to impede the further development of rec-ommender systems. In this paper we propose an approach that addresses this impasse by formulating a novel evalua-tion concept adopting aspects from recommender systems research and industry. Our model can express the quality of a recommender algorithm from three perspectives, the end consumer (user), the service provider and the vendor (business and technique for both). We review current bench- marking activities and point out their shortcomings, which are addressed by our model. We also explain how our 3D benchmarking framework would apply to a specific use case.

Original languageEnglish
Title of host publicationCEUR Workshop Proceedings
Pages21-23
Number of pages3
Volume910
Publication statusPublished - 2012
EventWorkshop on Recommendation Utility Evaluation: Beyond RMSE, RUE 2012 - Workshop at the 6th ACM International Conference on Recommender Systems, RecSys 2012 - Dublin, Ireland
Duration: Sep 9 2012Sep 9 2012

Other

OtherWorkshop on Recommendation Utility Evaluation: Beyond RMSE, RUE 2012 - Workshop at the 6th ACM International Conference on Recommender Systems, RecSys 2012
CountryIreland
CityDublin
Period9/9/129/9/12

Fingerprint

Recommender systems
Benchmarking
Industry

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Said, A., Tikk, D., Shi, Y., Larson, M., Stumpf, K., & Cremonesi, P. (2012). Recommender systems evaluation: A 3D benchmark. In CEUR Workshop Proceedings (Vol. 910, pp. 21-23)

Recommender systems evaluation : A 3D benchmark. / Said, Alan; Tikk, D.; Shi, Yue; Larson, Martha; Stumpf, Klara; Cremonesi, Paolo.

CEUR Workshop Proceedings. Vol. 910 2012. p. 21-23.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Said, A, Tikk, D, Shi, Y, Larson, M, Stumpf, K & Cremonesi, P 2012, Recommender systems evaluation: A 3D benchmark. in CEUR Workshop Proceedings. vol. 910, pp. 21-23, Workshop on Recommendation Utility Evaluation: Beyond RMSE, RUE 2012 - Workshop at the 6th ACM International Conference on Recommender Systems, RecSys 2012, Dublin, Ireland, 9/9/12.
Said A, Tikk D, Shi Y, Larson M, Stumpf K, Cremonesi P. Recommender systems evaluation: A 3D benchmark. In CEUR Workshop Proceedings. Vol. 910. 2012. p. 21-23
Said, Alan ; Tikk, D. ; Shi, Yue ; Larson, Martha ; Stumpf, Klara ; Cremonesi, Paolo. / Recommender systems evaluation : A 3D benchmark. CEUR Workshop Proceedings. Vol. 910 2012. pp. 21-23
@inproceedings{98604619f5a04290a879d75843843720,
title = "Recommender systems evaluation: A 3D benchmark",
abstract = "Recommender systems add value to vast content resources by matching users with items of interest. In recent years, immense progress has been made in recommendation tech- niques. The evaluation of these has however not been matched and is threatening to impede the further development of rec-ommender systems. In this paper we propose an approach that addresses this impasse by formulating a novel evalua-tion concept adopting aspects from recommender systems research and industry. Our model can express the quality of a recommender algorithm from three perspectives, the end consumer (user), the service provider and the vendor (business and technique for both). We review current bench- marking activities and point out their shortcomings, which are addressed by our model. We also explain how our 3D benchmarking framework would apply to a specific use case.",
author = "Alan Said and D. Tikk and Yue Shi and Martha Larson and Klara Stumpf and Paolo Cremonesi",
year = "2012",
language = "English",
volume = "910",
pages = "21--23",
booktitle = "CEUR Workshop Proceedings",

}

TY - GEN

T1 - Recommender systems evaluation

T2 - A 3D benchmark

AU - Said, Alan

AU - Tikk, D.

AU - Shi, Yue

AU - Larson, Martha

AU - Stumpf, Klara

AU - Cremonesi, Paolo

PY - 2012

Y1 - 2012

N2 - Recommender systems add value to vast content resources by matching users with items of interest. In recent years, immense progress has been made in recommendation tech- niques. The evaluation of these has however not been matched and is threatening to impede the further development of rec-ommender systems. In this paper we propose an approach that addresses this impasse by formulating a novel evalua-tion concept adopting aspects from recommender systems research and industry. Our model can express the quality of a recommender algorithm from three perspectives, the end consumer (user), the service provider and the vendor (business and technique for both). We review current bench- marking activities and point out their shortcomings, which are addressed by our model. We also explain how our 3D benchmarking framework would apply to a specific use case.

AB - Recommender systems add value to vast content resources by matching users with items of interest. In recent years, immense progress has been made in recommendation tech- niques. The evaluation of these has however not been matched and is threatening to impede the further development of rec-ommender systems. In this paper we propose an approach that addresses this impasse by formulating a novel evalua-tion concept adopting aspects from recommender systems research and industry. Our model can express the quality of a recommender algorithm from three perspectives, the end consumer (user), the service provider and the vendor (business and technique for both). We review current bench- marking activities and point out their shortcomings, which are addressed by our model. We also explain how our 3D benchmarking framework would apply to a specific use case.

UR - http://www.scopus.com/inward/record.url?scp=84891953003&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891953003&partnerID=8YFLogxK

M3 - Conference contribution

VL - 910

SP - 21

EP - 23

BT - CEUR Workshop Proceedings

ER -