Search and Society: Reimagining Information Access for Radical Futures
Information Retrieval Research Journal (IRRJ) | March 2025
Bhaskar Mitra
Principal Researcher
Publications
Publications by Year
-
Overview of the TREC 2024 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, To Eun Kim, Yifan He, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | March 2025
LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
2025 Web Search and Data Mining | March 2025
Tip of the Tongue Query Elicitation for Simulated Evaluation
Yifan He, To Eun Kim, Fernando Diaz, Jaime Arguello, Bhaskar Mitra
2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | February 2025
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
February 2025
Recall, Robustness, and Lexicographic Evaluation
Fernando Diaz, Michael D. Ekstrand, Bhaskar Mitra
ACM Transactions on Recommender Systems | February 2025
Emancipatory Information Retrieval
February 2025
JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
2025 The Web Conference | January 2025
SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
2025 The Web Conference | January 2025
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Bhaskar Mitra, Henriette Cramer, Olya Gurevich
Information Access in the Era of Generative AI, 7 | Published by Springer | 2025
-
Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
September 2024
LLMJudge: LLMs for Relevance Judgments
Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
August 2024
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
August 2024
Large Language Models Can Accurately Predict Searcher Preferences
Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
LLM4Eval: Large Language Model for Evaluation in IR
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
Workshop description
Report on The Search Futures Workshop at ECIR 2024
Leif Azzopardi, Charles L. A. Clarke, Paul Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
SIGIR Forum | July 2024
ReNeuIR at SIGIR 2024: The Third Workshop on Reaching Efficiency in Neural Information Retrieval
Maik Fröbe, Joel Mackenzie, Bhaskar Mitra, Franco Maria Nardini, Martin Potthast
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | June 2024
Synthetic Test Collections for Retrieval Evaluation
Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
Towards Group-aware Search Success
Haolun Wu, Bhaskar Mitra, Nick Craswell
International Conference on the Theory of Information Retrieval | April 2024
A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers
Anna Gausen, Bhaskar Mitra, Siân Lindley
ACM Conference on Fairness, Accountability, and Transparency | March 2024
The Search Futures Workshop
Leif Azzopardi, Charles L. A. Clarke, Paul B. Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
European Conference on Information Retrieval | March 2024
Result Diversification in Search and Recommendation: A Survey
Haolun Wu, Yansen Zhang, Chen Ma, Fuyuan Lyu, Bowei He, Bhaskar Mitra, Xue Liu
IEEE Transactions on Knowledge and Data Engineering | March 2024
Overview of the TREC 2023 Deep Learning Track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | February 2024
Overview of the TREC 2023 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | February 2024
Learning to Extract Structured Entities Using Language Models
Haolun Wu, Ye Yuan, Liana Mikaelyan, Alexander Meulemans, Xue Liu, James Hensman, Bhaskar Mitra
2024 Empirical Methods in Natural Language Processing | February 2024
Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
Karina Cortiñas-Lorenzo, Siân Lindley, Ida Larsen-Ledet, Bhaskar Mitra
January 2024
-
DiSK: A Diffusion Model for Structured Knowledge
Ouail Kitouni, Niklas Nolte, James Hensman, Bhaskar Mitra
December 2023
Co-audit: tools to help humans double-check AI-generated content
Andy Gordon, Carina Negreanu, José Cambronero, Rasika Mudumbai Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson (nee Murfet), Advait Sarkar, Stephanie Simmons, Jack Williams, Ben Zorn
October 2023
Patterns of gender-specializing query reformulation
Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
Overview of the TREC 2022 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | March 2023
De-Biasing Relevance Judgements for Fair Ranking
Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 45th European Conference on Information Retrieval (ECIR) | January 2023
Taking Search to Task
Chirag Shah, Ryen W. White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, Nicholas Belkin
Proceedings of the 8th ACM SIGIR Conference On Human Information Interaction And Retrieval | January 2023
-
Ethical and Social Considerations in Automatic Expert Identification and People Recommendation in Organizational Knowledge Management Systems
Ida Larsen-Ledet, Bhaskar Mitra, Siân Lindley
Proceedings of the 5th FAccTRec Workshop on Responsible Recommendation at RecSys 2022 | September 2022
A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
ACM Transactions on Information Systems (TOIS) | August 2022
Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
June 2022
Overview of the TREC 2021 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Text REtrieval Conference (TREC) | May 2022
Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Joint Multisided Exposure Fairness for Recommendation
Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
Chirag Shah, Torsten Suel, Fernando Diaz, Bhaskar Mitra, Bárbara Poblete, Hussein Suleman, Suzan Verberne
SIGIR Forum | February 2022, Vol 55(2)
Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
Exposing Query Identification for Search Transparency
Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
2022 The Web Conference | January 2022
-
Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
European Conference on Information Retrieval | November 2021
Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
European Conference on Information Retrieval | November 2021
MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
SIGIR Forum | July 2021, Vol 55(1)
Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
Text REtrieval Conference (TREC) | February 2021
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
-
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
December 2020
PhD Thesis: University College London | December 2020
Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Text REtrieval Conference (TREC) | November 2020
Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
August 2020
Conformer-Kernel with Query Term Independence for Document Retrieval
Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
July 2020
Evaluating Stochastic Rankings with Expected Exposure
Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
Best Long Research Paper Nominee
ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
Analyzing and Learning from User Interactions for Search Clarification
Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
Local Self-Attention over Long Text for Efficient Document Retrieval
Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
On the Reliability of Test Collections for Evaluating Systems of Different Types
Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
Overview of the TREC 2019 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
Text REtrieval Conference (TREC) | March 2020
-
Duet at TREC 2019 Deep Learning Track
Text REtrieval Conference (TREC) | December 2019
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
July 2019
An Axiomatic Approach to Regularizing Neural Ranking Models
Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
An updated duet model for passage re-ranking
March 2019
-
An Introduction to Neural Information Retrieval
Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
A Line in the Sand: Recommendation or Ad-hoc Retrieval?
Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
Proceedings of the Recommender Systems Challenge 2018 | July 2018
Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Best Short Research Paper
Optimizing Query Evaluations Using Reinforcement Learning for Web Search
Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
Advances in Information Retrieval
-
Neural Ranking Models with Multiple Document Fields
Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
Neural information retrieval: introduction to the special issue
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Information Retrieval Journal : Special issue Neural | November 2017
Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
SIGIR Forum | October 2017, Vol 52
Reply With: Proactive Recommendation of Email Attachments
Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
Benchmark for Complex Answer Retrieval
Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
Luandri: a Clean Lua Interface to the Indri Search Engine
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
Toward Incorporation of Relevant Documents in word2vec
Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
Neural Models for Information Retrieval
May 2017
Learning to Match Using Local and Distributed Representations of Text for Web Search
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 26th International Conference on World Wide Web | April 2017
Neural Text Embeddings for Information Retrieval
Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
-
Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
A Proposal for Evaluating Answer Distillation from Web Data
Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
Query Expansion with Locally-Trained Word Embeddings
Fernando Diaz, Bhaskar Mitra, Nick Craswell
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
Improving Document Ranking with Dual Word Embeddings
Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
WWW'16 | April 2016
A Dual Embedding Space Model for Document Ranking
Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
February 2016
This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
-
Query Auto-Completion for Rare Prefixes
Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
Exploring Session Context using Distributed Representations of Queries and Reformulations
Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
-
An Eye-tracking Study of User Interactions with Query Auto Completion
Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
An Introduction to Computational Networks and the Computational Network Toolkit
Dong Yu, Adam Eversole, Mike Seltzer, Kaisheng Yao, Oleksii Kuchaiev, Yu Zhang, Frank Seide, Zhiheng Huang, Brian Guenter, Huaming Wang, Jasha Droppo, Geoffrey Zweig, Chris Rossbach, Jie Gao, Andreas Stolcke, Jon Currey, Malcolm Slaney, Guoguo Chen, Amit Agarwal, Chris Basoglu, Marko Padmilac, Alexey Kamenev, Vladimir Ivanov, Scott Cypher, Hari Parthasarathi, Bhaskar Mitra, Baolin Peng, Xuedong Huang
MSR-TR-2014-112 | October 2014
On User Interactions with Query Auto-Completion
Bhaskar Mitra, Milad Shokouhi, Filip Radlinski, Katja Hofmann
Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2014
Publications by Research Area
-
Overview of the TREC 2024 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, To Eun Kim, Yifan He, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | March 2025
Tip of the Tongue Query Elicitation for Simulated Evaluation
Yifan He, To Eun Kim, Fernando Diaz, Jaime Arguello, Bhaskar Mitra
2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | February 2025
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
February 2025
JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
2025 The Web Conference | January 2025
SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
2025 The Web Conference | January 2025
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Bhaskar Mitra, Henriette Cramer, Olya Gurevich
Information Access in the Era of Generative AI, 7 | Published by Springer | 2025
Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
September 2024
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
August 2024
Large Language Models Can Accurately Predict Searcher Preferences
Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
Report on The Search Futures Workshop at ECIR 2024
Leif Azzopardi, Charles L. A. Clarke, Paul Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
SIGIR Forum | July 2024
ReNeuIR at SIGIR 2024: The Third Workshop on Reaching Efficiency in Neural Information Retrieval
Maik Fröbe, Joel Mackenzie, Bhaskar Mitra, Franco Maria Nardini, Martin Potthast
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | June 2024
Synthetic Test Collections for Retrieval Evaluation
Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
Towards Group-aware Search Success
Haolun Wu, Bhaskar Mitra, Nick Craswell
International Conference on the Theory of Information Retrieval | April 2024
A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers
Anna Gausen, Bhaskar Mitra, Siân Lindley
ACM Conference on Fairness, Accountability, and Transparency | March 2024
Overview of the TREC 2023 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | February 2024
Overview of the TREC 2023 Deep Learning Track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | February 2024
Learning to Extract Structured Entities Using Language Models
Haolun Wu, Ye Yuan, Liana Mikaelyan, Alexander Meulemans, Xue Liu, James Hensman, Bhaskar Mitra
2024 Empirical Methods in Natural Language Processing | February 2024
Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
Karina Cortiñas-Lorenzo, Siân Lindley, Ida Larsen-Ledet, Bhaskar Mitra
January 2024
DiSK: A Diffusion Model for Structured Knowledge
Ouail Kitouni, Niklas Nolte, James Hensman, Bhaskar Mitra
December 2023
Co-audit: tools to help humans double-check AI-generated content
Andy Gordon, Carina Negreanu, José Cambronero, Rasika Mudumbai Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson (nee Murfet), Advait Sarkar, Stephanie Simmons, Jack Williams, Ben Zorn
October 2023
Overview of the TREC 2022 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | March 2023
De-Biasing Relevance Judgements for Fair Ranking
Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 45th European Conference on Information Retrieval (ECIR) | January 2023
Ethical and Social Considerations in Automatic Expert Identification and People Recommendation in Organizational Knowledge Management Systems
Ida Larsen-Ledet, Bhaskar Mitra, Siân Lindley
Proceedings of the 5th FAccTRec Workshop on Responsible Recommendation at RecSys 2022 | September 2022
A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
ACM Transactions on Information Systems (TOIS) | August 2022
Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
June 2022
Overview of the TREC 2021 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Text REtrieval Conference (TREC) | May 2022
Joint Multisided Exposure Fairness for Recommendation
Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
Exposing Query Identification for Search Transparency
Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
2022 The Web Conference | January 2022
Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
European Conference on Information Retrieval | November 2021
Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
European Conference on Information Retrieval | November 2021
MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
SIGIR Forum | July 2021, Vol 55(1)
Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
Text REtrieval Conference (TREC) | February 2021
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
December 2020
PhD Thesis: University College London | December 2020
Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Text REtrieval Conference (TREC) | November 2020
Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
August 2020
Conformer-Kernel with Query Term Independence for Document Retrieval
Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
July 2020
Evaluating Stochastic Rankings with Expected Exposure
Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
Best Long Research Paper Nominee
ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
Analyzing and Learning from User Interactions for Search Clarification
Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
Local Self-Attention over Long Text for Efficient Document Retrieval
Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
On the Reliability of Test Collections for Evaluating Systems of Different Types
Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
Overview of the TREC 2019 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
Text REtrieval Conference (TREC) | March 2020
Duet at TREC 2019 Deep Learning Track
Text REtrieval Conference (TREC) | December 2019
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
July 2019
An Axiomatic Approach to Regularizing Neural Ranking Models
Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
An updated duet model for passage re-ranking
March 2019
An Introduction to Neural Information Retrieval
Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
A Line in the Sand: Recommendation or Ad-hoc Retrieval?
Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
Proceedings of the Recommender Systems Challenge 2018 | July 2018
Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Best Short Research Paper
Optimizing Query Evaluations Using Reinforcement Learning for Web Search
Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
Advances in Information Retrieval
Neural Ranking Models with Multiple Document Fields
Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
Neural information retrieval: introduction to the special issue
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Information Retrieval Journal : Special issue Neural | November 2017
Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
SIGIR Forum | October 2017, Vol 52
Reply With: Proactive Recommendation of Email Attachments
Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
Benchmark for Complex Answer Retrieval
Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
Luandri: a Clean Lua Interface to the Indri Search Engine
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
Toward Incorporation of Relevant Documents in word2vec
Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
Neural Models for Information Retrieval
May 2017
Learning to Match Using Local and Distributed Representations of Text for Web Search
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 26th International Conference on World Wide Web | April 2017
Neural Text Embeddings for Information Retrieval
Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
A Proposal for Evaluating Answer Distillation from Web Data
Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
Query Expansion with Locally-Trained Word Embeddings
Fernando Diaz, Bhaskar Mitra, Nick Craswell
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
Improving Document Ranking with Dual Word Embeddings
Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
WWW'16 | April 2016
A Dual Embedding Space Model for Document Ranking
Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
February 2016
This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
Query Auto-Completion for Rare Prefixes
Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
Exploring Session Context using Distributed Representations of Queries and Reformulations
Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
An Eye-tracking Study of User Interactions with Query Auto Completion
Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
An Introduction to Computational Networks and the Computational Network Toolkit
Dong Yu, Adam Eversole, Mike Seltzer, Kaisheng Yao, Oleksii Kuchaiev, Yu Zhang, Frank Seide, Zhiheng Huang, Brian Guenter, Huaming Wang, Jasha Droppo, Geoffrey Zweig, Chris Rossbach, Jie Gao, Andreas Stolcke, Jon Currey, Malcolm Slaney, Guoguo Chen, Amit Agarwal, Chris Basoglu, Marko Padmilac, Alexey Kamenev, Vladimir Ivanov, Scott Cypher, Hari Parthasarathi, Bhaskar Mitra, Baolin Peng, Xuedong Huang
MSR-TR-2014-112 | October 2014
-
Search and Society: Reimagining Information Access for Radical Futures
Information Retrieval Research Journal (IRRJ) | March 2025
Overview of the TREC 2024 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, To Eun Kim, Yifan He, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | March 2025
LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
2025 Web Search and Data Mining | March 2025
Tip of the Tongue Query Elicitation for Simulated Evaluation
Yifan He, To Eun Kim, Fernando Diaz, Jaime Arguello, Bhaskar Mitra
2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | February 2025
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
February 2025
Recall, Robustness, and Lexicographic Evaluation
Fernando Diaz, Michael D. Ekstrand, Bhaskar Mitra
ACM Transactions on Recommender Systems | February 2025
Emancipatory Information Retrieval
February 2025
JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
2025 The Web Conference | January 2025
SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
2025 The Web Conference | January 2025
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Bhaskar Mitra, Henriette Cramer, Olya Gurevich
Information Access in the Era of Generative AI, 7 | Published by Springer | 2025
Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
September 2024
LLMJudge: LLMs for Relevance Judgments
Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
August 2024
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
August 2024
LLM4Eval: Large Language Model for Evaluation in IR
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
Workshop description
Large Language Models Can Accurately Predict Searcher Preferences
Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
Report on The Search Futures Workshop at ECIR 2024
Leif Azzopardi, Charles L. A. Clarke, Paul Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
SIGIR Forum | July 2024
ReNeuIR at SIGIR 2024: The Third Workshop on Reaching Efficiency in Neural Information Retrieval
Maik Fröbe, Joel Mackenzie, Bhaskar Mitra, Franco Maria Nardini, Martin Potthast
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | June 2024
Synthetic Test Collections for Retrieval Evaluation
Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
Towards Group-aware Search Success
Haolun Wu, Bhaskar Mitra, Nick Craswell
International Conference on the Theory of Information Retrieval | April 2024
A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers
Anna Gausen, Bhaskar Mitra, Siân Lindley
ACM Conference on Fairness, Accountability, and Transparency | March 2024
The Search Futures Workshop
Leif Azzopardi, Charles L. A. Clarke, Paul B. Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
European Conference on Information Retrieval | March 2024
Result Diversification in Search and Recommendation: A Survey
Haolun Wu, Yansen Zhang, Chen Ma, Fuyuan Lyu, Bowei He, Bhaskar Mitra, Xue Liu
IEEE Transactions on Knowledge and Data Engineering | March 2024
Overview of the TREC 2023 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | February 2024
Overview of the TREC 2023 Deep Learning Track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | February 2024
Learning to Extract Structured Entities Using Language Models
Haolun Wu, Ye Yuan, Liana Mikaelyan, Alexander Meulemans, Xue Liu, James Hensman, Bhaskar Mitra
2024 Empirical Methods in Natural Language Processing | February 2024
Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
Karina Cortiñas-Lorenzo, Siân Lindley, Ida Larsen-Ledet, Bhaskar Mitra
January 2024
DiSK: A Diffusion Model for Structured Knowledge
Ouail Kitouni, Niklas Nolte, James Hensman, Bhaskar Mitra
December 2023
Patterns of gender-specializing query reformulation
Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
Overview of the TREC 2022 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | March 2023
De-Biasing Relevance Judgements for Fair Ranking
Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 45th European Conference on Information Retrieval (ECIR) | January 2023
Taking Search to Task
Chirag Shah, Ryen W. White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, Nicholas Belkin
Proceedings of the 8th ACM SIGIR Conference On Human Information Interaction And Retrieval | January 2023
Ethical and Social Considerations in Automatic Expert Identification and People Recommendation in Organizational Knowledge Management Systems
Ida Larsen-Ledet, Bhaskar Mitra, Siân Lindley
Proceedings of the 5th FAccTRec Workshop on Responsible Recommendation at RecSys 2022 | September 2022
A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
ACM Transactions on Information Systems (TOIS) | August 2022
Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
June 2022
Overview of the TREC 2021 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Text REtrieval Conference (TREC) | May 2022
Joint Multisided Exposure Fairness for Recommendation
Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
Chirag Shah, Torsten Suel, Fernando Diaz, Bhaskar Mitra, Bárbara Poblete, Hussein Suleman, Suzan Verberne
SIGIR Forum | February 2022, Vol 55(2)
Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
Exposing Query Identification for Search Transparency
Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
2022 The Web Conference | January 2022
Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
European Conference on Information Retrieval | November 2021
Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
European Conference on Information Retrieval | November 2021
MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
SIGIR Forum | July 2021, Vol 55(1)
Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
Text REtrieval Conference (TREC) | February 2021
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
December 2020
PhD Thesis: University College London | December 2020
Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Text REtrieval Conference (TREC) | November 2020
Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
August 2020
Conformer-Kernel with Query Term Independence for Document Retrieval
Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
July 2020
Evaluating Stochastic Rankings with Expected Exposure
Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
Best Long Research Paper Nominee
ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
Analyzing and Learning from User Interactions for Search Clarification
Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
Local Self-Attention over Long Text for Efficient Document Retrieval
Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
On the Reliability of Test Collections for Evaluating Systems of Different Types
Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
Overview of the TREC 2019 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
Text REtrieval Conference (TREC) | March 2020
Duet at TREC 2019 Deep Learning Track
Text REtrieval Conference (TREC) | December 2019
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
July 2019
An Axiomatic Approach to Regularizing Neural Ranking Models
Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
An updated duet model for passage re-ranking
March 2019
An Introduction to Neural Information Retrieval
Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
A Line in the Sand: Recommendation or Ad-hoc Retrieval?
Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
Proceedings of the Recommender Systems Challenge 2018 | July 2018
Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Best Short Research Paper
Optimizing Query Evaluations Using Reinforcement Learning for Web Search
Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
Advances in Information Retrieval
Neural Ranking Models with Multiple Document Fields
Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
Neural information retrieval: introduction to the special issue
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Information Retrieval Journal : Special issue Neural | November 2017
Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
SIGIR Forum | October 2017, Vol 52
Reply With: Proactive Recommendation of Email Attachments
Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
Benchmark for Complex Answer Retrieval
Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
Luandri: a Clean Lua Interface to the Indri Search Engine
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
Toward Incorporation of Relevant Documents in word2vec
Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
Neural Models for Information Retrieval
May 2017
Learning to Match Using Local and Distributed Representations of Text for Web Search
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 26th International Conference on World Wide Web | April 2017
Neural Text Embeddings for Information Retrieval
Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
A Proposal for Evaluating Answer Distillation from Web Data
Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
Query Expansion with Locally-Trained Word Embeddings
Fernando Diaz, Bhaskar Mitra, Nick Craswell
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
Improving Document Ranking with Dual Word Embeddings
Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
WWW'16 | April 2016
A Dual Embedding Space Model for Document Ranking
Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
February 2016
This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
Query Auto-Completion for Rare Prefixes
Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
Exploring Session Context using Distributed Representations of Queries and Reformulations
Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
An Eye-tracking Study of User Interactions with Query Auto Completion
Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
On User Interactions with Query Auto-Completion
Bhaskar Mitra, Milad Shokouhi, Filip Radlinski, Katja Hofmann
Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2014
-
Search and Society: Reimagining Information Access for Radical Futures
Information Retrieval Research Journal (IRRJ) | March 2025
Emancipatory Information Retrieval
February 2025
A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers
Anna Gausen, Bhaskar Mitra, Siân Lindley
ACM Conference on Fairness, Accountability, and Transparency | March 2024
Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
Karina Cortiñas-Lorenzo, Siân Lindley, Ida Larsen-Ledet, Bhaskar Mitra
January 2024
-
A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers
Anna Gausen, Bhaskar Mitra, Siân Lindley
ACM Conference on Fairness, Accountability, and Transparency | March 2024
Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
Karina Cortiñas-Lorenzo, Siân Lindley, Ida Larsen-Ledet, Bhaskar Mitra
January 2024
Co-audit: tools to help humans double-check AI-generated content
Andy Gordon, Carina Negreanu, José Cambronero, Rasika Mudumbai Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson (nee Murfet), Advait Sarkar, Stephanie Simmons, Jack Williams, Ben Zorn
October 2023
Patterns of gender-specializing query reformulation
Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
Ethical and Social Considerations in Automatic Expert Identification and People Recommendation in Organizational Knowledge Management Systems
Ida Larsen-Ledet, Bhaskar Mitra, Siân Lindley
Proceedings of the 5th FAccTRec Workshop on Responsible Recommendation at RecSys 2022 | September 2022
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
Query Auto-Completion for Rare Prefixes
Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
An Eye-tracking Study of User Interactions with Query Auto Completion
Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
-
Co-audit: tools to help humans double-check AI-generated content
Andy Gordon, Carina Negreanu, José Cambronero, Rasika Mudumbai Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson (nee Murfet), Advait Sarkar, Stephanie Simmons, Jack Williams, Ben Zorn
October 2023
-
An Introduction to Computational Networks and the Computational Network Toolkit
Dong Yu, Adam Eversole, Mike Seltzer, Kaisheng Yao, Oleksii Kuchaiev, Yu Zhang, Frank Seide, Zhiheng Huang, Brian Guenter, Huaming Wang, Jasha Droppo, Geoffrey Zweig, Chris Rossbach, Jie Gao, Andreas Stolcke, Jon Currey, Malcolm Slaney, Guoguo Chen, Amit Agarwal, Chris Basoglu, Marko Padmilac, Alexey Kamenev, Vladimir Ivanov, Scott Cypher, Hari Parthasarathi, Bhaskar Mitra, Baolin Peng, Xuedong Huang
MSR-TR-2014-112 | October 2014
Publications by Type
-
Overview of the TREC 2024 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, To Eun Kim, Yifan He, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | March 2025
LLM4Eval@WSDM 2025: Large Language Model for Evaluation in Information Retrieval
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L A Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
2025 Web Search and Data Mining | March 2025
Tip of the Tongue Query Elicitation for Simulated Evaluation
Yifan He, To Eun Kim, Fernando Diaz, Jaime Arguello, Bhaskar Mitra
2025 International ACM SIGIR Conference on Research and Development in Information Retrieval | February 2025
JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
2025 The Web Conference | January 2025
SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval
Hossein A. Rahmani, Xi Wang, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas
2025 The Web Conference | January 2025
LLM4Eval: Large Language Model for Evaluation in IR
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
Workshop description
Large Language Models Can Accurately Predict Searcher Preferences
Paul Thomas, Seth Spielman, Nick Craswell, Bhaskar Mitra
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2024
An earlier version of this paper appeared as arXiv preprint arXiv:2309.10621v1 [cs.IR].
ReNeuIR at SIGIR 2024: The Third Workshop on Reaching Efficiency in Neural Information Retrieval
Maik Fröbe, Joel Mackenzie, Bhaskar Mitra, Franco Maria Nardini, Martin Potthast
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | June 2024
Synthetic Test Collections for Retrieval Evaluation
Hossein A. Rahmani, Nick Craswell, Emine Yilmaz, Bhaskar Mitra, Daniel Campos
2024 International ACM SIGIR Conference on Research and Development in Information Retrieval | May 2024
Towards Group-aware Search Success
Haolun Wu, Bhaskar Mitra, Nick Craswell
International Conference on the Theory of Information Retrieval | April 2024
A Framework for Exploring the Consequences of AI-Mediated Enterprise Knowledge Access and Identifying Risks to Workers
Anna Gausen, Bhaskar Mitra, Siân Lindley
ACM Conference on Fairness, Accountability, and Transparency | March 2024
The Search Futures Workshop
Leif Azzopardi, Charles L. A. Clarke, Paul B. Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
European Conference on Information Retrieval | March 2024
Overview of the TREC 2023 Tip-of-the-Tongue Track
Jaime Arguello, Samarth Bhargav, Fernando Diaz, Evangelos Kanoulas, Bhaskar Mitra
Text REtrieval Conference (TREC) | February 2024
Overview of the TREC 2023 Deep Learning Track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Hossein A. Rahmani, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | February 2024
Learning to Extract Structured Entities Using Language Models
Haolun Wu, Ye Yuan, Liana Mikaelyan, Alexander Meulemans, Xue Liu, James Hensman, Bhaskar Mitra
2024 Empirical Methods in Natural Language Processing | February 2024
Patterns of gender-specializing query reformulation
Amifa Raj, Bhaskar Mitra, Nick Craswell, Michael D. Ekstrand
2023 International ACM SIGIR Conference on Research and Development in Information Retrieval | July 2023
Overview of the TREC 2022 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, Ian Soboroff
Text REtrieval Conference (TREC) | March 2023
De-Biasing Relevance Judgements for Fair Ranking
Amin Bigdeli, Negar Arabzadeh, Shirin Seyedsalehi, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 45th European Conference on Information Retrieval (ECIR) | January 2023
Taking Search to Task
Chirag Shah, Ryen W. White, Paul Thomas, Bhaskar Mitra, Shawon Sarkar, Nicholas Belkin
Proceedings of the 8th ACM SIGIR Conference On Human Information Interaction And Retrieval | January 2023
Ethical and Social Considerations in Automatic Expert Identification and People Recommendation in Organizational Knowledge Management Systems
Ida Larsen-Ledet, Bhaskar Mitra, Siân Lindley
Proceedings of the 5th FAccTRec Workshop on Responsible Recommendation at RecSys 2022 | September 2022
Overview of the TREC 2021 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Text REtrieval Conference (TREC) | May 2022
Joint Multisided Exposure Fairness for Recommendation
Bhaskar Mitra, Haolun Wu, Chen Ma, Fernando Diaz, Xue Liu
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Inconsistent Ranking Assumptions in Medical Search and Their Downstream Consequences
Daniel Cohen, Kevin Du, Bhaskar Mitra, Laura Mercurio, Navid Rekabsaz, Carsten Eickhoff
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Fostering Coopetition While Plugging Leaks: The Design and Implementation of the MS MARCO Leaderboards
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
2022 International ACM SIGIR Conference on Research and Development in Information Retrieval | April 2022
Bias-aware Fair Neural Ranking for Addressing Stereotypical Gender Biases
Shirin SeyedSalehi, Amin Bigdeli, Negar Arabzadeh, Bhaskar Mitra, Morteza Zihayat, Ebrahim Bagheri
Proceedings of the 25th International Conference on Extending Database Technology (EDBT) | February 2022
Exposing Query Identification for Search Transparency
Ruohan Li, Jianxiang Li, Bhaskar Mitra, Fernando Diaz, Asia Biega
2022 The Web Conference | January 2022
Less is Less: When Are Snippets Insufficient for Human vs Machine Relevance Estimation?
Gabriella Kazai, Bhaskar Mitra, Anlei Dong, Nick Craswell, Linjun Yang
European Conference on Information Retrieval | November 2021
Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
European Conference on Information Retrieval | November 2021
MS MARCO Chameleons: Challenging the MS MARCO Leaderboard with Extremely Obstinate Queries
Negar Arabzadeh, Bhaskar Mitra, Ebrahim Bagheri
Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM) | August 2021
Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
Sebastian Hofstatter, Bhaskar Mitra, Hamed Zamani, Nick Craswell, Allan Hanbury
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
Daniel Cohen, Bhaskar Mitra, Oleg Lesota, Navid Rekabsaz, Carsten Eickhoff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | May 2021
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees, Ian Soboroff
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | April 2021
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell, Bhaskar Mitra, Emine Yilmaz
Proceedings of the 44th international ACM SIGIR conference on Research & development in information retrieval | March 2021
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos
Text REtrieval Conference (TREC) | February 2021
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
Jaime Arguello, Adam Ferguson, Emery Fine, Bhaskar Mitra, Hamed Zamani, Fernando Diaz
Proceedings of the 6th international ACM SIGIR Conference on Human Information Interaction and Retrieval | January 2021
Conformer-Kernel with Query Term Independence at TREC 2020 Deep Learning Track
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani, Nick Craswell
Text REtrieval Conference (TREC) | November 2020
Evaluating Stochastic Rankings with Expected Exposure
Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, Ben Carterette
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | July 2020
Best Long Research Paper Nominee
ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search
Nick Craswell, Daniel Campos, Bhaskar Mitra, Emine Yilmaz, Bodo von Billerbeck
Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM) | June 2020
Analyzing and Learning from User Interactions for Search Clarification
Hamed Zamani, Bhaskar Mitra, Everest Chen, Gord Lueck, Fernando Diaz, Paul Bennett, Nick Craswell, Susan Dumais
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
Local Self-Attention over Long Text for Efficient Document Retrieval
Sebastian Hofstätter, Hamed Zamani, Bhaskar Mitra, Nick Craswell, Allan Hanbury
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | May 2020
On the Reliability of Test Collections for Evaluating Systems of Different Types
Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Daniel Campos
Proceedings of the 43rd international ACM SIGIR conference on Research & development in information retrieval | April 2020
Overview of the TREC 2019 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Ellen M. Voorhees
Text REtrieval Conference (TREC) | March 2020
Duet at TREC 2019 Deep Learning Track
Text REtrieval Conference (TREC) | December 2019
An Axiomatic Approach to Regularizing Neural Ranking Models
Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, Saurabh Tiwary
Proceedings of the 42nd international ACM SIGIR conference on Research & development in information retrieval | April 2019
A Line in the Sand: Recommendation or Ad-hoc Retrieval?
Surya Kallumadi, Bhaskar Mitra, Tereza Iofciu
Proceedings of the Recommender Systems Challenge 2018 | July 2018
Cross Domain Regularization for Neural Ranking Models Using Adversarial Learning
Daniel Cohen, Bhaskar Mitra, Katja Hofmann, W. Bruce Croft
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Best Short Research Paper
Optimizing Query Evaluations Using Reinforcement Learning for Web Search
Corby Rosset, Damien Jose, Gargi Ghosh, Bhaskar Mitra, Saurabh Tiwary
Proceedings of the 41st international ACM SIGIR conference on Research & development in information retrieval | May 2018
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 40th European Conference on IR Research (ECIR) | March 2018
Advances in Information Retrieval
Neural Ranking Models with Multiple Document Fields
Hamed Zamani, Bhaskar Mitra, Xia Song, Nick Craswell, Saurabh Tiwary
Proceedings of the 11th ACM International Conference on Web Search and Data Mining (WSDM'18) | November 2017
Reply With: Proactive Recommendation of Email Attachments
Christophe Van Gysel, Bhaskar Mitra, Matteo Venanzi, Roy Rosemarin, Grzegorz Kukla, Piotr Grudzien, Nicola Cancedda
Proceedings of the 26th ACM International Conference on Information and Knowledge Management (CIKM) | October 2017
Benchmark for Complex Answer Retrieval
Federico Nanni, Bhaskar Mitra, Matt Magnusson, Laura Dietz
Proceedings of 3rd ACM International Conference on the Theory of Information Retrieval | October 2017
SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval | August 2017
Luandri: a Clean Lua Interface to the Indri Search Engine
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 40th international ACM SIGIR conference on Research & development in information retrieval | August 2017
Toward Incorporation of Relevant Documents in word2vec
Navid Rekabsaz, Bhaskar Mitra, Mihai Lupu, Allan Hanbury
Proceedings of the SIGIR 2017 Workshop on Neural Information Retrieval (Neu-IR'17) | July 2017
Neural Networks for Information Retrieval
Tom Kenter, Alexey Borisov, Christophe Van Gysel, Mostafa Dehghani, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2017
Learning to Match Using Local and Distributed Representations of Text for Web Search
Bhaskar Mitra, Fernando Diaz, Nick Craswell
Proceedings of the 26th International Conference on World Wide Web | April 2017
Neural Text Embeddings for Information Retrieval
Proceedings of the 10th ACM International Conference on Web Search and Data Mining (WSDM'17) | February 2017
Neu-IR: The SIGIR 2016 Workshop on Neural Information Retrieval
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval | July 2016
A Proposal for Evaluating Answer Distillation from Web Data
Bhaskar Mitra, Grady Simon, Jianfeng Gao, Nick Craswell, Li Deng
Proceedings of the Second WebQA Workshop (SIGIR 2016) | July 2016
Query Expansion with Locally-Trained Word Embeddings
Fernando Diaz, Bhaskar Mitra, Nick Craswell
Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics | June 2016
Improving Document Ranking with Dual Word Embeddings
Eric Nalisnick, Bhaskar Mitra, Nick Craswell, Rich Caruana
WWW'16 | April 2016
Query Auto-Completion for Rare Prefixes
Proceedings of the 24th ACM International Conference on Information and Knowledge Management (CIKM) | October 2015
Exploring Session Context using Distributed Representations of Queries and Reformulations
Proceedings of the 38th international ACM SIGIR conference on Research & development in information retrieval | August 2015
An Eye-tracking Study of User Interactions with Query Auto Completion
Katja Hofmann, Bhaskar Mitra, Filip Radlinski, Milad Shokouhi
Proceedings of the 23rd ACM International Conference on Information and Knowledge Management (CIKM) | November 2014
On User Interactions with Query Auto-Completion
Bhaskar Mitra, Milad Shokouhi, Filip Radlinski, Katja Hofmann
Proceedings of the ACM International Conference on Research and Development in Information Retrieval (SIGIR) | July 2014
-
Judging the Judges: A Collection of LLM-Generated Relevance Judgements
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
February 2025
Emancipatory Information Retrieval
February 2025
Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024
Hossein A. Rahmani, Clemencia Siro, Mohammad Aliannejadi, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
August 2024
Through the Looking-Glass: Transparency Implications and Challenges in Enterprise AI Knowledge Systems
Karina Cortiñas-Lorenzo, Siân Lindley, Ida Larsen-Ledet, Bhaskar Mitra
January 2024
DiSK: A Diffusion Model for Structured Knowledge
Ouail Kitouni, Niklas Nolte, James Hensman, Bhaskar Mitra
December 2023
Co-audit: tools to help humans double-check AI-generated content
Andy Gordon, Carina Negreanu, José Cambronero, Rasika Mudumbai Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson (nee Murfet), Advait Sarkar, Stephanie Simmons, Jack Williams, Ben Zorn
October 2023
Are We There Yet? A Decision Framework for Replacing Term-Based Retrieval with Dense Retrieval Systems
Sebastian Hofstätter, Nick Craswell, Bhaskar Mitra, Hamed Zamani, Allan Hanbury
June 2022
Semantic Product Search for Matching Structured Product Catalogs in E-Commerce
Jason Ingyu Choi, Surya Kallumadi, Bhaskar Mitra, Eugene Agichtein, Faizan Javed
August 2020
Conformer-Kernel with Query Term Independence for Document Retrieval
Bhaskar Mitra, Sebastian Hofstätter, Hamed Zamani, Nick Craswell
July 2020
Incorporating Query Term Independence Assumption for Efficient Retrieval and Ranking using Deep Neural Networks
Bhaskar Mitra, Corby Rosset, David Hawking, Nick Craswell, Fernando Diaz, Emine Yilmaz
July 2019
An updated duet model for passage re-ranking
March 2019
-
Proceedings of The First Workshop on Large Language Models for Evaluation in Information Retrieval (LLM4Eval 2024)
Clemencia Siro, Mohammad Aliannejadi, Hossein A. Rahmani, Nick Craswell, Charles L. A. Clarke, Guglielmo Faggioli, Bhaskar Mitra, Paul Thomas, Emine Yilmaz
September 2024
LLMJudge: LLMs for Relevance Judgments
Hossein A Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra, Paul Thomas, Charles L A Clarke, Mohammad Aliannejadi, Clemencia Siro, Guglielmo Faggioli
August 2024
Neural Models for Information Retrieval
May 2017
A Dual Embedding Space Model for Document Ranking
Bhaskar Mitra, Eric Nalisnick, Nick Craswell, Rich Caruana
February 2016
This paper is an extended evaluation and analysis of the model proposed in a poster to appear in WWW'16, April 11 - 15, 2016, Montreal, Canada.
-
Search and Society: Reimagining Information Access for Radical Futures
Information Retrieval Research Journal (IRRJ) | March 2025
Recall, Robustness, and Lexicographic Evaluation
Fernando Diaz, Michael D. Ekstrand, Bhaskar Mitra
ACM Transactions on Recommender Systems | February 2025
Report on The Search Futures Workshop at ECIR 2024
Leif Azzopardi, Charles L. A. Clarke, Paul Kantor, Bhaskar Mitra, Johanne R. Trippas, Zhaochun Ren
SIGIR Forum | July 2024
Result Diversification in Search and Recommendation: A Survey
Haolun Wu, Yansen Zhang, Chen Ma, Fuyuan Lyu, Bowei He, Bhaskar Mitra, Xue Liu
IEEE Transactions on Knowledge and Data Engineering | March 2024
A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
Haolun Wu, Chen Ma, Bhaskar Mitra, Fernando Diaz, Xue Liu
ACM Transactions on Information Systems (TOIS) | August 2022
Report on the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)
Chirag Shah, Torsten Suel, Fernando Diaz, Bhaskar Mitra, Bárbara Poblete, Hussein Suleman, Suzan Verberne
SIGIR Forum | February 2022, Vol 55(2)
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval (Dissertation Abstract)
SIGIR Forum | July 2021, Vol 55(1)
Report on the First HIPstIR Workshop on the Future of Information Retrieval
Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
An Introduction to Neural Information Retrieval
Foundations and Trends® in Information Retrieval | December 2018, Vol 13(1): pp. 1-126
Neural information retrieval: introduction to the special issue
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
Information Retrieval Journal : Special issue Neural | November 2017
Report on the Second SIGIR Workshop on Neural Information Retrieval (Neu-IR’17)
Nick Craswell, W. Bruce Croft, Maarten de Rijke, Jiafeng Guo, Bhaskar Mitra
SIGIR Forum | October 2017, Vol 52
Report on the SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR)
Nick Craswell, W. Bruce Croft, Jiafeng Guo, Maarten de Rijke, Bhaskar Mitra
SIGIR Forum | December 2016, Vol 50(2): pp. 96-103
-
Sociotechnical Implications of Generative Artificial Intelligence for Information Access
Bhaskar Mitra, Henriette Cramer, Olya Gurevich
Information Access in the Era of Generative AI, 7 | Published by Springer | 2025
-
Neural Methods for Effective, Efficient, and Exposure-Aware Information Retrieval
December 2020
PhD Thesis: University College London | December 2020
-
An Introduction to Computational Networks and the Computational Network Toolkit
Dong Yu, Adam Eversole, Mike Seltzer, Kaisheng Yao, Oleksii Kuchaiev, Yu Zhang, Frank Seide, Zhiheng Huang, Brian Guenter, Huaming Wang, Jasha Droppo, Geoffrey Zweig, Chris Rossbach, Jie Gao, Andreas Stolcke, Jon Currey, Malcolm Slaney, Guoguo Chen, Amit Agarwal, Chris Basoglu, Marko Padmilac, Alexey Kamenev, Vladimir Ivanov, Scott Cypher, Hari Parthasarathi, Bhaskar Mitra, Baolin Peng, Xuedong Huang
MSR-TR-2014-112 | October 2014