Chairman : Nicola Ferro

Participants : Shane Culpepper – Grace Hui Yang – Jochen Leidner – Miguel Martinez

Abstract:

IR is an heavily experimental discipline with a long-lasting tradition in evaluation and system performance measurement. Nevertheless, there are two major shortcomings: we would need more insightful and richer explanations of IR system performance and we should devise techniques for  prediction of IR system performance and the ahead design of IR systems, meant as the capability of designing them as to meet an expected level of performance before actually needing to implement them for discovering what are their performance. The panel will discuss whether performance explanation and performance prediction are possible, to what extent we have already achieved them and what are the next steps, and, especially, whether they are a real need or just an academic exercise.

Shane Culpepper (https://culpepper.io) completed his PhD at​ ​the University of Melbourne in 2008. He is currently a Vice-Chancellor’s Principal Research Fellow and Associate Professor at RMIT University where he is the​ ​Director of the Centre for Information Discovery. His research ​primarily ​focuses ​on efficiency and effectiveness trade-offs in large scale search systems. Research interests include information retrieval, text indexing, data compression, system evaluation, information discovery, learning to rank, natural language processing, algorithm engineering, and scalable distributed/parallel computing.

Dr. Grace Hui Yang is an Associate Professor in the Department of Computer Science at Georgetown University. Dr. Yang is leading the InfoSense (Information Retrieval and Sense- Making) group at Georgetown University, Washington D.C., U.S.A.. Dr. Yang obtained her Ph.D. from the Language Technologies Institute, Carnegie Mellon University in 2011. Dr. Yang’s current research interests include reinforcement learning, deep learning, dynamic search, search engine evaluation, privacy-preserving information retrieval, and information organization. Prior to this, she have conducted research on question answering, ontology construction, near-duplicate detection, multimedia information retrieval and opinion and sentiment detection. Dr. Yang’s research is supported by the Defense Advanced Defense Advanced Research Projects Agency and National Science Foundation.  Dr. Yang is a recipient of the National Science Foundation (NSF) Faculty Early Career Development Program (CAREER) Award. Dr. Yang have co-chaired the SIGIR 2013-2014 Doctoral Consortium, SIGIR 2017 Workshop, WSDM 2017 Workshop, ICTIR 2017 Workshop, CIKM 2015 Tutorial, ICTIR 2018 Short Paper and SIGIR 2018 Demonstration Paper Program Committees. Dr. Yang served on the editorial board of Information Retrieval Journal 2014 to 2017. She served as an area chair/Senior PC for SIGIR 2014-2018 and ACL 2016. Grace also co-organized the TREC Dynamic Domain Track 2015 to 2017. Home page: http://www.cs.georgetown.edu/~huiyang

Dr Jochen Leidner is Director of Research at Thomson Reuters, the parent company that owns the Reuters News agency. He is also a Royal Academy of Engineering Visiting Professor of Data Analytics at the University of Sheffield. He is a computer scientist and computational linguist by training, and he leads the London site of its global Corporate Research & Development organization, which he built up in 2013. He joined Thomson Reuters in 2008 in various research capacities and has worked in the UK, Switzerland, and the US.  Prior to that, he held a Royal Society Edinburgh Fellowship at the University of Edinburgh, where he was Principal Investigator on a mobile question answering project. He also held founding roles in startups and worked as a software developer for SAP. He holds a Master’s degree in computational linguistics, English language and literature and computer science from Friedrich-Alexander-University Erlangen-Nuremberg.  In 2002 he obtained a Master of Philosophy in Engineering (MPhil) at the University of Cambridge in Computer Speech, Text and Internet Technologies.  In 2007, he obtained a PhD in Informatics with his thesis ‘Toponym Resolution in Text’, which also won the First ACM SIGIR Doctoral Consortium Award In 2015 and again in 2016, he was twice winner of the Thomson Reuters Inventor of the Year award for the best patent application across the company globally. He has edited, authored or co-authored several dozen publications and over a dozen patent applications/patents. In 2016, his paper on social media mining of medical drug side effects won the ‘Best Method Paper’ award at the London meeting of the Social Media Society. His research focuses on improving information access to or using textual data, especially information extraction, and question answering. He is also interested in methodology and ethics.

Dr. Miguel Martinez is the Co-founder and Chief Data Scientist at Signal Media, a fast-growing UK company that analyses in real-time millions of news articles per day in order to improve the quality of business intelligence and decision making across organisations. During the last five years, Miguel has led the Signal Research team, building an R&D group with strong university collaborations, with the main goal of transforming the best research principles, models and algorithms from academia into real, scalable products. Research interests among others include news processing, information retrieval, text analysis, text classification, entity linking, natural language processing and system evaluation.
During his time at Signal, the company has grown from three to 70+ people and raised more than £15M in funding. Miguel has been awarded the Business Leader of Tomorrow Award in 2014 by Innovate UK and was included in Bloomberg’s list of UK Business Innovators in 2016. The team has also won the Best Demonstration award in ECIR’15 and they are the main organisers of the NewsIR workshop.