Learning to Account for Good Abandonment in Search Success Metrics

Abandonment in web search has been widely used as a proxy
to measure user satisfaction. Initially it was considered a
signal of dissatisfaction, however with search engines mov-
ing towards providing answer-like results, a new category of
abandonment was introduced and referred to as Good Aban-
donment. Predicting good abandonment is a hard problem
and it was the subject of several previous studies. All those
studies have focused, though, on predicting good abandon-
ment in oine settings using manually labeled data. Thus,
it remained a challenge how to have an online metric that
accounts for good abandonment. In this work we describe
how a search success metric can be augmented to account for
good abandonment sessions using a machine learned metric
that depends on user’s viewport information. We use real
user trac from millions of users to evaluate the proposed
metric in an A/B experiment. We show that taking good
abandonment into consideration has a signicant eect on
the overall performance of the online metric.