... between au-tomatic summaries (the x (a, a ) factor in the in- equality). Finally, adding elements to A can onlyincrease the chances of finding a pair of automaticsummaries satisfying the condition ... may changeby adding, for instance, additional gold standards? In this paper, we introduce a probabilistic framework, QARLA, that addresses such issues.Given a set of manual summaries and another ... comparesummaries.An evaluation framework should include, atleast:• A measure QM,X (a) ∈ [0, 1] that estimates the quality of an automatic summary a, us-ing the similarity metrics in X to compare the summary...