SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Fix missing reference titles in retrieval evaluation of RAG (#7817)

Browse Source
This commit is contained in:
Quentin Lhoest
2020-10-16 10:15:49 +02:00
committed by GitHub
parent 464b53f5e4
commit 466115b279
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
examples/rag/eval_rag.py
View File

@@ -72,7 +72,7 @@ def get_precision_at_k(args, preds_path, gold_data_path):
em = total = 0 em = total = 0
for hypo, reference in zip(hypos, references): for hypo, reference in zip(hypos, references):
hypo_provenance = set(hypo.split("\t")[:k]) hypo_provenance = set(hypo.split("\t")[:k])
ref_provenance = set(reference.split("\t")[1 : (k + 1)]) ref_provenance = set(reference.split("\t"))
total += 1 total += 1
em += len(hypo_provenance & ref_provenance) / k em += len(hypo_provenance & ref_provenance) / k