SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Update Seq2Seq QA example script to use SQuAD metric. (#14335)

Browse Source 
* Update postporcessing accordingly to use SQuAD metric.

* Update assets accordingly based on SQuAD metrics.

* Fix function naming error.
This commit is contained in:
karthikrangasai
2021-11-09 18:34:23 +05:30
committed by GitHub
parent be4a6c64dc
commit 4f24058c58
3 changed files with 201 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
examples/pytorch/test_examples.py
View File

@@ -274,10 +274,8 @@ class ExamplesTests(TestCasePlus):
with patch.object(sys, "argv", testargs):
run_squad_seq2seq.main()
result = get_results(tmp_dir)
self.assertGreaterEqual(result["eval_rouge1"], 10)
self.assertGreaterEqual(result["eval_rouge2"], 10)
self.assertGreaterEqual(result["eval_rougeL"], 10)
self.assertGreaterEqual(result["eval_rougeLsum"], 10)
self.assertGreaterEqual(result["eval_f1"], 30)
self.assertGreaterEqual(result["eval_exact"], 30)
def test_run_swag(self):
stream_handler = logging.StreamHandler(sys.stdout)