SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Add an examples folder for code downstream tasks (#18679)

Browse Source 
* add examples subfolder

* mention examples in codeparrot readme

* use Trainer optimizer and scheduler type and add output_dir as argument

* add example of text-to-python and python-to-text models

* mention the downstream examples in the readme

* fix typo
This commit is contained in:
Loubna Ben Allal
2022-08-18 18:24:24 +02:00
committed by GitHub
parent a123eee9df
commit bbbb453e58
5 changed files with 200 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
examples/research_projects/codeparrot/scripts/minhash_deduplication.py
View File

@@ -63,7 +63,6 @@ class DuplicationIndex:
self._index.insert(code_key, min_hash)
if len(close_duplicates) > 0:
for base_duplicate in close_duplicates:
if base_duplicate in self._duplicate_clusters:
self._duplicate_clusters[base_duplicate].add(code_key)