SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
3,542 Commits 5 Branches 239 Tags
cd21d8bc00ec79a09ef313d738a424923a87b9ad
Commit Graph

387 Commits

Author SHA1 Message Date
Mohamed El-Geish
cd21d8bc00 Typo in warning message (#3219) 
`T5Tokenizer` instead of `XLNetTokenizer`
 2020-03-19 09:49:25 -04:00
Matthew Goldey
8d3e218ea6 fix typo in docstring demonstrating usage (#3213) 2020-03-19 09:47:54 -04:00
Patrick von Platen
cec3cdda15 Fix input ids can be none attn mask (#3345) 
* fix issue 3289

* fix attention mask if input_ids None behavior
 2020-03-19 09:55:17 +01:00
Lysandre Debut
d6afbd323d XLM-R Tokenizer now passes common tests + Integration tests (#3198) 
* XLM-R now passes common tests + Integration tests

* Correct mask index

* Model input names

* Style

* Remove text preprocessing

* Unneccessary import
 2020-03-18 09:52:49 -04:00
Patrick von Platen
292186a3e7 Adding LM Head to Transfo-XL and first step to fixing problem with Adaptive Embeddings in TransfoXL (#3286) 
* first commit

* work in progress

* make language generation task pass

* update to working version for LM

* delete print

* remove dead code

* make style
 2020-03-18 09:24:27 -04:00
Patrick von Platen
ddb10c6447 improve doctstring (#3327) 2020-03-18 13:24:09 +01:00
Sam Shleifer
38a555a83c Add Summarization to Pipelines (#3128) 
* passing

* Undo stupid chg

* docs

* undo rename

* delete-cruft

* only import if you have torch

* Dont rely on dict ordering

* Fix dict ordering upstream

* docstring link

* docstring link

* remove trailing comma for 3.5 compat

* new name

* delegate kwarging

* Update kwargs
 2020-03-17 18:04:21 -04:00
Patrick von Platen
e8f44af5bf [generate] do_sample default back to False (#3298) 
* change do_samples back

* None better default as boolean

* adapt do_sample to True in test example

* make style
 2020-03-17 10:52:37 -04:00
Thomas Wolf
2187c49f5c CPU/GPU memory benchmarking utilities - Remove support for python 3.5 (now only 3.6+) (#3186) 
* memory benchmark rss

* have both forward pass and line-by-line mem tracing

* cleaned up tracing

* refactored and cleaning up API

* no f-strings yet...

* add GPU mem logging

* fix GPU memory monitoring

* style and quality

* clean up and doc

* update with comments

* Switching to python 3.6+

* fix quality
 2020-03-17 10:17:11 -04:00
Patrick von Platen
4759176313 add camembert for Question answering for examples 2020-03-16 14:42:11 -04:00
Sam Shleifer
11573231c6 [BART] generation_mode as a kwarg not a class attribute (#3278) 2020-03-16 12:47:53 -04:00
Julien Plu
879e1d3234 Add TF2 version of FlauBERT (#2700) 
* Add TF2 version of FlauBERT

* Add TF2 version of FlauBERT

* Add documentation

* Apply style and quality

* Apply style once again

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-03-16 09:29:21 -04:00
Patrick von Platen
af471ce5e8 Improved Error message when loading config/model with .from_pretrained() (#3247) 
* better error message

* better error message

* update to model identifier instead of url

* update to model identifier instead of ur
 2020-03-16 09:48:30 +01:00
Sam Shleifer
5ea8ba67b4 [BART] Remove unused kwargs (#3279) 
* Remove unused kwargs
* dont call forward in tests
 2020-03-15 23:00:44 -04:00
Thomas Wolf
3814e167d9 Merge pull request #3225 from patrickvonplaten/finalize_merge_bart_generate_into_default_generate 
Complete merge Seq-2-Seq generation into default generation
 2020-03-14 15:08:59 +01:00
Sam Shleifer
2bd79e23de [BART] FP16 testing fixes (#3266) 2020-03-13 19:48:26 -04:00
Patrick von Platen
ab756f713c add gpt2-xl for tf 2020-03-13 16:40:35 -04:00
Patrick von Platen
6047f46b19 re-add eos token to get good bart results 2020-03-12 20:17:50 +01:00
Patrick von Platen
c11160114a small clean-up 2020-03-12 20:02:35 +01:00
Sam Shleifer
2e81b9d8d7 Bart: update example for #3140 compatibility (#3233) 
* Update bart example docs
 2020-03-12 10:36:37 -04:00
Patrick von Platen
a332cc9f7f finalize generation merge 2020-03-11 11:53:36 +01:00
Patrick von Platen
1ba21f96ca fix bug in tf no_repeat_ngram_size 2020-03-11 11:06:56 +01:00
Patrick von Platen
d997ac7810 fix typo 2020-03-11 11:06:56 +01:00
Patrick von Platen
7351a8dbaf re-add scoring filtering 2020-03-11 11:06:56 +01:00
Patrick von Platen
9b8ee8cea0 delete print and make style 2020-03-11 11:06:56 +01:00
Patrick von Platen
ca1330f0b2 do not mess with the negative sign 2020-03-11 11:06:56 +01:00
Patrick von Platen
10989715d0 rename variable 2020-03-11 11:06:56 +01:00
Patrick von Platen
cf06290565 remove ipdb 2020-03-11 11:06:56 +01:00
Patrick von Platen
374deef48d fixed typo 2020-03-11 11:06:56 +01:00
Patrick von Platen
a2c8e516c2 fix torch to tf translation 2020-03-11 11:06:56 +01:00
Patrick von Platen
ca2047bc35 refactor variable naming and improve tf generate in line with torch generate 2020-03-11 11:06:56 +01:00
patrickvonplaten
41b437ea3a add draft version of propsoed changes for ROGUE score 2020-03-11 11:06:56 +01:00
patrickvonplaten
629aac92ec do not allow do_sample and weird force bos token things 2020-03-11 11:06:56 +01:00
patrickvonplaten
d880a5fbde finalized PR 2020-03-11 11:06:56 +01:00
patrickvonplaten
2acfe63964 best current version and make style 2020-03-11 11:06:56 +01:00
patrickvonplaten
c62444da39 fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen
333affcb81 add current changes 2020-03-11 11:06:56 +01:00
Patrick von Platen
7a11e925cf work in progress 2020-03-11 11:06:56 +01:00
Patrick von Platen
7cba11fb9b better naming 2020-03-11 11:06:56 +01:00
Patrick von Platen
ff648221bd fix conflicts 2020-03-11 11:06:56 +01:00
Patrick von Platen
c0d9dd3ba9 refactored code a bit and made more generic 2020-03-11 11:06:56 +01:00
Patrick von Platen
d8e2b3c547 fix conflicts 2020-03-11 11:06:56 +01:00
Julien Chaumond
270dfa1c8e [dialogpt] conversion script 
Reference: https://github.com/huggingface/transformers/pull/1778#issuecomment-567675530

cc @patrickvonplaten and @dreasysnail
 2020-03-10 15:09:29 -04:00
Patrick von Platen
31f2437f07 Merge pull request #3191 from patrickvonplaten/add_integration_tests_lm_generate_torch_tf 
Add integration tests lm generate torch tf
 2020-03-10 11:29:17 +01:00
Julien Chaumond
cbf8f5d32b [model upload] Support for organizations 2020-03-09 17:33:57 -04:00
Lysandre Debut
5164ea91a7 Skipping outputs (#3116) 
* Minimal example

* Proposal 2

* Proposal 2 for fast tokenizers

* Typings

* Docs

* Revert "Docs" for easier review

This reverts commit eaf0f97062e809887704a542144c537f769d5223.

* Remove unnecessary assignments

* Tests

* Fix faulty type

* Remove prints

* return_outputs -> model_input_names

* Revert "Revert "Docs" for easier review"

This reverts commit 6fdc69408102bf695797f2dfddbb6350c6b9e722.

* code quality
 2020-03-09 13:48:58 -04:00
Patrick von Platen
847d370301 fix typo 2020-03-09 16:18:29 +01:00
Patrick von Platen
3e624c64ca fix repetition penalty mask in tf 2020-03-09 14:55:11 +01:00
Patrick von Platen
4620caa864 fix if use lang embeddings in tf xlm 2020-03-09 11:18:54 +01:00
Lysandre Debut
0ae91c80aa Change back pipeline signatures (#3105) 
* Change back pipeline signatures

* String types for non-imported objects
 2020-03-06 17:26:18 -05:00