JB (Don)
78a2b19fc8
Show a warning for missing attention masks when pad_token_id is not None ( #24510 )
...
* Adding warning messages to BERT for missing attention masks
These warning messages when there are pad tokens within the input ids and
no attention masks are given. The warning message should only show up once.
* Adding warning messages to BERT for missing attention masks
These warning messages are shown when the pad_token_id is not None
and no attention masks are given. The warning message should only
show up once.
* Ran fix copies to copy over the changes to some of the other models
* Add logger.warning_once.cache_clear() to the test
* Shows warning when there are no attention masks and input_ids start/end with pad tokens
* Using warning_once() instead and fix indexing in input_ids check
---------
Co-authored-by: JB Lau <hckyn@voyager2.local >
2023-06-30 08:19:39 -04:00
Arthur
b52a03cd3b
⚠️ ⚠️ [T5Tokenize] Fix T5 family tokenizers ⚠️ ⚠️ ( #24565 )
...
* don't add space before single letter chars that don't have a merge
* fix the fix
* fixup
* add a test
* more testing
* fixup
* hack to make sure fast is also fixed
* update switch transformers test
* revert convert slow
* Update src/transformers/models/t5/tokenization_t5.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com >
* add typechecking
* quality
---------
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com >
2023-06-30 07:00:43 +02:00
amyeroberts
b324557aac
Removal of deprecated vision methods and specify deprecation versions ( #24570 )
...
* Removal of deprecated methods and specify versions
* Fix tests
2023-06-29 15:09:51 +01:00
Yih-Dar
77db28dc52
Update some torchscript tests after #24505 ( #24566 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-29 16:05:24 +02:00
Sanchit Gandhi
1c1c90756d
Add Musicgen ( #24109 )
...
* Add Audiocraft
* add cross attention
* style
* add for lm
* convert and verify
* introduce t5
* split configs
* load t5 + lm
* clean conversion
* copy from t5
* style
* start pattern provider
* make generation work
* style
* fix pos embs
* propagate shape changes
* propagate shape changes
* style
* delay pattern: pad tokens at end
* audiocraft -> musicgen
* fix inits
* add mdx
* style
* fix pad token in processor
* override generate and add todos
* add init to test
* undo pattern delay mask after gen
* remove cfg logits processor
* remove cfg logits processor
* remove logits processor in favour of mask
* clean pos embs
* make fix copies
* update readmes
* clean pos emb
* refactor encoder/decoder
* make fix copies
* update conversion
* fix config imports
* update config docs
* make style
* send pattern mask to device
* pattern mask with delay
* recover prompted audio tokens
* fix docstrings
* laydown test file
* pattern edge case
* remove t5 ref
* add processing class
* config refactor
* better pattern comment
* check if mask is not present
* check if mask is not present
* refactor to auto class
* remove encoder configs
* fix processor
* processor import
* start updating conversion
* start updating tests
* make style
* convert t5, encodec, lm
* convert as composite
* also convert processor
* run generate
* classifier free gen
* comments and clean up
* make style
* docs for logit proc
* docstring for uncond gen
* start lm tests
* work tests
* let the lm generate
* refactor: reshape inside forward
* undo greedy loop changes
* from_enc_dec -> from_sub_model
* fix input id shapes in docstrings
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* undo generate changes
* from sub model config
* Update src/transformers/models/musicgen/modeling_musicgen.py
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* make generate work again
* generate uncond -> get uncond inputs
* remove prefix allowed tokens fn
* better error message
* logit proc checks
* Apply suggestions from code review
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
* make decoder only tests work
* composite fast tests
* make style
* uncond generation
* feat extr padding
* make audio prompt work
* fix inputs docstrings
* unconditional inputs: dict -> model output
* clean up tests
* more clean up tests
* make style
* t5 encoder -> auto text encoder
* remove comments
* deal with frames
* fix auto text
* slow tests
* nice mdx
* remove can generate
* todo - hub id
* convert m/l
* make fix copies
* only import generation with torch
* ignore decoder from tests
* don't wrap uncond inputs
* make style
* cleaner uncond inputs
* add example to musicgen forward
* fix docs
* ignore MusicGen Model/ForConditionalGeneration in auto mapping
* add doc section to toctree
* add to doc tests
* add processor tests
* fix push to hub in conversion
* tips for decoder only loading
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com >
* fix conversion for s / m / l checkpoints
* import stopping criteria from module
* remove from pipeline tests
* fix uncond docstring
* decode audio method
* fix docs
* org: sanchit-gandhi -> facebook
* fix max pos embeddings
* remove auto doc (not compatible with shapes)
* bump max pos emb
* make style
* fix doc
* fix config doc
* fix config doc
* ignore musicgen config from docstring
* make style
* fix config
* fix config for doctest
* consistent from_sub_models
* don't automap decoder
* fix mdx save audio file
* fix mdx save audio file
* processor batch decode for audio
* remove keys to ignore
* update doc md
* update generation config
* allow changes for default generation config
* update tests
* make style
* fix docstring for uncond
* fix processor test
* fix processor test
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com >
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com >
2023-06-29 14:48:59 +01:00
amyeroberts
ae454f41d4
Update old existing feature extractor references ( #24552 )
...
* Update old existing feature extractor references
* Typo
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Address comments from review - update 'feature extractor'
Co-authored by: Yih-Dar <2521628+ydshieh@users.noreply.github.com >
2023-06-29 10:17:36 +01:00
Yih-Dar
fd6735102a
Make PT/Flax tests could be run on GPU ( #24557 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-28 20:11:01 +02:00
Younes Belkada
33b5ef5cdf
[InstructBlip] Add instruct blip int8 test ( #24555 )
...
* add 8bit instructblip test
* update tests
2023-06-28 19:06:30 +02:00
Yih-Dar
b0651655be
Update EncodecIntegrationTest ( #24553 )
...
* fix
* fix
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-28 18:01:41 +02:00
Sylvain Gugger
89b6ee49fd
Finishing tidying keys to ignore on load ( #24535 )
2023-06-27 21:35:15 -04:00
Sylvain Gugger
8e5d1619b3
Clean load keys ( #24505 )
...
* Preliminary work on some models
* Fix test load missing and make sure nonpersistent buffers are tested
* Always ignore nonpersistent buffers if in state_dict
* Treat models
* More models
* Treat remaining models
* Fix quality
* Fix tests
* Remove draft
* This test is not needed anymore
* Fix copies
* Fix last test
* Newly added models
* Fix last tests
* Address review comments
2023-06-27 14:45:40 -04:00
Sebastian
06910f5a76
[T5] Add T5ForQuestionAnswering and MT5ForQuestionAnswering ( #24481 )
...
* Adding T5ForQuestionAnswering
* Changed weight initialization that results in better initial loss when fine-tuning
* Update to class variables
* Running make fixup
* Running make fix-copies
* Remove model_parallel
* Adding MT5ForQuestionAnswering
* Adding docs
* Fix wrong doc
* Update src/transformers/models/mt5/modeling_mt5.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com >
* Update src/transformers/models/t5/modeling_t5.py
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com >
* File formatting
* Undoing change
---------
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com >
2023-06-27 10:07:06 -04:00
Xiaoli Wang
239ace152b
Fix TypeError: Object of type int64 is not JSON serializable ( #24340 )
...
* Fix TypeError: Object of type int64 is not JSON serializable
* Convert numpy.float64 and numpy.int64 to float and int for json serialization
* Black reformatted examples/pytorch/token-classification/run_ner_no_trainer.py
* * make style
2023-06-27 12:15:49 +01:00
Yih-Dar
850cf4af0c
Compute dropout_probability only in training mode ( #24486 )
...
* fix
* fix
* fix
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-26 18:36:47 +02:00
Yih-Dar
3ca022238b
Update InstructBlipModelIntegrationTest ( #24490 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-26 14:37:12 +02:00
Matthijs Hollemans
3b84d86b57
add missing alignment_heads to Whisper integration test ( #24487 )
...
add missing alignment heads
2023-06-26 11:50:10 +02:00
NielsRogge
868363abb9
Add InstructBLIP ( #23460 )
...
* Squash 88 commits
* Use markdown
* Remove mdx files due to bad rebase
* Fix modeling files due to bad rebase
* Fix style
* Update comment
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-26 11:23:57 +02:00
Yih-Dar
2898fd3968
Fix some TFWhisperModelIntegrationTests ( #24428 )
...
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* Update src/transformers/models/whisper/modeling_tf_whisper.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* Update src/transformers/models/whisper/modeling_tf_whisper.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2023-06-23 14:27:49 +02:00
Younes Belkada
3ce3385c47
Revert "Fix gradient checkpointing + fp16 autocast for most models" ( #24420 )
...
Revert "Fix gradient checkpointing + fp16 autocast for most models (#24247 )"
This reverts commit 285a48011d
2023-06-22 16:11:27 +02:00
Matthijs Hollemans
cd927a4736
add word-level timestamps to Whisper ( #23205 )
...
* let's go!
* initial implementation of token-level timestamps
* only return a single timestamp per token
* remove token probabilities
* fix return type
* fix doc comment
* strip special tokens
* rename
* revert to not stripping special tokens
* only support models that have alignment_heads
* add integration test
* consistently name it token-level timestamps
* small DTW tweak
* initial support for ASR pipeline
* fix pipeline doc comments
* resolve token timestamps in pipeline with chunking
* change warning when no final timestamp is found
* return word-level timestamps
* fixup
* fix bug that skipped final word in each chunk
* fix failing unit tests
* merge punctuations into the words
* also return word tokens
* also return token indices
* add (failing) unit test for combine_tokens_into_words
* make combine_tokens_into_words private
* restore OpenAI's punctuation rules
* add pipeline tests
* make requested changes
* PR review changes
* fix failing pipeline test
* small stuff from PR
* only return words and their timestamps, not segments
* move alignment_heads into generation config
* forgot to set alignment_heads in pipeline tests
* tiny comment fix
* grr
2023-06-21 17:48:21 +02:00
Younes Belkada
285a48011d
Fix gradient checkpointing + fp16 autocast for most models ( #24247 )
...
* fix gc bug
* continue PoC on OPT
* fixes
* 🤯
* fix tests
* remove pytest.mark
* fixup
* forward contrib credits from discussions
* forward contrib credits from discussions
* reverting changes on untouched files.
---------
Co-authored-by: zhaoqf123 <zhaoqf123@users.noreply.github.com >
Co-authored-by: 7eu7d7 <7eu7d7@users.noreply.github.com >
2023-06-21 17:04:59 +02:00
Patrick von Platen
b0513b013b
[Wav2Vec2 - MMS] Correct directly loading adapters weights ( #24335 )
...
* Correct direct lang loading
* correct more
* revert black
* Use tie weights instead=
* add tests
* add tests
* make style
2023-06-20 19:39:52 +02:00
Arthur
e5c760d636
[GPTNeoX] Nit in config ( #24349 )
...
* add raise value error for attention size
* nits to fix test_config
* style
2023-06-20 19:19:19 +02:00
Yih-Dar
83dc5762e7
Skip a tapas (tokenization) test in past CI ( #24378 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-20 18:35:45 +02:00
Yih-Dar
297d769d0e
Better test name and enable pipeline test for pix2struct ( #24377 )
...
* best test name forever
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-20 18:29:30 +02:00
Yih-Dar
0527c1c0ea
Add a check in ImageToTextPipeline._forward ( #24373 )
...
* fix
* fix
* fix
* Update src/transformers/pipelines/image_to_text.py
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com >
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com >
2023-06-20 18:07:34 +02:00
Sylvain Gugger
dc4449918d
Rename test to be more accurate ( #24374 )
2023-06-20 11:54:55 -04:00
Sanchit Gandhi
6c1344444a
[Whisper] Make tests faster ( #24105 )
2023-06-20 16:01:56 +01:00
Yih-Dar
c23d131eab
Update tiny models for pipeline testing. ( #24364 )
...
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-20 14:43:10 +02:00
Matt
56efbf4301
TensorFlow CI fixes ( #24360 )
...
* Fix saved_model_creation_extended
* Skip the BLIP model creation test for now
* Fix TF SAM test
* Fix longformer tests
* Fix Wav2Vec2
* Add a skip for XLNet
* make fixup
* make fix-copies
* Add comments
2023-06-20 12:59:21 +01:00
Matt
3403712958
Big TF test cleanup ( #24282 )
...
* Fix one BLIP arg not being optional, remove misspelled arg
* Remove the lxmert test overrides and just use the base test_saved_model_creation
* saved_model_creation fixes and re-enabling tests across the board
* Remove unnecessary skip
* Stop caching sinusoidal embeddings in speech_to_text
* Fix transfo_xl compilation
* Fix transfo_xl compilation
* Fix the conditionals in xglm
* Set the save spec only when building
* Clarify comment
* Move comment correctly
* Correct embeddings generation for speech2text
* Mark RAG generation tests as @slow
* Remove redundant else:
* Add comment to clarify the save_spec line in build()
* Fix size tests for XGLM at last!
* make fixup
* Remove one band_part operation
* Mark test_keras_fit as @slow
2023-06-16 15:40:49 +01:00
Yih-Dar
896a58de15
Byebye pytorch 1.9 ( #24080 )
...
byebye
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-16 16:38:23 +02:00
Matt
62d71f4083
Fix functional TF Whisper and modernize tests ( #24301 )
...
* Revert whisper change and modify the test_compile_tf_model test
* make fixup
* Tweak test slightly
* Add functional model saving to test
* Ensure TF can infer shapes for data2vec
* Add override for efficientformer
* Mark test as slow
2023-06-16 14:43:43 +01:00
Sanchit Gandhi
4124a09f8b
[EnCodec] Changes for 32kHz ckpt ( #24296 )
...
* [EnCodec] Changes for 32kHz ckpt
* Update src/transformers/models/encodec/convert_encodec_checkpoint_to_pytorch.py
* Update src/transformers/models/encodec/convert_encodec_checkpoint_to_pytorch.py
2023-06-15 14:36:19 +01:00
Matthijs Hollemans
0c3fdccf2f
[WIP] add EnCodec model ( #23655 )
...
* boilerplate stuff
* messing around with the feature extractor
* fix feature extractor
* unit tests for feature extractor
* rename speech to audio
* quick-and-dirty import of Meta's code
* import weights (sort of)
* cleaning up
* more cleaning up
* move encoder/decoder args into config
* cleanup model
* rename EnCodec -> Encodec
* RVQ parameters in config
* add slow test
* add lstm init and test_init
* Add save & load
* finish EncodecModel
* remove decoder_input_values as they are ont used anywhere (not removed from doc yet)
* fix test feature extraction model name
* Add better slow test
* Fix tests
* some fixup and cleaning
* Improve further
* cleaning up quantizer
* fix up conversion script
* test don't pass, _encode_fram does not work
* update tests with output per encode and decode
* more cleanup
* rename _codebook
* remove old config cruft
* ratios & hop_length
* use ModuleList instead of Sequential
* clean up resnet block
* update types
* update tests
* fixup
* quick cleanup
* fix padding
* more styl,ing
* add patrick feedback
* fix copies
* fixup
* fix lstm
* fix shape issues
* fixup
* rename conv layers
* fixup
* fix decoding
* small conv refactoring
* remove norm_params
* simplify conv layers
* rename conv layers
* stuff
* Clean up
* Add padding logic
use padding mask
small conv refactoring
remove norm_params
simplify conv layers
rename conv layers
stuff
add batched test
update
Clean up
merge and update for padding
fix padding
fixup
* clean up more
* clean up more
* More clean ups
* cleanup convolutions
* typo
* fix typos
* fixup
* build PR doc?
* start refactoring docstring
* fix don't pad when no strid and chunk
* update docstring
* update docstring
* nits
* update going to lunch
* update config and model
* fix broken testse (becaue of the config changes)
* fix scale computation
* fixu[
* only return dict if speciefied or if config returns it
* remove todos
* update defaults in config
* update conversion script
* fix doctest
* more docstring + fixup
* nits on batched_tests
* more nits
* Apply suggestions from code review
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
* update basxed on review
* fix update
* updaet tests
* Apply suggestions from code review
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com >
* fixup
* add overlap and chunl_length_s
* cleanup feature extraction
* teste edge cases truncation and padding
* correct processor values
* update config encodec, nits
* fix tests
* fixup
* fix 24Hz test
* elle tests are green
* fix fixup
* Apply suggestions from code review
* revert readme changes
* fixup
* add example
* use facebook checkpoints
* fix typo
* no pipeline tests
* use slef.pad everywhere we can
* Apply suggestions from code review
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* update based on review
* update
* update mdx
* fix bug and tests
* fixup
* fix doctest
* remove comment
* more nits
* add more coverage for `test_truncation_and_padding`
* fixup
* add last test
* fix text
* nits
* Update tests/models/encodec/test_modeling_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* take care of the last comments
* typo
* fix test
* nits
* fixup
* Update src/transformers/models/encodec/feature_extraction_encodec.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
---------
Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com >
Co-authored-by: arthur.zucker@gmail.com <arthur.zucker@gmail.com >
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com >
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com >
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2023-06-14 18:57:23 +02:00
Yih-Dar
a04ebc8b33
Pix2StructImageProcessor requires torch>=1.11.0 (#24270 )
...
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-14 17:05:40 +02:00
Joao Gante
4626df5077
TF: CTRL with native embedding layers ( #23456 )
2023-06-14 14:39:02 +01:00
Yih-Dar
233113149b
Skip GPT-J fx tests for torch < 1.12 ( #24256 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-13 20:33:26 +02:00
Matt
3bd1fe4315
Stop storing references to bound methods via tf.function ( #24146 )
...
* Stop storing references to bound methods in tf.functions
* Remove the gc.collect calls now that we resolved the underlying problem
* Remove the default signature from model.serving entirely, big cleanup
* Remove _prune_signature as self.input_signature can prune itself
* Restore serving docstring
* Update int support test to check the input signature
* Make sure other tests also use model.input_signature and not serving.input_signature
* Restore _prune_signature
* Remove the doctest GC now it's no longer needed
* Correct core tests to use the pruned sig
* order lines correctly in core tests
* Add eager_serving back with a deprecation warning
2023-06-13 19:04:22 +01:00
Yih-Dar
cf561d7cf1
Add torch >=1.12 requirement for Tapas ( #24251 )
...
* fix
* fix
* fix
* Update src/transformers/models/tapas/modeling_tapas.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com >
2023-06-13 19:19:40 +02:00
Joao Gante
7bb6933b9d
TF: standardize test_model_common_attributes for language models ( #23457 )
2023-06-13 17:51:37 +01:00
Yih-Dar
74b846cacf
Update (TF)SamModelIntegrationTest ( #24199 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-13 14:28:14 +02:00
Yih-Dar
4fe9716a79
Skip RWKV test in past CI ( #24204 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-12 18:14:15 +02:00
Yih-Dar
dadc9fb427
Update GPTNeoXLanguageGenerationTest ( #24193 )
...
* fix
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-12 15:37:12 +02:00
Yih-Dar
e26c6f03be
Fix Wav2Vec2 CI OOM ( #24190 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-12 11:39:04 +02:00
Younes Belkada
62fe753325
[SAM] Fix sam slow test ( #24140 )
...
* fix sam test
* update pipeline typehint
2023-06-09 16:22:09 +02:00
Yih-Dar
847b47c0ee
Fix XGLM OOM on CI ( #24123 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-09 15:20:59 +02:00
Yih-Dar
b8fe259f16
Fix SAM OOM issue on CI ( #24125 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-09 15:07:08 +02:00
Yih-Dar
707023d155
Fix TF Rag OOM issue ( #24122 )
...
fix
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-09 15:03:11 +02:00
Yih-Dar
2e2088f24b
Avoid GPT-2 daily CI job OOM (in TF tests) ( #24106 )
...
* fix
* fix
---------
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com >
2023-06-08 18:21:09 +02:00
