Commit Graph

11108 Commits

Author SHA1 Message Date
Joao Gante
e0b825a8d0 Generate: contrastive search test updates (#19787)
* contrastive search test updates

* make fixup
2022-10-21 19:10:08 +01:00
Sylvain Gugger
c4a997cd85 Use None to detect if truncation was unset (#19794)
* Use None to detect if truncation was unset

* Fix repo consistency
2022-10-21 12:53:37 -04:00
Serhii Hamotskyi
2e5c6f5975 Fix error/typo in docstring of TokenClassificationPipeline (#19798) 2022-10-21 12:53:16 -04:00
Alara Dirik
cca51aa151 Fix image segmentation pipeline errors, resolve backward compatibility issues (#19768)
* Fix panoptic segmentation and pipeline
* Update ImageSegmentationPipeline tests and reenable test_small_model_pt
* Resolve backward compatibility issues
2022-10-21 18:09:58 +03:00
Sylvain Gugger
b58d4f70f6 Fix nightly test setup (#19792) 2022-10-21 10:26:30 -04:00
Yih-Dar
3a1aeea3c5 Fix CTRL test_torchscrip_xxx CI by updating _create_and_check_torchscript (#19786)
* Run inputs before trace

* Run inputs before trace

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-21 16:23:13 +02:00
Hao Wang
31565ff0fd Add sentencepiece to BertJapaneseTokenizer (#19769)
* support sentencepiece for bertjapanesetokenizer

* add test vocab file for sentencepiece, bertjapanesetokenizer

* make BasicTokenizer be identical to transformers.models.bert.tokenization_bert.BasicTokenizer

* fix missing of \n in comment

* fix init argument missing in tests

* make spm_file be optional, exclude spiece.model from tests/fixtures, and add description comments

* make comment length less than 119

* apply doc style check
2022-10-21 10:04:49 -04:00
Davi Alves
2ebf4e6a7b [ custom_models.mdx ] - Translated to Portuguese the custom models tutorial. (#19779) 2022-10-21 09:48:19 -04:00
ftorres16
c1f009ad9a Update training.mdx (#19791) 2022-10-21 09:46:44 -04:00
Sylvain Gugger
9151e649a5 Make public versions of private tensor utils (#19775)
* Make public versions of private utils

* I need sleep
2022-10-21 09:34:01 -04:00
Yih-Dar
3aaabaa214 Update ImageToTextPipelineTests.test_small_model_tf (#19785)
* update expected values for the correct TF checkpoint

* Run test

* Clean up

* fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-21 14:35:20 +02:00
Kashif Rasul
7487829a23 Added support for multivariate independent emission heads (#19453)
* Added support for multivariate independent emission heads

* fix typo

* rename distr_cls

* scale is a vector for multivariate

* set affine transform event_dim

* fix typo

* added variable

* added beta in the config

* set beta

* remove beta-nll option in nll
2022-10-21 08:32:10 -04:00
Matt
a5da6f1817 Add warning about restarting runtime to import errors (#19774)
* Add warning about restarting runtime to import errors

* Fix some linebreaks
2022-10-21 11:52:29 +01:00
Yih-Dar
84f6bee5da PT <-> TF for composite models (#19732)
* First step of PT->TF for composite models

* Update the tests

* For VisionEncoderDecoderModel

* Fix

* Fix

* Add comment

* Fix

* clean up import

* Save memory

* For (TF)EncoderDecoderModel

* For (TF)EncoderDecoderModel

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-21 12:40:39 +02:00
Yih-Dar
12ce2941c7 Fix docker image build (#19759)
* Use 2 jobs for the docker image build (latest torch + DS)

* fix

* Add comment

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-20 20:36:13 +02:00
Yih-Dar
15fd39ea0e Install tf2onnx dev version (#19755)
* pin tf2onnx<=1.12.0

* Install tf2onnx main

* Pin to a specific commit

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-20 20:24:39 +02:00
Joao Gante
5ed9bd1896 TF: sample generation compatible with XLA and dynamic batch sizes (#19773) 2022-10-20 19:01:22 +01:00
Lianmin Zheng
c186e816bd [FLAX] Add dtype to embedding for gpt2 model (#18462)
* [FLAX] Add dtype to embedding for gpt2 model

* lint
2022-10-20 18:15:49 +02:00
Chino
baa00f65ae Fix exception thrown using MishActivation (#19739)
* Fix exception thrown using MishActivation

* Update activations.py
2022-10-20 09:13:35 -04:00
Rohit Gupta
2dd1b8f0c5 adding key pair dataset (#19765) 2022-10-20 09:05:49 -04:00
IMvision12
17d7aec895 Update modeling_layoutlmv3.py (#19753) 2022-10-20 13:47:17 +01:00
Nicolas Patry
a40386669f image-segmentation pipeline: re-enable small_model_pt test. (#19716)
* Re-enable `small_model_pt`.

Re-enable `small_model_pt`.

Enabling the current test with the current values.

Debugging the values on the CI.

More logs ? Printing doesn't work ?

Using the CI values instead. Seems to be a Pillow sensitivity.

* Update src/transformers/pipelines/image_segmentation.py

Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>

Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
2022-10-20 11:57:11 +02:00
Rohith ND
eb98da9880 [Doctest] OpenAIGPTConfig and OPTConfig (#19763) 2022-10-20 10:22:00 +02:00
Partho
506355ca75 [Doctest] SpeechToTextTransformer2 Config for doctest (#19756) 2022-10-20 10:19:06 +02:00
Partho
123f65eea6 [Doctest] SqueezeBERT Config for doctest (#19758) 2022-10-20 10:16:39 +02:00
Partho
cc03063366 [Doctest] SpeechToTextTransformer Config for doctest (#19757) 2022-10-20 10:15:07 +02:00
Mohammed Faizan Ahmed
bbe2c8b126 All broken links were fixed in contributing file (#19760) 2022-10-19 16:44:03 -04:00
shreya24ag
5602a3ae1e Fixed spacing errors (#19754)
Co-authored-by: Shreya <>
2022-10-19 14:54:30 -04:00
Xabier Lahuerta Vazquez
0a03741590 [Doctest] Add configuration_detr.py (#19752) 2022-10-19 18:13:34 +02:00
Xabier Lahuerta Vazquez
65d36ee861 [Doctest] Add configuration_decision_transformer.py (#19751) 2022-10-19 18:12:34 +02:00
amyeroberts
5041bc3511 Image transforms add center crop (#19718)
* Add center crop to transforms library

* Return PIL images if PIL image input by default

* Fixup and add docstring

* Trigger CI

* Update src/transformers/image_transforms.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Update src/transformers/image_transforms.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* PR comments - move comments; unindent

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-10-19 16:15:01 +01:00
Sylvain Gugger
44a40c1466 Fix cache version file creation (#19750) 2022-10-19 10:55:57 -04:00
Yih-Dar
bed2edb99f Specify TF framework explicitly in more pipeline tests (#19748)
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
2022-10-19 16:24:03 +02:00
JuanCarlosPi
c206fc8779 [Doctest] Add configuration_wavlm.py (#19749)
* Change the import order of the model and configuration classes

* Add (with random weights) in the comment before model initialization

* Add configuration_wavlm to doctest
2022-10-19 16:10:13 +02:00
raghavanone
b17a5e0074 Fix issue #19300 (#19483)
* Fix issue #19300

* Fixing import order

* Fix issue #19300

* Fix formatting issues

* Update src/transformers/trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Refactor method

* Refactor method

* Fix the issue of sending wrong output dir

* Remove unused code

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2022-10-19 09:55:37 -04:00
IMvision12
d2ed8134f1 Update modeling_markuplm.py (#19723) 2022-10-19 13:46:11 +01:00
Rohith ND
7df0751cc6 [Doctest] GPTNeoConfig , GPTNeoXConfig , GPTNeoXJapaneseConfig (#19741) 2022-10-19 14:22:41 +02:00
GMFTBY
71786b10c5 Adding the state-of-the-art contrastive search decoding methods for the codebase of generation_utils.py (#19477)
* add: the contrastive search for generaton_utils

* add: testing scripts for contrastive search under examples/text-generation

* update the quality of codes

* revise the docstring; make the generation_contrastive_search.py scripts;

* revise the examples/pytorch/text-generation/run_generation_contrastive_search.py to the auto-APIs format

* revise the necessary documents

* fix: revise the docstring of generation_contrastive_search.py

* Fix the code indentation

* fix: revise the nits and examples in contrastive_search docstring.

* fix the copyright

* delete generation_contrastive_search.py

* revise the logic in contrastive_search

* update the intergration test and the docstring

* run the tests over

* add the slow decorate to the contrastive_search intergrate test

* add more test

* do the style, quality, consistency checks
2022-10-19 10:17:46 +01:00
Partho
fc5fdc109d [Doctest] Add configuration_clip.py (#19647)
* CLIP Config for doctest

* add doc example to CLIPConfig

* add from_text_vision_configs example

* added comment explaining objective
2022-10-19 09:51:26 +02:00
AymenBer99
c9a0da1e12 [Doctest] XLM Config for doctest (#19685) 2022-10-19 07:10:30 +02:00
JuanCarlosPi
eccbdbcd4d [Doctest] Add wav2vec2_conformer for doctest (#19734) 2022-10-19 06:47:41 +02:00
Steven Liu
32670805fc Update contribution guide (#19700)
* update the contribution guide

* apply review feedback

* fix checkboxes

* checkbox fix #2

* clarify force push
2022-10-18 17:20:12 -07:00
Sylvain Gugger
ebee0a2794 Remove debug statement 2022-10-18 13:58:09 -04:00
Rohith ND
fa8ed9ca76 [Doctest] Add doctest for FlavaConfig and FNetConfig (#19724) 2022-10-18 19:56:49 +02:00
amyeroberts
31ec424b3d Add decorator to flaky test (#19674) 2022-10-18 18:51:37 +01:00
Sylvain Gugger
a929f81e92 Repo utils test (#19696)
* Create repo utils test job

* Last occurence

* Add tests for tests_fetcher

* Better filtering

* Let's learn more

* Should fix

* Should fix

* Remove debug

* Style

* WiP

WiP

WiP

WiP

WiP

WiP

WiP

WiP

WiP

* Quality

* address review comments

* Fix link
2022-10-18 13:47:36 -04:00
David Yang
a23819ed6a Clean up deprecation warnings (#19654)
* Clean up deprecation warnings

Notes:
Changed some strings in tests to raw strings, which will change the literal content of the strings as they are fed into whatever machine handles them.
Test cases for past in the past/past_key_values switch changed/removed due to warning of impending removal

* Add PILImageResampling abstraction for PIL.Image.Resampling
2022-10-18 13:34:47 -04:00
Younes Belkada
af556a09f6 add accelerate support for Whisper (#19697) 2022-10-18 18:25:49 +02:00
Sylvain Gugger
fb0bd7b7a8 Fix activations being all the same module (#19728) 2022-10-18 11:56:45 -04:00
NielsRogge
14fe3e0410 Add docs (#19729)
Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local>
2022-10-18 17:42:46 +02:00