SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity
2,942 Commits 5 Branches 239 Tags
1b59b57b57010e6119282f3dbf37f8c7c6d6313e
Commit Graph

74 Commits

Author SHA1 Message Date
alberduris
81d6841b4b GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
alberduris
dd4df80f0b Moved the encoded_prompts to correct device 2020-01-06 15:11:12 +01:00
Aymeric Augustin
0ffc8eaf53 Enforce target version for black. 
This should stabilize formatting.
 2020-01-05 12:52:14 -05:00
Aymeric Augustin
10724a8123 Run the slow tests every Monday morning. 2019-12-24 09:09:43 +01:00
Aymeric Augustin
8a6881822a Run some tests on Python 3.7. 
This will improve version coverage.
 2019-12-23 21:06:23 +01:00
Aymeric Augustin
76a1417f2a Include all optional dependencies in extras. 
Take advantage of this to simplify the Circle CI configuration.

Don't bother with tensorboardX: it's a fallback for PyTorch < 1.1.0.
 2019-12-23 19:14:31 +01:00
Aymeric Augustin
23dad8447c Install deps from setup.py for building docs. 
requirements.txt isn't up to date.
 2019-12-23 17:06:32 +01:00
Thomas Wolf
ba2378ced5 Merge pull request #2264 from upura/fix-doclink 
Fix doc link in README
 2019-12-23 12:31:00 +01:00
Aymeric Augustin
0dddc1494d Remove py3 marker. 2019-12-22 18:38:56 +01:00
Aymeric Augustin
6be7cdda66 Move source code inside a src subdirectory. 
This prevents transformers from being importable simply because the CWD
is the root of the git repository, while not being importable from other
directories. That led to inconsistent behavior, especially in examples.

Once you fetch this commit, in your dev environment, you must run:

    $ pip uninstall transformers
    $ pip install -e .
 2019-12-22 14:15:13 +01:00
Aymeric Augustin
ced0a94204 Switch test files to the standard test_*.py scheme. 2019-12-22 14:15:13 +01:00
Aymeric Augustin
067395d5c5 Move tests outside of library. 2019-12-22 13:47:17 +01:00
Aymeric Augustin
c11b3e2926 Sort imports for optional third-party libraries. 
These libraries aren't always installed in the virtual environment where
isort is running. Declaring them properly avoids mixing these
third-party imports with local imports.
 2019-12-22 11:19:13 +01:00
Aymeric Augustin
577a03664d Enforce flake8 in CI. 2019-12-22 11:00:04 +01:00
Aymeric Augustin
9e80fc7b2f Enforce isort in CI. 
We need https://github.com/timothycrosley/isort/pull/1000 but there's no
release with this fix yet, so we'll install from GitHub.
 2019-12-22 10:59:00 +01:00
upura
9d00f78f16 fix doc link 2019-12-22 16:07:05 +09:00
Aymeric Augustin
6e5291a915 Enforce black in CI. 2019-12-21 17:53:18 +01:00
Aymeric Augustin
343c094f21 Run examples separately from tests. 
This optimizes the total run time of the Circle CI test suite.
 2019-12-21 08:43:19 +01:00
Aymeric Augustin
80caf79d07 Prevent excessive parallelism in PyTorch. 
We're already using as many processes in parallel as we have CPU cores.
Furthermore, the number of core may be incorrectly calculated as 36
(we've seen this in pytest-xdist) which make compound the problem.

PyTorch performance craters without this.
 2019-12-21 08:43:19 +01:00
Aymeric Augustin
bb3bfa2d29 Distribute tests from the same file to the same worker. 
This should prevent two issues:

- hitting API rate limits for tests that hit the HF API
- multiplying the cost of expensive test setups
 2019-12-21 08:43:19 +01:00
Aymeric Augustin
29cbab98f0 Parallelize tests on Circle CI. 
Set the number of CPUs manually based on the Circle CI resource class,
or else we're getting 36 CPUs, which is far too much (perhaps that's
the underlying hardware and not what Circle CI allocates to us).

Don't parallelize the custom tokenizers tests because they take less
than one second to run and parallelization actually makes them slower.
 2019-12-21 08:43:19 +01:00
thomwolf
15dda5ea32 remove python 2 tests for circle-ci cc @aaugustin @julien-c @LysandreJik 2019-12-20 13:20:41 +01:00
thomwolf
e4baa68ddb tick-tock cc @julien-c 2019-12-19 20:37:26 +01:00
Thomas Wolf
d5712f7cac Merge branch 'master' into check-link-validity 2019-12-12 08:00:51 +01:00
Julien Chaumond
371c5ddfad Py2 tests for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond
5505cf7014 Run tests on Py2 too, for Lysandre 2019-12-11 18:32:27 -05:00
Julien Chaumond
9cb97c0c0f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond
95854c4a2f Actually run the tests 2019-12-11 18:32:27 -05:00
Julien Chaumond
d2100428d3 Update to new test infra and only run conditionally 2019-12-11 18:32:27 -05:00
Masatoshi Suzuki
597ba7feb3 Support testing Japanese BERT tokenizers 2019-12-11 18:32:27 -05:00
Rémi Louf
f230d91b43 check the validity of links 
We add a script and a CI workflow to check that all download links
present in the source code are valid.
 2019-12-06 09:41:28 +01:00
Lysandre
45d767297a Updated v2.2.0 doc 2019-11-27 10:12:20 -05:00
Lysandre
cc7968227e Updated v2.2.0 doc 2019-11-26 15:52:25 -05:00
Lysandre
44b82c777f Updated v2.2.0 doc 2019-11-26 15:15:11 -05:00
Lysandre
7c6000e412 Updated v2.2.0 doc 2019-11-26 14:55:29 -05:00
Lysandre
ae98d45991 Release: v2.2.0 2019-11-26 14:12:44 -05:00
Lysandre
be7f2aacce [CI][DOC] Don't rebuild if folder exists - Correct directory. 2019-11-14 14:54:44 -05:00
Lysandre
8f8d69716a [CI][DOC] Don't rebuild if folder exists. 2019-11-14 14:48:21 -05:00
Lysandre Debut
6e85bccafc Fixed typo 2019-10-22 18:07:01 -04:00
Lysandre
fbcc5ff9fb Change branch to master 2019-10-22 18:01:10 -04:00
Lysandre
69eba0ab19 Edit script path 2019-10-22 17:53:52 -04:00
Lysandre
bc3e57d551 Multi version doc deployment 2019-10-22 17:51:30 -04:00
thomwolf
2a4fef837a move Circle-CI from TF2-rc0 to official TF2 2019-10-10 15:57:35 +02:00
Julien Chaumond
22d2fded2c [docs] Fix doc auto-deploy 
Co-Authored-By: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2019-09-26 18:22:45 -04:00
thomwolf
31c23bd5ee [BIG] pytorch-transformers => transformers 2019-09-26 10:15:53 +02:00
thomwolf
0f091062d4 Merge branch 'glue-example' into tf2 2019-09-25 10:21:52 +02:00
thomwolf
2167e366ba update circleCi 2019-09-24 13:27:45 +02:00
thomwolf
e9a103c17a bidirectional conversion TF <=> PT - extended tests 2019-09-24 13:25:50 +02:00
LysandreJik
11ac4b9555 [CI] Symbolic link for documentation 2019-09-11 10:13:44 +02:00
thomwolf
e30579f764 no pytest version checking 2019-09-08 15:02:06 +03:00