Add model like (#14992)

* Add new model like command * Bad doc-styler * black and doc-styler, stop fighting! * black and doc-styler, stop fighting! * At last * Clean up * Typo * Bad doc-styler * Bad doc-styler * All good maybe? * Use constants * Add doc and type hints * More cleaning * Add doc * Fix Copied from * Doc template * Use typing.Pattern instead * Framework-specific files * Fixes * Select frameworks clean model init * Deal with frameworks in main init * fixes * Last fix * Prompt user for info * Delete exemple config * Last fixes * Add test config * Fix bug with model_type included in each other * Fixes * More fixes * More fixes * Adapt config * Remove print statements * Will fix tokenization later, leave it broken for now * Add test * Quality * Try this way * Debug * Maybe by setting the path? * Let's try another way * It should go better when actually passing the arg... * Remove debug statements and style * Fix config * Add tests * Test require the three backends * intermediate commit * Revamp pattern replacements and start work on feature extractors * Adapt model info * Finalize code for processors * Fix in main init additions * Finish questionnaire for processing classes * Fix file name * Fix for real * Fix patterns * Style * Remove needless warnings * Copied from should work now. * Include Copied form in blocks * Add test * More fixes and tests * Apply suggestions from code review Co-authored-by: Lysandre Debut <lysandre@huggingface.co> * Address review comment Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
2022-01-24 15:25:10 -05:00
parent 457dd4392b
commit 81156d20cd
6 changed files with 2902 additions and 0 deletions
--- a/.github/workflows/add-model-like.yml
+++ b/.github/workflows/add-model-like.yml
@@ -0,0 +1,60 @@
+name: Add model like runner
+
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+    paths:
+      - "src/**"
+      - "tests/**"
+      - ".github/**"
+    types: [opened, synchronize, reopened]
+
+jobs:
+  run_tests_templates:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+
+      - name: Loading cache.
+        uses: actions/cache@v2
+        id: cache
+        with:
+          path: ~/.cache/pip
+          key: v1-tests_model_like
+          restore-keys: |
+            v1-tests_model_like-${{ hashFiles('setup.py') }}
+            v1-tests_model_like
+
+      - name: Install dependencies
+        run: |
+          pip install --upgrade pip!=21.3
+          sudo apt -y update && sudo apt install -y libsndfile1-dev
+          pip install .[dev]
+
+      - name: Create model files
+        run: |
+          transformers-cli add-new-model-like --config_file tests/fixtures/add_distilbert_like_config.json --path_to_repo .
+          make style
+          make fix-copies
+
+      - name: Run all PyTorch modeling test
+        run: |
+          python -m pytest -n 2 --dist=loadfile -s --make-reports=tests_new_models tests/test_modeling_bert_new.py
+
+      - name: Run style changes
+        run: |
+          git fetch origin master:master
+          make style && make quality && make repo-consistency
+
+      - name: Failure short reports
+        if: ${{ always() }}
+        run: cat reports/tests_new_models_failures_short.txt
+
+      - name: Test suite reports artifacts
+        if: ${{ always() }}
+        uses: actions/upload-artifact@v2
+        with:
+          name: run_all_tests_new_models_test_reports
+          path: reports