SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

Simplify soft dependencies and update the dummy-creation process (#36827)

Browse Source 
* Reverse dependency map shouldn't be created when test_all is set

* [test_all] Remove dummies

* Modular fixes

* Update utils/check_repo.py

Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>

* [test_all] Better docs

* [test_all] Update src/transformers/commands/chat.py

Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>

* [test_all] Remove deprecated AdaptiveEmbeddings from the tests

* [test_all] Doc builder

* [test_all] is_dummy

* [test_all] Import utils

* [test_all] Doc building should not require all deps

---------

Co-authored-by: Pablo Montalvo <39954772+molbap@users.noreply.github.com>
Co-authored-by: Joao Gante <joaofranciscocardosogante@gmail.com>
This commit is contained in:
Lysandre Debut
2025-04-11 11:08:36 +02:00
committed by GitHub
parent 931126b929
commit 54a123f068
295 changed files with 2167 additions and 27702 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
utils/tests_fetcher.py
View File

@@ -741,10 +741,13 @@ def get_module_dependencies(module_fname: str, cache: Dict[str, List[str]] = Non
# Add imports via `define_import_structure` after the #35167 as we remove explicit import in `__init__.py`
from transformers.utils.import_utils import define_import_structure
new_imported_modules_2 = define_import_structure(PATH_TO_REPO / module)
new_imported_modules_from_import_structure = define_import_structure(PATH_TO_REPO / module)
for mapping in new_imported_modules_2.values():
for mapping in new_imported_modules_from_import_structure.values():
for _module, _imports in mapping.items():
# Import Structure returns _module keys as import paths rather than local paths
# We replace with os.path.sep so that it's Windows-compatible
_module = _module.replace(".", os.path.sep)
_module = module.replace("__init__.py", f"{_module}.py")
new_imported_modules.append((_module, list(_imports)))
@@ -1038,18 +1041,18 @@ def infer_tests_to_run(
"""
if not test_all:
modified_files = get_modified_python_files(diff_with_last_commit=diff_with_last_commit)
reverse_map = create_reverse_dependency_map()
impacted_files = modified_files.copy()
for f in modified_files:
if f in reverse_map:
impacted_files.extend(reverse_map[f])
else:
modified_files = [str(k) for k in PATH_TO_TESTS.glob("*/*") if str(k).endswith(".py") and "test_" in str(k)]
impacted_files = modified_files = [
str(k) for k in PATH_TO_TESTS.glob("*/*") if str(k).endswith(".py") and "test_" in str(k)
]
print("\n### test_all is TRUE, FETCHING ALL FILES###\n")
print(f"\n### MODIFIED FILES ###\n{_print_list(modified_files)}")
# Create the map that will give us all impacted modules.
reverse_map = create_reverse_dependency_map()
impacted_files = modified_files.copy()
for f in modified_files:
if f in reverse_map:
impacted_files.extend(reverse_map[f])
# Remove duplicates
impacted_files = sorted(set(impacted_files))
print(f"\n### IMPACTED FILES ###\n{_print_list(impacted_files)}")