Anahita Bhiwandiwalla
3a6e4a221c
Add BridgeTower model (#20775)
* Commit with BTModel and latest HF code
* Placeholder classes for BTForMLM and BTForITR
* Importing Bert classes from transformers
* Removed objectives.py and dist_utils.py
* Removed swin_transformer.py
* Add image normalization, BridgeTowerForImageAndTextRetrieval
* Add center_crop
* Removing bert tokenizer and LCI references
* Tested config loading from HF transformers hub
* Removed state_dict updates and added path to hub
* Enable center crop
* Getting image_size from config, renaming num_heads and num_layers
* Handling max_length in BridgeTowerProcessor
* Add BridgeTowerForMaskedLM
* Add doc string for BridgeTowerConfig
* Add doc strings for BT config, processor, image processor
* Adding docs, removed swin
* Removed convert_bridgetower_original_to_pytorch.py
* Added doc files for bridgetower, removed is_vision
* Add support attention_mask=None and BridgeTowerModelOutput
* Fix formatting
* Fixes with 'make style', 'make quality', 'make fixup'
* Remove downstream tasks from BridgeTowerModel
* Formatting fixes, add return_dict to BT models
* Clean up after doc_test
* Update BTModelOutput return type, fix todo in doc
* Remove loss_names from init
* implement tests and update tuples returned by models
* Add image reference to bridgetower.mdx
* after make fix-copies, make fixup, make style, make quality, make repo-consistency
* Rename class names with BridgeTower prefix
* Fix for image_size in BTImageProcessor
* implement feature extraction bridgetower tests
* Update image_mean and image_std to be list
* remove unused import
* Removed old comments
* Rework CLIP
* update config in tests followed config update
* Formatting fixes
* Add copied from for BridgeTowerPredictionHeadTransform
* Update bridgetower.mdx
* Update test_feature_extraction_bridgetower.py
* Update bridgetower.mdx
* BridgeTowerForMaskedLM is conditioned on image too
* Add BridgeTowerForMaskedLM
* Fixes
* Call post_init to init weights
* Move freeze layers into method
* Remove BTFeatureExtractor, add BT under multimodal models
* Remove BTFeatureExtractor, add BT under multimodal models
* Code review feedback - cleanup
* Rename variables
* Formatting and style to PR review feedback
* Move center crop after resize
* Use named parameters
* Style fix for modeling_bridgetower.py
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
* Update src/transformers/models/bridgetower/modeling_bridgetower.py
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Rename config params, copy BERT classes, clean comments
* Cleanup irtr
* Replace Roberta imports, add BTTextConfig and Model
* Update docs, add visionconfig, consistent arg names
* make fixup
* Comments for forward in BTModel and make fixup
* correct tests
* Remove inconsistent roberta copied from
* Add BridgeTowerTextModel to dummy_pt_objects.py
* Add BridgeTowerTextModel to IGNORE_NON_TESTED
* Update docs for BT Text and Vision Configs
* Treat BridgeTowerTextModel as a private model
* BridgeTowerTextModel as private
* Run make fix-copies
* Adding BTTextModel to PRIVATE_MODELS
* Fix for issue with BT Text and Image configs
* make style changes
* Update README_ja.md
Add から to BridgeTower's description
* Clean up config, .mdx and arg names
* Fix init_weights. Remove nn.Sequential
* Formatting and style fixes
* Re-add tie_word_embeddings in config
* update test implementation
* update style
* remove commented out
* fix style
* Update README with abs for BridgeTower
* fix style
* fix mdx file
* Update bridgetower.mdx
* Update img src in bridgetower.mdx
* Update README.md
* Update README.md
* resolve style failed
* Update _toctree.yml
* Update README_ja.md
* Removed mlp_ratio, rename feats, rename BTCLIPModel
* Replace BTCLIP with BTVisionModel,pass in vision_config to BTVisionModel
* Add test_initialization support
* Add support for output_hidden_states
* Update support for output_hidden_states
* Add support for output_attentions
* Add docstring for output_hidden_states
* update tests
* add bridgetowervisionmodel as private model
* rerun the PR test
* Remove model_type, pass configs to classes, renames
* Change self.device to use weight device
* Remove image_size
* Style check fixes
* Add hidden_size and num_hidden_layers to BridgeTowerTransformer
* Update device setting
* cosmetic update
* trigger test again
* trigger tests again
* Update test_modeling_bridgetower.py
trigger tests again
* Update test_modeling_bridgetower.py
* minor update
* re-trigger tests
* Update docs/source/en/model_doc/bridgetower.mdx
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
* Remove pad, update max_text_len, doc cleanup, pass eps to LayerNorm
* Added copied to, some more review feedback
* make fixup
* Use BridgeTowerVisionEmbeddings
* Code cleanup
* Fixes for BridgeTowerVisionEmbeddings
* style checks
* re-tests
* fix embedding
* address comment on init file
* retrigger tests
* update import prepare_image_inputs
* update test_image_processing_bridgetower.py to reflect test_image_processing_common.py
* retrigger tests
Co-authored-by: Shaoyen Tseng <shao-yen.tseng@intel.com>
Co-authored-by: Tiep Le <tiep.le@intel.com>
Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
Co-authored-by: Tiep Le <97980157+tileintel@users.noreply.github.com>
2023-01-25 14:04:32 -05:00
..
2022-04-04 10:25:46 -04:00
2023-01-04 09:18:57 +01:00
2023-01-17 17:42:33 +01:00
2022-12-16 14:22:46 +01:00
2022-12-07 18:05:39 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-29 17:42:15 -04:00
2022-04-04 10:25:46 -04:00
2022-11-01 11:09:53 -07:00
2022-04-04 10:25:46 -04:00
2022-11-04 11:32:44 -04:00
2022-11-04 11:32:44 -04:00
2022-12-05 10:12:03 -05:00
2023-01-17 17:18:56 +01:00
2022-11-04 11:32:44 -04:00
2022-11-04 11:32:44 -04:00
2023-01-18 11:24:37 +01:00
2022-11-07 09:19:04 -05:00
2022-04-04 10:25:46 -04:00
2023-01-25 14:04:32 -05:00
2022-04-13 11:36:54 +02:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-11-30 19:22:23 +01:00
2023-01-17 17:18:56 +01:00
2022-11-09 18:31:22 +01:00
2022-06-24 17:10:38 +02:00
2022-11-30 10:24:03 +00:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-05-02 12:47:39 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-05-10 16:21:44 -04:00
2022-11-15 13:26:07 -05:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-10-28 13:16:07 -07:00
2023-01-17 17:18:56 +01:00
2022-11-29 10:38:01 +00:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-01-20 11:35:42 +03:00
2022-05-03 11:26:19 +02:00
2022-07-27 10:08:59 +02:00
2022-09-09 07:36:46 -04:00
2022-10-31 21:32:58 -04:00
2022-12-27 02:26:14 -05:00
2022-04-04 10:25:46 -04:00
2022-11-08 19:54:41 +00:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:42:33 +01:00
2023-01-17 17:18:56 +01:00
2022-11-29 10:02:40 -05:00
2022-04-04 10:25:46 -04:00
2022-09-14 10:17:40 -04:00
2022-05-24 09:31:10 -04:00
2022-12-12 13:12:13 -05:00
2022-11-09 15:34:08 +00:00
2023-01-19 13:05:59 -05:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-11-10 21:05:27 +01:00
2022-09-02 14:48:19 +02:00
2022-11-08 19:54:41 +00:00
2022-11-08 19:54:41 +00:00
2022-04-04 10:25:46 -04:00
2022-11-04 11:32:44 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-11-07 09:19:04 -05:00
2022-08-16 10:20:46 -05:00
2022-08-01 11:09:47 -04:00
2022-04-04 10:25:46 -04:00
2022-09-29 13:27:58 +02:00
2022-07-29 08:09:09 -04:00
2022-10-24 17:54:23 +02:00
2023-01-25 12:34:43 +01:00
2023-01-16 20:37:07 +03:00
2022-07-29 08:09:09 -04:00
2022-12-27 18:37:25 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-11-07 09:19:04 -05:00
2022-06-29 09:30:55 -04:00
2023-01-17 17:18:56 +01:00
2022-06-23 12:36:22 -04:00
2022-07-29 08:09:09 -04:00
2022-04-04 10:25:46 -04:00
2023-01-25 12:34:43 +01:00
2022-12-12 09:43:43 -05:00
2022-10-10 09:30:59 -04:00
2023-01-03 19:25:09 +03:00
2022-09-02 19:54:02 +02:00
2022-04-04 10:25:46 -04:00
2022-11-08 19:54:41 +00:00
2022-04-04 10:25:46 -04:00
2022-07-29 08:09:09 -04:00
2023-01-17 17:18:56 +01:00
2022-11-04 11:32:44 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-12-19 09:30:17 +01:00
2022-10-27 11:33:15 -07:00
2022-11-08 10:03:43 -05:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-11-09 15:34:08 +00:00
2022-11-03 14:18:45 +01:00
2022-07-29 08:09:09 -04:00
2022-11-07 09:19:04 -05:00
2022-04-04 10:25:46 -04:00
2022-12-16 16:24:01 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-11-15 13:06:45 +01:00
2022-11-09 15:34:08 +00:00
2022-11-07 09:19:04 -05:00
2022-11-30 14:50:55 +00:00
2022-11-14 01:04:26 -05:00
2022-04-08 10:57:51 +02:00
2022-09-30 15:32:59 -04:00
2022-12-02 09:13:25 +01:00
2022-05-17 19:07:43 -04:00
2022-04-04 10:25:46 -04:00
2022-11-30 14:50:55 +00:00
2022-06-21 10:24:50 +02:00
2022-05-17 00:43:16 +02:00
2022-05-17 00:43:16 +02:00
2023-01-17 17:42:33 +01:00
2023-01-17 17:18:56 +01:00
2022-11-30 14:50:55 +00:00
2022-11-08 19:54:41 +00:00
2022-11-30 14:50:55 +00:00
2022-11-07 09:19:04 -05:00
2022-11-30 14:50:55 +00:00
2022-12-07 11:03:39 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00
2022-06-15 18:34:15 +02:00
2022-10-28 13:28:18 -07:00
2022-05-17 00:43:16 +02:00
2022-11-09 15:34:08 +00:00
2023-01-17 17:18:56 +01:00
2022-08-24 10:51:05 +01:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-18 07:49:39 -05:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2022-04-04 10:25:46 -04:00
2023-01-17 17:18:56 +01:00
2022-04-04 10:25:46 -04:00