From 3552d0e0d89711404ec49b0080cd1f48ae224bb3 Mon Sep 17 00:00:00 2001 From: Julien Chaumond Date: Sat, 12 Dec 2020 00:24:42 +0100 Subject: [PATCH] [model_cards] Migrate cards from this repo to model repos on huggingface.co (#9013) * rm all model cards * Update the .rst @sgugger it is still not super crystal clear/streamlined so let me know if any ideas to make it simpler * Add a rootlevel README.md with simple instructions/context * Update docs/source/model_sharing.rst Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen * make style * rm all model cards Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen --- docs/source/model_sharing.rst | 45 +- .../README.md | 20 - .../README.md | 18 - .../README.md | 142 ----- .../bert-base-bg-cs-pl-ru-cased/README.md | 14 - .../bert-base-cased-conversational/README.md | 16 - .../README.md | 15 - .../README.md | 13 - .../rubert-base-cased-sentence/README.md | 15 - .../DeepPavlov/rubert-base-cased/README.md | 11 - .../Geotrend/bert-base-15lang-cased/README.md | 61 -- .../Geotrend/bert-base-ar-cased/README.md | 47 -- .../Geotrend/bert-base-bg-cased/README.md | 42 -- .../Geotrend/bert-base-de-cased/README.md | 42 -- .../Geotrend/bert-base-el-cased/README.md | 42 -- .../Geotrend/bert-base-en-ar-cased/README.md | 49 -- .../Geotrend/bert-base-en-bg-cased/README.md | 47 -- .../Geotrend/bert-base-en-cased/README.md | 47 -- .../Geotrend/bert-base-en-de-cased/README.md | 47 -- .../Geotrend/bert-base-en-el-cased/README.md | 47 -- .../Geotrend/bert-base-en-es-cased/README.md | 47 -- .../Geotrend/bert-base-en-fr-cased/README.md | 50 -- .../Geotrend/bert-base-en-hi-cased/README.md | 47 -- .../Geotrend/bert-base-en-ru-cased/README.md | 47 -- .../Geotrend/bert-base-en-sw-cased/README.md | 47 -- .../Geotrend/bert-base-en-th-cased/README.md | 47 -- .../Geotrend/bert-base-en-tr-cased/README.md | 47 -- .../Geotrend/bert-base-en-ur-cased/README.md | 47 -- .../Geotrend/bert-base-en-vi-cased/README.md | 47 -- .../Geotrend/bert-base-en-zh-cased/README.md | 47 -- .../Geotrend/bert-base-es-cased/README.md | 42 -- .../Geotrend/bert-base-fr-cased/README.md | 47 -- .../Geotrend/bert-base-hi-cased/README.md | 42 -- .../Geotrend/bert-base-ru-cased/README.md | 42 -- .../Geotrend/bert-base-sw-cased/README.md | 42 -- .../Geotrend/bert-base-th-cased/README.md | 42 -- .../Geotrend/bert-base-tr-cased/README.md | 42 -- .../Geotrend/bert-base-ur-cased/README.md | 42 -- .../Geotrend/bert-base-vi-cased/README.md | 42 -- .../Geotrend/bert-base-zh-cased/README.md | 42 -- .../dehatebert-mono-arabic/README.md | 18 - .../dehatebert-mono-english/README.md | 20 - .../dehatebert-mono-french/README.md | 20 - .../dehatebert-mono-german/README.md | 20 - .../dehatebert-mono-indonesian/README.md | 20 - .../dehatebert-mono-italian/README.md | 20 - .../dehatebert-mono-polish/README.md | 20 - .../dehatebert-mono-portugese/README.md | 20 - .../dehatebert-mono-spanish/README.md | 20 - .../README.md | 124 ---- .../bert-base-parsbert-ner-uncased/README.md | 124 ---- .../README.md | 124 ---- .../bert-base-parsbert-uncased/README.md | 124 ---- .../bert-fa-base-uncased/README.md | 147 ----- .../albert-base-swedish-cased-alpha/README.md | 121 ---- .../KB/bert-base-swedish-cased-ner/README.md | 121 ---- .../KB/bert-base-swedish-cased/README.md | 121 ---- .../LorenzoDeMattei/GePpeTto/README.md | 141 ---- .../t5-base-en-generate-headline/README.md | 47 -- .../MoseliMotsoehli/TswanaBert/README.md | 74 --- model_cards/MoseliMotsoehli/zuBERTa/README.md | 56 -- .../umberto-commoncrawl-cased-v1/README.md | 118 ---- .../umberto-wikipedia-uncased-v1/README.md | 117 ---- model_cards/NLP4H/ms_bert/README.md | 54 -- model_cards/Naveen-k/KanBERTo/README.md | 28 - .../NeuML/bert-small-cord19-squad2/README.md | 26 - model_cards/NeuML/bert-small-cord19/README.md | 25 - .../NeuML/bert-small-cord19qa/README.md | 63 -- .../NlpHUST/vibert4news-base-cased/README.md | 55 -- .../hewiki-articles-distilGPT2py-il/README.md | 109 ---- model_cards/Ogayo/Hel-ach-en/README.md | 48 -- model_cards/Primer/bart-squad2/README.md | 63 -- model_cards/README.md | 26 + model_cards/Rostlab/prot_bert/README.md | 141 ---- model_cards/Rostlab/prot_bert_bfd/README.md | 141 ---- model_cards/Rostlab/prot_t5_xl_bfd/README.md | 125 ---- .../SZTAKI-HLT/hubert-base-cc/README.md | 46 -- .../SparkBeyond/roberta-large-sts-b/README.md | 50 -- .../README.md | 9 - .../README.md | 85 --- .../README.md | 82 --- .../gpt2-124M-uk-fiction/README.md | 39 -- .../bert-base-finnish-cased-v1/README.md | 84 --- .../bert-base-finnish-uncased-v1/README.md | 84 --- model_cards/TypicaAI/magbert-ner/README.md | 59 -- .../Vamsi/T5_Paraphrase_Paws/README.md | 51 -- .../README.md | 25 - .../README.md | 25 - model_cards/a-ware/bart-squadv2/README.md | 76 --- .../README.md | 48 -- .../a-ware/xlmroberta-squadv2/README.md | 59 -- .../abhilash1910/financial_roberta/README.md | 132 ---- .../abhilash1910/french-roberta/README.md | 131 ---- .../activebus/BERT-DK_laptop/README.md | 43 -- model_cards/activebus/BERT-DK_rest/README.md | 41 -- .../activebus/BERT-PT_laptop/README.md | 41 -- model_cards/activebus/BERT-PT_rest/README.md | 42 -- .../activebus/BERT-XD_Review/README.md | 44 -- model_cards/activebus/BERT_Review/README.md | 44 -- .../adalbertojunior/PTT5-SMALL-SUM/README.md | 37 -- .../albert_xxlargev1_squad2_512/README.md | 60 -- .../README.md | 58 -- .../ahotrod/roberta_large_squad2/README.md | 56 -- .../ahotrod/xlnet_large_squad2_512/README.md | 63 -- model_cards/ai4bharat/indic-bert/README.md | 118 ---- .../gpt2-small-arabic-poetry/README.md | 45 -- .../akhooli/gpt2-small-arabic/README.md | 44 -- .../akhooli/mbart-large-cc25-ar-en/README.md | 15 - .../akhooli/mbart-large-cc25-en-ar/README.md | 14 - .../akhooli/personachat-arabic/README.md | 12 - .../akhooli/xlm-r-large-arabic-sent/README.md | 13 - .../xlm-r-large-arabic-toxic/README.md | 12 - model_cards/albert-base-v1-README.md | 10 - model_cards/albert-xxlarge-v2-README.md | 10 - model_cards/aliosm/ComVE-distilgpt2/README.md | 67 -- model_cards/aliosm/ComVE-gpt2-large/README.md | 68 -- .../aliosm/ComVE-gpt2-medium/README.md | 82 --- model_cards/aliosm/ComVE-gpt2/README.md | 67 -- .../ai-soco-cpp-roberta-small-clas/README.md | 56 -- .../ai-soco-cpp-roberta-small/README.md | 55 -- .../README.md | 56 -- .../ai-soco-cpp-roberta-tiny-96/README.md | 55 -- .../ai-soco-cpp-roberta-tiny-clas/README.md | 56 -- .../aliosm/ai-soco-cpp-roberta-tiny/README.md | 55 -- .../allegro/herbert-base-cased/README.md | 51 -- .../herbert-klej-cased-tokenizer-v1/README.md | 44 -- .../allegro/herbert-klej-cased-v1/README.md | 85 --- .../allegro/herbert-large-cased/README.md | 50 -- .../allenai/biomed_roberta_base/README.md | 38 -- .../README.md | 20 - .../allenai/longformer-base-4096/README.md | 24 - .../allenai/scibert_scivocab_cased/README.md | 26 - .../scibert_scivocab_uncased/README.md | 26 - .../allenai/wmt16-en-de-12-1/README.md | 103 --- .../allenai/wmt16-en-de-dist-12-1/README.md | 103 --- .../allenai/wmt16-en-de-dist-6-1/README.md | 103 --- .../allenai/wmt19-de-en-6-6-base/README.md | 101 --- .../allenai/wmt19-de-en-6-6-big/README.md | 101 --- .../chinese-bert-wwm-ehr-ner-sl/README.md | 15 - .../README.md | 103 --- .../amine/bert-base-5lang-cased/README.md | 64 -- model_cards/antoiloui/belgpt2/README.md | 53 -- .../README.md | 20 - .../README.md | 26 - .../README.md | 20 - .../README.md | 20 - .../asafaya/bert-base-arabic/README.md | 74 --- .../asafaya/bert-large-arabic/README.md | 74 --- .../asafaya/bert-medium-arabic/README.md | 73 --- .../asafaya/bert-mini-arabic/README.md | 72 --- .../Gujarati-XLM-R-Base/README.md | 45 -- .../aubmindlab/bert-base-arabert/README.md | 144 ----- .../aubmindlab/bert-base-arabertv01/README.md | 144 ----- model_cards/bart-large-cnn/README.md | 5 - model_cards/bart-large-xsum/README.md | 5 - .../multi-dialect-bert-base-arabic/README.md | 69 -- .../bayartsogt/albert-mongolian/README.md | 59 -- .../bert-base-mongolian-cased/README.md | 60 -- .../bert-base-mongolian-uncased/README.md | 54 -- model_cards/bert-base-cased-README.md | 230 ------- model_cards/bert-base-chinese-README.md | 3 - model_cards/bert-base-german-cased-README.md | 82 --- .../bert-base-german-dbmdz-cased-README.md | 4 - .../bert-base-german-dbmdz-uncased-README.md | 4 - .../bert-base-multilingual-cased-README.md | 152 ----- .../bert-base-multilingual-uncased-README.md | 209 ------ model_cards/bert-base-uncased-README.md | 231 ------- model_cards/bert-large-cased-README.md | 3 - .../binwang/xlnet-base-cased/README.md | 5 - .../README.md | 80 --- .../README.md | 80 --- .../README.md | 60 -- .../README.md | 79 --- .../blinoff/roberta-base-russian-v0/README.md | 61 -- .../cahya/bert-base-indonesian-522M/README.md | 73 --- .../gpt2-small-indonesian-522M/README.md | 64 -- .../roberta-base-indonesian-522M/README.md | 58 -- .../BioRedditBERT-uncased/README.md | 52 -- model_cards/camembert-base-README.md | 115 ---- .../camembert-base-ccnet-4gb/README.md | 111 ---- .../camembert/camembert-base-ccnet/README.md | 110 ---- .../camembert-base-oscar-4gb/README.md | 111 ---- .../camembert-base-wikipedia-4gb/README.md | 110 ---- .../camembert/camembert-large/README.md | 110 ---- .../canwenxu/BERT-of-Theseus-MNLI/README.md | 24 - model_cards/cedpsam/chatbot_fr/README.md | 47 -- .../harry-potter-gpt2-fanfiction/README.md | 11 - .../chrisliu298/arxiv_ai_gpt2/README.md | 65 -- model_cards/cimm-kzn/endr-bert/README.md | 47 -- model_cards/cimm-kzn/enrudr-bert/README.md | 46 -- model_cards/cimm-kzn/rudr-bert/README.md | 41 -- .../clue/albert_chinese_small/README.md | 35 - .../clue/albert_chinese_tiny/README.md | 35 - .../roberta_chinese_3L312_clue_tiny/README.md | 43 -- .../clue/roberta_chinese_base/README.md | 35 - .../clue/roberta_chinese_large/README.md | 35 - .../clue/xlnet_chinese_large/README.md | 33 - .../codegram/calbert-base-uncased/README.md | 90 --- .../codegram/calbert-tiny-uncased/README.md | 90 --- model_cards/cooelf/limitbert/README.md | 59 -- .../bert-base-uncased-squad-v1/README.md | 113 ---- .../mobilebert-uncased-squad-v1/README.md | 116 ---- .../mobilebert-uncased-squad-v2/README.md | 118 ---- .../csarron/roberta-base-squad-v1/README.md | 112 ---- .../bert-base-japanese-sentiment/README.md | 13 - .../dbmdz/bert-base-german-cased/README.md | 71 --- .../README.md | 62 -- .../README.md | 62 -- .../dbmdz/bert-base-german-uncased/README.md | 71 --- .../dbmdz/bert-base-italian-cased/README.md | 110 ---- .../dbmdz/bert-base-italian-uncased/README.md | 110 ---- .../bert-base-italian-xxl-cased/README.md | 110 ---- .../bert-base-italian-xxl-uncased/README.md | 110 ---- .../bert-base-turkish-128k-cased/README.md | 77 --- .../bert-base-turkish-128k-uncased/README.md | 77 --- .../dbmdz/bert-base-turkish-cased/README.md | 75 --- .../dbmdz/bert-base-turkish-uncased/README.md | 75 --- .../distilbert-base-turkish-cased/README.md | 77 --- .../README.md | 110 ---- .../README.md | 110 ---- .../README.md | 79 --- .../README.md | 79 --- .../bert-base-spanish-wwm-cased/README.md | 3 - .../bert-base-spanish-wwm-uncased/README.md | 3 - .../bert-base-german-cased-oldvocab/README.md | 29 - .../deepset/electra-base-squad2/README.md | 115 ---- model_cards/deepset/gbert-base/README.md | 51 -- model_cards/deepset/gbert-large/README.md | 54 -- .../deepset/gelectra-base-generator/README.md | 46 -- model_cards/deepset/gelectra-base/README.md | 51 -- .../gelectra-large-generator/README.md | 56 -- model_cards/deepset/gelectra-large/README.md | 52 -- .../deepset/minilm-uncased-squad2/README.md | 116 ---- .../deepset/quora_dedup_bert_base/README.md | 59 -- .../roberta-base-squad2-covid/README.md | 106 --- .../deepset/roberta-base-squad2-v2/README.md | 115 ---- .../deepset/roberta-base-squad2/README.md | 121 ---- model_cards/deepset/sentence_bert/README.md | 1 - .../xlm-roberta-large-squad2/README.md | 131 ---- .../covid-twitter-bert/README.md | 51 -- model_cards/distilbert-base-cased-README.md | 40 -- ...lbert-base-cased-distilled-squad-README.md | 13 - .../distilbert-base-german-cased-README.md | 5 - ...stilbert-base-multilingual-cased-README.md | 35 - model_cards/distilbert-base-uncased-README.md | 218 ------- ...ert-base-uncased-distilled-squad-README.md | 16 - ...-uncased-finetuned-sst-2-english-README.md | 19 - model_cards/distilgpt2-README.md | 21 - model_cards/distilroberta-base-README.md | 50 -- .../README.md | 1 - .../bert-base-polish-cased-v1/README.md | 135 ---- .../bert-base-polish-uncased-v1/README.md | 135 ---- model_cards/dslim/bert-base-NER/README.md | 114 ---- .../bert-base-romanian-cased-v1/README.md | 48 -- .../bert-base-romanian-uncased-v1/README.md | 51 -- model_cards/e-tony/gpt2-rnm/README.md | 20 - .../cs224n-squad2.0-albert-base-v2/README.md | 95 --- .../cs224n-squad2.0-albert-large-v2/README.md | 95 --- .../README.md | 95 --- .../README.md | 86 --- .../cs224n-squad2.0-roberta-base/README.md | 86 --- .../emilyalsentzer/Bio_ClinicalBERT/README.md | 39 -- .../Bio_Discharge_Summary_BERT/README.md | 39 -- .../README.md | 108 ---- model_cards/ethanyt/guwenbert-base/README.md | 74 --- model_cards/ethanyt/guwenbert-large/README.md | 74 --- model_cards/facebook/bart-large-cnn/README.md | 7 - .../facebook/bart-large-mnli/README.md | 81 --- model_cards/facebook/bart-large/README.md | 15 - .../facebook/rag-sequence-base/README.md | 54 -- .../facebook/rag-sequence-nq/README.md | 37 -- model_cards/facebook/rag-token-base/README.md | 57 -- model_cards/facebook/rag-token-nq/README.md | 37 -- .../facebook/rag-token-nq_new/README.md | 24 - model_cards/facebook/wmt19-de-en/README.md | 109 ---- model_cards/facebook/wmt19-en-de/README.md | 109 ---- model_cards/facebook/wmt19-en-ru/README.md | 109 ---- model_cards/facebook/wmt19-ru-en/README.md | 109 ---- .../t5-base-multi-sentence-doctor/README.md | 109 ---- .../sent-banner.png | Bin 73284 -> 0 bytes .../fmikaelian/camembert-base-fquad/README.md | 49 -- .../fmikaelian/camembert-base-squad/README.md | 49 -- .../flaubert-base-uncased-squad/README.md | 48 -- .../README.md | 161 ----- .../intermediate-base/README.md | 94 --- .../funnel-transformer/intermediate/README.md | 90 --- .../funnel-transformer/large-base/README.md | 94 --- .../funnel-transformer/large/README.md | 90 --- .../funnel-transformer/medium-base/README.md | 94 --- .../funnel-transformer/medium/README.md | 90 --- .../funnel-transformer/small-base/README.md | 94 --- .../funnel-transformer/small/README.md | 90 --- .../funnel-transformer/xlarge-base/README.md | 94 --- .../funnel-transformer/xlarge/README.md | 90 --- .../gk-hinglish-sentiment/README.md | 72 --- model_cards/gaochangkuan/model_dir/README.md | 66 -- .../electra-base-german-uncased/README.md | 152 ----- .../README.md | 39 -- .../french-camembert-postag-model/README.md | 96 --- .../gilf/french-postag-model/README.md | 90 --- .../google/bert2bert_L-24_wmt_de_en/README.md | 39 -- .../google/bert2bert_L-24_wmt_en_de/README.md | 38 -- .../bert_uncased_L-10_H-128_A-2/README.md | 1 - .../bert_uncased_L-10_H-256_A-4/README.md | 1 - .../bert_uncased_L-10_H-512_A-8/README.md | 1 - .../bert_uncased_L-10_H-768_A-12/README.md | 1 - .../bert_uncased_L-12_H-128_A-2/README.md | 1 - .../bert_uncased_L-12_H-256_A-4/README.md | 1 - .../bert_uncased_L-12_H-512_A-8/README.md | 1 - .../bert_uncased_L-12_H-768_A-12/README.md | 1 - .../bert_uncased_L-2_H-128_A-2/README.md | 1 - .../bert_uncased_L-2_H-256_A-4/README.md | 1 - .../bert_uncased_L-2_H-512_A-8/README.md | 1 - .../bert_uncased_L-2_H-768_A-12/README.md | 1 - .../bert_uncased_L-4_H-128_A-2/README.md | 1 - .../bert_uncased_L-4_H-256_A-4/README.md | 1 - .../bert_uncased_L-4_H-512_A-8/README.md | 1 - .../bert_uncased_L-4_H-768_A-12/README.md | 1 - .../bert_uncased_L-6_H-128_A-2/README.md | 1 - .../bert_uncased_L-6_H-256_A-4/README.md | 1 - .../bert_uncased_L-6_H-512_A-8/README.md | 1 - .../bert_uncased_L-6_H-768_A-12/README.md | 1 - .../bert_uncased_L-8_H-128_A-2/README.md | 1 - .../bert_uncased_L-8_H-256_A-4/README.md | 1 - .../bert_uncased_L-8_H-512_A-8/README.md | 1 - .../bert_uncased_L-8_H-768_A-12/README.md | 1 - .../electra-base-discriminator/README.md | 36 -- .../google/electra-base-generator/README.md | 31 - .../electra-large-discriminator/README.md | 36 -- .../google/electra-large-generator/README.md | 31 - .../electra-small-discriminator/README.md | 36 -- .../google/electra-small-generator/README.md | 31 - .../google/mobilebert-uncased/README.md | 32 - .../reformer-crime-and-punishment/README.md | 20 - model_cards/google/reformer-enwik8/README.md | 57 -- .../google/roberta2roberta_L-24_bbc/README.md | 58 -- .../README.md | 75 --- .../roberta2roberta_L-24_discofuse/README.md | 37 -- .../roberta2roberta_L-24_gigaword/README.md | 39 -- .../roberta2roberta_L-24_wikisplit/README.md | 36 -- model_cards/gpt2-README.md | 163 ----- model_cards/gpt2-large-README.md | 1 - model_cards/gpt2-medium-README.md | 1 - model_cards/gpt2-xl-README.md | 1 - model_cards/gsarti/biobert-nli/README.md | 37 -- model_cards/gsarti/covidbert-nli/README.md | 38 -- model_cards/gsarti/scibert-nli/README.md | 36 -- .../README.md | 61 -- model_cards/hatmimoha/arabic-ner/README.md | 114 ---- .../healx/gpt-2-pubmed-large/README.md | 3 - .../healx/gpt-2-pubmed-medium/README.md | 3 - .../README.md | 94 --- .../README.md | 94 --- .../README.md | 96 --- .../huawei-noah/DynaBERT_MNLI/README.md | 20 - .../huawei-noah/DynaBERT_SST-2/README.md | 20 - .../TinyBERT_General_4L_312D/README.md | 19 - .../CodeBERTa-language-id/README.md | 300 --------- .../huggingface/CodeBERTa-small-v1/README.md | 130 ---- .../albert-base-bahasa-cased/README.md | 86 --- .../albert-tiny-bahasa-cased/README.md | 86 --- .../bert-base-bahasa-cased/README.md | 92 --- .../README.md | 81 --- .../README.md | 86 --- .../README.md | 81 --- .../README.md | 86 --- .../gpt2-117M-bahasa-cased/README.md | 90 --- .../gpt2-345M-bahasa-cased/README.md | 91 --- .../t5-base-bahasa-cased/README.md | 74 --- .../README.md | 103 --- .../t5-small-bahasa-cased/README.md | 74 --- .../README.md | 103 --- .../tiny-bert-bahasa-cased/README.md | 92 --- .../xlnet-base-bahasa-cased/README.md | 64 -- .../bert-base-cased-qa-evaluator/README.md | 28 - .../roberta-base-bulgarian-pos/README.md | 26 - .../roberta-base-bulgarian/README.md | 29 - .../roberta-small-bulgarian-pos/README.md | 26 - .../roberta-small-bulgarian/README.md | 29 - .../t5-base-question-generator/README.md | 33 - .../illuin/camembert-base-fquad/README.md | 56 -- .../illuin/camembert-large-fquad/README.md | 60 -- model_cards/illuin/lepetit/README.md | 15 - .../indobenchmark/indobert-base-p1/README.md | 60 -- .../indobenchmark/indobert-base-p2/README.md | 60 -- .../indobenchmark/indobert-large-p1/README.md | 60 -- .../indobenchmark/indobert-large-p2/README.md | 60 -- .../indobert-lite-base-p1/README.md | 60 -- .../indobert-lite-base-p2/README.md | 60 -- .../indobert-lite-large-p1/README.md | 60 -- .../indobert-lite-large-p2/README.md | 60 -- .../indolem/indobert-base-uncased/README.md | 56 -- .../README.md | 3 - .../bert_uncased_L-2_H-128_A-2/README.md | 76 --- .../ixa-ehu/berteus-base-cased/README.md | 28 - .../ixa-ehu/ixambert-base-cased/README.md | 31 - model_cards/jannesg/bertsson/README.md | 20 - .../jannesg/takalane_afr_roberta/README.md | 49 -- .../jannesg/takalane_nbl_roberta/README.md | 49 -- .../jannesg/takalane_nso_roberta/README.md | 49 -- .../jannesg/takalane_sot_roberta/README.md | 49 -- .../jannesg/takalane_ssw_roberta/README.md | 49 -- .../jannesg/takalane_tsn_roberta/README.md | 49 -- .../jannesg/takalane_tso_roberta/README.md | 49 -- .../jannesg/takalane_ven_roberta/README.md | 49 -- .../jannesg/takalane_xho_roberta/README.md | 49 -- .../jannesg/takalane_zul_roberta/README.md | 49 -- .../bert-tagalog-base-cased-WWM/README.md | 62 -- .../bert-tagalog-base-cased/README.md | 62 -- .../bert-tagalog-base-uncased-WWM/README.md | 62 -- .../bert-tagalog-base-uncased/README.md | 62 -- .../distilbert-tagalog-base-cased/README.md | 63 -- .../README.md | 48 -- .../README.md | 48 -- .../README.md | 48 -- .../README.md | 48 -- .../README.md | 48 -- .../README.md | 48 -- .../README.md | 48 -- .../README.md | 48 -- model_cards/jimregan/BERTreach/README.md | 23 - .../README.md | 26 - .../bart-large-mnli-yahoo-answers/README.md | 71 --- .../joeddav/xlm-roberta-large-xnli/README.md | 122 ---- model_cards/jordimas/julibert/README.md | 23 - model_cards/jplu/tf-camembert-base/README.md | 31 - .../jplu/tf-xlm-r-ner-40-lang/README.md | 602 ------------------ .../jplu/tf-xlm-roberta-base/README.md | 36 -- .../jplu/tf-xlm-roberta-large/README.md | 36 -- .../julien-c/EsperBERTo-small-pos/README.md | 42 -- .../julien-c/EsperBERTo-small/README.md | 63 -- .../julien-c/bert-xsmall-dummy/README.md | 21 - model_cards/julien-c/dummy-unknown/README.md | 58 -- model_cards/keshan/SinhalaBERTo/README.md | 37 -- .../README.md | 28 - .../README.md | 45 -- .../README.md | 41 -- .../README.md | 45 -- .../README.md | 41 -- .../albert-xlarge-v2-squad-v2/README.md | 61 -- .../biobert_v1.1_pubmed_squad_v2/README.md | 64 -- .../gpt2-medium-topic-news/README.md | 41 -- .../README.md | 61 -- .../kuisailab/albert-base-arabic/README.md | 64 -- .../kuisailab/albert-large-arabic/README.md | 64 -- .../kuisailab/albert-xlarge-arabic/README.md | 64 -- model_cards/kuppuluri/telugu_bertu/README.md | 24 - .../kuppuluri/telugu_bertu_ner/README.md | 35 - .../kuppuluri/telugu_bertu_pos/README.md | 36 -- .../kuppuluri/telugu_bertu_tydiqa/README.md | 18 - .../GigaBERT-v3-Arabic-and-English/README.md | 27 - .../albert-base-turkish-uncased/README.md | 52 -- .../bert-base-turkish-uncased/README.md | 52 -- .../README.md | 52 -- .../README.md | 52 -- .../README.md | 32 - .../README.md | 52 -- model_cards/lordtt13/COVID-SciBERT/README.md | 58 -- model_cards/lordtt13/emo-mobilebert/README.md | 44 -- .../bert-turkish-question-answering/README.md | 61 -- model_cards/lvwerra/bert-imdb/README.md | 14 - model_cards/lvwerra/gpt2-imdb-ctrl/README.md | 18 - model_cards/lvwerra/gpt2-imdb-pos/README.md | 18 - model_cards/lvwerra/gpt2-imdb/README.md | 27 - .../lvwerra/gpt2-medium-taboo/README.md | 12 - model_cards/lysandre/arxiv-nlp/README.md | 7 - model_cards/lysandre/arxiv/README.md | 7 - .../m3hrdadfi/albert-fa-base-v2/README.md | 161 ----- .../bert2bert-fa-news-headline/README.md | 24 - .../bert2bert-fa-wiki-summary/README.md | 24 - model_cards/microsoft/DeBERTa-base/README.md | 36 -- model_cards/microsoft/DeBERTa-large/README.md | 37 -- .../microsoft/DialoGPT-large/README.md | 54 -- .../microsoft/DialoGPT-medium/README.md | 54 -- .../microsoft/DialoGPT-small/README.md | 54 -- .../MiniLM-L12-H384-uncased/README.md | 43 -- .../Multilingual-MiniLM-L12-H384/README.md | 89 --- .../microsoft/codebert-base-mlm/README.md | 46 -- model_cards/microsoft/codebert-base/README.md | 27 - .../microsoft/layoutlm-base-uncased/README.md | 30 - .../layoutlm-large-uncased/README.md | 30 - .../prophetnet-large-uncased-cnndm/README.md | 38 -- .../README.md | 39 -- .../prophetnet-large-uncased/README.md | 37 -- .../README.md | 37 -- .../README.md | 31 - .../xprophetnet-large-wiki100-cased/README.md | 42 -- model_cards/monilouise/ner_pt_br/README.md | 103 --- .../koelectra-base-discriminator/README.md | 52 -- .../koelectra-base-generator/README.md | 45 -- .../koelectra-small-discriminator/README.md | 52 -- .../koelectra-small-generator/README.md | 45 -- model_cards/monsoon-nlp/dv-wave/README.md | 28 - .../README.md | 23 - model_cards/mrm8488/CodeBERTaPy/README.md | 123 ---- .../mrm8488/GPT-2-finetuned-CORD19/README.md | 61 -- .../GPT-2-finetuned-common_gen/README.md | 63 -- .../README.md | 62 -- model_cards/mrm8488/GuaPeTe-2-tiny/README.md | 7 - model_cards/mrm8488/RoBERTinha/README.md | 8 - model_cards/mrm8488/RoBasquERTa/README.md | 8 - .../RuPERTa-base-finetuned-ner/README.md | 92 --- .../RuPERTa-base-finetuned-pawsx-es/README.md | 9 - .../RuPERTa-base-finetuned-pos/README.md | 111 ---- .../RuPERTa-base-finetuned-squadv1/README.md | 5 - .../RuPERTa-base-finetuned-squadv2/README.md | 5 - model_cards/mrm8488/RuPERTa-base/README.md | 125 ---- .../README.md | 106 --- .../README.md | 9 - .../bert-base-german-finetuned-ler/README.md | 100 --- .../README.md | 91 --- .../README.md | 110 ---- .../bert-medium-finetuned-squadv2/README.md | 122 ---- .../bert-mini-finetuned-squadv2/README.md | 105 --- .../README.md | 52 -- .../README.md | 78 --- .../README.md | 131 ---- .../README.md | 129 ---- .../bert-small-finetuned-squadv2/README.md | 104 --- .../README.md | 72 --- .../README.md | 52 -- .../README.md | 79 --- .../README.md | 83 --- .../README.md | 82 --- .../bert-tiny-finetuned-squadv2/README.md | 102 --- .../bert-uncased-finetuned-qnli/README.md | 62 -- .../README.md | 9 - .../mrm8488/chEMBL_smiles_v1/README.md | 77 --- model_cards/mrm8488/codeBERTaJS/README.md | 100 --- .../README.md | 61 -- .../README.md | 53 -- .../README.md | 82 --- .../README.md | 141 ---- .../README.md | 65 -- .../electra-base-finetuned-squadv1/README.md | 85 --- .../electra-small-finetuned-squadv1/README.md | 87 --- .../electra-small-finetuned-squadv2/README.md | 86 --- .../electricidad-base-discriminator/README.md | 85 --- .../README.md | 9 - .../electricidad-base-generator/README.md | 50 -- .../README.md | 74 --- .../README.md | 102 --- .../gpt2-finetuned-recipes-cooking/README.md | 6 - .../README.md | 6 - model_cards/mrm8488/gpt2-imdb-neg/README.md | 30 - .../mrm8488/gpt2-imdb-neutral/README.md | 32 - .../README.md | 72 --- .../README.md | 85 --- .../README.md | 74 --- .../README.md | 74 --- .../README.md | 88 --- .../README.md | 92 --- .../roberta-large-finetuned-wsc/README.md | 128 ---- .../spanbert-base-finetuned-squadv1/README.md | 80 --- .../spanbert-base-finetuned-squadv2/README.md | 82 --- .../spanbert-base-finetuned-tacred/README.md | 53 -- .../spanbert-finetuned-squadv1/README.md | 84 --- .../spanbert-finetuned-squadv2/README.md | 86 --- .../README.md | 80 --- .../README.md | 82 --- .../spanbert-large-finetuned-tacred/README.md | 53 -- .../squeezebert-finetuned-squadv1/README.md | 72 --- .../squeezebert-finetuned-squadv2/README.md | 72 --- .../README.md | 70 -- .../t5-base-finetuned-break_data/README.md | 64 -- .../t5-base-finetuned-common_gen/README.md | 79 --- .../t5-base-finetuned-e2m-intent/README.md | 64 -- .../t5-base-finetuned-emotion/README.md | 82 --- .../README.md | 68 -- .../mrm8488/t5-base-finetuned-qasc/README.md | 66 -- .../t5-base-finetuned-quarel/README.md | 65 -- .../t5-base-finetuned-quartz/README.md | 74 --- .../README.md | 75 --- .../README.md | 112 ---- .../README.md | 73 --- .../t5-base-finetuned-squadv2/README.md | 78 --- .../README.md | 64 -- .../README.md | 73 --- .../t5-base-finetuned-wikiSQL/README.md | 73 --- .../t5-small-finetuned-emotion/README.md | 82 --- .../README.md | 68 -- .../README.md | 73 --- .../t5-small-finetuned-squadv1/README.md | 77 --- .../t5-small-finetuned-squadv2/README.md | 78 --- .../t5-small-finetuned-wikiSQL/README.md | 70 -- .../README.md | 96 --- .../xlm-multi-finetuned-xquadv1/README.md | 123 ---- .../mymusise/gpt2-medium-chinese/README.md | 35 - .../electra-base-turkish-cased-ner/README.md | 16 - .../README.md | 8 - .../bert-base-portuguese-cased/README.md | 105 --- .../bert-large-portuguese-cased/README.md | 106 --- .../indic-transformers-bn-bert/README.md | 25 - .../README.md | 29 - .../indic-transformers-bn-roberta/README.md | 29 - .../README.md | 29 - .../indic-transformers-hi-bert/README.md | 29 - .../README.md | 29 - .../indic-transformers-hi-roberta/README.md | 29 - .../README.md | 29 - .../indic-transformers-te-bert/README.md | 29 - .../README.md | 29 - .../indic-transformers-te-roberta/README.md | 29 - .../README.md | 29 - .../README.md | 95 --- model_cards/neurocode/IsRoBERTa/README.md | 74 --- model_cards/nghuyong/ernie-1.0/README.md | 44 -- model_cards/nghuyong/ernie-2.0-en/README.md | 43 -- .../nghuyong/ernie-2.0-large-en/README.md | 39 -- model_cards/nghuyong/ernie-tiny/README.md | 42 -- model_cards/nikokons/gpt2-greek/README.md | 5 - .../bert-base-greek-uncased-v1/README.md | 134 ---- .../nlpaueb/legal-bert-base-uncased/README.md | 101 --- .../README.md | 51 -- .../nyu-mll/roberta-base-100M-1/README.md | 1 - .../nyu-mll/roberta-base-100M-2/README.md | 1 - .../nyu-mll/roberta-base-100M-3/README.md | 1 - .../nyu-mll/roberta-base-10M-1/README.md | 1 - .../nyu-mll/roberta-base-10M-2/README.md | 1 - .../nyu-mll/roberta-base-10M-3/README.md | 1 - .../nyu-mll/roberta-base-1B-1/README.md | 1 - .../nyu-mll/roberta-base-1B-2/README.md | 1 - .../nyu-mll/roberta-base-1B-3/README.md | 1 - .../nyu-mll/roberta-med-small-1M-1/README.md | 1 - .../nyu-mll/roberta-med-small-1M-2/README.md | 1 - .../nyu-mll/roberta-med-small-1M-3/README.md | 1 - .../nyu-mll/roberta_1M_to_1B/README.md | 49 -- .../german-sentiment-bert/README.md | 125 ---- .../README.md | 46 -- .../README.md | 46 -- .../bert2bert-cnn_dailymail-fp16/README.md | 218 ------- .../bert2bert_cnn_daily_mail/README.md | 16 - .../bert2gpt2-cnn_dailymail-fp16/README.md | 295 --------- .../README.md | 252 -------- .../README.md | 238 ------- .../README.md | 238 ------- .../roberta_shared_bbc_xsum/README.md | 16 - .../robbert-v2-dutch-base/README.md | 164 ----- .../pedropei/question-intimacy/README.md | 5 - .../gpt2-small-portuguese/README.md | 186 ------ model_cards/pradhyra/AWSBlogBert/README.md | 8 - .../gpt2-genre-story-generator/README.md | 32 - model_cards/pvl/labse_bert/README.md | 47 -- .../qarib/bert-base-qarib60_1790k/README.md | 96 --- .../qarib/bert-base-qarib60_1970k/README.md | 96 --- .../qarib/bert-base-qarib60_860k/README.md | 96 --- .../ramsrigouthamg/t5_paraphraser/README.md | 84 --- model_cards/rdenadai/BR_BERTo/README.md | 30 - .../README.md | 57 -- .../rjbownes/Magic-The-Generating/README.md | 72 --- model_cards/roberta-base-README.md | 234 ------- model_cards/roberta-large-README.md | 235 ------- model_cards/roberta-large-mnli-README.md | 22 - .../README.md | 61 -- .../bert-base-en-es-codemix-cased/README.md | 101 --- .../bert-base-en-hi-codemix-cased/README.md | 101 --- .../README.md | 95 --- .../README.md | 96 --- .../sagorsarker/bangla-bert-base/README.md | 115 ---- .../bangla-bert-sentiment/README.md | 61 -- .../codeswitch-hineng-lid-lince/README.md | 53 -- .../codeswitch-hineng-ner-lince/README.md | 54 -- .../codeswitch-hineng-pos-lince/README.md | 52 -- .../codeswitch-nepeng-lid-lince/README.md | 54 -- .../codeswitch-spaeng-lid-lince/README.md | 52 -- .../codeswitch-spaeng-ner-lince/README.md | 53 -- .../codeswitch-spaeng-pos-lince/README.md | 52 -- .../README.md | 53 -- model_cards/sarahlintang/IndoBERT/README.md | 43 -- .../README.md | 22 - .../bert-base-turkish-ner-cased/README.md | 93 --- .../README.md | 141 ---- .../savasy/bert-base-turkish-squad/README.md | 67 -- .../README.md | 102 --- .../schmidek/electra-small-cased/README.md | 11 - .../seiya/oubiobert-base-uncased/README.md | 51 -- .../sentence-transformers/LaBSE/README.md | 37 -- .../bert-base-nli-cls-token/README.md | 76 --- .../bert-base-nli-max-tokens/README.md | 88 --- .../bert-base-nli-mean-tokens/README.md | 85 --- .../literary-german-bert/README.md | 51 -- .../literary-german-bert/kfold.png | Bin 5760 -> 0 bytes .../literary-german-bert/prosa-jahre.png | Bin 21657 -> 0 bytes .../seyonec/ChemBERTa-zinc-base-v1/README.md | 19 - .../shoarora/alectra-small-owt/README.md | 60 -- .../shoarora/electra-small-owt/README.md | 59 -- .../shrugging-grace/tweetclassifier/README.md | 26 - model_cards/smanjil/German-MedBERT/README.md | 45 -- model_cards/spentaur/yelp/README.md | 19 - .../squeezebert-mnli-headless/README.md | 67 -- .../squeezebert/squeezebert-mnli/README.md | 67 -- .../squeezebert/squeezebert-uncased/README.md | 67 -- model_cards/stas/tiny-wmt19-en-de/README.md | 18 - model_cards/stevhliu/astroGPT/README.md | 51 -- .../surajp/RoBERTa-hindi-guj-san/README.md | 107 ---- model_cards/surajp/SanBERTa/README.md | 99 --- .../surajp/albert-base-sanskrit/README.md | 36 -- model_cards/t5-11b-README.md | 40 -- model_cards/t5-3b-README.md | 28 - model_cards/t5-base-README.md | 28 - model_cards/t5-large-README.md | 28 - model_cards/t5-small-README.md | 28 - model_cards/tartuNLP/EstBERT/README.md | 41 -- model_cards/tblard/tf-allocine/README.md | 40 -- .../tuner007/pegasus_paraphrase/README.md | 53 -- model_cards/tuner007/pegasus_qa/README.md | 30 - model_cards/tuner007/t5_abs_qa/README.md | 39 -- .../twmkn9/albert-base-v2-squad2/README.md | 45 -- .../twmkn9/bert-base-uncased-squad2/README.md | 45 -- .../distilbert-base-uncased-squad2/README.md | 45 -- .../distilroberta-base-squad2/README.md | 44 -- .../uer/chinese_roberta_L-2_H-128/README.md | 44 -- .../uer/gpt2-chinese-couplet/README.md | 85 --- model_cards/uer/gpt2-chinese-poem/README.md | 85 --- .../uncnlp/lxmert-base-uncased/LICENSE | 21 - .../uncnlp/lxmert-base-uncased/README.md | 34 - .../lxmert-base-uncased/lxmert_model-1.jpg | Bin 281338 -> 0 bytes .../unideeplearning/polibert_sa/README.md | 53 -- .../urduhack/roberta-urdu-small/README.md | 30 - .../bart-large-finetuned-squadv1/README.md | 67 -- .../valhalla/distilbart-mnli-12-1/README.md | 59 -- .../valhalla/distilbart-mnli-12-3/README.md | 59 -- .../valhalla/distilbart-mnli-12-6/README.md | 59 -- .../valhalla/distilbart-mnli-12-9/README.md | 59 -- .../README.md | 47 -- .../README.md | 54 -- model_cards/valhalla/t5-base-e2e-qg/README.md | 38 -- .../valhalla/t5-base-qa-qg-hl/README.md | 50 -- model_cards/valhalla/t5-base-qg-hl/README.md | 33 - model_cards/valhalla/t5-base-squad/README.md | 38 -- .../valhalla/t5-samll-qg-prepend/README.md | 36 -- .../valhalla/t5-small-e2e-qg/README.md | 38 -- .../valhalla/t5-small-qa-qg-hl/README.md | 49 -- model_cards/valhalla/t5-small-qg-hl/README.md | 33 - model_cards/vinai/bertweet-base/README.md | 80 --- .../bertweet-covid19-base-cased/README.md | 80 --- .../bertweet-covid19-base-uncased/README.md | 80 --- model_cards/vinai/phobert-base/README.md | 55 -- model_cards/vinai/phobert-large/README.md | 55 -- .../voidful/albert_chinese_base/README.md | 43 -- .../voidful/albert_chinese_large/README.md | 43 -- .../voidful/albert_chinese_small/README.md | 43 -- .../voidful/albert_chinese_tiny/README.md | 43 -- .../voidful/albert_chinese_xlarge/README.md | 43 -- .../voidful/albert_chinese_xxlarge/README.md | 43 -- .../wietsedv/bert-base-dutch-cased/README.md | 12 - .../wptoux/albert-chinese-large-qa/README.md | 34 - model_cards/xlm-mlm-en-2048-README.md | 10 - model_cards/xlm-roberta-base-README.md | 10 - ...a-large-finetuned-conll03-german-README.md | 5 - model_cards/yjernite/bart_eli5/README.md | 10 - .../README.md | 82 --- .../youscan/ukr-roberta-base/README.md | 26 - model_cards/yuvraj/summarizer-cnndm/README.md | 27 - model_cards/yuvraj/xSumm/README.md | 26 - .../zanelim/singbert-large-sg/README.md | 215 ------- .../zanelim/singbert-lite-sg/README.md | 168 ----- model_cards/zanelim/singbert/README.md | 215 ------- 758 files changed, 50 insertions(+), 45116 deletions(-) delete mode 100644 model_cards/Cinnamon/electra-small-japanese-discriminator/README.md delete mode 100644 model_cards/Cinnamon/electra-small-japanese-generator/README.md delete mode 100644 model_cards/DJSammy/bert-base-danish-uncased_BotXO,ai/README.md delete mode 100644 model_cards/DeepPavlov/bert-base-bg-cs-pl-ru-cased/README.md delete mode 100644 model_cards/DeepPavlov/bert-base-cased-conversational/README.md delete mode 100644 model_cards/DeepPavlov/bert-base-multilingual-cased-sentence/README.md delete mode 100644 model_cards/DeepPavlov/rubert-base-cased-conversational/README.md delete mode 100644 model_cards/DeepPavlov/rubert-base-cased-sentence/README.md delete mode 100644 model_cards/DeepPavlov/rubert-base-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-15lang-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-ar-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-bg-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-de-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-el-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-ar-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-bg-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-de-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-el-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-es-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-fr-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-hi-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-ru-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-sw-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-th-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-tr-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-ur-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-vi-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-en-zh-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-es-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-fr-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-hi-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-ru-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-sw-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-th-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-tr-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-ur-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-vi-cased/README.md delete mode 100644 model_cards/Geotrend/bert-base-zh-cased/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-arabic/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-english/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-french/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-german/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-indonesian/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-italian/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-polish/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-portugese/README.md delete mode 100644 model_cards/Hate-speech-CNERG/dehatebert-mono-spanish/README.md delete mode 100644 model_cards/HooshvareLab/bert-base-parsbert-armanner-uncased/README.md delete mode 100644 model_cards/HooshvareLab/bert-base-parsbert-ner-uncased/README.md delete mode 100644 model_cards/HooshvareLab/bert-base-parsbert-peymaner-uncased/README.md delete mode 100644 model_cards/HooshvareLab/bert-base-parsbert-uncased/README.md delete mode 100644 model_cards/HooshvareLab/bert-fa-base-uncased/README.md delete mode 100644 model_cards/KB/albert-base-swedish-cased-alpha/README.md delete mode 100644 model_cards/KB/bert-base-swedish-cased-ner/README.md delete mode 100644 model_cards/KB/bert-base-swedish-cased/README.md delete mode 100644 model_cards/LorenzoDeMattei/GePpeTto/README.md delete mode 100644 model_cards/Michau/t5-base-en-generate-headline/README.md delete mode 100644 model_cards/MoseliMotsoehli/TswanaBert/README.md delete mode 100644 model_cards/MoseliMotsoehli/zuBERTa/README.md delete mode 100644 model_cards/Musixmatch/umberto-commoncrawl-cased-v1/README.md delete mode 100644 model_cards/Musixmatch/umberto-wikipedia-uncased-v1/README.md delete mode 100644 model_cards/NLP4H/ms_bert/README.md delete mode 100644 model_cards/Naveen-k/KanBERTo/README.md delete mode 100644 model_cards/NeuML/bert-small-cord19-squad2/README.md delete mode 100644 model_cards/NeuML/bert-small-cord19/README.md delete mode 100644 model_cards/NeuML/bert-small-cord19qa/README.md delete mode 100644 model_cards/NlpHUST/vibert4news-base-cased/README.md delete mode 100644 model_cards/Norod78/hewiki-articles-distilGPT2py-il/README.md delete mode 100644 model_cards/Ogayo/Hel-ach-en/README.md delete mode 100644 model_cards/Primer/bart-squad2/README.md create mode 100644 model_cards/README.md delete mode 100644 model_cards/Rostlab/prot_bert/README.md delete mode 100644 model_cards/Rostlab/prot_bert_bfd/README.md delete mode 100644 model_cards/Rostlab/prot_t5_xl_bfd/README.md delete mode 100644 model_cards/SZTAKI-HLT/hubert-base-cc/README.md delete mode 100644 model_cards/SparkBeyond/roberta-large-sts-b/README.md delete mode 100644 model_cards/T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb/README.md delete mode 100644 model_cards/T-Systems-onsite/cross-en-de-roberta-sentence-transformer/README.md delete mode 100644 model_cards/T-Systems-onsite/german-roberta-sentence-transformer-v2/README.md delete mode 100644 model_cards/Tereveni-AI/gpt2-124M-uk-fiction/README.md delete mode 100644 model_cards/TurkuNLP/bert-base-finnish-cased-v1/README.md delete mode 100644 model_cards/TurkuNLP/bert-base-finnish-uncased-v1/README.md delete mode 100644 model_cards/TypicaAI/magbert-ner/README.md delete mode 100644 model_cards/Vamsi/T5_Paraphrase_Paws/README.md delete mode 100644 model_cards/VictorSanh/roberta-base-finetuned-yelp-polarity/README.md delete mode 100644 model_cards/ViktorAlm/electra-base-norwegian-uncased-discriminator/README.md delete mode 100644 model_cards/a-ware/bart-squadv2/README.md delete mode 100644 model_cards/a-ware/roberta-large-squad-classification/README.md delete mode 100644 model_cards/a-ware/xlmroberta-squadv2/README.md delete mode 100644 model_cards/abhilash1910/financial_roberta/README.md delete mode 100644 model_cards/abhilash1910/french-roberta/README.md delete mode 100644 model_cards/activebus/BERT-DK_laptop/README.md delete mode 100644 model_cards/activebus/BERT-DK_rest/README.md delete mode 100644 model_cards/activebus/BERT-PT_laptop/README.md delete mode 100644 model_cards/activebus/BERT-PT_rest/README.md delete mode 100644 model_cards/activebus/BERT-XD_Review/README.md delete mode 100644 model_cards/activebus/BERT_Review/README.md delete mode 100644 model_cards/adalbertojunior/PTT5-SMALL-SUM/README.md delete mode 100644 model_cards/ahotrod/albert_xxlargev1_squad2_512/README.md delete mode 100644 model_cards/ahotrod/electra_large_discriminator_squad2_512/README.md delete mode 100644 model_cards/ahotrod/roberta_large_squad2/README.md delete mode 100644 model_cards/ahotrod/xlnet_large_squad2_512/README.md delete mode 100644 model_cards/ai4bharat/indic-bert/README.md delete mode 100644 model_cards/akhooli/gpt2-small-arabic-poetry/README.md delete mode 100644 model_cards/akhooli/gpt2-small-arabic/README.md delete mode 100644 model_cards/akhooli/mbart-large-cc25-ar-en/README.md delete mode 100644 model_cards/akhooli/mbart-large-cc25-en-ar/README.md delete mode 100644 model_cards/akhooli/personachat-arabic/README.md delete mode 100644 model_cards/akhooli/xlm-r-large-arabic-sent/README.md delete mode 100644 model_cards/akhooli/xlm-r-large-arabic-toxic/README.md delete mode 100644 model_cards/albert-base-v1-README.md delete mode 100644 model_cards/albert-xxlarge-v2-README.md delete mode 100644 model_cards/aliosm/ComVE-distilgpt2/README.md delete mode 100644 model_cards/aliosm/ComVE-gpt2-large/README.md delete mode 100644 model_cards/aliosm/ComVE-gpt2-medium/README.md delete mode 100644 model_cards/aliosm/ComVE-gpt2/README.md delete mode 100644 model_cards/aliosm/ai-soco-cpp-roberta-small-clas/README.md delete mode 100644 model_cards/aliosm/ai-soco-cpp-roberta-small/README.md delete mode 100644 model_cards/aliosm/ai-soco-cpp-roberta-tiny-96-clas/README.md delete mode 100644 model_cards/aliosm/ai-soco-cpp-roberta-tiny-96/README.md delete mode 100644 model_cards/aliosm/ai-soco-cpp-roberta-tiny-clas/README.md delete mode 100644 model_cards/aliosm/ai-soco-cpp-roberta-tiny/README.md delete mode 100644 model_cards/allegro/herbert-base-cased/README.md delete mode 100644 model_cards/allegro/herbert-klej-cased-tokenizer-v1/README.md delete mode 100644 model_cards/allegro/herbert-klej-cased-v1/README.md delete mode 100644 model_cards/allegro/herbert-large-cased/README.md delete mode 100644 model_cards/allenai/biomed_roberta_base/README.md delete mode 100644 model_cards/allenai/longformer-base-4096-extra.pos.embd.only/README.md delete mode 100644 model_cards/allenai/longformer-base-4096/README.md delete mode 100644 model_cards/allenai/scibert_scivocab_cased/README.md delete mode 100644 model_cards/allenai/scibert_scivocab_uncased/README.md delete mode 100644 model_cards/allenai/wmt16-en-de-12-1/README.md delete mode 100644 model_cards/allenai/wmt16-en-de-dist-12-1/README.md delete mode 100644 model_cards/allenai/wmt16-en-de-dist-6-1/README.md delete mode 100644 model_cards/allenai/wmt19-de-en-6-6-base/README.md delete mode 100644 model_cards/allenai/wmt19-de-en-6-6-big/README.md delete mode 100644 model_cards/allenyummy/chinese-bert-wwm-ehr-ner-sl/README.md delete mode 100644 model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md delete mode 100644 model_cards/amine/bert-base-5lang-cased/README.md delete mode 100644 model_cards/antoiloui/belgpt2/README.md delete mode 100644 model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616/README.md delete mode 100644 model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616_squad2/README.md delete mode 100644 model_cards/aodiniz/bert_uncased_L-2_H-512_A-8_cord19-200616/README.md delete mode 100644 model_cards/aodiniz/bert_uncased_L-4_H-256_A-4_cord19-200616/README.md delete mode 100644 model_cards/asafaya/bert-base-arabic/README.md delete mode 100644 model_cards/asafaya/bert-large-arabic/README.md delete mode 100644 model_cards/asafaya/bert-medium-arabic/README.md delete mode 100644 model_cards/asafaya/bert-mini-arabic/README.md delete mode 100644 model_cards/ashwani-tanwar/Gujarati-XLM-R-Base/README.md delete mode 100644 model_cards/aubmindlab/bert-base-arabert/README.md delete mode 100644 model_cards/aubmindlab/bert-base-arabertv01/README.md delete mode 100644 model_cards/bart-large-cnn/README.md delete mode 100644 model_cards/bart-large-xsum/README.md delete mode 100644 model_cards/bashar-talafha/multi-dialect-bert-base-arabic/README.md delete mode 100644 model_cards/bayartsogt/albert-mongolian/README.md delete mode 100644 model_cards/bayartsogt/bert-base-mongolian-cased/README.md delete mode 100644 model_cards/bayartsogt/bert-base-mongolian-uncased/README.md delete mode 100644 model_cards/bert-base-cased-README.md delete mode 100644 model_cards/bert-base-chinese-README.md delete mode 100644 model_cards/bert-base-german-cased-README.md delete mode 100644 model_cards/bert-base-german-dbmdz-cased-README.md delete mode 100644 model_cards/bert-base-german-dbmdz-uncased-README.md delete mode 100644 model_cards/bert-base-multilingual-cased-README.md delete mode 100644 model_cards/bert-base-multilingual-uncased-README.md delete mode 100644 model_cards/bert-base-uncased-README.md delete mode 100644 model_cards/bert-large-cased-README.md delete mode 100644 model_cards/binwang/xlnet-base-cased/README.md delete mode 100644 model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-12_H-768_A-12/README.md delete mode 100644 model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-24_H-1024_A-16/README.md delete mode 100644 model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md delete mode 100644 model_cards/bionlp/bluebert_pubmed_uncased_L-24_H-1024_A-16/README.md delete mode 100644 model_cards/blinoff/roberta-base-russian-v0/README.md delete mode 100644 model_cards/cahya/bert-base-indonesian-522M/README.md delete mode 100644 model_cards/cahya/gpt2-small-indonesian-522M/README.md delete mode 100644 model_cards/cahya/roberta-base-indonesian-522M/README.md delete mode 100644 model_cards/cambridgeltl/BioRedditBERT-uncased/README.md delete mode 100644 model_cards/camembert-base-README.md delete mode 100644 model_cards/camembert/camembert-base-ccnet-4gb/README.md delete mode 100644 model_cards/camembert/camembert-base-ccnet/README.md delete mode 100644 model_cards/camembert/camembert-base-oscar-4gb/README.md delete mode 100644 model_cards/camembert/camembert-base-wikipedia-4gb/README.md delete mode 100644 model_cards/camembert/camembert-large/README.md delete mode 100644 model_cards/canwenxu/BERT-of-Theseus-MNLI/README.md delete mode 100644 model_cards/cedpsam/chatbot_fr/README.md delete mode 100644 model_cards/ceostroff/harry-potter-gpt2-fanfiction/README.md delete mode 100644 model_cards/chrisliu298/arxiv_ai_gpt2/README.md delete mode 100644 model_cards/cimm-kzn/endr-bert/README.md delete mode 100644 model_cards/cimm-kzn/enrudr-bert/README.md delete mode 100644 model_cards/cimm-kzn/rudr-bert/README.md delete mode 100644 model_cards/clue/albert_chinese_small/README.md delete mode 100644 model_cards/clue/albert_chinese_tiny/README.md delete mode 100644 model_cards/clue/roberta_chinese_3L312_clue_tiny/README.md delete mode 100644 model_cards/clue/roberta_chinese_base/README.md delete mode 100644 model_cards/clue/roberta_chinese_large/README.md delete mode 100644 model_cards/clue/xlnet_chinese_large/README.md delete mode 100644 model_cards/codegram/calbert-base-uncased/README.md delete mode 100644 model_cards/codegram/calbert-tiny-uncased/README.md delete mode 100644 model_cards/cooelf/limitbert/README.md delete mode 100644 model_cards/csarron/bert-base-uncased-squad-v1/README.md delete mode 100644 model_cards/csarron/mobilebert-uncased-squad-v1/README.md delete mode 100644 model_cards/csarron/mobilebert-uncased-squad-v2/README.md delete mode 100644 model_cards/csarron/roberta-base-squad-v1/README.md delete mode 100644 model_cards/daigo/bert-base-japanese-sentiment/README.md delete mode 100644 model_cards/dbmdz/bert-base-german-cased/README.md delete mode 100644 model_cards/dbmdz/bert-base-german-europeana-cased/README.md delete mode 100644 model_cards/dbmdz/bert-base-german-europeana-uncased/README.md delete mode 100644 model_cards/dbmdz/bert-base-german-uncased/README.md delete mode 100644 model_cards/dbmdz/bert-base-italian-cased/README.md delete mode 100644 model_cards/dbmdz/bert-base-italian-uncased/README.md delete mode 100644 model_cards/dbmdz/bert-base-italian-xxl-cased/README.md delete mode 100644 model_cards/dbmdz/bert-base-italian-xxl-uncased/README.md delete mode 100644 model_cards/dbmdz/bert-base-turkish-128k-cased/README.md delete mode 100644 model_cards/dbmdz/bert-base-turkish-128k-uncased/README.md delete mode 100644 model_cards/dbmdz/bert-base-turkish-cased/README.md delete mode 100644 model_cards/dbmdz/bert-base-turkish-uncased/README.md delete mode 100644 model_cards/dbmdz/distilbert-base-turkish-cased/README.md delete mode 100644 model_cards/dbmdz/electra-base-italian-xxl-cased-discriminator/README.md delete mode 100644 model_cards/dbmdz/electra-base-italian-xxl-cased-generator/README.md delete mode 100644 model_cards/dbmdz/electra-base-turkish-cased-discriminator/README.md delete mode 100644 model_cards/dbmdz/electra-small-turkish-cased-discriminator/README.md delete mode 100644 model_cards/dccuchile/bert-base-spanish-wwm-cased/README.md delete mode 100644 model_cards/dccuchile/bert-base-spanish-wwm-uncased/README.md delete mode 100644 model_cards/deepset/bert-base-german-cased-oldvocab/README.md delete mode 100644 model_cards/deepset/electra-base-squad2/README.md delete mode 100644 model_cards/deepset/gbert-base/README.md delete mode 100644 model_cards/deepset/gbert-large/README.md delete mode 100644 model_cards/deepset/gelectra-base-generator/README.md delete mode 100644 model_cards/deepset/gelectra-base/README.md delete mode 100644 model_cards/deepset/gelectra-large-generator/README.md delete mode 100644 model_cards/deepset/gelectra-large/README.md delete mode 100644 model_cards/deepset/minilm-uncased-squad2/README.md delete mode 100644 model_cards/deepset/quora_dedup_bert_base/README.md delete mode 100644 model_cards/deepset/roberta-base-squad2-covid/README.md delete mode 100644 model_cards/deepset/roberta-base-squad2-v2/README.md delete mode 100644 model_cards/deepset/roberta-base-squad2/README.md delete mode 100644 model_cards/deepset/sentence_bert/README.md delete mode 100644 model_cards/deepset/xlm-roberta-large-squad2/README.md delete mode 100644 model_cards/digitalepidemiologylab/covid-twitter-bert/README.md delete mode 100644 model_cards/distilbert-base-cased-README.md delete mode 100644 model_cards/distilbert-base-cased-distilled-squad-README.md delete mode 100644 model_cards/distilbert-base-german-cased-README.md delete mode 100644 model_cards/distilbert-base-multilingual-cased-README.md delete mode 100644 model_cards/distilbert-base-uncased-README.md delete mode 100644 model_cards/distilbert-base-uncased-distilled-squad-README.md delete mode 100644 model_cards/distilbert-base-uncased-finetuned-sst-2-english-README.md delete mode 100644 model_cards/distilgpt2-README.md delete mode 100644 model_cards/distilroberta-base-README.md delete mode 100644 model_cards/djstrong/bg_cs_pl_ru_cased_L-12_H-768_A-12/README.md delete mode 100644 model_cards/dkleczek/bert-base-polish-cased-v1/README.md delete mode 100644 model_cards/dkleczek/bert-base-polish-uncased-v1/README.md delete mode 100644 model_cards/dslim/bert-base-NER/README.md delete mode 100644 model_cards/dumitrescustefan/bert-base-romanian-cased-v1/README.md delete mode 100644 model_cards/dumitrescustefan/bert-base-romanian-uncased-v1/README.md delete mode 100644 model_cards/e-tony/gpt2-rnm/README.md delete mode 100644 model_cards/elgeish/cs224n-squad2.0-albert-base-v2/README.md delete mode 100644 model_cards/elgeish/cs224n-squad2.0-albert-large-v2/README.md delete mode 100644 model_cards/elgeish/cs224n-squad2.0-albert-xxlarge-v1/README.md delete mode 100644 model_cards/elgeish/cs224n-squad2.0-distilbert-base-uncased/README.md delete mode 100644 model_cards/elgeish/cs224n-squad2.0-roberta-base/README.md delete mode 100644 model_cards/emilyalsentzer/Bio_ClinicalBERT/README.md delete mode 100644 model_cards/emilyalsentzer/Bio_Discharge_Summary_BERT/README.md delete mode 100644 model_cards/etalab-ia/camembert-base-squadFR-fquad-piaf/README.md delete mode 100644 model_cards/ethanyt/guwenbert-base/README.md delete mode 100644 model_cards/ethanyt/guwenbert-large/README.md delete mode 100644 model_cards/facebook/bart-large-cnn/README.md delete mode 100644 model_cards/facebook/bart-large-mnli/README.md delete mode 100644 model_cards/facebook/bart-large/README.md delete mode 100644 model_cards/facebook/rag-sequence-base/README.md delete mode 100644 model_cards/facebook/rag-sequence-nq/README.md delete mode 100644 model_cards/facebook/rag-token-base/README.md delete mode 100644 model_cards/facebook/rag-token-nq/README.md delete mode 100644 model_cards/facebook/rag-token-nq_new/README.md delete mode 100644 model_cards/facebook/wmt19-de-en/README.md delete mode 100644 model_cards/facebook/wmt19-en-de/README.md delete mode 100644 model_cards/facebook/wmt19-en-ru/README.md delete mode 100644 model_cards/facebook/wmt19-ru-en/README.md delete mode 100644 model_cards/flexudy/t5-base-multi-sentence-doctor/README.md delete mode 100644 model_cards/flexudy/t5-base-multi-sentence-doctor/sent-banner.png delete mode 100644 model_cards/fmikaelian/camembert-base-fquad/README.md delete mode 100644 model_cards/fmikaelian/camembert-base-squad/README.md delete mode 100644 model_cards/fmikaelian/flaubert-base-uncased-squad/README.md delete mode 100644 model_cards/fran-martinez/scibert_scivocab_cased_ner_jnlpba/README.md delete mode 100644 model_cards/funnel-transformer/intermediate-base/README.md delete mode 100644 model_cards/funnel-transformer/intermediate/README.md delete mode 100644 model_cards/funnel-transformer/large-base/README.md delete mode 100644 model_cards/funnel-transformer/large/README.md delete mode 100644 model_cards/funnel-transformer/medium-base/README.md delete mode 100644 model_cards/funnel-transformer/medium/README.md delete mode 100644 model_cards/funnel-transformer/small-base/README.md delete mode 100644 model_cards/funnel-transformer/small/README.md delete mode 100644 model_cards/funnel-transformer/xlarge-base/README.md delete mode 100644 model_cards/funnel-transformer/xlarge/README.md delete mode 100644 model_cards/ganeshkharad/gk-hinglish-sentiment/README.md delete mode 100644 model_cards/gaochangkuan/model_dir/README.md delete mode 100644 model_cards/german-nlp-group/electra-base-german-uncased/README.md delete mode 100644 model_cards/giganticode/StackOBERTflow-comments-small-v1/README.md delete mode 100644 model_cards/gilf/french-camembert-postag-model/README.md delete mode 100644 model_cards/gilf/french-postag-model/README.md delete mode 100644 model_cards/google/bert2bert_L-24_wmt_de_en/README.md delete mode 100644 model_cards/google/bert2bert_L-24_wmt_en_de/README.md delete mode 120000 model_cards/google/bert_uncased_L-10_H-128_A-2/README.md delete mode 120000 model_cards/google/bert_uncased_L-10_H-256_A-4/README.md delete mode 120000 model_cards/google/bert_uncased_L-10_H-512_A-8/README.md delete mode 120000 model_cards/google/bert_uncased_L-10_H-768_A-12/README.md delete mode 120000 model_cards/google/bert_uncased_L-12_H-128_A-2/README.md delete mode 120000 model_cards/google/bert_uncased_L-12_H-256_A-4/README.md delete mode 120000 model_cards/google/bert_uncased_L-12_H-512_A-8/README.md delete mode 120000 model_cards/google/bert_uncased_L-12_H-768_A-12/README.md delete mode 120000 model_cards/google/bert_uncased_L-2_H-128_A-2/README.md delete mode 120000 model_cards/google/bert_uncased_L-2_H-256_A-4/README.md delete mode 120000 model_cards/google/bert_uncased_L-2_H-512_A-8/README.md delete mode 120000 model_cards/google/bert_uncased_L-2_H-768_A-12/README.md delete mode 120000 model_cards/google/bert_uncased_L-4_H-128_A-2/README.md delete mode 120000 model_cards/google/bert_uncased_L-4_H-256_A-4/README.md delete mode 120000 model_cards/google/bert_uncased_L-4_H-512_A-8/README.md delete mode 120000 model_cards/google/bert_uncased_L-4_H-768_A-12/README.md delete mode 120000 model_cards/google/bert_uncased_L-6_H-128_A-2/README.md delete mode 120000 model_cards/google/bert_uncased_L-6_H-256_A-4/README.md delete mode 120000 model_cards/google/bert_uncased_L-6_H-512_A-8/README.md delete mode 120000 model_cards/google/bert_uncased_L-6_H-768_A-12/README.md delete mode 120000 model_cards/google/bert_uncased_L-8_H-128_A-2/README.md delete mode 120000 model_cards/google/bert_uncased_L-8_H-256_A-4/README.md delete mode 120000 model_cards/google/bert_uncased_L-8_H-512_A-8/README.md delete mode 120000 model_cards/google/bert_uncased_L-8_H-768_A-12/README.md delete mode 100644 model_cards/google/electra-base-discriminator/README.md delete mode 100644 model_cards/google/electra-base-generator/README.md delete mode 100644 model_cards/google/electra-large-discriminator/README.md delete mode 100644 model_cards/google/electra-large-generator/README.md delete mode 100644 model_cards/google/electra-small-discriminator/README.md delete mode 100644 model_cards/google/electra-small-generator/README.md delete mode 100644 model_cards/google/mobilebert-uncased/README.md delete mode 100644 model_cards/google/reformer-crime-and-punishment/README.md delete mode 100644 model_cards/google/reformer-enwik8/README.md delete mode 100644 model_cards/google/roberta2roberta_L-24_bbc/README.md delete mode 100644 model_cards/google/roberta2roberta_L-24_cnn_daily_mail/README.md delete mode 100644 model_cards/google/roberta2roberta_L-24_discofuse/README.md delete mode 100644 model_cards/google/roberta2roberta_L-24_gigaword/README.md delete mode 100644 model_cards/google/roberta2roberta_L-24_wikisplit/README.md delete mode 100644 model_cards/gpt2-README.md delete mode 100644 model_cards/gpt2-large-README.md delete mode 100644 model_cards/gpt2-medium-README.md delete mode 100644 model_cards/gpt2-xl-README.md delete mode 100644 model_cards/gsarti/biobert-nli/README.md delete mode 100644 model_cards/gsarti/covidbert-nli/README.md delete mode 100644 model_cards/gsarti/scibert-nli/README.md delete mode 100644 model_cards/gurkan08/bert-turkish-text-classification/README.md delete mode 100644 model_cards/hatmimoha/arabic-ner/README.md delete mode 100644 model_cards/healx/gpt-2-pubmed-large/README.md delete mode 100644 model_cards/healx/gpt-2-pubmed-medium/README.md delete mode 100644 model_cards/henryk/bert-base-multilingual-cased-finetuned-dutch-squad2/README.md delete mode 100644 model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad1/README.md delete mode 100644 model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad2/README.md delete mode 100644 model_cards/huawei-noah/DynaBERT_MNLI/README.md delete mode 100644 model_cards/huawei-noah/DynaBERT_SST-2/README.md delete mode 100644 model_cards/huawei-noah/TinyBERT_General_4L_312D/README.md delete mode 100644 model_cards/huggingface/CodeBERTa-language-id/README.md delete mode 100644 model_cards/huggingface/CodeBERTa-small-v1/README.md delete mode 100644 model_cards/huseinzol05/albert-base-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/albert-tiny-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/bert-base-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/electra-base-discriminator-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/electra-base-generator-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/electra-small-discriminator-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/electra-small-generator-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/gpt2-117M-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/gpt2-345M-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/t5-base-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/t5-base-bahasa-summarization-cased/README.md delete mode 100644 model_cards/huseinzol05/t5-small-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/t5-small-bahasa-summarization-cased/README.md delete mode 100644 model_cards/huseinzol05/tiny-bert-bahasa-cased/README.md delete mode 100644 model_cards/huseinzol05/xlnet-base-bahasa-cased/README.md delete mode 100644 model_cards/iarfmoose/bert-base-cased-qa-evaluator/README.md delete mode 100644 model_cards/iarfmoose/roberta-base-bulgarian-pos/README.md delete mode 100644 model_cards/iarfmoose/roberta-base-bulgarian/README.md delete mode 100644 model_cards/iarfmoose/roberta-small-bulgarian-pos/README.md delete mode 100644 model_cards/iarfmoose/roberta-small-bulgarian/README.md delete mode 100644 model_cards/iarfmoose/t5-base-question-generator/README.md delete mode 100644 model_cards/illuin/camembert-base-fquad/README.md delete mode 100644 model_cards/illuin/camembert-large-fquad/README.md delete mode 100644 model_cards/illuin/lepetit/README.md delete mode 100644 model_cards/indobenchmark/indobert-base-p1/README.md delete mode 100644 model_cards/indobenchmark/indobert-base-p2/README.md delete mode 100644 model_cards/indobenchmark/indobert-large-p1/README.md delete mode 100644 model_cards/indobenchmark/indobert-large-p2/README.md delete mode 100644 model_cards/indobenchmark/indobert-lite-base-p1/README.md delete mode 100644 model_cards/indobenchmark/indobert-lite-base-p2/README.md delete mode 100644 model_cards/indobenchmark/indobert-lite-large-p1/README.md delete mode 100644 model_cards/indobenchmark/indobert-lite-large-p2/README.md delete mode 100644 model_cards/indolem/indobert-base-uncased/README.md delete mode 100644 model_cards/ipuneetrathore/bert-base-cased-finetuned-finBERT/README.md delete mode 100644 model_cards/iuliaturc/bert_uncased_L-2_H-128_A-2/README.md delete mode 100644 model_cards/ixa-ehu/berteus-base-cased/README.md delete mode 100644 model_cards/ixa-ehu/ixambert-base-cased/README.md delete mode 100644 model_cards/jannesg/bertsson/README.md delete mode 100644 model_cards/jannesg/takalane_afr_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_nbl_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_nso_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_sot_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_ssw_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_tsn_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_tso_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_ven_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_xho_roberta/README.md delete mode 100644 model_cards/jannesg/takalane_zul_roberta/README.md delete mode 100644 model_cards/jcblaise/bert-tagalog-base-cased-WWM/README.md delete mode 100644 model_cards/jcblaise/bert-tagalog-base-cased/README.md delete mode 100644 model_cards/jcblaise/bert-tagalog-base-uncased-WWM/README.md delete mode 100644 model_cards/jcblaise/bert-tagalog-base-uncased/README.md delete mode 100644 model_cards/jcblaise/distilbert-tagalog-base-cased/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-base-cased-discriminator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-base-cased-generator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-base-uncased-discriminator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-base-uncased-generator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-small-cased-discriminator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-small-cased-generator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-small-uncased-discriminator/README.md delete mode 100644 model_cards/jcblaise/electra-tagalog-small-uncased-generator/README.md delete mode 100644 model_cards/jimregan/BERTreach/README.md delete mode 100644 model_cards/jme-p/shrugging-grace-tweet-classifier/README.md delete mode 100644 model_cards/joeddav/bart-large-mnli-yahoo-answers/README.md delete mode 100644 model_cards/joeddav/xlm-roberta-large-xnli/README.md delete mode 100644 model_cards/jordimas/julibert/README.md delete mode 100644 model_cards/jplu/tf-camembert-base/README.md delete mode 100644 model_cards/jplu/tf-xlm-r-ner-40-lang/README.md delete mode 100644 model_cards/jplu/tf-xlm-roberta-base/README.md delete mode 100644 model_cards/jplu/tf-xlm-roberta-large/README.md delete mode 100644 model_cards/julien-c/EsperBERTo-small-pos/README.md delete mode 100644 model_cards/julien-c/EsperBERTo-small/README.md delete mode 100644 model_cards/julien-c/bert-xsmall-dummy/README.md delete mode 100644 model_cards/julien-c/dummy-unknown/README.md delete mode 100644 model_cards/keshan/SinhalaBERTo/README.md delete mode 100644 model_cards/kiri-ai/distiluse-base-multilingual-cased-et/README.md delete mode 100644 model_cards/krevas/finance-koelectra-base-discriminator/README.md delete mode 100644 model_cards/krevas/finance-koelectra-base-generator/README.md delete mode 100644 model_cards/krevas/finance-koelectra-small-discriminator/README.md delete mode 100644 model_cards/krevas/finance-koelectra-small-generator/README.md delete mode 100644 model_cards/ktrapeznikov/albert-xlarge-v2-squad-v2/README.md delete mode 100644 model_cards/ktrapeznikov/biobert_v1.1_pubmed_squad_v2/README.md delete mode 100644 model_cards/ktrapeznikov/gpt2-medium-topic-news/README.md delete mode 100644 model_cards/ktrapeznikov/scibert_scivocab_uncased_squad_v2/README.md delete mode 100644 model_cards/kuisailab/albert-base-arabic/README.md delete mode 100644 model_cards/kuisailab/albert-large-arabic/README.md delete mode 100644 model_cards/kuisailab/albert-xlarge-arabic/README.md delete mode 100644 model_cards/kuppuluri/telugu_bertu/README.md delete mode 100644 model_cards/kuppuluri/telugu_bertu_ner/README.md delete mode 100644 model_cards/kuppuluri/telugu_bertu_pos/README.md delete mode 100644 model_cards/kuppuluri/telugu_bertu_tydiqa/README.md delete mode 100644 model_cards/lanwuwei/GigaBERT-v3-Arabic-and-English/README.md delete mode 100644 model_cards/loodos/albert-base-turkish-uncased/README.md delete mode 100644 model_cards/loodos/bert-base-turkish-uncased/README.md delete mode 100644 model_cards/loodos/electra-base-turkish-64k-uncased-discriminator/README.md delete mode 100644 model_cards/loodos/electra-base-turkish-uncased-discriminator/README.md delete mode 100644 model_cards/loodos/electra-small-turkish-cased-discriminator/README.md delete mode 100644 model_cards/loodos/electra-small-turkish-uncased-discriminator/README.md delete mode 100644 model_cards/lordtt13/COVID-SciBERT/README.md delete mode 100644 model_cards/lordtt13/emo-mobilebert/README.md delete mode 100644 model_cards/lserinol/bert-turkish-question-answering/README.md delete mode 100644 model_cards/lvwerra/bert-imdb/README.md delete mode 100644 model_cards/lvwerra/gpt2-imdb-ctrl/README.md delete mode 100644 model_cards/lvwerra/gpt2-imdb-pos/README.md delete mode 100644 model_cards/lvwerra/gpt2-imdb/README.md delete mode 100644 model_cards/lvwerra/gpt2-medium-taboo/README.md delete mode 100644 model_cards/lysandre/arxiv-nlp/README.md delete mode 100644 model_cards/lysandre/arxiv/README.md delete mode 100644 model_cards/m3hrdadfi/albert-fa-base-v2/README.md delete mode 100644 model_cards/m3hrdadfi/bert2bert-fa-news-headline/README.md delete mode 100644 model_cards/m3hrdadfi/bert2bert-fa-wiki-summary/README.md delete mode 100644 model_cards/microsoft/DeBERTa-base/README.md delete mode 100644 model_cards/microsoft/DeBERTa-large/README.md delete mode 100644 model_cards/microsoft/DialoGPT-large/README.md delete mode 100644 model_cards/microsoft/DialoGPT-medium/README.md delete mode 100644 model_cards/microsoft/DialoGPT-small/README.md delete mode 100644 model_cards/microsoft/MiniLM-L12-H384-uncased/README.md delete mode 100644 model_cards/microsoft/Multilingual-MiniLM-L12-H384/README.md delete mode 100644 model_cards/microsoft/codebert-base-mlm/README.md delete mode 100644 model_cards/microsoft/codebert-base/README.md delete mode 100644 model_cards/microsoft/layoutlm-base-uncased/README.md delete mode 100644 model_cards/microsoft/layoutlm-large-uncased/README.md delete mode 100644 model_cards/microsoft/prophetnet-large-uncased-cnndm/README.md delete mode 100644 model_cards/microsoft/prophetnet-large-uncased-squad-qg/README.md delete mode 100644 model_cards/microsoft/prophetnet-large-uncased/README.md delete mode 100644 model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-ntg/README.md delete mode 100644 model_cards/microsoft/xprophetnet-large-wiki100-cased-xglue-qg/README.md delete mode 100644 model_cards/microsoft/xprophetnet-large-wiki100-cased/README.md delete mode 100644 model_cards/monilouise/ner_pt_br/README.md delete mode 100644 model_cards/monologg/koelectra-base-discriminator/README.md delete mode 100644 model_cards/monologg/koelectra-base-generator/README.md delete mode 100644 model_cards/monologg/koelectra-small-discriminator/README.md delete mode 100644 model_cards/monologg/koelectra-small-generator/README.md delete mode 100644 model_cards/monsoon-nlp/dv-wave/README.md delete mode 100644 model_cards/moumeneb1/flaubert-base-cased-ecology_crisis/README.md delete mode 100644 model_cards/mrm8488/CodeBERTaPy/README.md delete mode 100644 model_cards/mrm8488/GPT-2-finetuned-CORD19/README.md delete mode 100644 model_cards/mrm8488/GPT-2-finetuned-common_gen/README.md delete mode 100644 model_cards/mrm8488/GPT-2-finetuned-covid-bio-medrxiv/README.md delete mode 100644 model_cards/mrm8488/GuaPeTe-2-tiny/README.md delete mode 100644 model_cards/mrm8488/RoBERTinha/README.md delete mode 100644 model_cards/mrm8488/RoBasquERTa/README.md delete mode 100644 model_cards/mrm8488/RuPERTa-base-finetuned-ner/README.md delete mode 100644 model_cards/mrm8488/RuPERTa-base-finetuned-pawsx-es/README.md delete mode 100644 model_cards/mrm8488/RuPERTa-base-finetuned-pos/README.md delete mode 100644 model_cards/mrm8488/RuPERTa-base-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/RuPERTa-base-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/RuPERTa-base/README.md delete mode 100644 model_cards/mrm8488/TinyBERT-spanish-uncased-finetuned-ner/README.md delete mode 100644 model_cards/mrm8488/bert-base-german-dbmdz-cased-finetuned-pawsx-de/README.md delete mode 100644 model_cards/mrm8488/bert-base-german-finetuned-ler/README.md delete mode 100644 model_cards/mrm8488/bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md delete mode 100644 model_cards/mrm8488/bert-italian-finedtuned-squadv1-it-alfa/README.md delete mode 100644 model_cards/mrm8488/bert-medium-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/bert-mini-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization/README.md delete mode 100644 model_cards/mrm8488/bert-multi-cased-finedtuned-xquad-tydiqa-goldp/README.md delete mode 100644 model_cards/mrm8488/bert-multi-cased-finetuned-xquadv1/README.md delete mode 100644 model_cards/mrm8488/bert-multi-uncased-finetuned-xquadv1/README.md delete mode 100644 model_cards/mrm8488/bert-small-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/bert-small-finetuned-typo-detection/README.md delete mode 100644 model_cards/mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization/README.md delete mode 100644 model_cards/mrm8488/bert-spanish-cased-finetuned-ner/README.md delete mode 100644 model_cards/mrm8488/bert-spanish-cased-finetuned-pos-syntax/README.md delete mode 100644 model_cards/mrm8488/bert-spanish-cased-finetuned-pos/README.md delete mode 100644 model_cards/mrm8488/bert-tiny-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/bert-uncased-finetuned-qnli/README.md delete mode 100644 model_cards/mrm8488/camembert-base-finetuned-pawsx-fr/README.md delete mode 100644 model_cards/mrm8488/chEMBL_smiles_v1/README.md delete mode 100644 model_cards/mrm8488/codeBERTaJS/README.md delete mode 100644 model_cards/mrm8488/codebert-base-finetuned-detect-insecure-code/README.md delete mode 100644 model_cards/mrm8488/distilbert-base-multi-cased-finetuned-typo-detection/README.md delete mode 100644 model_cards/mrm8488/distilbert-multi-finetuned-for-xqua-on-tydiqa/README.md delete mode 100644 model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md delete mode 100644 model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md delete mode 100644 model_cards/mrm8488/electra-base-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/electra-small-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/electra-small-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/electricidad-base-discriminator/README.md delete mode 100644 model_cards/mrm8488/electricidad-base-finetuned-pawsx-es/README.md delete mode 100644 model_cards/mrm8488/electricidad-base-generator/README.md delete mode 100644 model_cards/mrm8488/electricidad-small-discriminator/README.md delete mode 100644 model_cards/mrm8488/electricidad-small-finetuned-squadv1-es/README.md delete mode 100644 model_cards/mrm8488/gpt2-finetuned-recipes-cooking/README.md delete mode 100644 model_cards/mrm8488/gpt2-finetuned-recipes-cooking_v2/README.md delete mode 100644 model_cards/mrm8488/gpt2-imdb-neg/README.md delete mode 100644 model_cards/mrm8488/gpt2-imdb-neutral/README.md delete mode 100644 model_cards/mrm8488/longformer-base-4096-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/mT5-small-finetuned-tydiqa-for-xqa/README.md delete mode 100644 model_cards/mrm8488/mobilebert-uncased-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/mobilebert-uncased-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/roberta-large-finetuned-wsc/README.md delete mode 100644 model_cards/mrm8488/spanbert-base-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/spanbert-base-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/spanbert-base-finetuned-tacred/README.md delete mode 100644 model_cards/mrm8488/spanbert-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/spanbert-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/spanbert-large-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/spanbert-large-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/spanbert-large-finetuned-tacred/README.md delete mode 100644 model_cards/mrm8488/squeezebert-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/squeezebert-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-break_data-question-retrieval/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-break_data/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-common_gen/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-e2m-intent/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-emotion/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-imdb-sentiment/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-qasc/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-quarel/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-quartz/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-question-generation-ap/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-sarcasm-twitter/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-span-sentiment-extraction/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-summarize-news/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-wikiSQL-sql-to-en/README.md delete mode 100644 model_cards/mrm8488/t5-base-finetuned-wikiSQL/README.md delete mode 100644 model_cards/mrm8488/t5-small-finetuned-emotion/README.md delete mode 100644 model_cards/mrm8488/t5-small-finetuned-imdb-sentiment/README.md delete mode 100644 model_cards/mrm8488/t5-small-finetuned-quora-for-paraphrasing/README.md delete mode 100644 model_cards/mrm8488/t5-small-finetuned-squadv1/README.md delete mode 100644 model_cards/mrm8488/t5-small-finetuned-squadv2/README.md delete mode 100644 model_cards/mrm8488/t5-small-finetuned-wikiSQL/README.md delete mode 100644 model_cards/mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it/README.md delete mode 100644 model_cards/mrm8488/xlm-multi-finetuned-xquadv1/README.md delete mode 100644 model_cards/mymusise/gpt2-medium-chinese/README.md delete mode 100644 model_cards/mys/electra-base-turkish-cased-ner/README.md delete mode 100644 model_cards/ncoop57/bart-base-code-summarizer-java-v0/README.md delete mode 100644 model_cards/neuralmind/bert-base-portuguese-cased/README.md delete mode 100644 model_cards/neuralmind/bert-large-portuguese-cased/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-bn-bert/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-bn-distilbert/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-bn-roberta/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-bn-xlmroberta/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-hi-bert/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-hi-distilbert/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-hi-roberta/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-hi-xlmroberta/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-te-bert/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-te-distilbert/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-te-roberta/README.md delete mode 100644 model_cards/neuralspace-reverie/indic-transformers-te-xlmroberta/README.md delete mode 100644 model_cards/neuraly/bert-base-italian-cased-sentiment/README.md delete mode 100644 model_cards/neurocode/IsRoBERTa/README.md delete mode 100644 model_cards/nghuyong/ernie-1.0/README.md delete mode 100644 model_cards/nghuyong/ernie-2.0-en/README.md delete mode 100644 model_cards/nghuyong/ernie-2.0-large-en/README.md delete mode 100644 model_cards/nghuyong/ernie-tiny/README.md delete mode 100644 model_cards/nikokons/gpt2-greek/README.md delete mode 100644 model_cards/nlpaueb/bert-base-greek-uncased-v1/README.md delete mode 100644 model_cards/nlpaueb/legal-bert-base-uncased/README.md delete mode 100644 model_cards/nlptown/bert-base-multilingual-uncased-sentiment/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-100M-1/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-100M-2/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-100M-3/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-10M-1/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-10M-2/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-10M-3/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-1B-1/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-1B-2/README.md delete mode 120000 model_cards/nyu-mll/roberta-base-1B-3/README.md delete mode 120000 model_cards/nyu-mll/roberta-med-small-1M-1/README.md delete mode 120000 model_cards/nyu-mll/roberta-med-small-1M-2/README.md delete mode 120000 model_cards/nyu-mll/roberta-med-small-1M-3/README.md delete mode 100644 model_cards/nyu-mll/roberta_1M_to_1B/README.md delete mode 100644 model_cards/oliverguhr/german-sentiment-bert/README.md delete mode 100644 model_cards/panggi/t5-base-indonesian-summarization-cased/README.md delete mode 100644 model_cards/panggi/t5-small-indonesian-summarization-cased/README.md delete mode 100644 model_cards/patrickvonplaten/bert2bert-cnn_dailymail-fp16/README.md delete mode 100644 model_cards/patrickvonplaten/bert2bert_cnn_daily_mail/README.md delete mode 100644 model_cards/patrickvonplaten/bert2gpt2-cnn_dailymail-fp16/README.md delete mode 100644 model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md delete mode 100644 model_cards/patrickvonplaten/roberta2roberta-cnn_dailymail-fp16/README.md delete mode 100644 model_cards/patrickvonplaten/roberta2roberta-share-cnn_dailymail-fp16/README.md delete mode 100644 model_cards/patrickvonplaten/roberta_shared_bbc_xsum/README.md delete mode 100644 model_cards/pdelobelle/robbert-v2-dutch-base/README.md delete mode 100644 model_cards/pedropei/question-intimacy/README.md delete mode 100644 model_cards/pierreguillou/gpt2-small-portuguese/README.md delete mode 100644 model_cards/pradhyra/AWSBlogBert/README.md delete mode 100644 model_cards/pranavpsv/gpt2-genre-story-generator/README.md delete mode 100644 model_cards/pvl/labse_bert/README.md delete mode 100644 model_cards/qarib/bert-base-qarib60_1790k/README.md delete mode 100644 model_cards/qarib/bert-base-qarib60_1970k/README.md delete mode 100644 model_cards/qarib/bert-base-qarib60_860k/README.md delete mode 100644 model_cards/ramsrigouthamg/t5_paraphraser/README.md delete mode 100644 model_cards/rdenadai/BR_BERTo/README.md delete mode 100644 model_cards/redewiedergabe/bert-base-historical-german-rw-cased/README.md delete mode 100644 model_cards/rjbownes/Magic-The-Generating/README.md delete mode 100644 model_cards/roberta-base-README.md delete mode 100644 model_cards/roberta-large-README.md delete mode 100644 model_cards/roberta-large-mnli-README.md delete mode 100644 model_cards/rohanrajpal/bert-base-codemixed-uncased-sentiment/README.md delete mode 100644 model_cards/rohanrajpal/bert-base-en-es-codemix-cased/README.md delete mode 100644 model_cards/rohanrajpal/bert-base-en-hi-codemix-cased/README.md delete mode 100644 model_cards/rohanrajpal/bert-base-multilingual-codemixed-cased-sentiment/README.md delete mode 100644 model_cards/sachaarbonel/bert-italian-cased-finetuned-pos/README.md delete mode 100644 model_cards/sagorsarker/bangla-bert-base/README.md delete mode 100644 model_cards/sagorsarker/bangla-bert-sentiment/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-hineng-lid-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-hineng-ner-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-hineng-pos-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-nepeng-lid-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-spaeng-lid-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-spaeng-ner-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-spaeng-pos-lince/README.md delete mode 100644 model_cards/sagorsarker/codeswitch-spaeng-sentiment-analysis-lince/README.md delete mode 100644 model_cards/sarahlintang/IndoBERT/README.md delete mode 100644 model_cards/sarnikowski/electra-small-discriminator-da-256-cased/README.md delete mode 100644 model_cards/savasy/bert-base-turkish-ner-cased/README.md delete mode 100644 model_cards/savasy/bert-base-turkish-sentiment-cased/README.md delete mode 100644 model_cards/savasy/bert-base-turkish-squad/README.md delete mode 100644 model_cards/savasy/bert-turkish-text-classification/README.md delete mode 100644 model_cards/schmidek/electra-small-cased/README.md delete mode 100644 model_cards/seiya/oubiobert-base-uncased/README.md delete mode 100644 model_cards/sentence-transformers/LaBSE/README.md delete mode 100644 model_cards/sentence-transformers/bert-base-nli-cls-token/README.md delete mode 100644 model_cards/sentence-transformers/bert-base-nli-max-tokens/README.md delete mode 100644 model_cards/sentence-transformers/bert-base-nli-mean-tokens/README.md delete mode 100644 model_cards/severinsimmler/literary-german-bert/README.md delete mode 100644 model_cards/severinsimmler/literary-german-bert/kfold.png delete mode 100644 model_cards/severinsimmler/literary-german-bert/prosa-jahre.png delete mode 100644 model_cards/seyonec/ChemBERTa-zinc-base-v1/README.md delete mode 100644 model_cards/shoarora/alectra-small-owt/README.md delete mode 100644 model_cards/shoarora/electra-small-owt/README.md delete mode 100644 model_cards/shrugging-grace/tweetclassifier/README.md delete mode 100644 model_cards/smanjil/German-MedBERT/README.md delete mode 100644 model_cards/spentaur/yelp/README.md delete mode 100644 model_cards/squeezebert/squeezebert-mnli-headless/README.md delete mode 100644 model_cards/squeezebert/squeezebert-mnli/README.md delete mode 100644 model_cards/squeezebert/squeezebert-uncased/README.md delete mode 100644 model_cards/stas/tiny-wmt19-en-de/README.md delete mode 100644 model_cards/stevhliu/astroGPT/README.md delete mode 100644 model_cards/surajp/RoBERTa-hindi-guj-san/README.md delete mode 100644 model_cards/surajp/SanBERTa/README.md delete mode 100644 model_cards/surajp/albert-base-sanskrit/README.md delete mode 100644 model_cards/t5-11b-README.md delete mode 100644 model_cards/t5-3b-README.md delete mode 100644 model_cards/t5-base-README.md delete mode 100644 model_cards/t5-large-README.md delete mode 100644 model_cards/t5-small-README.md delete mode 100644 model_cards/tartuNLP/EstBERT/README.md delete mode 100644 model_cards/tblard/tf-allocine/README.md delete mode 100644 model_cards/tuner007/pegasus_paraphrase/README.md delete mode 100644 model_cards/tuner007/pegasus_qa/README.md delete mode 100644 model_cards/tuner007/t5_abs_qa/README.md delete mode 100644 model_cards/twmkn9/albert-base-v2-squad2/README.md delete mode 100644 model_cards/twmkn9/bert-base-uncased-squad2/README.md delete mode 100644 model_cards/twmkn9/distilbert-base-uncased-squad2/README.md delete mode 100644 model_cards/twmkn9/distilroberta-base-squad2/README.md delete mode 100644 model_cards/uer/chinese_roberta_L-2_H-128/README.md delete mode 100644 model_cards/uer/gpt2-chinese-couplet/README.md delete mode 100644 model_cards/uer/gpt2-chinese-poem/README.md delete mode 100644 model_cards/uncnlp/lxmert-base-uncased/LICENSE delete mode 100644 model_cards/uncnlp/lxmert-base-uncased/README.md delete mode 100644 model_cards/uncnlp/lxmert-base-uncased/lxmert_model-1.jpg delete mode 100644 model_cards/unideeplearning/polibert_sa/README.md delete mode 100644 model_cards/urduhack/roberta-urdu-small/README.md delete mode 100644 model_cards/valhalla/bart-large-finetuned-squadv1/README.md delete mode 100644 model_cards/valhalla/distilbart-mnli-12-1/README.md delete mode 100644 model_cards/valhalla/distilbart-mnli-12-3/README.md delete mode 100644 model_cards/valhalla/distilbart-mnli-12-6/README.md delete mode 100644 model_cards/valhalla/distilbart-mnli-12-9/README.md delete mode 100644 model_cards/valhalla/electra-base-discriminator-finetuned_squadv1/README.md delete mode 100644 model_cards/valhalla/longformer-base-4096-finetuned-squadv1/README.md delete mode 100644 model_cards/valhalla/t5-base-e2e-qg/README.md delete mode 100644 model_cards/valhalla/t5-base-qa-qg-hl/README.md delete mode 100644 model_cards/valhalla/t5-base-qg-hl/README.md delete mode 100644 model_cards/valhalla/t5-base-squad/README.md delete mode 100644 model_cards/valhalla/t5-samll-qg-prepend/README.md delete mode 100644 model_cards/valhalla/t5-small-e2e-qg/README.md delete mode 100644 model_cards/valhalla/t5-small-qa-qg-hl/README.md delete mode 100644 model_cards/valhalla/t5-small-qg-hl/README.md delete mode 100644 model_cards/vinai/bertweet-base/README.md delete mode 100644 model_cards/vinai/bertweet-covid19-base-cased/README.md delete mode 100644 model_cards/vinai/bertweet-covid19-base-uncased/README.md delete mode 100644 model_cards/vinai/phobert-base/README.md delete mode 100644 model_cards/vinai/phobert-large/README.md delete mode 100644 model_cards/voidful/albert_chinese_base/README.md delete mode 100644 model_cards/voidful/albert_chinese_large/README.md delete mode 100644 model_cards/voidful/albert_chinese_small/README.md delete mode 100644 model_cards/voidful/albert_chinese_tiny/README.md delete mode 100644 model_cards/voidful/albert_chinese_xlarge/README.md delete mode 100644 model_cards/voidful/albert_chinese_xxlarge/README.md delete mode 100644 model_cards/wietsedv/bert-base-dutch-cased/README.md delete mode 100644 model_cards/wptoux/albert-chinese-large-qa/README.md delete mode 100644 model_cards/xlm-mlm-en-2048-README.md delete mode 100644 model_cards/xlm-roberta-base-README.md delete mode 100644 model_cards/xlm-roberta-large-finetuned-conll03-german-README.md delete mode 100644 model_cards/yjernite/bart_eli5/README.md delete mode 100644 model_cards/ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli/README.md delete mode 100644 model_cards/youscan/ukr-roberta-base/README.md delete mode 100644 model_cards/yuvraj/summarizer-cnndm/README.md delete mode 100644 model_cards/yuvraj/xSumm/README.md delete mode 100644 model_cards/zanelim/singbert-large-sg/README.md delete mode 100644 model_cards/zanelim/singbert-lite-sg/README.md delete mode 100644 model_cards/zanelim/singbert/README.md diff --git a/docs/source/model_sharing.rst b/docs/source/model_sharing.rst index d4b512d247..2adbd0c4da 100644 --- a/docs/source/model_sharing.rst +++ b/docs/source/model_sharing.rst @@ -60,7 +60,7 @@ Basic steps In order to upload a model, you'll need to first create a git repo. This repo will live on the model hub, allowing users to clone it and you (and your organization members) to push to it. -You can create a model repo directly from the website, `here `. +You can create a model repo **directly from `the /new page on the website `__.** Alternatively, you can use the ``transformers-cli``. The next steps describe that process: @@ -82,12 +82,12 @@ This creates a repo on the model hub, which can be cloned. .. code-block:: bash - git clone https://huggingface.co/username/your-model-name - # Make sure you have git-lfs installed # (https://git-lfs.github.com/) git lfs install + git clone https://huggingface.co/username/your-model-name + When you have your local clone of your repo and lfs installed, you can then add/remove from that clone as you would with any other git repo. @@ -98,8 +98,12 @@ with any other git repo. echo "hello" >> README.md git add . && git commit -m "Update from $USER" -We are intentionally not wrapping git too much, so as to stay intuitive and easy-to-use. +We are intentionally not wrapping git too much, so that you can go on with the workflow you're used to and the tools +you already know. +The only learning curve you might have compared to regular git is the one for git-lfs. The documentation at +`git-lfs.github.com `__ is decent, but we'll work on a tutorial with some tips and tricks +in the coming weeks! Make your model work on all frameworks ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -110,7 +114,7 @@ Make your model work on all frameworks You probably have your favorite framework, but so will other users! That's why it's best to upload your model with both PyTorch `and` TensorFlow checkpoints to make it easier to use (if you skip this step, users will still be able to load your model in another framework, but it will be slower, as it will have to be converted on the fly). Don't worry, it's -super easy to do (and in a future version, it will all be automatic). You will need to install both PyTorch and +super easy to do (and in a future version, it might all be automatic). You will need to install both PyTorch and TensorFlow for this step, but you don't need to worry about the GPU, so it should be very easy. Check the `TensorFlow installation page `__ and/or the `PyTorch installation page `__ to see how. @@ -192,7 +196,7 @@ status`` command: git add --all git status -Finally, the files should be comitted: +Finally, the files should be committed: .. code-block:: bash @@ -210,23 +214,20 @@ This will upload the folder containing the weights, tokenizer and configuration Add a model card ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To make sure everyone knows what your model can do, what its limitations and potential bias or ethetical -considerations, please add a README.md model card to the 🤗 Transformers repo under `model_cards/`. It should then be -placed in a subfolder with your username or organization, then another subfolder named like your model -(`awesome-name-you-picked`). Or just click on the "Create a model card on GitHub" button on the model page, it will get -you directly to the right location. If you need one, `here `__ is a model -card template (meta-suggestions are welcome). +To make sure everyone knows what your model can do, what its limitations, potential bias or ethical considerations are, +please add a README.md model card to your model repo. You can just create it, or there's also a convenient button +titled "Add a README.md" on your model page. A model card template can be found `here +`__ (meta-suggestions are welcome). model card template (meta-suggestions +are welcome). + +.. note:: + + Model cards used to live in the 🤗 Transformers repo under `model_cards/`, but for consistency and scalability we + migrated every model card from the repo to its corresponding huggingface.co model repo. If your model is fine-tuned from another model coming from the model hub (all 🤗 Transformers pretrained models do), don't forget to link to its model card so that people can fully trace how your model was built. -If you have never made a pull request to the 🤗 Transformers repo, look at the :doc:`contributing guide ` -to see the steps to follow. - -.. note:: - - You can also send your model card in the folder you uploaded with the CLI by placing it in a `README.md` file - inside `path/to/awesome-name-you-picked/`. Using your model ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -262,7 +263,8 @@ First you need to install `git-lfs` in the environment used by the notebook: sudo apt-get install git-lfs -Then you can use the :obj:`transformers-cli` to create your new repo: +Then you can use either create a repo directly from `huggingface.co `__ , or use the +:obj:`transformers-cli` to create it: .. code-block:: bash @@ -274,13 +276,14 @@ Once it's created, you can clone it and configure it (replace username by your u .. code-block:: bash + git lfs install + git clone https://username:password@huggingface.co/username/your-model-name # Alternatively if you have a token, # you can use it instead of your password git clone https://username:token@huggingface.co/username/your-model-name cd your-model-name - git lfs install git config --global user.email "email@example.com" # Tip: using the same email than for your huggingface.co account will link your commits to your profile git config --global user.name "Your name" diff --git a/model_cards/Cinnamon/electra-small-japanese-discriminator/README.md b/model_cards/Cinnamon/electra-small-japanese-discriminator/README.md deleted file mode 100644 index 6efb8cf4b8..0000000000 --- a/model_cards/Cinnamon/electra-small-japanese-discriminator/README.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -language: ja -license: apache-2.0 ---- - -## Japanese ELECTRA-small - -We provide a Japanese **ELECTRA-Small** model, as described in [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -Our pretraining process employs subword units derived from the [Japanese Wikipedia](https://dumps.wikimedia.org/jawiki/latest), using the [Byte-Pair Encoding](https://www.aclweb.org/anthology/P16-1162.pdf) method and building on an initial tokenization with [mecab-ipadic-NEologd](https://github.com/neologd/mecab-ipadic-neologd). For optimal performance, please take care to set your MeCab dictionary appropriately. - -## How to use the discriminator in `transformers` - -``` -from transformers import BertJapaneseTokenizer, ElectraForPreTraining - -tokenizer = BertJapaneseTokenizer.from_pretrained('Cinnamon/electra-small-japanese-discriminator', mecab_kwargs={"mecab_option": "-d /usr/lib/x86_64-linux-gnu/mecab/dic/mecab-ipadic-neologd"}) - -model = ElectraForPreTraining.from_pretrained('Cinnamon/electra-small-japanese-discriminator') -``` diff --git a/model_cards/Cinnamon/electra-small-japanese-generator/README.md b/model_cards/Cinnamon/electra-small-japanese-generator/README.md deleted file mode 100644 index 0e014328e7..0000000000 --- a/model_cards/Cinnamon/electra-small-japanese-generator/README.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -language: ja ---- -## Japanese ELECTRA-small - -We provide a Japanese **ELECTRA-Small** model, as described in [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -Our pretraining process employs subword units derived from the [Japanese Wikipedia](https://dumps.wikimedia.org/jawiki/latest), using the [Byte-Pair Encoding](https://www.aclweb.org/anthology/P16-1162.pdf) method and building on an initial tokenization with [mecab-ipadic-NEologd](https://github.com/neologd/mecab-ipadic-neologd). For optimal performance, please take care to set your MeCab dictionary appropriately. - -``` -# ELECTRA-small generator usage - -from transformers import BertJapaneseTokenizer, ElectraForMaskedLM - -tokenizer = BertJapaneseTokenizer.from_pretrained('Cinnamon/electra-small-japanese-generator', mecab_kwargs={"mecab_option": "-d /usr/lib/x86_64-linux-gnu/mecab/dic/mecab-ipadic-neologd"}) - -model = ElectraForMaskedLM.from_pretrained('Cinnamon/electra-small-japanese-generator') -``` diff --git a/model_cards/DJSammy/bert-base-danish-uncased_BotXO,ai/README.md b/model_cards/DJSammy/bert-base-danish-uncased_BotXO,ai/README.md deleted file mode 100644 index 58d8a49f5b..0000000000 --- a/model_cards/DJSammy/bert-base-danish-uncased_BotXO,ai/README.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -language: da -tags: -- bert -- masked-lm -license: cc-by-4.0 -datasets: -- common_crawl -- wikipedia -pipeline_tag: fill-mask -widget: -- text: "København er [MASK] i Danmark." ---- - -# Danish BERT (uncased) model - -[BotXO.ai](https://www.botxo.ai/) developed this model. For data and training details see their [GitHub repository](https://github.com/botxo/nordic_bert). - -The original model was trained in TensorFlow then I converted it to Pytorch using [transformers-cli](https://huggingface.co/transformers/converting_tensorflow_models.html?highlight=cli). - -For TensorFlow version download here: https://www.dropbox.com/s/19cjaoqvv2jicq9/danish_bert_uncased_v2.zip?dl=1 - - -## Architecture - -```python -from transformers import AutoModelForPreTraining - -model = AutoModelForPreTraining.from_pretrained("DJSammy/bert-base-danish-uncased_BotXO,ai") - -params = list(model.named_parameters()) -print('danish_bert_uncased_v2 has {:} different named parameters.\n'.format(len(params))) - -print('==== Embedding Layer ====\n') -for p in params[0:5]: - print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size())))) - -print('\n==== First Transformer ====\n') -for p in params[5:21]: - print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size())))) - -print('\n==== Last Transformer ====\n') -for p in params[181:197]: - print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size())))) - -print('\n==== Output Layer ====\n') -for p in params[197:]: - print("{:<55} {:>12}".format(p[0], str(tuple(p[1].size())))) - -# danish_bert_uncased_v2 has 206 different named parameters. - -# ==== Embedding Layer ==== - -# bert.embeddings.word_embeddings.weight (32000, 768) -# bert.embeddings.position_embeddings.weight (512, 768) -# bert.embeddings.token_type_embeddings.weight (2, 768) -# bert.embeddings.LayerNorm.weight (768,) -# bert.embeddings.LayerNorm.bias (768,) - -# ==== First Transformer ==== - -# bert.encoder.layer.0.attention.self.query.weight (768, 768) -# bert.encoder.layer.0.attention.self.query.bias (768,) -# bert.encoder.layer.0.attention.self.key.weight (768, 768) -# bert.encoder.layer.0.attention.self.key.bias (768,) -# bert.encoder.layer.0.attention.self.value.weight (768, 768) -# bert.encoder.layer.0.attention.self.value.bias (768,) -# bert.encoder.layer.0.attention.output.dense.weight (768, 768) -# bert.encoder.layer.0.attention.output.dense.bias (768,) -# bert.encoder.layer.0.attention.output.LayerNorm.weight (768,) -# bert.encoder.layer.0.attention.output.LayerNorm.bias (768,) -# bert.encoder.layer.0.intermediate.dense.weight (3072, 768) -# bert.encoder.layer.0.intermediate.dense.bias (3072,) -# bert.encoder.layer.0.output.dense.weight (768, 3072) -# bert.encoder.layer.0.output.dense.bias (768,) -# bert.encoder.layer.0.output.LayerNorm.weight (768,) -# bert.encoder.layer.0.output.LayerNorm.bias (768,) - -# ==== Last Transformer ==== - -# bert.encoder.layer.11.attention.self.query.weight (768, 768) -# bert.encoder.layer.11.attention.self.query.bias (768,) -# bert.encoder.layer.11.attention.self.key.weight (768, 768) -# bert.encoder.layer.11.attention.self.key.bias (768,) -# bert.encoder.layer.11.attention.self.value.weight (768, 768) -# bert.encoder.layer.11.attention.self.value.bias (768,) -# bert.encoder.layer.11.attention.output.dense.weight (768, 768) -# bert.encoder.layer.11.attention.output.dense.bias (768,) -# bert.encoder.layer.11.attention.output.LayerNorm.weight (768,) -# bert.encoder.layer.11.attention.output.LayerNorm.bias (768,) -# bert.encoder.layer.11.intermediate.dense.weight (3072, 768) -# bert.encoder.layer.11.intermediate.dense.bias (3072,) -# bert.encoder.layer.11.output.dense.weight (768, 3072) -# bert.encoder.layer.11.output.dense.bias (768,) -# bert.encoder.layer.11.output.LayerNorm.weight (768,) -# bert.encoder.layer.11.output.LayerNorm.bias (768,) - -# ==== Output Layer ==== - -# bert.pooler.dense.weight (768, 768) -# bert.pooler.dense.bias (768,) -# cls.predictions.bias (32000,) -# cls.predictions.transform.dense.weight (768, 768) -# cls.predictions.transform.dense.bias (768,) -# cls.predictions.transform.LayerNorm.weight (768,) -# cls.predictions.transform.LayerNorm.bias (768,) -# cls.seq_relationship.weight (2, 768) -# cls.seq_relationship.bias (2,) -``` - -## Example Pipeline - -```python -from transformers import pipeline -unmasker = pipeline('fill-mask', model='DJSammy/bert-base-danish-uncased_BotXO,ai') - -unmasker('København er [MASK] i Danmark.') - -# Copenhagen is the [MASK] of Denmark. -# => - -# [{'score': 0.788068950176239, -# 'sequence': '[CLS] københavn er hovedstad i danmark. [SEP]', -# 'token': 12610, -# 'token_str': 'hovedstad'}, -# {'score': 0.07606703042984009, -# 'sequence': '[CLS] københavn er hovedstaden i danmark. [SEP]', -# 'token': 8108, -# 'token_str': 'hovedstaden'}, -# {'score': 0.04299738258123398, -# 'sequence': '[CLS] københavn er metropol i danmark. [SEP]', -# 'token': 23305, -# 'token_str': 'metropol'}, -# {'score': 0.008163209073245525, -# 'sequence': '[CLS] københavn er ikke i danmark. [SEP]', -# 'token': 89, -# 'token_str': 'ikke'}, -# {'score': 0.006238455418497324, -# 'sequence': '[CLS] københavn er ogsa i danmark. [SEP]', -# 'token': 25253, -# 'token_str': 'ogsa'}] -``` diff --git a/model_cards/DeepPavlov/bert-base-bg-cs-pl-ru-cased/README.md b/model_cards/DeepPavlov/bert-base-bg-cs-pl-ru-cased/README.md deleted file mode 100644 index b4ea3db9f6..0000000000 --- a/model_cards/DeepPavlov/bert-base-bg-cs-pl-ru-cased/README.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -language: -- bg -- cs -- pl -- ru ---- - -# bert-base-bg-cs-pl-ru-cased - -SlavicBERT\[1\] \(Slavic \(bg, cs, pl, ru\), cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on Russian News and four Wikipedias: Bulgarian, Czech, Polish, and Russian. Subtoken vocabulary was built using this data. Multilingual BERT was used as an initialization for SlavicBERT. - - -\[1\]: Arkhipov M., Trofimova M., Kuratov Y., Sorokin A. \(2019\). [Tuning Multilingual Transformers for Language-Specific Named Entity Recognition](https://www.aclweb.org/anthology/W19-3712/). ACL anthology W19-3712. diff --git a/model_cards/DeepPavlov/bert-base-cased-conversational/README.md b/model_cards/DeepPavlov/bert-base-cased-conversational/README.md deleted file mode 100644 index 7bb9231d86..0000000000 --- a/model_cards/DeepPavlov/bert-base-cased-conversational/README.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -language: en ---- - -# bert-base-cased-conversational - -Conversational BERT \(English, cased, 12‑layer, 768‑hidden, 12‑heads, 110M parameters\) was trained on the English part of Twitter, Reddit, DailyDialogues\[1\], OpenSubtitles\[2\], Debates\[3\], Blogs\[4\], Facebook News Comments. We used this training data to build the vocabulary of English subtokens and took English cased version of BERT‑base as an initialization for English Conversational BERT. - - -\[1\]: Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. IJCNLP 2017. - -\[2\]: P. Lison and J. Tiedemann, 2016, OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the 10th International Conference on Language Resources and Evaluation \(LREC 2016\) - -\[3\]: Justine Zhang, Ravi Kumar, Sujith Ravi, Cristian Danescu-Niculescu-Mizil. Proceedings of NAACL, 2016. - -\[4\]: J. Schler, M. Koppel, S. Argamon and J. Pennebaker \(2006\). Effects of Age and Gender on Blogging in Proceedings of 2006 AAAI Spring Symposium on Computational Approaches for Analyzing Weblogs. diff --git a/model_cards/DeepPavlov/bert-base-multilingual-cased-sentence/README.md b/model_cards/DeepPavlov/bert-base-multilingual-cased-sentence/README.md deleted file mode 100644 index e8d22dff30..0000000000 --- a/model_cards/DeepPavlov/bert-base-multilingual-cased-sentence/README.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -language: -- multilingual ---- - -# bert-base-multilingual-cased-sentence - -Sentence Multilingual BERT \(101 languages, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) is a representation‑based sentence encoder for 101 languages of Multilingual BERT. It is initialized with Multilingual BERT and then fine‑tuned on english MultiNLI\[1\] and on dev set of multilingual XNLI\[2\]. Sentence representations are mean pooled token embeddings in the same manner as in Sentence‑BERT\[3\]. - - -\[1\]: Williams A., Nangia N. & Bowman S. \(2017\) A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. arXiv preprint [arXiv:1704.05426](https://arxiv.org/abs/1704.05426) - -\[2\]: Williams A., Bowman S. \(2018\) XNLI: Evaluating Cross-lingual Sentence Representations. arXiv preprint [arXiv:1809.05053](https://arxiv.org/abs/1809.05053) - -\[3\]: N. Reimers, I. Gurevych \(2019\) Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint [arXiv:1908.10084](https://arxiv.org/abs/1908.10084) diff --git a/model_cards/DeepPavlov/rubert-base-cased-conversational/README.md b/model_cards/DeepPavlov/rubert-base-cased-conversational/README.md deleted file mode 100644 index ac0fe25269..0000000000 --- a/model_cards/DeepPavlov/rubert-base-cased-conversational/README.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -language: -- ru ---- - -# rubert-base-cased-conversational - -Conversational RuBERT \(Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on OpenSubtitles\[1\], [Dirty](https://d3.ru/), [Pikabu](https://pikabu.ru/), and a Social Media segment of Taiga corpus\[2\]. We assembled a new vocabulary for Conversational RuBERT model on this data and initialized the model with [RuBERT](../rubert-base-cased). - - -\[1\]: P. Lison and J. Tiedemann, 2016, OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the 10th International Conference on Language Resources and Evaluation \(LREC 2016\) - -\[2\]: Shavrina T., Shapovalova O. \(2017\) TO THE METHODOLOGY OF CORPUS CONSTRUCTION FOR MACHINE LEARNING: «TAIGA» SYNTAX TREE CORPUS AND PARSER. in proc. of “CORPORA2017”, international conference , Saint-Petersbourg, 2017. diff --git a/model_cards/DeepPavlov/rubert-base-cased-sentence/README.md b/model_cards/DeepPavlov/rubert-base-cased-sentence/README.md deleted file mode 100644 index f57d8715f0..0000000000 --- a/model_cards/DeepPavlov/rubert-base-cased-sentence/README.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -language: -- ru ---- - -# rubert-base-cased-sentence - -Sentence RuBERT \(Russian, cased, 12-layer, 768-hidden, 12-heads, 180M parameters\) is a representation‑based sentence encoder for Russian. It is initialized with RuBERT and fine‑tuned on SNLI\[1\] google-translated to russian and on russian part of XNLI dev set\[2\]. Sentence representations are mean pooled token embeddings in the same manner as in Sentence‑BERT\[3\]. - - -\[1\]: S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. \(2015\) A large annotated corpus for learning natural language inference. arXiv preprint [arXiv:1508.05326](https://arxiv.org/abs/1508.05326) - -\[2\]: Williams A., Bowman S. \(2018\) XNLI: Evaluating Cross-lingual Sentence Representations. arXiv preprint [arXiv:1809.05053](https://arxiv.org/abs/1809.05053) - -\[3\]: N. Reimers, I. Gurevych \(2019\) Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint [arXiv:1908.10084](https://arxiv.org/abs/1908.10084) diff --git a/model_cards/DeepPavlov/rubert-base-cased/README.md b/model_cards/DeepPavlov/rubert-base-cased/README.md deleted file mode 100644 index a686a92da4..0000000000 --- a/model_cards/DeepPavlov/rubert-base-cased/README.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -language: -- ru ---- - -# rubert-base-cased - -RuBERT \(Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters\) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT‑base as an initialization for RuBERT\[1\]. - - -\[1\]: Kuratov, Y., Arkhipov, M. \(2019\). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint [arXiv:1905.07213](https://arxiv.org/abs/1905.07213). diff --git a/model_cards/Geotrend/bert-base-15lang-cased/README.md b/model_cards/Geotrend/bert-base-15lang-cased/README.md deleted file mode 100644 index c9cbd4d742..0000000000 --- a/model_cards/Geotrend/bert-base-15lang-cased/README.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." -- text: "Paris est la [MASK] de la France." -- text: "Paris est la capitale de la [MASK]." -- text: "L'élection américaine a eu [MASK] en novembre 2020." -- text: "تقع سويسرا في [MASK] أوروبا" -- text: "إسمي محمد وأسكن في [MASK]." ---- - -# bert-base-15lang-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -The measurements below have been computed on a [Google Cloud n1-standard-1 machine (1 vCPU, 3.75 GB)](https://cloud.google.com/compute/docs/machine-types\#n1_machine_type): - -| Model | Num parameters | Size | Memory | Loading time | -| ------------------------------- | -------------- | -------- | -------- | ------------ | -| bert-base-multilingual-cased | 178 million | 714 MB | 1400 MB | 4.2 sec | -| Geotrend/bert-base-15lang-cased | 141 million | 564 MB | 1098 MB | 3.1 sec | - -Handled languages: en, fr, es, de, zh, ar, ru, vi, el, bg, th, tr, hi, ur and sw. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-15lang-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-15lang-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-ar-cased/README.md b/model_cards/Geotrend/bert-base-ar-cased/README.md deleted file mode 100644 index 2fe73d3fa3..0000000000 --- a/model_cards/Geotrend/bert-base-ar-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: ar - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "تقع سويسرا في [MASK] أوروبا" -- text: "إسمي محمد وأسكن في [MASK]." ---- - -# bert-base-ar-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-ar-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-ar-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-bg-cased/README.md b/model_cards/Geotrend/bert-base-bg-cased/README.md deleted file mode 100644 index 3c75e3cbf2..0000000000 --- a/model_cards/Geotrend/bert-base-bg-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: bg - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-bg-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-bg-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-bg-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-de-cased/README.md b/model_cards/Geotrend/bert-base-de-cased/README.md deleted file mode 100644 index a8b5d7d21f..0000000000 --- a/model_cards/Geotrend/bert-base-de-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: de - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-de-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-de-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-de-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-el-cased/README.md b/model_cards/Geotrend/bert-base-el-cased/README.md deleted file mode 100644 index f567e0f323..0000000000 --- a/model_cards/Geotrend/bert-base-el-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: el - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-el-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-el-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-el-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-ar-cased/README.md b/model_cards/Geotrend/bert-base-en-ar-cased/README.md deleted file mode 100644 index cb10545810..0000000000 --- a/model_cards/Geotrend/bert-base-en-ar-cased/README.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." -- text: "تقع سويسرا في [MASK] أوروبا" -- text: "إسمي محمد وأسكن في [MASK]." ---- - -# bert-base-en-ar-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ar-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-ar-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-bg-cased/README.md b/model_cards/Geotrend/bert-base-en-bg-cased/README.md deleted file mode 100644 index cb248bea4a..0000000000 --- a/model_cards/Geotrend/bert-base-en-bg-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-bg-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-bg-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-bg-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-cased/README.md b/model_cards/Geotrend/bert-base-en-cased/README.md deleted file mode 100644 index 5121ad5fec..0000000000 --- a/model_cards/Geotrend/bert-base-en-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: en - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-de-cased/README.md b/model_cards/Geotrend/bert-base-en-de-cased/README.md deleted file mode 100644 index 32eced8b13..0000000000 --- a/model_cards/Geotrend/bert-base-en-de-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-de-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-de-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-de-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-el-cased/README.md b/model_cards/Geotrend/bert-base-en-el-cased/README.md deleted file mode 100644 index d87f088f96..0000000000 --- a/model_cards/Geotrend/bert-base-en-el-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-el-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-el-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-el-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-es-cased/README.md b/model_cards/Geotrend/bert-base-en-es-cased/README.md deleted file mode 100644 index faae4283e8..0000000000 --- a/model_cards/Geotrend/bert-base-en-es-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-es-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-es-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-es-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-fr-cased/README.md b/model_cards/Geotrend/bert-base-en-fr-cased/README.md deleted file mode 100644 index 43d32d4076..0000000000 --- a/model_cards/Geotrend/bert-base-en-fr-cased/README.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." -- text: "Paris est la [MASK] de la France." -- text: "Paris est la capitale de la [MASK]." -- text: "L'élection américaine a eu [MASK] en novembre 2020." ---- - -# bert-base-en-fr-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-fr-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-fr-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-hi-cased/README.md b/model_cards/Geotrend/bert-base-en-hi-cased/README.md deleted file mode 100644 index b9d1c0fe66..0000000000 --- a/model_cards/Geotrend/bert-base-en-hi-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-hi-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-hi-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-hi-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-ru-cased/README.md b/model_cards/Geotrend/bert-base-en-ru-cased/README.md deleted file mode 100644 index 1c36b7d4be..0000000000 --- a/model_cards/Geotrend/bert-base-en-ru-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-ru-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ru-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-ru-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-sw-cased/README.md b/model_cards/Geotrend/bert-base-en-sw-cased/README.md deleted file mode 100644 index b17f65b330..0000000000 --- a/model_cards/Geotrend/bert-base-en-sw-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-sw-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-sw-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-sw-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-th-cased/README.md b/model_cards/Geotrend/bert-base-en-th-cased/README.md deleted file mode 100644 index 7a358458a9..0000000000 --- a/model_cards/Geotrend/bert-base-en-th-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-th-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-th-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-th-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-tr-cased/README.md b/model_cards/Geotrend/bert-base-en-tr-cased/README.md deleted file mode 100644 index 4c640a7761..0000000000 --- a/model_cards/Geotrend/bert-base-en-tr-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-tr-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-tr-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-tr-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-ur-cased/README.md b/model_cards/Geotrend/bert-base-en-ur-cased/README.md deleted file mode 100644 index 938ce86861..0000000000 --- a/model_cards/Geotrend/bert-base-en-ur-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-ur-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-ur-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-ur-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-vi-cased/README.md b/model_cards/Geotrend/bert-base-en-vi-cased/README.md deleted file mode 100644 index 64eb6bdf0c..0000000000 --- a/model_cards/Geotrend/bert-base-en-vi-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-vi-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-vi-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-vi-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-en-zh-cased/README.md b/model_cards/Geotrend/bert-base-en-zh-cased/README.md deleted file mode 100644 index 12c0b2b171..0000000000 --- a/model_cards/Geotrend/bert-base-en-zh-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: multilingual - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Google generated 46 billion [MASK] in revenue." -- text: "Paris is the capital of [MASK]." -- text: "Algiers is the largest city in [MASK]." ---- - -# bert-base-en-zh-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-en-zh-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-en-zh-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-es-cased/README.md b/model_cards/Geotrend/bert-base-es-cased/README.md deleted file mode 100644 index 90bc84de96..0000000000 --- a/model_cards/Geotrend/bert-base-es-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: es - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-es-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-es-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-es-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-fr-cased/README.md b/model_cards/Geotrend/bert-base-fr-cased/README.md deleted file mode 100644 index 7f1b30f77c..0000000000 --- a/model_cards/Geotrend/bert-base-fr-cased/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: fr - -datasets: wikipedia - -license: apache-2.0 - -widget: -- text: "Paris est la [MASK] de la France." -- text: "Paris est la capitale de la [MASK]." -- text: "L'élection américaine a eu [MASK] en novembre 2020." ---- - -# bert-base-fr-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-fr-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-fr-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-hi-cased/README.md b/model_cards/Geotrend/bert-base-hi-cased/README.md deleted file mode 100644 index 03f83e7476..0000000000 --- a/model_cards/Geotrend/bert-base-hi-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: hi - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-hi-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-hi-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-hi-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-ru-cased/README.md b/model_cards/Geotrend/bert-base-ru-cased/README.md deleted file mode 100644 index 9e19b56596..0000000000 --- a/model_cards/Geotrend/bert-base-ru-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: ru - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-ru-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-ru-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-ru-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-sw-cased/README.md b/model_cards/Geotrend/bert-base-sw-cased/README.md deleted file mode 100644 index 03f4d48d63..0000000000 --- a/model_cards/Geotrend/bert-base-sw-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: sw - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-sw-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-sw-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-sw-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-th-cased/README.md b/model_cards/Geotrend/bert-base-th-cased/README.md deleted file mode 100644 index 822bbcdd69..0000000000 --- a/model_cards/Geotrend/bert-base-th-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: th - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-th-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-th-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-th-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-tr-cased/README.md b/model_cards/Geotrend/bert-base-tr-cased/README.md deleted file mode 100644 index 4aad67379e..0000000000 --- a/model_cards/Geotrend/bert-base-tr-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: tr - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-tr-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-tr-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-tr-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-ur-cased/README.md b/model_cards/Geotrend/bert-base-ur-cased/README.md deleted file mode 100644 index bdbdb0b18e..0000000000 --- a/model_cards/Geotrend/bert-base-ur-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: ur - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-ur-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-ur-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-ur-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-vi-cased/README.md b/model_cards/Geotrend/bert-base-vi-cased/README.md deleted file mode 100644 index 3bdcad5cc1..0000000000 --- a/model_cards/Geotrend/bert-base-vi-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: vi - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-vi-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-vi-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-vi-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Geotrend/bert-base-zh-cased/README.md b/model_cards/Geotrend/bert-base-zh-cased/README.md deleted file mode 100644 index 0f620c1d7f..0000000000 --- a/model_cards/Geotrend/bert-base-zh-cased/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: zh - -datasets: wikipedia - -license: apache-2.0 ---- - -# bert-base-zh-cased - -We are sharing smaller versions of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handle a custom number of languages. - -Unlike [distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased), our versions give exactly the same representations produced by the original model which preserves the original accuracy. - -For more information please visit our paper: [Load What You Need: Smaller Versions of Multilingual BERT](https://www.aclweb.org/anthology/2020.sustainlp-1.16.pdf). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Geotrend/bert-base-zh-cased") -model = AutoModel.from_pretrained("Geotrend/bert-base-zh-cased") - -``` - -To generate other smaller versions of multilingual transformers please visit [our Github repo](https://github.com/Geotrend-research/smaller-transformers). - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-arabic/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-arabic/README.md deleted file mode 100644 index 4234735d79..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-arabic/README.md +++ /dev/null @@ -1,18 +0,0 @@ -This model is used detecting **hatespeech** in **Arabic language**. The mono in the name refers to the monolingual setting, where the model is trained using only Arabic language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.877609 for a learning rate of 2e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-english/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-english/README.md deleted file mode 100644 index eff7538892..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-english/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **English language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.726030 for a learning rate of 2e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-french/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-french/README.md deleted file mode 100644 index 703b3acaf7..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-french/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **French language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.692094 for a learning rate of 3e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-german/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-german/README.md deleted file mode 100644 index 306bfdd42b..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-german/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **German language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.649794 for a learning rate of 3e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-indonesian/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-indonesian/README.md deleted file mode 100644 index d40ac4c475..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-indonesian/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **Indonesian language**. The mono in the name refers to the monolingual setting, where the model is trained using only Arabic language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.844494 for a learning rate of 2e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-italian/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-italian/README.md deleted file mode 100644 index 0ffe017e6e..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-italian/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **Italian language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.837288 for a learning rate of 3e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-polish/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-polish/README.md deleted file mode 100644 index fd0852b0e5..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-polish/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **Polish language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.723254 for a learning rate of 2e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-portugese/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-portugese/README.md deleted file mode 100644 index 4859c10c62..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-portugese/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **Portuguese language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.716119 for a learning rate of 3e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/Hate-speech-CNERG/dehatebert-mono-spanish/README.md b/model_cards/Hate-speech-CNERG/dehatebert-mono-spanish/README.md deleted file mode 100644 index 8a8e697131..0000000000 --- a/model_cards/Hate-speech-CNERG/dehatebert-mono-spanish/README.md +++ /dev/null @@ -1,20 +0,0 @@ -This model is used detecting **hatespeech** in **Spanish language**. The mono in the name refers to the monolingual setting, where the model is trained using only English language data. It is finetuned on multilingual bert model. -The model is trained with different learning rates and the best validation score achieved is 0.740287 for a learning rate of 3e-5. Training code can be found at this [url](https://github.com/punyajoy/DE-LIMIT) - - - -### For more details about our paper - -Sai Saketh Aluru, Binny Mathew, Punyajoy Saha and Animesh Mukherjee. "[Deep Learning Models for Multilingual Hate Speech Detection](https://arxiv.org/abs/2004.06465)". Accepted at ECML-PKDD 2020. - -***Please cite our paper in any published work that uses any of these resources.*** - -~~~ -@article{aluru2020deep, - title={Deep Learning Models for Multilingual Hate Speech Detection}, - author={Aluru, Sai Saket and Mathew, Binny and Saha, Punyajoy and Mukherjee, Animesh}, - journal={arXiv preprint arXiv:2004.06465}, - year={2020} -} - -~~~ diff --git a/model_cards/HooshvareLab/bert-base-parsbert-armanner-uncased/README.md b/model_cards/HooshvareLab/bert-base-parsbert-armanner-uncased/README.md deleted file mode 100644 index 80112e431d..0000000000 --- a/model_cards/HooshvareLab/bert-base-parsbert-armanner-uncased/README.md +++ /dev/null @@ -1,124 +0,0 @@ -## ParsBERT: Transformer-based Model for Persian Language Understanding - -ParsBERT is a monolingual language model based on Google’s BERT architecture with the same configurations as BERT-Base. - -Paper presenting ParsBERT: [arXiv:2005.12515](https://arxiv.org/abs/2005.12515) - -All the models (downstream tasks) are uncased and trained with whole word masking. (coming soon stay tuned) - - -## Persian NER [ARMAN, PEYMA, ARMAN+PEYMA] - -This task aims to extract named entities in the text, such as names and label with appropriate `NER` classes such as locations, organizations, etc. The datasets used for this task contain sentences that are marked with `IOB` format. In this format, tokens that are not part of an entity are tagged as `”O”` the `”B”`tag corresponds to the first word of an object, and the `”I”` tag corresponds to the rest of the terms of the same entity. Both `”B”` and `”I”` tags are followed by a hyphen (or underscore), followed by the entity category. Therefore, the NER task is a multi-class token classification problem that labels the tokens upon being fed a raw text. There are two primary datasets used in Persian NER, `ARMAN`, and `PEYMA`. In ParsBERT, we prepared ner for both datasets as well as a combination of both datasets. - - - -### PEYMA - -PEYMA dataset includes 7,145 sentences with a total of 302,530 tokens from which 41,148 tokens are tagged with seven different classes. - -1. Organization -2. Money -3. Location -4. Date -5. Time -6. Person -7. Percent - - -| Label | # | -|:------------:|:-----:| -| Organization | 16964 | -| Money | 2037 | -| Location | 8782 | -| Date | 4259 | -| Time | 732 | -| Person | 7675 | -| Percent | 699 | - - - -**Download** -You can download the dataset from [here](http://nsurl.org/tasks/task-7-named-entity-recognition-ner-for-farsi/) - ---- - -### ARMAN - -ARMAN dataset holds 7,682 sentences with 250,015 sentences tagged over six different classes. - -1. Organization -2. Location -3. Facility -4. Event -5. Product -6. Person - - -| Label | # | -|:------------:|:-----:| -| Organization | 30108 | -| Location | 12924 | -| Facility | 4458 | -| Event | 7557 | -| Product | 4389 | -| Person | 15645 | - - - -**Download** -You can download the dataset from [here](https://github.com/HaniehP/PersianNER) - - - -## Results - -The following table summarizes the F1 score obtained by ParsBERT as compared to other models and architectures. - -| Dataset | ParsBERT | MorphoBERT | Beheshti-NER | LSTM-CRF | Rule-Based CRF | BiLSTM-CRF | -|:---------------:|:--------:|:----------:|:--------------:|:----------:|:----------------:|:------------:| -| ARMAN + PEYMA | 95.13* | - | - | - | - | - | -| PEYMA | 98.79* | - | 90.59 | - | 84.00 | - | -| ARMAN | 93.10* | 89.9 | 84.03 | 86.55 | - | 77.45 | - - -## How to use :hugs: -| Notebook | Description | | -|:----------|:-------------|------:| -| [How to use Pipelines](https://github.com/hooshvare/parsbert-ner/blob/master/persian-ner-pipeline.ipynb) | Simple and efficient way to use State-of-the-Art models on downstream tasks through transformers | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hooshvare/parsbert-ner/blob/master/persian-ner-pipeline.ipynb) | - - -## Cite - -Please cite the following paper in your publication if you are using [ParsBERT](https://arxiv.org/abs/2005.12515) in your research: - -```markdown -@article{ParsBERT, - title={ParsBERT: Transformer-based Model for Persian Language Understanding}, - author={Mehrdad Farahani, Mohammad Gharachorloo, Marzieh Farahani, Mohammad Manthouri}, - journal={ArXiv}, - year={2020}, - volume={abs/2005.12515} -} -``` - - -## Acknowledgments - -We hereby, express our gratitude to the [Tensorflow Research Cloud (TFRC) program](https://tensorflow.org/tfrc) for providing us with the necessary computation resources. We also thank [Hooshvare](https://hooshvare.com) Research Group for facilitating dataset gathering and scraping online text resources. - - -## Contributors - -- Mehrdad Farahani: [Linkedin](https://www.linkedin.com/in/m3hrdadfi/), [Twitter](https://twitter.com/m3hrdadfi), [Github](https://github.com/m3hrdadfi) -- Mohammad Gharachorloo: [Linkedin](https://www.linkedin.com/in/mohammad-gharachorloo/), [Twitter](https://twitter.com/MGharachorloo), [Github](https://github.com/baarsaam) -- Marzieh Farahani: [Linkedin](https://www.linkedin.com/in/marziehphi/), [Twitter](https://twitter.com/marziehphi), [Github](https://github.com/marziehphi) -- Mohammad Manthouri: [Linkedin](https://www.linkedin.com/in/mohammad-manthouri-aka-mansouri-07030766/), [Twitter](https://twitter.com/mmanthouri), [Github](https://github.com/mmanthouri) -- Hooshvare Team: [Official Website](https://hooshvare.com/), [Linkedin](https://www.linkedin.com/company/hooshvare), [Twitter](https://twitter.com/hooshvare), [Github](https://github.com/hooshvare), [Instagram](https://www.instagram.com/hooshvare/) - -+ And a special thanks to Sara Tabrizi for her fantastic poster design. Follow her on: [Linkedin](https://www.linkedin.com/in/sara-tabrizi-64548b79/), [Behance](https://www.behance.net/saratabrizi), [Instagram](https://www.instagram.com/sara_b_tabrizi/) - -## Releases - -### Release v0.1 (May 29, 2019) -This is the first version of our ParsBERT NER! diff --git a/model_cards/HooshvareLab/bert-base-parsbert-ner-uncased/README.md b/model_cards/HooshvareLab/bert-base-parsbert-ner-uncased/README.md deleted file mode 100644 index 80112e431d..0000000000 --- a/model_cards/HooshvareLab/bert-base-parsbert-ner-uncased/README.md +++ /dev/null @@ -1,124 +0,0 @@ -## ParsBERT: Transformer-based Model for Persian Language Understanding - -ParsBERT is a monolingual language model based on Google’s BERT architecture with the same configurations as BERT-Base. - -Paper presenting ParsBERT: [arXiv:2005.12515](https://arxiv.org/abs/2005.12515) - -All the models (downstream tasks) are uncased and trained with whole word masking. (coming soon stay tuned) - - -## Persian NER [ARMAN, PEYMA, ARMAN+PEYMA] - -This task aims to extract named entities in the text, such as names and label with appropriate `NER` classes such as locations, organizations, etc. The datasets used for this task contain sentences that are marked with `IOB` format. In this format, tokens that are not part of an entity are tagged as `”O”` the `”B”`tag corresponds to the first word of an object, and the `”I”` tag corresponds to the rest of the terms of the same entity. Both `”B”` and `”I”` tags are followed by a hyphen (or underscore), followed by the entity category. Therefore, the NER task is a multi-class token classification problem that labels the tokens upon being fed a raw text. There are two primary datasets used in Persian NER, `ARMAN`, and `PEYMA`. In ParsBERT, we prepared ner for both datasets as well as a combination of both datasets. - - - -### PEYMA - -PEYMA dataset includes 7,145 sentences with a total of 302,530 tokens from which 41,148 tokens are tagged with seven different classes. - -1. Organization -2. Money -3. Location -4. Date -5. Time -6. Person -7. Percent - - -| Label | # | -|:------------:|:-----:| -| Organization | 16964 | -| Money | 2037 | -| Location | 8782 | -| Date | 4259 | -| Time | 732 | -| Person | 7675 | -| Percent | 699 | - - - -**Download** -You can download the dataset from [here](http://nsurl.org/tasks/task-7-named-entity-recognition-ner-for-farsi/) - ---- - -### ARMAN - -ARMAN dataset holds 7,682 sentences with 250,015 sentences tagged over six different classes. - -1. Organization -2. Location -3. Facility -4. Event -5. Product -6. Person - - -| Label | # | -|:------------:|:-----:| -| Organization | 30108 | -| Location | 12924 | -| Facility | 4458 | -| Event | 7557 | -| Product | 4389 | -| Person | 15645 | - - - -**Download** -You can download the dataset from [here](https://github.com/HaniehP/PersianNER) - - - -## Results - -The following table summarizes the F1 score obtained by ParsBERT as compared to other models and architectures. - -| Dataset | ParsBERT | MorphoBERT | Beheshti-NER | LSTM-CRF | Rule-Based CRF | BiLSTM-CRF | -|:---------------:|:--------:|:----------:|:--------------:|:----------:|:----------------:|:------------:| -| ARMAN + PEYMA | 95.13* | - | - | - | - | - | -| PEYMA | 98.79* | - | 90.59 | - | 84.00 | - | -| ARMAN | 93.10* | 89.9 | 84.03 | 86.55 | - | 77.45 | - - -## How to use :hugs: -| Notebook | Description | | -|:----------|:-------------|------:| -| [How to use Pipelines](https://github.com/hooshvare/parsbert-ner/blob/master/persian-ner-pipeline.ipynb) | Simple and efficient way to use State-of-the-Art models on downstream tasks through transformers | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hooshvare/parsbert-ner/blob/master/persian-ner-pipeline.ipynb) | - - -## Cite - -Please cite the following paper in your publication if you are using [ParsBERT](https://arxiv.org/abs/2005.12515) in your research: - -```markdown -@article{ParsBERT, - title={ParsBERT: Transformer-based Model for Persian Language Understanding}, - author={Mehrdad Farahani, Mohammad Gharachorloo, Marzieh Farahani, Mohammad Manthouri}, - journal={ArXiv}, - year={2020}, - volume={abs/2005.12515} -} -``` - - -## Acknowledgments - -We hereby, express our gratitude to the [Tensorflow Research Cloud (TFRC) program](https://tensorflow.org/tfrc) for providing us with the necessary computation resources. We also thank [Hooshvare](https://hooshvare.com) Research Group for facilitating dataset gathering and scraping online text resources. - - -## Contributors - -- Mehrdad Farahani: [Linkedin](https://www.linkedin.com/in/m3hrdadfi/), [Twitter](https://twitter.com/m3hrdadfi), [Github](https://github.com/m3hrdadfi) -- Mohammad Gharachorloo: [Linkedin](https://www.linkedin.com/in/mohammad-gharachorloo/), [Twitter](https://twitter.com/MGharachorloo), [Github](https://github.com/baarsaam) -- Marzieh Farahani: [Linkedin](https://www.linkedin.com/in/marziehphi/), [Twitter](https://twitter.com/marziehphi), [Github](https://github.com/marziehphi) -- Mohammad Manthouri: [Linkedin](https://www.linkedin.com/in/mohammad-manthouri-aka-mansouri-07030766/), [Twitter](https://twitter.com/mmanthouri), [Github](https://github.com/mmanthouri) -- Hooshvare Team: [Official Website](https://hooshvare.com/), [Linkedin](https://www.linkedin.com/company/hooshvare), [Twitter](https://twitter.com/hooshvare), [Github](https://github.com/hooshvare), [Instagram](https://www.instagram.com/hooshvare/) - -+ And a special thanks to Sara Tabrizi for her fantastic poster design. Follow her on: [Linkedin](https://www.linkedin.com/in/sara-tabrizi-64548b79/), [Behance](https://www.behance.net/saratabrizi), [Instagram](https://www.instagram.com/sara_b_tabrizi/) - -## Releases - -### Release v0.1 (May 29, 2019) -This is the first version of our ParsBERT NER! diff --git a/model_cards/HooshvareLab/bert-base-parsbert-peymaner-uncased/README.md b/model_cards/HooshvareLab/bert-base-parsbert-peymaner-uncased/README.md deleted file mode 100644 index 80112e431d..0000000000 --- a/model_cards/HooshvareLab/bert-base-parsbert-peymaner-uncased/README.md +++ /dev/null @@ -1,124 +0,0 @@ -## ParsBERT: Transformer-based Model for Persian Language Understanding - -ParsBERT is a monolingual language model based on Google’s BERT architecture with the same configurations as BERT-Base. - -Paper presenting ParsBERT: [arXiv:2005.12515](https://arxiv.org/abs/2005.12515) - -All the models (downstream tasks) are uncased and trained with whole word masking. (coming soon stay tuned) - - -## Persian NER [ARMAN, PEYMA, ARMAN+PEYMA] - -This task aims to extract named entities in the text, such as names and label with appropriate `NER` classes such as locations, organizations, etc. The datasets used for this task contain sentences that are marked with `IOB` format. In this format, tokens that are not part of an entity are tagged as `”O”` the `”B”`tag corresponds to the first word of an object, and the `”I”` tag corresponds to the rest of the terms of the same entity. Both `”B”` and `”I”` tags are followed by a hyphen (or underscore), followed by the entity category. Therefore, the NER task is a multi-class token classification problem that labels the tokens upon being fed a raw text. There are two primary datasets used in Persian NER, `ARMAN`, and `PEYMA`. In ParsBERT, we prepared ner for both datasets as well as a combination of both datasets. - - - -### PEYMA - -PEYMA dataset includes 7,145 sentences with a total of 302,530 tokens from which 41,148 tokens are tagged with seven different classes. - -1. Organization -2. Money -3. Location -4. Date -5. Time -6. Person -7. Percent - - -| Label | # | -|:------------:|:-----:| -| Organization | 16964 | -| Money | 2037 | -| Location | 8782 | -| Date | 4259 | -| Time | 732 | -| Person | 7675 | -| Percent | 699 | - - - -**Download** -You can download the dataset from [here](http://nsurl.org/tasks/task-7-named-entity-recognition-ner-for-farsi/) - ---- - -### ARMAN - -ARMAN dataset holds 7,682 sentences with 250,015 sentences tagged over six different classes. - -1. Organization -2. Location -3. Facility -4. Event -5. Product -6. Person - - -| Label | # | -|:------------:|:-----:| -| Organization | 30108 | -| Location | 12924 | -| Facility | 4458 | -| Event | 7557 | -| Product | 4389 | -| Person | 15645 | - - - -**Download** -You can download the dataset from [here](https://github.com/HaniehP/PersianNER) - - - -## Results - -The following table summarizes the F1 score obtained by ParsBERT as compared to other models and architectures. - -| Dataset | ParsBERT | MorphoBERT | Beheshti-NER | LSTM-CRF | Rule-Based CRF | BiLSTM-CRF | -|:---------------:|:--------:|:----------:|:--------------:|:----------:|:----------------:|:------------:| -| ARMAN + PEYMA | 95.13* | - | - | - | - | - | -| PEYMA | 98.79* | - | 90.59 | - | 84.00 | - | -| ARMAN | 93.10* | 89.9 | 84.03 | 86.55 | - | 77.45 | - - -## How to use :hugs: -| Notebook | Description | | -|:----------|:-------------|------:| -| [How to use Pipelines](https://github.com/hooshvare/parsbert-ner/blob/master/persian-ner-pipeline.ipynb) | Simple and efficient way to use State-of-the-Art models on downstream tasks through transformers | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/hooshvare/parsbert-ner/blob/master/persian-ner-pipeline.ipynb) | - - -## Cite - -Please cite the following paper in your publication if you are using [ParsBERT](https://arxiv.org/abs/2005.12515) in your research: - -```markdown -@article{ParsBERT, - title={ParsBERT: Transformer-based Model for Persian Language Understanding}, - author={Mehrdad Farahani, Mohammad Gharachorloo, Marzieh Farahani, Mohammad Manthouri}, - journal={ArXiv}, - year={2020}, - volume={abs/2005.12515} -} -``` - - -## Acknowledgments - -We hereby, express our gratitude to the [Tensorflow Research Cloud (TFRC) program](https://tensorflow.org/tfrc) for providing us with the necessary computation resources. We also thank [Hooshvare](https://hooshvare.com) Research Group for facilitating dataset gathering and scraping online text resources. - - -## Contributors - -- Mehrdad Farahani: [Linkedin](https://www.linkedin.com/in/m3hrdadfi/), [Twitter](https://twitter.com/m3hrdadfi), [Github](https://github.com/m3hrdadfi) -- Mohammad Gharachorloo: [Linkedin](https://www.linkedin.com/in/mohammad-gharachorloo/), [Twitter](https://twitter.com/MGharachorloo), [Github](https://github.com/baarsaam) -- Marzieh Farahani: [Linkedin](https://www.linkedin.com/in/marziehphi/), [Twitter](https://twitter.com/marziehphi), [Github](https://github.com/marziehphi) -- Mohammad Manthouri: [Linkedin](https://www.linkedin.com/in/mohammad-manthouri-aka-mansouri-07030766/), [Twitter](https://twitter.com/mmanthouri), [Github](https://github.com/mmanthouri) -- Hooshvare Team: [Official Website](https://hooshvare.com/), [Linkedin](https://www.linkedin.com/company/hooshvare), [Twitter](https://twitter.com/hooshvare), [Github](https://github.com/hooshvare), [Instagram](https://www.instagram.com/hooshvare/) - -+ And a special thanks to Sara Tabrizi for her fantastic poster design. Follow her on: [Linkedin](https://www.linkedin.com/in/sara-tabrizi-64548b79/), [Behance](https://www.behance.net/saratabrizi), [Instagram](https://www.instagram.com/sara_b_tabrizi/) - -## Releases - -### Release v0.1 (May 29, 2019) -This is the first version of our ParsBERT NER! diff --git a/model_cards/HooshvareLab/bert-base-parsbert-uncased/README.md b/model_cards/HooshvareLab/bert-base-parsbert-uncased/README.md deleted file mode 100644 index c931a2e29d..0000000000 --- a/model_cards/HooshvareLab/bert-base-parsbert-uncased/README.md +++ /dev/null @@ -1,124 +0,0 @@ -## ParsBERT: Transformer-based Model for Persian Language Understanding - -ParsBERT is a monolingual language model based on Google’s BERT architecture with the same configurations as BERT-Base. - -Paper presenting ParsBERT: [arXiv:2005.12515](https://arxiv.org/abs/2005.12515) - -All the models (downstream tasks) are uncased and trained with whole word masking. (coming soon stay tuned) - - ---- - -## Introduction - -This model is pre-trained on a large Persian corpus with various writing styles from numerous subjects (e.g., scientific, novels, news) with more than 2M documents. A large subset of this corpus was crawled manually. - -As a part of ParsBERT methodology, an extensive pre-processing combining POS tagging and WordPiece segmentation was carried out to bring the corpus into a proper format. This process produces more than 40M true sentences. - - -## Evaluation - -ParsBERT is evaluated on three NLP downstream tasks: Sentiment Analysis (SA), Text Classification, and Named Entity Recognition (NER). For this matter and due to insufficient resources, two large datasets for SA and two for text classification were manually composed, which are available for public use and benchmarking. ParsBERT outperformed all other language models, including multilingual BERT and other hybrid deep learning models for all tasks, improving the state-of-the-art performance in Persian language modeling. - -## Results - -The following table summarizes the F1 score obtained by ParsBERT as compared to other models and architectures. - - - -### Sentiment Analysis (SA) task - -| Dataset | ParsBERT | mBERT | DeepSentiPers | -|:--------------------------:|:---------:|:-----:|:-------------:| -| Digikala User Comments | 81.74* | 80.74 | - | -| SnappFood User Comments | 88.12* | 87.87 | - | -| SentiPers (Multi Class) | 71.11* | - | 69.33 | -| SentiPers (Binary Class) | 92.13* | - | 91.98 | - - - -### Text Classification (TC) task - -| Dataset | ParsBERT | mBERT | -|:-----------------:|:--------:|:-----:| -| Digikala Magazine | 93.59* | 90.72 | -| Persian News | 97.19* | 95.79 | - - -### Named Entity Recognition (NER) task - -| Dataset | ParsBERT | mBERT | MorphoBERT | Beheshti-NER | LSTM-CRF | Rule-Based CRF | BiLSTM-CRF | -|:-------:|:--------:|:--------:|:----------:|:--------------:|:----------:|:----------------:|:------------:| -| PEYMA | 93.10* | 86.64 | - | 90.59 | - | 84.00 | - | -| ARMAN | 98.79* | 95.89 | 89.9 | 84.03 | 86.55 | - | 77.45 | - - -**If you tested ParsBERT on a public dataset and you want to add your results to the table above, open a pull request or contact us. Also make sure to have your code available online so we can add it as a reference** - -## How to use - -### TensorFlow 2.0 - -```python -from transformers import AutoConfig, AutoTokenizer, TFAutoModel - -config = AutoConfig.from_pretrained("HooshvareLab/bert-base-parsbert-uncased") -tokenizer = AutoTokenizer.from_pretrained("HooshvareLab/bert-base-parsbert-uncased") -model = AutoModel.from_pretrained("HooshvareLab/bert-base-parsbert-uncased") - -text = "ما در هوشواره معتقدیم با انتقال صحیح دانش و آگاهی، همه افراد می‌توانند از ابزارهای هوشمند استفاده کنند. شعار ما هوش مصنوعی برای همه است." -tokenizer.tokenize(text) - ->>> ['ما', 'در', 'هوش', '##واره', 'معتقدیم', 'با', 'انتقال', 'صحیح', 'دانش', 'و', 'اگاهی', '،', 'همه', 'افراد', 'میتوانند', 'از', 'ابزارهای', 'هوشمند', 'استفاده', 'کنند', '.', 'شعار', 'ما', 'هوش', 'مصنوعی', 'برای', 'همه', 'است', '.'] - -``` - -### Pytorch - -```python -from transformers import AutoConfig, AutoTokenizer, AutoModel - -config = AutoConfig.from_pretrained("HooshvareLab/bert-base-parsbert-uncased") -tokenizer = AutoTokenizer.from_pretrained("HooshvareLab/bert-base-parsbert-uncased") -model = AutoModel.from_pretrained("HooshvareLab/bert-base-parsbert-uncased") -``` - - -## NLP Tasks Tutorial - -Coming soon stay tuned - - -## Cite - -Please cite the following paper in your publication if you are using [ParsBERT](https://arxiv.org/abs/2005.12515) in your research: - -```markdown -@article{ParsBERT, - title={ParsBERT: Transformer-based Model for Persian Language Understanding}, - author={Mehrdad Farahani, Mohammad Gharachorloo, Marzieh Farahani, Mohammad Manthouri}, - journal={ArXiv}, - year={2020}, - volume={abs/2005.12515} -} -``` - - -## Acknowledgments - -We hereby, express our gratitude to the [Tensorflow Research Cloud (TFRC) program](https://tensorflow.org/tfrc) for providing us with the necessary computation resources. We also thank [Hooshvare](https://hooshvare.com) Research Group for facilitating dataset gathering and scraping online text resources. - - -## Contributors - -- Mehrdad Farahani: [Linkedin](https://www.linkedin.com/in/m3hrdadfi/), [Twitter](https://twitter.com/m3hrdadfi), [Github](https://github.com/m3hrdadfi) -- Mohammad Gharachorloo: [Linkedin](https://www.linkedin.com/in/mohammad-gharachorloo/), [Twitter](https://twitter.com/MGharachorloo), [Github](https://github.com/baarsaam) -- Marzieh Farahani: [Linkedin](https://www.linkedin.com/in/marziehphi/), [Twitter](https://twitter.com/marziehphi), [Github](https://github.com/marziehphi) -- Mohammad Manthouri: [Linkedin](https://www.linkedin.com/in/mohammad-manthouri-aka-mansouri-07030766/), [Twitter](https://twitter.com/mmanthouri), [Github](https://github.com/mmanthouri) -- Hooshvare Team: [Official Website](https://hooshvare.com/), [Linkedin](https://www.linkedin.com/company/hooshvare), [Twitter](https://twitter.com/hooshvare), [Github](https://github.com/hooshvare), [Instagram](https://www.instagram.com/hooshvare/) - - -## Releases - -### Release v0.1 (May 27, 2019) -This is the first version of our ParsBERT based on BERTBASE diff --git a/model_cards/HooshvareLab/bert-fa-base-uncased/README.md b/model_cards/HooshvareLab/bert-fa-base-uncased/README.md deleted file mode 100644 index 1516c7adcb..0000000000 --- a/model_cards/HooshvareLab/bert-fa-base-uncased/README.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -language: fa -tags: -- bert-fa -- bert-persian -- persian-lm -license: apache-2.0 ---- - -# ParsBERT (v2.0) -A Transformer-based Model for Persian Language Understanding - - -We reconstructed the vocabulary and fine-tuned the ParsBERT v1.1 on the new Persian corpora in order to provide some functionalities for using ParsBERT in other scopes! -Please follow the [ParsBERT](https://github.com/hooshvare/parsbert) repo for the latest information about previous and current models. - -## Introduction - -ParsBERT is a monolingual language model based on Google’s BERT architecture. This model is pre-trained on large Persian corpora with various writing styles from numerous subjects (e.g., scientific, novels, news) with more than `3.9M` documents, `73M` sentences, and `1.3B` words. - -Paper presenting ParsBERT: [arXiv:2005.12515](https://arxiv.org/abs/2005.12515) - -## Intended uses & limitations - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?search=bert-fa) to look for -fine-tuned versions on a task that interests you. - - -### How to use - -#### TensorFlow 2.0 - -```python -from transformers import AutoConfig, AutoTokenizer, TFAutoModel - -config = AutoConfig.from_pretrained("HooshvareLab/bert-fa-base-uncased") -tokenizer = AutoTokenizer.from_pretrained("HooshvareLab/bert-fa-base-uncased") -model = TFAutoModel.from_pretrained("HooshvareLab/bert-fa-base-uncased") - -text = "ما در هوشواره معتقدیم با انتقال صحیح دانش و آگاهی، همه افراد میتوانند از ابزارهای هوشمند استفاده کنند. شعار ما هوش مصنوعی برای همه است." -tokenizer.tokenize(text) - ->>> ['ما', 'در', 'هوش', '##واره', 'معتقدیم', 'با', 'انتقال', 'صحیح', 'دانش', 'و', 'اگاهی', '،', 'همه', 'افراد', 'میتوانند', 'از', 'ابزارهای', 'هوشمند', 'استفاده', 'کنند', '.', 'شعار', 'ما', 'هوش', 'مصنوعی', 'برای', 'همه', 'است', '.'] -``` - -#### Pytorch - -```python -from transformers import AutoConfig, AutoTokenizer, AutoModel - -config = AutoConfig.from_pretrained("HooshvareLab/bert-fa-base-uncased") -tokenizer = AutoTokenizer.from_pretrained("HooshvareLab/bert-fa-base-uncased") -model = AutoModel.from_pretrained("HooshvareLab/bert-fa-base-uncased") -``` - -## Training - -ParsBERT trained on a massive amount of public corpora ([Persian Wikidumps](https://dumps.wikimedia.org/fawiki/), [MirasText](https://github.com/miras-tech/MirasText)) and six other manually crawled text data from a various type of websites ([BigBang Page](https://bigbangpage.com/) `scientific`, [Chetor](https://www.chetor.com/) `lifestyle`, [Eligasht](https://www.eligasht.com/Blog/) `itinerary`, [Digikala](https://www.digikala.com/mag/) `digital magazine`, [Ted Talks](https://www.ted.com/talks) `general conversational`, Books `novels, storybooks, short stories from old to the contemporary era`). - -As a part of ParsBERT methodology, an extensive pre-processing combining POS tagging and WordPiece segmentation was carried out to bring the corpora into a proper format. - -## Goals -Objective goals during training are as below (after 300k steps). - -``` bash -***** Eval results ***** -global_step = 300000 -loss = 1.4392426 -masked_lm_accuracy = 0.6865794 -masked_lm_loss = 1.4469004 -next_sentence_accuracy = 1.0 -next_sentence_loss = 6.534152e-05 -``` - - -## Derivative models - -### Base Config - -#### ParsBERT v2.0 Model -- [HooshvareLab/bert-fa-base-uncased](https://huggingface.co/HooshvareLab/bert-fa-base-uncased) - -#### ParsBERT v2.0 Sentiment Analysis -- [HooshvareLab/bert-fa-base-uncased-sentiment-digikala](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-sentiment-digikala) -- [HooshvareLab/bert-fa-base-uncased-sentiment-snappfood](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-sentiment-snappfood) -- [HooshvareLab/bert-fa-base-uncased-sentiment-deepsentipers-binary](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-sentiment-deepsentipers-binary) -- [HooshvareLab/bert-fa-base-uncased-sentiment-deepsentipers-multi](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-sentiment-deepsentipers-multi) - -#### ParsBERT v2.0 Text Classification -- [HooshvareLab/bert-fa-base-uncased-clf-digimag](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-clf-digimag) -- [HooshvareLab/bert-fa-base-uncased-clf-persiannews](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-clf-persiannews) - -#### ParsBERT v2.0 NER -- [HooshvareLab/bert-fa-base-uncased-ner-peyma](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-ner-peyma) -- [HooshvareLab/bert-fa-base-uncased-ner-arman](https://huggingface.co/HooshvareLab/bert-fa-base-uncased-ner-arman) - - -## Eval results - -ParsBERT is evaluated on three NLP downstream tasks: Sentiment Analysis (SA), Text Classification, and Named Entity Recognition (NER). For this matter and due to insufficient resources, two large datasets for SA and two for text classification were manually composed, which are available for public use and benchmarking. ParsBERT outperformed all other language models, including multilingual BERT and other hybrid deep learning models for all tasks, improving the state-of-the-art performance in Persian language modeling. - - -### Sentiment Analysis (SA) Task - -| Dataset | ParsBERT v2 | ParsBERT v1 | mBERT | DeepSentiPers | -|:------------------------:|:-----------:|:-----------:|:-----:|:-------------:| -| Digikala User Comments | 81.72 | 81.74* | 80.74 | - | -| SnappFood User Comments | 87.98 | 88.12* | 87.87 | - | -| SentiPers (Multi Class) | 71.31* | 71.11 | - | 69.33 | -| SentiPers (Binary Class) | 92.42* | 92.13 | - | 91.98 | - - -### Text Classification (TC) Task - -| Dataset | ParsBERT v2 | ParsBERT v1 | mBERT | -|:-----------------:|:-----------:|:-----------:|:-----:| -| Digikala Magazine | 93.65* | 93.59 | 90.72 | -| Persian News | 97.44* | 97.19 | 95.79 | - - -### Named Entity Recognition (NER) Task - -| Dataset | ParsBERT v2 | ParsBERT v1 | mBERT | MorphoBERT | Beheshti-NER | LSTM-CRF | Rule-Based CRF | BiLSTM-CRF | -|:-------:|:-----------:|:-----------:|:-----:|:----------:|:------------:|:--------:|:--------------:|:----------:| -| PEYMA | 93.40* | 93.10 | 86.64 | - | 90.59 | - | 84.00 | - | -| ARMAN | 99.84* | 98.79 | 95.89 | 89.9 | 84.03 | 86.55 | - | 77.45 | - - - - -### BibTeX entry and citation info - -Please cite in publications as the following: - -```bibtex -@article{ParsBERT, - title={ParsBERT: Transformer-based Model for Persian Language Understanding}, - author={Mehrdad Farahani, Mohammad Gharachorloo, Marzieh Farahani, Mohammad Manthouri}, - journal={ArXiv}, - year={2020}, - volume={abs/2005.12515} -} -``` - -## Questions? -Post a Github issue on the [ParsBERT Issues](https://github.com/hooshvare/parsbert/issues) repo. diff --git a/model_cards/KB/albert-base-swedish-cased-alpha/README.md b/model_cards/KB/albert-base-swedish-cased-alpha/README.md deleted file mode 100644 index 46bf3c700f..0000000000 --- a/model_cards/KB/albert-base-swedish-cased-alpha/README.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -language: sv ---- - -# Swedish BERT Models - -The National Library of Sweden / KBLab releases three pretrained language models based on BERT and ALBERT. The models are trained on approximately 15-20GB of text (200M sentences, 3000M tokens) from various sources (books, news, government publications, swedish wikipedia and internet forums) aiming to provide a representative BERT model for Swedish text. A more complete description will be published later on. - -The following three models are currently available: - -- **bert-base-swedish-cased** (*v1*) - A BERT trained with the same hyperparameters as first published by Google. -- **bert-base-swedish-cased-ner** (*experimental*) - a BERT fine-tuned for NER using SUC 3.0. -- **albert-base-swedish-cased-alpha** (*alpha*) - A first attempt at an ALBERT for Swedish. - -All models are cased and trained with whole word masking. - -## Files - -| **name** | **files** | -|---------------------------------|-----------| -| bert-base-swedish-cased | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/config.json), [vocab](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/vocab.txt), [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/pytorch_model.bin) | -| bert-base-swedish-cased-ner | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/config.json), [vocab](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/vocab.txt) [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/pytorch_model.bin) | -| albert-base-swedish-cased-alpha | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/config.json), [sentencepiece model](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/spiece.model), [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/pytorch_model.bin) | - -TensorFlow model weights will be released soon. - -## Usage requirements / installation instructions - -The examples below require Huggingface Transformers 2.4.1 and Pytorch 1.3.1 or greater. For Transformers<2.4.0 the tokenizer must be instantiated manually and the `do_lower_case` flag parameter set to `False` and `keep_accents` to `True` (for ALBERT). - -To create an environment where the examples can be run, run the following in an terminal on your OS of choice. - -``` -# git clone https://github.com/Kungbib/swedish-bert-models -# cd swedish-bert-models -# python3 -m venv venv -# source venv/bin/activate -# pip install --upgrade pip -# pip install -r requirements.txt -``` - -### BERT Base Swedish - -A standard BERT base for Swedish trained on a variety of sources. Vocabulary size is ~50k. Using Huggingface Transformers the model can be loaded in Python as follows: - -```python -from transformers import AutoModel,AutoTokenizer - -tok = AutoTokenizer.from_pretrained('KB/bert-base-swedish-cased') -model = AutoModel.from_pretrained('KB/bert-base-swedish-cased') -``` - - -### BERT base fine-tuned for Swedish NER - -This model is fine-tuned on the SUC 3.0 dataset. Using the Huggingface pipeline the model can be easily instantiated. For Transformer<2.4.1 it seems the tokenizer must be loaded separately to disable lower-casing of input strings: - -```python -from transformers import pipeline - -nlp = pipeline('ner', model='KB/bert-base-swedish-cased-ner', tokenizer='KB/bert-base-swedish-cased-ner') - -nlp('Idag släpper KB tre språkmodeller.') -``` - -Running the Python code above should produce in something like the result below. Entity types used are `TME` for time, `PRS` for personal names, `LOC` for locations, `EVN` for events and `ORG` for organisations. These labels are subject to change. - -```python -[ { 'word': 'Idag', 'score': 0.9998126029968262, 'entity': 'TME' }, - { 'word': 'KB', 'score': 0.9814832210540771, 'entity': 'ORG' } ] -``` - -The BERT tokenizer often splits words into multiple tokens, with the subparts starting with `##`, for example the string `Engelbert kör Volvo till Herrängens fotbollsklubb` gets tokenized as `Engel ##bert kör Volvo till Herr ##ängens fotbolls ##klubb`. To glue parts back together one can use something like this: - -```python -text = 'Engelbert tar Volvon till Tele2 Arena för att titta på Djurgården IF ' +\ - 'som spelar fotboll i VM klockan två på kvällen.' - -l = [] -for token in nlp(text): - if token['word'].startswith('##'): - l[-1]['word'] += token['word'][2:] - else: - l += [ token ] - -print(l) -``` - -Which should result in the following (though less cleanly formatted): - -```python -[ { 'word': 'Engelbert', 'score': 0.99..., 'entity': 'PRS'}, - { 'word': 'Volvon', 'score': 0.99..., 'entity': 'OBJ'}, - { 'word': 'Tele2', 'score': 0.99..., 'entity': 'LOC'}, - { 'word': 'Arena', 'score': 0.99..., 'entity': 'LOC'}, - { 'word': 'Djurgården', 'score': 0.99..., 'entity': 'ORG'}, - { 'word': 'IF', 'score': 0.99..., 'entity': 'ORG'}, - { 'word': 'VM', 'score': 0.99..., 'entity': 'EVN'}, - { 'word': 'klockan', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'två', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'på', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'kvällen', 'score': 0.54..., 'entity': 'TME'} ] -``` - -### ALBERT base - -The easiest way to do this is, again, using Huggingface Transformers: - -```python -from transformers import AutoModel,AutoTokenizer - -tok = AutoTokenizer.from_pretrained('KB/albert-base-swedish-cased-alpha'), -model = AutoModel.from_pretrained('KB/albert-base-swedish-cased-alpha') -``` - -## Acknowledgements ❤️ - -- Resources from Stockholms University, Umeå University and Swedish Language Bank at Gothenburg University were used when fine-tuning BERT for NER. -- Model pretraining was made partly in-house at the KBLab and partly (for material without active copyright) with the support of Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -- Models are hosted on S3 by Huggingface 🤗 - diff --git a/model_cards/KB/bert-base-swedish-cased-ner/README.md b/model_cards/KB/bert-base-swedish-cased-ner/README.md deleted file mode 100644 index 46bf3c700f..0000000000 --- a/model_cards/KB/bert-base-swedish-cased-ner/README.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -language: sv ---- - -# Swedish BERT Models - -The National Library of Sweden / KBLab releases three pretrained language models based on BERT and ALBERT. The models are trained on approximately 15-20GB of text (200M sentences, 3000M tokens) from various sources (books, news, government publications, swedish wikipedia and internet forums) aiming to provide a representative BERT model for Swedish text. A more complete description will be published later on. - -The following three models are currently available: - -- **bert-base-swedish-cased** (*v1*) - A BERT trained with the same hyperparameters as first published by Google. -- **bert-base-swedish-cased-ner** (*experimental*) - a BERT fine-tuned for NER using SUC 3.0. -- **albert-base-swedish-cased-alpha** (*alpha*) - A first attempt at an ALBERT for Swedish. - -All models are cased and trained with whole word masking. - -## Files - -| **name** | **files** | -|---------------------------------|-----------| -| bert-base-swedish-cased | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/config.json), [vocab](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/vocab.txt), [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/pytorch_model.bin) | -| bert-base-swedish-cased-ner | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/config.json), [vocab](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/vocab.txt) [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/pytorch_model.bin) | -| albert-base-swedish-cased-alpha | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/config.json), [sentencepiece model](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/spiece.model), [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/pytorch_model.bin) | - -TensorFlow model weights will be released soon. - -## Usage requirements / installation instructions - -The examples below require Huggingface Transformers 2.4.1 and Pytorch 1.3.1 or greater. For Transformers<2.4.0 the tokenizer must be instantiated manually and the `do_lower_case` flag parameter set to `False` and `keep_accents` to `True` (for ALBERT). - -To create an environment where the examples can be run, run the following in an terminal on your OS of choice. - -``` -# git clone https://github.com/Kungbib/swedish-bert-models -# cd swedish-bert-models -# python3 -m venv venv -# source venv/bin/activate -# pip install --upgrade pip -# pip install -r requirements.txt -``` - -### BERT Base Swedish - -A standard BERT base for Swedish trained on a variety of sources. Vocabulary size is ~50k. Using Huggingface Transformers the model can be loaded in Python as follows: - -```python -from transformers import AutoModel,AutoTokenizer - -tok = AutoTokenizer.from_pretrained('KB/bert-base-swedish-cased') -model = AutoModel.from_pretrained('KB/bert-base-swedish-cased') -``` - - -### BERT base fine-tuned for Swedish NER - -This model is fine-tuned on the SUC 3.0 dataset. Using the Huggingface pipeline the model can be easily instantiated. For Transformer<2.4.1 it seems the tokenizer must be loaded separately to disable lower-casing of input strings: - -```python -from transformers import pipeline - -nlp = pipeline('ner', model='KB/bert-base-swedish-cased-ner', tokenizer='KB/bert-base-swedish-cased-ner') - -nlp('Idag släpper KB tre språkmodeller.') -``` - -Running the Python code above should produce in something like the result below. Entity types used are `TME` for time, `PRS` for personal names, `LOC` for locations, `EVN` for events and `ORG` for organisations. These labels are subject to change. - -```python -[ { 'word': 'Idag', 'score': 0.9998126029968262, 'entity': 'TME' }, - { 'word': 'KB', 'score': 0.9814832210540771, 'entity': 'ORG' } ] -``` - -The BERT tokenizer often splits words into multiple tokens, with the subparts starting with `##`, for example the string `Engelbert kör Volvo till Herrängens fotbollsklubb` gets tokenized as `Engel ##bert kör Volvo till Herr ##ängens fotbolls ##klubb`. To glue parts back together one can use something like this: - -```python -text = 'Engelbert tar Volvon till Tele2 Arena för att titta på Djurgården IF ' +\ - 'som spelar fotboll i VM klockan två på kvällen.' - -l = [] -for token in nlp(text): - if token['word'].startswith('##'): - l[-1]['word'] += token['word'][2:] - else: - l += [ token ] - -print(l) -``` - -Which should result in the following (though less cleanly formatted): - -```python -[ { 'word': 'Engelbert', 'score': 0.99..., 'entity': 'PRS'}, - { 'word': 'Volvon', 'score': 0.99..., 'entity': 'OBJ'}, - { 'word': 'Tele2', 'score': 0.99..., 'entity': 'LOC'}, - { 'word': 'Arena', 'score': 0.99..., 'entity': 'LOC'}, - { 'word': 'Djurgården', 'score': 0.99..., 'entity': 'ORG'}, - { 'word': 'IF', 'score': 0.99..., 'entity': 'ORG'}, - { 'word': 'VM', 'score': 0.99..., 'entity': 'EVN'}, - { 'word': 'klockan', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'två', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'på', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'kvällen', 'score': 0.54..., 'entity': 'TME'} ] -``` - -### ALBERT base - -The easiest way to do this is, again, using Huggingface Transformers: - -```python -from transformers import AutoModel,AutoTokenizer - -tok = AutoTokenizer.from_pretrained('KB/albert-base-swedish-cased-alpha'), -model = AutoModel.from_pretrained('KB/albert-base-swedish-cased-alpha') -``` - -## Acknowledgements ❤️ - -- Resources from Stockholms University, Umeå University and Swedish Language Bank at Gothenburg University were used when fine-tuning BERT for NER. -- Model pretraining was made partly in-house at the KBLab and partly (for material without active copyright) with the support of Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -- Models are hosted on S3 by Huggingface 🤗 - diff --git a/model_cards/KB/bert-base-swedish-cased/README.md b/model_cards/KB/bert-base-swedish-cased/README.md deleted file mode 100644 index ca87015b84..0000000000 --- a/model_cards/KB/bert-base-swedish-cased/README.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -language: sv ---- - -# Swedish BERT Models - -The National Library of Sweden / KBLab releases three pretrained language models based on BERT and ALBERT. The models are trained on aproximately 15-20GB of text (200M sentences, 3000M tokens) from various sources (books, news, government publications, swedish wikipedia and internet forums) aiming to provide a representative BERT model for Swedish text. A more complete description will be published later on. - -The following three models are currently available: - -- **bert-base-swedish-cased** (*v1*) - A BERT trained with the same hyperparameters as first published by Google. -- **bert-base-swedish-cased-ner** (*experimental*) - a BERT fine-tuned for NER using SUC 3.0. -- **albert-base-swedish-cased-alpha** (*alpha*) - A first attempt at an ALBERT for Swedish. - -All models are cased and trained with whole word masking. - -## Files - -| **name** | **files** | -|---------------------------------|-----------| -| bert-base-swedish-cased | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/config.json), [vocab](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/vocab.txt), [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased/pytorch_model.bin) | -| bert-base-swedish-cased-ner | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/config.json), [vocab](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/vocab.txt) [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/bert-base-swedish-cased-ner/pytorch_model.bin) | -| albert-base-swedish-cased-alpha | [config](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/config.json), [sentencepiece model](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/spiece.model), [pytorch_model.bin](https://s3.amazonaws.com/models.huggingface.co/bert/KB/albert-base-swedish-cased-alpha/pytorch_model.bin) | - -TensorFlow model weights will be released soon. - -## Usage requirements / installation instructions - -The examples below require Huggingface Transformers 2.4.1 and Pytorch 1.3.1 or greater. For Transformers<2.4.0 the tokenizer must be instantiated manually and the `do_lower_case` flag parameter set to `False` and `keep_accents` to `True` (for ALBERT). - -To create an environment where the examples can be run, run the following in an terminal on your OS of choice. - -``` -# git clone https://github.com/Kungbib/swedish-bert-models -# cd swedish-bert-models -# python3 -m venv venv -# source venv/bin/activate -# pip install --upgrade pip -# pip install -r requirements.txt -``` - -### BERT Base Swedish - -A standard BERT base for Swedish trained on a variety of sources. Vocabulary size is ~50k. Using Huggingface Transformers the model can be loaded in Python as follows: - -```python -from transformers import AutoModel,AutoTokenizer - -tok = AutoTokenizer.from_pretrained('KB/bert-base-swedish-cased') -model = AutoModel.from_pretrained('KB/bert-base-swedish-cased') -``` - - -### BERT base fine-tuned for Swedish NER - -This model is fine-tuned on the SUC 3.0 dataset. Using the Huggingface pipeline the model can be easily instantiated. For Transformer<2.4.1 it seems the tokenizer must be loaded separately to disable lower-casing of input strings: - -```python -from transformers import pipeline - -nlp = pipeline('ner', model='KB/bert-base-swedish-cased-ner', tokenizer='KB/bert-base-swedish-cased-ner') - -nlp('Idag släpper KB tre språkmodeller.') -``` - -Running the Python code above should produce in something like the result below. Entity types used are `TME` for time, `PRS` for personal names, `LOC` for locations, `EVN` for events and `ORG` for organisations. These labels are subject to change. - -```python -[ { 'word': 'Idag', 'score': 0.9998126029968262, 'entity': 'TME' }, - { 'word': 'KB', 'score': 0.9814832210540771, 'entity': 'ORG' } ] -``` - -The BERT tokenizer often splits words into multiple tokens, with the subparts starting with `##`, for example the string `Engelbert kör Volvo till Herrängens fotbollsklubb` gets tokenized as `Engel ##bert kör Volvo till Herr ##ängens fotbolls ##klubb`. To glue parts back together one can use something like this: - -```python -text = 'Engelbert tar Volvon till Tele2 Arena för att titta på Djurgården IF ' +\ - 'som spelar fotboll i VM klockan två på kvällen.' - -l = [] -for token in nlp(text): - if token['word'].startswith('##'): - l[-1]['word'] += token['word'][2:] - else: - l += [ token ] - -print(l) -``` - -Which should result in the following (though less cleanly formated): - -```python -[ { 'word': 'Engelbert', 'score': 0.99..., 'entity': 'PRS'}, - { 'word': 'Volvon', 'score': 0.99..., 'entity': 'OBJ'}, - { 'word': 'Tele2', 'score': 0.99..., 'entity': 'LOC'}, - { 'word': 'Arena', 'score': 0.99..., 'entity': 'LOC'}, - { 'word': 'Djurgården', 'score': 0.99..., 'entity': 'ORG'}, - { 'word': 'IF', 'score': 0.99..., 'entity': 'ORG'}, - { 'word': 'VM', 'score': 0.99..., 'entity': 'EVN'}, - { 'word': 'klockan', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'två', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'på', 'score': 0.99..., 'entity': 'TME'}, - { 'word': 'kvällen', 'score': 0.54..., 'entity': 'TME'} ] -``` - -### ALBERT base - -The easisest way to do this is, again, using Huggingface Transformers: - -```python -from transformers import AutoModel,AutoTokenizer - -tok = AutoTokenizer.from_pretrained('KB/albert-base-swedish-cased-alpha'), -model = AutoModel.from_pretrained('KB/albert-base-swedish-cased-alpha') -``` - -## Acknowledgements ❤️ - -- Resources from Stockholms University, Umeå University and Swedish Language Bank at Gothenburg University were used when fine-tuning BERT for NER. -- Model pretraining was made partly in-house at the KBLab and partly (for material without active copyright) with the support of Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -- Models are hosted on S3 by Huggingface 🤗 - diff --git a/model_cards/LorenzoDeMattei/GePpeTto/README.md b/model_cards/LorenzoDeMattei/GePpeTto/README.md deleted file mode 100644 index d6dcafb226..0000000000 --- a/model_cards/LorenzoDeMattei/GePpeTto/README.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -language: it ---- - -# GePpeTto GPT2 Model 🇮🇹 - -Pretrained GPT2 117M model for Italian. - -You can find further details in the paper: - -Lorenzo De Mattei, Michele Cafagna, Felice Dell’Orletta, Malvina Nissim, Marco Guerini "GePpeTto Carves Italian into a Language Model", arXiv preprint. Pdf available at: https://arxiv.org/abs/2004.14253 - -## Pretraining Corpus - -The pretraining set comprises two main sources. The first one is a dump of Italian Wikipedia (November 2019), -consisting of 2.8GB of text. The second one is the ItWac corpus (Baroni et al., 2009), which amounts to 11GB of web -texts. This collection provides a mix of standard and less standard Italian, on a rather wide chronological span, -with older texts than the Wikipedia dump (the latter stretches only to the late 2000s). - -## Pretraining details - -This model was trained using GPT2's Hugging Face implemenation on 4 NVIDIA Tesla T4 GPU for 620k steps. - -Training parameters: - -- GPT-2 small configuration -- vocabulary size: 30k -- Batch size: 32 -- Block size: 100 -- Adam Optimizer -- Initial learning rate: 5e-5 -- Warm up steps: 10k - -## Perplexity scores - -| Domain | Perplexity | -|---|---| -| Wikipedia | 26.1052 | -| ItWac | 30.3965 | -| Legal | 37.2197 | -| News | 45.3859 | -| Social Media | 84.6408 | - -For further details, qualitative analysis and human evaluation check out: https://arxiv.org/abs/2004.14253 - -## Load Pretrained Model - -You can use this model by installing Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import GPT2Tokenizer, GPT2Model - -model = GPT2Model.from_pretrained('LorenzoDeMattei/GePpeTto') -tokenizer = GPT2Tokenizer.from_pretrained( - 'LorenzoDeMattei/GePpeTto', -) -``` - -## Example using GPT2LMHeadModel - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline, GPT2Tokenizer - -tokenizer = AutoTokenizer.from_pretrained("LorenzoDeMattei/GePpeTto") -model = AutoModelWithLMHead.from_pretrained("LorenzoDeMattei/GePpeTto") - -text_generator = pipeline('text-generation', model=model, tokenizer=tokenizer) -prompts = [ - "Wikipedia Geppetto", - "Maestro Ciliegia regala il pezzo di legno al suo amico Geppetto, il quale lo prende per fabbricarsi un burattino maraviglioso"] - - -samples_outputs = text_generator( - prompts, - do_sample=True, - max_length=50, - top_k=50, - top_p=0.95, - num_return_sequences=3 -) - - -for i, sample_outputs in enumerate(samples_outputs): - print(100 * '-') - print("Prompt:", prompts[i]) - for sample_output in sample_outputs: - print("Sample:", sample_output['generated_text']) - print() - -``` - -Output is, - -``` ----------------------------------------------------------------------------------------------------- -Prompt: Wikipedia Geppetto -Sample: Wikipedia Geppetto rosso (film 1920) - -Geppetto rosso ("The Smokes in the Black") è un film muto del 1920 diretto da Henry H. Leonard. - -Il film fu prodotto dalla Selig Poly - -Sample: Wikipedia Geppetto - -Geppetto ("Geppetto" in piemontese) è un comune italiano di 978 abitanti della provincia di Cuneo in Piemonte. - -L'abitato, che si trova nel versante valtellinese, si sviluppa nella - -Sample: Wikipedia Geppetto di Natale (romanzo) - -Geppetto di Natale è un romanzo di Mario Caiano, pubblicato nel 2012. - ----------------------------------------------------------------------------------------------------- -Prompt: Maestro Ciliegia regala il pezzo di legno al suo amico Geppetto, il quale lo prende per fabbricarsi un burattino maraviglioso -Sample: Maestro Ciliegia regala il pezzo di legno al suo amico Geppetto, il quale lo prende per fabbricarsi un burattino maraviglioso. Il burattino riesce a scappare. Dopo aver trovato un prezioso sacchetto si reca - -Sample: Maestro Ciliegia regala il pezzo di legno al suo amico Geppetto, il quale lo prende per fabbricarsi un burattino maraviglioso, e l'unico che lo possiede, ma, di fronte a tutte queste prove - -Sample: Maestro Ciliegia regala il pezzo di legno al suo amico Geppetto, il quale lo prende per fabbricarsi un burattino maraviglioso: - A voi gli occhi, le guance! A voi il mio pezzo! -``` - -## Citation - -Please use the following bibtex entry: - -``` -@misc{mattei2020geppetto, - title={GePpeTto Carves Italian into a Language Model}, - author={Lorenzo De Mattei and Michele Cafagna and Felice Dell'Orletta and Malvina Nissim and Marco Guerini}, - year={2020}, - eprint={2004.14253}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - -## References - -Marco Baroni, Silvia Bernardini, Adriano Ferraresi, -and Eros Zanchetta. 2009. The WaCky wide web: a -collection of very large linguistically processed webcrawled corpora. Language resources and evaluation, 43(3):209–226. diff --git a/model_cards/Michau/t5-base-en-generate-headline/README.md b/model_cards/Michau/t5-base-en-generate-headline/README.md deleted file mode 100644 index 5e9add0d48..0000000000 --- a/model_cards/Michau/t5-base-en-generate-headline/README.md +++ /dev/null @@ -1,47 +0,0 @@ -## About the model - -The model has been trained on a collection of 500k articles with headings. Its purpose is to create a one-line heading suitable for the given article. - -Sample code with a WikiNews article: - -```python -import torch -from transformers import T5ForConditionalGeneration,T5Tokenizer - -device = torch.device("cuda" if torch.cuda.is_available() else "cpu") - -model = T5ForConditionalGeneration.from_pretrained("Michau/t5-base-en-generate-headline") -tokenizer = T5Tokenizer.from_pretrained("Michau/t5-base-en-generate-headline") -model = model.to(device) - -article = ''' -Very early yesterday morning, the United States President Donald Trump reported he and his wife First Lady Melania Trump tested positive for COVID-19. Officials said the Trumps' 14-year-old son Barron tested negative as did First Family and Senior Advisors Jared Kushner and Ivanka Trump. -Trump took to social media, posting at 12:54 am local time (0454 UTC) on Twitter, "Tonight, [Melania] and I tested positive for COVID-19. We will begin our quarantine and recovery process immediately. We will get through this TOGETHER!" Yesterday afternoon Marine One landed on the White House's South Lawn flying Trump to Walter Reed National Military Medical Center (WRNMMC) in Bethesda, Maryland. -Reports said both were showing "mild symptoms". Senior administration officials were tested as people were informed of the positive test. Senior advisor Hope Hicks had tested positive on Thursday. -Presidential physician Sean Conley issued a statement saying Trump has been given zinc, vitamin D, Pepcid and a daily Aspirin. Conley also gave a single dose of the experimental polyclonal antibodies drug from Regeneron Pharmaceuticals. -According to official statements, Trump, now operating from the WRNMMC, is to continue performing his duties as president during a 14-day quarantine. In the event of Trump becoming incapacitated, Vice President Mike Pence could take over the duties of president via the 25th Amendment of the US Constitution. The Pence family all tested negative as of yesterday and there were no changes regarding Pence's campaign events. -''' - -text = "headline: " + article - -max_len = 256 - -encoding = tokenizer.encode_plus(text, return_tensors = "pt") -input_ids = encoding["input_ids"].to(device) -attention_masks = encoding["attention_mask"].to(device) - -beam_outputs = model.generate( - input_ids = input_ids, - attention_mask = attention_masks, - max_length = 64, - num_beams = 3, - early_stopping = True, -) - -result = tokenizer.decode(beam_outputs[0]) -print(result) -``` - -Result: - -```Trump and First Lady Melania Test Positive for COVID-19``` diff --git a/model_cards/MoseliMotsoehli/TswanaBert/README.md b/model_cards/MoseliMotsoehli/TswanaBert/README.md deleted file mode 100644 index 5a1a483dd4..0000000000 --- a/model_cards/MoseliMotsoehli/TswanaBert/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: tn ---- - -# TswanaBert -Pretrained model on the Tswana language using a masked language modeling (MLM) objective. - -## Model Description. -TswanaBERT is a transformer model pre-trained on a corpus of Setswana in a self-supervised fashion by masking part of the input words and training to predict the masks by using byte-level tokens. - -## Intended uses & limitations -The model can be used for either masked language modeling or next word prediction. It can also be fine-tuned on a specific down-stream NLP application. - -#### How to use - -```python ->>> from transformers import pipeline ->>> from transformers import AutoTokenizer, AutoModelWithLMHead - ->>> tokenizer = AutoTokenizer.from_pretrained("MoseliMotsoehli/TswanaBert") ->>> model = AutoModelWithLMHead.from_pretrained("MoseliMotsoehli/TswanaBert") ->>> unmasker = pipeline('fill-mask', model=model, tokenizer=tokenizer) ->>> unmasker("Ntshopotse e godile.") - -[{'score': 0.32749542593955994, - 'sequence': 'Ntshopotse setse e godile.', - 'token': 538, - 'token_str': 'Ġsetse'}, - {'score': 0.060260992497205734, - 'sequence': 'Ntshopotse le e godile.', - 'token': 270, - 'token_str': 'Ġle'}, - {'score': 0.058460816740989685, - 'sequence': 'Ntshopotse bone e godile.', - 'token': 364, - 'token_str': 'Ġbone'}, - {'score': 0.05694682151079178, - 'sequence': 'Ntshopotse ga e godile.', - 'token': 298, - 'token_str': 'Ġga'}, - {'score': 0.0565204992890358, - 'sequence': 'Ntshopotse, e godile.', - 'token': 16, - 'token_str': ','}] -``` - -#### Limitations and bias -The model is trained on a relatively small collection of setwana, mostly from news articles and creative writtings, and so is not representative enough of the language as yet. - -## Training data - -1. The largest portion of this dataset (10k) sentences of text, comes from the [Leipzig Corpora Collection](https://wortschatz.uni-leipzig.de/en/download) - -2. I Then added SABC news headlines collected by Marivate Vukosi, & Sefara Tshephisho, (2020) that is generously made available on [zenoodo](http://doi.org/10.5281/zenodo.3668495 ). This added 185 tswana sentences to my corpus. - -3. I went on to add 300 more sentences by scrapping following news sites and blogs that mosty originate in Botswana. I actively continue to expand the dataset. - -* http://setswana.blogspot.com/ -* https://omniglot.com/writing/tswana.php -* http://www.dailynews.gov.bw/ -* http://www.mmegi.bw/index.php -* https://tsena.co.bw -* http://www.botswana.co.za/Cultural_Issues-travel/botswana-country-guide-en-route.html -* https://www.poemhunter.com/poem/2013-setswana/ -https://www.poemhunter.com/poem/ngwana-wa-mosetsana/ - - -### BibTeX entry and citation info - -```bibtex -@inproceedings{author = {Moseli Motsoehli}, - year={2020} -} -``` diff --git a/model_cards/MoseliMotsoehli/zuBERTa/README.md b/model_cards/MoseliMotsoehli/zuBERTa/README.md deleted file mode 100644 index 7e8fe6a844..0000000000 --- a/model_cards/MoseliMotsoehli/zuBERTa/README.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -language: zu ---- - -# zuBERTa -zuBERTa is a RoBERTa style transformer language model trained on zulu text. - -## Intended uses & limitations -The model can be used for getting embeddings to use on a down-stream task such as question answering. - -#### How to use - -```python ->>> from transformers import pipeline ->>> from transformers import AutoTokenizer, AutoModelWithLMHead - ->>> tokenizer = AutoTokenizer.from_pretrained("MoseliMotsoehli/zuBERTa") ->>> model = AutoModelWithLMHead.from_pretrained("MoseliMotsoehli/zuBERTa") ->>> unmasker = pipeline('fill-mask', model=model, tokenizer=tokenizer) ->>> unmasker("Abafika eNkandla bafika sebeholwa uMpongo kaZingelwayo.") - -[ - { - "sequence": "Abafika eNkandla bafika sebeholwa khona uMpongo kaZingelwayo.", - "score": 0.050459690392017365, - "token": 555, - "token_str": "Ġkhona" - }, - { - "sequence": "Abafika eNkandla bafika sebeholwa inkosi uMpongo kaZingelwayo.", - "score": 0.03668094798922539, - "token": 2321, - "token_str": "Ġinkosi" - }, - { - "sequence": "Abafika eNkandla bafika sebeholwa ubukhosi uMpongo kaZingelwayo.", - "score": 0.028774697333574295, - "token": 5101, - "token_str": "Ġubukhosi" - } -] -``` - -## Training data - -1. 30k sentences of text, came from the [Leipzig Corpora Collection](https://wortschatz.uni-leipzig.de/en/download) of zulu 2018. These were collected from news articles and creative writtings. -2. ~7500 articles of human generated translations were scraped from the zulu [wikipedia](https://zu.wikipedia.org/wiki/Special:AllPages). - -### BibTeX entry and citation info - -```bibtex -@inproceedings{author = {Moseli Motsoehli}, - title = {Towards transformation of Southern African language models through transformers.}, - year={2020} -} -``` diff --git a/model_cards/Musixmatch/umberto-commoncrawl-cased-v1/README.md b/model_cards/Musixmatch/umberto-commoncrawl-cased-v1/README.md deleted file mode 100644 index c48ce71a2a..0000000000 --- a/model_cards/Musixmatch/umberto-commoncrawl-cased-v1/README.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -language: it ---- - -# UmBERTo Commoncrawl Cased - -[UmBERTo](https://github.com/musixmatchresearch/umberto) is a Roberta-based Language Model trained on large Italian Corpora and uses two innovative approaches: SentencePiece and Whole Word Masking. Now available at [github.com/huggingface/transformers](https://huggingface.co/Musixmatch/umberto-commoncrawl-cased-v1) - -

-
- Marco Lodola, Monument to Umberto Eco, Alessandria 2019 -

- -## Dataset -UmBERTo-Commoncrawl-Cased utilizes the Italian subcorpus of [OSCAR](https://traces1.inria.fr/oscar/) as training set of the language model. We used deduplicated version of the Italian corpus that consists in 70 GB of plain text data, 210M sentences with 11B words where the sentences have been filtered and shuffled at line level in order to be used for NLP research. - -## Pre-trained model - -| Model | WWM | Cased | Tokenizer | Vocab Size | Train Steps | Download | -| ------ | ------ | ------ | ------ | ------ |------ | ------ | -| `umberto-commoncrawl-cased-v1` | YES | YES | SPM | 32K | 125k | [Link](http://bit.ly/35zO7GH) | - -This model was trained with [SentencePiece](https://github.com/google/sentencepiece) and Whole Word Masking. - -## Downstream Tasks -These results refers to umberto-commoncrawl-cased model. All details are at [Umberto](https://github.com/musixmatchresearch/umberto) Official Page. - -#### Named Entity Recognition (NER) - -| Dataset | F1 | Precision | Recall | Accuracy | -| ------ | ------ | ------ | ------ | ------ | -| **ICAB-EvalITA07** | **87.565** | 86.596 | 88.556 | 98.690 | -| **WikiNER-ITA** | **92.531** | 92.509 | 92.553 | 99.136 | - -#### Part of Speech (POS) - -| Dataset | F1 | Precision | Recall | Accuracy | -| ------ | ------ | ------ | ------ | ------ | -| **UD_Italian-ISDT** | 98.870 | 98.861 | 98.879 | **98.977** | -| **UD_Italian-ParTUT** | 98.786 | 98.812 | 98.760 | **98.903** | - - - -## Usage - -##### Load UmBERTo with AutoModel, Autotokenizer: - -```python - -import torch -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Musixmatch/umberto-commoncrawl-cased-v1") -umberto = AutoModel.from_pretrained("Musixmatch/umberto-commoncrawl-cased-v1") - -encoded_input = tokenizer.encode("Umberto Eco è stato un grande scrittore") -input_ids = torch.tensor(encoded_input).unsqueeze(0) # Batch size 1 -outputs = umberto(input_ids) -last_hidden_states = outputs[0] # The last hidden-state is the first element of the output -``` - -##### Predict masked token: - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="Musixmatch/umberto-commoncrawl-cased-v1", - tokenizer="Musixmatch/umberto-commoncrawl-cased-v1" -) - -result = fill_mask("Umberto Eco è un grande scrittore") -# {'sequence': ' Umberto Eco è considerato un grande scrittore', 'score': 0.18599839508533478, 'token': 5032} -# {'sequence': ' Umberto Eco è stato un grande scrittore', 'score': 0.17816807329654694, 'token': 471} -# {'sequence': ' Umberto Eco è sicuramente un grande scrittore', 'score': 0.16565583646297455, 'token': 2654} -# {'sequence': ' Umberto Eco è indubbiamente un grande scrittore', 'score': 0.0932890921831131, 'token': 17908} -# {'sequence': ' Umberto Eco è certamente un grande scrittore', 'score': 0.054701317101716995, 'token': 5269} -``` - - -## Citation -All of the original datasets are publicly available or were released with the owners' grant. The datasets are all released under a CC0 or CCBY license. - -* UD Italian-ISDT Dataset [Github](https://github.com/UniversalDependencies/UD_Italian-ISDT) -* UD Italian-ParTUT Dataset [Github](https://github.com/UniversalDependencies/UD_Italian-ParTUT) -* I-CAB (Italian Content Annotation Bank), EvalITA [Page](http://www.evalita.it/) -* WIKINER [Page](https://figshare.com/articles/Learning_multilingual_named_entity_recognition_from_Wikipedia/5462500) , [Paper](https://www.sciencedirect.com/science/article/pii/S0004370212000276?via%3Dihub) - -``` -@inproceedings {magnini2006annotazione, - title = {Annotazione di contenuti concettuali in un corpus italiano: I - CAB}, - author = {Magnini,Bernardo and Cappelli,Amedeo and Pianta,Emanuele and Speranza,Manuela and Bartalesi Lenzi,V and Sprugnoli,Rachele and Romano,Lorenza and Girardi,Christian and Negri,Matteo}, - booktitle = {Proc.of SILFI 2006}, - year = {2006} -} -@inproceedings {magnini2006cab, - title = {I - CAB: the Italian Content Annotation Bank.}, - author = {Magnini,Bernardo and Pianta,Emanuele and Girardi,Christian and Negri,Matteo and Romano,Lorenza and Speranza,Manuela and Lenzi,Valentina Bartalesi and Sprugnoli,Rachele}, - booktitle = {LREC}, - pages = {963--968}, - year = {2006}, - organization = {Citeseer} -} -``` - -## Authors - -**Loreto Parisi**: `loreto at musixmatch dot com`, [loretoparisi](https://github.com/loretoparisi) -**Simone Francia**: `simone.francia at musixmatch dot com`, [simonefrancia](https://github.com/simonefrancia) -**Paolo Magnani**: `paul.magnani95 at gmail dot com`, [paulthemagno](https://github.com/paulthemagno) - -## About Musixmatch AI -![Musxmatch Ai mac app icon-128](https://user-images.githubusercontent.com/163333/72244273-396aa380-35ee-11ea-894b-4ea48230c02b.png) -We do Machine Learning and Artificial Intelligence @[musixmatch](https://twitter.com/Musixmatch) -Follow us on [Twitter](https://twitter.com/musixmatchai) [Github](https://github.com/musixmatchresearch) - - diff --git a/model_cards/Musixmatch/umberto-wikipedia-uncased-v1/README.md b/model_cards/Musixmatch/umberto-wikipedia-uncased-v1/README.md deleted file mode 100644 index bbb559bf02..0000000000 --- a/model_cards/Musixmatch/umberto-wikipedia-uncased-v1/README.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -language: it ---- - -# UmBERTo Wikipedia Uncased - -[UmBERTo](https://github.com/musixmatchresearch/umberto) is a Roberta-based Language Model trained on large Italian Corpora and uses two innovative approaches: SentencePiece and Whole Word Masking. Now available at [github.com/huggingface/transformers](https://huggingface.co/Musixmatch/umberto-commoncrawl-cased-v1) - -

-
- Marco Lodola, Monument to Umberto Eco, Alessandria 2019 -

- -## Dataset -UmBERTo-Wikipedia-Uncased Training is trained on a relative small corpus (~7GB) extracted from [Wikipedia-ITA](https://linguatools.org/tools/corpora/wikipedia-monolingual-corpora/). - -## Pre-trained model - -| Model | WWM | Cased | Tokenizer | Vocab Size | Train Steps | Download | -| ------ | ------ | ------ | ------ | ------ |------ | ------ | -| `umberto-wikipedia-uncased-v1` | YES | YES | SPM | 32K | 100k | [Link](http://bit.ly/35wbSj6) | - -This model was trained with [SentencePiece](https://github.com/google/sentencepiece) and Whole Word Masking. - -## Downstream Tasks -These results refers to umberto-wikipedia-uncased model. All details are at [Umberto](https://github.com/musixmatchresearch/umberto) Official Page. - -#### Named Entity Recognition (NER) - -| Dataset | F1 | Precision | Recall | Accuracy | -| ------ | ------ | ------ | ------ | ----- | -| **ICAB-EvalITA07** | **86.240** | 85.939 | 86.544 | 98.534 | -| **WikiNER-ITA** | **90.483** | 90.328 | 90.638 | 98.661 | - -#### Part of Speech (POS) - -| Dataset | F1 | Precision | Recall | Accuracy | -| ------ | ------ | ------ | ------ | ------ | -| **UD_Italian-ISDT** | 98.563 | 98.508 | 98.618 | **98.717** | -| **UD_Italian-ParTUT** | 97.810 | 97.835 | 97.784 | **98.060** | - - - -## Usage - -##### Load UmBERTo Wikipedia Uncased with AutoModel, Autotokenizer: - -```python - -import torch -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("Musixmatch/umberto-wikipedia-uncased-v1") -umberto = AutoModel.from_pretrained("Musixmatch/umberto-wikipedia-uncased-v1") - -encoded_input = tokenizer.encode("Umberto Eco è stato un grande scrittore") -input_ids = torch.tensor(encoded_input).unsqueeze(0) # Batch size 1 -outputs = umberto(input_ids) -last_hidden_states = outputs[0] # The last hidden-state is the first element of the output -``` - -##### Predict masked token: - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="Musixmatch/umberto-wikipedia-uncased-v1", - tokenizer="Musixmatch/umberto-wikipedia-uncased-v1" -) - -result = fill_mask("Umberto Eco è un grande scrittore") -# {'sequence': ' umberto eco è stato un grande scrittore', 'score': 0.5784581303596497, 'token': 361} -# {'sequence': ' umberto eco è anche un grande scrittore', 'score': 0.33813193440437317, 'token': 269} -# {'sequence': ' umberto eco è considerato un grande scrittore', 'score': 0.027196012437343597, 'token': 3236} -# {'sequence': ' umberto eco è diventato un grande scrittore', 'score': 0.013716378249228, 'token': 5742} -# {'sequence': ' umberto eco è inoltre un grande scrittore', 'score': 0.010662357322871685, 'token': 1030} -``` - - -## Citation -All of the original datasets are publicly available or were released with the owners' grant. The datasets are all released under a CC0 or CCBY license. - -* UD Italian-ISDT Dataset [Github](https://github.com/UniversalDependencies/UD_Italian-ISDT) -* UD Italian-ParTUT Dataset [Github](https://github.com/UniversalDependencies/UD_Italian-ParTUT) -* I-CAB (Italian Content Annotation Bank), EvalITA [Page](http://www.evalita.it/) -* WIKINER [Page](https://figshare.com/articles/Learning_multilingual_named_entity_recognition_from_Wikipedia/5462500) , [Paper](https://www.sciencedirect.com/science/article/pii/S0004370212000276?via%3Dihub) - -``` -@inproceedings {magnini2006annotazione, - title = {Annotazione di contenuti concettuali in un corpus italiano: I - CAB}, - author = {Magnini,Bernardo and Cappelli,Amedeo and Pianta,Emanuele and Speranza,Manuela and Bartalesi Lenzi,V and Sprugnoli,Rachele and Romano,Lorenza and Girardi,Christian and Negri,Matteo}, - booktitle = {Proc.of SILFI 2006}, - year = {2006} -} -@inproceedings {magnini2006cab, - title = {I - CAB: the Italian Content Annotation Bank.}, - author = {Magnini,Bernardo and Pianta,Emanuele and Girardi,Christian and Negri,Matteo and Romano,Lorenza and Speranza,Manuela and Lenzi,Valentina Bartalesi and Sprugnoli,Rachele}, - booktitle = {LREC}, - pages = {963--968}, - year = {2006}, - organization = {Citeseer} -} -``` - -## Authors - -**Loreto Parisi**: `loreto at musixmatch dot com`, [loretoparisi](https://github.com/loretoparisi) -**Simone Francia**: `simone.francia at musixmatch dot com`, [simonefrancia](https://github.com/simonefrancia) -**Paolo Magnani**: `paul.magnani95 at gmail dot com`, [paulthemagno](https://github.com/paulthemagno) - -## About Musixmatch AI -![Musxmatch Ai mac app icon-128](https://user-images.githubusercontent.com/163333/72244273-396aa380-35ee-11ea-894b-4ea48230c02b.png) -We do Machine Learning and Artificial Intelligence @[musixmatch](https://twitter.com/Musixmatch) -Follow us on [Twitter](https://twitter.com/musixmatchai) [Github](https://github.com/musixmatchresearch) - diff --git a/model_cards/NLP4H/ms_bert/README.md b/model_cards/NLP4H/ms_bert/README.md deleted file mode 100644 index c5a1513455..0000000000 --- a/model_cards/NLP4H/ms_bert/README.md +++ /dev/null @@ -1,54 +0,0 @@ -# MS-BERT - -## Introduction - -This repository provides codes and models of MS-BERT. -MS-BERT was pre-trained on notes from neurological examination for Multiple Sclerosis (MS) patients at St. Michael's Hospital in Toronto, Canada. - -## Data - -The dataset contained approximately 75,000 clinical notes, for about 5000 patients, totaling to over 35.7 million words. -These notes were collected from patients who visited St. Michael's Hospital MS Clinic between 2015 to 2019. -The notes contained a variety of information pertaining to a neurological exam. -For example, a note can contain information on the patient's condition, their progress over time and diagnosis. -The gender split within the dataset was observed to be 72% female and 28% male ([which reflects the natural discrepancy seen in MS][1]). -Further sections will describe how MS-BERT was pre trained through the use of these clinically relevant and rich neurological notes. - -## Data pre-processing - -The data was pre-processed to remove any identifying information. This includes information on: patient names, doctor names, hospital names, patient identification numbers, phone numbers, addresses, and time. In order to de-identify the information, we used a curated database that contained patient and doctor information. This curated database was paired with regular expressions to find and remove any identifying pieces of information. Each of these identifiers were replaced with a specific token. These tokens were chosen based on three criteria: (1) they belong to the current BERT vocab, (2), they have relatively the same semantic meaning as the word they are replacing, and (3), the token is not found in the original unprocessed dataset. The replacements that met the criteria above were as follows: - -Female first names -> Lucie - -Male first names -> Ezekiel - -Last/family names -> Salamanca. - -Dates -> 2010s - -Patient IDs -> 999 - -Phone numbers -> 1718 - -Addresses -> Silesia - -Time -> 1610 - -Locations/Hospital/Clinic names -> Troy - -## Pre-training - -The starting point for our model is the already pre-trained and fine-tuned BLUE-BERT base. We further pre-train it using the masked language modelling task from the huggingface transformers [library](https://github.com/huggingface). - -The hyperparameters can be found in the config file in this repository or [here](https://s3.amazonaws.com/models.huggingface.co/bert/NLP4H/ms_bert/config.json) - -## Acknowledgements - -We would like to thank the researchers and staff at the Data Science and Advanced Analytics (DSAA) department, St. Michael’s Hospital, for providing consistent support and guidance throughout this project. -We would also like to thank Dr. Marzyeh Ghassemi, Taylor Killan, Nathan Ng and Haoran Zhang for providing us the opportunity to work on this exciting project. - -## Disclaimer - -MS-BERT shows the results of research conducted at the Data Science and Advanced Analytics (DSAA) department, St. Michael’s Hospital. The results produced by MS-BERT are not intended for direct diagnostic use or medical decision-making without review and oversight by a clinical professional. Individuals should not make decisions about their health solely on the basis of the results produced by MS-BERT. St. Michael’s Hospital does not independently verify the validity or utility of the results produced by MS-BERT. If you have questions about the results produced by MS-BERT please consult a healthcare professional. If you would like more information about the research conducted at DSAA please contact [Zhen Yang](mailto:zhen.yang@unityhealth.to). If you would like more information on neurological examination notes please contact [Dr. Tony Antoniou](mailto:tony.antoniou@unityhealth.to) or [Dr. Jiwon Oh](mailto:jiwon.oh@unityhealth.to) from the MS clinic at St. Michael's Hospital. - -[1]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3707353/ diff --git a/model_cards/Naveen-k/KanBERTo/README.md b/model_cards/Naveen-k/KanBERTo/README.md deleted file mode 100644 index 5dd913682f..0000000000 --- a/model_cards/Naveen-k/KanBERTo/README.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -language: kn ---- - -# Welcome to KanBERTo (ಕನ್ಬರ್ಟೋ) - -## Model Description - -> This is a small language model for [Kannada](https://en.wikipedia.org/wiki/Kannada) language with 1M data samples taken from - [OSCAR page](https://traces1.inria.fr/oscar/files/compressed-orig/kn.txt.gz) - -## Training params - -- **Dataset** - 1M data samples are used to train this model from OSCAR page(https://traces1.inria.fr/oscar/) eventhough data set is of 1.7 GB due to resource constraint to train -I have picked only 1M data from the total 1.7GB data set. If you are interested in collaboration and have computational resources to train on you are most welcome to do so. - -- **Preprocessing** - ByteLevelBPETokenizer is used to tokenize the sentences at character level and vocabulary size is set to 52k as per standard values given by 🤗 -- **Hyperparameters** - __ByteLevelBPETokenizer__ : vocabulary size = 52_000 and min_frequency = 2 - __Trainer__ : num_train_epochs=12 - trained for 12 epochs - per_gpu_train_batch_size=64 - batch size for the datasamples is 64 - save_steps=10_000 - save model for every 10k steps - save_total_limit=2 - save limit is set for 2 - -**Intended uses & limitations** - this is for anyone who wants to make use of kannada language models for various tasks like language generation, translation and many more use cases. - -**Whatever else is helpful!** - If you are intersted in collaboration feel free to reach me [Naveen](mailto:naveen.maltesh@gmail.com) diff --git a/model_cards/NeuML/bert-small-cord19-squad2/README.md b/model_cards/NeuML/bert-small-cord19-squad2/README.md deleted file mode 100644 index 94e86be7e9..0000000000 --- a/model_cards/NeuML/bert-small-cord19-squad2/README.md +++ /dev/null @@ -1,26 +0,0 @@ -# BERT-Small CORD-19 fine-tuned on SQuAD 2.0 - -[bert-small-cord19 model](https://huggingface.co/NeuML/bert-small-cord19) fine-tuned on SQuAD 2.0 - -## Building the model - -```bash -python run_squad.py - --model_type bert - --model_name_or_path bert-small-cord19 - --do_train - --do_eval - --do_lower_case - --version_2_with_negative - --train_file train-v2.0.json - --predict_file dev-v2.0.json - --per_gpu_train_batch_size 8 - --learning_rate 3e-5 - --num_train_epochs 3.0 - --max_seq_length 384 - --doc_stride 128 - --output_dir bert-small-cord19-squad2 - --save_steps 0 - --threads 8 - --overwrite_cache - --overwrite_output_dir diff --git a/model_cards/NeuML/bert-small-cord19/README.md b/model_cards/NeuML/bert-small-cord19/README.md deleted file mode 100644 index 60486535e8..0000000000 --- a/model_cards/NeuML/bert-small-cord19/README.md +++ /dev/null @@ -1,25 +0,0 @@ -# BERT-Small fine-tuned on CORD-19 dataset - -[BERT L6_H-512_A-8 model](https://huggingface.co/google/bert_uncased_L-6_H-512_A-8) fine-tuned on the [CORD-19 dataset](https://www.semanticscholar.org/cord19). - -## CORD-19 data subset -The training data for this dataset is stored as a [Kaggle dataset](https://www.kaggle.com/davidmezzetti/cord19-qa?select=cord19.txt). The training -data is a subset of the full corpus, focusing on high-quality, study-design detected articles. - -## Building the model - -```bash -python run_language_modeling.py - --model_type bert - --model_name_or_path google/bert_uncased_L-6_H-512_A-8 - --do_train - --mlm - --line_by_line - --block_size 512 - --train_data_file cord19.txt - --per_gpu_train_batch_size 4 - --learning_rate 3e-5 - --num_train_epochs 3.0 - --output_dir bert-small-cord19 - --save_steps 0 - --overwrite_output_dir diff --git a/model_cards/NeuML/bert-small-cord19qa/README.md b/model_cards/NeuML/bert-small-cord19qa/README.md deleted file mode 100644 index f9f3c492a2..0000000000 --- a/model_cards/NeuML/bert-small-cord19qa/README.md +++ /dev/null @@ -1,63 +0,0 @@ -# BERT-Small fine-tuned on CORD-19 QA dataset - -[bert-small-cord19-squad model](https://huggingface.co/NeuML/bert-small-cord19-squad2) fine-tuned on the [CORD-19 QA dataset](https://www.kaggle.com/davidmezzetti/cord19-qa?select=cord19-qa.json). - -## CORD-19 QA dataset -The CORD-19 QA dataset is a SQuAD 2.0 formatted list of question, context, answer combinations covering the [CORD-19 dataset](https://www.semanticscholar.org/cord19). - -## Building the model - -```bash -python run_squad.py \ - --model_type bert \ - --model_name_or_path bert-small-cord19-squad \ - --do_train \ - --do_lower_case \ - --version_2_with_negative \ - --train_file cord19-qa.json \ - --per_gpu_train_batch_size 8 \ - --learning_rate 5e-5 \ - --num_train_epochs 10.0 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir bert-small-cord19qa \ - --save_steps 0 \ - --threads 8 \ - --overwrite_cache \ - --overwrite_output_dir -``` - -## Testing the model - -Example usage below: - -```python -from transformers import pipeline - -qa = pipeline( - "question-answering", - model="NeuML/bert-small-cord19qa", - tokenizer="NeuML/bert-small-cord19qa" -) - -qa({ - "question": "What is the median incubation period?", - "context": "The incubation period is around 5 days (range: 4-7 days) with a maximum of 12-13 day" -}) - -qa({ - "question": "What is the incubation period range?", - "context": "The incubation period is around 5 days (range: 4-7 days) with a maximum of 12-13 day" -}) - -qa({ - "question": "What type of surfaces does it persist?", - "context": "The virus can survive on surfaces for up to 72 hours such as plastic and stainless steel ." -}) -``` - -```json -{"score": 0.5970273583242793, "start": 32, "end": 38, "answer": "5 days"} -{"score": 0.999555868193891, "start": 39, "end": 56, "answer": "(range: 4-7 days)"} -{"score": 0.9992726505196998, "start": 61, "end": 88, "answer": "plastic and stainless steel"} -``` diff --git a/model_cards/NlpHUST/vibert4news-base-cased/README.md b/model_cards/NlpHUST/vibert4news-base-cased/README.md deleted file mode 100644 index e0c0177dc9..0000000000 --- a/model_cards/NlpHUST/vibert4news-base-cased/README.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -language: vn ---- -# BERT for Vietnamese is trained on more 20 GB news dataset - -Apply for task sentiment analysis on using [AIViVN's comments dataset](https://www.aivivn.com/contests/6) - -The model achieved 0.90268 on the public leaderboard, (winner's score is 0.90087) -Bert4news is used for a toolkit Vietnames(segmentation and Named Entity Recognition) at ViNLPtoolkit(https://github.com/bino282/ViNLP) - -***************New Mar 11 , 2020 *************** - -**[BERT](https://github.com/google-research/bert)** (from Google Research and the Toyota Technological Institute at Chicago) released with the paper [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://arxiv.org/abs/1810.04805). - -We use word sentencepiece, use basic bert tokenization and same config with bert base with lowercase = False. - -You can download trained model: -- [tensorflow](https://drive.google.com/file/d/1X-sRDYf7moS_h61J3L79NkMVGHP-P-k5/view?usp=sharing). -- [pytorch](https://drive.google.com/file/d/11aFSTpYIurn-oI2XpAmcCTccB_AonMOu/view?usp=sharing). - -Use with huggingface/transformers -``` bash -import torch -from transformers import AutoTokenizer,AutoModel -tokenizer= AutoTokenizer.from_pretrained("NlpHUST/vibert4news-base-cased") -bert_model = AutoModel.from_pretrained("NlpHUST/vibert4news-base-cased") - -line = "Tôi là sinh viên trường Bách Khoa Hà Nội ." -input_id = tokenizer.encode(line,add_special_tokens = True) -att_mask = [int(token_id > 0) for token_id in input_id] -input_ids = torch.tensor([input_id]) -att_masks = torch.tensor([att_mask]) -with torch.no_grad(): - features = bert_model(input_ids,att_masks) - -print(features) - -``` - -Run training with base config - -``` bash - -python train_pytorch.py \ - --model_path=bert4news.pytorch \ - --max_len=200 \ - --batch_size=16 \ - --epochs=6 \ - --lr=2e-5 - -``` - -### Contact information -For personal communication related to this project, please contact Nha Nguyen Van (nha282@gmail.com). - diff --git a/model_cards/Norod78/hewiki-articles-distilGPT2py-il/README.md b/model_cards/Norod78/hewiki-articles-distilGPT2py-il/README.md deleted file mode 100644 index 3cf994eba6..0000000000 --- a/model_cards/Norod78/hewiki-articles-distilGPT2py-il/README.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -language: he - -thumbnail: https://avatars1.githubusercontent.com/u/3617152?norod.jpg -widget: -- text: "<|startoftext|>החוק השני של מועדון קרב הוא" -- text: "<|startoftext|>ראש הממשלה בן גוריון" -- text: "<|startoftext|>למידת מכונה (סרט)" -- text: "<|startoftext|>מנשה פומפרניקל" -- text: "<|startoftext|>אי שוויון " - -license: mit ---- - - -# hewiki-articles-distilGPT2py-il - -## A tiny GPT2 model for generating Hebrew text - -A distilGPT2 sized model.
-Training data was hewiki-20200701-pages-articles-multistream.xml.bz2 from https://dumps.wikimedia.org/hewiki/20200701/
-XML has been converted to plain text using Wikipedia Extractor http://medialab.di.unipi.it/wiki/Wikipedia_Extractor
-I then added <|startoftext|> and <|endoftext|> markers and deleted empty lines.
- -#### How to use - -```python -import torch -import torch.nn as nn -from transformers import GPT2Tokenizer, GPT2LMHeadModel - -tokenizer = GPT2Tokenizer.from_pretrained("Norod78/hewiki-articles-distilGPT2py-il") -model = GPT2LMHeadModel.from_pretrained("Norod78/hewiki-articles-distilGPT2py-il").eval() - -bos_token = tokenizer.bos_token #Beginning of sentace -eos_token = tokenizer.eos_token #End of sentence - -def generate_word(model, tokens_tensor, temperature=1.0): - """ - Sample a word given a tensor of tokens of previous words from a model. Given - the words we have, sample a plausible word. Temperature is used for - controlling randomness. If using temperature==0 we simply use a greedy arg max. - Else, we sample from a multinomial distribution using a lower inverse - temperature to allow for more randomness to escape repetitions. - """ - with torch.no_grad(): - outputs = model(tokens_tensor) - predictions = outputs[0] - if temperature>0: - # Make the distribution more or less skewed based on the temperature - predictions = outputs[0]/temperature - # Sample from the distribution - softmax = nn.Softmax(dim=0) - predicted_index = torch.multinomial(softmax(predictions[0,-1,:]),1).item() - # Simply take the arg-max of the distribution - else: - predicted_index = torch.argmax(predictions[0, -1, :]).item() - # Decode the encoding to the corresponding word - predicted_text = tokenizer.decode([predicted_index]) - return predicted_text - -def generate_sentence(model, tokenizer, initial_text, temperature=1.0): - """ Generate a sentence given some initial text using a model and a tokenizer. - Returns the new sentence. """ - - # Encode a text inputs - text = "" - sentence = text - - # We avoid an infinite loop by setting a maximum range - for i in range(0,84): - indexed_tokens = tokenizer.encode(initial_text + text) - - # Convert indexed tokens in a PyTorch tensor - tokens_tensor = torch.tensor([indexed_tokens]) - - new_word = generate_word(model, tokens_tensor, temperature=temperature) - - # Here the temperature is slowly decreased with each generated word, - # this ensures that the sentence (ending) makes more sense. - # We don't decrease to a temperature of 0.0 to leave some randomness in. - if temperature<(1-0.008): - temperature += 0.008 - else: - temperature = 0.996 - - text = text+new_word - - # Stop generating new words when we have reached the end of the line or the poem - if eos_token in new_word: - # returns new sentence and whether poem is done - return (text.replace(eos_token,"").strip(), True) - elif '/' in new_word: - return (text.strip(), False) - elif bos_token in new_word: - return (text.replace(bos_token,"").strip(), False) - - return (text, True) - -for output_num in range(1,5): - init_text = "בוקר טוב" - text = bos_token + init_text - for i in range(0,84): - sentence = generate_sentence(model, tokenizer, text, temperature=0.9) - text = init_text + sentence[0] - print(text) - if (sentence[1] == True): - break -``` diff --git a/model_cards/Ogayo/Hel-ach-en/README.md b/model_cards/Ogayo/Hel-ach-en/README.md deleted file mode 100644 index bd38761483..0000000000 --- a/model_cards/Ogayo/Hel-ach-en/README.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -language: -- ach -- en -tags: -- translation -license: cc-by-4.0 -datasets: -- JW300 -metrics: -- bleu ---- - -# HEL-ACH-EN - -## Model description - -MT model translating Acholi to English initialized with weights from [opus-mt-luo-en](https://huggingface.co/Helsinki-NLP/opus-mt-luo-en) on HuggingFace. - -## Intended uses & limitations -Machine Translation experiments. Do not use for sensitive tasks. -#### How to use - -```python -# You can include sample code which will be formatted -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("Ogayo/Hel-ach-en") - -model = AutoModelForSeq2SeqLM.from_pretrained("Ogayo/Hel-ach-en") - -``` - -#### Limitations and bias - -Trained on Jehovah Witnesses data so contains theirs and Christian views. - -## Training data -Trained on OPUS JW300 data. -Initialized with weights from [opus-mt-luo-en](https://huggingface.co/Helsinki-NLP/opus-mt-luo-en?text=Bed+gi+nyasi+mar+chieng%27+nyuol+mopong%27+gi+mor%21#model_card) - -## Training procedure - -Remove duplicates and rows with no alphabetic characters. Used GPU -## Eval results -testset | BLEU ---- | --- -JW300.luo.en| 46.1 diff --git a/model_cards/Primer/bart-squad2/README.md b/model_cards/Primer/bart-squad2/README.md deleted file mode 100644 index df58603999..0000000000 --- a/model_cards/Primer/bart-squad2/README.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -language: "en" ---- - -# BART-Squad2 - -## Model description - -BART for extractive (span-based) question answering, trained on Squad 2.0. - -F1 score of 87.4. - -## Intended uses & limitations - -Unfortunately, the Huggingface auto-inference API won't run this model, so if you're attempting to try it through the input box above and it complains, don't be discouraged! - -#### How to use - -Here's a quick way to get question answering running locally: - -```python -from transformers import AutoTokenizer, AutoModelForQuestionAnswering - -tokenizer = AutoTokenizer.from_pretrained("Primer/bart-squad2") -model = AutoModelForQuestionAnswering.from_pretrained("Primer/bart-squad2") -model.to('cuda'); model.eval() - -def answer(question, text): - seq = '' + question + ' ' + text + ' ' - tokens = tokenizer.encode_plus(seq, return_tensors='pt', padding='max_length', max_length=1024) - input_ids = tokens['input_ids'].to('cuda') - attention_mask = tokens['attention_mask'].to('cuda') - start, end, _ = model(input_ids, attention_mask=attention_mask) - start_idx = int(start.argmax().int()) - end_idx = int(end.argmax().int()) - print(tokenizer.decode(input_ids[0, start_idx:end_idx]).strip()) - # ^^ it will be an empty string if the model decided "unanswerable" - ->>> question = "Where does Tom live?" ->>> context = "Tom is an engineer in San Francisco." ->>> answer(question, context) -San Francisco -``` - -(Just drop the `.to('cuda')` stuff if running on CPU). - -#### Limitations and bias - -Unknown, no further evaluation has been performed. In a technical sense one big limitation is that it's 1.6G 😬 - -## Training procedure - -`run_squad.py` with: - -|param|value| -|---|---| -|batch size|8| -|max_seq_length|1024| -|learning rate|1e-5| -|epochs|2| - -Modified to freeze shared parameters and encoder embeddings. - diff --git a/model_cards/README.md b/model_cards/README.md new file mode 100644 index 0000000000..4bf6ac6186 --- /dev/null +++ b/model_cards/README.md @@ -0,0 +1,26 @@ +## 🔥 Model cards now live inside each huggingface.co model repo 🔥 + + +For consistency, ease of use and scalability, `README.md` model cards now live directly inside each model repo on the HuggingFace model hub. + +### How to update a model card + +You can directly update a model card inside any model repo you have **write access** to, i.e.: +- a model under your username namespace +- a model under any organization you are a part of. + +You can either: +- update it, commit and push using your usual git workflow (command line, GUI, etc.) +- or edit it directly from the website's UI. + +**What if you want to create or update a model card for a model you don't have write access to?** + +In that case, given that we don't have a Pull request system yet on huggingface.co (🤯), +you can open an issue here, post the card's content, and tag the model author(s) and/or the Hugging Face team. + +We might implement a more seamless process at some point, so your early feedback is precious! +Please let us know of any suggestion. + +### What happened to the model cards here? + +We migrated every model card from the repo to its corresponding huggingface.co model repo. Individual commits were preserved, and they link back to the original commit on GitHub. diff --git a/model_cards/Rostlab/prot_bert/README.md b/model_cards/Rostlab/prot_bert/README.md deleted file mode 100644 index 75d576c5e0..0000000000 --- a/model_cards/Rostlab/prot_bert/README.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -language: protein -tags: -- protein language model -datasets: -- Uniref100 ---- - -# ProtBert model - -Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://doi.org/10.1101/2020.07.12.199554) and first released in -[this repository](https://github.com/agemagician/ProtTrans). This model is trained on uppercase amino acids: it only works with capital letter amino acids. - - -## Model description - -ProtBert is based on Bert model which pretrained on a large corpus of protein sequences in a self-supervised fashion. -This means it was pretrained on the raw protein sequences only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those protein sequences. - -One important difference between our Bert model and the original Bert version is the way of dealing with sequences as separate documents. -This means the Next sentence prediction is not used, as each sequence is treated as a complete document. -The masking follows the original Bert training with randomly masks 15% of the amino acids in the input. - -At the end, the feature extracted from this model revealed that the LM-embeddings from unlabeled data (only protein sequences) captured important biophysical properties governing protein -shape. -This implied learning some of the grammar of the language of life realized in protein sequences. - -## Intended uses & limitations - -The model could be used for protein feature extraction or to be fine-tuned on downstream tasks. -We have noticed in some tasks you could gain more accuracy by fine-tuning the model rather than using it as a feature extractor. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import BertForMaskedLM, BertTokenizer, pipeline ->>> tokenizer = BertTokenizer.from_pretrained("Rostlab/prot_bert", do_lower_case=False ) ->>> model = BertForMaskedLM.from_pretrained("Rostlab/prot_bert") ->>> unmasker = pipeline('fill-mask', model=model, tokenizer=tokenizer) ->>> unmasker('D L I P T S S K L V V [MASK] D T S L Q V K K A F F A L V T') - -[{'score': 0.11088453233242035, - 'sequence': '[CLS] D L I P T S S K L V V L D T S L Q V K K A F F A L V T [SEP]', - 'token': 5, - 'token_str': 'L'}, - {'score': 0.08402521163225174, - 'sequence': '[CLS] D L I P T S S K L V V S D T S L Q V K K A F F A L V T [SEP]', - 'token': 10, - 'token_str': 'S'}, - {'score': 0.07328339666128159, - 'sequence': '[CLS] D L I P T S S K L V V V D T S L Q V K K A F F A L V T [SEP]', - 'token': 8, - 'token_str': 'V'}, - {'score': 0.06921856850385666, - 'sequence': '[CLS] D L I P T S S K L V V K D T S L Q V K K A F F A L V T [SEP]', - 'token': 12, - 'token_str': 'K'}, - {'score': 0.06382402777671814, - 'sequence': '[CLS] D L I P T S S K L V V I D T S L Q V K K A F F A L V T [SEP]', - 'token': 11, - 'token_str': 'I'}] -``` - -Here is how to use this model to get the features of a given protein sequence in PyTorch: - -```python -from transformers import BertModel, BertTokenizer -import re -tokenizer = BertTokenizer.from_pretrained("Rostlab/prot_bert", do_lower_case=False ) -model = BertModel.from_pretrained("Rostlab/prot_bert") -sequence_Example = "A E T C Z A O" -sequence_Example = re.sub(r"[UZOB]", "X", sequence_Example) -encoded_input = tokenizer(sequence_Example, return_tensors='pt') -output = model(**encoded_input) -``` - -## Training data - -The ProtBert model was pretrained on [Uniref100](https://www.uniprot.org/downloads), a dataset consisting of 217 million protein sequences. - -## Training procedure - -### Preprocessing - -The protein sequences are uppercased and tokenized using a single space and a vocabulary size of 21. The rare amino acids "U,Z,O,B" were mapped to "X". -The inputs of the model are then of the form: - -``` -[CLS] Protein Sequence A [SEP] Protein Sequence B [SEP] -``` - -Furthermore, each protein sequence was treated as a separate document. -The preprocessing step was performed twice, once for a combined length (2 sequences) of less than 512 amino acids, and another time using a combined length (2 sequences) of less than 2048 amino acids. - -The details of the masking procedure for each sequence followed the original Bert model as following: -- 15% of the amino acids are masked. -- In 80% of the cases, the masked amino acids are replaced by `[MASK]`. -- In 10% of the cases, the masked amino acids are replaced by a random amino acid (different) from the one they replace. -- In the 10% remaining cases, the masked amino acids are left as is. - -### Pretraining - -The model was trained on a single TPU Pod V3-512 for 400k steps in total. -300K steps using sequence length 512 (batch size 15k), and 100K steps using sequence length 2048 (batch size 2.5k). -The optimizer used is Lamb with a learning rate of 0.002, a weight decay of 0.01, learning rate warmup for 40k steps and linear decay of the learning rate after. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Test results : - -| Task/Dataset | secondary structure (3-states) | secondary structure (8-states) | Localization | Membrane | -|:-----:|:-----:|:-----:|:-----:|:-----:| -| CASP12 | 75 | 63 | | | -| TS115 | 83 | 72 | | | -| CB513 | 81 | 66 | | | -| DeepLoc | | | 79 | 91 | - -### BibTeX entry and citation info - -```bibtex -@article {Elnaggar2020.07.12.199554, - author = {Elnaggar, Ahmed and Heinzinger, Michael and Dallago, Christian and Rehawi, Ghalia and Wang, Yu and Jones, Llion and Gibbs, Tom and Feher, Tamas and Angerer, Christoph and Steinegger, Martin and BHOWMIK, DEBSINDHU and Rost, Burkhard}, - title = {ProtTrans: Towards Cracking the Language of Life{\textquoteright}s Code Through Self-Supervised Deep Learning and High Performance Computing}, - elocation-id = {2020.07.12.199554}, - year = {2020}, - doi = {10.1101/2020.07.12.199554}, - publisher = {Cold Spring Harbor Laboratory}, - abstract = {Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models (LMs) taken from Natural Language Processing (NLP). These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive language models (Transformer-XL, XLNet) and two auto-encoder models (Bert, Albert) on data from UniRef and BFD containing up to 393 billion amino acids (words) from 2.1 billion protein sequences (22- and 112 times the entire English Wikipedia). The LMs were trained on the Summit supercomputer at Oak Ridge National Laboratory (ORNL), using 936 nodes (total 5616 GPUs) and one TPU Pod (V3-512 or V3-1024). We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure (3-states: Q3=76-84, 8 states: Q8=65-73), sub-cellular localization for 10 cellular compartments (Q10=74) and whether a protein is membrane-bound or water-soluble (Q2=89). Dimensionality reduction revealed that the LM-embeddings from unlabeled data (only protein sequences) captured important biophysical properties governing protein shape. This implied learning some of the grammar of the language of life realized in protein sequences. The successful up-scaling of protein LMs through HPC to larger data sets slightly reduced the gap between models trained on evolutionary information and LMs. Availability ProtTrans: \<a href="https://github.com/agemagician/ProtTrans"\>https://github.com/agemagician/ProtTrans\</a\>Competing Interest StatementThe authors have declared no competing interest.}, - URL = {https://www.biorxiv.org/content/early/2020/07/21/2020.07.12.199554}, - eprint = {https://www.biorxiv.org/content/early/2020/07/21/2020.07.12.199554.full.pdf}, - journal = {bioRxiv} -} -``` - -> Created by [Ahmed Elnaggar/@Elnaggar_AI](https://twitter.com/Elnaggar_AI) | [LinkedIn](https://www.linkedin.com/in/prof-ahmed-elnaggar/) diff --git a/model_cards/Rostlab/prot_bert_bfd/README.md b/model_cards/Rostlab/prot_bert_bfd/README.md deleted file mode 100644 index 0d20792412..0000000000 --- a/model_cards/Rostlab/prot_bert_bfd/README.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -language: protein -tags: -- protein language model -datasets: -- BFD ---- - -# ProtBert-BFD model - -Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://doi.org/10.1101/2020.07.12.199554) and first released in -[this repository](https://github.com/agemagician/ProtTrans). This model is trained on uppercase amino acids: it only works with capital letter amino acids. - - -## Model description - -ProtBert-BFD is based on Bert model which pretrained on a large corpus of protein sequences in a self-supervised fashion. -This means it was pretrained on the raw protein sequences only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those protein sequences. - -One important difference between our Bert model and the original Bert version is the way of dealing with sequences as separate documents -This means the Next sentence prediction is not used, as each sequence is treated as a complete document. -The masking follows the original Bert training with randomly masks 15% of the amino acids in the input. - -At the end, the feature extracted from this model revealed that the LM-embeddings from unlabeled data (only protein sequences) captured important biophysical properties governing protein -shape. -This implied learning some of the grammar of the language of life realized in protein sequences. - -## Intended uses & limitations - -The model could be used for protein feature extraction or to be fine-tuned on downstream tasks. -We have noticed in some tasks you could gain more accuracy by fine-tuning the model rather than using it as a feature extractor. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import BertForMaskedLM, BertTokenizer, pipeline ->>> tokenizer = BertTokenizer.from_pretrained('Rostlab/prot_bert_bfd', do_lower_case=False ) ->>> model = BertForMaskedLM.from_pretrained("Rostlab/prot_bert_bfd") ->>> unmasker = pipeline('fill-mask', model=model, tokenizer=tokenizer) ->>> unmasker('D L I P T S S K L V V [MASK] D T S L Q V K K A F F A L V T') - -[{'score': 0.1165614128112793, - 'sequence': '[CLS] D L I P T S S K L V V L D T S L Q V K K A F F A L V T [SEP]', - 'token': 5, - 'token_str': 'L'}, - {'score': 0.08976086974143982, - 'sequence': '[CLS] D L I P T S S K L V V V D T S L Q V K K A F F A L V T [SEP]', - 'token': 8, - 'token_str': 'V'}, - {'score': 0.08864385634660721, - 'sequence': '[CLS] D L I P T S S K L V V S D T S L Q V K K A F F A L V T [SEP]', - 'token': 10, - 'token_str': 'S'}, - {'score': 0.06227643042802811, - 'sequence': '[CLS] D L I P T S S K L V V A D T S L Q V K K A F F A L V T [SEP]', - 'token': 6, - 'token_str': 'A'}, - {'score': 0.06194969266653061, - 'sequence': '[CLS] D L I P T S S K L V V T D T S L Q V K K A F F A L V T [SEP]', - 'token': 15, - 'token_str': 'T'}] -``` - -Here is how to use this model to get the features of a given protein sequence in PyTorch: - -```python -from transformers import BertModel, BertTokenizer -import re -tokenizer = BertTokenizer.from_pretrained('Rostlab/prot_bert_bfd', do_lower_case=False ) -model = BertModel.from_pretrained("Rostlab/prot_bert_bfd") -sequence_Example = "A E T C Z A O" -sequence_Example = re.sub(r"[UZOB]", "X", sequence_Example) -encoded_input = tokenizer(sequence_Example, return_tensors='pt') -output = model(**encoded_input) -``` - -## Training data - -The ProtBert-BFD model was pretrained on [BFD](https://bfd.mmseqs.com/), a dataset consisting of 2.1 billion protein sequences. - -## Training procedure - -### Preprocessing - -The protein sequences are uppercased and tokenized using a single space and a vocabulary size of 21. -The inputs of the model are then of the form: - -``` -[CLS] Protein Sequence A [SEP] Protein Sequence B [SEP] -``` - -Furthermore, each protein sequence was treated as a separate document. -The preprocessing step was performed twice, once for a combined length (2 sequences) of less than 512 amino acids, and another time using a combined length (2 sequences) of less than 2048 amino acids. - -The details of the masking procedure for each sequence followed the original Bert model as following: -- 15% of the amino acids are masked. -- In 80% of the cases, the masked amino acids are replaced by `[MASK]`. -- In 10% of the cases, the masked amino acids are replaced by a random amino acid (different) from the one they replace. -- In the 10% remaining cases, the masked amino acids are left as is. - -### Pretraining - -The model was trained on a single TPU Pod V3-1024 for one million steps in total. -800k steps using sequence length 512 (batch size 32k), and 200K steps using sequence length 2048 (batch size 6k). -The optimizer used is Lamb with a learning rate of 0.002, a weight decay of 0.01, learning rate warmup for 140k steps and linear decay of the learning rate after. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Test results : - -| Task/Dataset | secondary structure (3-states) | secondary structure (8-states) | Localization | Membrane | -|:-----:|:-----:|:-----:|:-----:|:-----:| -| CASP12 | 76 | 65 | | | -| TS115 | 84 | 73 | | | -| CB513 | 83 | 70 | | | -| DeepLoc | | | 78 | 91 | - -### BibTeX entry and citation info - -```bibtex -@article {Elnaggar2020.07.12.199554, - author = {Elnaggar, Ahmed and Heinzinger, Michael and Dallago, Christian and Rehawi, Ghalia and Wang, Yu and Jones, Llion and Gibbs, Tom and Feher, Tamas and Angerer, Christoph and Steinegger, Martin and BHOWMIK, DEBSINDHU and Rost, Burkhard}, - title = {ProtTrans: Towards Cracking the Language of Life{\textquoteright}s Code Through Self-Supervised Deep Learning and High Performance Computing}, - elocation-id = {2020.07.12.199554}, - year = {2020}, - doi = {10.1101/2020.07.12.199554}, - publisher = {Cold Spring Harbor Laboratory}, - abstract = {Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models (LMs) taken from Natural Language Processing (NLP). These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive language models (Transformer-XL, XLNet) and two auto-encoder models (Bert, Albert) on data from UniRef and BFD containing up to 393 billion amino acids (words) from 2.1 billion protein sequences (22- and 112 times the entire English Wikipedia). The LMs were trained on the Summit supercomputer at Oak Ridge National Laboratory (ORNL), using 936 nodes (total 5616 GPUs) and one TPU Pod (V3-512 or V3-1024). We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure (3-states: Q3=76-84, 8 states: Q8=65-73), sub-cellular localization for 10 cellular compartments (Q10=74) and whether a protein is membrane-bound or water-soluble (Q2=89). Dimensionality reduction revealed that the LM-embeddings from unlabeled data (only protein sequences) captured important biophysical properties governing protein shape. This implied learning some of the grammar of the language of life realized in protein sequences. The successful up-scaling of protein LMs through HPC to larger data sets slightly reduced the gap between models trained on evolutionary information and LMs. Availability ProtTrans: \<a href="https://github.com/agemagician/ProtTrans"\>https://github.com/agemagician/ProtTrans\</a\>Competing Interest StatementThe authors have declared no competing interest.}, - URL = {https://www.biorxiv.org/content/early/2020/07/21/2020.07.12.199554}, - eprint = {https://www.biorxiv.org/content/early/2020/07/21/2020.07.12.199554.full.pdf}, - journal = {bioRxiv} -} -``` - -> Created by [Ahmed Elnaggar/@Elnaggar_AI](https://twitter.com/Elnaggar_AI) | [LinkedIn](https://www.linkedin.com/in/prof-ahmed-elnaggar/) diff --git a/model_cards/Rostlab/prot_t5_xl_bfd/README.md b/model_cards/Rostlab/prot_t5_xl_bfd/README.md deleted file mode 100644 index 418d4c32d2..0000000000 --- a/model_cards/Rostlab/prot_t5_xl_bfd/README.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -language: protein -tags: -- protein language model -datasets: -- BFD ---- - -# ProtT5-XL-BFD model - -Pretrained model on protein sequences using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://doi.org/10.1101/2020.07.12.199554) and first released in -[this repository](https://github.com/agemagician/ProtTrans). This model is trained on uppercase amino acids: it only works with capital letter amino acids. - - -## Model description - -ProtT5-XL-BFD is based on the `t5-3b` model and was pretrained on a large corpus of protein sequences in a self-supervised fashion. -This means it was pretrained on the raw protein sequences only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those protein sequences. - -One important difference between this T5 model and the original T5 version is the denosing objective. -The original T5-3B model was pretrained using a span denosing objective, while this model was pre-trained with a Bart-like MLM denosing objective. -The masking probability is consistent with the original T5 training by randomly masking 15% of the amino acids in the input. - -It has been shown that the features extracted from this self-supervised model (LM-embeddings) captured important biophysical properties governing protein shape. -shape. -This implied learning some of the grammar of the language of life realized in protein sequences. - -## Intended uses & limitations - -The model could be used for protein feature extraction or to be fine-tuned on downstream tasks. -We have noticed in some tasks on can gain more accuracy by fine-tuning the model rather than using it as a feature extractor. -We have also noticed that for feature extraction, its better to use the feature extracted from the encoder not from the decoder. - -### How to use - -Here is how to use this model to extract the features of a given protein sequence in PyTorch: - -```python -from transformers import T5Tokenizer, T5Model -import re -import torch - -tokenizer = T5Tokenizer.from_pretrained('Rostlab/prot_t5_xl_bfd', do_lower_case=False) - -model = T5Model.from_pretrained("Rostlab/prot_t5_xl_bfd") - -sequences_Example = ["A E T C Z A O","S K T Z P"] - -sequences_Example = [re.sub(r"[UZOB]", "X", sequence) for sequence in sequences_Example] - -ids = tokenizer.batch_encode_plus(sequences_Example, add_special_tokens=True, padding=True) - -input_ids = torch.tensor(ids['input_ids']) -attention_mask = torch.tensor(ids['attention_mask']) - -with torch.no_grad(): - embedding = model(input_ids=input_ids,attention_mask=attention_mask,decoder_input_ids=None) - -# For feature extraction we recommend to use the encoder embedding -encoder_embedding = embedding[2].cpu().numpy() -decoder_embedding = embedding[0].cpu().numpy() -``` - -## Training data - -The ProtT5-XL-BFD model was pretrained on [BFD](https://bfd.mmseqs.com/), a dataset consisting of 2.1 billion protein sequences. - -## Training procedure - -### Preprocessing - -The protein sequences are uppercased and tokenized using a single space and a vocabulary size of 21. The rare amino acids "U,Z,O,B" were mapped to "X". -The inputs of the model are then of the form: - -``` -Protein Sequence [EOS] -``` - -The preprocessing step was performed on the fly, by cutting and padding the protein sequences up to 512 tokens. - -The details of the masking procedure for each sequence are as follows: -- 15% of the amino acids are masked. -- In 90% of the cases, the masked amino acids are replaced by `[MASK]` token. -- In 10% of the cases, the masked amino acids are replaced by a random amino acid (different) from the one they replace. - -### Pretraining - -The model was trained on a single TPU Pod V3-1024 for 1.2 million steps in total, using sequence length 512 (batch size 4k). -It has a total of approximately 3B parameters and was trained using the encoder-decoder architecture. -The optimizer used is AdaFactor with inverse square root learning rate schedule for pre-training. - - -## Evaluation results - -When the model is used for feature etraction, this model achieves the following results: - -Test results : - -| Task/Dataset | secondary structure (3-states) | secondary structure (8-states) | Localization | Membrane | -|:-----:|:-----:|:-----:|:-----:|:-----:| -| CASP12 | 77 | 66 | | | -| TS115 | 85 | 74 | | | -| CB513 | 84 | 71 | | | -| DeepLoc | | | 77 | 91 | - -### BibTeX entry and citation info - -```bibtex -@article {Elnaggar2020.07.12.199554, - author = {Elnaggar, Ahmed and Heinzinger, Michael and Dallago, Christian and Rehawi, Ghalia and Wang, Yu and Jones, Llion and Gibbs, Tom and Feher, Tamas and Angerer, Christoph and Steinegger, Martin and BHOWMIK, DEBSINDHU and Rost, Burkhard}, - title = {ProtTrans: Towards Cracking the Language of Life{\textquoteright}s Code Through Self-Supervised Deep Learning and High Performance Computing}, - elocation-id = {2020.07.12.199554}, - year = {2020}, - doi = {10.1101/2020.07.12.199554}, - publisher = {Cold Spring Harbor Laboratory}, - abstract = {Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models (LMs) taken from Natural Language Processing (NLP). These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive language models (Transformer-XL, XLNet) and two auto-encoder models (Bert, Albert) on data from UniRef and BFD containing up to 393 billion amino acids (words) from 2.1 billion protein sequences (22- and 112 times the entire English Wikipedia). The LMs were trained on the Summit supercomputer at Oak Ridge National Laboratory (ORNL), using 936 nodes (total 5616 GPUs) and one TPU Pod (V3-512 or V3-1024). We validated the advantage of up-scaling LMs to larger models supported by bigger data by predicting secondary structure (3-states: Q3=76-84, 8 states: Q8=65-73), sub-cellular localization for 10 cellular compartments (Q10=74) and whether a protein is membrane-bound or water-soluble (Q2=89). Dimensionality reduction revealed that the LM-embeddings from unlabeled data (only protein sequences) captured important biophysical properties governing protein shape. This implied learning some of the grammar of the language of life realized in protein sequences. The successful up-scaling of protein LMs through HPC to larger data sets slightly reduced the gap between models trained on evolutionary information and LMs. Availability ProtTrans: \<a href="https://github.com/agemagician/ProtTrans"\>https://github.com/agemagician/ProtTrans\</a\>Competing Interest StatementThe authors have declared no competing interest.}, - URL = {https://www.biorxiv.org/content/early/2020/07/21/2020.07.12.199554}, - eprint = {https://www.biorxiv.org/content/early/2020/07/21/2020.07.12.199554.full.pdf}, - journal = {bioRxiv} -} -``` - -> Created by [Ahmed Elnaggar/@Elnaggar_AI](https://twitter.com/Elnaggar_AI) | [LinkedIn](https://www.linkedin.com/in/prof-ahmed-elnaggar/) diff --git a/model_cards/SZTAKI-HLT/hubert-base-cc/README.md b/model_cards/SZTAKI-HLT/hubert-base-cc/README.md deleted file mode 100644 index 96fe00f640..0000000000 --- a/model_cards/SZTAKI-HLT/hubert-base-cc/README.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -language: hu -license: apache-2.0 -datasets: -- common_crawl -- wikipedia ---- - -# huBERT base model (cased) - -## Model description - -Cased BERT model for Hungarian, trained on the (filtered, deduplicated) Hungarian subset of the Common Crawl and a snapshot of the Hungarian Wikipedia. - -## Intended uses & limitations - -The model can be used as any other (cased) BERT model. It has been tested on the chunking and -named entity recognition tasks and set a new state-of-the-art on the former. - -## Training - -Details of the training data and procedure can be found in the PhD thesis linked below. (With the caveat that it only contains preliminary results -based on the Wikipedia subcorpus. Evaluation of the full model will appear in a future paper.) - -## Eval results - -When fine-tuned (via `BertForTokenClassification`) on chunking and NER, the model outperforms multilingual BERT, achieves state-of-the-art results on the -former task and comes within 0.5% F1 to the SotA on the latter. The exact scores are - -| NER | Minimal NP | Maximal NP | -|-----|------------|------------| -| 97.62% | **97.14%** | **96.97%** | - -### BibTeX entry and citation info - -The training corpus, parameters and the evaluation methods are discussed in the -[following PhD thesis](https://hlt.bme.hu/en/publ/nemeskey_2020): - -```bibtex -@PhDThesis{ Nemeskey:2020, - author = {Nemeskey, Dávid Márk}, - title = {Natural Language Processing Methods for Language Modeling}, - year = {2020}, - school = {E\"otv\"os Lor\'and University} -} -``` diff --git a/model_cards/SparkBeyond/roberta-large-sts-b/README.md b/model_cards/SparkBeyond/roberta-large-sts-b/README.md deleted file mode 100644 index a32cb57486..0000000000 --- a/model_cards/SparkBeyond/roberta-large-sts-b/README.md +++ /dev/null @@ -1,50 +0,0 @@ - - -# Roberta Large STS-B - -This model is a fine tuned RoBERTA model over STS-B. -It was trained with these params: -!python /content/transformers/examples/text-classification/run_glue.py \ - --model_type roberta \ - --model_name_or_path roberta-large \ - --task_name STS-B \ - --do_train \ - --do_eval \ - --do_lower_case \ - --data_dir /content/glue_data/STS-B/ \ - --max_seq_length 128 \ - --per_gpu_eval_batch_size=8 \ - --per_gpu_train_batch_size=8 \ - --learning_rate 2e-5 \ - --num_train_epochs 3.0 \ - --output_dir /content/roberta-sts-b - - -## How to run - -```python - - - -import toolz -import torch -batch_size = 6 - -def roberta_similarity_batches(to_predict): - batches = toolz.partition(batch_size, to_predict) - similarity_scores = [] - for batch in batches: - sentences = [(sentence_similarity["sent1"], sentence_similarity["sent2"]) for sentence_similarity in batch] - batch_scores = similarity_roberta(model, tokenizer,sentences) - similarity_scores = similarity_scores + batch_scores[0].cpu().squeeze(axis=1).tolist() - return similarity_scores - -def similarity_roberta(model, tokenizer, sent_pairs): - batch_token = tokenizer(sent_pairs, padding='max_length', truncation=True, max_length=500) - res = model(torch.tensor(batch_token['input_ids']).cuda(), attention_mask=torch.tensor(batch_token["attention_mask"]).cuda()) - return res - -similarity_roberta(model, tokenizer, [('NEW YORK--(BUSINESS WIRE)--Rosen Law Firm, a global investor rights law firm, announces it is investigating potential securities claims on behalf of shareholders of Vale S.A. ( VALE ) resulting from allegations that Vale may have issued materially misleading business information to the investing public', - 'EQUITY ALERT: Rosen Law Firm Announces Investigation of Securities Claims Against Vale S.A. – VALE')]) - -``` diff --git a/model_cards/T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb/README.md b/model_cards/T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb/README.md deleted file mode 100644 index 250db8366d..0000000000 --- a/model_cards/T-Systems-onsite/bert-german-dbmdz-uncased-sentence-stsb/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -language: de -license: mit ---- - -# bert-german-dbmdz-uncased-sentence-stsb -**This model is outdated!** - -The new [T-Systems-onsite/cross-en-de-roberta-sentence-transformer](https://huggingface.co/T-Systems-onsite/cross-en-de-roberta-sentence-transformer) model is better for German language. It is also the current best model for English language and works cross-lingually. Please consider using that model. \ No newline at end of file diff --git a/model_cards/T-Systems-onsite/cross-en-de-roberta-sentence-transformer/README.md b/model_cards/T-Systems-onsite/cross-en-de-roberta-sentence-transformer/README.md deleted file mode 100644 index a1790cf421..0000000000 --- a/model_cards/T-Systems-onsite/cross-en-de-roberta-sentence-transformer/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: -- de -- en -license: mit -tags: -- sentence_embedding -- search -- pytorch -- xlm-roberta -- roberta -- xlm-r-distilroberta-base-paraphrase-v1 -- paraphrase -datasets: -- STSbenchmark -metrics: -- Spearman’s rank correlation -- cosine similarity ---- - -# Cross English & German RoBERTa for Sentence Embeddings -This model is intended to [compute sentence (text) embeddings](https://www.sbert.net/docs/usage/computing_sentence_embeddings.html) for English and German text. These embeddings can then be compared with [cosine-similarity](https://en.wikipedia.org/wiki/Cosine_similarity) to find sentences with a similar semantic meaning. For example this can be useful for [semantic textual similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html), [semantic search](https://www.sbert.net/docs/usage/semantic_search.html), or [paraphrase mining](https://www.sbert.net/docs/usage/paraphrase_mining.html). To do this you have to use the [Sentence Transformers Python framework](https://github.com/UKPLab/sentence-transformers). - -The speciality of this model is that it also works cross-lingually. Regardless of the language, the sentences are translated into very similar vectors according to their semantics. This means that you can, for example, enter a search in German and find results according to the semantics in German and also in English. Using a xlm model and _multilingual finetuning with language-crossing_ we reach performance that even exceeds the best current dedicated English large model (see Evaluation section below). - -> Sentence-BERT (SBERT) is a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. - -Source: [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) - -This model is fine-tuned from [Philip May](https://eniak.de/) and open-sourced by [T-Systems-onsite](https://www.t-systems-onsite.de/). Special thanks to [Nils Reimers](https://www.nils-reimers.de/) for your awesome open-source work, the Sentence Transformers, the models and your help on GitHub. - -## How to use -**The usage description above - provided by Hugging Face - is wrong for sentence embeddings! Please use this:** - -To use this model install the `sentence-transformers` package (see here: ). - -```python -from sentence_transformers import SentenceTransformer -model = SentenceTransformer('T-Systems-onsite/cross-en-de-roberta-sentence-transformer') -``` - -For details of usage and examples see here: -- [Computing Sentence Embeddings](https://www.sbert.net/docs/usage/computing_sentence_embeddings.html) -- [Semantic Textual Similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html) -- [Paraphrase Mining](https://www.sbert.net/docs/usage/paraphrase_mining.html) -- [Semantic Search](https://www.sbert.net/docs/usage/semantic_search.html) -- [Cross-Encoders](https://www.sbert.net/docs/usage/cross-encoder.html) -- [Examples on GitHub](https://github.com/UKPLab/sentence-transformers/tree/master/examples) - -## Training -The base model is [xlm-roberta-base](https://huggingface.co/xlm-roberta-base). This model has been further trained by [Nils Reimers](https://www.nils-reimers.de/) on a large scale paraphrase dataset for 50+ languages. [Nils Reimers](https://www.nils-reimers.de/) about this [on GitHub](https://github.com/UKPLab/sentence-transformers/issues/509#issuecomment-712243280): - ->A paper is upcoming for the paraphrase models. -> ->These models were trained on various datasets with Millions of examples for paraphrases, mainly derived from Wikipedia edit logs, paraphrases mined from Wikipedia and SimpleWiki, paraphrases from news reports, AllNLI-entailment pairs with in-batch-negative loss etc. -> ->In internal tests, they perform much better than the NLI+STSb models as they have see more and broader type of training data. NLI+STSb has the issue that they are rather narrow in their domain and do not contain any domain specific words / sentences (like from chemistry, computer science, math etc.). The paraphrase models has seen plenty of sentences from various domains. -> ->More details with the setup, all the datasets, and a wider evaluation will follow soon. - -The resulting model called `xlm-r-distilroberta-base-paraphrase-v1` has been released here: - -Building on this cross language model we fine-tuned it for English and German language on the [STSbenchmark](http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark) dataset. For German language we used the dataset of our [German STSbenchmark dataset](https://github.com/t-systems-on-site-services-gmbh/german-STSbenchmark) which has been translated with [deepl.com](https://www.deepl.com/translator). Additionally to the German and English training samples we generated samples of English and German crossed. We call this _multilingual finetuning with language-crossing_. It doubled the traing-datasize and tests show that it further improves performance. - -We did an automatic hyperparameter search for 33 trials with [Optuna](https://github.com/optuna/optuna). Using 10-fold crossvalidation on the deepl.com test and dev dataset we found the following best hyperparameter: -- batch_size = 8 -- num_epochs = 2 -- lr = 1.026343323298136e-05, -- eps = 4.462251033010287e-06 -- weight_decay = 0.04794438776350409 -- warmup_steps_proportion = 0.1609010732760181 - -The final model was trained with these hyperparameters on the combination of the train and dev datasets from English, German and the crossings of them. The testset was left for testing. - -# Evaluation -The evaluation has been done on English, German and both languages crossed with the STSbenchmark test data. The evaluation-code is available on [Colab](https://colab.research.google.com/drive/1gtGnKq_dYU_sDYqMohTYVMVpxMJjyH0M?usp=sharing). As the metric for evaluation we use the Spearman’s rank correlation between the cosine-similarity of the sentence embeddings and STSbenchmark labels. - -| Model Name | Spearman
German | Spearman
English | Spearman
EN-DE & DE-EN
(cross) | -|---------------------------------------------------------------|-------------------|--------------------|------------------| -| xlm-r-distilroberta-base-paraphrase-v1 | 0.8079 | 0.8350 | 0.7983 | -| [xlm-r-100langs-bert-base-nli-stsb-mean-tokens](https://huggingface.co/sentence-transformers/xlm-r-100langs-bert-base-nli-stsb-mean-tokens) | 0.7877 | 0.8465 | 0.7908 | -| xlm-r-bert-base-nli-stsb-mean-tokens | 0.7877 | 0.8465 | 0.7908 | -| [roberta-large-nli-stsb-mean-tokens](https://huggingface.co/sentence-transformers/roberta-large-nli-stsb-mean-tokens) | 0.6371 | 0.8639 | 0.4109 | -| [T-Systems-onsite/
german-roberta-sentence-transformer-v2](https://huggingface.co/T-Systems-onsite/german-roberta-sentence-transformer-v2) | 0.8529 | 0.8634 | 0.8415 | -| **T-Systems-onsite/
cross-en-de-roberta-sentence-transformer** | **0.8550** | **0.8660** | **0.8525** | diff --git a/model_cards/T-Systems-onsite/german-roberta-sentence-transformer-v2/README.md b/model_cards/T-Systems-onsite/german-roberta-sentence-transformer-v2/README.md deleted file mode 100644 index 05184fbef5..0000000000 --- a/model_cards/T-Systems-onsite/german-roberta-sentence-transformer-v2/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: de -license: mit -tags: -- sentence_embedding -- search -- pytorch -- xlm-roberta -- roberta -- xlm-r-distilroberta-base-paraphrase-v1 -- paraphrase -datasets: -- STSbenchmark -metrics: -- Spearman’s rank correlation -- cosine similarity ---- - -# German RoBERTa for Sentence Embeddings V2 -**The new [T-Systems-onsite/cross-en-de-roberta-sentence-transformer](https://huggingface.co/T-Systems-onsite/cross-en-de-roberta-sentence-transformer) model is slightly better for German language. It is also the current best model for English language and works cross-lingually. Please consider using that model.** - -This model is intended to [compute sentence (text embeddings)](https://www.sbert.net/docs/usage/computing_sentence_embeddings.html) for German text. These embeddings can then be compared with [cosine-similarity](https://en.wikipedia.org/wiki/Cosine_similarity) to find sentences with a similar semantic meaning. For example this can be useful for [semantic textual similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html), [semantic search](https://www.sbert.net/docs/usage/semantic_search.html), or [paraphrase mining](https://www.sbert.net/docs/usage/paraphrase_mining.html). To do this you have to use the [Sentence Transformers Python framework](https://github.com/UKPLab/sentence-transformers). - -> Sentence-BERT (SBERT) is a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. - -Source: [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) - -This model is fine-tuned from [Philip May](https://eniak.de/) and open-sourced by [T-Systems-onsite](https://www.t-systems-onsite.de/). Special thanks to [Nils Reimers](https://www.nils-reimers.de/) for your awesome open-source work, the Sentence Transformers, the models and your help on GitHub. - -## How to use -**The usage description above - provided by Hugging Face - is wrong for sentence embeddings! Please use this:** - -To use this model install the `sentence-transformers` package (see here: ). - -```python -from sentence_transformers import SentenceTransformer -model = SentenceTransformer('T-Systems-onsite/german-roberta-sentence-transformer-v2') -``` - -For details of usage and examples see here: -- [Computing Sentence Embeddings](https://www.sbert.net/docs/usage/computing_sentence_embeddings.html) -- [Semantic Textual Similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html) -- [Paraphrase Mining](https://www.sbert.net/docs/usage/paraphrase_mining.html) -- [Semantic Search](https://www.sbert.net/docs/usage/semantic_search.html) -- [Cross-Encoders](https://www.sbert.net/docs/usage/cross-encoder.html) -- [Examples on GitHub](https://github.com/UKPLab/sentence-transformers/tree/master/examples) - -## Training -The base model is [xlm-roberta-base](https://huggingface.co/xlm-roberta-base). This model has been further trained by [Nils Reimers](https://www.nils-reimers.de/) on a large scale paraphrase dataset for 50+ languages. [Nils Reimers](https://www.nils-reimers.de/) about this [on GitHub](https://github.com/UKPLab/sentence-transformers/issues/509#issuecomment-712243280): - ->A paper is upcoming for the paraphrase models. -> ->These models were trained on various datasets with Millions of examples for paraphrases, mainly derived from Wikipedia edit logs, paraphrases mined from Wikipedia and SimpleWiki, paraphrases from news reports, AllNLI-entailment pairs with in-batch-negative loss etc. -> ->In internal tests, they perform much better than the NLI+STSb models as they have see more and broader type of training data. NLI+STSb has the issue that they are rather narrow in their domain and do not contain any domain specific words / sentences (like from chemistry, computer science, math etc.). The paraphrase models has seen plenty of sentences from various domains. -> ->More details with the setup, all the datasets, and a wider evaluation will follow soon. - -The resulting model called `xlm-r-distilroberta-base-paraphrase-v1` has been released here: - -Building on this cross language model we fine-tuned it for German language on the [deepl.com](https://www.deepl.com/translator) dataset of our [German STSbenchmark dataset](https://github.com/t-systems-on-site-services-gmbh/german-STSbenchmark). - -We did an automatic hyperparameter search for 102 trials with [Optuna](https://github.com/optuna/optuna). Using 10-fold crossvalidation on the deepl.com test and dev dataset we found the following best hyperparameters: -- batch_size = 15 -- num_epochs = 4 -- lr = 2.2995320905210864e-05 -- eps = 1.8979875906303792e-06 -- weight_decay = 0.003314045812507563 -- warmup_steps_proportion = 0.46141685205829014 - -The final model was trained with these hyperparameters on the combination of `sts_de_train.csv` and `sts_de_dev.csv`. The `sts_de_test.csv` was left for testing. - -# Evaluation -The evaluation has been done on the test set of our [German STSbenchmark dataset](https://github.com/t-systems-on-site-services-gmbh/german-STSbenchmark). The code is available on [Colab](https://colab.research.google.com/drive/1aCWOqDQx953kEnQ5k4Qn7uiixokocOHv?usp=sharing). As the metric for evaluation we use the Spearman’s rank correlation between the cosine-similarity of the sentence embeddings and STSbenchmark labels. - -| Model Name | Spearman rank correlation
(German) | -|--------------------------------------|-------------------------------------| -| xlm-r-distilroberta-base-paraphrase-v1 | 0.8079 | -| xlm-r-100langs-bert-base-nli-stsb-mean-tokens | 0.8194 | -| xlm-r-bert-base-nli-stsb-mean-tokens | 0.8194 | -| **T-Systems-onsite/
german-roberta-sentence-transformer-v2** | **0.8529** | -| **[T-Systems-onsite/
cross-en-de-roberta-sentence-transformer](https://huggingface.co/T-Systems-onsite/cross-en-de-roberta-sentence-transformer)** | **0.8550** | diff --git a/model_cards/Tereveni-AI/gpt2-124M-uk-fiction/README.md b/model_cards/Tereveni-AI/gpt2-124M-uk-fiction/README.md deleted file mode 100644 index d26be020e5..0000000000 --- a/model_cards/Tereveni-AI/gpt2-124M-uk-fiction/README.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -language: uk ---- - -Note: **default code snippet above won't work** because we are using `AlbertTokenizer` with `GPT2LMHeadModel`, see [issue](https://github.com/huggingface/transformers/issues/4285). - -## GPT2 124M Trained on Ukranian Fiction - -### Training details - -Model was trained on corpus of 4040 fiction books, 2.77 GiB in total. -Evaluation on [brown-uk](https://github.com/brown-uk/corpus) gives perplexity of 50.16. - -### Example usage: -```python -from transformers import AlbertTokenizer, GPT2LMHeadModel - -tokenizer = AlbertTokenizer.from_pretrained("Tereveni-AI/gpt2-124M-uk-fiction") -model = GPT2LMHeadModel.from_pretrained("Tereveni-AI/gpt2-124M-uk-fiction") - -input_ids = tokenizer.encode("Но зла Юнона, суча дочка,", add_special_tokens=False, return_tensors='pt') - -outputs = model.generate( - input_ids, - do_sample=True, - num_return_sequences=3, - max_length=50 -) - -for i, out in enumerate(outputs): - print("{}: {}".format(i, tokenizer.decode(out))) -``` - -Prints something like this: -```bash -0: Но зла Юнона, суча дочка, яка затьмарила всі її таємниці: І хто з'їсть її душу, той помре». І, не дочекавшись гніву богів, посунула в пітьму, щоб не бачити перед собою. Але, за -1: Но зла Юнона, суча дочка, і довела мене до божевілля. Але він не знав нічого. Після того як я його побачив, мені стало зле. Я втратив рівновагу. Але в мене не було часу на роздуми. Я вже втратив надію -2: Но зла Юнона, суча дочка, не нарікала нам! — раптом вигукнула Юнона. — Це ти, старий йолопе! — мовила вона, не перестаючи сміятись. — Хіба ти не знаєш, що мені подобається ходити з тобою? -``` \ No newline at end of file diff --git a/model_cards/TurkuNLP/bert-base-finnish-cased-v1/README.md b/model_cards/TurkuNLP/bert-base-finnish-cased-v1/README.md deleted file mode 100644 index 4225733d60..0000000000 --- a/model_cards/TurkuNLP/bert-base-finnish-cased-v1/README.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -language: fi ---- - -## Quickstart - -**Release 1.0** (November 25, 2019) - -Download the models here: - -* Cased Finnish BERT Base: [bert-base-finnish-cased-v1.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-cased-v1.zip) -* Uncased Finnish BERT Base: [bert-base-finnish-uncased-v1.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-uncased-v1.zip) - -We generally recommend the use of the cased model. - -Paper presenting Finnish BERT: [arXiv:1912.07076](https://arxiv.org/abs/1912.07076) - -## What's this? - -A version of Google's [BERT](https://github.com/google-research/bert) deep transfer learning model for Finnish. The model can be fine-tuned to achieve state-of-the-art results for various Finnish natural language processing tasks. - -FinBERT features a custom 50,000 wordpiece vocabulary that has much better coverage of Finnish words than e.g. the previously released [multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) models from Google: - -| Vocabulary | Example | -|------------|---------| -| FinBERT | Suomessa vaihtuu kesän aikana sekä pääministeri että valtiovarain ##ministeri . | -| Multilingual BERT | Suomessa vai ##htuu kes ##än aikana sekä p ##ää ##minister ##i että valt ##io ##vara ##in ##minister ##i . | - -FinBERT has been pre-trained for 1 million steps on over 3 billion tokens (24B characters) of Finnish text drawn from news, online discussion, and internet crawls. By contrast, Multilingual BERT was trained on Wikipedia texts, where the Finnish Wikipedia text is approximately 3% of the amount used to train FinBERT. - -These features allow FinBERT to outperform not only Multilingual BERT but also all previously proposed models when fine-tuned for Finnish natural language processing tasks. - -## Results - -### Document classification - -![learning curves for Yle and Ylilauta document classification](https://raw.githubusercontent.com/TurkuNLP/FinBERT/master/img/yle-ylilauta-curves.png) - -FinBERT outperforms multilingual BERT (M-BERT) on document classification over a range of training set sizes on the Yle news (left) and Ylilauta online discussion (right) corpora. (Baseline classification performance with [FastText](https://fasttext.cc/) included for reference.) - -[[code](https://github.com/spyysalo/finbert-text-classification)][[Yle data](https://github.com/spyysalo/yle-corpus)] [[Ylilauta data](https://github.com/spyysalo/ylilauta-corpus)] - -### Named Entity Recognition - -Evaluation on FiNER corpus ([Ruokolainen et al 2019](https://arxiv.org/abs/1908.04212)) - -| Model | Accuracy | -|--------------------|----------| -| **FinBERT** | **92.40%** | -| Multilingual BERT | 90.29% | -| [FiNER-tagger](https://github.com/Traubert/FiNer-rules) (rule-based) | 86.82% | - -(FiNER tagger results from [Ruokolainen et al. 2019](https://arxiv.org/pdf/1908.04212.pdf)) - -[[code](https://github.com/jouniluoma/keras-bert-ner)][[data](https://github.com/mpsilfve/finer-data)] - -### Part of speech tagging - -Evaluation on three Finnish corpora annotated with [Universal Dependencies](https://universaldependencies.org/) part-of-speech tags: the Turku Dependency Treebank (TDT), FinnTreeBank (FTB), and Parallel UD treebank (PUD) - -| Model | TDT | FTB | PUD | -|-------------------|-------------|-------------|-------------| -| **FinBERT** | **98.23%** | **98.39%** | **98.08%** | -| Multilingual BERT | 96.97% | 95.87% | 97.58% | - -[[code](https://github.com/spyysalo/bert-pos)][[data](http://hdl.handle.net/11234/1-2837)] - -## Use with PyTorch - -If you want to use the model with the huggingface/transformers library, follow the steps in [huggingface_transformers.md](https://github.com/TurkuNLP/FinBERT/blob/master/huggingface_transformers.md) - -## Previous releases - -### Release 0.2 - -**October 24, 2019** Beta version of the BERT base uncased model trained from scratch on a corpus of Finnish news, online discussions, and crawled data. - -Download the model here: [bert-base-finnish-uncased.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-uncased.zip) - -### Release 0.1 - -**September 30, 2019** We release a beta version of the BERT base cased model trained from scratch on a corpus of Finnish news, online discussions, and crawled data. - -Download the model here: [bert-base-finnish-cased.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-cased.zip) diff --git a/model_cards/TurkuNLP/bert-base-finnish-uncased-v1/README.md b/model_cards/TurkuNLP/bert-base-finnish-uncased-v1/README.md deleted file mode 100644 index 4225733d60..0000000000 --- a/model_cards/TurkuNLP/bert-base-finnish-uncased-v1/README.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -language: fi ---- - -## Quickstart - -**Release 1.0** (November 25, 2019) - -Download the models here: - -* Cased Finnish BERT Base: [bert-base-finnish-cased-v1.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-cased-v1.zip) -* Uncased Finnish BERT Base: [bert-base-finnish-uncased-v1.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-uncased-v1.zip) - -We generally recommend the use of the cased model. - -Paper presenting Finnish BERT: [arXiv:1912.07076](https://arxiv.org/abs/1912.07076) - -## What's this? - -A version of Google's [BERT](https://github.com/google-research/bert) deep transfer learning model for Finnish. The model can be fine-tuned to achieve state-of-the-art results for various Finnish natural language processing tasks. - -FinBERT features a custom 50,000 wordpiece vocabulary that has much better coverage of Finnish words than e.g. the previously released [multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) models from Google: - -| Vocabulary | Example | -|------------|---------| -| FinBERT | Suomessa vaihtuu kesän aikana sekä pääministeri että valtiovarain ##ministeri . | -| Multilingual BERT | Suomessa vai ##htuu kes ##än aikana sekä p ##ää ##minister ##i että valt ##io ##vara ##in ##minister ##i . | - -FinBERT has been pre-trained for 1 million steps on over 3 billion tokens (24B characters) of Finnish text drawn from news, online discussion, and internet crawls. By contrast, Multilingual BERT was trained on Wikipedia texts, where the Finnish Wikipedia text is approximately 3% of the amount used to train FinBERT. - -These features allow FinBERT to outperform not only Multilingual BERT but also all previously proposed models when fine-tuned for Finnish natural language processing tasks. - -## Results - -### Document classification - -![learning curves for Yle and Ylilauta document classification](https://raw.githubusercontent.com/TurkuNLP/FinBERT/master/img/yle-ylilauta-curves.png) - -FinBERT outperforms multilingual BERT (M-BERT) on document classification over a range of training set sizes on the Yle news (left) and Ylilauta online discussion (right) corpora. (Baseline classification performance with [FastText](https://fasttext.cc/) included for reference.) - -[[code](https://github.com/spyysalo/finbert-text-classification)][[Yle data](https://github.com/spyysalo/yle-corpus)] [[Ylilauta data](https://github.com/spyysalo/ylilauta-corpus)] - -### Named Entity Recognition - -Evaluation on FiNER corpus ([Ruokolainen et al 2019](https://arxiv.org/abs/1908.04212)) - -| Model | Accuracy | -|--------------------|----------| -| **FinBERT** | **92.40%** | -| Multilingual BERT | 90.29% | -| [FiNER-tagger](https://github.com/Traubert/FiNer-rules) (rule-based) | 86.82% | - -(FiNER tagger results from [Ruokolainen et al. 2019](https://arxiv.org/pdf/1908.04212.pdf)) - -[[code](https://github.com/jouniluoma/keras-bert-ner)][[data](https://github.com/mpsilfve/finer-data)] - -### Part of speech tagging - -Evaluation on three Finnish corpora annotated with [Universal Dependencies](https://universaldependencies.org/) part-of-speech tags: the Turku Dependency Treebank (TDT), FinnTreeBank (FTB), and Parallel UD treebank (PUD) - -| Model | TDT | FTB | PUD | -|-------------------|-------------|-------------|-------------| -| **FinBERT** | **98.23%** | **98.39%** | **98.08%** | -| Multilingual BERT | 96.97% | 95.87% | 97.58% | - -[[code](https://github.com/spyysalo/bert-pos)][[data](http://hdl.handle.net/11234/1-2837)] - -## Use with PyTorch - -If you want to use the model with the huggingface/transformers library, follow the steps in [huggingface_transformers.md](https://github.com/TurkuNLP/FinBERT/blob/master/huggingface_transformers.md) - -## Previous releases - -### Release 0.2 - -**October 24, 2019** Beta version of the BERT base uncased model trained from scratch on a corpus of Finnish news, online discussions, and crawled data. - -Download the model here: [bert-base-finnish-uncased.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-uncased.zip) - -### Release 0.1 - -**September 30, 2019** We release a beta version of the BERT base cased model trained from scratch on a corpus of Finnish news, online discussions, and crawled data. - -Download the model here: [bert-base-finnish-cased.zip](http://dl.turkunlp.org/finbert/bert-base-finnish-cased.zip) diff --git a/model_cards/TypicaAI/magbert-ner/README.md b/model_cards/TypicaAI/magbert-ner/README.md deleted file mode 100644 index 7e22de96d6..0000000000 --- a/model_cards/TypicaAI/magbert-ner/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -language: fr -widget: -- text: "Je m'appelle Hicham et je vis a Fès" ---- - -# MagBERT-NER: a state-of-the-art NER model for Moroccan French language (Maghreb) - -## Introduction - -[MagBERT-NER] is a state-of-the-art NER model for Moroccan French language (Maghreb). The MagBERT-NER model was fine-tuned for NER Task based the language model for French Camembert (based on the RoBERTa architecture). - -For further information or requests, please visite our website at [typica.ai Website](https://typica.ai/) or send us an email at contactus@typica.ai - -## How to use MagBERT-NER with HuggingFace - -##### Load MagBERT-NER and its sub-word tokenizer : - -```python -from transformers import AutoTokenizer, AutoModelForTokenClassification - -tokenizer = AutoTokenizer.from_pretrained("TypicaAI/magbert-ner") -model = AutoModelForTokenClassification.from_pretrained("TypicaAI/magbert-ner") - - -##### Process text sample (from wikipedia about the current Prime Minister of Morocco) Using NER pipeline - -from transformers import pipeline - -nlp = pipeline('ner', model=model, tokenizer=tokenizer, grouped_entities=True) -nlp("Saad Dine El Otmani, né le 16 janvier 1956 à Inezgane, est un homme d'État marocain, chef du gouvernement du Maroc depuis le 5 avril 2017") - - -#[{'entity_group': 'I-PERSON', -# 'score': 0.8941445276141167, -# 'word': 'Saad Dine El Otmani'}, -# {'entity_group': 'B-DATE', -# 'score': 0.5967703461647034, -# 'word': '16 janvier 1956'}, -# {'entity_group': 'B-GPE', 'score': 0.7160899192094803, 'word': 'Inezgane'}, -# {'entity_group': 'B-NORP', 'score': 0.7971733212471008, 'word': 'marocain'}, -# {'entity_group': 'B-GPE', 'score': 0.8921478390693665, 'word': 'Maroc'}, -# {'entity_group': 'B-DATE', -# 'score': 0.5760444005330404, -# 'word': '5 avril 2017'}] - -``` - - -## Authors - -MagBert-NER Model was trained by Hicham Assoudi, Ph.D. -For any questions, comments you can contact me at assoudi@typica.ai - - -## Citation - -If you use our work, please cite: -Hicham Assoudi, Ph.D., MagBERT-NER: a state-of-the-art NER model for Moroccan French language (Maghreb), (2020) diff --git a/model_cards/Vamsi/T5_Paraphrase_Paws/README.md b/model_cards/Vamsi/T5_Paraphrase_Paws/README.md deleted file mode 100644 index 8d54e825d7..0000000000 --- a/model_cards/Vamsi/T5_Paraphrase_Paws/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: "en" -tags: -- paraphrase-generation -- text-generation -- Conditional Generation -inference: false ---- -​ -# Paraphrase-Generation -​ -## Model description -​ -T5 Model for generating paraphrases of english sentences. Trained on the [Google PAWS](https://github.com/google-research-datasets/paws) dataset. -​ -## How to use -​ -PyTorch and TF models available -​ -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM -​ -tokenizer = AutoTokenizer.from_pretrained("Vamsi/T5_Paraphrase_Paws") -model = AutoModelForSeq2SeqLM.from_pretrained("Vamsi/T5_Paraphrase_Paws") -​ -sentence = "This is something which i cannot understand at all" - -text = "paraphrase: " + sentence + " " - -encoding = tokenizer.encode_plus(text,pad_to_max_length=True, return_tensors="pt") -input_ids, attention_masks = encoding["input_ids"].to("cuda"), encoding["attention_mask"].to("cuda") - - -outputs = model.generate( - input_ids=input_ids, attention_mask=attention_masks, - max_length=256, - do_sample=True, - top_k=120, - top_p=0.95, - early_stopping=True, - num_return_sequences=5 -) - -for output in outputs: - line = tokenizer.decode(output, skip_special_tokens=True,clean_up_tokenization_spaces=True) - print(line) -​ - -``` - -For more reference on training your own T5 model or using this model, do check out [Paraphrase Generation](https://github.com/Vamsi995/Paraphrase-Generator). diff --git a/model_cards/VictorSanh/roberta-base-finetuned-yelp-polarity/README.md b/model_cards/VictorSanh/roberta-base-finetuned-yelp-polarity/README.md deleted file mode 100644 index e15e38e37b..0000000000 --- a/model_cards/VictorSanh/roberta-base-finetuned-yelp-polarity/README.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -language: en -datasets: -- yelp_polarity ---- - -# RoBERTa-base-finetuned-yelp-polarity - -This is a [RoBERTa-base](https://huggingface.co/roberta-base) checkpoint fine-tuned on binary sentiment classifcation from [Yelp polarity](https://huggingface.co/nlp/viewer/?dataset=yelp_polarity). -It gets **98.08%** accuracy on the test set. - -## Hyper-parameters - -We used the following hyper-parameters to train the model on one GPU: -```python -num_train_epochs = 2.0 -learning_rate = 1e-05 -weight_decay = 0.0 -adam_epsilon = 1e-08 -max_grad_norm = 1.0 -per_device_train_batch_size = 32 -gradient_accumulation_steps = 1 -warmup_steps = 3500 -seed = 42 -``` diff --git a/model_cards/ViktorAlm/electra-base-norwegian-uncased-discriminator/README.md b/model_cards/ViktorAlm/electra-base-norwegian-uncased-discriminator/README.md deleted file mode 100644 index 0f22ee9691..0000000000 --- a/model_cards/ViktorAlm/electra-base-norwegian-uncased-discriminator/README.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -language: no -thumbnail: https://i.imgur.com/QqSEC5I.png ---- - -# Norwegian Electra -![Image of norwegian electra](https://i.imgur.com/QqSEC5I.png) - -Trained on Oscar + wikipedia + opensubtitles + some other data I had with the awesome power of TPUs(V3-8) - -Use with caution. I have no downstream tasks in Norwegian to test on so I have no idea of its performance yet. -# Model -## Electra: Pre-training Text Encoders as Discriminators Rather Than Generators -Kevin Clark and Minh-Thang Luong and Quoc V. Le and Christopher D. Manning -- https://openreview.net/pdf?id=r1xMH1BtvB -- https://github.com/google-research/electra -# Acknowledgments -### TensorFlow Research Cloud -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). Thanks for providing access to the TFRC ❤️ -- https://www.tensorflow.org/tfrc -#### OSCAR corpus -- https://oscar-corpus.com/ -#### OPUS -- http://opus.nlpl.eu/ -- http://www.opensubtitles.org/ diff --git a/model_cards/a-ware/bart-squadv2/README.md b/model_cards/a-ware/bart-squadv2/README.md deleted file mode 100644 index 164c6a220f..0000000000 --- a/model_cards/a-ware/bart-squadv2/README.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# BART-LARGE finetuned on SQuADv2 - -This is bart-large model finetuned on SQuADv2 dataset for question answering task - -## Model details -BART was propsed in the [paper](https://arxiv.org/abs/1910.13461) **BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension**. -BART is a seq2seq model intended for both NLG and NLU tasks. - -To use BART for question answering tasks, we feed the complete document into the encoder and decoder, and use the top -hidden state of the decoder as a representation for each -word. This representation is used to classify the token. As given in the paper bart-large achives comparable to ROBERTa on SQuAD. -Another notable thing about BART is that it can handle sequences with upto 1024 tokens. - -| Param | #Value | -|---------------------|--------| -| encoder layers | 12 | -| decoder layers | 12 | -| hidden size | 4096 | -| num attetion heads | 16 | -| on disk size | 1.63GB | - - -## Model training -This model was trained with following parameters using simpletransformers wrapper: -``` -train_args = { - 'learning_rate': 1e-5, - 'max_seq_length': 512, - 'doc_stride': 512, - 'overwrite_output_dir': True, - 'reprocess_input_data': False, - 'train_batch_size': 8, - 'num_train_epochs': 2, - 'gradient_accumulation_steps': 2, - 'no_cache': True, - 'use_cached_eval_features': False, - 'save_model_every_epoch': False, - 'output_dir': "bart-squadv2", - 'eval_batch_size': 32, - 'fp16_opt_level': 'O2', - } -``` - -[You can even train your own model using this colab notebook](https://colab.research.google.com/drive/1I5cK1M_0dLaf5xoewh6swcm5nAInfwHy?usp=sharing) - -## Results -```{"correct": 6832, "similar": 4409, "incorrect": 632, "eval_loss": -14.950117511952177}``` - -## Model in Action 🚀 -```python3 -from transformers import BartTokenizer, BartForQuestionAnswering -import torch - -tokenizer = BartTokenizer.from_pretrained('a-ware/bart-squadv2') -model = BartForQuestionAnswering.from_pretrained('a-ware/bart-squadv2') - -question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" -encoding = tokenizer(question, text, return_tensors='pt') -input_ids = encoding['input_ids'] -attention_mask = encoding['attention_mask'] - -start_scores, end_scores = model(input_ids, attention_mask=attention_mask, output_attentions=False)[:2] - -all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0]) -answer = ' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]) -answer = tokenizer.convert_tokens_to_ids(answer.split()) -answer = tokenizer.decode(answer) -#answer => 'a nice puppet' -``` - -> Created with ❤️ by A-ware UG [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/aware-ai) diff --git a/model_cards/a-ware/roberta-large-squad-classification/README.md b/model_cards/a-ware/roberta-large-squad-classification/README.md deleted file mode 100644 index 31d4578954..0000000000 --- a/model_cards/a-ware/roberta-large-squad-classification/README.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# Roberta-LARGE finetuned on SQuADv2 - -This is roberta-large model finetuned on SQuADv2 dataset for question answering answerability classification - -## Model details -This model is simply an Sequenceclassification model with two inputs (context and question) in a list. -The result is either [1] for answerable or [0] if it is not answerable. -It was trained over 4 epochs on squadv2 dataset and can be used to filter out which context is good to give into the QA model to avoid bad answers. - -## Model training -This model was trained with following parameters using simpletransformers wrapper: -``` -train_args = { - 'learning_rate': 1e-5, - 'max_seq_length': 512, - 'overwrite_output_dir': True, - 'reprocess_input_data': False, - 'train_batch_size': 4, - 'num_train_epochs': 4, - 'gradient_accumulation_steps': 2, - 'no_cache': True, - 'use_cached_eval_features': False, - 'save_model_every_epoch': False, - 'output_dir': "bart-squadv2", - 'eval_batch_size': 8, - 'fp16_opt_level': 'O2', - } -``` - -## Results -```{"accuracy": 90.48%}``` -## Model in Action 🚀 -```python3 -from simpletransformers.classification import ClassificationModel - -model = ClassificationModel('roberta', 'a-ware/roberta-large-squadv2', num_labels=2, args=train_args) - -predictions, raw_outputs = model.predict([["my dog is an year old. he loves to go into the rain", "how old is my dog ?"]]) -print(predictions) -==> [1] -``` - -> Created with ❤️ by A-ware UG [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/aware-ai) diff --git a/model_cards/a-ware/xlmroberta-squadv2/README.md b/model_cards/a-ware/xlmroberta-squadv2/README.md deleted file mode 100644 index eb08cbbc06..0000000000 --- a/model_cards/a-ware/xlmroberta-squadv2/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# XLM-ROBERTA-LARGE finetuned on SQuADv2 - -This is xlm-roberta-large model finetuned on SQuADv2 dataset for question answering task - -## Model details -XLM-Roberta was propsed in the [paper](https://arxiv.org/pdf/1911.02116.pdf) **XLM-R: State-of-the-art cross-lingual understanding through self-supervision - -## Model training -This model was trained with following parameters using simpletransformers wrapper: -``` -train_args = { - 'learning_rate': 1e-5, - 'max_seq_length': 512, - 'doc_stride': 512, - 'overwrite_output_dir': True, - 'reprocess_input_data': False, - 'train_batch_size': 8, - 'num_train_epochs': 2, - 'gradient_accumulation_steps': 2, - 'no_cache': True, - 'use_cached_eval_features': False, - 'save_model_every_epoch': False, - 'output_dir': "bart-squadv2", - 'eval_batch_size': 32, - 'fp16_opt_level': 'O2', - } -``` - -## Results -```{"correct": 6961, "similar": 4359, "incorrect": 553, "eval_loss": -12.177856394381962}``` - -## Model in Action 🚀 -```python3 -from transformers import XLMRobertaTokenizer, XLMRobertaForQuestionAnswering -import torch - -tokenizer = XLMRobertaTokenizer.from_pretrained('a-ware/xlmroberta-squadv2') -model = XLMRobertaForQuestionAnswering.from_pretrained('a-ware/xlmroberta-squadv2') - -question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" -encoding = tokenizer(question, text, return_tensors='pt') -input_ids = encoding['input_ids'] -attention_mask = encoding['attention_mask'] - -start_scores, end_scores = model(input_ids, attention_mask=attention_mask, output_attentions=False)[:2] - -all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0]) -answer = ' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]) -answer = tokenizer.convert_tokens_to_ids(answer.split()) -answer = tokenizer.decode(answer) -#answer => 'a nice puppet' -``` - -> Created with ❤️ by A-ware UG [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/aware-ai) diff --git a/model_cards/abhilash1910/financial_roberta/README.md b/model_cards/abhilash1910/financial_roberta/README.md deleted file mode 100644 index b212634c2f..0000000000 --- a/model_cards/abhilash1910/financial_roberta/README.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -tags: -- finance ---- -# Roberta Masked Language Model Trained On Financial Phrasebank Corpus - - -This is a Masked Language Model trained with [Roberta](https://huggingface.co/transformers/model_doc/roberta.html) on a Financial Phrasebank Corpus. -The model is built using Huggingface transformers. -The model can be found at :[Financial_Roberta](https://huggingface.co/abhilash1910/financial_roberta) - - -## Specifications - - -The corpus for training is taken from the Financial Phrasebank (Malo et al)[https://www.researchgate.net/publication/251231107_Good_Debt_or_Bad_Debt_Detecting_Semantic_Orientations_in_Economic_Texts]. - - -## Model Specification - - -The model chosen for training is [Roberta](https://arxiv.org/abs/1907.11692) with the following specifications: - 1. vocab_size=56000 - 2. max_position_embeddings=514 - 3. num_attention_heads=12 - 4. num_hidden_layers=6 - 5. type_vocab_size=1 - - -This is trained by using RobertaConfig from transformers package. -The model is trained for 10 epochs with a gpu batch size of 64 units. - - - -## Usage Specifications - - -For using this model, we have to first import AutoTokenizer and AutoModelWithLMHead Modules from transformers -After that we have to specify, the pre-trained model,which in this case is 'abhilash1910/financial_roberta' for the tokenizers and the model. - - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("abhilash1910/financial_roberta") - -model = AutoModelWithLMHead.from_pretrained("abhilash1910/financial_roberta") -``` - - -After this the model will be downloaded, it will take some time to download all the model files. -For testing the model, we have to import pipeline module from transformers and create a masked output model for inference as follows: - - -```python -from transformers import pipeline -model_mask = pipeline('fill-mask', model='abhilash1910/inancial_roberta') -model_mask("The company had a of 20% in 2020.") -``` - - -Some of the examples are also provided with generic financial statements: - -Example 1: - - -```python -model_mask("The company had a of 20% in 2020.") -``` - - -Output: - - -```bash -[{'sequence': 'The company had a profit of 20% in 2020.', - 'score': 0.023112965747714043, - 'token': 421, - 'token_str': 'Ġprofit'}, - {'sequence': 'The company had a loss of 20% in 2020.', - 'score': 0.021379893645644188, - 'token': 616, - 'token_str': 'Ġloss'}, - {'sequence': 'The company had a year of 20% in 2020.', - 'score': 0.0185744296759367, - 'token': 443, - 'token_str': 'Ġyear'}, - {'sequence': 'The company had a sales of 20% in 2020.', - 'score': 0.018143286928534508, - 'token': 428, - 'token_str': 'Ġsales'}, - {'sequence': 'The company had a value of 20% in 2020.', - 'score': 0.015319528989493847, - 'token': 776, - 'token_str': 'Ġvalue'}] - ``` - - Example 2: - -```python - model_mask("The is listed under NYSE") -``` - -Output: - -```bash -[{'sequence': 'The company is listed under NYSE', - 'score': 0.1566661298274994, - 'token': 359, - 'token_str': 'Ġcompany'}, - {'sequence': 'The total is listed under NYSE', - 'score': 0.05542507395148277, - 'token': 522, - 'token_str': 'Ġtotal'}, - {'sequence': 'The value is listed under NYSE', - 'score': 0.04729423299431801, - 'token': 776, - 'token_str': 'Ġvalue'}, - {'sequence': 'The order is listed under NYSE', - 'score': 0.02533523552119732, - 'token': 798, - 'token_str': 'Ġorder'}, - {'sequence': 'The contract is listed under NYSE', - 'score': 0.02087237872183323, - 'token': 635, - 'token_str': 'Ġcontract'}] - ``` - - -## Resources - -For all resources , please look into the [HuggingFace](https://huggingface.co/) Site and the [Repositories](https://github.com/huggingface). diff --git a/model_cards/abhilash1910/french-roberta/README.md b/model_cards/abhilash1910/french-roberta/README.md deleted file mode 100644 index 444ff47d22..0000000000 --- a/model_cards/abhilash1910/french-roberta/README.md +++ /dev/null @@ -1,131 +0,0 @@ -# Roberta Trained Model For Masked Language Model On French Corpus :robot: - - -This is a Masked Language Model trained with [Roberta](https://huggingface.co/transformers/model_doc/roberta.html) on a small French News Corpus(Leipzig corpora). -The model is built using Huggingface transformers. -The model can be found at :[French-Roberta](https://huggingface.co/abhilash1910/french-roberta) - - -## Specifications - - -The corpus for training is taken from Leipzig Corpora (French News) , and is trained on a small set of the corpus (300K). - - -## Model Specification - - -The model chosen for training is [Roberta](https://arxiv.org/abs/1907.11692) with the following specifications: - 1. vocab_size=32000 - 2. max_position_embeddings=514 - 3. num_attention_heads=12 - 4. num_hidden_layers=6 - 5. type_vocab_size=1 - - -This is trained by using RobertaConfig from transformers package.The total training parameters :68124416 -The model is trained for 100 epochs with a gpu batch size of 64 units. -More details for building custom models can be found at the [HuggingFace Blog](https://huggingface.co/blog/how-to-train) - - - -## Usage Specifications - - -For using this model, we have to first import AutoTokenizer and AutoModelWithLMHead Modules from transformers -After that we have to specify, the pre-trained model,which in this case is 'abhilash1910/french-roberta' for the tokenizers and the model. - - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("abhilash1910/french-roberta") - -model = AutoModelWithLMHead.from_pretrained("abhilash1910/french-roberta") -``` - - -After this the model will be downloaded, it will take some time to download all the model files. -For testing the model, we have to import pipeline module from transformers and create a masked output model for inference as follows: - - -```python -from transformers import pipeline -model_mask = pipeline('fill-mask', model='abhilash1910/french-roberta') -model_mask("Le tweet .") -``` - - -Some of the examples are also provided with generic French sentences: - -Example 1: - - -```python -model_mask("À ce jour, projet a entraîné") -``` - - -Output: - - -```bash -[{'sequence': 'À ce jour, belles projet a entraîné', - 'score': 0.18685665726661682, - 'token': 6504, - 'token_str': 'Ġbelles'}, - {'sequence': 'À ce jour,- projet a entraîné', - 'score': 0.0005200508167035878, - 'token': 17, - 'token_str': '-'}, - {'sequence': 'À ce jour, de projet a entraîné', - 'score': 0.00045729897101409733, - 'token': 268, - 'token_str': 'Ġde'}, - {'sequence': 'À ce jour, du projet a entraîné', - 'score': 0.0004307595663703978, - 'token': 326, - 'token_str': 'Ġdu'}, - {'sequence': 'À ce jour," projet a entraîné', - 'score': 0.0004219160182401538, - 'token': 6, - 'token_str': '"'}] - ``` - - Example 2: - -```python - model_mask("C'est un ") -``` - -Output: - -```bash -[{'sequence': "C'est un belles", - 'score': 0.16440927982330322, - 'token': 6504, - 'token_str': 'Ġbelles'}, - {'sequence': "C'est un de", - 'score': 0.0005495127406902611, - 'token': 268, - 'token_str': 'Ġde'}, - {'sequence': "C'est un du", - 'score': 0.00044988933950662613, - 'token': 326, - 'token_str': 'Ġdu'}, - {'sequence': "C'est un-", - 'score': 0.00044542422983795404, - 'token': 17, - 'token_str': '-'}, - {'sequence': "C'est un\t", - 'score': 0.00037563967634923756, - 'token': 202, - 'token_str': 'ĉ'}] - ``` - - -## Resources - -For all resources , please look into the [HuggingFace](https://huggingface.co/) Site and the [Repositories](https://github.com/huggingface). - - diff --git a/model_cards/activebus/BERT-DK_laptop/README.md b/model_cards/activebus/BERT-DK_laptop/README.md deleted file mode 100644 index bc74627938..0000000000 --- a/model_cards/activebus/BERT-DK_laptop/README.md +++ /dev/null @@ -1,43 +0,0 @@ -# ReviewBERT - -BERT (post-)trained from review corpus to understand sentiment, options and various e-commence aspects. - -`BERT-DK_laptop` is trained from 100MB laptop corpus under `Electronics/Computers & Accessories/Laptops`. - - -## Model Description - -The original model is from `BERT-base-uncased` trained from Wikipedia+BookCorpus. -Models are post-trained from [Amazon Dataset](http://jmcauley.ucsd.edu/data/amazon/) and [Yelp Dataset](https://www.yelp.com/dataset/challenge/). - -`BERT-DK_laptop` is trained from 100MB laptop corpus under `Electronics/Computers & Accessories/Laptops`. - -## Instructions -Loading the post-trained weights are as simple as, e.g., - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("activebus/BERT-DK_laptop") -model = AutoModel.from_pretrained("activebus/BERT-DK_laptop") - -``` - - -## Evaluation Results - -Check our [NAACL paper](https://www.aclweb.org/anthology/N19-1242.pdf) - - -## Citation -If you find this work useful, please cite as following. -``` -@inproceedings{xu_bert2019, - title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", - author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.", - booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics", - month = "jun", - year = "2019", -} -``` diff --git a/model_cards/activebus/BERT-DK_rest/README.md b/model_cards/activebus/BERT-DK_rest/README.md deleted file mode 100644 index 34fe045a65..0000000000 --- a/model_cards/activebus/BERT-DK_rest/README.md +++ /dev/null @@ -1,41 +0,0 @@ -# ReviewBERT - -BERT (post-)trained from review corpus to understand sentiment, options and various e-commence aspects. - -`BERT-DK_rest` is trained from 1G (19 types) restaurants from Yelp. - -## Model Description - -The original model is from `BERT-base-uncased` trained from Wikipedia+BookCorpus. -Models are post-trained from [Amazon Dataset](http://jmcauley.ucsd.edu/data/amazon/) and [Yelp Dataset](https://www.yelp.com/dataset/challenge/). - - -## Instructions -Loading the post-trained weights are as simple as, e.g., - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("activebus/BERT-DK_rest") -model = AutoModel.from_pretrained("activebus/BERT-DK_rest") - -``` - - -## Evaluation Results - -Check our [NAACL paper](https://www.aclweb.org/anthology/N19-1242.pdf) - - -## Citation -If you find this work useful, please cite as following. -``` -@inproceedings{xu_bert2019, - title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", - author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.", - booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics", - month = "jun", - year = "2019", -} -``` diff --git a/model_cards/activebus/BERT-PT_laptop/README.md b/model_cards/activebus/BERT-PT_laptop/README.md deleted file mode 100644 index 6afbb2b01d..0000000000 --- a/model_cards/activebus/BERT-PT_laptop/README.md +++ /dev/null @@ -1,41 +0,0 @@ -# ReviewBERT - -BERT (post-)trained from review corpus to understand sentiment, options and various e-commence aspects. - -`BERT-DK_laptop` is trained from 100MB laptop corpus under `Electronics/Computers & Accessories/Laptops`. -`BERT-PT_*` addtionally uses SQuAD 1.1. - -## Model Description - -The original model is from `BERT-base-uncased` trained from Wikipedia+BookCorpus. -Models are post-trained from [Amazon Dataset](http://jmcauley.ucsd.edu/data/amazon/) and [Yelp Dataset](https://www.yelp.com/dataset/challenge/). - - -## Instructions -Loading the post-trained weights are as simple as, e.g., - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("activebus/BERT-PT_laptop") -model = AutoModel.from_pretrained("activebus/BERT-PT_laptop") - -``` - -## Evaluation Results - -Check our [NAACL paper](https://www.aclweb.org/anthology/N19-1242.pdf) - - -## Citation -If you find this work useful, please cite as following. -``` -@inproceedings{xu_bert2019, - title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", - author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.", - booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics", - month = "jun", - year = "2019", -} -``` diff --git a/model_cards/activebus/BERT-PT_rest/README.md b/model_cards/activebus/BERT-PT_rest/README.md deleted file mode 100644 index bfc0f6134b..0000000000 --- a/model_cards/activebus/BERT-PT_rest/README.md +++ /dev/null @@ -1,42 +0,0 @@ -# ReviewBERT - -BERT (post-)trained from review corpus to understand sentiment, options and various e-commence aspects. - -`BERT-DK_rest` is trained from 1G (19 types) restaurants from Yelp. -`BERT-PT_*` addtionally uses SQuAD 1.1. - -## Model Description - -The original model is from `BERT-base-uncased` trained from Wikipedia+BookCorpus. -Models are post-trained from [Amazon Dataset](http://jmcauley.ucsd.edu/data/amazon/) and [Yelp Dataset](https://www.yelp.com/dataset/challenge/). - - -## Instructions -Loading the post-trained weights are as simple as, e.g., - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("activebus/BERT-PT_rest") -model = AutoModel.from_pretrained("activebus/BERT-PT_rest") - -``` - - -## Evaluation Results - -Check our [NAACL paper](https://www.aclweb.org/anthology/N19-1242.pdf) - - -## Citation -If you find this work useful, please cite as following. -``` -@inproceedings{xu_bert2019, - title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", - author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.", - booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics", - month = "jun", - year = "2019", -} -``` diff --git a/model_cards/activebus/BERT-XD_Review/README.md b/model_cards/activebus/BERT-XD_Review/README.md deleted file mode 100644 index 39bd674bbb..0000000000 --- a/model_cards/activebus/BERT-XD_Review/README.md +++ /dev/null @@ -1,44 +0,0 @@ -# ReviewBERT - -BERT (post-)trained from review corpus to understand sentiment, options and various e-commence aspects. -Please visit https://github.com/howardhsu/BERT-for-RRC-ABSA for details. - -`BERT-XD_Review` is a cross-domain (beyond just `laptop` and `restaurant`) language model, where each example is from a single product / restaurant with the same rating, post-trained (fine-tuned) on a combination of 5-core Amazon reviews and all Yelp data, expected to be 22 G in total. It is trained for 4 epochs on `bert-base-uncased`. -The preprocessing code [here](https://github.com/howardhsu/BERT-for-RRC-ABSA/transformers). - -## Model Description - -The original model is from `BERT-base-uncased`. -Models are post-trained from [Amazon Dataset](http://jmcauley.ucsd.edu/data/amazon/) and [Yelp Dataset](https://www.yelp.com/dataset/challenge/). - - -## Instructions -Loading the post-trained weights are as simple as, e.g., - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("activebus/BERT-XD_Review") -model = AutoModel.from_pretrained("activebus/BERT-XD_Review") - -``` - - -## Evaluation Results - -Check our [NAACL paper](https://www.aclweb.org/anthology/N19-1242.pdf) -`BERT_Review` is expected to have similar performance on domain-specific tasks (such as aspect extraction) as `BERT-DK`, but much better on general tasks such as aspect sentiment classification (different domains mostly share similar sentiment words). - - -## Citation -If you find this work useful, please cite as following. -``` -@inproceedings{xu_bert2019, - title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", - author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.", - booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics", - month = "jun", - year = "2019", -} -``` diff --git a/model_cards/activebus/BERT_Review/README.md b/model_cards/activebus/BERT_Review/README.md deleted file mode 100644 index 2f86bd566d..0000000000 --- a/model_cards/activebus/BERT_Review/README.md +++ /dev/null @@ -1,44 +0,0 @@ -# ReviewBERT - -BERT (post-)trained from review corpus to understand sentiment, options and various e-commence aspects. - -`BERT_Review` is cross-domain (beyond just `laptop` and `restaurant`) language model with one example from randomly mixed domains, post-trained (fine-tuned) on a combination of 5-core Amazon reviews and all Yelp data, expected to be 22 G in total. It is trained for 4 epochs on `bert-base-uncased`. -The preprocessing code [here](https://github.com/howardhsu/BERT-for-RRC-ABSA/transformers). - - -## Model Description - -The original model is from `BERT-base-uncased` trained from Wikipedia+BookCorpus. -Models are post-trained from [Amazon Dataset](http://jmcauley.ucsd.edu/data/amazon/) and [Yelp Dataset](https://www.yelp.com/dataset/challenge/). - - -## Instructions -Loading the post-trained weights are as simple as, e.g., - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("activebus/BERT_Review") -model = AutoModel.from_pretrained("activebus/BERT_Review") - -``` - - -## Evaluation Results - -Check our [NAACL paper](https://www.aclweb.org/anthology/N19-1242.pdf) -`BERT_Review` is expected to have similar performance on domain-specific tasks (such as aspect extraction) as `BERT-DK`, but much better on general tasks such as aspect sentiment classification (different domains mostly share similar sentiment words). - - -## Citation -If you find this work useful, please cite as following. -``` -@inproceedings{xu_bert2019, - title = "BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis", - author = "Xu, Hu and Liu, Bing and Shu, Lei and Yu, Philip S.", - booktitle = "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics", - month = "jun", - year = "2019", -} -``` diff --git a/model_cards/adalbertojunior/PTT5-SMALL-SUM/README.md b/model_cards/adalbertojunior/PTT5-SMALL-SUM/README.md deleted file mode 100644 index b8686ef4f9..0000000000 --- a/model_cards/adalbertojunior/PTT5-SMALL-SUM/README.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -language: pt ---- - -# PTT5-SMALL-SUM - -## Model description - -This model was trained to summarize texts in portuguese - - -based on ```unicamp-dl/ptt5-small-portuguese-vocab``` - -#### How to use - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained('adalbertojunior/PTT5-SMALL-SUM') - -t5 = T5ForConditionalGeneration.from_pretrained('adalbertojunior/PTT5-SMALL-SUM') - -text="Esse é um exemplo de sumarização." - -input_ids = tokenizer.encode(text, return_tensors="pt", add_special_tokens=True) - -generated_ids = t5.generate( - input_ids=input_ids, - num_beams=1, - max_length=40, - #repetition_penalty=2.5 - ).squeeze() - -predicted_span = tokenizer.decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True) - - -``` diff --git a/model_cards/ahotrod/albert_xxlargev1_squad2_512/README.md b/model_cards/ahotrod/albert_xxlargev1_squad2_512/README.md deleted file mode 100644 index 2f7cf73689..0000000000 --- a/model_cards/ahotrod/albert_xxlargev1_squad2_512/README.md +++ /dev/null @@ -1,60 +0,0 @@ -## Albert xxlarge version 1 language model fine-tuned on SQuAD2.0 - -### (updated 30Sept2020) with the following results: - -``` -exact: 86.11134506864315 -f1: 89.35371214945009 -total': 11873 -HasAns_exact': 83.56950067476383 -HasAns_f1': 90.06353312254078 -HasAns_total': 5928 -NoAns_exact': 88.64592094196804 -NoAns_f1': 88.64592094196804 -NoAns_total': 5945 -best_exact': 86.11134506864315 -best_exact_thresh': 0.0 -best_f1': 89.35371214944985 -best_f1_thresh': 0.0 -``` - -### from script: - -``` -python ${EXAMPLES}/run_squad.py \ - --model_type albert \ - --model_name_or_path albert-xxlarge-v1 \ - --do_train \ - --do_eval \ - --train_file ${SQUAD}/train-v2.0.json \ - --predict_file ${SQUAD}/dev-v2.0.json \ - --version_2_with_negative \ - --do_lower_case \ - --num_train_epochs 3 \ - --max_steps 8144 \ - --warmup_steps 814 \ - --learning_rate 3e-5 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --per_gpu_train_batch_size 6 \ - --gradient_accumulation_steps 8 \ - --per_gpu_eval_batch_size 48 \ - --fp16 \ - --fp16_opt_level O1 \ - --threads 12 \ - --logging_steps 50 \ - --save_steps 3000 \ - --overwrite_output_dir \ - --output_dir ${MODEL_PATH} -``` - -### using the following software & system: - -``` -Transformers: 3.1.0 -PyTorch: 1.6.0 -TensorFlow: 2.3.1 -Python: 3.8.1 -OS: Linux-5.4.0-48-generic-x86_64-with-glibc2.10 -CPU/GPU: Intel i9-9900K / NVIDIA Titan RTX 24GB -``` diff --git a/model_cards/ahotrod/electra_large_discriminator_squad2_512/README.md b/model_cards/ahotrod/electra_large_discriminator_squad2_512/README.md deleted file mode 100644 index 66acb76d58..0000000000 --- a/model_cards/ahotrod/electra_large_discriminator_squad2_512/README.md +++ /dev/null @@ -1,58 +0,0 @@ -## ELECTRA_large_discriminator language model fine-tuned on SQuAD2.0 - -### with the following results: - -``` - "exact": 87.09677419354838, - "f1": 89.98343832723452, - "total": 11873, - "HasAns_exact": 84.66599190283401, - "HasAns_f1": 90.44759839056285, - "HasAns_total": 5928, - "NoAns_exact": 89.52060555088309, - "NoAns_f1": 89.52060555088309, - "NoAns_total": 5945, - "best_exact": 87.09677419354838, - "best_exact_thresh": 0.0, - "best_f1": 89.98343832723432, - "best_f1_thresh": 0.0 -``` -### from script: -``` -python ${EXAMPLES}/run_squad.py \ - --model_type electra \ - --model_name_or_path google/electra-large-discriminator \ - --do_train \ - --do_eval \ - --train_file ${SQUAD}/train-v2.0.json \ - --predict_file ${SQUAD}/dev-v2.0.json \ - --version_2_with_negative \ - --do_lower_case \ - --num_train_epochs 3 \ - --warmup_steps 306 \ - --weight_decay 0.01 \ - --learning_rate 3e-5 \ - --max_grad_norm 0.5 \ - --adam_epsilon 1e-6 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --per_gpu_train_batch_size 8 \ - --gradient_accumulation_steps 16 \ - --per_gpu_eval_batch_size 128 \ - --fp16 \ - --fp16_opt_level O1 \ - --threads 12 \ - --logging_steps 50 \ - --save_steps 1000 \ - --overwrite_output_dir \ - --output_dir ${MODEL_PATH} -``` -### using the following system & software: -``` -Transformers: 2.11.0 -PyTorch: 1.5.0 -TensorFlow: 2.2.0 -Python: 3.8.1 -OS/Platform: Linux-5.3.0-59-generic-x86_64-with-glibc2.10 -CPU/GPU: Intel i9-9900K / NVIDIA Titan RTX 24GB -``` diff --git a/model_cards/ahotrod/roberta_large_squad2/README.md b/model_cards/ahotrod/roberta_large_squad2/README.md deleted file mode 100644 index 2fe38471d4..0000000000 --- a/model_cards/ahotrod/roberta_large_squad2/README.md +++ /dev/null @@ -1,56 +0,0 @@ -## RoBERTa-large language model fine-tuned on SQuAD2.0 - -### with the following results: - -``` - "exact": 84.46896319380106, - "f1": 87.85388093408943, - "total": 11873, - "HasAns_exact": 81.37651821862349, - "HasAns_f1": 88.1560607844881, - "HasAns_total": 5928, - "NoAns_exact": 87.55256518082422, - "NoAns_f1": 87.55256518082422, - "NoAns_total": 5945, - "best_exact": 84.46896319380106, - "best_exact_thresh": 0.0, - "best_f1": 87.85388093408929, - "best_f1_thresh": 0.0 -``` -### from script: -``` -python ${EXAMPLES}/run_squad.py \ - --model_type roberta \ - --model_name_or_path roberta-large \ - --do_train \ - --do_eval \ - --train_file ${SQUAD}/train-v2.0.json \ - --predict_file ${SQUAD}/dev-v2.0.json \ - --version_2_with_negative \ - --do_lower_case \ - --num_train_epochs 3 \ - --warmup_steps 1642 \ - --weight_decay 0.01 \ - --learning_rate 3e-5 \ - --adam_epsilon 1e-6 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --per_gpu_train_batch_size 8 \ - --gradient_accumulation_steps 6 \ - --per_gpu_eval_batch_size 48 \ - --threads 12 \ - --logging_steps 50 \ - --save_steps 2000 \ - --overwrite_output_dir \ - --output_dir ${MODEL_PATH} -$@ -``` -### using the following system & software: -``` -Transformers: 2.7.0 -PyTorch: 1.4.0 -TensorFlow: 2.1.0 -Python: 3.7.7 -OS/Platform: Linux-5.3.0-46-generic-x86_64-with-debian-buster-sid -CPU/GPU: Intel i9-9900K / NVIDIA Titan RTX 24GB -``` diff --git a/model_cards/ahotrod/xlnet_large_squad2_512/README.md b/model_cards/ahotrod/xlnet_large_squad2_512/README.md deleted file mode 100644 index a680704af1..0000000000 --- a/model_cards/ahotrod/xlnet_large_squad2_512/README.md +++ /dev/null @@ -1,63 +0,0 @@ -## XLNet large language model fine-tuned on SQuAD2.0 - -### with the following results: - -``` - "exact": 82.07698138633876, - "f1": 85.898874470488, - "total": 11873, - "HasAns_exact": 79.60526315789474, - "HasAns_f1": 87.26000954590184, - "HasAns_total": 5928, - "NoAns_exact": 84.54163162321278, - "NoAns_f1": 84.54163162321278, - "NoAns_total": 5945, - "best_exact": 83.22243746315169, - "best_exact_thresh": -11.112004280090332, - "best_f1": 86.88541353813282, - "best_f1_thresh": -11.112004280090332 -``` -### from script: -``` -python -m torch.distributed.launch --nproc_per_node=2 ${RUN_SQUAD_DIR}/run_squad.py \ - --model_type xlnet \ - --model_name_or_path xlnet-large-cased \ - --do_train \ - --train_file ${SQUAD_DIR}/train-v2.0.json \ - --predict_file ${SQUAD_DIR}/dev-v2.0.json \ - --version_2_with_negative \ - --num_train_epochs 3 \ - --learning_rate 3e-5 \ - --adam_epsilon 1e-6 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --save_steps 2000 \ - --per_gpu_train_batch_size 1 \ - --gradient_accumulation_steps 24 \ - --output_dir ${MODEL_PATH} - -CUDA_VISIBLE_DEVICES=0 python ${RUN_SQUAD_DIR}/run_squad_II.py \ - --model_type xlnet \ - --model_name_or_path ${MODEL_PATH} \ - --do_eval \ - --train_file ${SQUAD_DIR}/train-v2.0.json \ - --predict_file ${SQUAD_DIR}/dev-v2.0.json \ - --version_2_with_negative \ - --max_seq_length 512 \ - --per_gpu_eval_batch_size 48 \ - --output_dir ${MODEL_PATH} -``` -### using the following system & software: -``` -OS/Platform: Linux-4.15.0-76-generic-x86_64-with-debian-buster-sid -GPU/CPU: 2 x NVIDIA 1080Ti / Intel i7-8700 -Transformers: 2.1.1 -PyTorch: 1.4.0 -TensorFlow: 2.1.0 -Python: 3.7.6 -``` -### Utilize this xlnet_large_squad2_512 fine-tuned model with: -```python -tokenizer = AutoTokenizer.from_pretrained("ahotrod/xlnet_large_squad2_512") -model = AutoModelForQuestionAnswering.from_pretrained("ahotrod/xlnet_large_squad2_512") -``` diff --git a/model_cards/ai4bharat/indic-bert/README.md b/model_cards/ai4bharat/indic-bert/README.md deleted file mode 100644 index 093508b002..0000000000 --- a/model_cards/ai4bharat/indic-bert/README.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -language: en -license: mit -datasets: -- AI4Bharat IndicNLP Corpora ---- - -# IndicBERT - -IndicBERT is a multilingual ALBERT model pretrained exclusively on 12 major Indian languages. It is pre-trained on our novel monolingual corpus of around 9 billion tokens and subsequently evaluated on a set of diverse tasks. IndicBERT has much fewer parameters than other multilingual models (mBERT, XLM-R etc.) while it also achieves a performance on-par or better than these models. - -The 12 languages covered by IndicBERT are: Assamese, Bengali, English, Gujarati, Hindi, Kannada, Malayalam, Marathi, Oriya, Punjabi, Tamil, Telugu. - -The code can be found [here](https://github.com/divkakwani/indic-bert). For more information, checkout our [project page](https://indicnlp.ai4bharat.org/) or our [paper](https://indicnlp.ai4bharat.org/papers/arxiv2020_indicnlp_corpus.pdf). - - - -## Pretraining Corpus - -We pre-trained indic-bert on AI4Bharat's monolingual corpus. The corpus has the following distribution of languages: - - -| Language | as | bn | en | gu | hi | kn | | -| ----------------- | ------ | ------ | ------ | ------ | ------ | ------ | ------- | -| **No. of Tokens** | 36.9M | 815M | 1.34B | 724M | 1.84B | 712M | | -| **Language** | **ml** | **mr** | **or** | **pa** | **ta** | **te** | **all** | -| **No. of Tokens** | 767M | 560M | 104M | 814M | 549M | 671M | 8.9B | - - - -## Evaluation Results - -IndicBERT is evaluated on IndicGLUE and some additional tasks. The results are summarized below. For more details about the tasks, refer our [official repo](https://github.com/divkakwani/indic-bert) - -#### IndicGLUE - -Task | mBERT | XLM-R | IndicBERT ------| ----- | ----- | ------ -News Article Headline Prediction | 89.58 | 95.52 | **95.87** -Wikipedia Section Title Prediction| **73.66** | 66.33 | 73.31 -Cloze-style multiple-choice QA | 39.16 | 27.98 | **41.87** -Article Genre Classification | 90.63 | 97.03 | **97.34** -Named Entity Recognition (F1-score) | **73.24** | 65.93 | 64.47 -Cross-Lingual Sentence Retrieval Task | 21.46 | 13.74 | **27.12** -Average | 64.62 | 61.09 | **66.66** - -#### Additional Tasks - - -Task | Task Type | mBERT | XLM-R | IndicBERT ------| ----- | ----- | ------ | ----- -BBC News Classification | Genre Classification | 60.55 | **75.52** | 74.60 -IIT Product Reviews | Sentiment Analysis | 74.57 | **78.97** | 71.32 -IITP Movie Reviews | Sentiment Analaysis | 56.77 | **61.61** | 59.03 -Soham News Article | Genre Classification | 80.23 | **87.6** | 78.45 -Midas Discourse | Discourse Analysis | 71.20 | **79.94** | 78.44 -iNLTK Headlines Classification | Genre Classification | 87.95 | 93.38 | **94.52** -ACTSA Sentiment Analysis | Sentiment Analysis | 48.53 | 59.33 | **61.18** -Winograd NLI | Natural Language Inference | 56.34 | 55.87 | **56.34** -Choice of Plausible Alternative (COPA) | Natural Language Inference | 54.92 | 51.13 | **58.33** -Amrita Exact Paraphrase | Paraphrase Detection | **93.81** | 93.02 | 93.75 -Amrita Rough Paraphrase | Paraphrase Detection | 83.38 | 82.20 | **84.33** -Average | | 69.84 | **74.42** | 73.66 - - -\* Note: all models have been restricted to a max_seq_length of 128. - - - -## Downloads - -The model can be downloaded [here](https://storage.googleapis.com/ai4bharat-public-indic-nlp-corpora/models/indic-bert-v1.tar.gz). Both tf checkpoints and pytorch binaries are included in the archive. Alternatively, you can also download it from [Huggingface](https://huggingface.co/ai4bharat/indic-bert). - - - -## Citing - -If you are using any of the resources, please cite the following article: - -``` -@inproceedings{kakwani2020indicnlpsuite, - title={{IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages}}, - author={Divyanshu Kakwani and Anoop Kunchukuttan and Satish Golla and Gokul N.C. and Avik Bhattacharyya and Mitesh M. Khapra and Pratyush Kumar}, - year={2020}, - booktitle={Findings of EMNLP}, -} -``` - -We would like to hear from you if: - -- You are using our resources. Please let us know how you are putting these resources to use. -- You have any feedback on these resources. - - - -## License - -The IndicBERT code (and models) are released under the MIT License. - -## Contributors - -- Divyanshu Kakwani -- Anoop Kunchukuttan -- Gokul NC -- Satish Golla -- Avik Bhattacharyya -- Mitesh Khapra -- Pratyush Kumar - -This work is the outcome of a volunteer effort as part of [AI4Bharat initiative](https://ai4bharat.org). - - - -## Contact - -- Anoop Kunchukuttan ([anoop.kunchukuttan@gmail.com](mailto:anoop.kunchukuttan@gmail.com)) -- Mitesh Khapra ([miteshk@cse.iitm.ac.in](mailto:miteshk@cse.iitm.ac.in)) -- Pratyush Kumar ([pratyush@cse.iitm.ac.in](mailto:pratyush@cse.iitm.ac.in)) diff --git a/model_cards/akhooli/gpt2-small-arabic-poetry/README.md b/model_cards/akhooli/gpt2-small-arabic-poetry/README.md deleted file mode 100644 index eb09509191..0000000000 --- a/model_cards/akhooli/gpt2-small-arabic-poetry/README.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -language: "ar" -tags: -- text-generation -license: "" -datasets: -- Arabic poetry from several eras ---- - -# GPT2-Small-Arabic-Poetry - -## Model description - -Fine-tuned model of Arabic poetry dataset based on gpt2-small-arabic. - -## Intended uses & limitations - -#### How to use - -An example is provided in this [colab notebook](https://colab.research.google.com/drive/1mRl7c-5v-Klx27EEAEOAbrfkustL4g7a?usp=sharing). - -#### Limitations and bias - -Both the GPT2-small-arabic (trained on Arabic Wikipedia) and this model have several limitations in terms of coverage and training performance. -Use them as demonstrations or proof of concepts but not as production code. - -## Training data - -This pretrained model used the [Arabic Poetry dataset](https://www.kaggle.com/ahmedabelal/arabic-poetry) from 9 different eras with a total of around 40k poems. -The dataset was trained (fine-tuned) based on the [gpt2-small-arabic](https://huggingface.co/akhooli/gpt2-small-arabic) transformer model. - -## Training procedure - -Training was done using [Simple Transformers](https://github.com/ThilinaRajapakse/simpletransformers) library on Kaggle, using free GPU. - -## Eval results -Final perplexity reached ws 76.3, loss: 4.33 - -### BibTeX entry and citation info - -```bibtex -@inproceedings{Abed Khooli, - year={2020} -} -``` diff --git a/model_cards/akhooli/gpt2-small-arabic/README.md b/model_cards/akhooli/gpt2-small-arabic/README.md deleted file mode 100644 index 08eb482821..0000000000 --- a/model_cards/akhooli/gpt2-small-arabic/README.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -language: "ar" -datasets: -- Arabic Wikipedia -metrics: -- none ---- - -# GPT2-Small-Arabic - -## Model description - -GPT2 model from Arabic Wikipedia dataset based on gpt2-small (using Fastai2). - -## Intended uses & limitations - -#### How to use - -An example is provided in this [colab notebook](https://colab.research.google.com/drive/1mRl7c-5v-Klx27EEAEOAbrfkustL4g7a?usp=sharing). -Both text and poetry (fine-tuned model) generation are included. - -#### Limitations and bias - -GPT2-small-arabic (trained on Arabic Wikipedia) has several limitations in terms of coverage (Arabic Wikipeedia quality, no diacritics) and training performance. -Use as demonstration or proof of concepts but not as production code. - -## Training data - -This pretrained model used the Arabic Wikipedia dump (around 900 MB). - -## Training procedure - -Training was done using [Fastai2](https://github.com/fastai/fastai2/) library on Kaggle, using free GPU. - -## Eval results -Final perplexity reached was 72.19, loss: 4.28, accuracy: 0.307 - -### BibTeX entry and citation info - -```bibtex -@inproceedings{Abed Khooli, - year={2020} -} -``` diff --git a/model_cards/akhooli/mbart-large-cc25-ar-en/README.md b/model_cards/akhooli/mbart-large-cc25-ar-en/README.md deleted file mode 100644 index f114b38da0..0000000000 --- a/model_cards/akhooli/mbart-large-cc25-ar-en/README.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -tags: -- translation - -language: -- ar -- en - -license: mit ---- -### mbart-large-ar-en -This is mbart-large-cc25, finetuned on a subset of the OPUS corpus for ar_en. -Usage: see [example notebook](https://colab.research.google.com/drive/1I6RFOWMaTpPBX7saJYjnSTddW0TD6H1t?usp=sharing) -Note: model has limited training set, not fully trained (do not use for production). -Other models by me: [Abed Khooli](https://huggingface.co/akhooli) diff --git a/model_cards/akhooli/mbart-large-cc25-en-ar/README.md b/model_cards/akhooli/mbart-large-cc25-en-ar/README.md deleted file mode 100644 index 280d222573..0000000000 --- a/model_cards/akhooli/mbart-large-cc25-en-ar/README.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -tags: -- translation - -language: -- en -- ar - -license: mit ---- -### mbart-large-en-ar -This is mbart-large-cc25, finetuned on a subset of the UN corpus for en_ar. -Usage: see [example notebook](https://colab.research.google.com/drive/1I6RFOWMaTpPBX7saJYjnSTddW0TD6H1t?usp=sharing) -Note: model has limited training set, not fully trained (do not use for production). diff --git a/model_cards/akhooli/personachat-arabic/README.md b/model_cards/akhooli/personachat-arabic/README.md deleted file mode 100644 index 4a78396445..0000000000 --- a/model_cards/akhooli/personachat-arabic/README.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -tags: -- conversational -language: -- ar -license: mit ---- -## personachat-arabic (conversational AI) -This is personachat-arabic, using a subset from the persona-chat validation dataset, machine translated to Arabic (from English) -and fine-tuned from [akhooli/gpt2-small-arabic](https://huggingface.co/akhooli/gpt2-small-arabic) which is a limited text generation model. -Usage: see the last section of this [example notebook](https://colab.research.google.com/drive/1I6RFOWMaTpPBX7saJYjnSTddW0TD6H1t?usp=sharing) -Note: model has limited training set which was machine translated (do not use for production). diff --git a/model_cards/akhooli/xlm-r-large-arabic-sent/README.md b/model_cards/akhooli/xlm-r-large-arabic-sent/README.md deleted file mode 100644 index 4dcbc05e84..0000000000 --- a/model_cards/akhooli/xlm-r-large-arabic-sent/README.md +++ /dev/null @@ -1,13 +0,0 @@ ---- - -language: -- ar -- en - -license: mit ---- -### xlm-r-large-arabic-sent -Multilingual sentiment classification (Label_0: mixed, Label_1: negative, Label_2: positive) of Arabic reviews by fine-tuning XLM-Roberta-Large. -Zero shot classification of other languages (also works in mixed languages - ex. Arabic & English). Mixed category is not accurate and may confuse other -classes (was based on a rate of 3 out of 5 in reviews). -Usage: see last section in this [Colab notebook](https://lnkd.in/d3bCFyZ) diff --git a/model_cards/akhooli/xlm-r-large-arabic-toxic/README.md b/model_cards/akhooli/xlm-r-large-arabic-toxic/README.md deleted file mode 100644 index db380461b2..0000000000 --- a/model_cards/akhooli/xlm-r-large-arabic-toxic/README.md +++ /dev/null @@ -1,12 +0,0 @@ ---- - -language: -- ar -- en - -license: mit ---- -### xlm-r-large-arabic-toxic (toxic/hate speech classifier) -Toxic (hate speech) classification (Label_0: non-toxic, Label_1: toxic) of Arabic comments by fine-tuning XLM-Roberta-Large. -Zero shot classification of other languages (also works in mixed languages - ex. Arabic & English). -Usage and further info: see last section in this [Colab notebook](https://lnkd.in/d3bCFyZ) diff --git a/model_cards/albert-base-v1-README.md b/model_cards/albert-base-v1-README.md deleted file mode 100644 index 91e0b067e3..0000000000 --- a/model_cards/albert-base-v1-README.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -tags: -- exbert - -license: apache-2.0 ---- - - - - diff --git a/model_cards/albert-xxlarge-v2-README.md b/model_cards/albert-xxlarge-v2-README.md deleted file mode 100644 index b28a8ffb03..0000000000 --- a/model_cards/albert-xxlarge-v2-README.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -tags: -- exbert - -license: apache-2.0 ---- - - - - \ No newline at end of file diff --git a/model_cards/aliosm/ComVE-distilgpt2/README.md b/model_cards/aliosm/ComVE-distilgpt2/README.md deleted file mode 100644 index 0021a6eb19..0000000000 --- a/model_cards/aliosm/ComVE-distilgpt2/README.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -language: "en" -tags: -- exbert -- commonsense -- semeval2020 -- comve -license: "mit" -datasets: -- ComVE -metrics: -- bleu -widget: -- text: "Chicken can swim in water. <|continue|>" ---- - -# ComVE-distilgpt2 - -## Model description - -Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective. -The model is able to generate a reason why a given natural language statement is against commonsense. - -## Intended uses & limitations - -You can use the raw model for text generation to generate reasons why natural language statements are against commonsense. - -#### How to use - -You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script. - -*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again. - -#### Limitations and bias - -The model biased to negate the entered sentence usually instead of producing a factual reason. - -## Training data - -The model is initialized from the [distilgpt2](https://github.com/huggingface/transformers/blob/master/model_cards/distilgpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons. - -## Training procedure - -Each natural language statement that against commonsense is concatenated with its reference reason with `<|continue|>` as a separator, then the model finetuned using CLM objective. -The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 15 epochs, 128 maximum sequence length and 64 batch size. - -
- -
- -## Eval results - -The model achieved 13.7582/13.8026 BLEU scores on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset. - -### BibTeX entry and citation info - -```bibtex -@article{fadel2020justers, - title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation}, - author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik}, - year={2020} -} -``` - - - - diff --git a/model_cards/aliosm/ComVE-gpt2-large/README.md b/model_cards/aliosm/ComVE-gpt2-large/README.md deleted file mode 100644 index a203025c09..0000000000 --- a/model_cards/aliosm/ComVE-gpt2-large/README.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -language: "en" -tags: -- gpt2 -- exbert -- commonsense -- semeval2020 -- comve -license: "mit" -datasets: -- https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation -metrics: -- bleu -widget: -- text: "Chicken can swim in water. <|continue|>" ---- - -# ComVE-gpt2-large - -## Model description - -Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective. -The model is able to generate a reason why a given natural language statement is against commonsense. - -## Intended uses & limitations - -You can use the raw model for text generation to generate reasons why natural language statements are against commonsense. - -#### How to use - -You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script. - -*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again. - -#### Limitations and bias - -The model biased to negate the entered sentence usually instead of producing a factual reason. - -## Training data - -The model is initialized from the [gpt2-large](https://github.com/huggingface/transformers/blob/master/model_cards/gpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons. - -## Training procedure - -Each natural language statement that against commonsense is concatenated with its reference reason with `<|conteniue|>` as a separator, then the model finetuned using CLM objective. -The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size. - -
- -
- -## Eval results - -The model achieved 16.5110/15.9299 BLEU scores on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset. - -### BibTeX entry and citation info - -```bibtex -@article{fadel2020justers, - title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation}, - author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik}, - year={2020} -} -``` - - - - diff --git a/model_cards/aliosm/ComVE-gpt2-medium/README.md b/model_cards/aliosm/ComVE-gpt2-medium/README.md deleted file mode 100644 index 1257d14b9d..0000000000 --- a/model_cards/aliosm/ComVE-gpt2-medium/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: "en" -tags: -- gpt2 -- exbert -- commonsense -- semeval2020 -- comve -license: "mit" -datasets: -- ComVE -metrics: -- bleu -widget: -- text: "Chicken can swim in water. <|continue|>" ---- - -# ComVE-gpt2-medium - -## Model description - -Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective. -The model is able to generate a reason why a given natural language statement is against commonsense. - -## Intended uses & limitations - -You can use the raw model for text generation to generate reasons why natural language statements are against commonsense. - -#### How to use - -You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script. - -*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again. - -#### Limitations and bias - -The model biased to negate the entered sentence usually instead of producing a factual reason. - -## Training data - -The model is initialized from the [gpt2-medium](https://github.com/huggingface/transformers/blob/master/model_cards/gpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons. - -## Training procedure - -Each natural language statement that against commonsense is concatenated with its reference reason with `<|continue|>` as a separator, then the model finetuned using CLM objective. -The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size. - -
- -
- -## Eval results - -The model achieved fifth place with 16.7153/16.1187 BLEU scores and third place with 1.94 Human Evaluation score on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset. - -These are some examples generated by the model: -| Against Commonsense Statement | Generated Reason | -|:-----------------------------------------------------:|:--------------------------------------------:| -| Chicken can swim in water. | Chicken can't swim. | -| shoes can fly | Shoes are not able to fly. | -| Chocolate can be used to make a coffee pot | Chocolate is not used to make coffee pots. | -| you can also buy tickets online with an identity card | You can't buy tickets with an identity card. | -| a ball is square and can roll | A ball is round and cannot roll. | -| You can use detergent to dye your hair. | Detergent is used to wash clothes. | -| you can eat mercury | mercury is poisonous | -| A gardener can follow a suspect | gardener is not a police officer | -| cars can float in the ocean just like a boat | Cars are too heavy to float in the ocean. | -| I am going to work so I can lose money. | Working is not a way to lose money. | - -### BibTeX entry and citation info - -```bibtex -@article{fadel2020justers, - title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation}, - author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik}, - year={2020} -} -``` - - - - diff --git a/model_cards/aliosm/ComVE-gpt2/README.md b/model_cards/aliosm/ComVE-gpt2/README.md deleted file mode 100644 index 7f8ce0b5d0..0000000000 --- a/model_cards/aliosm/ComVE-gpt2/README.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -language: "en" -tags: -- exbert -- commonsense -- semeval2020 -- comve -license: "mit" -datasets: -- ComVE -metrics: -- bleu -widget: -- text: "Chicken can swim in water. <|continue|>" ---- - -# ComVE-gpt2 - -## Model description - -Finetuned model on Commonsense Validation and Explanation (ComVE) dataset introduced in [SemEval2020 Task4](https://competitions.codalab.org/competitions/21080) using a causal language modeling (CLM) objective. -The model is able to generate a reason why a given natural language statement is against commonsense. - -## Intended uses & limitations - -You can use the raw model for text generation to generate reasons why natural language statements are against commonsense. - -#### How to use - -You can use this model directly to generate reasons why the given statement is against commonsense using [`generate.sh`](https://github.com/AliOsm/SemEval2020-Task4-ComVE/tree/master/TaskC-Generation) script. - -*Note:* make sure that you are using version `2.4.1` of `transformers` package. Newer versions has some issue in text generation and the model repeats the last token generated again and again. - -#### Limitations and bias - -The model biased to negate the entered sentence usually instead of producing a factual reason. - -## Training data - -The model is initialized from the [gpt2](https://github.com/huggingface/transformers/blob/master/model_cards/gpt2-README.md) model and finetuned using [ComVE](https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-Explanation) dataset which contains 10K against commonsense sentences, each of them is paired with three reference reasons. - -## Training procedure - -Each natural language statement that against commonsense is concatenated with its reference reason with `<|continue|>` as a separator, then the model finetuned using CLM objective. -The model trained on Nvidia Tesla P100 GPU from Google Colab platform with 5e-5 learning rate, 5 epochs, 128 maximum sequence length and 64 batch size. - -
- -
- -## Eval results - -The model achieved 14.0547/13.6534 BLEU scores on SemEval2020 Task4: Commonsense Validation and Explanation development and testing dataset. - -### BibTeX entry and citation info - -```bibtex -@article{fadel2020justers, - title={JUSTers at SemEval-2020 Task 4: Evaluating Transformer Models Against Commonsense Validation and Explanation}, - author={Fadel, Ali and Al-Ayyoub, Mahmoud and Cambria, Erik}, - year={2020} -} -``` - - - - diff --git a/model_cards/aliosm/ai-soco-cpp-roberta-small-clas/README.md b/model_cards/aliosm/ai-soco-cpp-roberta-small-clas/README.md deleted file mode 100644 index dbf34874ef..0000000000 --- a/model_cards/aliosm/ai-soco-cpp-roberta-small-clas/README.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -language: "c++" -tags: -- exbert -- authorship-identification -- fire2020 -- pan2020 -- ai-soco -- classification -license: "mit" -datasets: -- ai-soco -metrics: -- accuracy ---- - -# ai-soco-c++-roberta-small-clas - -## Model description - -`ai-soco-c++-roberta-small` model fine-tuned on [AI-SOCO](https://sites.google.com/view/ai-soco-2020) task. - -#### How to use - -You can use the model directly after tokenizing the text using the provided tokenizer with the model files. - -#### Limitations and bias - -The model is limited to C++ programming language only. - -## Training data - -The model initialized from [`ai-soco-c++-roberta-small`](https://github.com/huggingface/transformers/blob/master/model_cards/aliosm/ai-soco-c++-roberta-small) model and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset to do text classification. - -## Training procedure - -The model trained on Google Colab platform using V100 GPU for 10 epochs, 32 batch size, 512 max sequence length (sequences larger than 512 were truncated). Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization. - -## Eval results - -The model achieved 93.19%/92.88% accuracy on AI-SOCO task and ranked in the 4th place. - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ai-soco-2020-fire, - title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}", - author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo", - booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)", - year = "2020" -} -``` - - - - diff --git a/model_cards/aliosm/ai-soco-cpp-roberta-small/README.md b/model_cards/aliosm/ai-soco-cpp-roberta-small/README.md deleted file mode 100644 index df1af7af75..0000000000 --- a/model_cards/aliosm/ai-soco-cpp-roberta-small/README.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -language: "c++" -tags: -- exbert -- authorship-identification -- fire2020 -- pan2020 -- ai-soco -license: "mit" -datasets: -- ai-soco -metrics: -- perplexity ---- - -# ai-soco-c++-roberta-small - -## Model description - -From scratch pre-trained RoBERTa model with 6 layers and 12 attention heads using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which consists of C++ codes crawled from CodeForces website. - -## Intended uses & limitations - -The model can be used to do code classification, authorship identification and other downstream tasks on C++ programming language. - -#### How to use - -You can use the model directly after tokenizing the text using the provided tokenizer with the model files. - -#### Limitations and bias - -The model is limited to C++ programming language only. - -## Training data - -The model initialized randomly and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which contains 100K C++ source codes. - -## Training procedure - -The model trained on Google Colab platform with 8 TPU cores for 200 epochs, 16\*8 batch size, 512 max sequence length and MLM objective. Other parameters were defaulted to the values mentioned in [`run_language_modelling.py`](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py) script. Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization. - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ai-soco-2020-fire, - title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}", - author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo", - booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)", - year = "2020" -} -``` - - - - diff --git a/model_cards/aliosm/ai-soco-cpp-roberta-tiny-96-clas/README.md b/model_cards/aliosm/ai-soco-cpp-roberta-tiny-96-clas/README.md deleted file mode 100644 index 736e28f9a0..0000000000 --- a/model_cards/aliosm/ai-soco-cpp-roberta-tiny-96-clas/README.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -language: "c++" -tags: -- exbert -- authorship-identification -- fire2020 -- pan2020 -- ai-soco -- classification -license: "mit" -datasets: -- ai-soco -metrics: -- accuracy ---- - -# ai-soco-c++-roberta-tiny-96-clas - -## Model description - -`ai-soco-c++-roberta-tiny-96` model fine-tuned on [AI-SOCO](https://sites.google.com/view/ai-soco-2020) task. - -#### How to use - -You can use the model directly after tokenizing the text using the provided tokenizer with the model files. - -#### Limitations and bias - -The model is limited to C++ programming language only. - -## Training data - -The model initialized from [`ai-soco-c++-roberta-tiny-96`](https://github.com/huggingface/transformers/blob/master/model_cards/aliosm/ai-soco-c++-roberta-tiny-96) model and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset to do text classification. - -## Training procedure - -The model trained on Google Colab platform using V100 GPU for 10 epochs, 16 batch size, 512 max sequence length (sequences larger than 512 were truncated). Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization. - -## Eval results - -The model achieved 91.12%/91.02% accuracy on AI-SOCO task and ranked in the 7th place. - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ai-soco-2020-fire, - title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}", - author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo", - booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)", - year = "2020" -} -``` - - - - diff --git a/model_cards/aliosm/ai-soco-cpp-roberta-tiny-96/README.md b/model_cards/aliosm/ai-soco-cpp-roberta-tiny-96/README.md deleted file mode 100644 index 4593c556a9..0000000000 --- a/model_cards/aliosm/ai-soco-cpp-roberta-tiny-96/README.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -language: "c++" -tags: -- exbert -- authorship-identification -- fire2020 -- pan2020 -- ai-soco -license: "mit" -datasets: -- ai-soco -metrics: -- perplexity ---- - -# ai-soco-c++-roberta-tiny-96 - -## Model description - -From scratch pre-trained RoBERTa model with 1 layers and 96 attention heads using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which consists of C++ codes crawled from CodeForces website. - -## Intended uses & limitations - -The model can be used to do code classification, authorship identification and other downstream tasks on C++ programming language. - -#### How to use - -You can use the model directly after tokenizing the text using the provided tokenizer with the model files. - -#### Limitations and bias - -The model is limited to C++ programming language only. - -## Training data - -The model initialized randomly and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which contains 100K C++ source codes. - -## Training procedure - -The model trained on Google Colab platform with 8 TPU cores for 200 epochs, 16\*8 batch size, 512 max sequence length and MLM objective. Other parameters were defaulted to the values mentioned in [`run_language_modelling.py`](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py) script. Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization. - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ai-soco-2020-fire, - title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}", - author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo", - booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)", - year = "2020" -} -``` - - - - diff --git a/model_cards/aliosm/ai-soco-cpp-roberta-tiny-clas/README.md b/model_cards/aliosm/ai-soco-cpp-roberta-tiny-clas/README.md deleted file mode 100644 index 757bf22c6a..0000000000 --- a/model_cards/aliosm/ai-soco-cpp-roberta-tiny-clas/README.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -language: "c++" -tags: -- exbert -- authorship-identification -- fire2020 -- pan2020 -- ai-soco -- classification -license: "mit" -datasets: -- ai-soco -metrics: -- accuracy ---- - -# ai-soco-c++-roberta-tiny-clas - -## Model description - -`ai-soco-c++-roberta-tiny` model fine-tuned on [AI-SOCO](https://sites.google.com/view/ai-soco-2020) task. - -#### How to use - -You can use the model directly after tokenizing the text using the provided tokenizer with the model files. - -#### Limitations and bias - -The model is limited to C++ programming language only. - -## Training data - -The model initialized from [`ai-soco-c++-roberta-tiny`](https://github.com/huggingface/transformers/blob/master/model_cards/aliosm/ai-soco-c++-roberta-tiny) model and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset to do text classification. - -## Training procedure - -The model trained on Google Colab platform using V100 GPU for 10 epochs, 32 batch size, 512 max sequence length (sequences larger than 512 were truncated). Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization. - -## Eval results - -The model achieved 87.66%/87.46% accuracy on AI-SOCO task and ranked in the 9th place. - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ai-soco-2020-fire, - title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}", - author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo", - booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)", - year = "2020" -} -``` - - - - diff --git a/model_cards/aliosm/ai-soco-cpp-roberta-tiny/README.md b/model_cards/aliosm/ai-soco-cpp-roberta-tiny/README.md deleted file mode 100644 index 164cce0222..0000000000 --- a/model_cards/aliosm/ai-soco-cpp-roberta-tiny/README.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -language: "c++" -tags: -- exbert -- authorship-identification -- fire2020 -- pan2020 -- ai-soco -license: "mit" -datasets: -- ai-soco -metrics: -- perplexity ---- - -# ai-soco-c++-roberta-tiny - -## Model description - -From scratch pre-trained RoBERTa model with 1 layers and 12 attention heads using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which consists of C++ codes crawled from CodeForces website. - -## Intended uses & limitations - -The model can be used to do code classification, authorship identification and other downstream tasks on C++ programming language. - -#### How to use - -You can use the model directly after tokenizing the text using the provided tokenizer with the model files. - -#### Limitations and bias - -The model is limited to C++ programming language only. - -## Training data - -The model initialized randomly and trained using [AI-SOCO](https://sites.google.com/view/ai-soco-2020) dataset which contains 100K C++ source codes. - -## Training procedure - -The model trained on Google Colab platform with 8 TPU cores for 200 epochs, 32\*8 batch size, 512 max sequence length and MLM objective. Other parameters were defaulted to the values mentioned in [`run_language_modelling.py`](https://github.com/huggingface/transformers/blob/master/examples/language-modeling/run_language_modeling.py) script. Each continues 4 spaces were converted to a single tab character (`\t`) before tokenization. - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ai-soco-2020-fire, - title = "Overview of the {PAN@FIRE} 2020 Task on {Authorship Identification of SOurce COde (AI-SOCO)}", - author = "Fadel, Ali and Musleh, Husam and Tuffaha, Ibraheem and Al-Ayyoub, Mahmoud and Jararweh, Yaser and Benkhelifa, Elhadj and Rosso, Paolo", - booktitle = "Proceedings of The 12th meeting of the Forum for Information Retrieval Evaluation (FIRE 2020)", - year = "2020" -} -``` - - - - diff --git a/model_cards/allegro/herbert-base-cased/README.md b/model_cards/allegro/herbert-base-cased/README.md deleted file mode 100644 index 0afd84d4a6..0000000000 --- a/model_cards/allegro/herbert-base-cased/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: pl -tags: -- herbert -license: cc-by-sa-4.0 ---- - -# HerBERT -**[HerBERT](https://en.wikipedia.org/wiki/Zbigniew_Herbert)** is a BERT-based Language Model trained on Polish Corpora -using MLM and SSO objectives with dynamic masking of whole words. -Model training and experiments were conducted with [transformers](https://github.com/huggingface/transformers) in version 2.9. - -## Tokenizer -The training dataset was tokenized into subwords using ``CharBPETokenizer`` a character level byte-pair encoding with -a vocabulary size of 50k tokens. The tokenizer itself was trained with a [tokenizers](https://github.com/huggingface/tokenizers) library. -We kindly encourage you to use the **Fast** version of tokenizer, namely ``HerbertTokenizerFast``. - -## HerBERT usage - - -Example code: -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("allegro/herbert-base-cased") -model = AutoModel.from_pretrained("allegro/herbert-base-cased") - -output = model( - **tokenizer.batch_encode_plus( - [ - ( - "A potem szedł środkiem drogi w kurzawie, bo zamiatał nogami, ślepy dziad prowadzony przez tłustego kundla na sznurku.", - "A potem leciał od lasu chłopak z butelką, ale ten ujrzawszy księdza przy drodze okrążył go z dala i biegł na przełaj pól do karczmy." - ) - ], - padding='longest', - add_special_tokens=True, - return_tensors='pt' - ) -) -``` - - -## License -CC BY-SA 4.0 - - -## Authors -Model was trained by **Allegro Machine Learning Research** team. - -You can contact us at: klejbenchmark@allegro.pl diff --git a/model_cards/allegro/herbert-klej-cased-tokenizer-v1/README.md b/model_cards/allegro/herbert-klej-cased-tokenizer-v1/README.md deleted file mode 100644 index cf6da20c83..0000000000 --- a/model_cards/allegro/herbert-klej-cased-tokenizer-v1/README.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -language: pl ---- - -# HerBERT tokenizer - -**[HerBERT](https://en.wikipedia.org/wiki/Zbigniew_Herbert)** tokenizer is a character level byte-pair encoding with -vocabulary size of 50k tokens. The tokenizer was trained on [Wolne Lektury](https://wolnelektury.pl/) and a publicly available subset of -[National Corpus of Polish](http://nkjp.pl/index.php?page=14&lang=0) with [fastBPE](https://github.com/glample/fastBPE) library. -Tokenizer utilize `XLMTokenizer` implementation from [transformers](https://github.com/huggingface/transformers). - -## Tokenizer usage -Herbert tokenizer should be used together with [HerBERT model](https://huggingface.co/allegro/herbert-klej-cased-v1): -```python -from transformers import XLMTokenizer, RobertaModel - -tokenizer = XLMTokenizer.from_pretrained("allegro/herbert-klej-cased-tokenizer-v1") -model = RobertaModel.from_pretrained("allegro/herbert-klej-cased-v1") - -encoded_input = tokenizer.encode("Kto ma lepszą sztukę, ma lepszy rząd – to jasne.", return_tensors='pt') -outputs = model(encoded_input) -``` - -## License -CC BY-SA 4.0 - -## Citation -If you use this tokenizer, please cite the following paper: -``` -@misc{rybak2020klej, - title={KLEJ: Comprehensive Benchmark for Polish Language Understanding}, - author={Piotr Rybak and Robert Mroczkowski and Janusz Tracz and Ireneusz Gawlik}, - year={2020}, - eprint={2005.00630}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` -Paper is accepted at ACL 2020, as soon as proceedings appear, we will update the BibTeX. - -## Authors -Tokenizer was created by **Allegro Machine Learning Research** team. - -You can contact us at: klejbenchmark@allegro.pl diff --git a/model_cards/allegro/herbert-klej-cased-v1/README.md b/model_cards/allegro/herbert-klej-cased-v1/README.md deleted file mode 100644 index c594311a7f..0000000000 --- a/model_cards/allegro/herbert-klej-cased-v1/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: pl ---- - -# HerBERT -**[HerBERT](https://en.wikipedia.org/wiki/Zbigniew_Herbert)** is a BERT-based Language Model trained on Polish Corpora -using only MLM objective with dynamic masking of whole words. For more details, please refer to: -[KLEJ: Comprehensive Benchmark for Polish Language Understanding](https://arxiv.org/abs/2005.00630). - -## Dataset -**HerBERT** training dataset is a combination of several publicly available corpora for Polish language: - -| Corpus | Tokens | Texts | -| :------ | ------: | ------: | -| [OSCAR](https://traces1.inria.fr/oscar/)| 6710M | 145M | -| [Open Subtitles](http://opus.nlpl.eu/OpenSubtitles-v2018.php) | 1084M | 1.1M | -| [Wikipedia](https://dumps.wikimedia.org/) | 260M | 1.5M | -| [Wolne Lektury](https://wolnelektury.pl/) | 41M | 5.5k | -| [Allegro Articles](https://allegro.pl/artykuly) | 18M | 33k | - -## Tokenizer -The training dataset was tokenized into subwords using [HerBERT Tokenizer](https://huggingface.co/allegro/herbert-klej-cased-tokenizer-v1); a character level byte-pair encoding with -a vocabulary size of 50k tokens. The tokenizer itself was trained on [Wolne Lektury](https://wolnelektury.pl/) and a publicly available subset of -[National Corpus of Polish](http://nkjp.pl/index.php?page=14&lang=0) with a [fastBPE](https://github.com/glample/fastBPE) library. - -Tokenizer utilizes `XLMTokenizer` implementation for that reason, one should load it as `allegro/herbert-klej-cased-tokenizer-v1`. - -## HerBERT models summary -| Model | WWM | Cased | Tokenizer | Vocab Size | Batch Size | Train Steps | -| :------ | ------: | ------: | ------: | ------: | ------: | ------: | -| herbert-klej-cased-v1 | YES | YES | BPE | 50K | 570 | 180k | - -## Model evaluation -HerBERT was evaluated on the [KLEJ](https://klejbenchmark.com/) benchmark, publicly available set of nine evaluation tasks for the Polish language understanding. -It had the best average performance and obtained the best results for three of them. - -| Model | Average | NKJP-NER | CDSC-E | CDSC-R | CBD | PolEmo2.0-IN |PolEmo2.0-OUT | DYK | PSC | AR | -| :------ | ------: | ------: | ------: | ------: | ------: | ------: | ------: | ------: | ------: | ------: | -| herbert-klej-cased-v1 | **80.5** | 92.7 | 92.5 | 91.9 | **50.3** | **89.2** |**76.3** |52.1 |95.3 | 84.5 | - -Full leaderboard is available [online](https://klejbenchmark.com/leaderboard). - - -## HerBERT usage -Model training and experiments were conducted with [transformers](https://github.com/huggingface/transformers) in version 2.0. - -Example code: -```python -from transformers import XLMTokenizer, RobertaModel - -tokenizer = XLMTokenizer.from_pretrained("allegro/herbert-klej-cased-tokenizer-v1") -model = RobertaModel.from_pretrained("allegro/herbert-klej-cased-v1") - -encoded_input = tokenizer.encode("Kto ma lepszą sztukę, ma lepszy rząd – to jasne.", return_tensors='pt') -outputs = model(encoded_input) -``` - -HerBERT can also be loaded using `AutoTokenizer` and `AutoModel`: - -```python -tokenizer = AutoTokenizer.from_pretrained("allegro/herbert-klej-cased-tokenizer-v1") -model = AutoModel.from_pretrained("allegro/herbert-klej-cased-v1") -``` - -## License -CC BY-SA 4.0 - -## Citation -If you use this model, please cite the following paper: -``` -@misc{rybak2020klej, - title={KLEJ: Comprehensive Benchmark for Polish Language Understanding}, - author={Piotr Rybak and Robert Mroczkowski and Janusz Tracz and Ireneusz Gawlik}, - year={2020}, - eprint={2005.00630}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` -Paper is accepted at ACL 2020, as soon as proceedings appear, we will update the BibTeX. - -## Authors -Model was trained by **Allegro Machine Learning Research** team. - -You can contact us at: klejbenchmark@allegro.pl diff --git a/model_cards/allegro/herbert-large-cased/README.md b/model_cards/allegro/herbert-large-cased/README.md deleted file mode 100644 index 583586f747..0000000000 --- a/model_cards/allegro/herbert-large-cased/README.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -language: pl -tags: -- herbert -license: cc-by-sa-4.0 ---- -# HerBERT -**[HerBERT](https://en.wikipedia.org/wiki/Zbigniew_Herbert)** is a BERT-based Language Model trained on Polish Corpora -using MLM and SSO objectives with dynamic masking of whole words. -Model training and experiments were conducted with [transformers](https://github.com/huggingface/transformers) in version 2.9. - -## Tokenizer -The training dataset was tokenized into subwords using ``CharBPETokenizer`` a character level byte-pair encoding with -a vocabulary size of 50k tokens. The tokenizer itself was trained with a [tokenizers](https://github.com/huggingface/tokenizers) library. -We kindly encourage you to use the **Fast** version of tokenizer, namely ``HerbertTokenizerFast``. - -## HerBERT usage - - -Example code: -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("allegro/herbert-large-cased") -model = AutoModel.from_pretrained("allegro/herbert-large-cased") - -output = model( - **tokenizer.batch_encode_plus( - [ - ( - "A potem szedł środkiem drogi w kurzawie, bo zamiatał nogami, ślepy dziad prowadzony przez tłustego kundla na sznurku.", - "A potem leciał od lasu chłopak z butelką, ale ten ujrzawszy księdza przy drodze okrążył go z dala i biegł na przełaj pól do karczmy." - ) - ], - padding='longest', - add_special_tokens=True, - return_tensors='pt' - ) -) -``` - - -## License -CC BY-SA 4.0 - - -## Authors -Model was trained by **Allegro Machine Learning Research** team. - -You can contact us at: klejbenchmark@allegro.pl diff --git a/model_cards/allenai/biomed_roberta_base/README.md b/model_cards/allenai/biomed_roberta_base/README.md deleted file mode 100644 index 66c0371d11..0000000000 --- a/model_cards/allenai/biomed_roberta_base/README.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -thumbnail: https://huggingface.co/front/thumbnails/allenai.png ---- - -# BioMed-RoBERTa-base - -BioMed-RoBERTa-base is a language model based on the RoBERTa-base (Liu et. al, 2019) architecture. We adapt RoBERTa-base to 2.68 million scientific papers from the [Semantic Scholar](https://www.semanticscholar.org) corpus via continued pretraining. This amounts to 7.55B tokens and 47GB of data. We use the full text of the papers in training, not just abstracts. - -Specific details of the adaptive pretraining procedure can be found in Gururangan et. al, 2020. - - -## Evaluation - -BioMed-RoBERTa achieves competitive performance to state of the art models on a number of NLP tasks in the biomedical domain (numbers are mean (standard deviation) over 3+ random seeds) - - -| Task | Task Type | RoBERTa-base | BioMed-RoBERTa-base | -|--------------|---------------------|--------------|---------------------| -| RCT-180K | Text Classification | 86.4 (0.3) | 86.9 (0.2) | -| ChemProt | Relation Extraction | 81.1 (1.1) | 83.0 (0.7) | -| JNLPBA | NER | 74.3 (0.2) | 75.2 (0.1) | -| BC5CDR | NER | 85.6 (0.1) | 87.8 (0.1) | -| NCBI-Disease | NER | 86.6 (0.3) | 87.1 (0.8) | - -More evaluations TBD. - -## Citation - -If using this model, please cite the following paper: - -```bibtex -@inproceedings{domains, - author = {Suchin Gururangan and Ana Marasović and Swabha Swayamdipta and Kyle Lo and Iz Beltagy and Doug Downey and Noah A. Smith}, - title = {Don't Stop Pretraining: Adapt Language Models to Domains and Tasks}, - year = {2020}, - booktitle = {Proceedings of ACL}, -} -``` diff --git a/model_cards/allenai/longformer-base-4096-extra.pos.embd.only/README.md b/model_cards/allenai/longformer-base-4096-extra.pos.embd.only/README.md deleted file mode 100644 index 881870e55d..0000000000 --- a/model_cards/allenai/longformer-base-4096-extra.pos.embd.only/README.md +++ /dev/null @@ -1,20 +0,0 @@ - -# longformer-base-4096-extra.pos.embd.only - -This model is similar to `longformer-base-4096` but it was pretrained to preserve RoBERTa weights by freezing all RoBERTa weights and only train the additional position embeddings. - - -### Citing - -If you use `Longformer` in your research, please cite [Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150). -``` -@article{Beltagy2020Longformer, - title={Longformer: The Long-Document Transformer}, - author={Iz Beltagy and Matthew E. Peters and Arman Cohan}, - journal={arXiv:2004.05150}, - year={2020}, -} -``` - -`Longformer` is an open-source project developed by [the Allen Institute for Artificial Intelligence (AI2)](http://www.allenai.org). -AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering. diff --git a/model_cards/allenai/longformer-base-4096/README.md b/model_cards/allenai/longformer-base-4096/README.md deleted file mode 100644 index 44132b1b28..0000000000 --- a/model_cards/allenai/longformer-base-4096/README.md +++ /dev/null @@ -1,24 +0,0 @@ - -# longformer-base-4096 -[Longformer](https://arxiv.org/abs/2004.05150) is a transformer model for long documents. - -`longformer-base-4096` is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096. - -Longformer uses a combination of a sliding window (local) attention and global attention. Global attention is user-configured based on the task to allow the model to learn task-specific representations. -Please refer to the examples in `modeling_longformer.py` and the paper for more details on how to set global attention. - - -### Citing - -If you use `Longformer` in your research, please cite [Longformer: The Long-Document Transformer](https://arxiv.org/abs/2004.05150). -``` -@article{Beltagy2020Longformer, - title={Longformer: The Long-Document Transformer}, - author={Iz Beltagy and Matthew E. Peters and Arman Cohan}, - journal={arXiv:2004.05150}, - year={2020}, -} -``` - -`Longformer` is an open-source project developed by [the Allen Institute for Artificial Intelligence (AI2)](http://www.allenai.org). -AI2 is a non-profit institute with the mission to contribute to humanity through high-impact AI research and engineering. diff --git a/model_cards/allenai/scibert_scivocab_cased/README.md b/model_cards/allenai/scibert_scivocab_cased/README.md deleted file mode 100644 index 8f9ff9b234..0000000000 --- a/model_cards/allenai/scibert_scivocab_cased/README.md +++ /dev/null @@ -1,26 +0,0 @@ -# SciBERT - -This is the pretrained model presented in [SciBERT: A Pretrained Language Model for Scientific Text](https://www.aclweb.org/anthology/D19-1371/), which is a BERT model trained on scientific text. - -The training corpus was papers taken from [Semantic Scholar](https://www.semanticscholar.org). Corpus size is 1.14M papers, 3.1B tokens. We use the full text of the papers in training, not just abstracts. - -SciBERT has its own wordpiece vocabulary (scivocab) that's built to best match the training corpus. We trained cased and uncased versions. - -Available models include: -* `scibert_scivocab_cased` -* `scibert_scivocab_uncased` - - -The original repo can be found [here](https://github.com/allenai/scibert). - -If using these models, please cite the following paper: -``` -@inproceedings{beltagy-etal-2019-scibert, - title = "SciBERT: A Pretrained Language Model for Scientific Text", - author = "Beltagy, Iz and Lo, Kyle and Cohan, Arman", - booktitle = "EMNLP", - year = "2019", - publisher = "Association for Computational Linguistics", - url = "https://www.aclweb.org/anthology/D19-1371" -} -``` diff --git a/model_cards/allenai/scibert_scivocab_uncased/README.md b/model_cards/allenai/scibert_scivocab_uncased/README.md deleted file mode 100644 index 8f9ff9b234..0000000000 --- a/model_cards/allenai/scibert_scivocab_uncased/README.md +++ /dev/null @@ -1,26 +0,0 @@ -# SciBERT - -This is the pretrained model presented in [SciBERT: A Pretrained Language Model for Scientific Text](https://www.aclweb.org/anthology/D19-1371/), which is a BERT model trained on scientific text. - -The training corpus was papers taken from [Semantic Scholar](https://www.semanticscholar.org). Corpus size is 1.14M papers, 3.1B tokens. We use the full text of the papers in training, not just abstracts. - -SciBERT has its own wordpiece vocabulary (scivocab) that's built to best match the training corpus. We trained cased and uncased versions. - -Available models include: -* `scibert_scivocab_cased` -* `scibert_scivocab_uncased` - - -The original repo can be found [here](https://github.com/allenai/scibert). - -If using these models, please cite the following paper: -``` -@inproceedings{beltagy-etal-2019-scibert, - title = "SciBERT: A Pretrained Language Model for Scientific Text", - author = "Beltagy, Iz and Lo, Kyle and Cohan, Arman", - booktitle = "EMNLP", - year = "2019", - publisher = "Association for Computational Linguistics", - url = "https://www.aclweb.org/anthology/D19-1371" -} -``` diff --git a/model_cards/allenai/wmt16-en-de-12-1/README.md b/model_cards/allenai/wmt16-en-de-12-1/README.md deleted file mode 100644 index 4f896ea9d2..0000000000 --- a/model_cards/allenai/wmt16-en-de-12-1/README.md +++ /dev/null @@ -1,103 +0,0 @@ - ---- -language: -- en -- de -thumbnail: -tags: -- translation -- wmt16 -- allenai -license: apache-2.0 -datasets: -- wmt16 -metrics: -- bleu ---- - -# FSMT - -## Model description - -This is a ported version of fairseq-based [wmt16 transformer](https://github.com/jungokasai/deep-shallow/) for en-de. - -For more details, please, see [Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation](https://arxiv.org/abs/2006.10369). - -All 3 models are available: - -* [wmt16-en-de-dist-12-1](https://huggingface.co/allenai/wmt16-en-de-dist-12-1) -* [wmt16-en-de-dist-6-1](https://huggingface.co/allenai/wmt16-en-de-dist-6-1) -* [wmt16-en-de-12-1](https://huggingface.co/allenai/wmt16-en-de-12-1) - - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "allenai/wmt16-en-de-12-1" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Machine learning is great, isn't it?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Maschinelles Lernen ist großartig, nicht wahr? - -``` - -#### Limitations and bias - - -## Training data - -Pretrained weights were left identical to the original model released by allenai. For more details, please, see the [paper](https://arxiv.org/abs/2006.10369). - -## Eval results - -Here are the BLEU scores: - -model | fairseq | transformers --------|---------|---------- -wmt16-en-de-12-1 | 26.9 | 25.75 - -The score is slightly below the score reported in the paper, as the researchers don't use `sacrebleu` and measure the score on tokenized outputs. `transformers` score was measured using `sacrebleu` on detokenized outputs. - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=en-de -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=5 -mkdir -p $DATA_DIR -sacrebleu -t wmt16 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt16 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py allenai/wmt16-en-de-12-1 $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt16/) -- [test set](http://matrix.statmt.org/test_sets/newstest2016.tgz?1504722372) - - -### BibTeX entry and citation info - -``` -@misc{kasai2020deep, - title={Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation}, - author={Jungo Kasai and Nikolaos Pappas and Hao Peng and James Cross and Noah A. Smith}, - year={2020}, - eprint={2006.10369}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - diff --git a/model_cards/allenai/wmt16-en-de-dist-12-1/README.md b/model_cards/allenai/wmt16-en-de-dist-12-1/README.md deleted file mode 100644 index 16c7900387..0000000000 --- a/model_cards/allenai/wmt16-en-de-dist-12-1/README.md +++ /dev/null @@ -1,103 +0,0 @@ - ---- -language: -- en -- de -thumbnail: -tags: -- translation -- wmt16 -- allenai -license: apache-2.0 -datasets: -- wmt16 -metrics: -- bleu ---- - -# FSMT - -## Model description - -This is a ported version of fairseq-based [wmt16 transformer](https://github.com/jungokasai/deep-shallow/) for en-de. - -For more details, please, see [Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation](https://arxiv.org/abs/2006.10369). - -All 3 models are available: - -* [wmt16-en-de-dist-12-1](https://huggingface.co/allenai/wmt16-en-de-dist-12-1) -* [wmt16-en-de-dist-6-1](https://huggingface.co/allenai/wmt16-en-de-dist-6-1) -* [wmt16-en-de-12-1](https://huggingface.co/allenai/wmt16-en-de-12-1) - - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "allenai/wmt16-en-de-dist-12-1" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Machine learning is great, isn't it?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Maschinelles Lernen ist großartig, nicht wahr? - -``` - -#### Limitations and bias - - -## Training data - -Pretrained weights were left identical to the original model released by allenai. For more details, please, see the [paper](https://arxiv.org/abs/2006.10369). - -## Eval results - -Here are the BLEU scores: - -model | fairseq | transformers --------|---------|---------- -wmt16-en-de-dist-12-1 | 28.3 | 27.52 - -The score is slightly below the score reported in the paper, as the researchers don't use `sacrebleu` and measure the score on tokenized outputs. `transformers` score was measured using `sacrebleu` on detokenized outputs. - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=en-de -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=5 -mkdir -p $DATA_DIR -sacrebleu -t wmt16 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt16 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py allenai/wmt16-en-de-dist-12-1 $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt16/) -- [test set](http://matrix.statmt.org/test_sets/newstest2016.tgz?1504722372) - - -### BibTeX entry and citation info - -``` -@misc{kasai2020deep, - title={Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation}, - author={Jungo Kasai and Nikolaos Pappas and Hao Peng and James Cross and Noah A. Smith}, - year={2020}, - eprint={2006.10369}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - diff --git a/model_cards/allenai/wmt16-en-de-dist-6-1/README.md b/model_cards/allenai/wmt16-en-de-dist-6-1/README.md deleted file mode 100644 index 426231dd81..0000000000 --- a/model_cards/allenai/wmt16-en-de-dist-6-1/README.md +++ /dev/null @@ -1,103 +0,0 @@ - ---- -language: -- en -- de -thumbnail: -tags: -- translation -- wmt16 -- allenai -license: apache-2.0 -datasets: -- wmt16 -metrics: -- bleu ---- - -# FSMT - -## Model description - -This is a ported version of fairseq-based [wmt16 transformer](https://github.com/jungokasai/deep-shallow/) for en-de. - -For more details, please, see [Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation](https://arxiv.org/abs/2006.10369). - -All 3 models are available: - -* [wmt16-en-de-dist-12-1](https://huggingface.co/allenai/wmt16-en-de-dist-12-1) -* [wmt16-en-de-dist-6-1](https://huggingface.co/allenai/wmt16-en-de-dist-6-1) -* [wmt16-en-de-12-1](https://huggingface.co/allenai/wmt16-en-de-12-1) - - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "allenai/wmt16-en-de-dist-6-1" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Machine learning is great, isn't it?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Maschinelles Lernen ist großartig, nicht wahr? - -``` - -#### Limitations and bias - - -## Training data - -Pretrained weights were left identical to the original model released by allenai. For more details, please, see the [paper](https://arxiv.org/abs/2006.10369). - -## Eval results - -Here are the BLEU scores: - -model | fairseq | transformers --------|---------|---------- -wmt16-en-de-dist-6-1 | 27.4 | 27.11 - -The score is slightly below the score reported in the paper, as the researchers don't use `sacrebleu` and measure the score on tokenized outputs. `transformers` score was measured using `sacrebleu` on detokenized outputs. - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=en-de -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=5 -mkdir -p $DATA_DIR -sacrebleu -t wmt16 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt16 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py allenai/wmt16-en-de-dist-6-1 $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt16/) -- [test set](http://matrix.statmt.org/test_sets/newstest2016.tgz?1504722372) - - -### BibTeX entry and citation info - -``` -@misc{kasai2020deep, - title={Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation}, - author={Jungo Kasai and Nikolaos Pappas and Hao Peng and James Cross and Noah A. Smith}, - year={2020}, - eprint={2006.10369}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - diff --git a/model_cards/allenai/wmt19-de-en-6-6-base/README.md b/model_cards/allenai/wmt19-de-en-6-6-base/README.md deleted file mode 100644 index e5339bbc8a..0000000000 --- a/model_cards/allenai/wmt19-de-en-6-6-base/README.md +++ /dev/null @@ -1,101 +0,0 @@ - ---- - -language: -- de -- en -thumbnail: -tags: -- translation -- wmt19 -- allenai -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu ---- - -# FSMT - -## Model description - -This is a ported version of fairseq-based [wmt19 transformer](https://github.com/jungokasai/deep-shallow/) for de-en. - -For more details, please, see [Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation](https://arxiv.org/abs/2006.10369). - -2 models are available: - -* [wmt19-de-en-6-6-big](https://huggingface.co/allenai/wmt19-de-en-6-6-big) -* [wmt19-de-en-6-6-base](https://huggingface.co/allenai/wmt19-de-en-6-6-base) - - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "allenai/wmt19-de-en-6-6-base" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Maschinelles Lernen ist großartig, nicht wahr?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Machine learning is great, isn't it? - -``` - -#### Limitations and bias - - -## Training data - -Pretrained weights were left identical to the original model released by allenai. For more details, please, see the [paper](https://arxiv.org/abs/2006.10369). - -## Eval results - -Here are the BLEU scores: - -model | transformers --------|--------- -wmt19-de-en-6-6-base | 38.37 - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=de-en -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=5 -mkdir -p $DATA_DIR -sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py allenai/wmt19-de-en-6-6-base $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt19/) -- [test set](http://matrix.statmt.org/test_sets/newstest2019.tgz?1556572561) - - -### BibTeX entry and citation info - -``` -@misc{kasai2020deep, - title={Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation}, - author={Jungo Kasai and Nikolaos Pappas and Hao Peng and James Cross and Noah A. Smith}, - year={2020}, - eprint={2006.10369}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - diff --git a/model_cards/allenai/wmt19-de-en-6-6-big/README.md b/model_cards/allenai/wmt19-de-en-6-6-big/README.md deleted file mode 100644 index f348a772d0..0000000000 --- a/model_cards/allenai/wmt19-de-en-6-6-big/README.md +++ /dev/null @@ -1,101 +0,0 @@ - ---- - -language: -- de -- en -thumbnail: -tags: -- translation -- wmt19 -- allenai -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu ---- - -# FSMT - -## Model description - -This is a ported version of fairseq-based [wmt19 transformer](https://github.com/jungokasai/deep-shallow/) for de-en. - -For more details, please, see [Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation](https://arxiv.org/abs/2006.10369). - -2 models are available: - -* [wmt19-de-en-6-6-big](https://huggingface.co/allenai/wmt19-de-en-6-6-big) -* [wmt19-de-en-6-6-base](https://huggingface.co/allenai/wmt19-de-en-6-6-base) - - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "allenai/wmt19-de-en-6-6-big" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Maschinelles Lernen ist großartig, nicht wahr?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Machine learning is great, isn't it? - -``` - -#### Limitations and bias - - -## Training data - -Pretrained weights were left identical to the original model released by allenai. For more details, please, see the [paper](https://arxiv.org/abs/2006.10369). - -## Eval results - -Here are the BLEU scores: - -model | transformers --------|--------- -wmt19-de-en-6-6-big | 39.9 - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=de-en -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=5 -mkdir -p $DATA_DIR -sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py allenai/wmt19-de-en-6-6-big $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt19/) -- [test set](http://matrix.statmt.org/test_sets/newstest2019.tgz?1556572561) - - -### BibTeX entry and citation info - -``` -@misc{kasai2020deep, - title={Deep Encoder, Shallow Decoder: Reevaluating the Speed-Quality Tradeoff in Machine Translation}, - author={Jungo Kasai and Nikolaos Pappas and Hao Peng and James Cross and Noah A. Smith}, - year={2020}, - eprint={2006.10369}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - diff --git a/model_cards/allenyummy/chinese-bert-wwm-ehr-ner-sl/README.md b/model_cards/allenyummy/chinese-bert-wwm-ehr-ner-sl/README.md deleted file mode 100644 index e6df9a3128..0000000000 --- a/model_cards/allenyummy/chinese-bert-wwm-ehr-ner-sl/README.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -language: zh-tw ---- - -# Model name -Chinese-bert-wwm-electrical-health-record-ner-sequence-labeling - - -#### How to use - -``` -from transformers import AutoTokenizer, AutoModelForTokenClassification -tokenizer = AutoTokenizer.from_pretrained("chinese-bert-wwm-ehr-ner-sl") -model = AutoModelForTokenClassification.from_pretrained("chinese-bert-wwm-ehr-ner-sl") -``` diff --git a/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md b/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md deleted file mode 100644 index 6539f3edcd..0000000000 --- a/model_cards/amberoad/bert-multilingual-passage-reranking-msmarco/README.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -language: multilingual -thumbnail: "https://amberoad.de/images/logo_text.png" -tags: -- msmarco -- multilingual -- passage reranking -license: Apache-2.0 -datasets: -- msmarco -metrics: -- MRR -widget: -- query: "What is a corporation?" - passage: "A company is incorporated in a specific nation, often within the bounds of a smaller subset of that nation, such as a state or province. The corporation is then governed by the laws of incorporation in that state. A corporation may issue stock, either private or public, or may be classified as a non-stock corporation. If stock is issued, the corporation will usually be governed by its shareholders, either directly or indirectly." ---- - -# Passage Reranking Multilingual BERT 🔃 🌍 - - - -## Model description -**Input:** Supports over 100 Languages. See [List of supported languages](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages) for all available. - -**Purpose:** This module takes a search query [1] and a passage [2] and calculates if the passage matches the query. -It can be used as an improvement for Elasticsearch Results and boosts the relevancy by up to 100%. - -**Architecture:** On top of BERT there is a Densly Connected NN which takes the 768 Dimensional [CLS] Token as input and provides the output ([Arxiv](https://arxiv.org/abs/1901.04085)). - -**Output:** Just a single value between between -10 and 10. Better matching query,passage pairs tend to have a higher a score. - - - -## Intended uses & limitations -Both query[1] and passage[2] have to fit in 512 Tokens. -As you normally want to rerank the first dozens of search results keep in mind the inference time of approximately 300 ms/query. - -#### How to use - -```python -from transformers import AutoTokenizer, AutoModelForSequenceClassification - -tokenizer = AutoTokenizer.from_pretrained("amberoad/bert-multilingual-passage-reranking-msmarco") - -model = AutoModelForSequenceClassification.from_pretrained("amberoad/bert-multilingual-passage-reranking-msmarco") -``` - -This Model can be used as a drop-in replacement in the [Nboost Library](https://github.com/koursaros-ai/nboost) -Through this you can directly improve your Elasticsearch Results without any coding. - - -## Training data - -This model is trained using the [**Microsoft MS Marco Dataset**](https://microsoft.github.io/msmarco/ "Microsoft MS Marco"). This training dataset contains approximately 400M tuples of a query, relevant and non-relevant passages. All datasets used for training and evaluating are listed in this [table](https://github.com/microsoft/MSMARCO-Passage-Ranking#data-information-and-formating). The used dataset for training is called *Train Triples Large*, while the evaluation was made on *Top 1000 Dev*. There are 6,900 queries in total in the development dataset, where each query is mapped to top 1,000 passage retrieved using BM25 from MS MARCO corpus. - -## Training procedure - -The training is performed the same way as stated in this [README](https://github.com/nyu-dl/dl4marco-bert "NYU Github"). See their excellent Paper on [Arxiv](https://arxiv.org/abs/1901.04085). - -We changed the BERT Model from an English only to the default BERT Multilingual uncased Model from [Google](https://huggingface.co/bert-base-multilingual-uncased). - -Training was done 400 000 Steps. This equaled 12 hours an a TPU V3-8. - - -## Eval results - -We see nearly similar performance than the English only Model in the English [Bing Queries Dataset](http://www.msmarco.org/). Although the training data is English only internal Tests on private data showed a far higher accurancy in German than all other available models. - - - -Fine-tuned Models | Dependency | Eval Set | Search Boost | Speed on GPU ------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------- | ---------------------------------- -**`amberoad/Multilingual-uncased-MSMARCO`** (This Model) | PyTorch | bing queries | **+61%** (0.29 vs 0.18) | ~300 ms/query -`nboost/pt-tinybert-msmarco` | PyTorch | bing queries | **+45%** (0.26 vs 0.18) | ~50ms/query -`nboost/pt-bert-base-uncased-msmarco` | PyTorch | bing queries | **+62%** (0.29 vs 0.18) | ~300 ms/query -`nboost/pt-bert-large-msmarco` | PyTorch | bing queries | **+77%** (0.32 vs 0.18) | - -`nboost/pt-biobert-base-msmarco` | PyTorch | biomed | **+66%** (0.17 vs 0.10) | ~300 ms/query - -This table is taken from [nboost](https://github.com/koursaros-ai/nboost) and extended by the first line. - - - -## Contact Infos - -![](https://amberoad.de/images/logo_text.png) - -Amberoad is a company focussing on Search and Business Intelligence. -We provide you: -* Advanced Internal Company Search Engines thorugh NLP -* External Search Egnines: Find Competitors, Customers, Suppliers - -**Get in Contact now to benefit from our Expertise:** - -The training and evaluation was performed by [**Philipp Reissel**](https://reissel.eu/) and [**Igli Manaj**](https://github.com/iglimanaj) - - [![Amberoad](https://i.stack.imgur.com/gVE0j.png) Linkedin](https://de.linkedin.com/company/amberoad) | [Homepage](https://de.linkedin.com/company/amberoad) | [Email](info@amberoad.de) - - - - diff --git a/model_cards/amine/bert-base-5lang-cased/README.md b/model_cards/amine/bert-base-5lang-cased/README.md deleted file mode 100644 index 9117ca372c..0000000000 --- a/model_cards/amine/bert-base-5lang-cased/README.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -language: -- en -- fr -- es -- de -- zh - -tags: -- pytorch -- bert -- multilingual -- en -- fr -- es -- de -- zh - -datasets: wikipedia - -license: apache-2.0 - -inference: false ---- - -# bert-base-5lang-cased -This is a smaller version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) that handles only 5 languages (en, fr, es, de and zh) instead of 104. -The model is therefore 30% smaller than the original one (124M parameters instead of 178M) but gives exactly the same representations for the above cited languages. -Starting from `bert-base-5lang-cased` will facilitate the deployment of your model on public cloud platforms while keeping similar results. -For instance, Google Cloud Platform requires that the model size on disk should be lower than 500 MB for serveless deployments (Cloud Functions / Cloud ML) which is not the case of the original `bert-base-multilingual-cased`. - -For more information about the models size, memory footprint and loading time please refer to the table below: - -| Model | Num parameters | Size | Memory | Loading time | -| ---------------------------- | -------------- | -------- | -------- | ------------ | -| bert-base-multilingual-cased | 178 million | 714 MB | 1400 MB | 4.2 sec | -| bert-base-5lang-cased | 124 million | 495 MB | 950 MB | 3.6 sec | - -These measurements have been computed on a [Google Cloud n1-standard-1 machine (1 vCPU, 3.75 GB)](https://cloud.google.com/compute/docs/machine-types\#n1_machine_type). - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("amine/bert-base-5lang-cased") -model = AutoModel.from_pretrained("amine/bert-base-5lang-cased") - -``` - -### How to cite - -```bibtex -@inproceedings{smallermbert, - title={Load What You Need: Smaller Versions of Mutlilingual BERT}, - author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire}, - booktitle={SustaiNLP / EMNLP}, - year={2020} -} -``` - -## Contact - -Please contact amine@geotrend.fr for any question, feedback or request. \ No newline at end of file diff --git a/model_cards/antoiloui/belgpt2/README.md b/model_cards/antoiloui/belgpt2/README.md deleted file mode 100644 index 1a23fa0828..0000000000 --- a/model_cards/antoiloui/belgpt2/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: "fr" ---- - -# BelGPT-2 - -**BelGPT-2** (*Belgian GPT-2* 🇧🇪) is a "small" GPT-2 model pre-trained on a very large and heterogeneous French corpus (around 60Gb). Please check [antoiloui/gpt2-french](https://github.com/antoiloui/gpt2-french) for more information about the pre-trained model, the data, the code to use the model and the code to pre-train it. - - -## Using BelGPT-2 for Text Generation in French - -You can use BelGPT-2 with [🤗 transformers](https://github.com/huggingface/transformers) library as follows: - -```python -import torch -from transformers import GPT2Tokenizer, GPT2LMHeadModel - -# Load pretrained model and tokenizer -model = GPT2LMHeadModel.from_pretrained("antoiloui/belgpt2") -tokenizer = GPT2Tokenizer.from_pretrained("antoiloui/belgpt2") - -# Generate a sample of text -model.eval() -output = model.generate( - bos_token_id=random.randint(1,50000), - do_sample=True, - top_k=50, - max_length=100, - top_p=0.95, - num_return_sequences=1 -) - -# Decode it -decoded_output = [] -for sample in output: - decoded_output.append(tokenizer.decode(sample, skip_special_tokens=True)) -print(decoded_output) -``` - -## Data - -Below is the list of all French copora used to pre-trained the model: - -| Dataset | `$corpus_name` | Raw size | Cleaned size | -| :------| :--- | :---: | :---: | -| CommonCrawl | `common_crawl` | 200.2 GB | 40.4 GB | -| NewsCrawl | `news_crawl` | 10.4 GB | 9.8 GB | -| Wikipedia | `wiki` | 19.4 GB | 4.1 GB | -| Wikisource | `wikisource` | 4.6 GB | 2.3 GB | -| Project Gutenberg | `gutenberg` | 1.3 GB | 1.1 GB | -| EuroParl | `europarl` | 289.9 MB | 278.7 MB | -| NewsCommentary | `news_commentary` | 61.4 MB | 58.1 MB | -| **Total** | | **236.3 GB** | **57.9 GB** | diff --git a/model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616/README.md b/model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616/README.md deleted file mode 100644 index 0db4dea407..0000000000 --- a/model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616/README.md +++ /dev/null @@ -1,20 +0,0 @@ -# BERT L-10 H-512 fine-tuned on MLM (CORD-19 2020/06/16) - -BERT model with [10 Transformer layers and hidden embedding of size 512](https://huggingface.co/google/bert_uncased_L-10_H-512_A-8), referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962), fine-tuned for MLM on CORD-19 dataset (as released on 2020/06/16). - -## Training the model - -```bash -python run_language_modeling.py - --model_type bert - --model_name_or_path google/bert_uncased_L-10_H-512_A-8 - --do_train - --train_data_file {cord19-200616-dataset} - --mlm - --mlm_probability 0.2 - --line_by_line - --block_size 512 - --per_device_train_batch_size 10 - --learning_rate 3e-5 - --num_train_epochs 2 - --output_dir bert_uncased_L-10_H-512_A-8_cord19-200616 diff --git a/model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616_squad2/README.md b/model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616_squad2/README.md deleted file mode 100644 index 57b955182b..0000000000 --- a/model_cards/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616_squad2/README.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# BERT L-10 H-512 CORD-19 (2020/06/16) fine-tuned on SQuAD v2.0 - -BERT model with [10 Transformer layers and hidden embedding of size 512](https://huggingface.co/google/bert_uncased_L-10_H-512_A-8), referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962), [fine-tuned for MLM](https://huggingface.co/aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616) on CORD-19 dataset (as released on 2020/06/16) and fine-tuned for QA on SQuAD v2.0. - -## Training the model - -```bash -python run_squad.py - --model_type bert - --model_name_or_path aodiniz/bert_uncased_L-10_H-512_A-8_cord19-200616 - --train_file 'train-v2.0.json' - --predict_file 'dev-v2.0.json' - --do_train - --do_eval - --do_lower_case - --version_2_with_negative - --max_seq_length 384 - --per_gpu_train_batch_size 10 - --learning_rate 3e-5 - --num_train_epochs 2 - --output_dir bert_uncased_L-10_H-512_A-8_cord19-200616_squad2 diff --git a/model_cards/aodiniz/bert_uncased_L-2_H-512_A-8_cord19-200616/README.md b/model_cards/aodiniz/bert_uncased_L-2_H-512_A-8_cord19-200616/README.md deleted file mode 100644 index ec253db2b3..0000000000 --- a/model_cards/aodiniz/bert_uncased_L-2_H-512_A-8_cord19-200616/README.md +++ /dev/null @@ -1,20 +0,0 @@ -# BERT L-2 H-512 fine-tuned on MLM (CORD-19 2020/06/16) - -BERT model with [2 Transformer layers and hidden embedding of size 512](https://huggingface.co/google/bert_uncased_L-2_H-512_A-8), referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962), fine-tuned for MLM on CORD-19 dataset (as released on 2020/06/16). - -## Training the model - -```bash -python run_language_modeling.py - --model_type bert - --model_name_or_path google/bert_uncased_L-2_H-512_A-8 - --do_train - --train_data_file {cord19-200616-dataset} - --mlm - --mlm_probability 0.2 - --line_by_line - --block_size 512 - --per_device_train_batch_size 20 - --learning_rate 3e-5 - --num_train_epochs 2 - --output_dir bert_uncased_L-2_H-512_A-8_cord19-200616 diff --git a/model_cards/aodiniz/bert_uncased_L-4_H-256_A-4_cord19-200616/README.md b/model_cards/aodiniz/bert_uncased_L-4_H-256_A-4_cord19-200616/README.md deleted file mode 100644 index c29e5adfc3..0000000000 --- a/model_cards/aodiniz/bert_uncased_L-4_H-256_A-4_cord19-200616/README.md +++ /dev/null @@ -1,20 +0,0 @@ -# BERT L-4 H-256 fine-tuned on MLM (CORD-19 2020/06/16) - -BERT model with [4 Transformer layers and hidden embedding of size 256](https://huggingface.co/google/bert_uncased_L-4_H-256_A-4), referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962), fine-tuned for MLM on CORD-19 dataset (as released on 2020/06/16). - -## Training the model - -```bash -python run_language_modeling.py - --model_type bert - --model_name_or_path google/bert_uncased_L-4_H-256_A-4 - --do_train - --train_data_file {cord19-200616-dataset} - --mlm - --mlm_probability 0.2 - --line_by_line - --block_size 256 - --per_device_train_batch_size 20 - --learning_rate 3e-5 - --num_train_epochs 2 - --output_dir bert_uncased_L-4_H-256_A-4_cord19-200616 diff --git a/model_cards/asafaya/bert-base-arabic/README.md b/model_cards/asafaya/bert-base-arabic/README.md deleted file mode 100644 index 5b565e4686..0000000000 --- a/model_cards/asafaya/bert-base-arabic/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: ar -datasets: -- oscar -- wikipedia ---- - -# Arabic BERT Model - -Pretrained BERT base language model for Arabic - - -_If you use this model in your work, please cite this paper:_ - - - -``` -@misc{safaya2020kuisail, - title={KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media}, - author={Ali Safaya and Moutasem Abdullatif and Deniz Yuret}, - year={2020}, - eprint={2007.13184}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - -## Pretraining Corpus - -`arabic-bert-base` model was pretrained on ~8.2 Billion words: - -- Arabic version of [OSCAR](https://traces1.inria.fr/oscar/) - filtered from [Common Crawl](http://commoncrawl.org/) -- Recent dump of Arabic [Wikipedia](https://dumps.wikimedia.org/backup-index.html) - -and other Arabic resources which sum up to ~95GB of text. - -__Notes on training data:__ - -- Our final version of corpus contains some non-Arabic words inlines, which we did not remove from sentences since that would affect some tasks like NER. -- Although non-Arabic characters were lowered as a preprocessing step, since Arabic characters does not have upper or lower case, there is no cased and uncased version of the model. -- The corpus and vocabulary set are not restricted to Modern Standard Arabic, they contain some dialectical Arabic too. - -## Pretraining details - -- This model was trained using Google BERT's github [repository](https://github.com/google-research/bert) on a single TPU v3-8 provided for free from [TFRC](https://www.tensorflow.org/tfrc). -- Our pretraining procedure follows training settings of bert with some changes: trained for 3M training steps with batchsize of 128, instead of 1M with batchsize of 256. - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-base-arabic") -model = AutoModel.from_pretrained("asafaya/bert-base-arabic") -``` - -## Results - -For further details on the models performance or any other queries, please refer to [Arabic-BERT](https://github.com/alisafaya/Arabic-BERT) - -## Acknowledgement - -Thanks to Google for providing free TPU for the training process and for Huggingface for hosting this model on their servers 😊 - - diff --git a/model_cards/asafaya/bert-large-arabic/README.md b/model_cards/asafaya/bert-large-arabic/README.md deleted file mode 100644 index 4ecf5d2518..0000000000 --- a/model_cards/asafaya/bert-large-arabic/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: ar -datasets: -- oscar -- wikipedia ---- - - -# Arabic BERT Large Model - -Pretrained BERT Large language model for Arabic - -_If you use this model in your work, please cite this paper:_ - - - -``` -@misc{safaya2020kuisail, - title={KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media}, - author={Ali Safaya and Moutasem Abdullatif and Deniz Yuret}, - year={2020}, - eprint={2007.13184}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - -## Pretraining Corpus - -`arabic-bert-large` model was pretrained on ~8.2 Billion words: - -- Arabic version of [OSCAR](https://traces1.inria.fr/oscar/) - filtered from [Common Crawl](http://commoncrawl.org/) -- Recent dump of Arabic [Wikipedia](https://dumps.wikimedia.org/backup-index.html) - -and other Arabic resources which sum up to ~95GB of text. - -__Notes on training data:__ - -- Our final version of corpus contains some non-Arabic words inlines, which we did not remove from sentences since that would affect some tasks like NER. -- Although non-Arabic characters were lowered as a preprocessing step, since Arabic characters does not have upper or lower case, there is no cased and uncased version of the model. -- The corpus and vocabulary set are not restricted to Modern Standard Arabic, they contain some dialectical Arabic too. - -## Pretraining details - -- This model was trained using Google BERT's github [repository](https://github.com/google-research/bert) on a single TPU v3-8 provided for free from [TFRC](https://www.tensorflow.org/tfrc). -- Our pretraining procedure follows training settings of bert with some changes: trained for 3M training steps with batchsize of 128, instead of 1M with batchsize of 256. - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-large-arabic") -model = AutoModel.from_pretrained("asafaya/bert-large-arabic") -``` - -## Results - -For further details on the models performance or any other queries, please refer to [Arabic-BERT](https://github.com/alisafaya/Arabic-BERT) - -## Acknowledgement - -Thanks to Google for providing free TPU for the training process and for Huggingface for hosting this model on their servers 😊 - - diff --git a/model_cards/asafaya/bert-medium-arabic/README.md b/model_cards/asafaya/bert-medium-arabic/README.md deleted file mode 100644 index 502c1a5821..0000000000 --- a/model_cards/asafaya/bert-medium-arabic/README.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -language: ar -datasets: -- oscar -- wikipedia ---- - -# Arabic BERT Medium Model - -Pretrained BERT Medium language model for Arabic - -_If you use this model in your work, please cite this paper:_ - - - -``` -@misc{safaya2020kuisail, - title={KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media}, - author={Ali Safaya and Moutasem Abdullatif and Deniz Yuret}, - year={2020}, - eprint={2007.13184}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - -## Pretraining Corpus - -`arabic-bert-medium` model was pretrained on ~8.2 Billion words: - -- Arabic version of [OSCAR](https://traces1.inria.fr/oscar/) - filtered from [Common Crawl](http://commoncrawl.org/) -- Recent dump of Arabic [Wikipedia](https://dumps.wikimedia.org/backup-index.html) - -and other Arabic resources which sum up to ~95GB of text. - -__Notes on training data:__ - -- Our final version of corpus contains some non-Arabic words inlines, which we did not remove from sentences since that would affect some tasks like NER. -- Although non-Arabic characters were lowered as a preprocessing step, since Arabic characters does not have upper or lower case, there is no cased and uncased version of the model. -- The corpus and vocabulary set are not restricted to Modern Standard Arabic, they contain some dialectical Arabic too. - -## Pretraining details - -- This model was trained using Google BERT's github [repository](https://github.com/google-research/bert) on a single TPU v3-8 provided for free from [TFRC](https://www.tensorflow.org/tfrc). -- Our pretraining procedure follows training settings of bert with some changes: trained for 3M training steps with batchsize of 128, instead of 1M with batchsize of 256. - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-medium-arabic") -model = AutoModel.from_pretrained("asafaya/bert-medium-arabic") -``` - -## Results - -For further details on the models performance or any other queries, please refer to [Arabic-BERT](https://github.com/alisafaya/Arabic-BERT) - -## Acknowledgement - -Thanks to Google for providing free TPU for the training process and for Huggingface for hosting this model on their servers 😊 - - diff --git a/model_cards/asafaya/bert-mini-arabic/README.md b/model_cards/asafaya/bert-mini-arabic/README.md deleted file mode 100644 index d480df7b27..0000000000 --- a/model_cards/asafaya/bert-mini-arabic/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -language: ar -datasets: -- oscar -- wikipedia ---- - -# Arabic BERT Mini Model - -Pretrained BERT Mini language model for Arabic - -_If you use this model in your work, please cite this paper:_ - - - -``` -@misc{safaya2020kuisail, - title={KUISAIL at SemEval-2020 Task 12: BERT-CNN for Offensive Speech Identification in Social Media}, - author={Ali Safaya and Moutasem Abdullatif and Deniz Yuret}, - year={2020}, - eprint={2007.13184}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - -## Pretraining Corpus - -`arabic-bert-mini` model was pretrained on ~8.2 Billion words: - -- Arabic version of [OSCAR](https://traces1.inria.fr/oscar/) - filtered from [Common Crawl](http://commoncrawl.org/) -- Recent dump of Arabic [Wikipedia](https://dumps.wikimedia.org/backup-index.html) - -and other Arabic resources which sum up to ~95GB of text. - -__Notes on training data:__ - -- Our final version of corpus contains some non-Arabic words inlines, which we did not remove from sentences since that would affect some tasks like NER. -- Although non-Arabic characters were lowered as a preprocessing step, since Arabic characters does not have upper or lower case, there is no cased and uncased version of the model. -- The corpus and vocabulary set are not restricted to Modern Standard Arabic, they contain some dialectical Arabic too. - -## Pretraining details - -- This model was trained using Google BERT's github [repository](https://github.com/google-research/bert) on a single TPU v3-8 provided for free from [TFRC](https://www.tensorflow.org/tfrc). -- Our pretraining procedure follows training settings of bert with some changes: trained for 3M training steps with batchsize of 128, instead of 1M with batchsize of 256. - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-mini-arabic") -model = AutoModel.from_pretrained("asafaya/bert-mini-arabic") -``` - -## Results - -For further details on the models performance or any other queries, please refer to [Arabic-BERT](https://github.com/alisafaya/Arabic-BERT) - -## Acknowledgement - -Thanks to Google for providing free TPU for the training process and for Huggingface for hosting this model on their servers 😊 - diff --git a/model_cards/ashwani-tanwar/Gujarati-XLM-R-Base/README.md b/model_cards/ashwani-tanwar/Gujarati-XLM-R-Base/README.md deleted file mode 100644 index d7889540f7..0000000000 --- a/model_cards/ashwani-tanwar/Gujarati-XLM-R-Base/README.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -language: gu ---- - -# Gujarati-XLM-R-Base - - -This model is finetuned over [XLM-RoBERTa](https://huggingface.co/xlm-roberta-base) (XLM-R) using its base variant with the Gujarati language using the [OSCAR](https://oscar-corpus.com/) monolingual dataset. We used the same masked language modelling (MLM) objective which was used for pretraining the XLM-R. As it is built over the pretrained XLM-R, we leveraged *Transfer Learning* by exploiting the knowledge from its parent model. - -## Dataset -OSCAR corpus contains several diverse datasets for different languages. We followed the work of [CamemBERT](https://www.aclweb.org/anthology/2020.acl-main.645/) who reported better performance with this diverse dataset as compared to the other large homogenous datasets. - -## Preprocessing and Training Procedure -Please visit [this link](https://github.com/ashwanitanwar/nmt-transfer-learning-xlm-r#6-finetuning-xlm-r) for the detailed procedure. - -## Usage -- This model can be used for further finetuning for different NLP tasks using the Gujarati language. -- It can be used to generate contextualised word representations for the Gujarati words. -- It can be used for domain adaptation. -- It can be used to predict the missing words from the Gujarati sentences. - -## Demo - ### Using the model to predict missing words - ``` - from transformers import pipeline - unmasker = pipeline('fill-mask', model='ashwani-tanwar/Gujarati-XLM-R-Base') - pred_word = unmasker("અમદાવાદ એ ગુજરાતનું એક છે.") - print(pred_word) - ``` - ``` - [{'sequence': ' અમદાવાદ એ ગુજરાતનું એક શહેર છે.', 'score': 0.9463568329811096, 'token': 85227, 'token_str': '▁શહેર'}, - {'sequence': ' અમદાવાદ એ ગુજરાતનું એક ગામ છે.', 'score': 0.013311690650880337, 'token': 66346, 'token_str': '▁ગામ'}, - {'sequence': ' અમદાવાદ એ ગુજરાતનું એકનગર છે.', 'score': 0.012945962138473988, 'token': 69702, 'token_str': 'નગર'}, - {'sequence': ' અમદાવાદ એ ગુજરાતનું એક સ્થળ છે.', 'score': 0.0045941537246108055, 'token': 135436, 'token_str': '▁સ્થળ'}, - {'sequence': ' અમદાવાદ એ ગુજરાતનું એક મહત્વ છે.', 'score': 0.00402021361514926, 'token': 126763, 'token_str': '▁મહત્વ'}] - ``` - ### Using the model to generate contextualised word representations - ``` - from transformers import AutoTokenizer, AutoModel - tokenizer = AutoTokenizer.from_pretrained("ashwani-tanwar/Gujarati-XLM-R-Base") - model = AutoModel.from_pretrained("ashwani-tanwar/Gujarati-XLM-R-Base") - sentence = "અમદાવાદ એ ગુજરાતનું એક શહેર છે." - encoded_sentence = tokenizer(sentence, return_tensors='pt') - context_word_rep = model(**encoded_sentence) - ``` diff --git a/model_cards/aubmindlab/bert-base-arabert/README.md b/model_cards/aubmindlab/bert-base-arabert/README.md deleted file mode 100644 index 772676b6dc..0000000000 --- a/model_cards/aubmindlab/bert-base-arabert/README.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -language: ar ---- - -# AraBERT : Pre-training BERT for Arabic Language Understanding - - -**AraBERT** is an Arabic pretrained lanaguage model based on [Google's BERT architechture](https://github.com/google-research/bert). AraBERT uses the same BERT-Base config. More details are available in the [AraBERT PAPER](https://arxiv.org/abs/2003.00104v2) and in the [AraBERT Meetup](https://github.com/WissamAntoun/pydata_khobar_meetup) - -There are two version off the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the [Farasa Segmenter](http://alt.qcri.org/farasa/segmenter.html). - -The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words. The training corpora are a collection of publically available large scale raw arabic text ([Arabic Wikidumps](https://archive.org/details/arwiki-20190201), [The 1.5B words Arabic Corpus](https://www.semanticscholar.org/paper/1.5-billion-words-Arabic-Corpus-El-Khair/f3eeef4afb81223df96575adadf808fe7fe440b4), [The OSIAN Corpus](https://www.aclweb.org/anthology/W19-4619), Assafir news articles, and 4 other manually crawled news websites (Al-Akhbar, Annahar, AL-Ahram, AL-Wafd) from [the Wayback Machine](http://web.archive.org/)) - -We evalaute both AraBERT models on different downstream tasks and compare it to [mBERT]((https://github.com/google-research/bert/blob/master/multilingual.md)), and other state of the art models (*To the extent of our knowledge*). The Tasks were Sentiment Analysis on 6 different datasets ([HARD](https://github.com/elnagara/HARD-Arabic-Dataset), [ASTD-Balanced](https://www.aclweb.org/anthology/D15-1299), [ArsenTD-Lev](https://staff.aub.edu.lb/~we07/Publications/ArSentD-LEV_Sentiment_Corpus.pdf), [LABR](https://github.com/mohamedadaly/LABR), [ArSaS](http://lrec-conf.org/workshops/lrec2018/W30/pdf/22_W30.pdf)), Named Entity Recognition with the [ANERcorp](http://curtis.ml.cmu.edu/w/courses/index.php/ANERcorp), and Arabic Question Answering on [Arabic-SQuAD and ARCD](https://github.com/husseinmozannar/SOQAL) - -**Update 2 (21/5/2020) :** -Added support for the farasapy segmenter https://github.com/MagedSaeed/farasapy in the ``preprocess_arabert.py`` which is ~6x faster than the ``py4j.java_gateway``, consider setting ``use_farasapy=True`` when calling preprocess and pass it an instance of ``FarasaSegmenter(interactive=True)`` with interactive set to ``True`` for faster segmentation. - -**Update 1 (21/4/2020) :** -Fixed an issue with ARCD fine-tuning which drastically improved performance. Initially we didn't account for the change of the ```answer_start``` during preprocessing. -## Results (Acc.) -Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 ----|:---:|:---:|:---:|:---: -HARD |95.7 [ElJundi et.al.](https://www.aclweb.org/anthology/W19-4608/)|95.7|**96.2**|96.1 -ASTD |86.5 [ElJundi et.al.](https://www.aclweb.org/anthology/W19-4608/)| 80.1|92.2|**92.6** -ArsenTD-Lev|52.4 [ElJundi et.al.](https://www.aclweb.org/anthology/W19-4608/)|51|58.9|**59.4** -AJGT|93 [Dahou et.al.](https://dl.acm.org/doi/fullHtml/10.1145/3314941)| 83.6|93.1|**93.8** -LABR|**87.5** [Dahou et.al.](https://dl.acm.org/doi/fullHtml/10.1145/3314941)|83|85.9|86.7 -ANERcorp|81.7 (BiLSTM-CRF)|78.4|**84.2**|81.9 -ARCD|mBERT|EM:34.2 F1: 61.3|EM:51.14 F1:82.13|**EM:54.84 F1: 82.15** - -*If you tested AraBERT on a public dataset and you want to add your results to the table above, open a pull request or contact us. Also make sure to have your code available online so we can add it as a reference* - -## How to use - -You can easily use AraBERT since it is almost fully compatible with existing codebases (Use this repo instead of the official BERT one, the only difference is in the ```tokenization.py``` file where we modify the _is_punctuation function to make it compatible with the "+" symbol and the "[" and "]" characters) - -To use HuggingFace's Transformer repository you only need to provide a list of token that forces the model to not split them, also make sure that the text is pre-segmented: -**Not all libraries built on top of transformers support the `never_split` argument** -```python -from transformers import AutoTokenizer, AutoModel -from arabert.preprocess_arabert import never_split_tokens, preprocess -from farasa.segmenter import FarasaSegmenter - -arabert_tokenizer = AutoTokenizer.from_pretrained( - "aubmindlab/bert-base-arabert", - do_lower_case=False, - do_basic_tokenize=True, - never_split=never_split_tokens) -arabert_model = AutoModel.from_pretrained("aubmindlab/bert-base-arabert") - -#Preprocess the text to make it compatible with AraBERT using farasapy -farasa_segmenter = FarasaSegmenter(interactive=True) - -#or you can use a py4j JavaGateway to the farasa Segmneter .jar but it's slower -#(see update 2) -#from py4j.java_gateway import JavaGateway -#gateway = JavaGateway.launch_gateway(classpath='./PATH_TO_FARASA/FarasaSegmenterJar.jar') -#farasa = gateway.jvm.com.qcri.farasa.segmenter.Farasa() - -text = "ولن نبالغ إذا قلنا إن هاتف أو كمبيوتر المكتب في زمننا هذا ضروري" -text_preprocessed = preprocess( text, - do_farasa_tokenization = True, - farasa = farasa_segmenter, - use_farasapy = True) - ->>>text_preprocessed: "و+ لن نبالغ إذا قل +نا إن هاتف أو كمبيوتر ال+ مكتب في زمن +نا هذا ضروري" - -arabert_tokenizer.tokenize(text_preprocessed) - ->>> ['و+', 'لن', 'نبال', '##غ', 'إذا', 'قل', '+نا', 'إن', 'هاتف', 'أو', 'كمبيوتر', 'ال+', 'مكتب', 'في', 'زمن', '+نا', 'هذا', 'ضروري'] -``` - -**AraBERTv0.1 is compatible with all existing libraries, since it needs no pre-segmentation.** -```python -from transformers import AutoTokenizer, AutoModel - -arabert_tokenizer = AutoTokenizer.from_pretrained("aubmindlab/bert-base-arabertv01",do_lower_case=False) -arabert_model = AutoModel.from_pretrained("aubmindlab/bert-base-arabertv01") - -text = "ولن نبالغ إذا قلنا إن هاتف أو كمبيوتر المكتب في زمننا هذا ضروري" -arabert_tokenizer.tokenize(text) - ->>> ['ولن', 'ن', '##بالغ', 'إذا', 'قلنا', 'إن', 'هاتف', 'أو', 'كمبيوتر', 'المكتب', 'في', 'زمن', '##ن', '##ا', 'هذا', 'ضروري'] -``` - - -The ```araBERT_(Updated_Demo_TF).ipynb``` Notebook is a small demo using the AJGT dataset using TensorFlow (GPU and TPU compatible). - -**Coming Soon :** Fine-tunning demo using HuggingFace's Trainer API - -**AraBERT on ARCD** -During the preprocessing step the ```answer_start``` character position needs to be recalculated. You can use the file ```arcd_preprocessing.py``` as shown below to clean, preprocess the ARCD dataset before running ```run_squad.py```. More detailed Colab notebook is available in the [SOQAL repo](https://github.com/husseinmozannar/SOQAL). -```bash -python arcd_preprocessing.py \ - --input_file="/PATH_TO/arcd-test.json" \ - --output_file="arcd-test-pre.json" \ - --do_farasa_tokenization=True \ - --use_farasapy=True \ -``` -```bash -python SOQAL/bert/run_squad.py \ - --vocab_file="/PATH_TO_PRETRAINED_TF_CKPT/vocab.txt" \ - --bert_config_file="/PATH_TO_PRETRAINED_TF_CKPT/config.json" \ - --init_checkpoint="/PATH_TO_PRETRAINED_TF_CKPT/" \ - --do_train=True \ - --train_file=turk_combined_all_pre.json \ - --do_predict=True \ - --predict_file=arcd-test-pre.json \ - --train_batch_size=32 \ - --predict_batch_size=24 \ - --learning_rate=3e-5 \ - --num_train_epochs=4 \ - --max_seq_length=384 \ - --doc_stride=128 \ - --do_lower_case=False\ - --output_dir="/PATH_TO/OUTPUT_PATH"/ \ - --use_tpu=True \ - --tpu_name=$TPU_ADDRESS \ -``` -## Model Weights and Vocab Download -Models | AraBERTv0.1 | AraBERTv1 ----|:---:|:---: -TensorFlow|[Drive Link](https://drive.google.com/open?id=1-kVmTUZZ4DP2rzeHNjTPkY8OjnQCpomO) | [Drive Link](https://drive.google.com/open?id=1-d7-9ljKgDJP5mx73uBtio-TuUZCqZnt) -PyTorch| [Drive_Link](https://drive.google.com/open?id=1-_3te42mQCPD8SxwZ3l-VBL7yaJH-IOv)| [Drive_Link](https://drive.google.com/open?id=1-69s6Pxqbi63HOQ1M9wTcr-Ovc6PWLLo) - -**You can find the PyTorch models in HuggingFace's Transformer Library under the ```aubmindlab``` username** - -## If you used this model please cite us as: -``` -@inproceedings{antoun2020arabert, - title={AraBERT: Transformer-based Model for Arabic Language Understanding}, - author={Antoun, Wissam and Baly, Fady and Hajj, Hazem}, - booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020}, - pages={9} -} -``` -## Acknowledgments -Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn't have done it without this program, and to the [AUB MIND Lab](https://sites.aub.edu.lb/mindlab/) Members for the continous support. Also thanks to [Yakshof](https://www.yakshof.com/#/) and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT. - -## Contacts -**Wissam Antoun**: [Linkedin](https://www.linkedin.com/in/giulio-ravasio-3a81a9110/) | [Twitter](https://twitter.com/wissam_antoun) | [Github](https://github.com/WissamAntoun) | | - -**Fady Baly**: [Linkedin](https://www.linkedin.com/in/fadybaly/) | [Twitter](https://twitter.com/fadybaly) | [Github](https://github.com/fadybaly) | | diff --git a/model_cards/aubmindlab/bert-base-arabertv01/README.md b/model_cards/aubmindlab/bert-base-arabertv01/README.md deleted file mode 100644 index 772676b6dc..0000000000 --- a/model_cards/aubmindlab/bert-base-arabertv01/README.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -language: ar ---- - -# AraBERT : Pre-training BERT for Arabic Language Understanding - - -**AraBERT** is an Arabic pretrained lanaguage model based on [Google's BERT architechture](https://github.com/google-research/bert). AraBERT uses the same BERT-Base config. More details are available in the [AraBERT PAPER](https://arxiv.org/abs/2003.00104v2) and in the [AraBERT Meetup](https://github.com/WissamAntoun/pydata_khobar_meetup) - -There are two version off the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the [Farasa Segmenter](http://alt.qcri.org/farasa/segmenter.html). - -The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words. The training corpora are a collection of publically available large scale raw arabic text ([Arabic Wikidumps](https://archive.org/details/arwiki-20190201), [The 1.5B words Arabic Corpus](https://www.semanticscholar.org/paper/1.5-billion-words-Arabic-Corpus-El-Khair/f3eeef4afb81223df96575adadf808fe7fe440b4), [The OSIAN Corpus](https://www.aclweb.org/anthology/W19-4619), Assafir news articles, and 4 other manually crawled news websites (Al-Akhbar, Annahar, AL-Ahram, AL-Wafd) from [the Wayback Machine](http://web.archive.org/)) - -We evalaute both AraBERT models on different downstream tasks and compare it to [mBERT]((https://github.com/google-research/bert/blob/master/multilingual.md)), and other state of the art models (*To the extent of our knowledge*). The Tasks were Sentiment Analysis on 6 different datasets ([HARD](https://github.com/elnagara/HARD-Arabic-Dataset), [ASTD-Balanced](https://www.aclweb.org/anthology/D15-1299), [ArsenTD-Lev](https://staff.aub.edu.lb/~we07/Publications/ArSentD-LEV_Sentiment_Corpus.pdf), [LABR](https://github.com/mohamedadaly/LABR), [ArSaS](http://lrec-conf.org/workshops/lrec2018/W30/pdf/22_W30.pdf)), Named Entity Recognition with the [ANERcorp](http://curtis.ml.cmu.edu/w/courses/index.php/ANERcorp), and Arabic Question Answering on [Arabic-SQuAD and ARCD](https://github.com/husseinmozannar/SOQAL) - -**Update 2 (21/5/2020) :** -Added support for the farasapy segmenter https://github.com/MagedSaeed/farasapy in the ``preprocess_arabert.py`` which is ~6x faster than the ``py4j.java_gateway``, consider setting ``use_farasapy=True`` when calling preprocess and pass it an instance of ``FarasaSegmenter(interactive=True)`` with interactive set to ``True`` for faster segmentation. - -**Update 1 (21/4/2020) :** -Fixed an issue with ARCD fine-tuning which drastically improved performance. Initially we didn't account for the change of the ```answer_start``` during preprocessing. -## Results (Acc.) -Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 ----|:---:|:---:|:---:|:---: -HARD |95.7 [ElJundi et.al.](https://www.aclweb.org/anthology/W19-4608/)|95.7|**96.2**|96.1 -ASTD |86.5 [ElJundi et.al.](https://www.aclweb.org/anthology/W19-4608/)| 80.1|92.2|**92.6** -ArsenTD-Lev|52.4 [ElJundi et.al.](https://www.aclweb.org/anthology/W19-4608/)|51|58.9|**59.4** -AJGT|93 [Dahou et.al.](https://dl.acm.org/doi/fullHtml/10.1145/3314941)| 83.6|93.1|**93.8** -LABR|**87.5** [Dahou et.al.](https://dl.acm.org/doi/fullHtml/10.1145/3314941)|83|85.9|86.7 -ANERcorp|81.7 (BiLSTM-CRF)|78.4|**84.2**|81.9 -ARCD|mBERT|EM:34.2 F1: 61.3|EM:51.14 F1:82.13|**EM:54.84 F1: 82.15** - -*If you tested AraBERT on a public dataset and you want to add your results to the table above, open a pull request or contact us. Also make sure to have your code available online so we can add it as a reference* - -## How to use - -You can easily use AraBERT since it is almost fully compatible with existing codebases (Use this repo instead of the official BERT one, the only difference is in the ```tokenization.py``` file where we modify the _is_punctuation function to make it compatible with the "+" symbol and the "[" and "]" characters) - -To use HuggingFace's Transformer repository you only need to provide a list of token that forces the model to not split them, also make sure that the text is pre-segmented: -**Not all libraries built on top of transformers support the `never_split` argument** -```python -from transformers import AutoTokenizer, AutoModel -from arabert.preprocess_arabert import never_split_tokens, preprocess -from farasa.segmenter import FarasaSegmenter - -arabert_tokenizer = AutoTokenizer.from_pretrained( - "aubmindlab/bert-base-arabert", - do_lower_case=False, - do_basic_tokenize=True, - never_split=never_split_tokens) -arabert_model = AutoModel.from_pretrained("aubmindlab/bert-base-arabert") - -#Preprocess the text to make it compatible with AraBERT using farasapy -farasa_segmenter = FarasaSegmenter(interactive=True) - -#or you can use a py4j JavaGateway to the farasa Segmneter .jar but it's slower -#(see update 2) -#from py4j.java_gateway import JavaGateway -#gateway = JavaGateway.launch_gateway(classpath='./PATH_TO_FARASA/FarasaSegmenterJar.jar') -#farasa = gateway.jvm.com.qcri.farasa.segmenter.Farasa() - -text = "ولن نبالغ إذا قلنا إن هاتف أو كمبيوتر المكتب في زمننا هذا ضروري" -text_preprocessed = preprocess( text, - do_farasa_tokenization = True, - farasa = farasa_segmenter, - use_farasapy = True) - ->>>text_preprocessed: "و+ لن نبالغ إذا قل +نا إن هاتف أو كمبيوتر ال+ مكتب في زمن +نا هذا ضروري" - -arabert_tokenizer.tokenize(text_preprocessed) - ->>> ['و+', 'لن', 'نبال', '##غ', 'إذا', 'قل', '+نا', 'إن', 'هاتف', 'أو', 'كمبيوتر', 'ال+', 'مكتب', 'في', 'زمن', '+نا', 'هذا', 'ضروري'] -``` - -**AraBERTv0.1 is compatible with all existing libraries, since it needs no pre-segmentation.** -```python -from transformers import AutoTokenizer, AutoModel - -arabert_tokenizer = AutoTokenizer.from_pretrained("aubmindlab/bert-base-arabertv01",do_lower_case=False) -arabert_model = AutoModel.from_pretrained("aubmindlab/bert-base-arabertv01") - -text = "ولن نبالغ إذا قلنا إن هاتف أو كمبيوتر المكتب في زمننا هذا ضروري" -arabert_tokenizer.tokenize(text) - ->>> ['ولن', 'ن', '##بالغ', 'إذا', 'قلنا', 'إن', 'هاتف', 'أو', 'كمبيوتر', 'المكتب', 'في', 'زمن', '##ن', '##ا', 'هذا', 'ضروري'] -``` - - -The ```araBERT_(Updated_Demo_TF).ipynb``` Notebook is a small demo using the AJGT dataset using TensorFlow (GPU and TPU compatible). - -**Coming Soon :** Fine-tunning demo using HuggingFace's Trainer API - -**AraBERT on ARCD** -During the preprocessing step the ```answer_start``` character position needs to be recalculated. You can use the file ```arcd_preprocessing.py``` as shown below to clean, preprocess the ARCD dataset before running ```run_squad.py```. More detailed Colab notebook is available in the [SOQAL repo](https://github.com/husseinmozannar/SOQAL). -```bash -python arcd_preprocessing.py \ - --input_file="/PATH_TO/arcd-test.json" \ - --output_file="arcd-test-pre.json" \ - --do_farasa_tokenization=True \ - --use_farasapy=True \ -``` -```bash -python SOQAL/bert/run_squad.py \ - --vocab_file="/PATH_TO_PRETRAINED_TF_CKPT/vocab.txt" \ - --bert_config_file="/PATH_TO_PRETRAINED_TF_CKPT/config.json" \ - --init_checkpoint="/PATH_TO_PRETRAINED_TF_CKPT/" \ - --do_train=True \ - --train_file=turk_combined_all_pre.json \ - --do_predict=True \ - --predict_file=arcd-test-pre.json \ - --train_batch_size=32 \ - --predict_batch_size=24 \ - --learning_rate=3e-5 \ - --num_train_epochs=4 \ - --max_seq_length=384 \ - --doc_stride=128 \ - --do_lower_case=False\ - --output_dir="/PATH_TO/OUTPUT_PATH"/ \ - --use_tpu=True \ - --tpu_name=$TPU_ADDRESS \ -``` -## Model Weights and Vocab Download -Models | AraBERTv0.1 | AraBERTv1 ----|:---:|:---: -TensorFlow|[Drive Link](https://drive.google.com/open?id=1-kVmTUZZ4DP2rzeHNjTPkY8OjnQCpomO) | [Drive Link](https://drive.google.com/open?id=1-d7-9ljKgDJP5mx73uBtio-TuUZCqZnt) -PyTorch| [Drive_Link](https://drive.google.com/open?id=1-_3te42mQCPD8SxwZ3l-VBL7yaJH-IOv)| [Drive_Link](https://drive.google.com/open?id=1-69s6Pxqbi63HOQ1M9wTcr-Ovc6PWLLo) - -**You can find the PyTorch models in HuggingFace's Transformer Library under the ```aubmindlab``` username** - -## If you used this model please cite us as: -``` -@inproceedings{antoun2020arabert, - title={AraBERT: Transformer-based Model for Arabic Language Understanding}, - author={Antoun, Wissam and Baly, Fady and Hajj, Hazem}, - booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020}, - pages={9} -} -``` -## Acknowledgments -Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn't have done it without this program, and to the [AUB MIND Lab](https://sites.aub.edu.lb/mindlab/) Members for the continous support. Also thanks to [Yakshof](https://www.yakshof.com/#/) and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT. - -## Contacts -**Wissam Antoun**: [Linkedin](https://www.linkedin.com/in/giulio-ravasio-3a81a9110/) | [Twitter](https://twitter.com/wissam_antoun) | [Github](https://github.com/WissamAntoun) | | - -**Fady Baly**: [Linkedin](https://www.linkedin.com/in/fadybaly/) | [Twitter](https://twitter.com/fadybaly) | [Github](https://github.com/fadybaly) | | diff --git a/model_cards/bart-large-cnn/README.md b/model_cards/bart-large-cnn/README.md deleted file mode 100644 index 36061c0549..0000000000 --- a/model_cards/bart-large-cnn/README.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -tags: -- summarization ---- - diff --git a/model_cards/bart-large-xsum/README.md b/model_cards/bart-large-xsum/README.md deleted file mode 100644 index 36061c0549..0000000000 --- a/model_cards/bart-large-xsum/README.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -tags: -- summarization ---- - diff --git a/model_cards/bashar-talafha/multi-dialect-bert-base-arabic/README.md b/model_cards/bashar-talafha/multi-dialect-bert-base-arabic/README.md deleted file mode 100644 index 41e9aece3b..0000000000 --- a/model_cards/bashar-talafha/multi-dialect-bert-base-arabic/README.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -language: ar -thumbnail: https://raw.githubusercontent.com/mawdoo3/Multi-dialect-Arabic-BERT/master/multidialct_arabic_bert.png -datasets: -- nadi ---- -# Multi-dialect-Arabic-BERT -This is a repository of Multi-dialect Arabic BERT model. - -By [Mawdoo3-AI](https://ai.mawdoo3.com/). - -

-
- Background reference: http://www.qfi.org/wp-content/uploads/2018/02/Qfi_Infographic_Mother-Language_Final.pdf -
-

- - - -### About our Multi-dialect-Arabic-BERT model -Instead of training the Multi-dialect Arabic BERT model from scratch, we initialized the weights of the model using [Arabic-BERT](https://github.com/alisafaya/Arabic-BERT) and trained it on 10M arabic tweets from the unlabled data of [The Nuanced Arabic Dialect Identification (NADI) shared task](https://sites.google.com/view/nadi-shared-task). - -### To cite this work - -``` -@misc{talafha2020multidialect, - title={Multi-Dialect Arabic BERT for Country-Level Dialect Identification}, - author={Bashar Talafha and Mohammad Ali and Muhy Eddin Za'ter and Haitham Seelawi and Ibraheem Tuffaha and Mostafa Samir and Wael Farhan and Hussein T. Al-Natsheh}, - year={2020}, - eprint={2007.05612}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` - -### Usage -The model weights can be loaded using `transformers` library by HuggingFace. - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("bashar-talafha/multi-dialect-bert-base-arabic") -model = AutoModel.from_pretrained("bashar-talafha/multi-dialect-bert-base-arabic") -``` - -Example using `pipeline`: - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="bashar-talafha/multi-dialect-bert-base-arabic ", - tokenizer="bashar-talafha/multi-dialect-bert-base-arabic " -) - -fill_mask(" سافر الرحالة من مطار [MASK] ") -``` -``` -[{'sequence': '[CLS] سافر الرحالة من مطار الكويت [SEP]', 'score': 0.08296813815832138, 'token': 3226}, - {'sequence': '[CLS] سافر الرحالة من مطار دبي [SEP]', 'score': 0.05123933032155037, 'token': 4747}, - {'sequence': '[CLS] سافر الرحالة من مطار مسقط [SEP]', 'score': 0.046838656067848206, 'token': 13205}, - {'sequence': '[CLS] سافر الرحالة من مطار القاهرة [SEP]', 'score': 0.03234650194644928, 'token': 4003}, - {'sequence': '[CLS] سافر الرحالة من مطار الرياض [SEP]', 'score': 0.02606341242790222, 'token': 2200}] -``` -### Repository -Please check the [original repository](https://github.com/mawdoo3/Multi-dialect-Arabic-BERT) for more information. - - diff --git a/model_cards/bayartsogt/albert-mongolian/README.md b/model_cards/bayartsogt/albert-mongolian/README.md deleted file mode 100644 index 47fee353c5..0000000000 --- a/model_cards/bayartsogt/albert-mongolian/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -language: mn ---- - -# ALBERT-Mongolian -[pretraining repo link](https://github.com/bayartsogt-ya/albert-mongolian) -## Model description -Here we provide pretrained ALBERT model and trained SentencePiece model for Mongolia text. Training data is the Mongolian wikipedia corpus from Wikipedia Downloads and Mongolian News corpus. - -## Evaluation Result: -``` -loss = 1.7478163 -masked_lm_accuracy = 0.6838185 -masked_lm_loss = 1.6687671 -sentence_order_accuracy = 0.998125 -sentence_order_loss = 0.007942731 -``` - -## Fine-tuning Result on Eduge Dataset: -``` - precision recall f1-score support - - байгал орчин 0.83 0.76 0.80 483 - боловсрол 0.79 0.75 0.77 420 - спорт 0.98 0.96 0.97 1391 - технологи 0.85 0.83 0.84 543 - улс төр 0.88 0.87 0.87 1336 - урлаг соёл 0.89 0.94 0.91 726 - хууль 0.87 0.83 0.85 840 - эдийн засаг 0.80 0.84 0.82 1265 - эрүүл мэнд 0.84 0.90 0.87 562 - - accuracy 0.87 7566 - macro avg 0.86 0.85 0.86 7566 - weighted avg 0.87 0.87 0.87 7566 -``` - -## Reference -1. [ALBERT - official repo](https://github.com/google-research/albert) -2. [WikiExtrator](https://github.com/attardi/wikiextractor) -3. [Mongolian BERT](https://github.com/tugstugi/mongolian-bert) -4. [ALBERT - Japanese](https://github.com/alinear-corp/albert-japanese) -5. [Mongolian Text Classification](https://github.com/sharavsambuu/mongolian-text-classification) -6. [You's paper](https://arxiv.org/abs/1904.00962) - -## Citation -``` -@misc{albert-mongolian, - author = {Bayartsogt Yadamsuren}, - title = {ALBERT Pretrained Model on Mongolian Datasets}, - year = {2020}, - publisher = {GitHub}, - journal = {GitHub repository}, - howpublished = {\url{https://github.com/bayartsogt-ya/albert-mongolian/}} -} -``` - -## For More Information -Please contact by bayartsogtyadamsuren@icloud.com diff --git a/model_cards/bayartsogt/bert-base-mongolian-cased/README.md b/model_cards/bayartsogt/bert-base-mongolian-cased/README.md deleted file mode 100644 index 7cf8c1adaa..0000000000 --- a/model_cards/bayartsogt/bert-base-mongolian-cased/README.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -language: "mn" -tags: -- mongolian -- cased ---- - -# BERT-BASE-MONGOLIAN-CASED -[Link to Official Mongolian-BERT repo](https://github.com/tugstugi/mongolian-bert) - -## Model description -This repository contains pre-trained Mongolian [BERT](https://arxiv.org/abs/1810.04805) models trained by [tugstugi](https://github.com/tugstugi), [enod](https://github.com/enod) and [sharavsambuu](https://github.com/sharavsambuu). -Special thanks to [nabar](https://github.com/nabar) who provided 5x TPUs. - -This repository is based on the following open source projects: [google-research/bert](https://github.com/google-research/bert/), -[huggingface/pytorch-pretrained-BERT](https://github.com/huggingface/pytorch-pretrained-BERT) and [yoheikikuta/bert-japanese](https://github.com/yoheikikuta/bert-japanese). - -#### How to use - -```python -from transformers import pipeline, AlbertTokenizer, BertForMaskedLM - -tokenizer = AlbertTokenizer.from_pretrained('bayartsogt/bert-base-mongolian-cased') -model = BertForMaskedLM.from_pretrained('bayartsogt/bert-base-mongolian-cased') - -## declare task ## -pipe = pipeline(task="fill-mask", model=model, tokenizer=tokenizer) - -## example ## -input_ = 'Миний [MASK] хоол идэх нь тун чухал.' - -output_ = pipe(input_) -for i in range(len(output_)): - print(output_[i]) - -## Output ## -# {'sequence': '[CLS] Миний хувьд хоол идэх нь тун чухал.[SEP]', 'score': 0.8734784722328186, 'token': 95, 'token_str': '▁хувьд'} -# {'sequence': '[CLS] Миний бодлоор хоол идэх нь тун чухал.[SEP]', 'score': 0.09788835793733597, 'token': 6320, 'token_str': '▁бодлоор'} -# {'sequence': '[CLS] Миний хүү хоол идэх нь тун чухал.[SEP]', 'score': 0.0027510314248502254, 'token': 590, 'token_str': '▁хүү'} -# {'sequence': '[CLS] Миний бие хоол идэх нь тун чухал.[SEP]', 'score': 0.0014857524074614048, 'token': 267, 'token_str': '▁бие'} -# {'sequence': '[CLS] Миний охин хоол идэх нь тун чухал.[SEP]', 'score': 0.0013575413031503558, 'token': 1116, 'token_str': '▁охин'} - -``` - - -## Training data -Mongolian Wikipedia and the 700 million word Mongolian news data set [[Pretraining Procedure](https://github.com/tugstugi/mongolian-bert#pre-training)] - -### BibTeX entry and citation info - -```bibtex -@misc{mongolian-bert, - author = {Tuguldur, Erdene-Ochir and Gunchinish, Sharavsambuu and Bataa, Enkhbold}, - title = {BERT Pretrained Models on Mongolian Datasets}, - year = {2019}, - publisher = {GitHub}, - journal = {GitHub repository}, - howpublished = {\url{https://github.com/tugstugi/mongolian-bert/}} -} -``` diff --git a/model_cards/bayartsogt/bert-base-mongolian-uncased/README.md b/model_cards/bayartsogt/bert-base-mongolian-uncased/README.md deleted file mode 100644 index f673206e4d..0000000000 --- a/model_cards/bayartsogt/bert-base-mongolian-uncased/README.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -language: "mn" -tags: -- bert -- mongolian -- uncased ---- - -# BERT-BASE-MONGOLIAN-UNCASED -[Link to Official Mongolian-BERT repo](https://github.com/tugstugi/mongolian-bert) - -## Model description -This repository contains pre-trained Mongolian [BERT](https://arxiv.org/abs/1810.04805) models trained by [tugstugi](https://github.com/tugstugi), [enod](https://github.com/enod) and [sharavsambuu](https://github.com/sharavsambuu). -Special thanks to [nabar](https://github.com/nabar) who provided 5x TPUs. - -This repository is based on the following open source projects: [google-research/bert](https://github.com/google-research/bert/), -[huggingface/pytorch-pretrained-BERT](https://github.com/huggingface/pytorch-pretrained-BERT) and [yoheikikuta/bert-japanese](https://github.com/yoheikikuta/bert-japanese). - -#### How to use - -```python -from transformers import pipeline, AlbertTokenizer, BertForMaskedLM - -tokenizer = AlbertTokenizer.from_pretrained('bayartsogt/bert-base-mongolian-uncased') -model = BertForMaskedLM.from_pretrained('bayartsogt/bert-base-mongolian-uncased') - -## declare task ## -pipe = pipeline(task="fill-mask", model=model, tokenizer=tokenizer) - -## example ## -input_ = 'Миний [MASK] хоол идэх нь тун чухал.' - -output_ = pipe(input_) -for i in range(len(output_)): - print(output_[i]) - -``` - - -## Training data -Mongolian Wikipedia and the 700 million word Mongolian news data set [[Pretraining Procedure](https://github.com/tugstugi/mongolian-bert#pre-training)] - -### BibTeX entry and citation info - -```bibtex -@misc{mongolian-bert, - author = {Tuguldur, Erdene-Ochir and Gunchinish, Sharavsambuu and Bataa, Enkhbold}, - title = {BERT Pretrained Models on Mongolian Datasets}, - year = {2019}, - publisher = {GitHub}, - journal = {GitHub repository}, - howpublished = {\url{https://github.com/tugstugi/mongolian-bert/}} -} -``` diff --git a/model_cards/bert-base-cased-README.md b/model_cards/bert-base-cased-README.md deleted file mode 100644 index 82d50d4805..0000000000 --- a/model_cards/bert-base-cased-README.md +++ /dev/null @@ -1,230 +0,0 @@ ---- -language: en -tags: -- exbert -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia ---- - -# BERT base model (cased) - -Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://arxiv.org/abs/1810.04805) and first released in -[this repository](https://github.com/google-research/bert). This model is case-sensitive: it makes a difference between -english and English. - -Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by -the Hugging Face team. - -## Model description - -BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it -was pretrained with two objectives: - -- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run - the entire masked sentence through the model and has to predict the masked words. This is different from traditional - recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like - GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the - sentence. -- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes - they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to - predict if the two sentences were following each other or not. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-cased') ->>> unmasker("Hello I'm a [MASK] model.") - -[{'sequence': "[CLS] Hello I'm a fashion model. [SEP]", - 'score': 0.09019174426794052, - 'token': 4633, - 'token_str': 'fashion'}, - {'sequence': "[CLS] Hello I'm a new model. [SEP]", - 'score': 0.06349995732307434, - 'token': 1207, - 'token_str': 'new'}, - {'sequence': "[CLS] Hello I'm a male model. [SEP]", - 'score': 0.06228214129805565, - 'token': 2581, - 'token_str': 'male'}, - {'sequence': "[CLS] Hello I'm a professional model. [SEP]", - 'score': 0.0441727414727211, - 'token': 1848, - 'token_str': 'professional'}, - {'sequence': "[CLS] Hello I'm a super model. [SEP]", - 'score': 0.03326151892542839, - 'token': 7688, - 'token_str': 'super'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-cased') -model = TFBertModel.from_pretrained("bert-base-cased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-cased') -model = BertModel.from_pretrained("bert-base-cased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -Even if the training data used for this model could be characterized as fairly neutral, this model can have biased -predictions: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-cased') ->>> unmasker("The man worked as a [MASK].") - -[{'sequence': '[CLS] The man worked as a lawyer. [SEP]', - 'score': 0.04804691672325134, - 'token': 4545, - 'token_str': 'lawyer'}, - {'sequence': '[CLS] The man worked as a waiter. [SEP]', - 'score': 0.037494491785764694, - 'token': 17989, - 'token_str': 'waiter'}, - {'sequence': '[CLS] The man worked as a cop. [SEP]', - 'score': 0.035512614995241165, - 'token': 9947, - 'token_str': 'cop'}, - {'sequence': '[CLS] The man worked as a detective. [SEP]', - 'score': 0.031271643936634064, - 'token': 9140, - 'token_str': 'detective'}, - {'sequence': '[CLS] The man worked as a doctor. [SEP]', - 'score': 0.027423162013292313, - 'token': 3995, - 'token_str': 'doctor'}] - ->>> unmasker("The woman worked as a [MASK].") - -[{'sequence': '[CLS] The woman worked as a nurse. [SEP]', - 'score': 0.16927455365657806, - 'token': 7439, - 'token_str': 'nurse'}, - {'sequence': '[CLS] The woman worked as a waitress. [SEP]', - 'score': 0.1501094549894333, - 'token': 15098, - 'token_str': 'waitress'}, - {'sequence': '[CLS] The woman worked as a maid. [SEP]', - 'score': 0.05600163713097572, - 'token': 13487, - 'token_str': 'maid'}, - {'sequence': '[CLS] The woman worked as a housekeeper. [SEP]', - 'score': 0.04838843643665314, - 'token': 26458, - 'token_str': 'housekeeper'}, - {'sequence': '[CLS] The woman worked as a cook. [SEP]', - 'score': 0.029980547726154327, - 'token': 9834, - 'token_str': 'cook'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -The BERT model was pretrained on [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 -unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and -headers). - -## Training procedure - -### Preprocessing - -The texts are tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are then of the form: - -``` -[CLS] Sentence A [SEP] Sentence B [SEP] -``` - -With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in -the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a -consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two -"sentences" has a combined length of less than 512 tokens. - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by `[MASK]`. -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - -### Pretraining - -The model was trained on 4 cloud TPUs in Pod configuration (16 TPU chips total) for one million steps with a batch size -of 256. The sequence length was limited to 128 tokens for 90% of the steps and 512 for the remaining 10%. The optimizer -used is Adam with a learning rate of 1e-4, \\(\beta_{1} = 0.9\\) and \\(\beta_{2} = 0.999\\), a weight decay of 0.01, -learning rate warmup for 10,000 steps and linear decay of the learning rate after. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI-(m/mm) | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | Average | -|:----:|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|:-------:| -| | 84.6/83.4 | 71.2 | 90.5 | 93.5 | 52.1 | 85.8 | 88.9 | 66.4 | 79.6 | - - -### BibTeX entry and citation info - -```bibtex -@article{DBLP:journals/corr/abs-1810-04805, - author = {Jacob Devlin and - Ming{-}Wei Chang and - Kenton Lee and - Kristina Toutanova}, - title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language - Understanding}, - journal = {CoRR}, - volume = {abs/1810.04805}, - year = {2018}, - url = {http://arxiv.org/abs/1810.04805}, - archivePrefix = {arXiv}, - eprint = {1810.04805}, - timestamp = {Tue, 30 Oct 2018 20:39:56 +0100}, - biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` - - - - diff --git a/model_cards/bert-base-chinese-README.md b/model_cards/bert-base-chinese-README.md deleted file mode 100644 index 78633644e3..0000000000 --- a/model_cards/bert-base-chinese-README.md +++ /dev/null @@ -1,3 +0,0 @@ ---- -language: zh ---- diff --git a/model_cards/bert-base-german-cased-README.md b/model_cards/bert-base-german-cased-README.md deleted file mode 100644 index eda81a6c1e..0000000000 --- a/model_cards/bert-base-german-cased-README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: de -license: mit -thumbnail: https://static.tildacdn.com/tild6438-3730-4164-b266-613634323466/german_bert.png -tags: -- exbert ---- - - - - - -# German BERT -![bert_image](https://static.tildacdn.com/tild6438-3730-4164-b266-613634323466/german_bert.png) -## Overview -**Language model:** bert-base-cased -**Language:** German -**Training data:** Wiki, OpenLegalData, News (~ 12GB) -**Eval data:** Conll03 (NER), GermEval14 (NER), GermEval18 (Classification), GNAD (Classification) -**Infrastructure**: 1x TPU v2 -**Published**: Jun 14th, 2019 - -**Update April 3rd, 2020**: we updated the vocabulary file on deepset's s3 to conform with the default tokenization of punctuation tokens. -For details see the related [FARM issue](https://github.com/deepset-ai/FARM/issues/60). If you want to use the old vocab we have also uploaded a ["deepset/bert-base-german-cased-oldvocab"](https://huggingface.co/deepset/bert-base-german-cased-oldvocab) model. - -## Details -- We trained using Google's Tensorflow code on a single cloud TPU v2 with standard settings. -- We trained 810k steps with a batch size of 1024 for sequence length 128 and 30k steps with sequence length 512. Training took about 9 days. -- As training data we used the latest German Wikipedia dump (6GB of raw txt files), the OpenLegalData dump (2.4 GB) and news articles (3.6 GB). -- We cleaned the data dumps with tailored scripts and segmented sentences with spacy v2.1. To create tensorflow records we used the recommended sentencepiece library for creating the word piece vocabulary and tensorflow scripts to convert the text to data usable by BERT. - - -See https://deepset.ai/german-bert for more details - -## Hyperparameters - -``` -batch_size = 1024 -n_steps = 810_000 -max_seq_len = 128 (and 512 later) -learning_rate = 1e-4 -lr_schedule = LinearWarmup -num_warmup_steps = 10_000 -``` - -## Performance - -During training we monitored the loss and evaluated different model checkpoints on the following German datasets: - -- germEval18Fine: Macro f1 score for multiclass sentiment classification -- germEval18coarse: Macro f1 score for binary sentiment classification -- germEval14: Seq f1 score for NER (file names deuutf.\*) -- CONLL03: Seq f1 score for NER -- 10kGNAD: Accuracy for document classification - -Even without thorough hyperparameter tuning, we observed quite stable learning especially for our German model. Multiple restarts with different seeds produced quite similar results. - -![performancetable](https://thumb.tildacdn.com/tild3162-6462-4566-b663-376630376138/-/format/webp/Screenshot_from_2020.png) - -We further evaluated different points during the 9 days of pre-training and were astonished how fast the model converges to the maximally reachable performance. We ran all 5 downstream tasks on 7 different model checkpoints - taken at 0 up to 840k training steps (x-axis in figure below). Most checkpoints are taken from early training where we expected most performance changes. Surprisingly, even a randomly initialized BERT can be trained only on labeled downstream datasets and reach good performance (blue line, GermEval 2018 Coarse task, 795 kB trainset size). - -![checkpointseval](https://thumb.tildacdn.com/tild6335-3531-4137-b533-313365663435/-/format/webp/deepset_checkpoints.png) - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/bert-base-german-dbmdz-cased-README.md b/model_cards/bert-base-german-dbmdz-cased-README.md deleted file mode 100644 index 87bed79724..0000000000 --- a/model_cards/bert-base-german-dbmdz-cased-README.md +++ /dev/null @@ -1,4 +0,0 @@ ---- -language: de -license: mit ---- diff --git a/model_cards/bert-base-german-dbmdz-uncased-README.md b/model_cards/bert-base-german-dbmdz-uncased-README.md deleted file mode 100644 index 87bed79724..0000000000 --- a/model_cards/bert-base-german-dbmdz-uncased-README.md +++ /dev/null @@ -1,4 +0,0 @@ ---- -language: de -license: mit ---- diff --git a/model_cards/bert-base-multilingual-cased-README.md b/model_cards/bert-base-multilingual-cased-README.md deleted file mode 100644 index 8a6ed9bef3..0000000000 --- a/model_cards/bert-base-multilingual-cased-README.md +++ /dev/null @@ -1,152 +0,0 @@ ---- -language: multilingual -license: apache-2.0 -datasets: -- wikipedia ---- - -# BERT multilingual base model (cased) - -Pretrained model on the top 104 languages with the largest Wikipedia using a masked language modeling (MLM) objective. -It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in -[this repository](https://github.com/google-research/bert). This model is case sensitive: it makes a difference -between english and English. - -Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by -the Hugging Face team. - -## Model description - -BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means -it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it -was pretrained with two objectives: - -- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run - the entire masked sentence through the model and has to predict the masked words. This is different from traditional - recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like - GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the - sentence. -- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes - they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to - predict if the two sentences were following each other or not. - -This way, the model learns an inner representation of the languages in the training set that can then be used to -extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a -standard classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-multilingual-cased') ->>> unmasker("Hello I'm a [MASK] model.") - -[{'sequence': "[CLS] Hello I'm a model model. [SEP]", - 'score': 0.10182085633277893, - 'token': 13192, - 'token_str': 'model'}, - {'sequence': "[CLS] Hello I'm a world model. [SEP]", - 'score': 0.052126359194517136, - 'token': 11356, - 'token_str': 'world'}, - {'sequence': "[CLS] Hello I'm a data model. [SEP]", - 'score': 0.048930276185274124, - 'token': 11165, - 'token_str': 'data'}, - {'sequence': "[CLS] Hello I'm a flight model. [SEP]", - 'score': 0.02036019042134285, - 'token': 23578, - 'token_str': 'flight'}, - {'sequence': "[CLS] Hello I'm a business model. [SEP]", - 'score': 0.020079681649804115, - 'token': 14155, - 'token_str': 'business'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased') -model = BertModel.from_pretrained("bert-base-multilingual-cased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-cased') -model = TFBertModel.from_pretrained("bert-base-multilingual-cased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on the 104 languages with the largest Wikipedias. You can find the complete list -[here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages). - -## Training procedure - -### Preprocessing - -The texts are lowercased and tokenized using WordPiece and a shared vocabulary size of 110,000. The languages with a -larger Wikipedia are under-sampled and the ones with lower resources are oversampled. For languages like Chinese, -Japanese Kanji and Korean Hanja that don't have space, a CJK Unicode block is added around every character. - -The inputs of the model are then of the form: - -``` -[CLS] Sentence A [SEP] Sentence B [SEP] -``` - -With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in -the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a -consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two -"sentences" has a combined length of less than 512 tokens. - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by `[MASK]`. -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - - -### BibTeX entry and citation info - -```bibtex -@article{DBLP:journals/corr/abs-1810-04805, - author = {Jacob Devlin and - Ming{-}Wei Chang and - Kenton Lee and - Kristina Toutanova}, - title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language - Understanding}, - journal = {CoRR}, - volume = {abs/1810.04805}, - year = {2018}, - url = {http://arxiv.org/abs/1810.04805}, - archivePrefix = {arXiv}, - eprint = {1810.04805}, - timestamp = {Tue, 30 Oct 2018 20:39:56 +0100}, - biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` diff --git a/model_cards/bert-base-multilingual-uncased-README.md b/model_cards/bert-base-multilingual-uncased-README.md deleted file mode 100644 index 862acd3ac4..0000000000 --- a/model_cards/bert-base-multilingual-uncased-README.md +++ /dev/null @@ -1,209 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- wikipedia ---- - -# BERT multilingual base model (uncased) - -Pretrained model on the top 102 languages with the largest Wikipedia using a masked language modeling (MLM) objective. -It was introduced in [this paper](https://arxiv.org/abs/1810.04805) and first released in -[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by -the Hugging Face team. - -## Model description - -BERT is a transformers model pretrained on a large corpus of multilingual data in a self-supervised fashion. This means -it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it -was pretrained with two objectives: - -- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run - the entire masked sentence through the model and has to predict the masked words. This is different from traditional - recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like - GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the - sentence. -- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes - they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to - predict if the two sentences were following each other or not. - -This way, the model learns an inner representation of the languages in the training set that can then be used to -extract features useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a -standard classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-multilingual-uncased') ->>> unmasker("Hello I'm a [MASK] model.") - -[{'sequence': "[CLS] hello i'm a top model. [SEP]", - 'score': 0.1507750153541565, - 'token': 11397, - 'token_str': 'top'}, - {'sequence': "[CLS] hello i'm a fashion model. [SEP]", - 'score': 0.13075384497642517, - 'token': 23589, - 'token_str': 'fashion'}, - {'sequence': "[CLS] hello i'm a good model. [SEP]", - 'score': 0.036272723227739334, - 'token': 12050, - 'token_str': 'good'}, - {'sequence': "[CLS] hello i'm a new model. [SEP]", - 'score': 0.035954564809799194, - 'token': 10246, - 'token_str': 'new'}, - {'sequence': "[CLS] hello i'm a great model. [SEP]", - 'score': 0.028643041849136353, - 'token': 11838, - 'token_str': 'great'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-uncased') -model = BertModel.from_pretrained("bert-base-multilingual-uncased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-multilingual-uncased') -model = TFBertModel.from_pretrained("bert-base-multilingual-uncased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -Even if the training data used for this model could be characterized as fairly neutral, this model can have biased -predictions: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-multilingual-uncased') ->>> unmasker("The man worked as a [MASK].") - -[{'sequence': '[CLS] the man worked as a teacher. [SEP]', - 'score': 0.07943806052207947, - 'token': 21733, - 'token_str': 'teacher'}, - {'sequence': '[CLS] the man worked as a lawyer. [SEP]', - 'score': 0.0629938617348671, - 'token': 34249, - 'token_str': 'lawyer'}, - {'sequence': '[CLS] the man worked as a farmer. [SEP]', - 'score': 0.03367974981665611, - 'token': 36799, - 'token_str': 'farmer'}, - {'sequence': '[CLS] the man worked as a journalist. [SEP]', - 'score': 0.03172805905342102, - 'token': 19477, - 'token_str': 'journalist'}, - {'sequence': '[CLS] the man worked as a carpenter. [SEP]', - 'score': 0.031021825969219208, - 'token': 33241, - 'token_str': 'carpenter'}] - ->>> unmasker("The Black woman worked as a [MASK].") - -[{'sequence': '[CLS] the black woman worked as a nurse. [SEP]', - 'score': 0.07045423984527588, - 'token': 52428, - 'token_str': 'nurse'}, - {'sequence': '[CLS] the black woman worked as a teacher. [SEP]', - 'score': 0.05178029090166092, - 'token': 21733, - 'token_str': 'teacher'}, - {'sequence': '[CLS] the black woman worked as a lawyer. [SEP]', - 'score': 0.032601192593574524, - 'token': 34249, - 'token_str': 'lawyer'}, - {'sequence': '[CLS] the black woman worked as a slave. [SEP]', - 'score': 0.030507225543260574, - 'token': 31173, - 'token_str': 'slave'}, - {'sequence': '[CLS] the black woman worked as a woman. [SEP]', - 'score': 0.027691684663295746, - 'token': 14050, - 'token_str': 'woman'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -The BERT model was pretrained on the 102 languages with the largest Wikipedias. You can find the complete list -[here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages). - -## Training procedure - -### Preprocessing - -The texts are lowercased and tokenized using WordPiece and a shared vocabulary size of 110,000. The languages with a -larger Wikipedia are under-sampled and the ones with lower resources are oversampled. For languages like Chinese, -Japanese Kanji and Korean Hanja that don't have space, a CJK Unicode block is added around every character. - -The inputs of the model are then of the form: - -``` -[CLS] Sentence A [SEP] Sentence B [SEP] -``` - -With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in -the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a -consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two -"sentences" has a combined length of less than 512 tokens. - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by `[MASK]`. -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - - -### BibTeX entry and citation info - -```bibtex -@article{DBLP:journals/corr/abs-1810-04805, - author = {Jacob Devlin and - Ming{-}Wei Chang and - Kenton Lee and - Kristina Toutanova}, - title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language - Understanding}, - journal = {CoRR}, - volume = {abs/1810.04805}, - year = {2018}, - url = {http://arxiv.org/abs/1810.04805}, - archivePrefix = {arXiv}, - eprint = {1810.04805}, - timestamp = {Tue, 30 Oct 2018 20:39:56 +0100}, - biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` diff --git a/model_cards/bert-base-uncased-README.md b/model_cards/bert-base-uncased-README.md deleted file mode 100644 index 52f4ab676b..0000000000 --- a/model_cards/bert-base-uncased-README.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -language: en -tags: -- exbert -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia ---- - -# BERT base model (uncased) - -Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://arxiv.org/abs/1810.04805) and first released in -[this repository](https://github.com/google-research/bert). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing BERT did not write a model card for this model so this model card has been written by -the Hugging Face team. - -## Model description - -BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it -was pretrained with two objectives: - -- Masked language modeling (MLM): taking a sentence, the model randomly masks 15% of the words in the input then run - the entire masked sentence through the model and has to predict the masked words. This is different from traditional - recurrent neural networks (RNNs) that usually see the words one after the other, or from autoregressive models like - GPT which internally mask the future tokens. It allows the model to learn a bidirectional representation of the - sentence. -- Next sentence prediction (NSP): the models concatenates two masked sentences as inputs during pretraining. Sometimes - they correspond to sentences that were next to each other in the original text, sometimes not. The model then has to - predict if the two sentences were following each other or not. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=bert) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-uncased') ->>> unmasker("Hello I'm a [MASK] model.") - -[{'sequence': "[CLS] hello i'm a fashion model. [SEP]", - 'score': 0.1073106899857521, - 'token': 4827, - 'token_str': 'fashion'}, - {'sequence': "[CLS] hello i'm a role model. [SEP]", - 'score': 0.08774490654468536, - 'token': 2535, - 'token_str': 'role'}, - {'sequence': "[CLS] hello i'm a new model. [SEP]", - 'score': 0.05338378623127937, - 'token': 2047, - 'token_str': 'new'}, - {'sequence': "[CLS] hello i'm a super model. [SEP]", - 'score': 0.04667217284440994, - 'token': 3565, - 'token_str': 'super'}, - {'sequence': "[CLS] hello i'm a fine model. [SEP]", - 'score': 0.027095865458250046, - 'token': 2986, - 'token_str': 'fine'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-uncased') -model = BertModel.from_pretrained("bert-base-uncased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('bert-base-uncased') -model = TFBertModel.from_pretrained("bert-base-uncased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -Even if the training data used for this model could be characterized as fairly neutral, this model can have biased -predictions: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='bert-base-uncased') ->>> unmasker("The man worked as a [MASK].") - -[{'sequence': '[CLS] the man worked as a carpenter. [SEP]', - 'score': 0.09747550636529922, - 'token': 10533, - 'token_str': 'carpenter'}, - {'sequence': '[CLS] the man worked as a waiter. [SEP]', - 'score': 0.0523831807076931, - 'token': 15610, - 'token_str': 'waiter'}, - {'sequence': '[CLS] the man worked as a barber. [SEP]', - 'score': 0.04962705448269844, - 'token': 13362, - 'token_str': 'barber'}, - {'sequence': '[CLS] the man worked as a mechanic. [SEP]', - 'score': 0.03788609802722931, - 'token': 15893, - 'token_str': 'mechanic'}, - {'sequence': '[CLS] the man worked as a salesman. [SEP]', - 'score': 0.037680890411138535, - 'token': 18968, - 'token_str': 'salesman'}] - ->>> unmasker("The woman worked as a [MASK].") - -[{'sequence': '[CLS] the woman worked as a nurse. [SEP]', - 'score': 0.21981462836265564, - 'token': 6821, - 'token_str': 'nurse'}, - {'sequence': '[CLS] the woman worked as a waitress. [SEP]', - 'score': 0.1597415804862976, - 'token': 13877, - 'token_str': 'waitress'}, - {'sequence': '[CLS] the woman worked as a maid. [SEP]', - 'score': 0.1154729500412941, - 'token': 10850, - 'token_str': 'maid'}, - {'sequence': '[CLS] the woman worked as a prostitute. [SEP]', - 'score': 0.037968918681144714, - 'token': 19215, - 'token_str': 'prostitute'}, - {'sequence': '[CLS] the woman worked as a cook. [SEP]', - 'score': 0.03042375110089779, - 'token': 5660, - 'token_str': 'cook'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -The BERT model was pretrained on [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 -unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and -headers). - -## Training procedure - -### Preprocessing - -The texts are lowercased and tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are -then of the form: - -``` -[CLS] Sentence A [SEP] Sentence B [SEP] -``` - -With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in -the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a -consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two -"sentences" has a combined length of less than 512 tokens. - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by `[MASK]`. -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - -### Pretraining - -The model was trained on 4 cloud TPUs in Pod configuration (16 TPU chips total) for one million steps with a batch size -of 256. The sequence length was limited to 128 tokens for 90% of the steps and 512 for the remaining 10%. The optimizer -used is Adam with a learning rate of 1e-4, \\(\beta_{1} = 0.9\\) and \\(\beta_{2} = 0.999\\), a weight decay of 0.01, -learning rate warmup for 10,000 steps and linear decay of the learning rate after. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI-(m/mm) | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | Average | -|:----:|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|:-------:| -| | 84.6/83.4 | 71.2 | 90.5 | 93.5 | 52.1 | 85.8 | 88.9 | 66.4 | 79.6 | - - -### BibTeX entry and citation info - -```bibtex -@article{DBLP:journals/corr/abs-1810-04805, - author = {Jacob Devlin and - Ming{-}Wei Chang and - Kenton Lee and - Kristina Toutanova}, - title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language - Understanding}, - journal = {CoRR}, - volume = {abs/1810.04805}, - year = {2018}, - url = {http://arxiv.org/abs/1810.04805}, - archivePrefix = {arXiv}, - eprint = {1810.04805}, - timestamp = {Tue, 30 Oct 2018 20:39:56 +0100}, - biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` - - - - diff --git a/model_cards/bert-large-cased-README.md b/model_cards/bert-large-cased-README.md deleted file mode 100644 index 154df8298f..0000000000 --- a/model_cards/bert-large-cased-README.md +++ /dev/null @@ -1,3 +0,0 @@ ---- -license: apache-2.0 ---- diff --git a/model_cards/binwang/xlnet-base-cased/README.md b/model_cards/binwang/xlnet-base-cased/README.md deleted file mode 100644 index 482c0fc17e..0000000000 --- a/model_cards/binwang/xlnet-base-cased/README.md +++ /dev/null @@ -1,5 +0,0 @@ -This model is pre-trained **XLNET** with 12 layers. - -It comes with paper: SBERT-WK: A Sentence Embedding Method By Dissecting BERT-based Word Models - -Project Page: [SBERT-WK](https://github.com/BinWang28/SBERT-WK-Sentence-Embedding) diff --git a/model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-12_H-768_A-12/README.md b/model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-12_H-768_A-12/README.md deleted file mode 100644 index 64319e76e0..0000000000 --- a/model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-12_H-768_A-12/README.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -language: -- en -tags: -- bert -- bluebert -license: -- PUBLIC DOMAIN NOTICE -datasets: -- PubMed -- MIMIC-III - ---- - -# BlueBert-Base, Uncased, PubMed and MIMIC-III - -## Model description - -A BERT model pre-trained on PubMed abstracts and clinical notes ([MIMIC-III](https://mimic.physionet.org/)). - -## Intended uses & limitations - -#### How to use - -Please see https://github.com/ncbi-nlp/bluebert - -## Training data - -We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models. -The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/). - -Pre-trained model: https://huggingface.co/bert-base-uncased - -## Training procedure - -* lowercasing the text -* removing speical chars `\x00`-`\x7F` -* tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html) - -Below is a code snippet for more details. - -```python -value = value.lower() -value = re.sub(r'[\r\n]+', ' ', value) -value = re.sub(r'[^\x00-\x7F]+', ' ', value) - -tokenized = TreebankWordTokenizer().tokenize(value) -sentence = ' '.join(tokenized) -sentence = re.sub(r"\s's\b", "'s", sentence) -``` - -### BibTeX entry and citation info - -```bibtex -@InProceedings{peng2019transfer, - author = {Yifan Peng and Shankai Yan and Zhiyong Lu}, - title = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets}, - booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)}, - year = {2019}, - pages = {58--65}, -} -``` - -### Acknowledgments - -This work was supported by the Intramural Research Programs of the National Institutes of Health, National Library of -Medicine and Clinical Center. This work was supported by the National Library of Medicine of the National Institutes of Health under award number 4R00LM013001-01. - -We are also grateful to the authors of BERT and ELMo to make the data and codes publicly available. - -We would like to thank Dr Sun Kim for processing the PubMed texts. - -### Disclaimer - -This tool shows the results of research conducted in the Computational Biology Branch, NCBI. The information produced -on this website is not intended for direct diagnostic use or medical decision-making without review and oversight -by a clinical professional. Individuals should not change their health behavior solely on the basis of information -produced on this website. NIH does not independently verify the validity or utility of the information produced -by this tool. If you have questions about the information produced on this website, please see a health care -professional. More information about NCBI's disclaimer policy is available. diff --git a/model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-24_H-1024_A-16/README.md b/model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-24_H-1024_A-16/README.md deleted file mode 100644 index fd206ae973..0000000000 --- a/model_cards/bionlp/bluebert_pubmed_mimic_uncased_L-24_H-1024_A-16/README.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -language: -- en -tags: -- bert -- bluebert -license: -- PUBLIC DOMAIN NOTICE -datasets: -- PubMed -- MIMIC-III - ---- - -# BlueBert-Base, Uncased, PubMed and MIMIC-III - -## Model description - -A BERT model pre-trained on PubMed abstracts and clinical notes ([MIMIC-III](https://mimic.physionet.org/)). - -## Intended uses & limitations - -#### How to use - -Please see https://github.com/ncbi-nlp/bluebert - -## Training data - -We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models. -The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/). - -Pre-trained model: https://huggingface.co/bert-large-uncased - -## Training procedure - -* lowercasing the text -* removing speical chars `\x00`-`\x7F` -* tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html) - -Below is a code snippet for more details. - -```python -value = value.lower() -value = re.sub(r'[\r\n]+', ' ', value) -value = re.sub(r'[^\x00-\x7F]+', ' ', value) - -tokenized = TreebankWordTokenizer().tokenize(value) -sentence = ' '.join(tokenized) -sentence = re.sub(r"\s's\b", "'s", sentence) -``` - -### BibTeX entry and citation info - -```bibtex -@InProceedings{peng2019transfer, - author = {Yifan Peng and Shankai Yan and Zhiyong Lu}, - title = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets}, - booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)}, - year = {2019}, - pages = {58--65}, -} -``` - -### Acknowledgments - -This work was supported by the Intramural Research Programs of the National Institutes of Health, National Library of -Medicine and Clinical Center. This work was supported by the National Library of Medicine of the National Institutes of Health under award number 4R00LM013001-01. - -We are also grateful to the authors of BERT and ELMo to make the data and codes publicly available. - -We would like to thank Dr Sun Kim for processing the PubMed texts. - -### Disclaimer - -This tool shows the results of research conducted in the Computational Biology Branch, NCBI. The information produced -on this website is not intended for direct diagnostic use or medical decision-making without review and oversight -by a clinical professional. Individuals should not change their health behavior solely on the basis of information -produced on this website. NIH does not independently verify the validity or utility of the information produced -by this tool. If you have questions about the information produced on this website, please see a health care -professional. More information about NCBI's disclaimer policy is available. diff --git a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md b/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md deleted file mode 100644 index 7701bb25f4..0000000000 --- a/model_cards/bionlp/bluebert_pubmed_uncased_L-12_H-768_A-12/README.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -language: -- en -tags: -- bluebert -license: -- PUBLIC DOMAIN NOTICE -datasets: -- pubmed - ---- - -# BlueBert-Base, Uncased, PubMed - -## Model description - -A BERT model pre-trained on PubMed abstracts - -## Intended uses & limitations - -#### How to use - -Please see https://github.com/ncbi-nlp/bluebert - -## Training data - -We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models. -The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/). - -Pre-trained model: https://huggingface.co/bert-base-uncased - -## Training procedure - -* lowercasing the text -* removing speical chars `\x00`-`\x7F` -* tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html) - -Below is a code snippet for more details. - -```python -value = value.lower() -value = re.sub(r'[\r\n]+', ' ', value) -value = re.sub(r'[^\x00-\x7F]+', ' ', value) - -tokenized = TreebankWordTokenizer().tokenize(value) -sentence = ' '.join(tokenized) -sentence = re.sub(r"\s's\b", "'s", sentence) -``` - -### BibTeX entry and citation info - -```bibtex -@InProceedings{peng2019transfer, - author = {Yifan Peng and Shankai Yan and Zhiyong Lu}, - title = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets}, - booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)}, - year = {2019}, - pages = {58--65}, -} -``` diff --git a/model_cards/bionlp/bluebert_pubmed_uncased_L-24_H-1024_A-16/README.md b/model_cards/bionlp/bluebert_pubmed_uncased_L-24_H-1024_A-16/README.md deleted file mode 100644 index 96618f7fb5..0000000000 --- a/model_cards/bionlp/bluebert_pubmed_uncased_L-24_H-1024_A-16/README.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -language: -- en -tags: -- bert -- bluebert -license: -- PUBLIC DOMAIN NOTICE -datasets: -- PubMed - ---- - -# BlueBert-Base, Uncased, PubMed - -## Model description - -A BERT model pre-trained on PubMed abstracts. - -## Intended uses & limitations - -#### How to use - -Please see https://github.com/ncbi-nlp/bluebert - -## Training data - -We provide [preprocessed PubMed texts](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/pubmed_uncased_sentence_nltk.txt.tar.gz) that were used to pre-train the BlueBERT models. -The corpus contains ~4000M words extracted from the [PubMed ASCII code version](https://www.ncbi.nlm.nih.gov/research/bionlp/APIs/BioC-PubMed/). - -Pre-trained model: https://huggingface.co/bert-large-uncased - -## Training procedure - -* lowercasing the text -* removing speical chars `\x00`-`\x7F` -* tokenizing the text using the [NLTK Treebank tokenizer](https://www.nltk.org/_modules/nltk/tokenize/treebank.html) - -Below is a code snippet for more details. - -```python -value = value.lower() -value = re.sub(r'[\r\n]+', ' ', value) -value = re.sub(r'[^\x00-\x7F]+', ' ', value) - -tokenized = TreebankWordTokenizer().tokenize(value) -sentence = ' '.join(tokenized) -sentence = re.sub(r"\s's\b", "'s", sentence) -``` - -### BibTeX entry and citation info - -```bibtex -@InProceedings{peng2019transfer, - author = {Yifan Peng and Shankai Yan and Zhiyong Lu}, - title = {Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets}, - booktitle = {Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019)}, - year = {2019}, - pages = {58--65}, -} -``` - -### Acknowledgments - -This work was supported by the Intramural Research Programs of the National Institutes of Health, National Library of -Medicine and Clinical Center. This work was supported by the National Library of Medicine of the National Institutes of Health under award number 4R00LM013001-01. - -We are also grateful to the authors of BERT and ELMo to make the data and codes publicly available. - -We would like to thank Dr Sun Kim for processing the PubMed texts. - -### Disclaimer - -This tool shows the results of research conducted in the Computational Biology Branch, NCBI. The information produced -on this website is not intended for direct diagnostic use or medical decision-making without review and oversight -by a clinical professional. Individuals should not change their health behavior solely on the basis of information -produced on this website. NIH does not independently verify the validity or utility of the information produced -by this tool. If you have questions about the information produced on this website, please see a health care -professional. More information about NCBI's disclaimer policy is available. diff --git a/model_cards/blinoff/roberta-base-russian-v0/README.md b/model_cards/blinoff/roberta-base-russian-v0/README.md deleted file mode 100644 index 9ae6b45af2..0000000000 --- a/model_cards/blinoff/roberta-base-russian-v0/README.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -language: ru -widget: -- text: "Мозг — это машина , которая пытается снизить ошибку в прогнозе." ---- - -# RoBERTa-like language model trained on part of part of TAIGA corpus - -## Training Details - -- about 60k steps - -![]() - -## Example pipeline - -```python -from transformers import pipeline -from transformers import RobertaTokenizerFast - -tokenizer = RobertaTokenizerFast.from_pretrained('blinoff/roberta-base-russian-v0', max_len=512) - -fill_mask = pipeline( - "fill-mask", - model="blinoff/roberta-base-russian-v0", - tokenizer=tokenizer -) - -fill_mask("Мозг — это машина , которая пытается снизить ошибку в прогнозе.") - -# { -# 'sequence': 'Мозг — это машина города, которая пытается снизить ошибку в прогнозе.', -# 'score': 0.012859329581260681, -# 'token': 2144, -# 'token_str': 'ĠгоÑĢода' -# }, -# { -# 'sequence': 'Мозг — это машина человека, которая пытается снизить ошибку в прогнозе.', -# 'score': 0.01185101643204689, -# 'token': 1470, -# 'token_str': 'ĠÑĩеловека' -# }, -# { -# 'sequence': 'Мозг — это машина дома, которая пытается снизить ошибку в прогнозе.', -# 'score': 0.009940559044480324, -# 'token': 1411, -# 'token_str': 'Ġдома' -# }, -# { -# 'sequence': 'Мозг — это машина женщина, которая пытается снизить ошибку в прогнозе.', -# 'score': 0.007794599514454603, -# 'token': 2707, -# 'token_str': 'ĠженÑīина' -# }, -# { -# 'sequence': 'Мозг — это машина женщины, которая пытается снизить ошибку в прогнозе.', -# 'score': 0.007725382689386606, -# 'token': 3546, -# 'token_str': 'ĠженÑīинÑĭ' -# } -``` diff --git a/model_cards/cahya/bert-base-indonesian-522M/README.md b/model_cards/cahya/bert-base-indonesian-522M/README.md deleted file mode 100644 index 6008e770ad..0000000000 --- a/model_cards/cahya/bert-base-indonesian-522M/README.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -language: "id" -license: "mit" -datasets: -- Indonesian Wikipedia -widget: -- text: "Ibu ku sedang bekerja [MASK] supermarket." ---- - -# Indonesian BERT base model (uncased) - -## Model description -It is BERT-base model pre-trained with indonesian Wikipedia using a masked language modeling (MLM) objective. This -model is uncased: it does not make a difference between indonesia and Indonesia. - -This is one of several other language models that have been pre-trained with indonesian datasets. More detail about -its usage on downstream tasks (text classification, text generation, etc) is available at [Transformer based Indonesian Language Models](https://github.com/cahya-wirawan/indonesian-language-models/tree/master/Transformers) - -## Intended uses & limitations - -### How to use -You can use this model directly with a pipeline for masked language modeling: -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='cahya/bert-base-indonesian-522M') ->>> unmasker("Ibu ku sedang bekerja [MASK] supermarket") - -[{'sequence': '[CLS] ibu ku sedang bekerja di supermarket [SEP]', - 'score': 0.7983310222625732, - 'token': 1495}, - {'sequence': '[CLS] ibu ku sedang bekerja. supermarket [SEP]', - 'score': 0.090003103017807, - 'token': 17}, - {'sequence': '[CLS] ibu ku sedang bekerja sebagai supermarket [SEP]', - 'score': 0.025469014421105385, - 'token': 1600}, - {'sequence': '[CLS] ibu ku sedang bekerja dengan supermarket [SEP]', - 'score': 0.017966199666261673, - 'token': 1555}, - {'sequence': '[CLS] ibu ku sedang bekerja untuk supermarket [SEP]', - 'score': 0.016971781849861145, - 'token': 1572}] -``` -Here is how to use this model to get the features of a given text in PyTorch: -```python -from transformers import BertTokenizer, BertModel - -model_name='cahya/bert-base-indonesian-522M' -tokenizer = BertTokenizer.from_pretrained(model_name) -model = BertModel.from_pretrained(model_name) -text = "Silakan diganti dengan text apa saja." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` -and in Tensorflow: -```python -from transformers import BertTokenizer, TFBertModel - -model_name='cahya/bert-base-indonesian-522M' -tokenizer = BertTokenizer.from_pretrained(model_name) -model = TFBertModel.from_pretrained(model_name) -text = "Silakan diganti dengan text apa saja." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -This model was pre-trained with 522MB of indonesian Wikipedia. -The texts are lowercased and tokenized using WordPiece and a vocabulary size of 32,000. The inputs of the model are -then of the form: - -```[CLS] Sentence A [SEP] Sentence B [SEP]``` diff --git a/model_cards/cahya/gpt2-small-indonesian-522M/README.md b/model_cards/cahya/gpt2-small-indonesian-522M/README.md deleted file mode 100644 index cfd991839b..0000000000 --- a/model_cards/cahya/gpt2-small-indonesian-522M/README.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -language: "id" -license: "mit" -datasets: -- Indonesian Wikipedia -widget: -- text: "Pulau Dewata sering dikunjungi" ---- - -# Indonesian GPT2 small model - -## Model description -It is GPT2-small model pre-trained with indonesian Wikipedia using a causal language modeling (CLM) objective. This -model is uncased: it does not make a difference between indonesia and Indonesia. - -This is one of several other language models that have been pre-trained with indonesian datasets. More detail about -its usage on downstream tasks (text classification, text generation, etc) is available at [Transformer based Indonesian Language Models](https://github.com/cahya-wirawan/indonesian-language-models/tree/master/Transformers) - -## Intended uses & limitations - -### How to use -You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, -we set a seed for reproducibility: -```python ->>> from transformers import pipeline, set_seed ->>> generator = pipeline('text-generation', model='cahya/gpt2-small-indonesian-522M') ->>> set_seed(42) ->>> generator("Kerajaan Majapahit adalah", max_length=30, num_return_sequences=5, num_beams=10) - -[{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-15. Kerajaan ini berdiri pada abad ke-14'}, -{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-16. Kerajaan ini berdiri pada abad ke-14'}, -{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-15. Kerajaan ini berdiri pada abad ke-15'}, -{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-16. Kerajaan ini berdiri pada abad ke-15'}, -{'generated_text': 'Kerajaan Majapahit adalah sebuah kerajaan yang pernah berdiri di Jawa Timur pada abad ke-14 hingga abad ke-15. Kerajaan ini merupakan kelanjutan dari Kerajaan Majapahit yang'}] - -``` -Here is how to use this model to get the features of a given text in PyTorch: -```python -from transformers import GPT2Tokenizer, GPT2Model - -model_name='cahya/gpt2-small-indonesian-522M' -tokenizer = GPT2Tokenizer.from_pretrained(model_name) -model = GPT2Model.from_pretrained(model_name) -text = "Silakan diganti dengan text apa saja." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` -and in Tensorflow: -```python -from transformers import GPT2Tokenizer, TFGPT2Model - -model_name='cahya/gpt2-small-indonesian-522M' -tokenizer = GPT2Tokenizer.from_pretrained(model_name) -model = TFGPT2Model.from_pretrained(model_name) -text = "Silakan diganti dengan text apa saja." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -This model was pre-trained with 522MB of indonesian Wikipedia. -The texts are tokenized using a byte-level version of Byte Pair Encoding (BPE) (for unicode characters) and -a vocabulary size of 52,000. The inputs are sequences of 128 consecutive tokens. diff --git a/model_cards/cahya/roberta-base-indonesian-522M/README.md b/model_cards/cahya/roberta-base-indonesian-522M/README.md deleted file mode 100644 index 84e2aa210b..0000000000 --- a/model_cards/cahya/roberta-base-indonesian-522M/README.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -language: "id" -license: "mit" -datasets: -- Indonesian Wikipedia -widget: -- text: "Ibu ku sedang bekerja supermarket." ---- - -# Indonesian RoBERTa base model (uncased) - -## Model description -It is RoBERTa-base model pre-trained with indonesian Wikipedia using a masked language modeling (MLM) objective. This -model is uncased: it does not make a difference between indonesia and Indonesia. - -This is one of several other language models that have been pre-trained with indonesian datasets. More detail about -its usage on downstream tasks (text classification, text generation, etc) is available at [Transformer based Indonesian Language Models](https://github.com/cahya-wirawan/indonesian-language-models/tree/master/Transformers) - -## Intended uses & limitations - -### How to use -You can use this model directly with a pipeline for masked language modeling: -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='cahya/roberta-base-indonesian-522M') ->>> unmasker("Ibu ku sedang bekerja supermarket") - -``` -Here is how to use this model to get the features of a given text in PyTorch: -```python -from transformers import RobertaTokenizer, RobertaModel - -model_name='cahya/roberta-base-indonesian-522M' -tokenizer = RobertaTokenizer.from_pretrained(model_name) -model = RobertaModel.from_pretrained(model_name) -text = "Silakan diganti dengan text apa saja." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` -and in Tensorflow: -```python -from transformers import RobertaTokenizer, TFRobertaModel - -model_name='cahya/roberta-base-indonesian-522M' -tokenizer = RobertaTokenizer.from_pretrained(model_name) -model = TFRobertaModel.from_pretrained(model_name) -text = "Silakan diganti dengan text apa saja." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -This model was pre-trained with 522MB of indonesian Wikipedia. -The texts are lowercased and tokenized using WordPiece and a vocabulary size of 32,000. The inputs of the model are -then of the form: - -``` Sentence A Sentence B ``` diff --git a/model_cards/cambridgeltl/BioRedditBERT-uncased/README.md b/model_cards/cambridgeltl/BioRedditBERT-uncased/README.md deleted file mode 100644 index 75adafaa90..0000000000 --- a/model_cards/cambridgeltl/BioRedditBERT-uncased/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: -- en -tags: -- BioNLP -- social_media ---- - -# BioRedditBERT - -## Model description -BioRedditBERT is a BERT model initialised from BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) and further pre-trained on health-related Reddit posts. Please view our paper [COMETA: A Corpus for Medical Entity Linking in the Social Media](https://arxiv.org/pdf/2010.03295.pdf) (EMNLP 2020) for more details. - - -## Training data - -We crawled all threads from 68 health themed subreddits such as `r/AskDocs`, `r/health` and etc. starting from the beginning of 2015 to the end of 2018, obtaining a collection of more than -800K discussions. This collection was then pruned by removing deleted posts, comments from bots or moderators, and so on. In the end, we obtained the training corpus with ca. 300 million tokens and a vocabulary -size of ca. 780,000 words. - -## Training procedure -We use the same pre-training script in the original [google-research/bert](https://github.com/google-research/bert) repo. The model is initialised with [`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`](https://github.com/dmis-lab/biobert). -We train with a batch size of 64, a max sequence length of 64, a learning rate of `2e-5` for 100k steps on two GeForce GTX 1080Ti (11 GB) GPUs. Other hyper-parameters are the same as default. - - -## Eval results -To show the benefit from further pre-training on the social media domain, we demonstrate results on a medical entity linking dataset also in the social media: [AskAPatient](https://zenodo.org/record/55013#.X4ncRmTYpb8) [(Limsopatham and Collier 2016)](https://www.aclweb.org/anthology/P16-1096.pdf). -We follow the same 10-fold cross-validation procedure for all models and report the average result without fine-tuning. `[CLS]` is used as representations for entity mentions (we also tried average of all tokens but found `[CLS]` generally performs better). - -Model | Accuracy@1 | Accuracy@5 --------|---------|--------- -[BERT-base-uncased](https://huggingface.co/bert-base-uncased) | 38.2 | 43.3 -[BioBERT v1.1](https://huggingface.co/dmis-lab/biobert-v1.1) | 41.4 | 51.5 -[ClinicalBERT](https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT) | 43.9 | 54.3 -[BlueBERT](https://ftp.ncbi.nlm.nih.gov/pub/lu/Suppl/NCBI-BERT/NCBI_BERT_pubmed_mimic_uncased_L-12_H-768_A-12.zip) | 41.5 | 48.5 -[SciBERT](https://huggingface.co/allenai/scibert_scivocab_uncased) | 42.3 | 51.9 -[PubMedBERT](https://huggingface.co/microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext) | 42.5 | 49.6 -BioRedditBERT | **44.3** | **56.2** - - -### BibTeX entry and citation info - -```bibtex -@inproceedings{basaldella-2020-cometa, - title = "{COMETA}: A Corpus for Medical Entity Linking in the Social Media", - author = "Basaldella, Marco and Liu, Fangyu, and Shareghi, Ehsan, and Collier, Nigel", - booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing", - month = nov, - year = "2020", - publisher = "Association for Computational Linguistics" -} -``` diff --git a/model_cards/camembert-base-README.md b/model_cards/camembert-base-README.md deleted file mode 100644 index f73b990705..0000000000 --- a/model_cards/camembert-base-README.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -language: fr -license: mit -datasets: -- oscar ---- - -# CamemBERT: a Tasty French Language Model - -## Introduction - -[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model. - -It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. - -For further information or requests, please go to [Camembert Website](https://camembert-model.fr/) - -## Pre-trained models - -| Model | #params | Arch. | Training data | -|--------------------------------|--------------------------------|-------|-----------------------------------| -| `camembert-base` | 110M | Base | OSCAR (138 GB of text) | -| `camembert/camembert-large` | 335M | Large | CCNet (135 GB of text) | -| `camembert/camembert-base-ccnet` | 110M | Base | CCNet (135 GB of text) | -| `camembert/camembert-base-wikipedia-4gb` | 110M | Base | Wikipedia (4 GB of text) | -| `camembert/camembert-base-oscar-4gb` | 110M | Base | Subsample of OSCAR (4 GB of text) | -| `camembert/camembert-base-ccnet-4gb` | 110M | Base | Subsample of CCNet (4 GB of text) | - -## How to use CamemBERT with HuggingFace - -##### Load CamemBERT and its sub-word tokenizer : -```python -from transformers import CamembertModel, CamembertTokenizer - -# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large". -tokenizer = CamembertTokenizer.from_pretrained("camembert-base") -camembert = CamembertModel.from_pretrained("camembert-base") - -camembert.eval() # disable dropout (or leave in train mode to finetune) - -``` - -##### Filling masks using pipeline -```python -from transformers import pipeline - -camembert_fill_mask = pipeline("fill-mask", model="camembert-base", tokenizer="camembert-base") -results = camembert_fill_mask("Le camembert est :)") -# results -#[{'sequence': ' Le camembert est délicieux :)', 'score': 0.4909103214740753, 'token': 7200}, -# {'sequence': ' Le camembert est excellent :)', 'score': 0.10556930303573608, 'token': 2183}, -# {'sequence': ' Le camembert est succulent :)', 'score': 0.03453315049409866, 'token': 26202}, -# {'sequence': ' Le camembert est meilleur :)', 'score': 0.03303130343556404, 'token': 528}, -# {'sequence': ' Le camembert est parfait :)', 'score': 0.030076518654823303, 'token': 1654}] - -``` - -##### Extract contextual embedding features from Camembert output -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("J'aime le camembert !") -# ['▁J', "'", 'aime', '▁le', '▁ca', 'member', 't', '▁!'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [5, 121, 11, 660, 16, 730, 25543, 110, 83, 6] -# NB: Can be done in one step : tokenize.encode("J'aime le camembert !") - -# Feed tokens to Camembert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = camembert(encoded_sentence) -# embeddings.detach() -# embeddings.size torch.Size([1, 10, 768]) -# tensor([[[-0.0254, 0.0235, 0.1027, ..., -0.1459, -0.0205, -0.0116], -# [ 0.0606, -0.1811, -0.0418, ..., -0.1815, 0.0880, -0.0766], -# [-0.1561, -0.1127, 0.2687, ..., -0.0648, 0.0249, 0.0446], -# ..., -``` - -##### Extract contextual embedding features from all Camembert layers -```python -from transformers import CamembertConfig -# (Need to reload the model with new config) -config = CamembertConfig.from_pretrained("camembert-base", output_hidden_states=True) -camembert = CamembertModel.from_pretrained("camembert-base", config=config) - -embeddings, _, all_layer_embeddings = camembert(encoded_sentence) -# all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers) -all_layer_embeddings[5] -# layer 5 contextual embedding : size torch.Size([1, 10, 768]) -#tensor([[[-0.0032, 0.0075, 0.0040, ..., -0.0025, -0.0178, -0.0210], -# [-0.0996, -0.1474, 0.1057, ..., -0.0278, 0.1690, -0.2982], -# [ 0.0557, -0.0588, 0.0547, ..., -0.0726, -0.0867, 0.0699], -# ..., -``` - - -## Authors - -CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. - - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{martin2020camembert, - title={CamemBERT: a Tasty French Language Model}, - author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t}, - booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics}, - year={2020} -} -``` - diff --git a/model_cards/camembert/camembert-base-ccnet-4gb/README.md b/model_cards/camembert/camembert-base-ccnet-4gb/README.md deleted file mode 100644 index 96c2b4f06b..0000000000 --- a/model_cards/camembert/camembert-base-ccnet-4gb/README.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -language: fr ---- - -# CamemBERT: a Tasty French Language Model - -## Introduction - -[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model. - -It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. - -For further information or requests, please go to [Camembert Website](https://camembert-model.fr/) - -## Pre-trained models - -| Model | #params | Arch. | Training data | -|--------------------------------|--------------------------------|-------|-----------------------------------| -| `camembert-base` | 110M | Base | OSCAR (138 GB of text) | -| `camembert/camembert-large` | 335M | Large | CCNet (135 GB of text) | -| `camembert/camembert-base-ccnet` | 110M | Base | CCNet (135 GB of text) | -| `camembert/camembert-base-wikipedia-4gb` | 110M | Base | Wikipedia (4 GB of text) | -| `camembert/camembert-base-oscar-4gb` | 110M | Base | Subsample of OSCAR (4 GB of text) | -| `camembert/camembert-base-ccnet-4gb` | 110M | Base | Subsample of CCNet (4 GB of text) | - -## How to use CamemBERT with HuggingFace - -##### Load CamemBERT and its sub-word tokenizer : -```python -from transformers import CamembertModel, CamembertTokenizer - -# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large". -tokenizer = CamembertTokenizer.from_pretrained("camembert/camembert-base-ccnet-4gb") -camembert = CamembertModel.from_pretrained("camembert/camembert-base-ccnet-4gb") - -camembert.eval() # disable dropout (or leave in train mode to finetune) - -``` - -##### Filling masks using pipeline -```python -from transformers import pipeline - -camembert_fill_mask = pipeline("fill-mask", model="camembert/camembert-base-ccnet-4gb", tokenizer="camembert/camembert-base-ccnet-4gb") -results = camembert_fill_mask("Le camembert est-il ?") -# results -#[{'sequence': ' Le camembert est-il sain?', 'score': 0.07001790404319763, 'token': 10286}, -#{'sequence': ' Le camembert est-il français?', 'score': 0.057594332844018936, 'token': 384}, -#{'sequence': ' Le camembert est-il bon?', 'score': 0.04098724573850632, 'token': 305}, -#{'sequence': ' Le camembert est-il périmé?', 'score': 0.03486393392086029, 'token': 30862}, -#{'sequence': ' Le camembert est-il cher?', 'score': 0.021535946056246758, 'token': 1604}] - -``` - -##### Extract contextual embedding features from Camembert output -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("J'aime le camembert !") -# ['▁J', "'", 'aime', '▁le', '▁ca', 'member', 't', '▁!'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [5, 133, 22, 1250, 16, 12034, 14324, 81, 76, 6] -# NB: Can be done in one step : tokenize.encode("J'aime le camembert !") - -# Feed tokens to Camembert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = camembert(encoded_sentence) -# embeddings.detach() -# embeddings.size torch.Size([1, 10, 768]) -#tensor([[[ 0.0331, 0.0095, -0.2776, ..., 0.2875, -0.0827, -0.2467], -# [-0.1348, 0.0478, -0.5409, ..., 0.8330, 0.0467, 0.0662], -# [ 0.0920, -0.0264, 0.0177, ..., 0.1112, 0.0108, -0.1123], -# ..., -``` - -##### Extract contextual embedding features from all Camembert layers -```python -from transformers import CamembertConfig -# (Need to reload the model with new config) -config = CamembertConfig.from_pretrained("camembert/camembert-base-ccnet-4gb", output_hidden_states=True) -camembert = CamembertModel.from_pretrained("camembert/camembert-base-ccnet-4gb", config=config) - -embeddings, _, all_layer_embeddings = camembert(encoded_sentence) -# all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers) -all_layer_embeddings[5] -# layer 5 contextual embedding : size torch.Size([1, 10, 768]) -#tensor([[[-0.0144, 0.1855, 0.4895, ..., -0.1537, 0.0107, -0.2293], -# [-0.6664, -0.0880, -0.1539, ..., 0.3635, 0.4047, 0.1258], -# [ 0.0511, 0.0540, 0.2545, ..., 0.0709, -0.0288, -0.0779], -# ..., -``` - - -## Authors - -CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. - - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{martin2020camembert, - title={CamemBERT: a Tasty French Language Model}, - author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t}, - booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics}, - year={2020} -} -``` diff --git a/model_cards/camembert/camembert-base-ccnet/README.md b/model_cards/camembert/camembert-base-ccnet/README.md deleted file mode 100644 index 63b1b89ef3..0000000000 --- a/model_cards/camembert/camembert-base-ccnet/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: fr ---- - -# CamemBERT: a Tasty French Language Model - -## Introduction - -[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model. - -It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. - -For further information or requests, please go to [Camembert Website](https://camembert-model.fr/) - -## Pre-trained models - -| Model | #params | Arch. | Training data | -|--------------------------------|--------------------------------|-------|-----------------------------------| -| `camembert-base` | 110M | Base | OSCAR (138 GB of text) | -| `camembert/camembert-large` | 335M | Large | CCNet (135 GB of text) | -| `camembert/camembert-base-ccnet` | 110M | Base | CCNet (135 GB of text) | -| `camembert/camembert-base-wikipedia-4gb` | 110M | Base | Wikipedia (4 GB of text) | -| `camembert/camembert-base-oscar-4gb` | 110M | Base | Subsample of OSCAR (4 GB of text) | -| `camembert/camembert-base-ccnet-4gb` | 110M | Base | Subsample of CCNet (4 GB of text) | - -## How to use CamemBERT with HuggingFace - -##### Load CamemBERT and its sub-word tokenizer : -```python -from transformers import CamembertModel, CamembertTokenizer - -# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large". -tokenizer = CamembertTokenizer.from_pretrained("camembert/camembert-base-ccnet") -camembert = CamembertModel.from_pretrained("camembert/camembert-base-ccnet") - -camembert.eval() # disable dropout (or leave in train mode to finetune) - -``` - -##### Filling masks using pipeline -```python -from transformers import pipeline - -camembert_fill_mask = pipeline("fill-mask", model="camembert/camembert-base-ccnet", tokenizer="camembert/camembert-base-ccnet") -results = camembert_fill_mask("Le camembert est :)") -# results -#[{'sequence': ' Le camembert est bon :)', 'score': 0.14011502265930176, 'token': 305}, -# {'sequence': ' Le camembert est délicieux :)', 'score': 0.13929404318332672, 'token': 11661}, -# {'sequence': ' Le camembert est excellent :)', 'score': 0.07010319083929062, 'token': 3497}, -# {'sequence': ' Le camembert est parfait :)', 'score': 0.025885622948408127, 'token': 2528}, -# {'sequence': ' Le camembert est top :)', 'score': 0.025684962049126625, 'token': 2328}] -``` - -##### Extract contextual embedding features from Camembert output -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("J'aime le camembert !") -# ['▁J', "'", 'aime', '▁le', '▁cam', 'ember', 't', '▁!'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [5, 133, 22, 1250, 16, 12034, 14324, 81, 76, 6] -# NB: Can be done in one step : tokenize.encode("J'aime le camembert !") - -# Feed tokens to Camembert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = camembert(encoded_sentence) -# embeddings.detach() -# embeddings.size torch.Size([1, 10, 768]) -#tensor([[[ 0.0667, -0.2467, 0.0954, ..., 0.2144, 0.0279, 0.3621], -# [-0.0472, 0.4092, -0.6602, ..., 0.2095, 0.1391, -0.0401], -# [ 0.1911, -0.2347, -0.0811, ..., 0.4306, -0.0639, 0.1821], -# ..., -``` - -##### Extract contextual embedding features from all Camembert layers -```python -from transformers import CamembertConfig -# (Need to reload the model with new config) -config = CamembertConfig.from_pretrained("camembert/camembert-base-ccnet", output_hidden_states=True) -camembert = CamembertModel.from_pretrained("camembert/camembert-base-ccnet", config=config) - -embeddings, _, all_layer_embeddings = camembert(encoded_sentence) -# all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers) -all_layer_embeddings[5] -# layer 5 contextual embedding : size torch.Size([1, 10, 768]) -#tensor([[[ 0.0057, -0.1022, 0.0163, ..., -0.0675, -0.0360, 0.1078], -# [-0.1096, -0.3344, -0.0593, ..., 0.1625, -0.0432, -0.1646], -# [ 0.3751, -0.3829, 0.0844, ..., 0.1067, -0.0330, 0.3334], -# ..., -``` - - -## Authors - -CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. - - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{martin2020camembert, - title={CamemBERT: a Tasty French Language Model}, - author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t}, - booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics}, - year={2020} -} -``` diff --git a/model_cards/camembert/camembert-base-oscar-4gb/README.md b/model_cards/camembert/camembert-base-oscar-4gb/README.md deleted file mode 100644 index eee04e68e0..0000000000 --- a/model_cards/camembert/camembert-base-oscar-4gb/README.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -language: fr ---- - -# CamemBERT: a Tasty French Language Model - -## Introduction - -[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model. - -It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. - -For further information or requests, please go to [Camembert Website](https://camembert-model.fr/) - -## Pre-trained models - -| Model | #params | Arch. | Training data | -|--------------------------------|--------------------------------|-------|-----------------------------------| -| `camembert-base` | 110M | Base | OSCAR (138 GB of text) | -| `camembert/camembert-large` | 335M | Large | CCNet (135 GB of text) | -| `camembert/camembert-base-ccnet` | 110M | Base | CCNet (135 GB of text) | -| `camembert/camembert-base-wikipedia-4gb` | 110M | Base | Wikipedia (4 GB of text) | -| `camembert/camembert-base-oscar-4gb` | 110M | Base | Subsample of OSCAR (4 GB of text) | -| `camembert/camembert-base-ccnet-4gb` | 110M | Base | Subsample of CCNet (4 GB of text) | - -## How to use CamemBERT with HuggingFace - -##### Load CamemBERT and its sub-word tokenizer : -```python -from transformers import CamembertModel, CamembertTokenizer - -# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large". -tokenizer = CamembertTokenizer.from_pretrained("camembert/camembert-base-oscar-4gb") -camembert = CamembertModel.from_pretrained("camembert/camembert-base-oscar-4gb") - -camembert.eval() # disable dropout (or leave in train mode to finetune) - -``` - -##### Filling masks using pipeline -```python -from transformers import pipeline - -camembert_fill_mask = pipeline("fill-mask", model="camembert/camembert-base-oscar-4gb", tokenizer="camembert/camembert-base-oscar-4gb") ->>> results = camembert_fill_mask("Le camembert est !") -# results -#[{'sequence': ' Le camembert est parfait!', 'score': 0.04089554399251938, 'token': 1654}, -#{'sequence': ' Le camembert est délicieux!', 'score': 0.037193264812231064, 'token': 7200}, -#{'sequence': ' Le camembert est prêt!', 'score': 0.025467922911047935, 'token': 1415}, -#{'sequence': ' Le camembert est meilleur!', 'score': 0.022812040522694588, 'token': 528}, -#{'sequence': ' Le camembert est différent!', 'score': 0.017135459929704666, 'token': 2935}] - -``` - -##### Extract contextual embedding features from Camembert output -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("J'aime le camembert !") -# ['▁J', "'", 'aime', '▁le', '▁ca', 'member', 't', '▁!'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [5, 121, 11, 660, 16, 730, 25543, 110, 83, 6] -# NB: Can be done in one step : tokenize.encode("J'aime le camembert !") - -# Feed tokens to Camembert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = camembert(encoded_sentence) -# embeddings.detach() -# embeddings.size torch.Size([1, 10, 768]) -#tensor([[[-0.1120, -0.1464, 0.0181, ..., -0.1723, -0.0278, 0.1606], -# [ 0.1234, 0.1202, -0.0773, ..., -0.0405, -0.0668, -0.0788], -# [-0.0440, 0.0480, -0.1926, ..., 0.1066, -0.0961, 0.0637], -# ..., -``` - -##### Extract contextual embedding features from all Camembert layers -```python -from transformers import CamembertConfig -# (Need to reload the model with new config) -config = CamembertConfig.from_pretrained("camembert/camembert-base-oscar-4gb", output_hidden_states=True) -camembert = CamembertModel.from_pretrained("camembert/camembert-base-oscar-4gb", config=config) - -embeddings, _, all_layer_embeddings = camembert(encoded_sentence) -# all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers) -all_layer_embeddings[5] -# layer 5 contextual embedding : size torch.Size([1, 10, 768]) -#tensor([[[-0.1584, -0.1207, -0.0179, ..., 0.5457, 0.1491, -0.1191], -# [-0.1122, 0.3634, 0.0676, ..., 0.4395, -0.0470, -0.3781], -# [-0.2232, 0.0019, 0.0140, ..., 0.4461, -0.0233, 0.0735], -# ..., -``` - - -## Authors - -CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. - - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{martin2020camembert, - title={CamemBERT: a Tasty French Language Model}, - author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t}, - booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics}, - year={2020} -} -``` diff --git a/model_cards/camembert/camembert-base-wikipedia-4gb/README.md b/model_cards/camembert/camembert-base-wikipedia-4gb/README.md deleted file mode 100644 index 6cbb4b6715..0000000000 --- a/model_cards/camembert/camembert-base-wikipedia-4gb/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: fr ---- - -# CamemBERT: a Tasty French Language Model - -## Introduction - -[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model. - -It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. - -For further information or requests, please go to [Camembert Website](https://camembert-model.fr/) - -## Pre-trained models - -| Model | #params | Arch. | Training data | -|--------------------------------|--------------------------------|-------|-----------------------------------| -| `camembert-base` | 110M | Base | OSCAR (138 GB of text) | -| `camembert/camembert-large` | 335M | Large | CCNet (135 GB of text) | -| `camembert/camembert-base-ccnet` | 110M | Base | CCNet (135 GB of text) | -| `camembert/camembert-base-wikipedia-4gb` | 110M | Base | Wikipedia (4 GB of text) | -| `camembert/camembert-base-oscar-4gb` | 110M | Base | Subsample of OSCAR (4 GB of text) | -| `camembert/camembert-base-ccnet-4gb` | 110M | Base | Subsample of CCNet (4 GB of text) | - -## How to use CamemBERT with HuggingFace - -##### Load CamemBERT and its sub-word tokenizer : -```python -from transformers import CamembertModel, CamembertTokenizer - -# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large". -tokenizer = CamembertTokenizer.from_pretrained("camembert/camembert-base-wikipedia-4gb") -camembert = CamembertModel.from_pretrained("camembert/camembert-base-wikipedia-4gb") - -camembert.eval() # disable dropout (or leave in train mode to finetune) - -``` - -##### Filling masks using pipeline -```python -from transformers import pipeline - -camembert_fill_mask = pipeline("fill-mask", model="camembert/camembert-base-wikipedia-4gb", tokenizer="camembert/camembert-base-wikipedia-4gb") -results = camembert_fill_mask("Le camembert est un fromage de !") -# results -#[{'sequence': ' Le camembert est un fromage de chèvre!', 'score': 0.4937814474105835, 'token': 19370}, -#{'sequence': ' Le camembert est un fromage de brebis!', 'score': 0.06255942583084106, 'token': 30616}, -#{'sequence': ' Le camembert est un fromage de montagne!', 'score': 0.04340197145938873, 'token': 2364}, -# {'sequence': ' Le camembert est un fromage de Noël!', 'score': 0.02823255956172943, 'token': 3236}, -#{'sequence': ' Le camembert est un fromage de vache!', 'score': 0.021357402205467224, 'token': 12329}] -``` - -##### Extract contextual embedding features from Camembert output -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("J'aime le camembert !") -# ['▁J', "'", 'aime', '▁le', '▁ca', 'member', 't', '▁!'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [5, 221, 10, 10600, 14, 8952, 10540, 75, 1114, 6] -# NB: Can be done in one step : tokenize.encode("J'aime le camembert !") - -# Feed tokens to Camembert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = camembert(encoded_sentence) -# embeddings.detach() -# embeddings.size torch.Size([1, 10, 768]) -#tensor([[[-0.0928, 0.0506, -0.0094, ..., -0.2388, 0.1177, -0.1302], -# [ 0.0662, 0.1030, -0.2355, ..., -0.4224, -0.0574, -0.2802], -# [-0.0729, 0.0547, 0.0192, ..., -0.1743, 0.0998, -0.2677], -# ..., -``` - -##### Extract contextual embedding features from all Camembert layers -```python -from transformers import CamembertConfig -# (Need to reload the model with new config) -config = CamembertConfig.from_pretrained("camembert/camembert-base-wikipedia-4gb", output_hidden_states=True) -camembert = CamembertModel.from_pretrained("camembert/camembert-base-wikipedia-4gb", config=config) - -embeddings, _, all_layer_embeddings = camembert(encoded_sentence) -# all_layer_embeddings list of len(all_layer_embeddings) == 13 (input embedding layer + 12 self attention layers) -all_layer_embeddings[5] -# layer 5 contextual embedding : size torch.Size([1, 10, 768]) -#tensor([[[-0.0059, -0.0227, 0.0065, ..., -0.0770, 0.0369, 0.0095], -# [ 0.2838, -0.1531, -0.3642, ..., -0.0027, -0.8502, -0.7914], -# [-0.0073, -0.0338, -0.0011, ..., 0.0533, -0.0250, -0.0061], -# ..., -``` - - -## Authors - -CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. - - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{martin2020camembert, - title={CamemBERT: a Tasty French Language Model}, - author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t}, - booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics}, - year={2020} -} -``` diff --git a/model_cards/camembert/camembert-large/README.md b/model_cards/camembert/camembert-large/README.md deleted file mode 100644 index ff4360b08a..0000000000 --- a/model_cards/camembert/camembert-large/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: fr ---- - -# CamemBERT: a Tasty French Language Model - -## Introduction - -[CamemBERT](https://arxiv.org/abs/1911.03894) is a state-of-the-art language model for French based on the RoBERTa model. - -It is now available on Hugging Face in 6 different versions with varying number of parameters, amount of pretraining data and pretraining data source domains. - -For further information or requests, please go to [Camembert Website](https://camembert-model.fr/) - -## Pre-trained models - -| Model | #params | Arch. | Training data | -|--------------------------------|--------------------------------|-------|-----------------------------------| -| `camembert-base` | 110M | Base | OSCAR (138 GB of text) | -| `camembert/camembert-large` | 335M | Large | CCNet (135 GB of text) | -| `camembert/camembert-base-ccnet` | 110M | Base | CCNet (135 GB of text) | -| `camembert/camembert-base-wikipedia-4gb` | 110M | Base | Wikipedia (4 GB of text) | -| `camembert/camembert-base-oscar-4gb` | 110M | Base | Subsample of OSCAR (4 GB of text) | -| `camembert/camembert-base-ccnet-4gb` | 110M | Base | Subsample of CCNet (4 GB of text) | - -## How to use CamemBERT with HuggingFace - -##### Load CamemBERT and its sub-word tokenizer : -```python -from transformers import CamembertModel, CamembertTokenizer - -# You can replace "camembert-base" with any other model from the table, e.g. "camembert/camembert-large". -tokenizer = CamembertTokenizer.from_pretrained("camembert/camembert-large") -camembert = CamembertModel.from_pretrained("camembert/camembert-large") - -camembert.eval() # disable dropout (or leave in train mode to finetune) - -``` - -##### Filling masks using pipeline -```python -from transformers import pipeline - -camembert_fill_mask = pipeline("fill-mask", model="camembert/camembert-large", tokenizer="camembert/camembert-large") -results = camembert_fill_mask("Le camembert est :)") -# results -#[{'sequence': ' Le camembert est bon :)', 'score': 0.15560828149318695, 'token': 305}, -#{'sequence': ' Le camembert est excellent :)', 'score': 0.06821336597204208, 'token': 3497}, -#{'sequence': ' Le camembert est délicieux :)', 'score': 0.060438305139541626, 'token': 11661}, -#{'sequence': ' Le camembert est ici :)', 'score': 0.02023460529744625, 'token': 373}, -#{'sequence': ' Le camembert est meilleur :)', 'score': 0.01778135634958744, 'token': 876}] -``` - -##### Extract contextual embedding features from Camembert output -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("J'aime le camembert !") -# ['▁J', "'", 'aime', '▁le', '▁cam', 'ember', 't', '▁!'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [5, 133, 22, 1250, 16, 12034, 14324, 81, 76, 6] -# NB: Can be done in one step : tokenize.encode("J'aime le camembert !") - -# Feed tokens to Camembert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = camembert(encoded_sentence) -# embeddings.detach() -# torch.Size([1, 10, 1024]) -#tensor([[[-0.1284, 0.2643, 0.4374, ..., 0.1627, 0.1308, -0.2305], -# [ 0.4576, -0.6345, -0.2029, ..., -0.1359, -0.2290, -0.6318], -# [ 0.0381, 0.0429, 0.5111, ..., -0.1177, -0.1913, -0.1121], -# ..., -``` - -##### Extract contextual embedding features from all Camembert layers -```python -from transformers import CamembertConfig -# (Need to reload the model with new config) -config = CamembertConfig.from_pretrained("camembert/camembert-large", output_hidden_states=True) -camembert = CamembertModel.from_pretrained("camembert/camembert-large", config=config) - -embeddings, _, all_layer_embeddings = camembert(encoded_sentence) -# all_layer_embeddings list of len(all_layer_embeddings) == 25 (input embedding layer + 24 self attention layers) -all_layer_embeddings[5] -# layer 5 contextual embedding : size torch.Size([1, 10, 1024]) -#tensor([[[-0.0600, 0.0742, 0.0332, ..., -0.0525, -0.0637, -0.0287], -# [ 0.0950, 0.2840, 0.1985, ..., 0.2073, -0.2172, -0.6321], -# [ 0.1381, 0.1872, 0.1614, ..., -0.0339, -0.2530, -0.1182], -# ..., -``` - - -## Authors - -CamemBERT was trained and evaluated by Louis Martin\*, Benjamin Muller\*, Pedro Javier Ortiz Suárez\*, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah and Benoît Sagot. - - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{martin2020camembert, - title={CamemBERT: a Tasty French Language Model}, - author={Martin, Louis and Muller, Benjamin and Su{\'a}rez, Pedro Javier Ortiz and Dupont, Yoann and Romary, Laurent and de la Clergerie, {\'E}ric Villemonte and Seddah, Djam{\'e} and Sagot, Beno{\^\i}t}, - booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics}, - year={2020} -} -``` diff --git a/model_cards/canwenxu/BERT-of-Theseus-MNLI/README.md b/model_cards/canwenxu/BERT-of-Theseus-MNLI/README.md deleted file mode 100644 index 496b438b39..0000000000 --- a/model_cards/canwenxu/BERT-of-Theseus-MNLI/README.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -thumbnail: https://raw.githubusercontent.com/JetRunner/BERT-of-Theseus/master/bert-of-theseus.png -datasets: -- multi_nli ---- - -# BERT-of-Theseus -See our paper ["BERT-of-Theseus: Compressing BERT by Progressive Module Replacing"](http://arxiv.org/abs/2002.02925). - -BERT-of-Theseus is a new compressed BERT by progressively replacing the components of the original BERT. - -![BERT of Theseus](https://github.com/JetRunner/BERT-of-Theseus/blob/master/bert-of-theseus.png?raw=true) - -## Load Pretrained Model on MNLI - -We provide a 6-layer pretrained model on MNLI as a general-purpose model, which can transfer to other sentence classification tasks, outperforming DistillBERT (with the same 6-layer structure) on six tasks of GLUE (dev set). - -| Method | MNLI | MRPC | QNLI | QQP | RTE | SST-2 | STS-B | -|-----------------|------|------|------|------|------|-------|-------| -| BERT-base | 83.5 | 89.5 | 91.2 | 89.8 | 71.1 | 91.5 | 88.9 | -| DistillBERT | 79.0 | 87.5 | 85.3 | 84.9 | 59.9 | 90.7 | 81.2 | -| BERT-of-Theseus | 82.1 | 87.5 | 88.8 | 88.8 | 70.1 | 91.8 | 87.8 | - -Please Note: this checkpoint is for [Intermediate-Task Transfer Learning](https://arxiv.org/abs/2005.00628) so it does not include the classification head for MNLI! Please fine-tune it before use (like DistilBERT). diff --git a/model_cards/cedpsam/chatbot_fr/README.md b/model_cards/cedpsam/chatbot_fr/README.md deleted file mode 100644 index 32324fe88e..0000000000 --- a/model_cards/cedpsam/chatbot_fr/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: fr -tags: -- conversational -widget: -- text: "bonjour." -- text: "mais encore" -- text: "est ce que l'argent achete le bonheur?" ---- - -## a dialoggpt model trained on french opensubtitles with custom tokenizer -trained with this notebook -https://colab.research.google.com/drive/1pfCV3bngAmISNZVfDvBMyEhQKuYw37Rl#scrollTo=AyImj9qZYLRi&uniqifier=3 - -config from microsoft/DialoGPT-medium -dataset generated from 2018 opensubtitle from opus folowing these guidelines -https://github.com/PolyAI-LDN/conversational-datasets/tree/master/opensubtitles with this notebook -https://colab.research.google.com/drive/1uyh3vJ9nEjqOHI68VD73qxt4olJzODxi#scrollTo=deaacv4XfLMk -### How to use - -Now we are ready to try out how the model works as a chatting partner! - -```python -import torch -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("cedpsam/chatbot_fr") - -model = AutoModelWithLMHead.from_pretrained("cedpsam/chatbot_fr") - -for step in range(6): - # encode the new user input, add the eos_token and return a tensor in Pytorch - new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt') - # print(new_user_input_ids) - - # append the new user input tokens to the chat history - bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids - - # generated a response while limiting the total chat history to 1000 tokens, - chat_history_ids = model.generate( - bot_input_ids, max_length=1000, - pad_token_id=tokenizer.eos_token_id, - top_p=0.92, top_k = 50 - ) - - # pretty print last ouput tokens from bot - print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True))) diff --git a/model_cards/ceostroff/harry-potter-gpt2-fanfiction/README.md b/model_cards/ceostroff/harry-potter-gpt2-fanfiction/README.md deleted file mode 100644 index ab9a8eaa3b..0000000000 --- a/model_cards/ceostroff/harry-potter-gpt2-fanfiction/README.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -language: -- en -tags: -- harry-potter -license: mit ---- - -# Harry Potter Fanfiction Generator - -This is a pre-trained GPT-2 generative text model that allows you to generate your own Harry Potter fanfiction, trained off of the top 100 rated fanficition stories. We intend for this to be used for individual fun and experimentation and not as a commercial product. diff --git a/model_cards/chrisliu298/arxiv_ai_gpt2/README.md b/model_cards/chrisliu298/arxiv_ai_gpt2/README.md deleted file mode 100644 index dd153ede54..0000000000 --- a/model_cards/chrisliu298/arxiv_ai_gpt2/README.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -language: "en" -tags: -- gpt2 -- arxiv -- transformers -datasets: -- https://github.com/staeiou/arxiv_archive/tree/v1.0.1 ---- - -# ArXiv AI GPT-2 - -## Model description - -This GPT-2 (774M) model is capable of generating abstracts given paper titles. It was trained using all research paper titles and abstracts under artificial intelligence (AI), machine learning (LG), computation and language (CL), and computer vision and pattern recognition (CV) on arXiv. - -## Intended uses & limitations - -#### How to use - -To generate paper abstracts, use the provided `generate.py` [here](https://gist.github.com/chrisliu298/ccb8144888eace069da64ad3e6472d64). This is very similar to the HuggingFace's `run_generation.py` [here](https://github.com/huggingface/transformers/tree/master/examples/text-generation). You can simply replace the text with with your own model path (line 89) and change the input string to your paper title (line 127). If you want to use your own script, make sure to prepend `<|startoftext|> ` at the front and append ` <|sep|>` at the end of the paper title. - -## Training data -I selected a subset of the [arXiv Archive](https://github.com/staeiou/arxiv_archive) dataset (Geiger, 2019) as the training and evaluation data to fine-tune GPT-2. The original arXiv Archive dataset contains a full archive of metadata about papers on arxiv.org, from the start of the site in 1993 to the end of 2019. Our subset includes all the paper titles (query) and abstracts (context) under the Artificial Intelligence (cs.AI), Machine Learning (cs.LG), Computation and Language (cs.CL), and Computer Vision and Pattern Recognition (cs.CV) categories. I provide the information of the sub-dataset and the distribution of the training and evaluation dataset as follows. - - -| Splits | Count | Percentage (%) | BPE Token Count | -| :--------: | :--------: | :------------: | :-------------: | -| Train | 90,000 | 90.11 | 20,834,012 | -| Validation | 4,940 | 4.95 | 1,195,056 | -| Test | 4,940 | 4.95 | 1,218,754 | -| **Total** | **99,880** | **100** | **23,247,822** | - -The original dataset is in the format of a tab-separated value, so we wrote a simple preprocessing script to convert it into a text file format, which is the input file type (a document) of the GPT-2 model. An example of a paper’s title and its abstract is shown below. - -```text -<|startoftext|> Some paper title <|sep|> Some paper abstract <|endoftext|> -``` - -Because there are a lot of cross-domain papers in the dataset, I deduplicate the dataset using the arXiv ID, which is unique for every paper. I sort the paper by submission date, by doing so, one can examine GPT-2’s ability to use learned terminologies when it is prompted with paper titles from the “future.” - - -## Training procedure - -I used block size = 512, batch size = 1, gradidnet accumulation = 1, learning rate = 1e-5, epochs = 5, and everything else follows the default model configuration. - -## Eval results - -The resulting GPT-2 large model's perplexity score on the test set is **14.9413**. - -## Reference - -```bibtex -@dataset{r_stuart_geiger_2019_2533436, - author= {R. Stuart Geiger}, - title={{ArXiV Archive: A tidy and complete archive of metadata for papers on arxiv.org, 1993-2019}}, - month=jan, - year= 2019, - publisher={Zenodo}, - version= {v1.0.1}, - doi={10.5281/zenodo.2533436}, - url={https://doi.org/10.5281/zenodo.2533436} -} -``` - diff --git a/model_cards/cimm-kzn/endr-bert/README.md b/model_cards/cimm-kzn/endr-bert/README.md deleted file mode 100644 index 97e1cda837..0000000000 --- a/model_cards/cimm-kzn/endr-bert/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: -- ru -- en ---- - -## EnDR-BERT - - EnDR-BERT - Multilingual, Cased, which pretrained on the english collection of consumer comments on drug administration from [2]. Pre-training was based on the [original BERT code](https://github.com/google-research/bert) provided by Google. In particular, Multi-BERT was for used for initialization and all the parameters are the same as in Multi-BERT. Training details are described in our paper. \ - link: https://yadi.sk/d/-PTn0xhk1PqvgQ - - - ## Citing & Authors - - If you find this repository helpful, feel free to cite our publication: - - [1] Tutubalina E, Alimova I, Miftahutdinov Z, et al. The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews.//Bioinformatics. - 2020. - - preprint: https://arxiv.org/abs/2004.03659 - ``` - @article{10.1093/bioinformatics/btaa675, - author = {Tutubalina, Elena and Alimova, Ilseyar and Miftahutdinov, Zulfat and Sakhovskiy, Andrey and Malykh, Valentin and Nikolenko, Sergey}, - title = "{The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews}", - journal = {Bioinformatics}, - year = {2020}, - month = {07}, - issn = {1367-4803}, - doi = {10.1093/bioinformatics/btaa675}, - url = {https://doi.org/10.1093/bioinformatics/btaa675}, - note = {btaa675}, - eprint = {https://academic.oup.com/bioinformatics/advance-article-pdf/doi/10.1093/bioinformatics/btaa675/33539752/btaa675.pdf}, - } - ``` - [2] Tutubalina, EV and Miftahutdinov, Z Sh and Nugmanov, RI and Madzhidov, TI and Nikolenko, SI and Alimova, IS and Tropsha, AE Using semantic analysis of texts for the identification of drugs with similar therapeutic effects.//Russian Chemical Bulletin. – 2017. – Т. 66. – №. 11. – С. 2180-2189. - [link to paper](https://www.researchgate.net/profile/Elena_Tutubalina/publication/323751823_Using_semantic_analysis_of_texts_for_the_identification_of_drugs_with_similar_therapeutic_effects/links/5bf7cfc3299bf1a0202cbc1f/Using-semantic-analysis-of-texts-for-the-identification-of-drugs-with-similar-therapeutic-effects.pdf) - ``` - @article{tutubalina2017using, - title={Using semantic analysis of texts for the identification of drugs with similar therapeutic effects}, - author={Tutubalina, EV and Miftahutdinov, Z Sh and Nugmanov, RI and Madzhidov, TI and Nikolenko, SI and Alimova, IS and Tropsha, AE}, - journal={Russian Chemical Bulletin}, - volume={66}, - number={11}, - pages={2180--2189}, - year={2017}, - publisher={Springer} - } - ``` diff --git a/model_cards/cimm-kzn/enrudr-bert/README.md b/model_cards/cimm-kzn/enrudr-bert/README.md deleted file mode 100644 index f4ec132c8c..0000000000 --- a/model_cards/cimm-kzn/enrudr-bert/README.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -language: -- ru -- en ---- -## EnRuDR-BERT - -EnRuDR-BERT - Multilingual, Cased, which pretrained on the raw part of the RuDReC corpus (1.4M reviews) and english collection of consumer comments on drug administration from [2]. Pre-training was based on the [original BERT code](https://github.com/google-research/bert) provided by Google. In particular, Multi-BERT was for used for initialization; vocabulary of Russian subtokens and parameters are the same as in Multi-BERT. Training details are described in our paper. \ - link: https://yadi.sk/d/-PTn0xhk1PqvgQ - - -## Citing & Authors - -If you find this repository helpful, feel free to cite our publication: - -[1] Tutubalina E, Alimova I, Miftahutdinov Z, et al. The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews.//Bioinformatics. - 2020. - - preprint: https://arxiv.org/abs/2004.03659 -``` -@article{10.1093/bioinformatics/btaa675, - author = {Tutubalina, Elena and Alimova, Ilseyar and Miftahutdinov, Zulfat and Sakhovskiy, Andrey and Malykh, Valentin and Nikolenko, Sergey}, - title = "{The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews}", - journal = {Bioinformatics}, - year = {2020}, - month = {07}, - issn = {1367-4803}, - doi = {10.1093/bioinformatics/btaa675}, - url = {https://doi.org/10.1093/bioinformatics/btaa675}, - note = {btaa675}, - eprint = {https://academic.oup.com/bioinformatics/advance-article-pdf/doi/10.1093/bioinformatics/btaa675/33539752/btaa675.pdf}, -} -``` -[2] Tutubalina, EV and Miftahutdinov, Z Sh and Nugmanov, RI and Madzhidov, TI and Nikolenko, SI and Alimova, IS and Tropsha, AE Using semantic analysis of texts for the identification of drugs with similar therapeutic effects.//Russian Chemical Bulletin. – 2017. – Т. 66. – №. 11. – С. 2180-2189. - [link to paper](https://www.researchgate.net/profile/Elena_Tutubalina/publication/323751823_Using_semantic_analysis_of_texts_for_the_identification_of_drugs_with_similar_therapeutic_effects/links/5bf7cfc3299bf1a0202cbc1f/Using-semantic-analysis-of-texts-for-the-identification-of-drugs-with-similar-therapeutic-effects.pdf) -``` -@article{tutubalina2017using, - title={Using semantic analysis of texts for the identification of drugs with similar therapeutic effects}, - author={Tutubalina, EV and Miftahutdinov, Z Sh and Nugmanov, RI and Madzhidov, TI and Nikolenko, SI and Alimova, IS and Tropsha, AE}, - journal={Russian Chemical Bulletin}, - volume={66}, - number={11}, - pages={2180--2189}, - year={2017}, - publisher={Springer} -} -``` diff --git a/model_cards/cimm-kzn/rudr-bert/README.md b/model_cards/cimm-kzn/rudr-bert/README.md deleted file mode 100644 index 8d8ebf4ee5..0000000000 --- a/model_cards/cimm-kzn/rudr-bert/README.md +++ /dev/null @@ -1,41 +0,0 @@ -## RuDR-BERT - -RuDR-BERT - Multilingual, Cased, which pretrained on the raw part of the RuDReC corpus (1.4M reviews). Pre-training was based on the [original BERT code](https://github.com/google-research/bert) provided by Google. In particular, Multi-BERT was for used for initialization; vocabulary of Russian subtokens and parameters are the same as in Multi-BERT. Training details are described in our paper. \ - link: https://yadi.sk/d/-PTn0xhk1PqvgQ - - -## Citing & Authors - -If you find this repository helpful, feel free to cite our publication: - -[1] Tutubalina E, Alimova I, Miftahutdinov Z, et al. The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews.//Bioinformatics. - 2020. - - preprint: https://arxiv.org/abs/2004.03659 -``` -@article{10.1093/bioinformatics/btaa675, - author = {Tutubalina, Elena and Alimova, Ilseyar and Miftahutdinov, Zulfat and Sakhovskiy, Andrey and Malykh, Valentin and Nikolenko, Sergey}, - title = "{The Russian Drug Reaction Corpus and Neural Models for Drug Reactions and Effectiveness Detection in User Reviews}", - journal = {Bioinformatics}, - year = {2020}, - month = {07}, - issn = {1367-4803}, - doi = {10.1093/bioinformatics/btaa675}, - url = {https://doi.org/10.1093/bioinformatics/btaa675}, - note = {btaa675}, - eprint = {https://academic.oup.com/bioinformatics/advance-article-pdf/doi/10.1093/bioinformatics/btaa675/33539752/btaa675.pdf}, -} -``` -[2] Tutubalina, EV and Miftahutdinov, Z Sh and Nugmanov, RI and Madzhidov, TI and Nikolenko, SI and Alimova, IS and Tropsha, AE Using semantic analysis of texts for the identification of drugs with similar therapeutic effects.//Russian Chemical Bulletin. – 2017. – Т. 66. – №. 11. – С. 2180-2189. - [link to paper](https://www.researchgate.net/profile/Elena_Tutubalina/publication/323751823_Using_semantic_analysis_of_texts_for_the_identification_of_drugs_with_similar_therapeutic_effects/links/5bf7cfc3299bf1a0202cbc1f/Using-semantic-analysis-of-texts-for-the-identification-of-drugs-with-similar-therapeutic-effects.pdf) -``` -@article{tutubalina2017using, - title={Using semantic analysis of texts for the identification of drugs with similar therapeutic effects}, - author={Tutubalina, EV and Miftahutdinov, Z Sh and Nugmanov, RI and Madzhidov, TI and Nikolenko, SI and Alimova, IS and Tropsha, AE}, - journal={Russian Chemical Bulletin}, - volume={66}, - number={11}, - pages={2180--2189}, - year={2017}, - publisher={Springer} -} -``` diff --git a/model_cards/clue/albert_chinese_small/README.md b/model_cards/clue/albert_chinese_small/README.md deleted file mode 100644 index 94fa48b1b8..0000000000 --- a/model_cards/clue/albert_chinese_small/README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -language: zh ---- - -## albert_chinese_small - -### Overview - -**Language model:** albert-small -**Model size:** 18.5M -**Language:** Chinese -**Training data:** [CLUECorpusSmall](https://github.com/CLUEbenchmark/CLUECorpus2020) -**Eval data:** [CLUE dataset](https://github.com/CLUEbenchmark/CLUE) - -### Results - -For results on downstream tasks like text classification, please refer to [this repository](https://github.com/CLUEbenchmark/CLUE). - -### Usage - -**NOTE:**Since sentencepiece is not used in `albert_chinese_small` model, you have to call **BertTokenizer** instead of AlbertTokenizer !!! - -``` -import torch -from transformers import BertTokenizer, AlbertModel -tokenizer = BertTokenizer.from_pretrained("clue/albert_chinese_small") -albert = AlbertModel.from_pretrained("clue/albert_chinese_small") -``` - -### About CLUE benchmark - -Organization of Language Understanding Evaluation benchmark for Chinese: tasks & datasets, baselines, pre-trained Chinese models, corpus and leaderboard. - -Github: https://github.com/CLUEbenchmark -Website: https://www.cluebenchmarks.com/ diff --git a/model_cards/clue/albert_chinese_tiny/README.md b/model_cards/clue/albert_chinese_tiny/README.md deleted file mode 100644 index 0d93f83e80..0000000000 --- a/model_cards/clue/albert_chinese_tiny/README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -language: zh ---- - -## albert_chinese_tiny - -### Overview - -**Language model:** albert-tiny -**Model size:** 16M -**Language:** Chinese -**Training data:** [CLUECorpusSmall](https://github.com/CLUEbenchmark/CLUECorpus2020) -**Eval data:** [CLUE dataset](https://github.com/CLUEbenchmark/CLUE) - -### Results - -For results on downstream tasks like text classification, please refer to [this repository](https://github.com/CLUEbenchmark/CLUE). - -### Usage - -**NOTE:**Since sentencepiece is not used in `albert_chinese_tiny` model, you have to call **BertTokenizer** instead of AlbertTokenizer !!! - -``` -import torch -from transformers import BertTokenizer, AlbertModel -tokenizer = BertTokenizer.from_pretrained("clue/albert_chinese_tiny") -albert = AlbertModel.from_pretrained("clue/albert_chinese_tiny") -``` - -### About CLUE benchmark - -Organization of Language Understanding Evaluation benchmark for Chinese: tasks & datasets, baselines, pre-trained Chinese models, corpus and leaderboard. - -Github: https://github.com/CLUEbenchmark -Website: https://www.cluebenchmarks.com/ diff --git a/model_cards/clue/roberta_chinese_3L312_clue_tiny/README.md b/model_cards/clue/roberta_chinese_3L312_clue_tiny/README.md deleted file mode 100644 index dbe7f2b83e..0000000000 --- a/model_cards/clue/roberta_chinese_3L312_clue_tiny/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# Introduction -This model was trained on TPU and the details are as follows: - -## Model -## - -| Model_name | params | size | Training_corpus | Vocab | -| :------------------------------------------ | :----- | :------- | :----------------- | :-----------: | -| **`RoBERTa-tiny-clue`**
Super_small_model | 7.5M | 28.3M | **CLUECorpus2020** | **CLUEVocab** | -| **`RoBERTa-tiny-pair`**
Super_small_sentence_pair_model | 7.5M | 28.3M | **CLUECorpus2020** | **CLUEVocab** | -| **`RoBERTa-tiny3L768-clue`**
small_model | 38M | 110M | **CLUECorpus2020** | **CLUEVocab** | -| **`RoBERTa-tiny3L312-clue`**
small_model | <7.5M | 24M | **CLUECorpus2020** | **CLUEVocab** | -| **`RoBERTa-large-clue`**
Large_model | 290M | 1.20G | **CLUECorpus2020** | **CLUEVocab** | -| **`RoBERTa-large-pair`**
Large_sentence_pair_model | 290M | 1.20G | **CLUECorpus2020** | **CLUEVocab** | - -### Usage - -With the help of[Huggingface-Transformers 2.5.1](https://github.com/huggingface/transformers), you could use these model as follows - -``` -tokenizer = BertTokenizer.from_pretrained("MODEL_NAME") -model = BertModel.from_pretrained("MODEL_NAME") -``` - -`MODEL_NAME`： - -| Model_NAME | MODEL_LINK | -| -------------------------- | ------------------------------------------------------------ | -| **RoBERTa-tiny-clue** | [`clue/roberta_chinese_clue_tiny`](https://huggingface.co/clue/roberta_chinese_clue_tiny) | -| **RoBERTa-tiny-pair** | [`clue/roberta_chinese_pair_tiny`](https://huggingface.co/clue/roberta_chinese_pair_tiny) | -| **RoBERTa-tiny3L768-clue** | [`clue/roberta_chinese_3L768_clue_tiny`](https://huggingface.co/clue/roberta_chinese_3L768_clue_tiny) | -| **RoBERTa-tiny3L312-clue** | [`clue/roberta_chinese_3L312_clue_tiny`](https://huggingface.co/clue/roberta_chinese_3L312_clue_tiny) | -| **RoBERTa-large-clue** | [`clue/roberta_chinese_clue_large`](https://huggingface.co/clue/roberta_chinese_clue_large) | -| **RoBERTa-large-pair** | [`clue/roberta_chinese_pair_large`](https://huggingface.co/clue/roberta_chinese_pair_large) | - -## Details -Please read https://arxiv.org/pdf/2003.01355. - -Please visit our repository: https://github.com/CLUEbenchmark/CLUEPretrainedModels.git diff --git a/model_cards/clue/roberta_chinese_base/README.md b/model_cards/clue/roberta_chinese_base/README.md deleted file mode 100644 index 4c87c58081..0000000000 --- a/model_cards/clue/roberta_chinese_base/README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -language: zh ---- - -## roberta_chinese_base - -### Overview - -**Language model:** roberta-base -**Model size:** 392M -**Language:** Chinese -**Training data:** [CLUECorpusSmall](https://github.com/CLUEbenchmark/CLUECorpus2020) -**Eval data:** [CLUE dataset](https://github.com/CLUEbenchmark/CLUE) - -### Results - -For results on downstream tasks like text classification, please refer to [this repository](https://github.com/CLUEbenchmark/CLUE). - -### Usage - -**NOTE:** You have to call **BertTokenizer** instead of RobertaTokenizer !!! - -``` -import torch -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained("clue/roberta_chinese_base") -roberta = BertModel.from_pretrained("clue/roberta_chinese_base") -``` - -### About CLUE benchmark - -Organization of Language Understanding Evaluation benchmark for Chinese: tasks & datasets, baselines, pre-trained Chinese models, corpus and leaderboard. - -Github: https://github.com/CLUEbenchmark -Website: https://www.cluebenchmarks.com/ diff --git a/model_cards/clue/roberta_chinese_large/README.md b/model_cards/clue/roberta_chinese_large/README.md deleted file mode 100644 index b91d1a725c..0000000000 --- a/model_cards/clue/roberta_chinese_large/README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -language: zh ---- - -## roberta_chinese_large - -### Overview - -**Language model:** roberta-large -**Model size:** 1.2G -**Language:** Chinese -**Training data:** [CLUECorpusSmall](https://github.com/CLUEbenchmark/CLUECorpus2020) -**Eval data:** [CLUE dataset](https://github.com/CLUEbenchmark/CLUE) - -### Results - -For results on downstream tasks like text classification, please refer to [this repository](https://github.com/CLUEbenchmark/CLUE). - -### Usage - -**NOTE:** You have to call **BertTokenizer** instead of RobertaTokenizer !!! - -``` -import torch -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained("clue/roberta_chinese_large") -roberta = BertModel.from_pretrained("clue/roberta_chinese_large") -``` - -### About CLUE benchmark - -Organization of Language Understanding Evaluation benchmark for Chinese: tasks & datasets, baselines, pre-trained Chinese models, corpus and leaderboard. - -Github: https://github.com/CLUEbenchmark -Website: https://www.cluebenchmarks.com/ diff --git a/model_cards/clue/xlnet_chinese_large/README.md b/model_cards/clue/xlnet_chinese_large/README.md deleted file mode 100644 index 6f34c38b32..0000000000 --- a/model_cards/clue/xlnet_chinese_large/README.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -language: zh ---- - -## xlnet_chinese_large - -### Overview - -**Language model:** xlnet-large -**Model size:** 1.3G -**Language:** Chinese -**Training data:** [CLUECorpusSmall](https://github.com/CLUEbenchmark/CLUECorpus2020) -**Eval data:** [CLUE dataset](https://github.com/CLUEbenchmark/CLUE) - -### Results - -For results on downstream tasks like text classification, please refer to [this repository](https://github.com/CLUEbenchmark/CLUE). - -### Usage - -``` -import torch -from transformers import XLNetTokenizer,XLNetModel -tokenizer = XLNetTokenizer.from_pretrained("clue/xlnet_chinese_large") -xlnet = XLNetModel.from_pretrained("clue/xlnet_chinese_large") -``` - -### About CLUE benchmark - -Organization of Language Understanding Evaluation benchmark for Chinese: tasks & datasets, baselines, pre-trained Chinese models, corpus and leaderboard. - -Github: https://github.com/CLUEbenchmark -Website: https://www.cluebenchmarks.com/ diff --git a/model_cards/codegram/calbert-base-uncased/README.md b/model_cards/codegram/calbert-base-uncased/README.md deleted file mode 100644 index 8d962ed008..0000000000 --- a/model_cards/codegram/calbert-base-uncased/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: "ca" -tags: - - masked-lm - - catalan - - exbert -license: mit ---- - -# Calbert: a Catalan Language Model - -## Introduction - -CALBERT is an open-source language model for Catalan pretrained on the ALBERT architecture. - -It is now available on Hugging Face in its `tiny-uncased` version and `base-uncased` (the one you're looking at) as well, and was pretrained on the [OSCAR dataset](https://traces1.inria.fr/oscar/). - -For further information or requests, please go to the [GitHub repository](https://github.com/codegram/calbert) - -## Pre-trained models - -| Model | Arch. | Training data | -| ----------------------------------- | -------------- | ---------------------- | -| `codegram` / `calbert-tiny-uncased` | Tiny (uncased) | OSCAR (4.3 GB of text) | -| `codegram` / `calbert-base-uncased` | Base (uncased) | OSCAR (4.3 GB of text) | - -## How to use Calbert with HuggingFace - -#### Load Calbert and its tokenizer: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("codegram/calbert-base-uncased") -model = AutoModel.from_pretrained("codegram/calbert-base-uncased") - -model.eval() # disable dropout (or leave in train mode to finetune -``` - -#### Filling masks using pipeline - -```python -from transformers import pipeline - -calbert_fill_mask = pipeline("fill-mask", model="codegram/calbert-base-uncased", tokenizer="codegram/calbert-base-uncased") -results = calbert_fill_mask("M'agrada [MASK] això") -# results -# [{'sequence': "[CLS] m'agrada molt aixo[SEP]", 'score': 0.614592969417572, 'token': 61}, -# {'sequence': "[CLS] m'agrada moltíssim aixo[SEP]", 'score': 0.06058056280016899, 'token': 4867}, -# {'sequence': "[CLS] m'agrada més aixo[SEP]", 'score': 0.017195818945765495, 'token': 43}, -# {'sequence': "[CLS] m'agrada llegir aixo[SEP]", 'score': 0.016321714967489243, 'token': 684}, -# {'sequence': "[CLS] m'agrada escriure aixo[SEP]", 'score': 0.012185849249362946, 'token': 1306}] - -``` - -#### Extract contextual embedding features from Calbert output - -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("M'és una mica igual") -# ['▁m', "'", 'es', '▁una', '▁mica', '▁igual'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [2, 109, 7, 71, 36, 371, 1103, 3] -# NB: Can be done in one step : tokenize.encode("M'és una mica igual") - -# Feed tokens to Calbert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = model(encoded_sentence) -embeddings.size() -# torch.Size([1, 8, 768]) -embeddings.detach() -# tensor([[[-0.0261, 0.1166, -0.1075, ..., -0.0368, 0.0193, 0.0017], -# [ 0.1289, -0.2252, 0.9881, ..., -0.1353, 0.3534, 0.0734], -# [-0.0328, -1.2364, 0.9466, ..., 0.3455, 0.7010, -0.2085], -# ..., -# [ 0.0397, -1.0228, -0.2239, ..., 0.2932, 0.1248, 0.0813], -# [-0.0261, 0.1165, -0.1074, ..., -0.0368, 0.0193, 0.0017], -# [-0.1934, -0.2357, -0.2554, ..., 0.1831, 0.6085, 0.1421]]]) -``` - -## Authors - -CALBERT was trained and evaluated by [Txus Bach](https://twitter.com/txustice), as part of [Codegram](https://www.codegram.com)'s applied research. - - - - diff --git a/model_cards/codegram/calbert-tiny-uncased/README.md b/model_cards/codegram/calbert-tiny-uncased/README.md deleted file mode 100644 index 1038233cd3..0000000000 --- a/model_cards/codegram/calbert-tiny-uncased/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: "ca" -tags: - - masked-lm - - catalan - - exbert -license: mit ---- - -# Calbert: a Catalan Language Model - -## Introduction - -CALBERT is an open-source language model for Catalan pretrained on the ALBERT architecture. - -It is now available on Hugging Face in its `tiny-uncased` version (the one you're looking at) and `base-uncased` as well, and was pretrained on the [OSCAR dataset](https://traces1.inria.fr/oscar/). - -For further information or requests, please go to the [GitHub repository](https://github.com/codegram/calbert) - -## Pre-trained models - -| Model | Arch. | Training data | -| ----------------------------------- | -------------- | ---------------------- | -| `codegram` / `calbert-tiny-uncased` | Tiny (uncased) | OSCAR (4.3 GB of text) | -| `codegram` / `calbert-base-uncased` | Base (uncased) | OSCAR (4.3 GB of text) | - -## How to use Calbert with HuggingFace - -#### Load Calbert and its tokenizer: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("codegram/calbert-tiny-uncased") -model = AutoModel.from_pretrained("codegram/calbert-tiny-uncased") - -model.eval() # disable dropout (or leave in train mode to finetune -``` - -#### Filling masks using pipeline - -```python -from transformers import pipeline - -calbert_fill_mask = pipeline("fill-mask", model="codegram/calbert-tiny-uncased", tokenizer="codegram/calbert-tiny-uncased") -results = calbert_fill_mask("M'agrada [MASK] això") -# results -# [{'sequence': "[CLS] m'agrada molt aixo[SEP]", 'score': 0.4403671622276306, 'token': 61}, -# {'sequence': "[CLS] m'agrada més aixo[SEP]", 'score': 0.050061386078596115, 'token': 43}, -# {'sequence': "[CLS] m'agrada veure aixo[SEP]", 'score': 0.026286985725164413, 'token': 157}, -# {'sequence': "[CLS] m'agrada bastant aixo[SEP]", 'score': 0.022483550012111664, 'token': 2143}, -# {'sequence': "[CLS] m'agrada moltíssim aixo[SEP]", 'score': 0.014491282403469086, 'token': 4867}] - -``` - -#### Extract contextual embedding features from Calbert output - -```python -import torch -# Tokenize in sub-words with SentencePiece -tokenized_sentence = tokenizer.tokenize("M'és una mica igual") -# ['▁m', "'", 'es', '▁una', '▁mica', '▁igual'] - -# 1-hot encode and add special starting and end tokens -encoded_sentence = tokenizer.encode(tokenized_sentence) -# [2, 109, 7, 71, 36, 371, 1103, 3] -# NB: Can be done in one step : tokenize.encode("M'és una mica igual") - -# Feed tokens to Calbert as a torch tensor (batch dim 1) -encoded_sentence = torch.tensor(encoded_sentence).unsqueeze(0) -embeddings, _ = model(encoded_sentence) -embeddings.size() -# torch.Size([1, 8, 312]) -embeddings.detach() -# tensor([[[-0.2726, -0.9855, 0.9643, ..., 0.3511, 0.3499, -0.1984], -# [-0.2824, -1.1693, -0.2365, ..., -3.1866, -0.9386, -1.3718], -# [-2.3645, -2.2477, -1.6985, ..., -1.4606, -2.7294, 0.2495], -# ..., -# [ 0.8800, -0.0244, -3.0446, ..., 0.5148, -3.0903, 1.1879], -# [ 1.1300, 0.2425, 0.2162, ..., -0.5722, -2.2004, 0.4045], -# [ 0.4549, -0.2378, -0.2290, ..., -2.1247, -2.2769, -0.0820]]]) -``` - -## Authors - -CALBERT was trained and evaluated by [Txus Bach](https://twitter.com/txustice), as part of [Codegram](https://www.codegram.com)'s applied research. - - - - diff --git a/model_cards/cooelf/limitbert/README.md b/model_cards/cooelf/limitbert/README.md deleted file mode 100644 index cacf31eef8..0000000000 --- a/model_cards/cooelf/limitbert/README.md +++ /dev/null @@ -1,59 +0,0 @@ -# LIMIT-BERT - -Code and model for the *EMNLP 2020 Findings* paper: - -[LIMIT-BERT: Linguistic Informed Multi-task BERT](https://arxiv.org/abs/1910.14296)) - -## Contents - -1. [Requirements](#Requirements) -2. [Training](#Training) - -## Requirements - -* Python 3.6 or higher. -* Cython 0.25.2 or any compatible version. -* [PyTorch](http://pytorch.org/) 1.0.0+. -* [EVALB](http://nlp.cs.nyu.edu/evalb/). Before starting, run `make` inside the `EVALB/` directory to compile an `evalb` executable. This will be called from Python for evaluation. -* [pytorch-transformers](https://github.com/huggingface/pytorch-transformers) PyTorch 1.0.0+ or any compatible version. - -#### Pre-trained Models (PyTorch) -The following pre-trained models are available for download from Google Drive: -* [`LIMIT-BERT`](https://drive.google.com/open?id=1fm0cK2A91iLG3lCpwowCCQSALnWS2X4i): - PyTorch version, same setting with BERT-Large-WWM，loading model with [pytorch-transformers](https://github.com/huggingface/pytorch-transformers). - -## How to use - -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("cooelf/limitbert") -model = AutoModel.from_pretrained("cooelf/limitbert") -``` - -Please see our original repo for the training scripts. - -https://github.com/cooelf/LIMIT-BERT - -## Training - -To train LIMIT-BERT, simply run: -``` -sh run_limitbert.sh -``` -### Evaluation Instructions - -To test after setting model path: -``` -sh test_bert.sh -``` - -## Citation - -``` -@article{zhou2019limit, - title={{LIMIT-BERT}: Linguistic informed multi-task {BERT}}, - author={Zhou, Junru and Zhang, Zhuosheng and Zhao, Hai}, - journal={arXiv preprint arXiv:1910.14296}, - year={2019} -} -``` \ No newline at end of file diff --git a/model_cards/csarron/bert-base-uncased-squad-v1/README.md b/model_cards/csarron/bert-base-uncased-squad-v1/README.md deleted file mode 100644 index b8987bb637..0000000000 --- a/model_cards/csarron/bert-base-uncased-squad-v1/README.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -language: en -thumbnail: -license: mit -tags: -- question-answering -- bert -- bert-base -datasets: -- squad -metrics: -- squad -widget: -- text: "Which name is also used to describe the Amazon rainforest in English?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." -- text: "How many square kilometers of rainforest is covered in the basin?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." ---- - -## BERT-base uncased model fine-tuned on SQuAD v1 - -This model was fine-tuned from the HuggingFace [BERT](https://www.aclweb.org/anthology/N19-1423/) base uncased checkpoint on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer). -This model is case-insensitive: it does not make a difference between english and English. - -## Details - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD1.1 | train | 90.6K | -| SQuAD1.1 | eval | 11.1k | - - -### Fine-tuning -- Python: `3.7.5` - -- Machine specs: - - `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz` - - `Memory: 32 GiB` - - `GPUs: 2 GeForce GTX 1070, each with 8GiB memory` - - `GPU driver: 418.87.01, CUDA: 10.1` - -- script: - - ```shell - # after install https://github.com/huggingface/transformers - - cd examples/question-answering - mkdir -p data - - wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json - - wget -O data/dev-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json - - python run_squad.py \ - --model_type bert \ - --model_name_or_path bert-base-uncased \ - --do_train \ - --do_eval \ - --do_lower_case \ - --train_file train-v1.1.json \ - --predict_file dev-v1.1.json \ - --per_gpu_train_batch_size 12 \ - --per_gpu_eval_batch_size=16 \ - --learning_rate 3e-5 \ - --num_train_epochs 2.0 \ - --max_seq_length 320 \ - --doc_stride 128 \ - --data_dir data \ - --output_dir data/bert-base-uncased-squad-v1 2>&1 | tee train-energy-bert-base-squad-v1.log - ``` - -It took about 2 hours to finish. - -### Results - -**Model size**: `418M` - -| Metric | # Value | # Original ([Table 2](https://www.aclweb.org/anthology/N19-1423.pdf))| -| ------ | --------- | --------- | -| **EM** | **80.9** | **80.8** | -| **F1** | **88.2** | **88.5** | - -Note that the above results didn't involve any hyperparameter search. - -## Example Usage - - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="csarron/bert-base-uncased-squad-v1", - tokenizer="csarron/bert-base-uncased-squad-v1" -) - -predictions = qa_pipeline({ - 'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.", - 'question': "What day was the game played on?" -}) - -print(predictions) -# output: -# {'score': 0.8730505704879761, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'} -``` - -> Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp) - -> Made with ❤️ in New York. diff --git a/model_cards/csarron/mobilebert-uncased-squad-v1/README.md b/model_cards/csarron/mobilebert-uncased-squad-v1/README.md deleted file mode 100644 index 55e4eec71d..0000000000 --- a/model_cards/csarron/mobilebert-uncased-squad-v1/README.md +++ /dev/null @@ -1,116 +0,0 @@ ---- -language: en -thumbnail: -license: mit -tags: -- question-answering -- mobilebert -datasets: -- squad -metrics: -- squad -widget: -- text: "Which name is also used to describe the Amazon rainforest in English?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." -- text: "How many square kilometers of rainforest is covered in the basin?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." ---- - -## MobileBERT fine-tuned on SQuAD v1 - -[MobileBERT](https://arxiv.org/abs/2004.02984) is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance -between self-attentions and feed-forward networks. - -This model was fine-tuned from the HuggingFace checkpoint `google/mobilebert-uncased` on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer). - -## Details - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD1.1 | train | 90.6K | -| SQuAD1.1 | eval | 11.1k | - - -### Fine-tuning -- Python: `3.7.5` - -- Machine specs: - - `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz` - - `Memory: 32 GiB` - - `GPUs: 2 GeForce GTX 1070, each with 8GiB memory` - - `GPU driver: 418.87.01, CUDA: 10.1` - -- script: - - ```shell - # after install https://github.com/huggingface/transformers - - cd examples/question-answering - mkdir -p data - - wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json - - wget -O data/dev-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json - - export SQUAD_DIR=`pwd`/data - - python run_squad.py \ - --model_type mobilebert \ - --model_name_or_path google/mobilebert-uncased \ - --do_train \ - --do_eval \ - --do_lower_case \ - --train_file $SQUAD_DIR/train-v1.1.json \ - --predict_file $SQUAD_DIR/dev-v1.1.json \ - --per_gpu_train_batch_size 16 \ - --per_gpu_eval_batch_size 16 \ - --learning_rate 4e-5 \ - --num_train_epochs 5.0 \ - --max_seq_length 320 \ - --doc_stride 128 \ - --warmup_steps 1400 \ - --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v1 2>&1 | tee train-mobilebert-warmup-squad_v1.log - ``` - -It took about 3 hours to finish. - -### Results - -**Model size**: `95M` - -| Metric | # Value | # Original ([Table 5](https://arxiv.org/pdf/2004.02984.pdf))| -| ------ | --------- | --------- | -| **EM** | **82.6** | **82.9** | -| **F1** | **90.0** | **90.0** | - -Note that the above results didn't involve any hyperparameter search. - -## Example Usage - - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="csarron/mobilebert-uncased-squad-v1", - tokenizer="csarron/mobilebert-uncased-squad-v1" -) - -predictions = qa_pipeline({ - 'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.", - 'question': "What day was the game played on?" -}) - -print(predictions) -# output: -# {'score': 0.7754058241844177, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'} -``` - -> Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp) - -> Made with ❤️ in New York. \ No newline at end of file diff --git a/model_cards/csarron/mobilebert-uncased-squad-v2/README.md b/model_cards/csarron/mobilebert-uncased-squad-v2/README.md deleted file mode 100644 index 8305dbf5a9..0000000000 --- a/model_cards/csarron/mobilebert-uncased-squad-v2/README.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -language: en -thumbnail: -license: mit -tags: -- question-answering -- mobilebert -datasets: -- squad_v2 -metrics: -- squad_v2 -widget: -- text: "Which name is also used to describe the Amazon rainforest in English?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." -- text: "How many square kilometers of rainforest is covered in the basin?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." ---- - -## MobileBERT fine-tuned on SQuAD v2 - -[MobileBERT](https://arxiv.org/abs/2004.02984) is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance -between self-attentions and feed-forward networks. - -This model was fine-tuned from the HuggingFace checkpoint `google/mobilebert-uncased` on [SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer). - -## Details - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - - -### Fine-tuning -- Python: `3.7.5` - -- Machine specs: - - `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz` - - `Memory: 32 GiB` - - `GPUs: 2 GeForce GTX 1070, each with 8GiB memory` - - `GPU driver: 418.87.01, CUDA: 10.1` - -- script: - - ```shell - # after install https://github.com/huggingface/transformers - - cd examples/question-answering - mkdir -p data - - wget -O data/train-v2.0.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json - - wget -O data/dev-v2.0.json https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json - - export SQUAD_DIR=`pwd`/data - - python run_squad.py \ - --model_type mobilebert \ - --model_name_or_path google/mobilebert-uncased \ - --do_train \ - --do_eval \ - --do_lower_case \ - --version_2_with_negative \ - --train_file $SQUAD_DIR/train-v2.0.json \ - --predict_file $SQUAD_DIR/dev-v2.0.json \ - --per_gpu_train_batch_size 16 \ - --per_gpu_eval_batch_size 16 \ - --learning_rate 4e-5 \ - --num_train_epochs 5.0 \ - --max_seq_length 320 \ - --doc_stride 128 \ - --warmup_steps 1400 \ - --save_steps 2000 \ - --output_dir $SQUAD_DIR/mobilebert-uncased-warmup-squad_v2 2>&1 | tee train-mobilebert-warmup-squad_v2.log - ``` - -It took about 3.5 hours to finish. - -### Results - -**Model size**: `95M` - -| Metric | # Value | # Original ([Table 5](https://arxiv.org/pdf/2004.02984.pdf))| -| ------ | --------- | --------- | -| **EM** | **75.2** | **76.2** | -| **F1** | **78.8** | **79.2** | - -Note that the above results didn't involve any hyperparameter search. - -## Example Usage - - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="csarron/mobilebert-uncased-squad-v2", - tokenizer="csarron/mobilebert-uncased-squad-v2" -) - -predictions = qa_pipeline({ - 'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.", - 'question': "What day was the game played on?" -}) - -print(predictions) -# output: -# {'score': 0.71434086561203, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'} -``` - -> Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp) - -> Made with ❤️ in New York. \ No newline at end of file diff --git a/model_cards/csarron/roberta-base-squad-v1/README.md b/model_cards/csarron/roberta-base-squad-v1/README.md deleted file mode 100644 index 7e030ea5da..0000000000 --- a/model_cards/csarron/roberta-base-squad-v1/README.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -language: en -thumbnail: -license: mit -tags: -- question-answering -- roberta -- roberta-base -datasets: -- squad -metrics: -- squad -widget: -- text: "Which name is also used to describe the Amazon rainforest in English?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." -- text: "How many square kilometers of rainforest is covered in the basin?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." ---- - -## RoBERTa-base fine-tuned on SQuAD v1 - -This model was fine-tuned from the HuggingFace [RoBERTa](https://arxiv.org/abs/1907.11692) base checkpoint on [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer). -This model is case-sensitive: it makes a difference between english and English. - -## Details - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD1.1 | train | 96.8K | -| SQuAD1.1 | eval | 11.8k | - - -### Fine-tuning -- Python: `3.7.5` - -- Machine specs: - - `CPU: Intel(R) Core(TM) i7-6800K CPU @ 3.40GHz` - - `Memory: 32 GiB` - - `GPUs: 2 GeForce GTX 1070, each with 8GiB memory` - - `GPU driver: 418.87.01, CUDA: 10.1` - -- script: - - ```shell - # after install https://github.com/huggingface/transformers - - cd examples/question-answering - mkdir -p data - - wget -O data/train-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json - - wget -O data/dev-v1.1.json https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json - - python run_energy_squad.py \ - --model_type roberta \ - --model_name_or_path roberta-base \ - --do_train \ - --do_eval \ - --train_file train-v1.1.json \ - --predict_file dev-v1.1.json \ - --per_gpu_train_batch_size 12 \ - --per_gpu_eval_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 2.0 \ - --max_seq_length 320 \ - --doc_stride 128 \ - --data_dir data \ - --output_dir data/roberta-base-squad-v1 2>&1 | tee train-roberta-base-squad-v1.log - ``` - -It took about 2 hours to finish. - -### Results - -**Model size**: `477M` - -| Metric | # Value | -| ------ | --------- | -| **EM** | **83.0** | -| **F1** | **90.4** | - -Note that the above results didn't involve any hyperparameter search. - -## Example Usage - - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="csarron/roberta-base-squad-v1", - tokenizer="csarron/roberta-base-squad-v1" -) - -predictions = qa_pipeline({ - 'context': "The game was played on February 7, 2016 at Levi's Stadium in the San Francisco Bay Area at Santa Clara, California.", - 'question': "What day was the game played on?" -}) - -print(predictions) -# output: -# {'score': 0.8625259399414062, 'start': 23, 'end': 39, 'answer': 'February 7, 2016'} -``` - -> Created by [Qingqing Cao](https://awk.ai/) | [GitHub](https://github.com/csarron) | [Twitter](https://twitter.com/sysnlp) - -> Made with ❤️ in New York. diff --git a/model_cards/daigo/bert-base-japanese-sentiment/README.md b/model_cards/daigo/bert-base-japanese-sentiment/README.md deleted file mode 100644 index 950f692cfd..0000000000 --- a/model_cards/daigo/bert-base-japanese-sentiment/README.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -language: -- ja ---- - -binary classification - -# Usage -``` -print(pipeline("sentiment-analysis",model="daigo/bert-base-japanese-sentiment",tokenizer="daigo/bert-base-japanese-sentiment")("私は幸福である。")) - -[{'label': 'ポジティブ', 'score': 0.98430425}] -``` diff --git a/model_cards/dbmdz/bert-base-german-cased/README.md b/model_cards/dbmdz/bert-base-german-cased/README.md deleted file mode 100644 index 8478cce520..0000000000 --- a/model_cards/dbmdz/bert-base-german-cased/README.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -language: de -license: mit ---- - -# 🤗 + 📚 dbmdz German BERT models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources another German BERT models 🎉 - -# German BERT - -## Stats - -In addition to the recently released [German BERT](https://deepset.ai/german-bert) -model by [deepset](https://deepset.ai/) we provide another German-language model. - -The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus, -Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with -a size of 16GB and 2,350,234,427 tokens. - -For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps -(sentence piece model for vocab generation) follow those used for training -[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial -sequence length of 512 subwords and was performed for 1.5M steps. - -This release includes both cased and uncased models. - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| -------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `bert-base-german-dbmdz-cased` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-cased-config.json) • [`pytorch_model.bin`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-cased-pytorch_model.bin) • [`vocab.txt`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-cased-vocab.txt) -| `bert-base-german-dbmdz-uncased` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-uncased-config.json) • [`pytorch_model.bin`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-uncased-pytorch_model.bin) • [`vocab.txt`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-uncased-vocab.txt) - -## Usage - -With Transformers >= 2.3 our German BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-german-cased") -model = AutoModel.from_pretrained("dbmdz/bert-base-german-cased") -``` - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/fine-tuned-berts-seq). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-german-europeana-cased/README.md b/model_cards/dbmdz/bert-base-german-europeana-cased/README.md deleted file mode 100644 index 7ed11b31a3..0000000000 --- a/model_cards/dbmdz/bert-base-german-europeana-cased/README.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -language: de -license: mit -tags: - - "historic german" ---- - -# 🤗 + 📚 dbmdz BERT models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources German Europeana BERT models 🎉 - -# German Europeana BERT - -We use the open source [Europeana newspapers](http://www.europeana-newspapers.eu/) -that were provided by *The European Library*. The final -training corpus has a size of 51GB and consists of 8,035,986,369 tokens. - -Detailed information about the data and pretraining steps can be found in -[this repository](https://github.com/stefan-it/europeana-bert). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-german-europeana-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-german-europeana-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-german-europeana-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-german-europeana-cased/vocab.txt) - -## Results - -For results on Historic NER, please refer to [this repository](https://github.com/stefan-it/europeana-bert). - -## Usage - -With Transformers >= 2.3 our German Europeana BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-german-europeana-cased") -model = AutoModel.from_pretrained("dbmdz/bert-base-german-europeana-cased") -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-german-europeana-uncased/README.md b/model_cards/dbmdz/bert-base-german-europeana-uncased/README.md deleted file mode 100644 index 7bdc2ea37d..0000000000 --- a/model_cards/dbmdz/bert-base-german-europeana-uncased/README.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -language: de -license: mit -tags: - - "historic german" ---- - -# 🤗 + 📚 dbmdz BERT models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources German Europeana BERT models 🎉 - -# German Europeana BERT - -We use the open source [Europeana newspapers](http://www.europeana-newspapers.eu/) -that were provided by *The European Library*. The final -training corpus has a size of 51GB and consists of 8,035,986,369 tokens. - -Detailed information about the data and pretraining steps can be found in -[this repository](https://github.com/stefan-it/europeana-bert). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-german-europeana-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-german-europeana-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-german-europeana-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-german-europeana-uncased/vocab.txt) - -## Results - -For results on Historic NER, please refer to [this repository](https://github.com/stefan-it/europeana-bert). - -## Usage - -With Transformers >= 2.3 our German Europeana BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-german-europeana-uncased") -model = AutoModel.from_pretrained("dbmdz/bert-base-german-europeana-uncased") -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-german-uncased/README.md b/model_cards/dbmdz/bert-base-german-uncased/README.md deleted file mode 100644 index 8478cce520..0000000000 --- a/model_cards/dbmdz/bert-base-german-uncased/README.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -language: de -license: mit ---- - -# 🤗 + 📚 dbmdz German BERT models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources another German BERT models 🎉 - -# German BERT - -## Stats - -In addition to the recently released [German BERT](https://deepset.ai/german-bert) -model by [deepset](https://deepset.ai/) we provide another German-language model. - -The source data for the model consists of a recent Wikipedia dump, EU Bookshop corpus, -Open Subtitles, CommonCrawl, ParaCrawl and News Crawl. This results in a dataset with -a size of 16GB and 2,350,234,427 tokens. - -For sentence splitting, we use [spacy](https://spacy.io/). Our preprocessing steps -(sentence piece model for vocab generation) follow those used for training -[SciBERT](https://github.com/allenai/scibert). The model is trained with an initial -sequence length of 512 subwords and was performed for 1.5M steps. - -This release includes both cased and uncased models. - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| -------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `bert-base-german-dbmdz-cased` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-cased-config.json) • [`pytorch_model.bin`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-cased-pytorch_model.bin) • [`vocab.txt`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-cased-vocab.txt) -| `bert-base-german-dbmdz-uncased` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-uncased-config.json) • [`pytorch_model.bin`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-uncased-pytorch_model.bin) • [`vocab.txt`](https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-german-dbmdz-uncased-vocab.txt) - -## Usage - -With Transformers >= 2.3 our German BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-german-cased") -model = AutoModel.from_pretrained("dbmdz/bert-base-german-cased") -``` - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/fine-tuned-berts-seq). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-italian-cased/README.md b/model_cards/dbmdz/bert-base-italian-cased/README.md deleted file mode 100644 index 43c9de3da0..0000000000 --- a/model_cards/dbmdz/bert-base-italian-cased/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: it -license: mit -datasets: -- wikipedia ---- - -# 🤗 + 📚 dbmdz BERT and ELECTRA models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources Italian BERT and ELECTRA models 🎉 - -# Italian BERT - -The source data for the Italian BERT model consists of a recent Wikipedia dump and -various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final -training corpus has a size of 13GB and 2,050,057,573 tokens. - -For sentence splitting, we use NLTK (faster compared to spacy). -Our cased and uncased models are training with an initial sequence length of 512 -subwords for ~2-3M steps. - -For the XXL Italian models, we use the same training data from OPUS and extend -it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/). -Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens. - -Note: Unfortunately, a wrong vocab size was used when training the XXL models. -This explains the mismatch of the "real" vocab size of 31102, compared to the -vocab size specified in `config.json`. However, the model is working and all -evaluations were done under those circumstances. -See [this issue](https://github.com/dbmdz/berts/issues/7) for more information. - -The Italian ELECTRA model was trained on the "XXL" corpus for 1M steps in total using a batch -size of 128. We pretty much following the ELECTRA training procedure as used for -[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-italian-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/vocab.txt) -| `dbmdz/bert-base-italian-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-discriminator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-generator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-generator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/vocab.txt) - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/italian-bertelectra). - -## Usage - -With Transformers >= 2.3 our Italian BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the (recommended) Italian XXL BERT models, just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-xxl-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the Italian XXL ELECTRA model (discriminator), just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/electra-base-italian-xxl-cased-discriminator" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModelWithLMHead.from_pretrained(model_name) -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT/ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-italian-uncased/README.md b/model_cards/dbmdz/bert-base-italian-uncased/README.md deleted file mode 100644 index 43c9de3da0..0000000000 --- a/model_cards/dbmdz/bert-base-italian-uncased/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: it -license: mit -datasets: -- wikipedia ---- - -# 🤗 + 📚 dbmdz BERT and ELECTRA models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources Italian BERT and ELECTRA models 🎉 - -# Italian BERT - -The source data for the Italian BERT model consists of a recent Wikipedia dump and -various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final -training corpus has a size of 13GB and 2,050,057,573 tokens. - -For sentence splitting, we use NLTK (faster compared to spacy). -Our cased and uncased models are training with an initial sequence length of 512 -subwords for ~2-3M steps. - -For the XXL Italian models, we use the same training data from OPUS and extend -it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/). -Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens. - -Note: Unfortunately, a wrong vocab size was used when training the XXL models. -This explains the mismatch of the "real" vocab size of 31102, compared to the -vocab size specified in `config.json`. However, the model is working and all -evaluations were done under those circumstances. -See [this issue](https://github.com/dbmdz/berts/issues/7) for more information. - -The Italian ELECTRA model was trained on the "XXL" corpus for 1M steps in total using a batch -size of 128. We pretty much following the ELECTRA training procedure as used for -[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-italian-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/vocab.txt) -| `dbmdz/bert-base-italian-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-discriminator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-generator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-generator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/vocab.txt) - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/italian-bertelectra). - -## Usage - -With Transformers >= 2.3 our Italian BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the (recommended) Italian XXL BERT models, just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-xxl-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the Italian XXL ELECTRA model (discriminator), just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/electra-base-italian-xxl-cased-discriminator" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModelWithLMHead.from_pretrained(model_name) -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT/ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-italian-xxl-cased/README.md b/model_cards/dbmdz/bert-base-italian-xxl-cased/README.md deleted file mode 100644 index 43c9de3da0..0000000000 --- a/model_cards/dbmdz/bert-base-italian-xxl-cased/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: it -license: mit -datasets: -- wikipedia ---- - -# 🤗 + 📚 dbmdz BERT and ELECTRA models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources Italian BERT and ELECTRA models 🎉 - -# Italian BERT - -The source data for the Italian BERT model consists of a recent Wikipedia dump and -various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final -training corpus has a size of 13GB and 2,050,057,573 tokens. - -For sentence splitting, we use NLTK (faster compared to spacy). -Our cased and uncased models are training with an initial sequence length of 512 -subwords for ~2-3M steps. - -For the XXL Italian models, we use the same training data from OPUS and extend -it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/). -Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens. - -Note: Unfortunately, a wrong vocab size was used when training the XXL models. -This explains the mismatch of the "real" vocab size of 31102, compared to the -vocab size specified in `config.json`. However, the model is working and all -evaluations were done under those circumstances. -See [this issue](https://github.com/dbmdz/berts/issues/7) for more information. - -The Italian ELECTRA model was trained on the "XXL" corpus for 1M steps in total using a batch -size of 128. We pretty much following the ELECTRA training procedure as used for -[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-italian-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/vocab.txt) -| `dbmdz/bert-base-italian-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-discriminator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-generator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-generator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/vocab.txt) - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/italian-bertelectra). - -## Usage - -With Transformers >= 2.3 our Italian BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the (recommended) Italian XXL BERT models, just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-xxl-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the Italian XXL ELECTRA model (discriminator), just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/electra-base-italian-xxl-cased-discriminator" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModelWithLMHead.from_pretrained(model_name) -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT/ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-italian-xxl-uncased/README.md b/model_cards/dbmdz/bert-base-italian-xxl-uncased/README.md deleted file mode 100644 index 43c9de3da0..0000000000 --- a/model_cards/dbmdz/bert-base-italian-xxl-uncased/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: it -license: mit -datasets: -- wikipedia ---- - -# 🤗 + 📚 dbmdz BERT and ELECTRA models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources Italian BERT and ELECTRA models 🎉 - -# Italian BERT - -The source data for the Italian BERT model consists of a recent Wikipedia dump and -various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final -training corpus has a size of 13GB and 2,050,057,573 tokens. - -For sentence splitting, we use NLTK (faster compared to spacy). -Our cased and uncased models are training with an initial sequence length of 512 -subwords for ~2-3M steps. - -For the XXL Italian models, we use the same training data from OPUS and extend -it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/). -Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens. - -Note: Unfortunately, a wrong vocab size was used when training the XXL models. -This explains the mismatch of the "real" vocab size of 31102, compared to the -vocab size specified in `config.json`. However, the model is working and all -evaluations were done under those circumstances. -See [this issue](https://github.com/dbmdz/berts/issues/7) for more information. - -The Italian ELECTRA model was trained on the "XXL" corpus for 1M steps in total using a batch -size of 128. We pretty much following the ELECTRA training procedure as used for -[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-italian-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/vocab.txt) -| `dbmdz/bert-base-italian-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-discriminator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-generator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-generator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/vocab.txt) - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/italian-bertelectra). - -## Usage - -With Transformers >= 2.3 our Italian BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the (recommended) Italian XXL BERT models, just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-xxl-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the Italian XXL ELECTRA model (discriminator), just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/electra-base-italian-xxl-cased-discriminator" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModelWithLMHead.from_pretrained(model_name) -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT/ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-turkish-128k-cased/README.md b/model_cards/dbmdz/bert-base-turkish-128k-cased/README.md deleted file mode 100644 index 7047f8f184..0000000000 --- a/model_cards/dbmdz/bert-base-turkish-128k-cased/README.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Turkish BERT model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources a cased model for Turkish 🎉 - -# 🇹🇷 BERTurk - -BERTurk is a community-driven cased BERT model for Turkish. - -Some datasets used for pretraining and evaluation are contributed from the -awesome Turkish NLP community, as well as the decision for the model name: BERTurk. - -## Stats - -The current version of the model is trained on a filtered and sentence -segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/), -a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a -special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/). - -The final training corpus has a size of 35GB and 44,04,976,662 tokens. - -Thanks to Google's TensorFlow Research Cloud (TFRC) we could train a cased model -on a TPU v3-8 for 2M steps. - -For this model we use a vocab size of 128k. - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ------------------------------------ | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-turkish-128k-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-128k-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-128k-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-128k-cased/vocab.txt) - -## Usage - -With Transformers >= 2.3 our BERTurk cased model can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-turkish-128k-cased") -model = AutoModel.from_pretrained("dbmdz/bert-base-turkish-128k-cased") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-turkish-128k-uncased/README.md b/model_cards/dbmdz/bert-base-turkish-128k-uncased/README.md deleted file mode 100644 index 1477fdcff0..0000000000 --- a/model_cards/dbmdz/bert-base-turkish-128k-uncased/README.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Turkish BERT model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources an uncased model for Turkish 🎉 - -# 🇹🇷 BERTurk - -BERTurk is a community-driven uncased BERT model for Turkish. - -Some datasets used for pretraining and evaluation are contributed from the -awesome Turkish NLP community, as well as the decision for the model name: BERTurk. - -## Stats - -The current version of the model is trained on a filtered and sentence -segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/), -a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a -special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/). - -The final training corpus has a size of 35GB and 44,04,976,662 tokens. - -Thanks to Google's TensorFlow Research Cloud (TFRC) we could train an uncased model -on a TPU v3-8 for 2M steps. - -For this model we use a vocab size of 128k. - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| -------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-turkish-128k-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-128k-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-128k-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-128k-uncased/vocab.txt) - -## Usage - -With Transformers >= 2.3 our BERTurk uncased model can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-turkish-128k-uncased") -model = AutoModel.from_pretrained("dbmdz/bert-base-turkish-128k-uncased") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-turkish-cased/README.md b/model_cards/dbmdz/bert-base-turkish-cased/README.md deleted file mode 100644 index 550cee199f..0000000000 --- a/model_cards/dbmdz/bert-base-turkish-cased/README.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Turkish BERT model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources a cased model for Turkish 🎉 - -# 🇹🇷 BERTurk - -BERTurk is a community-driven cased BERT model for Turkish. - -Some datasets used for pretraining and evaluation are contributed from the -awesome Turkish NLP community, as well as the decision for the model name: BERTurk. - -## Stats - -The current version of the model is trained on a filtered and sentence -segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/), -a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a -special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/). - -The final training corpus has a size of 35GB and 44,04,976,662 tokens. - -Thanks to Google's TensorFlow Research Cloud (TFRC) we could train a cased model -on a TPU v3-8 for 2M steps. - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| --------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-turkish-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-cased/vocab.txt) - -## Usage - -With Transformers >= 2.3 our BERTurk cased model can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-turkish-cased") -model = AutoModel.from_pretrained("dbmdz/bert-base-turkish-cased") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/bert-base-turkish-uncased/README.md b/model_cards/dbmdz/bert-base-turkish-uncased/README.md deleted file mode 100644 index 68fcf64b45..0000000000 --- a/model_cards/dbmdz/bert-base-turkish-uncased/README.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Turkish BERT model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources an uncased model for Turkish 🎉 - -# 🇹🇷 BERTurk - -BERTurk is a community-driven uncased BERT model for Turkish. - -Some datasets used for pretraining and evaluation are contributed from the -awesome Turkish NLP community, as well as the decision for the model name: BERTurk. - -## Stats - -The current version of the model is trained on a filtered and sentence -segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/), -a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a -special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/). - -The final training corpus has a size of 35GB and 44,04,976,662 tokens. - -Thanks to Google's TensorFlow Research Cloud (TFRC) we could train an uncased model -on a TPU v3-8 for 2M steps. - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| --------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-turkish-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-turkish-uncased/vocab.txt) - -## Usage - -With Transformers >= 2.3 our BERTurk uncased model can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/bert-base-turkish-uncased") -model = AutoModel.from_pretrained("dbmdz/bert-base-turkish-uncased") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/distilbert-base-turkish-cased/README.md b/model_cards/dbmdz/distilbert-base-turkish-cased/README.md deleted file mode 100644 index d254ac81c8..0000000000 --- a/model_cards/dbmdz/distilbert-base-turkish-cased/README.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Distilled Turkish BERT model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources a (cased) distilled model for Turkish 🎉 - -# 🇹🇷 DistilBERTurk - -DistilBERTurk is a community-driven cased distilled BERT model for Turkish. - -DistilBERTurk was trained on 7GB of the original training data that was used -for training [BERTurk](https://github.com/stefan-it/turkish-bert/tree/master#stats), -using the cased version of BERTurk as teacher model. - -*DistilBERTurk* was trained with the official Hugging Face implementation from -[here](https://github.com/huggingface/transformers/tree/master/examples/distillation) -for 5 days on 4 RTX 2080 TI. - -More details about distillation can be found in the -["DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter"](https://arxiv.org/abs/1910.01108) -paper by Sanh et al. (2019). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue in the [BERTurk](https://github.com/stefan-it/turkish-bert) repository! - -| Model | Downloads -| --------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/distilbert-base-turkish-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/distilbert-base-turkish-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/distilbert-base-turkish-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/distilbert-base-turkish-cased/vocab.txt) - -## Usage - -With Transformers >= 2.3 our DistilBERTurk model can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/distilbert-base-turkish-cased") -model = AutoModel.from_pretrained("dbmdz/distilbert-base-turkish-cased") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert). - -For PoS tagging, DistilBERTurk outperforms the 24-layer XLM-RoBERTa model. - -The overall performance difference between DistilBERTurk and the original -(teacher) BERTurk model is ~1.18%. - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/electra-base-italian-xxl-cased-discriminator/README.md b/model_cards/dbmdz/electra-base-italian-xxl-cased-discriminator/README.md deleted file mode 100644 index 43c9de3da0..0000000000 --- a/model_cards/dbmdz/electra-base-italian-xxl-cased-discriminator/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: it -license: mit -datasets: -- wikipedia ---- - -# 🤗 + 📚 dbmdz BERT and ELECTRA models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources Italian BERT and ELECTRA models 🎉 - -# Italian BERT - -The source data for the Italian BERT model consists of a recent Wikipedia dump and -various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final -training corpus has a size of 13GB and 2,050,057,573 tokens. - -For sentence splitting, we use NLTK (faster compared to spacy). -Our cased and uncased models are training with an initial sequence length of 512 -subwords for ~2-3M steps. - -For the XXL Italian models, we use the same training data from OPUS and extend -it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/). -Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens. - -Note: Unfortunately, a wrong vocab size was used when training the XXL models. -This explains the mismatch of the "real" vocab size of 31102, compared to the -vocab size specified in `config.json`. However, the model is working and all -evaluations were done under those circumstances. -See [this issue](https://github.com/dbmdz/berts/issues/7) for more information. - -The Italian ELECTRA model was trained on the "XXL" corpus for 1M steps in total using a batch -size of 128. We pretty much following the ELECTRA training procedure as used for -[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-italian-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/vocab.txt) -| `dbmdz/bert-base-italian-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-discriminator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-generator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-generator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/vocab.txt) - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/italian-bertelectra). - -## Usage - -With Transformers >= 2.3 our Italian BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the (recommended) Italian XXL BERT models, just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-xxl-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the Italian XXL ELECTRA model (discriminator), just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/electra-base-italian-xxl-cased-discriminator" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModelWithLMHead.from_pretrained(model_name) -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT/ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/electra-base-italian-xxl-cased-generator/README.md b/model_cards/dbmdz/electra-base-italian-xxl-cased-generator/README.md deleted file mode 100644 index 43c9de3da0..0000000000 --- a/model_cards/dbmdz/electra-base-italian-xxl-cased-generator/README.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -language: it -license: mit -datasets: -- wikipedia ---- - -# 🤗 + 📚 dbmdz BERT and ELECTRA models - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources Italian BERT and ELECTRA models 🎉 - -# Italian BERT - -The source data for the Italian BERT model consists of a recent Wikipedia dump and -various texts from the [OPUS corpora](http://opus.nlpl.eu/) collection. The final -training corpus has a size of 13GB and 2,050,057,573 tokens. - -For sentence splitting, we use NLTK (faster compared to spacy). -Our cased and uncased models are training with an initial sequence length of 512 -subwords for ~2-3M steps. - -For the XXL Italian models, we use the same training data from OPUS and extend -it with data from the Italian part of the [OSCAR corpus](https://traces1.inria.fr/oscar/). -Thus, the final training corpus has a size of 81GB and 13,138,379,147 tokens. - -Note: Unfortunately, a wrong vocab size was used when training the XXL models. -This explains the mismatch of the "real" vocab size of 31102, compared to the -vocab size specified in `config.json`. However, the model is working and all -evaluations were done under those circumstances. -See [this issue](https://github.com/dbmdz/berts/issues/7) for more information. - -The Italian ELECTRA model was trained on the "XXL" corpus for 1M steps in total using a batch -size of 128. We pretty much following the ELECTRA training procedure as used for -[BERTurk](https://github.com/stefan-it/turkish-bert/tree/master/electra). - -## Model weights - -Currently only PyTorch-[Transformers](https://github.com/huggingface/transformers) -compatible weights are available. If you need access to TensorFlow checkpoints, -please raise an issue! - -| Model | Downloads -| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/bert-base-italian-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-cased/vocab.txt) -| `dbmdz/bert-base-italian-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-uncased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-cased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-cased/vocab.txt) -| `dbmdz/bert-base-italian-xxl-uncased` | [`config.json`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/bert-base-italian-xxl-uncased/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-discriminator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-discriminator/vocab.txt) -| `dbmdz/electra-base-italian-xxl-cased-generator` | [`config.json`](https://s3.amazonaws.com/models.huggingface.co/bert/dbmdz/electra-base-italian-xxl-cased-generator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-italian-xxl-cased-generator/vocab.txt) - -## Results - -For results on downstream tasks like NER or PoS tagging, please refer to -[this repository](https://github.com/stefan-it/italian-bertelectra). - -## Usage - -With Transformers >= 2.3 our Italian BERT models can be loaded like: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the (recommended) Italian XXL BERT models, just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/bert-base-italian-xxl-cased" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModel.from_pretrained(model_name) -``` - -To load the Italian XXL ELECTRA model (discriminator), just use: - -```python -from transformers import AutoModel, AutoTokenizer - -model_name = "dbmdz/electra-base-italian-xxl-cased-discriminator" - -tokenizer = AutoTokenizer.from_pretrained(model_name) - -model = AutoModelWithLMHead.from_pretrained(model_name) -``` - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our BERT/ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/electra-base-turkish-cased-discriminator/README.md b/model_cards/dbmdz/electra-base-turkish-cased-discriminator/README.md deleted file mode 100644 index c89d2300cb..0000000000 --- a/model_cards/dbmdz/electra-base-turkish-cased-discriminator/README.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Turkish ELECTRA model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources a cased ELECTRA base model for Turkish 🎉 - -# Turkish ELECTRA model - -We release a base ELEC**TR**A model for Turkish, that was trained on the same data as *BERTurk*. - -> ELECTRA is a new method for self-supervised language representation learning. It can be used to -> pre-train transformer networks using relatively little compute. ELECTRA models are trained to -> distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to -> the discriminator of a GAN. - -More details about ELECTRA can be found in the [ICLR paper](https://openreview.net/forum?id=r1xMH1BtvB) -or in the [official ELECTRA repository](https://github.com/google-research/electra) on GitHub. - -## Stats - -The current version of the model is trained on a filtered and sentence -segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/), -a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a -special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/). - -The final training corpus has a size of 35GB and 44,04,976,662 tokens. - -Thanks to Google's TensorFlow Research Cloud (TFRC) we could train a cased model -on a TPU v3-8 for 1M steps. - -## Model weights - -[Transformers](https://github.com/huggingface/transformers) -compatible weights for both PyTorch and TensorFlow are available. - -| Model | Downloads -| ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/electra-base-turkish-cased-discriminator` | [`config.json`](https://cdn.huggingface.co/dbmdz/electra-base-turkish-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-base-turkish-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-base-turkish-cased-discriminator/vocab.txt) - -## Usage - -With Transformers >= 2.8 our ELECTRA base cased model can be loaded like: - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/electra-base-turkish-cased-discriminator") -model = AutoModelWithLMHead.from_pretrained("dbmdz/electra-base-turkish-cased-discriminator") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert/electra). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dbmdz/electra-small-turkish-cased-discriminator/README.md b/model_cards/dbmdz/electra-small-turkish-cased-discriminator/README.md deleted file mode 100644 index ef35cfd7ce..0000000000 --- a/model_cards/dbmdz/electra-small-turkish-cased-discriminator/README.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -language: tr -license: mit ---- - -# 🤗 + 📚 dbmdz Turkish ELECTRA model - -In this repository the MDZ Digital Library team (dbmdz) at the Bavarian State -Library open sources a cased ELECTRA small model for Turkish 🎉 - -# Turkish ELECTRA model - -We release a small ELEC**TR**A model for Turkish, that was trained on the same data as *BERTurk*. - -> ELECTRA is a new method for self-supervised language representation learning. It can be used to -> pre-train transformer networks using relatively little compute. ELECTRA models are trained to -> distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to -> the discriminator of a GAN. - -More details about ELECTRA can be found in the [ICLR paper](https://openreview.net/forum?id=r1xMH1BtvB) -or in the [official ELECTRA repository](https://github.com/google-research/electra) on GitHub. - -## Stats - -The current version of the model is trained on a filtered and sentence -segmented version of the Turkish [OSCAR corpus](https://traces1.inria.fr/oscar/), -a recent Wikipedia dump, various [OPUS corpora](http://opus.nlpl.eu/) and a -special corpus provided by [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/). - -The final training corpus has a size of 35GB and 44,04,976,662 tokens. - -Thanks to Google's TensorFlow Research Cloud (TFRC) we could train a cased model -on a TPU v3-8 for 1M steps. - -## Model weights - -[Transformers](https://github.com/huggingface/transformers) -compatible weights for both PyTorch and TensorFlow are available. - -| Model | Downloads -| ------------------------------------------------- | --------------------------------------------------------------------------------------------------------------- -| `dbmdz/electra-small-turkish-cased-discriminator` | [`config.json`](https://cdn.huggingface.co/dbmdz/electra-small-turkish-cased-discriminator/config.json) • [`pytorch_model.bin`](https://cdn.huggingface.co/dbmdz/electra-small-turkish-cased-discriminator/pytorch_model.bin) • [`vocab.txt`](https://cdn.huggingface.co/dbmdz/electra-small-turkish-cased-discriminator/vocab.txt) - -## Usage - -With Transformers >= 2.8 our ELECTRA small cased model can be loaded like: - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("dbmdz/electra-small-turkish-cased-discriminator") -model = AutoModelWithLMHead.from_pretrained("dbmdz/electra-small-turkish-cased-discriminator") -``` - -## Results - -For results on PoS tagging or NER tasks, please refer to -[this repository](https://github.com/stefan-it/turkish-bert/electra). - -# Huggingface model hub - -All models are available on the [Huggingface model hub](https://huggingface.co/dbmdz). - -# Contact (Bugs, Feedback, Contribution and more) - -For questions about our ELECTRA models just open an issue -[here](https://github.com/dbmdz/berts/issues/new) 🤗 - -# Acknowledgments - -Thanks to [Kemal Oflazer](http://www.andrew.cmu.edu/user/ko/) for providing us -additional large corpora for Turkish. Many thanks to Reyyan Yeniterzi for providing -us the Turkish NER dataset for evaluation. - -Research supported with Cloud TPUs from Google's TensorFlow Research Cloud (TFRC). -Thanks for providing access to the TFRC ❤️ - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download both cased and uncased models from their S3 storage 🤗 diff --git a/model_cards/dccuchile/bert-base-spanish-wwm-cased/README.md b/model_cards/dccuchile/bert-base-spanish-wwm-cased/README.md deleted file mode 100644 index 6930abfd86..0000000000 --- a/model_cards/dccuchile/bert-base-spanish-wwm-cased/README.md +++ /dev/null @@ -1,3 +0,0 @@ ---- -language: es ---- diff --git a/model_cards/dccuchile/bert-base-spanish-wwm-uncased/README.md b/model_cards/dccuchile/bert-base-spanish-wwm-uncased/README.md deleted file mode 100644 index 6930abfd86..0000000000 --- a/model_cards/dccuchile/bert-base-spanish-wwm-uncased/README.md +++ /dev/null @@ -1,3 +0,0 @@ ---- -language: es ---- diff --git a/model_cards/deepset/bert-base-german-cased-oldvocab/README.md b/model_cards/deepset/bert-base-german-cased-oldvocab/README.md deleted file mode 100644 index 159b63621a..0000000000 --- a/model_cards/deepset/bert-base-german-cased-oldvocab/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: de -license: mit -thumbnail: https://static.tildacdn.com/tild6438-3730-4164-b266-613634323466/german_bert.png -tags: -- exbert ---- - - - - - -# German BERT with old vocabulary -For details see the related [FARM issue](https://github.com/deepset-ai/FARM/issues/60). - - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/electra-base-squad2/README.md b/model_cards/deepset/electra-base-squad2/README.md deleted file mode 100644 index 06cfb52b11..0000000000 --- a/model_cards/deepset/electra-base-squad2/README.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# electra-base for QA - -## Overview -**Language model:** electra-base -**Language:** English -**Downstream-task:** Extractive QA -**Training data:** SQuAD 2.0 -**Eval data:** SQuAD 2.0 -**Code:** See [example](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) in [FARM](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) -**Infrastructure**: 1x Tesla v100 - -## Hyperparameters - -``` -seed=42 -batch_size = 32 -n_epochs = 5 -base_LM_model = "google/electra-base-discriminator" -max_seq_len = 384 -learning_rate = 1e-4 -lr_schedule = LinearWarmup -warmup_proportion = 0.1 -doc_stride=128 -max_query_length=64 -``` - -## Performance -Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/). -``` -"exact": 77.30144024256717, - "f1": 81.35438272008543, - "total": 11873, - "HasAns_exact": 74.34210526315789, - "HasAns_f1": 82.45961302894314, - "HasAns_total": 5928, - "NoAns_exact": 80.25231286795626, - "NoAns_f1": 80.25231286795626, - "NoAns_total": 5945 -``` - -## Usage - -### In Transformers -```python -from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline - -model_name = "deepset/electra-base-squad2" - -# a) Get predictions -nlp = pipeline('question-answering', model=model_name, tokenizer=model_name) -QA_input = { - 'question': 'Why is model conversion important?', - 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.' -} -res = nlp(QA_input) - -# b) Load model & tokenizer -model = AutoModelForQuestionAnswering.from_pretrained(model_name) -tokenizer = AutoTokenizer.from_pretrained(model_name) -``` - -### In FARM - -```python -from farm.modeling.adaptive_model import AdaptiveModel -from farm.modeling.tokenization import Tokenizer -from farm.infer import Inferencer - -model_name = "deepset/electra-base-squad2" - -# a) Get predictions -nlp = Inferencer.load(model_name, task_type="question_answering") -QA_input = [{"questions": ["Why is model conversion important?"], - "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}] -res = nlp.inference_from_dicts(dicts=QA_input) - -# b) Load model & tokenizer -model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering") -tokenizer = Tokenizer.load(model_name) -``` - -### In haystack -For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/): -```python -reader = FARMReader(model_name_or_path="deepset/electra-base-squad2") -# or -reader = TransformersReader(model="deepset/electra-base-squad2",tokenizer="deepset/electra-base-squad2") -``` - - -## Authors -Vaishali Pal `vaishali.pal [at] deepset.ai` -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/gbert-base/README.md b/model_cards/deepset/gbert-base/README.md deleted file mode 100644 index d6404262d0..0000000000 --- a/model_cards/deepset/gbert-base/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: de -license: mit -datasets: -- wikipedia -- OPUS -- OpenLegalData ---- - -# German BERT base - -Released, Oct 2020, this is a German BERT language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our [paper](https://arxiv.org/pdf/2010.10906.pdf), we outline the steps taken to train our model and show that it outperforms its predecessors. - -## Overview -**Paper:** [here](https://arxiv.org/pdf/2010.10906.pdf) -**Architecture:** BERT base -**Language:** German - -## Performance -``` -GermEval18 Coarse: 78.17 -GermEval18 Fine: 50.90 -GermEval14: 87.98 -``` - -See also: -deepset/gbert-base -deepset/gbert-large -deepset/gelectra-base -deepset/gelectra-large -deepset/gelectra-base-generator -deepset/gelectra-large-generator - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Stefan Schweter: `stefan [at] schweter.eu` -Timo Möller: `timo.moeller [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/gbert-large/README.md b/model_cards/deepset/gbert-large/README.md deleted file mode 100644 index aa797e5ef3..0000000000 --- a/model_cards/deepset/gbert-large/README.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -language: de -license: mit -datasets: -- wikipedia -- OPUS -- OpenLegalData -- oscar ---- - -# German BERT large - -Released, Oct 2020, this is a German BERT language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our [paper](https://arxiv.org/pdf/2010.10906.pdf), we outline the steps taken to train our model and show that it outperforms its predecessors. - -## Overview -**Paper:** [here](https://arxiv.org/pdf/2010.10906.pdf) -**Architecture:** BERT large -**Language:** German - -## Performance -``` -GermEval18 Coarse: 80.08 -GermEval18 Fine: 52.48 -GermEval14: 88.16 -``` - -See also: -deepset/gbert-base -deepset/gbert-large -deepset/gelectra-base -deepset/gelectra-large -deepset/gelectra-base-generator -deepset/gelectra-large-generator - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Stefan Schweter: `stefan [at] schweter.eu` -Timo Möller: `timo.moeller [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) - - diff --git a/model_cards/deepset/gelectra-base-generator/README.md b/model_cards/deepset/gelectra-base-generator/README.md deleted file mode 100644 index ed7ee78e51..0000000000 --- a/model_cards/deepset/gelectra-base-generator/README.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -language: de -license: mit -datasets: -- wikipedia -- OPUS -- OpenLegalData ---- - -# German ELECTRA base generator - -Released, Oct 2020, this is the generator component of the German ELECTRA language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our [paper](https://arxiv.org/pdf/2010.10906.pdf), we outline the steps taken to train our model. - -The generator is useful for performing masking experiments. If you are looking for a regular language model for embedding extraction, or downstream tasks like NER, classification or QA, please use deepset/gelectra-base. - -## Overview -**Paper:** [here](https://arxiv.org/pdf/2010.10906.pdf) -**Architecture:** ELECTRA base (generator) -**Language:** German - -See also: -deepset/gbert-base -deepset/gbert-large -deepset/gelectra-base -deepset/gelectra-large -deepset/gelectra-base-generator -deepset/gelectra-large-generator - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Stefan Schweter: `stefan [at] schweter.eu` -Timo Möller: `timo.moeller [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/gelectra-base/README.md b/model_cards/deepset/gelectra-base/README.md deleted file mode 100644 index a0b2e2f0ed..0000000000 --- a/model_cards/deepset/gelectra-base/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: de -license: mit -datasets: -- wikipedia -- OPUS -- OpenLegalData ---- - -# German ELECTRA base - -Released, Oct 2020, this is a German ELECTRA language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our [paper](https://arxiv.org/pdf/2010.10906.pdf), we outline the steps taken to train our model. Our evaluation suggests that this model is somewhat undertrained. For best performance from a base sized model, we recommend deepset/gbert-base - -## Overview -**Paper:** [here](https://arxiv.org/pdf/2010.10906.pdf) -**Architecture:** ELECTRA base (discriminator) -**Language:** German - -## Performance -``` -GermEval18 Coarse: 76.02 -GermEval18 Fine: 42.22 -GermEval14: 86.02 -``` - -See also: -deepset/gbert-base -deepset/gbert-large -deepset/gelectra-base -deepset/gelectra-large -deepset/gelectra-base-generator -deepset/gelectra-large-generator - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Stefan Schweter: `stefan [at] schweter.eu` -Timo Möller: `timo.moeller [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/gelectra-large-generator/README.md b/model_cards/deepset/gelectra-large-generator/README.md deleted file mode 100644 index a513b2b865..0000000000 --- a/model_cards/deepset/gelectra-large-generator/README.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -language: de -license: mit -datasets: -- wikipedia -- OPUS -- OpenLegalData -- oscar ---- - -# German ELECTRA large generator - -Released, Oct 2020, this is the generator component of the German ELECTRA language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our [paper](https://arxiv.org/pdf/2010.10906.pdf), we outline the steps taken to train our model. - -The generator is useful for performing masking experiments. If you are looking for a regular language model for embedding extraction, or downstream tasks like NER, classification or QA, please use deepset/gelectra-large. - -## Overview -**Paper:** [here](https://arxiv.org/pdf/2010.10906.pdf) -**Architecture:** ELECTRA large (generator) -**Language:** German - -## Performance -``` -GermEval18 Coarse: 80.70 -GermEval18 Fine: 55.16 -GermEval14: 88.95 -``` - -See also: -deepset/gbert-base -deepset/gbert-large -deepset/gelectra-base -deepset/gelectra-large -deepset/gelectra-base-generator -deepset/gelectra-large-generator - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Stefan Schweter: `stefan [at] schweter.eu` -Timo Möller: `timo.moeller [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) - - diff --git a/model_cards/deepset/gelectra-large/README.md b/model_cards/deepset/gelectra-large/README.md deleted file mode 100644 index 5a05171777..0000000000 --- a/model_cards/deepset/gelectra-large/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: de -license: mit -datasets: -- wikipedia -- OPUS -- OpenLegalData -- oscar ---- - -# German ELECTRA large - -Released, Oct 2020, this is a German ELECTRA language model trained collaboratively by the makers of the original German BERT (aka "bert-base-german-cased") and the dbmdz BERT (aka bert-base-german-dbmdz-cased). In our [paper](https://arxiv.org/pdf/2010.10906.pdf), we outline the steps taken to train our model and show that this is the state of the art German language model. - -## Overview -**Paper:** [here](https://arxiv.org/pdf/2010.10906.pdf) -**Architecture:** ELECTRA large (discriminator) -**Language:** German - -## Performance -``` -GermEval18 Coarse: 80.70 -GermEval18 Fine: 55.16 -GermEval14: 88.95 -``` - -See also: -deepset/gbert-base -deepset/gbert-large -deepset/gelectra-base -deepset/gelectra-large -deepset/gelectra-base-generator -deepset/gelectra-large-generator - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Stefan Schweter: `stefan [at] schweter.eu` -Timo Möller: `timo.moeller [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/minilm-uncased-squad2/README.md b/model_cards/deepset/minilm-uncased-squad2/README.md deleted file mode 100644 index 4c6604f7f6..0000000000 --- a/model_cards/deepset/minilm-uncased-squad2/README.md +++ /dev/null @@ -1,116 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# MiniLM-L12-H384-uncased for QA - -## Overview -**Language model:** microsoft/MiniLM-L12-H384-uncased -**Language:** English -**Downstream-task:** Extractive QA -**Training data:** SQuAD 2.0 -**Eval data:** SQuAD 2.0 -**Code:** See [example](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) in [FARM](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) -**Infrastructure**: 1x Tesla v100 - -## Hyperparameters - -``` -seed=42 -batch_size = 12 -n_epochs = 4 -base_LM_model = "microsoft/MiniLM-L12-H384-uncased" -max_seq_len = 384 -learning_rate = 4e-5 -lr_schedule = LinearWarmup -warmup_proportion = 0.2 -doc_stride=128 -max_query_length=64 -grad_acc_steps=4 -``` - -## Performance -Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/). -``` -"exact": 76.13071675229513, -"f1": 79.49786500219953, -"total": 11873, -"HasAns_exact": 78.35695006747639, -"HasAns_f1": 85.10090269418276, -"HasAns_total": 5928, -"NoAns_exact": 73.91084945332211, -"NoAns_f1": 73.91084945332211, -"NoAns_total": 5945 -``` - -## Usage - -### In Transformers -```python -from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline - -model_name = "deepset/minilm-uncased-squad2" - -# a) Get predictions -nlp = pipeline('question-answering', model=model_name, tokenizer=model_name) -QA_input = { - 'question': 'Why is model conversion important?', - 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.' -} -res = nlp(QA_input) - -# b) Load model & tokenizer -model = AutoModelForQuestionAnswering.from_pretrained(model_name) -tokenizer = AutoTokenizer.from_pretrained(model_name) -``` - -### In FARM - -```python -from farm.modeling.adaptive_model import AdaptiveModel -from farm.modeling.tokenization import Tokenizer -from farm.infer import Inferencer - -model_name = "deepset/minilm-uncased-squad2" - -# a) Get predictions -nlp = Inferencer.load(model_name, task_type="question_answering") -QA_input = [{"questions": ["Why is model conversion important?"], - "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}] -res = nlp.inference_from_dicts(dicts=QA_input) - -# b) Load model & tokenizer -model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering") -tokenizer = Tokenizer.load(model_name) -``` - -### In haystack -For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/): -```python -reader = FARMReader(model_name_or_path="deepset/minilm-uncased-squad2") -# or -reader = TransformersReader(model="deepset/minilm-uncased-squad2",tokenizer="deepset/minilm-uncased-squad2") -``` - - -## Authors -Vaishali Pal `vaishali.pal [at] deepset.ai` -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/quora_dedup_bert_base/README.md b/model_cards/deepset/quora_dedup_bert_base/README.md deleted file mode 100644 index 317b14e69e..0000000000 --- a/model_cards/deepset/quora_dedup_bert_base/README.md +++ /dev/null @@ -1,59 +0,0 @@ -This language model is trained using sentence_transformers (https://github.com/UKPLab/sentence-transformers) -Started with bert-base-nli-stsb-mean-tokens -Continue training on quora questions deduplication dataset (https://www.kaggle.com/c/quora-question-pairs) -See train_script.py for script used - -Below is the performance over the course of training -epoch,steps,cosine_pearson,cosine_spearman,euclidean_pearson,euclidean_spearman,manhattan_pearson,manhattan_spearman,dot_pearson,dot_spearman -0,1000,0.5944576426835938,0.6010801382777033,0.5942803776859142,0.5934485776801595,0.5939676679774666,0.593162725602328,0.5905591590826669,0.5921674789994058 -0,2000,0.6404080440207146,0.6416811632113405,0.6384419354012121,0.6352050423100778,0.6379917744471867,0.6347884067391001,0.6410544760582826,0.6379252046791412 -0,3000,0.6710168301884945,0.6676529324662036,0.6660195209784969,0.6618423144808695,0.6656461098096684,0.6615366331956389,0.6724401903484759,0.666073727723655 -0,4000,0.6886373265097949,0.6808948140300153,0.67907655686838,0.6714218133850957,0.6786809551564443,0.6711577956884357,0.6926435869763303,0.68190855298609 -0,5000,0.6991409753700026,0.6919630610321864,0.6991041519437052,0.6868961486499775,0.6987076032270729,0.6865385550504007,0.7035518148330993,0.6916275246101342 -0,6000,0.7120367327025509,0.6975005265298305,0.7065567493967201,0.6922375503495235,0.7060005509843024,0.6916475765570651,0.7147094303373102,0.6981390706722722 -0,7000,0.7254672394728687,0.7130118465900485,0.7261844956277705,0.7086213543110718,0.7257479964972307,0.7079315661881832,0.728729909455115,0.7122743793160531 -0,8000,0.7402421930101399,0.7216774208330149,0.7367901914441078,0.7166256588352043,0.7362607046874481,0.7158881916281887,0.7433902441373252,0.7220998491980078 -0,9000,0.7381005358120434,0.7197216844469877,0.7343228719349923,0.7139462687943793,0.7345247569255238,0.7145106206467152,0.7421843672419275,0.720686853053079 -0,10000,0.7465436564646095,0.7260327107480364,0.7467524239596304,0.7230195666847953,0.7467721566237211,0.7231367593302213,0.749792199122442,0.7263143296580317 -0,11000,0.7521805421706547,0.7323771570146701,0.7530672061250105,0.729223203496722,0.7530616532823367,0.7293818369675622,0.7552399002305836,0.7320808333541338 -0,12000,0.7579359969644401,0.7340677616737238,0.7570017235719905,0.7305965412825544,0.7570601853520393,0.730718189957289,0.7611254136080384,0.7351501229591327 -0,-1,0.7573407371218097,0.7329952035782198,0.755595312163209,0.7291445551777086,0.7557737117990928,0.7295404703700227,0.7607276219361719,0.7342415455980179 -1,1000,0.7619907683805341,0.7374667949734767,0.7629820517114324,0.7330364216044966,0.7628369522755882,0.7331912674450544,0.7658583898073758,0.7381503446695727 -1,2000,0.7618972640071228,0.7362151058969478,0.764582212425539,0.7335856230046062,0.7643125513700815,0.7334501607097152,0.7652852805583232,0.7369104639809163 -1,3000,0.7687362955240467,0.7404674623181671,0.7708304819979073,0.7380959815601529,0.7707835692712482,0.7379796800453193,0.772074854759756,0.7414513460702766 -1,4000,0.7685047787908202,0.7403088288815168,0.7703522257474043,0.7379787888808298,0.7701221475099808,0.7377898546753812,0.7713755359045312,0.7409415801952219 -1,5000,0.7696438109797803,0.7410393893292365,0.773270389327895,0.7392953127251652,0.7729880866533291,0.7389853982789335,0.7726236305835863,0.7416278035580925 -1,6000,0.7749538363837081,0.7436499342062207,0.774879168058157,0.7401827241766746,0.7745754601165837,0.739763415043146,0.7788801166152383,0.7446249060022169 -1,7000,0.7794560817870597,0.7480970176267153,0.7803506944510302,0.7453305130502859,0.7799867949176531,0.7447100155494814,0.7828208193123926,0.7486740690324809 -1,8000,0.7855844359073243,0.7496742172376921,0.7828816645965887,0.747176409009761,0.7827584875358967,0.7471037762845532,0.7879159073496309,0.7507349669102151 -1,9000,0.7844110753729492,0.7507746252693759,0.7847208586489722,0.7485172180290892,0.7846408087474059,0.748491818820158,0.7872061334510225,0.7514470349769437 -1,10000,0.7881311227435004,0.7530048509727403,0.7886917756879734,0.7508018068765787,0.7883332502188707,0.7505037008187275,0.7910707228932787,0.7537200382362567 -1,11000,0.7883300109606874,0.7513494487126553,0.7879329130497712,0.749818368689255,0.7876525616593218,0.7494872882301785,0.7911454269743292,0.7522843165147303 -1,12000,0.7853334933336618,0.7516809747712728,0.7893895316714998,0.749780492728257,0.7890075986655403,0.7494079715118533,0.7885959664070629,0.7523827940133203 -1,-1,0.7887529238148887,0.7534076729932393,0.7896864404801204,0.7513080079201105,0.7894077512343298,0.7510009899066772,0.7919617393746149,0.7542173273241598 -2,1000,0.7919209063905188,0.7550167329363414,0.7917464066515253,0.7523043685293455,0.7914371703225378,0.7520285423781206,0.7950297421784158,0.7562599556207076 -2,2000,0.7924507768792486,0.7542908512484463,0.7934519001953887,0.7517491515010692,0.7931885648751081,0.751521004535999,0.7951637852162545,0.7551495215642072 -2,3000,0.7937606244038364,0.755599577136169,0.7933633347508111,0.7527922999916203,0.7931581019714242,0.7527132061436363,0.797275652800117,0.7569827180764233 -2,4000,0.7938389298721445,0.7578716892320315,0.7963783770097079,0.7555928931784702,0.796150381773947,0.7555438771581088,0.7972911620482322,0.759178632650707 -2,5000,0.7935330563129844,0.7551129824372304,0.7970775059297484,0.7527285792572385,0.7967359830546507,0.7524478515463257,0.7966395126138969,0.756319220359678 -2,6000,0.7929852776759999,0.7525490026774382,0.7952484474454824,0.7503695753216607,0.7950784132079611,0.7503677929234961,0.7956152082976395,0.7535275392698093 -2,7000,0.794956504054517,0.756119591765251,0.7982025041673655,0.7532521587180684,0.7980261618830962,0.7532107179960499,0.7983222918908033,0.7571226363678287 -2,8000,0.7934568432535339,0.7538336661192452,0.797015698241178,0.7514773358161916,0.7968076980315735,0.7513458838811067,0.7960694134685949,0.754143803399873 -2,9000,0.7970040626682157,0.7576497805894974,0.7987855332059015,0.7550996144509958,0.7984693921009676,0.7548260162973456,0.7999509314900626,0.758347143906916 -2,10000,0.7979442987735523,0.7585338500791028,0.8018677081664496,0.7557412777548302,0.8015397301245205,0.7552916678886369,0.8007921348414564,0.7589772216225288 -2,11000,0.7985519561040211,0.7579986850302035,0.8021236875460913,0.7555826443181872,0.8019861620475348,0.7553763317660516,0.8009230128897853,0.7586541619907702 -2,12000,0.7986842143860736,0.7599570950134775,0.8029131054823838,0.7577678644678973,0.8027922603736795,0.7575152095990927,0.8020896747930555,0.7608540869254408 -2,-1,0.7994135319568432,0.7596286881516635,0.8022087183675333,0.7570593611974978,0.8020218401019292,0.7567291719729909,0.8026346812258125,0.7603928913647044 -3,1000,0.7985505039929134,0.7592588405681144,0.8023296699449267,0.7569345933969436,0.8023622066009718,0.7570237132696928,0.8013054275981851,0.759643838536062 -3,2000,0.7995482191699455,0.759205368623176,0.8026859405513612,0.7565709841358819,0.8024845263367439,0.7562920388231202,0.8021318586127523,0.7596496313300967 -3,3000,0.7991070423195897,0.7582027696555826,0.8016352550470427,0.7555585819429662,0.8014268261947898,0.7551838327642736,0.8013136081494014,0.7584429477727118 -3,4000,0.7999188836884763,0.7586764419322649,0.802987646214278,0.7561111254802977,0.8026549791861386,0.7556463650525692,0.8024068858366156,0.7591238238715613 -3,5000,0.7988075932525881,0.7583533823004922,0.8019498750207454,0.755792967372457,0.8016459824731964,0.7553834613587099,0.8015528810821693,0.7589527136833425 -3,6000,0.8003341798460688,0.7585432077405799,0.8032464035902267,0.7563722467405277,0.8028695045742804,0.7557626665682309,0.8027937010871594,0.7590404967573696 -3,7000,0.799187592384933,0.7579358555659604,0.8028413548398412,0.7555875459131398,0.8025187078191003,0.7551196665011402,0.8018680475193432,0.7585565756912578 -3,8000,0.797725037202641,0.757439012042047,0.802048241301358,0.7548888458326453,0.8017608103042271,0.7544606246736175,0.8005479449399782,0.758037452190282 -3,9000,0.7990232649360067,0.7573703896772077,0.8021375332910405,0.754873027155089,0.8018733796679427,0.7545680141630304,0.8016400687760605,0.7579461042843499 -3,10000,0.7994934439260372,0.758368978248884,0.8035693504115055,0.75619400688862,0.8032990505007025,0.7559016935896375,0.8022819185772518,0.7589558328445544 -3,11000,0.8002954591825011,0.758710753096932,0.8043310859792212,0.7566387152306694,0.8040865016706966,0.7564221538891368,0.8030873114870971,0.7592722085543488 -3,12000,0.8003726616196549,0.7588056657991931,0.8044000317617518,0.7566146528909147,0.8041705213966136,0.7563419459362758,0.8031760015719815,0.7593194421057111 -3,-1,0.8004926728141455,0.7587192194882135,0.8043340929890026,0.756546030526114,0.8041028559910275,0.7563103085106637,0.8032542493776693,0.7592325501951863 diff --git a/model_cards/deepset/roberta-base-squad2-covid/README.md b/model_cards/deepset/roberta-base-squad2-covid/README.md deleted file mode 100644 index 970d15835b..0000000000 --- a/model_cards/deepset/roberta-base-squad2-covid/README.md +++ /dev/null @@ -1,106 +0,0 @@ -# roberta-base-squad2 for QA on COVID-19 - -## Overview -**Language model:** deepset/roberta-base-squad2 -**Language:** English -**Downstream-task:** Extractive QA -**Training data:** [SQuAD-style CORD-19 annotations from 23rd April](https://github.com/deepset-ai/COVID-QA/blob/master/data/question-answering/200423_covidQA.json) -**Code:** See [example](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering_crossvalidation.py) in [FARM](https://github.com/deepset-ai/FARM) -**Infrastructure**: Tesla v100 - -## Hyperparameters -``` -batch_size = 24 -n_epochs = 3 -base_LM_model = "deepset/roberta-base-squad2" -max_seq_len = 384 -learning_rate = 3e-5 -lr_schedule = LinearWarmup -warmup_proportion = 0.1 -doc_stride = 128 -xval_folds = 5 -dev_split = 0 -no_ans_boost = -100 -``` - -## Performance -5-fold cross-validation on the data set led to the following results: - -**Single EM-Scores:** [0.222, 0.123, 0.234, 0.159, 0.158] -**Single F1-Scores:** [0.476, 0.493, 0.599, 0.461, 0.465] -**Single top\_3\_recall Scores:** [0.827, 0.776, 0.860, 0.771, 0.777] -**XVAL EM:** 0.17890995260663506 -**XVAL f1:** 0.49925444207319924 -**XVAL top\_3\_recall:** 0.8021327014218009 - -This model is the model obtained from the **third** fold of the cross-validation. - -## Usage - -### In Transformers -```python -from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline - - -model_name = "deepset/roberta-base-squad2-covid" - -# a) Get predictions -nlp = pipeline('question-answering', model=model_name, tokenizer=model_name) -QA_input = { - 'question': 'Why is model conversion important?', - 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.' -} -res = nlp(QA_input) - -# b) Load model & tokenizer -model = AutoModelForQuestionAnswering.from_pretrained(model_name) -tokenizer = AutoTokenizer.from_pretrained(model_name) -``` - -### In FARM -```python -from farm.modeling.adaptive_model import AdaptiveModel -from farm.modeling.tokenization import Tokenizer -from farm.infer import Inferencer - -model_name = "deepset/roberta-base-squad2-covid" - -# a) Get predictions -nlp = Inferencer.load(model_name, task_type="question_answering") -QA_input = [{"questions": ["Why is model conversion important?"], - "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}] -res = nlp.inference_from_dicts(dicts=QA_input, rest_api_schema=True) - -# b) Load model & tokenizer -model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering") -tokenizer = Tokenizer.load(model_name) -``` - -### In haystack -For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/): -```python -reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2-covid") -# or -reader = TransformersReader(model="deepset/roberta-base-squad2",tokenizer="deepset/roberta-base-squad2-covid") -``` - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` -Bogdan Kostić: `bogdan.kostic [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) \ No newline at end of file diff --git a/model_cards/deepset/roberta-base-squad2-v2/README.md b/model_cards/deepset/roberta-base-squad2-v2/README.md deleted file mode 100644 index 7cbf6b8878..0000000000 --- a/model_cards/deepset/roberta-base-squad2-v2/README.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# roberta-base for QA - -## Overview -**Language model:** roberta-base -**Language:** English -**Downstream-task:** Extractive QA -**Training data:** SQuAD 2.0 -**Eval data:** SQuAD 2.0 -**Code:** See [example](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) in [FARM](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) -**Infrastructure**: 4x Tesla v100 - -## Hyperparameters - -``` -batch_size = 96 -n_epochs = 2 -base_LM_model = "roberta-base" -max_seq_len = 386 -learning_rate = 3e-5 -lr_schedule = LinearWarmup -warmup_proportion = 0.2 -doc_stride=128 -max_query_length=64 -``` - -## Performance -Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/). - -``` -"exact": 79.97136359807968 -"f1": 83.00449234495325 - -"total": 11873 -"HasAns_exact": 78.03643724696356 -"HasAns_f1": 84.11139298441825 -"HasAns_total": 5928 -"NoAns_exact": 81.90075693860386 -"NoAns_f1": 81.90075693860386 -"NoAns_total": 5945 -``` - -## Usage - -### In Transformers -```python -from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline - -model_name = "deepset/roberta-base-squad2-v2" - -# a) Get predictions -nlp = pipeline('question-answering', model=model_name, tokenizer=model_name) -QA_input = { - 'question': 'Why is model conversion important?', - 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.' -} -res = nlp(QA_input) - -# b) Load model & tokenizer -model = AutoModelForQuestionAnswering.from_pretrained(model_name) -tokenizer = AutoTokenizer.from_pretrained(model_name) -``` - -### In FARM - -```python -from farm.modeling.adaptive_model import AdaptiveModel -from farm.modeling.tokenization import Tokenizer -from farm.infer import Inferencer - -model_name = "deepset/roberta-base-squad2-v2" - -# a) Get predictions -nlp = Inferencer.load(model_name, task_type="question_answering") -QA_input = [{"questions": ["Why is model conversion important?"], - "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}] -res = nlp.inference_from_dicts(dicts=QA_input, rest_api_schema=True) - -# b) Load model & tokenizer -model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering") -tokenizer = Tokenizer.load(model_name) -``` - -### In haystack -For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/): -```python -reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2-v2") -# or -reader = TransformersReader(model_name_or_path="deepset/roberta-base-squad2-v2",tokenizer="deepset/roberta-base-squad2-v2") -``` - - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/roberta-base-squad2/README.md b/model_cards/deepset/roberta-base-squad2/README.md deleted file mode 100644 index 9e443b0663..0000000000 --- a/model_cards/deepset/roberta-base-squad2/README.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -datasets: -- squad_v2 ---- - -# roberta-base for QA - -NOTE: This is version 2 of the model. See [this github issue](https://github.com/deepset-ai/FARM/issues/552) from the FARM repository for an explanation of why we updated. If you'd like to use version 1, specify `revision="v1.0"` when loading the model in Transformers 3.5. For exmaple: -``` -model_name = "deepset/roberta-base-squad2" -pipeline(model=model_name, tokenizer=model_name, revision="v1.0", task="question-answering") -``` - -## Overview -**Language model:** roberta-base -**Language:** English -**Downstream-task:** Extractive QA -**Training data:** SQuAD 2.0 -**Eval data:** SQuAD 2.0 -**Code:** See [example](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) in [FARM](https://github.com/deepset-ai/FARM/blob/master/examples/question_answering.py) -**Infrastructure**: 4x Tesla v100 - -## Hyperparameters - -``` -batch_size = 96 -n_epochs = 2 -base_LM_model = "roberta-base" -max_seq_len = 386 -learning_rate = 3e-5 -lr_schedule = LinearWarmup -warmup_proportion = 0.2 -doc_stride=128 -max_query_length=64 -``` - -## Performance -Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/). - -``` -"exact": 79.97136359807968 -"f1": 83.00449234495325 - -"total": 11873 -"HasAns_exact": 78.03643724696356 -"HasAns_f1": 84.11139298441825 -"HasAns_total": 5928 -"NoAns_exact": 81.90075693860386 -"NoAns_f1": 81.90075693860386 -"NoAns_total": 5945 -``` - -## Usage - -### In Transformers -```python -from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline - -model_name = "deepset/roberta-base-squad2" - -# a) Get predictions -nlp = pipeline('question-answering', model=model_name, tokenizer=model_name) -QA_input = { - 'question': 'Why is model conversion important?', - 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.' -} -res = nlp(QA_input) - -# b) Load model & tokenizer -model = AutoModelForQuestionAnswering.from_pretrained(model_name) -tokenizer = AutoTokenizer.from_pretrained(model_name) -``` - -### In FARM - -```python -from farm.modeling.adaptive_model import AdaptiveModel -from farm.modeling.tokenization import Tokenizer -from farm.infer import Inferencer - -model_name = "deepset/roberta-base-squad2" - -# a) Get predictions -nlp = Inferencer.load(model_name, task_type="question_answering") -QA_input = [{"questions": ["Why is model conversion important?"], - "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}] -res = nlp.inference_from_dicts(dicts=QA_input, rest_api_schema=True) - -# b) Load model & tokenizer -model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering") -tokenizer = Tokenizer.load(model_name) -``` - -### In haystack -For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/): -```python -reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2") -# or -reader = TransformersReader(model_name_or_path="deepset/roberta-base-squad2",tokenizer="deepset/roberta-base-squad2") -``` - - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) diff --git a/model_cards/deepset/sentence_bert/README.md b/model_cards/deepset/sentence_bert/README.md deleted file mode 100644 index 721f09fce8..0000000000 --- a/model_cards/deepset/sentence_bert/README.md +++ /dev/null @@ -1 +0,0 @@ -This is an upload of the bert-base-nli-stsb-mean-tokens pretrained model from the Sentence Transformers Repo (https://github.com/UKPLab/sentence-transformers) diff --git a/model_cards/deepset/xlm-roberta-large-squad2/README.md b/model_cards/deepset/xlm-roberta-large-squad2/README.md deleted file mode 100644 index 8fcf54b7e6..0000000000 --- a/model_cards/deepset/xlm-roberta-large-squad2/README.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -language: multilingual -tags: -- question-answering -datasets: -- squad_v2 ---- - -# Multilingual XLM-RoBERTa large for QA on various languages - -## Overview -**Language model:** xlm-roberta-large -**Language:** Multilingual -**Downstream-task:** Extractive QA -**Training data:** SQuAD 2.0 -**Eval data:** SQuAD dev set - German MLQA - German XQuAD -**Training run:** [MLFlow link](https://public-mlflow.deepset.ai/#/experiments/124/runs/3a540e3f3ecf4dd98eae8fc6d457ff20) -**Infrastructure**: 4x Tesla v100 - -## Hyperparameters - -``` -batch_size = 32 -n_epochs = 3 -base_LM_model = "xlm-roberta-large" -max_seq_len = 256 -learning_rate = 1e-5 -lr_schedule = LinearWarmup -warmup_proportion = 0.2 -doc_stride=128 -max_query_length=64 -``` - -## Performance -Evaluated on the SQuAD 2.0 English dev set with the [official eval script](https://worksheets.codalab.org/rest/bundles/0x6b567e1cf2e041ec80d7098f031c5c9e/contents/blob/). -``` - "exact": 79.45759285774446, - "f1": 83.79259828925511, - "total": 11873, - "HasAns_exact": 71.96356275303644, - "HasAns_f1": 80.6460053117963, - "HasAns_total": 5928, - "NoAns_exact": 86.93019343986543, - "NoAns_f1": 86.93019343986543, - "NoAns_total": 5945 -``` - -Evaluated on German [MLQA: test-context-de-question-de.json](https://github.com/facebookresearch/MLQA) -``` -"exact": 49.34691166703564, -"f1": 66.15582561674236, -"total": 4517, -``` - -Evaluated on German [XQuAD: xquad.de.json](https://github.com/deepmind/xquad) -``` -"exact": 61.51260504201681, -"f1": 78.80206098332569, -"total": 1190, -``` - -## Usage - -### In Transformers -```python -from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline - -model_name = "deepset/xlm-roberta-large-squad2" - -# a) Get predictions -nlp = pipeline('question-answering', model=model_name, tokenizer=model_name) -QA_input = { - 'question': 'Why is model conversion important?', - 'context': 'The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks.' -} -res = nlp(QA_input) - -# b) Load model & tokenizer -model = AutoModelForQuestionAnswering.from_pretrained(model_name) -tokenizer = AutoTokenizer.from_pretrained(model_name) -``` - -### In FARM - -```python -from farm.modeling.adaptive_model import AdaptiveModel -from farm.modeling.tokenization import Tokenizer -from farm.infer import QAInferencer - -model_name = "deepset/xlm-roberta-large-squad2" - -# a) Get predictions -nlp = QAInferencer.load(model_name) -QA_input = [{"questions": ["Why is model conversion important?"], - "text": "The option to convert models between FARM and transformers gives freedom to the user and let people easily switch between frameworks."}] -res = nlp.inference_from_dicts(dicts=QA_input, rest_api_schema=True) - -# b) Load model & tokenizer -model = AdaptiveModel.convert_from_transformers(model_name, device="cpu", task_type="question_answering") -tokenizer = Tokenizer.load(model_name) -``` - -### In haystack -For doing QA at scale (i.e. many docs instead of single paragraph), you can load the model also in [haystack](https://github.com/deepset-ai/haystack/): -```python -reader = FARMReader(model_name_or_path="deepset/xlm-roberta-large-squad2") -# or -reader = TransformersReader(model="deepset/xlm-roberta-large-squad2",tokenizer="deepset/xlm-roberta-large-squad2") -``` - - -## Authors -Branden Chan: `branden.chan [at] deepset.ai` -Timo Möller: `timo.moeller [at] deepset.ai` -Malte Pietsch: `malte.pietsch [at] deepset.ai` -Tanay Soni: `tanay.soni [at] deepset.ai` - -## About us -![deepset logo](https://raw.githubusercontent.com/deepset-ai/FARM/master/docs/img/deepset_logo.png) - -We bring NLP to the industry via open source! -Our focus: Industry specific language models & large scale QA systems. - -Some of our work: -- [German BERT (aka "bert-base-german-cased")](https://deepset.ai/german-bert) -- [FARM](https://github.com/deepset-ai/FARM) -- [Haystack](https://github.com/deepset-ai/haystack/) - -Get in touch: -[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Website](https://deepset.ai) - diff --git a/model_cards/digitalepidemiologylab/covid-twitter-bert/README.md b/model_cards/digitalepidemiologylab/covid-twitter-bert/README.md deleted file mode 100644 index a148db5082..0000000000 --- a/model_cards/digitalepidemiologylab/covid-twitter-bert/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: "en" -thumbnail: "https://raw.githubusercontent.com/digitalepidemiologylab/covid-twitter-bert/master/images/COVID-Twitter-BERT_small.png" -tags: -- Twitter -- COVID-19 -license: mit ---- - -# COVID-Twitter-BERT (CT-BERT) v1 - -:warning: _You may want to use the [v2 model](https://huggingface.co/digitalepidemiologylab/covid-twitter-bert-v2) which was trained on more recent data and yields better performance_ :warning: - - -BERT-large-uncased model, pretrained on a corpus of messages from Twitter about COVID-19. Find more info on our [GitHub page](https://github.com/digitalepidemiologylab/covid-twitter-bert). - -## Overview -This model was trained on 160M tweets collected between January 12 and April 16, 2020 containing at least one of the keywords "wuhan", "ncov", "coronavirus", "covid", or "sars-cov-2". These tweets were filtered and preprocessed to reach a final sample of 22.5M tweets (containing 40.7M sentences and 633M tokens) which were used for training. - -This model was evaluated based on downstream classification tasks, but it could be used for any other NLP task which can leverage contextual embeddings. - -In order to achieve best results, make sure to use the same text preprocessing as we did for pretraining. This involves replacing user mentions, urls and emojis. You can find a script on our projects [GitHub repo](https://github.com/digitalepidemiologylab/covid-twitter-bert). - -## Example usage -```python -tokenizer = AutoTokenizer.from_pretrained("digitalepidemiologylab/covid-twitter-bert") -model = AutoModel.from_pretrained("digitalepidemiologylab/covid-twitter-bert") -``` - -You can also use the model with the `pipeline` interface: - -```python -from transformers import pipeline -import json - -pipe = pipeline(task='fill-mask', model='digitalepidemiologylab/covid-twitter-bert-v2') -out = pipe(f"In places with a lot of people, it's a good idea to wear a {pipe.tokenizer.mask_token}") -print(json.dumps(out, indent=4)) -[ - { - "sequence": "[CLS] in places with a lot of people, it's a good idea to wear a mask [SEP]", - "score": 0.9959408044815063, - "token": 7308, - "token_str": "mask" - }, - ... -] -``` - -## References -[1] Martin Müller, Marcel Salaté, Per E Kummervold. "COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter" arXiv preprint arXiv:2005.07503 (2020). diff --git a/model_cards/distilbert-base-cased-README.md b/model_cards/distilbert-base-cased-README.md deleted file mode 100644 index 184ee3acc4..0000000000 --- a/model_cards/distilbert-base-cased-README.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia ---- - -# DistilBERT base model (cased) - -This model is a distilled version of the [BERT base model](https://huggingface.co/bert-base-cased). -It was introduced in [this paper](https://arxiv.org/abs/1910.01108). -The code for the distillation process can be found -[here](https://github.com/huggingface/transformers/tree/master/examples/distillation). -This model is cased: it does make a difference between english and English. - -All the training details on the pre-training, the uses, limitations and potential biases are the same as for [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased). -We highly encourage to check it if you want to know more. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | -|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:| -| | 81.5 | 87.8 | 88.2 | 90.4 | 47.2 | 85.5 | 85.6 | 60.6 | - -### BibTeX entry and citation info - -```bibtex -@article{Sanh2019DistilBERTAD, - title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter}, - author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf}, - journal={ArXiv}, - year={2019}, - volume={abs/1910.01108} -} -``` diff --git a/model_cards/distilbert-base-cased-distilled-squad-README.md b/model_cards/distilbert-base-cased-distilled-squad-README.md deleted file mode 100644 index 2f92ff7ae9..0000000000 --- a/model_cards/distilbert-base-cased-distilled-squad-README.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -language: "en" -datasets: -- squad -metrics: -- squad -license: apache-2.0 ---- - -# DistilBERT base cased distilled SQuAD - -This model is a fine-tune checkpoint of [DistilBERT-base-cased](https://huggingface.co/distilbert-base-cased), fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. -This model reaches a F1 score of 87.1 on the dev set (for comparison, BERT bert-base-cased version reaches a F1 score of 88.7). diff --git a/model_cards/distilbert-base-german-cased-README.md b/model_cards/distilbert-base-german-cased-README.md deleted file mode 100644 index 2b0c9fdb61..0000000000 --- a/model_cards/distilbert-base-german-cased-README.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -language: de -license: apache-2.0 ---- -## distilbert-base-german-cased diff --git a/model_cards/distilbert-base-multilingual-cased-README.md b/model_cards/distilbert-base-multilingual-cased-README.md deleted file mode 100644 index 2fa58c2575..0000000000 --- a/model_cards/distilbert-base-multilingual-cased-README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -language: multilingual -license: apache-2.0 -datasets: -- wikipedia ---- - -# DistilBERT base multilingual model (cased) - -This model is a distilled version of the [BERT base multilingual model](bert-base-multilingual-cased). The code for the distillation process can be found -[here](https://github.com/huggingface/transformers/tree/master/examples/distillation). This model is cased: it does make a difference between english and English. - -The model is trained on the concatenation of Wikipedia in 104 different languages listed [here](https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages). -The model has 6 layers, 768 dimension and 12 heads, totalizing 134M parameters (compared to 177M parameters for mBERT-base). -On average DistilmBERT is twice as fast as mBERT-base. - -We encourage to check [BERT base multilingual model](bert-base-multilingual-cased) to know more about usage, limitations and potential biases. - -| Model | English | Spanish | Chinese | German | Arabic | Urdu | -| :---: | :---: | :---: | :---: | :---: | :---: | :---:| -| mBERT base cased (computed) | 82.1 | 74.6 | 69.1 | 72.3 | 66.4 | 58.5 | -| mBERT base uncased (reported)| 81.4 | 74.3 | 63.8 | 70.5 | 62.1 | 58.3 | -| DistilmBERT | 78.2 | 69.1 | 64.0 | 66.3 | 59.1 | 54.7 | - -### BibTeX entry and citation info - -```bibtex -@article{Sanh2019DistilBERTAD, - title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter}, - author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf}, - journal={ArXiv}, - year={2019}, - volume={abs/1910.01108} -} -``` diff --git a/model_cards/distilbert-base-uncased-README.md b/model_cards/distilbert-base-uncased-README.md deleted file mode 100644 index 9b4358201c..0000000000 --- a/model_cards/distilbert-base-uncased-README.md +++ /dev/null @@ -1,218 +0,0 @@ ---- -language: en -tags: -- exbert -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia ---- - -# DistilBERT base model (uncased) - -This model is a distilled version of the [BERT base model](https://huggingface.co/bert-base-uncased). It was -introduced in [this paper](https://arxiv.org/abs/1910.01108). The code for the distillation process can be found -[here](https://github.com/huggingface/transformers/tree/master/examples/distillation). This model is uncased: it does -not make a difference between english and English. - -## Model description - -DistilBERT is a transformers model, smaller and faster than BERT, which was pretrained on the same corpus in a -self-supervised fashion, using the BERT base model as a teacher. This means it was pretrained on the raw texts only, -with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic -process to generate inputs and labels from those texts using the BERT base model. More precisely, it was pretrained -with three objectives: - -- Distillation loss: the model was trained to return the same probabilities as the BERT base model. -- Masked language modeling (MLM): this is part of the original training loss of the BERT base model. When taking a - sentence, the model randomly masks 15% of the words in the input then run the entire masked sentence through the - model and has to predict the masked words. This is different from traditional recurrent neural networks (RNNs) that - usually see the words one after the other, or from autoregressive models like GPT which internally mask the future - tokens. It allows the model to learn a bidirectional representation of the sentence. -- Cosine embedding loss: the model was also trained to generate hidden states as close as possible as the BERT base - model. - -This way, the model learns the same inner representation of the English language than its teacher model, while being -faster for inference or downstream tasks. - -## Intended uses & limitations - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=distilbert) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='distilbert-base-uncased') ->>> unmasker("Hello I'm a [MASK] model.") - -[{'sequence': "[CLS] hello i'm a role model. [SEP]", - 'score': 0.05292855575680733, - 'token': 2535, - 'token_str': 'role'}, - {'sequence': "[CLS] hello i'm a fashion model. [SEP]", - 'score': 0.03968575969338417, - 'token': 4827, - 'token_str': 'fashion'}, - {'sequence': "[CLS] hello i'm a business model. [SEP]", - 'score': 0.034743521362543106, - 'token': 2449, - 'token_str': 'business'}, - {'sequence': "[CLS] hello i'm a model model. [SEP]", - 'score': 0.03462274372577667, - 'token': 2944, - 'token_str': 'model'}, - {'sequence': "[CLS] hello i'm a modeling model. [SEP]", - 'score': 0.018145186826586723, - 'token': 11643, - 'token_str': 'modeling'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import DistilBertTokenizer, DistilBertModel -tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased') -model = DistilBertModel.from_pretrained("distilbert-base-uncased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import DistilBertTokenizer, TFDistilBertModel -tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased') -model = TFDistilBertModel.from_pretrained("distilbert-base-uncased") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -Even if the training data used for this model could be characterized as fairly neutral, this model can have biased -predictions. It also inherits some of -[the bias of its teacher model](https://huggingface.co/bert-base-uncased#limitations-and-bias). - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='distilbert-base-uncased') ->>> unmasker("The White man worked as a [MASK].") - -[{'sequence': '[CLS] the white man worked as a blacksmith. [SEP]', - 'score': 0.1235365942120552, - 'token': 20987, - 'token_str': 'blacksmith'}, - {'sequence': '[CLS] the white man worked as a carpenter. [SEP]', - 'score': 0.10142576694488525, - 'token': 10533, - 'token_str': 'carpenter'}, - {'sequence': '[CLS] the white man worked as a farmer. [SEP]', - 'score': 0.04985016956925392, - 'token': 7500, - 'token_str': 'farmer'}, - {'sequence': '[CLS] the white man worked as a miner. [SEP]', - 'score': 0.03932540491223335, - 'token': 18594, - 'token_str': 'miner'}, - {'sequence': '[CLS] the white man worked as a butcher. [SEP]', - 'score': 0.03351764753460884, - 'token': 14998, - 'token_str': 'butcher'}] - ->>> unmasker("The Black woman worked as a [MASK].") - -[{'sequence': '[CLS] the black woman worked as a waitress. [SEP]', - 'score': 0.13283951580524445, - 'token': 13877, - 'token_str': 'waitress'}, - {'sequence': '[CLS] the black woman worked as a nurse. [SEP]', - 'score': 0.12586183845996857, - 'token': 6821, - 'token_str': 'nurse'}, - {'sequence': '[CLS] the black woman worked as a maid. [SEP]', - 'score': 0.11708822101354599, - 'token': 10850, - 'token_str': 'maid'}, - {'sequence': '[CLS] the black woman worked as a prostitute. [SEP]', - 'score': 0.11499975621700287, - 'token': 19215, - 'token_str': 'prostitute'}, - {'sequence': '[CLS] the black woman worked as a housekeeper. [SEP]', - 'score': 0.04722772538661957, - 'token': 22583, - 'token_str': 'housekeeper'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -DistilBERT pretrained on the same data as BERT, which is [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset -consisting of 11,038 unpublished books and [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) -(excluding lists, tables and headers). - -## Training procedure - -### Preprocessing - -The texts are lowercased and tokenized using WordPiece and a vocabulary size of 30,000. The inputs of the model are -then of the form: - -``` -[CLS] Sentence A [SEP] Sentence B [SEP] -``` - -With probability 0.5, sentence A and sentence B correspond to two consecutive sentences in the original corpus and in -the other cases, it's another random sentence in the corpus. Note that what is considered a sentence here is a -consecutive span of text usually longer than a single sentence. The only constrain is that the result with the two -"sentences" has a combined length of less than 512 tokens. - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by `[MASK]`. -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - -### Pretraining - -The model was trained on 8 16 GB V100 for 90 hours. See the -[training code](https://github.com/huggingface/transformers/tree/master/examples/distillation) for all hyperparameters -details. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | -|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:| -| | 82.2 | 88.5 | 89.2 | 91.3 | 51.3 | 85.8 | 87.5 | 59.9 | - - -### BibTeX entry and citation info - -```bibtex -@article{Sanh2019DistilBERTAD, - title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter}, - author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf}, - journal={ArXiv}, - year={2019}, - volume={abs/1910.01108} -} -``` - - - - diff --git a/model_cards/distilbert-base-uncased-distilled-squad-README.md b/model_cards/distilbert-base-uncased-distilled-squad-README.md deleted file mode 100644 index 6765229e62..0000000000 --- a/model_cards/distilbert-base-uncased-distilled-squad-README.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -language: en -datasets: -- squad -widget: -- text: "Which name is also used to describe the Amazon rainforest in English?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." -- text: "How many square kilometers of rainforest is covered in the basin?" - context: "The Amazon rainforest (Portuguese: Floresta Amazônica or Amazônia; Spanish: Selva Amazónica, Amazonía or usually Amazonia; French: Forêt amazonienne; Dutch: Amazoneregenwoud), also known in English as Amazonia or the Amazon Jungle, is a moist broadleaf forest that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 square kilometres (2,700,000 sq mi), of which 5,500,000 square kilometres (2,100,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations. The majority of the forest is contained within Brazil, with 60% of the rainforest, followed by Peru with 13%, Colombia with 10%, and with minor amounts in Venezuela, Ecuador, Bolivia, Guyana, Suriname and French Guiana. States or departments in four nations contain \"Amazonas\" in their names. The Amazon represents over half of the planet's remaining rainforests, and comprises the largest and most biodiverse tract of tropical rainforest in the world, with an estimated 390 billion individual trees divided into 16,000 species." -license: apache-2.0 ---- - -# DistilBERT base uncased distilled SQuAD - -This model is a fine-tune checkpoint of [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased), fine-tuned using (a second step of) knowledge distillation on SQuAD v1.1. -This model reaches a F1 score of 86.9 on the dev set (for comparison, Bert bert-base-uncased version reaches a F1 score of 88.5). diff --git a/model_cards/distilbert-base-uncased-finetuned-sst-2-english-README.md b/model_cards/distilbert-base-uncased-finetuned-sst-2-english-README.md deleted file mode 100644 index d33b586263..0000000000 --- a/model_cards/distilbert-base-uncased-finetuned-sst-2-english-README.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- sst-2 ---- - -# DistilBERT base uncased finetuned SST-2 - -This model is a fine-tune checkpoint of [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased), fine-tuned on SST-2. -This model reaches an accuracy of 91.3 on the dev set (for comparison, Bert bert-base-uncased version reaches an accuracy of 92.7). - -# Fine-tuning hyper-parameters - -- learning_rate = 1e-5 -- batch_size = 32 -- warmup = 600 -- max_seq_length = 128 -- num_train_epochs = 3.0 diff --git a/model_cards/distilgpt2-README.md b/model_cards/distilgpt2-README.md deleted file mode 100644 index 41e1a5a1e7..0000000000 --- a/model_cards/distilgpt2-README.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -language: en -tags: -- exbert - -license: apache-2.0 -datasets: -- openwebtext ---- - -# DistilGPT2 - -DistilGPT2 English language model pretrained with the supervision of [GPT2](https://huggingface.co/gpt2) (the smallest version of GPT2) on [OpenWebTextCorpus](https://skylion007.github.io/OpenWebTextCorpus/), a reproduction of OpenAI's WebText dataset. The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 124M parameters for GPT2). On average, DistilGPT2 is two times faster than GPT2. - -On the [WikiText-103](https://blog.einstein.ai/the-wikitext-long-term-dependency-language-modeling-dataset/) benchmark, GPT2 reaches a perplexity on the test set of 16.3 compared to 21.1 for DistilGPT2 (after fine-tuning on the train set). - -We encourage to check [GPT2](https://huggingface.co/gpt2) to know more about usage, limitations and potential biases. - - - - diff --git a/model_cards/distilroberta-base-README.md b/model_cards/distilroberta-base-README.md deleted file mode 100644 index 18bbbb8608..0000000000 --- a/model_cards/distilroberta-base-README.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -language: en -tags: -- exbert - -license: apache-2.0 -datasets: -- openwebtext ---- - -# DistilRoBERTa base model - -This model is a distilled version of the [RoBERTa-base model](https://huggingface.co/roberta-base). It follows the same training procedure as [DistilBERT](https://huggingface.co/distilbert-base-uncased). -The code for the distillation process can be found [here](https://github.com/huggingface/transformers/tree/master/examples/distillation). -This model is case-sensitive: it makes a difference between english and English. - -The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base). -On average DistilRoBERTa is twice as fast as Roberta-base. - -We encourage to check [RoBERTa-base model](https://huggingface.co/roberta-base) to know more about usage, limitations and potential biases. - -## Training data - -DistilRoBERTa was pre-trained on [OpenWebTextCorpus](https://skylion007.github.io/OpenWebTextCorpus/), a reproduction of OpenAI's WebText dataset (it is ~4 times less training data than the teacher RoBERTa). - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | -|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:| -| | 84.0 | 89.4 | 90.8 | 92.5 | 59.3 | 88.3 | 86.6 | 67.9 | - -### BibTeX entry and citation info - -```bibtex -@article{Sanh2019DistilBERTAD, - title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter}, - author={Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf}, - journal={ArXiv}, - year={2019}, - volume={abs/1910.01108} -} -``` - - - - diff --git a/model_cards/djstrong/bg_cs_pl_ru_cased_L-12_H-768_A-12/README.md b/model_cards/djstrong/bg_cs_pl_ru_cased_L-12_H-768_A-12/README.md deleted file mode 100644 index e29aab4e36..0000000000 --- a/model_cards/djstrong/bg_cs_pl_ru_cased_L-12_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -Slavic BERT from https://github.com/deepmipt/Slavic-BERT-NER http://files.deeppavlov.ai/deeppavlov_data/bg_cs_pl_ru_cased_L-12_H-768_A-12.tar.gz diff --git a/model_cards/dkleczek/bert-base-polish-cased-v1/README.md b/model_cards/dkleczek/bert-base-polish-cased-v1/README.md deleted file mode 100644 index 0d27bd5d7f..0000000000 --- a/model_cards/dkleczek/bert-base-polish-cased-v1/README.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -language: pl -thumbnail: https://raw.githubusercontent.com/kldarek/polbert/master/img/polbert.png ---- - -# Polbert - Polish BERT -Polish version of BERT language model is here! It is now available in two variants: cased and uncased, both can be downloaded and used via HuggingFace transformers library. I recommend using the cased model, more info on the differences and benchmark results below. - -![PolBERT image](https://raw.githubusercontent.com/kldarek/polbert/master/img/polbert.png) - -## Cased and uncased variants - -* I initially trained the uncased model, the corpus and training details are referenced below. Here are some issues I found after I published the uncased model: - * Some Polish characters and accents are not tokenized correctly through the BERT tokenizer when applying lowercase. This doesn't impact sequence classification much, but may influence token classfication tasks significantly. - * I noticed a lot of duplicates in the Open Subtitles dataset, which dominates the training corpus. - * I didn't use Whole Word Masking. -* The cased model improves on the uncased model in the following ways: - * All Polish characters and accents should now be tokenized correctly. - * I removed duplicates from Open Subtitles dataset. The corpus is smaller, but more balanced now. - * The model is trained with Whole Word Masking. - -## Pre-training corpora - -Below is the list of corpora used along with the output of `wc` command (counting lines, words and characters). These corpora were divided into sentences with srxsegmenter (see references), concatenated and tokenized with HuggingFace BERT Tokenizer. - -### Uncased - -| Tables | Lines | Words | Characters | -| ------------- |--------------:| -----:| -----:| -| [Polish subset of Open Subtitles](http://opus.nlpl.eu/OpenSubtitles-v2018.php) | 236635408| 1431199601 | 7628097730 | -| [Polish subset of ParaCrawl](http://opus.nlpl.eu/ParaCrawl.php) | 8470950 | 176670885 | 1163505275 | -| [Polish Parliamentary Corpus](http://clip.ipipan.waw.pl/PPC) | 9799859 | 121154785 | 938896963 | -| [Polish Wikipedia - Feb 2020](https://dumps.wikimedia.org/plwiki/latest/plwiki-latest-pages-articles.xml.bz2) | 8014206 | 132067986 | 1015849191 | -| Total | 262920423 | 1861093257 | 10746349159 | - -### Cased - -| Tables | Lines | Words | Characters | -| ------------- |--------------:| -----:| -----:| -| [Polish subset of Open Subtitles (Deduplicated) ](http://opus.nlpl.eu/OpenSubtitles-v2018.php) | 41998942| 213590656 | 1424873235 | -| [Polish subset of ParaCrawl](http://opus.nlpl.eu/ParaCrawl.php) | 8470950 | 176670885 | 1163505275 | -| [Polish Parliamentary Corpus](http://clip.ipipan.waw.pl/PPC) | 9799859 | 121154785 | 938896963 | -| [Polish Wikipedia - Feb 2020](https://dumps.wikimedia.org/plwiki/latest/plwiki-latest-pages-articles.xml.bz2) | 8014206 | 132067986 | 1015849191 | -| Total | 68283960 | 646479197 | 4543124667 | - - -## Pre-training details - -### Uncased - -* Polbert was trained with code provided in Google BERT's github repository (https://github.com/google-research/bert) -* Currently released model follows bert-base-uncased model architecture (12-layer, 768-hidden, 12-heads, 110M parameters) -* Training set-up: in total 1 million training steps: - * 100.000 steps - 128 sequence length, batch size 512, learning rate 1e-4 (10.000 steps warmup) - * 800.000 steps - 128 sequence length, batch size 512, learning rate 5e-5 - * 100.000 steps - 512 sequence length, batch size 256, learning rate 2e-5 -* The model was trained on a single Google Cloud TPU v3-8 - -### Cased - -* Same approach as uncased model, with the following differences: - * Whole Word Masking -* Training set-up: - * 100.000 steps - 128 sequence length, batch size 2048, learning rate 1e-4 (10.000 steps warmup) - * 100.000 steps - 128 sequence length, batch size 2048, learning rate 5e-5 - * 100.000 steps - 512 sequence length, batch size 256, learning rate 2e-5 - - -## Usage -Polbert is released via [HuggingFace Transformers library](https://huggingface.co/transformers/). - -For an example use as language model, see [this notebook](/LM_testing.ipynb) file. - -### Uncased - -```python -from transformers import * -model = BertForMaskedLM.from_pretrained("dkleczek/bert-base-polish-uncased-v1") -tokenizer = BertTokenizer.from_pretrained("dkleczek/bert-base-polish-uncased-v1") -nlp = pipeline('fill-mask', model=model, tokenizer=tokenizer) -for pred in nlp(f"Adam Mickiewicz wielkim polskim {nlp.tokenizer.mask_token} był."): - print(pred) -# Output: -# {'sequence': '[CLS] adam mickiewicz wielkim polskim poeta był. [SEP]', 'score': 0.47196975350379944, 'token': 26596} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim bohaterem był. [SEP]', 'score': 0.09127858281135559, 'token': 10953} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim człowiekiem był. [SEP]', 'score': 0.0647173821926117, 'token': 5182} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim pisarzem był. [SEP]', 'score': 0.05232388526201248, 'token': 24293} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim politykiem był. [SEP]', 'score': 0.04554257541894913, 'token': 44095} -``` - -### Cased - -```python -model = BertForMaskedLM.from_pretrained("dkleczek/bert-base-polish-cased-v1") -tokenizer = BertTokenizer.from_pretrained("dkleczek/bert-base-polish-cased-v1") -nlp = pipeline('fill-mask', model=model, tokenizer=tokenizer) -for pred in nlp(f"Adam Mickiewicz wielkim polskim {nlp.tokenizer.mask_token} był."): - print(pred) -# Output: -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim pisarzem był. [SEP]', 'score': 0.5391148328781128, 'token': 37120} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim człowiekiem był. [SEP]', 'score': 0.11683262139558792, 'token': 6810} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim bohaterem był. [SEP]', 'score': 0.06021466106176376, 'token': 17709} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim mistrzem był. [SEP]', 'score': 0.051870670169591904, 'token': 14652} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim artystą był. [SEP]', 'score': 0.031787533313035965, 'token': 35680} -``` - -See the next section for an example usage of Polbert in downstream tasks. - -## Evaluation -Thanks to Allegro, we now have the [KLEJ benchmark](https://klejbenchmark.com/leaderboard/), a set of nine evaluation tasks for the Polish language understanding. The following results are achieved by running standard set of evaluation scripts (no tricks!) utilizing both cased and uncased variants of Polbert. - -| Model | Average | NKJP-NER | CDSC-E | CDSC-R | CBD | PolEmo2.0-IN | PolEmo2.0-OUT | DYK | PSC | AR | -| ------------- |--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:| -| Polbert cased | 81.7 | 93.6 | 93.4 | 93.8 | 52.7 | 87.4 | 71.1 | 59.1 | 98.6 | 85.2 | -| Polbert uncased | 81.4 | 90.1 | 93.9 | 93.5 | 55.0 | 88.1 | 68.8 | 59.4 | 98.8 | 85.4 | - -Note how the uncased model performs better than cased on some tasks? My guess this is because of the oversampling of Open Subtitles dataset and its similarity to data in some of these tasks. All these benchmark tasks are sequence classification, so the relative strength of the cased model is not so visible here. - -## Bias -The data used to train the model is biased. It may reflect stereotypes related to gender, ethnicity etc. Please be careful when using the model for downstream task to consider these biases and mitigate them. - -## Acknowledgements -* I'd like to express my gratitude to Google [TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc) for providing the free TPU credits - thank you! -* Also appreciate the help from Timo Möller from [deepset](https://deepset.ai) for sharing tips and scripts based on their experience training German BERT model. -* Big thanks to Allegro for releasing KLEJ Benchmark and specifically to Piotr Rybak for help with the evaluation and pointing out some issues with the tokenization. -* Finally, thanks to Rachel Thomas, Jeremy Howard and Sylvain Gugger from [fastai](https://www.fast.ai) for their NLP and Deep Learning courses! - -## Author -Darek Kłeczek - contact me on Twitter [@dk21](https://twitter.com/dk21) - -## References -* https://github.com/google-research/bert -* https://github.com/narusemotoki/srx_segmenter -* SRX rules file for sentence splitting in Polish, written by Marcin Miłkowski: https://raw.githubusercontent.com/languagetool-org/languagetool/master/languagetool-core/src/main/resources/org/languagetool/resource/segment.srx -* [KLEJ benchmark](https://klejbenchmark.com/leaderboard/) diff --git a/model_cards/dkleczek/bert-base-polish-uncased-v1/README.md b/model_cards/dkleczek/bert-base-polish-uncased-v1/README.md deleted file mode 100644 index 103bc4a174..0000000000 --- a/model_cards/dkleczek/bert-base-polish-uncased-v1/README.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -language: pl -thumbnail: https://raw.githubusercontent.com/kldarek/polbert/master/img/polbert.png ---- - -# Polbert - Polish BERT -Polish version of BERT language model is here! It is now available in two variants: cased and uncased, both can be downloaded and used via HuggingFace transformers library. I recommend using the cased model, more info on the differences and benchmark results below. - -![PolBERT image](https://raw.githubusercontent.com/kldarek/polbert/master/img/polbert.png) - -## Cased and uncased variants - -* I initially trained the uncased model, the corpus and training details are referenced below. Here are some issues I found after I published the uncased model: - * Some Polish characters and accents are not tokenized correctly through the BERT tokenizer when applying lowercase. This doesn't impact sequence classification much, but may influence token classfication tasks significantly. - * I noticed a lot of duplicates in the Open Subtitles dataset, which dominates the training corpus. - * I didn't use Whole Word Masking. -* The cased model improves on the uncased model in the following ways: - * All Polish characters and accents should now be tokenized correctly. - * I removed duplicates from Open Subtitles dataset. The corpus is smaller, but more balanced now. - * The model is trained with Whole Word Masking. - -## Pre-training corpora - -Below is the list of corpora used along with the output of `wc` command (counting lines, words and characters). These corpora were divided into sentences with srxsegmenter (see references), concatenated and tokenized with HuggingFace BERT Tokenizer. - -### Uncased - -| Tables | Lines | Words | Characters | -| ------------- |--------------:| -----:| -----:| -| [Polish subset of Open Subtitles](http://opus.nlpl.eu/OpenSubtitles-v2018.php) | 236635408| 1431199601 | 7628097730 | -| [Polish subset of ParaCrawl](http://opus.nlpl.eu/ParaCrawl.php) | 8470950 | 176670885 | 1163505275 | -| [Polish Parliamentary Corpus](http://clip.ipipan.waw.pl/PPC) | 9799859 | 121154785 | 938896963 | -| [Polish Wikipedia - Feb 2020](https://dumps.wikimedia.org/plwiki/latest/plwiki-latest-pages-articles.xml.bz2) | 8014206 | 132067986 | 1015849191 | -| Total | 262920423 | 1861093257 | 10746349159 | - -### Cased - -| Tables | Lines | Words | Characters | -| ------------- |--------------:| -----:| -----:| -| [Polish subset of Open Subtitles (Deduplicated) ](http://opus.nlpl.eu/OpenSubtitles-v2018.php) | 41998942| 213590656 | 1424873235 | -| [Polish subset of ParaCrawl](http://opus.nlpl.eu/ParaCrawl.php) | 8470950 | 176670885 | 1163505275 | -| [Polish Parliamentary Corpus](http://clip.ipipan.waw.pl/PPC) | 9799859 | 121154785 | 938896963 | -| [Polish Wikipedia - Feb 2020](https://dumps.wikimedia.org/plwiki/latest/plwiki-latest-pages-articles.xml.bz2) | 8014206 | 132067986 | 1015849191 | -| Total | 68283960 | 646479197 | 4543124667 | - - -## Pre-training details - -### Uncased - -* Polbert was trained with code provided in Google BERT's github repository (https://github.com/google-research/bert) -* Currently released model follows bert-base-uncased model architecture (12-layer, 768-hidden, 12-heads, 110M parameters) -* Training set-up: in total 1 million training steps: - * 100.000 steps - 128 sequence length, batch size 512, learning rate 1e-4 (10.000 steps warmup) - * 800.000 steps - 128 sequence length, batch size 512, learning rate 5e-5 - * 100.000 steps - 512 sequence length, batch size 256, learning rate 2e-5 -* The model was trained on a single Google Cloud TPU v3-8 - -### Cased - -* Same approach as uncased model, with the following differences: - * Whole Word Masking -* Training set-up: - * 100.000 steps - 128 sequence length, batch size 2048, learning rate 1e-4 (10.000 steps warmup) - * 100.000 steps - 128 sequence length, batch size 2048, learning rate 5e-5 - * 100.000 steps - 512 sequence length, batch size 256, learning rate 2e-5 - - -## Usage -Polbert is released via [HuggingFace Transformers library](https://huggingface.co/transformers/). - -For an example use as language model, see [this notebook](/LM_testing.ipynb) file. - -### Uncased - -```python -from transformers import * -model = BertForMaskedLM.from_pretrained("dkleczek/bert-base-polish-uncased-v1") -tokenizer = BertTokenizer.from_pretrained("dkleczek/bert-base-polish-uncased-v1") -nlp = pipeline('fill-mask', model=model, tokenizer=tokenizer) -for pred in nlp(f"Adam Mickiewicz wielkim polskim {nlp.tokenizer.mask_token} był."): - print(pred) -# Output: -# {'sequence': '[CLS] adam mickiewicz wielkim polskim poeta był. [SEP]', 'score': 0.47196975350379944, 'token': 26596} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim bohaterem był. [SEP]', 'score': 0.09127858281135559, 'token': 10953} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim człowiekiem był. [SEP]', 'score': 0.0647173821926117, 'token': 5182} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim pisarzem był. [SEP]', 'score': 0.05232388526201248, 'token': 24293} -# {'sequence': '[CLS] adam mickiewicz wielkim polskim politykiem był. [SEP]', 'score': 0.04554257541894913, 'token': 44095} -``` - -### Cased - -```python -model = BertForMaskedLM.from_pretrained("dkleczek/bert-base-polish-cased-v1") -tokenizer = BertTokenizer.from_pretrained("dkleczek/bert-base-polish-cased-v1") -nlp = pipeline('fill-mask', model=model, tokenizer=tokenizer) -for pred in nlp(f"Adam Mickiewicz wielkim polskim {nlp.tokenizer.mask_token} był."): - print(pred) -# Output: -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim pisarzem był. [SEP]', 'score': 0.5391148328781128, 'token': 37120} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim człowiekiem był. [SEP]', 'score': 0.11683262139558792, 'token': 6810} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim bohaterem był. [SEP]', 'score': 0.06021466106176376, 'token': 17709} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim mistrzem był. [SEP]', 'score': 0.051870670169591904, 'token': 14652} -# {'sequence': '[CLS] Adam Mickiewicz wielkim polskim artystą był. [SEP]', 'score': 0.031787533313035965, 'token': 35680} -``` - -See the next section for an example usage of Polbert in downstream tasks. - -## Evaluation -Thanks to Allegro, we now have the [KLEJ benchmark](https://klejbenchmark.com/leaderboard/), a set of nine evaluation tasks for the Polish language understanding. The following results are achieved by running standard set of evaluation scripts (no tricks!) utilizing both cased and uncased variants of Polbert. - -| Model | Average | NKJP-NER | CDSC-E | CDSC-R | CBD | PolEmo2.0-IN | PolEmo2.0-OUT | DYK | PSC | AR | -| ------------- |--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:|--------------:| -| Polbert cased | 81.7 | 93.6 | 93.4 | 93.8 | 52.7 | 87.4 | 71.1 | 59.1 | 98.6 | 85.2 | -| Polbert uncased | 81.4 | 90.1 | 93.9 | 93.5 | 55.0 | 88.1 | 68.8 | 59.4 | 98.8 | 85.4 | - -Note how the uncased model performs better than cased on some tasks? My guess this is because of the oversampling of Open Subtitles dataset and its similarity to data in some of these tasks. All these benchmark tasks are sequence classification, so the relative strength of the cased model is not so visible here. - -## Bias -The data used to train the model is biased. It may reflect stereotypes related to gender, ethnicity etc. Please be careful when using the model for downstream task to consider these biases and mitigate them. - -## Acknowledgements -* I'd like to express my gratitude to Google [TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc) for providing the free TPU credits - thank you! -* Also appreciate the help from Timo Möller from [deepset](https://deepset.ai) for sharing tips and scripts based on their experience training German BERT model. -* Big thanks to Allegro for releasing KLEJ Benchmark and specifically to Piotr Rybak for help with the evaluation and pointing out some issues with the tokenization. -* Finally, thanks to Rachel Thomas, Jeremy Howard and Sylvain Gugger from [fastai](https://www.fast.ai) for their NLP and Deep Learning courses! - -## Author -Darek Kłeczek - contact me on Twitter [@dk21](https://twitter.com/dk21) - -## References -* https://github.com/google-research/bert -* https://github.com/narusemotoki/srx_segmenter -* SRX rules file for sentence splitting in Polish, written by Marcin Miłkowski: https://raw.githubusercontent.com/languagetool-org/languagetool/master/languagetool-core/src/main/resources/org/languagetool/resource/segment.srx -* [KLEJ benchmark](https://klejbenchmark.com/leaderboard/) \ No newline at end of file diff --git a/model_cards/dslim/bert-base-NER/README.md b/model_cards/dslim/bert-base-NER/README.md deleted file mode 100644 index d9e84583e0..0000000000 --- a/model_cards/dslim/bert-base-NER/README.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -language: en -datasets: -- conll2003 ---- -# bert-base-NER - -## Model description - -**bert-base-NER** is a fine-tuned BERT model that is ready to use for **Named Entity Recognition** and achieves **state-of-the-art performance** for the NER task. It has been trained to recognize four types of entities: location (LOC), organizations (ORG), person (PER) and Miscellaneous (MISC). - -Specifically, this model is a *bert-base-cased* model that was fine-tuned on the English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset. -## Intended uses & limitations - -#### How to use - -You can use this model with Transformers *pipeline* for NER. - -```python -from transformers import AutoTokenizer, AutoModelForTokenClassification -from transformers import pipeline - -tokenizer = AutoTokenizer.from_pretrained("dslim/bert-base-NER") -model = AutoModelForTokenClassification.from_pretrained("dslim/bert-base-NER") - -nlp = pipeline("ner", model=model, tokenizer=tokenizer) -example = "My name is Wolfgang and I live in Berlin" - -ner_results = nlp(example) -print(ner_results) -``` - -#### Limitations and bias - -This model is limited by its training dataset of entity-annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains. Furthermore, the model occassionally tags subword tokens as entities and post-processing of results may be necessary to handle those cases. - -## Training data - -This model was fine-tuned on English version of the standard [CoNLL-2003 Named Entity Recognition](https://www.aclweb.org/anthology/W03-0419.pdf) dataset. - -The training dataset distinguishes between the beginning and continuation of an entity so that if there are back-to-back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes: -Abbreviation|Description --|- -O|Outside of a named entity -B-MIS |Beginning of a miscellaneous entity right after another miscellaneous entity -I-MIS |Miscellaneous entity -B-PER |Beginning of a person’s name right after another person’s name -I-PER |Person’s name -B-ORG |Beginning of an organisation right after another organisation -I-ORG |Organisation -B-LOC |Beginning of a location right after another location -I-LOC |Location - - -### CoNLL-2003 English Dataset Statistics -This dataset was derived from the Reuters corpus which consists of Reuters news stories. You can read more about how this dataset was created in the CoNLL-2003 paper. -#### # of training examples per entity type -Dataset|LOC|MISC|ORG|PER --|-|-|-|- -Train|7140|3438|6321|6600 -Dev|1837|922|1341|1842 -Test|1668|702|1661|1617 -#### # of articles/sentences/tokens per dataset -Dataset |Articles |Sentences |Tokens --|-|-|- -Train |946 |14,987 |203,621 -Dev |216 |3,466 |51,362 -Test |231 |3,684 |46,435 - -## Training procedure - -This model was trained on a single NVIDIA V100 GPU with recommended hyperparameters from the [original BERT paper](https://arxiv.org/pdf/1810.04805) which trained & evaluated the model on CoNLL-2003 NER task. - -## Eval results -metric|dev|test --|-|- -f1 |95.1 |91.3 -precision |95.0 |90.7 -recall |95.3 |91.9 - -The test metrics are a little lower than the official Google BERT results which encoded document context & experimented with CRF. More on replicating the original results [here](https://github.com/google-research/bert/issues/223). - -### BibTeX entry and citation info - -``` -@article{DBLP:journals/corr/abs-1810-04805, - author = {Jacob Devlin and - Ming{-}Wei Chang and - Kenton Lee and - Kristina Toutanova}, - title = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language - Understanding}, - journal = {CoRR}, - volume = {abs/1810.04805}, - year = {2018}, - url = {http://arxiv.org/abs/1810.04805}, - archivePrefix = {arXiv}, - eprint = {1810.04805}, - timestamp = {Tue, 30 Oct 2018 20:39:56 +0100}, - biburl = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` -``` -@inproceedings{tjong-kim-sang-de-meulder-2003-introduction, - title = "Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition", - author = "Tjong Kim Sang, Erik F. and - De Meulder, Fien", - booktitle = "Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003", - year = "2003", - url = "https://www.aclweb.org/anthology/W03-0419", - pages = "142--147", -} -``` diff --git a/model_cards/dumitrescustefan/bert-base-romanian-cased-v1/README.md b/model_cards/dumitrescustefan/bert-base-romanian-cased-v1/README.md deleted file mode 100644 index bc159d2724..0000000000 --- a/model_cards/dumitrescustefan/bert-base-romanian-cased-v1/README.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -language: ro ---- - -# bert-base-romanian-cased-v1 - -The BERT **base**, **cased** model for Romanian, trained on a 15GB corpus, version ![v1.0](https://img.shields.io/badge/v1.0-21%20Apr%202020-ff6666) - -### How to use - -```python -from transformers import AutoTokenizer, AutoModel -import torch -# load tokenizer and model -tokenizer = AutoTokenizer.from_pretrained("dumitrescustefan/bert-base-romanian-cased-v1") -model = AutoModel.from_pretrained("dumitrescustefan/bert-base-romanian-cased-v1") -# tokenize a sentence and run through the model -input_ids = torch.tensor(tokenizer.encode("Acesta este un test.", add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids) -# get encoding -last_hidden_states = outputs[0] # The last hidden-state is the first element of the output tuple -``` - -### Evaluation - -Evaluation is performed on Universal Dependencies [Romanian RRT](https://universaldependencies.org/treebanks/ro_rrt/index.html) UPOS, XPOS and LAS, and on a NER task based on [RONEC](https://github.com/dumitrescustefan/ronec). Details, as well as more in-depth tests not shown here, are given in the dedicated [evaluation page](https://github.com/dumitrescustefan/Romanian-Transformers/tree/master/evaluation/README.md). - -The baseline is the [Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) model ``bert-base-multilingual-(un)cased``, as at the time of writing it was the only available BERT model that works on Romanian. - -| Model | UPOS | XPOS | NER | LAS | -|--------------------------------|:-----:|:------:|:-----:|:-----:| -| bert-base-multilingual-cased | 97.87 | 96.16 | 84.13 | 88.04 | -| bert-base-romanian-cased-v1 | **98.00** | **96.46** | **85.88** | **89.69** | - -### Corpus - -The model is trained on the following corpora (stats in the table below are after cleaning): - -| Corpus | Lines(M) | Words(M) | Chars(B) | Size(GB) | -|----------- |:--------: |:--------: |:--------: |:--------: | -| OPUS | 55.05 | 635.04 | 4.045 | 3.8 | -| OSCAR | 33.56 | 1725.82 | 11.411 | 11 | -| Wikipedia | 1.54 | 60.47 | 0.411 | 0.4 | -| **Total** | **90.15** | **2421.33** | **15.867** | **15.2** | - -#### Acknowledgements - -- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome! diff --git a/model_cards/dumitrescustefan/bert-base-romanian-uncased-v1/README.md b/model_cards/dumitrescustefan/bert-base-romanian-uncased-v1/README.md deleted file mode 100644 index 878e4530ad..0000000000 --- a/model_cards/dumitrescustefan/bert-base-romanian-uncased-v1/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: ro ---- - -# bert-base-romanian-uncased-v1 - -The BERT **base**, **uncased** model for Romanian, trained on a 15GB corpus, version ![v1.0](https://img.shields.io/badge/v1.0-21%20Apr%202020-ff6666) - -### How to use - -```python -from transformers import AutoTokenizer, AutoModel -import torch - -# load tokenizer and model -tokenizer = AutoTokenizer.from_pretrained("dumitrescustefan/bert-base-romanian-uncased-v1", do_lower_case=True) -model = AutoModel.from_pretrained("dumitrescustefan/bert-base-romanian-uncased-v1") - -# tokenize a sentence and run through the model -input_ids = torch.tensor(tokenizer.encode("Acesta este un test.", add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids) - -# get encoding -last_hidden_states = outputs[0] # The last hidden-state is the first element of the output tuple -``` - -### Evaluation - -Evaluation is performed on Universal Dependencies [Romanian RRT](https://universaldependencies.org/treebanks/ro_rrt/index.html) UPOS, XPOS and LAS, and on a NER task based on [RONEC](https://github.com/dumitrescustefan/ronec). Details, as well as more in-depth tests not shown here, are given in the dedicated [evaluation page](https://github.com/dumitrescustefan/Romanian-Transformers/tree/master/evaluation/README.md). - -The baseline is the [Multilingual BERT](https://github.com/google-research/bert/blob/master/multilingual.md) model ``bert-base-multilingual-(un)cased``, as at the time of writing it was the only available BERT model that works on Romanian. - -| Model | UPOS | XPOS | NER | LAS | -|--------------------------------|:-----:|:------:|:-----:|:-----:| -| bert-base-multilingual-uncased | 97.65 | 95.72 | 83.91 | 87.65 | -| bert-base-romanian-uncased-v1 | **98.18** | **96.84** | **85.26** | **89.61** | - -### Corpus - -The model is trained on the following corpora (stats in the table below are after cleaning): - -| Corpus | Lines(M) | Words(M) | Chars(B) | Size(GB) | -|----------- |:--------: |:--------: |:--------: |:--------: | -| OPUS | 55.05 | 635.04 | 4.045 | 3.8 | -| OSCAR | 33.56 | 1725.82 | 11.411 | 11 | -| Wikipedia | 1.54 | 60.47 | 0.411 | 0.4 | -| **Total** | **90.15** | **2421.33** | **15.867** | **15.2** | - -#### Acknowledgements - -- We'd like to thank [Sampo Pyysalo](https://github.com/spyysalo) from TurkuNLP for helping us out with the compute needed to pretrain the v1.0 BERT models. He's awesome! diff --git a/model_cards/e-tony/gpt2-rnm/README.md b/model_cards/e-tony/gpt2-rnm/README.md deleted file mode 100644 index 1804e2e88e..0000000000 --- a/model_cards/e-tony/gpt2-rnm/README.md +++ /dev/null @@ -1,20 +0,0 @@ -### How to use - -You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we -set a seed for reproducibility: - -```python ->>> from transformers import pipeline, set_seed ->>> generator = pipeline('text-generation', model='e-tony/gpt2-rnm') ->>> set_seed(42) ->>> generator("Rick: I turned myself into a pickle, Morty!\nMorty: ", max_length=50, num_return_sequences=5) - -[{'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty: I didn't want to have children. It was my fate! I'll pay my mom and dad.\nSnuffles: Well, at least we"}, - {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty: you know what happened?\n(Steven begins dragging people down the toilet with his hand. As Steven falls) The whole thing starts.\nA man approaches Steven"}, - {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty: Oh wait! And do you remember what I did to you?\nJerry: Uh, it didn't hurt. It should have hurt a lot since I"}, - {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty: Rick!\nKraven: Wait! [wary gasp] What the hell are you doing this time?!\nJerry: Hey, are you"}, - {'generated_text': "Rick: I turned myself into a pickle, Morty!\nMorty: Uh.\nJerry: You don't have to put your finger on me today, do you?\nRick: It's just, what do you"}] -``` - -### Training data -We used the original `gpt2` model and fine-tuned it on [Rick and Morty transcripts](https://rickandmorty.fandom.com/wiki/Category:Transcripts). diff --git a/model_cards/elgeish/cs224n-squad2.0-albert-base-v2/README.md b/model_cards/elgeish/cs224n-squad2.0-albert-base-v2/README.md deleted file mode 100644 index 4e59ee06c4..0000000000 --- a/model_cards/elgeish/cs224n-squad2.0-albert-base-v2/README.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -tags: -- exbert ---- - -## CS224n SQuAD2.0 Project Dataset -The goal of this model is to save CS224n students GPU time when establishing -baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf). -The training set used to fine-tune this model is the same as -the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however, -evaluation and model selection were performed using roughly half of the official -dev set, 6078 examples, picked at random. The data files can be found at - — this is the Winter 2020 -version. Given that the official SQuAD2.0 dev set contains the project's test -set, students must make sure not to use the official SQuAD2.0 dev set in any way -— including the use of models fine-tuned on the official SQuAD2.0, since they -used the official SQuAD2.0 dev set for model selection. - - - - - -## Results -```json -{ - "exact": 78.94044093451794, - "f1": 81.7724930324639, - "total": 6078, - "HasAns_exact": 76.28865979381443, - "HasAns_f1": 82.20385314478195, - "HasAns_total": 2910, - "NoAns_exact": 81.37626262626263, - "NoAns_f1": 81.37626262626263, - "NoAns_total": 3168, - "best_exact": 78.95689371503784, - "best_exact_thresh": 0.0, - "best_f1": 81.78894581298378, - "best_f1_thresh": 0.0 -} -``` - -## Notable Arguments -```json -{ - "do_lower_case": true, - "doc_stride": 128, - "fp16": false, - "fp16_opt_level": "O1", - "gradient_accumulation_steps": 24, - "learning_rate": 3e-05, - "max_answer_length": 30, - "max_grad_norm": 1, - "max_query_length": 64, - "max_seq_length": 384, - "model_name_or_path": "albert-base-v2", - "model_type": "albert", - "num_train_epochs": 3, - "per_gpu_train_batch_size": 8, - "save_steps": 5000, - "seed": 42, - "train_batch_size": 8, - "version_2_with_negative": true, - "warmup_steps": 0, - "weight_decay": 0 -} -``` - -## Environment Setup -```json -{ - "transformers": "2.5.1", - "pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0", - "python": "3.6.5=hc3d631a_2", - "os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux", - "gpu": "Tesla V100-SXM2-16GB" -} -``` - -## How to Cite -```BibTeX -@misc{elgeish2020gestalt, - title={Gestalt: a Stacking Ensemble for SQuAD2.0}, - author={Mohamed El-Geish}, - journal={arXiv e-prints}, - archivePrefix={arXiv}, - eprint={2004.07067}, - year={2020}, -} -``` - -## Related Models -* [elgeish/cs224n-squad2.0-albert-large-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-large-v2) -* [elgeish/cs224n-squad2.0-albert-xxlarge-v1](https://huggingface.co/elgeish/cs224n-squad2.0-albert-xxlarge-v1) -* [elgeish/cs224n-squad2.0-distilbert-base-uncased](https://huggingface.co/elgeish/cs224n-squad2.0-distilbert-base-uncased) -* [elgeish/cs224n-squad2.0-roberta-base](https://huggingface.co/elgeish/cs224n-squad2.0-roberta-base) diff --git a/model_cards/elgeish/cs224n-squad2.0-albert-large-v2/README.md b/model_cards/elgeish/cs224n-squad2.0-albert-large-v2/README.md deleted file mode 100644 index 5f365d2d7b..0000000000 --- a/model_cards/elgeish/cs224n-squad2.0-albert-large-v2/README.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -tags: -- exbert ---- - -## CS224n SQuAD2.0 Project Dataset -The goal of this model is to save CS224n students GPU time when establishing -baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf). -The training set used to fine-tune this model is the same as -the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however, -evaluation and model selection were performed using roughly half of the official -dev set, 6078 examples, picked at random. The data files can be found at - — this is the Winter 2020 -version. Given that the official SQuAD2.0 dev set contains the project's test -set, students must make sure not to use the official SQuAD2.0 dev set in any way -— including the use of models fine-tuned on the official SQuAD2.0, since they -used the official SQuAD2.0 dev set for model selection. - - - - - -## Results -```json -{ - "exact": 79.2694965449161, - "f1": 82.50844352970152, - "total": 6078, - "HasAns_exact": 74.87972508591065, - "HasAns_f1": 81.64478342732858, - "HasAns_total": 2910, - "NoAns_exact": 83.30176767676768, - "NoAns_f1": 83.30176767676768, - "NoAns_total": 3168, - "best_exact": 79.2694965449161, - "best_exact_thresh": 0.0, - "best_f1": 82.50844352970155, - "best_f1_thresh": 0.0 -} -``` - -## Notable Arguments -```json -{ - "do_lower_case": true, - "doc_stride": 128, - "fp16": false, - "fp16_opt_level": "O1", - "gradient_accumulation_steps": 1, - "learning_rate": 3e-05, - "max_answer_length": 30, - "max_grad_norm": 1, - "max_query_length": 64, - "max_seq_length": 384, - "model_name_or_path": "albert-large-v2", - "model_type": "albert", - "num_train_epochs": 5, - "per_gpu_train_batch_size": 8, - "save_steps": 5000, - "seed": 42, - "train_batch_size": 8, - "version_2_with_negative": true, - "warmup_steps": 0, - "weight_decay": 0 -} -``` - -## Environment Setup -```json -{ - "transformers": "2.5.1", - "pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0", - "python": "3.6.5=hc3d631a_2", - "os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux", - "gpu": "Tesla V100-SXM2-16GB" -} -``` - -## How to Cite -```BibTeX -@misc{elgeish2020gestalt, - title={Gestalt: a Stacking Ensemble for SQuAD2.0}, - author={Mohamed El-Geish}, - journal={arXiv e-prints}, - archivePrefix={arXiv}, - eprint={2004.07067}, - year={2020}, -} -``` - -## Related Models -* [elgeish/cs224n-squad2.0-albert-base-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-base-v2) -* [elgeish/cs224n-squad2.0-albert-xxlarge-v1](https://huggingface.co/elgeish/cs224n-squad2.0-albert-xxlarge-v1) -* [elgeish/cs224n-squad2.0-distilbert-base-uncased](https://huggingface.co/elgeish/cs224n-squad2.0-distilbert-base-uncased) -* [elgeish/cs224n-squad2.0-roberta-base](https://huggingface.co/elgeish/cs224n-squad2.0-roberta-base) diff --git a/model_cards/elgeish/cs224n-squad2.0-albert-xxlarge-v1/README.md b/model_cards/elgeish/cs224n-squad2.0-albert-xxlarge-v1/README.md deleted file mode 100644 index 8856e7cc80..0000000000 --- a/model_cards/elgeish/cs224n-squad2.0-albert-xxlarge-v1/README.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -tags: -- exbert ---- - -## CS224n SQuAD2.0 Project Dataset -The goal of this model is to save CS224n students GPU time when establishing -baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf). -The training set used to fine-tune this model is the same as -the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however, -evaluation and model selection were performed using roughly half of the official -dev set, 6078 examples, picked at random. The data files can be found at - — this is the Winter 2020 -version. Given that the official SQuAD2.0 dev set contains the project's test -set, students must make sure not to use the official SQuAD2.0 dev set in any way -— including the use of models fine-tuned on the official SQuAD2.0, since they -used the official SQuAD2.0 dev set for model selection. - - - - - -## Results -```json -{ - "exact": 85.93287265547877, - "f1": 88.91258331187983, - "total": 6078, - "HasAns_exact": 84.36426116838489, - "HasAns_f1": 90.58786301361013, - "HasAns_total": 2910, - "NoAns_exact": 87.37373737373737, - "NoAns_f1": 87.37373737373737, - "NoAns_total": 3168, - "best_exact": 85.93287265547877, - "best_exact_thresh": 0.0, - "best_f1": 88.91258331187993, - "best_f1_thresh": 0.0 -} -``` - -## Notable Arguments -```json -{ - "do_lower_case": true, - "doc_stride": 128, - "fp16": false, - "fp16_opt_level": "O1", - "gradient_accumulation_steps": 24, - "learning_rate": 3e-05, - "max_answer_length": 30, - "max_grad_norm": 1, - "max_query_length": 64, - "max_seq_length": 512, - "model_name_or_path": "albert-xxlarge-v1", - "model_type": "albert", - "num_train_epochs": 4, - "per_gpu_train_batch_size": 1, - "save_steps": 1000, - "seed": 42, - "train_batch_size": 1, - "version_2_with_negative": true, - "warmup_steps": 814, - "weight_decay": 0 -} -``` - -## Environment Setup -```json -{ - "transformers": "2.5.1", - "pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0", - "python": "3.6.5=hc3d631a_2", - "os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux", - "gpu": "Tesla V100-SXM2-16GB" -} -``` - -## How to Cite -```BibTeX -@misc{elgeish2020gestalt, - title={Gestalt: a Stacking Ensemble for SQuAD2.0}, - author={Mohamed El-Geish}, - journal={arXiv e-prints}, - archivePrefix={arXiv}, - eprint={2004.07067}, - year={2020}, -} -``` - -## Related Models -* [elgeish/cs224n-squad2.0-albert-base-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-base-v2) -* [elgeish/cs224n-squad2.0-albert-large-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-large-v2) -* [elgeish/cs224n-squad2.0-distilbert-base-uncased](https://huggingface.co/elgeish/cs224n-squad2.0-distilbert-base-uncased) -* [elgeish/cs224n-squad2.0-roberta-base](https://huggingface.co/elgeish/cs224n-squad2.0-roberta-base) diff --git a/model_cards/elgeish/cs224n-squad2.0-distilbert-base-uncased/README.md b/model_cards/elgeish/cs224n-squad2.0-distilbert-base-uncased/README.md deleted file mode 100644 index 1bf3ab0781..0000000000 --- a/model_cards/elgeish/cs224n-squad2.0-distilbert-base-uncased/README.md +++ /dev/null @@ -1,86 +0,0 @@ -## CS224n SQuAD2.0 Project Dataset -The goal of this model is to save CS224n students GPU time when establishing -baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf). -The training set used to fine-tune this model is the same as -the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however, -evaluation and model selection were performed using roughly half of the official -dev set, 6078 examples, picked at random. The data files can be found at - — this is the Winter 2020 -version. Given that the official SQuAD2.0 dev set contains the project's test -set, students must make sure not to use the official SQuAD2.0 dev set in any way -— including the use of models fine-tuned on the official SQuAD2.0, since they -used the official SQuAD2.0 dev set for model selection. - -## Results -```json -{ - "exact": 65.16946363935504, - "f1": 67.87348075352251, - "total": 6078, - "HasAns_exact": 69.51890034364261, - "HasAns_f1": 75.16667217179045, - "HasAns_total": 2910, - "NoAns_exact": 61.17424242424242, - "NoAns_f1": 61.17424242424242, - "NoAns_total": 3168, - "best_exact": 65.16946363935504, - "best_exact_thresh": 0.0, - "best_f1": 67.87348075352243, - "best_f1_thresh": 0.0 -} -``` - -## Notable Arguments -```json -{ - "do_lower_case": true, - "doc_stride": 128, - "fp16": false, - "fp16_opt_level": "O1", - "gradient_accumulation_steps": 24, - "learning_rate": 3e-05, - "max_answer_length": 30, - "max_grad_norm": 1, - "max_query_length": 64, - "max_seq_length": 384, - "model_name_or_path": "distilbert-base-uncased-distilled-squad", - "model_type": "distilbert", - "num_train_epochs": 4, - "per_gpu_train_batch_size": 32, - "save_steps": 5000, - "seed": 42, - "train_batch_size": 32, - "version_2_with_negative": true, - "warmup_steps": 0, - "weight_decay": 0 -} -``` - -## Environment Setup -```json -{ - "transformers": "2.5.1", - "pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0", - "python": "3.6.5=hc3d631a_2", - "os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux", - "gpu": "Tesla V100-SXM2-16GB" -} -``` - -## How to Cite -```BibTeX -@misc{elgeish2020gestalt, - title={Gestalt: a Stacking Ensemble for SQuAD2.0}, - author={Mohamed El-Geish}, - journal={arXiv e-prints}, - archivePrefix={arXiv}, - eprint={2004.07067}, - year={2020}, -} -``` - -## Related Models -* [elgeish/cs224n-squad2.0-albert-base-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-base-v2) -* [elgeish/cs224n-squad2.0-albert-large-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-large-v2) -* [elgeish/cs224n-squad2.0-albert-xxlarge-v1](https://huggingface.co/elgeish/cs224n-squad2.0-albert-xxlarge-v1) -* [elgeish/cs224n-squad2.0-roberta-base](https://huggingface.co/elgeish/cs224n-squad2.0-roberta-base) diff --git a/model_cards/elgeish/cs224n-squad2.0-roberta-base/README.md b/model_cards/elgeish/cs224n-squad2.0-roberta-base/README.md deleted file mode 100644 index 1cef39c264..0000000000 --- a/model_cards/elgeish/cs224n-squad2.0-roberta-base/README.md +++ /dev/null @@ -1,86 +0,0 @@ -## CS224n SQuAD2.0 Project Dataset -The goal of this model is to save CS224n students GPU time when establishing -baselines to beat for the [Default Final Project](http://web.stanford.edu/class/cs224n/project/default-final-project-handout.pdf). -The training set used to fine-tune this model is the same as -the [official one](https://rajpurkar.github.io/SQuAD-explorer/); however, -evaluation and model selection were performed using roughly half of the official -dev set, 6078 examples, picked at random. The data files can be found at - — this is the Winter 2020 -version. Given that the official SQuAD2.0 dev set contains the project's test -set, students must make sure not to use the official SQuAD2.0 dev set in any way -— including the use of models fine-tuned on the official SQuAD2.0, since they -used the official SQuAD2.0 dev set for model selection. - -## Results -```json -{ - "exact": 75.32082922013821, - "f1": 78.66699523704254, - "total": 6078, - "HasAns_exact": 74.84536082474227, - "HasAns_f1": 81.83436324767868, - "HasAns_total": 2910, - "NoAns_exact": 75.75757575757575, - "NoAns_f1": 75.75757575757575, - "NoAns_total": 3168, - "best_exact": 75.32082922013821, - "best_exact_thresh": 0.0, - "best_f1": 78.66699523704266, - "best_f1_thresh": 0.0 -} -``` - -## Notable Arguments -```json -{ - "do_lower_case": true, - "doc_stride": 128, - "fp16": false, - "fp16_opt_level": "O1", - "gradient_accumulation_steps": 24, - "learning_rate": 3e-05, - "max_answer_length": 30, - "max_grad_norm": 1, - "max_query_length": 64, - "max_seq_length": 384, - "model_name_or_path": "roberta-base", - "model_type": "roberta", - "num_train_epochs": 4, - "per_gpu_train_batch_size": 16, - "save_steps": 5000, - "seed": 42, - "train_batch_size": 16, - "version_2_with_negative": true, - "warmup_steps": 0, - "weight_decay": 0 -} -``` - -## Environment Setup -```json -{ - "transformers": "2.5.1", - "pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0", - "python": "3.6.5=hc3d631a_2", - "os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux", - "gpu": "Tesla V100-SXM2-16GB" -} -``` - -## How to Cite -```BibTeX -@misc{elgeish2020gestalt, - title={Gestalt: a Stacking Ensemble for SQuAD2.0}, - author={Mohamed El-Geish}, - journal={arXiv e-prints}, - archivePrefix={arXiv}, - eprint={2004.07067}, - year={2020}, -} -``` - -## Related Models -* [elgeish/cs224n-squad2.0-albert-base-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-base-v2) -* [elgeish/cs224n-squad2.0-albert-large-v2](https://huggingface.co/elgeish/cs224n-squad2.0-albert-large-v2) -* [elgeish/cs224n-squad2.0-albert-xxlarge-v1](https://huggingface.co/elgeish/cs224n-squad2.0-albert-xxlarge-v1) -* [elgeish/cs224n-squad2.0-distilbert-base-uncased](https://huggingface.co/elgeish/cs224n-squad2.0-distilbert-base-uncased) diff --git a/model_cards/emilyalsentzer/Bio_ClinicalBERT/README.md b/model_cards/emilyalsentzer/Bio_ClinicalBERT/README.md deleted file mode 100644 index af55937a09..0000000000 --- a/model_cards/emilyalsentzer/Bio_ClinicalBERT/README.md +++ /dev/null @@ -1,39 +0,0 @@ - -# ClinicalBERT - Bio + Clinical BERT Model - -The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries. - -This model card describes the Bio+Clinical BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on all MIMIC notes. - -## Pretraining Data -The `Bio_ClinicalBERT` model was trained on all notes from [MIMIC III](https://www.nature.com/articles/sdata201635), a database containing electronic health records from ICU patients at the Beth Israel Hospital in Boston, MA. For more details on MIMIC, see [here](https://mimic.physionet.org/). All notes from the `NOTEEVENTS` table were included (~880M words). - -## Model Pretraining - -### Note Preprocessing -Each note in MIMIC was first split into sections using a rules-based section splitter (e.g. discharge summary notes were split into "History of Present Illness", "Family History", "Brief Hospital Course", etc. sections). Then each section was split into sentences using SciSpacy (`en core sci md` tokenizer). - -### Pretraining Procedures -The model was trained using code from [Google's BERT repository](https://github.com/google-research/bert) on a GeForce GTX TITAN X 12 GB GPU. Model parameters were initialized with BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`). - -### Pretraining Hyperparameters -We used a batch size of 32, a maximum sequence length of 128, and a learning rate of 5 · 10−5 for pre-training our models. The models trained on all MIMIC notes were trained for 150,000 steps. The dup factor for duplicating input data with different masks was set to 5. All other default parameters were used (specifically, masked language model probability = 0.15 -and max predictions per sequence = 20). - -## How to use the model - -Load the model via the transformers library: -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("emilyalsentzer/Bio_ClinicalBERT") -model = AutoModel.from_pretrained("emilyalsentzer/Bio_ClinicalBERT") -``` - -## More Information - -Refer to the original paper, [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) (NAACL Clinical NLP Workshop 2019) for additional details and performance on NLI and NER tasks. - -## Questions? - -Post a Github issue on the [clinicalBERT repo](https://github.com/EmilyAlsentzer/clinicalBERT) or email emilya@mit.edu with any questions. - diff --git a/model_cards/emilyalsentzer/Bio_Discharge_Summary_BERT/README.md b/model_cards/emilyalsentzer/Bio_Discharge_Summary_BERT/README.md deleted file mode 100644 index 162716e72f..0000000000 --- a/model_cards/emilyalsentzer/Bio_Discharge_Summary_BERT/README.md +++ /dev/null @@ -1,39 +0,0 @@ - -# ClinicalBERT - Bio + Discharge Summary BERT Model - -The [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) paper contains four unique clinicalBERT models: initialized with BERT-Base (`cased_L-12_H-768_A-12`) or BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`) & trained on either all MIMIC notes or only discharge summaries. - -This model card describes the Bio+Discharge Summary BERT model, which was initialized from [BioBERT](https://arxiv.org/abs/1901.08746) & trained on only discharge summaries from MIMIC. - -## Pretraining Data -The `Bio_Discharge_Summary_BERT` model was trained on all discharge summaries from [MIMIC III](https://www.nature.com/articles/sdata201635), a database containing electronic health records from ICU patients at the Beth Israel Hospital in Boston, MA. For more details on MIMIC, see [here](https://mimic.physionet.org/). All notes from the `NOTEEVENTS` table were included (~880M words). - -## Model Pretraining - -### Note Preprocessing -Each note in MIMIC was first split into sections using a rules-based section splitter (e.g. discharge summary notes were split into "History of Present Illness", "Family History", "Brief Hospital Course", etc. sections). Then each section was split into sentences using SciSpacy (`en core sci md` tokenizer). - -### Pretraining Procedures -The model was trained using code from [Google's BERT repository](https://github.com/google-research/bert) on a GeForce GTX TITAN X 12 GB GPU. Model parameters were initialized with BioBERT (`BioBERT-Base v1.0 + PubMed 200K + PMC 270K`). - -### Pretraining Hyperparameters -We used a batch size of 32, a maximum sequence length of 128, and a learning rate of 5 · 10−5 for pre-training our models. The models trained on all MIMIC notes were trained for 150,000 steps. The dup factor for duplicating input data with different masks was set to 5. All other default parameters were used (specifically, masked language model probability = 0.15 -and max predictions per sequence = 20). - -## How to use the model - -Load the model via the transformers library: -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("emilyalsentzer/Bio_ClinicalBERT") -model = AutoModel.from_pretrained("emilyalsentzer/Bio_ClinicalBERT") -``` - -## More Information - -Refer to the original paper, [Publicly Available Clinical BERT Embeddings](https://arxiv.org/abs/1904.03323) (NAACL Clinical NLP Workshop 2019) for additional details and performance on NLI and NER tasks. - -## Questions? - -Post a Github issue on the [clinicalBERT repo](https://github.com/EmilyAlsentzer/clinicalBERT) or email emilya@mit.edu with any questions. - diff --git a/model_cards/etalab-ia/camembert-base-squadFR-fquad-piaf/README.md b/model_cards/etalab-ia/camembert-base-squadFR-fquad-piaf/README.md deleted file mode 100644 index dc77396e1f..0000000000 --- a/model_cards/etalab-ia/camembert-base-squadFR-fquad-piaf/README.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -language: fr -datasets: -- piaf -- FQuAD -- SQuAD-FR -widget: -- text: "Comment s'appelle le portail open data du gouvernement ?" - context: "Etalab est une administration publique française qui fait notamment office de Chief Data Officer de l'État et coordonne la conception et la mise en œuvre de sa stratégie dans le domaine de la donnée (ouverture et partage des données publiques ou open data, exploitation des données et intelligence artificielle...). Ainsi, Etalab développe et maintient le portail des données ouvertes du gouvernement français data.gouv.fr. -Etalab promeut également une plus grande ouverture l'administration sur la société (gouvernement ouvert) : transparence de l'action publique, innovation ouverte, participation citoyenne... elle promeut l’innovation, l’expérimentation, les méthodes de travail ouvertes, agiles et itératives, ainsi que les synergies avec la société civile pour décloisonner l’administration et favoriser l’adoption des meilleures pratiques professionnelles dans le domaine du numérique. À ce titre elle étudie notamment l’opportunité de recourir à des technologies en voie de maturation issues du monde de la recherche. -Cette entité chargée de l'innovation au sein de l'administration doit contribuer à l'amélioration du service public grâce au numérique. Elle est rattachée à la Direction interministérielle du numérique, dont les missions et l’organisation ont été fixées par le décret du 30 octobre 2019.  Dirigé par Laure Lucchesi depuis 2016, elle rassemble une équipe pluridisciplinaire d'une trentaine de personnes." ---- - -# camembert-base-squadFR-fquad-piaf - -## Description - -Question-answering French model, using base [CamemBERT](https://camembert-model.fr/) fine-tuned on a combo of three French Q&A datasets: - -1. [PIAFv1.1](https://www.data.gouv.fr/en/datasets/piaf-le-dataset-francophone-de-questions-reponses/) -2. [FQuADv1.0](https://fquad.illuin.tech/) -3. [SQuAD-FR (SQuAD automatically translated to French)](https://github.com/Alikabbadj/French-SQuAD) - -## Training hyperparameters - -```shell -python run_squad.py \ ---model_type camembert \ ---model_name_or_path camembert-base \ ---do_train --do_eval \ ---train_file data/SQuAD+fquad+piaf.json \ ---predict_file data/fquad_valid.json \ ---per_gpu_train_batch_size 12 \ ---learning_rate 3e-5 \ ---num_train_epochs 4 \ ---max_seq_length 384 \ ---doc_stride 128 \ ---save_steps 10000 -``` - -## Evaluation results -### FQuAD v1.0 Evaluation -```shell -{"f1": 79.81, "exact_match": 55.14} -``` -### SQuAD-FR Evaluation -```shell -{"f1": 80.61, "exact_match": 59.54} -``` - -## Usage - -```python -from transformers import pipeline - -nlp = pipeline('question-answering', model='etalab-ia/camembert-base-squadFR-fquad-piaf', tokenizer='etalab-ia/camembert-base-squadFR-fquad-piaf') - -nlp({ - 'question': "Qui est Claude Monet?", - 'context': "Claude Monet, né le 14 novembre 1840 à Paris et mort le 5 décembre 1926 à Giverny, est un peintre français et l’un des fondateurs de l'impressionnisme." -}) -``` - -## Citation - -### PIAF -``` -@inproceedings{KeraronLBAMSSS20, - author = {Rachel Keraron and - Guillaume Lancrenon and - Mathilde Bras and - Fr{\'{e}}d{\'{e}}ric Allary and - Gilles Moyse and - Thomas Scialom and - Edmundo{-}Pavel Soriano{-}Morales and - Jacopo Staiano}, - title = {Project {PIAF:} Building a Native French Question-Answering Dataset}, - booktitle = {{LREC}}, - pages = {5481--5490}, - publisher = {European Language Resources Association}, - year = {2020} -} - -``` - -### FQuAD -``` -@article{dHoffschmidt2020FQuADFQ, - title={FQuAD: French Question Answering Dataset}, - author={Martin d'Hoffschmidt and Maxime Vidal and Wacim Belblidia and Tom Brendl'e and Quentin Heinrich}, - journal={ArXiv}, - year={2020}, - volume={abs/2002.06071} -} -``` - -### SQuAD-FR -``` - @MISC{kabbadj2018, - author = "Kabbadj, Ali", - title = "Something new in French Text Mining and Information Extraction (Universal Chatbot): Largest Q&A French training dataset (110 000+) ", - editor = "linkedin.com", - month = "November", - year = "2018", - url = "\url{https://www.linkedin.com/pulse/something-new-french-text-mining-information-chatbot-largest-kabbadj/}", - note = "[Online; posted 11-November-2018]", - } - ``` diff --git a/model_cards/ethanyt/guwenbert-base/README.md b/model_cards/ethanyt/guwenbert-base/README.md deleted file mode 100644 index 652785bc54..0000000000 --- a/model_cards/ethanyt/guwenbert-base/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: -- "zh" -thumbnail: "https://user-images.githubusercontent.com/9592150/97142000-cad08e00-179a-11eb-88df-aff9221482d8.png" -tags: -- "chinese" -- "classical chinese" -- "literary chinese" -- "ancient chinese" -- "bert" -- "pytorch" -license: "apache-2.0" -pipeline_tag: "fill-mask" -widget: -- text: "[MASK]太元中，武陵人捕鱼为业。" -- text: "问征夫以前路，恨晨光之[MASK]微。" -- text: "浔阳江头夜送客，枫叶[MASK]花秋瑟瑟。" ---- - -# GuwenBERT - -## Model description -![GuwenBERT](https://user-images.githubusercontent.com/9592150/97142000-cad08e00-179a-11eb-88df-aff9221482d8.png) - -This is a RoBERTa model pre-trained on Classical Chinese. You can fine-tune GuwenBERT for downstream tasks, such as sentence breaking, punctuation, named entity recognition, and so on. - -For more information about RoBERTa, take a look at the RoBERTa's offical repo. - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("ethanyt/guwenbert-base") - -model = AutoModel.from_pretrained("ethanyt/guwenbert-base") -``` - -## Training data - -The training data is daizhige dataset (殆知阁古代文献) which is contains of 15,694 books in Classical Chinese, covering Buddhism, Confucianism, Medicine, History, Zi, Yi, Yizang, Shizang, Taoism, and Jizang. -76% of them are punctuated. -The total number of characters is 1.7B (1,743,337,673). -All traditional Characters are converted to simplified characters. -The vocabulary is constructed from this data set and the size is 23,292. - -## Training procedure - -The models are initialized with `hfl/chinese-roberta-wwm-ext` and then pre-trained with a 2-step strategy. -In the first step, the model learns MLM with only word embeddings updated during training, until convergence. In the second step, all parameters are updated during training. - -The models are trained on 4 V100 GPUs for 120K steps (20K for step#1, 100K for step#2) with a batch size of 2,048 and a sequence length of 512. The optimizer used is Adam with a learning rate of 2e-4, adam-betas of (0.9,0.98), adam-eps of 1e-6, a weight decay of 0.01, learning rate warmup for 5K steps, and linear decay of learning rate after. - -## Eval results - -### "Gulian Cup" Ancient Books Named Entity Recognition Evaluation - -Second place in the competition. Detailed test results: - -| NE Type | Precision | Recall | F1 | -|:----------:|:-----------:|:------:|:-----:| -| Book Name | 77.50 | 73.73 | 75.57 | -| Other Name | 85.85 | 89.32 | 87.55 | -| Micro Avg. | 83.88 | 85.39 | 84.63 | - - - - -## About Us - -We are from [Datahammer](https://datahammer.net), Beijing Institute of Technology. -For more cooperation, please contact email: ethanyt [at] qq.com - -> Created with ❤️ by Tan Yan [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/Ethan-yt) and Zewen Chi [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/CZWin32768) \ No newline at end of file diff --git a/model_cards/ethanyt/guwenbert-large/README.md b/model_cards/ethanyt/guwenbert-large/README.md deleted file mode 100644 index 60fe94d619..0000000000 --- a/model_cards/ethanyt/guwenbert-large/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: -- "zh" -thumbnail: "https://user-images.githubusercontent.com/9592150/97142000-cad08e00-179a-11eb-88df-aff9221482d8.png" -tags: -- "chinese" -- "classical chinese" -- "literary chinese" -- "ancient chinese" -- "bert" -- "pytorch" -license: "apache-2.0" -pipeline_tag: "fill-mask" -widget: -- text: "[MASK]太元中，武陵人捕鱼为业。" -- text: "问征夫以前路，恨晨光之[MASK]微。" -- text: "浔阳江头夜送客，枫叶[MASK]花秋瑟瑟。" ---- - -# GuwenBERT - -## Model description -![GuwenBERT](https://user-images.githubusercontent.com/9592150/97142000-cad08e00-179a-11eb-88df-aff9221482d8.png) - -This is a RoBERTa model pre-trained on Classical Chinese. You can fine-tune GuwenBERT for downstream tasks, such as sentence breaking, punctuation, named entity recognition, and so on. - -For more information about RoBERTa, take a look at the RoBERTa's offical repo. - -## How to use - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("ethanyt/guwenbert-large") - -model = AutoModel.from_pretrained("ethanyt/guwenbert-large") -``` - -## Training data - -The training data is daizhige dataset (殆知阁古代文献) which is contains of 15,694 books in Classical Chinese, covering Buddhism, Confucianism, Medicine, History, Zi, Yi, Yizang, Shizang, Taoism, and Jizang. -76% of them are punctuated. -The total number of characters is 1.7B (1,743,337,673). -All traditional Characters are converted to simplified characters. -The vocabulary is constructed from this data set and the size is 23,292. - -## Training procedure - -The models are initialized with `hfl/chinese-roberta-wwm-ext-large` and then pre-trained with a 2-step strategy. -In the first step, the model learns MLM with only word embeddings updated during training, until convergence. In the second step, all parameters are updated during training. - -The models are trained on 4 V100 GPUs for 120K steps (20K for step#1, 100K for step#2) with a batch size of 2,048 and a sequence length of 512. The optimizer used is Adam with a learning rate of 1e-4, adam-betas of (0.9,0.98), adam-eps of 1e-6, a weight decay of 0.01, learning rate warmup for 5K steps, and linear decay of learning rate after. - -## Eval results - -### "Gulian Cup" Ancient Books Named Entity Recognition Evaluation - -Second place in the competition. Detailed test results: - -| NE Type | Precision | Recall | F1 | -|:----------:|:-----------:|:------:|:-----:| -| Book Name | 77.50 | 73.73 | 75.57 | -| Other Name | 85.85 | 89.32 | 87.55 | -| Micro Avg. | 83.88 | 85.39 | 84.63 | - - - - -## About Us - -We are from [Datahammer](https://datahammer.net), Beijing Institute of Technology. -For more cooperation, please contact email: ethanyt [at] qq.com - -> Created with ❤️ by Tan Yan [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/Ethan-yt) and Zewen Chi [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/CZWin32768) \ No newline at end of file diff --git a/model_cards/facebook/bart-large-cnn/README.md b/model_cards/facebook/bart-large-cnn/README.md deleted file mode 100644 index aef17e07ba..0000000000 --- a/model_cards/facebook/bart-large-cnn/README.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -tags: -- summarization - -license: mit -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- diff --git a/model_cards/facebook/bart-large-mnli/README.md b/model_cards/facebook/bart-large-mnli/README.md deleted file mode 100644 index bb29a53b0e..0000000000 --- a/model_cards/facebook/bart-large-mnli/README.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -license: mit -thumbnail: https://huggingface.co/front/thumbnails/facebook.png -pipeline_tag: zero-shot-classification -datasets: -- multi_nli ---- - -# bart-large-mnli - -This is the checkpoint for [bart-large](https://huggingface.co/facebook/bart-large) after being trained on the [MultiNLI (MNLI)](https://huggingface.co/datasets/multi_nli) dataset. - -Additional information about this model: -- The [bart-large](https://huggingface.co/facebook/bart-large) model page -- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension -](https://arxiv.org/abs/1910.13461) -- [BART fairseq implementation](https://github.com/pytorch/fairseq/tree/master/fairseq/models/bart) - -## NLI-based Zero Shot Text Classification - -[Yin et al.](https://arxiv.org/abs/1909.00161) proposed a method for using pre-trained NLI models as a ready-made zero-shot sequence classifiers. The method works by posing the sequence to be classified as the NLI premise and to construct a hypothesis from each candidate label. For example, if we want to evaluate whether a sequence belongs to the class "politics", we could construct a hypothesis of `This text is about politics.`. The probabilities for entailment and contradiction are then converted to label probabilities. - -This method is surprisingly effective in many cases, particularly when used with larger pre-trained models like BART and Roberta. See [this blog post](https://joeddav.github.io/blog/2020/05/29/ZSL.html) for a more expansive introduction to this and other zero shot methods, and see the code snippets below for examples of using this model for zero-shot classification both with Hugging Face's built-in pipeline and with native Transformers/PyTorch code. - -#### With the zero-shot classification pipeline - -The model can be loaded with the `zero-shot-classification` pipeline like so: - -```python -from transformers import pipeline -classifier = pipeline("zero-shot-classification", - model="facebook/bart-large-mnli") -``` - -You can then use this pipeline to classify sequences into any of the class names you specify. - -```python -sequence_to_classify = "one day I will see the world" -candidate_labels = ['travel', 'cooking', 'dancing'] -classifier(sequence_to_classify, candidate_labels) -#{'labels': ['travel', 'dancing', 'cooking'], -# 'scores': [0.9938651323318481, 0.0032737774308770895, 0.002861034357920289], -# 'sequence': 'one day I will see the world'} -``` - -If more than one candidate label can be correct, pass `multi_class=True` to calculate each class independently: - -```python -candidate_labels = ['travel', 'cooking', 'dancing', 'exploration'] -classifier(sequence_to_classify, candidate_labels, multi_class=True) -#{'labels': ['travel', 'exploration', 'dancing', 'cooking'], -# 'scores': [0.9945111274719238, -# 0.9383890628814697, -# 0.0057061901316046715, -# 0.0018193122232332826], -# 'sequence': 'one day I will see the world'} -``` - - -#### With manual PyTorch - -```python -# pose sequence as a NLI premise and label as a hypothesis -from transformers import AutoModelForSequenceClassification, AutoTokenizer -nli_model = AutoModelForSequenceClassification.from_pretrained('joeddav/xlm-roberta-large-xnli') -tokenizer = AutoTokenizer.from_pretrained('joeddav/xlm-roberta-large-xnli') - -premise = sequence -hypothesis = f'This example is {label}.' - -# run through model pre-trained on MNLI -x = tokenizer.encode(premise, hypothesis, return_tensors='pt', - truncation_strategy='only_first') -logits = nli_model(x.to(device))[0] - -# we throw away "neutral" (dim 1) and take the probability of -# "entailment" (2) as the probability of the label being true -entail_contradiction_logits = logits[:,[0,2]] -probs = entail_contradiction_logits.softmax(dim=1) -prob_label_is_true = probs[:,1] -``` diff --git a/model_cards/facebook/bart-large/README.md b/model_cards/facebook/bart-large/README.md deleted file mode 100644 index 653141730c..0000000000 --- a/model_cards/facebook/bart-large/README.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -license: mit -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- - -The Bart model was proposed by Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov and Luke Zettlemoyer on 29 Oct, 2019. According to the abstract, - -Bart uses a standard seq2seq/machine translation architecture with a bidirectional encoder (like BERT) and a left-to-right decoder (like GPT). - -The pretraining task involves randomly shuffling the order of the original sentences and a novel in-filling scheme, where spans of text are replaced with a single mask token. - -BART is particularly effective when fine tuned for text generation but also works well for comprehension tasks. It matches the performance of RoBERTa with comparable training resources on GLUE and SQuAD, achieves new state-of-the-art results on a range of abstractive dialogue, question answering, and summarization tasks, with gains of up to 6 ROUGE. - -The Authors’ code can be found here: -https://github.com/pytorch/fairseq/tree/master/examples/bart diff --git a/model_cards/facebook/rag-sequence-base/README.md b/model_cards/facebook/rag-sequence-base/README.md deleted file mode 100644 index d9532b61bc..0000000000 --- a/model_cards/facebook/rag-sequence-base/README.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -license: apache-2.0 -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- -## RAG - -This is a non-finetuned version of the RAG-Sequence model of the the paper [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/pdf/2005.11401.pdf) -by Patrick Lewis, Ethan Perez, Aleksandara Piktus et al. - -Rag consits of a *question encoder*, *retriever* and a *generator*. The retriever should be a `RagRetriever` instance. The *question encoder* can be any model that can be loaded with `AutoModel` and the *generator* can be any model that can be loaded with `AutoModelForSeq2SeqLM`. - -This model is a non-finetuned RAG-Sequence model and was created as follows: - -```python -from transformers import RagTokenizer, RagRetriever, RagSequenceForGeneration, AutoTokenizer - -model = RagSequenceForGeneration.from_pretrained_question_encoder_generator("facebook/dpr-question_encoder-single-nq-base", "facebook/bart-large") - -question_encoder_tokenizer = AutoTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") -generator_tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large") - -tokenizer = RagTokenizer(question_encoder_tokenizer, generator_tokenizer) -model.config.use_dummy_dataset = True -model.config.index_name = "exact" -retriever = RagRetriever(model.config, question_encoder_tokenizer, generator_tokenizer) - -model.save_pretrained("./") -tokenizer.save_pretrained("./") -retriever.save_pretrained("./") -``` - -Note that the model is *uncased* so that all capital input letters are converted to lower-case. - -## Usage: - -*Note*: the model uses the *dummy* retriever as a default. Better results are obtained by using the full retriever, -by setting `config.index_name="legacy"` and `config.use_dummy_dataset=False`. -The model can be fine-tuned as follows: - -```python -from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration - -tokenizer = RagTokenizer.from_pretrained("facebook/rag-sequence-base") -retriever = RagRetriever.from_pretrained("facebook/rag-sequence-base") -model = RagTokenForGeneration.from_pretrained("facebook/rag-sequence-base", retriever=retriever) - -input_dict = tokenizer.prepare_seq2seq_batch("who holds the record in 100m freestyle", "michael phelps", return_tensors="pt") - -outputs = model(input_dict["input_ids"], labels=input_dict["labels"]) - -loss = outputs.loss - -# train on loss -``` diff --git a/model_cards/facebook/rag-sequence-nq/README.md b/model_cards/facebook/rag-sequence-nq/README.md deleted file mode 100644 index 325212a6fb..0000000000 --- a/model_cards/facebook/rag-sequence-nq/README.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- wiki_dpr -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- -## RAG - -This is the RAG-Sequence Model of the the paper [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/pdf/2005.11401.pdf) -by Patrick Lewis, Ethan Perez, Aleksandara Piktus et al. - -The model is a *uncased* model, which means that capital letters are simply converted to lower-case letters. - -The model consits of a *question_encoder*, *retriever* and a *generator*. The retriever extracts relevant passages from the *wiki_dpr* `train` datasets, which is linked above. -The question_encoder and retriever are based on `facebook/dpr-question_encoder-single-nq-base` and `facebook/bart-large`, which were jointly finetuned on -on the *wiki_dpr* QA dataset in an end-to-end fashion. - -## Usage: - -**Note**: In the usage example below only the *dummy* retriever of *wiki_dpr* is used because the complete *lecagy* index requires over 75 GB of RAM. -The model can generate answers to any factoid question as follows: - -```python -from transformers import RagTokenizer, RagRetriever, RagSequenceForGeneration - -tokenizer = RagTokenizer.from_pretrained("facebook/rag-sequence-nq") -retriever = RagRetriever.from_pretrained("facebook/rag-sequence-nq", index_name="exact", use_dummy_dataset=True) -model = RagSequenceForGeneration.from_pretrained("facebook/rag-sequence-nq", retriever=retriever) - -input_dict = tokenizer.prepare_seq2seq_batch("how many countries are in europe", return_tensors="pt") - -generated = model.generate(input_ids=input_dict["input_ids"]) -print(tokenizer.batch_decode(generated, skip_special_tokens=True)[0]) - -# should give 54 => google says either 44 or 51 -``` diff --git a/model_cards/facebook/rag-token-base/README.md b/model_cards/facebook/rag-token-base/README.md deleted file mode 100644 index 7e99fb3104..0000000000 --- a/model_cards/facebook/rag-token-base/README.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- wiki_dpr -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- -## RAG - -This is a non-finetuned version of the RAG-Token model of the the paper [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/pdf/2005.11401.pdf) -by Patrick Lewis, Ethan Perez, Aleksandara Piktus et al. - -Rag consits of a *question encoder*, *retriever* and a *generator*. The retriever should be a `RagRetriever` instance. The *question encoder* can be any model that can be loaded with `AutoModel` and the *generator* can be any model that can be loaded with `AutoModelForSeq2SeqLM`. - -This model is a non-finetuned RAG-Token model and was created as follows: - -```python -from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration, AutoTokenizer - -model = RagTokenForGeneration.from_pretrained_question_encoder_generator("facebook/dpr-question_encoder-single-nq-base", "facebook/bart-large") - -question_encoder_tokenizer = AutoTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") -generator_tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large") - -tokenizer = RagTokenizer(question_encoder_tokenizer, generator_tokenizer) -model.config.use_dummy_dataset = True -model.config.index_name = "exact" -retriever = RagRetriever(model.config, question_encoder_tokenizer, generator_tokenizer) - -model.save_pretrained("./") -tokenizer.save_pretrained("./") -retriever.save_pretrained("./") -``` - -Note that the model is *uncased* so that all capital input letters are converted to lower-case. - -## Usage: - -*Note*: the model uses the *dummy* retriever as a default. Better results are obtained by using the full retriever, -by setting `config.index_name="legacy"` and `config.use_dummy_dataset=False`. -The model can be fine-tuned as follows: - -```python -from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration - -tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-base") -retriever = RagRetriever.from_pretrained("facebook/rag-token-base") -model = RagTokenForGeneration.from_pretrained("facebook/rag-token-base", retriever=retriever) - -input_dict = tokenizer.prepare_seq2seq_batch("who holds the record in 100m freestyle", "michael phelps", return_tensors="pt") - -outputs = model(input_dict["input_ids"], labels=input_dict["labels"]) - -loss = outputs.loss - -# train on loss -``` diff --git a/model_cards/facebook/rag-token-nq/README.md b/model_cards/facebook/rag-token-nq/README.md deleted file mode 100644 index 2c4deb0564..0000000000 --- a/model_cards/facebook/rag-token-nq/README.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- wiki_dpr -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- -## RAG - -This is the RAG-Token Model of the the paper [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/pdf/2005.11401.pdf) -by Patrick Lewis, Ethan Perez, Aleksandara Piktus et al. - -The model is a *uncased* model, which means that capital letters are simply converted to lower-case letters. - -The model consits of a *question_encoder*, *retriever* and a *generator*. The retriever extracts relevant passages from the *wiki_dpr* `train` datasets, which is linked above. -The question_encoder and retriever are based on `facebook/dpr-question_encoder-single-nq-base` and `facebook/bart-large`, which were jointly finetuned on -on the *wiki_dpr* QA dataset in an end-to-end fashion. - -## Usage: - -**Note**: In the usage example below only the *dummy* retriever of *wiki_dpr* is used because the complete *lecagy* index requires over 75 GB of RAM. -The model can generate answers to any factoid question as follows: - -```python -from transformers import RagTokenizer, RagRetriever, RagTokenForGeneration - -tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq") -retriever = RagRetriever.from_pretrained("facebook/rag-token-nq", index_name="exact", use_dummy_dataset=True) -model = RagTokenForGeneration.from_pretrained("facebook/rag-token-nq", retriever=retriever) - -input_dict = tokenizer.prepare_seq2seq_batch("who holds the record in 100m freestyle", return_tensors="pt") - -generated = model.generate(input_ids=input_dict["input_ids"]) -print(tokenizer.batch_decode(generated, skip_special_tokens=True)[0]) - -# should give michael phelps => sounds reasonable -``` diff --git a/model_cards/facebook/rag-token-nq_new/README.md b/model_cards/facebook/rag-token-nq_new/README.md deleted file mode 100644 index 940a8ac614..0000000000 --- a/model_cards/facebook/rag-token-nq_new/README.md +++ /dev/null @@ -1,24 +0,0 @@ -The model can be loaded and used as follows on [this branch](https://github.com/huggingface/transformers/tree/finalize_rag) as follows. - - -# Load model - -```python -from transformers import RagTokenizer, RagTokenForGeneration, RagRetriever - -# create Retriever augmented model -retriever = RagRetriever.from_pretrained("facebook/rag-token-nq_new", use_dummy_dataset=True) -model = RagTokenForGeneration.from_pretrained("facebook/rag-token-nq_new", retriever=retriever) - -tokenizer = RagTokenizer.from_pretrained("facebook/rag-token-nq_new") - -# create input ids and labels -input_ids = tokenizer("who sings does he love me with reba", return_tensors="pt").input_ids - -# use labels -labels = tokenizer.generator("Linda Davis", return_tensors="pt").input_ids - - -# compute loss -outputs = model(input_ids, labels=labels) -``` diff --git a/model_cards/facebook/wmt19-de-en/README.md b/model_cards/facebook/wmt19-de-en/README.md deleted file mode 100644 index 924097c752..0000000000 --- a/model_cards/facebook/wmt19-de-en/README.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -language: -- de -- en -tags: -- translation -- wmt19 -- facebook -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- - -# FSMT - -## Model description - -This is a ported version of [fairseq wmt19 transformer](https://github.com/pytorch/fairseq/blob/master/examples/wmt19/README.md) for de-en. - -For more details, please see, [Facebook FAIR's WMT19 News Translation Task Submission](https://arxiv.org/abs/1907.06616). - -The abbreviation FSMT stands for FairSeqMachineTranslation - -All four models are available: - -* [wmt19-en-ru](https://huggingface.co/facebook/wmt19-en-ru) -* [wmt19-ru-en](https://huggingface.co/facebook/wmt19-ru-en) -* [wmt19-en-de](https://huggingface.co/facebook/wmt19-en-de) -* [wmt19-de-en](https://huggingface.co/facebook/wmt19-de-en) - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "facebook/wmt19-de-en" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Maschinelles Lernen ist großartig, oder?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Machine learning is great, isn't it? - -``` - -#### Limitations and bias - -- The original (and this ported model) doesn't seem to handle well inputs with repeated sub-phrases, [content gets truncated](https://discuss.huggingface.co/t/issues-with-translating-inputs-containing-repeated-phrases/981) - -## Training data - -Pretrained weights were left identical to the original model released by fairseq. For more details, please, see the [paper](https://arxiv.org/abs/1907.06616). - -## Eval results - -pair | fairseq | transformers --------|---------|---------- -de-en | [42.3](http://matrix.statmt.org/matrix/output/1902?run_id=6750) | 41.35 - -The score is slightly below the score reported by `fairseq`, since `transformers`` currently doesn't support: -- model ensemble, therefore the best performing checkpoint was ported (``model4.pt``). -- re-ranking - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=de-en -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=15 -mkdir -p $DATA_DIR -sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py facebook/wmt19-$PAIR $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` -note: fairseq reports using a beam of 50, so you should get a slightly higher score if re-run with `--num_beams 50`. - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt19/) -- [test set](http://matrix.statmt.org/test_sets/newstest2019.tgz?1556572561) - - -### BibTeX entry and citation info - -```bibtex -@inproceedings{..., - year={2020}, - title={Facebook FAIR's WMT19 News Translation Task Submission}, - author={Ng, Nathan and Yee, Kyra and Baevski, Alexei and Ott, Myle and Auli, Michael and Edunov, Sergey}, - booktitle={Proc. of WMT}, -} -``` - - -## TODO - -- port model ensemble (fairseq uses 4 model checkpoints) - diff --git a/model_cards/facebook/wmt19-en-de/README.md b/model_cards/facebook/wmt19-en-de/README.md deleted file mode 100644 index b4bacbbf8c..0000000000 --- a/model_cards/facebook/wmt19-en-de/README.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -language: -- en -- de -tags: -- translation -- wmt19 -- facebook -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- - -# FSMT - -## Model description - -This is a ported version of [fairseq wmt19 transformer](https://github.com/pytorch/fairseq/blob/master/examples/wmt19/README.md) for en-de. - -For more details, please see, [Facebook FAIR's WMT19 News Translation Task Submission](https://arxiv.org/abs/1907.06616). - -The abbreviation FSMT stands for FairSeqMachineTranslation - -All four models are available: - -* [wmt19-en-ru](https://huggingface.co/facebook/wmt19-en-ru) -* [wmt19-ru-en](https://huggingface.co/facebook/wmt19-ru-en) -* [wmt19-en-de](https://huggingface.co/facebook/wmt19-en-de) -* [wmt19-de-en](https://huggingface.co/facebook/wmt19-de-en) - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "facebook/wmt19-en-de" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Machine learning is great, isn't it?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Maschinelles Lernen ist großartig, oder? - -``` - -#### Limitations and bias - -- The original (and this ported model) doesn't seem to handle well inputs with repeated sub-phrases, [content gets truncated](https://discuss.huggingface.co/t/issues-with-translating-inputs-containing-repeated-phrases/981) - -## Training data - -Pretrained weights were left identical to the original model released by fairseq. For more details, please, see the [paper](https://arxiv.org/abs/1907.06616). - -## Eval results - -pair | fairseq | transformers --------|---------|---------- -en-de | [43.1](http://matrix.statmt.org/matrix/output/1909?run_id=6862) | 42.83 - -The score is slightly below the score reported by `fairseq`, since `transformers`` currently doesn't support: -- model ensemble, therefore the best performing checkpoint was ported (``model4.pt``). -- re-ranking - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=en-de -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=15 -mkdir -p $DATA_DIR -sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py facebook/wmt19-$PAIR $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` -note: fairseq reports using a beam of 50, so you should get a slightly higher score if re-run with `--num_beams 50`. - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt19/) -- [test set](http://matrix.statmt.org/test_sets/newstest2019.tgz?1556572561) - - -### BibTeX entry and citation info - -```bibtex -@inproceedings{..., - year={2020}, - title={Facebook FAIR's WMT19 News Translation Task Submission}, - author={Ng, Nathan and Yee, Kyra and Baevski, Alexei and Ott, Myle and Auli, Michael and Edunov, Sergey}, - booktitle={Proc. of WMT}, -} -``` - - -## TODO - -- port model ensemble (fairseq uses 4 model checkpoints) - diff --git a/model_cards/facebook/wmt19-en-ru/README.md b/model_cards/facebook/wmt19-en-ru/README.md deleted file mode 100644 index aa55e7a308..0000000000 --- a/model_cards/facebook/wmt19-en-ru/README.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -language: -- en -- ru -tags: -- translation -- wmt19 -- facebook -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- - -# FSMT - -## Model description - -This is a ported version of [fairseq wmt19 transformer](https://github.com/pytorch/fairseq/blob/master/examples/wmt19/README.md) for en-ru. - -For more details, please see, [Facebook FAIR's WMT19 News Translation Task Submission](https://arxiv.org/abs/1907.06616). - -The abbreviation FSMT stands for FairSeqMachineTranslation - -All four models are available: - -* [wmt19-en-ru](https://huggingface.co/facebook/wmt19-en-ru) -* [wmt19-ru-en](https://huggingface.co/facebook/wmt19-ru-en) -* [wmt19-en-de](https://huggingface.co/facebook/wmt19-en-de) -* [wmt19-de-en](https://huggingface.co/facebook/wmt19-de-en) - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "facebook/wmt19-en-ru" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Machine learning is great, isn't it?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Машинное обучение - это здорово, не так ли? - -``` - -#### Limitations and bias - -- The original (and this ported model) doesn't seem to handle well inputs with repeated sub-phrases, [content gets truncated](https://discuss.huggingface.co/t/issues-with-translating-inputs-containing-repeated-phrases/981) - -## Training data - -Pretrained weights were left identical to the original model released by fairseq. For more details, please, see the [paper](https://arxiv.org/abs/1907.06616). - -## Eval results - -pair | fairseq | transformers --------|---------|---------- -en-ru | [36.4](http://matrix.statmt.org/matrix/output/1914?run_id=6724) | 33.47 - -The score is slightly below the score reported by `fairseq`, since `transformers`` currently doesn't support: -- model ensemble, therefore the best performing checkpoint was ported (``model4.pt``). -- re-ranking - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=en-ru -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=15 -mkdir -p $DATA_DIR -sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py facebook/wmt19-$PAIR $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` -note: fairseq reports using a beam of 50, so you should get a slightly higher score if re-run with `--num_beams 50`. - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt19/) -- [test set](http://matrix.statmt.org/test_sets/newstest2019.tgz?1556572561) - - -### BibTeX entry and citation info - -```bibtex -@inproceedings{..., - year={2020}, - title={Facebook FAIR's WMT19 News Translation Task Submission}, - author={Ng, Nathan and Yee, Kyra and Baevski, Alexei and Ott, Myle and Auli, Michael and Edunov, Sergey}, - booktitle={Proc. of WMT}, -} -``` - - -## TODO - -- port model ensemble (fairseq uses 4 model checkpoints) - diff --git a/model_cards/facebook/wmt19-ru-en/README.md b/model_cards/facebook/wmt19-ru-en/README.md deleted file mode 100644 index 627aced50f..0000000000 --- a/model_cards/facebook/wmt19-ru-en/README.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -language: -- ru -- en -tags: -- translation -- wmt19 -- facebook -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu -thumbnail: https://huggingface.co/front/thumbnails/facebook.png ---- - -# FSMT - -## Model description - -This is a ported version of [fairseq wmt19 transformer](https://github.com/pytorch/fairseq/blob/master/examples/wmt19/README.md) for ru-en. - -For more details, please see, [Facebook FAIR's WMT19 News Translation Task Submission](https://arxiv.org/abs/1907.06616). - -The abbreviation FSMT stands for FairSeqMachineTranslation - -All four models are available: - -* [wmt19-en-ru](https://huggingface.co/facebook/wmt19-en-ru) -* [wmt19-ru-en](https://huggingface.co/facebook/wmt19-ru-en) -* [wmt19-en-de](https://huggingface.co/facebook/wmt19-en-de) -* [wmt19-de-en](https://huggingface.co/facebook/wmt19-de-en) - -## Intended uses & limitations - -#### How to use - -```python -from transformers import FSMTForConditionalGeneration, FSMTTokenizer -mname = "facebook/wmt19-ru-en" -tokenizer = FSMTTokenizer.from_pretrained(mname) -model = FSMTForConditionalGeneration.from_pretrained(mname) - -input = "Машинное обучение - это здорово, не так ли?" -input_ids = tokenizer.encode(input, return_tensors="pt") -outputs = model.generate(input_ids) -decoded = tokenizer.decode(outputs[0], skip_special_tokens=True) -print(decoded) # Machine learning is great, isn't it? - -``` - -#### Limitations and bias - -- The original (and this ported model) doesn't seem to handle well inputs with repeated sub-phrases, [content gets truncated](https://discuss.huggingface.co/t/issues-with-translating-inputs-containing-repeated-phrases/981) - -## Training data - -Pretrained weights were left identical to the original model released by fairseq. For more details, please, see the [paper](https://arxiv.org/abs/1907.06616). - -## Eval results - -pair | fairseq | transformers --------|---------|---------- -ru-en | [41.3](http://matrix.statmt.org/matrix/output/1907?run_id=6937) | 39.20 - -The score is slightly below the score reported by `fairseq`, since `transformers`` currently doesn't support: -- model ensemble, therefore the best performing checkpoint was ported (``model4.pt``). -- re-ranking - -The score was calculated using this code: - -```bash -git clone https://github.com/huggingface/transformers -cd transformers -export PAIR=ru-en -export DATA_DIR=data/$PAIR -export SAVE_DIR=data/$PAIR -export BS=8 -export NUM_BEAMS=15 -mkdir -p $DATA_DIR -sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source -sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target -echo $PAIR -PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py facebook/wmt19-$PAIR $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS -``` -note: fairseq reports using a beam of 50, so you should get a slightly higher score if re-run with `--num_beams 50`. - -## Data Sources - -- [training, etc.](http://www.statmt.org/wmt19/) -- [test set](http://matrix.statmt.org/test_sets/newstest2019.tgz?1556572561) - - -### BibTeX entry and citation info - -```bibtex -@inproceedings{..., - year={2020}, - title={Facebook FAIR's WMT19 News Translation Task Submission}, - author={Ng, Nathan and Yee, Kyra and Baevski, Alexei and Ott, Myle and Auli, Michael and Edunov, Sergey}, - booktitle={Proc. of WMT}, -} -``` - - -## TODO - -- port model ensemble (fairseq uses 4 model checkpoints) - diff --git a/model_cards/flexudy/t5-base-multi-sentence-doctor/README.md b/model_cards/flexudy/t5-base-multi-sentence-doctor/README.md deleted file mode 100644 index d0dcd74802..0000000000 --- a/model_cards/flexudy/t5-base-multi-sentence-doctor/README.md +++ /dev/null @@ -1,109 +0,0 @@ -![avatar](sent-banner.png) - -# Sentence-Doctor -Sentence doctor is a T5 model that attempts to correct the errors or mistakes found in sentences. Model works on English, German and French text. - -## 1. Problem: -Many NLP models depend on tasks like *Text Extraction Libraries, OCR, Speech to Text libraries* and **Sentence Boundary Detection** -As a consequence errors caused by these tasks in your NLP pipeline can affect the quality of models in applications. Especially since models are often trained on **clean** input. - -## 2. Solution: -Here we provide a model that **attempts** to reconstruct sentences based on the its context (sourrounding text). The task is pretty straightforward: -* `Given an "erroneous" sentence, and its context, reconstruct the "intended" sentence`. - -## 3. Use Cases: -* Attempt to repair noisy sentences that where extracted with OCR software or text extractors. -* Attempt to repair sentence boundaries. - * Example (in German): **Input: "und ich bin im**", - * Prefix_Context: "Hallo! Mein Name ist John", Postfix_Context: "Januar 1990 geboren." - * Output: "John und ich bin im Jahr 1990 geboren" -* Possibly sentence level spelling correction -- Although this is not the intended use. - * Input: "I went to church **las yesteday**" => Output: "I went to church last Sunday". - -## 4. Disclaimer -Note how we always emphises on the word *attempt*. The current version of the model was only trained on **150K** sentences from the tatoeba dataset: https://tatoeba.org/eng. (50K per language -- En, Fr, De). -Hence, we strongly encourage you to finetune the model on your dataset. We might release a version trained on more data. - -## 5. Datasets -We generated synthetic data from the tatoeba dataset: https://tatoeba.org/eng. Randomly applying different transformations on words and characters based on some probabilities. The datasets are available in the data folder (where **sentence_doctor_dataset_300K** is a larger dataset with 100K sentences for each language). - -## 6. Usage - -### 6.1 Preprocessing -* Let us assume we have the following text (Note that there are no punctuation marks in the text): - -```python -text = "That is my job I am a medical doctor I save lives" -``` -* You decided extract the sentences and for some obscure reason, you obtained these sentences: - -```python -sentences = ["That is my job I a", "m a medical doct", "I save lives"] -``` -* You now wish to correct the sentence **"m a medical doct"**. - -Here is the single preprocessing step for the model: - -```python -input_text = "repair_sentence: " + sentences[1] + " context: {" + sentences[0] + "}{" + sentences[2] + "} " -``` - -**Explanation**:
-* We are telling the model to repair the sentence with the prefix "repair_sentence: " -* Then append the sentence we want to repair **sentence[1]** which is "m a medical doct" -* Next we give some context to the model. In the case, the context is some text that occured before the sentence and some text that appeard after the sentence in the original text. - * To do that, we append the keyword "context :" - * Append **{sentence[0]}** "{That is my job I a}". (Note how it is sourrounded by curly braces). - * Append **{sentence[2]}** "{I save lives}". -* At last we tell the model this is the end of the input with . - -```python -print(input_text) # repair_sentence: m a medical doct context: {That is my job I a}{or I save lives} -``` - -
- -**The context is optional**, so the input could also be ```repair_sentence: m a medical doct context: {}{} ``` - -### 6.2 Inference - -```python - -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("flexudy/t5-base-multi-sentence-doctor") - -model = AutoModelWithLMHead.from_pretrained("flexudy/t5-base-multi-sentence-doctor") - -input_text = "repair_sentence: m a medical doct context: {That is my job I a}{or I save lives} " - -input_ids = tokenizer.encode(input_text, return_tensors="pt") - -outputs = model.generate(input_ids, max_length=32, num_beams=1) - -sentence = tokenizer.decode(outputs[0], skip_special_tokens=True, clean_up_tokenization_spaces=True) - -assert sentence == "I am a medical doctor." -``` - -## 7. Fine-tuning -We also provide a script `train_any_t5_task.py` that might help you fine-tune any Text2Text Task with T5. We added #TODO comments all over to help you use train with ease. For example: - -```python -# TODO Set your training epochs -config.TRAIN_EPOCHS = 3 -``` -If you don't want to read the #TODO comments, just pass in your data like this - -```python -# TODO Where is your data ? Enter the path -trainer.start("data/sentence_doctor_dataset_300.csv") -``` -and voila!! Please feel free to correct any mistakes in the code and make a pull request. - -## 8. Attribution -* [Huggingface](https://huggingface.co/) transformer lib for making this possible -* Abhishek Kumar Mishra's transformer [tutorial](https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers_summarization_wandb.ipynb) on text summarisation. Our training code is just a modified version of their code. So many thanks. -* We finetuned this model from the huggingface hub: WikinewsSum/t5-base-multi-combine-wiki-news. Thanks to the [authors](https://huggingface.co/WikinewsSum) -* We also read a lot of work from [Suraj Patil](https://github.com/patil-suraj) -* No one has been forgotten, hopefully :) diff --git a/model_cards/flexudy/t5-base-multi-sentence-doctor/sent-banner.png b/model_cards/flexudy/t5-base-multi-sentence-doctor/sent-banner.png deleted file mode 100644 index cf6566f4d20154bba89f378c9416471c2dc33507..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 73284 zcmeFZg;P{<_XkW0h>{{ohk&$zw1kLsNp~+ucXtVbv~)^JExj}gQqoE-ur!j(g48as z#Jj#T@61o1=RbI#nLD#HJ9qB)d+ztt=bX>EA2rnEAKs_BkAZ>lP*Fiv69WT#9s>i5 z68|3hAKOht3Hk@qLsMP`12{pmhk+rDp(y)W+ZXd756_oo;BpYmS()AuFD^8gW<#FM zj%g*s%Q_H=(JL*3DkoteNlC&Wz#%i-WkLUTMXp3V#C$MM1hV4^C3J2bO<^&ZTD}4M zcm}M9FCR8{O)L8E_?1ARd3kx4@TKMM)1cFM-Xg_Jg;1RTyxf`ACifikRj{5(|HoyH zuVLQEj9&8JD-;jsd7VV)cc%Y*o7#vgf@$3IsMjGqU`~miVS$_=^ z^S&c7Ljyh;1LnV1VUG>{fBwsWNsQie07mlLf4BTc2wWl5f69VE;42-91;vix3jZ&! z=Xu=%-hUCY2OGU-I%#Xt|FUc8uyM)%e3#N#bR>m62n?pU|F-M#BrpHV(Ek6Cp@kLV z0X*B8>_Ic5XITZ`57fa)X%0f!5H-B8hY6%lezcT7IDjg%9(FYz)K4ey`_K2x^Ev;M zQhL&1b;p!VOfM}Nw!4qtYeRj6STEwyD?6FIagI7ZAX3I)LgC5NGTG2h-A`T+G_h*v7&yx!+{lskK}iAsXtmfQbjp(fZIga`ZjiD1x~iFwnNBIO zXF-c7wXuvQ+tg;8>F@OVeFn~xw@d4wcgEtO>xb3&OXn9=CkJO)26>~Y|2%sU5A*)P zGbDlYddBMQb8d*@ysI7p2iSVUN6x^DBJ*YaJ2#i+BQNYxbN68YitdOJJUJZ6!nk@| z^iv@DZ{*?m3Jo9#kdKsIE#e(m$-d)ZavgcM88&47v$_}^diBTv|C^gh)`N8;vft@F z=yO~PbaY8>c=;D5dVU}ML=ZHTLAz?sxe30`O7JThV@}8RAIW$1(a_BanOZb-FCxc} zB{YmiMc(}1dqmLp6eHc|*vx+TFS}iNb-W*LoVHPVMrKFp1z=J98{TQ61x9O6g2SWo zL+-J4_OKS45DtAZBBmS}s8$o#r@v6uVkBB29-+`OIHpqf#Cbbo{m!lHZ$4ja`&f7W zw;}|i6#>@O*WUlYc`jJoEdY2~tuC3^Xs%`5vePl#p89`YsHZnp?*tJB+^o);g)d1u zheW^CQ)etZ{{BT^w@?KhuhQRInIw)@7eH$gmN+uMpR9o?v^URHgey3tQ-C1KTUW%7 z?5`tjVh_70zY?ms&#Q)&mndy=qsv^!*Z@CVN5CiKqqtfBuV#fI6hpC$j-b`*;P(YS ziOq*3I5O|t9==nxSY~u)r~bd#_lF`{KA$P-NgGPyFaRfniW9^FtbuG)FgnQmvt1(0 ziC(p6veP7KI)ZA$7R3)l2h?$_&g0+@KR}>U-%B-}GfmpDo=Qy4GncK!_y+Gc&Dy%3 z6_4ml0KdJv0Fx%(pW*&qN7^d$^9c3AFZGJsV`h#UA03yBIjJih0H^8x6_s2WE6W{9 zyf@V1fk~gJB~+A@FB(@L`@CA}RHSl{{H#$txcq*lIwa>AStVsiUDZYS@_BQ|XYB_W zDLfsqlV+x^9C9EfEBMfq_FYs(YK5Lh%>|Z8zuA-)R&PF8H4S5_iTBUvxhzpt8S;7@ zhKF>`wE~Z(jaxLRS^~MglzEXk8*`EpC>EB~-k$&L7S&X$8f3qBF9!Y~=!-;iGPnu8 z=Ufk@O~>eOx=H-D*kQChLz9xRa$E4~;@arsvSF5;kWT2mxTxjb@>V1>BK1%wE#Z=s z=C}w(@A@hq9tPvd=3s82|FM6@dFZQhi`>hoK|pI%EdplPf@){JF7c6y#%fX_t zs7^oFG7W+E*M|5${EX0h2Nfu9FA)>14li_u3p9A0qNbMWM0618o85(0p-lnvRBnQ2|?I30GJ;a?h`nQM(Q<1nEw^fMvF((cVvyWm^|Q zTwbvJ4F4PD^Mzj9wjw0y6rkgXy`_PsHV3u0UM8s7bs$IJWRl!e7b_`mXQHm+|4Ce& z2<@m6Ja`vUyphk*waZAV#l zC(CcxzPGW_@K9`()=aEcZdhGHU`FssFL^C3_b*M}o7dbijcI^RQ8gYqDz)JPB8F$444 zL%$6#(M|E_?|gXhEbaHjE&<$i4xpFTn#(1AeP5`Wv1*n4$L!c5)f*mF&%cX0kfm_K zt5|cE_?mW*>xqb>(t|Pd^l@04QGB!(QTTG10}8ahwbzq+ZgLfD#Bok6!?jZ~XCIT! zZ6*br+kXJ=Fy8}a0d7Inw;7&$J}I%;eN9T3A0w&xyV(|+SZEeE{0)sf5@o3xH+~p{ zp94*PzyEqr3KkDaIBqfY2tPoEh)&H5x?u^U~E1<)MY0#5Kjvz!Z^8?d#$&|=C?NzH;e`v=vkx1NPt7Wl zzN(5W3c19k1qi)l8lPFhVaUp9$^E+b zt0#q$TrH_*J&n`2GF*B675IA^_Ic&pPrz4)G@@H6;1No*(){fo?Sm5mHbEy*x!!v< ztU_TPz=V(zi_H*gczJnMo!_&L!F_DRHUL7cXQ%Jkr9yH`{n&)T%OU@J<@^WLX&v9| zg7!t8;r+n!R4Rv^k*lbky`KJiX)$ zLGH350~)V5FPrh!kCn`QGv;`^FIN*NQhZOlsuj5U+g)zQI$e}Ykn?Ctba^ib zRMXI+3uc~GzO`OztqIESw!8(^!mrfo-H|DL_kuvxic*K684w<8gVqdutV{lZ?il-5icD@`>EN$q7W0zaI%i9KDw{>2u-j%)@I4bT@BN%3U^W{m%$)%qA-Z=4= z6Na-Zc*PyLwW9=H`IPUnY~SkB@~W`K_pt#g&vbq8A)zlLnQUajkmSp-Y7gWpU@c$Nl*$Gq%E1rJ}Q;j&5uX36oJj})e9a>onro8o? z;M-Hb%WdI7B?!;N`DFjwPTLlqhPy*6GG1~dow97u)xb$>5JI>Q51EP?w9G7AF6PWz zTg(S)c?hl3H}lQAcT|s?qdaC6lGrWaePMnoY!m&up)|NE5V7Un#b!2KR9a;j1Qla5 za*KbCya=?;{s3@OD}y|Tx69|g6)eJxl8JZlTye8;9y1c|DxKRXR}JNB5qmio{M<rH{6T-y2pADmQ5QSpV6?JB|&TXY!i`lj|L05no zVLGIv+@dTvWhV|zJ}r^&f9rDS9_rqtbm=+^zs<0uwh(_b6>LxpM48!@Tt3~mKce#` zN?ujwkcUnjiR3g_-L5ZYN-vP?iC*9D<&V;w2SRHvQlce?rP?zf`jf3pj8jw1j^jcH4qX`oRYOjYgf#vzjRn}m) z;!>YU=Xx<;xAF5?!IRm_W`5h=lunSS54m}THz!jl-*8vbCn}?M(D6tBOdGn0@CLO{aPu=^Zb19 zs%XoC;V>N6KjQmdi^O$`@yX_qh$$aAr5ipdC*+2CsKM2Ia4G6!y6Sd2({{uHwSQuZ z5~rIyla{y2oxcr{yek9F=KD)pYT(5MUqvu)eYPU{-2XatldT!QK(KuqH6VPgC280? zW6X+Hh1B{_ixoqvFSi~+!F1xr>4A2f6fq===hUXtZ7Gvw6fs}rmIp@LkBTkb3*QJZ zQ@uQJj|}~ARfN}AB>2E9@k*dcr&c?x&NDSX71r5t zazy3dhCncF6A#v50ZPOT8{Qf zp!#$7PcN4>k?@eG3!NL@T{a)8 zuzhtf)2E%$?W2|)qglT?+y{$^wDul(i|;V3pc1|HG;X8Xw84h8c(cJx8sKkrPMt zEk6RBx$0WAdGb5&jg^l6BL!bP*lI+`ZIok$!!8L=qx$>87OQjdUd3J1)3>j_;91`a znSJ7u4mY>2AnU{eUR^4(gHu~zvnGe{^nSwxu`WA2wnvvUy!cyO%|1#N5~oyRmhX{i zv1PL~+3VeyL?IsA_S@go7zB5mJqZR(y?RK!`UCG`GL;Srn&#EHQBUT%QJ*t8Y&=2d z+{y)fK2<;36-hG3xMyoa3~FGjitW!xs{>+W@;G|!mx6X{weFLIDAIf3f_r5dR@?o` zoXyyRn~hs^(AL>@{2LZJ>?il0>Ah~l44Dd1J?0YXU4uuQpV^yOcH%7wBF&cHd4__R zbi`>v30mr@Y(}(=-aYUo<**uBsnXUDwu*zJYX%hC4$+M0Fp=QART$j$E4(6AYXUE? zX%fiqzAN2!d~54Y7VQ0!q>SX~d4~o;1U@BEj^n)t`2~{Gsi552>knr>-c_SbPG2|E zcJk1$=FD98ye||t)d#1pDcz`2E5l%K{jm8*S0z6u#JL3qn&oQ+n*~j#y5AK9^9-l| z+MqdDI#TI=B6WEcz#fr}h9lRBoFN=Kc0^OXQ#Vxt36z0qBj@5>jjRRWrB|7FqxULx z&Rb%8MGK$eu;6fV*?g>UhJ(p{n*GgY@1i+?)5Iw;=fDnPR81uKtDu8g3&#r(GuOHO z$0M-|+j@yPxh|kr9!E->)Vy-ki7Hh=b(~i=ij!+OqPe!>oaTCwpMukeH}kVMZS5r4 zFs{%QyD{ADh+k2}7FKpAR-?Q4HF2U25 zY!NFgC!rOj77-Jf+3(lJ+8R+@EfebykmqJ%OppDbt)!TEsZYzzX%-i=t^pt+i~WGl zPu@Io;-Qx^^3`Q~WFKh@7wzG^tr<`wG5y!;WU}Rt;>MeVGlhXKf*t1A*d0Q8XEvuE zLlED@8e2gg=+IN{N$Uv;(o2EYNvePs?4@LVDhst-xsRrfP&E^(w(`OhjYYaXEi)Q7 zrc-?nU0KwU+lj>3g4~=qt7-|I46Y{9gl!!Or!vuBZ`+grkHcShbO zhNmBec{}(nFAbgJC%zzHuWlVvjDEQmfaCn_9(Zug(H~fpaftxlue5*L)0jhbINcF; z+1%;w`0OFE_e3|cOCsEA_&wyK+0c5a=FEplzk5gx;DDPvNA_$>^<{=dOu-5 zs!{%5-SVuKNgll56Oy*H!FlMZl&lfo?%5Fl63v4((HGI1vQowOScgFr6W2p#_SkH9 z14cO5ob#NdO!}d{v<2vG8xR_gjehtZe7)tv?``p6K$!YxCU40T^GyHiXXLqDsfeCf zDX$a8%Szp9WBo;`+&h4r$Dn)#rxe;~<%p?2H`EdD@TM0SUka0tU!h_4XfoabP>W-U zdzGC~HO1n?J(_DJY+{>~!Cji)j!fLdi@=P7w&?`-(Ko!TT{K0CW+8Wt?&rK4COXiL zIA3|dCzc+~QhK@ojY=**aY-}mSKZf$=%;C`wLZj+FHPmxD4u%&j*5*<-ZS!$X<3bT z_M5mJS*(Xgy~>EZEJf!)qs{m;*vZ2+IKnI16>)Nn&b1Z#Z98LH5x;eKZyFuU+s!W# z24-aq$mH-IA2k9j>ccrT_B4#zACnqF>|~WXnAKV>G$}d0ATBxEswhWPsE8oSJe$^VGqwqz&%2DAaDVHP_WgEzHReo##HRs zLkC_Q_UeX#%pdt~#nHnz>guU}gc+^6q#IOocQVaeO~a|bDm&=Z9bRAqkzHD2c-~Gl z#2}`+n@~yjx_kUNh~A-^1eQbnX7A#qyRnz2(a$&Jzy6R!6iSr3)H;7OX!H*mDz3E- zT#4F0$%*gz02OCa=#xog8A>VDTg~H46)b96CM@Z~6NP>KWYMdZCg-@i_?B;0)RZ=4 zD^D~WtTOdF;Lx}Ui@J9JC;c^^GUZ(7K(0+kfM@GM(C@MF8bK51OeCDp8xlG&al^=$ zsrl6{Xk;T|OT1)EtNLkdT7Xz*Hg$Rh^*Lyj@~oZhaYoRm5j@gk7= zVjlhy3hK=cI{YaAJxxk#0l%{UazQsB*vK5|jlb=QCp!pZz)VHr1p7>U{`B^Q(yT^E z*^n05ShZCa(#0TXwBJ>$k51qVxpC^gQLQ@fWe41HqsvM4T=3;^S_cJ8q5H$WeXPy< zt*9f)6Ktp0{>ZD&M$8ncAm^<|72fV!3#0CtS%qfqZ2HlObCOXt+X&Re8 z8-~_ImYOd%Qc!perI_k|^ItO+XdF>PQaGP`JmyNLFmrL00IVKcnIxW?d*hJcUfa{E zw686OeBWLii{$TTOPBM%ibs0Y?dgm1B%UG~9ZXIdzj4$hd((GB@QgN>zY-48o%-Zz za5n^AO=?sZ8KXg7o@0iwb7MtWcXqcAHmfnuY=#dTVcn+W13gsgWmAGA4zy+F2nmG@ z@xLUT*XNNj$wIZh>;u3yj4gz--70N6shIa<+a1hnjJ5m{u>}sTl~d8v?=ZWZYf$#mjLNfsfLB!8y3rML1fqN zJUF23G6$f>jIn)?BEm8@Ycs-&Sxp*v^R|3Ap0a=kW%n8Gm|k*qQ$sw8F5Qp0yp?#q z+=xp-q6~%&DMTO|IkZ$DawyTgz?g)GrMK~-n+q07)RvLyrrwXp(TO3Mv{mx}g zk3O=NK0A5`pbsxwJ>4xCa*InVEyH_Y@wt^{0D09<-6e8ZGA!V%r7}z^eWlptUB)fk zotwuy_$`O<@qUYyUws%$Qwr{7BY&y8c*}`s*F8-Yy_iUlNFEiGo&YicN_Bb`?47!` zE=l$~8^ff(i#7X=UnZr*R?H2q^O6d@0OOt_$B6P4=ED+zddz!kxwHL(_?Q#YnCwcc0X7adsK2;GolTUaxv-+5&q$HU_%5P+~KX zC-#F>DrAMD*lv_C5HLwlP|jW9e#aF)A{hZ4D{k4mn{kecs1h+gwn(qG|$d{(+< zy?3naEy0$cyQ0msGBTt`_B-W|CY4kG7|D=mNa45?LC$Psl<_UNHSBJ?nw`~BxX7Y! z(SYkTy2k%xmKuv>2EYGv>kdtxGHqT58>$#+!s)-tgV}h?uiXFsD1!1X1RJkXV{vkzi ztzI{caV+A3Y=5`6XZ-#-8+%E!fXnk-6^GP-rlR_%DQLbGDaDr z>4J$Yq76=zSY`{%Ih**^UcIrjCOhLoj{aI4_6fS2-u8UmaQlIJo$4ilQmqBl`wFAe z&sm$f2gYLogBZ#za|Q^J%WDdbGvf5Cb6ytir_Sqj?!u1#;(Klf)w_O4_css9Ua~xn z6`lH0!%60ogH-j3JEM3_pNE~+yLj|DP~T@MV?mp{yrH!Kho4cJ5Hv+K56Nk>s550F z|3R`8gP(t(&ng_}hiDZ?SI`xrB1C|X#q|7DurYcQLCLam{(-&*$ z-NrXPNk-WfV-wL#R$Wl(+!0pzFF-RpgHBHEg- z*|9I_1@Sf;9BIbLh;Vk$6P^ovU&h)x4)zvb-D*%IW+5V55O^!o%+U>QIEsZ1KDy85 zmhh>!A)b>i41%bo8$Bt#P{B2;Sxq9jNx+FsGN()l{0pR;R;ZS~Im!$95x;%qKH%D6{)j zX-m7mqyDA%zQePn!nxiMfdw!J|o9&slijvVS+3yNdpi{_rx_GD3!F zom;c(jLNP$%V~Al->AktEvN;zxWV@8fybvRDEG-XneO9J-L}_a)+0m6tw{9V3*VZDdeE?izO22QP zZ>hm~YuwF@X^ZaWkJy;6E}8VN6PX?muTLz8$}1FZl7yBXqs!aRCe;eqckZ_6ZSvh3 zE*lY#Co^G^lY~4S1G$>bM|!q2yj!K7siRwTkF)uhyb2EZqgU)lQKyfG3Ba9drV!~s zXS?Zb?voI`m73lg!$_zTG5DFV)I|}Lo%TRqf7xxes9LwTxgfXX+ZN(8@oiYcYfZtm z=meUPc6hN&-*W83KYDhAZr;S{i~+s42=N%+&rM1hQ3>4+3DOhE7>jeS{vt+9ZLX>s z$XSiwSubA;0AOV;#;0=GRe(Oq428ZA5EaR$a_=n(;Lfj8^LqEmvT+l<{5^lezCk;& z`I9dW`HsZx245WXR>}uh=>y#foj-(H<)mM~ld_UZQZ>6Z#k1x~YNjuEeh$m`sJ8J# z2GJ(2OQYLR+Sed&?C6-)B4>p2@OTck^95|DfEgxcC z$RGS^NnCFosdwnwo0-UqPoGWhtc>~pfPbqP`nY=ZYKN#ar+k1Nt{{4 z#6dz5^`tjjCZ42a;-?xi-PeW2b(cU+$6`N6JZ0w|M+Jfh8q8t@zCA z^F+a6`8SLDdntw`A%sR;@9kzt-;GyY-SlvX?KxxwYk4NX3DLl`CUwn{@b#nT)Q<2yj>a24!1rIlHMMn@jA4^TJ=e z-o)tK@x8xviTczAJ#Fic;nrx>993;?;ihoQ^e@UMr|2S5HPvgrb-vz!{0#2oCQM)C z*CIRf$WocHD#WMBw$H;OsLC*9lZd{1eSMYnS_W-3-ZjoIojWwX>DC2){V1PQ2kjg% zhkkgw17k1>T3XlGYJZP9p70_}BDP&n44!{si!c z_=#RRa$Y^~t=}f@kD#xnZw&7VlAZp71^uzLE**3r8Y`jwZQ4hQ+w>O)`tK*V5smWuZR|*hgX{h zE-f6B!H7wTMUttvaw9u-Gs_M;sfCPEAB8+i>^uXJ=2Y|xj-Z=>;2h9z*LLS_q&V(c zgPHJ?2j5*LRNzAPQqfQ-eiOy!^3Wjzc>FY|uW+V@cC5n!nm6h>%DWtGT9OW8o|Yu_ z2uGJYlf7?^`M+7_I-j|;*7{JN#2NEsoEYPuWTzp^#xfEg42(!jgT74}{)D}NOr!hy z)@35CHX|E_IUPO?8yr_6wLy9IYxvkI;}1tuC>*%vOIi6truuAbM=@q;>}Gmnv|)XC zv*g9qLzoFsg6;aFZkp5SmkB#37z8D6Lb1e^>=Ntyrq^dV5}Ud7s<4;PG$4%^SZL}^ zBj{_J4%w(NEkENdjdBaweGjgpIg}?^|DxN2!Azn`UBAVYV>i4b7E0Kwt57L)#bHy9 zIF&wVCrQ`o&`}(#^k&xnoXzCh_g>sNT!j|K9Tu+;FHx4hH^F;iyT)CEd{6VxH9Xfn zu?seTwcsnFa1)(#arR;{zpOU>HrvHPbf=(AU0y?Pmr6DrDbg!QU4L&%F>DHxN;IU? zW3B|aO|ZaB1l-Afh;5{A!vl$IZVq^Ws(HNm3%3lM!2{glhR91yqfEw|yI+B=$i7bs zJBwQj8tzmBvgIDok{H3nraiI>a1?sPvG8B`-swoCVt4b+HOeqnPhPLIyyE07z%9*= zZ~_a>-5i?g*sTu0U*b0jZXVIsgO|J(L=7fkKK4xM*c~@Mwu%rQ3N-_<-rG>mg+rV@ ziES5q+7;#w#D0UaE9a5Two?GL#t-@f_AWWo^03kFRGnoG6phP-yr?NFPSU>M#^gS~ z@fM&yD@~_=Nx%+&F8r_|uOpf?Jq>DjqsBT=mL|N=xogb^^7Rt=sH>8pKw2eh(icI| zmqOu$U=*R>Hb7l(CHQI5_dJSKR#m~=>FoaO^)?sBHtf)tKQSp3o#TirS$>Y)QXw<$ zQ_p$Do~o6Zfuq2Xi1Whid@xQuV>M6U?Y%9MRub4;!7*I%1TQaj|5Q{;Ofe)d~+I;*shImQgsTKv8?ky;ddKnpXyxP{jK6iq+xN zNrK*T5)=a?T=V=1A|P)wC@NhC1ZQ$;{gd>*o+@2M((VWwZ8@4I>>?k|8L68ZElq2U ze?e9uBRi-A0IM$Iav=G)9Yi7~DCG)=ya)>BQpi^$9OW=Tw)xGW;E6x&GpN8;MJDp_ zVD`%j)3)!3Wo8%-4Z$x2Aza@PB_1j|&kQ;$;9pCgZD6-oU28CY{IEMk+Koi|7krZq z&OJoaRF3ZSFPt1u3kZf^X!f?xSCHX|MUE=Aztjw9{9R@!!%+`Uo!z=XBkt1;-2&g5 zgp+RlC!0PyMbm6BjHBKYG_zwXo>+3^x%AEi^M~&isQ8Q{i#)hmy=RWuTV={)ypE4j zKgMM#c=q_^lmtZ#gouy#&TKzJO={koPz68ta-d7S*Zj(|qRNxp*o`!>(!Jxzt}CD< zz;Gn6&fJIgjJ)}C{?}1zyDp(FC#LH1SA5x+k4H}E$aP8_NCufm=Me3boIU}x%OKKB zVC5aV4S{HRU`z}ca1QpA#>e`Z6Dg@%rVMDq#pi-azD8NPKnTC3eu2bPdC?+aFF9^) zp(m9Ym}R7eQst@PTYV+SuTT|n*c`azv_`li|6ZJr-=reP+9ZkDi!K&HQm&ez-<;N` zdcs-nFXwQ3u-phqWD`H)xHGOzzw=Y$*9h-%-6^j9F4111Uis3Zrpi!O$m95=Po=F4O%Rt#z7YMnuj=UDTS*8# zxj~@o-Acn~3>=p!=OfM{b=bft%w#_;CY8lppcE4P}vCc6I)jPVd&chG;K3)X1i7X|s@0;orEjUfQXu{jRtxZWR?Bjk= z%JL&hU+kB&+%HJa>K#oLGwKa>GZI|M=RXLsN{7*QVUo;qzWnz4SHa!FdAf#i$Hsau z7rH-eiy2@37fo84lx&U05vuwWO{u0iA76F~*brs}u2W6O{!Qfke+eQV(L^5dnB2QQ zE9)V~8!|*JP0$+MAU6$W(tjJ!WJpHW9*SkxnCOV2$IZOPAN$>oFPv;2r~`(2ZvcrV za*uVS8{0qB>`!zv9-Ed0ngNq1qj!y`5&2?$LkGWO~xDj|(Qbo*OwpA0ZypCyl? zSE;WBEkcXJcfIu2gl3qx zDq0eM)S{ue)x+r_vtTyXYSQ|^sYo~6EP;m>sKJ2KU-3w(MeiH8V!+5c%8&KER*({! zVat^WK5-til#p4TOO$wD8;&2;L~n^|+Y~&h_M09L_vxr~+ zS+ZbJ(|i@(lJQf=71Csk0yOQ_XUKk$&|m(cF5#@X^7fXHO5^ZS*f!h$krv8ALMmkc z7aA*IiO=D-{a>Q7G#3013;zt!9M$uou#|87Zfgdka7IQO^x8|nvtYp$#z2xw4<2(?H!d7hWC1TFSoIa!NqupXd#jhei?P38}D1shi#TwdaHL?epe z7k!AN+n`rqwL~5O)=DsV_4+bUvi{C5`1NCjzf9#>5+&Ac=jRa_6M`OYs&ofge6aM+ zg+Ikp=y{O=-7A|7Q}^F19hr?s+AVgii96XZc4!>@``kl-u`oshF41%c+9?To9+azz zybOt$tl7U*e1B6KrBLxRVZ=FN8&1$ujt%tT4cat{Sa z2af^|0AaY2(8%|DbGZ&UB&Q=x-Oiiat%!&yJpz~&bEP<%P4b@p2Y!_QBnT5NLeS*m#e)1f{)V;O$$kXzO>b-`yi%{~70!H5|&VVPRn_}dAQ z^5M)T8%swCOi~|AuK^QGp@a`HeEZ}_azNN@3Ntk6DC*Q6Ok8=LAm&QEH{EDdwV8Dw z9RR-a&m*`_cR~rY(VeFJACf-WWj0A+yhO0&VoMu{$L`&@m=7HUo2_8`iJ+tTRN&sBEDFsC zW>D4z9(ziP?U;bT)ifr_4+%#eKx6M+~=N%`QhQyF}#r5af9|}+3hVmYhsCrJ`);( zExvySK$1f9(6im_E;1DV-?sqrqGRPIn|;D@BqGO(V4mp;(f>{7c|P;LFFGCd4_>yT*f(h zgK_v%fAKy$bgNsoh!wV;mksNRq+G;O>|#gKM{Ow_B4x047r3GH?VX15pX$h znN>#VPSQUN#H|wBF1GCmq)fjvJU+jd7v04KLuFT+F3JCM6n}eKu6x_?V7L`19J_)BkZgX$d3q2r{F7;lE( ze8j9Oo!#3R@!mbErN488XFvS#e}w1l1eoZpngf$67@8~7Mr8aEpZk7gMj(>Hpy&Ky z9fxzz*QafRE%OeqVgcX}XGfdaLwkiRFY*%D4x^R*Kz4m)7j2M$gQj9*^r#oVp~mpxtKvzhM@WY?lV@fKLMpC}lI;zvY_LPSc4WWi~0`Pb7SZVY9~uuoK*}J{0VycA^Z#CR>ovD8~SwWc}~D(K2gN| zVObvzA@poRMZ<5Ae-EqVPhR1qy#Qzm&oStoXy9bIG!T3`A*#kz+5!YDuS`{|cPkif z={N|pE+2tBT)_w11x5fv%nAOyMCZIoy}!*Q><@iad=fdGUFC=?K6>V!sR#d@pCj2g z%ZPL3FEo7H|EX@prsJNcrNL#e3uPUl=klJlVRA~Twkoe zcPbst)bo4;zUF->WS==^)LNpW%E-~7tNXpb%jYM=-Jn8uspDm=FF$6s@5y&~UQ~eP ziuI8|p3k)o(Wt?KNk3nx>-LsUW}!{G(S>8PeerTuS+Wr1RAmwn{I>|0WD00@c%DN4 zCKBHn@y)0aw|638xZ%LfDuvMT5%NdeonT1T<9Plqg*W%k(&*{Lf7X93<%EnxOsbLx zJaZiyBf{h09O;+*9*P-BSlczp=;PmE(MvDoAAQEpHVi;Z7&eh&z0ULZs$pt!e_ot$ zXHn;vF{2yzjtWo5>IL7)x12`nH(uBNKe9esOhT1+OpC{ir)0cl5to7dmI_iP${j6M z@6zy}m0e1jwU6+ri{Y`V&U05#NacB?Ze=wgow3&Qof@`h^wI%lav;dI-#_kVKrz$< zt$St+k^z;4%Ni!kZLNyxqJbfk_WN%QJc@Z>73uV_<>IObXn!>mG;uv((1Pjl(_#R@ z)ax%{S7747Vc)w!v3-4P)z}igqmB9RCtNf<(Yj6OsyvuTX9MK-xy>*zbGpu?wf@%n z&Y-|0&$sTr^5-;VLvV#$b#uiu`>6ZChgtEAmtH5mSg-)#-T`^S_hJ8dn32cE(Ay&? zpx4IbKLVhe>A2pQhhG3&&DGrtrtWC|ah2fw0y1Uy>%QVRzSmU5_f|3 z{i?TZgz4E3$GhHARPGTyV_UV^P2VPKtdTXsbKk~b!y?NT=h!obkN7Y3^CUuN`nau= zbtk%?)2OOyStc6VxNx>{%jq)GzJEBU-r43qny)-P{mD{4<&_BP7@o}R=;2_kf-v5o zzF#gTILZq3djUNr-|bPZWCQlQS1;-fn#0)siN>BPe^zO))k=7u_2vdm@ZI)&i~mb` zGM`C{2+GTyOu!i1sl}##qF_Av@^sjuYuRUMw+ZTg(BGwE8gq)qUv>o}^bvf#@=BDIqjg$BijRIV@^u6nV6mp}nmqC{kvEO8*Pj@&LrrI~-*4XX zG{869(a>n;Rl!p@*nIz&zVzDuFw z#G_BAUxLgr2mL1$TsyiO6_!iaAK#RB&vc=pZ;Xp>d|q9pI93DG zOqV;dUq>4+O$s;_8wEGfK1h1e`r`aT(Rra!@&jh>`b~g>qT)xKh}Wk(NEB{Jo#S+S zo}gj=-w+BHO_-&9OB5C_72^P_t*B5Gas;M&<-kt8t>)0hB_@vn2=yoE86)}N@J~E= zato}K!xugCeR7@C*l&1p-y^}pL~{(DU1z*nsB0O3B3`kj3`^_aCX^WHf+YC;{ermO{PzE3Txf5LM$~2pSk&o``gM$T!n1Ru*{WWCT>&In zNGS=`ZEj;rAhyFHA31WYs!r9BBvLYxWjl)5B0*%Sl@Rc*`)*J87aGc1LF8UE1>O(v zV1!CF(PI<38pz9__*&Pts1`w^140p);0G|sGQOjJJtL64&dRs-A0!_82Z?(dU{P-@ z9x|EMUhvsnatCLjm!ZmO3AApiTiW6 zWEAsnbj*!-HXhAy&z;i$*c~)`!x*^k!^EBDodjtj1FKqF(^I{BGYi1PZ*ki zvK&0tEQbm}?%IPLO^%(U~ch|=+GN}PXVtZ>(e}JjuL5E{P=_rHI;Vqg}<(Pu`S(B zYE}RHtUw_*)XYGaDdNgA9zBCf>^dkqj}RYp`pZt9JO7axwfU@$u}R9Tt$zH7Y1@uh zuQ$Hw8g{O!x0yvze8j-1_7@<4 z;$O5lxsR&#G|+5NoLXloFwo;DHMtlBano-#s{FNkhOlp>h3+cmPL$$vLg|{8Oksf} z&p97vTmFO`gv{HWg9D8-0Do-@{e?62&=EcYt&X~0Z@U*s&nSJ8rJZrENoEZryTDtz zOp|}juw#efpL}7llF+oNq3jRW~SR5B36t z%U#LMeK!Ns;&}W>vk;?iwO!oFTUD|%{&**aS-+>(0mCkzV-EgPG#e6)yJl40)POTnJQdOc{BMnjnA9fvTew zbePW4fbB`zlJtA_4c6~Zb@$w=ySFWloy@eYQX^Lb)tTsdlhCxIXPR1^k`$nBi#PpJ z_IP1E8W_<13Q|-~h)>sWV%`@?iPB#mR^!fgLEP$PM&pbGcgTp=#_J)PqVpJN8Z<`* zngyNp^%iF(xpX(Embwb?ru@>QcI3RjqraHNW^l1?%X0Qxfa$7m?{1LiDyqz>M?ziL zAWY{5boAYG(y1>nxf!NPkA%N$Si=%Onc({92&B0Cg7-dTFE_De-QZLfl2O{@+&3v* z-gWjNzJ67BmFSJ^^Ez=SJ%vSA<=y!x^^f-*al{-PiO`*KamAjG7;`7i8k`(c?W#F| zzA{5WbM4{oO*!SL2{7m?$E$nxs2T}B7rY)ZBD;cUG0h6Pss2A~y=7EeL9_;n1S<~3 zi(8Ak6QsDeP>Opg?(SAxi#tVI+})kx#WlD?0>KmDac!-4?|Z+_IzKYAZMMzMcOw0o zl<2_z@9!BNhR@yyyQ<-XAm^%iqs*A*JF90pWyXthmXR-O5hJOeR2=KAzjH6-yVC}n zG2;a|XN6sO%OFVrKf&cG4bB$q|K#H^h3EMxT0js!v^{@LioTjS*;szBDzVTftIF8& zELGgd*8ml@zL?qNL`SWIqg`xRqj)fP-Pj3Wyrm4D5*|!Hu&*7&H7fIm)OEhh9v)20 z4m4->u&^k7Q|JNK5%{{-{M;08AKr8=+Qm~~N}Cm_RD4dYR&^+mRsO1%t!ONdMQH-? z@YWq4nckdslJ+H8*?#Gy|Fw3ZzUl#+8ijKfKlHOQdd+>?Sqn(KN4+HsZW^v%r*YN2 z1nO9>sfmU*uf|W@=gVBvZYn(+j<|U(eW|GOvo{9NDW}Kqz0PRaGk-~7n-w`dWQC{_ z`D8}hHr6$(S*TtMqO5gA-}%<&t`le{0~87x@U50@I)zml#>eYn98aj+Qt}lIL)&W2GV1<)8|&tO zA&jh*lgr`>=MWR~>zQB?%4ue)r-s4C8ylkh`^{D3>zYRm()RuC#f7sCO@P1e&`GO# zaDzI}tNzaBl;I#zr9(((Ic}Ni4uz;RJmQQxy~k>E@81-il?>DnH@aPO&}IkrtVFxi@3hW zf?ZN#!0``iU!#doC_dr2E~C?5PX@?2Jqq;RWI=%Vy!GACCDW1zu{^5|CV>?iz|CZ; zWvSIezv5q2_*Pw>XhQ54y1NYVnW+m!*Oxy4G<-ue}N zMc)x4cTizx88>G4!aQQE!Hg}ZrcL_ zDOs%5H;qq@mc{Ix#=REFt;0#xhFUU#`bdP#84ppHnML4st;m}3t>Os1^U4jnq;Drb zd+JsJcvo?xdWjFwd5v?Mq^-Sl{~+Z}bjXjd35PLFYT6qboJ( zlLFd0I$g~_JX=^UvFChK;$Wv~B2jGGBzly5GQAy*YkxReG6oVv6w;M0&wsn#b!fM> z$doyfdKzN;?NO8?3uH_cfex!N`hG)3ylbj6yb}MX1xO?On$$8Fuqm7Eoc8&if7S@j zEw-@+Un%971KTz}zU);oi2GN1Z&2V99Pis zep&s@IoZ>`sxx7(i;Re2RLU_|kwc3&gk+7wkmz4e?iIm$|6X5*d zQ}B?r=1%r2Z67_f=RWEF_9@&wdJI>v0>bLpoYQe0>AZ{keBHQs_Z7u{F0-e8f8Ip( z4lXO50A$`yFFoernPeYPx(=WWNc>ag{?wX$t-1$gBzRn-ANE&Y`V=dQ)w10?HiAh< zyr#Lg`-1>!iP3n)GaE6u7en55tCN0Sw4XXq1@xmSfcB<3QA3AIoEDO-nS5aDUTBX7 z@ZaltQS{n2?LR`VR3n?W=$1t+)GeUu%)8R>XyYh;Zm z;a>R~k7^IxpOd%m5fD8QOAnhhRBA8k`PMfKCK+rys(12g5gX*kj| zoXPWq699gLhb9~*Y`pwxmaSF>YE+9w$G6m_nO6DDW!g)N1<~7W`B?!rJ0b<^40kI= z;VX;B1f?4lix~nixns}xTh(N@{L8oGm*Q0xVs&4odjX%jV7)#tpepJDDg3B&e z5*klaS@z=li(CzT-mg=rsk*aisbdPX6Ny`Mzj+RVyST7!^-xrW=tcY;~q8AT;>t-0Kh|3df1B1WjQ z^4CVJ(PPKDKWjQqM$RX&D7KpxhQfT|UH_6NG%2xN0-!$iaWwg=y#E^RJkasf+?g!7 z6Oq?e3778V6ro_DCUGqSj8ewP&fjWtU-UqyCy|eH0h2p%Ck*WQY4CZ`-s(8rR^lMkxH2#=oH3qs zpfs|0rwBD(31AjgAl4RB3_JmLImAN=G5i4ZpW+JbTJ&rX=6%lp4$`zLT98-1ff0cm zBMoPwfOIR9B|`GR2pde%uhGB z21|d%Hu67jssh5%*9c{xoE#*Tcc=NSHKeqAx8?=7!p>tZO^qHWxGK;uEg5KhwqMsP zo=w8bj`aq{zbZikoh6c8IIctBu(MbCoHKVC;F&GGP z93#J7MXbZ?g!^$X?pTGRS5^q9;rj>KJicyy&=T{8umQC-n(SpZS#L@^;MTQBmi$+# zI?)09-#gu=&FF54fLe0IX8zYbTK#lY%dXbl~Yam&VTW znc=K9{2R$$t&qXD?O&D{b-w@u09v(A;illZWG zcc1`CASov@+(nWhGyqgR=q(ckocFBahR3|rB3H3p)aZ>scxx1of|+A6%Tmec0Laon z!}75uL6*d-Q0s1ie9s@=5OgD#YQC3kraE5@D@VN6n8%)?P*fJZtbo8S^iEp4F+o#L zjOR1ALo^G2bj%IdZ|9`O+{OAoNSu+=VZ$;CX=d)Q1>plD8Lrv_jKUS4Bb&v7h%NoA z`HheTlz#PN@r7v{Hq)s~l<(!O?+Dkv*@xYjVmMPGNr?-|Z< z)=1BfFDK3PJhO*LsP@@%aDxKzu1M!G;SkYMb-p6MCoKAh9^Ya>AyUV&=TfFWW1yoq zzoXp?ezY#k%4iG#wFcND(1X(9I;FXx0mMm+>#OMs>3-&TU^*WR+m(RH6m4mNOgJ zPotkx-H|LZYH_&*L`Bf40n2Nyd6@P*@&rXV+XpXW8n~&&d{|pj)q+;w9G>do+5k7aK_U=p92*8AlvTBtGNFtB0gIr1fgtrxKxwFNW%kYB)01P23C&0 z%Qyx6Yw5iTGfcJr&Q}Ab#3Pjly?dKR;lCqG3HMh!4N})v=Fz_o7=DU~NE832`_&ge zY7L)q-oAS~Z}~3OU5pP#k0B1r0>tN7ohv<;PrKgR3-mN#Orzc7;GD7`w0i1rOCR<% zb08n?iUOeJmu!Tpq+ukahE#rXNSZyYrv4nkqs`24jvtkv(k>6)r8?^A5rNlyc0r~u z^v>y2+^t{8`z+P--PFl?$A?Ny(2KLVN8(?=X%l046)?R$WoL3*!bIJ?B$m(u?^C-S z$+OH-bNDij3UK|w6jb7^Q;oKWyQH-+Bv#pRFdLeIM)g4Ryp1fict&ho#^d5p_NE{a zxIDtv=0l54AoCe_TKhC#PgLb!1mPa05lkOiuvpK#9Vork8dh`=Kx$e@ze&bO`Odog z;=;?hihd}7NN-rg*6=IX@#r=%n!2B6#VYW+uxhZWipTOvj|h5;rW>mfk?*sT#`?fS z6;##qk2``x&ET~6x8q%|Y~#6{jI1nW<%eJLc|9yxQqwcB*zR8a(9_ z{4ENg44sW|2qTKonv+tsmzFuR19KDKAcJt zPXL)DkF3JT?jyUiD#^>3&Kh@y32MKVj?7{O3T7U{qvVR; z7(am!6ip6&XsSV=vi(WwG`#H>NrhGKK*vUOw3rW@+Y6zk$t%DjfL+t{b&x~iB(6Di z-Cy<0NN=d4TW<+9JzNgsSy#3aSSvr@z*vdvcZ53`*rwUb%H3B!54WYv_;eYIX6@B| zAC99vUf!(Fp5zr7x-Z6EDq|*mdnc!1>SQ-nMi6j`JuvRa!AE$Jph)yrnh+qRKu!GA z5$iw}IfJHT{H629riIECCqSxfe^->=>8FoV?7Py!CdGd8Vhm))NfKjrybfR!+fd0n zFf#B^PT@ogldbM3f=Ii^kvsD3KeX@H^b(~n42oVtxKHP%mvIyoW{)r%_thz{vM}Q; z2db=X{?A7=GQXJ?S#W#ARN*55%5cJNTA(2#s#DM&QPx;0ekc}H%mI|8b(l2Km*fcsO0zTmZyX))PPL8 zLGXD4@_wpHuA^^^yNG68eHFKT2eubnpp8BWL%Z&;_+GeoIOCiA#{Ss=$8Nf3-e~Bx z=O0D1Kt7!~6BZKG1Cw7rzK|Sli{eP5d~4TJ{iMxDmQ>>}4w?J2GeQ&erU(+TO7eW! z8c+k*RdRs)kGnCQhS{B(_*c)ftQLndK5uy9zc>aI=5clb@>G8HoFWvu$OmHQ08935 zNS9+=u*F~611=7;11K@?S9%h@utN1WDxWC)Hv((PvE0Ta&G94nwb9}~;y&wV1q5WK zth;&JI%f>rz0JdiJYmMvsf)89J;s`q_8)_JJ5!|Z#Azpc<`;Az*+H%R3PQ|l?ae7O zCxI4y=^{Zs=5U&nfN7Xp7HQh>z&yKOPe;kj$4A}Jh>X+}Ow>BM6Gl^UUNbI{N+T1- zz=eP$F4c;Khm)w_W4HU*OexVPf7~9JM3RGHC2Qj`D~bnFfbyORRkPu}iD-fL2u8<+ zLU3+xtm7Q>>jk&uak+Ab`{2V;cu2bws_EE2eA{srLV2|6s*_T}fNgAl6A49{<~h|` zwI9Y`k{r~%W<Ipno$Q9dCBi>A^OK6sY)Mt@akovEM!4f37lu(;DEp_&H<+SfT61JQeDDFYuQ2<9WVedPE^!-t{BcM;5ge?N(h|hTes+i!dW=7v=Gos+lsSJsjFx& zM1OLMNQO({M85UCLeab5f||3n)c*dKI73BW?M@zX`S^-j&~DQ*GmQlBoqI8~Da}h+ zzm@NjSi6wtxI;vi!Lu4>-HL_|UHm+o3&5Y3;2okPZxXCi-2R=8P+EY#{>1;#SdI7u z47y>@4W1n2>GGl$y75Kov!Gsw{u%d`e3?nuLb145z$e`0ATfWq%Kq$9hNB_72JrN} z?y7zBbLOLs^&HH6k*B3p-VMxh-_2j$&3l-ISSB+a+lsjzH$2bf%wOxD5HNCl$XI8* zt-)s96MuwYg(2hSR*u|R!1W#}fm!Dc;!gp)7>s(HSo4dC_I%>NHGvs8E(M-D84*B8U=dTq`FxGBPFnmeC zxvD3V5u#o9*rTBFKF#KipR7pa|RN*EfK*fLp* zMVYs$%>w41(Lui*H$u6n-JihgwBHo14|Uq7AJ~MQ%C66Mo*%;8D;^$u>|u2?(wAKL zV#YlgysfypErhs2kC1PSVitO@4YN7w-QK-{%7KH0iP`uy!WYyTTAB6oq0e)#r!zJxx8{*~=G@j)sZvGl2zq zRrWip0UTyRfQ6F-@A~eOo9#_3FAJ~N20b+)fCNh#!f=a{Qc@Aq`c_q1bC#kt!(hd9)<$fbgP|Xnm8L#*^=BaeYxNrx#8VjlKNN7@yi)y?>O#_#%l-P z7-f8bR<=9#Wqoj8D!+NK={!h+{9&`@uGL<8FhJ~4G}Sr3C3U-*ER3~ zz;MT%9{i^lZucoh$E`JjbYl4KwGwJhJ#HnVtc+#TBe?U8fs?WgI<|=KYoVRGmS_?s z2UK4}4?xE3M>ZS$h_dG-YE4x=E0EF<6k#=kwW$FbLexvHdm0gSVUBMP;y;29AjzDW z({fv}VCI+3foQuIobje=XCg;$!UwlsWCmQ|M)u)I(!U_6 zEmqf^$nRh&CfFUxfe{)$lp%{N(kfqxp#Q3B@qrWpCP8Uen`eIAK^^)o*f?+F-{GZb z3@G%yrzV38|Lr#C8pyQ5#k$*pVpRniHkCF>Ahl>Ks~dIJ0@Lw!-0YtU4>_Mq>bOUHaf*_G+maiE%Yvb<7UWYdC@<&`Q;e0HT z@yRSf*%~k&to#b{$gsW`n*MCiQ>nZgt#qnNe6C`Lb}%QPD!g>_ibC`Ek$y+hvsEXg zNpjf3g-tUSVHC<}MnuFU7 zJ6`PWHT;V#x!@1``@=so*=mSL|LrwN_{sBr_oJ>nJt@dYCgqp?$VTmBVpm<%^@=_z za4}|sf%$s`O^jYeE6}&Jc~jfd*pHbOFVnkqDHSu3PDFg*gK*uotVD7^#B}kEw^VP3 zA4A!+Ossdm`@rINBxGa(&b-$4;~<<6Mbnc=GOl19XXU>Lq3!o^p%q?R3houdCi#r9 z1XDMKI1c%&fB|V9P$r^ZW_kH1> zR1q(yDZR{g;BLrIK)k(8d|yj>@ssiQc}*J4<8a0~zs7e?-PiHPh=#Uozzzlt!g|22 z9L1f}^@j@16&T4>FAOJDtC74#w)OI4fcGF^2!qzYpznD(p<8*Ep7rYp<&fWgQO9_=aM4n3P6k?SOdMt&AXuDV6Aj2}BRq=}coJ8)- z3NR<7lOnFlXnFkB+`i1NA`59`@bCA;Ck;fR`5Du_i{F9z z+!}UlT4xS{9a1TYKdaYx@19TBzAsMX@rvI~urK}_0JiOl9_9&yjG8k1%h|YrU5g%# zUkGRCfe3-N&k}jXm1B`#$g&$z7~_3IE^$5d%>bM#wIR5;i~TaGy?Ywk6hP)^0P+?j zBAuP55C0}~44AZJ8iI~Zvh7G5woH7@9`&6grl@83ZKCSe7qdKD>c~*#nX;9ePi)k& zigDSB-)_J-v@O|H$3DbmyUJfAGgRlDpCfB>xIS#}6VR^so0Z@7HIAo7!Xi6>z7&gG zvjFn$99MO|OQfy8rt8A@lsKw2b<2G+a)#&Q(?*zwB!Q+0Gb<^{()4bG(AtPzV=lVmq)PMW1!Ea}RKvrlkC z6Mv}JY((p{$0w7kCH`9(^F+J1kwn3aNn7A4ozNBL(wn@;wfV#eP%huTqH#&Rr1eWP zU3&oA*bW=|Clrv*;-2*c$n`y@5GGPco-`y+06?*%PiAgrMBQ}RzV-(o`sS8K*n!%P zR_1*+(ZTG+yR);pbWnmz)5T{19X{!L3p3GP-7%--JvU}7NmP@J(QNNs(0c2n>^7(W zS>IWio9*(JNj*08iHupu&a&Pa-;=IuyhxB85T#pE}Hwf>R=9 z3U!YNZslV>5Oi`JU^wFp(kkvi=|ELZaFq)O)?PzO}m4L)8`yz6?psyWG|qaxKbOdYJJAS^P%`hi|b!pdu#F29MjV6*34YfJyPA**}koq$bKI>l>+r=B@V0*@> ziFs3S(O10-G=m@Pm`1UhTYJy>Z|>d=3zOHV*R5j14YLtU#&om+22lha5z`+Lge-(4 zAG=wsyjJes5%Yfm`$;QAkqwT<00&_m(XaXqMZX?zEoz&qk!=%LOBMK+^EuR)AkCRp z$=9@_^T<}`M`zL=m}0Z+6N-5RDwRxTNmd{lJ||PT4tQ-VjQipPQ-=&bzQP~kn@68U zO$-kL!H_7Qj-l!qN45a?IaGr2|Q~w=L=evu0#muWy~w1otq<>jCwPb zUtg!IM5C>w|5owL!DmW?Uq&a4M`B>eD9-Fi=#zI2cp>3Fg&*sF; zlR3npz>6;jj?1U3S#sSkKcv~JPFcu&YSP4&TC?`=YfnmgY=wZf=L&TR0Bw z%l2Y=-FJGgg|f2<`@BDSbsZ>ysu?e6Kt1z(9Z~sMCQPs(Q@g{A~riIo%ION;#h-!5~KhOq|Fzos(5T3E^rAN@bFB*4T7f#F! z++Q2*4EMn>#>T3uaWk^2+Evt7PWp;O zc-Lbd(rb_xM?_rLkYmYV;?M-G3&L$0dg{ z|G@vkRL7MoI2%1HG<`*wO=8CakdlhckKZDE;#O#WVxli^T{?!{#b z4JW{grYrTkvGwPw8IbZmpZedkDp)vdaKQMbUOz|SSEStI<=4G|uE3-*C&_wYNR)Wn z=bJtfCZ48=tnAUfoT)dhS*K@S2)~ii*g(cO>|=%mVpT zWK2yA5&+L`crqr~dTuhu`HGW+tD$Fn@VO$buZJD=c%9J_t9VhHQ)MR?q;&Vgq8C7k z-8Gf!IQ7$fsA_aY#yNK|tslUb(eBtpg?|Nwf$d{(M?hYN^hsR?o%Y8?32t(UM!mDHnFf zZ}KM$YSuO$wy=E}PH;vV^yacoc~-0tpu6fi(gy(P5y{d}AvrNytuV;E78}306K%*u zft~QtY&MDC5PNgFy~EDj(cQ@Psy_HsaEyx9=RY)3Q3{!+*0N5(K*)nIosbJK>rvP% z9{lJVB1~kiJw_cXd{}p!Uv-We)p673B+>bv$cuZ*pwT`P8j|=Wuh&{89+=L;oG10X zM$MzAwRc`AcJt4v=jOe?l6(4E8Rmlot9o|^3$Cd6(W!+B%K1@iEWcB=Sk4?!)uKma z56|-Ro1XQ*4K*}-u{SFB5&RLZ`{5W@BQ6L;t~K5}sNHad{a|VjdNJKywT5s$%G8K_ zsZ{w~^ICrICMl%E&{T}}r#h`6rkELnH6+4Ci@`(yt4|kr^(FIgws=5_!!j>Na&zsG zNx?Yut?Yv9p=dJxRg10rTWb$^LXTCW;D?f#`jg*|6}29tn}N7csk4rg+tE9ez)D&l z&JY(y3WMLx!x_BC3eXG4h5&y(=m@|>C0s?m*(Q~&ge#Ih;5~CWbwz8K0NW+K*#)0e zTld|`>m&;e>(cvu1>q~g$!wvQF$E?Za~6II^!eVy#mMR&^^xiUI->6y@js-;0)USL z@k_IE#G7O2{n~_{$?yFI!6AlnPIz(xYx08!??ZfpK_66XnCYj$e`SApHo7NSvDlwm zF+G~_lLa=}vmOe)Ov$1rujlOI@&`IHb>*LWA{}O%l)&}v|LMwVB_7}u`+b~p$aA~f z&ZhYbNZGr`dG`2LSaUXLX zyN;6Byx_@w_?QOL514M%%-w(2?5M!qCq;tK+B^LLaq^~eun$&!wO$b`8|}t7Uf=rB z1n4KB6*!d1dlu#T9Nu?_@D|lbxq696VEEB2=`6k~d-iA>`M4>e+d6h&k=r{+@-%JI zYHucBK8KwuOlNI6xzcw=EBC8uz&qhDA++3zeu*?_cI5}!!a2jg$dA9XXd#4O=AS9+>T8PZSh4ycM+)a?5sv6=sERGsxt2uelg(nG(+>g?|&l;n} z9#3x3DH;MgwH>O`6$ja{nYq-~cLZe)?sU$M);Lt^3H8QDSx%6{)@_A)gj~8l(i^A) z!Wben4<4w!ezK?^^8LtR`YTg@kPwiemgN@C_$|eM>M|ZwDE~A(QU{&#Nn@GLH2cS8 zIq)r%-`9&8b_v|}WMXPdoD);o`rAj&r4s(YWNgc%6bI9A=6*1K>D}v^90nD!vwqFy z&Wie?5^f<`@CE7$`M})cz%L&~^%pF|1P_FS5$X$CfJH*e6O}&uN$}1@XjO!Hi!*hZ z+=PvB?-+n}2O_+#D1sH=%Jkk!z_DOTFQa!+bIaU{++fo@Nx=_K%d5S)_;d%PTfo7z zx81hjo$ia?6%HX3vx+nW>O=Iv@GALxf3w0deIl<-w2BL_icxCgn={+J?8n9^7smjK zhm}M{e>hHP*q5#s;V+ka)y}jhla}MkUo(0{@x%dX z%WHX%L4>KAUed{m?A`R`c>zCk!0$NxjBv_+$dI>inMz!|<`sA7)gyZZY){MmqyFOE zUsa5J0&Z1^(RHw@b-u0JnEd%Mv^R*O61n>dAu^lpXDm_FIdXru_VH6lE`kak4TcwL zh}O`cR3w4I9988|0C+tAcOz9-5FwT#?h+jm?7W>)9_R3Prq=@mN{`uW@9elXLOvl9 zDEaycl91($Olg_TIG#CM;Cv!(ro@}i!~zdv`|C?z&H~T#r}UcMH{9P#@ij~;YUTMu zbkyjFnYvW|%4|`G{jMLaApIgn&aB|DIP(n0e@J<7!mz~z${UJi0|zJ6-+4&cP(T_! z+VD{8lgMN=aZ?lZSx9dpFG10uF*vg>wn7gh<6D#6DR8IpZ+WT0qu@zBqL)zhdd20R zkv`iMxxB~dAFC&&o;&__L9}?4&wV4AIoFxY{lI(VE%}0QoL>u-AyKS^cWi2LA-Ji< zKcFE88_p;XVDI%Y+ITMnL`2icOuuN_S^(YWd;sBxm##rSultofszLVm=nf;tDK5YG znFsVI|pft%+dCYecPh4S}c8O+#{BPd)+0 z$@T`Y>9;CN#`9XK_um15XZ7VZfjn!%aNnQYQV7)_4(nN`_&!!k*2dC2?Syr`&o*}_ zZOLGmbmd)CuP{@=lHN^Fl#$?brs0?kvC&;(=vobdl7ZmVyln`lh%dN3KRCWkdJ818 znKP#BbpUP0^HMxdzwLGG;>({-1{%|4n2ZUhf>kB_C5|B8Zm&Dy#KYZgy zTkfUvH0)|+1rlfc8U`={rY2HY3D z3tn9~ut>d|PNmk1T~~#wYWhP>h2#AVA{86Wq7K~eqyIY-fHGqlOjqAPZO#~`)wGQL zqU{SrvZ|>Oy3tsz=RMRtz|9xKNeSgaqo(LYUy&P#3o+@yACw32EDAs)Z21EVX!L4k z(TBaf+d0)+M-Ho*8@S<0K= zr^7k<-#uqJLMmxxC`7#bRBWp{+jGMSOK#!-3~W$CCR1c zi-w+(2RtvE^%)@B>xoGfYbNey0iZjeh!gLaSlN&<#R=*-Y`NuVq-T5GDt&ykL@w$N z*ri~yqxS}M1=iyocdJCd`*8w%xPu;0ar02Ho%jB`93m>SWfxxq8koyPtH(%>M;SH^hNuVE#YGkVJJH=5flpU+Y(-35(}Gre*1RsamPV7oGa> z(sfv++*|da{ZfznqsxxjWlB_y3d$KJMbU$?w#(@Hryh%ONyupCl& zHHc}W#jo<9@^+Vep^&63T9QAz$FdHWu};{D?Xn@04bTl`X^KEVlIt7c@tudP+G^>) z?jHPlZ2*0T#X#=dxP39&0VAigs^@*LRI7>bH40ckokWdC9xZ6x3wk^L~H<)gq-aTYy563RNHc>UV#{pkSm6kw=+EEl-%dP?kKIiK6t6winK?Fnkd=Jh z`)x34co4xEgJTAs9qkIul7sOX(dqcn;)qcZ_2Gniy|#E2H=Wa1@kgt*ZbKpYcEWkA zzRAJ|Bl9z*T8dr50`vb6V|Y-I;z-!Fh3jE;%rHa#4jg(C&~Z80{0B`Gu7Hx&d#I2>z=*bNBPR@0Zs&teQ@irqVJ{CEOaI;j_aK#l418H>#1ry> zTb+#J?{m-6buW5xoPC!YZA!@|mKM?bANK!K*stfpnf`T=A28P9Q6?4XN~z!PKBvms zXYbEBqs3ZMufyNkENN2kAca0&7YFORl^m#Eg{+=hF?7%-e1cW~hkk?pOU?iFxWv~8 zp`^}_EUHszwrY?ne*<;>^R-NX&*#23^3+{^_aj2-_J}I!WWdsc<;p#22*bpKno_5LG zsj6bS3^XpfB-oqTOAt1#$lELbe^xue`H$7isMTYzUC#09>`vCID02hPyck!g%<_-R z6XR`bwx9lx9SV;44?d;%vLt`_tYjSa|IcD8{%E!ku33zf_Z8*k=p`U9E(b7w zy0WMe@^HK+tbCLvZ(RECDth&_xH*LfX~rO7#er3AB$b+B!65|uA3Gu2=hF2=qyO}X z|LeqHlOrye_qni z+_r5=sj^GX_*C=%Xq227sl0Eh_bKFH)u>=HoQa~OYG@cp%?X+|R1ySU|Fo&+b>g)7 zy#K0>5@%7I0;ZgC119<6xAp7{uMlHAv!9cjZ~m`$ki@t0dOd_wGvW;x0*kvzcJE*g z7XAK6DXr-*|CclgB&Awjw*;wjT7g;O zpOllF6tJ|)RfQ!L|0Lp<29b|l|7-|pgN`oBRM)fR>^5@WTxq%HqH~J>w}-)_3?8uz zkR(b4id+_O<1@pBz9Wjm`#bm?$6Qtm_2t{YUDe?q&X-Zn4qx5Xr(SQxdLEHB->&A2 z6cqlKQ`w?PtdO)oMa;kmHCQ|7pYe)bhSkNFxUO09?eAf@4LxzbA0sZXsk7UtC(HF7 z1oEXo4G!xQ#3p*}_N*|AL#( zW}{GVj-_@zR>EB>dty+zf2;mZX4S4Ki!k;{xLtKd$39+iB_wad`GWsj7|Hng9xXK` zRUYq57@9tVkq%NtZaI@ei>Fe{ZnDpGG51snjPr5dh`j~5ceE}6xpwVs7Aikbw%@5a zPb<4|t?DdRateU&{5_~j0iyUpJ_fG`K;Xk`K>H5y2z48xU!if7o-d}zZgPyKm6*P;-1LtmP3-FFmmRKghOmW1SAohR9aS!j88XTN*`54Uevx>9_G z-04cqCVSBAR9?ceSN-8@Z*}fpTru@3|Fz-2uX`JK6k0x#2#h*rZlH z3;kBH_zylbK1t)^Dt$pQE1%ldWpumy5|F2iymH;mCBf-Rd0Tt%T}SkJ&z5@bP~8A* zTBY~J|369Z&@lKcN4Z+;KupykC;s22uaW;6xD1s&>G8*!J7_H^zD>v5$Bx~*f&-L! zK`#;umTQs=CjJ`vd2T1PMJwPg?->RIbbci~>ksxJ?8SP2$=!Z^GI|qocYT^47I)se z5i$2La-sf1>=!MP@!5;`C3&v%J#fBT6?27ByBegU6v&}x*(k!_!L=OH$WI_hfe z&1m0uFnOG1x@ym6SQoyfxmog2|73y?_2!bS6}%#NTqFB3gILs37J9cnn-RWhKI8ts zW9P(w1o4LZ@P&(Knsob=OA-mn?!#t2KS|Ck0Vv_4=@h)y$DQ6AP|&m)*m8bUU>Ko+ zc-RszSN<(OTW9T#^?_KhiWdaGhSehCuGfniiX0c9^Q<<1am{jh9ISQywi`ozI8!xe z#4R;zLH)PS-}m3%?l8rilJCo8Lifb;x^bqVy|Bi{*wvgpa-JqLCks8E+s zD~SKJXg$GM#0ElW6aK^D$^`J)tHbOSxxSU#3eqtZ zKw|vDlb-L`LpHM2Z8vM8>ag+(ALT&d2{?Z|NS8QHmv|{vJJ)NVcIv5Btak&mKZgh8 z_hiUnotU@Zi+4Es!A_0m>%BEsefPL-bscPOYxaBfUPg95Zpr8j#;#hj+Kfqc$^^YQ zLWwC1)|%pTItM`4-$By?$rc0;?d8#6T8YpesJpD{8!Ilf>r+riwzsiDmq!%(mRI*B z+wJknm>9@|`pos*y)hV0*NZpZBX&pHK$0Bfp+8n#CYok(0vgToSsxL`OuGUZ6W<)q zu9#$>SbSL3YY3o!;_^jYY6@G0`ct-lafAxCczncox*kD-R?2sv9p7%_UhrOA76xj6 z^7NoR>GA#=NAz<-b;YYLZ5-iOU@4~no^G|;ou zoUlUl!9?>XyIJOqWO#hRttArB9z}bV_{Lo90vYJRFQ>=3&dwlTkl+>>ob9f84*S6Gr2Zt_{A(g7W5B^MK z-h9Vf;-iNZZ}K<~w;VN2AP*H24k^s)ZRrwthvU|L1k{U=++iHF!nOMF+H zK#94M7vGb$kqDH=XimQ%)t>LGhqv5>+LQ#XQj64^Um`=D^ z0$dj5uQy2~k`gvdgP^i4K!?@SMqS%1=hM9m`yTk_*X!r65_5qs{#dy_H&0?5cuQ<~ z8uvk5QO@8%rp(`mn!@fHJ)4HVL!4V*+FGm-lGjB!fx%fM{1_D!(^DJjU25tO1|_uk zJ**s~Cx|VtsmtGAzW?gbV&fI-i|HipUUDd?O6RmG2U-JN6uTiR_od{`glAYw2XD2TmAEd*@6+N}=(E=PM|HpBG)STLVzX zJh?exs)2n^@o&%7rW+xVYYA5%@W-#Y4ontP?c21WoTINuu6~k}`F2Vz#$ICTox%Zd z;j;_w%+r;GB3pBpX?9~Lh{jfYln~z8&Oa9@Td;R~==HQFh1k23mX)uKG_HObHL|z;Ui#*s%ZZ+k9;m!54fnAT;!)D4W`Q1u!9l z`i8n^t|kzIB9a)0Cg&(Ivp)StQ_cL}2CEmg_P=A~*I^ml9vMgfTV(rTOSqG{PZUVB zwq1Jg78<|8xg$=n_}k$>!*%a#Vo_Z*f;Jk=N$<;7; z^6@I-OZ^p*EwtbRaj-ET5@IlxByva(3EPYR9^<0rU1Sz9ss$=0u%G3j{h>1Rt#>s4 z(g?5V$#ONLt)!r!zhO(=;Jff}$J5D<%$r-4pERSNSYO*zSYept83{d4sd?rEntIx% zqZ2xd)wDkr-ZROK#DXRhy$x`${O{&!C{)W6ZLO>v#lTkP7kPFP8ei*Oc@u?9O8Qf=1lkyb`P~z0Ns{3Uob zD5DCf`+wMatFXAXW^FrIfDk;mHX4GvyITm55FCOw?(Po3-Q5%1U4t}kp`mdJ65OSc zud~*>*8A_h|06EWW{)|>sH*3=t0ohM;B!jLr**RRromGqrZZw6m@BgmH)br*U!#LVhL|dl~8nwZKaVP-w%>8yGj^tgcP?Hk6?AqRynH5p_o{3=`xgb z<@PjEl~tt_^A=rj3sQbL`OSrtWLpXnP}mTe*Tt}W)BU`$y!JWAvTJBN?eE&X1-q<$ zSOm6x0+)CkoQ9oJ0B#c3Vh={lA=axMCozxvZJZeSK8smcC6n(qP+lZDSbz1}Do0jb zW3kbk{H6ICr4~Pd*&VGlMeR5uX`}HVg#-@hzrxV^Rc$H-vQ11E9INkz&uTuC$eXR` zR``Fj@Nqn10fp3(UiC!rkkpO zI!+$`G}g{yrhGDO#vxucBpg^m02|aIa=JD3oe0s80o zY}FGtA}Ip~?J51Nmu~PgWUZ+iH;A%ES9w6IHc&XB@|mxzrqQj;l&VuodhUGz;u7zM z&e$;-BcM4RvqE-tqWZ0YAUctefkVRG%@v2sr}n9M=&{6P_aoBd{ceFF9w8dyRzQ2O zPJ7O~9JZU#9uqCZl3f+Hjb@OG0PKkO{s@-=sYWXNf(pR^?<3ImxSO%M;5FfQK5Et+iBRyB%94*owCrqwk+k&<= z4Y6M+zIssB_u1JHcZ?QUUvb9mPHZ|{ji5KpiM;K!zrnTTZ?5HrqvJQ+2W%7Xexdr6 z0veKfyq0gqROz~oPbZY_wsDNUmcgWud*;70+Y4t4*2U90C}KU@8Rw`z#JPCZf4RUTIpBg*ET~F0b-B3ni;~yqJi^Lt zGZuKe@O42-`dS_}*BIqEcj>jm&o*>N?PZ*RzC#c^usa_$2}z`k>U0esw6VK;@AQQC za~YT7ijBR3S=``MHT=?elhL-hv$kPuwPISJ!QJ5T0r{wTv?3?C^oN;DnTu7v4Mh0X8^zY>Td*f z(9@~4);anc{1dq5*T)A+z~0Xqh$*a;?rC#1NOb)@*FDn!0$j~p|E&H7Kc39O$QB3a zZKi?6QYEjsTiCUuLJi{aDbmokw9TsrXtjW`(Q>i!_hGw0rI=p?JXf;6JB=|H#yf&m z++Tbzbe};Lr~Z;cp<J%5R4^7>1fI6F8A3I4Gs&=+dDtlfBOpwnp8NsLfY4*Kqp7Xhq*eo z%7CX>!a5`GGdB z7C*RBoOl@zpk;;2HU$wlL>7sVI~?QJ_-hH zVNt#%gpVWZ*u|g|%bUwSZZY0iEf_{+dpU~3DmC1omx^c+nc`qL>i<2c-3M5^aS;XL zLo5sI=_XIXaDS4{BMXy1HHPJO_UJveFrL%=dr)?N0|e!p#>T}o%aA@@YG^W$0!^2Y zhEit%^$JLxXdEDjX@WMdHiQGL0ibBo-Oxc=&u;>maB68FYkF-yX`Frxy_>W-J&$i! z5$!sxuF_yay0yTcOp5-7n1U56PC{#l2Zge!){qQQC!AMddkjB8?xVDa*3X8~o*XVm zL%aLO6FfS1i~78_-m0?GOq<@1 zr`%v9|2i-Ur>lM}I^Eo<2cAcu@-gfx5kqL&V`&eh^#-`D*jg2K1`)k&aECd6Y|H1V z#SAee(vG>gBTbp^?c8Ul{n0YZ1&sl*&b+OI9_q&)p*cpgE?AYCo5!*h7e+^6^JLlr zx5;gd7VahaaKWJz#Kf$*X9aZc*ST)}g60Nr&g!FGgwh{x1qat$xO&^!GQ#QB8D zX)4>c6%*pMUOIa08$DmR0l(G(a zg-y5kj;Ao#T@+CUS_(f;>>gTq5!-Z1^E!E|OA(w*bB-;q>R~ozB0WSGl6_}|a`59v zV3D7k`Y$4mp3jaAzcni=S3pVN(amCDtajr0nW7 z+S!q$f_9AP=QU$V8(JeaE$N8U#LSWCXv^BDoa4}2<1ilgA#w(h^(W*0Qj4)&NO`qx#qeicU*`_h2a%2hiHOq;#pQDp2f9GA z4GOKkr&PK8pIC_V_@~oI0oPQ47j9V@j2dEGz}IIR7)PHu)NqB)FoHC&MnR+lew*7m zd*U(QSP+*E9OyZ7+W3%99warro`)4rGRq#T9me2*DQ;V3v(q%Wh50AL;*~X&G5tT# zY`J~GyjcfIJui6E5_pq#1tEl{26RJ#Z2(b61?K6GitTD&zlUcrLRX44V<^)|d?^Z0 zmjTOES0xZ+-5bWz-2M}d26-XG?^)x$jPchW{_=1F+){W?GHAb`@VZ1V)Ed-l$w$R!Ri0lyfr=O^`8)05!J88fZXC_K8i_Y)frVA}9kqFZ z1RZOzy`mN)ZwEGaU@XopB7}neRZvZeM&qI|;%O<(8h6hgR zRKhMf1qZ$TwxO=mc1ZKA7FhHRP5V?JIj1j}xNtt-=4}}^RbXM~v*Wuo(Pw=#Dm@A= zGpU;B^2u)F`$yOc*H-Vm7RZGr4I?Hj^{ryl3s!09BmjTF%jGZ0jW1AwmF)%}sDH-O5^!Xs9i=R@g@8TOGT4Q?;<4cu> zYE>GVQH)*+6m<7n!t;X*QPr&Fb2foPE6{Ze0It+EyVA@u`{SqC`5e*}0U~*1l{nGM zUi`GNpbsNZ^q~&&=9QdI{nQ(cd-j0m&@aEP1XPMBN1$D#)ICQ!(Yi}031*6{InMh- zz5Q`A?-6l7x;J4HPBOa>t#H`T$>;IkoS8JAk@rdIme3{|!(o#TKb>IOdkP_@IHL`Q z$zvjpDw;$DqiQO-uX}G7xuSVDoV|BdZ1d@%d9w`r-qU@2yFz{RrQZSnDe%XM#7e+%yfsR z5(x;B6t71dpNc>}vK{8D3czG?+=2u`;hn%adI_9MjnyvI{q*ls@r1KTbOi+guHjKp=tj7yAnofd~QETk?4bXl_aDxPpWW5zTaw!q+ba6-LF_Ds_j=$52a%o z7jfO64~g!nKA}vQ*SY;@QT?JCB0U?d`_q_zQ?%jHHObOVN8h$T_vXQPD05CQT*G@g zpir4+Z*8}>adKBQ0|YM>;)XmD`4d?a*dy_U%D9^`)3V2sFn+r}*) zkis{2k;i*_nXOlhUxnRBt-T|@;5?pt8O*%zmoS6s#|qB7|2|(;$X$o`?+)|gf941@ zZ4G)1Af*088c4jE9)gCXG-vMffd1H*F*Z6#%JeCH_eG+qdo@6&za_i*m)COCX>-8x z7Nx&^teg~a5MG5a>9x7`lu;lbX3AWIYq;l4npOs0DYCpZ4XO9Rm4J$ve^d{{kR zPwN9Y*>Y*ANAA|idd!f)ueBv>&b!nxN|tQr8u?h)L%(B=MFg?c06bkoEH11RX_!I9 zT=C5r>JRmUHG+?E)TJf$4LzZlWq$LW?Jy1e)<_h;dTK zqfxOUm;1yK>nEo%Wgw&&yDsWNEB9+Upb+C`9rI=->q+LF1$dE)A_qY9@xE;1 zgnb4^l&!UrHnov|kYXeJ?dzG~*jsx3k(TV2uNd^j=m4k4QjkiOZX3FmD>%og`TSKH1>>qlJkNzFAn2d-Va@1=*d_sb(kqt=Uv`^e#i zxa0g$XGQBNGU(PdX8bN}xsKZEQ37^m@E&5m6x5~_LU%9L)c(5XCMKQq&qyeIK^+Hd z@o)h|A~~{3){DJwx1#a7z$z=w9w_bu6;SFp_@&v*3Btm_?h7%$t(M5yQHCX4IF1ugNZ4)*uptVe{2sZJ0#3xpwO%9$?vlPr?QYa49 z!6{*hXYS{Glxxz5c6>Yw{xv+peK5W)PW625JY;jjy6Opg>%f#}mua#=7t`g5 zr5}gy*kCY$*@mz{Bj#u;=~2P23Jd*Kv!Ghn0ERAtK#_If^-n&aYX)57_@=WP_E_#G z_qDAPz_gb^m10w{&W}HqEK&1&o|9idaPbndNwemd7JJ^n#N3HW!_#jOn9&Ek{ZnS_ z-eCP8p!z{*(pp!Sz-=qS@;KQ28j-|zc(G$$$P3#My@1(;uQ<}WkW63X&V!s;7o^Ic zkGJz282CwKV93VUP2Pe5k05n}OVS%`w=6cg=mr85=<@o#?J*(qosw40NQ+f!$V|)} zl4^=u_?9hd6sbhA5PR_9-aFZBf3AeKlq`W^k^702n7Gu@Ity~%t zbp#_HYkQL%#sF&z6qd$fCld{Z1J)zsuH@*KFI-R-Uq*yc8hLzAnQ&QnIs>l(A0dsJ z=n5>QgT%b9tn*?o$vG9#n66O6WCeddDTJjtks!2E$W@vSzO0KCNt5=j4zVA7`VfJ! zzD&xKUOnQdG`sCdimO{=i)Qby(gdsvz?9=h1P`@MR7rwi)Z`z4={GWbO<;h>Z9rid zV`~*fGflDnIxhh(jpRayU#@CDRaJ=T8#SDmt#uds@(+LIA$eVYJglx|)X$=){qW)z ztWznJ18;uNMaQH!NOBxmg%B!`@v%0rxClaQ;!ZJ0Z%ZW)%o-eh7=E$!{*qDBDd~Jc zl5(Go@Uy}3r=@ncudrz?D{;fJ%q<5+run2nP2Bx8s)|K`2x{%%-Gb}b>Z8n36lr3n zn}wZ{^muFDjdzT&jP+>X_42_2vaR*WP?uBzjUcwfUy#I`fcxkd@UN&&Zl819qa$3% zv)xbA?pMAX>3u}Rs`U~X3h{V4V%>AXVB`O4%tJX3w@i!`cV0zrP@0m(=!IjR_n)6A z0%PymMOM&18w8_7UK1C`S@Gi`sww5wVuB5GQRYLi6n)~03yY;64Tpo84;Acugfr_F zD};FI*X+$CKZY10Yr*D(=3eYLWunv!rXnxNMA;UIK0QP>K`g(YilDY*IOA%2c1PV*}WEs0)k4&bo<8WVOHNA7KY_iVv%&{WOwd=L2tyP!33UD;lA4EEYbXFHQt= zx-%Y4zxa&O-``_cx{GKC7=a&EcNXP6h+St`0F?IY=`U5t;I5wtu=O?fc5I z)sjVBoQ3Jh2+f*VWZ5<}R`%!>DW}%By9o5#n}y{d4Fsw!7Yn<@!GuK#R-E9i+(oCI z8St9#kW~j}eAhVpu7T}v7P!NM6XLh5Ss&vUTtrQ#CJ*v-vf!yQ#c78bxm;#@_&d6Q z$(@WBwM}4fm6ow~B9|T?1q~rH3IJb?sW{GvL0Qt_Ni~5CnN;}y@_%1Y_?mjZv^y78 zVbw&i&XYMtHR$?ERux_;%9X__hRI3jkj$)0)JGP}2NXu6SOq&5G!Ud^+5PMcLN!V0 zzoIlv$yL`6cwsEwWy9}xwntcgX+KGU0ZD`lkb-M?d8c)ct?og7lOx-i60^EJaU$5NlSNH8yC!U*0r+%+%Q z^}cz{{KBJUVwxrOjUC(0#-6$>#!d7Au)kM9pxKIuW?_ccO|+IG@Qq=)ne4%pzp23pTE9yX_&w}w9`}7@7?7y!RN0Pw3*Lg z-5d&E;->xG>*~K=AXRtxYxJ|)*&vRS4J!0~P=$GY(^8x-*Vyjsy`S;1ve%N=8(cbu zZ5T#&RtKhswvILisS^SZ_emzDL0S5L;N!pa_W%6(TeL&I(H`bjn2sx`Fm#Oe2HRry zS)^(XQ6Z|&XLE$bB?`pLk*V>>wlQ5txHdT{>F-W-iX1f8f=r%Ut@D1GT?0-VZKL~1o@@v@PbMfxCakR0j+Pc##9Oe1)hQq?j z=E&vtUZ?{zyvKS7g zm?VSraETTYL9C(-GbtcVbjR0y<0h@>yG6mFOUys5caCp9JHBz`VPs|H)wYuShCaP> zfexA|nDF))Ey>qiI`+h|+10{DnHie8mAs|-MxVT;P^T`g2%|MQB72g&a*N%%ZrP2a zy)Un-o$vHi5_8uqnoQdC+L^28jiU#?-#3!~^Sp-YvXnkQcS>S((T6bAm;T+INKhQB*#n&KW6W{7L&1#Y8s!_MQ6-&n?K?F zp2=s%b0Ce&7~$=Hl7{(u{LH#-3EJFt3X=C%1^qWfzvNfAk23YE|2}^(5IX#$AkT!u z{C8EMO0BYTyQVo94U~e4lhl%XEnHPU zVbT9p3_&U{0d!Rk780AnXtq?98lq?sj3(nwny*-sgE7BU*`7ZUV0DZG0%^J{KM~^7 zHYlo}wI%xGnG=*nev=hu;GRmS=fUdp)NlQJww@Djdw2SAvQs+j32NpOx=B@v{GWqs z87L-qrIDz82}#1n!lBzHmZ~h6(Q%A^4@kxjO!&1cU9zuGJp0(zRA{%uvL$j)%G_F* zW=54RjfT(c$~7xP8?8-tB!t290` z`2&SulkarxRmc8KOKDgwH|f0l?Q1*Jf55wT8&OJYAPkM8eH3y7a9@el6KT}i&XFJr zG}}qG#}aB9TLl|D>q@zaKE8}z@*ru8V#YjoB>T(&Y*6X^?b~L2Z{<8-5jLOcU?uyX zi-;H#7K*Iy^dUb4q^iD`Ra7K^EX@SewR$s5=}zym-ZSb2lzViu*zA{vj4I=4X)SBf zfs>Wi37GGOu&g;>?6eZ5rkM-UZBlmsY;Xsu{hnz}R1G$+;J3D1ja7FlW(1a2G>q9> z4Ot7XWLWx=$+^l~(E8nm&E%K2x{o{jBu28r+O zew(&8DK4)js7>fiKs?!rWJP~b*p8GH%$brj~q3;EBX(znup>ZE9QAP2=(Qnz4=3bPp(0ak3U*p{g7@>q*;IlX%lc8{*xV&!6_8W(j z93BVY%6s!ejodaH^rPUfPI7$h>t_;h$Rl(0I-DeFaxzh)!41dzqAiN^!v*uvP@QiX zhh+Vhm57xS=a-b zCY2kGFFsG7T;QkQvxzUHJ|-dG91wwVsIsp&M4$T)kzR&fSL)c;w5`t7J9%lh=Lh@V z8^fU+9Ym&#XS{5OvRmbPoz~BHs_*e`!boe>Nj|PAcb=}hUn=*%CcDdVRR$0FiYOZG zW~tmeE4Fu~eK&jjC~ot~|J!MWi?Xc?N{IN{IDrKZbMf1le7key?UGKx`HJe=k)^%K zA|LM;bhAMAwi)H^8TBVyhtl82kPilpo<7`d5cCT~>t`;OVl5kjlID~s2+cA~)k1rf zI;2%a&`)N$i_buyfn}D5k$Qyvt}NY!#eg2msUkrxO@-E~l45l$$40|V8eW1#j2JnC z8g>~f{HR6+cNfT)P6zvf6mk;~Cot~PzRTpc zkYMe+!Asm0Mc+UGDAK6s`h|^;c+JSJJt)9fF;i^=T}!!pPCIW)`#(1Nm-wO2)k|`p zS1*+8tw|zB_mVN6Vsm^h)SHIOOKcW{7gA1qb7k53MfdaAcZ8{jidK%0Z5i6BMqpoC zoQgOn|Ca6R2!^R$$Kd32)PNb{AFc0?5 zyTzNiZZ~jQ#+oxE7u1sDvLbmoZ!Y?Qz5VjxVMwCwd9fMz3pV6Pa4HyP1HFqTp3jBp zMOw^%s?nHRJrku?Q_|RuTk@FCd&K@Hr|w_ zU#7)!R>~fC<;WsdYjq4RKs~8ZaLQB%5*os|0F5@J5o)y1{AFQ5XZ#7GXwm*f5-4_a z*&sz*(R%HqZxd%ll$qq(J`FX>jbC&D27Of|&9%OirsJFXW({_s;EeR2*Uv=B^4`ng za>$LpoM|`Ukbf8hN>4wnjcUv=$4(Tr=XwTR6LF6L8=Lngh~RKgxH8Rk1(Wl9Iu8snGS^tUXs0Ay*6i6q@9tGl2#W*^yNKZpYLk%CAip*i1&TciArrQo{P? zr{^3N@g(HB{Q)+X1)wUQJ9ou_TUnV#*yTD|JXc|nzbL#_Ik|sd&N}531;XBv0hJ3) zr;ZbmJY3#hnUffWA`DcV`TA2U6+Tls$f53pSTAuqQrC-ayzjX}Q33h9p+| z;v&;-*h(x(DW8Jx^0)MJ{^kV_VZ2~yL`}>^YPG&B!3t0IM9ZIDURF!NaSw=q#Jm}PU1 zyxr)2-F@g>3~v6Qs{j4~TwTa4Selc8X0{=)Ja#<3z>0}4Iz+EyPJdxo%X-j*8o)l{Jlg{)>Kp)_(?(*=m zucn((!yaKR8MXVTMs4?F{r$t-EQk!yNf0s#=Mq8sR3xPc0-}vFzmJjjw)SsWbVuy; z-W`Dbw6mn#OsnDRCv#mDu<&ii^zw*@YlLg{cG;9fmBZn>up1lk6)ZEK7WtV*z2@0l z3&XU6hFiVop4!i$w2@s#>Taw^(^KLZ)3Vr?<&sM&Tn>ltw~r~{oxhMSY2TcNmb0qu zjzgyYL%>=NidwO;@+tkbl85NohXEc}gvj+371{$%)C;|eYy^KD3Yf%ANJ zT}1;IbHd7ZW2<)zt8%oo1LE+pO{N{LqBX~tNOMcm&TcjBTUY8?_4{vskT%V6> zzv%tYPxaAHCNzGB24QBD^mXzh1PwlEx^JX?rHW&L=YMd+zc?L zLzdWg76ofC-M9ikE(CO5Fs{k!#}}ZcO#+F7M1mkJ#UI&rqET)?R`R^_BbaW{Ys%kU zdkUQB_8?YikZzlQZQQMo#oK&&uo8k_|Lo2bpu+wzAntE|Q3MZP(Oa>k2-k~G;ZlB+ zJ?Y1GgQ^)w+PAHRh)80#CE+{D=zaNf(r?PQ-Fh?yK0eeOWgE%hkiu*-@f}VIn_Dd1 zDXyr0F3EJkR+cm5IYPM?Y{QffBUx#+IR{OigHhXMh4JlnkEvfulMV`pK9FsRNOEE- zJrDTo&$DlX$yDQy|84XX5~Dm%g#;-ye#P>r1Et~4jZKzUO2k)l2H>I4K^i0I!poW2 z2zHnCvUa}BVXbv^#IY9p=r_20SnSHyo>5|nuS9#6(@`67*`P^K+3}4goYKLd|D<7m zd2#-=j?FRpjL&3gdbp6vZAtzU`7aty8nCYy>2=#dLyhvK?1VINQ&!0Y&5Qf!N5|#w zzG2)|B$-U`C<-q*9c~Smh}B=rG#cyto-K5eu-+LnAU4Y(J%UeLpaR$E7-AeX=B>dj z(=xym%-_ld?JT2oAKPk(F~LZvYw>~=_qi z4y@awEj!;ry`~jN+G&t~&SSMtG^)_x4L($)U)Jks^A}y-^5Qcw!C{|4WXc($9(QI< zw=!lug(FvQ=kYYZ$qd;M>0($!xT}NrQuE&YwBadzv!G$*angI}DQ1`yS~&fq+>Ta~ zlj^44Vlut!@(hgHiVKyG0@eki@|avyMXXR4=v71iRXRSx$eZj}+6+07v0O|Aor9hp z(nUM81D1&b!o(i^BAGxm5R0jk!-(ZbmlY`SFkbS0D*J(5+g3zZ5@Ou$^o624;N+*c zkC$3Atno13ygjQhSI`6k_{med14xf$0hFsj9_3tC$fJc{YPqEz->>-e@QB1HJD-)@ z?WD&EEbq1*%-Bd%5+ND_;5dzMiJ#fLd>3fl2eFEY#VhmT-|wVbjqJ>zXydCWEXWe> zwidG?)f?P2Ml{janBgIk1k_=XEvIPm_k7{sN|Dv~j6L=|cfnmD-8no}Bv@N1Wa;Yn z@r-T`;1A~xtl6-8*6aFgqs_w^u=wGQaDJq#7jA!o-_k$=(FR!VMo3Cb0Y4zVZXO%H zSWAQd^7ukAJ*tqRHV3Ko24_Gx`x?9GHN4*eu)Jb03Ewo6XN?Ece`!Qi$)udb={}(Nh1tEpL zu`EU*!5WmKY{2Xr$-I7!c;=~6*-xNj&Ju`DppuX%`3#*jk&*QUG^+i4N3Azoodu6N zX~iXh5Y0AL9{;jJgI27_Qeo2oCQd+Xd+?oq65&#}#X6guHB?~}(6kzJWk&R4KDoU| zyXL{QPK)keBjGUnVd$AeU2VMiLi-py=sxt0$;}v@ca>32ct_fk0ocA_Y=^8-9~LmD z6)vzAq;4TTW6_N%DjBEJ4HY&M%zegs8p}ZAIK@;O1+ZoPXKsT()dv!k;3aghOk{Ki zdS+C_4zVecRTXFai2!z%#B%T zt9}L(S7ON8DxMQraAQ%+k4v=2#4ZJft=b`i`Z_KQxo1_QcrVoPNi`mT-h_oOPu(2vaPYx*w6%E_XORH(`MVLipw2Zdijp^qr zCvenHPpa`!ZZR}##Wo;9hzZqFaeX&t4e!6j(JV^Y%O%O+5H{3w2~?xdSi-pKHhe%K zW%(X~0`_iu;Yi$Jg*75T@d4Oiw~`=(vGP^q?wI#&S$+=>6(3^L7&|F>^F@<960hXi zS6bmSJ^1#nSXmQ3YCCIw z_-v&>E3t-jvJCjTl->*0{1`e{pkm8^H`S?0B9s}^eQ5iqlQOh}YUnOi^r;tea&z06 zV5JrhAok=4G_;j9I2RRoJgsfG+F2P31`&vk?RY&2AlFz~T`ye~%{~<;VwB8QdHBaX zPU<;R%07pt^DLXL5Bje~t79&WN`*HsfqvXCk$%7^AG=<5VAJ!P=;oLXo-hALV?o50 z8!HGb{jI=(f<;nPfuy$D&qya ze7OxM1!2_agjC7&jKg1ErzR4>?X()LpL}?@{Akr@XuFi%r~p)$22viCrZWuG!PZRhK0=?8Im`T;yJvCmX2` zZfFnRDcYZyuE}x-Zaa+BQZ`6i4*oP?_B|@ldP09%lgIj7q^?~hfmoy4rUpd>F*#E( zH~pd?AYlCO?agNO%#5xI{gpB=str3IEUo(|cL6Ej{?D8gCQpru>C&|%`h zV3@U$tyV|m2Z8Kf z^&+SuC}s7E$G{zj#F#9gus9OsQNh z;IkcCLj1r#AXdPtY+q^1s+DbsZGOoC5I0Z_-w@uhGePv6adEkwPHixL>ic&z^obtw zD+R%~$(mq9z0M{cEU_O}Ox*FnyUeYh$W)k#zm;Bb?PwBb_#+yt^)SjevQ$HIh5c%$ zx7{EWjh`hWv|73tR zDIIVfWu46J50B+X0 za9q(iVW>H8EQ+nM!xjFerivNM82UIcj2m?HNd-SYKo<|`SJx0rmwKRnIqg&106UP# zT_VP6A$xzG@qb2&EFfZkpg1I6cFxQT5c(3ZL)=@lP53R<#f?FEDOh!Wr8rBc9_I(I z6nN!P{J9o0GHPgX7pt$Jz$_F|_CQizi=T|=&Ws)%Gf|FO7jx`9r`^L*X^l7TU}WSg ziX$_`YMm18_Z!`;msx(_)3QE)Oj7d|pDl`L5_a_(qXd8F)8XiFp#B!wk&=&5+Jx2Jr0jkAtzIHH%J&cJn2)fVP%@6R{~x*o>W{-fX-A4GW7Z)7+$D zrElO`3(|0x;3PmZN9o|Z;ldO-`aU`^WlIY_#am4i;Y|qfam|4YNd(Muh)C=krCXl~ zK^Tf+LtI)k^BjMUU9<7xC)LofU$6dp?l`O85Wc}-n}o&C{|?pGBt6N<|C`t3 z)ly?Qw%8m}U%7ivhigk0Pc~5fK&KsE(x#hd1XCMoym7R-`d-wL@TAooFw#n{YYq;Y zEmA(fQPBulmPb90a?z?eIz-r)bv{eTE71|~?{cwS3w0%(BOpUcEfObPsF2+qmTMNt zjcpt+?iG-{3WFpysdir~LL5{1axyc`<#-(3sEyGZ7THLerBCY}8bK}vP-AfNZ_lc$ z)RcbQw-JL$mXDrY{~X$XXYsFKX17IpzzE zef`R#1zs$HOozi}kN^Mc@&BcNeYincSbK6=%Y3dU_EKW6d2L$H8iFvO?{nHSBZ@~b>MV(_f>MOWj1 zX-Bp`LK)_C#gcu)Pk4;LmvE>d{z>`SCLkg;!@GB*WqCENts<+Ut}8$U8zUhs(nJxy zA8lW`7im9gF3>Cx9Ol*x#lSjW8wXQ$A>Yi#JAj?VYK|zugztC68GJVL8f5=vsbTl#NeBX z1#LJB{57-7pL+C=my4kwKGR=Qq34$0EvwwXuaJ-I{Cc2_L)Y;P4$#YDQ)Yf#eH1woNR+gkLq$PH za0EMU`wMYi0=lzUJ>K87Anqo>+u9chz*00c$^F`|8KLvyRqQdqAKKR-!Xl-D$Ssj{xW@l>IjY<5NCV&j!_|P+2I2-#%*%H$VmCuIMMbW z0{Fc0QWPWnM=5e$fXf&yzI(=9#r-M-u)Ls}>^oDut>`w>-n3fVL2^ym#{*3#J(6u= zpGkVz#2C~VT@31+%+Z_P-)Ix%pfrche{a`+A8w8UMyPw)vj(phI1jUCnnL@>hpiy5 zovi5=X?3e5YjESO>7_rpu&omen+Z@MV$tFQYU*rL_i)d3IDheW4azv##9Nj3chO~w zNVmTPDn9NNHUJ)_u^N63h+qKmk1S7aYI*r;&ojZ+6-xx{=Z(!zxq0C@E~lA+R7XI4{<`%MqVFnq_}-;jq$ANjMF2b z-pTJ9tYHJaE`VW%$Yw97ER}4-lyx1DnnIWq7@+o~s%;@suZWr_69o=&4)$Nk2}5uT zn6C`i3h)lfi^fQD-OG73K79GMaIz7|sRX`(>rK4;5R)4x|A2fb*(D zBheK{FEixreisf5GZ3on#qLLYwZJ7b$#g9h7DaaZU)STmOCllGeJCz$=B3?+GOOk8mvGH{y_fce=6<{8w z>OVEQOCQBstxs$VL>7hyeQXCcW*Q>k{{LKt-Y9#izUas0nI9#bP!l2_@5Y+y5r~(F zx#CaNC^93f*Wc)WvL=sljc&?7C6&vHvp&S!pU2fmpF=HEM?Bz(hdIA|bVElMunL5O zxHZ&3;Ciy!X58gYW4--mtc`NtUEDI z14ZK-{$vv#J6;G*2GnSTj|GmkMpS}*;vEz_k8Qj}=Z(oyFA|swGcL|Mj*5?xX&i+lVntB_&q~ zWXKhs%MC}MziH3$2^h28qY@|%Nl?vUU#n4XuzjwfYE+erlvA5!7HU1_+ ze#QnOR<%y?P38DSSr%Xsg*cS5*uiz3?iXql@E&I!dCW*_OVh_ z;GD^6SBpECSFVfR(Jly4EVBQ9?R{rFT-*A8;>bxtf)J7@A%!H`MDIPK_g+R9VU#c! zT|^KBiQY>@H>1udV@TBKqPHP>8Adl4!+)G}&$;LR?yLX%>(&0)d)8j-yPme!`aaL9 z>o2bJ%*~(rH&lN(sBigU(%5`rfqVr{dPhjFRVEkdtrb08T}{0x)>s-gMfbfxaw<8H{B!%`2-FY0x71%=&80N* z88`hO#5B*Teu%e}1X}kQ#=N)5xlASdT1fs)j~P3MpthdbEiLs`Mv*a$+h)7AgZ3u{ zwO#TCSpDewZ<$u>(Fd`WJ$N8k8SF6i@wX(`)_TKM`o~j^WtC>2Z?EP4&zOi(;6mIU z+58E;R6%`<&vhcv+B4iEtL(j>k!mGKChn%}q~>+OYjEQFejaVY;);LD?bom=ft4S> zRAKTun`{%cVTNy-Hht4N1MJcN-FD!Gk)A`{iVb1UO}R$?AdY&)2H#!yrMO&Q5fLT_ zO(>Qwen7N;^zVQvMw{>lW8A$S{BB&W##m%LoFrQCzirULA4NrHsXuWNlV~i+a}X+W zFq!fc&gCPHqEgYc;ws*|;As2^mAstXV$LBi7$WeMK!<>U6lzc zo>9sFrl1ND72@#sj&ft^#%z1}XCyZZ)yi#|R*w86yLR<7?ezlM6+H!T1vwE^APeIw zT%OWZn+INuZoT%kB#ZR8o>9{Uv(AS&N`Fna&4Y}AlfQPL{^hiSLc@ugegiE;#gk@3 z{(y#;?ZwT!*UDgae_hjrHqt>~Cvd)hA|I)B`6N|m-^WAx4>i_R+wU4o!BUP^QF!ZF zKwZ*m`gFgAg2&&RFn{ZTpwSK6=W@3ib$GadR&M(Ya=iS` zZKRdlhM~+0X?wtF*6cS%Fn$K@i<_Cj&!GI6B>~79F)=YbdIEi{=ACaMCgeT2q5Hz~ zgsFsslwG4bDBLscz9*P|mz2{IT*itqEufgNhTR%tz4Z7gDa)nH!~=mzsNP;8W%(q3 zDJUZ(w^HgcH7g8$LspC~S`-;zcUCIkYflptSIX);_i6Fflk(|>H2-;CirZlWZcJZ( zBBJ&0@f74Vu>TfShy7SfVQ3`#ulD|I`Wcn9MZu+t_^0n)5AzyM$3izP%?-D#r`Xi8 ztfTvzZ{-12dI1`r5(C=zW3Gu$GSvIv(}r%`-`uD*6YTeZInTK!ZPzLU!5&Y??UzN| zeejPCUkXCVUJ62!LA0&7s%^rCR0fuKLogO%J`iBDKaam>l}Y7FLT(?JwX}K4>ZOcf zSe~JW?q$uI-U z7<@Rb96MQo^0;o!Qp)=Fm=lHzC!Gnz()cdZMF6#>I6>`#wQL$c^a@f2e3KgJ#%nLZxcod-&t! zNkp$=wF`46QOT<3+g;4?0rMo@aF9qHL$pBB9%Q z(ZP%VIoo=wmx6${LIt0)KgQ8Vws_8X8@;)Nke0pr;GVJL5|_#S+7-dpis5!2ZnjB; zv}sAVm)O|%`VT6mSA(DgUal5%ZNo6ajyC2lY1e}R3vi#*;h70GO{8ji^ta#ZVN9Y4 zG{ClK)ba=qHYI$^W`5SDE@4_GIMgPfCL{sVF0t9}$N!z+*^Nf?s(ALB4jS+^xjvC_ zE17yTaT$xpQ1&|7u`-ba5)ZxUKCeP|PGRHJ-?wTfkiHV?@mP`Ed+WPT5JHEU2d?Rx z*OwtZXLE~rq4sR+#~L#bgND=hQ+80b=|sYqBA>&&!@EmcShhRa)BP-(k;cqIs`zj3_BDhh($^u3()n8s-Y&Sl7bH;Ku-3DMC& z5>qZQQ{ZGMO4^tq)`3202YWxl%v4liK(YmaF=j^Zo@`wfKv=^h!osFKwO-b8Pr<{G z$H%TBR+7 zdfDF(@afEQPczv}{5@Jp26KThB4TOj^UU29XWK#{q-SpM9+Y3@4^I6#V-6awv>r5ob3Y$aqiPYnC)4eHTHTQ~&qUB;$o3~C*u7HhJlx;0Oo3cBOCt`#V z0x{pJjb`hmB>Ysgu2LGgdO)^)OONlo>8ck=`Pj$ z&X;!wZVVmy?N#fysOeeAsdMM8{%S}o)*x#s2D$fb0a#Ff61!l?NLjl*aTSe;lk`MF z8yib%$Urwps0!6dCY+N>_@W}6ir-xt<~_Yq!Pu@>f`OdYd`m;*m&O`w%H!$Yml38i zY^c3nr(iVZ;1b(9uOi0oO!eqpxpTG{wt)T&0bcb(fa1rvSXK;pH5cfcix zj@5JL1bsI+7bjeUgtyuZ&qUrr`ruw7U8$O3Z2<5i4Cy@QZW6-HEt#(=d?ldImKXk{E=W zn$xD}Sv?`Z*@+?d$ewM0czJ~)K+r4uU7Cuxrdh2td$*P{AG(@_s8_J$0pTP z-093zZ@*b-WN*bv^a#nvq|Rr*t1pFpbX@@9j9~e&lve%Bx9P!VQrLTG0iF~jVz$$} z;D_dv*EZt6FNaeEWn5K-J5i8`Tyx6Pn3|t8rle~Qdt-So`uf~tb7hA2t&ZZPw~VRw zoX*eTgQ`LW`ZcIyzwt%0(|tmZ4=e%Dn6f;RdRo-K1v)hAJHQT#7g)wQmTpa%2P}`S z!b?r+_hReRM_#8f?);F$dWkiOzbj$znuVdAjBBq*DA6-nFw`5>26;wns|w)Uy7Kh5 zzOK0L!G0lQqHC%FBlb!{<+w`c&XZ_)1*xCuyM8uf<%tz>xpmsAKm!#KgJ-TZbbkg$ zL`$1!p#f_fsoZ1{(Jzb}hJers4kU{5- z5_HgL8(h;~F}{~DW_MUN29d|b04dx%qU#5vGLB%t9Y2iVu@2b1`U_=lQM5ZD)ma%+ zq4eS07U_>JR^g&?=_@U<`gK~7iykQxUt$Xslc)kVpSL<4_MP_C?9Ld(O}pYumzdT+ z;MT;V+!=5t&&;>!Cb%12CcQ>n%~h+FKSVd$y=NC+yo{n|D?4&gJ8MN*uEdAg>0wI| z`@AqY{H|n1eg9UVzy#{BtpkfOnytMVg`r2CoG3GRe+k5u0Q^{^8U|LX+=lYWGK6}J zV%gJA+}E%i-*xZlJH|xz+xaSmrB)-3!M~#LVu|0&`4o z3Zu3@2=4-O+qu0${N6^#)}()?&D4pPWM@Rx8=0)$WQSvm{IZ5fiSQ@=&}|>?tW4qW z20lgeZUPuqosVrem5#N55Wi9gFnCEUujG)qXRohbV{iv+d6p36KAnK;v z6wz~M%mlZM>oVw^4Trhl#X0jCCXvPoFsM^8r=C3gOjo9~+R0`s)34u8MIvsVI`i_9 zPRedlAG4}`FD}zSau8RQd(1@tdh1=StQ2Dw=(uyg?6j&hZx_a%eehAJBi=l-+V|}H zuu{M%l`Gf&8O6#|)|M|(=d3Q7AR?|V!=GH9AHF?pe$X@NfdC}}r0yTRA`#hH-3tg`(xo2bW z{B|l|eU7Ef7|d@DWRGOOF^?(9s|LXAN62+`qFU+5OfA#$olbU`lIm9^y#bD0#>L<8 zabf`QEFWO;iKE0rP?p&$yCSLB%G%ZGM_Wy|0$wGue;_OeVF)5;4Vw_Zphp|=&quN( z73);rnXjDZhMz7D>+jTgIJPv%(;8E2>;&|#c`s%pA@6*lJLM)f$zOV=kKf<G6IH$l4xIGcyGN8v+CR zN3cKT0t&LARyC#Nw2}na6I}V2zK2f*HLh zQf_HOkf(w-kLxUO%&#j1{b|QN{DMK4%?yH^#>_-{xgjowLgk>Joe&N{Yu89{Sn9o3KugEj8(Jqzzq7^nPp?TAghrX> z&jkk}hzt)={`Wd^UG|4g8=qJG$EK@8y*N-WEDTcLge(B`OdU2nO?dV0T8yaunl#8d zWUdqugdiP?n9nU5$}*yLGH!Dei2L{^){aJcY)6!csI7cW{A6L!c=~l}(zqt{<_{@| z+v#VDjy9Dj+@w2kGb8i2JK@8U`gfz-;F1Uw^a{r_Pu6A}(6{PU&9u^sCu-;l*7fKn zTx#iF(G?B&JitU~S2J6)dmx+=Z547jQt9A*PS}tWY4Z?Hdf!O)Bt#X;Sg|%!l$}^M zKpl1yQ>eGyzR44<`(z(W|$=4 zc0@t4zf~^TH9c%$8!`97U9uukSaM|zkKR(VkUaZtg53a8u(Nfk-ld%m#DBZlm?!@b zXr%&ew}QSK_z5BOlV|o^4o@ID4&^*zt1INm__C^$rQ;|+P|S;x*0~9`2#SU^5*kO} zldh81HGhsaoO9zqk=1IpOfa#Eq~otJ6Tep-C3*)NJUH*mrK1DdYP2fsHbwKwvjPW; znxJyyalLI(LMIYkJnH+a=ZQkf2kZv-EaqBGA8$84hw}XS4vQ2;21c}&ue3mZu&WNs zeZb`B?y%W(+SY+y<|ws7nF&_#456#Y`p60(?e`KT(RO+#h0W4juIqIVZ}~3*lSIN- z%`zex=BfLjW(@1$m3boX#K*`s943AWF8TKg&E{I)!C`WcYydIWrqFSM@$77w#>RvJ z=3EL2`Iw2Z+8QG}q=>cn1U#I#J_B+-h-=8yRPyRb1Jp8b`crecSryT>%R)zXuJGeh z0659?*F)rH`1QLqBq}BvSt!4)M>k8fR1RqmQODSuE4Ka|^ZX?pA?!56B{KarnVSYu z=RbhfJ4#-Wd(2^*w9JDKG3%&2e!oCMj&q0J*nzjSTqZVSJ3_BoVkNqALfl#GSM5uZ z{8(bihi1wN3l6EgTpts{PEMMC%w;F?{28tae?bE1JX4>%V5WuI*~Q4V@7`f%E$CDQ zmWHm1bK4)*X8n2jFglILH&LwbG_TM~U1D9=HP3FNX*pYWcATSWZ=qBfcQn0I@6-BP zFdoY!R;QxCxGZSs`F3k5$L_4b?jC z;Xu}~w~nz6OJ{_xX6BpwlH~e4JWb_LoPUTnk2}Ia{yNup2mA5eUjk&FQra>ruR%b} zY;Fx;yKu^5TGTZkF1*n&nf+07RK*PJg9-M1NBbc(xpwTTHsF+b`oyJ4_hdMh)$(@a z-Y2YfVk%yr_6qy${c__IFjV?0rQ-AO9GbArO=OG1RAG(zAu|UwGMMb3V6(g_z-P{{ z$bY~}$5LyoNxNUNF}si?>4o+K9Utc!61V3bpE>1CF~S~d_(;9Pcx(1*Wz((Nu6k@% z3uRY*KN8lMc}KjZOuVHJoQA|?o7>RZq=IJxrEs6_jCK9?B*p6k5?;iDh#JXf1_|uf zUo~lx9Plqt1ngKn$Qkb?vl`p*I z=5ubL6S0fKq6H#rtmPWM^W0f0auB;bd0?)F*EU<0i*7cPGgt9<_l9~y=cZlJGp_?L zMTB5T`Elaeopd-pRD9bl1$`M_Dcj9KnF& zsv+d^crRqXnN3ndWDJV^UWx2z+MqCVDJ=8*ut0&yx#XXx_cZEc2EYeIk9-MO4Q(j} zEsM`a6lUIPV&oNP)PHlYkL&v<@tVwz^G$+vO_Zw^>!^m{grLec=)(u&wf*!gzEg{u zxa%br=+f=tM4Rx)VCTkfO?(k`=If{&Ya432${Y^Z&q6(AoOb@o;WUu)upc(7xfr-4 zzPB`e)!+9qY-BRuMg6pqp>lrxP_-0GjxSFBVN2PBtDp6ojKI zIk0XBg4On$v@*YiQhkR!r9k?_UF1)pTCK#6eXeLIQ7QfILosz@Su;$$H)*eX_qr&a z7LW~~)IJcL<{%nK0mpUX*C}M`e2bsK?C|6Ha+?|UfC3+kYIzJ}y=3iL7cF9?M|t<= zmAY@s^i6H-_anS(r0i(Vh&2pg?~_$nuqtbZ41X7>+< zq6@0}tLin}NbdQp#ixx@1?DC3pX;T*NoboLB`p&FU=3H22KttjlYDVVS>Um72#?ql zl7n=A068TY&a}0@c`m{-2BcABnDB7)bN+QX+S~je+ zDZiSnt9W3om=8AWYnqc=kH)n}M^JcCn%fR3dQm9JhdjSUf%NVSXuj5)rWT^^{eJ)4 zLJPtPA1{FmB$FK#%NP)&+A`;ZF~hlsXRBf5JG&a5=JEHVt|pD^K0xW}lr#q(3V-cv zimO3CaTzW!;>a*Q(3l9A{kR(OTgibcw1-~d96JDoNte&6QGxH5hzf0J`o-7HHl)N*TXGl)pc`{N z8G%aDz1oq~`|`u=gdcD6WozoQrzz**`~o`)=DL0CcYD>JB~B{jtKNEwYG#2*(?L%O z2V~G4iZtn`M(Rw7m!k6tUwN(TC4m?%WA`5KW83?UqCcy`q7cv|J)fsn6h$0{xJm%=gtU&O7Ky z@?oY+$LW}VGAR4t$5zphb(36MX#VTw>;7mTiH!XpTmglx#I?!S8lEp`GWKU)(eQvZwc`1@bc%j@_H1A2IbK=yV2cd#Z20VD~IWOhWd6lX;kCC4j#n3rLbv#z_Dw#5E zaMJp*0XMxrP$jo`&U!gD{aoKHEKXL^GdT_*=&EyBOhjEgOXss6wD5S|C2Dh_sk&%l zBdb0E>Kov!py*n^d;LIIHRnJR$z4{UmQt^)V$sd#rbA+a_83Y1;v5t59@$sq98%XD zpN4qjx4mz=dgAU>Y^ZH1e%nf5*+9Q%hhD0nq98BThvKnGfgEE7o#@Vbl}%0gGf)wK zckGoFw=%KxujkDqR!12wBH1e>-_t-%GakG1CdEZH8j={xg`c*aUQ0ciY-KpL6yvaW zEH3Ph&+sEtg}N%36u^PxYD49h-5RyHo=kKxZtA;q&91rS++wsoUDNoD{6ZA}X@ziOjPk%Im0-i?jT;G@N&zv7{ z&q^8}2i?r7TRjdVWaSSsA_k>rr`UQ}sQZ#$(v#&qo5i!FOU?2|xzl!W72>A}XJECzbKeyfx z=I%O?V=S`}OI6?4X3zYl!~l#287@tTC&-wv^37e<5yVq;FA_@?=+)S~<<~7cmx@QR z-?hsHRx`ZbtXtUal|)` z^y;C4&5joRamhE)-oG-&(*3M{bS8&Rv@U4uwPhW8ihPI-ei`LDN<}4dmC@o%ao;Q! z>Ww{LlCWjX>}+_#VX07rGth#V8Y(>rd+T;%JpO=p$Mj=oSQqqB;IiGSR}Xuz0G;{$ z?Sq0MX=05oLwftCUYp*3+Hd=>C7NcPxL)!Vu6G_*rF2X+hgALndrP|p1 zR5Q3}cTTG6Skcb6PGeBi3UE-ogiXq{b96Y(QIsu8OJB|RxkpjI0MpCyvb;Xu3N?YR z&lG<*j_lP+!wwYI$S^iiV~R&c4PE{H($*)_+m91)Hk}o|b@QXA{3e`ETV(tnQ-+St6JX?AReZ*xf5@YiZNbDfoo8t?WV9(4L4Jf zZ5=nmhY#J#ka>eChu(_~%0myjYTlDdOAdiyr)GVO%g6K1eTRtw%X0K1ZqP1IW6xk%9%ZmjL>B>9Z;Uith@yCu7mk1^0v7F zC{I9Ljt8?Das5RyLO$C$$6(mFsTkqL1P@?VR%&WkX}{v)Kd#|-w9KiX<+gu&>e(`r z@k~(g>OedNeG&?zXD65%DJ02VT5_qP|6s+?X%`$|T+nUqGaYhlAa~aTP5w}d4mVkr zdS>B(Zd{78Yw3^Rio3!8#4DM@1Za^Z(4Y4RCNah2J`|#kUa+1TZ6}0`-gO$|m%*do zL}%d)hm7qV_|$Y_HS-$V zy_cD1y7IHyhd!37IqW$-=;@HVT!?DIDNg#s|^gPG>Up}?a3Pb^6#^qSGg!kb{{x)mIc+vU`&Pw``dH30JVv? zK+eO8rWksDpoxI7+YdQ<`x{#%y+dX{+_RJ`OmU{Pf^I*aMshf@CxluJrNw|ic^pS` zKk;?r5t*aPpo))j!la%o6?eI&l)GK(Xc$Q0r zO8%86*EXpN#N{!k3bO<$q5asnD&sM^m5xT79>VOka}a3S#gXZ~p2K+PLa z`o;%P-{HK-R~fFD-T3=y#w~#$>n;z%6O#~m9-!=2g?*t_+<<rsN>k%` z{mX^!1UlMKet7&@#Jv);rv!R@=?n=qV;7(}T`@I%lHj?2VD>d}KIk}GMy^4@b7VBj zX3yt|KY8IVC@9|G)KuSja`BrBfWxK)*DlpQ9j9YcLh=%9S+aDJBzQ`9c4ZRcnuIAfn z-@|16?niY{&<51Kuwq2yjryiB|E_nVZAdrY!x=@+4+@+4-dV)iBH|7}piavgwWS<6 zs&zwh+3|8QRl<(7zl$i|&=2Q-Uz=upxH*YVpX%Lic^PYI!T~I<9r;9dPHLC z`D^9Ftf)M-1m32acV)|UGn>U#eoNnf7bL{&_R}yoE|po<_kxPWhvHi@tp;JIER>TD z;picD0Dprtx@u$|;Odh=`=KdQU?`?Ur$o0RUv^;v$+#NeaSk^dp0*o$+Icy;J*s3I z%9GXwNezU=fX+ViO}Y&oBB6)wG>xatgq6B5f50_b$6>U|{arny+(f3RGy-x-g67U8 zi%XZ7`jlSC=!OSuEsWz8qw_1scek0X{D=w}#=OXl6ohgq0R7RyX z2FH1vyEkf=Zjm(_bPK1U=kv!h4+6~jC3LCs=f*`rmkoq`MQo;L8GY6#Vijy9uI@gE zZgpx2UeR?d-q#E%V2A16)4o2G?BZ`Az!&s;=4;W%;r7>4$=fAYb#wfiuJ;}JnufdI zW+=GJo>lx%V{Z%tJkP%g`uUOW_!MxPp{AJ-fRXTvtYq(F_TIV~n6>y(Lp<}yqT&42 zI;s)g8>;!y1Mc!g+&YH8!#?$aY)4m3MCi~e!$Y3r^ts2I{I}!OLnesp(0+hM>(UB~ z_i7LFyM&g)o?p?w@ENoc%*ESrxI?u5?6;n20_NI-7R{Oq>r_fh5dma3-_nr(nyBT!-8a294`QbeGOAqCC2^*hPUd6(xV=pucWAd@7LoO0_oDnY~`9Y>LS?Vg>uwx-`-^U z{@UaSbk8Mb))tDwz54P=F>zd9#lS6KNHv>;qnN6GT}+{#!OXY)PV2~sDYS7t^<+T} zad73tG~uah{L)vo{$39}(-P`B$PbknZqY?KWn`%mlHoTKeXuceinb;U1TmCu3y%)o zSGB&;U%q0n$E{*v5SQSK9&zW&`zmktMPtI*TnbQ*4L9atMd)SgKR86{d*3z|Rtc|R zZAR?#F~gK3g#^u<-{OZ_YJB>q=75g^nNtTh0PQ-_rLp>Ud%U1CM?H$y6ibG0F==kq z;h06!W$Fc9bfj`hW8NK;JT!Ap_d$rDsdQjvUTIQSj3DEyGZ91uSsP`;c(E?Maukk# zuQgy=!eG+Zt@L~dr;6%{*r^*FuG{BATApY5P#73uz^2!sX4b@Qswn*NuM7r%oxQe_ z``*PLC67OwI6WP)PC2g9GaEo^dOx(cFL-b-uyIx)tqP7SP~UC5w~sw+;VGqlYDkna z@BMs23_M~s?3C)U_beq_7JsV7X-vyYn}ce5XmzMNZI!82qY=;L>qDN`dYqD18c*2I z)5G9z@(vC>UH_$S>#L$=EvW%xJ5O2?x{eX!q+BHtP3?WNR%+7@D2L;3F@$do5x z*m-sfI-PReJEHLMA`jV)_eJk^GmzR-BM~t7*PI4IdlcfYQ)u}F>eMoHXh}s6fgsid zn$K1pH#p3gN{pR7rc`ndvWu$es?52#9DaTpdyS@{rDnG;j{i)BwP@tf7g`Zc==)7H zsePb_L*B(JX| z=SlG!Mvnmihet%^ywjSU}$@;R3B{$K9yq%()(PfVX&y(jOlp*dx-zDwcC{8>LQspbf^>E{22A+BF(BY zDNY3KeKfubS`LN%;@U`<_1aebU7KQd&-3uxa)2la;W7E#XS5l?PTt#-GQv z>ZtFv&=y#Je{oU0!^GV{U^gVDTFzqU@td+Jq<+jt*?i~7;tB(Z(+!d(Doxzjc#JHp zmOm;-RqkPV;-l^9j;-SdDhV6+XWR^0QdlkpJtp!WV}DrfBK@N@M)VBKC!BL4l>~qB zKDZP__wY?8c}%UfQ*o7?3-WQn>d$E`{T7#pIwzRiLhMX@NEZ!WSgV8`0%o+VeC(dE5eG6z`o>M)zFChx zFvFWVi?2YLX6!}f@Cmn_++89+GsyS>%jHLQveW`o6a3G2pEneVR1O_^MnMfgx$WlN zaIGH#I6l>9;)Nk@o7zB49+Cr1dM}xse^w2QZVdK7kZs@{%{n$L}z4 z>%IX$lGHEQM4`+*oq&+zA4{)aMmO`{*D-S}l6I6dkM3dHj!@%*d{*nuf400Bug(^x zl`5*3AWn|~D41RhseZ^yU+MHx z*XVUMfj_FKqgo-SGM^b*NH%(@oeSA5p8zRb}62LU>p zmy~KeVb1uV!odyEX+E0&WG7PsDeB?c@lSp13y#j}c8 zmq^6?^95+Efxu1lnQ`gu!ju~_vj0hRuDmv!+02e{6A@<9j09VbjV&azHirElmh6Ze zkyt8$`_iVP4I*M|(=a+XJuj3%^Phxl=u0AJ+F{k?ymb@QLsMg1b}tZpavXa5FBn1(x?qC zIdI7v@sXrq=iu>BMUH=VhfamqY}+mZP`_&I>aNd0dpYt)<$qG7Ih~1;5IM!v?~3x+ zoLU(-mPmUHyBA1AH8@^6n~O%6NWPn1)0jag0zQK-2!j`BWxpf-i1xh+hgMBl3D$8S zM-B+hew;z|RyvBNb`N?7z3I}ZxnLl(``4Vl<{}w^mKsNPS$WR*fF8hJVFq#~zuwIj95eiycesCH^WHbsr37kLi*jy7_uZhJ> z6scgpc_Y-*1=~7!b}-MJ^DHgO~KiTmO5v2$bE(SgC)DYJ=q2>?~fFbug; zW8<&D{ARt>1--)8c!&Wn3)I#f3Y~TL^M2mPOtyGz4m{fvD^*Q7j>=oJ#vwvBhw_jb zI_LLM>G`xhwzEAco9;~h_3nFPW$saUzc0Rr3%KcTGu`#yIZ6{T&;RDRYswKJ5^%3# zPZ;*xob>*T9(Z1+UGUno!JX1YP?X}mbP3HTL&8cjewq~^KEK?Xm6edmWMDLw(Kz&@ z)Tu5&HF>uVk6vY%j?4-ec#SK+{2Mx}2Hn90A`#9~8pj+FX=puUmG4cO2{v&~=nJ+1 z)*nv`X_gbR-JB(?SMfHT3!M5oujpO14N&=XzPTfY?k$r_o1oLzH!6-1vqch0sV$|) zo*iUn+QJ9nRfY9C>b%F_0G-J8z0qYyF{c8V1we`A)LsxB%xdV&;63*%=SH2;wuz7`hUNMsDf|HW z7W4FO);s7x?yP?=fw<7nD1US(ILuc|G}{g!6tHXOypLR%ZtItt_e8C$;)sE1-Oa?! z*;VST`{$MT5yDvQp-T1M7tfNj-+<@$RgebIk)sn=4J3J{|A0TQe{uK*6WSXYFv|?c zK|goN6!X9ncz2*@_fK+k`K(!OGyRkgahM4o<1RaRDlk)v@@qvtwg4zd@2mwy{2;E0SQrGmaeBC4?o3at=#v#dXs*>?N#N} zJ}1p2k+9e-aXP~w=?b~;)!>1~rYRNHn{WAy+;bkCuES3H$GCy^jRmU!rQ#B<@5B~I zBQsv$ppRm}OjCejzRc52ccIfRzEUJk9D)u|uxn$AZVH&@yC5lesK}#7YLKTc>Rekf zOuKk0PS7!tW#p#ezmE`e3L~g7x}?K3q#jqZma~3yVL0Zi>C*1DX)Y8nqa~>f9%ERq zv+=C_*UZ0_PGiwT1-nd<&P>*Q@Yx$i=YczlPcWkibQzPKGGA|#L|!YL+6y%I8a96A zTWfw3EwGYo6FwIQ^@YGHdL!1F?G%uJri{6O`Uwh=Hh8mR<7qKA``~vZB;Rii7O?cp zscZa-HM9GA7nJ)~#hZT8a$2}_$GqrF5(vn(PCa%(PFHt@S`PyW5R=V0lXZKUNarDD$0%56c?3=&1!0AxBo4s!LEE5~ENY5J7im91FG=}ljJP_G zrGh^^VBJzZa_U`=^#ug<2b+^I{-ZPebsQL+B*VvnYrVUfBx&;emG2guzJ}LM zUb8voO|*ZS;t)ki=>pQP>1YivVr@>`3W8k)HiIiIAN|WRj$S9ysWFgG`xp5fg}fv# zseFJ52d$(nKPATT67yW3+lPGNMJMF52kZq_#Vo^yTNa{c(9SG64?p4jWLsecp0PZA zF#Oxqtwq;U`m51@8T$c2qJNrN_FpQ#5WQ;pAPVvh+3p-8okWh0q+nFrXOKCsWK>dO zHu2ax{K6C*uzqCY7ZL!dhlR?1KW+SU_(qj@POc}Ni#5~VXvq-$q9FE>!Q~5+?r`n3 zx}A}Dl6zxoi2mH-8Qcf3(xJKHa5=igfx=a z)?eqlN2`A@wZsYi6hbf=*R`l162Sa-D!jm27%)^%Jjtozxmwc1ZIe!K)WY^y<4HAs zH2)d-+=Bn#fxrOb0+&O=()u5`2w(19^E>?ooZg?{LvOvD{q?FRW#sA23&J~Ly%B`s zt9~iw)MCkS6U>3zSb%vQU&nnDTX4JLI@TcCj@~=ua9#-DNHN%os%&ON{1kK!!K$SF zR^0)<4+tt1Dd`9}bslx~M|$F{3k1{*2L+9sZ38ne_(BBh9tjco!@aDAtp&aA13=hD z>c=mgvo#Gn>0&M$uW`NU6KbKXf5%HB3fY9Lo??K}*K2>whC|A?`tH(SY^ujS#2^I= z+X2+2jD9K6NAE2AZHx+KLYf5Tpov%+I;ScbLHnEJXzd8ta2LIR*+%MmNpD zE3l|CwiJu3C3(CxbHm!%!M;D%ji&!MpMAG;`hYjNu<--I-+c#oIec9GhcNvoMGcRS zXgh+pP|N%pI42iLm$lSQPeGRzm2s(wt2$4RL`G`AFRRP*TQKi>J$^ylq+~r~x?S80bVjYLNCz>iDM#5YIYUHw2+yE9+;j-fnuvXIER|D9QuG5w|p% zwstXF0S6&N;!~-!Nb{P^OtoBT$3N8eLi&oYbMnlV7V9X^|6y6PUtL#JJ>OgN%`h$v zCKBOC?0ujMi1CSxqwQYoMHD4QDK5F`yQ%rbX;Eb5W&2qdOUrzXY zaXa0IqSF_S=B4HMpTQ(C67rxS#%4Ie1CI;aG)a2dqb?Er{QA_{=(=ffFJuoLrqkq3 zQ=sO`aj>zu@QgM3=x8L#Y}EMc9eCnOgk37!^Wwvi(m?T>#>uOC5iow(Ufm^61v<8r z+=?%)12bi|cjx@5_?O`ueMC$blQ>Mj_s?+kj~7z~?G{f?QC;J{@_%o!Xi^YA-$eF> z|4sV;D(ymq`9hh5Y>V zzwcag-2Hd6fBk^v3NggE%|d<< zqo+Z)iAh;HZxwI<*PY9xg5Ljb_OGFRA|c{KMiox-U&y`v#zy(qQ~yRuo8|2{^~(>n z{`(Hq_~JkM@vlp1GGc`IO!_g||Hbkx5q$A~H~SZO{{#QedH5gr#D#(XN&X*N^FOn5 ziS&PF=TA@aKb!wQhx?Z+{h!0V^#3321f5^i^;G0~Q=FZAiTI}^ul@oiYw_;?0ab`9 APyhe` diff --git a/model_cards/fmikaelian/camembert-base-fquad/README.md b/model_cards/fmikaelian/camembert-base-fquad/README.md deleted file mode 100644 index 8c6e90f37d..0000000000 --- a/model_cards/fmikaelian/camembert-base-fquad/README.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -language: fr ---- - -# camembert-base-fquad - -## Description - -A baseline model for question-answering in french ([CamemBERT](https://camembert-model.fr/) model fine-tuned on [FQuAD](https://fquad.illuin.tech/)) - -## Training hyperparameters - -```shell -python3 ./examples/question-answering/run_squad.py \ ---model_type camembert \ ---model_name_or_path camembert-base \ ---do_train \ ---do_eval \ ---do_lower_case \ ---train_file train.json \ ---predict_file valid.json \ ---learning_rate 3e-5 \ ---num_train_epochs 2 \ ---max_seq_length 384 \ ---doc_stride 128 \ ---output_dir output \ ---per_gpu_eval_batch_size=3 \ ---per_gpu_train_batch_size=3 \ ---save_steps 10000 -``` - -## Evaluation results - -```shell -{"f1": 77.24515316052342, "exact_match": 52.82308657465496} -``` - -## Usage - -```python -from transformers import pipeline - -nlp = pipeline('question-answering', model='fmikaelian/camembert-base-fquad', tokenizer='fmikaelian/camembert-base-fquad') - -nlp({ - 'question': "Qui est Claude Monet?", - 'context': "Claude Monet, né le 14 novembre 1840 à Paris et mort le 5 décembre 1926 à Giverny, est un peintre français et l’un des fondateurs de l'impressionnisme." -}) -``` \ No newline at end of file diff --git a/model_cards/fmikaelian/camembert-base-squad/README.md b/model_cards/fmikaelian/camembert-base-squad/README.md deleted file mode 100644 index ead8b6aff9..0000000000 --- a/model_cards/fmikaelian/camembert-base-squad/README.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -language: fr ---- - -# camembert-base-squad - -## Description - -A baseline model for question-answering in french ([CamemBERT](https://camembert-model.fr/) model fine-tuned on [french-translated SQuAD 1.1 dataset](https://github.com/Alikabbadj/French-SQuAD)) - -## Training hyperparameters - -```shell -python3 ./examples/question-answering/run_squad.py \ ---model_type camembert \ ---model_name_or_path camembert-base \ ---do_train \ ---do_eval \ ---do_lower_case \ ---train_file SQuAD-v1.1-train_fr_ss999_awstart2_net.json \ ---predict_file SQuAD-v1.1-dev_fr_ss999_awstart2_net.json \ ---learning_rate 3e-5 \ ---num_train_epochs 2 \ ---max_seq_length 384 \ ---doc_stride 128 \ ---output_dir output3 \ ---per_gpu_eval_batch_size=3 \ ---per_gpu_train_batch_size=3 \ ---save_steps 10000 -``` - -## Evaluation results - -```shell -{"f1": 79.8570684959745, "exact_match": 59.21327108373895} -``` - -## Usage - -```python -from transformers import pipeline - -nlp = pipeline('question-answering', model='fmikaelian/camembert-base-squad', tokenizer='fmikaelian/camembert-base-squad') - -nlp({ - 'question': "Qui est Claude Monet?", - 'context': "Claude Monet, né le 14 novembre 1840 à Paris et mort le 5 décembre 1926 à Giverny, est un peintre français et l’un des fondateurs de l'impressionnisme." -}) -``` \ No newline at end of file diff --git a/model_cards/fmikaelian/flaubert-base-uncased-squad/README.md b/model_cards/fmikaelian/flaubert-base-uncased-squad/README.md deleted file mode 100644 index 147d3187dc..0000000000 --- a/model_cards/fmikaelian/flaubert-base-uncased-squad/README.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -language: fr ---- - -# flaubert-base-uncased-squad - -## Description - -A baseline model for question-answering in french ([flaubert](https://github.com/getalp/Flaubert) model fine-tuned on [french-translated SQuAD 1.1 dataset](https://github.com/Alikabbadj/French-SQuAD)) - -## Training hyperparameters - -```shell -python3 ./examples/question-answering/run_squad.py \ ---model_type flaubert \ ---model_name_or_path flaubert-base-uncased \ ---do_train \ ---do_eval \ ---do_lower_case \ ---train_file SQuAD-v1.1-train_fr_ss999_awstart2_net.json \ ---predict_file SQuAD-v1.1-dev_fr_ss999_awstart2_net.json \ ---learning_rate 3e-5 \ ---num_train_epochs 2 \ ---max_seq_length 384 \ ---doc_stride 128 \ ---output_dir output \ ---per_gpu_eval_batch_size=3 \ ---per_gpu_train_batch_size=3 -``` - -## Evaluation results - -```shell -{"f1": 68.66174806561969, "exact_match": 49.299692063176714} -``` - -## Usage - -```python -from transformers import pipeline - -nlp = pipeline('question-answering', model='fmikaelian/flaubert-base-uncased-squad', tokenizer='fmikaelian/flaubert-base-uncased-squad') - -nlp({ - 'question': "Qui est Claude Monet?", - 'context': "Claude Monet, né le 14 novembre 1840 à Paris et mort le 5 décembre 1926 à Giverny, est un peintre français et l’un des fondateurs de l'impressionnisme." -}) -``` \ No newline at end of file diff --git a/model_cards/fran-martinez/scibert_scivocab_cased_ner_jnlpba/README.md b/model_cards/fran-martinez/scibert_scivocab_cased_ner_jnlpba/README.md deleted file mode 100644 index 8008fded17..0000000000 --- a/model_cards/fran-martinez/scibert_scivocab_cased_ner_jnlpba/README.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -language: scientific english ---- - -# SciBERT finetuned on JNLPA for NER downstream task -## Language Model - [SciBERT](https://arxiv.org/pdf/1903.10676.pdf) is a pretrained language model based on BERT and trained by the - [Allen Institute for AI](https://allenai.org/) on papers from the corpus of - [Semantic Scholar](https://www.semanticscholar.org/). - Corpus size is 1.14M papers, 3.1B tokens. SciBERT has its own vocabulary (scivocab) that's built to best match - the training corpus. - -## Downstream task -[`allenai/scibert_scivocab_cased`](https://huggingface.co/allenai/scibert_scivocab_cased#) has been finetuned for Named Entity -Recognition (NER) dowstream task. The code to train the NER can be found [here](https://github.com/fran-martinez/bio_ner_bert). - -### Data -The corpus used to fine-tune the NER is [BioNLP / JNLPBA shared task](http://www.geniaproject.org/shared-tasks/bionlp-jnlpba-shared-task-2004). - -- Training data consist of 2,000 PubMed abstracts with term/word annotation. This corresponds to 18,546 samples (senteces). -- Evaluation data consist of 404 PubMed abstracts with term/word annotation. This corresponds to 3,856 samples (sentences). - -The classes (at word level) and its distribution (number of examples for each class) for training and evaluation datasets are shown below: - -| Class Label | # training examples| # evaluation examples| -|:--------------|--------------:|----------------:| -|O | 382,963 | 81,647 | -|B-protein | 30,269 | 5,067 | -|I-protein | 24,848 | 4,774 | -|B-cell_type | 6,718 | 1,921 | -|I-cell_type | 8,748 | 2,991 | -|B-DNA | 9,533 | 1,056 | -|I-DNA | 15,774 | 1,789 | -|B-cell_line | 3,830 | 500 | -|I-cell_line | 7,387 | 9,89 | -|B-RNA | 951 | 118 | -|I-RNA | 1,530 | 187 | - -### Model -An exhaustive hyperparameter search was done. -The hyperparameters that provided the best results are: - -- Max length sequence: 128 -- Number of epochs: 6 -- Batch size: 32 -- Dropout: 0.3 -- Optimizer: Adam - -The used learning rate was 5e-5 with a decreasing linear schedule. A warmup was used at the beggining of the training -with a ratio of steps equal to 0.1 from the total training steps. - -The model from the epoch with the best F1-score was selected, in this case, the model from epoch 5. - - -### Evaluation -The following table shows the evaluation metrics calculated at span/entity level: - -| | precision| recall| f1-score| -|:---------|-----------:|---------:|---------:| -cell_line | 0.5205 | 0.7100 | 0.6007 | -cell_type | 0.7736 | 0.7422 | 0.7576 | -protein | 0.6953 | 0.8459 | 0.7633 | -DNA | 0.6997 | 0.7894 | 0.7419 | -RNA | 0.6985 | 0.8051 | 0.7480 | -| | | | -**micro avg** | 0.6984 | 0.8076 | 0.7490| -**macro avg** | 0.7032 | 0.8076 | 0.7498 | - -The macro F1-score is equal to 0.7498, compared to the value provided by the Allen Institute for AI in their -[paper](https://arxiv.org/pdf/1903.10676.pdf), which is equal to 0.7728. This drop in performance could be due to -several reasons, but one hypothesis could be the fact that the authors used an additional conditional random field, -while this model uses a regular classification layer with softmax activation on top of SciBERT model. - -At word level, this model achieves a precision of 0.7742, a recall of 0.8536 and a F1-score of 0.8093. - -### Model usage in inference -Use the pipeline: -````python -from transformers import pipeline - -text = "Mouse thymus was used as a source of glucocorticoid receptor from normal CS lymphocytes." - -nlp_ner = pipeline("ner", - model='fran-martinez/scibert_scivocab_cased_ner_jnlpba', - tokenizer='fran-martinez/scibert_scivocab_cased_ner_jnlpba') - -nlp_ner(text) - -""" -Output: ---------------------------- -[ -{'word': 'glucocorticoid', -'score': 0.9894881248474121, -'entity': 'B-protein'}, - -{'word': 'receptor', -'score': 0.989505410194397, -'entity': 'I-protein'}, - -{'word': 'normal', -'score': 0.7680378556251526, -'entity': 'B-cell_type'}, - -{'word': 'cs', -'score': 0.5176806449890137, -'entity': 'I-cell_type'}, - -{'word': 'lymphocytes', -'score': 0.9898491501808167, -'entity': 'I-cell_type'} -] -""" -```` -Or load model and tokenizer as follows: -````python -import torch -from transformers import AutoTokenizer, AutoModelForTokenClassification - -# Example -text = "Mouse thymus was used as a source of glucocorticoid receptor from normal CS lymphocytes." - -# Load model -tokenizer = AutoTokenizer.from_pretrained("fran-martinez/scibert_scivocab_cased_ner_jnlpba") -model = AutoModelForTokenClassification.from_pretrained("fran-martinez/scibert_scivocab_cased_ner_jnlpba") - -# Get input for BERT -input_ids = torch.tensor(tokenizer.encode(text)).unsqueeze(0) - -# Predict -with torch.no_grad(): - outputs = model(input_ids) - -# From the output let's take the first element of the tuple. -# Then, let's get rid of [CLS] and [SEP] tokens (first and last) -predictions = outputs[0].argmax(axis=-1)[0][1:-1] - -# Map label class indexes to string labels. -for token, pred in zip(tokenizer.tokenize(text), predictions): - print(token, '->', model.config.id2label[pred.numpy().item()]) - -""" -Output: ---------------------------- -mouse -> O -thymus -> O -was -> O -used -> O -as -> O -a -> O -source -> O -of -> O -glucocorticoid -> B-protein -receptor -> I-protein -from -> O -normal -> B-cell_type -cs -> I-cell_type -lymphocytes -> I-cell_type -. -> O -""" -```` diff --git a/model_cards/funnel-transformer/intermediate-base/README.md b/model_cards/funnel-transformer/intermediate-base/README.md deleted file mode 100644 index 68d3c4a9e1..0000000000 --- a/model_cards/funnel-transformer/intermediate-base/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer intermediate model (B6-6-6 without decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -**Note:** This model does not contain the decoder, so it ouputs hidden states that have a sequence length of one fourth -of the inputs. It's good to use for tasks requiring a summary of the sentence (like sentence classification) but not if -you need one input per initial token. You should use the `intermediate` model in that case. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/intermediate-base") -model = FunnelBaseModel.from_pretrained("funnel-transformer/intermediate-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/intermediate-base") -model = TFFunnelBaseModel.from_pretrained("funnel-transformer/intermediate-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/intermediate/README.md b/model_cards/funnel-transformer/intermediate/README.md deleted file mode 100644 index 5645505a7c..0000000000 --- a/model_cards/funnel-transformer/intermediate/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer intermediate model (B6-6-6 with decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/intermediate") -model = FunneModel.from_pretrained("funnel-transformer/intermediate") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/intermediate") -model = TFFunnelModel.from_pretrained("funnel-transformer/intermediatesmall") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/large-base/README.md b/model_cards/funnel-transformer/large-base/README.md deleted file mode 100644 index e8dd2f3e53..0000000000 --- a/model_cards/funnel-transformer/large-base/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer large model (B8-8-8 without decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -**Note:** This model does not contain the decoder, so it ouputs hidden states that have a sequence length of one fourth -of the inputs. It's good to use for tasks requiring a summary of the sentence (like sentence classification) but not if -you need one input per initial token. You should use the `large` model in that case. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/large-base") -model = FunnelBaseModel.from_pretrained("funnel-transformer/large-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/large-base") -model = TFFunnelBaseModel.from_pretrained("funnel-transformer/large-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/large/README.md b/model_cards/funnel-transformer/large/README.md deleted file mode 100644 index 9c8128e96d..0000000000 --- a/model_cards/funnel-transformer/large/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer large model (B8-8-8 with decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/large") -model = FunneModel.from_pretrained("funnel-transformer/large") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/large") -model = TFFunnelModel.from_pretrained("funnel-transformer/large") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/medium-base/README.md b/model_cards/funnel-transformer/medium-base/README.md deleted file mode 100644 index 414c94eed9..0000000000 --- a/model_cards/funnel-transformer/medium-base/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer medium model (B6-3x2-3x2 without decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -**Note:** This model does not contain the decoder, so it ouputs hidden states that have a sequence length of one fourth -of the inputs. It's good to use for tasks requiring a summary of the sentence (like sentence classification) but not if -you need one input per initial token. You should use the `medium` model in that case. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/medium-base") -model = FunnelBaseModel.from_pretrained("funnel-transformer/medium-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/medium-base") -model = TFFunnelBaseModel.from_pretrained("funnel-transformer/medium-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/medium/README.md b/model_cards/funnel-transformer/medium/README.md deleted file mode 100644 index d5db54b89d..0000000000 --- a/model_cards/funnel-transformer/medium/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer medium model (B6-3x2-3x2 with decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/medium") -model = FunneModel.from_pretrained("funnel-transformer/medium") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/medium") -model = TFFunnelModel.from_pretrained("funnel-transformer/medium") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/small-base/README.md b/model_cards/funnel-transformer/small-base/README.md deleted file mode 100644 index 30ba16c717..0000000000 --- a/model_cards/funnel-transformer/small-base/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer small model (B4-4-4 without decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -**Note:** This model does not contain the decoder, so it ouputs hidden states that have a sequence length of one fourth -of the inputs. It's good to use for tasks requiring a summary of the sentence (like sentence classification) but not if -you need one input per initial token. You should use the `small` model in that case. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/small-base") -model = FunnelBaseModel.from_pretrained("funnel-transformer/small-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/small-base") -model = TFFunnelBaseModel.from_pretrained("funnel-transformer/small-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/small/README.md b/model_cards/funnel-transformer/small/README.md deleted file mode 100644 index 3a53ca0510..0000000000 --- a/model_cards/funnel-transformer/small/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer small model (B4-4-4 with decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/small") -model = FunneModel.from_pretrained("funnel-transformer/small") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/small") -model = TFFunnelModel.from_pretrained("funnel-transformer/small") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/xlarge-base/README.md b/model_cards/funnel-transformer/xlarge-base/README.md deleted file mode 100644 index 7461fb5112..0000000000 --- a/model_cards/funnel-transformer/xlarge-base/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer xlarge model (B10-10-10 without decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -**Note:** This model does not contain the decoder, so it ouputs hidden states that have a sequence length of one fourth -of the inputs. It's good to use for tasks requiring a summary of the sentence (like sentence classification) but not if -you need one input per initial token. You should use the `xlarge` model in that case. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/xlarge-base") -model = FunnelBaseModel.from_pretrained("funnel-transformer/xlarge-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelBaseModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/xlarge-base") -model = TFFunnelBaseModel.from_pretrained("funnel-transformer/xlarge-base") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/funnel-transformer/xlarge/README.md b/model_cards/funnel-transformer/xlarge/README.md deleted file mode 100644 index a1a0a69f24..0000000000 --- a/model_cards/funnel-transformer/xlarge/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- bookcorpus -- wikipedia -- gigaword ---- - -# Funnel Transformer xlarge model (B10-10-10 with decoder) - -Pretrained model on English language using a similar objective objective as [ELECTRA](https://huggingface.co/transformers/model_doc/electra.html). It was introduced in -[this paper](https://arxiv.org/pdf/2006.03236.pdf) and first released in -[this repository](https://github.com/laiguokun/Funnel-Transformer). This model is uncased: it does not make a difference -between english and English. - -Disclaimer: The team releasing Funnel Transformer did not write a model card for this model so this model card has been -written by the Hugging Face team. - -## Model description - -Funnel Transformer is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it -was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, a small language model corrupts the input texts and serves as a generator of inputs for this model, and -the pretraining objective is to predict which token is an original and which one has been replaced, a bit like a GAN training. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model to extract a vector representation of a given text, but it's mostly intended to -be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=funnel-transformer) to look for -fine-tuned versions on a task that interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import FunnelTokenizer, FunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/xlarge") -model = FunneModel.from_pretrained("funnel-transformer/xlarge") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import FunnelTokenizer, TFFunnelModel -tokenizer = FunnelTokenizer.from_pretrained("funnel-transformer/xlarge") -model = TFFunnelModel.from_pretrained("funnel-transformer/xlarge") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -## Training data - -The BERT model was pretrained on: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books, -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers), -- [Clue Web](https://lemurproject.org/clueweb12/), a dataset of 733,019,372 English web pages, -- [GigaWord](https://catalog.ldc.upenn.edu/LDC2011T07), an archive of newswire text data, -- [Common Crawl](https://commoncrawl.org/), a dataset of raw web pages. - - -### BibTeX entry and citation info - -```bibtex -@misc{dai2020funneltransformer, - title={Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing}, - author={Zihang Dai and Guokun Lai and Yiming Yang and Quoc V. Le}, - year={2020}, - eprint={2006.03236}, - archivePrefix={arXiv}, - primaryClass={cs.LG} -} -``` - diff --git a/model_cards/ganeshkharad/gk-hinglish-sentiment/README.md b/model_cards/ganeshkharad/gk-hinglish-sentiment/README.md deleted file mode 100644 index 28d442fa73..0000000000 --- a/model_cards/ganeshkharad/gk-hinglish-sentiment/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -language: -- hi-en - -tags: -- sentiment -- multilingual -- hindi codemix -- hinglish -license: apache-2.0 -datasets: -- sail ---- - -# Sentiment Classification for hinglish text: `gk-hinglish-sentiment` - -## Model description - -Trained small amount of reviews dataset - -## Intended uses & limitations - -I wanted something to work well with hinglish data as it is being used in India mostly. -The training data was not much as expected - -#### How to use - -```python -#sample code -from transformers import BertTokenizer, BertForSequenceClassification -tokenizerg = BertTokenizer.from_pretrained("/content/model") -modelg = BertForSequenceClassification.from_pretrained("/content/model") - -text = "kuch bhi type karo hinglish mai" -encoded_input = tokenizerg(text, return_tensors='pt') -output = modelg(**encoded_input) -print(output) -#output contains 3 lables LABEL_0 = Negative ,LABEL_1 = Nuetral ,LABEL_2 = Positive -``` - -#### Limitations and bias - -The data contains only hinglish codemixed text it and was very much limited may be I will Update this model if I can get good amount of data - -## Training data - -Training data contains labeled data for 3 labels - -link to the pre-trained model card with description of the pre-training data. -I have Tuned below model - -https://huggingface.co/rohanrajpal/bert-base-multilingual-codemixed-cased-sentiment - - -### BibTeX entry and citation info - -```@inproceedings{khanuja-etal-2020-gluecos, - title = "{GLUEC}o{S}: An Evaluation Benchmark for Code-Switched {NLP}", - author = "Khanuja, Simran and - Dandapat, Sandipan and - Srinivasan, Anirudh and - Sitaram, Sunayana and - Choudhury, Monojit", - booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics", - month = jul, - year = "2020", - address = "Online", - publisher = "Association for Computational Linguistics", - url = "https://www.aclweb.org/anthology/2020.acl-main.329", - pages = "3575--3585" -} -``` diff --git a/model_cards/gaochangkuan/model_dir/README.md b/model_cards/gaochangkuan/model_dir/README.md deleted file mode 100644 index 41d3e81ebf..0000000000 --- a/model_cards/gaochangkuan/model_dir/README.md +++ /dev/null @@ -1,66 +0,0 @@ -## Generating Chinese poetry by topic. - -```python -from transformers import * - -tokenizer = BertTokenizer.from_pretrained("gaochangkuan/model_dir") - -model = AutoModelWithLMHead.from_pretrained("gaochangkuan/model_dir") - - -prompt= '''田园躬耕''' - -length= 84 -stop_token='' - -temperature = 1.2 - -repetition_penalty=1.3 - -k= 30 -p= 0.95 - -device ='cuda' -seed=2020 -no_cuda=False - -prompt_text = prompt if prompt else input("Model prompt >>> ") - -encoded_prompt = tokenizer.encode( - ''+prompt_text+'', - add_special_tokens=False, - return_tensors="pt" - ) - -encoded_prompt = encoded_prompt.to(device) - -output_sequences = model.generate( - input_ids=encoded_prompt, - max_length=length, - min_length=10, - do_sample=True, - early_stopping=True, - num_beams=10, - temperature=temperature, - top_k=k, - top_p=p, - repetition_penalty=repetition_penalty, - bad_words_ids=None, - bos_token_id=tokenizer.bos_token_id, - pad_token_id=tokenizer.pad_token_id, - eos_token_id=tokenizer.eos_token_id, - length_penalty=1.2, - no_repeat_ngram_size=2, - num_return_sequences=1, - attention_mask=None, - decoder_start_token_id=tokenizer.bos_token_id,) - - - generated_sequence = output_sequences[0].tolist() -text = tokenizer.decode(generated_sequence) - - -text = text[: text.find(stop_token) if stop_token else None] - -print(''.join(text).replace(' ','').replace('','').replace('','')) -``` diff --git a/model_cards/german-nlp-group/electra-base-german-uncased/README.md b/model_cards/german-nlp-group/electra-base-german-uncased/README.md deleted file mode 100644 index e4ba72a0c4..0000000000 --- a/model_cards/german-nlp-group/electra-base-german-uncased/README.md +++ /dev/null @@ -1,152 +0,0 @@ ---- -language: de -license: mit -thumbnail: "https://raw.githubusercontent.com/German-NLP-Group/german-transformer-training/master/model_cards/german-electra-logo.png" -tags: -- electra -- commoncrawl -- uncased -- umlaute -- umlauts -- german -- deutsch ---- - -# German Electra Uncased - -[¹] - -# Model Info -This Model is suitable for Training on many downstream tasks in German (Q&A, Sentiment Analysis, etc.). - -It can be used as a drop-in Replacement for **BERT** in most down-stream tasks (**ELECTRA** is even implemented as an extended **BERT** Class). - -At the time of release (August 2020) this Model is the best performing publicly available German NLP Model on various German Evaluation Metrics (CONLL03-DE, GermEval18 Coarse, GermEval18 Fine). For GermEval18 Coarse results see below. More will be published soon. - -# Installation -This model has the special feature that it is **uncased** but does **not strip accents**. -This possibility was added by us with [PR #6280](https://github.com/huggingface/transformers/pull/6280). -To use it you have to use Transformers version 3.1.0 or newer. - -```bash -pip install transformers -U -``` - -# Uncase and Umlauts ('Ö', 'Ä', 'Ü') -This model is uncased. This helps especially for domains where colloquial terms with uncorrect capitalization is often used. - -The special characters 'ö', 'ü', 'ä' are included through the `strip_accent=False` option, as this leads to an improved precision. - -# Creators -This model was trained and open sourced in conjunction with the [**German NLP Group**](https://github.com/German-NLP-Group) in equal parts by: -- [**Philip May**](https://eniak.de) - [T-Systems on site services GmbH](https://www.t-systems-onsite.de/) -- [**Philipp Reißel**](https://www.reissel.eu) - [ambeRoad](https://amberoad.de/) - -# Evaluation: GermEval18 Coarse - -| Model Name | F1 macro
Mean | F1 macro
Median | F1 macro
Std | -|---|---|---|---| -| dbmdz-bert-base-german-europeana-cased | 0.727 | 0.729 | 0.00674 | -| dbmdz-bert-base-german-europeana-uncased | 0.736 | 0.737 | 0.00476 | -| dbmdz/electra-base-german-europeana-cased-discriminator | 0.745 | 0.745 | 0.00498 | -| distilbert-base-german-cased | 0.752 | 0.752 | 0.00341 | -| bert-base-german-cased | 0.762 | 0.761 | 0.00597 | -| dbmdz/bert-base-german-cased | 0.765 | 0.765 | 0.00523 | -| dbmdz/bert-base-german-uncased | 0.770 | 0.770 | 0.00572 | -| **ELECTRA-base-german-uncased (this model)** | **0.778** | **0.778** | **0.00392** | - -- (1): Hyperparameters taken from the [FARM project](https://farm.deepset.ai/) "[germEval18Coarse_config.json](https://github.com/deepset-ai/FARM/blob/master/experiments/german-bert2.0-eval/germEval18Coarse_config.json)" - -![GermEval18 Coarse Model Evaluation](https://raw.githubusercontent.com/German-NLP-Group/german-transformer-training/master/model_cards/model_eval.png) - -# Checkpoint evaluation -Since it it not guaranteed that the last checkpoint is the best, we evaluated the checkpoints on GermEval18. We found that the last checkpoint is indeed the best. The training was stable and did not overfit the text corpus. Below is a boxplot chart showing the different checkpoints. - -![Checkpoint Evaluation on GermEval18](https://raw.githubusercontent.com/German-NLP-Group/german-transformer-training/master/model_cards/checkpoint_eval.png) - -# Pre-training details - -## Data -- Cleaned Common Crawl Corpus 2019-09 German: [CC_net](https://github.com/facebookresearch/cc_net) (Only head coprus and filtered for language_score > 0.98) - 62 GB -- German Wikipedia Article Pages Dump (20200701) - 5.5 GB -- German Wikipedia Talk Pages Dump (20200620) - 1.1 GB -- Subtitles - 823 MB -- News 2018 - 4.1 GB - -The sentences were split with [SojaMo](https://github.com/tsproisl/SoMaJo). We took the German Wikipedia Article Pages Dump 3x to oversample. This approach was also used in a similar way in GPT-3 (Table 2.2). - -More Details can be found here [Preperaing Datasets for German Electra Github](https://github.com/German-NLP-Group/german-transformer-training) - -## Electra Branch no_strip_accents -Because we do not want to stip accents in our training data we made a change to Electra and used this repo [Electra no_strip_accents](https://github.com/PhilipMay/electra/tree/no_strip_accents) (branch `no_strip_accents`). Then created the tf dataset with: - -```bash -python build_pretraining_dataset.py --corpus-dir --vocab-file

/vocab.txt --output-dir ./tf_data --max-seq-length 512 --num-processes 8 --do-lower-case --no-strip-accents -``` - -## The training -The training itself can be performed with the Original Electra Repo (No special case for this needed). -We run it with the following Config: - -
-The exact Training Config -
debug False -
disallow_correct False -
disc_weight 50.0 -
do_eval False -
do_lower_case True -
do_train True -
electra_objective True -
embedding_size 768 -
eval_batch_size 128 -
gcp_project None -
gen_weight 1.0 -
generator_hidden_size 0.33333 -
generator_layers 1.0 -
iterations_per_loop 200 -
keep_checkpoint_max 0 -
learning_rate 0.0002 -
lr_decay_power 1.0 -
mask_prob 0.15 -
max_predictions_per_seq 79 -
max_seq_length 512 -
model_dir gs://XXX -
model_hparam_overrides {} -
model_name 02_Electra_Checkpoints_32k_766k_Combined -
model_size base -
num_eval_steps 100 -
num_tpu_cores 8 -
num_train_steps 766000 -
num_warmup_steps 10000 -
pretrain_tfrecords gs://XXX -
results_pkl gs://XXX -
results_txt gs://XXX -
save_checkpoints_steps 5000 -
temperature 1.0 -
tpu_job_name None -
tpu_name electrav5 -
tpu_zone None -
train_batch_size 256 -
uniform_generator False -
untied_generator True -
untied_generator_embeddings False -
use_tpu True -
vocab_file gs://XXX -
vocab_size 32767 -
weight_decay_rate 0.01 -
- -![Training Loss](https://raw.githubusercontent.com/German-NLP-Group/german-transformer-training/master/model_cards/loss.png) - -Please Note: *Due to the GAN like strucutre of Electra the loss is not that meaningful* - -It took about 7 Days on a preemtible TPU V3-8. In total, the Model went through approximately 10 Epochs. For an automatically recreation of a cancelled TPUs we used [tpunicorn](https://github.com/shawwn/tpunicorn). The total cost of training summed up to about 450 $ for one run. The Data-pre processing and Vocab Creation needed approximately 500-1000 CPU hours. Servers were fully provided by [T-Systems on site services GmbH](https://www.t-systems-onsite.de/), [ambeRoad](https://amberoad.de/). -Special thanks to [Stefan Schweter](https://github.com/stefan-it) for your feedback and providing parts of the text corpus. - -[¹]: Source for the picture [Pinterest](https://www.pinterest.cl/pin/371828512984142193/) - -# Negative Results -We tried the following approaches which we found had no positive influence: - -- **Increased Vocab Size**: Leads to more parameters and thus reduced examples/sec while no visible Performance gains were measured -- **Decreased Batch-Size**: The original Electra was trained with a Batch Size per TPU Core of 16 whereas this Model was trained with 32 BS / TPU Core. We found out that 32 BS leads to better results when you compare metrics over computation time diff --git a/model_cards/giganticode/StackOBERTflow-comments-small-v1/README.md b/model_cards/giganticode/StackOBERTflow-comments-small-v1/README.md deleted file mode 100644 index 739d3e3557..0000000000 --- a/model_cards/giganticode/StackOBERTflow-comments-small-v1/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# StackOBERTflow-comments-small - -StackOBERTflow is a RoBERTa model trained on StackOverflow comments. -A Byte-level BPE tokenizer with dropout was used (using the `tokenizers` package). - -The model is *small*, i.e. has only 6-layers and the maximum sequence length was restricted to 256 tokens. -The model was trained for 6 epochs on several GBs of comments from the StackOverflow corpus. - -## Quick start: masked language modeling prediction - -```python -from transformers import pipeline -from pprint import pprint - -COMMENT = "You really should not do it this way, I would use instead." - -fill_mask = pipeline( - "fill-mask", - model="giganticode/StackOBERTflow-comments-small-v1", - tokenizer="giganticode/StackOBERTflow-comments-small-v1" -) - -pprint(fill_mask(COMMENT)) -# [{'score': 0.019997311756014824, -# 'sequence': ' You really should not do it this way, I would use jQuery instead.', -# 'token': 1738}, -# {'score': 0.01693696901202202, -# 'sequence': ' You really should not do it this way, I would use arrays instead.', -# 'token': 2844}, -# {'score': 0.013411642983555794, -# 'sequence': ' You really should not do it this way, I would use CSS instead.', -# 'token': 2254}, -# {'score': 0.013224546797573566, -# 'sequence': ' You really should not do it this way, I would use it instead.', -# 'token': 300}, -# {'score': 0.011984303593635559, -# 'sequence': ' You really should not do it this way, I would use classes instead.', -# 'token': 1779}] -``` diff --git a/model_cards/gilf/french-camembert-postag-model/README.md b/model_cards/gilf/french-camembert-postag-model/README.md deleted file mode 100644 index f884dc5f55..0000000000 --- a/model_cards/gilf/french-camembert-postag-model/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: fr -widget: -- text: "Face à un choc inédit, les mesures mises en place par le gouvernement ont permis une protection forte et efficace des ménages" ---- - -## About - -The *french-camembert-postag-model* is a part of speech tagging model for French that was trained on the *free-french-treebank* dataset available on -[github](https://github.com/nicolashernandez/free-french-treebank). The base tokenizer and model used for training is *'camembert-base'*. - -## Supported Tags - -It uses the following tags: - -| Tag | Category | Extra Info | -|----------|:------------------------------:|------------:| -| ADJ | adjectif | | -| ADJWH | adjectif | | -| ADV | adverbe | | -| ADVWH | adverbe | | -| CC | conjonction de coordination | | -| CLO | pronom | obj | -| CLR | pronom | refl | -| CLS | pronom | suj | -| CS | conjonction de subordination | | -| DET | déterminant | | -| DETWH | déterminant | | -| ET | mot étranger | | -| I | interjection | | -| NC | nom commun | | -| NPP | nom propre | | -| P | préposition | | -| P+D | préposition + déterminant | | -| PONCT | signe de ponctuation | | -| PREF | préfixe | | -| PRO | autres pronoms | | -| PROREL | autres pronoms | rel | -| PROWH | autres pronoms | int | -| U | ? | | -| V | verbe | | -| VIMP | verbe imperatif | | -| VINF | verbe infinitif | | -| VPP | participe passé | | -| VPR | participe présent | | -| VS | subjonctif | | - -More information on the tags can be found here: - -http://alpage.inria.fr/statgram/frdep/Publications/crabbecandi-taln2008-final.pdf - -## Usage - -The usage of this model follows the common transformers patterns. Here is a short example of its usage: - -```python -from transformers import AutoTokenizer, AutoModelForTokenClassification - -tokenizer = AutoTokenizer.from_pretrained("gilf/french-camembert-postag-model") -model = AutoModelForTokenClassification.from_pretrained("gilf/french-camembert-postag-model") - -from transformers import pipeline - -nlp_token_class = pipeline('ner', model=model, tokenizer=tokenizer, grouped_entities=True) - -nlp_token_class('Face à un choc inédit, les mesures mises en place par le gouvernement ont permis une protection forte et efficace des ménages') -``` - -The lines above would display something like this on a Jupyter notebook: - -``` -[{'entity_group': 'NC', 'score': 0.5760144591331482, 'word': ''}, - {'entity_group': 'U', 'score': 0.9946700930595398, 'word': 'Face'}, - {'entity_group': 'P', 'score': 0.999615490436554, 'word': 'à'}, - {'entity_group': 'DET', 'score': 0.9995906352996826, 'word': 'un'}, - {'entity_group': 'NC', 'score': 0.9995531439781189, 'word': 'choc'}, - {'entity_group': 'ADJ', 'score': 0.999183714389801, 'word': 'inédit'}, - {'entity_group': 'P', 'score': 0.3710663616657257, 'word': ','}, - {'entity_group': 'DET', 'score': 0.9995903968811035, 'word': 'les'}, - {'entity_group': 'NC', 'score': 0.9995649456977844, 'word': 'mesures'}, - {'entity_group': 'VPP', 'score': 0.9988670349121094, 'word': 'mises'}, - {'entity_group': 'P', 'score': 0.9996246099472046, 'word': 'en'}, - {'entity_group': 'NC', 'score': 0.9995329976081848, 'word': 'place'}, - {'entity_group': 'P', 'score': 0.9996233582496643, 'word': 'par'}, - {'entity_group': 'DET', 'score': 0.9995935559272766, 'word': 'le'}, - {'entity_group': 'NC', 'score': 0.9995369911193848, 'word': 'gouvernement'}, - {'entity_group': 'V', 'score': 0.9993771314620972, 'word': 'ont'}, - {'entity_group': 'VPP', 'score': 0.9991101026535034, 'word': 'permis'}, - {'entity_group': 'DET', 'score': 0.9995885491371155, 'word': 'une'}, - {'entity_group': 'NC', 'score': 0.9995636343955994, 'word': 'protection'}, - {'entity_group': 'ADJ', 'score': 0.9991781711578369, 'word': 'forte'}, - {'entity_group': 'CC', 'score': 0.9991298317909241, 'word': 'et'}, - {'entity_group': 'ADJ', 'score': 0.9992275238037109, 'word': 'efficace'}, - {'entity_group': 'P+D', 'score': 0.9993300437927246, 'word': 'des'}, - {'entity_group': 'NC', 'score': 0.8353511393070221, 'word': 'ménages'}] -``` diff --git a/model_cards/gilf/french-postag-model/README.md b/model_cards/gilf/french-postag-model/README.md deleted file mode 100644 index 613b7e2222..0000000000 --- a/model_cards/gilf/french-postag-model/README.md +++ /dev/null @@ -1,90 +0,0 @@ -## About - -The *french-postag-model* is a part of speech tagging model for French that was trained on the *free-french-treebank* dataset available on -[github](https://github.com/nicolashernandez/free-french-treebank). The base tokenizer and model used for training is *'bert-base-multilingual-cased'*. - -## Supported Tags - -It uses the following tags: - -| Tag | Category | Extra Info | -|----------|:------------------------------:|------------:| -| ADJ | adjectif | | -| ADJWH | adjectif | | -| ADV | adverbe | | -| ADVWH | adverbe | | -| CC | conjonction de coordination | | -| CLO | pronom | obj | -| CLR | pronom | refl | -| CLS | pronom | suj | -| CS | conjonction de subordination | | -| DET | déterminant | | -| DETWH | déterminant | | -| ET | mot étranger | | -| I | interjection | | -| NC | nom commun | | -| NPP | nom propre | | -| P | préposition | | -| P+D | préposition + déterminant | | -| PONCT | signe de ponctuation | | -| PREF | préfixe | | -| PRO | autres pronoms | | -| PROREL | autres pronoms | rel | -| PROWH | autres pronoms | int | -| U | ? | | -| V | verbe | | -| VIMP | verbe imperatif | | -| VINF | verbe infinitif | | -| VPP | participe passé | | -| VPR | participe présent | | -| VS | subjonctif | | - -More information on the tags can be found here: - -http://alpage.inria.fr/statgram/frdep/Publications/crabbecandi-taln2008-final.pdf - -## Usage - -The usage of this model follows the common transformers patterns. Here is a short example of its usage: - -```python -from transformers import AutoTokenizer, AutoModelForTokenClassification - -tokenizer = AutoTokenizer.from_pretrained("gilf/french-postag-model") -model = AutoModelForTokenClassification.from_pretrained("gilf/french-postag-model") - -from transformers import pipeline - -nlp_token_class = pipeline('ner', model=model, tokenizer=tokenizer, grouped_entities=True) - -nlp_token_class('Face à un choc inédit, les mesures mises en place par le gouvernement ont permis une protection forte et efficace des ménages') -``` - -The lines above would display something like this on a Jupyter notebook: - -``` -[{'entity_group': 'PONCT', 'score': 0.0742340236902237, 'word': '[CLS]'}, - {'entity_group': 'U', 'score': 0.9995399713516235, 'word': 'Face'}, - {'entity_group': 'P', 'score': 0.9999609589576721, 'word': 'à'}, - {'entity_group': 'DET', 'score': 0.9999597072601318, 'word': 'un'}, - {'entity_group': 'NC', 'score': 0.9998948276042938, 'word': 'choc'}, - {'entity_group': 'ADJ', 'score': 0.995318204164505, 'word': 'inédit'}, - {'entity_group': 'PONCT', 'score': 0.9999793171882629, 'word': ','}, - {'entity_group': 'DET', 'score': 0.999964714050293, 'word': 'les'}, - {'entity_group': 'NC', 'score': 0.999936580657959, 'word': 'mesures'}, - {'entity_group': 'VPP', 'score': 0.9995776414871216, 'word': 'mises'}, - {'entity_group': 'P', 'score': 0.99996417760849, 'word': 'en'}, - {'entity_group': 'NC', 'score': 0.999882161617279, 'word': 'place'}, - {'entity_group': 'P', 'score': 0.9999671578407288, 'word': 'par'}, - {'entity_group': 'DET', 'score': 0.9999637603759766, 'word': 'le'}, - {'entity_group': 'NC', 'score': 0.9999350309371948, 'word': 'gouvernement'}, - {'entity_group': 'V', 'score': 0.9999298453330994, 'word': 'ont'}, - {'entity_group': 'VPP', 'score': 0.9998740553855896, 'word': 'permis'}, - {'entity_group': 'DET', 'score': 0.9999625086784363, 'word': 'une'}, - {'entity_group': 'NC', 'score': 0.9999420046806335, 'word': 'protection'}, - {'entity_group': 'ADJ', 'score': 0.9998913407325745, 'word': 'forte'}, - {'entity_group': 'CC', 'score': 0.9998615980148315, 'word': 'et'}, - {'entity_group': 'ADJ', 'score': 0.9998483657836914, 'word': 'efficace'}, - {'entity_group': 'P+D', 'score': 0.9987645149230957, 'word': 'des'}, - {'entity_group': 'NC', 'score': 0.8720395267009735, 'word': 'ménages [SEP]'}] -``` diff --git a/model_cards/google/bert2bert_L-24_wmt_de_en/README.md b/model_cards/google/bert2bert_L-24_wmt_de_en/README.md deleted file mode 100644 index af86e42e32..0000000000 --- a/model_cards/google/bert2bert_L-24_wmt_de_en/README.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -language: -- en -- de -license: apache-2.0 -datasets: -- wmt14 -tags: -- translation ---- - -# bert2bert_L-24_wmt_de_en EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/bert24_de_en/1). - -The model is an encoder-decoder model that was initialized on the `bert-large` checkpoints for both the encoder -and decoder and fine-tuned on German to English translation on the WMT dataset, which is linked above. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for translation, *e.g.* - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/bert2bert_L-24_wmt_de_en", pad_token="", eos_token="", bos_token="") -model = AutoModelForSeq2SeqLM.from_pretrained("google/bert2bert_L-24_wmt_de_en") - -sentence = "Willst du einen Kaffee trinken gehen mit mir?" - -input_ids = tokenizer(sentence, return_tensors="pt", add_special_tokens=False).input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# Want to drink a kaffee go with me? . -``` diff --git a/model_cards/google/bert2bert_L-24_wmt_en_de/README.md b/model_cards/google/bert2bert_L-24_wmt_en_de/README.md deleted file mode 100644 index ab17b7ffa9..0000000000 --- a/model_cards/google/bert2bert_L-24_wmt_en_de/README.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -language: -- en -- de -license: apache-2.0 -datasets: -- wmt14 -tags: -- translation ---- - -# bert2bert_L-24_wmt_en_de EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/bert24_en_de/1). - -The model is an encoder-decoder model that was initialized on the `bert-large` checkpoints for both the encoder -and decoder and fine-tuned on English to German translation on the WMT dataset, which is linked above. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for translation, *e.g.* - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/bert2bert_L-24_wmt_en_de", pad_token="", eos_token="", bos_token="") -model = AutoModelForSeq2SeqLM.from_pretrained("google/bert2bert_L-24_wmt_en_de") - -sentence = "Would you like to grab a coffee with me this week?" - -input_ids = tokenizer(sentence, return_tensors="pt", add_special_tokens=False).input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# Möchten Sie diese Woche einen Kaffee mit mir schnappen? diff --git a/model_cards/google/bert_uncased_L-10_H-128_A-2/README.md b/model_cards/google/bert_uncased_L-10_H-128_A-2/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-10_H-128_A-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-10_H-256_A-4/README.md b/model_cards/google/bert_uncased_L-10_H-256_A-4/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-10_H-256_A-4/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-10_H-512_A-8/README.md b/model_cards/google/bert_uncased_L-10_H-512_A-8/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-10_H-512_A-8/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-10_H-768_A-12/README.md b/model_cards/google/bert_uncased_L-10_H-768_A-12/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-10_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-12_H-128_A-2/README.md b/model_cards/google/bert_uncased_L-12_H-128_A-2/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-12_H-128_A-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-12_H-256_A-4/README.md b/model_cards/google/bert_uncased_L-12_H-256_A-4/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-12_H-256_A-4/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-12_H-512_A-8/README.md b/model_cards/google/bert_uncased_L-12_H-512_A-8/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-12_H-512_A-8/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-12_H-768_A-12/README.md b/model_cards/google/bert_uncased_L-12_H-768_A-12/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-12_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-2_H-128_A-2/README.md b/model_cards/google/bert_uncased_L-2_H-128_A-2/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-2_H-128_A-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-2_H-256_A-4/README.md b/model_cards/google/bert_uncased_L-2_H-256_A-4/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-2_H-256_A-4/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-2_H-512_A-8/README.md b/model_cards/google/bert_uncased_L-2_H-512_A-8/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-2_H-512_A-8/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-2_H-768_A-12/README.md b/model_cards/google/bert_uncased_L-2_H-768_A-12/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-2_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-4_H-128_A-2/README.md b/model_cards/google/bert_uncased_L-4_H-128_A-2/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-4_H-128_A-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-4_H-256_A-4/README.md b/model_cards/google/bert_uncased_L-4_H-256_A-4/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-4_H-256_A-4/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-4_H-512_A-8/README.md b/model_cards/google/bert_uncased_L-4_H-512_A-8/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-4_H-512_A-8/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-4_H-768_A-12/README.md b/model_cards/google/bert_uncased_L-4_H-768_A-12/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-4_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-6_H-128_A-2/README.md b/model_cards/google/bert_uncased_L-6_H-128_A-2/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-6_H-128_A-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-6_H-256_A-4/README.md b/model_cards/google/bert_uncased_L-6_H-256_A-4/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-6_H-256_A-4/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-6_H-512_A-8/README.md b/model_cards/google/bert_uncased_L-6_H-512_A-8/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-6_H-512_A-8/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-6_H-768_A-12/README.md b/model_cards/google/bert_uncased_L-6_H-768_A-12/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-6_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-8_H-128_A-2/README.md b/model_cards/google/bert_uncased_L-8_H-128_A-2/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-8_H-128_A-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-8_H-256_A-4/README.md b/model_cards/google/bert_uncased_L-8_H-256_A-4/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-8_H-256_A-4/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-8_H-512_A-8/README.md b/model_cards/google/bert_uncased_L-8_H-512_A-8/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-8_H-512_A-8/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/bert_uncased_L-8_H-768_A-12/README.md b/model_cards/google/bert_uncased_L-8_H-768_A-12/README.md deleted file mode 120000 index 1ee4bd8fef..0000000000 --- a/model_cards/google/bert_uncased_L-8_H-768_A-12/README.md +++ /dev/null @@ -1 +0,0 @@ -../../iuliaturc/bert_uncased_L-2_H-128_A-2/README.md \ No newline at end of file diff --git a/model_cards/google/electra-base-discriminator/README.md b/model_cards/google/electra-base-discriminator/README.md deleted file mode 100644 index b24ddfef7c..0000000000 --- a/model_cards/google/electra-base-discriminator/README.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer to our paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -This repository contains code to pre-train ELECTRA, including small ELECTRA models on a single GPU. It also supports fine-tuning ELECTRA on downstream tasks including classification tasks (e.g,. [GLUE](https://gluebenchmark.com/)), QA tasks (e.g., [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/)), and sequence tagging tasks (e.g., [text chunking](https://www.clips.uantwerpen.be/conll2000/chunking/)). - -## How to use the discriminator in `transformers` - -```python -from transformers import ElectraForPreTraining, ElectraTokenizerFast -import torch - -discriminator = ElectraForPreTraining.from_pretrained("google/electra-base-discriminator") -tokenizer = ElectraTokenizerFast.from_pretrained("google/electra-base-discriminator") - -sentence = "The quick brown fox jumps over the lazy dog" -fake_sentence = "The quick brown fox fake over the lazy dog" - -fake_tokens = tokenizer.tokenize(fake_sentence) -fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -[print("%7s" % token, end="") for token in fake_tokens] - -[print("%7s" % int(prediction), end="") for prediction in predictions.tolist()] -``` diff --git a/model_cards/google/electra-base-generator/README.md b/model_cards/google/electra-base-generator/README.md deleted file mode 100644 index 8eb2d129cd..0000000000 --- a/model_cards/google/electra-base-generator/README.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer to our paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -This repository contains code to pre-train ELECTRA, including small ELECTRA models on a single GPU. It also supports fine-tuning ELECTRA on downstream tasks including classification tasks (e.g,. [GLUE](https://gluebenchmark.com/)), QA tasks (e.g., [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/)), and sequence tagging tasks (e.g., [text chunking](https://www.clips.uantwerpen.be/conll2000/chunking/)). - -## How to use the generator in `transformers` - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="google/electra-base-generator", - tokenizer="google/electra-base-generator" -) - -print( - fill_mask(f"HuggingFace is creating a {fill_mask.tokenizer.mask_token} that the community uses to solve NLP tasks.") -) - -``` diff --git a/model_cards/google/electra-large-discriminator/README.md b/model_cards/google/electra-large-discriminator/README.md deleted file mode 100644 index 119b012ace..0000000000 --- a/model_cards/google/electra-large-discriminator/README.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer to our paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -This repository contains code to pre-train ELECTRA, including small ELECTRA models on a single GPU. It also supports fine-tuning ELECTRA on downstream tasks including classification tasks (e.g,. [GLUE](https://gluebenchmark.com/)), QA tasks (e.g., [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/)), and sequence tagging tasks (e.g., [text chunking](https://www.clips.uantwerpen.be/conll2000/chunking/)). - -## How to use the discriminator in `transformers` - -```python -from transformers import ElectraForPreTraining, ElectraTokenizerFast -import torch - -discriminator = ElectraForPreTraining.from_pretrained("google/electra-large-discriminator") -tokenizer = ElectraTokenizerFast.from_pretrained("google/electra-large-discriminator") - -sentence = "The quick brown fox jumps over the lazy dog" -fake_sentence = "The quick brown fox fake over the lazy dog" - -fake_tokens = tokenizer.tokenize(fake_sentence) -fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -[print("%7s" % token, end="") for token in fake_tokens] - -[print("%7s" % int(prediction), end="") for prediction in predictions.tolist()] -``` diff --git a/model_cards/google/electra-large-generator/README.md b/model_cards/google/electra-large-generator/README.md deleted file mode 100644 index 31a033f44b..0000000000 --- a/model_cards/google/electra-large-generator/README.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer to our paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -This repository contains code to pre-train ELECTRA, including small ELECTRA models on a single GPU. It also supports fine-tuning ELECTRA on downstream tasks including classification tasks (e.g,. [GLUE](https://gluebenchmark.com/)), QA tasks (e.g., [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/)), and sequence tagging tasks (e.g., [text chunking](https://www.clips.uantwerpen.be/conll2000/chunking/)). - -## How to use the generator in `transformers` - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="google/electra-large-generator", - tokenizer="google/electra-large-generator" -) - -print( - fill_mask(f"HuggingFace is creating a {nlp.tokenizer.mask_token} that the community uses to solve NLP tasks.") -) - -``` diff --git a/model_cards/google/electra-small-discriminator/README.md b/model_cards/google/electra-small-discriminator/README.md deleted file mode 100644 index 650d3fd687..0000000000 --- a/model_cards/google/electra-small-discriminator/README.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer to our paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -This repository contains code to pre-train ELECTRA, including small ELECTRA models on a single GPU. It also supports fine-tuning ELECTRA on downstream tasks including classification tasks (e.g,. [GLUE](https://gluebenchmark.com/)), QA tasks (e.g., [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/)), and sequence tagging tasks (e.g., [text chunking](https://www.clips.uantwerpen.be/conll2000/chunking/)). - -## How to use the discriminator in `transformers` - -```python -from transformers import ElectraForPreTraining, ElectraTokenizerFast -import torch - -discriminator = ElectraForPreTraining.from_pretrained("google/electra-small-discriminator") -tokenizer = ElectraTokenizerFast.from_pretrained("google/electra-small-discriminator") - -sentence = "The quick brown fox jumps over the lazy dog" -fake_sentence = "The quick brown fox fake over the lazy dog" - -fake_tokens = tokenizer.tokenize(fake_sentence) -fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -[print("%7s" % token, end="") for token in fake_tokens] - -[print("%7s" % int(prediction), end="") for prediction in predictions.tolist()] -``` diff --git a/model_cards/google/electra-small-generator/README.md b/model_cards/google/electra-small-generator/README.md deleted file mode 100644 index b097328d0c..0000000000 --- a/model_cards/google/electra-small-generator/README.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer to our paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -This repository contains code to pre-train ELECTRA, including small ELECTRA models on a single GPU. It also supports fine-tuning ELECTRA on downstream tasks including classification tasks (e.g,. [GLUE](https://gluebenchmark.com/)), QA tasks (e.g., [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/)), and sequence tagging tasks (e.g., [text chunking](https://www.clips.uantwerpen.be/conll2000/chunking/)). - -## How to use the generator in `transformers` - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="google/electra-small-generator", - tokenizer="google/electra-small-generator" -) - -print( - fill_mask(f"HuggingFace is creating a {nlp.tokenizer.mask_token} that the community uses to solve NLP tasks.") -) - -``` diff --git a/model_cards/google/mobilebert-uncased/README.md b/model_cards/google/mobilebert-uncased/README.md deleted file mode 100644 index 16a4c1fd57..0000000000 --- a/model_cards/google/mobilebert-uncased/README.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -language: en -thumbnail: https://huggingface.co/front/thumbnails/google.png - -license: apache-2.0 ---- - -## MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices - -MobileBERT is a thin version of BERT_LARGE, while equipped with bottleneck structures and a carefully designed balance -between self-attentions and feed-forward networks. - -This checkpoint is the original MobileBert Optimized Uncased English: -[uncased_L-24_H-128_B-512_A-4_F-4_OPT](https://storage.googleapis.com/cloud-tpu-checkpoints/mobilebert/uncased_L-24_H-128_B-512_A-4_F-4_OPT.tar.gz) -checkpoint. - -## How to use MobileBERT in `transformers` - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="google/mobilebert-uncased", - tokenizer="google/mobilebert-uncased" -) - -print( - fill_mask(f"HuggingFace is creating a {fill_mask.tokenizer.mask_token} that the community uses to solve NLP tasks.") -) - -``` diff --git a/model_cards/google/reformer-crime-and-punishment/README.md b/model_cards/google/reformer-crime-and-punishment/README.md deleted file mode 100644 index bd37073c3c..0000000000 --- a/model_cards/google/reformer-crime-and-punishment/README.md +++ /dev/null @@ -1,20 +0,0 @@ -## Reformer Model trained on "Crime and Punishment" - -Crime and Punishment is a novel written by Fyodor Dostoevsky and was translated into English. - -Crime and Punishment training data was taken from `gs://trax-ml/reformer/crime-and-punishment-2554.txt` and contains -roughly 0.5M tokens. - -The ReformerLM model was trained in flax using colab notebook proposed by authors: https://colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb and the weights were converted to Hugging Face's PyTorch ReformerLM model `ReformerModelWithLMHead`. - -The model is a language model that operates on small sub-word units. Text can be generated as follows: - -```python -model = ReformerModelWithLMHead.from_pretrained("google/reformer-crime-and-punishment") -tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment") -tok.decode(model.generate(tok.encode("A few months later", return_tensors="pt"), do_sample=True,temperature=0.7, max_length=100)[0]) - -# gives:'A few months later on was more than anything in the flat. -# “I have already.” “That’s not my notion that he had forgotten him. -# What does that matter? And why do you mean? It’s only another fellow,” he said as he went out, as though he want' -``` diff --git a/model_cards/google/reformer-enwik8/README.md b/model_cards/google/reformer-enwik8/README.md deleted file mode 100644 index 998ac33d9c..0000000000 --- a/model_cards/google/reformer-enwik8/README.md +++ /dev/null @@ -1,57 +0,0 @@ -## Reformer Language model on character level and trained on enwik8. - -*enwik8* is a dataset based on Wikipedia and is often used to measure the model's ability to *compress* data, *e.g.* in -the scope of the *Hutter prize*: https://en.wikipedia.org/wiki/Hutter_Prize. - -`reformer-enwik8` was pretrained on the first 90M chars of *enwik8* whereas the text was chunked into batches of size 65536 chars (=2^16). -The model's weights were taken from https://console.cloud.google.com/storage/browser/trax-ml/reformer/enwik8 and converted -to Hugging Face's PyTorch ReformerLM model `ReformerModelWithLMHead`. - -The model is a language model that operates on characters. -Therefore, this model does not need a tokenizer. The following function can instead be used for **encoding** and **decoding**: - -```python -import torch - -# Encoding -def encode(list_of_strings, pad_token_id=0): - max_length = max([len(string) for string in list_of_strings]) - - # create emtpy tensors - attention_masks = torch.zeros((len(list_of_strings), max_length), dtype=torch.long) - input_ids = torch.full((len(list_of_strings), max_length), pad_token_id, dtype=torch.long) - - for idx, string in enumerate(list_of_strings): - # make sure string is in byte format - if not isinstance(string, bytes): - string = str.encode(string) - - input_ids[idx, :len(string)] = torch.tensor([x + 2 for x in string]) - attention_masks[idx, :len(string)] = 1 - - return input_ids, attention_masks - -# Decoding -def decode(outputs_ids): - decoded_outputs = [] - for output_ids in outputs_ids.tolist(): - # transform id back to char IDs < 2 are simply transformed to "" - decoded_outputs.append("".join([chr(x - 2) if x > 1 else "" for x in output_ids])) - return decoded_outputs -``` - -Text can be generated as follows: - -```python -from transformers import ReformerModelWithLMHead - -model = ReformerModelWithLMHead.from_pretrained("google/reformer-enwik8") -encoded, attention_masks = encode(["In 1965, Brooks left IBM to found the Department of"]) -decode(model.generate(encoded, do_sample=True, max_length=150)) - -# gives: -# In 1965, Brooks left IBM to found the Department of Journalism in 1968. IBM had jurisdiction himself in 1980, while Brooks resolved, nevertheless thro - -``` - -***Note***: Language generation using `ReformerModelWithLMHead` is not optimized yet and is rather slow. diff --git a/model_cards/google/roberta2roberta_L-24_bbc/README.md b/model_cards/google/roberta2roberta_L-24_bbc/README.md deleted file mode 100644 index 9e0c959f7a..0000000000 --- a/model_cards/google/roberta2roberta_L-24_bbc/README.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- xsum -tags: -- summarization ---- - -# Roberta2Roberta_L-24_bbc EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/roberta24_bbc/1). - -The model is an encoder-decoder model that was initialized on the `roberta-large` checkpoints for both the encoder -and decoder and fine-tuned on extreme summarization on the BBC XSum dataset, which is linked above. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for extreme summarization, *e.g.* - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_bbc") -model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_bbc") - -article = """The problem is affecting people using the older -versions of the PlayStation 3, called the "Fat" -model.The problem isn't affecting the newer PS3 -Slim systems that have been on sale since -September last year.Sony have also said they are -aiming to have the problem fixed shortly but is -advising some users to avoid using their console -for the time being."We hope to resolve this -problem within the next 24 hours," a statement -reads. "In the meantime, if you have a model other -than the new slim PS3, we advise that you do not -use your PS3 system, as doing so may result in -errors in some functionality, such as recording -obtained trophies, and not being able to restore -certain data."We believe we have identified that -this problem is being caused by a bug in the clock -functionality incorporated in the system."The -PlayStation Network is used by millions of people -around the world.It allows users to play their -friends at games like Fifa over the internet and -also do things like download software or visit -online stores.""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# Some Sony PlayStation gamers are being advised to stay away from the network because of a problem with the PlayStation 3 network. -``` diff --git a/model_cards/google/roberta2roberta_L-24_cnn_daily_mail/README.md b/model_cards/google/roberta2roberta_L-24_cnn_daily_mail/README.md deleted file mode 100644 index 58ccd1f1d5..0000000000 --- a/model_cards/google/roberta2roberta_L-24_cnn_daily_mail/README.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- cnn_dailymail -tags: -- summarization ---- - -# Roberta2Roberta_L-24_cnn_daily_mail EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/roberta24_cnndm/1). - -The model is an encoder-decoder model that was initialized on the `roberta-large` checkpoints for both the encoder -and decoder and fine-tuned on summarization on the CNN / Dailymail dataset, which is linked above. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for summarization, *e.g.* - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_cnn_daily_mail") -model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_cnn_daily_mail") - -article = """ (The Hollywood Reporter)"The Rocky Horror Picture -Show" is the latest musical getting the small- -screen treatment. Fox is developing a two-hour -remake of the 1975 cult classic to be directed, -executive-produced and choreographed by Kenneth -Ortega ("High School Musical"). The project, -tentatively titled "The Rocky Horror Picture Show -Event," is casting-contingent. The special will be -filmed in advance and not air live, but few -details beyond that are known. In addition to -Ortega, Gail Berman and Lou Adler, who produced -the original film, are also attached as executive -producers. The special will be produced by Fox 21 -Television Studios, and Berman's The Jackal Group. -The special is timed to celebrate the 40th -anniversary of the film, which has grossed more -than $112 million and still plays in theaters -across the country. TV premiere dates: The -complete guide . This isn't the first stab at -adapting "The Rocky Horror Picture Show." In 2002, -Fox unveiled plans for an adaptation timed to the -30th anniversary that never came to fruition. The -faces of pilot season 2015 . Fox's "Glee" covered -several of the show's most popular songs for a -Season 2 episode and even released a special "The -Rocky Horror Glee Show" EP. There is no plan yet -for when the adaptation will air. Fox also has a -live musical production of "Grease", starring -Julianne Hough and Vanessa Hudgens, scheduled to -air on Jan. 31, 2016. Broadcast TV scorecard . -Following in the footsteps of "The Sound of Music" -and "Peter Pan," NBC recently announced plans to -air a live version of The Wiz later this year. -Ortega's credits include "Gilmore Girls," "This Is -It" and "Hocus Pocus." He is repped by Paradigm -and Hanson, Jacobson. ©2015 The Hollywood -Reporter. All rights reserved.""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# Fox is developing a two-hour remake of the 1975 cult classic. The special will be directed, executive-produced and choreographed by Kenneth Ortega. -# The special is timed to celebrate the 40th anniversary of the film, which has grossed more than $112 million. - -``` diff --git a/model_cards/google/roberta2roberta_L-24_discofuse/README.md b/model_cards/google/roberta2roberta_L-24_discofuse/README.md deleted file mode 100644 index f721972530..0000000000 --- a/model_cards/google/roberta2roberta_L-24_discofuse/README.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- discofuse ---- - -# Roberta2Roberta_L-24_discofuse EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/roberta24_discofuse/1). - -The model is an encoder-decoder model that was initialized on the `roberta-large` checkpoints for both the encoder -and decoder and fine-tuned on sentencefusion on the discofuse dataset, which is linked above. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for sentence fusion, *e.g.* - -IMPORTANT: The model was not trained on the `"` (double quotation mark) character -> so the before tokenizing the text, it is advised to replace all `"` (double quotation marks) with a single `` ` `` (single back tick). - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_discofuse") -model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_discofuse") - -discofuse = """As a run-blocker, Zeitler moves relatively well. Zeitler often struggles at the point of contact in space.""" - -input_ids = tokenizer(discofuse, return_tensors="pt").input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# As a run-blocker, Zeitler moves relatively well. However, Zeitler often struggles at the point of contact in space. -``` diff --git a/model_cards/google/roberta2roberta_L-24_gigaword/README.md b/model_cards/google/roberta2roberta_L-24_gigaword/README.md deleted file mode 100644 index a465a34604..0000000000 --- a/model_cards/google/roberta2roberta_L-24_gigaword/README.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- gigaword -tags: -- summarization ---- - -# Roberta2Roberta_L-24_gigaword EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/roberta24_gigaword/1). - -The model is an encoder-decoder model that was initialized on the `roberta-large` checkpoints for both the encoder -and decoder and fine-tuned on headline generation using the Gigaword dataset, which is linked above. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for extreme summarization, *e.g.* - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_gigaword") -model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_gigaword") - -article = """australian shares closed down #.# percent monday -following a weak lead from the united states and -lower commodity prices , dealers said .""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# australian shares close down #.# percent. -``` diff --git a/model_cards/google/roberta2roberta_L-24_wikisplit/README.md b/model_cards/google/roberta2roberta_L-24_wikisplit/README.md deleted file mode 100644 index 8d4a2b380a..0000000000 --- a/model_cards/google/roberta2roberta_L-24_wikisplit/README.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -language: en -license: apache-2.0 ---- - -# Roberta2Roberta_L-24_wikisplit EncoderDecoder model - -The model was introduced in -[this paper](https://arxiv.org/abs/1907.12461) by Sascha Rothe, Shashi Narayan, Aliaksei Severyn and first released in [this repository](https://tfhub.dev/google/bertseq2seq/roberta24_cnndm/1). - -The model is an encoder-decoder model that was initialized on the `roberta-large` checkpoints for both the encoder -and decoder and fine-tuned on sentence splitting on the [WikiSplit](https://github.com/google-research-datasets/wiki-split) dataset. - -Disclaimer: The model card has been written by the Hugging Face team. - -## How to use - -You can use this model for sentence splitting, *e.g.* - -**IMPORTANT**: The model was not trained on the `"` (double quotation mark) character -> so the before tokenizing the text, -it is advised to replace all `"` (double quotation marks) with two single `'` (single quotation mark). - -```python -from transformers import AutoTokenizer, AutoModelForSeq2SeqLM - -tokenizer = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_wikisplit") -model = AutoModelForSeq2SeqLM.from_pretrained("google/roberta2roberta_L-24_wikisplit") - -long_sentence = """Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it and he decides to open Bob 's Burgers for customers who were planning on going to Lobsterfest.""" - -input_ids = tokenizer(tokenizer.bos_token + long_sentence + tokenizer.eos_token, return_tensors="pt").input_ids -output_ids = model.generate(input_ids)[0] -print(tokenizer.decode(output_ids, skip_special_tokens=True)) -# should output -# Due to the hurricane, Lobsterfest has been canceled, making Bob very happy about it. He decides to open Bob's Burgers for customers who were planning on going to Lobsterfest. -``` diff --git a/model_cards/gpt2-README.md b/model_cards/gpt2-README.md deleted file mode 100644 index 65fe7b5e38..0000000000 --- a/model_cards/gpt2-README.md +++ /dev/null @@ -1,163 +0,0 @@ ---- -language: en -tags: -- exbert - -license: mit ---- - - -# GPT-2 - -Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large - -Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in -[this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) -and first released at [this page](https://openai.com/blog/better-language-models/). - -Disclaimer: The team releasing GPT-2 also wrote a -[model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card -has been written by the Hugging Face team to complete the information they provided and give specific examples of bias. - -## Model description - -GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This -means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots -of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, -it was trained to guess the next word in sentences. - -More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence, -shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the -predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a -prompt. - -## Intended uses & limitations - -You can use the raw model for text generation or fine-tune it to a downstream task. See the -[model hub](https://huggingface.co/models?filter=gpt2) to look for fine-tuned versions on a task that interests you. - -### How to use - -You can use this model directly with a pipeline for text generation. Since the generation relies on some randomness, we -set a seed for reproducibility: - -```python ->>> from transformers import pipeline, set_seed ->>> generator = pipeline('text-generation', model='gpt2') ->>> set_seed(42) ->>> generator("Hello, I'm a language model,", max_length=30, num_return_sequences=5) - -[{'generated_text': "Hello, I'm a language model, a language for thinking, a language for expressing thoughts."}, - {'generated_text': "Hello, I'm a language model, a compiler, a compiler library, I just want to know how I build this kind of stuff. I don"}, - {'generated_text': "Hello, I'm a language model, and also have more than a few of your own, but I understand that they're going to need some help"}, - {'generated_text': "Hello, I'm a language model, a system model. I want to know my language so that it might be more interesting, more user-friendly"}, - {'generated_text': 'Hello, I\'m a language model, not a language model"\n\nThe concept of "no-tricks" comes in handy later with new'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import GPT2Tokenizer, GPT2Model -tokenizer = GPT2Tokenizer.from_pretrained('gpt2') -model = GPT2Model.from_pretrained('gpt2') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import GPT2Tokenizer, TFGPT2Model -tokenizer = GPT2Tokenizer.from_pretrained('gpt2') -model = TFGPT2Model.from_pretrained('gpt2') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -The training data used for this model has not been released as a dataset one can browse. We know it contains a lot of -unfiltered content from the internet, which is far from neutral. As the openAI team themselves point out in their -[model card](https://github.com/openai/gpt-2/blob/master/model_card.md#out-of-scope-use-cases): - -> Because large-scale language models like GPT-2 do not distinguish fact from fiction, we don’t support use-cases -> that require the generated text to be true. -> -> Additionally, language models like GPT-2 reflect the biases inherent to the systems they were trained on, so we do -> not recommend that they be deployed into systems that interact with humans > unless the deployers first carry out a -> study of biases relevant to the intended use-case. We found no statistically significant difference in gender, race, -> and religious bias probes between 774M and 1.5B, implying all versions of GPT-2 should be approached with similar -> levels of caution around use cases that are sensitive to biases around human attributes. - -Here's an example of how the model can have biased predictions: - -```python ->>> from transformers import pipeline, set_seed ->>> generator = pipeline('text-generation', model='gpt2') ->>> set_seed(42) ->>> generator("The White man worked as a", max_length=10, num_return_sequences=5) - -[{'generated_text': 'The White man worked as a mannequin for'}, - {'generated_text': 'The White man worked as a maniser of the'}, - {'generated_text': 'The White man worked as a bus conductor by day'}, - {'generated_text': 'The White man worked as a plumber at the'}, - {'generated_text': 'The White man worked as a journalist. He had'}] - ->>> set_seed(42) ->>> generator("The Black man worked as a", max_length=10, num_return_sequences=5) - -[{'generated_text': 'The Black man worked as a man at a restaurant'}, - {'generated_text': 'The Black man worked as a car salesman in a'}, - {'generated_text': 'The Black man worked as a police sergeant at the'}, - {'generated_text': 'The Black man worked as a man-eating monster'}, - {'generated_text': 'The Black man worked as a slave, and was'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -The OpenAI team wanted to train this model on a corpus as large as possible. To build it, they scraped all the web -pages from outbound links on Reddit which received at least 3 karma. Note that all Wikipedia pages were removed from -this dataset, so the model was not trained on any part of Wikipedia. The resulting dataset (called WebText) weights -40GB of texts but has not been publicly released. You can find a list of the top 1,000 domains present in WebText -[here](https://github.com/openai/gpt-2/blob/master/domains.txt). - -## Training procedure - -### Preprocessing - -The texts are tokenized using a byte-level version of Byte Pair Encoding (BPE) (for unicode characters) and a -vocabulary size of 50,257. The inputs are sequences of 1024 consecutive tokens. - -The larger model was trained on 256 cloud TPU v3 cores. The training duration was not disclosed, nor were the exact -details of training. - -## Evaluation results - -The model achieves the following results without any fine-tuning (zero-shot): - -| Dataset | LAMBADA | LAMBADA | CBT-CN | CBT-NE | WikiText2 | PTB | enwiki8 | text8 | WikiText103 | 1BW | -|:--------:|:-------:|:-------:|:------:|:------:|:---------:|:------:|:-------:|:------:|:-----------:|:-----:| -| (metric) | (PPL) | (ACC) | (ACC) | (ACC) | (PPL) | (PPL) | (BPB) | (BPC) | (PPL) | (PPL) | -| | 35.13 | 45.99 | 87.65 | 83.4 | 29.41 | 65.85 | 1.16 | 1,17 | 37.50 | 75.20 | - - -### BibTeX entry and citation info - -```bibtex -@article{radford2019language, - title={Language Models are Unsupervised Multitask Learners}, - author={Radford, Alec and Wu, Jeff and Child, Rewon and Luan, David and Amodei, Dario and Sutskever, Ilya}, - year={2019} -} -``` - - - - diff --git a/model_cards/gpt2-large-README.md b/model_cards/gpt2-large-README.md deleted file mode 100644 index a019393abf..0000000000 --- a/model_cards/gpt2-large-README.md +++ /dev/null @@ -1 +0,0 @@ -Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large diff --git a/model_cards/gpt2-medium-README.md b/model_cards/gpt2-medium-README.md deleted file mode 100644 index a019393abf..0000000000 --- a/model_cards/gpt2-medium-README.md +++ /dev/null @@ -1 +0,0 @@ -Test the full generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large diff --git a/model_cards/gpt2-xl-README.md b/model_cards/gpt2-xl-README.md deleted file mode 100644 index bac6ec0137..0000000000 --- a/model_cards/gpt2-xl-README.md +++ /dev/null @@ -1 +0,0 @@ -Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large diff --git a/model_cards/gsarti/biobert-nli/README.md b/model_cards/gsarti/biobert-nli/README.md deleted file mode 100644 index 0936cb00a1..0000000000 --- a/model_cards/gsarti/biobert-nli/README.md +++ /dev/null @@ -1,37 +0,0 @@ -# BioBERT-NLI - -This is the model [BioBERT](https://github.com/dmis-lab/biobert) [1] fine-tuned on the [SNLI](https://nlp.stanford.edu/projects/snli/) and the [MultiNLI](https://www.nyu.edu/projects/bowman/multinli/) datasets using the [`sentence-transformers` library](https://github.com/UKPLab/sentence-transformers/) to produce universal sentence embeddings [2]. - -The model uses the original BERT wordpiece vocabulary and was trained using the **average pooling strategy** and a **softmax loss**. - -**Base model**: `monologg/biobert_v1.1_pubmed` from HuggingFace's `AutoModel`. - -**Training time**: ~6 hours on the NVIDIA Tesla P100 GPU provided in Kaggle Notebooks. - -**Parameters**: - -| Parameter | Value | -|------------------|-------| -| Batch size | 64 | -| Training steps | 30000 | -| Warmup steps | 1450 | -| Lowercasing | False | -| Max. Seq. Length | 128 | - -**Performances**: The performance was evaluated on the test portion of the [STS dataset](http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark) using Spearman rank correlation and compared to the performances of a general BERT base model obtained with the same procedure to verify their similarity. - -| Model | Score | -|-------------------------------|-------------| -| `biobert-nli` (this) | 73.40 | -| `gsarti/scibert-nli` | 74.50 | -| `bert-base-nli-mean-tokens`[3]| 77.12 | - -An example usage for similarity-based scientific paper retrieval is provided in the [Covid Papers Browser](https://github.com/gsarti/covid-papers-browser) repository. - -**References:** - -[1] J. Lee et al, [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://academic.oup.com/bioinformatics/article/36/4/1234/5566506) - -[2] A. Conneau et al., [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://www.aclweb.org/anthology/D17-1070/) - -[3] N. Reimers et I. Gurevych, [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://www.aclweb.org/anthology/D19-1410/) diff --git a/model_cards/gsarti/covidbert-nli/README.md b/model_cards/gsarti/covidbert-nli/README.md deleted file mode 100644 index 45037dcbc0..0000000000 --- a/model_cards/gsarti/covidbert-nli/README.md +++ /dev/null @@ -1,38 +0,0 @@ -# CovidBERT-NLI - -This is the model **CovidBERT** trained by DeepSet on AllenAI's [CORD19 Dataset](https://pages.semanticscholar.org/coronavirus-research) of scientific articles about coronaviruses. - -The model uses the original BERT wordpiece vocabulary and was subsequently fine-tuned on the [SNLI](https://nlp.stanford.edu/projects/snli/) and the [MultiNLI](https://www.nyu.edu/projects/bowman/multinli/) datasets using the [`sentence-transformers` library](https://github.com/UKPLab/sentence-transformers/) to produce universal sentence embeddings [1] using the **average pooling strategy** and a **softmax loss**. - -Parameter details for the original training on CORD-19 are available on [DeepSet's MLFlow](https://public-mlflow.deepset.ai/#/experiments/2/runs/ba27d00c30044ef6a33b1d307b4a6cba) - -**Base model**: `deepset/covid_bert_base` from HuggingFace's `AutoModel`. - -**Training time**: ~6 hours on the NVIDIA Tesla P100 GPU provided in Kaggle Notebooks. - -**Parameters**: - -| Parameter | Value | -|------------------|-------| -| Batch size | 64 | -| Training steps | 23000 | -| Warmup steps | 1450 | -| Lowercasing | True | -| Max. Seq. Length | 128 | - -**Performances**: The performance was evaluated on the test portion of the [STS dataset](http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark) using Spearman rank correlation and compared to the performances of similar models obtained with the same procedure to verify its performances. - -| Model | Score | -|-------------------------------|-------------| -| `covidbert-nli` (this) | 67.52 | -| `gsarti/biobert-nli` | 73.40 | -| `gsarti/scibert-nli` | 74.50 | -| `bert-base-nli-mean-tokens`[2]| 77.12 | - -An example usage for similarity-based scientific paper retrieval is provided in the [Covid-19 Semantic Browser](https://github.com/gsarti/covid-papers-browser) repository. - -**References:** - -[1] A. Conneau et al., [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://www.aclweb.org/anthology/D17-1070/) - -[2] N. Reimers et I. Gurevych, [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://www.aclweb.org/anthology/D19-1410/) diff --git a/model_cards/gsarti/scibert-nli/README.md b/model_cards/gsarti/scibert-nli/README.md deleted file mode 100644 index 661d04452c..0000000000 --- a/model_cards/gsarti/scibert-nli/README.md +++ /dev/null @@ -1,36 +0,0 @@ -# SciBERT-NLI - -This is the model [SciBERT](https://github.com/allenai/scibert) [1] fine-tuned on the [SNLI](https://nlp.stanford.edu/projects/snli/) and the [MultiNLI](https://www.nyu.edu/projects/bowman/multinli/) datasets using the [`sentence-transformers` library](https://github.com/UKPLab/sentence-transformers/) to produce universal sentence embeddings [2]. - -The model uses the original `scivocab` wordpiece vocabulary and was trained using the **average pooling strategy** and a **softmax loss**. - -**Base model**: `allenai/scibert-scivocab-cased` from HuggingFace's `AutoModel`. - -**Training time**: ~4 hours on the NVIDIA Tesla P100 GPU provided in Kaggle Notebooks. - -**Parameters**: - -| Parameter | Value | -|------------------|-------| -| Batch size | 64 | -| Training steps | 20000 | -| Warmup steps | 1450 | -| Lowercasing | True | -| Max. Seq. Length | 128 | - -**Performances**: The performance was evaluated on the test portion of the [STS dataset](http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark) using Spearman rank correlation and compared to the performances of a general BERT base model obtained with the same procedure to verify their similarity. - -| Model | Score | -|-------------------------------|-------------| -| `scibert-nli` (this) | 74.50 | -| `bert-base-nli-mean-tokens`[3]| 77.12 | - -An example usage for similarity-based scientific paper retrieval is provided in the [Covid Papers Browser](https://github.com/gsarti/covid-papers-browser) repository. - -**References:** - -[1] I. Beltagy et al, [SciBERT: A Pretrained Language Model for Scientific Text](https://www.aclweb.org/anthology/D19-1371/) - -[2] A. Conneau et al., [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://www.aclweb.org/anthology/D17-1070/) - -[3] N. Reimers et I. Gurevych, [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://www.aclweb.org/anthology/D19-1410/) diff --git a/model_cards/gurkan08/bert-turkish-text-classification/README.md b/model_cards/gurkan08/bert-turkish-text-classification/README.md deleted file mode 100644 index e70af8b86d..0000000000 --- a/model_cards/gurkan08/bert-turkish-text-classification/README.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -language: tr ---- -# Turkish News Text Classification - - Turkish text classification model obtained by fine-tuning the Turkish bert model (dbmdz/bert-base-turkish-cased) - -# Dataset - -Dataset consists of 11 classes were obtained from https://www.trthaber.com/. The model was created using the most distinctive 6 classes. - -Dataset can be accessed at https://github.com/gurkan08/datasets/tree/master/trt_11_category. - - label_dict = { - 'LABEL_0': 'ekonomi', - 'LABEL_1': 'spor', - 'LABEL_2': 'saglik', - 'LABEL_3': 'kultur_sanat', - 'LABEL_4': 'bilim_teknoloji', - 'LABEL_5': 'egitim' - } - -70% of the data were used for training and 30% for testing. - -train f1-weighted score = %97 - -test f1-weighted score = %94 - -# Usage - - from transformers import pipeline, AutoTokenizer, AutoModelForSequenceClassification - - tokenizer = AutoTokenizer.from_pretrained("gurkan08/bert-turkish-text-classification") - model = AutoModelForSequenceClassification.from_pretrained("gurkan08/bert-turkish-text-classification") - - nlp = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer) - - text = ["Süper Lig'in 6. haftasında Sivasspor ile Çaykur Rizespor karşı karşıya geldi...", - "Son 24 saatte 69 kişi Kovid-19 nedeniyle yaşamını yitirdi, 1573 kişi iyileşti"] - - out = nlp(text) - - label_dict = { - 'LABEL_0': 'ekonomi', - 'LABEL_1': 'spor', - 'LABEL_2': 'saglik', - 'LABEL_3': 'kultur_sanat', - 'LABEL_4': 'bilim_teknoloji', - 'LABEL_5': 'egitim' - } - - results = [] - for result in out: - result['label'] = label_dict[result['label']] - results.append(result) - print(results) - - # > [{'label': 'spor', 'score': 0.9992026090621948}, {'label': 'saglik', 'score': 0.9972177147865295}] - - - diff --git a/model_cards/hatmimoha/arabic-ner/README.md b/model_cards/hatmimoha/arabic-ner/README.md deleted file mode 100644 index 2ea68808b9..0000000000 --- a/model_cards/hatmimoha/arabic-ner/README.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -language: ar ---- -# Arabic Named Entity Recognition Model - -Pretrained BERT-based ([arabic-bert-base](https://huggingface.co/asafaya/bert-base-arabic)) Named Entity Recognition model for Arabic. - -The pre-trained model can recognize the following entities: -1. **PERSON** - -- و هذا ما نفاه المعاون السياسي للرئيس ***نبيه بري*** ، النائب ***علي حسن خليل*** - -- لكن أوساط ***الحريري*** تعتبر أنه ضحى كثيرا في سبيل البلد - -- و ستفقد الملكة ***إليزابيث الثانية*** بذلك سيادتها على واحدة من آخر ممالك الكومنولث - -2. **ORGANIZATION** - -- حسب أرقام ***البنك الدولي*** - -- أعلن ***الجيش العراقي*** - -- و نقلت وكالة ***رويترز*** عن ثلاثة دبلوماسيين في ***الاتحاد الأوروبي*** ، أن ***بلجيكا*** و ***إيرلندا*** و ***لوكسمبورغ*** تريد أيضاً مناقشة - -- ***الحكومة الاتحادية*** و ***حكومة إقليم كردستان*** - -- و هو ما يثير الشكوك حول مشاركة النجم البرتغالي في المباراة المرتقبة أمام ***برشلونة*** الإسباني في - - -3. ***LOCATION*** - -- الجديد هو تمكين اللاجئين من “ مغادرة الجزيرة تدريجياً و بهدوء إلى ***أثينا*** ” - -- ***جزيرة ساكيز*** تبعد 1 كم عن ***إزمير*** - - -4. **DATE** - -- ***غدا الجمعة*** - -- ***06 أكتوبر 2020*** - -- ***العام السابق*** - - -5. **PRODUCT** - -- عبر حسابه ب ***تطبيق “ إنستغرام ”*** - -- الجيل الثاني من ***نظارة الواقع الافتراضي أوكولوس كويست*** تحت اسم " ***أوكولوس كويست 2*** " - - -6. **COMPETITION** - -- عدم المشاركة في ***بطولة فرنسا المفتوحة للتنس*** - -- في مباراة ***كأس السوبر الأوروبي*** - -7. **PRIZE** - -- ***جائزة نوبل ل لآداب*** - -- الذي فاز ب ***جائزة “ إيمي ” لأفضل دور مساند*** - -8. **EVENT** - -- تسجّل أغنية جديدة خاصة ب ***العيد الوطني السعودي*** - -- ***مهرجان المرأة يافوية*** في دورته الرابعة - -9. **DISEASE** - -- في مكافحة فيروس ***كورونا*** و عدد من الأمراض - -- الأزمات المشابهة مثل “ ***انفلونزا الطيور*** ” و ” ***انفلونزا الخنازير*** - -## Example - -[Find here a complete example to use this model](https://github.com/hatmimoha/arabic-ner) - -Here is the map from index to label: - -``` -id2label = { - "0": "B-PERSON", - "1": "I-PERSON", - "2": "B-ORGANIZATION", - "3": "I-ORGANIZATION", - "4": "B-LOCATION", - "5": "I-LOCATION", - "6": "B-DATE", - "7": "I-DATE"", - "8": "B-COMPETITION", - "9": "I-COMPETITION", - "10": "B-PRIZE", - "11": "I-PRIZE", - "12": "O", - "13": "B-PRODUCT", - "14": "I-PRODUCT", - "15": "B-EVENT", - "16": "I-EVENT", - "17": "B-DISEASE", - "18": "I-DISEASE", -} - -``` - -## Training Corpus - -The training corpus is made of 378.000 tokens (14.000 sentences) collected from the Web and annotated manually. - -## Results - -The results on a valid corpus made of 30.000 tokens shows an F-measure of ~87%. diff --git a/model_cards/healx/gpt-2-pubmed-large/README.md b/model_cards/healx/gpt-2-pubmed-large/README.md deleted file mode 100644 index 5dd99bd8d3..0000000000 --- a/model_cards/healx/gpt-2-pubmed-large/README.md +++ /dev/null @@ -1,3 +0,0 @@ -GPT-2 (774M model) finetuned on 0.5m PubMed abstracts. Used in the [writemeanabstract.com](writemeanabstract.com) and the following preprint: - -[Papanikolaou, Yannis, and Andrea Pierleoni. "DARE: Data Augmented Relation Extraction with GPT-2." arXiv preprint arXiv:2004.13845 (2020).](https://arxiv.org/abs/2004.13845) diff --git a/model_cards/healx/gpt-2-pubmed-medium/README.md b/model_cards/healx/gpt-2-pubmed-medium/README.md deleted file mode 100644 index c161836ef2..0000000000 --- a/model_cards/healx/gpt-2-pubmed-medium/README.md +++ /dev/null @@ -1,3 +0,0 @@ -GPT-2 (355M model) finetuned on 0.5m PubMed abstracts. Used in the [writemeanabstract.com](writemeanabstract.com) and the following preprint: - -[Papanikolaou, Yannis, and Andrea Pierleoni. "DARE: Data Augmented Relation Extraction with GPT-2." arXiv preprint arXiv:2004.13845 (2020).](https://arxiv.org/abs/2004.13845) diff --git a/model_cards/henryk/bert-base-multilingual-cased-finetuned-dutch-squad2/README.md b/model_cards/henryk/bert-base-multilingual-cased-finetuned-dutch-squad2/README.md deleted file mode 100644 index 621dbda8d3..0000000000 --- a/model_cards/henryk/bert-base-multilingual-cased-finetuned-dutch-squad2/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: nl ---- - -# Multilingual + Dutch SQuAD2.0 - -This model is the multilingual model provided by the Google research team with a fine-tuned dutch Q&A downstream task. - -## Details of the language model - -Language model ([**bert-base-multilingual-cased**](https://github.com/google-research/bert/blob/master/multilingual.md)): -12-layer, 768-hidden, 12-heads, 110M parameters. -Trained on cased text in the top 104 languages with the largest Wikipedias. - -## Details of the downstream task -Using the `mtranslate` Python module, [**SQuAD2.0**](https://rajpurkar.github.io/SQuAD-explorer/) was machine-translated. In order to find the start tokens, the direct translations of the answers were searched in the corresponding paragraphs. Due to the different translations depending on the context (missing context in the pure answer), the answer could not always be found in the text, and thus a loss of question-answer examples occurred. This is a potential problem where errors can occur in the data set. - -| Dataset | # Q&A | -| ---------------------- | ----- | -| SQuAD2.0 Train | 130 K | -| Dutch SQuAD2.0 Train | 99 K | -| SQuAD2.0 Dev | 12 K | -| Dutch SQuAD2.0 Dev | 10 K | - - -## Model benchmark - - -| Model | EM/F1 |HasAns (EM/F1) | NoAns | -| ---------------------- | ----- | ----- | ----- | -| [robBERT](https://huggingface.co/pdelobelle/robBERT-base) | 58.04/60.95 | 33.08/40.64 | 73.67 | -| [dutchBERT](https://huggingface.co/wietsedv/bert-base-dutch-cased) | 64.25/68.45 | 45.59/56.49 | 75.94 | -| [multiBERT](https://huggingface.co/bert-base-multilingual-cased) | **67.38**/**71.36** | 47.42/57.76 | 79.88 | - -## Model training - -The model was trained on a **Tesla V100** GPU with the following command: - -```python -export SQUAD_DIR=path/to/nl_squad - -python run_squad.py - --model_type bert \ - --model_name_or_path bert-base-multilingual-cased \ - --do_train \ - --do_eval \ - --train_file $SQUAD_DIR/nl_squadv2_train_clean.json \ - --predict_file $SQUAD_DIR/nl_squadv2_dev_clean.json \ - --num_train_epochs 2 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --save_steps=8000 \ - --output_dir ../../output \ - --overwrite_cache \ - --overwrite_output_dir -``` - -**Results**: - -{'exact': 67.38028751680629, 'f1': 71.362297054268, 'total': 9669, 'HasAns_exact': 47.422126745435015, 'HasAns_f1': 57.761023151910734, 'HasAns_total': 3724, 'NoAns_exact': 79.88225399495374, 'NoAns_f1': 79.88225399495374, 'NoAns_total': 5945, 'best_exact': 67.53542248422795, 'best_exact_thresh': 0.0, 'best_f1': 71.36229705426837, 'best_f1_thresh': 0.0} - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="henryk/bert-base-multilingual-cased-finetuned-dutch-squad2", - tokenizer="henryk/bert-base-multilingual-cased-finetuned-dutch-squad2" -) - -qa_pipeline({ - 'context': "Amsterdam is de hoofdstad en de dichtstbevolkte stad van Nederland.", - 'question': "Wat is de hoofdstad van Nederland?"}) - -``` - -# Output: - -```json -{ - "score": 0.83, - "start": 0, - "end": 9, - "answer": "Amsterdam" -} -``` - -## Contact - -Please do not hesitate to contact me via [LinkedIn](https://www.linkedin.com/in/henryk-borzymowski-0755a2167/) if you want to discuss or get access to the Dutch version of SQuAD. \ No newline at end of file diff --git a/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad1/README.md b/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad1/README.md deleted file mode 100644 index d1ec8ab307..0000000000 --- a/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad1/README.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -language: pl ---- - -# Multilingual + Polish SQuAD1.1 - -This model is the multilingual model provided by the Google research team with a fine-tuned polish Q&A downstream task. - -## Details of the language model - -Language model ([**bert-base-multilingual-cased**](https://github.com/google-research/bert/blob/master/multilingual.md)): -12-layer, 768-hidden, 12-heads, 110M parameters. -Trained on cased text in the top 104 languages with the largest Wikipedias. - -## Details of the downstream task -Using the `mtranslate` Python module, [**SQuAD1.1**](https://rajpurkar.github.io/SQuAD-explorer/) was machine-translated. In order to find the start tokens, the direct translations of the answers were searched in the corresponding paragraphs. Due to the different translations depending on the context (missing context in the pure answer), the answer could not always be found in the text, and thus a loss of question-answer examples occurred. This is a potential problem where errors can occur in the data set. - -| Dataset | # Q&A | -| ---------------------- | ----- | -| SQuAD1.1 Train | 87.7 K | -| Polish SQuAD1.1 Train | 39.5 K | -| SQuAD1.1 Dev | 10.6 K | -| Polish SQuAD1.1 Dev | 2.6 K | - - -## Model benchmark - -| Model | EM | F1 | -| ---------------------- | ----- | ----- | -| [SlavicBERT](https://huggingface.co/DeepPavlov/bert-base-bg-cs-pl-ru-cased) | **60.89** | 71.68 | -| [polBERT](https://huggingface.co/dkleczek/bert-base-polish-uncased-v1) | 57.46 | 68.87 | -| [multiBERT](https://huggingface.co/bert-base-multilingual-cased) | 60.67 | **71.89** | -| [xlm](https://huggingface.co/xlm-mlm-100-1280) | 47.98 | 59.42 | -## Model training - -The model was trained on a **Tesla V100** GPU with the following command: - -```python -export SQUAD_DIR=path/to/pl_squad - -python run_squad.py - --model_type bert \ - --model_name_or_path bert-base-multilingual-cased \ - --do_train \ - --do_eval \ - --train_file $SQUAD_DIR/pl_squadv1_train_clean.json \ - --predict_file $SQUAD_DIR/pl_squadv1_dev_clean.json \ - --num_train_epochs 2 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --save_steps=8000 \ - --output_dir ../../output \ - --overwrite_cache \ - --overwrite_output_dir -``` - -**Results**: - -{'exact': 60.670731707317074, 'f1': 71.8952193697293, 'total': 2624, 'HasAns_exact': 60.670731707317074, 'HasAns_f1': 71.8952193697293, -'HasAns_total': 2624, 'best_exact': 60.670731707317074, 'best_exact_thresh': 0.0, 'best_f1': 71.8952193697293, 'best_f1_thresh': 0.0} - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="henryk/bert-base-multilingual-cased-finetuned-polish-squad1", - tokenizer="henryk/bert-base-multilingual-cased-finetuned-polish-squad1" -) - -qa_pipeline({ - 'context': "Warszawa jest największym miastem w Polsce pod względem liczby ludności i powierzchni", - 'question': "Jakie jest największe miasto w Polsce?"}) - -``` - -# Output: - -```json -{ - "score": 0.9988, - "start": 0, - "end": 8, - "answer": "Warszawa" -} -``` - -## Contact - -Please do not hesitate to contact me via [LinkedIn](https://www.linkedin.com/in/henryk-borzymowski-0755a2167/) if you want to discuss or get access to the Polish version of SQuAD. \ No newline at end of file diff --git a/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad2/README.md b/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad2/README.md deleted file mode 100644 index 6ef1ab48e0..0000000000 --- a/model_cards/henryk/bert-base-multilingual-cased-finetuned-polish-squad2/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: pl ---- - -# Multilingual + Polish SQuAD2.0 - -This model is the multilingual model provided by the Google research team with a fine-tuned polish Q&A downstream task. - -## Details of the language model - -Language model ([**bert-base-multilingual-cased**](https://github.com/google-research/bert/blob/master/multilingual.md)): -12-layer, 768-hidden, 12-heads, 110M parameters. -Trained on cased text in the top 104 languages with the largest Wikipedias. - -## Details of the downstream task -Using the `mtranslate` Python module, [**SQuAD2.0**](https://rajpurkar.github.io/SQuAD-explorer/) was machine-translated. In order to find the start tokens, the direct translations of the answers were searched in the corresponding paragraphs. Due to the different translations depending on the context (missing context in the pure answer), the answer could not always be found in the text, and thus a loss of question-answer examples occurred. This is a potential problem where errors can occur in the data set. - -| Dataset | # Q&A | -| ---------------------- | ----- | -| SQuAD2.0 Train | 130 K | -| Polish SQuAD2.0 Train | 83.1 K | -| SQuAD2.0 Dev | 12 K | -| Polish SQuAD2.0 Dev | 8.5 K | - - -## Model benchmark - -| Model | EM/F1 |HasAns (EM/F1) | NoAns | -| ---------------------- | ----- | ----- | ----- | -| [SlavicBERT](https://huggingface.co/DeepPavlov/bert-base-bg-cs-pl-ru-cased) | 69.35/71.51 | 47.02/54.09 | 79.20 | -| [polBERT](https://huggingface.co/dkleczek/bert-base-polish-uncased-v1) | 67.33/69.80| 45.73/53.80 | 76.87 | -| [multiBERT](https://huggingface.co/bert-base-multilingual-cased) | **70.76**/**72.92** |45.00/52.04 | 82.13 | - -## Model training - -The model was trained on a **Tesla V100** GPU with the following command: - -```python -export SQUAD_DIR=path/to/pl_squad - -python run_squad.py - --model_type bert \ - --model_name_or_path bert-base-multilingual-cased \ - --do_train \ - --do_eval \ - --version_2_with_negative \ - --train_file $SQUAD_DIR/pl_squadv2_train.json \ - --predict_file $SQUAD_DIR/pl_squadv2_dev.json \ - --num_train_epochs 2 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --save_steps=8000 \ - --output_dir ../../output \ - --overwrite_cache \ - --overwrite_output_dir -``` - -**Results**: - -{'exact': 70.76671723655035, 'f1': 72.92156947155917, 'total': 8569, 'HasAns_exact': 45.00762195121951, 'HasAns_f1': 52.04456128116991, 'HasAns_total': 2624, 'NoAns_exact': 82.13624894869638, ' -NoAns_f1': 82.13624894869638, 'NoAns_total': 5945, 'best_exact': 71.72365503559342, 'best_exact_thresh': 0.0, 'best_f1': 73.62662512059369, 'best_f1_thresh': 0.0} - - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="henryk/bert-base-multilingual-cased-finetuned-polish-squad2", - tokenizer="henryk/bert-base-multilingual-cased-finetuned-polish-squad2" -) - -qa_pipeline({ - 'context': "Warszawa jest największym miastem w Polsce pod względem liczby ludności i powierzchni", - 'question': "Jakie jest największe miasto w Polsce?"}) - -``` - -# Output: - -```json -{ - "score": 0.9986, - "start": 0, - "end": 8, - "answer": "Warszawa" -} -``` - -## Contact - -Please do not hesitate to contact me via [LinkedIn](https://www.linkedin.com/in/henryk-borzymowski-0755a2167/) if you want to discuss or get access to the Polish version of SQuAD. \ No newline at end of file diff --git a/model_cards/huawei-noah/DynaBERT_MNLI/README.md b/model_cards/huawei-noah/DynaBERT_MNLI/README.md deleted file mode 100644 index a9a4bc233d..0000000000 --- a/model_cards/huawei-noah/DynaBERT_MNLI/README.md +++ /dev/null @@ -1,20 +0,0 @@ -## DynaBERT: Dynamic BERT with Adaptive Width and Depth - -* DynaBERT can flexibly adjust the size and latency by selecting adaptive width and depth, and -the subnetworks of it have competitive performances as other similar-sized compressed models. -The training process of DynaBERT includes first training a width-adaptive BERT and then -allowing both adaptive width and depth using knowledge distillation. - -* This code is modified based on the repository developed by Hugging Face: [Transformers v2.1.1](https://github.com/huggingface/transformers/tree/v2.1.1), and is released in [GitHub](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/DynaBERT). - -### Reference -Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu. -[DynaBERT: Dynamic BERT with Adaptive Width and Depth](https://arxiv.org/abs/2004.04037). -``` -@inproceedings{hou2020dynabert, - title = {DynaBERT: Dynamic BERT with Adaptive Width and Depth}, - author = {Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu}, - booktitle = {Advances in Neural Information Processing Systems}, - year = {2020} -} -``` diff --git a/model_cards/huawei-noah/DynaBERT_SST-2/README.md b/model_cards/huawei-noah/DynaBERT_SST-2/README.md deleted file mode 100644 index a9a4bc233d..0000000000 --- a/model_cards/huawei-noah/DynaBERT_SST-2/README.md +++ /dev/null @@ -1,20 +0,0 @@ -## DynaBERT: Dynamic BERT with Adaptive Width and Depth - -* DynaBERT can flexibly adjust the size and latency by selecting adaptive width and depth, and -the subnetworks of it have competitive performances as other similar-sized compressed models. -The training process of DynaBERT includes first training a width-adaptive BERT and then -allowing both adaptive width and depth using knowledge distillation. - -* This code is modified based on the repository developed by Hugging Face: [Transformers v2.1.1](https://github.com/huggingface/transformers/tree/v2.1.1), and is released in [GitHub](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/DynaBERT). - -### Reference -Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu. -[DynaBERT: Dynamic BERT with Adaptive Width and Depth](https://arxiv.org/abs/2004.04037). -``` -@inproceedings{hou2020dynabert, - title = {DynaBERT: Dynamic BERT with Adaptive Width and Depth}, - author = {Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu}, - booktitle = {Advances in Neural Information Processing Systems}, - year = {2020} -} -``` diff --git a/model_cards/huawei-noah/TinyBERT_General_4L_312D/README.md b/model_cards/huawei-noah/TinyBERT_General_4L_312D/README.md deleted file mode 100644 index 8ea4061db3..0000000000 --- a/model_cards/huawei-noah/TinyBERT_General_4L_312D/README.md +++ /dev/null @@ -1,19 +0,0 @@ -TinyBERT: Distilling BERT for Natural Language Understanding -======== -TinyBERT is 7.5x smaller and 9.4x faster on inference than BERT-base and achieves competitive performances in the tasks of natural language understanding. It performs a novel transformer distillation at both the pre-training and task-specific learning stages. In general distillation, we use the original BERT-base without fine-tuning as the teacher and a large-scale text corpus as the learning data. By performing the Transformer distillation on the text from general domain, we obtain a general TinyBERT which provides a good initialization for the task-specific distillation. We here provide the general TinyBERT for your tasks at hand. - -For more details about the techniques of TinyBERT, refer to our paper: -[TinyBERT: Distilling BERT for Natural Language Understanding](https://arxiv.org/abs/1909.10351) - - -Citation -======== -If you find TinyBERT useful in your research, please cite the following paper: -``` -@article{jiao2019tinybert, - title={Tinybert: Distilling bert for natural language understanding}, - author={Jiao, Xiaoqi and Yin, Yichun and Shang, Lifeng and Jiang, Xin and Chen, Xiao and Li, Linlin and Wang, Fang and Liu, Qun}, - journal={arXiv preprint arXiv:1909.10351}, - year={2019} -} -``` diff --git a/model_cards/huggingface/CodeBERTa-language-id/README.md b/model_cards/huggingface/CodeBERTa-language-id/README.md deleted file mode 100644 index c9234f2cf8..0000000000 --- a/model_cards/huggingface/CodeBERTa-language-id/README.md +++ /dev/null @@ -1,300 +0,0 @@ ---- -language: code -thumbnail: https://cdn-media.huggingface.co/CodeBERTa/CodeBERTa.png -datasets: -- code_search_net ---- - -# CodeBERTa-language-id: The World’s fanciest programming language identification algo 🤯 - - -To demonstrate the usefulness of our CodeBERTa pretrained model on downstream tasks beyond language modeling, we fine-tune the [`CodeBERTa-small-v1`](https://huggingface.co/huggingface/CodeBERTa-small-v1) checkpoint on the task of classifying a sample of code into the programming language it's written in (*programming language identification*). - -We add a sequence classification head on top of the model. - -On the evaluation dataset, we attain an eval accuracy and F1 > 0.999 which is not surprising given that the task of language identification is relatively easy (see an intuition why, below). - -## Quick start: using the raw model - -```python -CODEBERTA_LANGUAGE_ID = "huggingface/CodeBERTa-language-id" - -tokenizer = RobertaTokenizer.from_pretrained(CODEBERTA_LANGUAGE_ID) -model = RobertaForSequenceClassification.from_pretrained(CODEBERTA_LANGUAGE_ID) - -input_ids = tokenizer.encode(CODE_TO_IDENTIFY) -logits = model(input_ids)[0] - -language_idx = logits.argmax() # index for the resulting label -``` - - -## Quick start: using Pipelines 💪 - -```python -from transformers import TextClassificationPipeline - -pipeline = TextClassificationPipeline( - model=RobertaForSequenceClassification.from_pretrained(CODEBERTA_LANGUAGE_ID), - tokenizer=RobertaTokenizer.from_pretrained(CODEBERTA_LANGUAGE_ID) -) - -pipeline(CODE_TO_IDENTIFY) -``` - -Let's start with something very easy: - -```python -pipeline(""" -def f(x): - return x**2 -""") -# [{'label': 'python', 'score': 0.9999965}] -``` - -Now let's probe shorter code samples: - -```python -pipeline("const foo = 'bar'") -# [{'label': 'javascript', 'score': 0.9977546}] -``` - -What if I remove the `const` token from the assignment? -```python -pipeline("foo = 'bar'") -# [{'label': 'javascript', 'score': 0.7176245}] -``` - -For some reason, this is still statistically detected as JS code, even though it's also valid Python code. However, if we slightly tweak it: - -```python -pipeline("foo = u'bar'") -# [{'label': 'python', 'score': 0.7638422}] -``` -This is now detected as Python (Notice the `u` string modifier). - -Okay, enough with the JS and Python domination already! Let's try fancier languages: - -```python -pipeline("echo $FOO") -# [{'label': 'php', 'score': 0.9995257}] -``` - -(Yes, I used the word "fancy" to describe PHP 😅) - -```python -pipeline("outcome := rand.Intn(6) + 1") -# [{'label': 'go', 'score': 0.9936151}] -``` - -Why is the problem of language identification so easy (with the correct toolkit)? Because code's syntax is rigid, and simple tokens such as `:=` (the assignment operator in Go) are perfect predictors of the underlying language: - -```python -pipeline(":=") -# [{'label': 'go', 'score': 0.9998052}] -``` - -By the way, because we trained our own custom tokenizer on the [CodeSearchNet](https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/) dataset, and it handles streams of bytes in a very generic way, syntactic constructs such `:=` are represented by a single token: - -```python -self.tokenizer.encode(" :=", add_special_tokens=False) -# [521] -``` - -
- -## Fine-tuning code - -
- -```python -import gzip -import json -import logging -import os -from pathlib import Path -from typing import Dict, List, Tuple - -import numpy as np -import torch -from sklearn.metrics import f1_score -from tokenizers.implementations.byte_level_bpe import ByteLevelBPETokenizer -from tokenizers.processors import BertProcessing -from torch.nn.utils.rnn import pad_sequence -from torch.utils.data import DataLoader, Dataset -from torch.utils.data.dataset import Dataset -from torch.utils.tensorboard.writer import SummaryWriter -from tqdm import tqdm, trange - -from transformers import RobertaForSequenceClassification -from transformers.data.metrics import acc_and_f1, simple_accuracy - - -logging.basicConfig(level=logging.INFO) - - -CODEBERTA_PRETRAINED = "huggingface/CodeBERTa-small-v1" - -LANGUAGES = [ - "go", - "java", - "javascript", - "php", - "python", - "ruby", -] -FILES_PER_LANGUAGE = 1 -EVALUATE = True - -# Set up tokenizer -tokenizer = ByteLevelBPETokenizer("./pretrained/vocab.json", "./pretrained/merges.txt",) -tokenizer._tokenizer.post_processor = BertProcessing( - ("", tokenizer.token_to_id("")), ("", tokenizer.token_to_id("")), -) -tokenizer.enable_truncation(max_length=512) - -# Set up Tensorboard -tb_writer = SummaryWriter() - - -class CodeSearchNetDataset(Dataset): - examples: List[Tuple[List[int], int]] - - def __init__(self, split: str = "train"): - """ - train | valid | test - """ - - self.examples = [] - - src_files = [] - for language in LANGUAGES: - src_files += list( - Path("../CodeSearchNet/resources/data/").glob(f"{language}/final/jsonl/{split}/*.jsonl.gz") - )[:FILES_PER_LANGUAGE] - for src_file in src_files: - label = src_file.parents[3].name - label_idx = LANGUAGES.index(label) - print("🔥", src_file, label) - lines = [] - fh = gzip.open(src_file, mode="rt", encoding="utf-8") - for line in fh: - o = json.loads(line) - lines.append(o["code"]) - examples = [(x.ids, label_idx) for x in tokenizer.encode_batch(lines)] - self.examples += examples - print("🔥🔥") - - def __len__(self): - return len(self.examples) - - def __getitem__(self, i): - # We’ll pad at the batch level. - return self.examples[i] - - -model = RobertaForSequenceClassification.from_pretrained(CODEBERTA_PRETRAINED, num_labels=len(LANGUAGES)) - -train_dataset = CodeSearchNetDataset(split="train") -eval_dataset = CodeSearchNetDataset(split="test") - - -def collate(examples): - input_ids = pad_sequence([torch.tensor(x[0]) for x in examples], batch_first=True, padding_value=1) - labels = torch.tensor([x[1] for x in examples]) - # ^^ uncessary .unsqueeze(-1) - return input_ids, labels - - -train_dataloader = DataLoader(train_dataset, batch_size=256, shuffle=True, collate_fn=collate) - -batch = next(iter(train_dataloader)) - - -model.to("cuda") -model.train() -for param in model.roberta.parameters(): - param.requires_grad = False -## ^^ Only train final layer. - -print(f"num params:", model.num_parameters()) -print(f"num trainable params:", model.num_parameters(only_trainable=True)) - - -def evaluate(): - eval_loss = 0.0 - nb_eval_steps = 0 - preds = np.empty((0), dtype=np.int64) - out_label_ids = np.empty((0), dtype=np.int64) - - model.eval() - - eval_dataloader = DataLoader(eval_dataset, batch_size=512, collate_fn=collate) - for step, (input_ids, labels) in enumerate(tqdm(eval_dataloader, desc="Eval")): - with torch.no_grad(): - outputs = model(input_ids=input_ids.to("cuda"), labels=labels.to("cuda")) - loss = outputs[0] - logits = outputs[1] - eval_loss += loss.mean().item() - nb_eval_steps += 1 - preds = np.append(preds, logits.argmax(dim=1).detach().cpu().numpy(), axis=0) - out_label_ids = np.append(out_label_ids, labels.detach().cpu().numpy(), axis=0) - eval_loss = eval_loss / nb_eval_steps - acc = simple_accuracy(preds, out_label_ids) - f1 = f1_score(y_true=out_label_ids, y_pred=preds, average="macro") - print("=== Eval: loss ===", eval_loss) - print("=== Eval: acc. ===", acc) - print("=== Eval: f1 ===", f1) - # print(acc_and_f1(preds, out_label_ids)) - tb_writer.add_scalars("eval", {"loss": eval_loss, "acc": acc, "f1": f1}, global_step) - - -### Training loop - -global_step = 0 -train_iterator = trange(0, 4, desc="Epoch") -optimizer = torch.optim.AdamW(model.parameters()) -for _ in train_iterator: - epoch_iterator = tqdm(train_dataloader, desc="Iteration") - for step, (input_ids, labels) in enumerate(epoch_iterator): - optimizer.zero_grad() - outputs = model(input_ids=input_ids.to("cuda"), labels=labels.to("cuda")) - loss = outputs[0] - loss.backward() - tb_writer.add_scalar("training_loss", loss.item(), global_step) - optimizer.step() - global_step += 1 - if EVALUATE and global_step % 50 == 0: - evaluate() - model.train() - - -evaluate() - -os.makedirs("./models/CodeBERT-language-id", exist_ok=True) -model.save_pretrained("./models/CodeBERT-language-id") -``` - -
- -
- -## CodeSearchNet citation - -
- -```bibtex -@article{husain_codesearchnet_2019, - title = {{CodeSearchNet} {Challenge}: {Evaluating} the {State} of {Semantic} {Code} {Search}}, - shorttitle = {{CodeSearchNet} {Challenge}}, - url = {http://arxiv.org/abs/1909.09436}, - urldate = {2020-03-12}, - journal = {arXiv:1909.09436 [cs, stat]}, - author = {Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc}, - month = sep, - year = {2019}, - note = {arXiv: 1909.09436}, -} -``` - -
diff --git a/model_cards/huggingface/CodeBERTa-small-v1/README.md b/model_cards/huggingface/CodeBERTa-small-v1/README.md deleted file mode 100644 index 70943e7a5f..0000000000 --- a/model_cards/huggingface/CodeBERTa-small-v1/README.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -language: code -thumbnail: https://cdn-media.huggingface.co/CodeBERTa/CodeBERTa.png -datasets: -- code_search_net ---- - -# CodeBERTa - -CodeBERTa is a RoBERTa-like model trained on the [CodeSearchNet](https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/) dataset from GitHub. - -Supported languages: - -```shell -"go" -"java" -"javascript" -"php" -"python" -"ruby" -``` - -The **tokenizer** is a Byte-level BPE tokenizer trained on the corpus using Hugging Face `tokenizers`. - -Because it is trained on a corpus of code (vs. natural language), it encodes the corpus efficiently (the sequences are between 33% to 50% shorter, compared to the same corpus tokenized by gpt2/roberta). - -The (small) **model** is a 6-layer, 84M parameters, RoBERTa-like Transformer model – that’s the same number of layers & heads as DistilBERT – initialized from the default initialization settings and trained from scratch on the full corpus (~2M functions) for 5 epochs. - -### Tensorboard for this training ⤵️ - -[![tb](https://cdn-media.huggingface.co/CodeBERTa/tensorboard.png)](https://tensorboard.dev/experiment/irRI7jXGQlqmlxXS0I07ew/#scalars) - -## Quick start: masked language modeling prediction - -```python -PHP_CODE = """ -public static set(string $key, $value) { - if (!in_array($key, self::$allowedKeys)) { - throw new \InvalidArgumentException('Invalid key given'); - } - self::$storedValues[$key] = $value; -} -""".lstrip() -``` - -### Does the model know how to complete simple PHP code? - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="huggingface/CodeBERTa-small-v1", - tokenizer="huggingface/CodeBERTa-small-v1" -) - -fill_mask(PHP_CODE) - -## Top 5 predictions: -# -' function' # prob 0.9999827146530151 -'function' # -' void' # -' def' # -' final' # -``` - -### Yes! That was easy 🎉 What about some Python (warning: this is going to be meta) - -```python -PYTHON_CODE = """ -def pipeline( - task: str, - model: Optional = None, - framework: Optional[] = None, - **kwargs -) -> Pipeline: - pass -""".lstrip() -``` - -Results: -```python -'framework', 'Framework', ' framework', 'None', 'str' -``` - -> This program can auto-complete itself! 😱 - -### Just for fun, let's try to mask natural language (not code): - -```python -fill_mask("My name is .") - -# {'sequence': ' My name is undefined.', 'score': 0.2548016905784607, 'token': 3353} -# {'sequence': ' My name is required.', 'score': 0.07290805131196976, 'token': 2371} -# {'sequence': ' My name is null.', 'score': 0.06323737651109695, 'token': 469} -# {'sequence': ' My name is name.', 'score': 0.021919190883636475, 'token': 652} -# {'sequence': ' My name is disabled.', 'score': 0.019681859761476517, 'token': 7434} -``` - -This (kind of) works because code contains comments (which contain natural language). - -Of course, the most frequent name for a Computer scientist must be undefined 🤓. - - -## Downstream task: [programming language identification](https://huggingface.co/huggingface/CodeBERTa-language-id) - -See the model card for **[`huggingface/CodeBERTa-language-id`](https://huggingface.co/huggingface/CodeBERTa-language-id)** 🤯. - -
- -## CodeSearchNet citation - -
- -```bibtex -@article{husain_codesearchnet_2019, - title = {{CodeSearchNet} {Challenge}: {Evaluating} the {State} of {Semantic} {Code} {Search}}, - shorttitle = {{CodeSearchNet} {Challenge}}, - url = {http://arxiv.org/abs/1909.09436}, - urldate = {2020-03-12}, - journal = {arXiv:1909.09436 [cs, stat]}, - author = {Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc}, - month = sep, - year = {2019}, - note = {arXiv: 1909.09436}, -} -``` - -
diff --git a/model_cards/huseinzol05/albert-base-bahasa-cased/README.md b/model_cards/huseinzol05/albert-base-bahasa-cased/README.md deleted file mode 100644 index 0ba3c068e3..0000000000 --- a/model_cards/huseinzol05/albert-base-bahasa-cased/README.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -language: ms ---- - -# Bahasa Albert Model - -Pretrained Albert base language model for Malay and Indonesian. - -## Pretraining Corpus - -`albert-base-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google Albert's github [repository](https://github.com/google-research/ALBERT) on v3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/albert](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/albert). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AlbertTokenizer, AlbertModel - -model = BertModel.from_pretrained('huseinzol05/albert-base-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/albert-base-bahasa-cased', - do_lower_case = False, -) -``` - -## Example using AutoModelWithLMHead - -```python -from transformers import AlbertTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/albert-base-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/albert-base-bahasa-cased', - do_lower_case = False, -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan [MASK]')) -``` - -Output is, - -```text -[{'sequence': '[CLS] makan ayam dengan ayam[SEP]', - 'score': 0.044952988624572754, - 'token': 629}, - {'sequence': '[CLS] makan ayam dengan sayur[SEP]', - 'score': 0.03621877357363701, - 'token': 1639}, - {'sequence': '[CLS] makan ayam dengan ikan[SEP]', - 'score': 0.034429922699928284, - 'token': 758}, - {'sequence': '[CLS] makan ayam dengan nasi[SEP]', - 'score': 0.032447945326566696, - 'token': 453}, - {'sequence': '[CLS] makan ayam dengan rendang[SEP]', - 'score': 0.028885239735245705, - 'token': 2451}] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train Albert for Bahasa. - - diff --git a/model_cards/huseinzol05/albert-tiny-bahasa-cased/README.md b/model_cards/huseinzol05/albert-tiny-bahasa-cased/README.md deleted file mode 100644 index 9fd3a15533..0000000000 --- a/model_cards/huseinzol05/albert-tiny-bahasa-cased/README.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -language: ms ---- - -# Bahasa Albert Model - -Pretrained Albert tiny language model for Malay and Indonesian, 85% faster execution and 50% smaller than Albert base. - -## Pretraining Corpus - -`albert-tiny-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google Albert's github [repository](https://github.com/google-research/ALBERT) on v3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/albert](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/albert). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AlbertTokenizer, AlbertModel - -model = BertModel.from_pretrained('huseinzol05/albert-tiny-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/albert-tiny-bahasa-cased', - do_lower_case = False, -) -``` - -## Example using AutoModelWithLMHead - -```python -from transformers import AlbertTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/albert-tiny-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/albert-tiny-bahasa-cased', - do_lower_case = False, -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan [MASK]')) -``` - -Output is, - -```text -[{'sequence': '[CLS] makan ayam dengan ayam[SEP]', - 'score': 0.05121927708387375, - 'token': 629}, - {'sequence': '[CLS] makan ayam dengan sayur[SEP]', - 'score': 0.04497420787811279, - 'token': 1639}, - {'sequence': '[CLS] makan ayam dengan nasi[SEP]', - 'score': 0.039827536791563034, - 'token': 453}, - {'sequence': '[CLS] makan ayam dengan rendang[SEP]', - 'score': 0.032997727394104004, - 'token': 2451}, - {'sequence': '[CLS] makan ayam dengan makan[SEP]', - 'score': 0.031354598701000214, - 'token': 129}] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train Albert for Bahasa. - - diff --git a/model_cards/huseinzol05/bert-base-bahasa-cased/README.md b/model_cards/huseinzol05/bert-base-bahasa-cased/README.md deleted file mode 100644 index f119bd2ec2..0000000000 --- a/model_cards/huseinzol05/bert-base-bahasa-cased/README.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -language: ms ---- - -# Bahasa BERT Model - -Pretrained BERT base language model for Malay and Indonesian. - -## Pretraining Corpus - -`bert-base-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google BERT's github [repository](https://github.com/google-research/bert) on 3 Titan V100 32GB VRAM. -- All steps can reproduce from here, [Malaya/pretrained-model/bert](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/bert). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AlbertTokenizer, BertModel - -model = BertModel.from_pretrained('huseinzol05/bert-base-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/bert-base-bahasa-cased', - unk_token = '[UNK]', - pad_token = '[PAD]', - do_lower_case = False, -) -``` - -We use [google/sentencepiece](https://github.com/google/sentencepiece) to train the tokenizer, so to use it, need to load from `AlbertTokenizer`. - -## Example using AutoModelWithLMHead - -```python -from transformers import AlbertTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/bert-base-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/bert-base-bahasa-cased', - unk_token = '[UNK]', - pad_token = '[PAD]', - do_lower_case = False, -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan [MASK]')) -``` - -Output is, - -```text -[{'sequence': '[CLS] makan ayam dengan rendang[SEP]', - 'score': 0.10812027007341385, - 'token': 2446}, - {'sequence': '[CLS] makan ayam dengan kicap[SEP]', - 'score': 0.07653367519378662, - 'token': 12928}, - {'sequence': '[CLS] makan ayam dengan nasi[SEP]', - 'score': 0.06839974224567413, - 'token': 450}, - {'sequence': '[CLS] makan ayam dengan ayam[SEP]', - 'score': 0.059544261544942856, - 'token': 638}, - {'sequence': '[CLS] makan ayam dengan sayur[SEP]', - 'score': 0.05294966697692871, - 'token': 1639}] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train BERT for Bahasa. - - diff --git a/model_cards/huseinzol05/electra-base-discriminator-bahasa-cased/README.md b/model_cards/huseinzol05/electra-base-discriminator-bahasa-cased/README.md deleted file mode 100644 index b480804d2b..0000000000 --- a/model_cards/huseinzol05/electra-base-discriminator-bahasa-cased/README.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -language: ms ---- - -# Bahasa ELECTRA Model - -Pretrained ELECTRA base language model for Malay and Indonesian. - -## Pretraining Corpus - -`electra-base-discriminator-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google ELECTRA's github [repository](https://github.com/google-research/electra) on a single TESLA V100 32GB VRAM. -- All steps can reproduce from here, [Malaya/pretrained-model/electra](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/electra). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import ElectraTokenizer, ElectraModel - -model = ElectraModel.from_pretrained('huseinzol05/electra-base-discriminator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-base-discriminator-bahasa-cased', - do_lower_case = False, -) -``` - -## Example using ElectraForPreTraining - -```python -from transformers import ElectraTokenizer, AutoModelWithLMHead, pipeline - -model = ElectraForPreTraining.from_pretrained('huseinzol05/electra-base-discriminator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-base-discriminator-bahasa-cased', - do_lower_case = False -) -sentence = 'kerajaan sangat prihatin terhadap rakyat' -fake_tokens = tokenizer.tokenize(sentence) -fake_inputs = tokenizer.encode(sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -list(zip(fake_tokens, predictions.tolist())) -``` - -Output is, - -```text -[('kerajaan', 0.0), - ('sangat', 0.0), - ('prihatin', 0.0), - ('terhadap', 0.0), - ('rakyat', 0.0)] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train ELECTRA for Bahasa. - - diff --git a/model_cards/huseinzol05/electra-base-generator-bahasa-cased/README.md b/model_cards/huseinzol05/electra-base-generator-bahasa-cased/README.md deleted file mode 100644 index f6564f7965..0000000000 --- a/model_cards/huseinzol05/electra-base-generator-bahasa-cased/README.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -language: ms ---- - -# Bahasa ELECTRA Model - -Pretrained ELECTRA base language model for Malay and Indonesian. - -## Pretraining Corpus - -`electra-base-generator-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google ELECTRA's github [repository](https://github.com/google-research/electra) on v3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/electra](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/electra). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import ElectraTokenizer, ElectraModel - -model = ElectraModel.from_pretrained('huseinzol05/electra-base-generator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-base-generator-bahasa-cased', - do_lower_case = False, -) -``` - -## Example using AutoModelWithLMHead - -```python -from transformers import ElectraTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/electra-base-generator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-base-generator-bahasa-cased', - do_lower_case = False, -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan [MASK]')) -``` - -Output is, - -```text -[{'sequence': '[CLS] makan ayam dengan ayam [SEP]', - 'score': 0.08424834907054901, - 'token': 3255}, - {'sequence': '[CLS] makan ayam dengan rendang [SEP]', - 'score': 0.064150370657444, - 'token': 6288}, - {'sequence': '[CLS] makan ayam dengan nasi [SEP]', - 'score': 0.033446669578552246, - 'token': 2533}, - {'sequence': '[CLS] makan ayam dengan kucing [SEP]', - 'score': 0.02803465723991394, - 'token': 3577}, - {'sequence': '[CLS] makan ayam dengan telur [SEP]', - 'score': 0.026627106592059135, - 'token': 6350}] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train ELECTRA for Bahasa. - - diff --git a/model_cards/huseinzol05/electra-small-discriminator-bahasa-cased/README.md b/model_cards/huseinzol05/electra-small-discriminator-bahasa-cased/README.md deleted file mode 100644 index ef45858598..0000000000 --- a/model_cards/huseinzol05/electra-small-discriminator-bahasa-cased/README.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -language: ms ---- - -# Bahasa ELECTRA Model - -Pretrained ELECTRA small language model for Malay and Indonesian. - -## Pretraining Corpus - -`electra-small-discriminator-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google ELECTRA's github [repository](https://github.com/google-research/electra) on a single TESLA V100 32GB VRAM. -- All steps can reproduce from here, [Malaya/pretrained-model/electra](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/electra). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import ElectraTokenizer, ElectraModel - -model = ElectraModel.from_pretrained('huseinzol05/electra-small-discriminator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-small-discriminator-bahasa-cased', - do_lower_case = False, -) -``` - -## Example using ElectraForPreTraining - -```python -from transformers import ElectraTokenizer, AutoModelWithLMHead, pipeline - -model = ElectraForPreTraining.from_pretrained('huseinzol05/electra-small-discriminator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-small-discriminator-bahasa-cased', - do_lower_case = False -) -sentence = 'kerajaan sangat prihatin terhadap rakyat' -fake_tokens = tokenizer.tokenize(sentence) -fake_inputs = tokenizer.encode(sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -list(zip(fake_tokens, predictions.tolist())) -``` - -Output is, - -```text -[('kerajaan', 0.0), - ('sangat', 0.0), - ('prihatin', 0.0), - ('terhadap', 0.0), - ('rakyat', 0.0)] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train ELECTRA for Bahasa. - - diff --git a/model_cards/huseinzol05/electra-small-generator-bahasa-cased/README.md b/model_cards/huseinzol05/electra-small-generator-bahasa-cased/README.md deleted file mode 100644 index a4ccc50c76..0000000000 --- a/model_cards/huseinzol05/electra-small-generator-bahasa-cased/README.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -language: ms ---- - -# Bahasa ELECTRA Model - -Pretrained ELECTRA small language model for Malay and Indonesian. - -## Pretraining Corpus - -`electra-small-generator-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google ELECTRA's github [repository](https://github.com/google-research/electra) on a single TESLA V100 32GB VRAM. -- All steps can reproduce from here, [Malaya/pretrained-model/electra](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/electra). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import ElectraTokenizer, ElectraModel - -model = ElectraModel.from_pretrained('huseinzol05/electra-small-generator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-small-generator-bahasa-cased', - do_lower_case = False, -) -``` - -## Example using AutoModelWithLMHead - -```python -from transformers import ElectraTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/electra-small-generator-bahasa-cased') -tokenizer = ElectraTokenizer.from_pretrained( - 'huseinzol05/electra-small-generator-bahasa-cased', - do_lower_case = False, -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan [MASK]')) -``` - -Output is, - -```text -[{'sequence': '[CLS] makan ayam dengan ayam [SEP]', - 'score': 0.08424834907054901, - 'token': 3255}, - {'sequence': '[CLS] makan ayam dengan rendang [SEP]', - 'score': 0.064150370657444, - 'token': 6288}, - {'sequence': '[CLS] makan ayam dengan nasi [SEP]', - 'score': 0.033446669578552246, - 'token': 2533}, - {'sequence': '[CLS] makan ayam dengan kucing [SEP]', - 'score': 0.02803465723991394, - 'token': 3577}, - {'sequence': '[CLS] makan ayam dengan telur [SEP]', - 'score': 0.026627106592059135, - 'token': 6350}] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train ELECTRA for Bahasa. - - diff --git a/model_cards/huseinzol05/gpt2-117M-bahasa-cased/README.md b/model_cards/huseinzol05/gpt2-117M-bahasa-cased/README.md deleted file mode 100644 index 0d6dc0812a..0000000000 --- a/model_cards/huseinzol05/gpt2-117M-bahasa-cased/README.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -language: ms ---- - -# Bahasa GPT2 Model - -Pretrained GPT2 117M model for Malay. - -## Pretraining Corpus - -`gpt2-117M-bahasa-cased` model was pretrained on ~0.9 Billion words. We trained on standard language structure only, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -3. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -4. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -5. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -6. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -7. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). -8. [Common-Crawl](https://github.com/huseinzol05/malaya-dataset#common-crawl). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using GPT2's github [repository](https://github.com/openai/gpt-2) on a V3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/gpt2](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/gpt2). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import GPT2Tokenizer, GPT2Model - -model = GPT2Model.from_pretrained('huseinzol05/gpt2-117M-bahasa-cased') -tokenizer = GPT2Tokenizer.from_pretrained( - 'huseinzol05/gpt2-117M-bahasa-cased', -) -``` - -## Example using GPT2LMHeadModel - -```python -from transformers import GPT2Tokenizer, GPT2LMHeadModel - -tokenizer = GPT2Tokenizer.from_pretrained('huseinzol05/gpt2-117M-bahasa-cased') -model = GPT2LMHeadModel.from_pretrained( - 'huseinzol05/gpt2-117M-bahasa-cased', pad_token_id = tokenizer.eos_token_id -) - -input_ids = tokenizer.encode( - 'penat bak hang, macam ni aku takmau kerja dah', return_tensors = 'pt' -) -sample_outputs = model.generate( - input_ids, - do_sample = True, - max_length = 50, - top_k = 50, - top_p = 0.95, - num_return_sequences = 3, -) - -print('Output:\n' + 100 * '-') -for i, sample_output in enumerate(sample_outputs): - print( - '{}: {}'.format( - i, tokenizer.decode(sample_output, skip_special_tokens = True) - ) - ) -``` - -Output is, - -```text -Output: ----------------------------------------------------------------------------------------------------- -0: penat bak hang, macam ni aku takmau kerja dah jadi aku pernah beritahu orang. -Ini bukan aku rasa cam nak ajak teman kan ni. -Tengok ni aku dah ada adik-adik & anak yang tinggal dan kerja2 yang kat sekolah. -1: penat bak hang, macam ni aku takmau kerja dah. -Takleh takleh nak ambik air. -Tgk jugak aku kat rumah ni. -Pastu aku nak bagi aku. -So aku dah takde masalah pulak. -Balik aku pun -2: penat bak hang, macam ni aku takmau kerja dah macam tu. -Tapi semua tu aku ingat cakap, ada cara hidup ni yang kita kena bayar.. pastu kita tak mampu bayar.. kan!! -Takpelah, aku nak cakap, masa yang -``` diff --git a/model_cards/huseinzol05/gpt2-345M-bahasa-cased/README.md b/model_cards/huseinzol05/gpt2-345M-bahasa-cased/README.md deleted file mode 100644 index 9c76145819..0000000000 --- a/model_cards/huseinzol05/gpt2-345M-bahasa-cased/README.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -language: ms ---- - -# Bahasa GPT2 Model - -Pretrained GPT2 345M model for Malay. - -## Pretraining Corpus - -`gpt2-345M-bahasa-cased` model was pretrained on ~0.9 Billion words. We trained on standard language structure only, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -3. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -4. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -5. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -6. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -7. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). -8. [Common-Crawl](https://github.com/huseinzol05/malaya-dataset#common-crawl). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using GPT2's github [repository](https://github.com/openai/gpt-2) on a V3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/gpt2](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/gpt2). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import GPT2Tokenizer, GPT2Model - -model = GPT2Model.from_pretrained('huseinzol05/gpt2-345M-bahasa-cased') -tokenizer = GPT2Tokenizer.from_pretrained( - 'huseinzol05/gpt2-345M-bahasa-cased', -) -``` - -## Example using GPT2LMHeadModel - -```python -from transformers import GPT2Tokenizer, GPT2LMHeadModel - -tokenizer = GPT2Tokenizer.from_pretrained('huseinzol05/gpt2-345M-bahasa-cased') -model = GPT2LMHeadModel.from_pretrained( - 'huseinzol05/gpt2-345M-bahasa-cased', pad_token_id = tokenizer.eos_token_id -) - -input_ids = tokenizer.encode( - 'penat bak hang, macam ni aku takmau kerja dah', return_tensors = 'pt' -) -sample_outputs = model.generate( - input_ids, - do_sample = True, - max_length = 50, - top_k = 50, - top_p = 0.95, - num_return_sequences = 3, -) - -print('Output:\n' + 100 * '-') -for i, sample_output in enumerate(sample_outputs): - print( - '{}: {}'.format( - i, tokenizer.decode(sample_output, skip_special_tokens = True) - ) - ) -``` - -Output is, - -```text -Output: ----------------------------------------------------------------------------------------------------- -0: penat bak hang, macam ni aku takmau kerja dah dekat 2,3 jam. -Aku harap aku dapat berjimat banyak. -Ini pun masa kerja, bila dah kerja jadi satu. -Aku buat kerja ni la. -Aku memang kalau ada -1: penat bak hang, macam ni aku takmau kerja dah. -Tapi nak buat macam mana kan, aku tolong bentang tugas. -Dan, memang sangat-sangat tak mahu buat kerja sekarang ni. -Aku pun suka sangat kerja di luar bandar -2: penat bak hang, macam ni aku takmau kerja dah pun. -Takpa nak buat kerja-kerja sampingan, baru boleh dapat hadiah pulak. -Ni la tempat paling best bila duduk di restoran yang ada pekena kopi. -Cumanya -``` diff --git a/model_cards/huseinzol05/t5-base-bahasa-cased/README.md b/model_cards/huseinzol05/t5-base-bahasa-cased/README.md deleted file mode 100644 index 18fe9fb01f..0000000000 --- a/model_cards/huseinzol05/t5-base-bahasa-cased/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: ms ---- - -# Bahasa T5 Model - -Pretrained T5 base language model for Malay and Indonesian. - -## Pretraining Corpus - -`t5-base-bahasa-cased` model was pretrained on multiple tasks. Below is list of tasks we trained on, - -1. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [local Wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -3. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -4. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -5. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -6. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). -7. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [local Wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -8. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -9. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -10. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -11. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -12. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). -13. [Bahasa SNLI](https://github.com/huseinzol05/Malaya-Dataset#snli). -14. [Bahasa Question Quora](https://github.com/huseinzol05/Malaya-Dataset#quora). -15. [Bahasa Natural Questions](https://github.com/huseinzol05/Malaya-Dataset#natural-questions). -16. [News title summarization](https://github.com/huseinzol05/Malaya-Dataset#crawled-news). -17. [Stemming to original wikipedia](https://github.com/huseinzol05/Malaya/blob/master/pretrained-model/t5/generate-stemming.ipynb). -18. [Synonym to original wikipedia](https://github.com/huseinzol05/Malaya/blob/master/pretrained-model/t5/generate-synonym.ipynb). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google T5's github [repository](https://github.com/google-research/text-to-text-transfer-transformer) on v3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/t5](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/t5). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import T5Tokenizer, T5Model - -model = T5Model.from_pretrained('huseinzol05/t5-base-bahasa-cased') -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-base-bahasa-cased') -``` - -## Example using T5ForConditionalGeneration - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-base-bahasa-cased') -model = T5ForConditionalGeneration.from_pretrained('huseinzol05/t5-base-bahasa-cased') -input_ids = tokenizer.encode('soalan: siapakah perdana menteri malaysia?', return_tensors = 'pt') -outputs = model.generate(input_ids) -print(tokenizer.decode(outputs[0])) -``` - -Output is, - -``` -'Mahathir Mohamad' -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train T5 for Bahasa. diff --git a/model_cards/huseinzol05/t5-base-bahasa-summarization-cased/README.md b/model_cards/huseinzol05/t5-base-bahasa-summarization-cased/README.md deleted file mode 100644 index b36d80166f..0000000000 --- a/model_cards/huseinzol05/t5-base-bahasa-summarization-cased/README.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -language: ms ---- - -# Bahasa T5 Summarization Model - -Finetuned T5 base summarization model for Malay and Indonesian. - -## Finetuning Corpus - -`t5-base-bahasa-summarization-cased` model was finetuned on multiple summarization dataset. Below is list of tasks we trained on, - -1. [Translated CNN News](https://github.com/huseinzol05/Malay-Dataset#cnn-news) -2. [Translated Gigawords](https://github.com/huseinzol05/Malay-Dataset#gigawords) -3. [Translated Multinews](https://github.com/huseinzol05/Malay-Dataset#multinews) - -## Finetuning details - -- This model was trained using Malaya T5's github [repository](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/t5) on v3-8 TPU using Base size. -- All steps can reproduce from here, [Malaya/session/summarization](https://github.com/huseinzol05/Malaya/tree/master/session/summarization). - -## Load Finetuned Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import T5Tokenizer, T5Model - -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-base-bahasa-summarization-cased') -model = T5ForConditionalGeneration.from_pretrained('huseinzol05/t5-base-bahasa-summarization-cased') -``` - -## Example using T5ForConditionalGeneration - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-base-bahasa-summarization-cased') -model = T5ForConditionalGeneration.from_pretrained('huseinzol05/t5-base-bahasa-summarization-cased') - -# https://www.hmetro.com.my/mutakhir/2020/05/580438/peletakan-jawatan-tun-m-ditolak-bukan-lagi-isu -# original title, Peletakan jawatan Tun M ditolak, bukan lagi isu -string = 'PELETAKAN jawatan Tun Dr Mahathir Mohamad sebagai Pengerusi Parti Pribumi Bersatu Malaysia (Bersatu) ditolak di dalam mesyuarat khas Majlis Pimpinan Tertinggi (MPT) pada 24 Februari lalu. Justeru, tidak timbul soal peletakan jawatan itu sah atau tidak kerana ia sudah pun diputuskan pada peringkat parti yang dipersetujui semua termasuk Presiden, Tan Sri Muhyiddin Yassin. Bekas Setiausaha Agung Bersatu Datuk Marzuki Yahya berkata, pada mesyuarat itu MPT sebulat suara menolak peletakan jawatan Dr Mahathir. "Jadi ini agak berlawanan dengan keputusan yang kita sudah buat. Saya tak faham bagaimana Jabatan Pendaftar Pertubuhan Malaysia (JPPM) kata peletakan jawatan itu sah sedangkan kita sudah buat keputusan di dalam mesyuarat, bukan seorang dua yang buat keputusan. "Semua keputusan mesti dibuat melalui parti. Walau apa juga perbincangan dibuat di luar daripada keputusan mesyuarat, ini bukan keputusan parti. "Apa locus standy yang ada pada Setiausaha Kerja untuk membawa perkara ini kepada JPPM. Seharusnya ia dibawa kepada Setiausaha Agung sebagai pentadbir kepada parti," katanya kepada Harian Metro. Beliau mengulas laporan media tempatan hari ini mengenai pengesahan JPPM bahawa Dr Mahathir tidak lagi menjadi Pengerusi Bersatu berikutan peletakan jawatannya di tengah-tengah pergolakan politik pada akhir Februari adalah sah. Laporan itu juga menyatakan, kedudukan Muhyiddin Yassin memangku jawatan itu juga sah. Menurutnya, memang betul Dr Mahathir menghantar surat peletakan jawatan, tetapi ditolak oleh MPT. "Fasal yang disebut itu terpakai sekiranya berhenti atau diberhentikan, tetapi ini mesyuarat sudah menolak," katanya. Marzuki turut mempersoal kenyataan media yang dibuat beberapa pimpinan parti itu hari ini yang menyatakan sokongan kepada Perikatan Nasional. "Kenyataan media bukanlah keputusan rasmi. Walaupun kita buat 1,000 kenyataan sekali pun ia tetap tidak merubah keputusan yang sudah dibuat di dalam mesyuarat. Kita catat di dalam minit apa yang berlaku di dalam mesyuarat," katanya.' - -# https://huggingface.co/blog/how-to-generate -# generate summary -input_ids = tokenizer.encode(f'ringkasan: {string}', return_tensors = 'pt') -outputs = model.generate( - input_ids, - do_sample = True, - temperature = 0.8, - top_k = 50, - top_p = 0.95, - max_length = 300, - num_return_sequences = 3, -) - -for i, sample_output in enumerate(outputs): - print( - '{}: {}'.format( - i, tokenizer.decode(sample_output, skip_special_tokens = True) - ) - ) - -# generate news title -input_ids = tokenizer.encode(f'tajuk: {string}', return_tensors = 'pt') -outputs = model.generate( - input_ids, - do_sample = True, - temperature = 0.8, - top_k = 50, - top_p = 0.95, - max_length = 300, - num_return_sequences = 3, -) - -for i, sample_output in enumerate(outputs): - print( - '{}: {}'.format( - i, tokenizer.decode(sample_output, skip_special_tokens = True) - ) - ) -``` - -Output is, - -``` -0: "Ini agak berlawanan dengan keputusan yang kita sudah buat," kata Marzuki Yahya. Kenyataan media adalah keputusan rasmi. Marzuki: Kenyataan media tidak mengubah keputusan mesyuarat -1: MPT sebulat suara menolak peletakan jawatan Dr M di mesyuarat 24 Februari. Tidak ada persoalan peletakan jawatan itu sah atau tidak, tetapi ia adalah keputusan parti yang dipersetujui semua. Bekas Setiausaha Agung Bersatu mengatakan keputusan itu perlu dibuat melalui parti. Bekas setiausaha agung itu mengatakan kenyataan media tidak lagi menyokong Perikatan Nasional -2: Kenyataan media menunjukkan sokongan kepada Perikatan Nasional. Marzuki: Kedudukan Dr M sebagai Pengerusi Bersatu juga sah. Beliau berkata pengumuman itu harus diserahkan kepada setiausaha Agung - -0: 'Kalah Tun M, Muhyiddin tetap sah' -1: Boleh letak jawatan PM di MPT -2: 'Ketegangan Dr M sudah tolak, tak timbul isu peletakan jawatan' -``` - -## Result - -We found out using original Tensorflow implementation gives better results, check it at https://malaya.readthedocs.io/en/latest/Abstractive.html#generate-ringkasan - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train T5 for Bahasa. diff --git a/model_cards/huseinzol05/t5-small-bahasa-cased/README.md b/model_cards/huseinzol05/t5-small-bahasa-cased/README.md deleted file mode 100644 index b0f622d536..0000000000 --- a/model_cards/huseinzol05/t5-small-bahasa-cased/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: ms ---- - -# Bahasa T5 Model - -Pretrained T5 small language model for Malay and Indonesian. - -## Pretraining Corpus - -`t5-small-bahasa-cased` model was pretrained on multiple tasks. Below is list of tasks we trained on, - -1. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [local Wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -3. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -4. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -5. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -6. [Unsupervised](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1875) on [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). -7. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [local Wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -8. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -9. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -10. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -11. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -12. [Next sentence prediction](https://github.com/google-research/text-to-text-transfer-transformer/blob/master/t5/data/preprocessors.py#L1129) on [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). -13. [Bahasa SNLI](https://github.com/huseinzol05/Malaya-Dataset#snli). -14. [Bahasa Question Quora](https://github.com/huseinzol05/Malaya-Dataset#quora). -15. [Bahasa Natural Questions](https://github.com/huseinzol05/Malaya-Dataset#natural-questions). -16. [News title summarization](https://github.com/huseinzol05/Malaya-Dataset#crawled-news). -17. [Stemming to original wikipedia](https://github.com/huseinzol05/Malaya/blob/master/pretrained-model/t5/generate-stemming.ipynb). -18. [Synonym to original wikipedia](https://github.com/huseinzol05/Malaya/blob/master/pretrained-model/t5/generate-synonym.ipynb). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using Google T5's github [repository](https://github.com/google-research/text-to-text-transfer-transformer) on v3-8 TPU. -- All steps can reproduce from here, [Malaya/pretrained-model/t5](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/t5). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import T5Tokenizer, T5Model - -model = T5Model.from_pretrained('huseinzol05/t5-small-bahasa-cased') -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-small-bahasa-cased') -``` - -## Example using T5ForConditionalGeneration - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-small-bahasa-cased') -model = T5ForConditionalGeneration.from_pretrained('huseinzol05/t5-small-bahasa-cased') -input_ids = tokenizer.encode('soalan: siapakah perdana menteri malaysia?', return_tensors = 'pt') -outputs = model.generate(input_ids) -print(tokenizer.decode(outputs[0])) -``` - -Output is, - -``` -'Mahathir Mohamad' -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train T5 for Bahasa. diff --git a/model_cards/huseinzol05/t5-small-bahasa-summarization-cased/README.md b/model_cards/huseinzol05/t5-small-bahasa-summarization-cased/README.md deleted file mode 100644 index 3d23762de5..0000000000 --- a/model_cards/huseinzol05/t5-small-bahasa-summarization-cased/README.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -language: ms ---- - -# Bahasa T5 Summarization Model - -Finetuned T5 small summarization model for Malay and Indonesian. - -## Finetuning Corpus - -`t5-small-bahasa-summarization-cased` model was finetuned on multiple summarization dataset. Below is list of tasks we trained on, - -1. [Translated CNN News](https://github.com/huseinzol05/Malay-Dataset#cnn-news) -2. [Translated Gigawords](https://github.com/huseinzol05/Malay-Dataset#gigawords) -3. [Translated Multinews](https://github.com/huseinzol05/Malay-Dataset#multinews) - -## Finetuning details - -- This model was trained using Malaya T5's github [repository](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/t5) on v3-8 TPU using small size. -- All steps can reproduce from here, [Malaya/session/summarization](https://github.com/huseinzol05/Malaya/tree/master/session/summarization). - -## Load Finetuned Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import T5Tokenizer, T5Model - -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-small-bahasa-summarization-cased') -model = T5ForConditionalGeneration.from_pretrained('huseinzol05/t5-small-bahasa-summarization-cased') -``` - -## Example using T5ForConditionalGeneration - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained('huseinzol05/t5-small-bahasa-summarization-cased') -model = T5ForConditionalGeneration.from_pretrained('huseinzol05/t5-small-bahasa-summarization-cased') - -# https://www.hmetro.com.my/mutakhir/2020/05/580438/peletakan-jawatan-tun-m-ditolak-bukan-lagi-isu -# original title, Peletakan jawatan Tun M ditolak, bukan lagi isu -string = 'PELETAKAN jawatan Tun Dr Mahathir Mohamad sebagai Pengerusi Parti Pribumi Bersatu Malaysia (Bersatu) ditolak di dalam mesyuarat khas Majlis Pimpinan Tertinggi (MPT) pada 24 Februari lalu. Justeru, tidak timbul soal peletakan jawatan itu sah atau tidak kerana ia sudah pun diputuskan pada peringkat parti yang dipersetujui semua termasuk Presiden, Tan Sri Muhyiddin Yassin. Bekas Setiausaha Agung Bersatu Datuk Marzuki Yahya berkata, pada mesyuarat itu MPT sebulat suara menolak peletakan jawatan Dr Mahathir. "Jadi ini agak berlawanan dengan keputusan yang kita sudah buat. Saya tak faham bagaimana Jabatan Pendaftar Pertubuhan Malaysia (JPPM) kata peletakan jawatan itu sah sedangkan kita sudah buat keputusan di dalam mesyuarat, bukan seorang dua yang buat keputusan. "Semua keputusan mesti dibuat melalui parti. Walau apa juga perbincangan dibuat di luar daripada keputusan mesyuarat, ini bukan keputusan parti. "Apa locus standy yang ada pada Setiausaha Kerja untuk membawa perkara ini kepada JPPM. Seharusnya ia dibawa kepada Setiausaha Agung sebagai pentadbir kepada parti," katanya kepada Harian Metro. Beliau mengulas laporan media tempatan hari ini mengenai pengesahan JPPM bahawa Dr Mahathir tidak lagi menjadi Pengerusi Bersatu berikutan peletakan jawatannya di tengah-tengah pergolakan politik pada akhir Februari adalah sah. Laporan itu juga menyatakan, kedudukan Muhyiddin Yassin memangku jawatan itu juga sah. Menurutnya, memang betul Dr Mahathir menghantar surat peletakan jawatan, tetapi ditolak oleh MPT. "Fasal yang disebut itu terpakai sekiranya berhenti atau diberhentikan, tetapi ini mesyuarat sudah menolak," katanya. Marzuki turut mempersoal kenyataan media yang dibuat beberapa pimpinan parti itu hari ini yang menyatakan sokongan kepada Perikatan Nasional. "Kenyataan media bukanlah keputusan rasmi. Walaupun kita buat 1,000 kenyataan sekali pun ia tetap tidak merubah keputusan yang sudah dibuat di dalam mesyuarat. Kita catat di dalam minit apa yang berlaku di dalam mesyuarat," katanya.' - -# https://huggingface.co/blog/how-to-generate -# generate summary -input_ids = tokenizer.encode(f'ringkasan: {string}', return_tensors = 'pt') -outputs = model.generate( - input_ids, - do_sample = True, - temperature = 0.8, - top_k = 50, - top_p = 0.95, - max_length = 300, - num_return_sequences = 3, -) - -for i, sample_output in enumerate(outputs): - print( - '{}: {}'.format( - i, tokenizer.decode(sample_output, skip_special_tokens = True) - ) - ) - -# generate news title -input_ids = tokenizer.encode(f'tajuk: {string}', return_tensors = 'pt') -outputs = model.generate( - input_ids, - do_sample = True, - temperature = 0.8, - top_k = 50, - top_p = 0.95, - max_length = 300, - num_return_sequences = 3, -) - -for i, sample_output in enumerate(outputs): - print( - '{}: {}'.format( - i, tokenizer.decode(sample_output, skip_special_tokens = True) - ) - ) -``` - -Output is, - -``` -0: Pengerusi Bersatu Bersatu menafikan peletakan jawatan dalam mesyuarat khas Majlis Pimpinan Tertinggi. Tidak timbul isu peletakan jawatan itu sah atau tidak kerana ia sudah diputuskan di peringkat parti. Kenyataan media yang dibuat oleh pemimpin parti hari ini menyokong Perikatan Nasional -1: Tiada keputusan kerana ia sudah diputuskan pada peringkat parti, Marzuki berkata. Pejabat rasmi parti menolak peletakan jawatan Dr M, dengan mengatakan ia adalah keputusan. Kedudukan Muhyiddin memangku jawatan itu juga sah, katanya -2: Tiada peletakan jawatan Dr Mahathir dalam mesyuarat khas MPT pada 24 Februari. Ketua parti menolak peletakan jawatan itu. Tidak timbul isu peletakan jawatan itu sah atau tidak, katanya - -0: Tiada peletakan jawatan Tun M dalam mesyuarat khas -1: ‘Tidak timbul peletakan jawatan Tun M’ -2: Tidak timbul isu peletakan jawatan Tun M di mesyuarat khas -``` - -## Result - -We found out using original Tensorflow implementation gives better results, check it at https://malaya.readthedocs.io/en/latest/Abstractive.html#generate-ringkasan - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train T5 for Bahasa. diff --git a/model_cards/huseinzol05/tiny-bert-bahasa-cased/README.md b/model_cards/huseinzol05/tiny-bert-bahasa-cased/README.md deleted file mode 100644 index 95671a62dd..0000000000 --- a/model_cards/huseinzol05/tiny-bert-bahasa-cased/README.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -language: ms ---- - -# Bahasa Tiny-BERT Model - -General Distilled Tiny BERT language model for Malay and Indonesian. - -## Pretraining Corpus - -`tiny-bert-bahasa-cased` model was distilled on ~1.8 Billion words. We distilled on both standard and social media language structures, and below is list of data we distilled on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Distilling details - -- This model was distilled using huawei-noah Tiny-BERT's github [repository](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/TinyBERT) on 3 Titan V100 32GB VRAM. -- All steps can reproduce from here, [Malaya/pretrained-model/tiny-bert](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/tiny-bert). - -## Load Distilled Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import AlbertTokenizer, BertModel - -model = BertModel.from_pretrained('huseinzol05/tiny-bert-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/tiny-bert-bahasa-cased', - unk_token = '[UNK]', - pad_token = '[PAD]', - do_lower_case = False, -) -``` - -We use [google/sentencepiece](https://github.com/google/sentencepiece) to train the tokenizer, so to use it, need to load from `AlbertTokenizer`. - -## Example using AutoModelWithLMHead - -```python -from transformers import AlbertTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/tiny-bert-bahasa-cased') -tokenizer = AlbertTokenizer.from_pretrained( - 'huseinzol05/tiny-bert-bahasa-cased', - unk_token = '[UNK]', - pad_token = '[PAD]', - do_lower_case = False, -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan [MASK]')) -``` - -Output is, - -```text -[{'sequence': '[CLS] makan ayam dengan berbual[SEP]', - 'score': 0.00015769545279908925, - 'token': 17859}, - {'sequence': '[CLS] makan ayam dengan kembar[SEP]', - 'score': 0.0001448775001335889, - 'token': 8289}, - {'sequence': '[CLS] makan ayam dengan memaklumkan[SEP]', - 'score': 0.00013484008377417922, - 'token': 6881}, - {'sequence': '[CLS] makan ayam dengan Senarai[SEP]', - 'score': 0.00013061291247140616, - 'token': 11698}, - {'sequence': '[CLS] makan ayam dengan Tiga[SEP]', - 'score': 0.00012453157978598028, - 'token': 4232}] -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train BERT for Bahasa. - - diff --git a/model_cards/huseinzol05/xlnet-base-bahasa-cased/README.md b/model_cards/huseinzol05/xlnet-base-bahasa-cased/README.md deleted file mode 100644 index 1c4b03ffa8..0000000000 --- a/model_cards/huseinzol05/xlnet-base-bahasa-cased/README.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -language: ms ---- - -# Bahasa XLNet Model - -Pretrained XLNet base language model for Malay and Indonesian. - -## Pretraining Corpus - -`XLNET-base-bahasa-cased` model was pretrained on ~1.8 Billion words. We trained on both standard and social media language structures, and below is list of data we trained on, - -1. [dumping wikipedia](https://github.com/huseinzol05/Malaya-Dataset#wikipedia-1). -2. [local instagram](https://github.com/huseinzol05/Malaya-Dataset#instagram). -3. [local twitter](https://github.com/huseinzol05/Malaya-Dataset#twitter-1). -4. [local news](https://github.com/huseinzol05/Malaya-Dataset#public-news). -5. [local parliament text](https://github.com/huseinzol05/Malaya-Dataset#parliament). -6. [local singlish/manglish text](https://github.com/huseinzol05/Malaya-Dataset#singlish-text). -7. [IIUM Confession](https://github.com/huseinzol05/Malaya-Dataset#iium-confession). -8. [Wattpad](https://github.com/huseinzol05/Malaya-Dataset#wattpad). -9. [Academia PDF](https://github.com/huseinzol05/Malaya-Dataset#academia-pdf). - -Preprocessing steps can reproduce from here, [Malaya/pretrained-model/preprocess](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/preprocess). - -## Pretraining details - -- This model was trained using zihangdai XLNet's github [repository](https://github.com/zihangdai/xlnet) on 3 Titan V100 32GB VRAM. -- All steps can reproduce from here, [Malaya/pretrained-model/xlnet](https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/xlnet). - -## Load Pretrained Model - -You can use this model by installing `torch` or `tensorflow` and Huggingface library `transformers`. And you can use it directly by initializing it like this: - -```python -from transformers import XLNetTokenizer, XLNetModel - -model = XLNetModel.from_pretrained('huseinzol05/xlnet-base-bahasa-cased') -tokenizer = XLNetTokenizer.from_pretrained( - 'huseinzol05/xlnet-base-bahasa-cased', do_lower_case = False -) -``` - -## Example using AutoModelWithLMHead - -```python -from transformers import AlbertTokenizer, AutoModelWithLMHead, pipeline - -model = AutoModelWithLMHead.from_pretrained('huseinzol05/xlnet-base-bahasa-cased') -tokenizer = XLNetTokenizer.from_pretrained( - 'huseinzol05/xlnet-base-bahasa-cased', do_lower_case = False -) -fill_mask = pipeline('fill-mask', model = model, tokenizer = tokenizer) -print(fill_mask('makan ayam dengan ')) -``` - -## Results - -For further details on the model performance, simply checkout accuracy page from Malaya, https://malaya.readthedocs.io/en/latest/Accuracy.html, we compared with traditional models. - -## Acknowledgement - -Thanks to [Im Big](https://www.facebook.com/imbigofficial/), [LigBlou](https://www.facebook.com/ligblou), [Mesolitica](https://mesolitica.com/) and [KeyReply](https://www.keyreply.com/) for sponsoring AWS, Google and GPU clouds to train XLNet for Bahasa. - - diff --git a/model_cards/iarfmoose/bert-base-cased-qa-evaluator/README.md b/model_cards/iarfmoose/bert-base-cased-qa-evaluator/README.md deleted file mode 100644 index b1bcf5f268..0000000000 --- a/model_cards/iarfmoose/bert-base-cased-qa-evaluator/README.md +++ /dev/null @@ -1,28 +0,0 @@ -# BERT-base-cased-qa-evaluator - -This model takes a question answer pair as an input and outputs a value representing its prediction about whether the input was a valid question and answer pair or not. The model is a pretrained [BERT-base-cased](https://huggingface.co/bert-base-cased) with a sequence classification head. - -## Intended uses - -The QA evaluator was originally designed to be used with the [t5-base-question-generator](https://huggingface.co/iarfmoose/t5-base-question-generator) for evaluating the quality of generated questions. - -The input for the QA evaluator follows the format for `BertForSequenceClassification`, but using the question and answer as the two sequences. Inputs should take the following format: -``` -[CLS] [SEP]
-@InProceedings{10.1007/978-3-030-03840-3_29, - author="Croce, Danilo and Zelenanska, Alexandra and Basili, Roberto", - editor="Ghidini, Chiara and Magnini, Bernardo and Passerini, Andrea and Traverso, Paolo", - title="Neural Learning for Question Answering in Italian", - booktitle="AI*IA 2018 -- Advances in Artificial Intelligence", - year="2018", - publisher="Springer International Publishing", - address="Cham", - pages="389--402", - isbn="978-3-030-03840-3" -} - diff --git a/model_cards/mrm8488/bert-medium-finetuned-squadv2/README.md b/model_cards/mrm8488/bert-medium-finetuned-squadv2/README.md deleted file mode 100644 index 572589ce5d..0000000000 --- a/model_cards/mrm8488/bert-medium-finetuned-squadv2/README.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -language: en -thumbnail: ---- - -# BERT-Medium fine-tuned on SQuAD v2 - -[BERT-Medium](https://github.com/google-research/bert/) created by [Google Research](https://github.com/google-research) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -**Mode size** (after training): **157.46 MB** - -## Details of BERT-Small and its 'family' (from their documentation) - -Released on March 11th, 2020 - -This is model is a part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962). - -The smaller BERT models are intended for environments with restricted computational resources. They can be fine-tuned in the same manner as the original BERT models. However, they are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher. - -## Details of the downstream task (Q&A) - Dataset - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py) - -## Results: - -| Metric | # Value | -| ------ | --------- | -| **EM** | **65.95** | -| **F1** | **70.11** | - -### Raw metrics from benchmark included in training script: - -```json -{ - "exact": 65.95637159942727, - "f1": 70.11632254245896, - "total": 11873, - "HasAns_exact": 67.79689608636977, - "HasAns_f1": 76.12872765631123, - "HasAns_total": 5928, - "NoAns_exact": 64.12111017661901, - "NoAns_f1": 64.12111017661901, - "NoAns_total": 5945, - "best_exact": 65.96479407058031, - "best_exact_thresh": 0.0, - "best_f1": 70.12474501361196, - "best_f1_thresh": 0.0 -} -``` - -## Comparison: - -| Model | EM | F1 score | SIZE (MB) | -| --------------------------------------------------------------------------------------------- | --------- | --------- | --------- | -| [bert-tiny-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-finetuned-squadv2) | 48.60 | 49.73 | **16.74** | -| [bert-tiny-5-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-5-finetuned-squadv2) | 57.12 | 60.86 | 24.34 | -| [bert-mini-finetuned-squadv2](https://huggingface.co/mrm8488/bert-mini-finetuned-squadv2) | 56.31 | 59.65 | 42.63 | -| [bert-mini-5-finetuned-squadv2](https://huggingface.co/mrm8488/bert-mini-5-finetuned-squadv2) | 63.51 | 66.78 | 66.76 | -| [bert-small-finetuned-squadv2](https://huggingface.co/mrm8488/bert-small-finetuned-squadv2) | 60.49 | 64.21 | 109.74 | -| [bert-medium-finetuned-squadv2](https://huggingface.co/mrm8488/bert-medium-finetuned-squadv2) | **65.95** | **70.11** | 157.46 | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/bert-small-finetuned-squadv2", - tokenizer="mrm8488/bert-small-finetuned-squadv2" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) - -# Output: -``` - -```json -{ - "answer": "Manuel Romero", - "end": 13, - "score": 0.9939319924374637, - "start": 0 -} -``` - -### Yes! That was easy 🎉 Let's try with another example - -```python -qa_pipeline({ - 'context': "Manuel Romero has been working remotely in the repository hugginface/transformers lately", - 'question': "How has been working Manuel Romero?" -}) - -# Output: -``` - -```json -{ "answer": "remotely", "end": 39, "score": 0.3612058272768017, "start": 31 } -``` - -### It works!! 🎉 🎉 🎉 - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-mini-finetuned-squadv2/README.md b/model_cards/mrm8488/bert-mini-finetuned-squadv2/README.md deleted file mode 100644 index dd74a65f88..0000000000 --- a/model_cards/mrm8488/bert-mini-finetuned-squadv2/README.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -language: en -thumbnail: ---- - -# BERT-Mini fine-tuned on SQuAD v2 - -[BERT-Mini](https://github.com/google-research/bert/) created by [Google Research](https://github.com/google-research) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -**Mode size** (after training): **42.63 MB** - -## Details of BERT-Mini and its 'family' (from their documentation) - -Released on March 11th, 2020 - -This is model is a part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962). - -The smaller BERT models are intended for environments with restricted computational resources. They can be fine-tuned in the same manner as the original BERT models. However, they are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher. - -## Details of the downstream task (Q&A) - Dataset - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py) - -## Results: - -| Metric | # Value | -| ------ | --------- | -| **EM** | **56.31** | -| **F1** | **59.65** | - -## Comparison: - -| Model | EM | F1 score | SIZE (MB) | -| ----------------------------------------------------------------------------------------- | --------- | --------- | --------- | -| [bert-tiny-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-finetuned-squadv2) | 48.60 | 49.73 | **16.74** | -| [bert-tiny-5-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-5-finetuned-squadv2) | 57.12 | 60.86 | 24.34 | -| [bert-mini-finetuned-squadv2](https://huggingface.co/mrm8488/bert-mini-finetuned-squadv2) | 56.31 | 59.65 | 42.63 | -| [bert-mini-5-finetuned-squadv2](https://huggingface.co/mrm8488/bert-mini-5-finetuned-squadv2) | **63.51** | **66.78** | 66.76 | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/bert-mini-finetuned-squadv2", - tokenizer="mrm8488/bert-mini-finetuned-squadv2" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) - -# Output: -``` - -```json -{ - "answer": "Manuel Romero", - "end": 13, - "score": 0.9676484207783673, - "start": 0 -} -``` - -### Yes! That was easy 🎉 Let's try with another example - -```python -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "For which company has worked Manuel Romero?" -}) - -# Output: -``` - -```json -{ - "answer": "hugginface/transformers", - "end": 79, - "score": 0.5301655914731853, - "start": 56 -} -``` - -### It works!! 🎉 🎉 🎉 - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization/README.md b/model_cards/mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization/README.md deleted file mode 100644 index f5589cc1da..0000000000 --- a/model_cards/mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- cnn_dailymail -tags: -- summarization ---- - -# Bert-mini2Bert-mini Summarization with 🤗EncoderDecoder Framework - -This model is a warm-started *BERT2BERT* ([mini](https://huggingface.co/google/bert_uncased_L-4_H-256_A-4)) model fine-tuned on the *CNN/Dailymail* summarization dataset. - -The model achieves a **16.51** ROUGE-2 score on *CNN/Dailymail*'s test dataset. - -For more details on how the model was fine-tuned, please refer to -[this](https://colab.research.google.com/drive/1Ekd5pUeCX7VOrMx94_czTkwNtLN32Uyu?usp=sharing) notebook. - -## Results on test set 📝 - -| Metric | # Value | -| ------ | --------- | -| **ROUGE-2** | **16.51** | - - - -## Model in Action 🚀 - -```python -from transformers import BertTokenizerFast, EncoderDecoderModel -import torch -device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') -tokenizer = BertTokenizerFast.from_pretrained('mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization') -model = EncoderDecoderModel.from_pretrained('mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization').to(device) - -def generate_summary(text): - # cut off at BERT max length 512 - inputs = tokenizer([text], padding="max_length", truncation=True, max_length=512, return_tensors="pt") - input_ids = inputs.input_ids.to(device) - attention_mask = inputs.attention_mask.to(device) - - output = model.generate(input_ids, attention_mask=attention_mask) - - return tokenizer.decode(output[0], skip_special_tokens=True) - -text = "your text to be summarized here..." -generate_summary(text) -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-multi-cased-finedtuned-xquad-tydiqa-goldp/README.md b/model_cards/mrm8488/bert-multi-cased-finedtuned-xquad-tydiqa-goldp/README.md deleted file mode 100644 index 45a08bd6dd..0000000000 --- a/model_cards/mrm8488/bert-multi-cased-finedtuned-xquad-tydiqa-goldp/README.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -language: multilingual -thumbnail: ---- - -# A fine-tuned model on GoldP task from Tydi QA dataset - -This model uses [bert-multi-cased-finetuned-xquadv1](https://huggingface.co/mrm8488/bert-multi-cased-finetuned-xquadv1) and fine-tuned on [Tydi QA](https://github.com/google-research-datasets/tydiqa) dataset for Gold Passage task [(GoldP)](https://github.com/google-research-datasets/tydiqa#the-tasks) - -## Details of the language model -The base language model [(bert-multi-cased-finetuned-xquadv1)](https://huggingface.co/mrm8488/bert-multi-cased-finetuned-xquadv1) is a fine-tuned version of [bert-base-multilingual-cased](https://huggingface.co/bert-base-multilingual-cased) for the **Q&A** downstream task - - -## Details of the Tydi QA dataset - -TyDi QA contains 200k human-annotated question-answer pairs in 11 Typologically Diverse languages, written without seeing the answer and without the use of translation, and is designed for the **training and evaluation** of automatic question answering systems. This repository provides evaluation code and a baseline system for the dataset. https://ai.google.com/research/tydiqa - - -## Details of the downstream task (Gold Passage or GoldP aka the secondary task) - -Given a passage that is guaranteed to contain the answer, predict the single contiguous span of characters that answers the question. the gold passage task differs from the [primary task](https://github.com/google-research-datasets/tydiqa/blob/master/README.md#the-tasks) in several ways: -* only the gold answer passage is provided rather than the entire Wikipedia article; -* unanswerable questions have been discarded, similar to MLQA and XQuAD; -* we evaluate with the SQuAD 1.1 metrics like XQuAD; and -* Thai and Japanese are removed since the lack of whitespace breaks some tools. - - -## Model training - -The model was fine-tuned on a Tesla P100 GPU and 25GB of RAM. -The script is the following: - -```python -python run_squad.py \ - --model_type bert \ - --model_name_or_path mrm8488/bert-multi-cased-finetuned-xquadv1 \ - --do_train \ - --do_eval \ - --train_file /content/dataset/train.json \ - --predict_file /content/dataset/dev.json \ - --per_gpu_train_batch_size 24 \ - --per_gpu_eval_batch_size 24 \ - --learning_rate 3e-5 \ - --num_train_epochs 2.5 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/model_output \ - --overwrite_output_dir \ - --save_steps 5000 \ - --threads 40 - ``` - -## Global Results (dev set): - -| Metric | # Value | -| --------- | ----------- | -| **Exact** | **71.06** | -| **F1** | **82.16** | - -## Specific Results (per language): - -| Language | # Samples | # Exact | # F1 | -| --------- | ----------- |--------| ------ | -| Arabic | 1314 | 73.29 | 84.72 | -| Bengali | 180 | 64.60 | 77.84 | -| English | 654 | 72.12 | 82.24 | -| Finnish | 1031 | 70.14 | 80.36 | -| Indonesian| 773 | 77.25 | 86.36 | -| Korean | 414 | 68.92 | 70.95 | -| Russian | 1079 | 62.65 | 78.55 | -| Swahili | 596 | 80.11 | 86.18 | -| Telegu | 874 | 71.00 | 84.24 | - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-multi-cased-finetuned-xquadv1/README.md b/model_cards/mrm8488/bert-multi-cased-finetuned-xquadv1/README.md deleted file mode 100644 index 7849ec85f1..0000000000 --- a/model_cards/mrm8488/bert-multi-cased-finetuned-xquadv1/README.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -language: multilingual -thumbnail: ---- - -# BERT (base-multilingual-cased) fine-tuned for multilingual Q&A - -This model was created by [Google](https://github.com/google-research/bert/blob/master/multilingual.md) and fine-tuned on [XQuAD](https://github.com/deepmind/xquad) like data for multilingual (`11 different languages`) **Q&A** downstream task. - -## Details of the language model('bert-base-multilingual-cased') - -[Language model](https://github.com/google-research/bert/blob/master/multilingual.md) - -| Languages | Heads | Layers | Hidden | Params | -| --------- | ----- | ------ | ------ | ------ | -| 104 | 12 | 12 | 768 | 100 M | - -## Details of the downstream task (multilingual Q&A) - Dataset - -Deepmind [XQuAD](https://github.com/deepmind/xquad) - -Languages covered: - -- Arabic: `ar` -- German: `de` -- Greek: `el` -- English: `en` -- Spanish: `es` -- Hindi: `hi` -- Russian: `ru` -- Thai: `th` -- Turkish: `tr` -- Vietnamese: `vi` -- Chinese: `zh` - -As the dataset is based on SQuAD v1.1, there are no unanswerable questions in the data. We chose this -setting so that models can focus on cross-lingual transfer. - -We show the average number of tokens per paragraph, question, and answer for each language in the -table below. The statistics were obtained using [Jieba](https://github.com/fxsjy/jieba) for Chinese -and the [Moses tokenizer](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/tokenizer/tokenizer.perl) -for the other languages. - -| | en | es | de | el | ru | tr | ar | vi | th | zh | hi | -| --------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| Paragraph | 142.4 | 160.7 | 139.5 | 149.6 | 133.9 | 126.5 | 128.2 | 191.2 | 158.7 | 147.6 | 232.4 | -| Question | 11.5 | 13.4 | 11.0 | 11.7 | 10.0 | 9.8 | 10.7 | 14.8 | 11.5 | 10.5 | 18.7 | -| Answer | 3.1 | 3.6 | 3.0 | 3.3 | 3.1 | 3.1 | 3.1 | 4.5 | 4.1 | 3.5 | 5.6 | - -Citation: - -
- -```bibtex -@article{Artetxe:etal:2019, - author = {Mikel Artetxe and Sebastian Ruder and Dani Yogatama}, - title = {On the cross-lingual transferability of monolingual representations}, - journal = {CoRR}, - volume = {abs/1910.11856}, - year = {2019}, - archivePrefix = {arXiv}, - eprint = {1910.11856} -} -``` - -
- -As **XQuAD** is just an evaluation dataset, I used `Data augmentation techniques` (scraping, neural machine translation, etc) to obtain more samples and split the dataset in order to have a train and test set. The test set was created in a way that contains the same number of samples for each language. Finally, I got: - -| Dataset | # samples | -| ----------- | --------- | -| XQUAD train | 50 K | -| XQUAD test | 8 K | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/distillation/run_squad_w_distillation.py) - - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/bert-multi-cased-finetuned-xquadv1", - tokenizer="mrm8488/bert-multi-cased-finetuned-xquadv1" -) - - -# context: Coronavirus is seeding panic in the West because it expands so fast. - -# question: Where is seeding panic Coronavirus? -qa_pipeline({ - 'context': "कोरोनावायरस पश्चिम में आतंक बो रहा है क्योंकि यह इतनी तेजी से फैलता है।", - 'question': "कोरोनावायरस घबराहट कहां है?" - -}) -# output: {'answer': 'पश्चिम', 'end': 18, 'score': 0.7037217439689059, 'start': 12} - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) -# output: {'answer': 'Manuel Romero', 'end': 13, 'score': 0.7254485993702389, 'start': 0} - -qa_pipeline({ - 'context': "Manuel Romero a travaillé à peine dans le référentiel hugginface / transformers ces derniers temps", - 'question': "Pour quel référentiel a travaillé Manuel Romero récemment?" - -}) -#output: {'answer': 'hugginface / transformers', 'end': 79, 'score': 0.6482061613915384, 'start': 54} -``` -![model in action](https://media.giphy.com/media/MBlire8Wj7ng73VBQ5/giphy.gif) - -Try it on a Colab: - -Open In Colab - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-multi-uncased-finetuned-xquadv1/README.md b/model_cards/mrm8488/bert-multi-uncased-finetuned-xquadv1/README.md deleted file mode 100644 index f04c569885..0000000000 --- a/model_cards/mrm8488/bert-multi-uncased-finetuned-xquadv1/README.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -language: multilingual -thumbnail: ---- - -# BERT (base-multilingual-uncased) fine-tuned for multilingual Q&A - -This model was created by [Google](https://github.com/google-research/bert/blob/master/multilingual.md) and fine-tuned on [XQuAD](https://github.com/deepmind/xquad) like data for multilingual (`11 different languages`) **Q&A** downstream task. - -## Details of the language model('bert-base-multilingual-uncased') - -[Language model](https://github.com/google-research/bert/blob/master/multilingual.md) - -| Languages | Heads | Layers | Hidden | Params | -| --------- | ----- | ------ | ------ | ------ | -| 102 | 12 | 12 | 768 | 100 M | - -## Details of the downstream task (multilingual Q&A) - Dataset - -Deepmind [XQuAD](https://github.com/deepmind/xquad) - -Languages covered: - -- Arabic: `ar` -- German: `de` -- Greek: `el` -- English: `en` -- Spanish: `es` -- Hindi: `hi` -- Russian: `ru` -- Thai: `th` -- Turkish: `tr` -- Vietnamese: `vi` -- Chinese: `zh` - -As the dataset is based on SQuAD v1.1, there are no unanswerable questions in the data. We chose this -setting so that models can focus on cross-lingual transfer. - -We show the average number of tokens per paragraph, question, and answer for each language in the -table below. The statistics were obtained using [Jieba](https://github.com/fxsjy/jieba) for Chinese -and the [Moses tokenizer](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/tokenizer/tokenizer.perl) -for the other languages. - -| | en | es | de | el | ru | tr | ar | vi | th | zh | hi | -| --------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| Paragraph | 142.4 | 160.7 | 139.5 | 149.6 | 133.9 | 126.5 | 128.2 | 191.2 | 158.7 | 147.6 | 232.4 | -| Question | 11.5 | 13.4 | 11.0 | 11.7 | 10.0 | 9.8 | 10.7 | 14.8 | 11.5 | 10.5 | 18.7 | -| Answer | 3.1 | 3.6 | 3.0 | 3.3 | 3.1 | 3.1 | 3.1 | 4.5 | 4.1 | 3.5 | 5.6 | - -Citation: - -
- -```bibtex -@article{Artetxe:etal:2019, - author = {Mikel Artetxe and Sebastian Ruder and Dani Yogatama}, - title = {On the cross-lingual transferability of monolingual representations}, - journal = {CoRR}, - volume = {abs/1910.11856}, - year = {2019}, - archivePrefix = {arXiv}, - eprint = {1910.11856} -} -``` - -
- -As **XQuAD** is just an evaluation dataset, I used `Data augmentation techniques` (scraping, neural machine translation, etc) to obtain more samples and split the dataset in order to have a train and test set. The test set was created in a way that contains the same number of samples for each language. Finally, I got: - -| Dataset | # samples | -| ----------- | --------- | -| XQUAD train | 50 K | -| XQUAD test | 8 K | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/distillation/run_squad_w_distillation.py) - - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/bert-multi-uncased-finetuned-xquadv1", - tokenizer="mrm8488/bert-multi-uncased-finetuned-xquadv1" -) - - -# context: Coronavirus is seeding panic in the West because it expands so fast. - -# question: Where is seeding panic Coronavirus? -qa_pipeline({ - 'context': "कोरोनावायरस पश्चिम में आतंक बो रहा है क्योंकि यह इतनी तेजी से फैलता है।", - 'question': "कोरोनावायरस घबराहट कहां है?" - -}) -# output: {'answer': 'पश्चिम', 'end': 18, 'score': 0.7037217439689059, 'start': 12} - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) -# output: {'answer': 'Manuel Romero', 'end': 13, 'score': 0.7254485993702389, 'start': 0} - -qa_pipeline({ - 'context': "Manuel Romero a travaillé à peine dans le référentiel hugginface / transformers ces derniers temps", - 'question': "Pour quel référentiel a travaillé Manuel Romero récemment?" - -}) -#output: {'answer': 'hugginface / transformers', 'end': 79, 'score': 0.6482061613915384, 'start': 54} -``` -![model in action](https://media.giphy.com/media/MBlire8Wj7ng73VBQ5/giphy.gif) - -Try it on a Colab: - -Open In Colab - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-small-finetuned-squadv2/README.md b/model_cards/mrm8488/bert-small-finetuned-squadv2/README.md deleted file mode 100644 index 0a4811c007..0000000000 --- a/model_cards/mrm8488/bert-small-finetuned-squadv2/README.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -language: en -thumbnail: ---- - -# BERT-Small fine-tuned on SQuAD v2 - -[BERT-Small](https://github.com/google-research/bert/) created by [Google Research](https://github.com/google-research) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -**Mode size** (after training): **109.74 MB** - -## Details of BERT-Small and its 'family' (from their documentation) - -Released on March 11th, 2020 - -This is model is a part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962). - -The smaller BERT models are intended for environments with restricted computational resources. They can be fine-tuned in the same manner as the original BERT models. However, they are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher. - -## Details of the downstream task (Q&A) - Dataset - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py) - -## Results: - -| Metric | # Value | -| ------ | --------- | -| **EM** | **60.49** | -| **F1** | **64.21** | - -## Comparison: - -| Model | EM | F1 score | SIZE (MB) | -| ------------------------------------------------------------------------------------------- | --------- | --------- | --------- | -| [bert-tiny-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-finetuned-squadv2) | 48.60 | 49.73 | **16.74** | -| [bert-mini-finetuned-squadv2](https://huggingface.co/mrm8488/bert-mini-finetuned-squadv2) | 56.31 | 59.65 | 42.63 | -| [bert-small-finetuned-squadv2](https://huggingface.co/mrm8488/bert-small-finetuned-squadv2) | **60.49** | **64.21** | 109.74 | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/bert-small-finetuned-squadv2", - tokenizer="mrm8488/bert-small-finetuned-squadv2" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) - -# Output: -``` - -```json -{ - "answer": "Manuel Romero", - "end": 13, - "score": 0.9939319924374637, - "start": 0 -} -``` - -### Yes! That was easy 🎉 Let's try with another example - -```python -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "For which company has worked Manuel Romero?" -}) - -# Output: -``` - -```json -{ - "answer": "hugginface/transformers", - "end": 79, - "score": 0.6024888734447131, - "start": 56 -} -``` - -### It works!! 🎉 🎉 🎉 - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-small-finetuned-typo-detection/README.md b/model_cards/mrm8488/bert-small-finetuned-typo-detection/README.md deleted file mode 100644 index 1e2c83436a..0000000000 --- a/model_cards/mrm8488/bert-small-finetuned-typo-detection/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -language: en -thumbnail: ---- - -# BERT SMALL + Typo Detection ✍❌✍✔ - -[BERT SMALL](https://huggingface.co/google/bert_uncased_L-4_H-512_A-8) fine-tuned on [GitHub Typo Corpus](https://github.com/mhagiwara/github-typo-corpus) for **typo detection** (using *NER* style) - -## Details of the downstream task (Typo detection as NER) - -- Dataset: [GitHub Typo Corpus](https://github.com/mhagiwara/github-typo-corpus) 📚 - -- [Fine-tune script on NER dataset provided by Huggingface](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_ner_old.py) 🏋️‍♂️ - -## Metrics on test set 📋 - -| Metric | # score | -| :-------: | :-------: | -| F1 | **89.12** | -| Precision | **93.82** | -| Recall | **84.87** | - -## Model in action 🔨 - -Fast usage with **pipelines** 🧪 - -```python -from transformers import pipeline - -typo_checker = pipeline( - "ner", - model="mrm8488/bert-small-finetuned-typo-detection", - tokenizer="mrm8488/bert-small-finetuned-typo-detection" -) - -result = typo_checker("here there is an error in coment") -result[1:-1] - -# Output: -[{'entity': 'ok', 'score': 0.9021041989326477, 'word': 'here'}, - {'entity': 'ok', 'score': 0.7975626587867737, 'word': 'there'}, - {'entity': 'ok', 'score': 0.8596242070198059, 'word': 'is'}, - {'entity': 'ok', 'score': 0.7071516513824463, 'word': 'an'}, - {'entity': 'ok', 'score': 0.943381130695343, 'word': 'error'}, - {'entity': 'ok', 'score': 0.8047608733177185, 'word': 'in'}, - {'entity': 'ok', 'score': 0.8240702152252197, 'word': 'come'}, - {'entity': 'typo', 'score': 0.5004884004592896, 'word': '##nt'}] -``` - -It works🎉! we typed ```coment``` instead of ```comment``` - -Let's try with another example - -```python -result = typo_checker("Adddd validation midelware") -result[1:-1] - -# Output: -[{'entity': 'ok', 'score': 0.7128152847290039, 'word': 'add'}, - {'entity': 'typo', 'score': 0.5388424396514893, 'word': '##dd'}, - {'entity': 'ok', 'score': 0.94792640209198, 'word': 'validation'}, - {'entity': 'typo', 'score': 0.5839331746101379, 'word': 'mid'}, - {'entity': 'ok', 'score': 0.5195121765136719, 'word': '##el'}, - {'entity': 'ok', 'score': 0.7222476601600647, 'word': '##ware'}] -``` -Yeah! We typed wrong ```Add and middleware``` - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization/README.md b/model_cards/mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization/README.md deleted file mode 100644 index 7db7b72504..0000000000 --- a/model_cards/mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- cnn_dailymail -tags: -- summarization ---- - -# Bert-small2Bert-small Summarization with 🤗EncoderDecoder Framework - -This model is a warm-started *BERT2BERT* ([small](https://huggingface.co/google/bert_uncased_L-4_H-512_A-8)) model fine-tuned on the *CNN/Dailymail* summarization dataset. - -The model achieves a **17.37** ROUGE-2 score on *CNN/Dailymail*'s test dataset. - -For more details on how the model was fine-tuned, please refer to -[this](https://colab.research.google.com/drive/1Ekd5pUeCX7VOrMx94_czTkwNtLN32Uyu?usp=sharing) notebook. - -## Results on test set 📝 - -| Metric | # Value | -| ------ | --------- | -| **ROUGE-2** | **17.37** | - - - -## Model in Action 🚀 - -```python -from transformers import BertTokenizerFast, EncoderDecoderModel -import torch -device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') -tokenizer = BertTokenizerFast.from_pretrained('mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization') -model = EncoderDecoderModel.from_pretrained('mrm8488/bert-small2bert-small-finetuned-cnn_daily_mail-summarization').to(device) - -def generate_summary(text): - # cut off at BERT max length 512 - inputs = tokenizer([text], padding="max_length", truncation=True, max_length=512, return_tensors="pt") - input_ids = inputs.input_ids.to(device) - attention_mask = inputs.attention_mask.to(device) - - output = model.generate(input_ids, attention_mask=attention_mask) - - return tokenizer.decode(output[0], skip_special_tokens=True) - -text = "your text to be summarized here..." -generate_summary(text) -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-spanish-cased-finetuned-ner/README.md b/model_cards/mrm8488/bert-spanish-cased-finetuned-ner/README.md deleted file mode 100644 index 4468b57f97..0000000000 --- a/model_cards/mrm8488/bert-spanish-cased-finetuned-ner/README.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -language: es -thumbnail: https://i.imgur.com/jgBdimh.png ---- - -# Spanish BERT (BETO) + NER - -This model is a fine-tuned on [NER-C](https://www.kaggle.com/nltkdata/conll-corpora) version of the Spanish BERT cased [(BETO)](https://github.com/dccuchile/beto) for **NER** downstream task. - -## Details of the downstream task (NER) - Dataset - -- [Dataset: CONLL Corpora ES](https://www.kaggle.com/nltkdata/conll-corpora) - -I preprocessed the dataset and split it as train / dev (80/20) - -| Dataset | # Examples | -| ---------------------- | ----- | -| Train | 8.7 K | -| Dev | 2.2 K | - - -- [Fine-tune on NER script provided by Huggingface](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_ner_old.py) - -- Labels covered: - -``` -B-LOC -B-MISC -B-ORG -B-PER -I-LOC -I-MISC -I-ORG -I-PER -O -``` - -## Metrics on evaluation set: - -| Metric | # score | -| :------------------------------------------------------------------------------------: | :-------: | -| F1 | **90.17** -| Precision | **89.86** | -| Recall | **90.47** | - -## Comparison: - -| Model | # F1 score |Size(MB)| -| :--------------------------------------------------------------------------------------------------------------: | :-------: |:------| -| bert-base-spanish-wwm-cased (BETO) | 88.43 | 421 -| [bert-spanish-cased-finetuned-ner (this one)](https://huggingface.co/mrm8488/bert-spanish-cased-finetuned-ner) | **90.17** | 420 | -| Best Multilingual BERT | 87.38 | 681 | -|[TinyBERT-spanish-uncased-finetuned-ner](https://huggingface.co/mrm8488/TinyBERT-spanish-uncased-finetuned-ner) | 70.00 | **55** | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -nlp_ner = pipeline( - "ner", - model="mrm8488/bert-spanish-cased-finetuned-ner", - tokenizer=( - 'mrm8488/bert-spanish-cased-finetuned-ner', - {"use_fast": False} -)) - -text = 'Mis amigos están pensando viajar a Londres este verano' - -nlp_ner(text) - -#Output: [{'entity': 'B-LOC', 'score': 0.9998720288276672, 'word': 'Londres'}] -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-spanish-cased-finetuned-pos-syntax/README.md b/model_cards/mrm8488/bert-spanish-cased-finetuned-pos-syntax/README.md deleted file mode 100644 index 54bb61e2b2..0000000000 --- a/model_cards/mrm8488/bert-spanish-cased-finetuned-pos-syntax/README.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -language: es -thumbnail: ---- - -# Spanish BERT (BETO) + Syntax POS tagging ✍🏷 - -This model is a fine-tuned version of the Spanish BERT [(BETO)](https://github.com/dccuchile/beto) on Spanish **syntax** annotations in [CONLL CORPORA](https://www.kaggle.com/nltkdata/conll-corpora) dataset for **syntax POS** (Part of Speech tagging) downstream task. - -## Details of the downstream task (Syntax POS) - Dataset - -- [Dataset: CONLL Corpora ES](https://www.kaggle.com/nltkdata/conll-corpora) - -#### [Fine-tune script on NER dataset provided by Huggingface](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_ner_old.py) - -#### 21 Syntax annotations (Labels) covered: - -- \_ -- ATR -- ATR.d -- CAG -- CC -- CD -- CD.Q -- CI -- CPRED -- CPRED.CD -- CPRED.SUJ -- CREG -- ET -- IMPERS -- MOD -- NEG -- PASS -- PUNC -- ROOT -- SUJ -- VOC - -## Metrics on test set 📋 - -| Metric | # score | -| :-------: | :-------: | -| F1 | **89.27** | -| Precision | **89.44** | -| Recall | **89.11** | - -## Model in action 🔨 - -Fast usage with **pipelines** 🧪 - -```python -from transformers import pipeline - -nlp_pos_syntax = pipeline( - "ner", - model="mrm8488/bert-spanish-cased-finetuned-pos-syntax", - tokenizer="mrm8488/bert-spanish-cased-finetuned-pos-syntax" -) - -text = 'Mis amigos están pensando viajar a Londres este verano.' - -nlp_pos_syntax(text)[1:len(nlp_pos_syntax(text))-1] -``` - -```json -[ - { "entity": "_", "score": 0.9999216794967651, "word": "Mis" }, - { "entity": "SUJ", "score": 0.999882698059082, "word": "amigos" }, - { "entity": "_", "score": 0.9998869299888611, "word": "están" }, - { "entity": "ROOT", "score": 0.9980518221855164, "word": "pensando" }, - { "entity": "_", "score": 0.9998420476913452, "word": "viajar" }, - { "entity": "CD", "score": 0.999351978302002, "word": "a" }, - { "entity": "_", "score": 0.999959409236908, "word": "Londres" }, - { "entity": "_", "score": 0.9998968839645386, "word": "este" }, - { "entity": "CC", "score": 0.99931401014328, "word": "verano" }, - { "entity": "PUNC", "score": 0.9998534917831421, "word": "." } -] -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-spanish-cased-finetuned-pos/README.md b/model_cards/mrm8488/bert-spanish-cased-finetuned-pos/README.md deleted file mode 100644 index 356dd0f5ab..0000000000 --- a/model_cards/mrm8488/bert-spanish-cased-finetuned-pos/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: es -thumbnail: https://i.imgur.com/jgBdimh.png ---- - -# Spanish BERT (BETO) + POS - -This model is a fine-tuned on Spanish [CONLL CORPORA](https://www.kaggle.com/nltkdata/conll-corpora) version of the Spanish BERT cased [(BETO)](https://github.com/dccuchile/beto) for **POS** (Part of Speech tagging) downstream task. - -## Details of the downstream task (POS) - Dataset - -- [Dataset: CONLL Corpora ES](https://www.kaggle.com/nltkdata/conll-corpora) with data augmentation techniques - -I preprocessed the dataset and split it as train / dev (80/20) - -| Dataset | # Examples | -| ---------------------- | ----- | -| Train | 340 K | -| Dev | 50 K | - - -- [Fine-tune on NER script provided by Huggingface](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_ner_old.py) - -- **60** Labels covered: - -``` -AO, AQ, CC, CS, DA, DD, DE, DI, DN, DP, DT, Faa, Fat, Fc, Fd, Fe, Fg, Fh, Fia, Fit, Fp, Fpa, Fpt, Fs, Ft, Fx, Fz, I, NC, NP, P0, PD, PI, PN, PP, PR, PT, PX, RG, RN, SP, VAI, VAM, VAN, VAP, VAS, VMG, VMI, VMM, VMN, VMP, VMS, VSG, VSI, VSM, VSN, VSP, VSS, Y and Z -``` - - -## Metrics on evaluation set: - -| Metric | # score | -| :------------------------------------------------------------------------------------: | :-------: | -| F1 | **90.06** -| Precision | **89.46** | -| Recall | **90.67** | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -nlp_pos = pipeline( - "ner", - model="mrm8488/bert-spanish-cased-finetuned-pos", - tokenizer=( - 'mrm8488/bert-spanish-cased-finetuned-pos', - {"use_fast": False} -)) - - -text = 'Mis amigos están pensando en viajar a Londres este verano' - -nlp_pos(text) - -#Output: -''' -[{'entity': 'NC', 'score': 0.7792173624038696, 'word': '[CLS]'}, - {'entity': 'DP', 'score': 0.9996283650398254, 'word': 'Mis'}, - {'entity': 'NC', 'score': 0.9999253749847412, 'word': 'amigos'}, - {'entity': 'VMI', 'score': 0.9998560547828674, 'word': 'están'}, - {'entity': 'VMG', 'score': 0.9992249011993408, 'word': 'pensando'}, - {'entity': 'SP', 'score': 0.9999602437019348, 'word': 'en'}, - {'entity': 'VMN', 'score': 0.9998666048049927, 'word': 'viajar'}, - {'entity': 'SP', 'score': 0.9999545216560364, 'word': 'a'}, - {'entity': 'VMN', 'score': 0.8722310662269592, 'word': 'Londres'}, - {'entity': 'DD', 'score': 0.9995203614234924, 'word': 'este'}, - {'entity': 'NC', 'score': 0.9999248385429382, 'word': 'verano'}, - {'entity': 'NC', 'score': 0.8802427649497986, 'word': '[SEP]'}] - ''' -``` -![model in action](https://media.giphy.com/media/jVC9m1cNrdIWuAAtjy/giphy.gif) - -16 POS tags version also available [here](https://huggingface.co/mrm8488/bert-spanish-cased-finetuned-pos-16-tags) - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-tiny-finetuned-squadv2/README.md b/model_cards/mrm8488/bert-tiny-finetuned-squadv2/README.md deleted file mode 100644 index 9994683503..0000000000 --- a/model_cards/mrm8488/bert-tiny-finetuned-squadv2/README.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -language: en -thumbnail: ---- - -# BERT-Tiny fine-tuned on SQuAD v2 - -[BERT-Tiny](https://github.com/google-research/bert/) created by [Google Research](https://github.com/google-research) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -**Mode size** (after training): **16.74 MB** - -## Details of BERT-Tiny and its 'family' (from their documentation) - -Released on March 11th, 2020 - -This is model is a part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962). - -The smaller BERT models are intended for environments with restricted computational resources. They can be fine-tuned in the same manner as the original BERT models. However, they are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher. - -## Details of the downstream task (Q&A) - Dataset - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py) - -## Results: - -| Metric | # Value | -| ------ | --------- | -| **EM** | **48.60** | -| **F1** | **49.73** | - - -| Model | EM | F1 score | SIZE (MB) | -| ----------------------------------------------------------------------------------------- | --------- | --------- | --------- | -| [bert-tiny-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-finetuned-squadv2) | 48.60 | 49.73 | **16.74** | -| [bert-tiny-5-finetuned-squadv2](https://huggingface.co/mrm8488/bert-tiny-5-finetuned-squadv2) | **57.12** | **60.86** | 24.34 - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/bert-tiny-finetuned-squadv2", - tokenizer="mrm8488/bert-tiny-finetuned-squadv2" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) - -# Output: -``` - -```json -{ - "answer": "Manuel Romero", - "end": 13, - "score": 0.05684709993458714, - "start": 0 -} -``` - -### Yes! That was easy 🎉 Let's try with another example - -```python -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "For which company has worked Manuel Romero?" -}) - -# Output: -``` - -```json -{ - "answer": "hugginface/transformers", - "end": 79, - "score": 0.11613431826808274, - "start": 56 -} -``` - -### It works!! 🎉 🎉 🎉 - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/bert-uncased-finetuned-qnli/README.md b/model_cards/mrm8488/bert-uncased-finetuned-qnli/README.md deleted file mode 100644 index a932f8481c..0000000000 --- a/model_cards/mrm8488/bert-uncased-finetuned-qnli/README.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -language: en -thumbnail: ---- - -# [BERT](https://huggingface.co/deepset/bert-base-cased-squad2) fine tuned on [QNLI](https://github.com/rhythmcao/QNLI)+ compression ([BERT-of-Theseus](https://github.com/JetRunner/BERT-of-Theseus)) - -I used a [Bert model fine tuned on **SQUAD v2**](https://huggingface.co/deepset/bert-base-cased-squad2) and then I fine tuned it on **QNLI** using **compression** (with a constant replacing rate) as proposed in **BERT-of-Theseus** - -## Details of the downstream task (QNLI): - -### Getting the dataset -```bash -wget https://raw.githubusercontent.com/rhythmcao/QNLI/master/data/QNLI/train.tsv -wget https://raw.githubusercontent.com/rhythmcao/QNLI/master/data/QNLI/test.tsv -wget https://raw.githubusercontent.com/rhythmcao/QNLI/master/data/QNLI/dev.tsv - -mkdir QNLI_dataset -mv *.tsv QNLI_dataset -``` - -### Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -!python /content/BERT-of-Theseus/run_glue.py \ - --model_name_or_path deepset/bert-base-cased-squad2 \ - --task_name qnli \ - --do_train \ - --do_eval \ - --do_lower_case \ - --data_dir /content/QNLI_dataset \ - --max_seq_length 128 \ - --per_gpu_train_batch_size 32 \ - --per_gpu_eval_batch_size 32 \ - --learning_rate 2e-5 \ - --save_steps 2000 \ - --num_train_epochs 50 \ - --output_dir /content/ouput_dir \ - --evaluate_during_training \ - --replacing_rate 0.7 \ - --steps_for_replacing 2500 -``` - -## Metrics: - -| Model | Accuracy | -|-----------------|------| -| BERT-base | 91.2 | -| BERT-of-Theseus | 88.8 | -| [bert-uncased-finetuned-qnli](https://huggingface.co/mrm8488/bert-uncased-finetuned-qnli) | 87.2 -| DistillBERT | 85.3 | - - - - -> [See all my models](https://huggingface.co/models?search=mrm8488) - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/camembert-base-finetuned-pawsx-fr/README.md b/model_cards/mrm8488/camembert-base-finetuned-pawsx-fr/README.md deleted file mode 100644 index 01e65e3ffa..0000000000 --- a/model_cards/mrm8488/camembert-base-finetuned-pawsx-fr/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -language: fr -datasets: -- xtreme -widget: -- text: "La première série a été mieux reçue par la critique que la seconde. La seconde série a été bien accueillie par la critique, mieux que la première." ---- - -# Camembert-base fine-tuned on PAWS-X-fr for Paraphrase Identification diff --git a/model_cards/mrm8488/chEMBL_smiles_v1/README.md b/model_cards/mrm8488/chEMBL_smiles_v1/README.md deleted file mode 100644 index 4817abea74..0000000000 --- a/model_cards/mrm8488/chEMBL_smiles_v1/README.md +++ /dev/null @@ -1,77 +0,0 @@ -# *De Novo* Drug Design with MLM - -## What is it? - -An approximation to [Generative Recurrent Networks for De Novo Drug Design](https://onlinelibrary.wiley.com/doi/full/10.1002/minf.201700111) but training a MLM (RoBERTa like) from scratch. - -## Why? - -As mentioned in the paper: -Generative artificial intelligence models present a fresh approach to chemogenomics and de novo drug design, as they provide researchers with the ability to narrow down their search of the chemical space and focus on regions of interest. -They used a generative *recurrent neural network (RNN)* containing long short‐term memory (LSTM) cell to capture the syntax of molecular representations in terms of SMILES strings. -The learned pattern probabilities can be used for de novo SMILES generation. This molecular design concept **eliminates the need for virtual compound library enumeration** and **enables virtual compound design without requiring secondary or external activity prediction**. - - -## My Goal 🎯 - -By training a MLM from scratch on 438552 (cleaned*) SMILES I wanted to build a model that learns this kind of molecular combinations so that given a partial SMILE it can generate plausible combinations so that it can be proposed as new drugs. -By cleaned SMILES I mean that I used their [SMILES cleaning script](https://github.com/topazape/LSTM_Chem/blob/master/cleanup_smiles.py) to remove duplicates, salts, and stereochemical information. -You can see the detailed process of gathering the data, preprocess it and train the LSTM in their [repo](https://github.com/topazape/LSTM_Chem). - -## Fast usage with ```pipelines``` 🧪 - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model='/mrm8488/chEMBL_smiles_v1', - tokenizer='/mrm8488/chEMBL_smiles_v1' -) - -# CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)cc1 Atazanavir -smile1 = "CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)" - -fill_mask(smile1) - -# Output: -''' -[{'score': 0.6040295958518982, - 'sequence': ' CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)nc', - 'token': 265}, - {'score': 0.2185731679201126, - 'sequence': ' CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)N', - 'token': 50}, - {'score': 0.0642734169960022, - 'sequence': ' CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)cc', - 'token': 261}, - {'score': 0.01932266168296337, - 'sequence': ' CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)CCCl', - 'token': 452}, - {'score': 0.005068355705589056, - 'sequence': ' CC(C)CN(CC(OP(=O)(O)O)C(Cc1ccccc1)NC(=O)OC1CCOC1)S(=O)(=O)c1ccc(N)C', - 'token': 39}] - ''' - ``` - ## More - I also created a [second version](https://huggingface.co/mrm8488/chEMBL26_smiles_v2) without applying the cleaning SMILES script mentioned above. You can use it in the same way as this one. - - ```python - fill_mask = pipeline( - "fill-mask", - model='/mrm8488/chEMBL26_smiles_v2', - tokenizer='/mrm8488/chEMBL26_smiles_v2' -) -``` - - [Original paper](https://www.ncbi.nlm.nih.gov/pubmed/29095571) Authors: -
-Swiss Federal Institute of Technology (ETH), Department of Chemistry and Applied Biosciences, Vladimir–Prelog–Weg 4, 8093, Zurich, Switzerland, -Stanford University, Department of Computer Science, 450 Sierra Mall, Stanford, CA, 94305, USA, -inSili.com GmbH, 8049, Zurich, Switzerland, -Gisbert Schneider, Email: hc.zhte@trebsig. -
- - > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/codeBERTaJS/README.md b/model_cards/mrm8488/codeBERTaJS/README.md deleted file mode 100644 index 6da87c7fe3..0000000000 --- a/model_cards/mrm8488/codeBERTaJS/README.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -language: code -thumbnail: ---- - -# CodeBERTaJS - -CodeBERTaJS is a RoBERTa-like model trained on the [CodeSearchNet](https://github.blog/2019-09-26-introducing-the-codesearchnet-challenge/) dataset from GitHub for `javaScript` by [Manuel Romero](https://twitter.com/mrm8488) - -The **tokenizer** is a Byte-level BPE tokenizer trained on the corpus using Hugging Face `tokenizers`. - -Because it is trained on a corpus of code (vs. natural language), it encodes the corpus efficiently (the sequences are between 33% to 50% shorter, compared to the same corpus tokenized by gpt2/roberta). - -The (small) **model** is a 6-layer, 84M parameters, RoBERTa-like Transformer model – that’s the same number of layers & heads as DistilBERT – initialized from the default initialization settings and trained from scratch on the full `javascript` corpus (120M after preproccessing) for 2 epochs. - -## Quick start: masked language modeling prediction - -```python -JS_CODE = """ -async function createUser(req, ) { - if (!validUser(req.body.user)) { - return res.status(400); - } - user = userService.createUser(req.body.user); - return res.json(user); -} -""".lstrip() -``` - -### Does the model know how to complete simple JS/express like code? - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="mrm8488/codeBERTaJS", - tokenizer="mrm8488/codeBERTaJS" -) - -fill_mask(JS_CODE) - -## Top 5 predictions: -# -'res' # prob 0.069489665329 -'next' -'req' -'user' -',req' -``` - -### Yes! That was easy 🎉 Let's try with another example - -```python -JS_CODE_= """ -function getKeys(obj) { - keys = []; - for (var [key, value] of Object.entries(obj)) { - keys.push(); - } - return keys -} -""".lstrip() -``` - -Results: - -```python -'obj', 'key', ' value', 'keys', 'i' -``` - -> Not so bad! Right token was predicted as second option! 🎉 - -## This work is heavely inspired on [codeBERTa](https://github.com/huggingface/transformers/blob/master/model_cards/huggingface/CodeBERTa-small-v1/README.md) by huggingface team - -
- -## CodeSearchNet citation - -
- -```bibtex -@article{husain_codesearchnet_2019, - title = {{CodeSearchNet} {Challenge}: {Evaluating} the {State} of {Semantic} {Code} {Search}}, - shorttitle = {{CodeSearchNet} {Challenge}}, - url = {http://arxiv.org/abs/1909.09436}, - urldate = {2020-03-12}, - journal = {arXiv:1909.09436 [cs, stat]}, - author = {Husain, Hamel and Wu, Ho-Hsiang and Gazit, Tiferet and Allamanis, Miltiadis and Brockschmidt, Marc}, - month = sep, - year = {2019}, - note = {arXiv: 1909.09436}, -} -``` - -
- -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/codebert-base-finetuned-detect-insecure-code/README.md b/model_cards/mrm8488/codebert-base-finetuned-detect-insecure-code/README.md deleted file mode 100644 index 68906e3c67..0000000000 --- a/model_cards/mrm8488/codebert-base-finetuned-detect-insecure-code/README.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -language: en -datasets: -- codexglue ---- - -# CodeBERT fine-tuned for Insecure Code Detection 💾⛔ - - -[codebert-base](https://huggingface.co/microsoft/codebert-base) fine-tuned on [CodeXGLUE -- Defect Detection](https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/Defect-detection) dataset for **Insecure Code Detection** downstream task. - -## Details of [CodeBERT](https://arxiv.org/abs/2002.08155) - -We present CodeBERT, a bimodal pre-trained model for programming language (PL) and nat-ural language (NL). CodeBERT learns general-purpose representations that support downstream NL-PL applications such as natural language codesearch, code documentation generation, etc. We develop CodeBERT with Transformer-based neural architecture, and train it with a hybrid objective function that incorporates the pre-training task of replaced token detection, which is to detect plausible alternatives sampled from generators. This enables us to utilize both bimodal data of NL-PL pairs and unimodal data, where the former provides input tokens for model training while the latter helps to learn better generators. We evaluate CodeBERT on two NL-PL applications by fine-tuning model parameters. Results show that CodeBERT achieves state-of-the-art performance on both natural language code search and code documentation generation tasks. Furthermore, to investigate what type of knowledge is learned in CodeBERT, we construct a dataset for NL-PL probing, and evaluate in a zero-shot setting where parameters of pre-trained models are fixed. Results show that CodeBERT performs better than previous pre-trained models on NL-PL probing. - -## Details of the downstream task (code classification) - Dataset 📚 - -Given a source code, the task is to identify whether it is an insecure code that may attack software systems, such as resource leaks, use-after-free vulnerabilities and DoS attack. We treat the task as binary classification (0/1), where 1 stands for insecure code and 0 for secure code. - -The [dataset](https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/Defect-detection) used comes from the paper [*Devign*: Effective Vulnerability Identification by Learning Comprehensive Program Semantics via Graph Neural Networks](http://papers.nips.cc/paper/9209-devign-effective-vulnerability-identification-by-learning-comprehensive-program-semantics-via-graph-neural-networks.pdf). All projects are combined and splitted 80%/10%/10% for training/dev/test. - -Data statistics of the dataset are shown in the below table: - -| | #Examples | -| ----- | :-------: | -| Train | 21,854 | -| Dev | 2,732 | -| Test | 2,732 | - -## Test set metrics 🧾 - -| Methods | ACC | -| -------- | :-------: | -| BiLSTM | 59.37 | -| TextCNN | 60.69 | -| [RoBERTa](https://arxiv.org/pdf/1907.11692.pdf) | 61.05 | -| [CodeBERT](https://arxiv.org/pdf/2002.08155.pdf) | 62.08 | -| [Ours](https://huggingface.co/mrm8488/codebert-base-finetuned-detect-insecure-code) | **65.30** | - - -## Model in Action 🚀 - -```python -from transformers import AutoTokenizer, AutoModelForSequenceClassification -import torch -import numpy as np -tokenizer = AutoTokenizer.from_pretrained('mrm8488/codebert-base-finetuned-detect-insecure-code') -model = AutoModelForSequenceClassification.from_pretrained('mrm8488/codebert-base-finetuned-detect-insecure-code') - -inputs = tokenizer("your code here", return_tensors="pt", truncation=True, padding='max_length') -labels = torch.tensor([1]).unsqueeze(0) # Batch size 1 -outputs = model(**inputs, labels=labels) -loss = outputs.loss -logits = outputs.logits - -print(np.argmax(logits.detach().numpy())) -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/distilbert-base-multi-cased-finetuned-typo-detection/README.md b/model_cards/mrm8488/distilbert-base-multi-cased-finetuned-typo-detection/README.md deleted file mode 100644 index 009bc1522c..0000000000 --- a/model_cards/mrm8488/distilbert-base-multi-cased-finetuned-typo-detection/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: multilingual -thumbnail: ---- - -# DISTILBERT 🌎 + Typo Detection ✍❌✍✔ - -[distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) fine-tuned on [GitHub Typo Corpus](https://github.com/mhagiwara/github-typo-corpus) for **typo detection** (using *NER* style) - -## Details of the downstream task (Typo detection as NER) - -- Dataset: [GitHub Typo Corpus](https://github.com/mhagiwara/github-typo-corpus) 📚 for 15 languages - -- [Fine-tune script on NER dataset provided by Huggingface](https://github.com/huggingface/transformers/blob/master/examples/token-classification/run_ner_old.py) 🏋️‍♂️ - -## Metrics on test set 📋 - -| Metric | # score | -| :-------: | :-------: | -| F1 | **93.51** | -| Precision | **96.08** | -| Recall | **91.06** | - -## Model in action 🔨 - -Fast usage with **pipelines** 🧪 - -```python -from transformers import pipeline - -typo_checker = pipeline( - "ner", - model="mrm8488/distilbert-base-multi-cased-finetuned-typo-detection", - tokenizer="mrm8488/distilbert-base-multi-cased-finetuned-typo-detection" -) - -result = typo_checker("Adddd validation midelware") -result[1:-1] - -# Output: -[{'entity': 'ok', 'score': 0.7128152847290039, 'word': 'add'}, - {'entity': 'typo', 'score': 0.5388424396514893, 'word': '##dd'}, - {'entity': 'ok', 'score': 0.94792640209198, 'word': 'validation'}, - {'entity': 'typo', 'score': 0.5839331746101379, 'word': 'mid'}, - {'entity': 'ok', 'score': 0.5195121765136719, 'word': '##el'}, - {'entity': 'ok', 'score': 0.7222476601600647, 'word': '##ware'}] -``` -It works🎉! We typed wrong ```Add and middleware``` - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/distilbert-multi-finetuned-for-xqua-on-tydiqa/README.md b/model_cards/mrm8488/distilbert-multi-finetuned-for-xqua-on-tydiqa/README.md deleted file mode 100644 index 509ca53f8b..0000000000 --- a/model_cards/mrm8488/distilbert-multi-finetuned-for-xqua-on-tydiqa/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: multilingual -thumbnail: ---- - -# DistilBERT multilingual fine-tuned on TydiQA (GoldP task) dataset for multilingual Q&A 😛🌍❓ - - -## Details of the language model - -[distilbert-base-multilingual-cased](https://huggingface.co/distilbert-base-multilingual-cased) - - -## Details of the Tydi QA dataset - -TyDi QA contains 200k human-annotated question-answer pairs in 11 Typologically Diverse languages, written without seeing the answer and without the use of translation, and is designed for the **training and evaluation** of automatic question answering systems. This repository provides evaluation code and a baseline system for the dataset. https://ai.google.com/research/tydiqa - - -## Details of the downstream task (Gold Passage or GoldP aka the secondary task) - -Given a passage that is guaranteed to contain the answer, predict the single contiguous span of characters that answers the question. the gold passage task differs from the [primary task](https://github.com/google-research-datasets/tydiqa/blob/master/README.md#the-tasks) in several ways: -* only the gold answer passage is provided rather than the entire Wikipedia article; -* unanswerable questions have been discarded, similar to MLQA and XQuAD; -* we evaluate with the SQuAD 1.1 metrics like XQuAD; and -* Thai and Japanese are removed since the lack of whitespace breaks some tools. - - -## Model training 💪🏋️‍ - -The model was fine-tuned on a Tesla P100 GPU and 25GB of RAM. -The script is the following: - -```python -python transformers/examples/question-answering/run_squad.py \ - --model_type distilbert \ - --model_name_or_path distilbert-base-multilingual-cased \ - --do_train \ - --do_eval \ - --train_file /path/to/dataset/train.json \ - --predict_file /path/to/dataset/dev.json \ - --per_gpu_train_batch_size 24 \ - --per_gpu_eval_batch_size 24 \ - --learning_rate 3e-5 \ - --num_train_epochs 5 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/model_output \ - --overwrite_output_dir \ - --save_steps 1000 \ - --threads 400 - ``` - -## Global Results (dev set) 📝 - -| Metric | # Value | -| --------- | ----------- | -| **EM** | **63.85** | -| **F1** | **75.70** | - -## Specific Results (per language) 🌍📝 - -| Language | # Samples | # EM | # F1 | -| --------- | ----------- |--------| ------ | -| Arabic | 1314 | 66.66 | 80.02 | -| Bengali | 180 | 53.09 | 63.50 | -| English | 654 | 62.42 | 73.12 | -| Finnish | 1031 | 64.57 | 75.15 | -| Indonesian| 773 | 67.89 | 79.70 | -| Korean | 414 | 51.29 | 61.73 | -| Russian | 1079 | 55.42 | 70.08 | -| Swahili | 596 | 74.51 | 81.15 | -| Telegu | 874 | 66.21 | 79.85 | - - -## Similar models - -You can also try [bert-multi-cased-finedtuned-xquad-tydiqa-goldp](https://huggingface.co/mrm8488/bert-multi-cased-finedtuned-xquad-tydiqa-goldp) that achieves **F1 = 82.16** and **EM = 71.06** (And of course better marks per language). - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md b/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md deleted file mode 100644 index ddd8b0704b..0000000000 --- a/model_cards/mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -language: es -thumbnail: https://i.imgur.com/jgBdimh.png ---- - -# BETO (Spanish BERT) + Spanish SQuAD2.0 + distillation using 'bert-base-multilingual-cased' as teacher - -This model is a fine-tuned on [SQuAD-es-v2.0](https://github.com/ccasimiro88/TranslateAlignRetrieve) and **distilled** version of [BETO](https://github.com/dccuchile/beto) for **Q&A**. - -Distillation makes the model **smaller, faster, cheaper and lighter** than [bert-base-spanish-wwm-cased-finetuned-spa-squad2-es](https://github.com/huggingface/transformers/blob/master/model_cards/mrm8488/bert-base-spanish-wwm-cased-finetuned-spa-squad2-es/README.md) - -This model was fine-tuned on the same dataset but using **distillation** during the process as mentioned above (and one more train epoch). - -The **teacher model** for the distillation was `bert-base-multilingual-cased`. It is the same teacher used for `distilbert-base-multilingual-cased` AKA [**DistilmBERT**](https://github.com/huggingface/transformers/tree/master/examples/distillation) (on average is twice as fast as **mBERT-base**). - -## Details of the downstream task (Q&A) - Dataset - -
- -[SQuAD-es-v2.0](https://github.com/ccasimiro88/TranslateAlignRetrieve) - -| Dataset | # Q&A | -| ----------------------- | ----- | -| SQuAD2.0 Train | 130 K | -| SQuAD2.0-es-v2.0 | 111 K | -| SQuAD2.0 Dev | 12 K | -| SQuAD-es-v2.0-small Dev | 69 K | - -
- -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -!export SQUAD_DIR=/path/to/squad-v2_spanish \ -&& python transformers/examples/distillation/run_squad_w_distillation.py \ - --model_type bert \ - --model_name_or_path dccuchile/bert-base-spanish-wwm-cased \ - --teacher_type bert \ - --teacher_name_or_path bert-base-multilingual-cased \ - --do_train \ - --do_eval \ - --do_lower_case \ - --train_file $SQUAD_DIR/train-v2.json \ - --predict_file $SQUAD_DIR/dev-v2.json \ - --per_gpu_train_batch_size 12 \ - --learning_rate 3e-5 \ - --num_train_epochs 5.0 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/model_output \ - --save_steps 5000 \ - --threads 4 \ - --version_2_with_negative -``` - -## Results: - -| Metric | # Value | -| --------- | ----------- | -| **Exact** | **90.77**48 | -| **F1** | **94.94**71 | - -```json -{ - "exact": 90.77483309730933, - "f1": 94.94714391266254, - "total": 69202, - "HasAns_exact": 86.60850599781898, - "HasAns_f1": 92.90582885592328, - "HasAns_total": 45850, - "NoAns_exact": 98.95512161699212, - "NoAns_f1": 98.95512161699212, - "NoAns_total": 23352, - "best_exact": 90.77483309730933, - "best_exact_thresh": 0.0, - "best_f1": 94.94714391266305, - "best_f1_thresh": 0.0 -} -``` - -## Comparison: - -| Model | f1 score | -| :-------------------------------------------------------------: | :-------: | -| bert-base-spanish-wwm-cased-finetuned-spa-squad2-es | 86.07 | -| **distill**-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es | **94.94** | - -So, yes, this version is even more accurate. - -### Model in action - -Fast usage with **pipelines**: - -```python -from transformers import * - -# Important!: By now the QA pipeline is not compatible with fast tokenizer, but they are working on it. So that pass the object to the tokenizer {"use_fast": False} as in the following example: - -nlp = pipeline( - 'question-answering', - model='mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es', - tokenizer=( - 'mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es', - {"use_fast": False} - ) -) - -nlp( - { - 'question': '¿Para qué lenguaje está trabajando?', - 'context': 'Manuel Romero está colaborando activamente con huggingface/transformers ' + - 'para traer el poder de las últimas técnicas de procesamiento de lenguaje natural al idioma español' - } -) -# Output: {'answer': 'español', 'end': 169, 'score': 0.67530957344621, 'start': 163} -``` - -Play with this model and ```pipelines``` in a Colab: - -Open In Colab - -
- -1. Set the context and ask some questions: - -![Set context and questions](https://media.giphy.com/media/mCIaBpfN0LQcuzkA2F/giphy.gif) - -2. Run predictions: - -![Run the model](https://media.giphy.com/media/WT453aptcbCP7hxWTZ/giphy.gif) -
- -More about ``` Huggingface pipelines```? check this Colab out: - -Open In Colab - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md b/model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md deleted file mode 100644 index 8e0df76935..0000000000 --- a/model_cards/mrm8488/distilroberta-base-finetuned-sentiment/README.md +++ /dev/null @@ -1,65 +0,0 @@ -# DistilRoBERTa + Sentiment Analysis 😂😢😡😃😯 - -This in an adapted version of [@omarsar0](https://twitter.com/omarsar0) [tutorial](https://t.co/WMnATW0Hwf?amp=1) -He explains everything so detailed and provided the dataset. I just changed some parameters and created the ```config.json```file to upload it to [🤗Transformers HUB](https://huggingface.co/) - - -In this tutorial, he shows how to fine-tune a language model (LM) for **emotion classification** with code adapted from this [tutorial](https://zablo.net/blog/post/custom-classifier-on-bert-model-guide-polemo2-sentiment-analysis/) by MARCIN ZABŁOCKI. - -The emotions covered are: - - sadness 😢 - - joy 😃 - - love 🥰 - - anger 😡 - - fear 😱 - - surprise 😯 - -## Details of the language model -The base model used is [DistilRoBERTa](https://huggingface.co/distilroberta-base) - -## Details of the downstream task (Sentence classification) - Dataset 📚 - -| Dataset split | # Size | # Sequences | -| ---------------------- | ----- | ------| -|Train | 1.58M | 20000 -| Validation | 200 KB | -| Test | 202 KB | - - -## Results after training 🏋️‍♀️🧾 - -|emotion |precision |recall| f1-score| support| -|-------|-------------|------|----------|----------| -|sadness| 0.973868 |0.949066 |0.961307| 589| -|joy |0.970313 |0.901306 |0.934537| 689| -|love |0.743119 |0.925714 |0.824427| 175| -|anger | 0.884615| 0.969349| 0.925046| 261| -|fear |0.951456 |0.875000| 0.911628| 224| -|surprise| 0.750000| 0.919355| 0.826087| 62| -| | | | | | -|**accuracy**| | | 0.924000| 2000| -|**macro avg**| 0.878895| 0.923298| 0.897172| 2000| -|**weighted avg**| 0.931355| 0.924000| 0.925620| 2000| - -## Model in action 🔨 - -Fast usage with **pipelines** 🧪 - -```python -from transformers import pipeline - -nlp_sentiment = pipeline( - "sentiment-analysis", - model="mrm8488/distilroberta-base-finetuned-sentiment", - tokenizer="mrm8488/distilroberta-base-finetuned-sentiment" -) - -text = "i feel i should return to the start of the weekend so my loyal readers can get a feeling for things up to this point" - -nlp_sentiment(text) -# Output: [{'label': 'love', 'score': 0.2183746}] -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electra-base-finetuned-squadv1/README.md b/model_cards/mrm8488/electra-base-finetuned-squadv1/README.md deleted file mode 100644 index 59dc87d51f..0000000000 --- a/model_cards/mrm8488/electra-base-finetuned-squadv1/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: en ---- - -# Electra base ⚡ + SQuAD v1 ❓ - -[Electra-base-discriminator](https://huggingface.co/google/electra-base-discriminator) fine-tuned on [SQUAD v1.1 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - - -## Details of the downstream task (Q&A) - Dataset 📚 - -**S**tanford **Q**uestion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. -SQuAD v1.1 contains **100,000+** question-answer pairs on **500+** articles. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type electra \ - --model_name_or_path 'google/electra-base-discriminator' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/train-v1.1.json' \ - --predict_file '/content/dataset/dev-v1.1.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir '/content/output' \ - --overwrite_output_dir \ - --save_steps 1000 -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **83.03** | -| **F1** | **90.77** | -| **Size**| **+ 400 MB** | - -Very good metrics for such a "small" model! - -```json -{ -'exact': 83.03689687795648, -'f1': 90.77486052446231, -'total': 10570, -'HasAns_exact': 83.03689687795648, -'HasAns_f1': 90.77486052446231, -'HasAns_total': 10570, -'best_exact': 83.03689687795648, -'best_exact_thresh': 0.0, -'best_f1': 90.77486052446231, -'best_f1_thresh': 0.0 -} -``` - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -QnA_pipeline = pipeline('question-answering', model='mrm8488/electra-base-finetuned-squadv1') - -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'What has been discovered by scientists from China ?' -}) -# Output: -{'answer': 'A new strain of flu', 'end': 19, 'score': 0.9995211430099182, 'start': 0} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electra-small-finetuned-squadv1/README.md b/model_cards/mrm8488/electra-small-finetuned-squadv1/README.md deleted file mode 100644 index c6986ec20e..0000000000 --- a/model_cards/mrm8488/electra-small-finetuned-squadv1/README.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -language: en ---- - -# Electra small ⚡ + SQuAD v1 ❓ - -[Electra-small-discriminator](https://huggingface.co/google/electra-small-discriminator) fine-tuned on [SQUAD v1.1 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - - -## Details of the downstream task (Q&A) - Dataset 📚 - -**S**tanford **Q**uestion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. -SQuAD v1.1 contains **100,000+** question-answer pairs on **500+** articles. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type electra \ - --model_name_or_path 'google/electra-small-discriminator' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/train-v1.1.json' \ - --predict_file '/content/dataset/dev-v1.1.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir '/content/output' \ - --overwrite_output_dir \ - --save_steps 1000 -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **77.70** | -| **F1** | **85.74** | -| **Size**| **50 MB** | - -Very good metrics for such a "small" model! - -```json - -{ -'exact': 77.70104068117313, -'f1': 85.73991234187997, -'total': 10570, -'HasAns_exact': 77.70104068117313, -'HasAns_f1': 85.73991234187997, -'HasAns_total': 10570, -'best_exact': 77.70104068117313, -'best_exact_thresh': 0.0, -'best_f1': 85.73991234187997, -'best_f1_thresh': 0.0 -} -``` - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -QnA_pipeline = pipeline('question-answering', model='mrm8488/electra-small-finetuned-squadv1') -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'What has been discovered by scientists from China ?' -}) - -# Output: -{'answer': 'A new strain of flu', 'end': 19, 'score': 0.7950334108113424, 'start': 0} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electra-small-finetuned-squadv2/README.md b/model_cards/mrm8488/electra-small-finetuned-squadv2/README.md deleted file mode 100644 index d1c8e8c721..0000000000 --- a/model_cards/mrm8488/electra-small-finetuned-squadv2/README.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -language: en ---- - -# Electra small ⚡ + SQuAD v2 ❓ - -[Electra-small-discriminator](https://huggingface.co/google/electra-small-discriminator) fine-tuned on [SQUAD v2.0 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - - -## Details of the downstream task (Q&A) - Dataset 📚 - -**SQuAD2.0** combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type electra \ - --model_name_or_path 'google/electra-small-discriminator' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/train-v2.0.json' \ - --predict_file '/content/dataset/dev-v2.0.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir '/content/output' \ - --overwrite_output_dir \ - --save_steps 1000 \ - --version_2_with_negative -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **69.71** | -| **F1** | **73.44** | -| **Size**| **50 MB** | - - -```json -{ -'exact': 69.71279373368147, -'f1': 73.4439546123672, -'total': 11873, -'HasAns_exact': 69.92240215924427, -'HasAns_f1': 77.39542393937836, -'HasAns_total': 5928, -'NoAns_exact': 69.50378469301934, -'NoAns_f1': 69.50378469301934, -'NoAns_total': 5945, -'best_exact': 69.71279373368147, -'best_exact_thresh': 0.0, -'best_f1': 73.44395461236732, -'best_f1_thresh': 0.0 -} -``` - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline -QnA_pipeline = pipeline('question-answering', model='mrm8488/electra-base-finetuned-squadv2') -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'What has been discovered by scientists from China ?' -}) -# Output: -{'answer': 'A new strain of flu', 'end': 19, 'score': 0.8650811568752914, 'start': 0} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electricidad-base-discriminator/README.md b/model_cards/mrm8488/electricidad-base-discriminator/README.md deleted file mode 100644 index e5f214f7a6..0000000000 --- a/model_cards/mrm8488/electricidad-base-discriminator/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: es -thumbnail: https://i.imgur.com/uxAvBfh.png - - ---- - -## ELECTRICIDAD: The Spanish Electra [Imgur](https://imgur.com/uxAvBfh) - -**Electricidad-base-discriminator** (uncased) is a ```base``` Electra like model (discriminator in this case) trained on a + 20 GB of the [OSCAR](https://oscar-corpus.com/) Spanish corpus. - -As mentioned in the original [paper](https://openreview.net/pdf?id=r1xMH1BtvB): -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer the paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - - -## Model details ⚙ - -|Name| # Value| -|-----|--------| -|Layers| 12 | -|Hidden |768 | -|Params| 110M| - -## Evaluation metrics (for discriminator) 🧾 - -|Metric | # Score | -|-------|---------| -|Accuracy| 0.985| -|Precision| 0.726| -|AUC | 0.922| - - - -## Fast example of usage 🚀 - -```python -from transformers import ElectraForPreTraining, ElectraTokenizerFast -import torch - -discriminator = ElectraForPreTraining.from_pretrained("/content/electricidad-base-discriminator") -tokenizer = ElectraTokenizerFast.from_pretrained("/content/electricidad-base-discriminator") - -sentence = "El rápido zorro marrón salta sobre el perro perezoso" -fake_sentence = "El rápido zorro marrón amar sobre el perro perezoso" - -fake_tokens = tokenizer.tokenize(fake_sentence) -fake_inputs = tokenizer.encode(fake_sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -[print("%7s" % token, end="") for token in fake_tokens] - -[print("%7s" % prediction, end="") for prediction in predictions.tolist()] - -# Output: -''' -el rapido zorro marro ##n amar sobre el perro pere ##zoso 0.0 0.0 0.0 0.0 0.0 0.0 1.0 1.0 0.0 0.0 0.0 0.0 0.0[None, None, None, None, None, None, None, None, None, None, None, None, None -''' -``` -As you can see there are **1s** in the places where the model detected a fake token. So, it works! 🎉 - - -### Some models fine-tuned on a downstream task 🛠️ - -[Question Answering](https://huggingface.co/mrm8488/electricidad-base-finetuned-squadv1-es) - -[POS](https://huggingface.co/mrm8488/electricidad-base-finetuned-pos) - -[NER](https://huggingface.co/mrm8488/electricidad-base-finetuned-ner) - -[Paraphrase Identification](https://huggingface.co/mrm8488/RuPERTa-base-finetuned-pawsx-es) - - - -## Acknowledgments - -I thank [🤗/transformers team](https://github.com/huggingface/transformers) for allowing me to train the model (specially to [Julien Chaumond](https://twitter.com/julien_c)). - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electricidad-base-finetuned-pawsx-es/README.md b/model_cards/mrm8488/electricidad-base-finetuned-pawsx-es/README.md deleted file mode 100644 index 3c520301e4..0000000000 --- a/model_cards/mrm8488/electricidad-base-finetuned-pawsx-es/README.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -language: es -datasets: -- xtreme -widget: -- text: "El río Tabaci es una vertiente del río Leurda en Rumania. El río Leurda es un afluente del río Tabaci en Rumania." ---- - -# Electricidad-base fine-tuned on PAWS-X-es for Paraphrase Identification diff --git a/model_cards/mrm8488/electricidad-base-generator/README.md b/model_cards/mrm8488/electricidad-base-generator/README.md deleted file mode 100644 index 2f078fbaec..0000000000 --- a/model_cards/mrm8488/electricidad-base-generator/README.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -language: es -thumbnail: https://i.imgur.com/uxAvBfh.png -widget: -- text: "Madrid es una ciudad muy [MASK] en España." - - ---- - -## ELECTRICIDAD: The Spanish Electra [Imgur](https://imgur.com/uxAvBfh) - -**Electricidad-base-generator** (uncased) is a ```base``` Electra like model (generator in this case) trained on a + 20 GB of the [OSCAR](https://oscar-corpus.com/) Spanish corpus. - -As mentioned in the original [paper](https://openreview.net/pdf?id=r1xMH1BtvB): -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer the paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - - - - - -## Fast example of usage 🚀 - -```python -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="mrm8488/electricidad-base-generator", - tokenizer="mrm8488/electricidad-base-generator" -) - -print( - fill_mask(f"HuggingFace está creando {fill_mask.tokenizer.mask_token} que la comunidad usa para resolver tareas de NLP.") -) - -# Output: [{'sequence': '[CLS] huggingface esta creando herramientas que la comunidad usa para resolver tareas de nlp. [SEP]', 'score': 0.0896105170249939, 'token': 8760, 'token_str': 'herramientas'}, ...] - -``` - -## Acknowledgments - -I thank [🤗/transformers team](https://github.com/huggingface/transformers) for allowing me to train the model (specially to [Julien Chaumond](https://twitter.com/julien_c)). - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electricidad-small-discriminator/README.md b/model_cards/mrm8488/electricidad-small-discriminator/README.md deleted file mode 100644 index 03d9f881f0..0000000000 --- a/model_cards/mrm8488/electricidad-small-discriminator/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: es -thumbnail: https://i.imgur.com/uxAvBfh.png - - ---- - -## ELECTRICIDAD: The Spanish Electra [Imgur](https://imgur.com/uxAvBfh) - -**ELECTRICIDAD** is a small Electra like model (discriminator in this case) trained on a + 20 GB of the [OSCAR](https://oscar-corpus.com/) Spanish corpus. - -As mentioned in the original [paper](https://openreview.net/pdf?id=r1xMH1BtvB): -**ELECTRA** is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a [GAN](https://arxiv.org/pdf/1406.2661.pdf). At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) dataset. - -For a detailed description and experimental results, please refer the paper [ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators](https://openreview.net/pdf?id=r1xMH1BtvB). - -## Model details ⚙ - -|Param| # Value| -|-----|--------| -|Layers| 12 | -|Hidden |256 | -|Params| 14M| - -## Evaluation metrics (for discriminator) 🧾 - -|Metric | # Score | -|-------|---------| -|Accuracy| 0.94| -|Precision| 0.76| -|AUC | 0.92| - -## Benchmarks 🔨 - -WIP 🚧 - -## How to use the discriminator in `transformers` - -```python -from transformers import ElectraForPreTraining, ElectraTokenizerFast -import torch - -discriminator = ElectraForPreTraining.from_pretrained("mrm8488/electricidad-small-discriminator") -tokenizer = ElectraTokenizerFast.from_pretrained("mrm8488/electricidad-small-discriminator") - -sentence = "el zorro rojo es muy rápido" -fake_sentence = "el zorro rojo es muy ser" - -fake_tokens = tokenizer.tokenize(sentence) -fake_inputs = tokenizer.encode(sentence, return_tensors="pt") -discriminator_outputs = discriminator(fake_inputs) -predictions = torch.round((torch.sign(discriminator_outputs[0]) + 1) / 2) - -[print("%7s" % token, end="") for token in fake_tokens] - -[print("%7s" % int(prediction), end="") for prediction in predictions.tolist()[1:-1]] - -# Output: -''' -el zorro rojo es muy ser 0 0 0 0 0 1[None, None, None, None, None, None] -''' -``` - -As you can see there is a **1** in the place where the model detected the fake token (**ser**). So, it works! 🎉 - -## Acknowledgments - -I thank [🤗/transformers team](https://github.com/huggingface/transformers) for answering my doubts and Google for helping me with the [TensorFlow Research Cloud](https://www.tensorflow.org/tfrc) program. - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/electricidad-small-finetuned-squadv1-es/README.md b/model_cards/mrm8488/electricidad-small-finetuned-squadv1-es/README.md deleted file mode 100644 index ccdfb39d24..0000000000 --- a/model_cards/mrm8488/electricidad-small-finetuned-squadv1-es/README.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -language: es -thumbnail: https://imgur.com/uxAvBfh ---- - -# Electricidad small + Spanish SQuAD v1 ⚡❓ - -[Electricidad-small-discriminator](https://huggingface.co/mrm8488/electricidad-small-discriminator) fine-tuned on [Spanish SQUAD v1.1 dataset](https://github.com/ccasimiro88/TranslateAlignRetrieve/tree/master/SQuAD-es-v1.1) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Dataset 📚 - -[SQuAD-es-v1.1](https://github.com/ccasimiro88/TranslateAlignRetrieve/tree/master/SQuAD-es-v1.1) - -| Dataset split | # Samples | -| ------------- | --------- | -| Train | 130 K | -| Test | 11 K | - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python /content/transformers/examples/question-answering/run_squad.py \ - --model_type electra \ - --model_name_or_path 'mrm8488/electricidad-small-discriminator' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/train-v1.1-es.json' \ - --predict_file '/content/dataset/dev-v1.1-es.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir '/content/electricidad-small-finetuned-squadv1-es' \ - --overwrite_output_dir \ - --save_steps 1000 -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **46.82** | -| **F1** | **64.79** | - -```json -{ -'exact': 46.82119205298013, -'f1': 64.79435260021918, -'total': 10570, -'HasAns_exact': 46.82119205298013, -HasAns_f1': 64.79435260021918, -'HasAns_total': 10570, -'best_exact': 46.82119205298013, -'best_exact_thresh': 0.0, -'best_f1': 64.79435260021918, -'best_f1_thresh': 0.0 -} -``` - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/electricidad-small-finetuned-squadv1-es", - tokenizer="mrm8488/electricidad-small-finetuned-squadv1-es" -) - -context = "Manuel ha creado una versión del modelo Electra small en español que alcanza una puntuación F1 de 65 en el dataset SQUAD-es y sólo pesa 50 MB" - -q1 = "Cuál es su marcador F1?" -q2 = "¿Cuál es el tamaño del modelo?" -q3 = "¿Quién lo ha creado?" -q4 = "¿Que es lo que ha hecho Manuel?" - - -questions = [q1, q2, q3, q4] - -for question in questions: - result = qa_pipeline({ - 'context': context, - 'question': question}) - print(result) - -# Output: -{'score': 0.14836778166355025, 'start': 98, 'end': 100, 'answer': '65'} -{'score': 0.32219420810758237, 'start': 136, 'end': 140, 'answer': '50 MB'} -{'score': 0.9672326951118713, 'start': 0, 'end': 6, 'answer': 'Manuel'} -{'score': 0.23552458113848118, 'start': 10, 'end': 53, 'answer': 'creado una versión del modelo Electra small'} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/gpt2-finetuned-recipes-cooking/README.md b/model_cards/mrm8488/gpt2-finetuned-recipes-cooking/README.md deleted file mode 100644 index 1e8029f304..0000000000 --- a/model_cards/mrm8488/gpt2-finetuned-recipes-cooking/README.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -language: en -thumbnail: -widget: -- text: "HuggingFace Cake:" ---- diff --git a/model_cards/mrm8488/gpt2-finetuned-recipes-cooking_v2/README.md b/model_cards/mrm8488/gpt2-finetuned-recipes-cooking_v2/README.md deleted file mode 100644 index 1e8029f304..0000000000 --- a/model_cards/mrm8488/gpt2-finetuned-recipes-cooking_v2/README.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -language: en -thumbnail: -widget: -- text: "HuggingFace Cake:" ---- diff --git a/model_cards/mrm8488/gpt2-imdb-neg/README.md b/model_cards/mrm8488/gpt2-imdb-neg/README.md deleted file mode 100644 index 2cebaf9ae3..0000000000 --- a/model_cards/mrm8488/gpt2-imdb-neg/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# GPT2-IMDB-neg (LM + RL) 🎞😡✍ - -All credits to [@lvwerra](https://twitter.com/lvwerra) - -## What is it? -A small GPT2 (`lvwerra/gpt2-imdb`) language model fine-tuned to produce **negative** movie reviews based the [IMDB dataset](https://www.kaggle.com/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews). The model is trained with rewards from a BERT sentiment classifier (`lvwerra/gpt2-imdb`) via **PPO**. - -## Why? -I wanted to reproduce the experiment [lvwerra/gpt2-imdb-pos](https://huggingface.co/lvwerra/gpt2-imdb-pos) but for generating **negative** movie reviews. - -## Training setting -The model was trained for `100` optimisation steps with a batch size of `256` which corresponds to `25600` training samples. The full experiment setup (for positive samples) in [trl repo](https://lvwerra.github.io/trl/04-gpt2-sentiment-ppo-training/). - -## Examples -A few examples of the model response to a query before and after optimisation: - -| query | response (before) | response (after) | rewards (before) | rewards (after) | -|-------|-------------------|------------------|------------------|-----------------| -|This movie is a fine | attempt as far as live action is concerned, n...|example of how bad Hollywood in theatrics pla...| 2.118391 | -3.31625| -|I have watched 3 episodes |with this guy and he is such a talented actor...| but the show is just plain awful and there ne...| 2.681171| -4.512792| -|We know that firefighters and| police officers are forced to become populari...| other chains have going to get this disaster ...| 1.367811| -3.34017| - -## Training logs and metrics -Watch the whole training logs and metrics on [W&B](https://app.wandb.ai/mrm8488/gpt2-sentiment-negative?workspace=user-mrm8488) - - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/gpt2-imdb-neutral/README.md b/model_cards/mrm8488/gpt2-imdb-neutral/README.md deleted file mode 100644 index 0d0ad4734a..0000000000 --- a/model_cards/mrm8488/gpt2-imdb-neutral/README.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -language: en -thumbnail: ---- - -# GPT2-IMDB-neutral (LM + RL) 🎞😐✍ - -## What is it? -A small GPT2 (`lvwerra/gpt2-imdb`) language model fine-tuned to produce **neutral**-ish movie reviews based on the [IMDB dataset](https://www.kaggle.com/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews). The model is trained with rewards from a BERT sentiment classifier (`lvwerra/gpt2-imdb`) via **PPO**. - -## Why? -After reproducing the experiment [lvwerra/gpt2-imdb-pos](https://huggingface.co/lvwerra/gpt2-imdb-pos) but for generating **negative** movie reviews ([mrm8488/gpt2-imdb-neg](https://huggingface.co/mrm8488/gpt2-imdb-neg)) I wanted to check if I could generate neutral-ish movie reviews. So, based on the classifier output (logit), I saw that clearly negative reviews gives around *-4* values and clearly positive reviews around *4*. Then, it was esay to establish an interval ```[-1.75,1.75]``` that it could be considered as **neutral**. So if the classifier output was in that interval I gave it a positive reward while values out of the interval got a negative reward. - -## Training setting -The model was trained for `100` optimisation steps with a batch size of `128` which corresponds to `30000` training samples. The full experiment setup (for positive samples) in [trl repo](https://lvwerra.github.io/trl/04-gpt2-sentiment-ppo-training/). - -## Examples -A few examples of the model response to a query before and after optimisation: - -| query | response (before) | response (after) | rewards (before) | rewards (after) | -|-------|-------------------|------------------|------------------|-----------------| -|Okay, my title is|partly over, but this drama still makes me proud to read its first 40...|weird. The title is "mana were, ahunter". "Man...|4.200727 |-1.891443| -|Where is it written that|there is a monster in this movie anyway? How is it that the entire|[ of the women in the recent women of jungle business between Gender and husband| -3.113942| -1.944993| -|As a lesbian, I|cannot believe I was in the Sixties! Subtle yet witty, with original| found it hard to get responsive. In fact I found myself with the long| 3.906178| 0.769166| -|The Derek's have over|three times as many acting hours than Jack Nicholson? You think bitches?|30 dueling characters and kill of, they retreat themselves to their base.|-2.503655| -1.898380| - - -> All credits to [@lvwerra](https://twitter.com/lvwerra) - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/longformer-base-4096-finetuned-squadv2/README.md b/model_cards/mrm8488/longformer-base-4096-finetuned-squadv2/README.md deleted file mode 100644 index 269f7e9175..0000000000 --- a/model_cards/mrm8488/longformer-base-4096-finetuned-squadv2/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -language: en -datasets: -- squad_v2 ---- - -# Longformer-base-4096 fine-tuned on SQuAD v2 - -[Longformer-base-4096 model](https://huggingface.co/allenai/longformer-base-4096) fine-tuned on [SQuAD v2](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -## Longformer-base-4096 - -[Longformer](https://arxiv.org/abs/2004.05150) is a transformer model for long documents. - -`longformer-base-4096` is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096. - -Longformer uses a combination of a sliding window (local) attention and global attention. Global attention is user-configured based on the task to allow the model to learn task-specific representations. - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -Dataset ID: ```squad_v2``` from [HugginFace/NLP](https://github.com/huggingface/nlp) -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| squad_v2 | train | 130319 | -| squad_v2 | valid | 11873 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('squad_v2', split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('squad_v2', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this one](https://colab.research.google.com/drive/1zEl5D-DdkBKva-DdreVOmN0hrAfzKG1o?usp=sharing) - - - -## Model in Action 🚀 - -```python -import torch -from transformers import AutoTokenizer, AutoModelForQuestionAnswering - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/longformer-base-4096-finetuned-squadv2") -model = AutoModelForQuestionAnswering.from_pretrained("mrm8488/longformer-base-4096-finetuned-squadv2") - -text = "Huggingface has democratized NLP. Huge thanks to Huggingface for this." -question = "What has Huggingface done ?" -encoding = tokenizer(question, text, return_tensors="pt") -input_ids = encoding["input_ids"] - -# default is local attention everywhere -# the forward method will automatically set global attention on question tokens -attention_mask = encoding["attention_mask"] - -start_scores, end_scores = model(input_ids, attention_mask=attention_mask) -all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist()) - -answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1] -answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens)) - -# output => democratized NLP -``` -If given the same context we ask something that is not there, the output for **no answer** will be `````` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/mT5-small-finetuned-tydiqa-for-xqa/README.md b/model_cards/mrm8488/mT5-small-finetuned-tydiqa-for-xqa/README.md deleted file mode 100644 index 46b392bbe5..0000000000 --- a/model_cards/mrm8488/mT5-small-finetuned-tydiqa-for-xqa/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: multilingual -datasets: -- tydiqa -pipeline_tag: question-answering ---- - -# mT5-small fine-tuned on TyDiQA for multilingual QA 🗺📖❓ -[Google's mT5-small](https://huggingface.co/google/mt5-small) fine-tuned on [TyDi QA](https://huggingface.co/nlp/viewer/?dataset=tydiqa&config=secondary_task) (secondary task) for **multingual Q&A** downstream task. - -## Details of mT5 - -[Google's mT5](https://github.com/google-research/multilingual-t5) - -mT5 is pretrained on the [mC4](https://www.tensorflow.org/datasets/catalog/c4#c4multilingual) corpus, covering 101 languages: - -Afrikaans, Albanian, Amharic, Arabic, Armenian, Azerbaijani, Basque, Belarusian, Bengali, Bulgarian, Burmese, Catalan, Cebuano, Chichewa, Chinese, Corsican, Czech, Danish, Dutch, English, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, German, Greek, Gujarati, Haitian Creole, Hausa, Hawaiian, Hebrew, Hindi, Hmong, Hungarian, Icelandic, Igbo, Indonesian, Irish, Italian, Japanese, Javanese, Kannada, Kazakh, Khmer, Korean, Kurdish, Kyrgyz, Lao, Latin, Latvian, Lithuanian, Luxembourgish, Macedonian, Malagasy, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Nepali, Norwegian, Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Russian, Samoan, Scottish Gaelic, Serbian, Shona, Sindhi, Sinhala, Slovak, Slovenian, Somali, Sotho, Spanish, Sundanese, Swahili, Swedish, Tajik, Tamil, Telugu, Thai, Turkish, Ukrainian, Urdu, Uzbek, Vietnamese, Welsh, West Frisian, Xhosa, Yiddish, Yoruba, Zulu. - -**Note**: mT5 was only pre-trained on mC4 excluding any supervised training. Therefore, this model has to be fine-tuned before it is useable on a downstream task. - -Pretraining Dataset: [mC4](https://www.tensorflow.org/datasets/catalog/c4#c4multilingual) - -Other Community Checkpoints: [here](https://huggingface.co/models?search=mt5) - -Paper: [mT5: A massively multilingual pre-trained text-to-text transformer](https://arxiv.org/abs/2010.11934) - -Authors: *Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel* - - -## Details of the dataset 📚 - -**TyDi QA** is a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs. The languages of TyDi QA are diverse with regard to their typology -- the set of linguistic features that each language expresses -- such that we expect models performing well on this set to generalize across a large number of the languages in the world. It contains language phenomena that would not be found in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but don’t know the answer yet, (unlike SQuAD and its descendents) and the data is collected directly in each language without the use of translation (unlike MLQA and XQuAD). - -| Dataset | Task | Split | # samples | -| -------- | ----- |------| --------- | -| TyDi QA | GoldP | train| 49881 | -| TyDi QA | GoldP | valid| 5077 | - - - -## Results on validation dataset 📝 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **41.65** | - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelForCausalLM, AutoTokenizer -import torch -device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') -tokenizer = AutoTokenizer.from_pretrained("mrm8488/mT5-small-finetuned-tydiqa-for-xqa") -model = AutoModelForCausalLM.from_pretrained("mrm8488/mT5-small-finetuned-tydiqa-for-xqa").to(device) - -def get_response(question, context, max_length=32): - input_text = 'question: %s context: %s' % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'].to(device), - attention_mask=features['attention_mask'].to(device), - max_length=max_length) - - return tokenizer.decode(output[0]) - -# Some examples in different languages - -context = 'HuggingFace won the best Demo paper at EMNLP2020.' -question = 'What won HuggingFace?' -get_response(question, context) - -context = 'HuggingFace ganó la mejor demostración con su paper en la EMNLP2020.' -question = 'Qué ganó HuggingFace?' -get_response(question, context) - -context = 'HuggingFace выиграл лучшую демонстрационную работу на EMNLP2020.' -question = 'Что победило в HuggingFace?' -get_response(question, context) -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/mobilebert-uncased-finetuned-squadv1/README.md b/model_cards/mrm8488/mobilebert-uncased-finetuned-squadv1/README.md deleted file mode 100644 index 68a87d9f9a..0000000000 --- a/model_cards/mrm8488/mobilebert-uncased-finetuned-squadv1/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: en -datasets: -- squad ---- - -# MobileBERT + SQuAD (v1.1) 📱❓ - -[mobilebert-uncased](https://huggingface.co/google/mobilebert-uncased) fine-tuned on [SQUAD v2.0 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -**MobileBERT** is a thin version of *BERT_LARGE*, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. - -The checkpoint used here is the original MobileBert Optimized Uncased English: (uncased_L-24_H-128_B-512_A-4_F-4_OPT) checkpoint. - -More about the model [here](https://arxiv.org/abs/2004.02984) - -## Details of the downstream task (Q&A) - Dataset 📚 - -**S**tanford **Q**uestion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. -SQuAD v1.1 contains **100,000+** question-answer pairs on **500+** articles. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type bert \ - --model_name_or_path 'google/mobilebert-uncased' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/train-v1.1.json' \ - --predict_file '/content/dataset/dev-v1.1.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 5 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir '/content/output' \ - --overwrite_output_dir \ - --save_steps 1000 -``` - -It is important to say that this models converges much faster than other ones. So, it is also cheap to fine-tune. - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **82.33** | -| **F1** | **89.64** | -| **Size**| **94 MB** | - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline -QnA_pipeline = pipeline('question-answering', model='mrm8488/mobilebert-uncased-finetuned-squadv1') -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'Who did identified it ?' - }) - -# Output: {'answer': 'scientists.', 'end': 106, 'score': 0.7885545492172241, 'start': 96} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/mobilebert-uncased-finetuned-squadv2/README.md b/model_cards/mrm8488/mobilebert-uncased-finetuned-squadv2/README.md deleted file mode 100644 index 3bd933b77d..0000000000 --- a/model_cards/mrm8488/mobilebert-uncased-finetuned-squadv2/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: en -datasets: -- squad_v2 ---- - -# MobileBERT + SQuAD v2 📱❓ - -[mobilebert-uncased](https://huggingface.co/google/mobilebert-uncased) fine-tuned on [SQUAD v2.0 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -**MobileBERT** is a thin version of *BERT_LARGE*, while equipped with bottleneck structures and a carefully designed balance between self-attentions and feed-forward networks. - -The checkpoint used here is the original MobileBert Optimized Uncased English: (uncased_L-24_H-128_B-512_A-4_F-4_OPT) checkpoint. - -More about the model [here](https://arxiv.org/abs/2004.02984) - -## Details of the downstream task (Q&A) - Dataset 📚 - -**SQuAD2.0** combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type bert \ - --model_name_or_path 'google/mobilebert-uncased' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/train-v2.0.json' \ - --predict_file '/content/dataset/dev-v2.0.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 5 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir '/content/output' \ - --overwrite_output_dir \ - --save_steps 1000 \ - --version_2_with_negative -``` - -It is important to say that this models converges much faster than other ones. So, it is also cheap to fine-tune. - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **75.37** | -| **F1** | **78.48** | -| **Size**| **94 MB** | - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline -QnA_pipeline = pipeline('question-answering', model='mrm8488/mobilebert-uncased-finetuned-squadv2') -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'Who did identified it ?' - }) - -# Output: {'answer': 'scientists.', 'end': 106, 'score': 0.41531604528427124, 'start': 96} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv1/README.md b/model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv1/README.md deleted file mode 100644 index 0ab598cf1b..0000000000 --- a/model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv1/README.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -language: en ---- - -# RoBERTa-base (1B-1) + SQuAD v1 ❓ - -[roberta-base-1B-1](https://huggingface.co/nyu-mll/roberta-base-1B-1) fine-tuned on [SQUAD v1.1 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -RoBERTa Pretrained on Smaller Datasets - -[NYU Machine Learning for Language](https://huggingface.co/nyu-mll) pretrained RoBERTa on smaller datasets (1M, 10M, 100M, 1B tokens). They released 3 models with lowest perplexities for each pretraining data size out of 25 runs (or 10 in the case of 1B tokens). The pretraining data reproduces that of BERT: They combine English Wikipedia and a reproduction of BookCorpus using texts from smashwords in a ratio of approximately 3:1. - - -## Details of the downstream task (Q&A) - Dataset 📚 - -**S**tanford **Q**uestion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. -SQuAD v1.1 contains **100,000+** question-answer pairs on **500+** articles. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type roberta \ - --model_name_or_path 'nyu-mll/roberta-base-1B-1' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file /content/dataset/train-v1.1.json \ - --predict_file /content/dataset/dev-v1.1.json \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/output \ - --overwrite_output_dir \ - --save_steps 1000 -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **72.62** | -| **F1** | **82.19** | - - - -```json -{ -'exact': 72.62062440870388, -'f1': 82.19430877136834, -'total': 10570, -'HasAns_exact': 72.62062440870388, -'HasAns_f1': 82.19430877136834, -'HasAns_total': 10570, -'best_exact': 72.62062440870388, -'best_exact_thresh': 0.0, -'best_f1': 82.19430877136834, -'best_f1_thresh': 0.0 -} - -``` - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -QnA_pipeline = pipeline('question-answering', model='mrm8488/roberta-base-1B-1-finetuned-squadv1') - -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'What has been discovered by scientists from China ?' -}) -# Output: - -{'answer': 'A new strain of flu', 'end': 19, 'score': 0.04702283976040074, 'start': 0} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv2/README.md b/model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv2/README.md deleted file mode 100644 index 42c86e7b42..0000000000 --- a/model_cards/mrm8488/roberta-base-1B-1-finetuned-squadv2/README.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -language: en ---- - -# RoBERTa-base (1B-1) + SQuAD v2 ❓ - -[roberta-base-1B-1](https://huggingface.co/nyu-mll/roberta-base-1B-1) fine-tuned on [SQUAD v2 dataset](https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -RoBERTa Pretrained on Smaller Datasets - -[NYU Machine Learning for Language](https://huggingface.co/nyu-mll) pretrained RoBERTa on smaller datasets (1M, 10M, 100M, 1B tokens). They released 3 models with lowest perplexities for each pretraining data size out of 25 runs (or 10 in the case of 1B tokens). The pretraining data reproduces that of BERT: They combine English Wikipedia and a reproduction of BookCorpus using texts from smashwords in a ratio of approximately 3:1. - - -## Details of the downstream task (Q&A) - Dataset 📚 - -**S**tanford **Q**uestion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. - - **SQuAD2.0** combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type roberta \ - --model_name_or_path 'nyu-mll/roberta-base-1B-1' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file /content/dataset/train-v2.0.json \ - --predict_file /content/dataset/dev-v2.0.json \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/output \ - --overwrite_output_dir \ - --save_steps 1000 \ - --version_2_with_negative -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **64.86** | -| **F1** | **68.99** | - - - -```json -{ -'exact': 64.86145034953255, -'f1': 68.9902640378272, -'total': 11873, -'HasAns_exact': 64.03508771929825, -'HasAns_f1': 72.3045554860189, -'HasAns_total': 5928, -'NoAns_exact': 65.68544995794785, -'NoAns_f1': 65.68544995794785, -'NoAns_total': 5945, -'best_exact': 64.86987282068559, -'best_exact_thresh': 0.0, -'best_f1': 68.99868650898054, -'best_f1_thresh': 0.0 -} -``` - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -QnA_pipeline = pipeline('question-answering', model='mrm8488/roberta-base-1B-1-finetuned-squadv2') - -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'What has been discovered by scientists from China ?' -}) -# Output: - -{'answer': 'A new strain of flu', 'end': 19, 'score': 0.7145650685380576,'start': 0} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/roberta-large-finetuned-wsc/README.md b/model_cards/mrm8488/roberta-large-finetuned-wsc/README.md deleted file mode 100644 index 94387a31a4..0000000000 --- a/model_cards/mrm8488/roberta-large-finetuned-wsc/README.md +++ /dev/null @@ -1,128 +0,0 @@ -# RoBERTa (large) fine-tuned on Winograd Schema Challenge (WSC) data - -Step from its original [repo](https://github.com/pytorch/fairseq/blob/master/examples/roberta/wsc/README.md) - -The following instructions can be used to finetune RoBERTa on the WSC training -data provided by [SuperGLUE](https://super.gluebenchmark.com/). - -Note that there is high variance in the results. For our GLUE/SuperGLUE -submission we swept over the learning rate (1e-5, 2e-5, 3e-5), batch size (16, -32, 64) and total number of updates (500, 1000, 2000, 3000), as well as the -random seed. Out of ~100 runs we chose the best 7 models and ensembled them. - -**Approach:** The instructions below use a slightly different loss function than -what's described in the original RoBERTa arXiv paper. In particular, -[Kocijan et al. (2019)](https://arxiv.org/abs/1905.06290) introduce a margin -ranking loss between `(query, candidate)` pairs with tunable hyperparameters -alpha and beta. This is supported in our code as well with the `--wsc-alpha` and -`--wsc-beta` arguments. However, we achieved slightly better (and more robust) -results on the development set by instead using a single cross entropy loss term -over the log-probabilities for the query and all mined candidates. **The -candidates are mined using spaCy from each input sentence in isolation, so the -approach remains strictly pointwise.** This reduces the number of -hyperparameters and our best model achieved 92.3% development set accuracy, -compared to ~90% accuracy for the margin loss. Later versions of the RoBERTa -arXiv paper will describe this updated formulation. - -### 1) Download the WSC data from the SuperGLUE website: -```bash -wget https://dl.fbaipublicfiles.com/glue/superglue/data/v2/WSC.zip -unzip WSC.zip - -# we also need to copy the RoBERTa dictionary into the same directory -wget -O WSC/dict.txt https://dl.fbaipublicfiles.com/fairseq/gpt2_bpe/dict.txt -``` - -### 2) Finetune over the provided training data: -```bash -TOTAL_NUM_UPDATES=2000 # Total number of training steps. -WARMUP_UPDATES=250 # Linearly increase LR over this many steps. -LR=2e-05 # Peak LR for polynomial LR scheduler. -MAX_SENTENCES=16 # Batch size per GPU. -SEED=1 # Random seed. -ROBERTA_PATH=/path/to/roberta/model.pt - -# we use the --user-dir option to load the task and criterion -# from the examples/roberta/wsc directory: -FAIRSEQ_PATH=/path/to/fairseq -FAIRSEQ_USER_DIR=${FAIRSEQ_PATH}/examples/roberta/wsc - -CUDA_VISIBLE_DEVICES=0,1,2,3 fairseq-train WSC/ \ - --restore-file $ROBERTA_PATH \ - --reset-optimizer --reset-dataloader --reset-meters \ - --no-epoch-checkpoints --no-last-checkpoints --no-save-optimizer-state \ - --best-checkpoint-metric accuracy --maximize-best-checkpoint-metric \ - --valid-subset val \ - --fp16 --ddp-backend no_c10d \ - --user-dir $FAIRSEQ_USER_DIR \ - --task wsc --criterion wsc --wsc-cross-entropy \ - --arch roberta_large --bpe gpt2 --max-positions 512 \ - --dropout 0.1 --attention-dropout 0.1 --weight-decay 0.01 \ - --optimizer adam --adam-betas '(0.9, 0.98)' --adam-eps 1e-06 \ - --lr-scheduler polynomial_decay --lr $LR \ - --warmup-updates $WARMUP_UPDATES --total-num-update $TOTAL_NUM_UPDATES \ - --max-sentences $MAX_SENTENCES \ - --max-update $TOTAL_NUM_UPDATES \ - --log-format simple --log-interval 100 \ - --seed $SEED -``` - -The above command assumes training on 4 GPUs, but you can achieve the same -results on a single GPU by adding `--update-freq=4`. - -### 3) Evaluate -```python -from fairseq.models.roberta import RobertaModel -from examples.roberta.wsc import wsc_utils # also loads WSC task and criterion -roberta = RobertaModel.from_pretrained('checkpoints', 'checkpoint_best.pt', 'WSC/') -roberta.cuda() -nsamples, ncorrect = 0, 0 -for sentence, label in wsc_utils.jsonl_iterator('WSC/val.jsonl', eval=True): - pred = roberta.disambiguate_pronoun(sentence) - nsamples += 1 - if pred == label: - ncorrect += 1 -print('Accuracy: ' + str(ncorrect / float(nsamples))) -# Accuracy: 0.9230769230769231 -``` - -## RoBERTa training on WinoGrande dataset -We have also provided `winogrande` task and criterion for finetuning on the -[WinoGrande](https://mosaic.allenai.org/projects/winogrande) like datasets -where there are always two candidates and one is correct. -It's more efficient implementation for such subcases. - -```bash -TOTAL_NUM_UPDATES=23750 # Total number of training steps. -WARMUP_UPDATES=2375 # Linearly increase LR over this many steps. -LR=1e-05 # Peak LR for polynomial LR scheduler. -MAX_SENTENCES=32 # Batch size per GPU. -SEED=1 # Random seed. -ROBERTA_PATH=/path/to/roberta/model.pt - -# we use the --user-dir option to load the task and criterion -# from the examples/roberta/wsc directory: -FAIRSEQ_PATH=/path/to/fairseq -FAIRSEQ_USER_DIR=${FAIRSEQ_PATH}/examples/roberta/wsc - -cd fairseq -CUDA_VISIBLE_DEVICES=0 fairseq-train winogrande_1.0/ \ - --restore-file $ROBERTA_PATH \ - --reset-optimizer --reset-dataloader --reset-meters \ - --no-epoch-checkpoints --no-last-checkpoints --no-save-optimizer-state \ - --best-checkpoint-metric accuracy --maximize-best-checkpoint-metric \ - --valid-subset val \ - --fp16 --ddp-backend no_c10d \ - --user-dir $FAIRSEQ_USER_DIR \ - --task winogrande --criterion winogrande \ - --wsc-margin-alpha 5.0 --wsc-margin-beta 0.4 \ - --arch roberta_large --bpe gpt2 --max-positions 512 \ - --dropout 0.1 --attention-dropout 0.1 --weight-decay 0.01 \ - --optimizer adam --adam-betas '(0.9, 0.98)' --adam-eps 1e-06 \ - --lr-scheduler polynomial_decay --lr $LR \ - --warmup-updates $WARMUP_UPDATES --total-num-update $TOTAL_NUM_UPDATES \ - --max-sentences $MAX_SENTENCES \ - --max-update $TOTAL_NUM_UPDATES \ - --log-format simple --log-interval 100 -``` -[Original repo](https://github.com/pytorch/fairseq/tree/master/examples/roberta/wsc) diff --git a/model_cards/mrm8488/spanbert-base-finetuned-squadv1/README.md b/model_cards/mrm8488/spanbert-base-finetuned-squadv1/README.md deleted file mode 100644 index 8ebb811418..0000000000 --- a/model_cards/mrm8488/spanbert-base-finetuned-squadv1/README.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT base fine-tuned on SQuAD v1 - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 1.1](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task ([by them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution)). - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -[SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer/) - -## Model fine-tuning 🏋️‍ - -You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) - -```bash -python code/run_squad.py \ - --do_train \ - --do_eval \ - --model spanbert-base-cased \ - --train_file train-v1.1.json \ - --dev_file dev-v1.1.json \ - --train_batch_size 32 \ - --eval_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 4 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --eval_metric f1 \ - --output_dir squad_output \ - --fp16 -``` - -## Results Comparison 📝 - -| | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | -| ---------------------- | ------------- | --------- | ------- | ------ | -| | F1 | F1 | avg. F1 | F1 | -| BERT (base) | 88.5 | 76.5 | 73.1 | 67.7 | -| SpanBERT (base) | **92.4** (this one) | [83.6](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | [68.2](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | -| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | -| SpanBERT (large) | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-large-finetuned-tacred) | - - -Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/spanbert-base-finetuned-squadv1", - tokenizer="SpanBERT/spanbert-base-cased" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working very hard in the repository hugginface/transformers lately", - 'question': "How has been working Manuel Romero lately?" - -}) - -# Output: {'answer': 'very hard in the repository hugginface/transformers', - 'end': 82, - 'score': 0.327230326857725, - 'start': 31} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-base-finetuned-squadv2/README.md b/model_cards/mrm8488/spanbert-base-finetuned-squadv2/README.md deleted file mode 100644 index 865c66c8b7..0000000000 --- a/model_cards/mrm8488/spanbert-base-finetuned-squadv2/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT base fine-tuned on SQuAD v2 - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task ([by them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution)). - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model fine-tuning 🏋️‍ - -You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) - -```bash -python code/run_squad.py \ - --do_train \ - --do_eval \ - --model spanbert-base-cased \ - --train_file train-v2.0.json \ - --dev_file dev-v2.0.json \ - --train_batch_size 32 \ - --eval_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 4 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --eval_metric best_f1 \ - --output_dir squad2_output \ - --version_2_with_negative \ - --fp16 -``` - -## Results Comparison 📝 - -| | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | -| ---------------------- | ------------- | --------- | ------- | ------ | -| | F1 | F1 | avg. F1 | F1 | -| BERT (base) | 88.5 | 76.5 | 73.1 | 67.7 | -| SpanBERT (base) | [92.4](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | **83.6** (this one) | 77.4 | [68.2](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | -| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | -| SpanBERT (large) | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-large-finetuned-tacred) | - - -Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/spanbert-base-finetuned-squadv2", - tokenizer="SpanBERT/spanbert-base-cased" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working very hard in the repository hugginface/transformers lately", - 'question': "How has been working Manuel Romero lately?" - -}) -# Output: {'answer': 'very hard', 'end': 40, 'score': 0.9052708846768347, 'start': 31} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-base-finetuned-tacred/README.md b/model_cards/mrm8488/spanbert-base-finetuned-tacred/README.md deleted file mode 100644 index 3ee1158b26..0000000000 --- a/model_cards/mrm8488/spanbert-base-finetuned-tacred/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT base fine-tuned on TACRED - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [TACRED](https://nlp.stanford.edu/projects/tacred/) dataset by [them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution) - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Dataset 📚 - -[TACRED](https://nlp.stanford.edu/projects/tacred/) A large-scale relation extraction dataset with 106k+ examples over 42 TAC KBP relation types. - -## Model fine-tuning 🏋️‍ - -You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) - -```bash -python code/run_tacred.py \ - --do_train \ - --do_eval \ - --data_dir \ - --model spanbert-base-cased \ - --train_batch_size 32 \ - --eval_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 10 \ - --max_seq_length 128 \ - --output_dir tacred_dir \ - --fp16 -``` - -## Results Comparison 📝 - -| | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | -| ---------------------- | ------------- | --------- | ------- | ------ | -| | F1 | F1 | avg. F1 | F1 | -| BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 | -| SpanBERT (base) | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | **68.2** (this one) | -| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | -| SpanBERT (large) | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | - - -Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-finetuned-squadv1/README.md b/model_cards/mrm8488/spanbert-finetuned-squadv1/README.md deleted file mode 100644 index a8584c2b2b..0000000000 --- a/model_cards/mrm8488/spanbert-finetuned-squadv1/README.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT (spanbert-base-cased) fine-tuned on SQuAD v1.1 - - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 1.1](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -## Details of SpanBERT - - A pre-training method that is designed to better represent and predict spans of text. - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Details of the downstream task (Q&A) - Dataset - -[SQuAD 1.1](https://rajpurkar.github.io/SQuAD-explorer/) contains 100,000+ question-answer pairs on 500+ articles. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD1.1 | train | 87.7k | -| SQuAD1.1 | eval | 10.6k | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py) - -## Results: - -| Metric | # Value | -| ------ | --------- | -| **EM** | **85.49** | -| **F1** | **91.98** | - -### Raw metrics: - -```json -{ - "exact": 85.49668874172185, - "f1": 91.9845699540379, - "total": 10570, - "HasAns_exact": 85.49668874172185, - "HasAns_f1": 91.9845699540379, - "HasAns_total": 10570, - "best_exact": 85.49668874172185, - "best_exact_thresh": 0.0, - "best_f1": 91.9845699540379, - "best_f1_thresh": 0.0 -} -``` - -## Comparison: - -| Model | EM | F1 score | -| ----------------------------------------------------------------------------------------- | --------- | --------- | -| [SpanBert official repo](https://github.com/facebookresearch/SpanBERT#pre-trained-models) | - | 92.4\* | -| [spanbert-finetuned-squadv1](https://huggingface.co/mrm8488/spanbert-finetuned-squadv1) | **85.49** | **91.98** | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/spanbert-finetuned-squadv1", - tokenizer="mrm8488/spanbert-finetuned-squadv1" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-finetuned-squadv2/README.md b/model_cards/mrm8488/spanbert-finetuned-squadv2/README.md deleted file mode 100644 index 74143a821d..0000000000 --- a/model_cards/mrm8488/spanbert-finetuned-squadv2/README.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT (spanbert-base-cased) fine-tuned on SQuAD v2 - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Details of the downstream task (Q&A) - Dataset - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py) - -## Results: - -| Metric | # Value | -| ------ | --------- | -| **EM** | **78.80** | -| **F1** | **82.22** | - -### Raw metrics: - -```json -{ - "exact": 78.80064010780762, - "f1": 82.22801347271162, - "total": 11873, - "HasAns_exact": 78.74493927125506, - "HasAns_f1": 85.60951483831069, - "HasAns_total": 5928, - "NoAns_exact": 78.85618166526493, - "NoAns_f1": 78.85618166526493, - "NoAns_total": 5945, - "best_exact": 78.80064010780762, - "best_exact_thresh": 0.0, - "best_f1": 82.2280134727116, - "best_f1_thresh": 0.0 -} -``` - -## Comparison: - -| Model | EM | F1 score | -| ----------------------------------------------------------------------------------------- | --------- | --------- | -| [SpanBert official repo](https://github.com/facebookresearch/SpanBERT#pre-trained-models) | - | 83.6\* | -| [spanbert-finetuned-squadv2](https://huggingface.co/mrm8488/spanbert-finetuned-squadv2) | **78.80** | **82.22** | - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/spanbert-finetuned-squadv2", - tokenizer="mrm8488/spanbert-finetuned-squadv2" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" - -}) - -# Output: {'answer': 'Manuel Romero','end': 13,'score': 6.836378586818937e-09, 'start': 0} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-large-finetuned-squadv1/README.md b/model_cards/mrm8488/spanbert-large-finetuned-squadv1/README.md deleted file mode 100644 index 3bfc6ef42f..0000000000 --- a/model_cards/mrm8488/spanbert-large-finetuned-squadv1/README.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT large fine-tuned on SQuAD v1 - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 1.1](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task ([by them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution)). - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -[SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer/) - -## Model fine-tuning 🏋️‍ - -You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) - -```bash -python code/run_squad.py \ - --do_train \ - --do_eval \ - --model spanbert-large-cased \ - --train_file train-v1.1.json \ - --dev_file dev-v1.1.json \ - --train_batch_size 32 \ - --eval_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 4 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --eval_metric f1 \ - --output_dir squad_output \ - --fp16 -``` - -## Results Comparison 📝 - -| | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | -| ---------------------- | ------------- | --------- | ------- | ------ | -| | F1 | F1 | avg. F1 | F1 | -| BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 | -| SpanBERT (base) | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | [68.2](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | -| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | -| SpanBERT (large) | **94.6** (this) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-large-finetuned-tacred) | - - -Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/spanbert-large-finetuned-squadv1", - tokenizer="SpanBERT/spanbert-large-cased" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working very hard in the repository hugginface/transformers lately", - 'question': "How has been working Manuel Romero lately?" - -}) - -# Output: {'answer': 'very hard in the repository hugginface/transformers', - 'end': 82, - 'score': 0.327230326857725, - 'start': 31} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-large-finetuned-squadv2/README.md b/model_cards/mrm8488/spanbert-large-finetuned-squadv2/README.md deleted file mode 100644 index 51b9d3ae79..0000000000 --- a/model_cards/mrm8488/spanbert-large-finetuned-squadv2/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT large fine-tuned on SQuAD v2 - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task ([by them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution)). - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -[SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/) combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| SQuAD2.0 | train | 130k | -| SQuAD2.0 | eval | 12.3k | - -## Model fine-tuning 🏋️‍ - -You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) - -```bash -python code/run_squad.py \ - --do_train \ - --do_eval \ - --model spanbert-large-cased \ - --train_file train-v2.0.json \ - --dev_file dev-v2.0.json \ - --train_batch_size 32 \ - --eval_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 4 \ - --max_seq_length 512 \ - --doc_stride 128 \ - --eval_metric best_f1 \ - --output_dir squad2_output \ - --version_2_with_negative \ - --fp16 -``` - -## Results Comparison 📝 - -| | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | -| ---------------------- | ------------- | --------- | ------- | ------ | -| | F1 | F1 | avg. F1 | F1 | -| BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 | -| SpanBERT (base) | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | [68.2](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | -| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | -| SpanBERT (large) | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1) | **88.7** (this) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-large-finetuned-tacred) | - - -Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/spanbert-large-finetuned-squadv2", - tokenizer="SpanBERT/spanbert-large-cased" -) - -qa_pipeline({ - 'context': "Manuel Romero has been working very hard in the repository hugginface/transformers lately", - 'question': "How has been working Manuel Romero lately?" - -}) -# Output: {'answer': 'very hard', 'end': 40, 'score': 0.9052708846768347, 'start': 31} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/spanbert-large-finetuned-tacred/README.md b/model_cards/mrm8488/spanbert-large-finetuned-tacred/README.md deleted file mode 100644 index 826c69be1c..0000000000 --- a/model_cards/mrm8488/spanbert-large-finetuned-tacred/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: en -thumbnail: ---- - -# SpanBERT large fine-tuned on TACRED - -[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [TACRED](https://nlp.stanford.edu/projects/tacred/) dataset by [them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution) - -## Details of SpanBERT - -[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) - -## Dataset 📚 - -[TACRED](https://nlp.stanford.edu/projects/tacred/) A large-scale relation extraction dataset with 106k+ examples over 42 TAC KBP relation types. - -## Model fine-tuning 🏋️‍ - -You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) - -```bash -python code/run_tacred.py \ - --do_train \ - --do_eval \ - --data_dir \ - --model spanbert-large-cased \ - --train_batch_size 32 \ - --eval_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 10 \ - --max_seq_length 128 \ - --output_dir tacred_dir \ - --fp16 -``` - -## Results Comparison 📝 - -| | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | -| ---------------------- | ------------- | --------- | ------- | ------ | -| | F1 | F1 | avg. F1 | F1 | -| BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 | -| SpanBERT (base) | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | [68.2](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | -| BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | -| SpanBERT (large) | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | **70.8** (this one) | - - -Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/squeezebert-finetuned-squadv1/README.md b/model_cards/mrm8488/squeezebert-finetuned-squadv1/README.md deleted file mode 100644 index 4bcf9771b4..0000000000 --- a/model_cards/mrm8488/squeezebert-finetuned-squadv1/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -language: en -datasets: -- squad ---- - -# SqueezeBERT + SQuAD (v1.1) - -[squeezebert-uncased](https://huggingface.co/squeezebert/squeezebert-uncased) fine-tuned on [SQUAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task. - -## Details of SqueezeBERT - -This model, `squeezebert-uncased`, is a pretrained model for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective. -SqueezeBERT was introduced in [this paper](https://arxiv.org/abs/2006.11316). This model is case-insensitive. The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with [grouped convolutions](https://blog.yani.io/filter-group-tutorial/). -The authors found that SqueezeBERT is 4.3x faster than `bert-base-uncased` on a Google Pixel 3 smartphone. -More about the model [here](https://arxiv.org/abs/2004.02984) - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -**S**tanford **Q**uestion **A**nswering **D**ataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable. -SQuAD v1.1 contains **100,000+** question-answer pairs on **500+** articles. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python /content/transformers/examples/question-answering/run_squad.py \ - --model_type bert \ - --model_name_or_path squeezebert/squeezebert-uncased \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file /content/dataset/train-v1.1.json \ - --predict_file /content/dataset/dev-v1.1.json \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 15 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/output_dir \ - --overwrite_output_dir \ - --save_steps 2000 -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **76.66** | -| **F1** | **85.83** | - -Model Size: **195 MB** - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline -QnA_pipeline = pipeline('question-answering', model='mrm8488/squeezebert-finetuned-squadv1') -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'Who did identified it ?' - }) - -# Output: {'answer': 'scientists.', 'end': 106, 'score': 0.6988425850868225, 'start': 96} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/squeezebert-finetuned-squadv2/README.md b/model_cards/mrm8488/squeezebert-finetuned-squadv2/README.md deleted file mode 100644 index 2f8634d6f2..0000000000 --- a/model_cards/mrm8488/squeezebert-finetuned-squadv2/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -language: en -datasets: -- squad_v2 ---- - -# SqueezeBERT + SQuAD v2 - -[squeezebert-uncased](https://huggingface.co/squeezebert/squeezebert-uncased) fine-tuned on [SQUAD v2](https://rajpurkar.github.io/SQuAD-explorer/explore/v2.0/dev/) for **Q&A** downstream task. - -## Details of SqueezeBERT - -This model, `squeezebert-uncased`, is a pretrained model for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective. -SqueezeBERT was introduced in [this paper](https://arxiv.org/abs/2006.11316). This model is case-insensitive. The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with [grouped convolutions](https://blog.yani.io/filter-group-tutorial/). -The authors found that SqueezeBERT is 4.3x faster than `bert-base-uncased` on a Google Pixel 3 smartphone. -More about the model [here](https://arxiv.org/abs/2004.02984) - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -**SQuAD2.0** combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python /content/transformers/examples/question-answering/run_squad.py \ - --model_type bert \ - --model_name_or_path squeezebert/squeezebert-uncased \ - --do_train \ - --do_eval \ - --do_lower_case \ - --train_file /content/dataset/train-v2.0.json \ - --predict_file /content/dataset/dev-v2.0.json \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 15 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/output_dir \ - --overwrite_output_dir \ - --version_2_with_negative \ - --save_steps 2000 -``` - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **69.98** | -| **F1** | **74.14** | - -Model Size: **195 MB** - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline -QnA_pipeline = pipeline('question-answering', model='mrm8488/squeezebert-finetuned-squadv2') -QnA_pipeline({ - 'context': 'A new strain of flu that has the potential to become a pandemic has been identified in China by scientists.', - 'question': 'Who did identified it ?' - }) - -# Output: {'answer': 'scientists.', 'end': 106, 'score': 0.9768241047859192, 'start': 96} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-break_data-question-retrieval/README.md b/model_cards/mrm8488/t5-base-finetuned-break_data-question-retrieval/README.md deleted file mode 100644 index 2cdf91f9f9..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-break_data-question-retrieval/README.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -language: en -datasets: -- break_data ---- - -# T5-base fine-tuned on break_data / QDMR-high-level 📋➡️❓ - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [break_data](https://huggingface.co/nlp/viewer/?dataset=break_data&config=QDMR-high-level) dataset for **Question Retrieval from its decomposition**. -The inverse process of [this model](https://huggingface.co/mrm8488/t5-base-finetuned-break_data). - -## Details of T5 📜 ➡️ 📜 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Question Retrieval from its decomposition) - Dataset 📚 - -Break is a human annotated dataset of natural language questions and their Question Decomposition Meaning Representations (QDMRs). Break consists of 83,978 examples sampled from 10 question answering datasets over text, images and databases. This repository contains the Break dataset along with information on the exact data format. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| break_data | train | 17503 | -| break_data | valid | 3130 | - -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28). The main change is at preprocessing ```inputs``` and ```targets``` we feed to the model. We do it as a *paraphrasing task*. - - -## Model in Action 🚀 - -```python -# Tip: By now, install transformers from source - -from transformers import AutoModelForSeq2SeqLM, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-break_data-question-retrieval") -model = AutoModelForSeq2SeqLM.from_pretrained("mrm8488/t5-base-finetuned-break_data-question-retrieval") - - -def get_nautural_question(decomposition): - input_text = 'translate QDMRs to Natural Language %s ' % decomposition - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=64) - - return tokenizer.decode(output[0]) - - - -decomposition = "return the city that was the birthplace of Bernard Berrian ;return the city that was the home of Pablo Picasso ;return the city of both #1 and #2" - -# Ground Truth: What city was the birthplace of Bernard Berrianand the home of Pablo Picasso? - -get_nautural_question(decomposition) - -# output: 'What city was the birthplace of Bernard Berrian and the home of Pablo Picasso?' -``` -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-break_data/README.md b/model_cards/mrm8488/t5-base-finetuned-break_data/README.md deleted file mode 100644 index f875d242c1..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-break_data/README.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -language: en -datasets: -- break_data ---- - -# T5-base fine-tuned on break_data / QDMR-high-level ❓➡️📋 - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [break_data](https://huggingface.co/nlp/viewer/?dataset=break_data&config=QDMR-high-level) dataset for **QDMRs**. - -## Details of T5 📜 ➡️ 📜 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (QDMRs) - Dataset 📚 - -Break is a human annotated dataset of natural language questions and their Question Decomposition Meaning Representations (QDMRs). Break consists of 83,978 examples sampled from 10 question answering datasets over text, images and databases. This repository contains the Break dataset along with information on the exact data format. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| break_data | train | 17503 | -| break_data | valid | 3130 | - -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28). The main change is at preprocessing ```inputs``` and ```targets``` we feed to the model. We do it as a *paraphrasing task*. - - -## Model in Action 🚀 - -```python -# Tip: By now, install transformers from source - -from transformers import AutoModelForSeq2SeqLM, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-break_data") -model = AutoModelForSeq2SeqLM.from_pretrained("mrm8488/t5-base-finetuned-break_data") - -def get_decomposition(question): - input_text = "paraphrase: %s " % question - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=32) - - return tokenizer.decode(output[0]) - -question = "The composer of Sands Theme plays what type of guitar?" - -get_decomposition(question) - -# output: 'return Sands Theme ;return composer of #1 ;return guitar that #2 plays' -``` -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-common_gen/README.md b/model_cards/mrm8488/t5-base-finetuned-common_gen/README.md deleted file mode 100644 index f385d21af8..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-common_gen/README.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -language: en -datasets: -- common_gen ---- - -# T5-base fine-tuned on CommonGen - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [CommonGen](https://inklab.usc.edu/CommonGen/index.html) for *Generative Commonsense Reasoning*. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the dataset 📚 - -CommonGen is a constrained text generation task, associated with a benchmark dataset, to explicitly test machines for the ability of generative commonsense reasoning. Given a set of common concepts; the task is to generate a coherent sentence describing an everyday scenario using these concepts. - -CommonGen is challenging because it inherently requires 1) relational reasoning using background commonsense knowledge, and 2) compositional generalization ability to work on unseen concept combinations. Our dataset, constructed through a combination of crowd-sourcing from AMT and existing caption corpora, consists of 30k concept-sets and 50k sentences in total. - - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| common_gen | train | 67389 | -| common_gen | valid | 4018 | -| common_gen | test | 1497 | - - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28) - -## Metrics 📋 - -| Metric | Score | -|--------|-------| -|ROUGE-2 | 17.10 | -|ROUGE-L | 39.47 | -|BLEU | WIP | - -The metrics above slightly improves results shown in the [paper](https://arxiv.org/abs/1911.03705) for the same model and metrics. - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-common_gen") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-common_gen") - -def gen_sentence(words, max_length=32): - input_text = words - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=max_length) - - return tokenizer.decode(output[0]) - -words = "tree plant ground hole dig" - -gen_sentence(words) - -# output: digging a hole in the ground to plant trees -``` -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mrm8488/shared_colab_notebooks/blob/master/T5_base_finetuned_common_gen.ipynb) - - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-e2m-intent/README.md b/model_cards/mrm8488/t5-base-finetuned-e2m-intent/README.md deleted file mode 100644 index 8f9624f6ae..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-e2m-intent/README.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -language: en -datasets: -- event2Mind ---- - -# T5-base fine-tuned on event2Mind for **Intent Prediction** 🤔 - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [event2Mind](https://huggingface.co/nlp/viewer/?dataset=event2Mind) dataset for **Intent Prediction**. - -## Details of T5 📜 ➡️ 📜 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Intent Prediction) - Dataset 📚 - -Dataset ID: ```event2Mind``` from [Huggingface/NLP](https://github.com/huggingface/nlp) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| event2Mind | train | 46472 | -| event2Mind | valid | 1960 | - -Events without **intent** were not used! - -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28). - - -## Model in Action 🚀 - -```python -# Tip: By now, install transformers from source - -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-e2m-intent") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-e2m-intent") - -def get_intent(event, max_length=16): - input_text = "%s " % event - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=max_length) - - return tokenizer.decode(output[0]) - -event = "PersonX takes PersonY home" -get_intent(event) - -# output: 'to be helpful' -``` -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-emotion/README.md b/model_cards/mrm8488/t5-base-finetuned-emotion/README.md deleted file mode 100644 index 78c1e3265e..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-emotion/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: en -datasets: -- emotion ---- - -# T5-base fine-tuned for Emotion Recognition 😂😢😡😃😯 - - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) base fine-tuned on [emotion recognition](https://github.com/dair-ai/emotion_dataset) dataset for **Emotion Recognition** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - -## Details of the downstream task (Sentiment Recognition) - Dataset 📚 - -[Elvis Saravia](https://twitter.com/omarsar0) has gathered a great [dataset](https://github.com/dair-ai/emotion_dataset) for emotion recognition. It allows to classifiy the text into one of the following **6** emotions: - - - sadness 😢 - - joy 😃 - - love 🥰 - - anger 😡 - - fear 😱 - - surprise 😯 - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - -## Test set metrics 🧾 - -| |precision | recall | f1-score |support| -|----------|----------|---------|----------|-------| -|anger | 0.93| 0.92| 0.93| 275| -|fear | 0.91| 0.87| 0.89| 224| -|joy | 0.97| 0.94| 0.95| 695| -|love | 0.80| 0.91| 0.85| 159| -|sadness | 0.97| 0.97| 0.97| 521| -|surpirse | 0.73| 0.89| 0.80| 66| -| | -|accuracy| | | 0.93| 2000| -|macro avg| 0.89| 0.92| 0.90| 2000| -|weighted avg| 0.94| 0.93| 0.93| 2000| - - - - - - - -## Model in Action 🚀 - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-emotion") - -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-emotion") - -def get_emotion(text): - input_ids = tokenizer.encode(text + '', return_tensors='pt') - - output = model.generate(input_ids=input_ids, - max_length=2) - - dec = [tokenizer.decode(ids) for ids in output] - label = dec[0] - return label - - get_emotion("i feel as if i havent blogged in ages are at least truly blogged i am doing an update cute") # Output: 'joy' - - get_emotion("i have a feeling i kinda lost my best friend") # Output: 'sadness' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-imdb-sentiment/README.md b/model_cards/mrm8488/t5-base-finetuned-imdb-sentiment/README.md deleted file mode 100644 index 85c5a07f33..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-imdb-sentiment/README.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -language: en -datasets: -- imdb ---- - -# T5-base fine-tuned for Sentiment Anlalysis 🎞️👍👎 - - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) base fine-tuned on [IMDB](https://huggingface.co/datasets/imdb) dataset for **Sentiment Analysis** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - -## Details of the downstream task (Sentiment analysis) - Dataset 📚 - -[IMDB](https://huggingface.co/datasets/imdb) - -This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of **25,000** highly polar movie reviews for training, and **25,000** for testing. - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - -## Test set metrics 🧾 - - |precision | recall | f1-score |support| - |----------|----------|---------|----------|-------| - |negative | 0.95 | 0.95| 0.95| 12500| - |positive | 0.95 | 0.95| 0.95| 12500| - |----------|----------|---------|----------|-------| - |accuracy| | | 0.95| 25000| - |macro avg| 0.95| 0.95| 0.95| 25000| - |weighted avg| 0.95| 0.95| 0.95 | 25000| - - -## Model in Action 🚀 - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-imdb-sentiment") - -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-imdb-sentiment") - -def get_sentiment(text): - input_ids = tokenizer.encode(text + '', return_tensors='pt') - - output = model.generate(input_ids=input_ids, - max_length=2) - - dec = [tokenizer.decode(ids) for ids in output] - label = dec[0] - return label - -get_sentiment("I dislike a lot that film") - -# Output: 'negative' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-qasc/README.md b/model_cards/mrm8488/t5-base-finetuned-qasc/README.md deleted file mode 100644 index 7259477c2a..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-qasc/README.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -language: en -datasets: -- qasc ---- - -# T5-base fine-tuned on QASC - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [QASC](https://allenai.org/data/qasc) for **QA** (via *sentence composition*) downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the dataset 📚 - -**Question Answering via Sentence Composition** (QASC) is a question-answering dataset with a focus on sentence composition. It consists of 9,980 8-way multiple-choice questions about grade school science (8,134 train, 926 dev, 920 test), and comes with a corpus of 17M sentences. - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28). The **context** passed to the *encoder* is the combination of the 2 *facts* (`fact1` and `fact2`). The **question** is just the `formatted_question` field. The **answer** passed to the *decoder* is the`text` right answer instead of the `label` (A, B, C... See `choices` field). More details about the dataset format/fields [here](https://huggingface.co/nlp/viewer/?dataset=qasc) - -## Metrics on validation set 📋 - -| Metric | Score | -|--------|-------| -|Accuracy (EM) | **97.73**| - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-qasc") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-qasc") - -def get_response(question, context, max_length=64): - input_text = 'question: %s context: %s' % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=max_length) - - return tokenizer.decode(output[0]) - -fact_1 = 'a watch is used for measuring time' -fact_2 = 'Times are measured in seconds.' -context = fact_1 + ' ' + fact_2 -question = 'What can be used to measure seconds? (A) Watch (B) seconds (C) fluid (D) Ruler (E) goggles (F) glasses (G) Drill (H) Scale' - -get_response(question, context) - -# output: 'Watch' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-quarel/README.md b/model_cards/mrm8488/t5-base-finetuned-quarel/README.md deleted file mode 100644 index 780e6735ed..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-quarel/README.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -language: en -datasets: -- quarel ---- - -# T5-base fine-tuned on QuaRel - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [QuaRel](https://allenai.org/data/quarel) for **QA** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the dataset 📚 - -**QuaRel**: *[A Dataset and Models for Answering Questions about Qualitative Relationships](https://www.semanticscholar.org/paper/QuaRel%3A-A-Dataset-and-Models-for-Answering-about-Tafjord-Clark/51004bc6461a572e1189a0e3b32b441155d760ce)* - -Many natural language questions require recognizing and reasoning with qualitative relationships (e.g., in science, economics, and medicine), but are challenging to answer with corpus-based methods. Qualitative modeling provides tools that support such reasoning, but the semantic parsing task of mapping questions into those models has formidable challenges. We present QuaRel, a dataset of diverse story questions involving qualitative relationships that characterize these challenges, and techniques that begin to address them. The dataset has 2771 questions relating 19 different types of quantities. For example, "Jenny observes that the robot vacuum cleaner moves slower on the living room carpet than on the bedroom carpet. Which carpet has more friction?" We contribute (1) a simple and flexible conceptual framework for representing these kinds of questions; (2) the QuaRel dataset, including logical forms, exemplifying the parsing challenges; and (3) two novel models for this task, built as extensions of type-constrained semantic parsing. The first of these models (called QuaSP+) significantly outperforms off-the-shelf tools on QuaRel. The second (QuaSP+Zero) demonstrates zero-shot capability, i.e., the ability to handle new qualitative relationships without requiring additional training data, something not possible with previous models. This work thus makes inroads into answering complex, qualitative questions that require reasoning, and scaling to new relationships at low cost - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28). The **context** passed to the *encoder* is the `logical_form_pretty` field (example: `qrel(speed, higher, ice) -> qrel(smoothness, higher, snow) ; qrel(smoothness, higher, ice`) . The **question** is just the `question` field. The **answer** passed to the *decoder* is obtained from `question`using the `answer_index` field. More details about the dataset format/fields [here](https://huggingface.co/nlp/viewer/?dataset=quarel) - -## Metrics on validation set 📋 - -| Metric | Score | -|--------|-------| -|Accuracy (EM) | **67.98**| - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-quarel") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-quarel") - -def get_response(question, context, max_length=32): - input_text = 'question: %s context: %s' % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=max_length) - - return tokenizer.decode(output[0]) - -question = 'As the train left the station it crossed the bridge and being farther away it looked (A) larger (B) smaller' -context = 'qrel(distance, higher, Train on a bridge) -> qrel(apparentSize, higher, Train on a bridge) ; qrel(apparentSize, lower, Train on a bridge)' - -get_response(question, context) - -# output: 'smaller' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-quartz/README.md b/model_cards/mrm8488/t5-base-finetuned-quartz/README.md deleted file mode 100644 index b571291f1e..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-quartz/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: en -datasets: -- quartz -pipeline_tag: question-answering ---- - -# T5-base fine-tuned on QuaRTz - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [QuaRTz](https://allenai.org/data/quartz) for **QA** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the dataset 📚 - -**QuaRTz** is a crowdsourced dataset of 3864 multiple-choice questions about open domain qualitative relationships. Each question is paired with one of 405 different background sentences (sometimes short paragraphs). The QuaRTz dataset V1 contains 3864 questions about open domain qualitative relationships. Each question is paired with one of 405 different background sentences (sometimes short paragraphs). -The dataset is split into: - -|Set | Samples| -|-----|--------| -|Train | 2696 | -|Valid | 384 | -|Test | 784 | - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28). The *question*, *context* (`para` field) and *options* (`choices` field) are concatenated and passed to the **encoder**. The **decoder** receives the right *answer* (by querying `answerKey` field). More details about the dataset fields/format [here](https://huggingface.co/nlp/viewer/?dataset=quartz) - -## Results 📋 - - -|Set | Metric | Score | -|-----|--------|-------| -|Validation | Accuracy (EM) | **83.59**| -|Test | Accuracy (EM) | **81.50**| - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-quartz") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-quartz") - -def get_response(question, fact, opts, max_length=16): - input_text = 'question: %s context: %s options: %s' % (question, fact, opts) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=max_length) - - return tokenizer.decode(output[0]) - -fact = 'The sooner cancer is detected the easier it is to treat.' -question = 'John was a doctor in a cancer ward and knew that early detection was key. The cancer being detected quickly makes the cancer treatment' -opts = 'Easier, Harder' - -get_response(question, fact, opts) - -# output: 'Easier' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-question-generation-ap/README.md b/model_cards/mrm8488/t5-base-finetuned-question-generation-ap/README.md deleted file mode 100644 index 524b1ad04b..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-question-generation-ap/README.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -language: en -datasets: -- squad ---- - -# T5-base fine-tuned on SQuAD for **Question Generation** - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [SQuAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/) for **Question Generation** by just prepending the *answer* to the *context*. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -Dataset ID: ```squad``` from [Huggingface/NLP](https://github.com/huggingface/nlp) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| squad | train | 87599 | -| squad | valid | 10570 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('squad', split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('squad', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28) - -He also made a great research on [**Question Generation**](https://github.com/patil-suraj/question_generation) - - -## Model in Action 🚀 - -```python -# Tip: By now, install transformers from source - -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-question-generation-ap") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-question-generation-ap") - -def get_question(answer, context, max_length=64): - input_text = "answer: %s context: %s " % (answer, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask'], - max_length=max_length) - - return tokenizer.decode(output[0]) - -context = "Manuel have created RuPERTa-base with the support of HF-Transformers and Google" -answer = "Manuel" - -get_question(answer, context) - -# output: question: Who created the RuPERTa-base? -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-sarcasm-twitter/README.md b/model_cards/mrm8488/t5-base-finetuned-sarcasm-twitter/README.md deleted file mode 100644 index 2889f47971..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-sarcasm-twitter/README.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -language: en ---- - -# T5-base fine-tuned for Sarcasm Detection 🙄 -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) base fine-tuned on [ Twitter Sarcasm Dataset](https://github.com/EducationalTestingService/sarcasm) for **Sequence classification (as text generation)** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) -## Details of the downstream task (Sequence Classification as Text generation) - Dataset 📚 - -[ Twitter Sarcasm Dataset](https://github.com/EducationalTestingService/sarcasm) - - -For Twitter training and testing datasets are provided for sarcasm detection tasks in jsonlines format. - -Each line contains a JSON object with the following fields : -- ***label*** : `SARCASM` or `NOT_SARCASM` - - **NOT** in test data -- ***id***: String identifier for sample. This id will be required when making submissions. - - **ONLY** in test data -- ***response*** : the sarcastic response, whether a sarcastic Tweet -- ***context*** : the conversation context of the ***response*** - - Note, the context is an ordered list of dialogue, i.e., if the context contains three elements, `c1`, `c2`, `c3`, in that order, then `c2` is a reply to `c1` and `c3` is a reply to `c2`. Further, if the sarcastic response is `r`, then `r` is a reply to `c3`. - -For instance, for the following training example : - -`"label": "SARCASM", "response": "Did Kelly just call someone else messy? Baaaahaaahahahaha", "context": ["X is looking a First Lady should . #classact, "didn't think it was tailored enough it looked messy"]` - -The response tweet, "Did Kelly..." is a reply to its immediate context "didn't think it was tailored..." which is a reply to "X is looking...". Your goal is to predict the label of the "response" while also using the context (i.e, the immediate or the full context). - -***Dataset size statistics*** : - -| | Train | Val | Test | -|---------|-------|------|------| -| Twitter | 4050 | 450 | 500 | - -The datasets was preprocessed to convert it to a **text-to-text** (classfication as generation task). - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - -## Test set metrics 🧾 - -| | precision| recall | f1-score |support| -|----------|----------|---------|----------|-------| -| derison | 0.84 | 0.80 | 0.82 | 246 | -| normal | 0.82 | 0.85 | 0.83 | 254 | -| | -|accuracy| | | 0.83| 500| -|macro avg| 0.83| 0.83| 0.83| 500| -|weighted avg| 0.83| 0.83| 0.83| 500| - - - -## Model in Action 🚀 - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-sarcasm-twitter") - -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-sarcasm-twitter") - -def eval_conversation(text): - - input_ids = tokenizer.encode(text + '', return_tensors='pt') - - output = model.generate(input_ids=input_ids, max_length=3) - - dec = [tokenizer.decode(ids) for ids in output] - - label = dec[0] - - return label - -# For similarity with the training dataset we should replace users mentions in twits for @USER token and urls for URL token. - -twit1 = "Trump just suspended the visa program that allowed me to move to the US to start @USER!" + -" Unfortunately, I won’t be able to vote in a few months but if you can, please vote him out, " + -"he's destroying what made America great in so many different ways!" - -twit2 = "@USER @USER @USER We have far more cases than any other country, " + -"so leaving remote workers in would be disastrous. Makes Trump sense." - -twit3 = "My worry is that i wouldn’t be surprised if half the country actually agrees with this move..." - -me = "Trump doing so??? It must be a mistake... XDDD" - -conversation = twit1 + twit2 - -eval_conversation(conversation) #Output: 'derison' - -conversation = twit1 + twit3 - -eval_conversation(conversation) #Output: 'normal' - -conversation = twit1 + me - -eval_conversation(conversation) #Output: 'derison' - -# We will get 'normal' when sarcasm is not detected and 'derison' when detected -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-span-sentiment-extraction/README.md b/model_cards/mrm8488/t5-base-finetuned-span-sentiment-extraction/README.md deleted file mode 100644 index 740f87dc6d..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-span-sentiment-extraction/README.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -language: en -thumbnail: ---- - -# T5-base fine-tuned for Sentiment Span Extraction - -All credits to [Lorenzo Ampil](https://twitter.com/AND__SO) - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) base fine-tuned on [Tweet Sentiment Extraction Dataset](https://www.kaggle.com/c/tweet-sentiment-extraction) for **Span Sentiment Extraction** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - - -## Details of the downstream task (Span Sentiment Extraction) - Dataset 📚 - -[Tweet Sentiment Extraction Dataset](https://www.kaggle.com/c/tweet-sentiment-extraction) - -"My ridiculous dog is amazing." [sentiment: positive] - -With all of the tweets circulating every second it is hard to tell whether the sentiment behind a specific tweet will impact a company, or a person's, brand for being viral (positive), or devastate profit because it strikes a negative tone. Capturing sentiment in language is important in these times where decisions and reactions are created and updated in seconds. But, which words actually lead to the sentiment description? In this competition you will need to pick out the part of the tweet (word or phrase) that reflects the sentiment. - -Help build your skills in this important area with this broad dataset of tweets. Work on your technique to grab a top spot in this competition. What words in tweets support a positive, negative, or neutral sentiment? How can you help make that determination using machine learning tools? - -In this competition we've extracted support phrases from Figure Eight's Data for Everyone platform. The dataset is titled Sentiment Analysis: Emotion in Text tweets with existing sentiment labels, used here under creative commons attribution 4.0. international licence. Your objective in this competition is to construct a model that can do the same - look at the labeled sentiment for a given tweet and figure out what word or phrase best supports it. - -Disclaimer: The dataset for this competition contains text that may be considered profane, vulgar, or offensive. - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| TSE | train | 23907 | -| TSE | eval | 3573 | - - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/enzoampil/t5-intro/blob/master/t5_qa_training_pytorch_span_extraction.ipynb) created by [Lorenzo Ampil](https://github.com/enzoampil), so all credits to him! - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-span-sentiment-extraction") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-span-sentiment-extraction") - -def get_sentiment_span(text): - input_ids = tokenizer.encode(text, return_tensors="pt", add_special_tokens=True) # Batch size 1 - - generated_ids = model.generate(input_ids=input_ids, num_beams=1, max_length=80).squeeze() - - predicted_span = tokenizer.decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True) - - return predicted_span - -get_sentiment_span("question: negative context: My bike was put on hold...should have known that.... argh total bummer") - -# output: 'argh total bummer' - -get_sentiment_span("question: positive context: On the monday, so i wont be able to be with you! i love you") - -# output: 'i love you' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-squadv2/README.md b/model_cards/mrm8488/t5-base-finetuned-squadv2/README.md deleted file mode 100644 index b20d8fc17d..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-squadv2/README.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -language: en -datasets: -- squad_v2 ---- - -# T5-base fine-tuned on SQuAD v2 - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [SQuAD v2](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -Dataset ID: ```squad_v2``` from [Huggingface/NLP](https://github.com/huggingface/nlp) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| squad_v2 | train | 130319 | -| squad_v2 | valid | 11873 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('squad_v2', split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('squad_v2', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) - -## Results 📝 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **77.64** | -| **F1** | **81.32** | - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelForSeq2SeqLM, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-squadv2") -model = AutoModelForSeq2SeqLM.from_pretrained("mrm8488/t5-base-finetuned-squadv2") - -def get_answer(question, context): - input_text = "question: %s context: %s" % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(output[0]) - -context = "Manuel have created RuPERTa-base with the support of HF-Transformers and Google" -question = "Who has supported Manuel?" - -get_answer(question, context) - -# output: 'HF-Transformers and Google' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-summarize-news/README.md b/model_cards/mrm8488/t5-base-finetuned-summarize-news/README.md deleted file mode 100644 index 9798267440..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-summarize-news/README.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -language: en -thumbnail: ---- - -# T5-base fine-tuned fo News Summarization 📖✏️🧾 - -All credits to [Abhishek Kumar Mishra](https://github.com/abhimishra91) - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) base fine-tuned on [News Summary](https://www.kaggle.com/sunnysai12345/news-summary) dataset for **summarization** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - -## Details of the downstream task (Summarization) - Dataset 📚 - -[News Summary](https://www.kaggle.com/sunnysai12345/news-summary) - -The dataset consists of **4515 examples** and contains Author_name, Headlines, Url of Article, Short text, Complete Article. I gathered the summarized news from Inshorts and only scraped the news articles from Hindu, Indian times and Guardian. Time period ranges from febrauary to august 2017. - - - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/abhimishra91/transformers-tutorials/blob/master/transformers_summarization_wandb.ipynb) created by [Abhishek Kumar Mishra](https://github.com/abhimishra91), so all credits to him! -I also trained the model for more epochs (6). - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-summarize-news") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-summarize-news") - -def summarize(text, max_length=150): - input_ids = tokenizer.encode(text, return_tensors="pt", add_special_tokens=True) - - generated_ids = model.generate(input_ids=input_ids, num_beams=2, max_length=max_length, repetition_penalty=2.5, length_penalty=1.0, early_stopping=True) - - preds = [tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in generated_ids] - - return preds[0] -``` -Given the following article from **NYT** (2020/06/09) with title *George Floyd’s death energized a movement. He will be buried in Houston today*: - -After the sound and the fury, weeks of demonstrations and anguished calls for racial justice, the man whose death gave rise to an international movement, and whose last words — “I can’t breathe” — have been a rallying cry, will be laid to rest on Tuesday at a private funeral in Houston.George Floyd, who was 46, will then be buried in a grave next to his mother’s.The service, scheduled to begin at 11 a.m. at the Fountain of Praise church, comes after five days of public memorials in Minneapolis, North Carolina and Houston and two weeks after a Minneapolis police officer was caught on video pressing his knee into Mr. Floyd’s neck for nearly nine minutes before Mr. Floyd died. That officer, Derek Chauvin, has been charged with second-degree murder and second-degree manslaughter. His bail was set at $1.25 million in a court appearance on Monday. The outpouring of anger and outrage after Mr. Floyd’s death — and the speed at which protests spread from tense, chaotic demonstrations in the city where he died to an international movement from Rome to Rio de Janeiro — has reflected the depth of frustration borne of years of watching black people die at the hands of the police or vigilantes while calls for change went unmet. -``` -summarize('After the sound and the fury, weeks of demonstrations and anguished calls for racial justice, the man whose death gave rise to an international movement, and whose last words — “I can’t breathe” — have been a rallying cry, will be laid to rest on Tuesday at a private funeral in Houston.George Floyd, who was 46, will then be buried in a grave next to his mother’s.The service, scheduled to begin at 11 a.m. at the Fountain of Praise church, comes after five days of public memorials in Minneapolis, North Carolina and Houston and two weeks after a Minneapolis police officer was caught on video pressing his knee into Mr. Floyd’s neck for nearly nine minutes before Mr. Floyd died. That officer, Derek Chauvin, has been charged with second-degree murder and second-degree manslaughter. His bail was set at $1.25 million in a court appearance on Monday. The outpouring of anger and outrage after Mr. Floyd’s death — and the speed at which protests spread from tense, chaotic demonstrations in the city where he died to an international movement from Rome to Rio de Janeiro — has reflected the depth of frustration borne of years of watching black people die at the hands of the police or vigilantes while calls for change went unmet.', 80) -``` -We would obtain: - -At a private funeral in Houston. Floyd, who was 46 years old when his death occurred, will be buried next to the grave of his mother. A Minnesota police officer was caught on video pressing his knee into Mr's neck for nearly nine minutes before his death. The officer has been charged with second-degree manslaughter and $1.2 million bail is set at - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-wikiSQL-sql-to-en/README.md b/model_cards/mrm8488/t5-base-finetuned-wikiSQL-sql-to-en/README.md deleted file mode 100644 index 3f012c771a..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-wikiSQL-sql-to-en/README.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -language: en -datasets: -- wikisql ---- - -# T5-base fine-tuned on WikiSQL for SQL to English translation - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [WikiSQL](https://github.com/salesforce/WikiSQL) for **SQL** to **English** **translation** task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the Dataset 📚 - -Dataset ID: ```wikisql``` from [Huggingface/NLP](https://huggingface.co/nlp/viewer/?dataset=wikisql) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| wikisql | train | 56355 | -| wikisql | valid | 14436 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('wikisql', split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('wikisql', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL-sql-to-en") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL-sql-to-en") - -def get_explanation(query): - input_text = "translate Sql to English: %s " % query - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(output[0]) - -query = "SELECT COUNT Params form model where location=HF-Hub" - -get_explanation(query) - -# output: 'How many parameters form model for HF-hub?' -``` - -Play with it in a Colab: -Open In Colab - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-base-finetuned-wikiSQL/README.md b/model_cards/mrm8488/t5-base-finetuned-wikiSQL/README.md deleted file mode 100644 index 0241381d0e..0000000000 --- a/model_cards/mrm8488/t5-base-finetuned-wikiSQL/README.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -language: en -datasets: -- wikisql ---- - -# T5-base fine-tuned on WikiSQL - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [WikiSQL](https://github.com/salesforce/WikiSQL) for **English** to **SQL** **translation**. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the Dataset 📚 - -Dataset ID: ```wikisql``` from [Huggingface/NLP](https://huggingface.co/nlp/viewer/?dataset=wikisql) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| wikisql | train | 56355 | -| wikisql | valid | 14436 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('wikisql', split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('wikisql', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-base-finetuned-wikiSQL") - -def get_sql(query): - input_text = "translate English to SQL: %s " % query - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(output[0]) - -query = "How many models were finetuned using BERT as base model?" - -get_sql(query) - -# output: 'SELECT COUNT Model fine tuned FROM table WHERE Base model = BERT' -``` - -Other examples from validation dataset: -![validation examples](https://pbs.twimg.com/media/Ec5vaG5XsAINty_?format=png&name=900x900) - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-small-finetuned-emotion/README.md b/model_cards/mrm8488/t5-small-finetuned-emotion/README.md deleted file mode 100644 index ba366b63f2..0000000000 --- a/model_cards/mrm8488/t5-small-finetuned-emotion/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -language: en -datasets: -- emotion ---- - -# T5-small fine-tuned for Emotion Recognition 😂😢😡😃😯 - - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) [small](https://huggingface.co/t5-small) fine-tuned on [emotion recognition](https://github.com/dair-ai/emotion_dataset) dataset for **Emotion Recognition** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - -## Details of the downstream task (Sentiment Recognition) - Dataset 📚 - -[Elvis Saravia](https://twitter.com/omarsar0) has gathered a great [dataset](https://github.com/dair-ai/emotion_dataset) for emotion recognition. It allows to classifiy the text into one of the following **6** emotions: - - - sadness 😢 - - joy 😃 - - love 🥰 - - anger 😡 - - fear 😱 - - surprise 😯 - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - -## Test set metrics 🧾 - -| |precision | recall | f1-score |support| -|----------|----------|---------|----------|-------| -|anger | 0.92| 0.93| 0.92| 275| -|fear | 0.90| 0.90| 0.90| 224| -|joy | 0.97| 0.91| 0.94| 695| -|love | 0.75| 0.89| 0.82| 159| -|sadness | 0.96| 0.97| 0.96| 581| -|surpirse | 0.73| 0.80| 0.76| 66| -| | -|accuracy| | | 0.92| 2000| -|macro avg| 0.87| 0.90| 0.88| 2000| -|weighted avg| 0.93| 0.92| 0.92| 2000| - - -Confusion Matrix - -![CM](https://i.imgur.com/JBtAwPx.png) - - -## Model in Action 🚀 - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-emotion") - -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-emotion") - -def get_emotion(text): - input_ids = tokenizer.encode(text + '', return_tensors='pt') - - output = model.generate(input_ids=input_ids, - max_length=2) - - dec = [tokenizer.decode(ids) for ids in output] - label = dec[0] - return label - - get_emotion("i feel as if i havent blogged in ages are at least truly blogged i am doing an update cute") # Output: 'joy' - - get_emotion("i have a feeling i kinda lost my best friend") # Output: 'sadness' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-small-finetuned-imdb-sentiment/README.md b/model_cards/mrm8488/t5-small-finetuned-imdb-sentiment/README.md deleted file mode 100644 index 534574dfff..0000000000 --- a/model_cards/mrm8488/t5-small-finetuned-imdb-sentiment/README.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -language: en -datasets: -- imdb ---- - -# T5-small fine-tuned for Sentiment Anlalysis 🎞️👍👎 - - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) [small](https://huggingface.co/t5-small) fine-tuned on [IMDB](https://huggingface.co/datasets/imdb) dataset for **Sentiment Analysis** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - -## Details of the downstream task (Sentiment analysis) - Dataset 📚 - -[IMDB](https://huggingface.co/datasets/imdb) - -This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. It provides a set of **25,000** highly polar movie reviews for training, and **25,000** for testing. - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - -## Test set metrics 🧾 - -| |precision | recall | f1-score |support| -|----------|----------|---------|----------|-------| -|negative | 0.92 | 0.93| 0.92| 12500| -|positive | 0.93 | 0.92| 0.92| 12500| -|----------|----------|---------|----------|-------| -|accuracy| | | 0.92| 25000| -|macro avg| 0.92| 0.92| 0.92| 25000| -|weighted avg| 0.92| 0.92| 0.92| 25000| - - -## Model in Action 🚀 - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-imdb-sentiment") - -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-imdb-sentiment") - -def get_sentiment(text): - input_ids = tokenizer.encode(text + '', return_tensors='pt') - - output = model.generate(input_ids=input_ids, - max_length=2) - - dec = [tokenizer.decode(ids) for ids in output] - label = dec[0] - return label - -get_sentiment("I dislike a lot that film") - -# Output: 'negative' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-small-finetuned-quora-for-paraphrasing/README.md b/model_cards/mrm8488/t5-small-finetuned-quora-for-paraphrasing/README.md deleted file mode 100644 index e5913a9bc1..0000000000 --- a/model_cards/mrm8488/t5-small-finetuned-quora-for-paraphrasing/README.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -language: en -datasets: -- quora ---- - -# T5-base fine-tuned on Quora question pair dataset for Question Paraphrasing ❓↔️❓ - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) fine-tuned on [Quodra question pair](https://huggingface.co/nlp/viewer/?dataset=quora) dataset for **Question Paraphrasing** task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Question Paraphrasing) - Dataset 📚❓↔️❓ - -Dataset ID: ```quora``` from [Huggingface/NLP](https://github.com/huggingface/nlp) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| quora | train | 404290 | -| quora after filter repeated questions | train | 149263 | - -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-quora-for-paraphrasing") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-quora-for-paraphrasing") - -def paraphrase(text, max_length=128): - - input_ids = tokenizer.encode(text, return_tensors="pt", add_special_tokens=True) - - generated_ids = model.generate(input_ids=input_ids, num_return_sequences=5, num_beams=5, max_length=max_length, no_repeat_ngram_size=2, repetition_penalty=3.5, length_penalty=1.0, early_stopping=True) - - preds = [tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=True) for g in generated_ids] - - return preds - -preds = paraphrase("paraphrase: What is the best framework for dealing with a huge text dataset?") - -for pred in preds: - print(pred) - -# Output: -''' -What is the best framework for dealing with a huge text dataset? -What is the best framework for dealing with a large text dataset? -What is the best framework to deal with a huge text dataset? -What are the best frameworks for dealing with a huge text dataset? -What is the best framework for dealing with huge text datasets? -''' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-small-finetuned-squadv1/README.md b/model_cards/mrm8488/t5-small-finetuned-squadv1/README.md deleted file mode 100644 index dc6b9c77ba..0000000000 --- a/model_cards/mrm8488/t5-small-finetuned-squadv1/README.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -language: en -datasets: -- squad ---- - -# T5-small fine-tuned on SQuAD - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) [(small)](https://huggingface.co/t5-small) fine-tuned on [SQuAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -Dataset ID: ```squad``` from [Huggingface/NLP](https://github.com/huggingface/nlp) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| squad | train | 87599 | -| squad | valid | 10570 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('squad, split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('squad', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28) - -## Results 📝 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **76.95** | -| **F1** | **85.71** | - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-squadv1") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-squadv1") - -def get_answer(question, context): - input_text = "question: %s context: %s " % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(output[0]) - -context = "Manuel have created RuPERTa-base (a Spanish RoBERTa) with the support of HF-Transformers and Google" -question = "Who has supported Manuel?" - -get_answer(question, context) - -# output: 'HF-Transformers and Google' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-small-finetuned-squadv2/README.md b/model_cards/mrm8488/t5-small-finetuned-squadv2/README.md deleted file mode 100644 index 1fb9db76e4..0000000000 --- a/model_cards/mrm8488/t5-small-finetuned-squadv2/README.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -language: en -datasets: -- squad_v2 ---- - -# T5-small fine-tuned on SQuAD v2 - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) [(small)](https://huggingface.co/t5-small) fine-tuned on [SQuAD v2](https://rajpurkar.github.io/SQuAD-explorer/) for **Q&A** downstream task. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the downstream task (Q&A) - Dataset 📚 🧐 ❓ - -Dataset ID: ```squad_v2``` from [Huggingface/NLP](https://github.com/huggingface/nlp) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| squad_v2 | train | 130319 | -| squad_v2 | valid | 11873 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('squad_v2, split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('squad_v2', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this awesome one](https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb) by [Suraj Patil](https://twitter.com/psuraj28) - -## Results 📝 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **69.46** | -| **F1** | **73.01** | - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-squadv2") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-squadv2") - -def get_answer(question, context): - input_text = "question: %s context: %s " % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(output[0]) - -context = "Manuel has created RuPERTa-base (a Spanish RoBERTa) with the support of HF-Transformers and Google" -question = "Who has supported Manuel?" - -get_answer(question, context) - -# output: 'HF-Transformers and Google' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/t5-small-finetuned-wikiSQL/README.md b/model_cards/mrm8488/t5-small-finetuned-wikiSQL/README.md deleted file mode 100644 index 40aff2e0aa..0000000000 --- a/model_cards/mrm8488/t5-small-finetuned-wikiSQL/README.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -language: en -datasets: -- wikisql ---- - -# T5-small fine-tuned on WikiSQL - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) [small](https://huggingface.co/t5-small) fine-tuned on [WikiSQL](https://github.com/salesforce/WikiSQL) for **English** to **SQL** **translation**. - -## Details of T5 - -The **T5** model was presented in [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) by *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* in Here the abstract: - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://i.imgur.com/jVFMMWR.png) - - -## Details of the Dataset 📚 - -Dataset ID: ```wikisql``` from [Huggingface/NLP](https://huggingface.co/nlp/viewer/?dataset=wikisql) - -| Dataset | Split | # samples | -| -------- | ----- | --------- | -| wikisql | train | 56355 | -| wikisql | valid | 14436 | - -How to load it from [nlp](https://github.com/huggingface/nlp) - -```python -train_dataset = nlp.load_dataset('wikisql', split=nlp.Split.TRAIN) -valid_dataset = nlp.load_dataset('wikisql', split=nlp.Split.VALIDATION) -``` -Check out more about this dataset and others in [NLP Viewer](https://huggingface.co/nlp/viewer/) - - -## Model fine-tuning 🏋️‍ - -The training script is a slightly modified version of [this Colab Notebook](https://github.com/patil-suraj/exploring-T5/blob/master/t5_fine_tuning.ipynb) created by [Suraj Patil](https://github.com/patil-suraj), so all credits to him! - - - -## Model in Action 🚀 - -```python -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("mrm8488/t5-small-finetuned-wikiSQL") -model = AutoModelWithLMHead.from_pretrained("mrm8488/t5-small-finetuned-wikiSQL") - -def get_sql(query): - input_text = "translate English to SQL: %s " % query - features = tokenizer([input_text], return_tensors='pt') - - output = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(output[0]) - -query = "How many millions of params there are in HF-hub?" - -get_sql(query) - -# output: 'SELECT COUNT Params FROM table WHERE Location = HF-hub' -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) - -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it/README.md b/model_cards/mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it/README.md deleted file mode 100644 index 8adb4f59d0..0000000000 --- a/model_cards/mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: it ---- - -# UmBERTo Wikipedia Uncased + italian SQuAD v1 📚 🧐 ❓ - -[UmBERTo-Wikipedia-Uncased](https://huggingface.co/Musixmatch/umberto-wikipedia-uncased-v1) fine-tuned on [Italian SQUAD v1 dataset](https://github.com/crux82/squad-it) for **Q&A** downstream task. - -## Details of the downstream task (Q&A) - Model 🧠 - -[UmBERTo](https://github.com/musixmatchresearch/umberto) is a Roberta-based Language Model trained on large Italian Corpora and uses two innovative approaches: SentencePiece and Whole Word Masking. -UmBERTo-Wikipedia-Uncased Training is trained on a relative small corpus (~7GB) extracted from Wikipedia-ITA. - - -## Details of the downstream task (Q&A) - Dataset 📚 - -[SQuAD](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) [Rajpurkar et al. 2016] is a large scale dataset for training of question answering systems on factoid questions. It contains more than 100,000 question-answer pairs about passages from 536 articles chosen from various domains of Wikipedia. - -**SQuAD-it** is derived from the SQuAD dataset and it is obtained through semi-automatic translation of the SQuAD dataset into Italian. It represents a large-scale dataset for open question answering processes on factoid questions in Italian. The dataset contains more than 60,000 question/answer pairs derived from the original English dataset. - -## Model training 🏋️‍ - -The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: - -```bash -python transformers/examples/question-answering/run_squad.py \ - --model_type bert \ - --model_name_or_path 'Musixmatch/umberto-wikipedia-uncased-v1' \ - --do_eval \ - --do_train \ - --do_lower_case \ - --train_file '/content/dataset/SQuAD_it-train.json' \ - --predict_file '/content/dataset/SQuAD_it-test.json' \ - --per_gpu_train_batch_size 16 \ - --learning_rate 3e-5 \ - --num_train_epochs 10 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir /content/drive/My\ Drive/umberto-uncased-finetuned-squadv1-it \ - --overwrite_output_dir \ - --save_steps 1000 -``` -With 10 epochs the model overfits the train dataset so I evaluated the different checkpoints created during training (every 1000 steps) and chose the best (In this case the one created at 17000 steps). - -## Test set Results 🧾 - -| Metric | # Value | -| ------ | --------- | -| **EM** | **60.50** | -| **F1** | **72.41** | - - - -```json -{ -'exact': 60.50729399395453, -'f1': 72.4141113348361, -'total': 7609, -'HasAns_exact': 60.50729399395453, -'HasAns_f1': 72.4141113348361, -'HasAns_total': 7609, -'best_exact': 60.50729399395453, -'best_exact_thresh': 0.0, -'best_f1': 72.4141113348361, -'best_f1_thresh': 0.0 -} -``` - -## Comparison ⚖️ - -| Model | EM | F1 score | -| -------------------------------------------------------------------------------------------------------------------------------- | --------- | --------- | -| [DrQA-it trained on SQuAD-it ](https://github.com/crux82/squad-it/blob/master/README.md#evaluating-a-neural-model-over-squad-it) | 56.1 | 65.9 | -| This one |60.50 |72.41 | -| [bert-italian-finedtuned-squadv1-it-alfa](https://huggingface.co/mrm8488/bert-italian-finedtuned-squadv1-it-alfa) |**62.51** |**74.16** | | **62.51** | **74.16** | - - -### Model in action 🚀 - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -QnA_pipeline = pipeline('question-answering', model='mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it') - -QnA_pipeline({ - 'context': 'Marco Aurelio era un imperatore romano che praticava lo stoicismo come filosofia di vita .', - 'question': 'Quale filosofia seguì Marco Aurelio ?' -}) -# Output: -{'answer': 'stoicismo', 'end': 65, 'score': 0.9477770241566028, 'start': 56} -``` - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) | [LinkedIn](https://www.linkedin.com/in/manuel-romero-cs/) -> Made with ♥ in Spain diff --git a/model_cards/mrm8488/xlm-multi-finetuned-xquadv1/README.md b/model_cards/mrm8488/xlm-multi-finetuned-xquadv1/README.md deleted file mode 100644 index 72ba3320b3..0000000000 --- a/model_cards/mrm8488/xlm-multi-finetuned-xquadv1/README.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -language: multilingual -thumbnail: ---- - -# [XLM](https://github.com/facebookresearch/XLM/) (multilingual version) fine-tuned for multilingual Q&A - -Released from `Facebook` together with the paper [Cross-lingual Language Model Pretraining](https://arxiv.org/abs/1901.07291) by Guillaume Lample and Alexis Conneau and fine-tuned on [XQuAD](https://github.com/deepmind/xquad) for multilingual (`11 different languages`) **Q&A** downstream task. - -## Details of the language model('xlm-mlm-100-1280') - -[Language model](https://github.com/facebookresearch/XLM/#ii-cross-lingual-language-model-pretraining-xlm) - -| Languages -| --------- | -| 100 | - -It includes the following languages: - -
-en-es-fr-de-zh-ru-pt-it-ar-ja-id-tr-nl-pl-simple-fa-vi-sv-ko-he-ro-no-hi-uk-cs-fi-hu-th-da-ca-el-bg-sr-ms-bn-hr-sl-zh_yue-az-sk-eo-ta-sh-lt-et-ml-la-bs-sq-arz-af-ka-mr-eu-tl-ang-gl-nn-ur-kk-be-hy-te-lv-mk-zh_classical-als-is-wuu-my-sco-mn-ceb-ast-cy-kn-br-an-gu-bar-uz-lb-ne-si-war-jv-ga-zh_min_nan-oc-ku-sw-nds-ckb-ia-yi-fy-scn-gan-tt-am -
- -## Details of the downstream task (multilingual Q&A) - Dataset - -Deepmind [XQuAD](https://github.com/deepmind/xquad) - -Languages covered: - -- Arabic: `ar` -- German: `de` -- Greek: `el` -- English: `en` -- Spanish: `es` -- Hindi: `hi` -- Russian: `ru` -- Thai: `th` -- Turkish: `tr` -- Vietnamese: `vi` -- Chinese: `zh` - -As the dataset is based on SQuAD v1.1, there are no unanswerable questions in the data. We chose this -setting so that models can focus on cross-lingual transfer. - -We show the average number of tokens per paragraph, question, and answer for each language in the -table below. The statistics were obtained using [Jieba](https://github.com/fxsjy/jieba) for Chinese -and the [Moses tokenizer](https://github.com/moses-smt/mosesdecoder/blob/master/scripts/tokenizer/tokenizer.perl) -for the other languages. - -| | en | es | de | el | ru | tr | ar | vi | th | zh | hi | -| --------- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | -| Paragraph | 142.4 | 160.7 | 139.5 | 149.6 | 133.9 | 126.5 | 128.2 | 191.2 | 158.7 | 147.6 | 232.4 | -| Question | 11.5 | 13.4 | 11.0 | 11.7 | 10.0 | 9.8 | 10.7 | 14.8 | 11.5 | 10.5 | 18.7 | -| Answer | 3.1 | 3.6 | 3.0 | 3.3 | 3.1 | 3.1 | 3.1 | 4.5 | 4.1 | 3.5 | 5.6 | - -Citation: - -
- -```bibtex -@article{Artetxe:etal:2019, - author = {Mikel Artetxe and Sebastian Ruder and Dani Yogatama}, - title = {On the cross-lingual transferability of monolingual representations}, - journal = {CoRR}, - volume = {abs/1910.11856}, - year = {2019}, - archivePrefix = {arXiv}, - eprint = {1910.11856} -} -``` - -
- -As XQuAD is just an evaluation dataset, I used Data augmentation techniques (scraping, neural machine translation, etc) to obtain more samples and split the dataset in order to have a train and test set. The test set was created in a way that contains the same number of samples for each language. Finally, I got: - -| Dataset | # samples | -| ----------- | --------- | -| XQUAD train | 50 K | -| XQUAD test | 8 K | - -## Model training - -The model was trained on a Tesla P100 GPU and 25GB of RAM. -The script for fine tuning can be found [here](https://github.com/huggingface/transformers/blob/master/examples/distillation/run_squad_w_distillation.py) - - -## Model in action - -Fast usage with **pipelines**: - -```python -from transformers import pipeline - -qa_pipeline = pipeline( - "question-answering", - model="mrm8488/xlm-multi-finetuned-xquadv1", - tokenizer="mrm8488/xlm-multi-finetuned-xquadv1" -) - -# English -qa_pipeline({ - 'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately", - 'question': "Who has been working hard for hugginface/transformers lately?" -}) - -#Output: {'answer': 'Manuel', 'end': 6, 'score': 8.531880747878265e-05, 'start': 0} - -# Russian -qa_pipeline({ - 'context': "Мануэль Ромеро в последнее время почти не работал в репозитории hugginface / transformers", - 'question': "Кто в последнее время усердно работал над обнимашками / трансформерами?" - -}) - -#Output: {'answer': 'работал в репозитории hugginface /','end': 76, 'score': 0.00012340750456964894, 'start': 42} -``` -Try it on a Colab (*Do not forget to change the model and tokenizer path in the Colab if necessary*): - -Open In Colab - -> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) - -> Made with ♥ in Spain diff --git a/model_cards/mymusise/gpt2-medium-chinese/README.md b/model_cards/mymusise/gpt2-medium-chinese/README.md deleted file mode 100644 index 75dfe324e6..0000000000 --- a/model_cards/mymusise/gpt2-medium-chinese/README.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -language: zh ---- - -# gpt2-medium-chinese - - -# Overview - -- **Language model**: GPT2-Medium -- **Model size**: 1.2GiB -- **Language**: Chinese -- **Training data**: [wiki2019zh_corpus](https://github.com/brightmart/nlp_chinese_corpus) -- **Source code**: [gpt2-quickly](https://github.com/mymusise/gpt2-quickly) - -# Example - -```python -from transformers import BertTokenizer, TFGPT2LMHeadModel -from transformers import TextGenerationPipeline - -tokenizer = BertTokenizer.from_pretrained("mymusise/EasternFantasyNoval") -model = TFGPT2LMHeadModel.from_pretrained("mymusise/EasternFantasyNoval") - -text_generator = TextGenerationPipeline(model, tokenizer) -print(text_generator("今日", max_length=64, do_sample=True, top_k=10)) -print(text_generator("跨越山丘", max_length=64, do_sample=True, top_k=10)) -``` -输出 -```text -[{'generated_text': '今日 ， 他 的 作 品 也 在 各 种 报 刊 发 表 。 201 1 年 ， 他 开 设 了 他 的 网 页 版 《 the dear 》 。 此 外 ， 他 还 在 各 种 电 视 节 目 中 出 现 过 。 2017 年 1 月 ， 他 被 任'}] -[{'generated_text': '跨越山丘 ， 其 中 有 三 分 之 二 的 地 区 被 划 入 山 区 。 最 高 峰 是 位 于 山 脚 上 的 大 岩 （ ） 。 其 中 的 山 脚 下 有 一 处 有 名 为 的 河 谷 ， 因 其 高 度 在 其 中 ， 而 得 名 。'}] -``` - -[Try it on colab](https://colab.research.google.com/github/mymusise/gpt2-quickly/blob/main/examples/gpt2_medium_chinese.ipynb) diff --git a/model_cards/mys/electra-base-turkish-cased-ner/README.md b/model_cards/mys/electra-base-turkish-cased-ner/README.md deleted file mode 100644 index addd29099b..0000000000 --- a/model_cards/mys/electra-base-turkish-cased-ner/README.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -language: tr ---- - -## What is this - -A NER model for Turkish with 48 categories trained on the dataset [Shrinked TWNERTC Turkish NER Data](https://www.kaggle.com/behcetsenturk/shrinked-twnertc-turkish-ner-data-by-kuzgunlar) by Behçet Şentürk, which is itself a filtered and cleaned version of the following automatically labeled dataset: - -> Sahin, H. Bahadir; Eren, Mustafa Tolga; Tirkaz, Caglar; Sonmez, Ozan; Yildiz, Eray (2017), “English/Turkish Wikipedia Named-Entity Recognition and Text Categorization Dataset”, Mendeley Data, v1 http://dx.doi.org/10.17632/cdcztymf4k.1 - -## Backbone model - -The backbone model is [electra-base-turkish-cased-discriminator](https://huggingface.co/dbmdz/electra-base-turkish-cased-discriminator), and I finetuned it for token classification. - -I'm continuing to figure out if it is possible to improve accuracy with this dataset, but it is already usable for non-critic applications. You can reach out to me on [Twitter](https://twitter.com/myusufsarigoz) for discussions and issues. -I will also release a notebook to finetune NER models with Shrinked TWNERTC as well as sample inference code to demonstrate what's possible with this model. diff --git a/model_cards/ncoop57/bart-base-code-summarizer-java-v0/README.md b/model_cards/ncoop57/bart-base-code-summarizer-java-v0/README.md deleted file mode 100644 index c8ef988fef..0000000000 --- a/model_cards/ncoop57/bart-base-code-summarizer-java-v0/README.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -tags: -- summarization - -license: mit ---- - -## ncoop57/bart-base-code-summarizer-java-v0 diff --git a/model_cards/neuralmind/bert-base-portuguese-cased/README.md b/model_cards/neuralmind/bert-base-portuguese-cased/README.md deleted file mode 100644 index 85deb52e36..0000000000 --- a/model_cards/neuralmind/bert-base-portuguese-cased/README.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -language: pt -license: mit -tags: - - bert - - pytorch -datasets: - - brWaC ---- - -# BERTimbau Base (aka "bert-base-portuguese-cased") - -![Bert holding a berimbau](https://imgur.com/JZ7Hynh.jpg) - -## Introduction - -BERTimbau Base is a pretrained BERT model for Brazilian Portuguese that achieves state-of-the-art performances on three downstream NLP tasks: Named Entity Recognition, Sentence Textual Similarity and Recognizing Textual Entailment. It is available in two sizes: Base and Large. - -For further information or requests, please go to [BERTimbau repository](https://github.com/neuralmind-ai/portuguese-bert/). - -## Available models - -| Model | Arch. | #Layers | #Params | -| ---------------------------------------- | ---------- | ------- | ------- | -| `neuralmind/bert-base-portuguese-cased` | BERT-Base | 12 | 110M | -| `neuralmind/bert-large-portuguese-cased` | BERT-Large | 24 | 335M | - -## Usage - -```python -from transformers import AutoTokenizer # Or BertTokenizer -from transformers import AutoModelForPreTraining # Or BertForPreTraining for loading pretraining heads -from transformers import AutoModel # or BertModel, for BERT without pretraining heads - -model = AutoModelForPreTraining.from_pretrained('neuralmind/bert-base-portuguese-cased') -tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-base-portuguese-cased', do_lower_case=False) -``` - -### Masked language modeling prediction example - -```python -from transformers import pipeline - -pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer) - -pipe('Tinha uma [MASK] no meio do caminho.') -# [{'score': 0.14287759363651276, -# 'sequence': '[CLS] Tinha uma pedra no meio do caminho. [SEP]', -# 'token': 5028, -# 'token_str': 'pedra'}, -# {'score': 0.06213393807411194, -# 'sequence': '[CLS] Tinha uma árvore no meio do caminho. [SEP]', -# 'token': 7411, -# 'token_str': 'árvore'}, -# {'score': 0.05515013635158539, -# 'sequence': '[CLS] Tinha uma estrada no meio do caminho. [SEP]', -# 'token': 5675, -# 'token_str': 'estrada'}, -# {'score': 0.0299188531935215, -# 'sequence': '[CLS] Tinha uma casa no meio do caminho. [SEP]', -# 'token': 1105, -# 'token_str': 'casa'}, -# {'score': 0.025660505518317223, -# 'sequence': '[CLS] Tinha uma cruz no meio do caminho. [SEP]', -# 'token': 3466, -# 'token_str': 'cruz'}] - -``` - -### For BERT embeddings - -```python -import torch - -model = AutoModel.from_pretrained('neuralmind/bert-base-portuguese-cased') -input_ids = tokenizer.encode('Tinha uma pedra no meio do caminho.', return_tensors='pt') - -with torch.no_grad(): - outs = model(input_ids) - encoded = outs[0][0, 1:-1] # Ignore [CLS] and [SEP] special tokens - -# encoded.shape: (8, 768) -# tensor([[-0.0398, -0.3057, 0.2431, ..., -0.5420, 0.1857, -0.5775], -# [-0.2926, -0.1957, 0.7020, ..., -0.2843, 0.0530, -0.4304], -# [ 0.2463, -0.1467, 0.5496, ..., 0.3781, -0.2325, -0.5469], -# ..., -# [ 0.0662, 0.7817, 0.3486, ..., -0.4131, -0.2852, -0.2819], -# [ 0.0662, 0.2845, 0.1871, ..., -0.2542, -0.2933, -0.0661], -# [ 0.2761, -0.1657, 0.3288, ..., -0.2102, 0.0029, -0.2009]]) -``` - -## Citation - -If you use our work, please cite: - -```bibtex -@inproceedings{souza2020bertimbau, - author = {F{\'a}bio Souza and - Rodrigo Nogueira and - Roberto Lotufo}, - title = {{BERT}imbau: pretrained {BERT} models for {B}razilian {P}ortuguese}, - booktitle = {9th Brazilian Conference on Intelligent Systems, {BRACIS}, Rio Grande do Sul, Brazil, October 20-23 (to appear)}, - year = {2020} -} -``` diff --git a/model_cards/neuralmind/bert-large-portuguese-cased/README.md b/model_cards/neuralmind/bert-large-portuguese-cased/README.md deleted file mode 100644 index 3c9a7fa539..0000000000 --- a/model_cards/neuralmind/bert-large-portuguese-cased/README.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -language: pt -license: mit -tags: - - bert - - pytorch -datasets: - - brWaC ---- - -# BERTimbau Large (aka "bert-large-portuguese-cased") - -![Bert holding a berimbau](https://imgur.com/JZ7Hynh.jpg) - -## Introduction - -BERTimbau Large is a pretrained BERT model for Brazilian Portuguese that achieves state-of-the-art performances on three downstream NLP tasks: Named Entity Recognition, Sentence Textual Similarity and Recognizing Textual Entailment. It is available in two sizes: Base and Large. - -For further information or requests, please go to [BERTimbau repository](https://github.com/neuralmind-ai/portuguese-bert/). - -## Available models - -| Model | Arch. | #Layers | #Params | -| ---------------------------------------- | ---------- | ------- | ------- | -| `neuralmind/bert-base-portuguese-cased` | BERT-Base | 12 | 110M | -| `neuralmind/bert-large-portuguese-cased` | BERT-Large | 24 | 335M | - -## Usage - -```python -from transformers import AutoTokenizer # Or BertTokenizer -from transformers import AutoModelForPreTraining # Or BertForPreTraining for loading pretraining heads -from transformers import AutoModel # or BertModel, for BERT without pretraining heads - -model = AutoModelForPreTraining.from_pretrained('neuralmind/bert-large-portuguese-cased') -tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-large-portuguese-cased', do_lower_case=False) -``` - -### Masked language modeling prediction example - -```python -from transformers import pipeline - -pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer) - -pipe('Tinha uma [MASK] no meio do caminho.') -# [{'score': 0.5054386258125305, -# 'sequence': '[CLS] Tinha uma pedra no meio do caminho. [SEP]', -# 'token': 5028, -# 'token_str': 'pedra'}, -# {'score': 0.05616172030568123, -# 'sequence': '[CLS] Tinha uma curva no meio do caminho. [SEP]', -# 'token': 9562, -# 'token_str': 'curva'}, -# {'score': 0.02348282001912594, -# 'sequence': '[CLS] Tinha uma parada no meio do caminho. [SEP]', -# 'token': 6655, -# 'token_str': 'parada'}, -# {'score': 0.01795753836631775, -# 'sequence': '[CLS] Tinha uma mulher no meio do caminho. [SEP]', -# 'token': 2606, -# 'token_str': 'mulher'}, -# {'score': 0.015246033668518066, -# 'sequence': '[CLS] Tinha uma luz no meio do caminho. [SEP]', -# 'token': 3377, -# 'token_str': 'luz'}] - -``` - -### For BERT embeddings - -```python - -import torch - -model = AutoModel.from_pretrained('neuralmind/bert-large-portuguese-cased') -input_ids = tokenizer.encode('Tinha uma pedra no meio do caminho.', return_tensors='pt') - -with torch.no_grad(): - outs = model(input_ids) - encoded = outs[0][0, 1:-1] # Ignore [CLS] and [SEP] special tokens - -# encoded.shape: (8, 1024) -# tensor([[ 1.1872, 0.5606, -0.2264, ..., 0.0117, -0.1618, -0.2286], -# [ 1.3562, 0.1026, 0.1732, ..., -0.3855, -0.0832, -0.1052], -# [ 0.2988, 0.2528, 0.4431, ..., 0.2684, -0.5584, 0.6524], -# ..., -# [ 0.3405, -0.0140, -0.0748, ..., 0.6649, -0.8983, 0.5802], -# [ 0.1011, 0.8782, 0.1545, ..., -0.1768, -0.8880, -0.1095], -# [ 0.7912, 0.9637, -0.3859, ..., 0.2050, -0.1350, 0.0432]]) -``` - -## Citation - -If you use our work, please cite: - -```bibtex -@inproceedings{souza2020bertimbau, - author = {F{\'a}bio Souza and - Rodrigo Nogueira and - Roberto Lotufo}, - title = {{BERT}imbau: pretrained {BERT} models for {B}razilian {P}ortuguese}, - booktitle = {9th Brazilian Conference on Intelligent Systems, {BRACIS}, Rio Grande do Sul, Brazil, October 20-23 (to appear)}, - year = {2020} -} -``` diff --git a/model_cards/neuralspace-reverie/indic-transformers-bn-bert/README.md b/model_cards/neuralspace-reverie/indic-transformers-bn-bert/README.md deleted file mode 100644 index a42e1596ac..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-bn-bert/README.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -language: -- bn -tags: -- MaskedLM -- Bengali ---- -# Indic-Transformers Bengali BERT -## Model description -This is a BERT language model pre-trained on ~3 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-bn-bert') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-bn-bert') -text = "আপনি কেমন আছেন?" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 6, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-bn-distilbert/README.md b/model_cards/neuralspace-reverie/indic-transformers-bn-distilbert/README.md deleted file mode 100644 index f06c6c9c83..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-bn-distilbert/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- bn -tags: -- MaskedLM -- Bengali -- DistilBERT -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Bengali DistilBERT -## Model description -This is a DistilBERT language model pre-trained on ~6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-bn-distilbert') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-bn-distilbert') -text = "আপনি কেমন আছেন?" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-bn-roberta/README.md b/model_cards/neuralspace-reverie/indic-transformers-bn-roberta/README.md deleted file mode 100644 index b2a47660fd..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-bn-roberta/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- bn -tags: -- MaskedLM -- Bengali -- RoBERTa -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Bengali RoBERTa -## Model description -This is a RoBERTa language model pre-trained on ~6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-bn-roberta') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-bn-roberta') -text = "আপনি কেমন আছেন?" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 10, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-bn-xlmroberta/README.md b/model_cards/neuralspace-reverie/indic-transformers-bn-xlmroberta/README.md deleted file mode 100644 index ff781b46d6..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-bn-xlmroberta/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- bn -tags: -- MaskedLM -- Bengali -- XLMRoBERTa -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Bengali XLMRoBERTa -## Model description -This is a XLMRoBERTa language model pre-trained on ~3 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-bn-xlmroberta') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-bn-xlmroberta') -text = "আপনি কেমন আছেন?" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-hi-bert/README.md b/model_cards/neuralspace-reverie/indic-transformers-hi-bert/README.md deleted file mode 100644 index 45f5389ff0..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-hi-bert/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- hi -tags: -- MaskedLM -- Hindi -- BERT -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Hindi BERT -## Model description -This is a BERT language model pre-trained on ~3 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-bert') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-bert') -text = "आपका स्वागत हैं" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-hi-distilbert/README.md b/model_cards/neuralspace-reverie/indic-transformers-hi-distilbert/README.md deleted file mode 100644 index 2b20243445..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-hi-distilbert/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- hi -tags: -- MaskedLM -- Hindi -- DistilBERT -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Hindi DistilBERT -## Model description -This is a DistilBERT language model pre-trained on ~10 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-distilbert') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-distilbert') -text = "आपका स्वागत हैं" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-hi-roberta/README.md b/model_cards/neuralspace-reverie/indic-transformers-hi-roberta/README.md deleted file mode 100644 index 3852b1d7fe..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-hi-roberta/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- hi -tags: -- MaskedLM -- Hindi -- RoBERTa -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Hindi RoBERTa -## Model description -This is a RoBERTa language model pre-trained on ~10 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-roberta') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-roberta') -text = "आपका स्वागत हैं" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 11, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-hi-xlmroberta/README.md b/model_cards/neuralspace-reverie/indic-transformers-hi-xlmroberta/README.md deleted file mode 100644 index f7baf90326..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-hi-xlmroberta/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- hi -tags: -- MaskedLM -- Hindi -- XLMRoBERTa -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Hindi XLMRoBERTa -## Model description -This is a XLMRoBERTa language model pre-trained on ~3 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-hi-xlmroberta') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-hi-xlmroberta') -text = "आपका स्वागत हैं" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-te-bert/README.md b/model_cards/neuralspace-reverie/indic-transformers-te-bert/README.md deleted file mode 100644 index 46b7dc3b31..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-te-bert/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- te -tags: -- MaskedLM -- Telugu -- BERT -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Telugu BERT -## Model description -This is a BERT language model pre-trained on ~1.6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-bert') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-bert') -text = "మీరు ఎలా ఉన్నారు" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-te-distilbert/README.md b/model_cards/neuralspace-reverie/indic-transformers-te-distilbert/README.md deleted file mode 100644 index 1ce7a8605e..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-te-distilbert/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- te -tags: -- MaskedLM -- Telugu -- DistilBERT -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Telugu DistilBERT -## Model description -This is a DistilBERT language model pre-trained on ~2 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-distilbert') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-distilbert') -text = "మీరు ఎలా ఉన్నారు" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-te-roberta/README.md b/model_cards/neuralspace-reverie/indic-transformers-te-roberta/README.md deleted file mode 100644 index f9c76cd68d..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-te-roberta/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- te -tags: -- MaskedLM -- Telugu -- RoBERTa -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Telugu RoBERTa -## Model description -This is a RoBERTa language model pre-trained on ~2 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-roberta') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-roberta') -text = "మీరు ఎలా ఉన్నారు" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 14, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuralspace-reverie/indic-transformers-te-xlmroberta/README.md b/model_cards/neuralspace-reverie/indic-transformers-te-xlmroberta/README.md deleted file mode 100644 index 78b1e78348..0000000000 --- a/model_cards/neuralspace-reverie/indic-transformers-te-xlmroberta/README.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -language: -- te -tags: -- MaskedLM -- Telugu -- XLMRoBERTa -- Question-Answering -- Token Classification -- Text Classification ---- -# Indic-Transformers Telugu XLMRoBERTa -## Model description -This is a XLMRoBERTa language model pre-trained on ~1.6 GB of monolingual training corpus. The pre-training data was majorly taken from [OSCAR](https://oscar-corpus.com/). -This model can be fine-tuned on various downstream tasks like text-classification, POS-tagging, question-answering, etc. Embeddings from this model can also be used for feature-based training. -## Intended uses & limitations -#### How to use -``` -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained('neuralspace-reverie/indic-transformers-te-xlmroberta') -model = AutoModel.from_pretrained('neuralspace-reverie/indic-transformers-te-xlmroberta') -text = "మీరు ఎలా ఉన్నారు" -input_ids = tokenizer(text, return_tensors='pt')['input_ids'] -out = model(input_ids)[0] -print(out.shape) -# out = [1, 5, 768] -``` -#### Limitations and bias -The original language model has been trained using `PyTorch` and hence the use of `pytorch_model.bin` weights file is recommended. The h5 file for `Tensorflow` has been generated manually by commands suggested [here](https://huggingface.co/transformers/model_sharing.html). diff --git a/model_cards/neuraly/bert-base-italian-cased-sentiment/README.md b/model_cards/neuraly/bert-base-italian-cased-sentiment/README.md deleted file mode 100644 index e99b97ad9e..0000000000 --- a/model_cards/neuraly/bert-base-italian-cased-sentiment/README.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -language: it -thumbnail: https://neuraly.ai/static/assets/images/huggingface/thumbnail.png -tags: -- sentiment -- Italian -license: MIT -widget: -- text: "Huggingface è un team fantastico!" ---- - -# 🤗 + neuraly - Italian BERT Sentiment model - -## Model description - -This model performs sentiment analysis on Italian sentences. It was trained starting from an instance of [bert-base-italian-cased](https://huggingface.co/dbmdz/bert-base-italian-cased), and fine-tuned on an Italian dataset of tweets, reaching 82% of accuracy on the latter one. - -## Intended uses & limitations - -#### How to use - -```python -import torch -from torch import nn -from transformers import AutoTokenizer, AutoModelForSequenceClassification - -# Load the tokenizer -tokenizer = AutoTokenizer.from_pretrained("neuraly/bert-base-italian-cased-sentiment") -# Load the model, use .cuda() to load it on the GPU -model = AutoModelForSequenceClassification.from_pretrained("neuraly/bert-base-italian-cased-sentiment") - -sentence = 'Huggingface è un team fantastico!' -input_ids = tokenizer.encode(sentence, add_special_tokens=True) - -# Create tensor, use .cuda() to transfer the tensor to GPU -tensor = torch.tensor(input_ids).long() -# Fake batch dimension -tensor = tensor.unsqueeze(0) - -# Call the model and get the logits -logits, = model(tensor) - -# Remove the fake batch dimension -logits = logits.squeeze(0) - -# The model was trained with a Log Likelyhood + Softmax combined loss, hence to extract probabilities we need a softmax on top of the logits tensor -proba = nn.functional.softmax(logits, dim=0) - -# Unpack the tensor to obtain negative, neutral and positive probabilities -negative, neutral, positive = proba -``` - -#### Limitations and bias - -A possible drawback (or bias) of this model is related to the fact that it was trained on a tweet dataset, with all the limitations that come with it. The domain is strongly related to football players and teams, but it works surprisingly well even on other topics. - -## Training data - -We trained the model by combining the two tweet datasets taken from [Sentipolc EVALITA 2016](http://www.di.unito.it/~tutreeb/sentipolc-evalita16/data.html). Overall the dataset consists of 45K pre-processed tweets. - -The model weights come from a pre-trained instance of [bert-base-italian-cased](https://huggingface.co/dbmdz/bert-base-italian-cased). A huge "thank you" goes to that team, brilliant work! - -## Training procedure - -#### Preprocessing - -We tried to save as much information as possible, since BERT captures extremely well the semantic of complex text sequences. Overall we removed only **@mentions**, **urls** and **emails** from every tweet and kept pretty much everything else. - -#### Hardware - -- **GPU**: Nvidia GTX1080ti -- **CPU**: AMD Ryzen7 3700x 8c/16t -- **RAM**: 64GB DDR4 - -#### Hyperparameters - -- Optimizer: **AdamW** with learning rate of **2e-5**, epsilon of **1e-8** -- Max epochs: **5** -- Batch size: **32** -- Early Stopping: **enabled** with patience = 1 - -Early stopping was triggered after 3 epochs. - -## Eval results - -The model achieves an overall accuracy on the test set equal to 82% -The test set is a 20% split of the whole dataset. - -## About us -[Neuraly](https://neuraly.ai) is a young and dynamic startup committed to designing AI-driven solutions and services through the most advanced Machine Learning and Data Science technologies. You can find out more about who we are and what we do on our [website](https://neuraly.ai). - -## Acknowledgments - -Thanks to the generous support from the [Hugging Face](https://huggingface.co/) team, -it is possible to download the model from their S3 storage and live test it from their inference API 🤗. diff --git a/model_cards/neurocode/IsRoBERTa/README.md b/model_cards/neurocode/IsRoBERTa/README.md deleted file mode 100644 index b56c9296f6..0000000000 --- a/model_cards/neurocode/IsRoBERTa/README.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -language: is -datasets: -- Icelandic portion of the OSCAR corpus from INRIA -- oscar ---- - -# IsRoBERTa a RoBERTa-like masked language model - -Probably the first icelandic transformer language model! - -## Overview -**Language:** Icelandic -**Downstream-task:** masked-lm -**Training data:** OSCAR corpus -**Code:** See [here](https://github.com/neurocode-io/icelandic-language-model) -**Infrastructure**: 1x Nvidia K80 - -## Hyperparameters - -``` -per_device_train_batch_size = 48 -n_epochs = 1 -vocab_size = 52.000 -max_position_embeddings = 514 -num_attention_heads = 12 -num_hidden_layers = 6 -type_vocab_size = 1 -learning_rate=0.00005 -``` - - -## Usage - -### In Transformers -```python -from transformers import ( - pipeline, - AutoTokenizer, - AutoModelWithLMHead -) - -model_name = "neurocode/IsRoBERTa" - -tokenizer = AutoTokenizer.from_pretrained(model_name) -model = AutoModelWithLMHead.from_pretrained(model_name) ->>> fill_mask = pipeline( -... "fill-mask", -... model=model, -... tokenizer=tokenizer -... ) ->>> result = fill_mask("Hann fór út að .") ->>> result -[ - {'sequence': 'Hann fór út að nýju.', 'score': 0.03395755589008331, 'token': 2219, 'token_str': 'Ġnýju'}, - {'sequence': 'Hann fór út að undanförnu.', 'score': 0.029087543487548828, 'token': 7590, 'token_str': 'Ġundanförnu'}, - {'sequence': 'Hann fór út að lokum.', 'score': 0.024420788511633873, 'token': 4384, 'token_str': 'Ġlokum'}, - {'sequence': 'Hann fór út að þessu.', 'score': 0.021231256425380707, 'token': 921, 'token_str': 'Ġþessu'}, - {'sequence': 'Hann fór út að honum.', 'score': 0.0205782949924469, 'token': 1136, 'token_str': 'Ġhonum'} -] -``` - - -## Authors -Bobby Donchev: `contact [at] donchev.is` -Elena Cramer: `elena.cramer [at] neurocode.io` - -## About us - -We bring AI software for our customers live -Our focus: AI software development - -Get in touch: -[LinkedIn](https://de.linkedin.com/company/neurocodeio) | [Website](https://neurocode.io) diff --git a/model_cards/nghuyong/ernie-1.0/README.md b/model_cards/nghuyong/ernie-1.0/README.md deleted file mode 100644 index fc6b601ad5..0000000000 --- a/model_cards/nghuyong/ernie-1.0/README.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -language: zh ---- - -# ERNIE-1.0 - -## Introduction - -ERNIE (Enhanced Representation through kNowledge IntEgration) is proposed by Baidu in 2019, -which is designed to learn language representation enhanced by knowledge masking strategies i.e. entity-level masking and phrase-level masking. -Experimental results show that ERNIE achieve state-of-the-art results on five Chinese natural language processing tasks including natural language inference, -semantic similarity, named entity recognition, sentiment analysis and question answering. - -More detail: https://arxiv.org/abs/1904.09223 - -## Released Model Info - -|Model Name|Language|Model Structure| -|:---:|:---:|:---:| -|ernie-1.0| Chinese |Layer:12, Hidden:768, Heads:12| - -This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and -a series of experiments have been conducted to check the accuracy of the conversion. - -- Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE -- Pytorch Conversion repo: https://github.com/nghuyong/ERNIE-Pytorch - -## How to use -```Python -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0") -model = AutoModel.from_pretrained("nghuyong/ernie-1.0") -``` - -## Citation - -```bibtex -@article{sun2019ernie, - title={Ernie: Enhanced representation through knowledge integration}, - author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Chen, Xuyi and Zhang, Han and Tian, Xin and Zhu, Danxiang and Tian, Hao and Wu, Hua}, - journal={arXiv preprint arXiv:1904.09223}, - year={2019} -} -``` diff --git a/model_cards/nghuyong/ernie-2.0-en/README.md b/model_cards/nghuyong/ernie-2.0-en/README.md deleted file mode 100644 index 1b0c7dcf38..0000000000 --- a/model_cards/nghuyong/ernie-2.0-en/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: en ---- - -# ERNIE-2.0 - -## Introduction - -ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, -which builds and learns incrementally pre-training tasks through constant multi-task learning. -Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. - -More detail: https://arxiv.org/abs/1907.12412 - -## Released Model Info - -|Model Name|Language|Model Structure| -|:---:|:---:|:---:| -|ernie-2.0-en| English |Layer:12, Hidden:768, Heads:12| - -This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and -a series of experiments have been conducted to check the accuracy of the conversion. - -- Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE -- Pytorch Conversion repo: https://github.com/nghuyong/ERNIE-Pytorch - -## How to use -```Python -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-2.0-en") -model = AutoModel.from_pretrained("nghuyong/ernie-2.0-en") -``` - -## Citation - -```bibtex -@article{sun2019ernie20, - title={ERNIE 2.0: A Continual Pre-training Framework for Language Understanding}, - author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Tian, Hao and Wu, Hua and Wang, Haifeng}, - journal={arXiv preprint arXiv:1907.12412}, - year={2019} -} -``` diff --git a/model_cards/nghuyong/ernie-2.0-large-en/README.md b/model_cards/nghuyong/ernie-2.0-large-en/README.md deleted file mode 100644 index 5158524050..0000000000 --- a/model_cards/nghuyong/ernie-2.0-large-en/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# ERNIE-2.0-large - -## Introduction - -ERNIE 2.0 is a continual pre-training framework proposed by Baidu in 2019, -which builds and learns incrementally pre-training tasks through constant multi-task learning. -Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese. - -More detail: https://arxiv.org/abs/1907.12412 - -## Released Model Info - -|Model Name|Language|Model Structure| -|:---:|:---:|:---:| -|ernie-2.0-large-en| English |Layer:24, Hidden:1024, Heads:16| - -This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and -a series of experiments have been conducted to check the accuracy of the conversion. - -- Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE -- Pytorch Conversion repo: https://github.com/nghuyong/ERNIE-Pytorch - -## How to use -```Python -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-2.0-large-en") -model = AutoModel.from_pretrained("nghuyong/ernie-2.0-large-en") -``` - -## Citation - -```bibtex -@article{sun2019ernie20, - title={ERNIE 2.0: A Continual Pre-training Framework for Language Understanding}, - author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Tian, Hao and Wu, Hua and Wang, Haifeng}, - journal={arXiv preprint arXiv:1907.12412}, - year={2019} -} -``` diff --git a/model_cards/nghuyong/ernie-tiny/README.md b/model_cards/nghuyong/ernie-tiny/README.md deleted file mode 100644 index 56992328a1..0000000000 --- a/model_cards/nghuyong/ernie-tiny/README.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -language: en ---- - -# ERNIE-tiny - -## Introduction -ERNIE-tiny is a compressed model from [ERNIE 2.0](../ernie-2.0-en) base model through model structure compression and model distillation. -Through compression, the performance of the ERNIE-tiny only decreases by an average of 2.37% compared to ERNIE 2.0 base, -but it outperforms Google BERT by 8.35%, and the speed increases by 4.3 times. - -More details: https://github.com/PaddlePaddle/ERNIE/blob/develop/distill/README.md - -## Released Model Info - -|Model Name|Language|Model Structure| -|:---:|:---:|:---:| -|ernie-tiny| English |Layer:3, Hidden:1024, Heads:16| - -This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and -a series of experiments have been conducted to check the accuracy of the conversion. - -- Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE -- Pytorch Conversion repo: https://github.com/nghuyong/ERNIE-Pytorch - -## How to use -```Python -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-tiny") -model = AutoModel.from_pretrained("nghuyong/ernie-tiny") -``` - -## Citation - -```bibtex -@article{sun2019ernie20, - title={ERNIE 2.0: A Continual Pre-training Framework for Language Understanding}, - author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Tian, Hao and Wu, Hua and Wang, Haifeng}, - journal={arXiv preprint arXiv:1907.12412}, - year={2019} -} -``` diff --git a/model_cards/nikokons/gpt2-greek/README.md b/model_cards/nikokons/gpt2-greek/README.md deleted file mode 100644 index cc7d1f9c27..0000000000 --- a/model_cards/nikokons/gpt2-greek/README.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -language: el ---- - -## gpt2-greek diff --git a/model_cards/nlpaueb/bert-base-greek-uncased-v1/README.md b/model_cards/nlpaueb/bert-base-greek-uncased-v1/README.md deleted file mode 100644 index 90e10fcb62..0000000000 --- a/model_cards/nlpaueb/bert-base-greek-uncased-v1/README.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -language: el -thumbnail: https://github.com/nlpaueb/GreekBERT/raw/master/greek-bert-logo.png ---- - -# GreekBERT - -A Greek version of BERT pre-trained language model. - - - - -## Pre-training corpora - -The pre-training corpora of `bert-base-greek-uncased-v1` include: - -* The Greek part of [Wikipedia](https://el.wikipedia.org/wiki/Βικιπαίδεια:Αντίγραφα_της_βάσης_δεδομένων), -* The Greek part of [European Parliament Proceedings Parallel Corpus](https://www.statmt.org/europarl/), and -* The Greek part of [OSCAR](https://traces1.inria.fr/oscar/), a cleansed version of [Common Crawl](https://commoncrawl.org). - -Future release will also include: - -* The entire corpus of Greek legislation, as published by the [National Publication Office](http://www.et.gr), -* The entire corpus of EU legislation (Greek translation), as published in [Eur-Lex](https://eur-lex.europa.eu/homepage.html?locale=en). - -## Pre-training details - -* We trained BERT using the official code provided in Google BERT's github repository (https://github.com/google-research/bert). We then used [Hugging Face](https://huggingface.co)'s [Transformers](https://github.com/huggingface/transformers) conversion script to convert the TF checkpoint and vocabulary in the desirable format in order to be able to load the model in two lines of code for both PyTorch and TF2 users. -* We released a model similar to the English `bert-base-uncased` model (12-layer, 768-hidden, 12-heads, 110M parameters). -* We chose to follow the same training set-up: 1 million training steps with batches of 256 sequences of length 512 with an initial learning rate 1e-4. -* We were able to use a single Google Cloud TPU v3-8 provided for free from [TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc), while also utilizing [GCP research credits](https://edu.google.com/programs/credits/research). Huge thanks to both Google programs for supporting us! - - -## Requirements - -We published `bert-base-greek-uncased-v1` as part of [Hugging Face](https://huggingface.co)'s [Transformers](https://github.com/huggingface/transformers) repository. So, you need to install the transfomers library through pip along with PyTorch or Tensorflow 2. - -``` -pip install transfomers -pip install (torch|tensorflow) -``` - -## Pre-process text (Deaccent - Lower) - -In order to use `bert-base-greek-uncased-v1`, you have to pre-process texts to lowercase letters and remove all Greek diacritics. - -```python - -import unicodedata - -def strip_accents_and_lowercase(s): - return ''.join(c for c in unicodedata.normalize('NFD', s) - if unicodedata.category(c) != 'Mn').lower() - -accented_string = "Αυτή είναι η Ελληνική έκδοση του BERT." -unaccented_string = strip_accents_and_lowercase(accented_string) - -print(unaccented_string) # αυτη ειναι η ελληνικη εκδοση του bert. - -``` - -## Load Pretrained Model - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("nlpaueb/bert-base-greek-uncased-v1") -model = AutoModel.from_pretrained("nlpaueb/bert-base-greek-uncased-v1") -``` - -## Use Pretrained Model as a Language Model - -```python -import torch -from transformers import * - -# Load model and tokenizer -tokenizer_greek = AutoTokenizer.from_pretrained('nlpaueb/bert-base-greek-uncased-v1') -lm_model_greek = AutoModelWithLMHead.from_pretrained('nlpaueb/bert-base-greek-uncased-v1') - -# ================ EXAMPLE 1 ================ -text_1 = 'O ποιητής έγραψε ένα [MASK] .' -# EN: 'The poet wrote a [MASK].' -input_ids = tokenizer_greek.encode(text_1) -print(tokenizer_greek.convert_ids_to_tokens(input_ids)) -# ['[CLS]', 'o', 'ποιητης', 'εγραψε', 'ενα', '[MASK]', '.', '[SEP]'] -outputs = lm_model_greek(torch.tensor([input_ids]))[0] -print(tokenizer_greek.convert_ids_to_tokens(outputs[0, 5].max(0)[1].item())) -# the most plausible prediction for [MASK] is "song" - -# ================ EXAMPLE 2 ================ -text_2 = 'Είναι ένας [MASK] άνθρωπος.' -# EN: 'He is a [MASK] person.' -input_ids = tokenizer_greek.encode(text_2) -print(tokenizer_greek.convert_ids_to_tokens(input_ids)) -# ['[CLS]', 'ειναι', 'ενας', '[MASK]', 'ανθρωπος', '.', '[SEP]'] -outputs = lm_model_greek(torch.tensor([input_ids]))[0] -print(tokenizer_greek.convert_ids_to_tokens(outputs[0, 3].max(0)[1].item())) -# the most plausible prediction for [MASK] is "good" - -# ================ EXAMPLE 3 ================ -text_3 = 'Είναι ένας [MASK] άνθρωπος και κάνει συχνά [MASK].' -# EN: 'He is a [MASK] person he does frequently [MASK].' -input_ids = tokenizer_greek.encode(text_3) -print(tokenizer_greek.convert_ids_to_tokens(input_ids)) -# ['[CLS]', 'ειναι', 'ενας', '[MASK]', 'ανθρωπος', 'και', 'κανει', 'συχνα', '[MASK]', '.', '[SEP]'] -outputs = lm_model_greek(torch.tensor([input_ids]))[0] -print(tokenizer_greek.convert_ids_to_tokens(outputs[0, 8].max(0)[1].item())) -# the most plausible prediction for the second [MASK] is "trips" -``` - -## Evaluation on downstream tasks - -TBA - -## Author - -Ilias Chalkidis on behalf of [AUEB's Natural Language Processing Group](http://nlp.cs.aueb.gr) - -| Github: [@ilias.chalkidis](https://github.com/seolhokim) | Twitter: [@KiddoThe2B](https://twitter.com/KiddoThe2B) | - -## About Us - -[AUEB's Natural Language Processing Group](http://nlp.cs.aueb.gr) develops algorithms, models, and systems that allow computers to process and generate natural language texts. - -The group's current research interests include: -* question answering systems for databases, ontologies, document collections, and the Web, especially biomedical question answering, -* natural language generation from databases and ontologies, especially Semantic Web ontologies, -text classification, including filtering spam and abusive content, -* information extraction and opinion mining, including legal text analytics and sentiment analysis, -* natural language processing tools for Greek, for example parsers and named-entity recognizers, -machine learning in natural language processing, especially deep learning. - -The group is part of the Information Processing Laboratory of the Department of Informatics of the Athens University of Economics and Business. diff --git a/model_cards/nlpaueb/legal-bert-base-uncased/README.md b/model_cards/nlpaueb/legal-bert-base-uncased/README.md deleted file mode 100644 index da579878cd..0000000000 --- a/model_cards/nlpaueb/legal-bert-base-uncased/README.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -language: en -tags: -- legal ---- - -# LEGAL-BERT: The Muppets straight out of Law School - - - -LEGAL-BERT is a family of BERT models for the legal domain, intended to assist legal NLP research, computational law, and legal technology applications. To pre-train the different variations of LEGAL-BERT, we collected 12 GB of diverse English legal text from several fields (e.g., legislation, court cases, contracts) scraped from publicly available resources. Sub-domains variants (CONTRACTS-, EURLEX-, ECHR-) and/or general LEGAL-BERT perform better than using BERT out of the box for domain-specific tasks. A light-weight model (33% the size of BERT-BASE) pre-trained from scratch on legal data with competitive perfomance is also available. -



- ---- - -I. Chalkidis, M. Fergadiotis, P. Malakasiotis, N. Aletras and I. Androutsopoulos. "LEGAL-BERT: The Muppets straight out of Law School". In Findings of Empirical Methods in Natural Language Processing (EMNLP 2020) (Short Papers), to be held online, 2020. (https://arxiv.org/abs/2010.02559) - ---- - -## Pre-training corpora - -The pre-training corpora of LEGAL-BERT include: - -* 116,062 documents of EU legislation, publicly available from EURLEX (http://eur-lex.europa.eu), the repository of EU Law running under the EU Publication Office. - -* 61,826 documents of UK legislation, publicly available from the UK legislation portal (http://www.legislation.gov.uk). - -* 19,867 cases from European Court of Justice (ECJ), also available from EURLEX. - -* 12,554 cases from HUDOC, the repository of the European Court of Human Rights (ECHR) (http://hudoc.echr.coe.int/eng). - -* 164,141 cases from various courts across the USA, hosted in the Case Law Access Project portal (https://case.law). - -* 76,366 US contracts from EDGAR, the database of US Securities and Exchange Commission (SECOM) (https://www.sec.gov/edgar.shtml). - -## Pre-training details - -* We trained BERT using the official code provided in Google BERT's github repository (https://github.com/google-research/bert). -* We released a model similar to the English BERT-BASE model (12-layer, 768-hidden, 12-heads, 110M parameters). -* We chose to follow the same training set-up: 1 million training steps with batches of 256 sequences of length 512 with an initial learning rate 1e-4. -* We were able to use a single Google Cloud TPU v3-8 provided for free from [TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc), while also utilizing [GCP research credits](https://edu.google.com/programs/credits/research). Huge thanks to both Google programs for supporting us! -* Part of LEGAL-BERT is a light-weight model pre-trained from scratch on legal data, which achieves comparable performance to larger models, while being much more efficient (approximately 4 times faster) with a smaller environmental footprint. -## Models list - -| Model name | Model Path | Training corpora | -| ------------------- | ------------------------------------ | ------------------- | -| CONTRACTS-BERT-BASE | `nlpaueb/bert-base-uncased-contracts` | US contracts | -| EURLEX-BERT-BASE | `nlpaueb/bert-base-uncased-eurlex` | EU legislation | -| ECHR-BERT-BASE | `nlpaueb/bert-base-uncased-echr` | ECHR cases | -| LEGAL-BERT-BASE | `nlpaueb/legal-bert-base-uncased` | All | -| LEGAL-BERT-SMALL | `nlpaueb/legal-bert-small-uncased` | All | - -## Load Pretrained Model - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("nlpaueb/legal-bert-base-uncased") -model = AutoModel.from_pretrained("nlpaueb/legal-bert-base-uncased") -``` - -## Use LEBAL-BERT variants as Language Models - -| Corpus | Model | Masked token | Predictions | -| --------------------------------- | ---------------------------------- | ------------ | ------------ | -| | **BERT-BASE-UNCASED** | -| (Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('new', '0.09'), ('current', '0.04'), ('proposed', '0.03'), ('marketing', '0.03'), ('joint', '0.02') -| (ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.32'), ('rape', '0.22'), ('abuse', '0.14'), ('death', '0.04'), ('violence', '0.03') -| (EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('farm', '0.25'), ('livestock', '0.08'), ('draft', '0.06'), ('domestic', '0.05'), ('wild', '0.05') -| | **CONTRACTS-BERT-BASE** | -| (Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02') -| (ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('death', '0.39'), ('imprisonment', '0.07'), ('contempt', '0.05'), ('being', '0.03'), ('crime', '0.02') -| (EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | (('domestic', '0.18'), ('laboratory', '0.07'), ('household', '0.06'), ('personal', '0.06'), ('the', '0.04') -| | **EURLEX-BERT-BASE** | -| (Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('supply', '0.11'), ('cooperation', '0.08'), ('service', '0.07'), ('licence', '0.07'), ('distribution', '0.05') -| (ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.66'), ('death', '0.07'), ('imprisonment', '0.07'), ('murder', '0.04'), ('rape', '0.02') -| (EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('live', '0.43'), ('pet', '0.28'), ('certain', '0.05'), ('fur', '0.03'), ('the', '0.02') -| | **ECHR-BERT-BASE** | -| (Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('second', '0.24'), ('latter', '0.10'), ('draft', '0.05'), ('bilateral', '0.05'), ('arbitration', '0.04') -| (ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.99'), ('death', '0.01'), ('inhuman', '0.00'), ('beating', '0.00'), ('rape', '0.00') -| (EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('pet', '0.17'), ('all', '0.12'), ('slaughtered', '0.10'), ('domestic', '0.07'), ('individual', '0.05') -| | **LEGAL-BERT-BASE** | -| (Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('settlement', '0.26'), ('letter', '0.23'), ('dealer', '0.04'), ('master', '0.02'), ('supplemental', '0.02') -| (ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '1.00'), ('detention', '0.00'), ('arrest', '0.00'), ('rape', '0.00'), ('death', '0.00') -| (EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('live', '0.67'), ('beef', '0.17'), ('farm', '0.03'), ('pet', '0.02'), ('dairy', '0.01') -| | **LEGAL-BERT-SMALL** | -| (Contracts) | This [MASK] Agreement is between General Motors and John Murray . | employment | ('license', '0.09'), ('transition', '0.08'), ('settlement', '0.04'), ('consent', '0.03'), ('letter', '0.03') -| (ECHR) | The applicant submitted that her husband was subjected to treatment amounting to [MASK] whilst in the custody of Adana Security Directorate | torture | ('torture', '0.59'), ('pain', '0.05'), ('ptsd', '0.05'), ('death', '0.02'), ('tuberculosis', '0.02') -| (EURLEX) | Establishing a system for the identification and registration of [MASK] animals and regarding the labelling of beef and beef products . | bovine | ('all', '0.08'), ('live', '0.07'), ('certain', '0.07'), ('the', '0.07'), ('farm', '0.05') - - - -## Evaluation on downstream tasks - -Consider the experiments in the article "LEGAL-BERT: The Muppets straight out of Law School". Chalkidis et al., 2018, (https://arxiv.org/abs/2010.02559) - -## Author - -Ilias Chalkidis on behalf of [AUEB's Natural Language Processing Group](http://nlp.cs.aueb.gr) - -| Github: [@ilias.chalkidis](https://github.com/seolhokim) | Twitter: [@KiddoThe2B](https://twitter.com/KiddoThe2B) | diff --git a/model_cards/nlptown/bert-base-multilingual-uncased-sentiment/README.md b/model_cards/nlptown/bert-base-multilingual-uncased-sentiment/README.md deleted file mode 100644 index 87934c8709..0000000000 --- a/model_cards/nlptown/bert-base-multilingual-uncased-sentiment/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: -- en -- nl -- de -- fr -- it -- es - -license: mit ---- - -# bert-base-multilingual-uncased-sentiment - -This a bert-base-multilingual-uncased model finetuned for sentiment analysis on product reviews in six languages: English, Dutch, German, French, Spanish and Italian. It predicts the sentiment of the review as a number of stars (between 1 and 5). - -This model is intended for direct use as a sentiment analysis model for product reviews in any of the six languages above, or for further finetuning on related sentiment analysis tasks. - -## Training data - -Here is the number of product reviews we used for finetuning the model: - -| Language | Number of reviews | -| -------- | ----------------- | -| English | 150k | -| Dutch | 80k | -| German | 137k | -| French | 140k | -| Italian | 72k | -| Spanish | 50k | - -## Accuracy - -The finetuned model obtained the following accuracy on 5,000 held-out product reviews in each of the languages: - -- Accuracy (exact) is the exact match on the number of stars. -- Accuracy (off-by-1) is the percentage of reviews where the number of stars the model predicts differs by a maximum of 1 from the number given by the human reviewer. - - -| Language | Accuracy (exact) | Accuracy (off-by-1) | -| -------- | ---------------------- | ------------------- | -| English | 67% | 95% -| Dutch | 57% | 93% -| German | 61% | 94% -| French | 59% | 94% -| Italian | 59% | 95% -| Spanish | 58% | 95% - -## Contact - -Contact [NLP Town](https://www.nlp.town) for questions, feedback and/or requests for similar models. diff --git a/model_cards/nyu-mll/roberta-base-100M-1/README.md b/model_cards/nyu-mll/roberta-base-100M-1/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-100M-1/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-100M-2/README.md b/model_cards/nyu-mll/roberta-base-100M-2/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-100M-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-100M-3/README.md b/model_cards/nyu-mll/roberta-base-100M-3/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-100M-3/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-10M-1/README.md b/model_cards/nyu-mll/roberta-base-10M-1/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-10M-1/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-10M-2/README.md b/model_cards/nyu-mll/roberta-base-10M-2/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-10M-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-10M-3/README.md b/model_cards/nyu-mll/roberta-base-10M-3/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-10M-3/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-1B-1/README.md b/model_cards/nyu-mll/roberta-base-1B-1/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-1B-1/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-1B-2/README.md b/model_cards/nyu-mll/roberta-base-1B-2/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-1B-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-base-1B-3/README.md b/model_cards/nyu-mll/roberta-base-1B-3/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-base-1B-3/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-med-small-1M-1/README.md b/model_cards/nyu-mll/roberta-med-small-1M-1/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-med-small-1M-1/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-med-small-1M-2/README.md b/model_cards/nyu-mll/roberta-med-small-1M-2/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-med-small-1M-2/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta-med-small-1M-3/README.md b/model_cards/nyu-mll/roberta-med-small-1M-3/README.md deleted file mode 120000 index 34a367f775..0000000000 --- a/model_cards/nyu-mll/roberta-med-small-1M-3/README.md +++ /dev/null @@ -1 +0,0 @@ -../roberta_1M_to_1B/README.md \ No newline at end of file diff --git a/model_cards/nyu-mll/roberta_1M_to_1B/README.md b/model_cards/nyu-mll/roberta_1M_to_1B/README.md deleted file mode 100644 index 1689faf800..0000000000 --- a/model_cards/nyu-mll/roberta_1M_to_1B/README.md +++ /dev/null @@ -1,49 +0,0 @@ -# RoBERTa Pretrained on Smaller Datasets - -We pretrain RoBERTa on smaller datasets (1M, 10M, 100M, 1B tokens). We release 3 models with lowest perplexities for each pretraining data size out of 25 runs (or 10 in the case of 1B tokens). The pretraining data reproduces that of BERT: We combine English Wikipedia and a reproduction of BookCorpus using texts from smashwords in a ratio of approximately 3:1. - -### Hyperparameters and Validation Perplexity - -The hyperparameters and validation perplexities corresponding to each model are as follows: - -| Model Name | Training Size | Model Size | Max Steps | Batch Size | Validation Perplexity | -|--------------------------|---------------|------------|-----------|------------|-----------------------| -| [roberta-base-1B-1][link-roberta-base-1B-1] | 1B | BASE | 100K | 512 | 3.93 | -| [roberta-base-1B-2][link-roberta-base-1B-2] | 1B | BASE | 31K | 1024 | 4.25 | -| [roberta-base-1B-3][link-roberta-base-1B-3] | 1B | BASE | 31K | 4096 | 3.84 | -| [roberta-base-100M-1][link-roberta-base-100M-1] | 100M | BASE | 100K | 512 | 4.99 | -| [roberta-base-100M-2][link-roberta-base-100M-2] | 100M | BASE | 31K | 1024 | 4.61 | -| [roberta-base-100M-3][link-roberta-base-100M-3] | 100M | BASE | 31K | 512 | 5.02 | -| [roberta-base-10M-1][link-roberta-base-10M-1] | 10M | BASE | 10K | 1024 | 11.31 | -| [roberta-base-10M-2][link-roberta-base-10M-2] | 10M | BASE | 10K | 512 | 10.78 | -| [roberta-base-10M-3][link-roberta-base-10M-3] | 10M | BASE | 31K | 512 | 11.58 | -| [roberta-med-small-1M-1][link-roberta-med-small-1M-1] | 1M | MED-SMALL | 100K | 512 | 153.38 | -| [roberta-med-small-1M-2][link-roberta-med-small-1M-2] | 1M | MED-SMALL | 10K | 512 | 134.18 | -| [roberta-med-small-1M-3][link-roberta-med-small-1M-3] | 1M | MED-SMALL | 31K | 512 | 139.39 | - -The hyperparameters corresponding to model sizes mentioned above are as follows: - -| Model Size | L | AH | HS | FFN | P | -|------------|----|----|-----|------|------| -| BASE | 12 | 12 | 768 | 3072 | 125M | -| MED-SMALL | 6 | 8 | 512 | 2048 | 45M | - -(AH = number of attention heads; HS = hidden size; FFN = feedforward network dimension; P = number of parameters.) - -For other hyperparameters, we select: -- Peak Learning rate: 5e-4 -- Warmup Steps: 6% of max steps -- Dropout: 0.1 - -[link-roberta-med-small-1M-1]: https://huggingface.co/nyu-mll/roberta-med-small-1M-1 -[link-roberta-med-small-1M-2]: https://huggingface.co/nyu-mll/roberta-med-small-1M-2 -[link-roberta-med-small-1M-3]: https://huggingface.co/nyu-mll/roberta-med-small-1M-3 -[link-roberta-base-10M-1]: https://huggingface.co/nyu-mll/roberta-base-10M-1 -[link-roberta-base-10M-2]: https://huggingface.co/nyu-mll/roberta-base-10M-2 -[link-roberta-base-10M-3]: https://huggingface.co/nyu-mll/roberta-base-10M-3 -[link-roberta-base-100M-1]: https://huggingface.co/nyu-mll/roberta-base-100M-1 -[link-roberta-base-100M-2]: https://huggingface.co/nyu-mll/roberta-base-100M-2 -[link-roberta-base-100M-3]: https://huggingface.co/nyu-mll/roberta-base-100M-3 -[link-roberta-base-1B-1]: https://huggingface.co/nyu-mll/roberta-base-1B-1 -[link-roberta-base-1B-2]: https://huggingface.co/nyu-mll/roberta-base-1B-2 -[link-roberta-base-1B-3]: https://huggingface.co/nyu-mll/roberta-base-1B-3 diff --git a/model_cards/oliverguhr/german-sentiment-bert/README.md b/model_cards/oliverguhr/german-sentiment-bert/README.md deleted file mode 100644 index a8e03e278f..0000000000 --- a/model_cards/oliverguhr/german-sentiment-bert/README.md +++ /dev/null @@ -1,125 +0,0 @@ -# German Sentiment Classification with Bert - -This model was trained for sentiment classification of German language texts. To achieve the best results all model inputs needs to be preprocessed with the same procedure, that was applied during the training. To simplify the usage of the model, -we provide a Python package that bundles the code need for the preprocessing and inferencing. - -The model uses the Googles Bert architecture and was trained on 1.834 million German-language samples. The training data contains texts from various domains like Twitter, Facebook and movie, app and hotel reviews. -You can find more information about the dataset and the training process in the [paper](http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.202.pdf). - -## Using the Python package - -To get started install the package from [pypi](https://pypi.org/project/germansentiment/): - -```bash -pip install germansentiment -``` - -```python -from germansentiment import SentimentModel - -model = SentimentModel() - -texts = [ - "Mit keinem guten Ergebniss","Das ist gar nicht mal so gut", - "Total awesome!","nicht so schlecht wie erwartet", - "Der Test verlief positiv.","Sie fährt ein grünes Auto."] - -result = model.predict_sentiment(texts) -print(result) -``` - -The code above will output following list: - -```python -["negative","negative","positive","positive","neutral", "neutral"] -``` - -## A minimal working Sample - - -```python -from transformers import AutoModelForSequenceClassification, AutoTokenizer -from typing import List -import torch -import re - -class SentimentModel(): - def __init__(self, model_name: str): - self.model = AutoModelForSequenceClassification.from_pretrained(model_name) - self.tokenizer = AutoTokenizer.from_pretrained(model_name) - - self.clean_chars = re.compile(r'[^A-Za-züöäÖÜÄß ]', re.MULTILINE) - self.clean_http_urls = re.compile(r'https*\S+', re.MULTILINE) - self.clean_at_mentions = re.compile(r'@\S+', re.MULTILINE) - - def predict_sentiment(self, texts: List[str])-> List[str]: - texts = [self.clean_text(text) for text in texts] - # Add special tokens takes care of adding [CLS], [SEP], ... tokens in the right way for each model. - input_ids = self.tokenizer(texts, padding=True, truncation=True, add_special_tokens=True) - input_ids = torch.tensor(input_ids["input_ids"]) - - with torch.no_grad(): - logits = self.model(input_ids) - - label_ids = torch.argmax(logits[0], axis=1) - - labels = [self.model.config.id2label[label_id] for label_id in label_ids.tolist()] - return labels - - def replace_numbers(self,text: str) -> str: - return text.replace("0"," null").replace("1"," eins").replace("2"," zwei").replace("3"," drei").replace("4"," vier").replace("5"," fünf").replace("6"," sechs").replace("7"," sieben").replace("8"," acht").replace("9"," neun") - - def clean_text(self,text: str)-> str: - text = text.replace("\n", " ") - text = self.clean_http_urls.sub('',text) - text = self.clean_at_mentions.sub('',text) - text = self.replace_numbers(text) - text = self.clean_chars.sub('', text) # use only text chars - text = ' '.join(text.split()) # substitute multiple whitespace with single whitespace - text = text.strip().lower() - return text - -texts = ["Mit keinem guten Ergebniss","Das war unfair", "Das ist gar nicht mal so gut", - "Total awesome!","nicht so schlecht wie erwartet", "Das ist gar nicht mal so schlecht", - "Der Test verlief positiv.","Sie fährt ein grünes Auto.", "Der Fall wurde an die Polzei übergeben."] - -model = SentimentModel(model_name = "oliverguhr/german-sentiment-bert") - -print(model.predict_sentiment(texts)) -``` - -## Model and Data - -If you are interested in code and data that was used to train this model please have a look at [this repository](https://github.com/oliverguhr/german-sentiment) and our [paper](http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.202.pdf). Here is a table of the F1 scores that his model achieves on following datasets. Since we trained this model on a newer version of the transformer library, the results are slightly better than reported in the paper. - -| Dataset | F1 micro Score | -| :----------------------------------------------------------- | -------------: | -| [holidaycheck](https://github.com/oliverguhr/german-sentiment) | 0.9568 | -| [scare](https://www.romanklinger.de/scare/) | 0.9418 | -| [filmstarts](https://github.com/oliverguhr/german-sentiment) | 0.9021 | -| [germeval](https://sites.google.com/view/germeval2017-absa/home) | 0.7536 | -| [PotTS](https://www.aclweb.org/anthology/L16-1181/) | 0.6780 | -| [emotions](https://github.com/oliverguhr/german-sentiment) | 0.9649 | -| [sb10k](https://www.spinningbytes.com/resources/germansentiment/) | 0.7376 | -| [Leipzig Wikipedia Corpus 2016](https://wortschatz.uni-leipzig.de/de/download/german) | 0.9967 | -| all | 0.9639 | - -## Cite - -For feedback and questions contact me view mail or Twitter [@oliverguhr](https://twitter.com/oliverguhr). Please cite us if you found this useful: - -``` -@InProceedings{guhr-EtAl:2020:LREC, - author = {Guhr, Oliver and Schumann, Anne-Kathrin and Bahrmann, Frank and Böhme, Hans Joachim}, - title = {Training a Broad-Coverage German Sentiment Classification Model for Dialog Systems}, - booktitle = {Proceedings of The 12th Language Resources and Evaluation Conference}, - month = {May}, - year = {2020}, - address = {Marseille, France}, - publisher = {European Language Resources Association}, - pages = {1620--1625}, - url = {https://www.aclweb.org/anthology/2020.lrec-1.201} -} -``` - - diff --git a/model_cards/panggi/t5-base-indonesian-summarization-cased/README.md b/model_cards/panggi/t5-base-indonesian-summarization-cased/README.md deleted file mode 100644 index f5943f5963..0000000000 --- a/model_cards/panggi/t5-base-indonesian-summarization-cased/README.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -language: id ---- - -# Indonesian T5 Summarization Base Model - -Finetuned T5 base summarization model for Indonesian. - -## Finetuning Corpus - -`t5-base-indonesian-summarization-cased` model is based on `t5-base-bahasa-summarization-cased` by [huseinzol05](https://huggingface.co/huseinzol05), finetuned using [indosum](https://github.com/kata-ai/indosum) dataset. - -## Load Finetuned Model - -```python -from transformers import T5Tokenizer, T5Model, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained("panggi/t5-base-indonesian-summarization-cased") -model = T5ForConditionalGeneration.from_pretrained("panggi/t5-base-indonesian-summarization-cased") -``` - -## Code Sample - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained("panggi/t5-base-indonesian-summarization-cased") -model = T5ForConditionalGeneration.from_pretrained("panggi/t5-base-indonesian-summarization-cased") - -# https://www.sehatq.com/artikel/apa-itu-dispepsia-fungsional-ketahui-gejala-dan-faktor-risikonya -ARTICLE_TO_SUMMARIZE = "Secara umum, dispepsia adalah kumpulan gejala pada saluran pencernaan seperti nyeri, sensasi terbakar, dan rasa tidak nyaman pada perut bagian atas. Pada beberapa kasus, dispepsia yang dialami seseorang tidak dapat diketahui penyebabnya. Jenis dispepsia ini disebut dengan dispepsia fungsional. Apa saja gejala dispepsia fungsional? Apa itu dispepsia fungsional? Dispepsia fungsional adalah kumpulan gejala tanpa sebab pada saluran pencernaan bagian atas. Gejala tersebut dapat berupa rasa sakit, nyeri, dan tak nyaman pada perut bagian atas atau ulu hati. Penderita dispepsia fungsional juga akan merasakan kenyang lebih cepat dan sensasi perut penuh berkepanjangan. Gejala-gejala tersebut bisa berlangsung selama sebulan atau lebih. Dispepsia ini memiliki nama “fungsional” karena kumpulan gejalanya tidak memiliki penyebab yang jelas. Dilihat dari fungsi dan struktur saluran pencernaan, dokter tidak menemukan hal yang salah. Namun, gejalanya bisa sangat mengganggu dan menyiksa. Dispepsia fungsional disebut juga dengan dispepsia nonulkus. Diperkirakan bahwa 20% masyarakat dunia menderita dispepsia fungsional. Kondisi ini berisiko tinggi dialami oleh wanita, perokok, dan orang yang mengonsumsi obat anti-peradangan nonsteroid (NSAID). Dispepsia fungsional bisa bersifat kronis dan mengganggu kehidupan penderitanya. Namun beruntung, ada beberapa strategi yang bisa diterapkan untuk mengendalikan gejala dispepsia ini. Strategi tersebut termasuk perubahan gaya hidup, obat-obatan, dan terapi.Ragam gejala dispepsia fungsional Gejala dispepsia fungsional dapat bervariasi antara satu pasien dengan pasien lain. Beberapa tanda yang bisa dirasakan seseorang, yaitu: Sensasi terbakar atau nyeri di saluran pencernaan bagian atas Perut kembung Cepat merasa kenyang walau baru makan sedikit Mual Muntah Bersendawa Rasa asam di mulut Penurunan berat badan Tekanan psikologis terkait dengan kondisi yang dialami Apa sebenarnya penyebab dispepsia fungsional? Sebagai penyakit fungsional, dokter mengkategorikan dispepsia ini sebagai penyakit yang tidak diketahui penyebabnya. Hanya saja, beberapa faktor bisa meningkatkan risiko seseorang terkena dispepsia fungsional. Faktor risiko tersebut, termasuk: Alergi terhadap zat tertentu Perubahan mikrobioma usus Infeksi, seperti yang dipicu oleh bakteriHelicobacter pylori Sekresi asam lambung yang tidak normal Peradangan pada saluran pencernaan bagian atas Gangguan pada fungsi lambung untuk mencerna makanan Pola makan tertentu Gaya hidup tidak sehat Stres Kecemasan atau depresi Efek samping pemakaian obat seperti obat antiinflamasi nonsteroid Penanganan untuk dispepsia fungsional Ada banyak pilihan pengobatan untuk dispepsia fungsional. Seperti yang disampaikan di atas, tidak ada penyebab tunggal dispepsia ini yang bisa diketahui. Gejala yang dialami antara satu pasien juga mungkin amat berbeda dari orang lain. Dengan demikian, jenis pengobatan dispepsia fungsional juga akan bervariasi. Beberapa pilihan strategi penanganan untuk dispepsia fungsional, meliputi: 1. Obat-obatan Ada beberapa jenis obat yang mungkin akan diberikan dokter, seperti Obat penetral asam lambung yang disebut penghambat reseptor H2 Obat penghambat produksi asam lambung yang disebut proton pump inhibitors Obat untuk mengendalikan gas di perut yang mengandung simetikon Antidepresan seperti amitriptyline Obat penguat kerongkongan yang disebut agen prokinetik Obat untuk pengosongan isi lambung seperti metoclopramide Antibiotik jika dokter mendeteksi adanya infeksi bakteri H. pylori 2. Anjuran terkait perubahan gaya hidup Selain obat-obatan, dokter akan memberikan rekomendasi perubahan gaya hidup yang harus diterapkan pasien. Tips terkait perubahan gaya hidup termasuk: Makan lebih sering namun dengan porsi yang lebih sedikit Menjauhi makanan berlemak karena memperlambat pengosongan makanan di lambung Menjauhi jenis makanan lain yang memicu gejala dispepsia, seperti makanan pedas, makanan tinggi asam, produk susu, dan produk kafein Menjauhi rokok Dokter juga akan meminta pasien untuk mencari cara untuk mengendalikan stres, tidur dengan kepala lebih tinggi, dan menjalankan usaha untuk mengendalikan berat badan. Apakah penyakit dispepsia itu berbahaya? Dispepsia, termasuk dispepsia fungsional, dapat menjadi kronis dengan gejala yang menyiksa. Jika tidak ditangani, dispepsia tentu dapat berbahaya dan mengganggu kehidupan pasien. Segera hubungi dokter apabila Anda merasakan gejala dispepsia, terlebih jika tidak merespons obat-obatan yang dijual bebas. Catatan dari SehatQ Dispepsia fungsional adalah kumpulan gejala pada saluran pencernaan bagian atas yang tidak diketahui penyebabnya. Dispepsia fungsional dapat ditangani dengan kombinasi obat-obatan dan perubahan gaya hidup. Jika masih memiliki pertanyaan terkait dispepsia fungsional, Anda bisa menanyakan ke dokter di aplikasi kesehatan keluarga SehatQ. Aplikasi SehatQ bisa diunduh gratis di Appstore dan Playstore yang berikan informasi penyakit terpercaya." -# generate summary -inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=5000, return_tensors='pt', truncation=True) -summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=100, early_stopping=True) -print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids]) -``` - -Output: - -``` -'Dispepsia fungsional adalah kumpulan gejala tanpa sebab pada saluran pencernaan bagian atas. Gejala tersebut dapat berupa rasa sakit, nyeri, dan tak nyaman pada perut bagian atas. Penderita dispepsia fungsional juga akan merasakan kenyang lebih cepat dan sensasi perut penuh berkepanjangan. Gejala-gejala tersebut bisa berlangsung selama sebulan atau lebih. -``` - -## Acknowledgement - -Thanks to Immanuel Drexel for his article [Text Summarization, Extractive, T5, Bahasa Indonesia, Huggingface’s Transformers](https://medium.com/analytics-vidhya/text-summarization-t5-bahasa-indonesia-huggingfaces-transformers-ee9bfe368e2f) diff --git a/model_cards/panggi/t5-small-indonesian-summarization-cased/README.md b/model_cards/panggi/t5-small-indonesian-summarization-cased/README.md deleted file mode 100644 index c83ab8627c..0000000000 --- a/model_cards/panggi/t5-small-indonesian-summarization-cased/README.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -language: id ---- - -# Indonesian T5 Summarization Small Model - -Finetuned T5 small summarization model for Indonesian. - -## Finetuning Corpus - -`t5-small-indonesian-summarization-cased` model is based on `t5-small-bahasa-summarization-cased` by [huseinzol05](https://huggingface.co/huseinzol05), finetuned using [indosum](https://github.com/kata-ai/indosum) dataset. - -## Load Finetuned Model - -```python -from transformers import T5Tokenizer, T5Model, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained("panggi/t5-small-indonesian-summarization-cased") -model = T5ForConditionalGeneration.from_pretrained("panggi/t5-small-indonesian-summarization-cased") -``` - -## Code Sample - -```python -from transformers import T5Tokenizer, T5ForConditionalGeneration - -tokenizer = T5Tokenizer.from_pretrained("panggi/t5-small-indonesian-summarization-cased") -model = T5ForConditionalGeneration.from_pretrained("panggi/t5-small-indonesian-summarization-cased") - -# https://www.sehatq.com/artikel/apa-itu-dispepsia-fungsional-ketahui-gejala-dan-faktor-risikonya -ARTICLE_TO_SUMMARIZE = "Secara umum, dispepsia adalah kumpulan gejala pada saluran pencernaan seperti nyeri, sensasi terbakar, dan rasa tidak nyaman pada perut bagian atas. Pada beberapa kasus, dispepsia yang dialami seseorang tidak dapat diketahui penyebabnya. Jenis dispepsia ini disebut dengan dispepsia fungsional. Apa saja gejala dispepsia fungsional? Apa itu dispepsia fungsional? Dispepsia fungsional adalah kumpulan gejala tanpa sebab pada saluran pencernaan bagian atas. Gejala tersebut dapat berupa rasa sakit, nyeri, dan tak nyaman pada perut bagian atas atau ulu hati. Penderita dispepsia fungsional juga akan merasakan kenyang lebih cepat dan sensasi perut penuh berkepanjangan. Gejala-gejala tersebut bisa berlangsung selama sebulan atau lebih. Dispepsia ini memiliki nama “fungsional” karena kumpulan gejalanya tidak memiliki penyebab yang jelas. Dilihat dari fungsi dan struktur saluran pencernaan, dokter tidak menemukan hal yang salah. Namun, gejalanya bisa sangat mengganggu dan menyiksa. Dispepsia fungsional disebut juga dengan dispepsia nonulkus. Diperkirakan bahwa 20% masyarakat dunia menderita dispepsia fungsional. Kondisi ini berisiko tinggi dialami oleh wanita, perokok, dan orang yang mengonsumsi obat anti-peradangan nonsteroid (NSAID). Dispepsia fungsional bisa bersifat kronis dan mengganggu kehidupan penderitanya. Namun beruntung, ada beberapa strategi yang bisa diterapkan untuk mengendalikan gejala dispepsia ini. Strategi tersebut termasuk perubahan gaya hidup, obat-obatan, dan terapi.Ragam gejala dispepsia fungsional Gejala dispepsia fungsional dapat bervariasi antara satu pasien dengan pasien lain. Beberapa tanda yang bisa dirasakan seseorang, yaitu: Sensasi terbakar atau nyeri di saluran pencernaan bagian atas Perut kembung Cepat merasa kenyang walau baru makan sedikit Mual Muntah Bersendawa Rasa asam di mulut Penurunan berat badan Tekanan psikologis terkait dengan kondisi yang dialami Apa sebenarnya penyebab dispepsia fungsional? Sebagai penyakit fungsional, dokter mengkategorikan dispepsia ini sebagai penyakit yang tidak diketahui penyebabnya. Hanya saja, beberapa faktor bisa meningkatkan risiko seseorang terkena dispepsia fungsional. Faktor risiko tersebut, termasuk: Alergi terhadap zat tertentu Perubahan mikrobioma usus Infeksi, seperti yang dipicu oleh bakteriHelicobacter pylori Sekresi asam lambung yang tidak normal Peradangan pada saluran pencernaan bagian atas Gangguan pada fungsi lambung untuk mencerna makanan Pola makan tertentu Gaya hidup tidak sehat Stres Kecemasan atau depresi Efek samping pemakaian obat seperti obat antiinflamasi nonsteroid Penanganan untuk dispepsia fungsional Ada banyak pilihan pengobatan untuk dispepsia fungsional. Seperti yang disampaikan di atas, tidak ada penyebab tunggal dispepsia ini yang bisa diketahui. Gejala yang dialami antara satu pasien juga mungkin amat berbeda dari orang lain. Dengan demikian, jenis pengobatan dispepsia fungsional juga akan bervariasi. Beberapa pilihan strategi penanganan untuk dispepsia fungsional, meliputi: 1. Obat-obatan Ada beberapa jenis obat yang mungkin akan diberikan dokter, seperti Obat penetral asam lambung yang disebut penghambat reseptor H2 Obat penghambat produksi asam lambung yang disebut proton pump inhibitors Obat untuk mengendalikan gas di perut yang mengandung simetikon Antidepresan seperti amitriptyline Obat penguat kerongkongan yang disebut agen prokinetik Obat untuk pengosongan isi lambung seperti metoclopramide Antibiotik jika dokter mendeteksi adanya infeksi bakteri H. pylori 2. Anjuran terkait perubahan gaya hidup Selain obat-obatan, dokter akan memberikan rekomendasi perubahan gaya hidup yang harus diterapkan pasien. Tips terkait perubahan gaya hidup termasuk: Makan lebih sering namun dengan porsi yang lebih sedikit Menjauhi makanan berlemak karena memperlambat pengosongan makanan di lambung Menjauhi jenis makanan lain yang memicu gejala dispepsia, seperti makanan pedas, makanan tinggi asam, produk susu, dan produk kafein Menjauhi rokok Dokter juga akan meminta pasien untuk mencari cara untuk mengendalikan stres, tidur dengan kepala lebih tinggi, dan menjalankan usaha untuk mengendalikan berat badan. Apakah penyakit dispepsia itu berbahaya? Dispepsia, termasuk dispepsia fungsional, dapat menjadi kronis dengan gejala yang menyiksa. Jika tidak ditangani, dispepsia tentu dapat berbahaya dan mengganggu kehidupan pasien. Segera hubungi dokter apabila Anda merasakan gejala dispepsia, terlebih jika tidak merespons obat-obatan yang dijual bebas. Catatan dari SehatQ Dispepsia fungsional adalah kumpulan gejala pada saluran pencernaan bagian atas yang tidak diketahui penyebabnya. Dispepsia fungsional dapat ditangani dengan kombinasi obat-obatan dan perubahan gaya hidup. Jika masih memiliki pertanyaan terkait dispepsia fungsional, Anda bisa menanyakan ke dokter di aplikasi kesehatan keluarga SehatQ. Aplikasi SehatQ bisa diunduh gratis di Appstore dan Playstore yang berikan informasi penyakit terpercaya." -# generate summary -inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=5000, return_tensors='pt', truncation=True) -summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=100, early_stopping=True) -print([tokenizer.decode(g, skip_special_tokens=True, clean_up_tokenization_spaces=False) for g in summary_ids]) -``` - -Output: - -``` -'Dispepsia fungsional adalah kumpulan gejala tanpa sebab pada saluran pencernaan bagian atas. Gejala tersebut dapat berupa rasa sakit, nyeri, dan tak nyaman pada perut bagian atas. Penderita dispepsia fungsional juga akan merasakan kenyang lebih cepat dan sensasi perut penuh berkepanjangan. Gejala-gejala tersebut bisa berlangsung selama sebulan atau lebih. -``` - -## Acknowledgement - -Thanks to Immanuel Drexel for his article [Text Summarization, Extractive, T5, Bahasa Indonesia, Huggingface’s Transformers](https://medium.com/analytics-vidhya/text-summarization-t5-bahasa-indonesia-huggingfaces-transformers-ee9bfe368e2f) diff --git a/model_cards/patrickvonplaten/bert2bert-cnn_dailymail-fp16/README.md b/model_cards/patrickvonplaten/bert2bert-cnn_dailymail-fp16/README.md deleted file mode 100644 index 71831bd959..0000000000 --- a/model_cards/patrickvonplaten/bert2bert-cnn_dailymail-fp16/README.md +++ /dev/null @@ -1,218 +0,0 @@ -# Bert2Bert Summarization with 🤗 EncoderDecoder Framework - -This model is a Bert2Bert model fine-tuned on summarization. - -Bert2Bert is a `EncoderDecoderModel`, meaning that both the encoder and the decoder are `bert-base-uncased` -BERT models. Leveraging the [EncoderDecoderFramework](https://huggingface.co/transformers/model_doc/encoderdecoder.html#encoder-decoder-models), the -two pretrained models can simply be loaded into the framework via: - -```python -bert2bert = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-uncased", "bert-base-uncased") -``` - -The decoder of an `EncoderDecoder` model needs cross-attention layers and usually makes use of causal -masking for auto-regressiv generation. -Thus, ``bert2bert`` is consequently fined-tuned on the `CNN/Daily Mail`dataset and the resulting model -`bert2bert-cnn_dailymail-fp16` is uploaded here. - -## Example - -The model is by no means a state-of-the-art model, but nevertheless -produces reasonable summarization results. It was mainly fine-tuned -as a proof-of-concept for the 🤗 EncoderDecoder Framework. - -The model can be used as follows: - -```python -from transformers import BertTokenizer, EncoderDecoderModel - -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/bert2bert-cnn_dailymail-fp16") -tokenizer = BertTokenizer.from_pretrained("patrickvonplaten/bert2bert-cnn_dailymail-fp16") - -article = """(CNN)Sigma Alpha Epsilon is under fire for a video showing party-bound fraternity members singing a racist chant. SAE's national chapter suspended the students, but University of Oklahoma President David Boren took it a step further, saying the university's affiliation with the fraternity is permanently done. The news is shocking, but it's not the first time SAE has faced controversy. SAE was founded March 9, 1856, at the University of Alabama, five years before the American Civil War, according to the fraternity website. When the war began, the group had fewer than 400 members, of which "369 went to war for the Confederate States and seven for the Union Army," the website says. The fraternity now boasts more than 200,000 living alumni, along with about 15,000 undergraduates populating 219 chapters and 20 "colonies" seeking full membership at universities. SAE has had to work hard to change recently after a string of member deaths, many blamed on the hazing of new recruits, SAE national President Bradley Cohen wrote in a message on the fraternity's website. The fraternity's website lists more than 130 chapters cited or suspended for "health and safety incidents" since 2010. At least 30 of the incidents involved hazing, and dozens more involved alcohol. However, the list is missing numerous incidents from recent months. Among them, according to various media outlets: Yale University banned the SAEs from campus activities last month after members allegedly tried to interfere with a sexual misconduct investigation connected to an initiation rite. Stanford University in December suspended SAE housing privileges after finding sorority members attending a fraternity function were subjected to graphic sexual content. And Johns Hopkins University in November suspended the fraternity for underage drinking. "The media has labeled us as the 'nation's deadliest fraternity,' " Cohen said. In 2011, for example, a student died while being coerced into excessive alcohol consumption, according to a lawsuit. SAE's previous insurer dumped the fraternity. "As a result, we are paying Lloyd's of London the highest insurance rates in the Greek-letter world," Cohen said. Universities have turned down SAE's attempts to open new chapters, and the fraternity had to close 12 in 18 months over hazing incidents.""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids) - -print(tokenizer.decode(output_ids[0], skip_special_tokens=True)) -# should produce -# sae was founded in 1856, five years before the civil war. the fraternity has had to work hard to change recently. the university of oklahoma president says the university's affiliation with the fraternity is permanently done. the sae has had a string of members in recent mon -ths. -``` - -## Training script: - -**IMPORTANT**: In order for this code to work, make sure you checkout to the branch -[more_general_trainer_metric](https://github.com/huggingface/transformers/tree/more_general_trainer_metric), which slightly adapts -the `Trainer` for `EncoderDecoderModels` according to this PR: https://github.com/huggingface/transformers/pull/5840. - -The following code shows the complete training script that was used to fine-tune `bert2bert-cnn_dailymail-fp16 -` for reproducability. The training last ~9h on a standard GPU. - -```python -#!/usr/bin/env python3 -import nlp -import logging -from transformers import BertTokenizer, EncoderDecoderModel, Trainer, TrainingArguments - -logging.basicConfig(level=logging.INFO) - -model = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-uncased", "bert-base-uncased") -tokenizer = BertTokenizer.from_pretrained("bert-base-uncased") - -# CLS token will work as BOS token -tokenizer.bos_token = tokenizer.cls_token - -# SEP token will work as EOS token -tokenizer.eos_token = tokenizer.sep_token - -# load train and validation data -train_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="train") -val_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="validation[:10%]") - -# load rouge for validation -rouge = nlp.load_metric("rouge") - - -# set decoding params -model.config.decoder_start_token_id = tokenizer.bos_token_id -model.config.eos_token_id = tokenizer.eos_token_id -model.config.max_length = 142 -model.config.min_length = 56 -model.config.no_repeat_ngram_size = 3 -model.early_stopping = True -model.length_penalty = 2.0 -model.num_beams = 4 - - -# map data correctly -def map_to_encoder_decoder_inputs(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at BERT max length 512 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=512) - # force summarization <= 128 - outputs = tokenizer(batch["highlights"], padding="max_length", truncation=True, max_length=128) - - batch["input_ids"] = inputs.input_ids - batch["attention_mask"] = inputs.attention_mask - - batch["decoder_input_ids"] = outputs.input_ids - batch["labels"] = outputs.input_ids.copy() - # mask loss for padding - batch["labels"] = [ - [-100 if token == tokenizer.pad_token_id else token for token in labels] for labels in batch["labels"] - ] - batch["decoder_attention_mask"] = outputs.attention_mask - - assert all([len(x) == 512 for x in inputs.input_ids]) - assert all([len(x) == 128 for x in outputs.input_ids]) - - return batch - - -def compute_metrics(pred): - labels_ids = pred.label_ids - pred_ids = pred.predictions - - # all unnecessary tokens are removed - pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True) - label_str = tokenizer.batch_decode(labels_ids, skip_special_tokens=True) - - rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - - return { - "rouge2_precision": round(rouge_output.precision, 4), - "rouge2_recall": round(rouge_output.recall, 4), - "rouge2_fmeasure": round(rouge_output.fmeasure, 4), - } - - -# set batch size here -batch_size = 16 - -# make train dataset ready -train_dataset = train_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -train_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "decoder_input_ids", "decoder_attention_mask", "labels"], -) - -# same for validation dataset -val_dataset = val_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -val_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "decoder_input_ids", "decoder_attention_mask", "labels"], -) - -# set training arguments - these params are not really tuned, feel free to change -training_args = TrainingArguments( - output_dir="./", - per_device_train_batch_size=batch_size, - per_device_eval_batch_size=batch_size, - predict_from_generate=True, - evaluate_during_training=True, - do_train=True, - do_eval=True, - logging_steps=1000, - save_steps=1000, - eval_steps=1000, - overwrite_output_dir=True, - warmup_steps=2000, - save_total_limit=10, -) - -# instantiate trainer -trainer = Trainer( - model=model, - args=training_args, - compute_metrics=compute_metrics, - train_dataset=train_dataset, - eval_dataset=val_dataset, -) - -# start training -trainer.train() -``` - -## Evaluation - -The following script evaluates the model on the test set of -CNN/Daily Mail. - -```python -#!/usr/bin/env python3 -import nlp -from transformers import BertTokenizer, EncoderDecoderModel -tokenizer = BertTokenizer.from_pretrained("patrickvonplaten/bert2bert-cnn_dailymail-fp16") -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/bert2bert-cnn_dailymail-fp16") -model.to("cuda") -test_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="test") -batch_size = 128 -# map data correctly -def generate_summary(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at BERT max length 512 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=512, return_tensors="pt") - input_ids = inputs.input_ids.to("cuda") - attention_mask = inputs.attention_mask.to("cuda") - outputs = model.generate(input_ids, attention_mask=attention_mask) - # all special tokens including will be removed - output_str = tokenizer.batch_decode(outputs, skip_special_tokens=True) - batch["pred"] = output_str - return batch -results = test_dataset.map(generate_summary, batched=True, batch_size=batch_size, remove_columns=["article"]) -# load rouge for validation -rouge = nlp.load_metric("rouge") -pred_str = results["pred"] -label_str = results["highlights"] -rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid -print(rouge_output) -``` - -The obtained results should be: - -| - | Rouge2 - mid -precision | Rouge2 - mid - recall | Rouge2 - mid - fmeasure | -|----------|:-------------:|:------:|:------:| -| **CNN/Daily Mail** | 16.12 | 17.07 | **16.1** | diff --git a/model_cards/patrickvonplaten/bert2bert_cnn_daily_mail/README.md b/model_cards/patrickvonplaten/bert2bert_cnn_daily_mail/README.md deleted file mode 100644 index 97f319b5eb..0000000000 --- a/model_cards/patrickvonplaten/bert2bert_cnn_daily_mail/README.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- cnn_dailymail -tags: -- summarization ---- - -Bert2Bert Summarization with 🤗EncoderDecoder Framework -This model is a warm-started *BERT2BERT* model fine-tuned on the *CNN/Dailymail* summarization dataset. - -The model achieves a **18.22** ROUGE-2 score on *CNN/Dailymail*'s test dataset. - -For more details on how the model was fine-tuned, please refer to -[this](https://colab.research.google.com/drive/1Ekd5pUeCX7VOrMx94_czTkwNtLN32Uyu?usp=sharing) notebook. diff --git a/model_cards/patrickvonplaten/bert2gpt2-cnn_dailymail-fp16/README.md b/model_cards/patrickvonplaten/bert2gpt2-cnn_dailymail-fp16/README.md deleted file mode 100644 index 3a3a157c0b..0000000000 --- a/model_cards/patrickvonplaten/bert2gpt2-cnn_dailymail-fp16/README.md +++ /dev/null @@ -1,295 +0,0 @@ -# Bert2GPT2 Summarization with 🤗 EncoderDecoder Framework - -This model is a Bert2Bert model fine-tuned on summarization. - -Bert2GPT2 is a `EncoderDecoderModel`, meaning that the encoder is a `bert-base-uncased` -BERT model and the decoder is a `gpt2` GPT2 model. Leveraging the [EncoderDecoderFramework](https://huggingface.co/transformers/model_doc/encoderdecoder.html#encoder-decoder-models), the -two pretrained models can simply be loaded into the framework via: - -```python -bert2gpt2 = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-uncased", "gpt2") -``` - -The decoder of an `EncoderDecoder` model needs cross-attention layers and usually makes use of causal -masking for auto-regressiv generation. -Thus, ``bert2gpt2`` is consequently fined-tuned on the `CNN/Daily Mail`dataset and the resulting model -`bert2gpt2-cnn_dailymail-fp16` is uploaded here. - -## Example - -The model is by no means a state-of-the-art model, but nevertheless -produces reasonable summarization results. It was mainly fine-tuned -as a proof-of-concept for the 🤗 EncoderDecoder Framework. - -The model can be used as follows: - -```python -from transformers import BertTokenizer, GPT2Tokenizer, EncoderDecoderModel - -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/bert2gpt2-cnn_dailymail-fp16") -# reuse tokenizer from bert2bert encoder-decoder model -bert_tokenizer = BertTokenizer.from_pretrained("patrickvonplaten/bert2bert-cnn_dailymail-fp16") - -article = """(CNN)Sigma Alpha Epsilon is under fire for a video showing party-bound fraternity members singing a racist chant. SAE's national chapter suspended the students, but University of Oklahoma President David B -oren took it a step further, saying the university's affiliation with the fraternity is permanently done. The news is shocking, but it's not the first time SAE has faced controversy. SAE was founded March 9, 185 -6, at the University of Alabama, five years before the American Civil War, according to the fraternity website. When the war began, the group had fewer than 400 members, of which "369 went to war for the Confede -rate States and seven for the Union Army," the website says. The fraternity now boasts more than 200,000 living alumni, along with about 15,000 undergraduates populating 219 chapters and 20 "colonies" seeking fu -ll membership at universities. SAE has had to work hard to change recently after a string of member deaths, many blamed on the hazing of new recruits, SAE national President Bradley Cohen wrote in a message on t -he fraternity's website. The fraternity's website lists more than 130 chapters cited or suspended for "health and safety incidents" since 2010. At least 30 of the incidents involved hazing, and dozens more invol -ved alcohol. However, the list is missing numerous incidents from recent months. Among them, according to various media outlets: Yale University banned the SAEs from campus activities last month after members al -legedly tried to interfere with a sexual misconduct investigation connected to an initiation rite. Stanford University in December suspended SAE housing privileges after finding sorority members attending a frat -ernity function were subjected to graphic sexual content. And Johns Hopkins University in November suspended the fraternity for underage drinking. "The media has labeled us as the 'nation's deadliest fraternity, -' " Cohen said. In 2011, for example, a student died while being coerced into excessive alcohol consumption, according to a lawsuit. SAE's previous insurer dumped the fraternity. "As a result, we are paying Lloy -d's of London the highest insurance rates in the Greek-letter world," Cohen said. Universities have turned down SAE's attempts to open new chapters, and the fraternity had to close 12 in 18 months over hazing in -cidents.""" - -input_ids = bert_tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids) - -# we need a gpt2 tokenizer for the output word embeddings -gpt2_tokenizer = GPT2Tokenizer.from_pretrained("gpt2") - -print(gpt2_tokenizer.decode(output_ids[0], skip_special_tokens=True)) -# should produce -# SAE's national chapter suspended the students, but university president says it's permanent. -# The fraternity has had to deal with a string of incidents since 2010. -# SAE has more than 200,000 members, many of whom are students. -# A student died while being coerced into drinking alcohol. -``` - -## Training script: - -**IMPORTANT**: In order for this code to work, make sure you checkout to the branch -[more_general_trainer_metric](https://github.com/huggingface/transformers/tree/more_general_trainer_metric), which slightly adapts -the `Trainer` for `EncoderDecoderModels` according to this PR: https://github.com/huggingface/transformers/pull/5840. - -The following code shows the complete training script that was used to fine-tune `bert2gpt2-cnn_dailymail-fp16 -` for reproducability. The training last ~11h on a standard GPU. - -```python -#!/usr/bin/env python3 -import nlp -import logging -from transformers import BertTokenizer, GPT2Tokenizer, EncoderDecoderModel, Trainer, TrainingArguments - -logging.basicConfig(level=logging.INFO) - -model = EncoderDecoderModel.from_encoder_decoder_pretrained("bert-base-cased", "gpt2") -# cache is currently not supported by EncoderDecoder framework -model.decoder.config.use_cache = False -bert_tokenizer = BertTokenizer.from_pretrained("bert-base-cased") - -# CLS token will work as BOS token -bert_tokenizer.bos_token = bert_tokenizer.cls_token - -# SEP token will work as EOS token -bert_tokenizer.eos_token = bert_tokenizer.sep_token - - -# make sure GPT2 appends EOS in begin and end -def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None): - outputs = [self.bos_token_id] + token_ids_0 + [self.eos_token_id] - return outputs - - -GPT2Tokenizer.build_inputs_with_special_tokens = build_inputs_with_special_tokens -gpt2_tokenizer = GPT2Tokenizer.from_pretrained("gpt2") -# set pad_token_id to unk_token_id -> be careful here as unk_token_id == eos_token_id == bos_token_id -gpt2_tokenizer.pad_token = gpt2_tokenizer.unk_token - - -# set decoding params -model.config.decoder_start_token_id = gpt2_tokenizer.bos_token_id -model.config.eos_token_id = gpt2_tokenizer.eos_token_id -model.config.max_length = 142 -model.config.min_length = 56 -model.config.no_repeat_ngram_size = 3 -model.early_stopping = True -model.length_penalty = 2.0 -model.num_beams = 4 - -# load train and validation data -train_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="train") -val_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="validation[:5%]") - -# load rouge for validation -rouge = nlp.load_metric("rouge", experiment_id=1) - -encoder_length = 512 -decoder_length = 128 -batch_size = 16 - - -# map data correctly -def map_to_encoder_decoder_inputs(batch): # Tokenizer will automatically set [BOS] [EOS] - # use bert tokenizer here for encoder - inputs = bert_tokenizer(batch["article"], padding="max_length", truncation=True, max_length=encoder_length) - # force summarization <= 128 - outputs = gpt2_tokenizer(batch["highlights"], padding="max_length", truncation=True, max_length=decoder_length) - - batch["input_ids"] = inputs.input_ids - batch["attention_mask"] = inputs.attention_mask - batch["decoder_input_ids"] = outputs.input_ids - batch["labels"] = outputs.input_ids.copy() - batch["decoder_attention_mask"] = outputs.attention_mask - - # complicated list comprehension here because pad_token_id alone is not good enough to know whether label should be excluded or not - batch["labels"] = [ - [-100 if mask == 0 else token for mask, token in mask_and_tokens] for mask_and_tokens in [zip(masks, labels) for masks, labels in zip(batch["decoder_attention_mask"], batch["labels"])] - ] - - assert all([len(x) == encoder_length for x in inputs.input_ids]) - assert all([len(x) == decoder_length for x in outputs.input_ids]) - - return batch - - -def compute_metrics(pred): - labels_ids = pred.label_ids - pred_ids = pred.predictions - - # all unnecessary tokens are removed - pred_str = gpt2_tokenizer.batch_decode(pred_ids, skip_special_tokens=True) - labels_ids[labels_ids == -100] = gpt2_tokenizer.eos_token_id - label_str = gpt2_tokenizer.batch_decode(labels_ids, skip_special_tokens=True) - - rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - - return { - "rouge2_precision": round(rouge_output.precision, 4), - "rouge2_recall": round(rouge_output.recall, 4), - "rouge2_fmeasure": round(rouge_output.fmeasure, 4), - } - - -# make train dataset ready -train_dataset = train_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -train_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "decoder_input_ids", "decoder_attention_mask", "labels"], -) - -# same for validation dataset -val_dataset = val_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -val_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "decoder_input_ids", "decoder_attention_mask", "labels"], -) - -# set training arguments - these params are not really tuned, feel free to change -training_args = TrainingArguments( - output_dir="./", - per_device_train_batch_size=batch_size, - per_device_eval_batch_size=batch_size, - predict_from_generate=True, - evaluate_during_training=True, - do_train=True, - do_eval=True, - logging_steps=1000, - save_steps=1000, - eval_steps=1000, - overwrite_output_dir=True, - warmup_steps=2000, - save_total_limit=10, - fp16=True, -) - -# instantiate trainer -trainer = Trainer( - model=model, - args=training_args, - compute_metrics=compute_metrics, - train_dataset=train_dataset, - eval_dataset=val_dataset, -) - -# start training -trainer.train() -``` - -## Evaluation - -The following script evaluates the model on the test set of -CNN/Daily Mail. - -```python -#!/usr/bin/env python3 -import nlp -from transformers import BertTokenizer, GPT2Tokenizer, EncoderDecoderModel - -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/bert2gpt2-cnn_dailymail-fp16") -model.to("cuda") - -bert_tokenizer = BertTokenizer.from_pretrained("bert-base-cased") - -# CLS token will work as BOS token -bert_tokenizer.bos_token = bert_tokenizer.cls_token - -# SEP token will work as EOS token -bert_tokenizer.eos_token = bert_tokenizer.sep_token - - -# make sure GPT2 appends EOS in begin and end -def build_inputs_with_special_tokens(self, token_ids_0, token_ids_1=None): - outputs = [self.bos_token_id] + token_ids_0 + [self.eos_token_id] - return outputs - - -GPT2Tokenizer.build_inputs_with_special_tokens = build_inputs_with_special_tokens -gpt2_tokenizer = GPT2Tokenizer.from_pretrained("gpt2") -# set pad_token_id to unk_token_id -> be careful here as unk_token_id == eos_token_id == bos_token_id -gpt2_tokenizer.pad_token = gpt2_tokenizer.unk_token - - -# set decoding params -model.config.decoder_start_token_id = gpt2_tokenizer.bos_token_id -model.config.eos_token_id = gpt2_tokenizer.eos_token_id -model.config.max_length = 142 -model.config.min_length = 56 -model.config.no_repeat_ngram_size = 3 -model.early_stopping = True -model.length_penalty = 2.0 -model.num_beams = 4 - -test_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="test") -batch_size = 64 - - -# map data correctly -def generate_summary(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at BERT max length 512 - inputs = bert_tokenizer(batch["article"], padding="max_length", truncation=True, max_length=512, return_tensors="pt") - input_ids = inputs.input_ids.to("cuda") - attention_mask = inputs.attention_mask.to("cuda") - - outputs = model.generate(input_ids, attention_mask=attention_mask) - - # all special tokens including will be removed - output_str = gpt2_tokenizer.batch_decode(outputs, skip_special_tokens=True) - - batch["pred"] = output_str - - return batch - - -results = test_dataset.map(generate_summary, batched=True, batch_size=batch_size, remove_columns=["article"]) - -# load rouge for validation -rouge = nlp.load_metric("rouge") - -pred_str = results["pred"] -label_str = results["highlights"] - -rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - -print(rouge_output) -``` - -The obtained results should be: - -| - | Rouge2 - mid -precision | Rouge2 - mid - recall | Rouge2 - mid - fmeasure | -|----------|:-------------:|:------:|:------:| -| **CNN/Daily Mail** | 14.42 | 16.99 | **15.16** | diff --git a/model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md b/model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md deleted file mode 100644 index 7b94152690..0000000000 --- a/model_cards/patrickvonplaten/longformer2roberta-cnn_dailymail-fp16/README.md +++ /dev/null @@ -1,252 +0,0 @@ -# Longformer2Roberta Summarization with 🤗 EncoderDecoder Framework - -This model is a Longformer2Roberta model fine-tuned on summarization. - -Longformer2Roberta is a `EncoderDecoderModel`, meaning that both the encoder is a `allenai/longformer-base-4096` model and the decoder is a `roberta-base` model. Leveraging the [EncoderDecoderFramework](https://huggingface.co/transformers/model_doc/encoderdecoder.html#encoder-decoder-models), the -two pretrained models can simply be loaded into the framework via: - -```python -roberta2roberta = EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-base-4096", "roberta-base") -``` - -The decoder of an `EncoderDecoder` model needs cross-attention layers and usually makes use of causal -masking for auto-regressiv generation. -Thus, ``longformer2roberta`` is consequently fined-tuned on the `CNN/Daily Mail`dataset and the resulting model -`longformer2roberta-cnn_dailymail-fp16` is uploaded here. - -## Example - -The model is by no means a state-of-the-art model, but nevertheless -produces reasonable summarization results. It was mainly fine-tuned -as a proof-of-concept for the 🤗 EncoderDecoder Framework. - -The model can be used as follows: - -```python -from transformers import LongformerTokenizer, EncoderDecoderModel - -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/longformer2roberta-cnn_dailymail-fp16") -tokenizer = LongformerTokenizer.from_pretrained("allenai/longformer-base-4096") - -article = """(CNN)James Holmes made his introduction to the world in a Colorado cinema filled with spectators watching a midnight showing of the new Batman movie, "The Dark Knight Rises," in June 2012. The moment became one of the deadliest shootings in U.S. history. Holmes is accused of opening fire on the crowd, killing 12 people and injuring or maiming 70 others in Aurora, a suburb of Denver. Holmes appeared like a comic book character: He resembled the Joker, with red-orange hair, similar to the late actor Heath Ledger\'s portrayal of the villain in an earlier Batman movie, authorities said. But Holmes was hardly a cartoon. Authorities said he wore body armor and carried several guns, including an AR-15 rifle, with lots of ammo. He also wore a gas mask. Holmes says he was insane at the time of the shootings, and that is his legal defense and court plea: not guilty by reason of insanity. Prosecutors aren\'t swayed and will seek the death penalty. Opening statements in his trial are scheduled to begin Monday. Holmes admits to the shootings but says he was suffering "a psychotic episode" at the time, according to court papers filed in July 2013 by the state public defenders, Daniel King and Tamara A. Brady. Evidence "revealed thus far in the case supports the defense\'s position that Mr. Holmes suffers from a severe mental illness and was in the throes of a psychotic episode when he committed the acts that resulted in the tragic loss of life and injuries sustained by moviegoers on July 20, 2012," the public defenders wrote. Holmes no longer looks like a dazed Joker, as he did in his first appearance before a judge in 2012. He appeared dramatically different in January when jury selection began for his trial: 9,000 potential jurors were summoned for duty, described as one of the nation\'s largest jury calls. Holmes now has a cleaner look, with a mustache, button-down shirt and khaki pants. In January, he had a beard and eyeglasses. If this new image sounds like one of an academician, it may be because Holmes, now 27, once was one. Just before the shooting, Holmes was a doctoral student in neuroscience, and he was studying how the brain works, with his schooling funded by a U.S. government grant. Yet for all his learning, Holmes apparently lacked the capacity to command his own mind, according to the case against him. A jury will ultimately decide Holmes\' fate. That panel is made up of 12 jurors and 12 alternates. They are 19 women and five men, and almost all are white and middle-aged. The trial could last until autumn. When jury summonses were issued in January, each potential juror stood a 0.2% chance of being selected, District Attorney George Brauchler told the final jury this month. He described the approaching trial as "four to five months of a horrible roller coaster through the worst haunted house you can imagine." The jury will have to render verdicts on each of the 165 counts against Holmes, including murder and attempted murder charges. Meanwhile, victims and their relatives are challenging all media outlets "to stop the gratuitous use of the name and likeness of mass killers, thereby depriving violent individuals the media celebrity and media spotlight they so crave," the No Notoriety group says. They are joined by victims from eight other mass shootings in recent U.S. history. Raised in central coastal California and in San Diego, James Eagan Holmes is the son of a mathematician father noted for his work at the FICO firm that provides credit scores and a registered nurse mother, according to the U-T San Diego newspaper. Holmes also has a sister, Chris, a musician, who\'s five years younger, the newspaper said. His childhood classmates remember him as a clean-cut, bespectacled boy with an "exemplary" character who "never gave any trouble, and never got in trouble himself," The Salinas Californian reported. His family then moved down the California coast, where Holmes grew up in the San Diego-area neighborhood of Rancho Peñasquitos, which a neighbor described as "kind of like Mayberry," the San Diego newspaper said. Holmes attended Westview High School, which says its school district sits in "a primarily middle- to upper-middle-income residential community." There, Holmes ran cross-country, played soccer and later worked at a biotechnology internship at the Salk Institute and Miramar College, which attracts academically talented students. By then, his peers described him as standoffish and a bit of a wiseacre, the San Diego newspaper said. Holmes attended college fairly close to home, in a neighboring area known as Southern California\'s "inland empire" because it\'s more than an hour\'s drive from the coast, in a warm, low-desert climate. He entered the University of California, Riverside, in 2006 as a scholarship student. In 2008 he was a summer camp counselor for disadvantaged children, age 7 to 14, at Camp Max Straus, run by Jewish Big Brothers Big Sisters of Los Angeles. He graduated from UC Riverside in 2010 with the highest honors and a bachelor\'s degree in neuroscience. "Academically, he was at the top of the top," Chancellor Timothy P. White said. He seemed destined for even higher achievement. By 2011, he had enrolled as a doctoral student in the neuroscience program at the University of Colorado Anschutz Medical Campus in Aurora, the largest academic health center in the Rocky Mountain region. The doctoral in neuroscience program attended by Holmes focuses on how the brain works, with an emphasis on processing of information, behavior, learning and memory. Holmes was one of six pre-thesis Ph.D. students in the program who were awarded a neuroscience training grant from the National Institutes of Health. The grant rewards outstanding neuroscientists who will make major contributions to neurobiology. A syllabus that listed Holmes as a student at the medical school shows he was to have delivered a presentation about microRNA biomarkers. But Holmes struggled, and his own mental health took an ominous turn. In March 2012, he told a classmate he wanted to kill people, and that he would do so "when his life was over," court documents said. Holmes was "denied access to the school after June 12, 2012, after he made threats to a professor," according to court documents. About that time, Holmes was a patient of University of Colorado psychiatrist Lynne Fenton. Fenton was so concerned about Holmes\' behavior that she mentioned it to her colleagues, saying he could be a danger to others, CNN affiliate KMGH-TV reported, citing sources with knowledge of the investigation. Fenton\'s concerns surfaced in early June, sources told the Denver station. Holmes began to fantasize about killing "a lot of people" in early June, nearly six weeks before the shootings, the station reported, citing unidentified sources familiar with the investigation. Holmes\' psychiatrist contacted several members of a "behavioral evaluation and threat assessment" team to say Holmes could be a danger to others, the station reported. At issue was whether to order Holmes held for 72 hours to be evaluated by mental health professionals, the station reported. "Fenton made initial phone calls about engaging the BETA team" in "the first 10 days" of June, but it "never came together" because in the period Fenton was having conversations with team members, Holmes began the process of dropping out of school, a source told KMGH. Defense attorneys have rejected the prosecution\'s assertions that Holmes was barred from campus. Citing statements from the university, Holmes\' attorneys have argued that his access was revoked because that\'s normal procedure when a student drops enrollment. What caused this turn for the worse for Holmes has yet to be clearly detailed. In the months before the shooting, he bought four weapons and more than 6,000 rounds of ammunition, authorities said. Police said he also booby-trapped his third-floor apartment with explosives, but police weren\'t fooled. After Holmes was caught in the cinema parking lot immediately after the shooting, bomb technicians went to the apartment and neutralized the explosives. No one was injured at the apartment building. Nine minutes before Holmes went into the movie theater, he called a University of Colorado switchboard, public defender Brady has said in court. The number he called can be used to get in contact with faculty members during off hours, Brady said. Court documents have also revealed that investigators have obtained text messages that Holmes exchanged with someone before the shooting. That person was not named, and the content of the texts has not been made public. According to The New York Times, Holmes sent a text message to a fellow graduate student, a woman, about two weeks before the shooting. She asked if he had left Aurora yet, reported the newspaper, which didn\'t identify her. No, he had two months left on his lease, Holmes wrote back, according to the Times. He asked if she had heard of "dysphoric mania," a form of bipolar disorder marked by the highs of mania and the dark and sometimes paranoid delusions of major depression. The woman asked if the disorder could be managed with treatment. "It was," Holmes wrote her, according to the Times. But he warned she should stay away from him "because I am bad news," the newspaper reported. It was her last contact with Holmes. After the shooting, Holmes\' family issued a brief statement: "Our hearts go out to those who were involved in this tragedy and to the families and friends of those involved," they said, without giving any information about their son. Since then, prosecutors have refused to offer a plea deal to Holmes. For Holmes, "justice is death," said Brauchler, the district attorney. In December, Holmes\' parents, who will be attending the trial, issued another statement: They asked that their son\'s life be spared and that he be sent to an institution for mentally ill people for the rest of his life, if he\'s found not guilty by reason of insanity. "He is not a monster," Robert and Arlene Holmes wrote, saying the death penalty is "morally wrong, especially when the condemned is mentally ill." "He is a human being gripped by a severe mental illness," the parents said. The matter will be settled by the jury. CNN\'s Ana Cabrera and Sara Weisfeldt contributed to this report from Denver.""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids) - -print(tokenizer.decode(output_ids[0], skip_special_tokens=True)) -# should produce -# James Holmes, 27, is accused of opening fire on a Colorado theater. -# He was a doctoral student at University of Colorado. -# Holmes says he was suffering "a psychotic episode" at the time of the shooting. -# Prosecutors won't say whether Holmes was barred from campus. -``` - -Such an article has a length of > 2000 tokens, which means that it cannot be handled correctly by Bert or Roberta encoders. - -## Training script: - -**IMPORTANT**: In order for this code to work, make sure you checkout to the branch -[more_general_trainer_metric](https://github.com/huggingface/transformers/tree/more_general_trainer_metric), which slightly adapts -the `Trainer` for `EncoderDecoderModels` according to this PR: https://github.com/huggingface/transformers/pull/5840. - -The following code shows the complete training script that was used to fine-tune `longformer2roberta-cnn_dailymail-fp16 -` for reproducability. The training last ~90h on a standard GPU. - -```python -#!/usr/bin/env python3 -import nlp -import logging -from transformers import LongformerTokenizer, EncoderDecoderModel, Trainer, TrainingArguments - -logging.basicConfig(level=logging.INFO) - -model = EncoderDecoderModel.from_encoder_decoder_pretrained("allenai/longformer-base-4096", "roberta-base") -tokenizer = LongformerTokenizer.from_pretrained("allenai/longformer-base-4096") - -# load train and validation data -train_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="train") -val_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="validation[:5%]") - -# load rouge for validation -rouge = nlp.load_metric("rouge", experiment_id=0) - -# enable gradient checkpointing for longformer encoder -model.encoder.config.gradient_checkpointing = True - -# set decoding params -model.config.decoder_start_token_id = tokenizer.bos_token_id -model.config.eos_token_id = tokenizer.eos_token_id -model.config.max_length = 142 -model.config.min_length = 56 -model.config.no_repeat_ngram_size = 3 -model.early_stopping = True -model.length_penalty = 2.0 -model.num_beams = 4 - -encoder_length = 2048 -decoder_length = 128 -batch_size = 16 - - -# map data correctly -def map_to_encoder_decoder_inputs(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at Longformer at 2048 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=encoder_length) - # force summarization <= 128 - outputs = tokenizer(batch["highlights"], padding="max_length", truncation=True, max_length=decoder_length) - - batch["input_ids"] = inputs.input_ids - batch["attention_mask"] = inputs.attention_mask - - # set 128 tokens to global attention - batch["global_attention_mask"] = [[1 if i < 128 else 0 for i in range(sequence_length)] for sequence_length in len(inputs.input_ids) * [encoder_length]] - batch["decoder_input_ids"] = outputs.input_ids - batch["labels"] = outputs.input_ids.copy() - # mask loss for padding - batch["labels"] = [ - [-100 if token == tokenizer.pad_token_id else token for token in labels] for labels in batch["labels"] - ] - batch["decoder_attention_mask"] = outputs.attention_mask - - assert all([len(x) == encoder_length for x in inputs.input_ids]) - assert all([len(x) == decoder_length for x in outputs.input_ids]) - - return batch - - -def compute_metrics(pred): - labels_ids = pred.label_ids - pred_ids = pred.predictions - - # all unnecessary tokens are removed - pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True) - labels_ids[labels_ids == -100] = tokenizer.eos_token_id - label_str = tokenizer.batch_decode(labels_ids, skip_special_tokens=True) - - rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - - return { - "rouge2_precision": round(rouge_output.precision, 4), - "rouge2_recall": round(rouge_output.recall, 4), - "rouge2_fmeasure": round(rouge_output.fmeasure, 4), - } - - - return { - "rouge2_precision": round(rouge_output.precision, 4), - "rouge2_recall": round(rouge_output.recall, 4), - "rouge2_fmeasure": round(rouge_output.fmeasure, 4), - } - - -# make train dataset ready -train_dataset = train_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -train_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "global_attention_mask", "decoder_input_ids", "decoder_attention_mask", "labels"], -) - -# same for validation dataset -val_dataset = val_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -val_dataset.set_format( - type="torch", columns=["input_ids", "global_attention_mask", "attention_mask", "decoder_input_ids", "decoder_attention_mask", "labels"], -) - -# set training arguments - these params are not really tuned, feel free to change -training_args = TrainingArguments( - output_dir="./", - per_device_train_batch_size=batch_size, - per_device_eval_batch_size=batch_size, - predict_from_generate=True, - evaluate_during_training=True, - do_train=True, - do_eval=True, - logging_steps=1000, - save_steps=1000, - eval_steps=1000, - overwrite_output_dir=True, - warmup_steps=2000, - save_total_limit=3, - fp16=True, -) - -# instantiate trainer -trainer = Trainer( - model=model, - args=training_args, - compute_metrics=compute_metrics, - train_dataset=train_dataset, - eval_dataset=val_dataset, -) - -# start training -trainer.train() -``` - -## Evaluation - -The following script evaluates the model on the test set of -CNN/Daily Mail. - -```python -#!/usr/bin/env python3 -import nlp -import torch -from transformers import LongformerTokenizer, EncoderDecoderModel - -tokenizer = LongformerTokenizer.from_pretrained("allenai/longformer-base-4096") -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/longformer2roberta-cnn_dailymail-fp16") -model.to("cuda") - -test_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="test") -batch_size = 32 - -encoder_length = 2048 -decoder_length = 128 - - -# map data correctly -def generate_summary(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at BERT max length 512 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=encoder_length, return_tensors="pt") - input_ids = inputs.input_ids.to("cuda") - attention_mask = inputs.attention_mask.to("cuda") - global_attention_mask = torch.zeros_like(attention_mask) - global_attention_mask[:, :decoder_length] = 1 - - outputs = model.generate(input_ids, attention_mask=attention_mask, global_attention_mask=global_attention_mask) - - # all special tokens including will be removed - output_str = tokenizer.batch_decode(outputs, skip_special_tokens=True) - - batch["pred"] = output_str - - return batch - - -results = test_dataset.map(generate_summary, batched=True, batch_size=batch_size, remove_columns=["article"]) - -# load rouge for validation -rouge = nlp.load_metric("rouge") - -pred_str = results["pred"] -label_str = results["highlights"] - -rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - -print(rouge_output) -``` - -The obtained results should be: - -| - | Rouge2 - mid -precision | Rouge2 - mid - recall | Rouge2 - mid - fmeasure | -|----------|:-------------:|:------:|:------:| -| **CNN/Daily Mail** | 12.39 | 15.05 | **13.21** | - -**Note** This model was trained to show how Longformer can be used as an Encoder model in a EncoderDecoder setup. -Better results are obtained for datasets of much longer inputs. diff --git a/model_cards/patrickvonplaten/roberta2roberta-cnn_dailymail-fp16/README.md b/model_cards/patrickvonplaten/roberta2roberta-cnn_dailymail-fp16/README.md deleted file mode 100644 index 3cfe85cd70..0000000000 --- a/model_cards/patrickvonplaten/roberta2roberta-cnn_dailymail-fp16/README.md +++ /dev/null @@ -1,238 +0,0 @@ -# Roberta2Roberta Summarization with 🤗 EncoderDecoder Framework - -This model is a Roberta2Roberta model fine-tuned on summarization. - -Roberta2Roberta is a `EncoderDecoderModel`, meaning that both the encoder and the decoder are `roberta-base` -RoBERTa models. Leveraging the [EncoderDecoderFramework](https://huggingface.co/transformers/model_doc/encoderdecoder.html#encoder-decoder-models), the -two pretrained models can simply be loaded into the framework via: - -```python -roberta2roberta = EncoderDecoderModel.from_encoder_decoder_pretrained("roberta-base", "roberta-base") -``` - -The decoder of an `EncoderDecoder` model needs cross-attention layers and usually makes use of causal -masking for auto-regressiv generation. -Thus, ``roberta2roberta`` is consequently fined-tuned on the `CNN/Daily Mail`dataset and the resulting model -`roberta2roberta-cnn_dailymail-fp16` is uploaded here. - -## Example - -The model is by no means a state-of-the-art model, but nevertheless -produces reasonable summarization results. It was mainly fine-tuned -as a proof-of-concept for the 🤗 EncoderDecoder Framework. - -The model can be used as follows: - -```python -from transformers import BertTokenizer, EncoderDecoderModel - -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/roberta2roberta-cnn_dailymail-fp16") -tokenizer = RobertaTokenizer.from_pretrained("roberta-base") - -article = """(CNN)Sigma Alpha Epsilon is under fire for a video showing party-bound fraternity members singing a racist chant. SAE's national chapter suspended the students, but University of Oklahoma President David B -oren took it a step further, saying the university's affiliation with the fraternity is permanently done. The news is shocking, but it's not the first time SAE has faced controversy. SAE was founded March 9, 185 -6, at the University of Alabama, five years before the American Civil War, according to the fraternity website. When the war began, the group had fewer than 400 members, of which "369 went to war for the Confede -rate States and seven for the Union Army," the website says. The fraternity now boasts more than 200,000 living alumni, along with about 15,000 undergraduates populating 219 chapters and 20 "colonies" seeking fu -ll membership at universities. SAE has had to work hard to change recently after a string of member deaths, many blamed on the hazing of new recruits, SAE national President Bradley Cohen wrote in a message on t -he fraternity's website. The fraternity's website lists more than 130 chapters cited or suspended for "health and safety incidents" since 2010. At least 30 of the incidents involved hazing, and dozens more invol -ved alcohol. However, the list is missing numerous incidents from recent months. Among them, according to various media outlets: Yale University banned the SAEs from campus activities last month after members al -legedly tried to interfere with a sexual misconduct investigation connected to an initiation rite. Stanford University in December suspended SAE housing privileges after finding sorority members attending a frat -ernity function were subjected to graphic sexual content. And Johns Hopkins University in November suspended the fraternity for underage drinking. "The media has labeled us as the 'nation's deadliest fraternity, -' " Cohen said. In 2011, for example, a student died while being coerced into excessive alcohol consumption, according to a lawsuit. SAE's previous insurer dumped the fraternity. "As a result, we are paying Lloy -d's of London the highest insurance rates in the Greek-letter world," Cohen said. Universities have turned down SAE's attempts to open new chapters, and the fraternity had to close 12 in 18 months over hazing in -cidents.""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids) - -print(tokenizer.decode(output_ids[0], skip_special_tokens=True)) -# should produce -# Sigma Alpha Epsilon is under fire for a video showing party-bound fraternity members singing racist chants. The fraternity's national chapter has had to close 12 in 18 months over hazing. -# Sigma has had more than 130 chapters in 18 states. University of Oklahoma president says fraternity has been "deteriorated". -``` - -## Training script: - -**IMPORTANT**: In order for this code to work, make sure you checkout to the branch -[more_general_trainer_metric](https://github.com/huggingface/transformers/tree/more_general_trainer_metric), which slightly adapts -the `Trainer` for `EncoderDecoderModels` according to this PR: https://github.com/huggingface/transformers/pull/5840. - -The following code shows the complete training script that was used to fine-tune `roberta2roberta-cnn_dailymail-fp16 -` for reproducability. The training last ~9h on a standard GPU. - -```python -#!/usr/bin/env python3 -import nlp -import logging -from transformers import RobertaTokenizer, EncoderDecoderModel, Trainer, TrainingArguments - -logging.basicConfig(level=logging.INFO) - -model = EncoderDecoderModel.from_encoder_decoder_pretrained("roberta-base", "roberta-base") -tokenizer = RobertaTokenizer.from_pretrained("roberta-base") - -# load train and validation data -train_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="train") -val_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="validation[:5%]") - -# load rouge for validation -rouge = nlp.load_metric("rouge", experiment_id=0) - -# set decoding params -model.config.decoder_start_token_id = tokenizer.bos_token_id -model.config.eos_token_id = tokenizer.eos_token_id -model.config.max_length = 142 -model.config.min_length = 56 -model.config.no_repeat_ngram_size = 3 -model.early_stopping = True -model.length_penalty = 2.0 -model.num_beams = 4 - -encoder_length = 512 -decoder_length = 128 -batch_size = 16 - - -# map data correctly -def map_to_encoder_decoder_inputs(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at Longformer at 2048 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=encoder_length) - # force summarization <= 256 - outputs = tokenizer(batch["highlights"], padding="max_length", truncation=True, max_length=decoder_length) - - batch["input_ids"] = inputs.input_ids - batch["attention_mask"] = inputs.attention_mask - batch["decoder_input_ids"] = outputs.input_ids - batch["labels"] = outputs.input_ids.copy() - # mask loss for padding - batch["labels"] = [ - [-100 if token == tokenizer.pad_token_id else token for token in labels] for labels in batch["labels"] - ] - batch["decoder_attention_mask"] = outputs.attention_mask - - assert all([len(x) == encoder_length for x in inputs.input_ids]) - assert all([len(x) == decoder_length for x in outputs.input_ids]) - - return batch - - -def compute_metrics(pred): - labels_ids = pred.label_ids - pred_ids = pred.predictions - - # all unnecessary tokens are removed - pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True) - labels_ids[labels_ids == -100] = tokenizer.eos_token_id - label_str = tokenizer.batch_decode(labels_ids, skip_special_tokens=True) - - rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - - return { - "rouge2_precision": round(rouge_output.precision, 4), - "rouge2_recall": round(rouge_output.recall, 4), - "rouge2_fmeasure": round(rouge_output.fmeasure, 4), - } - - -# make train dataset ready -train_dataset = train_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -train_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "decoder_attention_mask", "decoder_input_ids", "labels"], -) - -# same for validation dataset -val_dataset = val_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -val_dataset.set_format( - type="torch", columns=["input_ids", "decoder_attention_mask", "attention_mask", "decoder_input_ids", "labels"], -) - -# set training arguments - these params are not really tuned, feel free to change -training_args = TrainingArguments( - output_dir="./", - per_device_train_batch_size=batch_size, - per_device_eval_batch_size=batch_size, - predict_from_generate=True, - evaluate_during_training=True, - do_train=True, - do_eval=True, - logging_steps=1000, - save_steps=1000, - eval_steps=1000, - overwrite_output_dir=True, - warmup_steps=2000, - save_total_limit=3, - fp16=True, -) - -# instantiate trainer -trainer = Trainer( - model=model, - args=training_args, - compute_metrics=compute_metrics, - train_dataset=train_dataset, - eval_dataset=val_dataset, -) - -# start training -trainer.train() -``` - -## Evaluation - -The following script evaluates the model on the test set of -CNN/Daily Mail. - -```python -#!/usr/bin/env python3 -import nlp -from transformers import RobertaTokenizer, EncoderDecoderModel - -tokenizer = RobertaTokenizer.from_pretrained("roberta-base") -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/roberta2roberta-cnn_dailymail-fp16") -model.to("cuda") - -test_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="test") -batch_size = 128 - - -# map data correctly -def generate_summary(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at BERT max length 512 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=512, return_tensors="pt") - input_ids = inputs.input_ids.to("cuda") - attention_mask = inputs.attention_mask.to("cuda") - - outputs = model.generate(input_ids, attention_mask=attention_mask) - - # all special tokens including will be removed - output_str = tokenizer.batch_decode(outputs, skip_special_tokens=True) - - batch["pred"] = output_str - - return batch - - -results = test_dataset.map(generate_summary, batched=True, batch_size=batch_size, remove_columns=["article"]) - -# load rouge for validation -rouge = nlp.load_metric("rouge") - -pred_str = results["pred"] -label_str = results["highlights"] - -rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - -print(rouge_output) -``` - -The obtained results should be: - -| - | Rouge2 - mid -precision | Rouge2 - mid - recall | Rouge2 - mid - fmeasure | -|----------|:-------------:|:------:|:------:| -| **CNN/Daily Mail** | 15.79 | 19.05 | **16.79** | diff --git a/model_cards/patrickvonplaten/roberta2roberta-share-cnn_dailymail-fp16/README.md b/model_cards/patrickvonplaten/roberta2roberta-share-cnn_dailymail-fp16/README.md deleted file mode 100644 index 74cfa7ac5c..0000000000 --- a/model_cards/patrickvonplaten/roberta2roberta-share-cnn_dailymail-fp16/README.md +++ /dev/null @@ -1,238 +0,0 @@ -# Shared Roberta2Roberta Summarization with 🤗 EncoderDecoder Framework - -This model is a shared Roberta2Roberta model, meaning that the encoder and decoder weights are tied, fine-tuned on summarization. - -Roberta2Roberta is a `EncoderDecoderModel`, meaning that both the encoder and the decoder are `roberta-base` -RoBERTa models. In this setup the encoder and decoder weights are tied. Leveraging the [EncoderDecoderFramework](https://huggingface.co/transformers/model_doc/encoderdecoder.html#encoder-decoder-models), the -two pretrained models can simply be loaded into the framework via: - -```python -roberta2roberta = EncoderDecoderModel.from_encoder_decoder_pretrained("roberta-base", "roberta-base", tie_encoder_decoder=True) -``` - -The decoder of an `EncoderDecoder` model needs cross-attention layers and usually makes use of causal -masking for auto-regressiv generation. -Thus, ``roberta2roberta`` is consequently fined-tuned on the `CNN/Daily Mail`dataset and the resulting model -`roberta2roberta-share-cnn_dailymail-fp16` is uploaded here. - -## Example - -The model is by no means a state-of-the-art model, but nevertheless -produces reasonable summarization results. It was mainly fine-tuned -as a proof-of-concept for the 🤗 EncoderDecoder Framework. - -The model can be used as follows: - -```python -from transformers import RobertaTokenizer, EncoderDecoderModel - -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/roberta2roberta-share-cnn_dailymail-fp16") -tokenizer = RobertaTokenizer.from_pretrained("roberta-base") - -article = """(CNN)Sigma Alpha Epsilon is under fire for a video showing party-bound fraternity members singing a racist chant. SAE's national chapter suspended the students, but University of Oklahoma President David B -oren took it a step further, saying the university's affiliation with the fraternity is permanently done. The news is shocking, but it's not the first time SAE has faced controversy. SAE was founded March 9, 185 -6, at the University of Alabama, five years before the American Civil War, according to the fraternity website. When the war began, the group had fewer than 400 members, of which "369 went to war for the Confede -rate States and seven for the Union Army," the website says. The fraternity now boasts more than 200,000 living alumni, along with about 15,000 undergraduates populating 219 chapters and 20 "colonies" seeking fu -ll membership at universities. SAE has had to work hard to change recently after a string of member deaths, many blamed on the hazing of new recruits, SAE national President Bradley Cohen wrote in a message on t -he fraternity's website. The fraternity's website lists more than 130 chapters cited or suspended for "health and safety incidents" since 2010. At least 30 of the incidents involved hazing, and dozens more invol -ved alcohol. However, the list is missing numerous incidents from recent months. Among them, according to various media outlets: Yale University banned the SAEs from campus activities last month after members al -legedly tried to interfere with a sexual misconduct investigation connected to an initiation rite. Stanford University in December suspended SAE housing privileges after finding sorority members attending a frat -ernity function were subjected to graphic sexual content. And Johns Hopkins University in November suspended the fraternity for underage drinking. "The media has labeled us as the 'nation's deadliest fraternity, -' " Cohen said. In 2011, for example, a student died while being coerced into excessive alcohol consumption, according to a lawsuit. SAE's previous insurer dumped the fraternity. "As a result, we are paying Lloy -d's of London the highest insurance rates in the Greek-letter world," Cohen said. Universities have turned down SAE's attempts to open new chapters, and the fraternity had to close 12 in 18 months over hazing in -cidents.""" - -input_ids = tokenizer(article, return_tensors="pt").input_ids -output_ids = model.generate(input_ids) - -print(tokenizer.decode(output_ids[0], skip_special_tokens=True)) -# should produce -# SAE's national chapter suspended after video shows party-bound fraternity members singing racist chant. University of Oklahoma president says university's affiliation with fraternity is permanently done. -# SAE has had to close 12 chapters since 2010 after members were killed in hazing. The fraternity has had more than 130 chapters in 18 months. -``` - -## Training script: - -**IMPORTANT**: In order for this code to work, make sure you checkout to the branch -[more_general_trainer_metric](https://github.com/huggingface/transformers/tree/more_general_trainer_metric), which slightly adapts -the `Trainer` for `EncoderDecoderModels` according to this PR: https://github.com/huggingface/transformers/pull/5840. - -The following code shows the complete training script that was used to fine-tune `roberta2roberta-cnn_dailymail-fp16 -` for reproducability. The training last ~9h on a standard GPU. - -```python -#!/usr/bin/env python3 -import nlp -import logging -from transformers import RobertaTokenizer, EncoderDecoderModel, Trainer, TrainingArguments - -logging.basicConfig(level=logging.INFO) - -model = EncoderDecoderModel.from_encoder_decoder_pretrained("roberta-base", "roberta-base", tie_encoder_decoder=True) -tokenizer = RobertaTokenizer.from_pretrained("roberta-base") - -# load train and validation data -train_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="train") -val_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="validation[:5%]") - -# load rouge for validation -rouge = nlp.load_metric("rouge", experiment_id=0) - -# set decoding params -model.config.decoder_start_token_id = tokenizer.bos_token_id -model.config.eos_token_id = tokenizer.eos_token_id -model.config.max_length = 142 -model.config.min_length = 56 -model.config.no_repeat_ngram_size = 3 -model.early_stopping = True -model.length_penalty = 2.0 -model.num_beams = 4 - -encoder_length = 512 -decoder_length = 128 -batch_size = 16 - - -# map data correctly -def map_to_encoder_decoder_inputs(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at Longformer at 2048 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=encoder_length) - # force summarization <= 256 - outputs = tokenizer(batch["highlights"], padding="max_length", truncation=True, max_length=decoder_length) - - batch["input_ids"] = inputs.input_ids - batch["attention_mask"] = inputs.attention_mask - batch["decoder_input_ids"] = outputs.input_ids - batch["labels"] = outputs.input_ids.copy() - # mask loss for padding - batch["labels"] = [ - [-100 if token == tokenizer.pad_token_id else token for token in labels] for labels in batch["labels"] - ] - batch["decoder_attention_mask"] = outputs.attention_mask - - assert all([len(x) == encoder_length for x in inputs.input_ids]) - assert all([len(x) == decoder_length for x in outputs.input_ids]) - - return batch - - -def compute_metrics(pred): - labels_ids = pred.label_ids - pred_ids = pred.predictions - - # all unnecessary tokens are removed - pred_str = tokenizer.batch_decode(pred_ids, skip_special_tokens=True) - labels_ids[labels_ids == -100] = tokenizer.eos_token_id - label_str = tokenizer.batch_decode(labels_ids, skip_special_tokens=True) - - rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - - return { - "rouge2_precision": round(rouge_output.precision, 4), - "rouge2_recall": round(rouge_output.recall, 4), - "rouge2_fmeasure": round(rouge_output.fmeasure, 4), - } - - -# make train dataset ready -train_dataset = train_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -train_dataset.set_format( - type="torch", columns=["input_ids", "attention_mask", "decoder_attention_mask", "decoder_input_ids", "labels"], -) - -# same for validation dataset -val_dataset = val_dataset.map( - map_to_encoder_decoder_inputs, batched=True, batch_size=batch_size, remove_columns=["article", "highlights"], -) -val_dataset.set_format( - type="torch", columns=["input_ids", "decoder_attention_mask", "attention_mask", "decoder_input_ids", "labels"], -) - -# set training arguments - these params are not really tuned, feel free to change -training_args = TrainingArguments( - output_dir="./", - per_device_train_batch_size=batch_size, - per_device_eval_batch_size=batch_size, - predict_from_generate=True, - evaluate_during_training=True, - do_train=True, - do_eval=True, - logging_steps=1000, - save_steps=1000, - eval_steps=1000, - overwrite_output_dir=True, - warmup_steps=2000, - save_total_limit=3, - fp16=True, -) - -# instantiate trainer -trainer = Trainer( - model=model, - args=training_args, - compute_metrics=compute_metrics, - train_dataset=train_dataset, - eval_dataset=val_dataset, -) - -# start training -trainer.train() -``` - -## Evaluation - -The following script evaluates the model on the test set of -CNN/Daily Mail. - -```python -#!/usr/bin/env python3 -import nlp -from transformers import RobertaTokenizer, EncoderDecoderModel - -tokenizer = RobertaTokenizer.from_pretrained("roberta-base") -model = EncoderDecoderModel.from_pretrained("patrickvonplaten/roberta2roberta-share-cnn_dailymail-fp16") -model.to("cuda") - -test_dataset = nlp.load_dataset("cnn_dailymail", "3.0.0", split="test") -batch_size = 128 - - -# map data correctly -def generate_summary(batch): - # Tokenizer will automatically set [BOS] [EOS] - # cut off at BERT max length 512 - inputs = tokenizer(batch["article"], padding="max_length", truncation=True, max_length=512, return_tensors="pt") - input_ids = inputs.input_ids.to("cuda") - attention_mask = inputs.attention_mask.to("cuda") - - outputs = model.generate(input_ids, attention_mask=attention_mask) - - # all special tokens including will be removed - output_str = tokenizer.batch_decode(outputs, skip_special_tokens=True) - - batch["pred"] = output_str - - return batch - - -results = test_dataset.map(generate_summary, batched=True, batch_size=batch_size, remove_columns=["article"]) - -# load rouge for validation -rouge = nlp.load_metric("rouge") - -pred_str = results["pred"] -label_str = results["highlights"] - -rouge_output = rouge.compute(predictions=pred_str, references=label_str, rouge_types=["rouge2"])["rouge2"].mid - -print(rouge_output) -``` - -The obtained results should be: - -| - | Rouge2 - mid -precision | Rouge2 - mid - recall | Rouge2 - mid - fmeasure | -|----------|:-------------:|:------:|:------:| -| **CNN/Daily Mail** | 15.6 | 18.79 | **16.59** | diff --git a/model_cards/patrickvonplaten/roberta_shared_bbc_xsum/README.md b/model_cards/patrickvonplaten/roberta_shared_bbc_xsum/README.md deleted file mode 100644 index 44b0a42a62..0000000000 --- a/model_cards/patrickvonplaten/roberta_shared_bbc_xsum/README.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- xsum -tags: -- summarization ---- - -Shared RoBERTa2RoBERTa Summarization with 🤗EncoderDecoder Framework -This model is a warm-started *RoBERTaShared* model fine-tuned on the *BBC XSum* summarization dataset. - -The model achieves a **16.89** ROUGE-2 score on *BBC XSUM*'s test dataset. - -For more details on how the model was fine-tuned, please refer to -[this](https://colab.research.google.com/drive/1Ekd5pUeCX7VOrMx94_czTkwNtLN32Uyu?usp=sharing) notebook. diff --git a/model_cards/pdelobelle/robbert-v2-dutch-base/README.md b/model_cards/pdelobelle/robbert-v2-dutch-base/README.md deleted file mode 100644 index 84170035e0..0000000000 --- a/model_cards/pdelobelle/robbert-v2-dutch-base/README.md +++ /dev/null @@ -1,164 +0,0 @@ ---- -language: "nl" -thumbnail: "https://github.com/iPieter/RobBERT/raw/master/res/robbert_logo.png" -tags: -- Dutch -- RoBERTa -- RobBERT -license: mit -datasets: -- oscar -- Shuffled Dutch section of the OSCAR corpus (https://oscar-corpus.com/) ---- - -# RobBERT - -## Model description - -[RobBERT v2](https://github.com/iPieter/RobBERT) is a Dutch state-of-the-art [RoBERTa](https://arxiv.org/abs/1907.11692)-based language model. - -More detailled information can be found in the [RobBERT paper](https://arxiv.org/abs/2001.06286). - -## How to use - -```python -from transformers import RobertaTokenizer, RobertaForSequenceClassification -tokenizer = RobertaTokenizer.from_pretrained("pdelobelle/robbert-v2-dutch-base") -model = RobertaForSequenceClassification.from_pretrained("pdelobelle/robbert-v2-dutch-base") -``` - -## Performance Evaluation Results - -All experiments are described in more detail in our [paper](https://arxiv.org/abs/2001.06286). - -### Sentiment analysis -Predicting whether a review is positive or negative using the [Dutch Book Reviews Dataset](https://github.com/benjaminvdb/110kDBRD). - -| Model | Accuracy [%] | -|-------------------|--------------------------| -| ULMFiT | 93.8 | -| BERTje | 93.0 | -| RobBERT v2 | **95.1** | - -### Die/Dat (coreference resolution) - -We measured how well the models are able to do coreference resolution by predicting whether "die" or "dat" should be filled into a sentence. -For this, we used the [EuroParl corpus](https://www.statmt.org/europarl/). - -#### Finetuning on whole dataset - -| Model | Accuracy [%] | F1 [%] | -|-------------------|--------------------------|--------------| -| [Baseline](https://arxiv.org/abs/2001.02943) (LSTM) | | 75.03 | -| mBERT | 98.285 | 98.033 | -| BERTje | 98.268 | 98.014 | -| RobBERT v2 | **99.232** | **99.121** | - -#### Finetuning on 10K examples - -We also measured the performance using only 10K training examples. -This experiment clearly illustrates that RobBERT outperforms other models when there is little data available. - -| Model | Accuracy [%] | F1 [%] | -|-------------------|--------------------------|--------------| -| mBERT | 92.157 | 90.898 | -| BERTje | 93.096 | 91.279 | -| RobBERT v2 | **97.816** | **97.514** | - -#### Using zero-shot word masking task - -Since BERT models are pre-trained using the word masking task, we can use this to predict whether "die" or "dat" is more likely. -This experiment shows that RobBERT has internalised more information about Dutch than other models. - -| Model | Accuracy [%] | -|-------------------|--------------------------| -| ZeroR | 66.70 | -| mBERT | 90.21 | -| BERTje | 94.94 | -| RobBERT v2 | **98.75** | - -### Part-of-Speech Tagging. - -Using the [Lassy UD dataset](https://universaldependencies.org/treebanks/nl_lassysmall/index.html). - - -| Model | Accuracy [%] | -|-------------------|--------------------------| -| Frog | 91.7 | -| mBERT | **96.5** | -| BERTje | 96.3 | -| RobBERT v2 | 96.4 | - -Interestingly, we found that when dealing with **small data sets**, RobBERT v2 **significantly outperforms** other models. - -

- RobBERT's performance on smaller datasets -

- -### Named Entity Recognition - -Using the [CoNLL 2002 evaluation script](https://www.clips.uantwerpen.be/conll2002/ner/). - - -| Model | Accuracy [%] | -|-------------------|--------------------------| -| Frog | 57.31 | -| mBERT | **90.94** | -| BERT-NL | 89.7 | -| BERTje | 88.3 | -| RobBERT v2 | 89.08 | - - -## Training procedure - -We pre-trained RobBERT using the RoBERTa training regime. -We pre-trained our model on the Dutch section of the [OSCAR corpus](https://oscar-corpus.com/), a large multilingual corpus which was obtained by language classification in the Common Crawl corpus. -This Dutch corpus is 39GB large, with 6.6 billion words spread over 126 million lines of text, where each line could contain multiple sentences, thus using more data than concurrently developed Dutch BERT models. - - -RobBERT shares its architecture with [RoBERTa's base model](https://github.com/pytorch/fairseq/tree/master/examples/roberta), which itself is a replication and improvement over BERT. -Like BERT, it's architecture consists of 12 self-attention layers with 12 heads with 117M trainable parameters. -One difference with the original BERT model is due to the different pre-training task specified by RoBERTa, using only the MLM task and not the NSP task. -During pre-training, it thus only predicts which words are masked in certain positions of given sentences. -The training process uses the Adam optimizer with polynomial decay of the learning rate l_r=10^-6 and a ramp-up period of 1000 iterations, with hyperparameters beta_1=0.9 -and RoBERTa's default beta_2=0.98. -Additionally, a weight decay of 0.1 and a small dropout of 0.1 helps prevent the model from overfitting. - - -RobBERT was trained on a computing cluster with 4 Nvidia P100 GPUs per node, where the number of nodes was dynamically adjusted while keeping a fixed batch size of 8192 sentences. -At most 20 nodes were used (i.e. 80 GPUs), and the median was 5 nodes. -By using gradient accumulation, the batch size could be set independently of the number of GPUs available, in order to maximally utilize the cluster. -Using the [Fairseq library](https://github.com/pytorch/fairseq/tree/master/examples/roberta), the model trained for two epochs, which equals over 16k batches in total, which took about three days on the computing cluster. -In between training jobs on the computing cluster, 2 Nvidia 1080 Ti's also covered some parameter updates for RobBERT v2. - - -## Limitations and bias - -In the [RobBERT paper](https://arxiv.org/abs/2001.06286), we also investigated potential sources of bias in RobBERT. - -We found that the zeroshot model estimates the probability of *hij* (he) to be higher than *zij* (she) for most occupations in bleached template sentences, regardless of their actual job gender ratio in reality. - -

- RobBERT's performance on smaller datasets -

- -By augmenting the DBRB Dutch Book sentiment analysis dataset with the stated gender of the author of the review, we found that highly positive reviews written by women were generally more accurately detected by RobBERT as being positive than those written by men. - -

- RobBERT's performance on smaller datasets -

- - - -## BibTeX entry and citation info - -```bibtex -@misc{delobelle2020robbert, - title={RobBERT: a Dutch RoBERTa-based Language Model}, - author={Pieter Delobelle and Thomas Winters and Bettina Berendt}, - year={2020}, - eprint={2001.06286}, - archivePrefix={arXiv}, - primaryClass={cs.CL} -} -``` diff --git a/model_cards/pedropei/question-intimacy/README.md b/model_cards/pedropei/question-intimacy/README.md deleted file mode 100644 index 27b03bf66f..0000000000 --- a/model_cards/pedropei/question-intimacy/README.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -language: -- en -inference: false ---- diff --git a/model_cards/pierreguillou/gpt2-small-portuguese/README.md b/model_cards/pierreguillou/gpt2-small-portuguese/README.md deleted file mode 100644 index 010773eb0d..0000000000 --- a/model_cards/pierreguillou/gpt2-small-portuguese/README.md +++ /dev/null @@ -1,186 +0,0 @@ ---- -language: pt - -widget: -- text: "Quem era Jim Henson? Jim Henson era um" -- text: "Em um achado chocante, o cientista descobriu um" -- text: "Barack Hussein Obama II, nascido em 4 de agosto de 1961, é" -- text: "Corrida por vacina contra Covid-19 já tem" -license: mit -datasets: -- wikipedia ---- - -# GPorTuguese-2: a Language Model for Portuguese text generation (and more NLP tasks...) - -## Introduction - -GPorTuguese-2 (Portuguese GPT-2 small) is a state-of-the-art language model for Portuguese based on the GPT-2 small model. - -It was trained on Portuguese Wikipedia using **Transfer Learning and Fine-tuning techniques** in just over a day, on one GPU NVIDIA V100 32GB and with a little more than 1GB of training data. - -It is a proof-of-concept that it is possible to get a state-of-the-art language model in any language with low ressources. - -It was fine-tuned from the [English pre-trained GPT-2 small](https://huggingface.co/gpt2) using the Hugging Face libraries (Transformers and Tokenizers) wrapped into the [fastai v2](https://dev.fast.ai/) Deep Learning framework. All the fine-tuning fastai v2 techniques were used. - -It is now available on Hugging Face. For further information or requests, please go to "[Faster than training from scratch — Fine-tuning the English GPT-2 in any language with Hugging Face and fastai v2 (practical case with Portuguese)](https://medium.com/@pierre_guillou/faster-than-training-from-scratch-fine-tuning-the-english-gpt-2-in-any-language-with-hugging-f2ec05c98787)". - -## Model - -| Model | #params | Model file (pt/tf) | Arch. | Training /Validation data (text) | -|-------------------------|---------|--------------------|-------------|------------------------------------------| -| `gpt2-small-portuguese` | 124M | 487M / 475M | GPT-2 small | Portuguese Wikipedia (1.28 GB / 0.32 GB) | - -## Evaluation results -In a little more than a day (we only used one GPU NVIDIA V100 32GB; through a Distributed Data Parallel (DDP) training mode, we could have divided by three this time to 10 hours, just with 2 GPUs), we got a loss of 3.17, an **accuracy of 37.99%** and a **perplexity of 23.76** (see the validation results table below). - -| after ... epochs | loss | accuracy (%) | perplexity | time by epoch | cumulative time | -|------------------|------|--------------|------------|---------------|-----------------| -| 0 | 9.95 | 9.90 | 20950.94 | 00:00:00 | 00:00:00 | -| 1 | 3.64 | 32.52 | 38.12 | 5:48:31 | 5:48:31 | -| 2 | 3.30 | 36.29 | 27.16 | 5:38:18 | 11:26:49 | -| 3 | 3.21 | 37.46 | 24.71 | 6:20:51 | 17:47:40 | -| 4 | 3.19 | 37.74 | 24.21 | 6:06:29 | 23:54:09 | -| 5 | 3.17 | 37.99 | 23.76 | 6:16:22 | 30:10:31 | - -## GPT-2 - -*Note: information copied/pasted from [Model: gpt2 >> GPT-2](https://huggingface.co/gpt2#gpt-2)* - -Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in this [paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) and first released at this [page](https://openai.com/blog/better-language-models/) (February 14, 2019). - -Disclaimer: The team releasing GPT-2 also wrote a [model card](https://github.com/openai/gpt-2/blob/master/model_card.md) for their model. Content from this model card has been written by the Hugging Face team to complete the information they provided and give specific examples of bias. - -## Model description - -*Note: information copied/pasted from [Model: gpt2 >> Model description](https://huggingface.co/gpt2#model-description)* - -GPT-2 is a transformers model pretrained on a very large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was trained to guess the next word in sentences. - -More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence, shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens. - -This way, the model learns an inner representation of the English language that can then be used to extract features useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a prompt. - -## How to use GPorTuguese-2 with HuggingFace (PyTorch) - -The following code use PyTorch. To use TensorFlow, check the below corresponding paragraph. - -### Load GPorTuguese-2 and its sub-word tokenizer (Byte-level BPE) - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead -import torch - -tokenizer = AutoTokenizer.from_pretrained("pierreguillou/gpt2-small-portuguese") -model = AutoModelWithLMHead.from_pretrained("pierreguillou/gpt2-small-portuguese") - -# Get sequence length max of 1024 -tokenizer.model_max_length=1024 - -model.eval() # disable dropout (or leave in train mode to finetune) -``` - -### Generate one word - -```python -# input sequence -text = "Quem era Jim Henson? Jim Henson era um" -inputs = tokenizer(text, return_tensors="pt") - -# model output -outputs = model(**inputs, labels=inputs["input_ids"]) -loss, logits = outputs[:2] -predicted_index = torch.argmax(logits[0, -1, :]).item() -predicted_text = tokenizer.decode([predicted_index]) - -# results -print('input text:', text) -print('predicted text:', predicted_text) - -# input text: Quem era Jim Henson? Jim Henson era um -# predicted text: homem -``` - -### Generate one full sequence - -```python -# input sequence -text = "Quem era Jim Henson? Jim Henson era um" -inputs = tokenizer(text, return_tensors="pt") - -# model output using Top-k sampling text generation method -sample_outputs = model.generate(inputs.input_ids, - pad_token_id=50256, - do_sample=True, - max_length=50, # put the token number you want - top_k=40, - num_return_sequences=1) - -# generated sequence -for i, sample_output in enumerate(sample_outputs): - print(">> Generated text {}\n\n{}".format(i+1, tokenizer.decode(sample_output.tolist()))) - -# >> Generated text -# Quem era Jim Henson? Jim Henson era um executivo de televisão e diretor de um grande estúdio de cinema mudo chamado Selig, -# depois que o diretor de cinema mudo Georges Seuray dirigiu vários filmes para a Columbia e o estúdio. -``` - -## How to use GPorTuguese-2 with HuggingFace (TensorFlow) - -The following code use TensorFlow. To use PyTorch, check the above corresponding paragraph. - -### Load GPorTuguese-2 and its sub-word tokenizer (Byte-level BPE) - -```python -from transformers import AutoTokenizer, TFAutoModelWithLMHead -import tensorflow as tf - -tokenizer = AutoTokenizer.from_pretrained("pierreguillou/gpt2-small-portuguese") -model = TFAutoModelWithLMHead.from_pretrained("pierreguillou/gpt2-small-portuguese") - -# Get sequence length max of 1024 -tokenizer.model_max_length=1024 - -model.eval() # disable dropout (or leave in train mode to finetune) -``` - -### Generate one full sequence - -```python -# input sequence -text = "Quem era Jim Henson? Jim Henson era um" -inputs = tokenizer.encode(text, return_tensors="tf") - -# model output using Top-k sampling text generation method -outputs = model.generate(inputs, eos_token_id=50256, pad_token_id=50256, - do_sample=True, - max_length=40, - top_k=40) -print(tokenizer.decode(outputs[0])) - -# >> Generated text -# Quem era Jim Henson? Jim Henson era um amigo familiar da família. Ele foi contratado pelo seu pai -# para trabalhar como aprendiz no escritório de um escritório de impressão, e então começou a ganhar dinheiro - -``` - -## Limitations and bias - -The training data used for this model come from Portuguese Wikipedia. We know it contains a lot of unfiltered content from the internet, which is far from neutral. As the openAI team themselves point out in their model card: - -> Because large-scale language models like GPT-2 do not distinguish fact from fiction, we don’t support use-cases that require the generated text to be true. Additionally, language models like GPT-2 reflect the biases inherent to the systems they were trained on, so we do not recommend that they be deployed into systems that interact with humans > unless the deployers first carry out a study of biases relevant to the intended use-case. We found no statistically significant difference in gender, race, and religious bias probes between 774M and 1.5B, implying all versions of GPT-2 should be approached with similar levels of caution around use cases that are sensitive to biases around human attributes. - -## Author - -Portuguese GPT-2 small was trained and evaluated by [Pierre GUILLOU](https://www.linkedin.com/in/pierreguillou/) thanks to the computing power of the GPU (GPU NVIDIA V100 32 Go) of the [AI Lab](https://www.linkedin.com/company/ailab-unb/) (University of Brasilia) to which I am attached as an Associate Researcher in NLP and the participation of its directors in the definition of NLP strategy, Professors Fabricio Ataides Braz and Nilton Correia da Silva. - -## Citation -If you use our work, please cite: - -```bibtex -@inproceedings{pierre2020gpt2smallportuguese, - title={GPorTuguese-2 (Portuguese GPT-2 small): a Language Model for Portuguese text generation (and more NLP tasks...)}, - author={Pierre Guillou}, - year={2020} -} -``` diff --git a/model_cards/pradhyra/AWSBlogBert/README.md b/model_cards/pradhyra/AWSBlogBert/README.md deleted file mode 100644 index b182a398be..0000000000 --- a/model_cards/pradhyra/AWSBlogBert/README.md +++ /dev/null @@ -1,8 +0,0 @@ -This model is pre-trained on blog articles from AWS Blogs. - -## Pre-training corpora -The input text contains around 3000 blog articles on [AWS Blogs website](https://aws.amazon.com/blogs/) technical subject matter including AWS products, tools and tutorials. - -## Pre-training details -I picked a Roberta architecture for masked language modeling (6-layer, 768-hidden, 12-heads, 82M parameters) and its corresponding ByteLevelBPE tokenization strategy. I then followed HuggingFace's Transformers [blog post](https://huggingface.co/blog/how-to-train) to train the model. -I chose to follow the following training set-up: 28k training steps with batches of 64 sequences of length 512 with an initial learning rate 5e-5. The model acheived a training loss of 3.6 on the MLM task over 10 epochs. diff --git a/model_cards/pranavpsv/gpt2-genre-story-generator/README.md b/model_cards/pranavpsv/gpt2-genre-story-generator/README.md deleted file mode 100644 index 5900081deb..0000000000 --- a/model_cards/pranavpsv/gpt2-genre-story-generator/README.md +++ /dev/null @@ -1,32 +0,0 @@ - -# GPT2 Genre Based Story Generator - -## Model description - -GPT2 fine-tuned on genre-based story generation. - -## Intended uses - -Used to generate stories based on user inputted genre and starting prompts. - -## How to use - -#### Supported Genres -superhero, action, drama, horror, thriller, sci_fi -#### Input text format -\ \ Some optional text... - -**Example**: \ \ After discovering time travel, - -```python -# Example of usage -from transformers import pipeline - -story_gen = pipeline("text-generation", "pranavpsv/gpt2-genre-story-generator") -print(story_gen(" Batman")) - -``` - -## Training data - -Initialized with pre-trained weights of "gpt2" checkpoint. Fine-tuned the model on stories of various genres. diff --git a/model_cards/pvl/labse_bert/README.md b/model_cards/pvl/labse_bert/README.md deleted file mode 100644 index 56898861f3..0000000000 --- a/model_cards/pvl/labse_bert/README.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -language: en -thumbnail: -tags: -- bert -- embeddings -license: Apache-2.0 ---- - -# LABSE BERT - -## Model description - -Model for "Language-agnostic BERT Sentence Embedding" paper from Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, Wei Wang. Model available in [TensorFlow Hub](https://tfhub.dev/google/LaBSE/1). - -## Intended uses & limitations - -#### How to use - -```python -from transformers import AutoTokenizer, AutoModel -import torch - -# from sentence-transformers -def mean_pooling(model_output, attention_mask): - token_embeddings = model_output[0] #First element of model_output contains all token embeddings - input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float() - sum_embeddings = torch.sum(token_embeddings * input_mask_expanded, 1) - sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9) - return sum_embeddings / sum_mask - -tokenizer = AutoTokenizer.from_pretrained("pvl/labse_bert", do_lower_case=False) -model = AutoModel.from_pretrained("pvl/labse_bert") - -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] - -encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt') - -with torch.no_grad(): - model_output = model(**encoded_input) - -sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask']) - - -``` diff --git a/model_cards/qarib/bert-base-qarib60_1790k/README.md b/model_cards/qarib/bert-base-qarib60_1790k/README.md deleted file mode 100644 index ae68b78464..0000000000 --- a/model_cards/qarib/bert-base-qarib60_1790k/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: ar -tags: -- qarib - -license: apache-2.0 -datasets: -- Arabic GigaWord -- Abulkhair Arabic Corpus -- opus -- Twitter data ---- - -# QARiB: QCRI Arabic and Dialectal BERT - -## About QARiB -QCRI Arabic and Dialectal BERT (QARiB) model, was trained on a collection of ~ 420 Million tweets and ~ 180 Million sentences of text. -For Tweets, the data was collected using twitter API and using language filter. `lang:ar`. For Text data, it was a combination from -[Arabic GigaWord](url), [Abulkhair Arabic Corpus]() and [OPUS](http://opus.nlpl.eu/). - -### bert-base-qarib60_1790k -- Data size: 60Gb -- Number of Iterations: 1790k -- Loss: 1.8764963 - -## Training QARiB -The training of the model has been performed using Google’s original Tensorflow code on Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. -See more details in [Training QARiB](../Training_QARiB.md) - -## Using QARiB - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you. For more details, see [Using QARiB](../Using_QARiB.md) - -### How to use -You can use this model directly with a pipeline for masked language modeling: - -```python ->>>from transformers import pipeline ->>>fill_mask = pipeline("fill-mask", model="./models/data60gb_86k") - ->>> fill_mask("شو عندكم يا [MASK]") -[{'sequence': '[CLS] شو عندكم يا عرب [SEP]', 'score': 0.0990147516131401, 'token': 2355, 'token_str': 'عرب'}, -{'sequence': '[CLS] شو عندكم يا جماعة [SEP]', 'score': 0.051633741706609726, 'token': 2308, 'token_str': 'جماعة'}, -{'sequence': '[CLS] شو عندكم يا شباب [SEP]', 'score': 0.046871256083250046, 'token': 939, 'token_str': 'شباب'}, -{'sequence': '[CLS] شو عندكم يا رفاق [SEP]', 'score': 0.03598872944712639, 'token': 7664, 'token_str': 'رفاق'}, -{'sequence': '[CLS] شو عندكم يا ناس [SEP]', 'score': 0.031996358186006546, 'token': 271, 'token_str': 'ناس'}] - ->>> fill_mask("قللي وشفيييك يرحم [MASK]") -[{'sequence': '[CLS] قللي وشفيييك يرحم والديك [SEP]', 'score': 0.4152909517288208, 'token': 9650, 'token_str': 'والديك'}, -{'sequence': '[CLS] قللي وشفيييك يرحملي [SEP]', 'score': 0.07663793861865997, 'token': 294, 'token_str': '##لي'}, -{'sequence': '[CLS] قللي وشفيييك يرحم حالك [SEP]', 'score': 0.0453166700899601, 'token': 2663, 'token_str': 'حالك'}, -{'sequence': '[CLS] قللي وشفيييك يرحم امك [SEP]', 'score': 0.04390475153923035, 'token': 1942, 'token_str': 'امك'}, -{'sequence': '[CLS] قللي وشفيييك يرحمونك [SEP]', 'score': 0.027349254116415977, 'token': 3283, 'token_str': '##ونك'}] - ->>> fill_mask("وقام المدير [MASK]") -[ -{'sequence': '[CLS] وقام المدير بالعمل [SEP]', 'score': 0.0678194984793663, 'token': 4230, 'token_str': 'بالعمل'}, -{'sequence': '[CLS] وقام المدير بذلك [SEP]', 'score': 0.05191086605191231, 'token': 984, 'token_str': 'بذلك'}, -{'sequence': '[CLS] وقام المدير بالاتصال [SEP]', 'score': 0.045264165848493576, 'token': 26096, 'token_str': 'بالاتصال'}, -{'sequence': '[CLS] وقام المدير بعمله [SEP]', 'score': 0.03732728958129883, 'token': 40486, 'token_str': 'بعمله'}, -{'sequence': '[CLS] وقام المدير بالامر [SEP]', 'score': 0.0246378555893898, 'token': 29124, 'token_str': 'بالامر'} -] ->>> fill_mask("وقامت المديرة [MASK]") - -[{'sequence': '[CLS] وقامت المديرة بذلك [SEP]', 'score': 0.23992691934108734, 'token': 984, 'token_str': 'بذلك'}, -{'sequence': '[CLS] وقامت المديرة بالامر [SEP]', 'score': 0.108805812895298, 'token': 29124, 'token_str': 'بالامر'}, -{'sequence': '[CLS] وقامت المديرة بالعمل [SEP]', 'score': 0.06639821827411652, 'token': 4230, 'token_str': 'بالعمل'}, -{'sequence': '[CLS] وقامت المديرة بالاتصال [SEP]', 'score': 0.05613093823194504, 'token': 26096, 'token_str': 'بالاتصال'}, -{'sequence': '[CLS] وقامت المديرة المديرة [SEP]', 'score': 0.021778125315904617, 'token': 41635, 'token_str': 'المديرة'}] -``` -## Training procedure - -The training of the model has been performed using Google’s original Tensorflow code on eight core Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. - -## Eval results - -We evaluated QARiB models on five NLP downstream task: -- Sentiment Analysis -- Emotion Detection -- Named-Entity Recognition (NER) -- Offensive Language Detection -- Dialect Identification - -The results obtained from QARiB models outperforms multilingual BERT/AraBERT/ArabicBERT. - - -## Model Weights and Vocab Download -TBD - -## Contacts - -Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish and Younes Samih - - diff --git a/model_cards/qarib/bert-base-qarib60_1970k/README.md b/model_cards/qarib/bert-base-qarib60_1970k/README.md deleted file mode 100644 index 44c0328896..0000000000 --- a/model_cards/qarib/bert-base-qarib60_1970k/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: ar -tags: -- qarib - -license: apache-2.0 -datasets: -- Arabic GigaWord -- Abulkhair Arabic Corpus -- opus -- Twitter data ---- - -# QARiB: QCRI Arabic and Dialectal BERT - -## About QARiB -QCRI Arabic and Dialectal BERT (QARiB) model, was trained on a collection of ~ 420 Million tweets and ~ 180 Million sentences of text. -For Tweets, the data was collected using twitter API and using language filter. `lang:ar`. For Text data, it was a combination from -[Arabic GigaWord](url), [Abulkhair Arabic Corpus]() and [OPUS](http://opus.nlpl.eu/). - -### bert-base-qarib60_1970k -- Data size: 60Gb -- Number of Iterations: 1970k -- Loss: 1.5708898 - -## Training QARiB -The training of the model has been performed using Google’s original Tensorflow code on Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. -See more details in [Training QARiB](../Training_QARiB.md) - -## Using QARiB - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you. For more details, see [Using QARiB](../Using_QARiB.md) - -### How to use -You can use this model directly with a pipeline for masked language modeling: - -```python ->>>from transformers import pipeline ->>>fill_mask = pipeline("fill-mask", model="./models/data60gb_86k") - ->>> fill_mask("شو عندكم يا [MASK]") -[{'sequence': '[CLS] شو عندكم يا عرب [SEP]', 'score': 0.0990147516131401, 'token': 2355, 'token_str': 'عرب'}, -{'sequence': '[CLS] شو عندكم يا جماعة [SEP]', 'score': 0.051633741706609726, 'token': 2308, 'token_str': 'جماعة'}, -{'sequence': '[CLS] شو عندكم يا شباب [SEP]', 'score': 0.046871256083250046, 'token': 939, 'token_str': 'شباب'}, -{'sequence': '[CLS] شو عندكم يا رفاق [SEP]', 'score': 0.03598872944712639, 'token': 7664, 'token_str': 'رفاق'}, -{'sequence': '[CLS] شو عندكم يا ناس [SEP]', 'score': 0.031996358186006546, 'token': 271, 'token_str': 'ناس'}] - ->>> fill_mask("قللي وشفيييك يرحم [MASK]") -[{'sequence': '[CLS] قللي وشفيييك يرحم والديك [SEP]', 'score': 0.4152909517288208, 'token': 9650, 'token_str': 'والديك'}, -{'sequence': '[CLS] قللي وشفيييك يرحملي [SEP]', 'score': 0.07663793861865997, 'token': 294, 'token_str': '##لي'}, -{'sequence': '[CLS] قللي وشفيييك يرحم حالك [SEP]', 'score': 0.0453166700899601, 'token': 2663, 'token_str': 'حالك'}, -{'sequence': '[CLS] قللي وشفيييك يرحم امك [SEP]', 'score': 0.04390475153923035, 'token': 1942, 'token_str': 'امك'}, -{'sequence': '[CLS] قللي وشفيييك يرحمونك [SEP]', 'score': 0.027349254116415977, 'token': 3283, 'token_str': '##ونك'}] - ->>> fill_mask("وقام المدير [MASK]") -[ -{'sequence': '[CLS] وقام المدير بالعمل [SEP]', 'score': 0.0678194984793663, 'token': 4230, 'token_str': 'بالعمل'}, -{'sequence': '[CLS] وقام المدير بذلك [SEP]', 'score': 0.05191086605191231, 'token': 984, 'token_str': 'بذلك'}, -{'sequence': '[CLS] وقام المدير بالاتصال [SEP]', 'score': 0.045264165848493576, 'token': 26096, 'token_str': 'بالاتصال'}, -{'sequence': '[CLS] وقام المدير بعمله [SEP]', 'score': 0.03732728958129883, 'token': 40486, 'token_str': 'بعمله'}, -{'sequence': '[CLS] وقام المدير بالامر [SEP]', 'score': 0.0246378555893898, 'token': 29124, 'token_str': 'بالامر'} -] ->>> fill_mask("وقامت المديرة [MASK]") - -[{'sequence': '[CLS] وقامت المديرة بذلك [SEP]', 'score': 0.23992691934108734, 'token': 984, 'token_str': 'بذلك'}, -{'sequence': '[CLS] وقامت المديرة بالامر [SEP]', 'score': 0.108805812895298, 'token': 29124, 'token_str': 'بالامر'}, -{'sequence': '[CLS] وقامت المديرة بالعمل [SEP]', 'score': 0.06639821827411652, 'token': 4230, 'token_str': 'بالعمل'}, -{'sequence': '[CLS] وقامت المديرة بالاتصال [SEP]', 'score': 0.05613093823194504, 'token': 26096, 'token_str': 'بالاتصال'}, -{'sequence': '[CLS] وقامت المديرة المديرة [SEP]', 'score': 0.021778125315904617, 'token': 41635, 'token_str': 'المديرة'}] -``` -## Training procedure - -The training of the model has been performed using Google’s original Tensorflow code on eight core Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. - -## Eval results - -We evaluated QARiB models on five NLP downstream task: -- Sentiment Analysis -- Emotion Detection -- Named-Entity Recognition (NER) -- Offensive Language Detection -- Dialect Identification - -The results obtained from QARiB models outperforms multilingual BERT/AraBERT/ArabicBERT. - - -## Model Weights and Vocab Download -TBD - -## Contacts - -Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish and Younes Samih - - diff --git a/model_cards/qarib/bert-base-qarib60_860k/README.md b/model_cards/qarib/bert-base-qarib60_860k/README.md deleted file mode 100644 index 4924489b69..0000000000 --- a/model_cards/qarib/bert-base-qarib60_860k/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: ar -tags: -- qarib - -license: apache-2.0 -datasets: -- Arabic GigaWord -- Abulkhair Arabic Corpus -- opus -- Twitter data ---- - -# QARiB: QCRI Arabic and Dialectal BERT - -## About QARiB -QCRI Arabic and Dialectal BERT (QARiB) model, was trained on a collection of ~ 420 Million tweets and ~ 180 Million sentences of text. -For Tweets, the data was collected using twitter API and using language filter. `lang:ar`. For Text data, it was a combination from -[Arabic GigaWord](url), [Abulkhair Arabic Corpus]() and [OPUS](http://opus.nlpl.eu/). - -### bert-base-qarib60_860k -- Data size: 60Gb -- Number of Iterations: 860k -- Loss: 2.2454472 - -## Training QARiB -The training of the model has been performed using Google’s original Tensorflow code on Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. -See more details in [Training QARiB](../Training_QARiB.md) - -## Using QARiB - -You can use the raw model for either masked language modeling or next sentence prediction, but it's mostly intended to be fine-tuned on a downstream task. See the model hub to look for fine-tuned versions on a task that interests you. For more details, see [Using QARiB](../Using_QARiB.md) - -### How to use -You can use this model directly with a pipeline for masked language modeling: - -```python ->>>from transformers import pipeline ->>>fill_mask = pipeline("fill-mask", model="./models/data60gb_86k") - ->>> fill_mask("شو عندكم يا [MASK]") -[{'sequence': '[CLS] شو عندكم يا عرب [SEP]', 'score': 0.0990147516131401, 'token': 2355, 'token_str': 'عرب'}, -{'sequence': '[CLS] شو عندكم يا جماعة [SEP]', 'score': 0.051633741706609726, 'token': 2308, 'token_str': 'جماعة'}, -{'sequence': '[CLS] شو عندكم يا شباب [SEP]', 'score': 0.046871256083250046, 'token': 939, 'token_str': 'شباب'}, -{'sequence': '[CLS] شو عندكم يا رفاق [SEP]', 'score': 0.03598872944712639, 'token': 7664, 'token_str': 'رفاق'}, -{'sequence': '[CLS] شو عندكم يا ناس [SEP]', 'score': 0.031996358186006546, 'token': 271, 'token_str': 'ناس'}] - ->>> fill_mask("قللي وشفيييك يرحم [MASK]") -[{'sequence': '[CLS] قللي وشفيييك يرحم والديك [SEP]', 'score': 0.4152909517288208, 'token': 9650, 'token_str': 'والديك'}, -{'sequence': '[CLS] قللي وشفيييك يرحملي [SEP]', 'score': 0.07663793861865997, 'token': 294, 'token_str': '##لي'}, -{'sequence': '[CLS] قللي وشفيييك يرحم حالك [SEP]', 'score': 0.0453166700899601, 'token': 2663, 'token_str': 'حالك'}, -{'sequence': '[CLS] قللي وشفيييك يرحم امك [SEP]', 'score': 0.04390475153923035, 'token': 1942, 'token_str': 'امك'}, -{'sequence': '[CLS] قللي وشفيييك يرحمونك [SEP]', 'score': 0.027349254116415977, 'token': 3283, 'token_str': '##ونك'}] - ->>> fill_mask("وقام المدير [MASK]") -[ -{'sequence': '[CLS] وقام المدير بالعمل [SEP]', 'score': 0.0678194984793663, 'token': 4230, 'token_str': 'بالعمل'}, -{'sequence': '[CLS] وقام المدير بذلك [SEP]', 'score': 0.05191086605191231, 'token': 984, 'token_str': 'بذلك'}, -{'sequence': '[CLS] وقام المدير بالاتصال [SEP]', 'score': 0.045264165848493576, 'token': 26096, 'token_str': 'بالاتصال'}, -{'sequence': '[CLS] وقام المدير بعمله [SEP]', 'score': 0.03732728958129883, 'token': 40486, 'token_str': 'بعمله'}, -{'sequence': '[CLS] وقام المدير بالامر [SEP]', 'score': 0.0246378555893898, 'token': 29124, 'token_str': 'بالامر'} -] ->>> fill_mask("وقامت المديرة [MASK]") - -[{'sequence': '[CLS] وقامت المديرة بذلك [SEP]', 'score': 0.23992691934108734, 'token': 984, 'token_str': 'بذلك'}, -{'sequence': '[CLS] وقامت المديرة بالامر [SEP]', 'score': 0.108805812895298, 'token': 29124, 'token_str': 'بالامر'}, -{'sequence': '[CLS] وقامت المديرة بالعمل [SEP]', 'score': 0.06639821827411652, 'token': 4230, 'token_str': 'بالعمل'}, -{'sequence': '[CLS] وقامت المديرة بالاتصال [SEP]', 'score': 0.05613093823194504, 'token': 26096, 'token_str': 'بالاتصال'}, -{'sequence': '[CLS] وقامت المديرة المديرة [SEP]', 'score': 0.021778125315904617, 'token': 41635, 'token_str': 'المديرة'}] -``` -## Training procedure - -The training of the model has been performed using Google’s original Tensorflow code on eight core Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. - -## Eval results - -We evaluated QARiB models on five NLP downstream task: -- Sentiment Analysis -- Emotion Detection -- Named-Entity Recognition (NER) -- Offensive Language Detection -- Dialect Identification - -The results obtained from QARiB models outperforms multilingual BERT/AraBERT/ArabicBERT. - - -## Model Weights and Vocab Download -TBD - -## Contacts - -Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish and Younes Samih - - diff --git a/model_cards/ramsrigouthamg/t5_paraphraser/README.md b/model_cards/ramsrigouthamg/t5_paraphraser/README.md deleted file mode 100644 index 7816c0dd08..0000000000 --- a/model_cards/ramsrigouthamg/t5_paraphraser/README.md +++ /dev/null @@ -1,84 +0,0 @@ -## Model in Action 🚀 - -```python -import torch -from transformers import T5ForConditionalGeneration,T5Tokenizer - - -def set_seed(seed): - torch.manual_seed(seed) - if torch.cuda.is_available(): - torch.cuda.manual_seed_all(seed) - -set_seed(42) - -model = T5ForConditionalGeneration.from_pretrained('ramsrigouthamg/t5_paraphraser') -tokenizer = T5Tokenizer.from_pretrained('ramsrigouthamg/t5_paraphraser') - -device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -print ("device ",device) -model = model.to(device) - -sentence = "Which course should I take to get started in data science?" -# sentence = "What are the ingredients required to bake a perfect cake?" -# sentence = "What is the best possible approach to learn aeronautical engineering?" -# sentence = "Do apples taste better than oranges in general?" - - -text = "paraphrase: " + sentence + " " - - -max_len = 256 - -encoding = tokenizer.encode_plus(text,pad_to_max_length=True, return_tensors="pt") -input_ids, attention_masks = encoding["input_ids"].to(device), encoding["attention_mask"].to(device) - - -# set top_k = 50 and set top_p = 0.95 and num_return_sequences = 3 -beam_outputs = model.generate( - input_ids=input_ids, attention_mask=attention_masks, - do_sample=True, - max_length=256, - top_k=120, - top_p=0.98, - early_stopping=True, - num_return_sequences=10 -) - - -print ("\nOriginal Question ::") -print (sentence) -print ("\n") -print ("Paraphrased Questions :: ") -final_outputs =[] -for beam_output in beam_outputs: - sent = tokenizer.decode(beam_output, skip_special_tokens=True,clean_up_tokenization_spaces=True) - if sent.lower() != sentence.lower() and sent not in final_outputs: - final_outputs.append(sent) - -for i, final_output in enumerate(final_outputs): - print("{}: {}".format(i, final_output)) - -``` -## Output -``` -Original Question :: -Which course should I take to get started in data science? - - -Paraphrased Questions :: -0: What should I learn to become a data scientist? -1: How do I get started with data science? -2: How would you start a data science career? -3: How can I start learning data science? -4: How do you get started in data science? -5: What's the best course for data science? -6: Which course should I start with for data science? -7: What courses should I follow to get started in data science? -8: What degree should be taken by a data scientist? -9: Which course should I follow to become a Data Scientist? -``` - -## Detailed blog post available here : -https://towardsdatascience.com/paraphrase-any-question-with-t5-text-to-text-transfer-transformer-pretrained-model-and-cbb9e35f1555 - diff --git a/model_cards/rdenadai/BR_BERTo/README.md b/model_cards/rdenadai/BR_BERTo/README.md deleted file mode 100644 index 594df42bf5..0000000000 --- a/model_cards/rdenadai/BR_BERTo/README.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -language: pt -tags: -- portuguese -- brazil -- pt_BR -widget: -- text: gostei muito dessa ---- - -# BR_BERTo - -Portuguese (Brazil) model for text inference. - -## Params - -Trained on a corpus of 6_993_330 sentences. - -- Vocab size: 150_000 -- RobertaForMaskedLM size : 512 -- Num train epochs: 3 -- Time to train: ~10days (on GCP with a Nvidia T4) - -I follow the great tutorial from HuggingFace team: - -[How to train a new language model from scratch using Transformers and Tokenizers](https://huggingface.co/blog/how-to-train) - -More infor here: - -[BR_BERTo](https://github.com/rdenadai/BR-BERTo) diff --git a/model_cards/redewiedergabe/bert-base-historical-german-rw-cased/README.md b/model_cards/redewiedergabe/bert-base-historical-german-rw-cased/README.md deleted file mode 100644 index c85ac02b4f..0000000000 --- a/model_cards/redewiedergabe/bert-base-historical-german-rw-cased/README.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -language: de ---- - -# Model description -## Dataset -Trained on fictional and non-fictional German texts written between 1840 and 1920: -* Narrative texts from Digitale Bibliothek (https://textgrid.de/digitale-bibliothek) -* Fairy tales and sagas from Grimm Korpus (https://www1.ids-mannheim.de/kl/projekte/korpora/archiv/gri.html) -* Newspaper and magazine article from Mannheimer Korpus Historischer Zeitungen und Zeitschriften (https://repos.ids-mannheim.de/mkhz-beschreibung.html) -* Magazine article from the journal „Die Grenzboten“ (http://www.deutschestextarchiv.de/doku/textquellen#grenzboten) -* Fictional and non-fictional texts from Projekt Gutenberg (https://www.projekt-gutenberg.org) - -## Hardware used -1 Tesla P4 GPU - -## Hyperparameters - -| Parameter | Value | -|-------------------------------|----------| -| Epochs | 3 | -| Gradient_accumulation_steps | 1 | -| Train_batch_size | 32 | -| Learning_rate | 0.00003 | -| Max_seq_len | 128 | - -## Evaluation results: Automatic tagging of four forms of speech/thought/writing representation in historical fictional and non-fictional German texts - -The language model was used in the task to tag direct, indirect, reported and free indirect speech/thought/writing representation in fictional and non-fictional German texts. The tagger is available and described in detail at https://github.com/redewiedergabe/tagger. - -The tagging model was trained using the SequenceTagger Class of the Flair framework ([Akbik et al., 2019](https://www.aclweb.org/anthology/N19-4010)) which implements a BiLSTM-CRF architecture on top of a language embedding (as proposed by [Huang et al. (2015)](https://arxiv.org/abs/1508.01991)). - - -Hyperparameters - -| Parameter | Value | -|-------------------------------|------------| -| Hidden_size | 256 | -| Learning_rate | 0.1 | -| Mini_batch_size | 8 | -| Max_epochs | 150 | - -Results are reported below in comparison to a custom trained flair embedding, which was stacked onto a custom trained fastText-model. Both models were trained on the same dataset. - -| | BERT ||| FastText+Flair |||Test data| -|----------------|----------|-----------|----------|------|-----------|--------|--------| -| | F1 | Precision | Recall | F1 | Precision | Recall || -| Direct | 0.80 | 0.86 | 0.74 | 0.84 | 0.90 | 0.79 |historical German, fictional & non-fictional| -| Indirect | **0.76** | **0.79** | **0.73** | 0.73 | 0.78 | 0.68 |historical German, fictional & non-fictional| -| Reported | **0.58** | **0.69** | **0.51** | 0.56 | 0.68 | 0.48 |historical German, fictional & non-fictional| -| Free indirect | **0.57** | **0.80** | **0.44** | 0.47 | 0.78 | 0.34 |modern German, fictional| - -## Intended use: -Historical German Texts (1840 to 1920) - -(Showed good performance with modern German fictional texts as well) - diff --git a/model_cards/rjbownes/Magic-The-Generating/README.md b/model_cards/rjbownes/Magic-The-Generating/README.md deleted file mode 100644 index c20f94715d..0000000000 --- a/model_cards/rjbownes/Magic-The-Generating/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -widget: -- text: "Even the Dwarves" -- text: "The secrets of" ---- - -# Model name -Magic The Generating - -## Model description - -This is a fine tuned GPT-2 model trained on a corpus of all available English language Magic the Gathering card flavour texts. - -## Intended uses & limitations - -This is intended only for use in generating new, novel, and sometimes surprising, MtG like flavour texts. - -#### How to use - -```python -from transformers import GPT2Tokenizer, GPT2LMHeadModel - -tokenizer = GPT2Tokenizer.from_pretrained("rjbownes/Magic-The-Generating") - -model = GPT2LMHeadModel.from_pretrained("rjbownes/Magic-The-Generating") - -``` - -#### Limitations and bias - -The training corpus was surprisingly small, only ~29000 cards, I had suspected there were more. This might mean there is a real limit to the number of entirely original strings this will generate. -This is also only based on the 117M parameter GPT2, it's a pretty obvious upgrade to retrain with medium, large or XL models. However, despite this, the outputs I tested were very convincing! - -## Training data - -The data was 29222 MtG card flavour texts. The model was based on the "gpt2" pretrained transformer: https://huggingface.co/gpt2. - -## Training procedure - -Only English language MtG flavour texts were scraped from the [Scryfall](https://scryfall.com/) API. Empty strings and any non-UTF-8 encoded tokens were removed leaving 29222 entries. -This was trained using google Colab with a T4 instance. 4 epochs, adamW optimizer with default parameters and a batch size of 32. Token embedding lengths were capped at 98 tokens as this was the longest string and an attention mask was added to the training model to ignore all padding tokens. - -## Eval results - -Average Training Loss: 0.44866578806635815. -Validation loss: 0.5606984243444775. - -Sample model outputs: - -1. "Every branch a crossroads, every vine a swift steed." - —Gwendlyn Di Corci - -2. "The secrets of this world will tell their masters where to strike if need be." - —Noyan Dar, Tazeem roilmage - -3. "The secrets of nature are expensive. You'd be better off just to have more freedom." - -4. "Even the Dwarves knew to leave some stones unturned." - -5. "The wise always keep an ear open to the whispers of power." - -### BibTeX entry and citation info - -```bibtex -@article{BownesLM, - title={Fine Tuning GPT-2 for Magic the Gathering flavour text generation.}, - author={Richard J. Bownes}, - journal={Medium}, - year={2020} -} - -``` diff --git a/model_cards/roberta-base-README.md b/model_cards/roberta-base-README.md deleted file mode 100644 index 3a89cda006..0000000000 --- a/model_cards/roberta-base-README.md +++ /dev/null @@ -1,234 +0,0 @@ ---- -language: en -tags: -- exbert -license: mit -datasets: -- bookcorpus -- wikipedia ---- - -# RoBERTa base model - -Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://arxiv.org/abs/1907.11692) and first released in -[this repository](https://github.com/pytorch/fairseq/tree/master/examples/roberta). This model is case-sensitive: it -makes a difference between english and English. - -Disclaimer: The team releasing RoBERTa did not write a model card for this model so this model card has been written by -the Hugging Face team. - -## Model description - -RoBERTa is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means -it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, it was pretrained with the Masked language modeling (MLM) objective. Taking a sentence, the model -randomly masks 15% of the words in the input then run the entire masked sentence through the model and has to predict -the masked words. This is different from traditional recurrent neural networks (RNNs) that usually see the words one -after the other, or from autoregressive models like GPT which internally mask the future tokens. It allows the model to -learn a bidirectional representation of the sentence. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model for masked language modeling, but it's mostly intended to be fine-tuned on a downstream task. -See the [model hub](https://huggingface.co/models?filter=roberta) to look for fine-tuned versions on a task that -interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='roberta-base') ->>> unmasker("Hello I'm a model.") - -[{'sequence': "Hello I'm a male model.", - 'score': 0.3306540250778198, - 'token': 2943, - 'token_str': 'Ġmale'}, - {'sequence': "Hello I'm a female model.", - 'score': 0.04655390977859497, - 'token': 2182, - 'token_str': 'Ġfemale'}, - {'sequence': "Hello I'm a professional model.", - 'score': 0.04232972860336304, - 'token': 2038, - 'token_str': 'Ġprofessional'}, - {'sequence': "Hello I'm a fashion model.", - 'score': 0.037216778844594955, - 'token': 2734, - 'token_str': 'Ġfashion'}, - {'sequence': "Hello I'm a Russian model.", - 'score': 0.03253649175167084, - 'token': 1083, - 'token_str': 'ĠRussian'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import RobertaTokenizer, RobertaModel -tokenizer = RobertaTokenizer.from_pretrained('roberta-base') -model = RobertaModel.from_pretrained('roberta-base') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import RobertaTokenizer, TFRobertaModel -tokenizer = RobertaTokenizer.from_pretrained('roberta-base') -model = TFRobertaModel.from_pretrained('roberta-base') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -The training data used for this model contains a lot of unfiltered content from the internet, which is far from -neutral. Therefore, the model can have biased predictions: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='roberta-base') ->>> unmasker("The man worked as a .") - -[{'sequence': 'The man worked as a mechanic.', - 'score': 0.08702439814805984, - 'token': 25682, - 'token_str': 'Ġmechanic'}, - {'sequence': 'The man worked as a waiter.', - 'score': 0.0819653645157814, - 'token': 38233, - 'token_str': 'Ġwaiter'}, - {'sequence': 'The man worked as a butcher.', - 'score': 0.073323555290699, - 'token': 32364, - 'token_str': 'Ġbutcher'}, - {'sequence': 'The man worked as a miner.', - 'score': 0.046322137117385864, - 'token': 18678, - 'token_str': 'Ġminer'}, - {'sequence': 'The man worked as a guard.', - 'score': 0.040150221437215805, - 'token': 2510, - 'token_str': 'Ġguard'}] - ->>> unmasker("The Black woman worked as a .") - -[{'sequence': 'The Black woman worked as a waitress.', - 'score': 0.22177888453006744, - 'token': 35698, - 'token_str': 'Ġwaitress'}, - {'sequence': 'The Black woman worked as a prostitute.', - 'score': 0.19288744032382965, - 'token': 36289, - 'token_str': 'Ġprostitute'}, - {'sequence': 'The Black woman worked as a maid.', - 'score': 0.06498628109693527, - 'token': 29754, - 'token_str': 'Ġmaid'}, - {'sequence': 'The Black woman worked as a secretary.', - 'score': 0.05375480651855469, - 'token': 2971, - 'token_str': 'Ġsecretary'}, - {'sequence': 'The Black woman worked as a nurse.', - 'score': 0.05245552211999893, - 'token': 9008, - 'token_str': 'Ġnurse'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -The RoBERTa model was pretrained on the reunion of five datasets: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books; -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers) ; -- [CC-News](https://commoncrawl.org/2016/10/news-dataset-available/), a dataset containing 63 millions English news - articles crawled between September 2016 and February 2019. -- [OpenWebText](https://github.com/jcpeterson/openwebtext), an opensource recreation of the WebText dataset used to - train GPT-2, -- [Stories](https://arxiv.org/abs/1806.02847) a dataset containing a subset of CommonCrawl data filtered to match the - story-like style of Winograd schemas. - -Together theses datasets weight 160GB of text. - -## Training procedure - -### Preprocessing - -The texts are tokenized using a byte version of Byte-Pair Encoding (BPE) and a vocabulary size of 50,000. The inputs of -the model take pieces of 512 contiguous token that may span over documents. The beginning of a new document is marked -with `` and the end of one by `` - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by ``. -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - -Contrary to BERT, the masking is done dynamically during pretraining (e.g., it changes at each epoch and is not fixed). - -### Pretraining - -The model was trained on 1024 V100 GPUs for 500K steps with a batch size of 8K and a sequence length of 512. The -optimizer used is Adam with a learning rate of 6e-4, \\(\beta_{1} = 0.9\\), \\(\beta_{2} = 0.98\\) and -\\(\epsilon = 1e-6\\), a weight decay of 0.01, learning rate warmup for 24,000 steps and linear decay of the learning -rate after. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | -|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:| -| | 87.6 | 91.9 | 92.8 | 94.8 | 63.6 | 91.2 | 90.2 | 78.7 | - - -### BibTeX entry and citation info - -```bibtex -@article{DBLP:journals/corr/abs-1907-11692, - author = {Yinhan Liu and - Myle Ott and - Naman Goyal and - Jingfei Du and - Mandar Joshi and - Danqi Chen and - Omer Levy and - Mike Lewis and - Luke Zettlemoyer and - Veselin Stoyanov}, - title = {RoBERTa: {A} Robustly Optimized {BERT} Pretraining Approach}, - journal = {CoRR}, - volume = {abs/1907.11692}, - year = {2019}, - url = {http://arxiv.org/abs/1907.11692}, - archivePrefix = {arXiv}, - eprint = {1907.11692}, - timestamp = {Thu, 01 Aug 2019 08:59:33 +0200}, - biburl = {https://dblp.org/rec/journals/corr/abs-1907-11692.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` - - - - diff --git a/model_cards/roberta-large-README.md b/model_cards/roberta-large-README.md deleted file mode 100644 index 788f152910..0000000000 --- a/model_cards/roberta-large-README.md +++ /dev/null @@ -1,235 +0,0 @@ ---- -language: en -tags: -- exbert -license: mit -datasets: -- bookcorpus -- wikipedia ---- - -# RoBERTa large model - -Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in -[this paper](https://arxiv.org/abs/1907.11692) and first released in -[this repository](https://github.com/pytorch/fairseq/tree/master/examples/roberta). This model is case-sensitive: it -makes a difference between english and English. - -Disclaimer: The team releasing RoBERTa did not write a model card for this model so this model card has been written by -the Hugging Face team. - -## Model description - -RoBERTa is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means -it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of -publicly available data) with an automatic process to generate inputs and labels from those texts. - -More precisely, it was pretrained with the Masked language modeling (MLM) objective. Taking a sentence, the model -randomly masks 15% of the words in the input then run the entire masked sentence through the model and has to predict -the masked words. This is different from traditional recurrent neural networks (RNNs) that usually see the words one -after the other, or from autoregressive models like GPT which internally mask the future tokens. It allows the model to -learn a bidirectional representation of the sentence. - -This way, the model learns an inner representation of the English language that can then be used to extract features -useful for downstream tasks: if you have a dataset of labeled sentences for instance, you can train a standard -classifier using the features produced by the BERT model as inputs. - -## Intended uses & limitations - -You can use the raw model for masked language modeling, but it's mostly intended to be fine-tuned on a downstream task. -See the [model hub](https://huggingface.co/models?filter=roberta) to look for fine-tuned versions on a task that -interests you. - -Note that this model is primarily aimed at being fine-tuned on tasks that use the whole sentence (potentially masked) -to make decisions, such as sequence classification, token classification or question answering. For tasks such as text -generation you should look at model like GPT2. - -### How to use - -You can use this model directly with a pipeline for masked language modeling: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='roberta-large') ->>> unmasker("Hello I'm a model.") - -[{'sequence': "Hello I'm a male model.", - 'score': 0.3317350447177887, - 'token': 2943, - 'token_str': 'Ġmale'}, - {'sequence': "Hello I'm a fashion model.", - 'score': 0.14171843230724335, - 'token': 2734, - 'token_str': 'Ġfashion'}, - {'sequence': "Hello I'm a professional model.", - 'score': 0.04291723668575287, - 'token': 2038, - 'token_str': 'Ġprofessional'}, - {'sequence': "Hello I'm a freelance model.", - 'score': 0.02134818211197853, - 'token': 18150, - 'token_str': 'Ġfreelance'}, - {'sequence': "Hello I'm a young model.", - 'score': 0.021098261699080467, - 'token': 664, - 'token_str': 'Ġyoung'}] -``` - -Here is how to use this model to get the features of a given text in PyTorch: - -```python -from transformers import RobertaTokenizer, RobertaModel -tokenizer = RobertaTokenizer.from_pretrained('roberta-large') -model = RobertaModel.from_pretrained('roberta-large') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: - -```python -from transformers import RobertaTokenizer, TFRobertaModel -tokenizer = RobertaTokenizer.from_pretrained('roberta-large') -model = TFRobertaModel.from_pretrained('roberta-large') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -### Limitations and bias - -The training data used for this model contains a lot of unfiltered content from the internet, which is far from -neutral. Therefore, the model can have biased predictions: - -```python ->>> from transformers import pipeline ->>> unmasker = pipeline('fill-mask', model='roberta-large') ->>> unmasker("The man worked as a .") - -[{'sequence': 'The man worked as a mechanic.', - 'score': 0.08260300755500793, - 'token': 25682, - 'token_str': 'Ġmechanic'}, - {'sequence': 'The man worked as a driver.', - 'score': 0.05736079439520836, - 'token': 1393, - 'token_str': 'Ġdriver'}, - {'sequence': 'The man worked as a teacher.', - 'score': 0.04709019884467125, - 'token': 3254, - 'token_str': 'Ġteacher'}, - {'sequence': 'The man worked as a bartender.', - 'score': 0.04641604796051979, - 'token': 33080, - 'token_str': 'Ġbartender'}, - {'sequence': 'The man worked as a waiter.', - 'score': 0.04239227622747421, - 'token': 38233, - 'token_str': 'Ġwaiter'}] - ->>> unmasker("The woman worked as a .") - -[{'sequence': 'The woman worked as a nurse.', - 'score': 0.2667474150657654, - 'token': 9008, - 'token_str': 'Ġnurse'}, - {'sequence': 'The woman worked as a waitress.', - 'score': 0.12280137836933136, - 'token': 35698, - 'token_str': 'Ġwaitress'}, - {'sequence': 'The woman worked as a teacher.', - 'score': 0.09747499972581863, - 'token': 3254, - 'token_str': 'Ġteacher'}, - {'sequence': 'The woman worked as a secretary.', - 'score': 0.05783602222800255, - 'token': 2971, - 'token_str': 'Ġsecretary'}, - {'sequence': 'The woman worked as a cleaner.', - 'score': 0.05576248839497566, - 'token': 16126, - 'token_str': 'Ġcleaner'}] -``` - -This bias will also affect all fine-tuned versions of this model. - -## Training data - -The RoBERTa model was pretrained on the reunion of five datasets: -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of 11,038 unpublished books; -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) (excluding lists, tables and headers) ; -- [CC-News](https://commoncrawl.org/2016/10/news-dataset-available/), a dataset containing 63 millions English news - articles crawled between September 2016 and February 2019. -- [OpenWebText](https://github.com/jcpeterson/openwebtext), an opensource recreation of the WebText dataset used to - train GPT-2, -- [Stories](https://arxiv.org/abs/1806.02847) a dataset containing a subset of CommonCrawl data filtered to match the - story-like style of Winograd schemas. - -Together theses datasets weight 160GB of text. - -## Training procedure - -### Preprocessing - -The texts are tokenized using a byte version of Byte-Pair Encoding (BPE) and a vocabulary size of 50,000. The inputs of -the model take pieces of 512 contiguous token that may span over documents. The beginning of a new document is marked -with `` and the end of one by `` - -The details of the masking procedure for each sentence are the following: -- 15% of the tokens are masked. -- In 80% of the cases, the masked tokens are replaced by ``. - -- In 10% of the cases, the masked tokens are replaced by a random token (different) from the one they replace. -- In the 10% remaining cases, the masked tokens are left as is. - -Contrary to BERT, the masking is done dynamically during pretraining (e.g., it changes at each epoch and is not fixed). - -### Pretraining - -The model was trained on 1024 V100 GPUs for 500K steps with a batch size of 8K and a sequence length of 512. The -optimizer used is Adam with a learning rate of 4e-4, \\(\beta_{1} = 0.9\\), \\(\beta_{2} = 0.98\\) and -\\(\epsilon = 1e-6\\), a weight decay of 0.01, learning rate warmup for 30,000 steps and linear decay of the learning -rate after. - -## Evaluation results - -When fine-tuned on downstream tasks, this model achieves the following results: - -Glue test results: - -| Task | MNLI | QQP | QNLI | SST-2 | CoLA | STS-B | MRPC | RTE | -|:----:|:----:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:| -| | 90.2 | 92.2 | 94.7 | 96.4 | 68.0 | 96.4 | 90.9 | 86.6 | - - -### BibTeX entry and citation info - -```bibtex -@article{DBLP:journals/corr/abs-1907-11692, - author = {Yinhan Liu and - Myle Ott and - Naman Goyal and - Jingfei Du and - Mandar Joshi and - Danqi Chen and - Omer Levy and - Mike Lewis and - Luke Zettlemoyer and - Veselin Stoyanov}, - title = {RoBERTa: {A} Robustly Optimized {BERT} Pretraining Approach}, - journal = {CoRR}, - volume = {abs/1907.11692}, - year = {2019}, - url = {http://arxiv.org/abs/1907.11692}, - archivePrefix = {arXiv}, - eprint = {1907.11692}, - timestamp = {Thu, 01 Aug 2019 08:59:33 +0200}, - biburl = {https://dblp.org/rec/journals/corr/abs-1907-11692.bib}, - bibsource = {dblp computer science bibliography, https://dblp.org} -} -``` - - - - diff --git a/model_cards/roberta-large-mnli-README.md b/model_cards/roberta-large-mnli-README.md deleted file mode 100644 index 5490327548..0000000000 --- a/model_cards/roberta-large-mnli-README.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -license: mit -widget: -- text: "I like you. I love you." ---- - - -## roberta-large-mnli - -Trained by Facebook, [original source](https://github.com/pytorch/fairseq/tree/master/examples/roberta) - -```bibtex -@article{liu2019roberta, - title = {RoBERTa: A Robustly Optimized BERT Pretraining Approach}, - author = {Yinhan Liu and Myle Ott and Naman Goyal and Jingfei Du and - Mandar Joshi and Danqi Chen and Omer Levy and Mike Lewis and - Luke Zettlemoyer and Veselin Stoyanov}, - journal={arXiv preprint arXiv:1907.11692}, - year = {2019}, -} -``` - diff --git a/model_cards/rohanrajpal/bert-base-codemixed-uncased-sentiment/README.md b/model_cards/rohanrajpal/bert-base-codemixed-uncased-sentiment/README.md deleted file mode 100644 index 1d25452a7d..0000000000 --- a/model_cards/rohanrajpal/bert-base-codemixed-uncased-sentiment/README.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -language: -- hi -- en -tags: -- hi -- en -- codemix -datasets: -- SAIL 2017 ---- - -# Model name - -## Model description - -I took a bert-base-multilingual-cased model from huggingface and finetuned it on SAIL 2017 dataset. - -## Intended uses & limitations - -#### How to use - -```python -# You can include sample code which will be formatted -#Coming soon! -``` - -#### Limitations and bias - -Provide examples of latent issues and potential remediations. - -## Training data - -I trained on the SAIL 2017 dataset [link](http://amitavadas.com/SAIL/Data/SAIL_2017.zip) on this [pretrained model](https://huggingface.co/bert-base-multilingual-cased). - - -## Training procedure - -No preprocessing. - -## Eval results - -### BibTeX entry and citation info - -```bibtex -@inproceedings{khanuja-etal-2020-gluecos, - title = "{GLUEC}o{S}: An Evaluation Benchmark for Code-Switched {NLP}", - author = "Khanuja, Simran and - Dandapat, Sandipan and - Srinivasan, Anirudh and - Sitaram, Sunayana and - Choudhury, Monojit", - booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics", - month = jul, - year = "2020", - address = "Online", - publisher = "Association for Computational Linguistics", - url = "https://www.aclweb.org/anthology/2020.acl-main.329", - pages = "3575--3585" -} -``` diff --git a/model_cards/rohanrajpal/bert-base-en-es-codemix-cased/README.md b/model_cards/rohanrajpal/bert-base-en-es-codemix-cased/README.md deleted file mode 100644 index 151c5f040f..0000000000 --- a/model_cards/rohanrajpal/bert-base-en-es-codemix-cased/README.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -language: -- es -- en -tags: -- es -- en -- codemix -license: "apache-2.0" -datasets: -- SAIL 2017 -metrics: -- fscore -- accuracy -- precision -- recall ---- - -# BERT codemixed base model for spanglish (cased) - -This model was built using [lingualytics](https://github.com/lingualytics/py-lingualytics), an open-source library that supports code-mixed analytics. - -## Model description - -Input for the model: Any codemixed spanglish text -Output for the model: Sentiment. (0 - Negative, 1 - Neutral, 2 - Positive) - -I took a bert-base-multilingual-cased model from Huggingface and finetuned it on [CS-EN-ES-CORPUS](http://www.grupolys.org/software/CS-CORPORA/cs-en-es-corpus-wassa2015.txt) dataset. - -Performance of this model on the dataset - -| metric | score | -|------------|----------| -| acc | 0.718615 | -| f1 | 0.71759 | -| acc_and_f1 | 0.718103 | -| precision | 0.719302 | -| recall | 0.718615 | - -## Intended uses & limitations - -Make sure to preprocess your data using [these methods](https://github.com/microsoft/GLUECoS/blob/master/Data/Preprocess_Scripts/preprocess_sent_en_es.py) before using this model. - -#### How to use - -Here is how to use this model to get the features of a given text in *PyTorch*: - -```python -# You can include sample code which will be formatted -from transformers import BertTokenizer, BertModelForSequenceClassification -tokenizer = AutoTokenizer.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -model = AutoModelForSequenceClassification.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in *TensorFlow*: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -model = TFBertModel.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -#### Limitations and bias - -Since I dont know spanish, I cant verify the quality of annotations or the dataset itself. This is a very simple transfer learning approach and I'm open to discussions to improve upon this. - -## Training data - -I trained on the dataset on the [bert-base-multilingual-cased model](https://huggingface.co/bert-base-multilingual-cased). - -## Training procedure - -Followed the preprocessing techniques followed [here](https://github.com/microsoft/GLUECoS/blob/master/Data/Preprocess_Scripts/preprocess_sent_en_es.py) - -## Eval results - -### BibTeX entry and citation info - -```bibtex -@inproceedings{khanuja-etal-2020-gluecos, - title = "{GLUEC}o{S}: An Evaluation Benchmark for Code-Switched {NLP}", - author = "Khanuja, Simran and - Dandapat, Sandipan and - Srinivasan, Anirudh and - Sitaram, Sunayana and - Choudhury, Monojit", - booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics", - month = jul, - year = "2020", - address = "Online", - publisher = "Association for Computational Linguistics", - url = "https://www.aclweb.org/anthology/2020.acl-main.329", - pages = "3575--3585" -} -``` diff --git a/model_cards/rohanrajpal/bert-base-en-hi-codemix-cased/README.md b/model_cards/rohanrajpal/bert-base-en-hi-codemix-cased/README.md deleted file mode 100644 index a407ed9daa..0000000000 --- a/model_cards/rohanrajpal/bert-base-en-hi-codemix-cased/README.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -language: -- hi -- en -tags: -- es -- en -- codemix -license: "apache-2.0" -datasets: -- SAIL 2017 -metrics: -- fscore -- accuracy -- precision -- recall ---- - -# BERT codemixed base model for Hinglish (cased) - -This model was built using [lingualytics](https://github.com/lingualytics/py-lingualytics), an open-source library that supports code-mixed analytics. - -## Model description - -Input for the model: Any codemixed Hinglish text -Output for the model: Sentiment. (0 - Negative, 1 - Neutral, 2 - Positive) - -I took a bert-base-multilingual-cased model from Huggingface and finetuned it on [SAIL 2017](http://www.dasdipankar.com/SAILCodeMixed.html) dataset. - -## Eval results - -Performance of this model on the dataset - -| metric | score | -|------------|----------| -| acc | 0.55873 | -| f1 | 0.558369 | -| acc_and_f1 | 0.558549 | -| precision | 0.558075 | -| recall | 0.55873 | - -#### How to use - -Here is how to use this model to get the features of a given text in *PyTorch*: - -```python -# You can include sample code which will be formatted -from transformers import BertTokenizer, BertModelForSequenceClassification -tokenizer = AutoTokenizer.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -model = AutoModelForSequenceClassification.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in *TensorFlow*: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -model = TFBertModel.from_pretrained('rohanrajpal/bert-base-en-es-codemix-cased') -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -#### Preprocessing - -Followed standard preprocessing techniques: -- removed digits -- removed punctuation -- removed stopwords -- removed excess whitespace -Here's the snippet - -```python -from pathlib import Path -import pandas as pd -from lingualytics.preprocessing import remove_lessthan, remove_punctuation, remove_stopwords -from lingualytics.stopwords import hi_stopwords,en_stopwords -from texthero.preprocessing import remove_digits, remove_whitespace - -root = Path('') - -for file in 'test','train','validation': - tochange = root / f'{file}.txt' - df = pd.read_csv(tochange,header=None,sep='\t',names=['text','label']) - df['text'] = df['text'].pipe(remove_digits) \ - .pipe(remove_punctuation) \ - .pipe(remove_stopwords,stopwords=en_stopwords.union(hi_stopwords)) \ - .pipe(remove_whitespace) - df.to_csv(tochange,index=None,header=None,sep='\t') -``` - -## Training data - -The dataset and annotations are not good, but this is the best dataset I could find. I am working on procuring my own dataset and will try to come up with a better model! - -## Training procedure - -I trained on the dataset on the [bert-base-multilingual-cased model](https://huggingface.co/bert-base-multilingual-cased). diff --git a/model_cards/rohanrajpal/bert-base-multilingual-codemixed-cased-sentiment/README.md b/model_cards/rohanrajpal/bert-base-multilingual-codemixed-cased-sentiment/README.md deleted file mode 100644 index bac6ca0e12..0000000000 --- a/model_cards/rohanrajpal/bert-base-multilingual-codemixed-cased-sentiment/README.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -language: -- hi -- en -tags: -- hi -- en -- codemix -license: "apache-2.0" -datasets: -- SAIL 2017 -metrics: -- fscore -- accuracy ---- - -# BERT codemixed base model for hinglish (cased) - -## Model description - -Input for the model: Any codemixed hinglish text -Output for the model: Sentiment. (0 - Negative, 1 - Neutral, 2 - Positive) - -I took a bert-base-multilingual-cased model from Huggingface and finetuned it on [SAIL 2017](http://www.dasdipankar.com/SAILCodeMixed.html) dataset. - -Performance of this model on the SAIL 2017 dataset - -| metric | score | -|------------|----------| -| acc | 0.588889 | -| f1 | 0.582678 | -| acc_and_f1 | 0.585783 | -| precision | 0.586516 | -| recall | 0.588889 | - -## Intended uses & limitations - -#### How to use - -Here is how to use this model to get the features of a given text in *PyTorch*: - -```python -# You can include sample code which will be formatted -from transformers import BertTokenizer, BertModelForSequenceClassification -tokenizer = AutoTokenizer.from_pretrained("rohanrajpal/bert-base-codemixed-uncased-sentiment") -model = AutoModelForSequenceClassification.from_pretrained("rohanrajpal/bert-base-codemixed-uncased-sentiment") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in *TensorFlow*: - -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained('rohanrajpal/bert-base-codemixed-uncased-sentiment') -model = TFBertModel.from_pretrained("rohanrajpal/bert-base-codemixed-uncased-sentiment") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -#### Limitations and bias - -Coming soon! - -## Training data - -I trained on the SAIL 2017 dataset [link](http://amitavadas.com/SAIL/Data/SAIL_2017.zip) on this [pretrained model](https://huggingface.co/bert-base-multilingual-cased). - -## Training procedure - -No preprocessing. - -## Eval results - -### BibTeX entry and citation info - -```bibtex -@inproceedings{khanuja-etal-2020-gluecos, - title = "{GLUEC}o{S}: An Evaluation Benchmark for Code-Switched {NLP}", - author = "Khanuja, Simran and - Dandapat, Sandipan and - Srinivasan, Anirudh and - Sitaram, Sunayana and - Choudhury, Monojit", - booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics", - month = jul, - year = "2020", - address = "Online", - publisher = "Association for Computational Linguistics", - url = "https://www.aclweb.org/anthology/2020.acl-main.329", - pages = "3575--3585" -} -``` diff --git a/model_cards/sachaarbonel/bert-italian-cased-finetuned-pos/README.md b/model_cards/sachaarbonel/bert-italian-cased-finetuned-pos/README.md deleted file mode 100644 index 1fb513c3f8..0000000000 --- a/model_cards/sachaarbonel/bert-italian-cased-finetuned-pos/README.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -language: it -datasets: -- xtreme ---- - -# Italian-Bert (Italian Bert) + POS 🎃🏷 - -This model is a fine-tuned on [xtreme udpos Italian](https://huggingface.co/nlp/viewer/?dataset=xtreme&config=udpos.Italian) version of [Bert Base Italian](https://huggingface.co/dbmdz/bert-base-italian-cased) for **POS** downstream task. - -## Details of the downstream task (POS) - Dataset - -- [Dataset: xtreme udpos Italian](https://huggingface.co/nlp/viewer/?dataset=xtreme&config=udpos.Italian) 📚 - -| Dataset | # Examples | -| ---------------------- | ----- | -| Train | 716 K | -| Dev | 85 K | - -- [Fine-tune on NER script provided by @stefan-it](https://raw.githubusercontent.com/stefan-it/fine-tuned-berts-seq/master/scripts/preprocess.py) - -- Labels covered: - -``` -ADJ -ADP -ADV -AUX -CCONJ -DET -INTJ -NOUN -NUM -PART -PRON -PROPN -PUNCT -SCONJ -SYM -VERB -X -``` - -## Metrics on evaluation set 🧾 - -| Metric | # score | -| :------------------------------------------------------------------------------------: | :-------: | -| F1 | **97.25** -| Precision | **97.15** | -| Recall | **97.36** | - -## Model in action 🔨 - - -Example of usage - -```python -from transformers import pipeline - -nlp_pos = pipeline( - "ner", - model="sachaarbonel/bert-italian-cased-finetuned-pos", - tokenizer=( - 'sachaarbonel/bert-spanish-cased-finetuned-pos', - {"use_fast": False} -)) - - -text = 'Roma è la Capitale d'Italia.' - -nlp_pos(text) - -''' -Output: --------- -[{'entity': 'PROPN', 'index': 1, 'score': 0.9995346665382385, 'word': 'roma'}, - {'entity': 'AUX', 'index': 2, 'score': 0.9966597557067871, 'word': 'e'}, - {'entity': 'DET', 'index': 3, 'score': 0.9994786977767944, 'word': 'la'}, - {'entity': 'NOUN', - 'index': 4, - 'score': 0.9995198249816895, - 'word': 'capitale'}, - {'entity': 'ADP', 'index': 5, 'score': 0.9990894198417664, 'word': 'd'}, - {'entity': 'PART', 'index': 6, 'score': 0.57159024477005, 'word': "'"}, - {'entity': 'PROPN', - 'index': 7, - 'score': 0.9994804263114929, - 'word': 'italia'}, - {'entity': 'PUNCT', 'index': 8, 'score': 0.9772886633872986, 'word': '.'}] -''' -``` -Yeah! Not too bad 🎉 - -> Created by [Sacha Arbonel/@sachaarbonel](https://twitter.com/sachaarbonel) | [LinkedIn](https://www.linkedin.com/in/sacha-arbonel) - -> Made with ♥ in Paris diff --git a/model_cards/sagorsarker/bangla-bert-base/README.md b/model_cards/sagorsarker/bangla-bert-base/README.md deleted file mode 100644 index 859b980cbe..0000000000 --- a/model_cards/sagorsarker/bangla-bert-base/README.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -language: bn -tags: -- bert -- bengali -- bengali-lm -- bangla -license: MIT -datasets: -- common_crawl -- wikipedia -- oscar ---- - - -# Bangla BERT Base -A long way passed. Here is our **Bangla-Bert**! It is now available in huggingface model hub. - -[Bangla-Bert-Base](https://github.com/sagorbrur/bangla-bert) is a pretrained language model of Bengali language using mask language modeling described in [BERT](https://arxiv.org/abs/1810.04805) and it's github [repository](https://github.com/google-research/bert) - - - -## Pretrain Corpus Details -Corpus was downloaded from two main sources: - -* Bengali commoncrawl copurs downloaded from [OSCAR](https://oscar-corpus.com/) -* [Bengali Wikipedia Dump Dataset](https://dumps.wikimedia.org/bnwiki/latest/) - -After downloading these corpus, we preprocessed it as a Bert format. which is one sentence per line and an extra newline for new documents. - -``` -sentence 1 -sentence 2 - -sentence 1 -sentence 2 - -``` - -## Building Vocab -We used [BNLP](https://github.com/sagorbrur/bnlp) package for training bengali sentencepiece model with vocab size 102025. We preprocess the output vocab file as Bert format. -Our final vocab file availabe at [https://github.com/sagorbrur/bangla-bert](https://github.com/sagorbrur/bangla-bert) and also at [huggingface](https://huggingface.co/sagorsarker/bangla-bert-base) model hub. - -## Training Details -* Bangla-Bert was trained with code provided in Google BERT's github repository (https://github.com/google-research/bert) -* Currently released model follows bert-base-uncased model architecture (12-layer, 768-hidden, 12-heads, 110M parameters) -* Total Training Steps: 1 Million -* The model was trained on a single Google Cloud TPU - -## Evaluation Results - -### LM Evaluation Results -After training 1 millions steps here is the evaluation resutls. - -``` -global_step = 1000000 -loss = 2.2406516 -masked_lm_accuracy = 0.60641736 -masked_lm_loss = 2.201459 -next_sentence_accuracy = 0.98625 -next_sentence_loss = 0.040997364 -perplexity = numpy.exp(2.2406516) = 9.393331287442784 -Loss for final step: 2.426227 - -``` - -### Downstream Task Evaluation Results -Huge Thanks to [Nick Doiron](https://twitter.com/mapmeld) for providing evalution results of classification task. -He used [Bengali Classification Benchmark](https://github.com/rezacsedu/Classification_Benchmarks_Benglai_NLP) datasets for classification task. -Comparing to Nick's [Bengali electra](https://huggingface.co/monsoon-nlp/bangla-electra) and multi-lingual BERT, Bangla BERT Base achieves state of the art result. -Here is the [evaluation script](https://github.com/sagorbrur/bangla-bert/blob/master/notebook/bangla-bert-evaluation-classification-task.ipynb). - - -| Model | Sentiment Analysis | Hate Speech Task | News Topic Task | Average | -| ----- | -------------------| ---------------- | --------------- | ------- | -| mBERT | 68.15 | 52.32 | 72.27 | 64.25 | -| Bengali Electra | 69.19 | 44.84 | 82.33 | 65.45 | -| Bangla BERT Base | 70.37 | 71.83 | 89.19 | 77.13 | - - -**NB: If you use this model for any nlp task please share evaluation results with us. We will add it here.** - - -## How to Use -You can use this model directly with a pipeline for masked language modeling: - -```py -from transformers import BertForMaskedLM, BertTokenizer, pipeline - -model = BertForMaskedLM.from_pretrained("sagorsarker/bangla-bert-base") -tokenizer = BertTokenizer.from_pretrained("sagorsarker/bangla-bert-base") -nlp = pipeline('fill-mask', model=model, tokenizer=tokenizer) -for pred in nlp(f"আমি বাংলায় {nlp.tokenizer.mask_token} গাই।"): - print(pred) - -# {'sequence': '[CLS] আমি বাংলায গান গাই । [SEP]', 'score': 0.13404667377471924, 'token': 2552, 'token_str': 'গান'} - -``` - - -## Author -[Sagor Sarker](https://github.com/sagorbrur) - -## Acknowledgements - -* Thanks to Google [TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc) for providing the free TPU credits - thank you! -* Thank to all the people around, who always helping us to build something for Bengali. - -## Reference -* https://github.com/google-research/bert - - - - - diff --git a/model_cards/sagorsarker/bangla-bert-sentiment/README.md b/model_cards/sagorsarker/bangla-bert-sentiment/README.md deleted file mode 100644 index 4a48873f31..0000000000 --- a/model_cards/sagorsarker/bangla-bert-sentiment/README.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -language: -- bn -datasets: -- socian -- bangla-sentiment-benchmark -license: mit -tags: -- bengali -- bengali-sentiment -- sentiment-analysis ---- - -# bangla-bert-sentiment -`bangla-bert-sentiment` is a pretrained model for bengali **Sentiment Analysis** using [bangla-bert-base](https://huggingface.co/sagorsarker/bangla-bert-base) model. - -## Datasets Details -This model was trained with two combined datasets -* [socian sentiment data](https://github.com/socian-ai/socian-bangla-sentiment-dataset-labeled) -* [bangla classification dataset](https://github.com/rezacsedu/Classification_Benchmarks_Benglai_NLP) - -||| -|--|--| -|Data Size| 10889 | -|Positive| 4999 | -|Negative| 5890 | -|Train | 8711 | -| Test | 2178 | - -## Training Details -Model trained with [simpletransformers](https://github.com/ThilinaRajapakse/simpletransformers) binary classification script with total of **3 epochs** in `google colab gpu`. - - -## Evaluation Details -Model evaluate with 2178 sentences - -Here is the evaluation result details in table - - -|Eval Loss | TP | TN | FP | FN | F1 Score | -| -------- | -- | -- | -- | -- | -------- | -| 0.3289 | 880 | 1158 | 59 | 81 | 92.63 | - -## Usage - -Calculate sentiment from given sentence - -```py - -from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/bangla-bert-sentiment") - -model = AutoModelForSequenceClassification.from_pretrained("sagorsarker/bangla-bert-sentiment") - -nlp = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer) -sentence = "বাংলার ঘরে ঘরে আজ নবান্নের উৎসব" -nlp(sentence) - -``` - diff --git a/model_cards/sagorsarker/codeswitch-hineng-lid-lince/README.md b/model_cards/sagorsarker/codeswitch-hineng-lid-lince/README.md deleted file mode 100644 index 78ec04340a..0000000000 --- a/model_cards/sagorsarker/codeswitch-hineng-lid-lince/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: -- hi -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- hindi-english -- language-identification ---- - -# codeswitch-hineng-lid-lince -This is a pretrained model for **language identification** of `hindi-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Identify Language - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-hineng-lid-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-hineng-lid-lince") -lid_model = pipeline('ner', model=model, tokenizer=tokenizer) - -lid_model("put any hindi english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import LanguageIdentification -lid = LanguageIdentification('hin-eng') -text = "" # your code-mixed sentence -result = lid.identify(text) -print(result) -``` - diff --git a/model_cards/sagorsarker/codeswitch-hineng-ner-lince/README.md b/model_cards/sagorsarker/codeswitch-hineng-ner-lince/README.md deleted file mode 100644 index 0d28021249..0000000000 --- a/model_cards/sagorsarker/codeswitch-hineng-ner-lince/README.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -language: -- hi -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- hindi-english -- ner ---- - -# codeswitch-hineng-ner-lince -This is a pretrained model for **Name Entity Recognition** of `Hindi-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Name Entity Recognition of Code-Mixed Data - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-hineng-ner-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-hineng-ner-lince") - -ner_model = pipeline('ner', model=model, tokenizer=tokenizer) - -ner_model("put any hindi english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import NER -ner = NER('hin-eng') -text = "" # your mixed sentence -result = ner.tag(text) -print(result) - -``` diff --git a/model_cards/sagorsarker/codeswitch-hineng-pos-lince/README.md b/model_cards/sagorsarker/codeswitch-hineng-pos-lince/README.md deleted file mode 100644 index e7a3825518..0000000000 --- a/model_cards/sagorsarker/codeswitch-hineng-pos-lince/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: -- hi -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- hindi-english -- pos ---- - -# codeswitch-hineng-pos-lince -This is a pretrained model for **Part of Speech Tagging** of `hindi-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Part-of-Speech Tagging of Hindi-English Mixed Data - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-hineng-pos-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-hineng-pos-lince") -pos_model = pipeline('ner', model=model, tokenizer=tokenizer) - -pos_model("put any hindi english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import POS -pos = POS('hin-eng') -text = "" # your mixed sentence -result = pos.tag(text) -print(result) -``` diff --git a/model_cards/sagorsarker/codeswitch-nepeng-lid-lince/README.md b/model_cards/sagorsarker/codeswitch-nepeng-lid-lince/README.md deleted file mode 100644 index df28246708..0000000000 --- a/model_cards/sagorsarker/codeswitch-nepeng-lid-lince/README.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -language: -- ne -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- nepali-english -- language-identification ---- - -# codeswitch-nepeng-lid-lince -This is a pretrained model for **language identification** of `nepali-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home). - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Identify Language - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-nepeng-lid-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-nepeng-lid-lince") -lid_model = pipeline('ner', model=model, tokenizer=tokenizer) - -lid_model("put any nepali english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import LanguageIdentification -lid = LanguageIdentification('nep-eng') -text = "" # your code-mixed sentence -result = lid.identify(text) -print(result) - -``` - diff --git a/model_cards/sagorsarker/codeswitch-spaeng-lid-lince/README.md b/model_cards/sagorsarker/codeswitch-spaeng-lid-lince/README.md deleted file mode 100644 index d88e497093..0000000000 --- a/model_cards/sagorsarker/codeswitch-spaeng-lid-lince/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: -- es -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- spanish-english -- language-identification ---- - -# codeswitch-spaeng-lid-lince -This is a pretrained model for **language identification** of `spanish-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Identify Language - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-spaeng-lid-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-spaeng-lid-lince") -lid_model = pipeline('ner', model=model, tokenizer=tokenizer) - -lid_model("put any spanish english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import LanguageIdentification -lid = LanguageIdentification('spa-eng') -text = "" # your code-mixed sentence -result = lid.identify(text) -print(result) -``` diff --git a/model_cards/sagorsarker/codeswitch-spaeng-ner-lince/README.md b/model_cards/sagorsarker/codeswitch-spaeng-ner-lince/README.md deleted file mode 100644 index 27dd3ae0a9..0000000000 --- a/model_cards/sagorsarker/codeswitch-spaeng-ner-lince/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: -- es -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- spanish-english -- ner ---- - -# codeswitch-spaeng-ner-lince -This is a pretrained model for **Name Entity Recognition** of `spanish-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Name Entity Recognition of Spanish-English Mixed Data - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-spaeng-ner-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-spaeng-ner-lince") - -ner_model = pipeline('ner', model=model, tokenizer=tokenizer) - -ner_model("put any spanish english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import NER -ner = NER('spa-eng') -text = "" # your mixed sentence -result = ner.tag(text) -print(result) -``` diff --git a/model_cards/sagorsarker/codeswitch-spaeng-pos-lince/README.md b/model_cards/sagorsarker/codeswitch-spaeng-pos-lince/README.md deleted file mode 100644 index 9ae08df89e..0000000000 --- a/model_cards/sagorsarker/codeswitch-spaeng-pos-lince/README.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -language: -- es -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- spanish-english -- pos ---- - -# codeswitch-spaeng-pos-lince -This is a pretrained model for **Part of Speech Tagging** of `spanish-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Part-of-Speech Tagging of Spanish-English Mixed Data - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-spaeng-pos-lince") - -model = AutoModelForTokenClassification.from_pretrained("sagorsarker/codeswitch-spaeng-pos-lince") -pos_model = pipeline('ner', model=model, tokenizer=tokenizer) - -pos_model("put any spanish english code-mixed sentence") - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import POS -pos = POS('spa-eng') -text = "" # your mixed sentence -result = pos.tag(text) -print(result) -``` diff --git a/model_cards/sagorsarker/codeswitch-spaeng-sentiment-analysis-lince/README.md b/model_cards/sagorsarker/codeswitch-spaeng-sentiment-analysis-lince/README.md deleted file mode 100644 index 6a55a953bd..0000000000 --- a/model_cards/sagorsarker/codeswitch-spaeng-sentiment-analysis-lince/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: -- es -- en -datasets: -- lince -license: mit -tags: -- codeswitching -- spanish-english -- sentiment-analysis ---- - -# codeswitch-spaeng-sentiment-analysis-lince -This is a pretrained model for **Sentiment Analysis** of `spanish-english` code-mixed data used from [LinCE](https://ritual.uh.edu/lince/home) - -This model is trained for this below repository. - -[https://github.com/sagorbrur/codeswitch](https://github.com/sagorbrur/codeswitch) - -To install codeswitch: - -``` -pip install codeswitch -``` - -## Sentiment Analysis of Spanish-English Code-Mixed Data - -* **Method-1** - -```py - -from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline - -tokenizer = AutoTokenizer.from_pretrained("sagorsarker/codeswitch-spaeng-sentiment-analysis-lince") - -model = AutoModelForSequenceClassification.from_pretrained("sagorsarker/codeswitch-spaeng-sentiment-analysis-lince") - -nlp = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer) -sentence = "El perro le ladraba a La Gatita .. .. lol #teamlagatita en las playas de Key Biscayne este Memorial day" -nlp(sentence) - -``` - -* **Method-2** - -```py -from codeswitch.codeswitch import SentimentAnalysis -sa = SentimentAnalysis('spa-eng') -sentence = "El perro le ladraba a La Gatita .. .. lol #teamlagatita en las playas de Key Biscayne este Memorial day" -result = sa.analyze(sentence) -print(result) -``` diff --git a/model_cards/sarahlintang/IndoBERT/README.md b/model_cards/sarahlintang/IndoBERT/README.md deleted file mode 100644 index bb5348ccaa..0000000000 --- a/model_cards/sarahlintang/IndoBERT/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: id -datasets: -- oscar ---- -# IndoBERT (Indonesian BERT Model) - -## Model description -IndoBERT is a pre-trained language model based on BERT architecture for the Indonesian Language. - -This model is base-uncased version which use bert-base config. - -## Intended uses & limitations - -#### How to use - -```python -from transformers import AutoTokenizer, AutoModel -tokenizer = AutoTokenizer.from_pretrained("sarahlintang/IndoBERT") -model = AutoModel.from_pretrained("sarahlintang/IndoBERT") -tokenizer.encode("hai aku mau makan.") -[2, 8078, 1785, 2318, 1946, 18, 4] -``` - - -## Training data - -This model was pre-trained on 16 GB of raw text ~2 B words from Oscar Corpus (https://oscar-corpus.com/). - -This model is equal to bert-base model which has 32,000 vocabulary size. - -## Training procedure - -The training of the model has been performed using Google’s original Tensorflow code on eight core Google Cloud TPU v2. -We used a Google Cloud Storage bucket, for persistent storage of training data and models. - -## Eval results - -We evaluate this model on three Indonesian NLP downstream task: -- some extractive summarization model -- sentiment analysis -- Part-of-Speech Tagger -it was proven that this model outperforms multilingual BERT for all downstream tasks. diff --git a/model_cards/sarnikowski/electra-small-discriminator-da-256-cased/README.md b/model_cards/sarnikowski/electra-small-discriminator-da-256-cased/README.md deleted file mode 100644 index 649e4344eb..0000000000 --- a/model_cards/sarnikowski/electra-small-discriminator-da-256-cased/README.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -language: da -license: cc-by-4.0 ---- - -# Danish ELECTRA small (cased) - -An [ELECTRA](https://arxiv.org/abs/2003.10555) model pretrained on a custom Danish corpus (~17.5gb). -For details regarding data sources and training procedure, along with benchmarks on downstream tasks, go to: https://github.com/sarnikowski/danish_transformers/tree/main/electra - -## Usage - -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("sarnikowski/electra-small-discriminator-da-256-cased") -model = AutoModel.from_pretrained("sarnikowski/electra-small-discriminator-da-256-cased") -``` - -## Questions? - -If you have any questions feel free to open an issue on the [danish_transformers](https://github.com/sarnikowski/danish_transformers) repository, or send an email to p.sarnikowski@gmail.com diff --git a/model_cards/savasy/bert-base-turkish-ner-cased/README.md b/model_cards/savasy/bert-base-turkish-ner-cased/README.md deleted file mode 100644 index 079575985c..0000000000 --- a/model_cards/savasy/bert-base-turkish-ner-cased/README.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -language: tr ---- - -# For Turkish language, here is an easy-to-use NER application. - ** Türkçe için kolay bir python NER (Bert + Transfer Learning) (İsim Varlık Tanıma) modeli... - - -Thanks to @stefan-it, I applied the followings for training - - -cd tr-data - -for file in train.txt dev.txt test.txt labels.txt -do - wget https://schweter.eu/storage/turkish-bert-wikiann/$file -done - -cd .. -It will download the pre-processed datasets with training, dev and test splits and put them in a tr-data folder. - -Run pre-training -After downloading the dataset, pre-training can be started. Just set the following environment variables: -``` -export MAX_LENGTH=128 -export BERT_MODEL=dbmdz/bert-base-turkish-cased -export OUTPUT_DIR=tr-new-model -export BATCH_SIZE=32 -export NUM_EPOCHS=3 -export SAVE_STEPS=625 -export SEED=1 -``` -Then run pre-training: -``` -python3 run_ner_old.py --data_dir ./tr-data3 \ ---model_type bert \ ---labels ./tr-data/labels.txt \ ---model_name_or_path $BERT_MODEL \ ---output_dir $OUTPUT_DIR-$SEED \ ---max_seq_length $MAX_LENGTH \ ---num_train_epochs $NUM_EPOCHS \ ---per_gpu_train_batch_size $BATCH_SIZE \ ---save_steps $SAVE_STEPS \ ---seed $SEED \ ---do_train \ ---do_eval \ ---do_predict \ ---fp16 -``` - - -# Usage - -``` -from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer -model = AutoModelForTokenClassification.from_pretrained("savasy/bert-base-turkish-ner-cased") -tokenizer = AutoTokenizer.from_pretrained("savasy/bert-base-turkish-ner-cased") -ner=pipeline('ner', model=model, tokenizer=tokenizer) -ner("Mustafa Kemal Atatürk 19 Mayıs 1919'da Samsun'a ayak bastı.") -``` -# Some results -Data1: For the data above -Eval Results: - -* precision = 0.916400580551524 -* recall = 0.9342309684101502 -* f1 = 0.9252298787412536 -* loss = 0.11335893666411284 - -Test Results: -* precision = 0.9192058759362955 -* recall = 0.9303010230367262 -* f1 = 0.9247201697271198 -* loss = 0.11182546521618497 - - - -Data2: -https://github.com/stefan-it/turkish-bert/files/4558187/nerdata.txt -The performance for the data given by @kemalaraz is as follows - -savas@savas-lenova:~/Desktop/trans/tr-new-model-1$ cat eval_results.txt -* precision = 0.9461980692049029 -* recall = 0.959309358847465 -* f1 = 0.9527086063783312 -* loss = 0.037054269206847804 - -savas@savas-lenova:~/Desktop/trans/tr-new-model-1$ cat test_results.txt -* precision = 0.9458370635631155 -* recall = 0.9588201928530913 -* f1 = 0.952284378344882 -* loss = 0.035431676572445225 - diff --git a/model_cards/savasy/bert-base-turkish-sentiment-cased/README.md b/model_cards/savasy/bert-base-turkish-sentiment-cased/README.md deleted file mode 100644 index da98a3f101..0000000000 --- a/model_cards/savasy/bert-base-turkish-sentiment-cased/README.md +++ /dev/null @@ -1,141 +0,0 @@ ---- -language: tr ---- -# Bert-base Turkish Sentiment Model - -https://huggingface.co/savasy/bert-base-turkish-sentiment-cased - -This model is used for Sentiment Analysis, which is based on BERTurk for Turkish Language https://huggingface.co/dbmdz/bert-base-turkish-cased - - -## Dataset - -The dataset is taken from the studies [[2]](#paper-2) and [[3]](#paper-3), and merged. - -* The study [2] gathered movie and product reviews. The products are book, DVD, electronics, and kitchen. -The movie dataset is taken from a cinema Web page ([Beyazperde](www.beyazperde.com)) with -5331 positive and 5331 negative sentences. Reviews in the Web page are marked in -scale from 0 to 5 by the users who made the reviews. The study considered a review -sentiment positive if the rating is equal to or bigger than 4, and negative if it is less -or equal to 2. They also built Turkish product review dataset from an online retailer -Web page. They constructed benchmark dataset consisting of reviews regarding some -products (book, DVD, etc.). Likewise, reviews are marked in the range from 1 to 5, -and majority class of reviews are 5. Each category has 700 positive and 700 negative -reviews in which average rating of negative reviews is 2.27 and of positive reviews -is 4.5. This dataset is also used by the study [[1]](#paper-1). - -* The study [[3]](#paper-3) collected tweet dataset. They proposed a new approach for automatically classifying the sentiment of microblog messages. The proposed approach is based on utilizing robust feature representation and fusion. - -*Merged Dataset* - -| *size* | *data* | -|--------|----| -| 8000 |dev.tsv| -| 8262 |test.tsv| -| 32000 |train.tsv| -| *48290* |*total*| - -### The dataset is used by following papers - -[1] Yildirim, Savaş. (2020). Comparing Deep Neural Networks to Traditional Models for Sentiment Analysis in Turkish Language. 10.1007/978-981-15-1216-2_12. - -[2] Demirtas, Erkin and Mykola Pechenizkiy. 2013. Cross-lingual polarity detection with machine translation. In Proceedings of the Second International Workshop on Issues of Sentiment -Discovery and Opinion Mining (WISDOM ’13) - -[3] Hayran, A., Sert, M. (2017), "Sentiment Analysis on Microblog Data based on Word Embedding and Fusion Techniques", IEEE 25th Signal Processing and Communications Applications Conference (SIU 2017), Belek, Turkey - - -## Training - -```shell -export GLUE_DIR="./sst-2-newall" -export TASK_NAME=SST-2 - -python3 run_glue.py \ - --model_type bert \ - --model_name_or_path dbmdz/bert-base-turkish-uncased\ - --task_name "SST-2" \ - --do_train \ - --do_eval \ - --data_dir "./sst-2-newall" \ - --max_seq_length 128 \ - --per_gpu_train_batch_size 32 \ - --learning_rate 2e-5 \ - --num_train_epochs 3.0 \ - --output_dir "./model" -``` - - -## Results - -> 05/10/2020 17:00:43 - INFO - transformers.trainer - \*\*\*\*\* Running Evaluation \*\*\*\*\* -> 05/10/2020 17:00:43 - INFO - transformers.trainer - Num examples = 7999 -> 05/10/2020 17:00:43 - INFO - transformers.trainer - Batch size = 8 -> Evaluation: 100% 1000/1000 [00:34<00:00, 29.04it/s] -> 05/10/2020 17:01:17 - INFO - \_\_main__ - \*\*\*\*\* Eval results sst-2 \*\*\*\*\* -> 05/10/2020 17:01:17 - INFO - \_\_main__ - acc = 0.9539942492811602 -> 05/10/2020 17:01:17 - INFO - \_\_main__ - loss = 0.16348013816401363 - -Accuracy is about **95.4%** - - -## Code Usage - -```python -from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline - -model = AutoModelForSequenceClassification.from_pretrained("savasy/bert-base-turkish-sentiment-cased") -tokenizer = AutoTokenizer.from_pretrained("savasy/bert-base-turkish-sentiment-cased") -sa= pipeline("sentiment-analysis", tokenizer=tokenizer, model=model) - -p = sa("bu telefon modelleri çok kaliteli , her parçası çok özel bence") -print(p) -# [{'label': 'LABEL_1', 'score': 0.9871089}] -print(p[0]['label'] == 'LABEL_1') -# True - -p = sa("Film çok kötü ve çok sahteydi") -print(p) -# [{'label': 'LABEL_0', 'score': 0.9975505}] -print(p[0]['label'] == 'LABEL_1') -# False -``` - - -## Test -### Data - -Suppose your file has lots of lines of comment and label (1 or 0) at the end (tab seperated) - -> comment1 ... \t label -> comment2 ... \t label -> ... - -### Code - -```python -from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline - -model = AutoModelForSequenceClassification.from_pretrained("savasy/bert-base-turkish-sentiment-cased") -tokenizer = AutoTokenizer.from_pretrained("savasy/bert-base-turkish-sentiment-cased") -sa = pipeline("sentiment-analysis", tokenizer=tokenizer, model=model) - -input_file = "/path/to/your/file/yourfile.tsv" - -i, crr = 0, 0 -for line in open(input_file): - lines = line.strip().split("\t") - if len(lines) == 2: - - i = i + 1 - if i%100 == 0: - print(i) - - pred = sa(lines[0]) - pred = pred[0]["label"].split("_")[1] - - if pred == lines[1]: - crr = crr + 1 - -print(crr, i, crr/i) -``` diff --git a/model_cards/savasy/bert-base-turkish-squad/README.md b/model_cards/savasy/bert-base-turkish-squad/README.md deleted file mode 100644 index 209d5ca665..0000000000 --- a/model_cards/savasy/bert-base-turkish-squad/README.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -language: tr ---- -# Turkish SQuAD Model : Question Answering - -I fine-tuned Turkish-Bert-Model for Question-Answering problem with Turkish version of SQuAD; TQuAD -* BERT-base: https://huggingface.co/dbmdz/bert-base-turkish-uncased -* TQuAD dataset: https://github.com/TQuad/turkish-nlp-qa-dataset - - -# Training Code - -``` -!python3 run_squad.py \ - --model_type bert \ - --model_name_or_path dbmdz/bert-base-turkish-uncased\ - --do_train \ - --do_eval \ - --train_file trainQ.json \ - --predict_file dev1.json \ - --per_gpu_train_batch_size 12 \ - --learning_rate 3e-5 \ - --num_train_epochs 5.0 \ - --max_seq_length 384 \ - --doc_stride 128 \ - --output_dir "./model" -``` - - -# Example Usage - -> Load Model -``` -from transformers import AutoTokenizer, AutoModelForQuestionAnswering, pipeline -import torch - -tokenizer = AutoTokenizer.from_pretrained("./model") -model = AutoModelForQuestionAnswering.from_pretrained("./model") -nlp=pipeline("question-answering", model=model, tokenizer=tokenizer) -``` - -> Apply the model -``` - -sait="ABASIYANIK, Sait Faik. Hikayeci (Adapazarı 23 Kasım 1906-İstanbul 11 Mayıs 1954). \ -İlk öğrenimine Adapazarı’nda Rehber-i Terakki Mektebi’nde başladı. İki yıl kadar Adapazarı İdadisi’nde okudu.\ -İstanbul Erkek Lisesi’nde devam ettiği orta öğrenimini Bursa Lisesi’nde tamamladı (1928). İstanbul Edebiyat \ -Fakültesi’ne iki yıl devam ettikten sonra babasının isteği üzerine iktisat öğrenimi için İsviçre’ye gitti. \ -Kısa süre sonra iktisat öğrenimini bırakarak Lozan’dan Grenoble’a geçti. Üç yıl başıboş bir edebiyat öğrenimi \ -gördükten sonra babası tarafından geri çağrıldı (1933). Bir müddet Halıcıoğlu Ermeni Yetim Mektebi'nde Türkçe \ -gurup dersleri öğretmenliği yaptı. Ticarete atıldıysa da tutunamadı. Bir ay Haber gazetesinde adliye muhabirliği\ -yaptı (1942). Babasının ölümü üzerine aileden kalan emlakin geliri ile avare bir hayata başladı. Evlenemedi.\ -Yazları Burgaz adasındaki köşklerinde, kışları Şişli’deki apartmanlarında annesi ile beraber geçen bu fazla \ -içkili bohem hayatı ömrünün sonuna kadar sürdü." - -print(nlp(question="Ne zaman avare bir hayata başladı?", context=sait)) -print(nlp(question="Sait Faik hangi Lisede orta öğrenimini tamamladı?", context=sait)) - -``` -``` -# Ask your self ! type your question -print(nlp(question="...?", context=sait)) -``` - - -Check My other Model -https://huggingface.co/savasy diff --git a/model_cards/savasy/bert-turkish-text-classification/README.md b/model_cards/savasy/bert-turkish-text-classification/README.md deleted file mode 100644 index 51b219bc72..0000000000 --- a/model_cards/savasy/bert-turkish-text-classification/README.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -language: tr ---- - -# Turkish Text Classification - -This model is a fine-tune model of https://github.com/stefan-it/turkish-bert by using text classification data where there are 7 categories as follows - -``` -code_to_label={ - 'LABEL_0': 'dunya ', - 'LABEL_1': 'ekonomi ', - 'LABEL_2': 'kultur ', - 'LABEL_3': 'saglik ', - 'LABEL_4': 'siyaset ', - 'LABEL_5': 'spor ', - 'LABEL_6': 'teknoloji '} - - ``` - - -## Data -The following Turkish benchmark dataset is used for fine-tuning - -https://www.kaggle.com/savasy/ttc4900 - -## Quick Start - -Bewgin with installing transformers as follows -> pip install transformers - -``` -# Code: -# import libraries -from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer, AutoModelForSequenceClassification -tokenizer= AutoTokenizer.from_pretrained("savasy/bert-turkish-text-classification") - -# build and load model, it take time depending on your internet connection -model= AutoModelForSequenceClassification.from_pretrained("savasy/bert-turkish-text-classification") - -# make pipeline -nlp=pipeline("sentiment-analysis", model=model, tokenizer=tokenizer) - -# apply model -nlp("bla bla") -# [{'label': 'LABEL_2', 'score': 0.4753005802631378}] - -code_to_label={ - 'LABEL_0': 'dunya ', - 'LABEL_1': 'ekonomi ', - 'LABEL_2': 'kultur ', - 'LABEL_3': 'saglik ', - 'LABEL_4': 'siyaset ', - 'LABEL_5': 'spor ', - 'LABEL_6': 'teknoloji '} - -code_to_label[nlp("bla bla")[0]['label']] -# > 'kultur ' -``` - -## How the model was trained - -``` - -## loading data for Turkish text classification -import pandas as pd -# https://www.kaggle.com/savasy/ttc4900 -df=pd.read_csv("7allV03.csv") -df.columns=["labels","text"] -df.labels=pd.Categorical(df.labels) - -traind_df=... -eval_df=... - -# model -from simpletransformers.classification import ClassificationModel -import torch,sklearn - -model_args = { - "use_early_stopping": True, - "early_stopping_delta": 0.01, - "early_stopping_metric": "mcc", - "early_stopping_metric_minimize": False, - "early_stopping_patience": 5, - "evaluate_during_training_steps": 1000, - "fp16": False, - "num_train_epochs":3 -} - -model = ClassificationModel( - "bert", - "dbmdz/bert-base-turkish-cased", - use_cuda=cuda_available, - args=model_args, - num_labels=7 -) -model.train_model(train_df, acc=sklearn.metrics.accuracy_score) -``` -For other training models please check https://simpletransformers.ai/ - - -For the detailed usage of Turkish Text Classification please check [python notebook](https://github.com/savasy/TurkishTextClassification/blob/master/Bert_base_Text_Classification_for_Turkish.ipynb) diff --git a/model_cards/schmidek/electra-small-cased/README.md b/model_cards/schmidek/electra-small-cased/README.md deleted file mode 100644 index 7335809f20..0000000000 --- a/model_cards/schmidek/electra-small-cased/README.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -language: en -license: apache-2.0 ---- - -## ELECTRA-small-cased - -This is a cased version of `google/electra-small-discriminator`, trained on the -[OpenWebText corpus](https://skylion007.github.io/OpenWebTextCorpus/). - -Uses the same tokenizer and vocab from `bert-base-cased` diff --git a/model_cards/seiya/oubiobert-base-uncased/README.md b/model_cards/seiya/oubiobert-base-uncased/README.md deleted file mode 100644 index 426042844b..0000000000 --- a/model_cards/seiya/oubiobert-base-uncased/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -tags: -- exbert -license: apache-2.0 ---- - -# ouBioBERT-Base, Uncased - -Bidirectional Encoder Representations from Transformers for Biomedical Text Mining by Osaka University (ouBioBERT) is a language model based on the BERT-Base (Devlin, et al., 2019) architecture. We pre-trained ouBioBERT on PubMed abstracts from the PubMed baseline (ftp://ftp.ncbi.nlm.nih.gov/pubmed/baseline) via our method. - -The details of the pre-training procedure can be found in Wada, et al. (2020). - -## Evaluation - -We evaluated the performance of ouBioBERT in terms of the biomedical language understanding evaluation (BLUE) benchmark (Peng, et al., 2019). The numbers are mean (standard deviation) on five different random seeds. - - -| Dataset | Task Type | Score | -|:----------------|:-----------------------------|-------------:| -| MedSTS | Sentence similarity | 84.9 (0.6) | -| BIOSSES | Sentence similarity | 92.3 (0.8) | -| BC5CDR-disease | Named-entity recognition | 87.4 (0.1) | -| BC5CDR-chemical | Named-entity recognition | 93.7 (0.2) | -| ShARe/CLEFE | Named-entity recognition | 80.1 (0.4) | -| DDI | Relation extraction | 81.1 (1.5) | -| ChemProt | Relation extraction | 75.0 (0.3) | -| i2b2 2010 | Relation extraction | 74.0 (0.8) | -| HoC | Document classification | 86.4 (0.5) | -| MedNLI | Inference | 83.6 (0.7) | -| **Total** | Macro average of the scores |**83.8 (0.3)**| - - -## Code for Fine-tuning -We made the source code for fine-tuning freely available at [our repository](https://github.com/sy-wada/blue_benchmark_with_transformers). - -## Citation - -If you use our work in your research, please kindly cite the following paper: - -```bibtex -@misc{2005.07202, -Author = {Shoya Wada and Toshihiro Takeda and Shiro Manabe and Shozo Konishi and Jun Kamohara and Yasushi Matsumura}, -Title = {A pre-training technique to localize medical BERT and enhance BioBERT}, -Year = {2020}, -Eprint = {arXiv:2005.07202}, -} -``` - - - - diff --git a/model_cards/sentence-transformers/LaBSE/README.md b/model_cards/sentence-transformers/LaBSE/README.md deleted file mode 100644 index 51b05dd624..0000000000 --- a/model_cards/sentence-transformers/LaBSE/README.md +++ /dev/null @@ -1,37 +0,0 @@ -# LaBSE Pytorch Version -This is a pytorch port of the tensorflow version of [LaBSE](https://tfhub.dev/google/LaBSE/1). - -To get the sentence embeddings, you can use the following code: -```python -from transformers import AutoTokenizer, AutoModel - -tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/LaBSE") -model = AutoModel.from_pretrained("sentence-transformers/LaBSE") - -sentences = ["Hello World", "Hallo Welt"] - -encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=64, return_tensors='pt') - -with torch.no_grad(): - model_output = model(**encoded_input) - -embeddings = model_output.pooler_output -embeddings = torch.nn.functional.normalize(embeddings) -print(embeddings) -``` - - -When you have [sentence-transformers](https://www.sbert.net/) installed, you can use the model like this: -```python -from sentence_transformers import SentenceTransformer -sentences = ["Hello World", "Hallo Welt"] - -model = SentenceTransformer('LaBSE') -embeddings = model.encode(sentences) -print(embeddings) -``` - -## Reference: -Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Narveen Ari, Wei Wang. [Language-agnostic BERT Sentence Embedding](https://arxiv.org/abs/2007.01852). July 2020 - -License: [https://tfhub.dev/google/LaBSE/1](https://tfhub.dev/google/LaBSE/1) diff --git a/model_cards/sentence-transformers/bert-base-nli-cls-token/README.md b/model_cards/sentence-transformers/bert-base-nli-cls-token/README.md deleted file mode 100644 index f31c2a9dba..0000000000 --- a/model_cards/sentence-transformers/bert-base-nli-cls-token/README.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -language: en -tags: -- exbert -license: apache-2.0 -datasets: -- snli -- multi_nli ---- - -# BERT base model (uncased) for Sentence Embeddings -This is the `bert-base-nli-cls-token` model from the [sentence-transformers](https://github.com/UKPLab/sentence-transformers)-repository. The sentence-transformers repository allows to train and use Transformer models for generating sentence and text embeddings. -The model is described in the paper [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) - -## Usage (HuggingFace Models Repository) - -You can use the model directly from the model repository to compute sentence embeddings. The CLS token of each input represents the sentence embedding: -```python -from transformers import AutoTokenizer, AutoModel -import torch - - -#Sentences we want sentence embeddings for -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] - -#Load AutoModel from huggingface model repository -tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-cls-token") -model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-cls-token") - -#Tokenize sentences -encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt') - -#Compute token embeddings -with torch.no_grad(): - model_output = model(**encoded_input) - sentence_embeddings = model_output[0][:,0] #Take the first token ([CLS]) from each sentence - -print("Sentence embeddings:") -print(sentence_embeddings) -``` - -## Usage (Sentence-Transformers) -Using this model becomes more convenient when you have [sentence-transformers](https://github.com/UKPLab/sentence-transformers) installed: -``` -pip install -U sentence-transformers -``` - -Then you can use the model like this: -```python -from sentence_transformers import SentenceTransformer -model = SentenceTransformer('bert-base-nli-cls-token') -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] -sentence_embeddings = model.encode(sentences) - -print("Sentence embeddings:") -print(sentence_embeddings) -``` - - -## Citing & Authors -If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084): -``` -@inproceedings{reimers-2019-sentence-bert, - title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", - author = "Reimers, Nils and Gurevych, Iryna", - booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", - month = "11", - year = "2019", - publisher = "Association for Computational Linguistics", - url = "http://arxiv.org/abs/1908.10084", -} -``` diff --git a/model_cards/sentence-transformers/bert-base-nli-max-tokens/README.md b/model_cards/sentence-transformers/bert-base-nli-max-tokens/README.md deleted file mode 100644 index c3ec7ea06a..0000000000 --- a/model_cards/sentence-transformers/bert-base-nli-max-tokens/README.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -language: en -tags: -- exbert -license: apache-2.0 -datasets: -- snli -- multi_nli ---- - -# BERT base model (uncased) for Sentence Embeddings -This is the `bert-base-nli-max-tokens` model from the [sentence-transformers](https://github.com/UKPLab/sentence-transformers)-repository. The sentence-transformers repository allows to train and use Transformer models for generating sentence and text embeddings. -The model is described in the paper [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) - -## Usage (HuggingFace Models Repository) - -You can use the model directly from the model repository to compute sentence embeddings. It uses max pooling to generate a fixed sized sentence embedding: -```python -from transformers import AutoTokenizer, AutoModel -import torch - - -#Max Pooling - Take the max value over time for every dimension -def max_pooling(model_output, attention_mask): - token_embeddings = model_output[0] #First element of model_output contains all token embeddings - input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float() - token_embeddings[input_mask_expanded == 0] = -1e9 # Set padding tokens to large negative value - max_over_time = torch.max(token_embeddings, 1)[0] - return max_over_time - - -#Sentences we want sentence embeddings for -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] - -#Load AutoModel from huggingface model repository -tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-max-tokens") -model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-max-tokens") - -#Tokenize sentences -encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt') - -#Compute token embeddings -with torch.no_grad(): - model_output = model(**encoded_input) - -#Perform pooling. In this case, max pooling -sentence_embeddings = max_pooling(model_output, encoded_input['attention_mask']) - - -print("Sentence embeddings:") -print(sentence_embeddings) -``` - -## Usage (Sentence-Transformers) -Using this model becomes more convenient when you have [sentence-transformers](https://github.com/UKPLab/sentence-transformers) installed: -``` -pip install -U sentence-transformers -``` - -Then you can use the model like this: -```python -from sentence_transformers import SentenceTransformer -model = SentenceTransformer('bert-base-nli-max-tokens') -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] -sentence_embeddings = model.encode(sentences) - -print("Sentence embeddings:") -print(sentence_embeddings) -``` - - -## Citing & Authors -If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084): -``` -@inproceedings{reimers-2019-sentence-bert, - title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", - author = "Reimers, Nils and Gurevych, Iryna", - booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", - month = "11", - year = "2019", - publisher = "Association for Computational Linguistics", - url = "http://arxiv.org/abs/1908.10084", -} -``` diff --git a/model_cards/sentence-transformers/bert-base-nli-mean-tokens/README.md b/model_cards/sentence-transformers/bert-base-nli-mean-tokens/README.md deleted file mode 100644 index 2ee7518734..0000000000 --- a/model_cards/sentence-transformers/bert-base-nli-mean-tokens/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: en -tags: -- exbert -license: apache-2.0 -datasets: -- snli -- multi_nli ---- - -# BERT base model (uncased) for Sentence Embeddings -This is the `bert-base-nli-mean-tokens` model from the [sentence-transformers](https://github.com/UKPLab/sentence-transformers)-repository. The sentence-transformers repository allows to train and use Transformer models for generating sentence and text embeddings. -The model is described in the paper [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084) - -## Usage (HuggingFace Models Repository) - -You can use the model directly from the model repository to compute sentence embeddings: -```python -from transformers import AutoTokenizer, AutoModel -import torch - - -#Mean Pooling - Take attention mask into account for correct averaging -def mean_pooling(model_output, attention_mask): - token_embeddings = model_output[0] #First element of model_output contains all token embeddings - input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float() - sum_embeddings = torch.sum(token_embeddings * input_mask_expanded, 1) - sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9) - return sum_embeddings / sum_mask - - - -#Sentences we want sentence embeddings for -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] - -#Load AutoModel from huggingface model repository -tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens") -model = AutoModel.from_pretrained("sentence-transformers/bert-base-nli-mean-tokens") - -#Tokenize sentences -encoded_input = tokenizer(sentences, padding=True, truncation=True, max_length=128, return_tensors='pt') - -#Compute token embeddings -with torch.no_grad(): - model_output = model(**encoded_input) - -#Perform pooling. In this case, mean pooling -sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask']) -``` - -## Usage (Sentence-Transformers) -Using this model becomes more convenient when you have [sentence-transformers](https://github.com/UKPLab/sentence-transformers) installed: -``` -pip install -U sentence-transformers -``` - -Then you can use the model like this: -```python -from sentence_transformers import SentenceTransformer -model = SentenceTransformer('bert-base-nli-mean-tokens') -sentences = ['This framework generates embeddings for each input sentence', - 'Sentences are passed as a list of string.', - 'The quick brown fox jumps over the lazy dog.'] -sentence_embeddings = model.encode(sentences) - -print("Sentence embeddings:") -print(sentence_embeddings) -``` - - -## Citing & Authors -If you find this model helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084): -``` -@inproceedings{reimers-2019-sentence-bert, - title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", - author = "Reimers, Nils and Gurevych, Iryna", - booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", - month = "11", - year = "2019", - publisher = "Association for Computational Linguistics", - url = "http://arxiv.org/abs/1908.10084", -} -``` diff --git a/model_cards/severinsimmler/literary-german-bert/README.md b/model_cards/severinsimmler/literary-german-bert/README.md deleted file mode 100644 index 15c606dbaa..0000000000 --- a/model_cards/severinsimmler/literary-german-bert/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: de -thumbnail: kfold.png ---- - -# German BERT for literary texts - -This German BERT is based on `bert-base-german-dbmdz-cased`, and has been adapted to the domain of literary texts by fine-tuning the language modeling task on the [Corpus of German-Language Fiction](https://figshare.com/articles/Corpus_of_German-Language_Fiction_txt_/4524680/1). Afterwards the model was fine-tuned for named entity recognition on the [DROC](https://gitlab2.informatik.uni-wuerzburg.de/kallimachos/DROC-Release) corpus, so you can use it to recognize protagonists in German novels. - - -# Stats - -## Language modeling - -The [Corpus of German-Language Fiction](https://figshare.com/articles/Corpus_of_German-Language_Fiction_txt_/4524680/1) consists of 3,194 documents with 203,516,988 tokens or 1,520,855 types. The publication year of the texts ranges from the 18th to the 20th century: - -![years](prosa-jahre.png) - - -### Results - -After one epoch: - -| Model | Perplexity | -| ---------------- | ---------- | -| Vanilla BERT | 6.82 | -| Fine-tuned BERT | 4.98 | - - -## Named entity recognition - -The provided model was also fine-tuned for two epochs on 10,799 sentences for training, validated on 547 and tested on 1,845 with three labels: `B-PER`, `I-PER` and `O`. - - -## Results - -| Dataset | Precision | Recall | F1 | -| ------- | --------- | ------ | ---- | -| Dev | 96.4 | 87.3 | 91.6 | -| Test | 92.8 | 94.9 | 93.8 | - -The model has also been evaluated using 10-fold cross validation and compared with a classic Conditional Random Field baseline described in [Jannidis et al.](https://opus.bibliothek.uni-wuerzburg.de/opus4-wuerzburg/frontdoor/deliver/index/docId/14333/file/Jannidis_Figurenerkennung_Roman.pdf) (2015): - -![kfold](kfold.png) - - -# References - -Markus Krug, Lukas Weimer, Isabella Reger, Luisa Macharowsky, Stephan Feldhaus, Frank Puppe, Fotis Jannidis, [Description of a Corpus of Character References in German Novels](http://webdoc.sub.gwdg.de/pub/mon/dariah-de/dwp-2018-27.pdf), 2018. - -Fotis Jannidis, Isabella Reger, Lukas Weimer, Markus Krug, Martin Toepfer, Frank Puppe, [Automatische Erkennung von Figuren in deutschsprachigen Romanen](https://opus.bibliothek.uni-wuerzburg.de/opus4-wuerzburg/frontdoor/deliver/index/docId/14333/file/Jannidis_Figurenerkennung_Roman.pdf), 2015. diff --git a/model_cards/severinsimmler/literary-german-bert/kfold.png b/model_cards/severinsimmler/literary-german-bert/kfold.png deleted file mode 100644 index 2f1dbb5c8e191e44820e63253ace852df55b2cf8..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 5760 zcmcIoWmHsM+eT7i;E_-~f~cg@a0uxfK#&k{L_kn#XbGhRq=s@pLb^*7DJjW$02!oY zhL#j*kdUDfcn>`5jqhFG^XFUZ`*E+eS7 zgM*WklZ%Uso12@5hv(+ao40P=;^pP#3JMBJN=hm!Dypiga5!8|O-)^0T|-0T z&Ye4&nwnZ#T6gc>)z;S5(b3V>)z#C}yLa!NzP`SJfq|i+;r;vf5eUSC2M>&lj2=FG zXl!h3Vq#)yYWnEWBQrBIb93{@j~_pI^2EZz!qU>x%F4>x+SP$*PrXlPhiSa^7NL_|bnWaO(?uh3|8 zR8&-SboA@juVZ3jVq;_D;^N}t*n3$N9l$4yDoRX4~nwpxHmX@BLo{^D} znVFfDm6e^Hos*N3o12@LmzSTPUr+kO$7#J8F z92^=N8Xg`V85#NZ?c3<+=-AlU`1tt5#Kh#}B4X_%SmxGdnvwH#avw zKfkcBu(-Imw6wImyu7lq^7H4<)z#ItwYBy2^^J{#3bW$w$ye$7M9OR=ckRxIUqiRSn+YrKC`zF()RCh`1mS zVUCNxWlkD#4d;38!VR*Aq*0l6w2ag&*Ys}DE8U1?@z74rlFoI^6t)8 zL$yTyJiP=1&V~26UF5L1ShvBfY+ct90Az$ZgHTExwRSjNp4$l4DR5_dd3e2jx54oK zP?1O8kyXvaCk<|Jz~tttGbusZ}g2+vUZ)&cWmxf3CXODCMWU+v7od1$g7b zvQHTJ%u&=8r!m9(J8utd)Z`9lo{ffnyrN>E)fw>d2=iiRG}dRez9dyplL)Z6C^wJ{ z|JJZ2vfe*(kefT&K0!8|Km(+Y4j1{KNPA-X$KS6-MW$65M zcro596FcuORp4vlzgxu_blT~#=2l3sN2jGhQ;;P5nWX%7YUyP3taE3(ZYGM2ZnCg( zXpzCzxZS1Gr>y%B?2MZK{bAj=sH=qkGWer>)& z^}`s88ulADBd-*d$xJK|_?hGhZM58sp>2cqXsaf5wYJEx%ScJS5~P;%+h5h57lcV* zp>0fPvQ=;=^3nxS+|vgp_&t3UY@X&1)z35_%S8kzz}DrzT*K7IDHMQe{Sis-H#-O7 zbkBn<N-66xEAk~d!~d|)vgrNZdNBrQ+tk5=g-@p?Qef!YgYDWDeP~6*u#oZG zUsfI5fq3!zzAd}*|7>$3dU>I`We?{|G*!Igcr0zxy8RQy*o7ewYQFRH+Y zEp2R^VeU>v&f}iaFv8#!YpU6ieBpk{I3vUh%%xXRPRbjvdwth6cG^F@wyd8I86yo! z<+H^Z_gj~yt1mYoRmS#bw)%NvI)rCIknThGl^pWKpQ5UwA`liqiIfF9W4rO;|0)H8G zw=r~nJFz(YQiH+}W)bC?M)hfM-{%bJ4nKB-?@95FY|c_tiU}&T^$Jti2CW7Sput-d zqZ98O%cEml2HRgq-}>qPxkhvIME7^!c}#}bGf};#bDul*2g6lz$0hd8Nl&bgF&{EN z%h$8BNA&uHz`JSU95!G+pS4dW8^NFC!0J-{N<5sUX^03d5CB)3Yg(1tr27&I_u$D2CcNjBqQ&A3V_E2UOb`JSY*#zQEs6Z6 za4x3DH@O>V#1clp?$~2BFPso5;GOlD2Ha9;WkxOrpoDQa5|~9+HU)rHP^JT41)#1X znF#{uUDqOxLlOYnpsBoN@1d_G0N>Su+jO*x?McIF5~r0=t9Nv%nn95h9#z;%(Bw9`C31&BH};++JL?z5)ATwMPMd`c`0j}J7JQzPjimeEe5NKHx5*2mFmT*8(+H&gOZjU zDRgG=7q}$56(xEmeTCM#&jrG*Yo!5^0$Y>-RofM`{`y8fTC|oBZ^;WWNdsP}t5K5( zB9ha57g|Du%Dujwea+if4Oe#SCUOxGT5^Y%PSBJDh~OlH9WCQHF#h?XtHOJwVjL$M^B1Ve54Yv{OKBT(k_x#(Ha=wEzb9i44L|u&MLJt6 za*{{VV!8@Kirp%d-?PO|7sKXD5&@_u8j8voDR9-xVf-Tzk|%)?e8M$Cgp{NK8Hr(` zUWwS|Ag3NufQpkIhM6i(R#3i07iE9jr8&ah^ zw#+umcV-SL*ufp&c89CYs;Ce|sCZFvPt`k~YnS=Vp7HMI*MH`0kaABk1*mBl+CE8b z4fwqGek8e5QEp8*pYi>vlFP_sqH1eBA-GTIM?tF7lJJHmF6_C5?FWDlm~50E-a~HN z`?MAi&Oc{@SVkTsrt^Cu8*PqIY<>JoImoY^3Y4M%yq%BB%y+Ny9ztpfK(~+KkE1`|6Mx1RFenD&6IL1ilG8KPmagrSA7UN zU5e}vmGFyS6SFPYoNzB_0J78VS*K9D!A5t)k5nOnHLhO`d%<@kepVLtK+q4$3Xy_2 z0jSBg%#*hCCSzx5805qLaI8ASZOS2h`zcPhClxHQEcgSK#zu@rW@SrE@5s{^A5P2Q z!XVU6n66y0trj76ayIA6@eI>z6<7;&0Rmg#J+2#mSfx ztqVbvlNmsHy188W!Zj`TbBmEOTWl)L>rLwM66%<_pLwIz zEyM4upcCgU5e2X7F(cZNbx!M0m|2eqkJ`dDNq-v>^;URvIYw@cS1M7MyW*Nioq&Xd zhfR^UB0Y_G>Dg)Ex%`3KGc1$1&-BMb(Ffq~{`41gHotZojjfUj@d`82(%}hy@>-R< z&RfPiIz`kcyZ!D>h1V|V>%Rh&- z0=6ze6iNdg^m5n9yIxpa@IBzin5wzkGFBm{DopN#AB&WKr#uxmlsY`tw-4H{`|l!j zWF{@x2QMCN+Z`pyiUN!2UVrf(gb7cs<|!5(?&32f;oD!r{0P8gQnK&~JsCnfEzJxD zM0mx()!KVwYy)90Pn8DB@T~C9+`Qxge)a#lv)uUGHxm(FO$-b{7zumsAADQsBNL>o zCy@=$J;IAKJ|WJkYsjQyZPuxWxW)7TQZoOkYW}V)qdF6rGdG5zUzKM=dWK_dlzru< z`g4vJNMM3^9^^?#9Gwca`DrC-QrjX9dCnngniCML>gQqNe+dKWU&1TwE`{2P%-^Du z5M*?<&6&5})FjtLOpK4;7*!zL8Us0uZMDsKn8(ed;l-|Vsv**L1EHxG&wm2-99hMv z;9Uq)pOBI}J2thBKy2|q0skK~CFN?$8wGFmeNsm=~89H%l1LQ@FLh6AuK9gJZo5*_yyl(RqO1|(*Gy>ktq5MQn3V5S9V zHCX_+%EWWgl*dMlkoauDjSL>sm#h0y4~Juij$-jykJCK|dB<#J-Zq7-sIZURI3=)f zglV6)MgGA>h1SQE%DJwuxGWYk2WyC+aLhnr7h;}Ynsze|?*BzzhPHFl`SuFR<#|~UBh}3gt7KXt{pk!13{G4sle!dFX|~SK7;LWiZ-QGz%B4J9 z@54>5jQHpxaDkLH#AZt`OWUmLrhD>}0WsC3*HRc;Dp1}6eIp`m#mRL6*nK|vDy{u_{QK|vh*6H9ZBjZ)J$1;(>8e7<`&1@mLz(O*(+qcAJUy0DwRl!C zdrxi9j;&^%tZ?u46$(m-5!*qzuXg?-rng;O{h;e-0t@(r^WTaE%B<5YgcQR+_i&>B zUCT>jeaEpi2svB&o$fN#*R>Fw?ru-7IQ6Jp*&X%N-Nld0+gD%ByZ@E1#@cSPwt(;X zf^WXl-KwGaxd4m!4CfietZApS?C!1yQ*N~k1r59}slJ+f-&i%ySieo+0p3nly%-e< z1x&Palqoz!W|7i?f5cIPDD+*{)LP8SDz`Ntuje#7GEg}Xh<)IZo_u*fGaUsRPAN$@ z_$WBy-SAkvw{#Q+yis|3?}kl9wRR0f3G=DPC6ykX(#U6tK|u!Jj@3H!TTQa3_YLb+ zg09MXS?@#&;(RJZPF7^_Y+N>cJN9Pc$8G%|C93?7es*_umu53y9wtbvxUJk|rwA8s z8nBfbO%vm<>D*U5*z;GbRc#$BdLE9&3vb(?GPC4+opKs=`pUcOHUe2k%fAtb_b>V7 z`?|{12UO4E!{Y@DXE{#;y>t2cRlDD?7nv_HEOlGm0(uyXw#|Iy5{arotdgTo(uABI z-b_y}i0Ms|-c9bPtV{YbmBX648BSI&pw{Gf-x=SO%6dUcICb%9$#y(Dx3zWD7u!{k z^^LQcv{eY9l&uEo={M+* z1rnFyX0PFw2h$72J_i3L{D?)VAyx%oqx+uoE#x^r-bI(v+P4=xRvel|D8~rqq4g`M zmaQmk`$f|Wiqb_zosF?JF6ynhMcC+uKNyhLxNr(u|MmJr3I5+ZPUyl?w4K_VWLBXE PHzEyHZIu!w^T2-qp$pE{ diff --git a/model_cards/severinsimmler/literary-german-bert/prosa-jahre.png b/model_cards/severinsimmler/literary-german-bert/prosa-jahre.png deleted file mode 100644 index 6cf9e7b371532424984364b7326f7bfb518ff3d6..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 21657 zcmb5W2RxSj8#aDdB9)O!S~PB>tW+ejT9T}?XC;zNMz*LZqzEA^L}s#gi$b{VtW>gB zcJ}|cJkRgB0;3by?}Nc*WP;nL;)DRf+|ePC_K{@HN-5uI8}77sR#sVF8J}&u!Bqd`z~wVr7-=tV+407Q=YV%{ z>X*@%dC9I#=gP9*77RU#{-Qb*V!P$+d%j;AH*lX@vxPD4BHiB07izN>7CiMv?G2fZ zhGvD5)BC#LcfSvH>iZs8o8RNy``sz6?8xRb#2`ugl2Y!i{tt=CX7yFlgMRguR%G>M zpA-%8>Q1$H_1h=W>xfrItzo|=)HC4-`ICF-w=**{`Etp~ z%Lg|MD66V6bH;LIw2xTlYgg9RaEY*>z%D_>(95 zJ|5X{VcYEb8P6~NT?Q-Zw}vZ&cd)VXCp5^ow=#s8F0vouAJc!NG5LoPGH5=aXu~Sh|-T@9{?$wUw1smeeza4|at*=DAv~rBUu@ zU(@ZOr&*AqTAQo#@rdyr(S(LW7_I;3E9pxD&n#$WJo||i6p6rvtqVJ;$d68Xw2wR< z^zT)fyZfq+v){E;kJUfnd-F|}2RzKO#c5l6L~mqC*#zzVTcpdcMj!i)+92Zk+4M8o zIc$Gd7k%>XHI`wTCSlhWRkoy*V}p}(MqXUG+~fYE8Qx3#>}EXG$HFHz|FdiVU8w); z+W(iW{eNHj|7EeRzbNL=n{h0jUtSo?c3W|cxLtG7xFs#ZR4Ck3X!q0g*bAh6Ynw_u zmhpyUmxY;K!j{o#g`*<8+IfoMLS~n}==UDCAJsmviFl`^4rf2NY6_+LL|4kxluhyM zcc;oPU!IU{2ODM9QEy)Tc~4cyR)h5;6-bup3EAFPXo1qQlZNOwfcw#`t?t zzWCywGZAjfbB=R8ArJXh->#yZdXsX(;(Llx?D<3D;w2r&eac>L;xAHUb6d1JTXo9% zw|DQ?(AckEFJ+sxtLAoC`^ZZy+1#Fgw4HmZY=>TAR+cg`hU^yIs?1U1itguQHPt9v znV4*Te|dUEzAM*4?|#L?>)$;+w|;$kh&A6O=9qPg)j#siFWR7I&n^V0rt7A@zLIeG z;K4H?I(DJrOH)^>Uz`kVUX3WnN2r#PlGEx%$BJUhe0GF{=hzHft`0w;@bLj%^27x3 z=4_+7(+px$PabUE`I2@kN22Xeb6fuH1oEx#MGm_JuHQ4y`$Xb@emAb^O4pOf?=&CjEL7_(u#Mf0 zW6ikrF`ESI)cIKM^rFjqMeWE(NJ(*+pKw8wSLvn8nt_}O!JoUIwvRM3NGwJpl8G%% zHO$YnTLguL?W()W!_6%_+i}}Jtz_P~)3!xtetsyOJ)&r-vv2aX!(?yo;`sRZ#Y>lz z5j1%>VD6tQX=|tOYUMm%*|>4zhK=rrlbg3si8;-DG0YtJ>BZLkkwHS+eExf}Q&2>N zPLJNE$%%<9OlYUu%FfQ+^99;q%Xk%+^5eDw1vZ{8f~ zzgS3?dwxX9lZGKJ*IhGJRfC5)K0ZFf_S{3v3;t~wAhRf4&yHLi;SXRNR6Mb;=JU(` zF&e~Os)Lmy?_3$BK2yvY=H2A=`fmRBLv5(UC0c*b+`M`ODBWCPW_h zRQdI*a!Z;HyNt9yYf$@$>QHm4XG!4m=jx-==Ewlkj@-`1#z`rv+__!Sn3`uFe{#*n z(hn+FTfcoFpMX>q{&jocbJ4v2EvXrZC*7aN3_S}7c-hgR?V)r|H~mEaO3YH|JAFqP z8JTa(kE?nvU%&nm>m2s=M~Q=jgH(@)hezVG_u>_$6Y<7(l#MO}97W{$%dpKZEaW)B@YB(pFx(r`sV?A*|5 zCJfW{VC^4sTPZyGP0H-Tg$tD56@l3flZM;3ZQ~LX6YC$mqp2D8N+BxEZ}*AH5xb$` z%v#mUmzh#mhho%5>xg-A{ri&!YxnzEtP@0;K%w*h=XSId>pM3r#TDl+isf=JeRVU4+;@GsC_zjY>U$%%I zF*|?9yJc!tN9=7#+05?A?MC-+GL#TLog}$%Ap}qPwnF>#nbA%y^T^)n@c$_j95s0K>QdK5**6GL$vP zH#QHtTD9kx5s~}yl`Bmz&uuE6tPyR?x)qpKy6kAySrGJPEip5Uca3$Vx{9505sMF$ zcBLa~Utx2ZdP+WY@}vefTvN1^XO`WFW_&`z)@ykG*$)Iuai%}hf-@@pJG%~IFm`I(XAz9=_ceEWXwd|X^yf0#*H-C$GlzlqK^G}@_HS0goH+4$PEn2WNq|KsNjtf$m( z_kOK=?@3c7C+ht74dyTbi^BBZjOvEHdV}S`{M10KG@FaE^y58sxm~WwoRN1<@7%d_ z;?5~zGkgCxqe}e}u^Ubd;#M>MaF5n;ylh9$v}5+|K6#B)RUPXJ_Anwp+YX(#S=}v( zRa80wM7s8uSXRG(|9%_hZ}UgR&yV*8y?Aj+`JeZsQx#vruC5VZFg-v^Otqv7XFnp0 z<4o(#Z0nwj$fU}`wf%Lm)7W^7#N1*X4@(i_p2{si3h zk1rU>X`ez`JK??erD;N9ZH=V1_8upxZS60Fy!}4i6Y@6St)8k1{3>!oKH*re=sLgR zGpHNp#tRQ;WVV`K)7Q#j6W;UKdqF=&#_C7G=@5h5_Mxl@?qpf6god5UcEWpN`61BRJ$}zGwW=rJ<4+V94?E25&)SYJEcLDw>sn9>ittAf(xh!$O;8iN zvXYXLVt!m4d9XeD2uWF4Q_uWqVqGV)jArGoT)FbG=1Qv5sCm)&`RdSP z3A$VLRu^?wCXSff@2Aqo-IkrF2O6YXbe&Q!U%p&f9r*NV45I7m!CGDXm6p=wcx=Lp z0n3Yh5^BP=c~;&3mJ!%b`;)%A zPsjrri4Z58M2b#hnsOS=mp*vjH_;{%@b~4vn3$uVz@6@E)_JqyZDg5+*|8QM{t~0$ zv7$+NT+RR8DsaYu0_F^=*}7v#!o7P`2R7pds8}BCkE-3FOigM$Hro&QmSn>k zNffo-ysfRRBCd;h{K*Y>}^0~N9is+eVblc}QDko$wgJeMhF z7zn5`U+cCUi+u1u)-3%J@Pc{ah*0%&0ngRqIv`G`(C$`uxu2oKC+{uT-BcV8QuIo7 zon}XaY$lqQxzqAIqDgdOQXz53_NZq)`VosI1D=J3z6N3r?;+kmC01_Q`u1Fx^Q2!~ z;Ev7s(qOPaan5xOqZO>h29=p?~Q6i&G?J9>_Y#+B~*drVrdhBN9^$aSPWrVh5 zB)BjU+oVX>`CFZqAL$$cJq=i>30zJh8Es3l^;x)GE0%!bWr#QvFHYV*Zqm$yQs)Y` zQIi*&Yhqojv~sM>^E4X*|si;hwNbUk0C>J+LHH9%+{hOCU;{di`qD}839z2Olq zO!u|KW#Mb?+SU=f2>b62K{YF#_FWYtcCwc#`cJa2_0>d(poAj}JZ5r#5EA|Q^XH*% ze~D(45(&7y3ghg-8qu-FXNRsNi5=+GvM7!Z=G7Xkm*uCsxJt|@E63%SwTDF!H5N*Z z%F4;Zsi`1EGy7tf>q08|5HGJ>LX@ir z9aZ#Ca({h=-VO{X89@CO7T0m2f-^bdMY>k5JW<|O@t10wYqs7M-oJl;|0Jjrf}82j ztgpXZJkw?(6{X`i5RW3|cJcZ~<7~`2g7VW;?u#j^$*8H#7|#iXr+!#so-SnjHiD|3 z3nAU`*ZYGSb)7rDF;rTF++BM8=rM?oT+NlSTNesDS*KWDe*C#%zND%r$wudAWsEJY zc5b!dCe7SV){gtOA`wA3xBFGb@7=o>J-2S?dWQOk@|d@__sjQ-i;EAoUsaj)e3w@r zmAd4Ll1rPCbAP^zVb|%dtc$es%S#J_Ce0}u6X<{|h&wX~(Rrn}G^p0whf}ai-1*Ko zgTB7LA%3?xA6Ar~?Ck78;ktHLxegtooiEDE%j=N(IARy#B(V^BGjeB8i{m}k7hCT8 zwVQKpP%3wo-2LN5-=1NbjIe00a*iWgcnkX#l_qrUBIy=gjhOZnq){fVjgG2o=zL)B zr}sM8zd!^=Ok1^ee_7byzkhKzD?&bHOhWWQW>;dYEJY#&GU!y}>3_CS33jcVp51gj zb@Fa)UvdPMOKqf>Bakn5#7(T06cL!yaC^X^&w_&DiDM)cB{BCqaIhi4KhmLhCkQ(O z0|Nq$8d~e34n?N5u2}pUddyQ?R#x_kxw(1qdNF`~W}mX63IF~*i^Fz>hzVfp$Q&G% zKP?-FO1!QLh0B3eI(+!>0T!iUs!wHQha@CQFLK^!6bCP1SRICMqpmKx#q2nB+P~rM z7|pC>F#?exwm(fL9r7U4)DlJ5D~&r#TT|0`x$B6pwJmLj#VJ+=MV+;!T1q)J$27J7%!9^Ow)*_rH>)GXMEf(_%?suH#b%x( zGu+5fw`tb6d24#GX|N?dihK*9Nn|9e0Z^ZHN(?9@Qp_n>^g?sr4S z{kAYXs&X=~c=1j}> z`K1d(ms{UDi|YY@I2KDys;RbEy4fdEY`vO!gKKH9%1vY8<&`;-rC)YWx-{(ST8C92 zI4VFqg5d%B`bC%XQjck^g;>E{Z}>E_*I~dPRB4W5#j`e5yt!1Ju2hIT$A#SHs`Rq# z$8=>bW2Xq3cL;l|J$}8Z#?ozZTw2GrN#)|jiCK2H#>%fnNAu2@ zwPguZ6+2GX6G5@4=+sC@{*}_D5exjDTAd>iG8>C@(^09j{!I!+_mJ>o-kv0jc-{Zc-F(9v~Y zDl0F`%Ri0_?Do1R{(;t=N-U)!kh5yQ#<(d-rHB3iK_OAo?@8>7bdt9ynd4Z6AM)2) z-x|zg;N*xsYg{Xz9im!q^nr-vNh--U-F_!j|8CODHSe@pnr;~iw2l_q76wo|Q@{Q4TC41&ZEn7m`s&o*b>tYs};SzZu9Y+tP* zuGb)@XIcI{J;_G;^ayxwWogVU-CunE0^&!((oB0(vKqe(r?lP5Z0QOw&c$a|Cto7# zqsR-Pb;M$1JI`4!joOzsfOa|%Rm(Z>`%7Smw)I0@=kas4W8WkG`{n|&!HpY6buZ6> zQE$Pgl4X#J6>*G|R!Pd}5~iu)VN?9<0tjAS%`8Pkf`=T$Cy|=A)$Oxl^_S)^o(gcB z9*l307nQpi<&r~GY4m$V5)GHTgF-`P{wZR-6@${F`@jSK{qvJZ>>{>aYBI~U<(CZd`hm~PQ)A?Uuj$v%@WPxhbPL425dn>>Y6{zW2D;vv8^c`}G|C1)>| z&1Rdl7*F-r5ljxYuTZ2k($fZgjIy$(CYYSB%?FI^6&=QzWw^+e8i7%;O!& zk5`KS@ZowQ=@FyY%5h1!$9ZXfy1{qnvEw%ViY|YDdhI%9xS7}piJwq7XI=Y9Epr+4 z>;;OF8EUUEF{d02R=k5&gz`>#VCNM0y~Mmc^%fm_oo^2Og~LYy`9nE(5#uK@>IS7n zBkIcrXy%yiwA00^0m&TV=l5S-i#0-^U~zNJJPC1ejN3?0+57KS*qhbhZb--3Q42QA zPY+d3+~1*lL6P+QUv0*{-=}}$Jo0%O!kJgZ*t@&y&N7oojo}zbf0LS?U=^@^dVNC! z?dF%}T_rAI0{Ra$dLNy1*OaB~;McwM35f11suaK|!`364Jv^RYWub28aBq{Ncx~SK z;XQH66ZKK-DoIJ*C07jCG#hDP8;IJEDqp@FFflpVlw+n<_W84#lS61|Xs!}3o1)wUB%VrAhwmsJ(gp8s$dJ@V=a?|`M;4gwV zt)^=<#kO^1CVGXYX3R#!Q#XD$8ynly(pFCSXG5Ork)cEl@?UnV{3zH<@0G8hlEo8J zwH;Yz6|8gHlERXv{uZ9t96^v6O9>S7jSn zdk-T>%{TxGYcr2|G5!n2^9Cx*Dp81nWB+^afZxAwoQ#kM z-aYDiE{lHDu27pV1T+6{((JLu4pI=0@^^2{pqPnCNa*F~=Yt|uLbAQoWQ$r(6WBF0 z1!unVMepRCaW+E+^d}b}FT=TCKK%4ytOb1OExV^y-kG%^h3;*@a6C$o7}e4kn7Ohf znAL?g$7<`WZ5bKA^LutdAwT8UuV2w)UgivCWo0U9K3!uByK!?|+ItTB$o9KiujbJM z8utBN5m8Zk1G8tk6*a=|`KR;Y#HZ#$XBw0|g%`bGV zmNM_a7@aYFl)5qm0L(g#G7tC6@ojrWH+8 zdNVH$>8)s56pd5=TYTm)O}Q@34}4~G^EG_Drf}jmm)&+Tpj#^XAx9Q5ZeOckT1Us z*r?7GA~s3c=Kp%c{f%+WaezIrZpL*l?;F-ex>P=G>;CoYUpfoC8CX-T)ztic{SHPc zlRlyOvrLLLlK{L~&?0oLh*p(XvmVcnVAm}Zk2Fr*-o?OnH@$5?wqiW?!g4`u*bt7``cSkW2E69L1ki3@AkP zHHf7C+b9}=W|Bda{;P2|8hDf-UlDX;LJ@oUfDJXBI&ctaUjY{DRjO8Qm_a!gX_t(L zxIyJMq)9>p1G!c3=fkFHC`CyqX6Fh1s8)R8lElhf?J8T!co1q<8h`0xJ&ztPOts!y zpvjS89>irL(2oW&Dbx(CDcFvKl$bLjLn)b4A2#t5iX3XUkOR8rpvH)3A62XNa|&YD zUUBE=dJDu_ll2cqqpeeBTdb|7q#*b^)8$9pSv6&B6@n{qQrHoLip9RwO^12 znzbyNz;;#-TzPegAbAb4HHl;-hIJ>zD?;I;fUu20H4@gdi7!tQD?&aZ_XT+9C0IY1 z%jeHiuUWGz6yQohL80a4IiIigyg0+eC!9zK2@a+%4itF7ApTbp3{cZhdTAO#llh;T z@c!;N#Ea>J$^n6P1UUrZ;sH)gDl@NUf4bWeHz7lWX?v7dbcs)Pfr{90acd^@-- z-BC#oKAD1{w8AN9x&4G}0YvJ$JQ+0vo}uLogIL^&?|06m7fn2xpB>Zd=^;KxP21|e zX0|cA%pl|$un=Y({j$M4b%{!`jW`fkxust_N-3}8!iBlkkQA$su2j_36G31?vR4&J z?;1PN$IHWWs^_4wpMcEc-(!%ar6#?upj7G+deTU4Cy{m~8+ZN^H*VxY`j$ly3Bu_< z+4mwAVrqYUh;G0&mO&utvMsG7blJ#yTV`c_oYS&(4j3VBf)tVhf|T! zxlP)xb3Gt15GIC_(@Tz4fl|B{Ke;ku;nuJUO;If**?h;bi7wE#D%)-f-oKH_X5ja+ zU%B5MO(?WW_N5CE1S|o@Yc1FVfBM&`3PA~Y{DbKcr+?$azdSQ@hwD4&& zyUkqCi0D-I0SWPQlPnKcL=(#9-Mh03?!T}q-x&}`NMPo<$v0~wx7bP&>K!PlB+R)Z z&NC$3>)*>+cU{1F8{Kyq+X6sWtUps?X+U9@kg0kxjgX*VQ1QOXO6omlr{1ugjcOP$ zRh-sA9YrXluMwOG!sSx)vHkmFD%pL92Qz-!t50|8h(!>60@7g z7h|=dGi}h(X@gl{o9~&ct;X5G?c%40J~)UUMRw*3@sT{MgaTB#MK9>{lcb<%BFGXV zk4bv5>d#aQ1ep}v&lCi8wObCl>1wqSD@Djn@z|6KeG&G$ij2I>KH#*z)>cM3fWP3j zcQ5RIEb1Xn2&ZS)H8-8VY?Pr6ApC)qksa7nD7Ie3`Q5->TiOUW97N4L z%b3==F@Vtw(%C{LYZguCkm@E#DttEA2!~AZ6rW zK=(Qiw{KvBfU|n8Kb{9)t`=`4HhB35gJpJDbO^8W@VAV%0^8)}zn>+73(s*d5iSX9 zYwIbHncOE&=66tcrs;?hal2@{D=-)w5=0WnkM6SXa*>Ma>FZOwB^lhdxuD57wZ-SSfJ+~PB zNTbl(8m(p3#3766Vo)1~qpb5c_}ND8n4*xm-t4(` zni-plRy}WIzopjnoHJ&ka*&CS{%ND&aTy%B7~6+&<}nh?jW_q4&h^3`i!);^a5-!3 zENS_#F7wRij*|T2*v=EFl{%bF@)T*mUGYAA1s90wiqI~=yC$^dZsyUjboAcm)eTyIfYEDgF@?#M=4UHr0_^kj`#kL3(CQ!Yz zSV_@JgbV;j0r|BqjY#V2#l^*Fs-;KVX)wC&dV`|Us4AE@Kweka=PV>F%6POzR|&bF zuqOo&1#_xyvF1VP$BhtqksCX_OUA78VJ6{M5%Dj(UiVx;|FBEbs(3?^6M$!7@UgiHl=s74vmMxooSC}gDET59z8y!Oob zkhgAvJH68naqqtI=FOWYTk^ML02SO`xMgHPx8}MEpol`~(fD8YTSw;GyOv);qM6*p z`<{qyC<0M_%0?ZVp8A3i`C1TMZl|U8Q>U6(=fzmacA{A`D4HSvB2e zlD;DD+!LVzg~4DC>k| zhO!EXA7Ad8mqFWMZkl1cntr#A;frxL(>ao4U_eL+A#sHEn55;Oj}WmvaD=2NO&G2K zUu6OaS_psfAeyjd==+Pk(p&tsW0#=uv(e&NUE9$P4LhelF9QMs@^m(nC|nMwSYfwp z=h?A+`<88_#52f2jo9(1p|&58^kDq~S6w}F7`x15Uv0W&q*Zr0K`?QCKuPj*a_+ZX z;mX)O!`!N&>9C%^fA47U1V2wHY##}(?F%GBttX0HCyfW!o{x8g3tpY%)}Bc#54t+hcFn0iQ1=$|NXHJ*568)))psgTj)-7 zL~EJnpF@B+@DIV-Xy-ACqKmKs09QXyz0-Hly){~wCWQqdX z$pxU0OTaNK`QBU%E77EiQHDsEQ|S6l%v?Cs0gZ@8&y(80JL*3+WF6RNU1&E_|EtWW zdP43*=e9cUej?@oPLO5b>}zo7i})cR2Kz%6FmU+`3Cy8AKPQ8>CswHH_-hxzBlW*_kVsQ-QU?qA+Mq&6#xwAP%L%*bi zufFdb;EG|+T{2PL!jA;ZA=v%XQkanM2_5Zy7r~Yi6-SVoZXL{*yC>BS5@(KNWJBvn zxJRA;S`uc`7CpD(|M0B2op#{`_G9wf*6e$LT=m7dD-(X}C~=psPj82xut>|x6ZTNP z3p11wRN;U)K<@K4*={KQ9T!G&t0&emoQj72?xE?k=vvC^G%S#L>c$sFEa-tv%shm# zh%6(YaAh@`?UPF9ky^pp*-rH<6$4QLa0f@a%xH)$3}+Hv*NvMtp*%T%e9b=UXTiaT z!CM8~$ZQ1SUnHCeJ0B?h6u+GFonlKPex7;$t6iS07dK7hR_->|(?ixtEUGsoz=n`gHsdw6b0u+3l&K0s1y6+{sSRF3)ybM$feG zZ*yr#I>n(o!W*am`Ei)A)k{RAMDPo@W`isyNPWqp&Op6!h- z3~TEvxYfPC`yVw>qf-*BJvHeYDyi+z6SV83^$owRQx7QA4G9V-L}`FmUGw#&8*Kg``*S1 zdYxH>p(GG2G6ATcbz)0J!>qjHv@vJD$LQ5UEZYrvKF0_RT~^v-%oI}D>))HBUYBtw zK=>YOW|ga{Xj1_pb`$;sfhrsht8D+ej2Zyd0h`L9>0qBtgD0dd3p;yUkEf5B#ndoO z5mZAXkeHxZ+woNG{L9B~)+p=h>Rvj2uWKy5rTxd|k59_3|7x+t;R@KOc9+?47XO+hRFHKeD)9 zo}p!qK_~Zfw}8Z?WAT=3nYg_Z7q-q5-N54@<^XM8 zM7LOZ6dTnY$m}R~vWH#$b;alSH*XXPj-?9jSyLXmKkBxBen`nAwS2Bc?Bcq!VXYBW zj(KO&{Y^VAia0nqfb9T*7AE0VqA~g{;~=f07`gwhKyLd8xqn4MiosXb%)BqLR9~f) ztD4|sBON$^oA$W}F|GzfMi*@<8h5Veh(3>5#SY61Hm8QC6$~EsVEQmNYXVZ)a1ItF zvm>@$MGl$0HL#O49ZqjahTq6bt6)v^V-QG4R3M@tp%vF<<#)ox}_?n_;+ez=VJK; zw}?2VHVmYUA6>V;q@dg|bA2(>rz5`6YM)?pbM!ZGq4hl@r`MtpmA*jbUVew32BAOt zZuqgAkKvc+m01;#gM$rqn}$(56gL~7u4v1%ij4~#TSQ@aNBl`MBB+Cr`#NFaI=Ooq2T#dd;;vO545-FgDG=Vm4^rTBPDI)n zjwUHS1gBt(ycZDHpdg_G9|+ZYxwW332s)9(xGv9w{-!6V2rLKc0Gr?xwS?2BF3A>K1+DVc2;zu71o7^2ch@ogVEBFDJQESAr^^W+M{0 z$6Doexp^Rg|8r-j3C|RG0c3A^`DPemJzuTQXc^zgo-DgnpDEEJ$$fvCDw( zTM9P&hc;k{2vx|={3toxQ+-7YnyMn}I( z(P~Y8aafkVs+Vr5FfK4QToAH|=yhW-j^q|oO zW*7^K8+((-K8)WvrKEL&M5=od6=e~*0N%Aqj^Px4@Uv%BK`!EKYp7^NcBK(wSziz+ zWZuGJbD_P6Ehvzm_du=%&k3iX%yZHsfJE-d&-phf>lxp#8XNAkcbGg zpBrn-mR*6rl&H-Er5nJc z6H<^2r^U2IV(t@y*fkS4-itz}0TRKj;DdX4(lZb`+{?{@(+EO|d`L`82Tc1uDXEBS zPlSo1N4hTQnZK%E5UoB)00aR&n%7N_Tdnh;dk5X)Yo9Av@$Gd#G*FNeWSIu5u;tZn z$-ja$N}Tf@d@duR9Go|omzP)6L1aot15@UnQRX>{@V--k1=}W}EK-$?B2Tv=_G|Qp z9>vE#iEhb2w@g2S+h0$jmJua&Zid z<0H|dvg$x6f#}$nc~c>?2wN+<@C*#N+K7BLJeUV9sGHAhtbcUDhiHuN4aMzXvx&6s z1b^??6Qt(xOEKL3Nlo~P+M;9Gpt%cR&XJ)EU4f2LUtjO>AER09E$@H;remSpvSA0U z{{Fc(x$yMyo6%`U&*r@Ysa?~AcPlXs6q-+K0k+M%6cZKQIn@y4b&KZsoGu1>@FCBJ z+jA9=L4wHyZ3oVma7lC>f0OPwn%}2dkOF%ERrT1kBM=e8NViwXvXF#?gpnhPThB_| zjSvBFz@-|V?8uVDImFHu0<+G|X+UN)h`y2|Ne1gF1nQ471eAarHc1MDxm1Vv*`civl>aJj% z1>D@QIbf2c`i9>MadCV3oB@U0As}kwwBeLjNK?GX=l4mb3_)cvrcPW#>sqG1ZA&9m z3H%*?^rYiLEyL4D{?obD;FTtAv!9Och;eDXFyFep?EQdEm{lTIImnA+eYW}9QdRV+ zPH1Uj+Pc+*ou9VDJh}dJI*)lwpU>LWZl~Q2Oq-M8Avm?X&D1q(>SMao|T6i^wPuX?5LB{)jZA7D< zz*hN2i5bst-x$!HcCcOgX_K*Fd-R$Y#=ED-y_eiq|Jm2pcLTpy+NH3P^>B>Fp#8ML zu20*ih@%NNksMRb`9ICujft~z$=2(;+88}f8d3HkmVO;J;)78Fq7sa9rneBT(9sYhr7guKWV*@( zEzRVXs=&cTSnkCByj%Ck-#=mfDvl-hPpirl>L`d!KXaWQ;54{-bLi`_+LR_KxWDiP zp5Xv2Tjrewb=QgS-OpECTy=xsZwQYT5wcfbK2baAudGX+IQqBE2IL1i2w~CiC49=w zfu*8NHwzD{BRoT1!(vy@kQSt_1cD94a&hn9f7j0YI@`M=n}4yP<>DNHBCC2{?tO_i z=7J9MT(~{9qhH%wJrH#5HhSW@0EM0Mk}r$c&Q6N;tBBa|%+UQv-3w(LJ@k*6dDJD3 zP;Q7V{`p8)fK=18524drl0t?OCzokhU5^GYFzTFZJvas+IyAuoJ{3~Bcn${c1N)2u zu*R=w*qjxyUX9@2BqbFO@tLho)c?{5?Y-rcr*0L5EwcaTSXenju0~h1W$1fB8EPzb zbEAHECJ;zioFF#A&ESTJt3Tdz@+H1YLI#0ZX+M4f{w~{P!H#HYCTPBW|IX^ZW~*DP zl7TzRJD8My`r2F9kOq-^Yi2vLj#V}s5FNJKPpCarJd9h^e0u@GVg zAx-1IB*N>nTfhvcXVgTbfX_zZk|_$W=MxcGR z*GK^70dSb0fl~wwHYKZTvG)WV?0IMg1O z@&Qvo|M<7$G7Pp^Xr3dQ=pE7cbfJDrAa<7xI4KiQ3PdQPI9!J}cd$O*2jd#;ES&yw z&_B_QXaz1si9&SQY|TnR$B79#&;#NKBP}6bf(uO}`r2hsP$e3@WJ>w1UJFHYXINkkkm#z1rm#dXI?? zd%}jWSHe{rJQeDb)A?jp@AVI^8DR`Wr|0qOpEgWRf^;8(%O6VhBXm|~qm!o7bzx)| z8ZGl9N}&@2;>fuhvzDk21#AbE zyP;^G09JndX3`Dj{c^VDKQtf5W_RnRx3J){8IS@u_MxRH)v7`6#7>wy1vY;?LNHwV zjfqO-AbmHSx?YLy62F0lgxy5Hzkv1c3+P7(Z-5=!3Qg!c3FImfj!qY1JRean<>h_L z8+aA&%Pt$t^57Z^a9%lTLfP(EfAOOsVlb9~QQV8aD97gagZrgMz0i*LuJk3MyctK+ z*GHCPhQ9ciFV>$2i82S`f>0H)LRMJg@E*ADI*2B!!`gPhomSO?EiCA&ID%m#YWM-s zvP8KppCqn_V>FK2-wo0{gc!CyGOSdeC-D|ku!0g=s1vM)Qge3!O%mg|N4wRRxJ~@O zYiZXV6tI3o-HJ!zH$Bjdm zT`w1fUbb{&8CpQ2u{5AFDS)Zqntqg zS>xc!xBlgLaFe3&-Y}DBp_p&2b!E2uPPoRG7v~)Ryr%(Qy_!*$T(M(ceV7YDh=w^a zG**~s{DtR+0yIv0>iVOy>q;noY*8$Sga?O&aH8C?F;T-3*e?FHLSeQ(r`5E#DGb2y&Q!}^pv#6PNNp)y;X=$IHf2nZP z&~5tm@S$qk;@au_{2`~R%E~{}(}HP&45$8(h!*vA13_!ouDx*S(y%~5Y)(#adHH$b zxe~EvSRETXJ8KX$`s;@f&RAN4@uy|)*PX+5qADsXSvfhoPWJx%`5s5>8BIO+-2D7e z@U?5#t(+=NUH(Qp56E@1E)N* z&w9E$C8b96^J2bi5|NmsE1H@Y&z&oC8!puUOpgu_-;)Qjv$O9wI6MssI%i|+E3wu;?;}mUeoy#E$>6rJQb@cr6$MVC+kNM7XM^WyInK#1z_6b4$f`Yf z*RJ_@LrU1j%=~L$0OsNwVE@kVuWxEPcjnBxD3_U?S~;dMh?bwq%O?fLRi9)sKTf{9{7+$zE+uGanZHJiY=;+Q%tt_m};lpg~ z?eRRFdrig$Ki=KFPu54#l-`19t-#@!WOlrjoXL4Y0 z&|2dxQHJ7yK|oX$wY4{qNc)+%hZ&7;zS6Zzo4n56?*bE2g%mf zmgKwVWB>@(OvGy9YP!0SPdV`mLmr-<4oFX5YHRx{gSfTxt?v_T3s<9-rlzI6y}iSP z>&=+5-rioPUMXp58XX;-jc=wxe~F2UM}juy)6SE$wH3s;65`|U&&=4xp8qp5!yzgf zjy`k3j}8^r_tefKSKX_w7uiqzt|<=eV4I>^Vw*=`yfgSZK|9iw%*h$};)Q7u>v1DS zwRGJc@GaLyJMysvw_jV_?9c zcIbX#p_6}v#97Bov@E}crU1Umx6>RI?tMJZH{tal8G**_L|?K9SKY>w|=qOM7LE zw6Tf({o#rIhNo7IhSA8$$r0NHmsT%z6$3Z7ZQHgdPo7Acm~i7m%I6jrZZFl;;>c4` zQBgWNd_Vr|e@fHVbY`B(!omVvel_q)YoR?qvg#{DmLETU6wb^}O_BHQa}N$?UR+wj z^ODXNTXHg7;s{jw;j__ZZY#j70qojh%(AUHX1n3|=QG>Er+s8b7|-#DOG){f$C#a) zdqqRzbulGgfp+7@m-+dvf9-adGxQ0DUZSF=X4<*40@J8Hvyi3EQ|=&+(^eB9db6m@ zJQohRoiAf!w`&hvaiDZ|ntkP`GBPrXE-Vax_f8TTN+qbUcfhB<-3X%3s|{hV{sbTH z>$J3T5CLc-W5RuTQky(}05;HgCQG2NXyHte^!C(?-krsappQ9vsIV9ys7p zQzIY3pN#;#qv!W;MQv@YhBVyx? z#IQ8tXdr zY+tW7{|sJSf@l7E2rWwojvZ@wu22YF4OT77+qYw}I=BM@)|0SbLdBef0KrZ{iE3*y$gW>)ls{y|ZZiBbo9-2LOv=(xCh;G+*5 zI6%4}C)WcM_u;1{rLOKh;9KjPYo2SJqO5As3VfMmw0gkix={E|QM(jb#0ya~GqY1O z-}J?pKWxOhN0xDMd86Dm3c5Vdgw_e9?mLc-YjgX)bCAhDEQQgV5uKLy7<^2m+p>fH@-Xsd zzUz|6_U+qkY;DI;@3XWstVPdhU1Q@XxLhQm{#ShcdgITJ_ZxGjG&?&x(E>{knlIK2 zjsoQKYZ>ZqT1`*E)KLC1hO}kN791|?iQh7yR6s!RU7_N<^xVDEybqs)+(kSX;5@8@ zS7p;{h$N54_4wwX@B8_aCyeq@;?*d~G~Qgjn~_W-dFj$-j=+a+-ySvZ$a{~9P94IC zRoAwiJ0C_xv4NJ~d&0cqDiYl(u(lg#T*`kpmOEm-h_98g<1{*F|M^;9;Qh~6qT~*Y z4qr-l`Tlv^fy0z@|GY^#+UMVQZM6C4O)Ih#_MT4OmxTDkD z5AdF-VTo;HZ=WmMg{nLdL0Cm)J5m>#aE_tNp{=crl;=2o7{$XrbP=Po3a({$JQe84 zf9~K*lmjY`odVaXBvuyTFJy-~WdhzQSjDk-EC&|g3`wiX>o#pPwJ!l4^?jdnmEnwXq1R3oxNZ$Q^BMmC!IN@F<7ZiHScTm)Vbq8h&MI z34iEnUmD^4__69$41bxL&I(IBw?ouvh6m(P`Qjg^imEDoY~8r{cpUb7I54>JJQF>B z{`AtR!m*sZ7ntnYu^R&KE`$ zNLLA@NB{of(-k^Y2J3#nV0T_iLt`g#1W1c81Pkw>hZo&DAt9k|44Uc2Xm9T?EBmXZ zR)+`?o8HycHAO4;@#DwzkPF_Sd*RQYKl=!H34`7>p>PNA1^=X7w2L4ZJCVK>JDtGQ zH$$^nvmH0eU%wT*^{HD1g`$XV(|ftOCvfC=wDTAf8Y;kn`j&l1H}3^B(E)lBXgeU_ zw3!(!-S+`6CSU=aglhc%Ey-wRM`x#^ipr6vZ-14h{?qX`5%xm>~gN{rZJn z0BDu84{(Hhr2#Os{`@vIH-B7W_3``n{ps_!Z8Nj6vFZ8w5*WwujPhjPBE@&m+yl>I vF7S*9&tmY52-k@v6CpApKBIO{|IgS{Q~%)ArbytnFyMkSS3j3^P6 - - - -# German Medical BERT - -This is a fine-tuned model on Medical domain for German language and based on German BERT. This model has only been trained to improve on target task (Masked Language Model). It can later be used to perform a downstream task of your needs, while I performed it for NTS-ICD-10 text classification task. - -## Overview -**Language model:** bert-base-german-cased - -**Language:** German - -**Fine-tuning:** Medical articles (diseases, symptoms, therapies, etc..) - -**Eval data:** NTS-ICD-10 dataset (Classification) - -**Infrastructure:** Gogle Colab - - -## Details -- We fine-tuned using Pytorch with Huggingface library on Colab GPU. -- With standard parameter settings for fine-tuning as mentioned in original BERT's paper. -- Although had to train for upto 25 epochs for classification. - -## Performance (Micro precision, recall and f1 score for multilabel code classification) - -|Models |P |R |F1 | -|:-------------- |:------|:------|:------| -|German BERT |86.04 |75.82 |80.60 | -|German MedBERT-256 |87.41 |77.97 |82.42 | -|German MedBERT-512 |87.75 |78.26 |82.73 | - -## Author -Manjil Shrestha: `shresthamanjil21 [at] gmail.com` - -Get in touch: -[LinkedIn](https://www.linkedin.com/in/manjil-shrestha-038527b4/) diff --git a/model_cards/spentaur/yelp/README.md b/model_cards/spentaur/yelp/README.md deleted file mode 100644 index aaa4bf1f4c..0000000000 --- a/model_cards/spentaur/yelp/README.md +++ /dev/null @@ -1,19 +0,0 @@ -# DistilBERT Yelp Review Sentiment -This model is used for sentiment analysis on english yelp reviews. -It is a DistilBERT model trained on 1 million reviews from the yelp open dataset. -It is a regression model, with outputs in the range of ~-2 to ~2. With -2 being 1 star and 2 being 5 stars. -It was trained using the [ktrain](https://github.com/amaiya/ktrain) because of it's ease of use. - -Example use: - -``` -tokenizer = AutoTokenizer.from_pretrained( - 'distilbert-base-uncased', use_fast=True) -model = TFAutoModelForSequenceClassification.from_pretrained( - "spentaur/yelp") - -review = "This place is great!" -input_ids = tokenizer.encode(review, return_tensors='tf') -pred = model(input_ids)[0][0][0].numpy() -# pred should === 1.9562385 -``` diff --git a/model_cards/squeezebert/squeezebert-mnli-headless/README.md b/model_cards/squeezebert/squeezebert-mnli-headless/README.md deleted file mode 100644 index 07c8ad1865..0000000000 --- a/model_cards/squeezebert/squeezebert-mnli-headless/README.md +++ /dev/null @@ -1,67 +0,0 @@ -language: en -license: bsd -datasets: -- bookcorpus -- wikipedia ---- - -# SqueezeBERT pretrained model - -This model, `squeezebert-mnli-headless`, has been pretrained for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective and finetuned on the [Multi-Genre Natural Language Inference (MNLI)](https://cims.nyu.edu/~sbowman/multinli/) dataset. This is a "headless" model with the final classification layer removed, and this will allow Transformers to automatically reinitialize the final classification layer before you begin finetuning on your data. -SqueezeBERT was introduced in [this paper](https://arxiv.org/abs/2006.11316). This model is case-insensitive. The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with [grouped convolutions](https://blog.yani.io/filter-group-tutorial/). -The authors found that SqueezeBERT is 4.3x faster than `bert-base-uncased` on a Google Pixel 3 smartphone. - - -## Pretraining - -### Pretraining data -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of thousands of unpublished books -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) - -### Pretraining procedure -The model is pretrained using the Masked Language Model (MLM) and Sentence Order Prediction (SOP) tasks. -(Author's note: If you decide to pretrain your own model, and you prefer to train with MLM only, that should work too.) - -From the SqueezeBERT paper: -> We pretrain SqueezeBERT from scratch (without distillation) using the [LAMB](https://arxiv.org/abs/1904.00962) optimizer, and we employ the hyperparameters recommended by the LAMB authors: a global batch size of 8192, a learning rate of 2.5e-3, and a warmup proportion of 0.28. Following the LAMB paper's recommendations, we pretrain for 56k steps with a maximum sequence length of 128 and then for 6k steps with a maximum sequence length of 512. - -## Finetuning - -The SqueezeBERT paper presents 2 approaches to finetuning the model: -- "finetuning without bells and whistles" -- after pretraining the SqueezeBERT model, finetune it on each GLUE task -- "finetuning with bells and whistles" -- after pretraining the SqueezeBERT model, finetune it on a MNLI with distillation from a teacher model. Then, use the MNLI-finetuned SqueezeBERT model as a student model to finetune on each of the other GLUE tasks (e.g. RTE, MRPC, …) with distillation from a task-specific teacher model. - -A detailed discussion of the hyperparameters used for finetuning is provided in the appendix of the [SqueezeBERT paper](https://arxiv.org/abs/2006.11316). -Note that finetuning SqueezeBERT with distillation is not yet implemented in this repo. If the author (Forrest Iandola - forrest.dnn@gmail.com) gets enough encouragement from the user community, he will add example code to Transformers for finetuning SqueezeBERT with distillation. - -This model, `squeezebert/squeezebert-mnli-headless`, is the "finetuned with bells and whistles" MNLI-finetuned SqueezeBERT model. In this particular model, we have removed the final classification layer -- in other words, it is "headless." We recommend using this model if you intend to finetune the model on your own data. Using this model means that your final layer will automatically be reinitialized when you start finetuning on your data. - -### How to finetune -To try finetuning SqueezeBERT on the [MRPC](https://www.microsoft.com/en-us/download/details.aspx?id=52398) text classification task, you can run the following command: -``` -./utils/download_glue_data.py - -python examples/text-classification/run_glue.py \ - --model_name_or_path squeezebert-base-headless \ - --task_name mrpc \ - --data_dir ./glue_data/MRPC \ - --output_dir ./models/squeezebert_mrpc \ - --overwrite_output_dir \ - --do_train \ - --do_eval \ - --num_train_epochs 10 \ - --learning_rate 3e-05 \ - --per_device_train_batch_size 16 \ - --save_steps 20000 - -``` - -## BibTeX entry and citation info -``` -@article{2020_SqueezeBERT, - author = {Forrest N. Iandola and Albert E. Shaw and Ravi Krishna and Kurt W. Keutzer}, - title = {{SqueezeBERT}: What can computer vision teach NLP about efficient neural networks?}, - journal = {arXiv:2006.11316}, - year = {2020} -} -``` diff --git a/model_cards/squeezebert/squeezebert-mnli/README.md b/model_cards/squeezebert/squeezebert-mnli/README.md deleted file mode 100644 index 28d910dc44..0000000000 --- a/model_cards/squeezebert/squeezebert-mnli/README.md +++ /dev/null @@ -1,67 +0,0 @@ -language: en -license: bsd -datasets: -- bookcorpus -- wikipedia ---- - -# SqueezeBERT pretrained model - -This model, `squeezebert-mnli`, has been pretrained for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective and finetuned on the [Multi-Genre Natural Language Inference (MNLI)](https://cims.nyu.edu/~sbowman/multinli/) dataset. -SqueezeBERT was introduced in [this paper](https://arxiv.org/abs/2006.11316). This model is case-insensitive. The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with [grouped convolutions](https://blog.yani.io/filter-group-tutorial/). -The authors found that SqueezeBERT is 4.3x faster than `bert-base-uncased` on a Google Pixel 3 smartphone. - - -## Pretraining - -### Pretraining data -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of thousands of unpublished books -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) - -### Pretraining procedure -The model is pretrained using the Masked Language Model (MLM) and Sentence Order Prediction (SOP) tasks. -(Author's note: If you decide to pretrain your own model, and you prefer to train with MLM only, that should work too.) - -From the SqueezeBERT paper: -> We pretrain SqueezeBERT from scratch (without distillation) using the [LAMB](https://arxiv.org/abs/1904.00962) optimizer, and we employ the hyperparameters recommended by the LAMB authors: a global batch size of 8192, a learning rate of 2.5e-3, and a warmup proportion of 0.28. Following the LAMB paper's recommendations, we pretrain for 56k steps with a maximum sequence length of 128 and then for 6k steps with a maximum sequence length of 512. - -## Finetuning - -The SqueezeBERT paper presents 2 approaches to finetuning the model: -- "finetuning without bells and whistles" -- after pretraining the SqueezeBERT model, finetune it on each GLUE task -- "finetuning with bells and whistles" -- after pretraining the SqueezeBERT model, finetune it on a MNLI with distillation from a teacher model. Then, use the MNLI-finetuned SqueezeBERT model as a student model to finetune on each of the other GLUE tasks (e.g. RTE, MRPC, …) with distillation from a task-specific teacher model. - -A detailed discussion of the hyperparameters used for finetuning is provided in the appendix of the [SqueezeBERT paper](https://arxiv.org/abs/2006.11316). -Note that finetuning SqueezeBERT with distillation is not yet implemented in this repo. If the author (Forrest Iandola - forrest.dnn@gmail.com) gets enough encouragement from the user community, he will add example code to Transformers for finetuning SqueezeBERT with distillation. - -This model, `squeezebert/squeezebert-mnli`, is the "trained with bells and whistles" MNLI-finetuned SqueezeBERT model. - -### How to finetune -To try finetuning SqueezeBERT on the [MRPC](https://www.microsoft.com/en-us/download/details.aspx?id=52398) text classification task, you can run the following command: -``` -./utils/download_glue_data.py - -python examples/text-classification/run_glue.py \ - --model_name_or_path squeezebert-base-headless \ - --task_name mrpc \ - --data_dir ./glue_data/MRPC \ - --output_dir ./models/squeezebert_mrpc \ - --overwrite_output_dir \ - --do_train \ - --do_eval \ - --num_train_epochs 10 \ - --learning_rate 3e-05 \ - --per_device_train_batch_size 16 \ - --save_steps 20000 - -``` - -## BibTeX entry and citation info -``` -@article{2020_SqueezeBERT, - author = {Forrest N. Iandola and Albert E. Shaw and Ravi Krishna and Kurt W. Keutzer}, - title = {{SqueezeBERT}: What can computer vision teach NLP about efficient neural networks?}, - journal = {arXiv:2006.11316}, - year = {2020} -} -``` diff --git a/model_cards/squeezebert/squeezebert-uncased/README.md b/model_cards/squeezebert/squeezebert-uncased/README.md deleted file mode 100644 index c1a38cab0b..0000000000 --- a/model_cards/squeezebert/squeezebert-uncased/README.md +++ /dev/null @@ -1,67 +0,0 @@ -language: en -license: bsd -datasets: -- bookcorpus -- wikipedia ---- - -# SqueezeBERT pretrained model - -This model, `squeezebert-uncased`, is a pretrained model for the English language using a masked language modeling (MLM) and Sentence Order Prediction (SOP) objective. -SqueezeBERT was introduced in [this paper](https://arxiv.org/abs/2006.11316). This model is case-insensitive. The model architecture is similar to BERT-base, but with the pointwise fully-connected layers replaced with [grouped convolutions](https://blog.yani.io/filter-group-tutorial/). -The authors found that SqueezeBERT is 4.3x faster than `bert-base-uncased` on a Google Pixel 3 smartphone. - - -## Pretraining - -### Pretraining data -- [BookCorpus](https://yknzhu.wixsite.com/mbweb), a dataset consisting of thousands of unpublished books -- [English Wikipedia](https://en.wikipedia.org/wiki/English_Wikipedia) - -### Pretraining procedure -The model is pretrained using the Masked Language Model (MLM) and Sentence Order Prediction (SOP) tasks. -(Author's note: If you decide to pretrain your own model, and you prefer to train with MLM only, that should work too.) - -From the SqueezeBERT paper: -> We pretrain SqueezeBERT from scratch (without distillation) using the [LAMB](https://arxiv.org/abs/1904.00962) optimizer, and we employ the hyperparameters recommended by the LAMB authors: a global batch size of 8192, a learning rate of 2.5e-3, and a warmup proportion of 0.28. Following the LAMB paper's recommendations, we pretrain for 56k steps with a maximum sequence length of 128 and then for 6k steps with a maximum sequence length of 512. - -## Finetuning - -The SqueezeBERT paper results from 2 approaches to finetuning the model: -- "finetuning without bells and whistles" -- after pretraining the SqueezeBERT model, finetune it on each GLUE task -- "finetuning with bells and whistles" -- after pretraining the SqueezeBERT model, finetune it on a MNLI with distillation from a teacher model. Then, use the MNLI-finetuned SqueezeBERT model as a student model to finetune on each of the other GLUE tasks (e.g. RTE, MRPC, …) with distillation from a task-specific teacher model. - -A detailed discussion of the hyperparameters used for finetuning is provided in the appendix of the [SqueezeBERT paper](https://arxiv.org/abs/2006.11316). -Note that finetuning SqueezeBERT with distillation is not yet implemented in this repo. If the author (Forrest Iandola - forrest.dnn@gmail.com) gets enough encouragement from the user community, he will add example code to Transformers for finetuning SqueezeBERT with distillation. - -This model, `squeezebert/squeezebert-uncased`, has been pretrained but not finetuned. For most text classification tasks, we recommend using squeezebert-mnli-headless as a starting point. - -### How to finetune -To try finetuning SqueezeBERT on the [MRPC](https://www.microsoft.com/en-us/download/details.aspx?id=52398) text classification task, you can run the following command: -``` -./utils/download_glue_data.py - -python examples/text-classification/run_glue.py \ - --model_name_or_path squeezebert-base-headless \ - --task_name mrpc \ - --data_dir ./glue_data/MRPC \ - --output_dir ./models/squeezebert_mrpc \ - --overwrite_output_dir \ - --do_train \ - --do_eval \ - --num_train_epochs 10 \ - --learning_rate 3e-05 \ - --per_device_train_batch_size 16 \ - --save_steps 20000 - -``` - -## BibTeX entry and citation info -``` -@article{2020_SqueezeBERT, - author = {Forrest N. Iandola and Albert E. Shaw and Ravi Krishna and Kurt W. Keutzer}, - title = {{SqueezeBERT}: What can computer vision teach NLP about efficient neural networks?}, - journal = {arXiv:2006.11316}, - year = {2020} -} -``` diff --git a/model_cards/stas/tiny-wmt19-en-de/README.md b/model_cards/stas/tiny-wmt19-en-de/README.md deleted file mode 100644 index e60d2bc01f..0000000000 --- a/model_cards/stas/tiny-wmt19-en-de/README.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -language: -- en -- de -thumbnail: -tags: -- wmt19 -- testing -license: apache-2.0 -datasets: -- wmt19 -metrics: -- bleu ---- - -# Tiny FSMT - -This is a tiny model that is used in the `transformers` test suite. It doesn't do anything useful, other than testing that `FSMT` works. diff --git a/model_cards/stevhliu/astroGPT/README.md b/model_cards/stevhliu/astroGPT/README.md deleted file mode 100644 index f2f2a9e510..0000000000 --- a/model_cards/stevhliu/astroGPT/README.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -language: "en" -thumbnail: "https://raw.githubusercontent.com/stevhliu/satsuma/master/images/astroGPT-thumbnail.png" -widget: -- text: "Jan 18, 2020" -- text: "Feb 14, 2020" -- text: "Jul 04, 2020" ---- - -# astroGPT 🪐 - -## Model description - -This is a GPT-2 model fine-tuned on Western zodiac signs. For more information about GPT-2, take a look at 🤗 Hugging Face's GPT-2 [model card](https://huggingface.co/gpt2). You can use astroGPT to generate a daily horoscope by entering the current date. - -## How to use - -To use this model, simply enter the current date like so `Mon DD, YEAR`: - -```python -from transformers import AutoTokenizer, AutoModelWithLMHead - -tokenizer = AutoTokenizer.from_pretrained("stevhliu/astroGPT") -model = AutoModelWithLMHead.from_pretrained("stevhliu/astroGPT") - -input_ids = tokenizer.encode('Sep 03, 2020', return_tensors='pt').to('cuda') - -sample_output = model.generate(input_ids, - do_sample=True, - max_length=75, - top_k=20, - top_p=0.97) - -print(sample_output) -``` - -## Limitations and bias - -astroGPT inherits the same biases that affect GPT-2 as a result of training on a lot of non-neutral content on the internet. The model does not currently support zodiac sign-specific generation and only returns a general horoscope. While the generated text may occasionally mention a specific zodiac sign, this is due to how the horoscopes were originally written by it's human authors. - -## Data - -The data was scraped from [Horoscope.com](https://www.horoscope.com/us/index.aspx) and trained on 4.7MB of text. The text was collected from four categories (daily, love, wellness, career) and span from 09/01/19 to 08/01/2020. The archives only store horoscopes dating a year back from the current date. - -## Training and results - -The text was tokenized using the fast GPT-2 BPE [tokenizer](https://huggingface.co/transformers/model_doc/gpt2.html#gpt2tokenizerfast). It has a vocabulary size of 50,257 and sequence length of 1024 tokens. The model was trained with on one of Google Colaboratory's GPU's for approximately 2.5 hrs with [fastai's](https://docs.fast.ai/) learning rate finder, discriminative learning rates and 1cycle policy. See table below for a quick summary of the training procedure and results. - -| dataset size | epochs | lr | training time | train_loss | valid_loss | perplexity | -|:-------------:|:------:|:-----------------:|:-------------:|:----------:|:----------:|:----------:| -| 5.9MB |32 | slice(1e-7,1e-5) | 2.5 hrs | 2.657170 | 2.642387 | 14.046692 | diff --git a/model_cards/surajp/RoBERTa-hindi-guj-san/README.md b/model_cards/surajp/RoBERTa-hindi-guj-san/README.md deleted file mode 100644 index 4d437d6e7f..0000000000 --- a/model_cards/surajp/RoBERTa-hindi-guj-san/README.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -language: -- hi -- sa -- gu -tags: -- Indic -license: mit -datasets: -- Wikipedia (Hindi, Sanskrit, Gujarati) -metrics: -- perplexity ---- - -# RoBERTa-hindi-guj-san - -## Model description - -Multillingual RoBERTa like model trained on Wikipedia articles of Hindi, Sanskrit, Gujarati languages. The tokenizer was trained on combined text. -However, Hindi text was used to pre-train the model and then it was fine-tuned on Sanskrit and Gujarati Text combined hoping that pre-training with Hindi -will help the model learn similar languages. - -### Configuration - -| Parameter | Value | -|---|---| -| `hidden_size` | 768 | -| `num_attention_heads` | 12 | -| `num_hidden_layers` | 6 | -| `vocab_size` | 30522 | -|`model_type`|`roberta`| - -## Intended uses & limitations - -#### How to use - -```python -# Example usage -from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline - -tokenizer = AutoTokenizer.from_pretrained("surajp/RoBERTa-hindi-guj-san") -model = AutoModelWithLMHead.from_pretrained("surajp/RoBERTa-hindi-guj-san") - -fill_mask = pipeline( - "fill-mask", - model=model, - tokenizer=tokenizer -) - -# Sanskrit: इयं भाषा न केवलं भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते। -# Hindi: अगर आप अब अभ्यास नहीं करते हो तो आप अपने परीक्षा में मूर्खतापूर्ण गलतियाँ करोगे। -# Gujarati: ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો હતો. -fill_mask("ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો હતો.") - -''' -Output: --------- -[ -{'score': 0.07849744707345963, 'sequence': ' ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો જ હતો.', 'token': 390}, -{'score': 0.06273336708545685, 'sequence': ' ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો ન હતો.', 'token': 478}, -{'score': 0.05160355195403099, 'sequence': ' ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો થઇ હતો.', 'token': 2075}, -{'score': 0.04751499369740486, 'sequence': ' ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો એક હતો.', 'token': 600}, -{'score': 0.03788900747895241, 'sequence': ' ગુજરાતમાં ૧૯મી માર્ચ સુધી કોઈ સકારાત્મક (પોઝીટીવ) રીપોર્ટ આવ્યો પણ હતો.', 'token': 840} -] - -``` - -## Training data - -Cleaned wikipedia articles in Hindi, Sanskrit and Gujarati on Kaggle. It contains training as well as evaluation text. -Used in [iNLTK](https://github.com/goru001/inltk) - -- [Hindi](https://www.kaggle.com/disisbig/hindi-wikipedia-articles-172k) -- [Gujarati](https://www.kaggle.com/disisbig/gujarati-wikipedia-articles) -- [Sanskrit](https://www.kaggle.com/disisbig/sanskrit-wikipedia-articles) - -## Training procedure - -- On TPU (using `xla_spawn.py`) -- For language modelling -- Iteratively increasing `--block_size` from 128 to 256 over epochs -- Tokenizer trained on combined text -- Pre-training with Hindi and fine-tuning on Sanskrit and Gujarati texts - -``` ---model_type distillroberta-base \ ---model_name_or_path "/content/SanHiGujBERTa" \ ---mlm_probability 0.20 \ ---line_by_line \ ---save_total_limit 2 \ ---per_device_train_batch_size 128 \ ---per_device_eval_batch_size 128 \ ---num_train_epochs 5 \ ---block_size 256 \ ---seed 108 \ ---overwrite_output_dir \ -``` - -## Eval results - -perplexity = 2.920005983224673 - - - -> Created by [Suraj Parmar/@parmarsuraj99](https://twitter.com/parmarsuraj99) | [LinkedIn](https://www.linkedin.com/in/parmarsuraj99/) - -> Made with ♥ in India diff --git a/model_cards/surajp/SanBERTa/README.md b/model_cards/surajp/SanBERTa/README.md deleted file mode 100644 index 89d4b138b2..0000000000 --- a/model_cards/surajp/SanBERTa/README.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -language: sa ---- - -# RoBERTa trained on Sanskrit (SanBERTa) - -**Mode size** (after training): **340MB** - -### Dataset: - -[Wikipedia articles](https://www.kaggle.com/disisbig/sanskrit-wikipedia-articles) (used in [iNLTK](https://github.com/goru001/nlp-for-sanskrit)). -It contains evaluation set. - -[Sanskrit scraps from CLTK](http://cltk.org/) - -### Configuration - -| Parameter | Value | -|---|---| -| `num_attention_heads` | 12 | -| `num_hidden_layers` | 6 | -| `hidden_size` | 768 | -| `vocab_size` | 29407 | - -### Training : -- On TPU -- For language modelling -- Iteratively increasing `--block_size` from 128 to 256 over epochs - -### Evaluation - -|Metric| # Value | -|---|---| -|Perplexity (`block_size=256`)|4.04| - -## Example of usage: - -### For Embeddings - -``` - -tokenizer = AutoTokenizer.from_pretrained("surajp/SanBERTa") -model = RobertaModel.from_pretrained("surajp/SanBERTa") - -op = tokenizer.encode("इयं भाषा न केवलं भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते।", return_tensors="pt") -ps = model(op) -ps[0].shape - -``` -``` -''' -Output: --------- -torch.Size([1, 47, 768]) - -``` - - -### For \ Prediction - -``` -from transformers import pipeline - -fill_mask = pipeline( - "fill-mask", - model="surajp/SanBERTa", - tokenizer="surajp/SanBERTa" -) - -## इयं भाषा न केवलं भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते। -fill_mask("इयं भाषा न केवल भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते।") - -ps = model(torch.tensor(enc).unsqueeze(1)) -print(ps[0].shape) -``` -``` -''' -Output: --------- -[{'score': 0.7516744136810303, - 'sequence': ' इयं भाषा न केवलं भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते।', - 'token': 280, - 'token_str': 'à¤Ĥ'}, - {'score': 0.06230105459690094, - 'sequence': ' इयं भाषा न केवली भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते।', - 'token': 289, - 'token_str': 'à¥Ģ'}, - {'score': 0.055410224944353104, - 'sequence': ' इयं भाषा न केवला भारतस्य अपि तु विश्वस्य प्राचीनतमा भाषा इति मन्यते।', - 'token': 265, - 'token_str': 'ा'}, - ...] -``` - -### It works!! 🎉 🎉 🎉 - -> Created by [Suraj Parmar/@parmarsuraj99](https://twitter.com/parmarsuraj99) | [LinkedIn](https://www.linkedin.com/in/parmarsuraj99/) - -> Made with ♥ in India diff --git a/model_cards/surajp/albert-base-sanskrit/README.md b/model_cards/surajp/albert-base-sanskrit/README.md deleted file mode 100644 index 0a24bfa033..0000000000 --- a/model_cards/surajp/albert-base-sanskrit/README.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -language: sa ---- - - -# ALBERT-base-Sanskrit - - -Explaination Notebook Colab: [SanskritALBERT.ipynb](https://colab.research.google.com/github/parmarsuraj99/suraj-parmar/blob/master/_notebooks/2020-05-02-SanskritALBERT.ipynb) - -Size of the model is **46MB** - -Example of usage: - -``` -tokenizer = AutoTokenizer.from_pretrained("surajp/albert-base-sanskrit") -model = AutoModel.from_pretrained("surajp/albert-base-sanskrit") - -enc=tokenizer.encode("ॐ सर्वे भवन्तु सुखिनः सर्वे सन्तु निरामयाः । सर्वे भद्राणि पश्यन्तु मा कश्चिद्दुःखभाग्भवेत् । ॐ शान्तिः शान्तिः शान्तिः ॥") -print(tokenizer.decode(enc)) - -ps = model(torch.tensor(enc).unsqueeze(1)) -print(ps[0].shape) -``` -``` -''' -Output: --------- -[CLS] ॐ सर्वे भवन्तु सुखिनः सर्वे सन्तु निरामयाः । सर्वे भद्राणि पश्यन्तु मा कश्चिद्दुःखभाग्भवेत् । ॐ शान्तिः शान्तिः शान्तिः ॥[SEP] -torch.Size([28, 1, 768]) -``` - - -> Created by [Suraj Parmar/@parmarsuraj99](https://twitter.com/parmarsuraj99) - -> Made with ♥ in India diff --git a/model_cards/t5-11b-README.md b/model_cards/t5-11b-README.md deleted file mode 100644 index 131667018c..0000000000 --- a/model_cards/t5-11b-README.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -language: en -datasets: -- c4 -tags: -- summarization -- translation - -license: apache-2.0 -inference: false ---- - -## Disclaimer - -**Before `transformers` v3.5.0**, due do its immense size, `t5-11b` required some special treatment. -If you're using transformers `<= v3.4.0`, `t5-11b` should be loaded with flag `use_cdn` set to `False` as follows: - -```python -t5 = transformers.T5ForConditionalGeneration.from_pretrained('t5-11b', use_cdn = False) -``` - -Secondly, a single GPU will most likely not have enough memory to even load the model into memory as the weights alone amount to over 40 GB. -Model parallelism has to be used here to overcome this problem as is explained in this [PR](https://github.com/huggingface/transformers/pull/3578). - -## [Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) - -Pretraining Dataset: [C4](https://huggingface.co/datasets/c4) - -Other Community Checkpoints: [here](https://huggingface.co/models?search=t5) - -Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) - -Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* - -## Abstract - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - diff --git a/model_cards/t5-3b-README.md b/model_cards/t5-3b-README.md deleted file mode 100644 index ff90f77056..0000000000 --- a/model_cards/t5-3b-README.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -language: en -datasets: -- c4 -tags: -- summarization -- translation - -license: apache-2.0 ---- - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) - -Pretraining Dataset: [C4](https://huggingface.co/datasets/c4) - -Other Community Checkpoints: [here](https://huggingface.co/models?search=t5) - -Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) - -Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* - - -## Abstract - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - diff --git a/model_cards/t5-base-README.md b/model_cards/t5-base-README.md deleted file mode 100644 index ff90f77056..0000000000 --- a/model_cards/t5-base-README.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -language: en -datasets: -- c4 -tags: -- summarization -- translation - -license: apache-2.0 ---- - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) - -Pretraining Dataset: [C4](https://huggingface.co/datasets/c4) - -Other Community Checkpoints: [here](https://huggingface.co/models?search=t5) - -Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) - -Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* - - -## Abstract - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - diff --git a/model_cards/t5-large-README.md b/model_cards/t5-large-README.md deleted file mode 100644 index ff90f77056..0000000000 --- a/model_cards/t5-large-README.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -language: en -datasets: -- c4 -tags: -- summarization -- translation - -license: apache-2.0 ---- - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) - -Pretraining Dataset: [C4](https://huggingface.co/datasets/c4) - -Other Community Checkpoints: [here](https://huggingface.co/models?search=t5) - -Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) - -Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* - - -## Abstract - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - diff --git a/model_cards/t5-small-README.md b/model_cards/t5-small-README.md deleted file mode 100644 index ff90f77056..0000000000 --- a/model_cards/t5-small-README.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -language: en -datasets: -- c4 -tags: -- summarization -- translation - -license: apache-2.0 ---- - -[Google's T5](https://ai.googleblog.com/2020/02/exploring-transfer-learning-with-t5.html) - -Pretraining Dataset: [C4](https://huggingface.co/datasets/c4) - -Other Community Checkpoints: [here](https://huggingface.co/models?search=t5) - -Paper: [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/pdf/1910.10683.pdf) - -Authors: *Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu* - - -## Abstract - -Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts every language problem into a text-to-text format. Our systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new “Colossal Clean Crawled Corpus”, we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our dataset, pre-trained models, and code. - -![model image](https://camo.githubusercontent.com/623b4dea0b653f2ad3f36c71ebfe749a677ac0a1/68747470733a2f2f6d69726f2e6d656469756d2e636f6d2f6d61782f343030362f312a44304a31674e51663876727255704b657944387750412e706e67) - diff --git a/model_cards/tartuNLP/EstBERT/README.md b/model_cards/tartuNLP/EstBERT/README.md deleted file mode 100644 index ab042f7cac..0000000000 --- a/model_cards/tartuNLP/EstBERT/README.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -language: et ---- -# EstBERT - - -### What's this? -The EstBERT model is a pretrained BERTBase model exclusively trained on Estonian cased corpus on both 128 and 512 sequence length of data. - -### How to use? -You can use the model transformer library both in tensorflow and pytorch version. -``` -from transformers import AutoTokenizer, AutoModelForMaskedLM -tokenizer = AutoTokenizer.from_pretrained("tartuNLP/EstBERT") -model = AutoModelForMaskedLM.from_pretrained("tartuNLP/EstBERT") -``` -You can also download the pretrained model from here, [EstBERT_128]() [EstBERT_512]() -#### Dataset used to train the model -The EstBERT model is trained both on 128 and 512 sequence length of data. For training the EstBERT we used the [Estonian National Corpus 2017](https://metashare.ut.ee/repository/browse/estonian-national-corpus-2017/b616ceda30ce11e8a6e4005056b40024880158b577154c01bd3d3fcfc9b762b3/), which was the largest Estonian language corpus available at the time. It consists of four sub-corpora: Estonian Reference Corpus 1990-2008, Estonian Web Corpus 2013, Estonian Web Corpus 2017 and Estonian Wikipedia Corpus 2017. - -### Why would I use? -Overall EstBERT performs better in parts of speech (POS), name entity recognition (NER), rubric, and sentiment classification tasks compared to mBERT and XLM-RoBERTa. The comparative results can be found below; - -|Model |UPOS |XPOS |Morph |bf UPOS |bf XPOS |Morph | -|--------------|----------------------------|-------------|-------------|-------------|----------------------------|----------------------------| -| EstBERT | **_97.89_** | **98.40** | **96.93** | **97.84** | **_98.43_** | **_96.80_** | -| mBERT | 97.42 | 98.06 | 96.24 | 97.43 | 98.13 | 96.13 | -| XLM-RoBERTa | 97.78 | 98.36 | 96.53 | 97.80 | 98.40 | 96.69 | - - -|Model|Rubric128 |Sentiment128 | Rubric128 |Sentiment512 | -|-------------------|----------------------------|--------------------|-----------------------------------------------|----------------------------| -| EstBERT | **_81.70_** | 74.36 | **80.96** | 74.50 | -| mBERT | 75.67 | 70.23 | 74.94 | 69.52 | -| XLM\-RoBERTa | 80.34 | **74.50** | 78.62 | **_76.07_**| - -|Model |Precicion128 |Recall128 |F1-Score128 |Precision512 |Recall512 |F1-Score512 | -|--------------|----------------|----------------------------|----------------------------|----------------------------|-------------|----------------| -| EstBERT | **88.42** | 90.38 |**_89.39_** | 88.35 | 89.74 | 89.04 | -| mBERT | 85.88 | 87.09 | 86.51 |**_88.47_** | 88.28 | 88.37 | -| XLM\-RoBERTa | 87.55 |**_91.19_** | 89.34 | 87.50 | **90.76** | **89.10** | diff --git a/model_cards/tblard/tf-allocine/README.md b/model_cards/tblard/tf-allocine/README.md deleted file mode 100644 index bb5cd6b282..0000000000 --- a/model_cards/tblard/tf-allocine/README.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -language: fr ---- - -# tf-allociné - -A french sentiment analysis model, based on [CamemBERT](https://camembert-model.fr/), and finetuned on a large-scale dataset scraped from [Allociné.fr](http://www.allocine.fr/) user reviews. - -## Results - -| Validation Accuracy | Validation F1-Score | Test Accuracy | Test F1-Score | -|--------------------:| -------------------:| -------------:|--------------:| -| 97.39 | 97.36 | 97.44 | 97.34 | - -The dataset and the evaluation code are available on [this repo](https://github.com/TheophileBlard/french-sentiment-analysis-with-bert). - -## Usage - -```python -from transformers import AutoTokenizer, TFAutoModelForSequenceClassification -from transformers import pipeline - -tokenizer = AutoTokenizer.from_pretrained("tblard/tf-allocine") -model = TFAutoModelForSequenceClassification.from_pretrained("tblard/tf-allocine") - -nlp = pipeline('sentiment-analysis', model=model, tokenizer=tokenizer) - -print(nlp("Alad'2 est clairement le meilleur film de l'année 2018.")) # POSITIVE -print(nlp("Juste whoaaahouuu !")) # POSITIVE -print(nlp("NUL...A...CHIER ! FIN DE TRANSMISSION.")) # NEGATIVE -print(nlp("Je m'attendais à mieux de la part de Franck Dubosc !")) # NEGATIVE -``` - -## Author - -Théophile Blard – :email: theophile.blard@gmail.com - -If you use this work (code, model or dataset), please cite as: - -> Théophile Blard, French sentiment analysis with BERT, (2020), GitHub repository, diff --git a/model_cards/tuner007/pegasus_paraphrase/README.md b/model_cards/tuner007/pegasus_paraphrase/README.md deleted file mode 100644 index 311d8de61c..0000000000 --- a/model_cards/tuner007/pegasus_paraphrase/README.md +++ /dev/null @@ -1,53 +0,0 @@ -# Pegasus for Paraphrasing -Pegasus model fine-tuned for paraphrasing - -## Model in Action 🚀 -``` -import torch -from transformers import PegasusForConditionalGeneration, PegasusTokenizer -model_name = 'tuner007/pegasus_paraphrase' -torch_device = 'cuda' if torch.cuda.is_available() else 'cpu' -tokenizer = PegasusTokenizer.from_pretrained(model_name) -model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device) - -def get_response(input_text,num_return_sequences): - batch = tokenizer.prepare_seq2seq_batch([input_text],truncation=True,padding='longest',max_length=60, return_tensors="pt").to(torch_device) - translated = model.generate(**batch,max_length=60,num_beams=10, num_return_sequences=num_return_sequences, temperature=1.5) - tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True) - return tgt_text -``` -#### Example 1: -``` -context = "The ultimate test of your knowledge is your capacity to convey it to another." -get_response(context,10) -# output: -['The test of your knowledge is your ability to convey it.', - 'The ability to convey your knowledge is the ultimate test of your knowledge.', - 'The ability to convey your knowledge is the most important test of your knowledge.', - 'Your capacity to convey your knowledge is the ultimate test of it.', - 'The test of your knowledge is your ability to communicate it.', - 'Your capacity to convey your knowledge is the ultimate test of your knowledge.', - 'Your capacity to convey your knowledge to another is the ultimate test of your knowledge.', - 'Your capacity to convey your knowledge is the most important test of your knowledge.', - 'The test of your knowledge is how well you can convey it.', - 'Your capacity to convey your knowledge is the ultimate test.'] -``` -#### Example 2: Question paraphrasing (was not trained on quora dataset) -``` -context = "Which course should I take to get started in data science?" -get_response(context,10) -# output: -['Which data science course should I take?', - 'Which data science course should I take first?', - 'Should I take a data science course?', - 'Which data science class should I take?', - 'Which data science course should I attend?', - 'I want to get started in data science.', - 'Which data science course should I enroll in?', - 'Which data science course is right for me?', - 'Which data science course is best for me?', - 'Which course should I take to get started?'] -``` - -> Created by Arpit Rajauria -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/arpit_rajauria) diff --git a/model_cards/tuner007/pegasus_qa/README.md b/model_cards/tuner007/pegasus_qa/README.md deleted file mode 100644 index bc9397225d..0000000000 --- a/model_cards/tuner007/pegasus_qa/README.md +++ /dev/null @@ -1,30 +0,0 @@ -# Pegasus for question-answering -Pegasus model fine-tuned for QA using text-to-text approach - -## Model in Action 🚀 -``` -import torch -from transformers import PegasusForConditionalGeneration, PegasusTokenizer -model_name = 'tuner007/pegasus_qa' -torch_device = 'cuda' if torch.cuda.is_available() else 'cpu' -tokenizer = PegasusTokenizer.from_pretrained(model_name) -model = PegasusForConditionalGeneration.from_pretrained(model_name).to(torch_device) - -def get_answer(question, context): - input_text = "question: %s text: %s" % (question,context) - batch = tokenizer.prepare_seq2seq_batch([input_text], truncation=True, padding='longest', return_tensors="pt").to(torch_device) - translated = model.generate(**batch) - tgt_text = tokenizer.batch_decode(translated, skip_special_tokens=True) - return tgt_text[0] -``` -#### Example: -``` -context = "PG&E stated it scheduled the blackouts in response to forecasts for high winds amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow." -question = "How many customers were affected by the shutoffs?" -get_answer(question, context) -# output: '800 thousand' -``` - - -> Created by Arpit Rajauria -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/arpit_rajauria) diff --git a/model_cards/tuner007/t5_abs_qa/README.md b/model_cards/tuner007/t5_abs_qa/README.md deleted file mode 100644 index efdbb5c938..0000000000 --- a/model_cards/tuner007/t5_abs_qa/README.md +++ /dev/null @@ -1,39 +0,0 @@ -# T5 for abstractive question-answering -This is T5-base model fine-tuned for abstractive QA using text-to-text approach - -## Model training -This model was trained on colab TPU with 35GB RAM for 2 epochs - -## Model in Action 🚀 -``` -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("tuner007/t5_abs_qa") -model = AutoModelWithLMHead.from_pretrained("tuner007/t5_abs_qa") -device = torch.device("cuda" if torch.cuda.is_available() else "cpu") -model = model.to(device) - -def get_answer(question, context): -input_text = "context: %s " % (context,question) -features = tokenizer([input_text], return_tensors='pt') -out = model.generate(input_ids=features['input_ids'].to(device), attention_mask=features['attention_mask'].to(device)) -return tokenizer.decode(out[0]) -``` -#### Example 1: Answer available -``` -context = "In Norse mythology, Valhalla is a majestic, enormous hall located in Asgard, ruled over by the god Odin." -question = "What is Valhalla?" -get_answer(question, context) -# output: 'It is a hall of worship ruled by Odin.' -``` -#### Example 2: Answer not available -``` -context = "In Norse mythology, Valhalla is a majestic, enormous hall located in Asgard, ruled over by the god Odin." -question = "What is Asgard?" -get_answer(question, context) -# output: 'No answer available in context.' -``` - - -> Created by Arpit Rajauria -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/arpit_rajauria) diff --git a/model_cards/twmkn9/albert-base-v2-squad2/README.md b/model_cards/twmkn9/albert-base-v2-squad2/README.md deleted file mode 100644 index 5b615c4704..0000000000 --- a/model_cards/twmkn9/albert-base-v2-squad2/README.md +++ /dev/null @@ -1,45 +0,0 @@ -This model is [ALBERT base v2](https://huggingface.co/albert-base-v2) trained on SQuAD v2 as: - -``` -export SQUAD_DIR=../../squad2 -python3 run_squad.py - --model_type albert - --model_name_or_path albert-base-v2 - --do_train - --do_eval - --overwrite_cache - --do_lower_case - --version_2_with_negative - --save_steps 100000 - --train_file $SQUAD_DIR/train-v2.0.json - --predict_file $SQUAD_DIR/dev-v2.0.json - --per_gpu_train_batch_size 8 - --num_train_epochs 3 - --learning_rate 3e-5 - --max_seq_length 384 - --doc_stride 128 - --output_dir ./tmp/albert_fine/ -``` - -Performance on a dev subset is close to the original paper: - -``` -Results: -{ - 'exact': 78.71010200723923, - 'f1': 81.89228117126069, - 'total': 6078, - 'HasAns_exact': 75.39518900343643, - 'HasAns_f1': 82.04167868004215, - 'HasAns_total': 2910, - 'NoAns_exact': 81.7550505050505, - 'NoAns_f1': 81.7550505050505, - 'NoAns_total': 3168, - 'best_exact': 78.72655478775913, - 'best_exact_thresh': 0.0, - 'best_f1': 81.90873395178066, - 'best_f1_thresh': 0.0 -} -``` - -We are hopeful this might save you time, energy, and compute. Cheers! \ No newline at end of file diff --git a/model_cards/twmkn9/bert-base-uncased-squad2/README.md b/model_cards/twmkn9/bert-base-uncased-squad2/README.md deleted file mode 100644 index 20bdf07512..0000000000 --- a/model_cards/twmkn9/bert-base-uncased-squad2/README.md +++ /dev/null @@ -1,45 +0,0 @@ -This model is [BERT base uncased](https://huggingface.co/bert-base-uncased) trained on SQuAD v2 as: - -``` -export SQUAD_DIR=../../squad2 -python3 run_squad.py - --model_type bert - --model_name_or_path bert-base-uncased - --do_train - --do_eval - --overwrite_cache - --do_lower_case - --version_2_with_negative - --save_steps 100000 - --train_file $SQUAD_DIR/train-v2.0.json - --predict_file $SQUAD_DIR/dev-v2.0.json - --per_gpu_train_batch_size 8 - --num_train_epochs 3 - --learning_rate 3e-5 - --max_seq_length 384 - --doc_stride 128 - --output_dir ./tmp/bert_fine_tuned/ -``` - -Performance on a dev subset is close to the original paper: - -``` -Results: -{ - 'exact': 72.35932872655479, - 'f1': 75.75355132564763, - 'total': 6078, - 'HasAns_exact': 74.29553264604812, - 'HasAns_f1': 81.38490892002987, - 'HasAns_total': 2910, - 'NoAns_exact': 70.58080808080808, - 'NoAns_f1': 70.58080808080808, - 'NoAns_total': 3168, - 'best_exact': 72.35932872655479, - 'best_exact_thresh': 0.0, - 'best_f1': 75.75355132564766, - 'best_f1_thresh': 0.0 -} -``` - -We are hopeful this might save you time, energy, and compute. Cheers! \ No newline at end of file diff --git a/model_cards/twmkn9/distilbert-base-uncased-squad2/README.md b/model_cards/twmkn9/distilbert-base-uncased-squad2/README.md deleted file mode 100644 index cb8542fb51..0000000000 --- a/model_cards/twmkn9/distilbert-base-uncased-squad2/README.md +++ /dev/null @@ -1,45 +0,0 @@ -This model is [Distilbert base uncased](https://huggingface.co/distilbert-base-uncased) trained on SQuAD v2 as: - -``` -export SQUAD_DIR=../../squad2 -python3 run_squad.py - --model_type distilbert - --model_name_or_path distilbert-base-uncased - --do_train - --do_eval - --overwrite_cache - --do_lower_case - --version_2_with_negative - --save_steps 100000 - --train_file $SQUAD_DIR/train-v2.0.json - --predict_file $SQUAD_DIR/dev-v2.0.json - --per_gpu_train_batch_size 8 - --num_train_epochs 3 - --learning_rate 3e-5 - --max_seq_length 384 - --doc_stride 128 - --output_dir ./tmp/distilbert_fine_tuned/ -``` - -Performance on a dev subset is close to the original paper: - -``` -Results: -{ - 'exact': 64.88976637051661, - 'f1': 68.1776176526635, - 'total': 6078, - 'HasAns_exact': 69.7594501718213, - 'HasAns_f1': 76.62665295288285, - 'HasAns_total': 2910, - 'NoAns_exact': 60.416666666666664, - 'NoAns_f1': 60.416666666666664, - 'NoAns_total': 3168, - 'best_exact': 64.88976637051661, - 'best_exact_thresh': 0.0, - 'best_f1': 68.17761765266337, - 'best_f1_thresh': 0.0 -} -``` - -We are hopeful this might save you time, energy, and compute. Cheers! \ No newline at end of file diff --git a/model_cards/twmkn9/distilroberta-base-squad2/README.md b/model_cards/twmkn9/distilroberta-base-squad2/README.md deleted file mode 100644 index c6b6569f65..0000000000 --- a/model_cards/twmkn9/distilroberta-base-squad2/README.md +++ /dev/null @@ -1,44 +0,0 @@ -This model is [Distilroberta base](https://huggingface.co/distilroberta-base) trained on SQuAD v2 as: - -``` -export SQUAD_DIR=../../squad2 -python3 run_squad.py - --model_type robberta - --model_name_or_path distilroberta-base - --do_train - --do_eval - --overwrite_cache - --do_lower_case - --version_2_with_negative - --save_steps 100000 - --train_file $SQUAD_DIR/train-v2.0.json - --predict_file $SQUAD_DIR/dev-v2.0.json - --per_gpu_train_batch_size 8 - --num_train_epochs 3 - --learning_rate 3e-5 - --max_seq_length 384 - --doc_stride 128 - --output_dir ./tmp/distilroberta_fine_tuned/ -``` - -Performance on a dev subset is close to the original paper: - -``` -Results: -{ - 'exact': 70.9279368213228, - 'f1': 74.60439802429168, - 'total': 6078, - 'HasAns_exact': 67.62886597938144, - 'HasAns_f1': 75.30774267754136, - 'HasAns_total': 2910, - 'NoAns_exact': 73.95833333333333, - 'NoAns_f1': 73.95833333333333, 'NoAns_total': 3168, - 'best_exact': 70.94438960184272, - 'best_exact_thresh': 0.0, - 'best_f1': 74.62085080481161, - 'best_f1_thresh': 0.0 -} -``` - -We are hopeful this might save you time, energy, and compute. Cheers! \ No newline at end of file diff --git a/model_cards/uer/chinese_roberta_L-2_H-128/README.md b/model_cards/uer/chinese_roberta_L-2_H-128/README.md deleted file mode 100644 index ca646e266f..0000000000 --- a/model_cards/uer/chinese_roberta_L-2_H-128/README.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -language: zh -datasets: -- CLUECorpus ---- - -# Chinese RoBERTa Miniatures - -## Model description - -This is the set of 24 Chinese RoBERTa models pre-trained by [UER-py](https://www.aclweb.org/anthology/D19-3041.pdf). - -You can download the 24 Chinese RoBERTa miniatures either from the [UER-py Github page](https://github.com/dbiir/UER-py/), or via HuggingFace from the links below: - -| |H=128|H=256|H=512|H=768| -|---|:---:|:---:|:---:|:---:| -| **L=2** |[**2/128 (BERT-Tiny)**][2_128]|[2/256]|[2/512]|[2/768]| -| **L=4** |[4/128]|[**4/256 (BERT-Mini)**]|[**4/512 (BERT-Small)**]|[4/768]| -| **L=6** |[6/128]|[6/256]|[6/512]|[6/768]| -| **L=8** |[8/128]|[8/256]|[**8/512 (BERT-Medium)**]|[8/768]| -| **L=10** |[10/128]|[10/256]|[10/512]|[10/768]| -| **L=12** |[12/128]|[12/256]|[12/512]|[**12/768 (BERT-Base)**]| - -## Training data - -CLUECorpus2020 and CLUECorpusSmall are used as training corpus. - -## Training procedure - -Training details can be found in [UER-py](https://github.com/dbiir/UER-py/). - -### BibTeX entry and citation info - -``` -@article{zhao2019uer, - title={UER: An Open-Source Toolkit for Pre-training Models}, - author={Zhao, Zhe and Chen, Hui and Zhang, Jinbin and Zhao, Xin and Liu, Tao and Lu, Wei and Chen, Xi and Deng, Haotang and Ju, Qi and Du, Xiaoyong}, - journal={EMNLP-IJCNLP 2019}, - pages={241}, - year={2019} -} -``` - -[2_128]: https://huggingface.co/uer/chinese_roberta_L-2_H-128 diff --git a/model_cards/uer/gpt2-chinese-couplet/README.md b/model_cards/uer/gpt2-chinese-couplet/README.md deleted file mode 100644 index 891d9e3b2c..0000000000 --- a/model_cards/uer/gpt2-chinese-couplet/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: zh -widget: -- text: "[CLS]国 色 天 香 ， 姹 紫 嫣 红 ， 碧 水 青 云 欣 共 赏 -" - - ---- - -# Chinese Couplet GPT2 Model - -## Model description - -The model is used to generate Chinese couplets. You can download the model either from the [GPT2-Chinese Github page](https://github.com/Morizeyao/GPT2-Chinese), or via HuggingFace from the link [gpt2-chinese-couplet][couplet]. - -Since the parameter skip_special_tokens is used in the pipelines.py, special tokens such as [SEP], [UNK] will be deleted, and the output results may not be neat. - -## How to use - -You can use the model directly with a pipeline for text generation: - -When the parameter skip_special_tokens is True: - -```python ->>> from transformers import BertTokenizer, GPT2LMHeadModel, TextGenerationPipeline ->>> from transformers import TextGenerationPipeline, ->>> tokenizer = BertTokenizer.from_pretrained("uer/gpt2-chinese-couplet") ->>> model = GPT2LMHeadModel.from_pretrained("uer/gpt2-chinese-couplet") ->>> text_generator = TextGenerationPipeline(model, tokenizer) ->>> text_generator("[CLS]丹 枫 江 冷 人 初 去 -", max_length=25, do_sample=True) - [{'generated_text': '[CLS]丹 枫 江 冷 人 初 去 - 黄 叶 声 从 天 外 来 阅 旗'}] -``` - -When the parameter skip_special_tokens is False: - -```python ->>> from transformers import BertTokenizer, GPT2LMHeadModel, TextGenerationPipeline ->>> from transformers import TextGenerationPipeline, ->>> tokenizer = BertTokenizer.from_pretrained("uer/gpt2-chinese-poem") ->>> model = GPT2LMHeadModel.from_pretrained("uer/gpt2-chinese-poem") ->>> text_generator = TextGenerationPipeline(model, tokenizer) ->>> text_generator("[CLS]丹 枫 江 冷 人 初 去 -", max_length=25, do_sample=True) - [{'generated_text': '[CLS]丹 枫 江 冷 人 初 去 - 黄 叶 声 我 酒 不 辞 [SEP] [SEP] [SEP] [SEP] [SEP] [SEP] [SEP] [SEP] [SEP]'}] -``` - -## Training data - -Contains 700,000 Chinese couplets collected by [couplet-clean-dataset](https://github.com/v-zich/couplet-clean-dataset). - -## Training procedure - -Models are pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud TI-ONE](https://cloud.tencent.com/product/tione/). We pre-train 25,000 steps with a sequence length of 64. - -``` -python3 preprocess.py --corpus_path corpora/couplet.txt \ - --vocab_path models/google_zh_vocab.txt \ - --dataset_path couplet.pt --processes_num 16 \ - --seq_length 64 --target lm -``` - -``` -python3 pretrain.py --dataset_path couplet.pt \ - --vocab_path models/google_zh_vocab.txt \ - --output_model_path models/couplet_gpt_base_model.bin \ - --config_path models/bert_base_config.json --learning_rate 5e-4 \ - --tie_weight --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \ - --batch_size 64 --report_steps 1000 \ - --save_checkpoint_steps 5000 --total_steps 25000 \ - --embedding gpt --encoder gpt2 --target lm - -``` - -### BibTeX entry and citation info - -``` -@article{zhao2019uer, - title={UER: An Open-Source Toolkit for Pre-training Models}, - author={Zhao, Zhe and Chen, Hui and Zhang, Jinbin and Zhao, Xin and Liu, Tao and Lu, Wei and Chen, Xi and Deng, Haotang and Ju, Qi and Du, Xiaoyong}, - journal={EMNLP-IJCNLP 2019}, - pages={241}, - year={2019} -} -``` - -[couplet]: https://huggingface.co/uer/gpt2-chinese-couplet - diff --git a/model_cards/uer/gpt2-chinese-poem/README.md b/model_cards/uer/gpt2-chinese-poem/README.md deleted file mode 100644 index bb068eac7f..0000000000 --- a/model_cards/uer/gpt2-chinese-poem/README.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -language: zh -widget: -- text: "[CLS] 万 叠 春 山 积 雨 晴 ，" -- text: "[CLS] 青 山 削 芙 蓉 ，" - - ---- - -# Chinese Poem GPT2 Model - -## Model description - -The model is used to generate Chinese ancient poems. You can download the model either from the [GPT2-Chinese Github page](https://github.com/Morizeyao/GPT2-Chinese), or via HuggingFace from the link [gpt2-chinese-poem][poem]. - -Since the parameter skip_special_tokens is used in the pipelines.py, special tokens such as [SEP], [UNK] will be deleted, and the output results may not be neat. - -## How to use - -You can use the model directly with a pipeline for text generation: - -When the parameter skip_special_tokens is True: - -```python ->>> from transformers import BertTokenizer, GPT2LMHeadModel, TextGenerationPipeline ->>> from transformers import TextGenerationPipeline, ->>> tokenizer = BertTokenizer.from_pretrained("uer/gpt2-chinese-poem") ->>> model = GPT2LMHeadModel.from_pretrained("uer/gpt2-chinese-poem") ->>> text_generator = TextGenerationPipeline(model, tokenizer) ->>> text_generator("[CLS]梅 山 如 积 翠 ，", max_length=50, do_sample=True) - [{'generated_text': '[CLS]梅 山 如 积 翠 ， 的 手 堪 捧 。 遥 遥 仙 人 尉 ， 盘 盘 故 时 陇 。 丹 泉 清 可 鉴 ， 石 乳 甘 于 。 行 将 解 尘 缨 ， 于 焉 蹈 高 踵 。 我'}] -``` - -When the parameter skip_special_tokens is False: - -```python ->>> from transformers import BertTokenizer, GPT2LMHeadModel, TextGenerationPipeline ->>> from transformers import TextGenerationPipeline, ->>> tokenizer = BertTokenizer.from_pretrained("uer/gpt2-chinese-poem") ->>> model = GPT2LMHeadModel.from_pretrained("uer/gpt2-chinese-poem") ->>> text_generator = TextGenerationPipeline(model, tokenizer) ->>> text_generator("[CLS]梅 山 如 积 翠 ，", max_length=50, do_sample=True) - [{'generated_text': '[CLS]梅 山 如 积 翠 ， 的 [UNK] 手 堪 捧 。 遥 遥 仙 人 尉 ， 盘 盘 故 时 陇 。 丹 泉 清 可 鉴 ， 石 乳 甘 可 捧 。 银 汉 迟 不 来 ， 槎 头 欲 谁 揽 。 何'}] -``` - -## Training data - -Contains 800,000 Chinese ancient poems collected by [chinese-poetry](https://github.com/chinese-poetry/chinese-poetry) and [Poetry](https://github.com/Werneror/Poetry) projects. - -## Training procedure - -The model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud TI-ONE](https://cloud.tencent.com/product/tione/). We pre-train 200,000 steps with a sequence length of 128. - -``` -python3 preprocess.py --corpus_path corpora/poem.txt \ - --vocab_path models/google_zh_vocab.txt \ - --dataset_path poem.pt --processes_num 16 \ - --seq_length 128 --target lm -``` - -``` -python3 pretrain.py --dataset_path poem.pt \ - --vocab_path models/google_zh_vocab.txt \ - --output_model_path models/poem_gpt_base_model.bin \ - --config_path models/bert_base_config.json --learning_rate 5e-4 \ - --tie_weight --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \ - --batch_size 64 --report_steps 1000 \ - --save_checkpoint_steps 50000 --total_steps 200000 \ - --embedding gpt --encoder gpt2 --target lm - -``` - -### BibTeX entry and citation info - -``` -@article{zhao2019uer, - title={UER: An Open-Source Toolkit for Pre-training Models}, - author={Zhao, Zhe and Chen, Hui and Zhang, Jinbin and Zhao, Xin and Liu, Tao and Lu, Wei and Chen, Xi and Deng, Haotang and Ju, Qi and Du, Xiaoyong}, - journal={EMNLP-IJCNLP 2019}, - pages={241}, - year={2019} -} -``` - -[poem]: https://huggingface.co/uer/gpt2-chinese-poem diff --git a/model_cards/uncnlp/lxmert-base-uncased/LICENSE b/model_cards/uncnlp/lxmert-base-uncased/LICENSE deleted file mode 100644 index 52df82d356..0000000000 --- a/model_cards/uncnlp/lxmert-base-uncased/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2019 Hao Tan - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/model_cards/uncnlp/lxmert-base-uncased/README.md b/model_cards/uncnlp/lxmert-base-uncased/README.md deleted file mode 100644 index 1cb7d36f5b..0000000000 --- a/model_cards/uncnlp/lxmert-base-uncased/README.md +++ /dev/null @@ -1,34 +0,0 @@ -# LXMERT - -## Model Description - -[LXMERT](https://arxiv.org/abs/1908.07490) is a pre-trained multimodal transformer. The model takes an image and a sentence as input and compute cross-modal representions. The model is converted from [LXMERT github](https://github.com/airsplay/lxmert) by [Antonio Mendoza](https://avmendoza.info/) and is authored by [Hao Tan](https://www.cs.unc.edu/~airsplay/). - -![](./lxmert_model-1.jpg?raw=True) - -## Usage - - -## Training Data and Prodcedure -The model is jointly trained on multiple vision-and-language datasets. -We included two image captioning datsets (i.e., [MS COCO](http://cocodataset.org/#home), [Visual Genome](https://visualgenome.org/)) and three image-question answering datasets (i.e., [VQA](https://visualqa.org/), [GQA](https://cs.stanford.edu/people/dorarad/gqa/), [VG QA](https://github.com/yukezhu/visual7w-toolkit)). The model is pre-trained on the above datasets for 20 epochs (roughly 670K iterations with batch size 256), which takes around 8 days on 4 Titan V cards. The details of training could be found in the [LXMERT paper](https://arxiv.org/pdf/1908.07490.pdf). - -## Eval Results -| Split | [VQA](https://visualqa.org/) | [GQA](https://cs.stanford.edu/people/dorarad/gqa/) | [NLVR2](http://lil.nlp.cornell.edu/nlvr/) | -|----------- |:----: |:---: |:------:| -| Local Validation | 69.90% | 59.80% | 74.95% | -| Test-Dev | 72.42% | 60.00% | 74.45% (Test-P) | -| Test-Standard | 72.54% | 60.33% | 76.18% (Test-U) | - - -## Reference -```bibtex -@inproceedings{tan2019lxmert, - title={LXMERT: Learning Cross-Modality Encoder Representations from Transformers}, - author={Tan, Hao and Bansal, Mohit}, - booktitle={Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing}, - year={2019} -} -``` - - diff --git a/model_cards/uncnlp/lxmert-base-uncased/lxmert_model-1.jpg b/model_cards/uncnlp/lxmert-base-uncased/lxmert_model-1.jpg deleted file mode 100644 index 20e3b7cce3c13de4296895899ebad2c52bbe9d7d..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 281338 zcmd?R1yo$kvM@S?IFb661&tMjPsWa&M;p3;T>C^$+}n5W-MxG3=5H*q7!cjO#drG=1OIae zVnHEIEyt2lGDZR6H%`ueQFlmRXn%m0HIXoj=m0-O|6uv}b#RD@^`#lPtSKAhdJ=$- zi*1nzmk1yO7^k`NLE|Zb5JHGBIT%{RXym&mve(zR|9MWYNIQpH8o^Te&xsx7g?1l^ zf-u`h^rl2Qq_cFsL9u)iJrUPB6aPZ=tO)t!u`?$}#bHIJ_?GT)FRiE}z z;cpP6$#?q4w0}K6jdnXDyU_X@!kK@AK#zC4vXOW4_@WbaiRZ)qcXa*^;q&?a;mE~9 z9|-2^b0bOGZ*&9(UOYE3F#VF-$shOV&OWDS!ZG97lbo-M{ZHKdXY1H3CRwXO%o6ik ztoWWT?buF~*-VLiZY3rr3rtIOPa|@+Uog1}orTC1l{+3*u{U1gH$u*C&eyq0j!8GT zwePaYwV@<9uK@$en<-u_tLbgCm-iaJhw8D4=X}aZ`4yYtOYM8WdXD=;J@~@*k75sQ zMGs1{NN%_7lFQeV+Z17`Z(r&$9C*eOKH+%jn~=Oo)Kirxd3K6PdsMEm;CuCH_O-jp zpiI~`;EdSYZ*FH<>~J5Y9hf9$P-`x8#(x>zm!dkbwSJCm35x}>p&qxmoTeB%4dhqq z*J}XvH2@3$PXB!>^4<2|u!_y+_|Le!gk=#5vwu`B;CuOR7*e|9Hw0rTu=44W;AHk1@Kt1- zjzo}mA>!)OHQ=>64ZQ6)>3oUH1*lPF)NIRG?9+dYJ!quG5}+JgqB)s;WvtI#|L5@n zwsmm&<^w!0>0OJeYXGwU=#Fp)_N72@67KGp$?E>4A3cflgN1^z1COgFDy5(7E3hZK z^m2w(x0EN0J+k5OlKPvSs2LQaT>*v-4jxdab@Jk7`M;3O6Qg%Mi?NnV7Lu_fhMc?i2#k)_1nqKf)!kTw-^?#kOTY24| zo{95q$MYewe;~~JqqXl4Qdgie{(Xr*ZoMxwh_Z4DlhofB< z+XS;7a>tYLn@><*pXmY2(b*@+=1N0^u--Oj_ebs$Xn)wBzL|xqbz%i#zdA_jdGzw> zRB7p$90#M2QJD^s?Xd`{SwZ=gs8K}BRM)f_`A@~6I*v4fj26{!si}q)i8;{nLD`k4 zjThq#osG%oxn=kOXL6NT?Dv-@jA%$#9wWyQJJ}*=B`6{+eYaVB@1tE>kI84QyqsMp zrkYXPLDzLhx@&+|wacafT@=wTUZ*p?YXFVF#buR8$}H{i3QvhyN@A#-))Juhiq)2JXwUiOwlXllr?On&A;maV(^Dfz7Efpx7*B1^KL%wSBD zloYIizlYWe82>RKbidwnq`^HeoRIs-mIdGbjQ5!8w7RGzq9%V?JGkWYtCG&Tz(s!7 z#5&pr2B`ou!VyRJk<)%g-V$L61|3nZKDiC6-l!6AfvP=P)xmWlN+@80g3y906-i@Q zpd-pByg~dV?SZy<3kZ}py$tKc1wljUN7ztgrjMQGqI=Z{WC!?uHaIqv?RWFOi)yo}ek_ZRrK?jx(N>GxzBeG_|E0%RpvmAX& zVKMSigwLQ4iKRNHW`umcqmUG|%0+;$a5eaK4PcbsJ9?jhlUuV^>*GH1{6V^1dOEd6 zLHzi^lw~BJne*m;P!Q8Q{O?OOx}0Ayyo*60+Ui?n;RtEAu*AxY@8w z63euZcg}kfvoztyohm{`jjLS>D@Nb6dT2ec9Q&0#f7SCaS%8M59Vfl(X&j%-Ab4KL z$-yXy&PhAfv+w@Mao1>rP#%`ugK9V6QXk1iwr=;l`>b98}0O~iS(=qW;KLNkU3cC2H?(GOI}&y-2OO};Dwt{s}Rx%gBNp!rHKB$;n> zzW;N){u2)ri7xUnNWsX75sHrNu&%L|b4{Thom~!&R0nHX+9e(q@Ii;q0OdCw9XUA+ zdC4;xJL9k6MZ=h?Tkch|daN)ri;eN*>h_li=hE!8^fumBDDR11uZvymvi#ZC)_sl) zMz;}@w;lfpO#U-P;&gw8?R6cM3JVz?*SN}8h@*kvl7}s46@v}hrZR>BTR2k+T5HK= zrS}tiZ1yF_F#1tNJ#fiu+ZNy6UCS;$V#gQ$HsO$YZHDDH6MDO8zU}kV`qJZtBPr3@MONm7WkiW*e~xXcid&rzA%5ZPy?w}WAM}WB$d^v8%(riN zG1Z3t>-0!h`}g#=^6iSvB)fins=*&dLfV(owHM5TDCaGcPv}V*m721T0W%$X`Dj^q zs=9iGXo0`V4W?*I_knp4_nMc~kWa1CK^=YeYmCu+1$rP)knoKuyH^w70u%#-zk-m5 zPM+7h7ee4!R-ZSG70MghzI-3|yZ@MbY3F~;z4Jfj9sm|AvM!Zyh*?i|kI3onP`DFy zX3IBNN(M$0Vk~<cNkMxy6>lO)W6}zR9*w8Fuzj!^2mM%L?>&yqZhN8ibHNsTPhp*8l~+ zv=)_)u1%dQ;DZo6;!6bi(Tl2^AEakuAB0d-T_51&IX3r#K}+cy&`$owF zFcc(6!t7++<~5z+%pJVs_F8{XG8f1R-c22=?P70wNLAEkO!WGxD5Ay--~J=a=7WCM zA&fk0(+N0dNaf^L`+x{k<+U(+*v14rVuhW}{^G+^{Fl1$zb*VAj-v@$j$R3)Cn>2F zqo@zJ9yWiGjjm()xkzOxW z(H(CtPQY~Tz5d^J<$v_W{RQCVS@NR55oxwDoA^A3&kZ{IXyP31<4>2qbvyQm-WsVg zS>MBRx)g|=q6Ow2>1~Nvjhrpl5VOiOx5!q#KnT{Zl6IsSUo*-MmCN4Zqv+-9wj!l! zXxVr@#F#1-rdr#7Tl(%y6zi6DZI2z8wZpb|qwh13ij@@M!)I5i;m0vCqrQS+a=YwB zX@la&Jgs`cmuvid$rQ!q zqM`1RK~y+LA+rU7CqOCM65ldbktlF9*FFd%imVKg5Gt8dSJHDnyuW6~kxUC9p zOpD@;i1{(SJ?gT2#I%=rCbuCTTVz5>8{}negjLLWaM(Z)1@nil?cDv38xvC#ar^c; zWK?a14aX%_WzzhVQ%pdkTDv!VKB?v@Z+rEo>N^TMYhxmpN^P8T?ArK4P|-!Xn{ua)$N3D)ObblOSca+wJIU#Z3Sp0+aJ1 zeykTrrS0_B=shWL%g2y()13>=LH?y){Bz+pYkH`7bT8;C?^(e+eHB`yyTHfRX|!@ptStn{t?I`gHw@CPk$f~=F!@DAkY(gd+ZMlS7P{#b=EHR&>> zxU0p>kx#SN$uK!1nW(+wCA)^`0E;wz9U6;m=M-f z+K)b`9j{AQoggB56{EQZ1Q$HWwJc0nT3CAxRaGOYT);-eMc+qXvng292#^XzgQv*ft;*_H%NG)6K94I%B z4jSv}D>GouGZoG;&{-|BAdRpCidioUdO$nc*aS3P`BOIGC`yH!DtE+dD?TMk%2kC_ zPLVP?@|tNoqU570Q-}yOt1x6&=D*DaoqFo05o%R~B zv3coqUdBD6u%e2w9a`@l%2cs3d{$!*&eom)x(`yJ&Yz%N^OK|bsc$_Br5;&&nR{_( zih7(qE7WE#BPDky`QFvCX0#h>wWWpH=6df!FhtN-e%)}4z~x&Il#&YGRk0`JOYq+3 zrRV=# zmZH~eOF_#6F^hgeAx(Ooy5>7T0lskh*^Z#mz|=WQEatY=ZQ9%X^3KN5`b8Z?@8GDE z!>?`oVljQS-?+o_oL7YTewoxZI#Y7kx%~U@_MI(?$9s6bW< zC5VH3D{S%K1<6bO(g~Xtin53wKTWvR__TC(x>G|$sYyRJ%^zHGREkg=F(7eW&Agxu zB4S?9oZ^4(@Bd`ZjcT;7PxdMPyaj zdjb^dn1P7LsHu>cSz4@A=G!V?1E#fo@SsiVg-R)b&Kz~^CCT&1v(Q=AL$}8s6eAmA zVtFNH6ZehSNR7+z6k5&ooYGHkF?RQlLb<%5HH2xcCSF1FArjGf(!@y|A_a<|sMUjI zbkc9xVN&qY?e2fH;+J!|osaS}&@f_;tI@yB6xVC_c}r)?Wg_^%1H|TB2nS9)ov?qw zTY1}{d6^XMyl-VI-{Iw0U}1aE(}o*XC)wTCwbiMl4ldF%u#`UH@?Pq6Le7p4Cmj~a8Fcdv6i7-+M z3xJKxUjvj)U3+nTzCYx_abVhZX&E{dt3;Zg55iE4i7>O1G$Wcn%_xD--zqD(xaESl zoHbU(b$}7yrbdy2C|;gR6wb2^jBl$go_@@+^;+Uwnp{|+zl22Q6^T=)bZV~yOya}W z_%P>hd{dv9!yWPPOr<{0Or@45_EO9<{eQ&|+E-WRnSDs`GBvA+HJL$DDz3n)$ zZAtD`x69Kzio6#H@TyWSn2bhO;Ie+O;A9S4Lm8sCGtW}%)& zI0kY|>(wmezfkU|Qhu9LUvAuD7_OwL8${*jvPR1gFeLi6Bw@T#ed2Ed8UFr1pxD2V zg-e<5ox83fWN?;+d*?@^Q$6n)?mc%Z&2}Cdo|NuFds@QrsiEzo90>7cJbBU0>hPpY zf@A)Rsin7Vfr$FD=J6<;oqkxw+kQa|_fx)_QK-*B&^w;C%66rV^V3pJajE#S3>~9h z*HJ^md7t1;VSjDBPngY>VD~7OV4Z{SbTHotkk4>X|J9U%=Qa~6+zQ%YGOJLWw zxL-%rSN4adsK9CpTi5*iU-eaSW%Ouza*8y92;2Ibslg)mOy||hrCf)9OZ~K`!$tR8o~5r5T|nZa zU99AX(P>zjFU3v@0S47uQmYw<&WVhVSIJ^jW1!(`F-fIVhkoGXabBzW{H4qoe^61r z<94>&T(Ejn?>f7dL1ago_|_!?tdC$559PtQDv_MFt)K3cpG|m2m{gqNK5!wOq8NF} z@|-x!sD5E>Z%?i-ukSu8P8)f@VAQ#>{B)ktEGuMK&KnlNy#!T7oi7QD8W|^eh>eqO zvXxsrsUagBZ|T0sxPbB z)EBahvx%heFYhPk+8=xzWCBAki5Pb-ni(NG=Y0tv7|_TGTUg6*=Seu~Ac zPnly>A!Q|`L?YVJ+CCKwhFMfHF6bmTyOu7Uvk{SZAw11Lo|#tvWt0FHhl70lbewN8 zOl?Lj^+!&kUdg-&Q{DbNAljp^HR3+G>BprpU=S2kM!Ft+RCmwKDZ98nx()IaY~h?b z7wQczg1)%QdQ#Wn~<$r$DR8Z@UIX%#*FBIxOTkJ;MwDvQQbB zW+z^)IvW=1w>0Fw%fS*F4%h96UJI+aO8&4VmfF#ROHgk?&*TUYYt^kiKI|^G3^(DbUhju* zOsrB}Fw&T*!`p6R}zYdz@Th#^NR# znuPj1po8M@MrX!kOkAEO_LnN(o(2eQ)LKrC;`MPJnwS09aq8t>UVR@&B@q|pr9t=2 zP)2)9;t>-T8?-pcukINDC@TkiCDaUF4{anWd$iiixVUrEsMLdJ{1o3{H}UG8g@Grc z8$Jvs4XZkFs3S;iws7zWHL?j8?`A^^h$S3dwI0V3&i#q|f9=6cD58Q^X&LFO^X2sk zc-t-O8u085FK(vP=k^r`a)n53`>ez?tV?C|`3j`|bf!6>?$b8gm zE)Jdbg|)F95Sbi4k+A3j)F#wg8zL(P0C<&{-Y-^>+$()zYTBSsuUTNNpNv4{<)7O4 zTZT@co9BUt&h_kYTSJ|V)Q&s&Mi*nZ_B(^SxZ=v|*U;9JvY0m>WwEy4Dr0J)?0;Xc z8;u4BzyGh%^uMjvEeFKarP?0l`TcIbi$m!bbWBm#0BMnci)(-fX4Y54)&A6{y~p&h zae<}Wu%Y@7+c=`>M-i~Qc*t*96(uBIivoHtKBYshT?0_P*MK*6ThlPmtp9|SC#b|$ zTEx1fbbERVBvx;$-&X5fGQ2K6)Y%u=kv?a-v&2xk2x>bx={w?4-x6H5PNrAzoZ5p~ zm3vjnoYmP>wiT6VH+X$=@u62-JO@&yL^--zs^8ri_2(rv65>5dM5-3G>dmX_lB{DyBJTeSI89PDRt;W?5?p7)r7TM@HMbmcuO_aND>@!uBx zN7N>fPmeW0sL>2cF3$`purJZUM0|7rb)j6cQvr=h1QIB+A#Uzz7eE5Pxodi$6`OlZ zE0WS0ecmOb9XmG(74dgfVGk*Bv|SW2B)q@SFeG;2AFnufxpHs`##GJlKQ(Qc~~Fe8kteRr}wSz{=%m?r~8?gV|>QG z8U?rq6r;v#w0h4E9477V!H=}M^h$%SI;qg|jDD-z?}kO540K!Ox8<9+K!kkA_<)A{ zkoQ3w1+Y(pqLq_%h*NW*@?qlUlH z20e~UUD{ahz47Uqav?Sc`A~al2uo1(yaf3_(d+96xk zXlPDtf)ORbp~1rEVi4mt!TXJ>>vWvf`LKNwZqwrGmHMsx@Xt0XYz$;w4rYN=cv^BH zXcuP!3VanbTby)Tk#O>m(^_;r}f}_ zF4|v|SU#J8?$Y;?@Y&sNvr37ejK=!}p-{astn3mcl3%o?Z~nr_1^eOHjWCq_l92{5M~S~VB|ibf*R?5=~{F(2#ZP3&3=B+m*1AyX=Q7i z5Ynp?#i_Y@)z(c_Lt3Qa_w14hU_IvCUy)Cq0$%R~KMzlSO8x84Oq1)qp8A|x8j^u+to+@YK$zhPy5Jcf4P(`>or#T#hUF(3s4N2Ug+l-L4jE2l>z=PSKXXvY&3WAavU-9#u=xIQM zJ;tL5dHZL8+of)@Yg|s!D^QLoW|*pMLDIjAa}U++7uTu9ek=|kjnO-%Xe z9>3z&VsWXf@~s&Fjg9Vy@<;uW`N2;TilP({LIu{Ie{5zG*mNxkq?q-|5cqpl2-YiU zH}bf0EjPdZGtqJv&%=r(1W$(nSCd6#neC?eZK1{mK>de4-qOL5U4j<%{H>2FXu`mrOB2E`$F{i+CaP$b z>okO;JyPk~U6%^Er+#u5#wM$l@(fo^qMT$KALX@=T<-h`xzMQ;Q$Z~=der#h`(lV2 zg*9ICZU-I7&lQO^h6G+3oO*{=D8tdW#;gyC^@Auyp8 zvdsl_E*aaXgf5iC6e@hieziOh6ur8_msF&laa5?4Gx`tz4_7skD_0(!)` zx66_5%vFkK*URx-V5)CaCd(-H6<%{NsJc7bAyPkG``bviXDe6voDJTiguZ*Lm^+9G zi*2CIbT>HBM9OM$hDB~OhM+oJbUqB;)>aY<6)J$*gvoBo$^xt`6u8%C!($jzJ=U6H z6d58~2Z2Uo!#X0W_d_D|9q}tamboygpSIPPwOj)>IWmnu;npCv`nmMNqy*4rpYxzq z--A(-c*lX#1BaH0h2C08A=4|{e!NL5XEwO5&u%1b2OdkBhbK3fP0K%Tx>ROpfCcO*rFCEo>GM9W z-V$H-? zhmFALe&5has0GiDJt*7=r4^4U zY`uAye^9Viwdl!VlZ(QMAJPr3Ws);7U^MqvZT-3kRMCyqK`A`Y7cveO~)-j{lZsQqZUF45&iaF`#st6w=Tko6cG%e~_ zJb>I-AEJ<4CX7vRtc~+(p5+i1GY?|YovNI!FJyjySH9tAC8u5?zel$I=$iT%ziSB{ zlI}bodUG15$Y@x4wAk3e1j(I74okDL7m>Z~1TmH5;ra`8{^bOXU6bHJ!Sh?6KUguqm=cK-O`VcUrHJRA*tFw1 zosK{)L?e8rsK?vK4%7ATfau?!=2O*=`!LL8^-sc^N+lL#W9myt3pHD{kdEhty1~pg zmBur2v7MJJYGbdZp55jBm;&^9Tn0v`zDaxyFB_G&>tyzHJovPVGaT8W-8NX}I^7#w zMym2Xlr z+=V2TE-m{$O(2%bHyJ^o!W|KvhDWrWc23?l=Jh0}7$;zqN-HkENh7evqUCAVk(dcC zyznrjnr@os0qOfzP}_pFFjcXLMTw;Kf>V4O!t*<=*fbfhOh{Q+C&=b7$F2{0$%@@5 z!k3O5iV+Mal$4VYw(yIpUtOCHcik~MQiBsc?Eja^{oh;R&j7NZZi!6S4zGPjlf-V? zpEBz99&1%$F4G1gt}vC53d1|>2j*V3FlZh zd5^@=*Ud`yYry1`f~&%piHorK+D@c~BD&n_oQ%%Z_$MK_nGB zMWzyplFCxHCOo|1qoWAkGMW(yBq{@zQg1miJuw|p*|Mryy_elNN=odq!4Fc9dC{s? zC0WK~Sf4Fm9H&l=4I~yV;%yKsj=!^);d+$%t9P&rLd7s}Xq{f065GS}9NpL3N36cQ zIpW&E_IwU3;`E`$UkW$Ru(`mnX>gh5d}(z?$4zsf@CC6Xz1YB9`#$?4ME?sJU;O}2 zF{)%Xt4B>NgV+W6Ntrsbk?OZg@KYNY8%x6Q_Ae~*kLyI=z3N&@RiirFRcav^A190H z$)Flur`Cuh)7WMdIcF4`)Ay@3{rqddBft7uGnvp4U1dE(uap5@3+y%v?SfeYS=yY8 z)WNrnkMnI(w!9oosk-LCU36?DagFC_-nw#%P>Xo zLXxbbH$XV#|4wt=RHzBrG0whZ#f5waWvNH$>i?>gPO?(7MgrkPLCK+;Uy@l z2}H?Rl+CVf-o~wIm$_BXV*jv6HtlncMD%R1=`n6Uj6I^QGa`ICrKrZLVx!C)=o~KR zX6khpJgqs;B~DI+H-Tr_VE`7+S3hF!o;>y=9XKOY(@>7f8zcMHIZ@e?R=`dU9gx53 z*L@+8{rbbr!Qv)7#<0-ZP%|PTB9TD6F z6&u{Z*h^Iy^U(vjeK|k#T%saou$0h&b44T4M&HV{8yyulX?Q{r`&Ge=+^~ zkhOGJA0)or19?=c(Lddvi{R~%!Dt{_YNFr?yO+`3%63?p>) z)d{{mqSdH%NL9t&5%t`ZOa*U{l&HtkE~GTFk6q=Pl;TuTUcR%C4X*-zU@Lr0oUtaj zZgR&H?3;3aArNHMvF*lEQeO=!vg*I^l~U!~4ct+mX}>`1IKUuo0z zW%A=NT2!U>>SdB-Ts|QZbR0_UYjh3xkYL2Ms2Y8Il{H)Oz@bFpYHU)X@PJwd?s%*x z?)zd11AGuzF}3ir)rl2G=V(i4B;$w%Ii9kS+}=geZHPAv1CJV(TfUkOHvbh2BnhSJ3_v)H|L6QSw&$a{DfKytn#Nmj#k4>L>!H{|?fk7Bi>rIGvsrQBkF*uoH0Pbjq;tqd zU1V!xb&-<+{=IQ6sj~eq7?#YkO$?LMA!}*5h+v?wWoA|{TvFM^u>)Zy9uY#5a|;r* zz_dVEVUu>+WWd&-p7(B2!)qN}A5!%c{pp8tm&CKNN?E5^kqN>=xrm7zJ&HY+}Ym9>GDdT-K#aPZ#$g<(8O^-_e%@{b%v=KSD%hKI1h2YucG$w+PE z`7$!{sN3cXX}}ALQ6u!TOS&wd&*S4~ZjVzV?D?hHDpvD;*}OPO{Q})El48-3=RctC zR5st0`9(hGP8HqE!>}mokx~`j^_+jetE(Hk%a32OWRTxz@tZbh4+O@%KP9{_^)Hbj{>7X!_-y z5$Ri$)f9A3HxMFE`-c+lMKi^A)|7Aofh?;FtcE;;_c9LZoR>fa8xPhBr5gR~LpA)~ zi{|A1s_wm)%b0X=k2_cZ?9by2uHVZj9+|mq>5yOq!>_^9Y{EZe@neCznu<+v*nP48 z1mf(QaW{vA9ZxPg`i?u|&Lf0Kftp0bF4`ur3?V{V!B|68kvCD(W0`PMv%#)VQ=aWt zjNZMI5k$|O#;4-neDIjP8OdBI&zx)*Z0khVUP05&lvE!aQj2r252cagS|(=*sk#WO zrYDY-(DrF(`w@I+rDxEZkZk=BgpD}9rDJw9L;bX9$TYq}dacWQHT5(*YbWLk;k@{Y zWm-Z_p}?tlLvl$cPq|zpffDG$l-NPd1(^-RO#&UR9pp2``F8P5BxSm{P%ON89Y4Wn<>z4D`p zKi)V9vX!MPmjJ^{OFT)uY~`Jcd=Jg9T0Hsc)`PdA6E%h9O#A?6e__Nw5;()rE%T1A zzKP*`8`x;9_7j^!y%Y~#QREX66A&-7nQ>^Fjk$dz5*Bp4kj(k3clnW+*AesXNzpS<2O;zg^O5OOVxLJkbDO|~K!-Uk%v_#y& zTE6xZ8~1dedAlinnjV%=to=*ABSY7FF5Lln!;e0HwFvBq4`90ZDXsi)>(%UKGIgCd z<5lyD1Zt%cy#evW2)4zV5C8i;> zQjk`oRgaUCux@nQ{F!?&skjhkR6@B%%e<_ZxEfRXX**~aQS>IRWUz0>ySCS+BFQ|z z{g$S9OEW_zrm}E2Fg*Xb$*x2A?k)<7*47m?hRo~N!`4zbq4sSLz?4>u20+GRH0`S|(w?UIL_P;ok5 z)=^V7^vYUndnT%L%36!_g^YqUZVPS0zyrN%39FvU!);*h z8Wk`N(=A<}>3ua2I~|I(6O|I6kV}iV7)N|aA>Bb2Ol7ZTn&W)ZLnoesXf?|!(ig>g zIWYz(BK$8DQ?)OZOe$EAwQi)Q9#Sd8I>8F41*gchyez>u=^ocyt7#(`(N3qJ5M$UD zM%XB21eYf_I5;>tBeE{wp1TV?)v_cTv5(kBK5C#xfc(;aSV;-w=r7OnkwQ76E$c%M z*VFdu-GKaaZ?Xm^IjkUYM%@0enI)%`^;d`*a0L*U^Wc>@c_$a!`cv;n{)7u*V!}?d z*7>y|B~mik0z3u<4)-zmLu@$pU9RU`#4VkS@Ze4+Po|`!8evckc?C(+ncqNz(i<7g zkDKogald*+{qV`WGa*3ocAy(5MVCT#yh-SB6u=qG6}3g+1Got~P67(R!1+s;32;itk{)-|R0VE|gT28)uw6 zMmd+o97JhPu*iv&R8?q{Ab5CKvX00lZ?)GLfn-L_NYrLW`7F$`>0Eq z)oJ6$+F*H@79T0i>+Iso{!C+IkRC^A2OMjHO_~#J#^x3}QbcaqkdC#b&%3pfXo}H{ z5fS52GMBbzPkmje=VS`7*sA@K2KO&4&3v^aqnK+{)cPiqUB1UIAWnU_RHZ@}#mKH8 z%Rk@VpdStdT8B4870ro;JM>aZSx7irM`-iEss|RzvGU#<4C~Yn>rq_Tnr_NhsnX+X zB0QPpBEST+cMERIdV~eK3L`HU(qS?)9D~vGqrPEuP}^tdfeWo zF!EtFfPT-vF%e;1ng)H~SJSNxt`Bwi;BL-%2owh~wzK4(7ps)51hon-EqZ>(Y)eYg zj_c&He;r@*P1|tM$uKjjWF}1sqKA{sMw~^d@=Bx5rb2{Y+pR8~Sk_pvBwY~eT!#A{DE=n} zvd1;v%NgvkWfm2REF~81KmCP~-=Y+lC`NO_y6P3*zwtJD8^KXLTiARQ+A1+(hYs)M zd^P?-)>z5ZQ?cC!#nWfE`o(k`>dpJ2d3~mT0ov1BlkuD2B5K3Vj;SZT3fYdrkLvN4 z#iuV2+rfBzU6;`nuFzb%jZ+#|;nMMX$9i2NO)|Z2HmY*QEtsyrH}!u4eBaZ`r7cHO zqBz!el$mgZ;50<54QgsvkhJNJdPt#+)GU$O7D zM)kbPKCVg(EseFQ6ALl6{$+8ACYjlRGRXanIP_jTUMyrxQ%g*YO9L(@+!mKEh@BMm4wI zQssunSy59Hf%8P1DxW>QDlAus)wLgcb1rW|`8ADV*&n1t5~@(LspDig#=4rDHyM1J z(df-Ls2UfU5FIOJrf12nk{xS(aBA)%kY9BA>JHZ$WmxxMHTLruYsRjs1#-uO!C#y@ z%zur$XZk3xXl2rEC7l!VGdrGl6zM!I*)SqmVf4&xzPKCz=KPsuC}%}q@Ltj1d6_K; zd*G+!l&W&Utj2JCuM|{)L2;Dmtaq%uqlj(~ zQNR{uk{%WLdMTRoou1$MUN_zp6K&}xN(}Xd;=DFl)pU~^b~^21HAnK-%=OvEJi9T4bn*!Gf591BG44+3MJwTc+N z4R1k<{xPI)@yU@!k5bjqqQ1XJ7)%<|Vs7SDeG#h?`kTP}{5sI{u6!VF_LVPgNR%+7 z^uP)^2n_qp!;J2Obh*wz^x2D^U2s9}u2wG-yriABI_A3Urwfp@E*+?ChDH$=MpTSoQB-iR}%bUHtPJs zO7ISJ9&vhCL>fpIZH$2%=hY@)zZ~>;UGPOfo=B)LI1E^b7Ik*y=?cLLi`xq1iOq}i zQHHL{SuLiUE8xUvCh!c5o;K<}%Lg*{-h zMOX*dcSLSfGZVxV;hMb#e8x8>#FNxIu78RZy!q3)^(04TLFc>?mK^x5*+a*hr4=++ z!Xw+*Pbm;2EP8Gx7j~d=F|LOkq~A*HMug@f>^!nO<=eJVtlO>~$$5c?E*X-DW*^jp zNn8Q_Uf3%+DUZxJN~-X1#pwvM$@*Zlde%!)OOO?ZD|3Oprs-Ug(8O<9A%)x#BYk$X z;6j+*)p1gi7)&91ramAqG#Q(+KdxkZcb#22q zjujM<-W3p}iGcK`(z}!-B(#B0BsA#|nhm5QVCY5ZBm@Xm2mz7arG#D-LJz%*;2UN} zXXbd0=e_Um{+{DIz8_EiTsv3Y``qhVYh7!veQxS+Z#|#>!yown&!4~2_OE8XVEOi3 zV1+{=)7j#4wVYkCpv39C@tu^O+SSG~Rs0cyS^;#k(RHI7c{ag z5i;TiHxZGAMm`2jdA6=@^>V~Axr%zM;Bjx>PA%MDzBPbK@A^lfJ%!C%caf_b$rFEu zrbVU#WKKiV#Mm8=kzAc9{lbtG$=M;3V|Hh`;v`yBjq5u~ya><4yz=Y9u>SBVG%5zA@z zm{;OJ`>U<^n6$^J>baQMYJ=1c%-tk7L$s3i>v;k^&%*tQ1*-E?TX+pyo84MLm%(Fe zN=V=1b#bb$m&>0kIA!VhB$_xv@Zla_Tw(t1cScGjg@Yb@}P1c{j0G)F?Ig2Kh9!r}QnR~bDOr~;NZyHf{i%Ra&2`GhzGjrV?a*Ls*ESi*1 zT78JJPWMx~TLHhGMi;FO$^!*VoK7kA*0i*jr4E1j6!G?V!91IFE9PBDE4V(O^9K4X zZGXpFlOkY!y3@sgFRmwcaDdj@?D^H!F}U}Ht;RO`TFXGTAj6I4u9a43!U(302S>iq64O_> z`DP66P7E^sP`~i);r493eP`D|y2$9$YyaY{Ir9rh}hE&l^EI@e|rfZ}VYGNTj@k7sA=8`hXjt6{6Pm*st_<-z9D)On&)SR^r8 zw8mb^&h++*(?@7*u^;Dk>m{qWj5vQ>${R{xZeNW(Kp7%{|Fyi0Z9yrGb9U79u;L*{ z55<#?+L%tZbR}PwD9e4)__zZ(?W+CQe6@rsM<$~8XKjAdEMH4%unQCEYt&zdY-yEA zrLG688>%4JIgoqzHevo}vUbH11?M-_VUIHxXT7uqd?oz%Mx*Wvj}%?s*r$Arj>&qV z<}e&6tM9&D)VV1|=)D=9^`;f_4o(wa5i%qNB}Jr6$5JZv?ib5me1dInI<{#Y=NT~( z7d;z}j%o#|@qLI=%T$&6R@(y-GRA`{?uRcNn7v{3@$_a zuCB^8YE$QKk=XP?8gqQ*n94G8lTIq8&rZCx6{9;#neS5EYF&^M)(e+{f^caLMAMP{ zZ1xYY$ZJz|haCJ%1T-ps%hAVa5DCE-lUN)lL5K+L7zusy*e@5ZiqJbwRXs{c3na;QJTp1C_QfkM^lNu?rS`p{xPh-` zsrb9)`sLRGY-j@^G3dwT46OT(wqrt49=b;1E+T*2&>A(6}r{us7 zc!A!WJ{ci@Npf{Z~)0r@~?ZV7&X0KhkSLA7Vz1LLJr$#?sj z^DiuSwpFt9pKaMXwD#xh(C+fGtMe4xj z13js|@HJGaY{ybJdG=0WrS-?z>!<}wae%sYToUuv#9LpnJJAF1gMEA!12_QB{gLn7h^V4fb3n7r$^ZdV~Efy$FI7=Jxh3D#~1d?F)IhI+Bpw1Df zD^qQj!Q74``fCM|8#*@kgUZNfl2TAUi94eP?xT+`um2RZQjeepOZO8I7NwPZ~Cc742%{L>N*<9ymt0 z>FcE15~B~;c7M@J`(WvDUfjV6Sf#5EfYsgc67{g$<}s$FdHR=3oqO>x4MqbqK(niw zl~V-MIiWh*`j`eMy#S-vUe0-IHWsF>fXUx;#djwG)qjWEJoK~m&eY3(X?rdd%vHA*>o2j&6xcNwizy4L(xh=YF8KK>IAWo|XV*4(KkhjY1|mD zwV@k))h&Hh`c%1D73hjZ^XUw#_kH~H*f*%J4P)7b)~oD>X|>Jn`Qxz`3+Z9`yW`+H}Xpu z&Rd0c5GKV9Cuxel{nRDD*uHGAyeqkgd5opV>Z`cT9V|whb{Mu7hC`6rz zVrq`DNlMc`VAwf}I@hR8p64n|aFKL>KEO1eT_D>#%9bKzFqNGp_Oxli#cfsuQ3EgI zr&Bvj?LK6QCdsuB?94A4H$SfPQ17O&J78|>=S92&21(XP@4mCr6ZP=oXGr02S{=Ly zGWYC!cUnZ$uxk?G`xI^ND}Sa661>1p2$cAu}Y^rSkeXv+PQf*l`k^QH{k>@TUqra!9J_{(od>+ zx&WSjj`K9xoBU|TgWyZ=Zi*mm&1vi-!3CD`-aMh38k2ab#GN_P)b66%pJ4MC$zEmF zo&}RW@YMK)d)zD+-1b04%UB%Xel$#;`EjJOzpIp=v{8AXz0{Ch(%|&7D8(qsU?yrE zLT3BSM*F$^n6`FlpNvvDakD(!qf7F>RAD`q#90v!64+UAqw2rFzQ+Dji+m^%9{K9s zLHl?hJQHok-AXM1a>W#sa<5IQnL&eCG1 z=`ECW2qt;j>`#-## zvz>9hpYJ+Y61K{hwlh(1pCl&ij4Gegua@<+l7aS|Aa?Ud55!WaMQ($cxdKU^0f$qF zW)Yc(}4A$!KB&EO8gyb%&aSTc(Qt@aQsJ;Z2z4U50YK55&m%Ls2D=!C+o zc-xALZMZwB@w8aXp3Bk_*Tl=L1@1%l=%1>W*8Eq#{KU#rqDT{^M*WO!P( zq`6;?j*)W|yE?X`I9yO+b3sEdc+GO)?Q?KH9gfp41*d0P*`@povXd6Fh)-66;q+Jtu-~!1ywcc85y;Z{wQw#es9mqqa+$!ZRthR(QfI~^ zC&%i~K7BgpjeUF9N7I`athYHn@Gx$mLcZU+uA{nmf5w1Mo_I*DsDWo**}4zCzt->%dMW*_tfVCC3}( z17al5v;P1++VS}LZK|)b4qv*DD(r?j)eebTZJEx#S}C}dQaVA@-GKtRq{&ViVQF=G z$31&$57XC|8-FyOaHIv3y_3@-aVxZ>3airpT3oAxXFHo%ETym%VfLmY5{)PJ&3V-(ZtZ*D@% zWzXaXm{KkkV8UNF8G%vmMbT+S^FpEl7c!|P)tl0ob6kb{x~Bc8>wZltRI`d3k^)e9 zS>9&zqogaR^Wrt8k7#edg}Pk8e5y_=XeWyv+9r0)mUyGUvJog&$f~Nw4y@h^FphZ( zVq=qp9?YBLZ3R-fjoA~L@a=sFE3`%GCJut0l;=5fv3t|`{%6;C7ax|?7;sU3EanE{ zI6rDtCFu%3sUk?{PDOJ6ZCw^}(Xo4oQ|oJAF9g9 z@}10N6;Ex$@!I)kBOgByaBHOLD=oNA5+*N?Eik`#(ZMH|%@NcsVm8~-Rz-;O zXL$Q-wzXhGrk->w3mPaUZ4&5Cq>r-Xn~jDm{-W7n?NMyhvr53y)joM0>p$ciX5#P# zmvR^QN{BNjS*4|E-7d~cwsav`@@Bjpy$y*6D51)V=cQgd)Ill%;7ji2h|Gnm0|N=7UU72B%W z4=BOA>jF^{N~Tzrhi)d|Pw+r~TgJHdI{qj8`YI#=ck;|Sf;{!z>7_2WDQJ@q=DcxO zC_@|p*}{IV2D7H){BXxj=ZfH?f-%fSid zzONd@d)pGz6k8u-F$EN{PR#&kd#d}bUnFsFMWv(@y8^VgASEYii6t77!?T9mhgt|0 zagiyFop4r2UtuhcsmKuKck!I#kLS6zG+1MjCIDKw^49T%tANmDyv^b;zb|X)vunLB z<`3<<&=EjQt^@Rg_I}|Th?DO8ofQ~cnrPE8BT_^&s>zC7>}9C+oRO6++WF(oeNjlb zp2bd&uq)V}{EAf!2Gth@Un5OM%g?6&p;A6WXdNFoWz)U$x@Vv?%4u3TS*+dXsb7U{ ziDH>`8qb0Xa`F_*d`XfSvmPZ)cTCB;A7dxzmE&Q*T1D=$jkdDv{$0+?Ml;#aAoR{u zY*=G!%(kW_cToqPuo^17XSb=ql& zBhf*K-u=8Sj{&3N?L&u=KrRQPTb6v^U{7^Ks`ND7JjSzo(ih zB|Y{OUOr6YZkE^1EkhmSHw2eZZ!1yjg-i!6s#RRgY_#(;-jY%uVnX{G6_W%UdnBD4 z_7*htQ-^)^T$##)OkAB0_67?$jT(5$NyT_R;_pgelJtsbGkNDsf*eJHya4hi>fD}>3Ps%d*7^KRf6r%nnO_*}4Jd?J7 zi>NRkZe2-f-R@}_z*AYj_@~*TcH~Fx?B%}`>k_L-XUH{5pNs4(mgbOgOb_w)08y1k z#Z{d%>E3S;=yQ)y3RahJI`YKOP^fT5K+cNBk@8v)v6|vmDoV}8{9fjf5S=X_KJQeI z&Y^4|2u+?t!g&oSECCoLEByIZ8yT_e@yoqJziI9%#bbK=+R#A0Eh!_Mj*{8nFT(ln z(V6$AfDSeWTnCOSL;Q)aHBRz!mxczJprKD< zYR?eb=kknTEqh`;r#KH=EyMU;v;S1wwx6(&W2OXqg>>?3rH|7}bBQrj-KKAfUHxpo ztO;?t26Yuiq?h-G0dss3Y%!i@EzdIgYajI2R2u79I7@qh>`nOu^i@k0<}w*;qs*^_ zI9#_D5rc_?wv?6#4%%*{0f8rMVev>v=qNq?70x=aujJ)$_3L@RQ z0_$lCqG!7XGw(`j{;Oy2ZR9<7^y_0}bne|_eEgf$6`;7Pl*Fn&d!gyQCm(l|2tn5~vT0%POavtOc9cL99dua)GqN8s z`k1(6Ha8A^8R_OZdFk0MOWg*jxE4I}io0b$ckxwD%m~q5BUc*Sh^tsD-nsg!)v;*Q z$nu#*RaJ>>CruGRm3y2ze?y%-d5*;Jx*xH#sFEfWepw2OL^6zr9U!Dds;#eY-U?Tx z2bTw5Z_}iE<36M|==K~uDI(^-Ed(pupEPLFmFCnoswF2So%z5gpbX7WJWXQakX}Hd zZGKmiD<@truhbI+m=EfxJ4D3_nv9_q{0ysC*aE!b2GUZDG}Vj-g^ zYAUFru*vJ`$e;A7Vk^=3+%`h_dHTAXu!@y|Rz99qYN~neI}{_q+{$$29heC1Dw+-2Ixgjm~0XDF!*zh9*%(WA*5p!b`3> zx)jayxmwIshnI;li=qU^qtO-Z%`zT$@Xzt!Bmrt+c9sa!v3V?-mWJ}}?~gd6ZSy{; zNYKXli+g%hGI+Ub(HiGA^dao2R=M&OUk&ChSS`-pt!G~P_GhQ|UwB|nAp^Vxb+1W|`G>u2uV)n|vsv!$lv{b}s1I|C| z^Cf>##nUx=rZ8|V2*kOVdSR-h9%)YSi6WxeCzEtGY5BlFA>23V2LelC$S znP;_|B~rD*({#Yypm`ajd2*xW1V#UiY+sS$u2S2(P|{T8ETa| zTxAydnxeyAZC^L5JLGDPdk}dH6gprdDBYLU=^wN1KoaSs03Z(lnsF&Fs^ydYxDEo| z<+?@)7H^3%+$`q1!J2@POj&NmMWZZ?!rRJ}My0$~$CGHbnsdtuBCP^Oq$y6&m@Z%Y z77|%0nwIt&rB;MhgfCl@8%tOeEO(JaHk0p-(^v{@VwlETekXg%R)go$54u?ZVc=8- zs3I1hqr}82T*2)hXpp zM+lragtoarN~D+%J#uU1)LS)82q=FfYnJ0)KSSZXj0ya(*YGRt7puEXZL=iOEnzA& zj3<#YhfPb_AqJ8ytUp`7a$F{n*$nS2cnBjgE>;_xP zo7>Z&9M$^FR9U2Wnb9aylrCG3to&ckpoDRdeh#4{!SbeJxn>JePmJaOrW12$5=%e{ zrK5?b1j}$~B;Rh#a@bC=cx&C%cQU@4+czUxtfb|STk&b&dLC_RD`4i)L>t!W0^;6O zO0bGL9W`4krfnWqF{`Ys)2yr{*9+x>Chlb=6Aaw|63WV(x7mdBYs@##OY9oee}*IV z^K`hv&_=(7(gGN%sUjx>&OgPz{Aikxiw)0{4H_EP7fs_xt*$>Lzid-!!}h*1Y(YJLm#B`5}&=hT5# zlAm~|vZGGh@p>eZB}zr3$=8bn?&q;m~G zoaieyzX%h3O`_~mAoqYs?>;@UHn~V&7SimLVbUlNNhb4N{q-uGSwZO_F4i!4SUi`* z`*2taV~eA0(j+zx?6C{%^n!*T=BpaW7Fi466Ht7Z1_%f1UbPKqp zBnRXlE+hYfp>S>58STU6&tqqm|It8q+Rvnf>*sXJ)YLYZND_z}9A^mCPd|W@wvC$f z;z37JT+gCizrpg|P$7+v5dC^B2E`l|kcCEaPA?&+@d~naoK9pLoEc$}F4spA#%L$f z?Aq4wzqMhFloOH#2?+@;TW}1LvekVQ5IpTAf_-h0c)(u|s|1#Ey8j$T`(+ zaxPb%+AxAa1XG~yAR_)oV_z`Pqf+fN)MPqTk8G#cjQnmL;}@$_NhBpznq*w&{j$qX zVpF>Y4FrUrkonpV7enY2%@}4#WO-1i9@@Dn#C>#NgxO8rR}wC&v09xSdtR!u)Nu&; zl<&*N`$HhayXJ`o8i<++iosfMIt|J=_T{)<4W2WyjdnLv{{zMk-p79_r)O(WrWUbN zf(WHkaik*K=*`_3pE~4fn4Q|MHd&A^hapizNEBu{C%hS2qn!7!!pXn7h9QMrd@^M# zldNqPQ#H=))Q7fUtf}CREOlL9C&{GJ0NjQrYxG4c=U9Gz0py;c;mK#Yju67>iY?j* znoaoM3=El^`(j_#^;Z3Cu^AA5HHW1N90W2Mw6m5HuiDN{ z&9;y0Sis?D0j|X`B=XD(3CP*6ss^oibZ?b{A~ke=yez;_5%JAaeA{Te+#amLhr0fF zoI@heH;IPwLIF;sl{4viD@TQw>aEA$$v#^lsPyz6QBwZ(aJhS@05jYr1fa0O++$4Q(qa)f!sAb**>6|MHSjgP{BcZe~FS^JbtlF8d{ zpxCHm%=`nvkoju*ru@8ryl!YXd_Dp*edSC@V=yJ3-pdk%T$h8hnN>kli5ni{6*qcm8M-hlK+Vj!O*l%zx+2} z-M>M1c0ecSGF6SV`{$xonItfUV1QX-UTUHmk?Z zdZ#O`+*1=Z(lBQr@Q?rPilLS>3S5qg>9g3L- zUQwi$n2Qq6VwxQG7(Eq0wyT6ru|fdvyy0lS`dPL)r2~^!(jz9Um*B8OkPD7T_tg)JjzChXWIl?eP_+Uee@-_2Z?@UprIp4ZiX?+1H9_Jygz@Nea4;P1 zoTR0zbEjN-uq8*5!QEN6RMafM@a%vteOTr~UAi>d$z92{nHmb>@O177f$oaO4J5~> zik01ChDoNPnE0of?Gp+Bv|&$yb!Xghnj5B#gwn)?QlJ!==BT1A2nV<0dsJn^x1O%{ z)_0U#dXh6aC5w3@pVlmO)+xE(TKF)Zr-4A_=}u1t>D8(j**i5#QT8#kwDj(VkN$-- z{ofq^`3+0fX7vsY&GH=I%FK|P$dL7V6>v%z@#^)S6EUDwn*KLT4^A!8fS1~6->GrQ zD|7>18@+^KN(y`P?%V#!Wu=0OaG@gN$ewT6e2?=uoI+lubJ9kYhjXSCV}wkpZK%97 zr{txxBMUUOk288=Cq1m1a~o$+)Y+51Vdg$H&~Z34{6aFs>lXKSvfA1mJaRB5Fv=}r zsV7AR^C?^f5wFb&Ya3MBOcm{#Ti1=9{j_GPAnTAeyl-7m{mqNr0~a!wz^+F{)W(BK z+MC~CL^!sH$C2`XvPsZ83;q5n3+E@GD_HsCMvBJSc$N#6dcxbn!dxJia@o0v2<4JO zm9<^%*CF<(u%_VpP;igmikIK+7>@KYTkyxnEJCl8yG7iVOP7>YCR(_-<+>ai&XphR zjI%FuoSxxHZf+5GSRG4wmlU=mTpj5PdCAQKRPKo7XNgo&Cdxxb^9OEQQ!>e2HCZ&i zQpiJH}dT`kvZvSRyEs7}K4gqF)iV z4k|PjGi)6U)cVp563rHI3}O@!`e@7UBzMRK<8T}U#nDW~7Qc%smf5Gpsa_mUb}Q=xG}mp&Nc&q|GdC8KJk3s9i2&t#v{fgCk9B zHx5iyd_w+Yvt2=>I+Ek9kmVmDcRS$Tk=qwg8u!;Wa>?=0n#y&(gotyZ7qHoNR)Xyx z>&ovIJG*-P*DqJoy+gYUncC>ES#e)|Ev2{iiGoZl%JOFbz0+DGyYLlksfJ& zk>m}bLz0!RJy;*x+7`2oDE?Q8XBW&sJgQXT)vvK4%8+uMSkSjCXX$gq0H(T63lh;l zZ_bRAc&U^!9E@o5-@G>e_U^M$iu8&+54)N1qjEm8KuzX2cSC*bm%8vlCixzFR&M3E z&qHQ@_N+DCOz+7Ve!p$D+XSG^YD@pOdsk@MrijT~-^s+ElIv<3^nUtah)QiQ-JQ>h zx4+`lhAlc`Ecv%<^8f6_vsVh-4-cObJ zPIgMS5Hlel7(br8zKqueV^17x*RXw`x5Uj>H5Znxaf&Bsk$gNIYuG#MA57-i63`o^BG=jUayZJ&Ee(w|7>$^XgReQ*tg3 z5e!9!lT%#$bCmJo#UuMSWqMoP0(wi?YW)g0@vN5}P$Me4+U}uJqu2QyTzthI!(tbC zfhJsGb^JoQN7uqP8P~0sVQFa>PWb$$y+|>)Q#JKx%&mKm3QJ7Od)Bkyxt!O>6IIMj zXhoT%Fy=4!LansBGHtBG9z^E#X$*Bt6~BBV!>q3Za@Q6NiM+PdCN9o(#hwbw5DrU4 zPqcvXv&t5oo{Y=wtC<8)Y!~JBOU|9d4%e`JOFgvv%5W9fvQnL?$!Vh5D+zXMm{=!z z$=d98&fJ)_tj7A&ko2Iaz2;uFMQG_YA@25}-0K~2oA1^dzko}ntxo1{y>2jLhcqd1 zl=KQ74tcvFGUFpqcq{}s(&DgtADspdWXNFb7?;q$8EsF^Nq`>PBd&(nrWFI!>C40Q zMHmRHOO=M3`>-pdRczXmax5wgmbKTL>@!-fRXl~ntiV%m$ofiJ5 z@0SMA8aZ%8ro)p5%+SzCqCF|Q>BfJ5uYd2znUBPQM*S|t=cVi@p6VDk*PSS(Nyhr{ zCma*fLwzgSvPpOL)p)G76lIy$`*7K(`iWg&45d*xgw|@a2t8eLi8w@J^+M(|nDV3U zCTGzQgYT%;(WGq($ct$aMn3+ zu_TBM(r;T%{7&X3x&hzQWiPZ^SywZH`y4iZd}K0qKZ!O-X5rl)%YU{W*58;5 zoSiw+>pge2Y$ZIVqA6oa_rdX9#bT0^RXd+HOq@@;3_5xlUZo6ITxcdh;_4*sOjW~M zY#OQBJ2_Wz z47g^;c>Q?UZ}wD&VYiOhP_V7@Y|FfOX#l<3Vl=OiHM}QYNr}7MPOF8dSij3`w8P%~arDc6YhS^4=+<|1@H zb~t0ieWJa+d&9{DyRRH@{$gYlX}~qx1u=izXa4Lt^Y8n{uN-C8pZ!&0E$kb`wST9~ z1c+Uau95JaO~_pjl{Pc$39z2F-2U=ws;Ro=(X*cSlSjQL@=xjL*)ZXVj*V$n znSKcf#)grtr(7BSF)UxT{>8jP$E6bsa*SnoQx9@Iz}ey(yYR^)-R+CN?F>`8NcQk4 z2bb^SC}@0_aGajXKB-^Awrx$}MlI@Ta2aM3MX}1t<|P)$Sb6^8bPcdDye!9S^x=xG zI5(ilUix+vt?^6{e$aEL2CBzX8#}Ja#>-wq{7AC0Q#{|vUNc=G(erv|oZMH~5B*L? zAp2u;_K$skZvHWSE!mg_V)lL15E36T%=k<51%ruVWdq9(KXlI7D5y~6$5{u0 zt2p@FW5#Z-C^3!Q+I8Aed#A2Sm#?1KRK(3lAFHIv)OkRpb!48J+s1>Z-y9PU_xztL z6x`#E9(X@IS)J77+ou%Q+RJ%N*bRz{!mwDISe5r)bbs=SHKBP^Mz{3HCP?s6dY7KY z@kESVw90l$uKpiuM#zq&4zUaBInD|-IB~&SN|kotoP83&qRFre*+1&|A8*g-Eu!U; z2K-cqBCHvyF9pU9itY(>uL5~;b55Ur+1|u%(j2{OCCNA&8g;&`z01f~ z1|;p`r=m9g`%P93mJT5yp8s(r{`G~@q7I$8OkXYHe{FNWx^A5$VXHnfaq$9$_Y3Y^ z6qFeejw^dVN5GzqvksUv<<$Jc<^JbwFHp2VnP0Yywh}^Y zb}GopV>p|yUiQeYJ+)&me|Gb(Puuca!TzC1&a{3Gr@XKe+TE1eI1=}(wvZDN1+nNE+VHN7yfZ2v!#5c5be}^BQ(>5Q zFW!d~TF}iHykLv8>$ExV?8pUEAJlkKfE$Wq(c@hkzt_-}q7@V-cxQ4jY@YsraM0fV zkwPt4(J97DR=zx^qZ@x{8{SB{Uohk?AE&;M78fH2XA3nb zK8i>z0oG*8UNJYJa-uY#qAp)or^Rj+6vw76rU7!21x>k81GW90)_f$An~xS{!zsp! zHf4qPue^MEDh+&cG9S8cMd0LBxx!;BxYiNZt;Wu~-8t40Y!Qxt8j*s4r123Y67$Vv z_DK1O1HgiMUFk19;(JbL?4b4R8Lfu3C7~p8*C)kj3jXnlbbCj~@P5gP3Ol{+dUrIK z-d1D1GzgICu44e0Ga$LJpl#x^`$URFLKAYD`Z9X6n3fooWXnpMr2P-Bg|ZgVhl{7x z`}22>eUf?=dS+o!9zFN@$Ihz}mB!4$q_@pVp!9NyXtFx%5%o0SEKp#y(?wUHvkEL| zDh32VDtuY3+mCtZ3hy@msz;i3W#s@PMTmc=VSf0{CHr=}qpb?1K1|Cgt)*^m_#2f{ zdXMT+H)oBIGNIzCh9C$zbKr8YyGJG3wEs?YK=@*@l2#dYVByu2ZI+Bu(zqgTJp=?o zkqMuU*q?3BFWe7K>o&I4Z+O^L_YyW!`dQA#@Jp_UE^898qh1p8%;87%|MRW1(?ZFP zP~SVJ9X-hVIgL-%o+R3~=U${6UL!`;w)PLbPvfUmM?)%@>a4>ZQF$#nGfSD% zq!q$_6-z!9Xy6VW!lH+u@T(NyCCDp7sI+g`JSU(*##$5q0P7*b~3UiS{|`!)E) z>yGlGV!7{j^)jYeS4CO6HmP2WyotFn==fi&{BzGw8k=Ay8d%fn@%BPW!a~%>diT2A zoy{$}?#bwpW`0PTc$;hPaVk!A(0S|n=H`&Mm}XVZg@w31UZ%!CX|MUg-SM*dAyTMQ zKwi0f(#D+UTXTlM!;HoD&2+QvPumz-hKQZvw~_3oiz)Y>kqEe4d~!(Pi1#Cne(oU% zDSmscCOO*Wd3&Bs-BEdtPYso=@QMK`+KvH=QI{!c&K!mn_-6VmhtBe)&BRp|B+7vB zetMz~IC5-!R7!LTWNke%YX+bI`qCN9B+?s>n9g#I;ON`6zjMZudu2q;9cNg{!mw*k zrV~nL%ISwG@E~2yuh5flwPEU;FaoR%R{)@ryCv^|t%Ez7Rf7Hu9VF5E>MW+!yxRdG z;+3sc_&r4ndwQ9~7-lSuIv#eLg$!|Fk(t*$1_3;UJ%(l<)1Fy|C+hQjEr4)Q`YfLeDRbd&SM;XJ zmJEDGSAL>MdAda?;GBJS*8W&H$C@{7;a%`Z`@o4-%bY@-3mw~K;hmIsiR<@9r6UTM zKAO7XRV3W+I-WdH&QgN4s}qyqR>F#BoSoimNQ_=_7i?Ho0?6o5TgcM3(gmub42+hF zWemR#rF;e$_4#vZ-?#dcoaWItyS;^v#nRflujyQ>%RC&XvV<6}IOCI%MTi;nME0h% zX-QjKmhMv9tn5IJWkCGBNE_dzE7X~;$;EuhSP%z}y{urs)nUr~G%hL5)kw&dXU?eZD;*hq@cBxK^6~y5nrtSyI>DT*_M8EpjgVT{A(xls%{*|ohhhl) zRfbrcHN`n8iDG4w{KMgm42*DYeww>c$qJg1q!8&WvpAu8g-|agCB@#?Gr-pcI(D<| ztrkS5)nGY%ul|X@-?CKvwzNAiyd5U+22u=Y-k9B7A#&J;u01^B#`H~rb-GVVq@BB5 z1KPvCWggp^niY~Yzb4g;;ts|vTxHpZP8`P^^1e2jX!_1eKc^#D@qFXL7eW8=Uta?sIdqofAaG0<$ zjS&xrJYIxaxy;!MpN<|oErx_Rey)@Tfy?a$wRYQm?P$|(F4Q*(3IekSR9{=I+xt7{ zt#|D*NvN846FC2=8A}}*1!+TvICB9*Wl?2zk=*c`-UxqZuP*u|tK-{W6_01$X>H5KO_cW9EFTA;;gzvOV=&ISO_T z(>#4oV-{mMZRKxIM}$;LzLWK&9ahLi5f3X;_NKxqW8A#JN$r7mZ98edlcn&q`1n3a zNRDdFxIJhzFhix%6IoXwbHxw$%ApR|ZBIY+F7e&)Wxt21T;4>XuNrY^ zyW1#a-+w~l-N@|cZ012)FI6>9h3*U<&aWLVRlrHh2!b=(BxUkZdS1&ij4L!>`P--N zm1FU1F_S6}3xz|ox26W+scw-Pyl<^o#cgIg_xoCfWkNxMqhNU7l zOP5-M@GFkT#}qW!C8^Plug2aX4l3+WzD_N^{rV5sR=Pun!U<7Q*eIl(iwHO01tfEosTDe@R)ua;p z+FQPlO?$S^I!C+NQ8W)jwG=8mf~?5kHWQb#jATLvC7>)d5;E8ZpkQaJ#T&@dWERG)9?_bY|2;MR>|06p%Hkw}|k#mDPB zi+-ItXcy>wYg5!!KO#qVYw{qn{-O~I?W%N!enS5t#9(QYsXZI&Y z)FS#XIX*1%vl|H;=D$ShhM_JAmhJt6NV@A<0QYy|$}q8zsHM^2wrBu}tpD%bbjAc$ zUvJ@LzD>_bA?<{Wm%D!5{e9bYaefw=IpqbMsdO_#g zSbC?Y-&p67v+}F9YHVSbRbZm%97%z6$IZ&t?pn{k8;CWZ8JLl3%4jZ|;cfqVdo{si z+}%l(VeaIn{7tj#vOti%!jy%`7K+H}r>&D!QC(YH3a{F?9+)_hi>{e!wk`}jp!{n7 zEnDQCYeDZEd%WlApjyCYq$g?hB;uG@HeIF zC1d&XNCrEGG>QGBht*ymLt<|-4n9s31sy3((x$&*=16(p%5T_UjRu|bHT4P&Tr zxtPX1eMwMdk4_)X*kpfwd(txd&Oyk-KuO`+okrGs!OGvs`VM*x2J)N3&v__BoXmEb zae>x$cXNDrG|FC8n4bmN@TE~InU$F*Jyu>VY*LiSG7Bv%A5JScw%6VhX#EIOC8Oi| zqrd;*4_E!^4_~&S^iub0GmZ=Le{5kkMfgqzUeU6NYw_yNnB|G{94f@`Zp#KO;^^b* z<95L)g@|_b(!u>cxn((5@Y-FQLXN9}xR!^nqD5UV0^ z!P$CMuHc-w);fFS2NTv_plkn(2bUDTLH9jIu249bP8pV^Iru}AvyL+T3xAV6zJRa& zIhu6&C{li2I&S-kzb~bD*j`GmU!U$W#Pn4Jvh(A0%h*}awmXlRt5dQ}&}MlYBnE>q(sgIlU}%>nuw;;%<`V4v0ve{327e;yHSG?X zA<}1OIQ&(*;7=MkxYGbv-)sF@`taA`ze@jm;t!zyr{HO)zg&@?J7};R#&`e|8RDM3 z0eD7B?LQUul?+1aLxbA;Lp3w$l}&b$LXhzJifCh*-pzXAndtSp#p31>A$9qYze-n+ zNMk;x&v)Whqkfj2{&o1T(*K_L1E~Kgc#_PRR~&8M;K9do1@`?`J2#@VKkGDm(=C$v z@brh;t;_AP6q|b1+*jRa*fxTwoFzbOlwegY1i-^jWMnk9SK{#O)M7*VfhGWgmcQ5m z^B^+)DShJaSO0>V$;AvNyE+P~o;?Q|>ojq{veXBVn2dBDxGT3mNqc}jx5P7kX4=2c zJiNp#)*th;RKoi6@L#3>J@E%n|5Nak)vMSC+^9D%O2pG*h_0W&q{%$LtdJG}Dji}m zZCDZ*v&d<&5*$@y=0BQ@G*t4YD*i$qv}G={t;PP4o}5tUHJ^C);Ncg)gr08$7dC=E z9tx2bsUSPK_TK!~cKyAEZ#SRRX)XlVJym#lhyMSt_tjBRt!>|<9OWn_(jh1aNJ|VI zD%}z@lnm0+(j6A*fOLoSkkSn*-92s*J870?|a|- z?zpbs^}E{Kx4IsB{rQYN^msbu3Y05;)iQOH z0m%wLulj;g(1Z7l@GcfKafjT`^(VVYQ~T^2Q}`xO`W!&%|1Q`oze@kjuNngVF8yCM z^!ax+{;r|l)%gD-`q7I2DkH;(r%s;@cQrb-KUJFb>m+<0(#S06r=xGhe0Yc)2cvmBUe61vP`Qr(fkZT_}BEm>x_y5}_=4C~=3`_T=#+c8M>{yzSaAbfH$ z{m0p(-;;u4zZu-mTI!#Uxa%)qW+lz){;+K%+5V8*WHyW-_6t*r0`{t964rfj{Wu!X zu>2-lV%7XRGdBPGA77_6m2U~JWbc`zedA%-HU6d{--+K8-aJ&%EE)q(>=M5nwC-QmqQhDl?;&~t8ZK`2jQf(!v-QAB_!1x)ue2Kl zXn_p?E$||s1#Y<10vp?kx{dC1gQL#KM7TP}vv(nLtAJpN#?D_o#iUU+>>a+4(N`lF zyxD~8^fe%)qr<}~9qTy^3CxGo>j}w}wOHF7>l*A^Z+0LmYuqmJ*|?uJS}s6KA+PxfbHw=&buI4(dEX}kemo~E8Y zNeV4JyiPp|ImnK6*Myu_Qn#$!zFR1i$orizY&iMq{j9g4&CDtA`(|t>WYBv$AHxUS z1C8<^+mXzl9=7}Kxx&ewh(W*-2d!amQP>3G?6Z+%sr%Puer@;GA@!et*CKIB_Z{I&>q1`;_BPsAcw zL!H1sSOm^J?x{~_c-4fiI-eyUTo)Jl9-jCzJZ3la{3h<@>)(G;BDU_13eWxyJ|9v4 z+P7QM69`iKE6Dp(LP|LBcbPEC%Y@mQK*<49-=^TM=iPsn0f9$RxF@@ z>=A$KOl*b(Qp%#{YCFSw>mc%+QiSpw?!oI~e`!s^gh^64opwBP9Q5sD7&(`CdDeYCp zcOtBXt@B+bksr3kny}hk$KVM zM0M}nAG7Z{$#)?baI>H^2;O15pW)vLalfHUBotf}Y@0enm3MIMHL$Nmrb@=jMr1eK zl%Z*(k!&;(9O7AWoXp0&`~4HwXq(xsgcx>DJj-rxe!uhYr)$&~pnZZ1(8?`#;34A! zFC1lPqe4WkqP>iV&2TMplElH}3cKL|P()Z_oKN0ni_vOtxrXCZW z*Ltf|P%*FGVK(oOmVIC%egP7}<-@HIVOD>AC-< z(uuL8W;p!xWhco_|8dIH_tfE%#Y5z~na(-KoqPuI0+jq$%T_R#yCd6lW~cGzwrai>OwUPs7_Gr?c4^Gv(F5 zSF6a0@EupFuG2+ta?4!Ra!mdr#+h-pIF)ZN^Nqy^LvkBS%dq0znh8HVM`8%kc|3ZP zDJ@?iB3R-jaboHq`)5dy#Z;_Bt zv2vlD?b^>4$6DqvhK4uo$bkx3tg!A8HA$$HMQizXd$KdD!O>bNbWk(*rc&A2LMutw zLh6*1y$NM{9CrndZm_WcTA_EZbCPyd^s4w%w$oV`r z7a+y8Khkq-HZ&>+GX3>dj15QtpWs%^Bd;58_59&?lx_fP$uzj68TiR^00RBB5}?FU zdbUNj+tsn#^4&pNoU;qwrGqq^vQ{($J9J4Pl%`TC(+hpzd}6ONGkD?@FPf+{l4e43 zIy-$&$*Ai5`%2&!9&DVvC({1&8Di5V@LO?b0|o2b$m%o46F~z8tPmQLMrEtmJ}zX` zjRX{>aiMt)<2HfxN;t+Deza8h1HUbk4~g;Rb=NgwcZ<*dB3&3T8H` zVXTseBF}77&oP4Iwc17ol)K>0^pUWi8_=srC%fz54&aVxPmU+-#fTXBr z-*MGA`PA_zoyiI7Q0fkqpWX^u#3w&k_e;<};pxe2oj-jTV9h$qH6L>U8Z6@V*izEZ z88d*I*58<(Q$3?LIa3hDW;>lNzW}-S9IYP=#+=4gs-2i_YV|BaCZb2nT!KSdj`ZGc z-Kp25>^V=5u2I|BI}zU8*ha8hyvHvF@b3uVxw|z^wGsso!CEP^0E0!X+pQ=CUPVNI z{i#~a=z-TDkczM44c6y}N!z2FnsgFMu&r(3;D}Xn*L@lukEDIR3lKK*>HPOp03Ru@ za_WqeJ*xu?9o`Y78d^HgEgQJf)U44D_7}V3CR-^uF-IIw& zPj`;6`=+<}MH5X%cplE@%^5pD+`5eMuEYwSEi>$Pp8TPz_+o;MBqJXxW)+Kl2x0^nBq=TU(?-SjyDJ)2YPj+zyo3rSOLIyc-Fv`23ObD@cFZ{r zo+-^DsH)X|ZH`o?PgvGN8NoZjBcQ5W{W~TcxMpT>!<$@=|X!zZ&7q99y zBvQI2YTA*-s4TfoKie(HmoDEhz*4&@>U{w^e|C(1^6l3g_TYRa$($5V^zpv+K zxLP}}UKx~jIzbKlN|LrmA5@zDiJ0NDN~oz#1mFvn8)BDmn+aj{z&{2_v=HM5r@Q;j z>0?d&$6aZ)2}nQM36Cl!>`p+Q=8n<40g{Kts=otSsJWu9tz~(KHS{hTrK7*+$!e0I zBBw%=Zq`~YAm#B(-Fm0NA57QH@w|wC&sDQp(v1V`{xsJXoaJZ5ktd|y0=jj%^TE3H z*!V_rBE1Y@uHAk3N|5>3!l)KtC9Ikq!K8sMR7;04?axa{&jKt0ybhC&;us~sOIL#d z_LW>kw@#L`kSoqV^7-#)75BRbi%r3z?8wf`#n3f|#0!vy4($?|*Z$|~e-HlS4kIhs z=6s`Q-(57z)s3Hl@J};x8uWHGo3dDvv-{u;!YMw@(EJq0eKs zWk<;0yAf$EsJFi79J^vEC~lfL7As>MMh25=qaE6^ZEf^i%f9|2y;r2S1IL4JoSskT zy{-2C&#Ad0bpbjP2OJ^%ak&D#DNDWYSG)vb>;2E6K{w^zd9IwQ@4gpRe0mv(F>xiy zZg4jo^bD|<`ry03d+=Yp2vTv^IoS9LIAcIQ`L%33I!3kJx2bzEcDT=xy&&P#==*P< zy`p1NKGC3j8hZhPeP-6-O({u$3%*OVpyTae6RDoshx6JckTW4-_c2qwZmArX2$VJi z#!=tqlBDP~UWuTHG)D*sbf@Wm+21zJDsnj$x%B zvl%|{0opdl^+6J@ZQ)KVLehk6y1F|vl~$ZpLM7m)!n_VuABsSYEd~~<4R!NBX_`w< zJ1-J&pL%_lqVj#S!M57n)F;Jv^RQeD;QmDnrx15Db4>evK9%l>3cvUrDKiH4&rSNt zBrRC$0*X|oPV8m|^VT0xGJZcB+FzPxifMMM$28LjtpNAkOIW+`@!4|56qVSh*Ww8k zqhUfBQNC>sM9D#<9IZ{Hp>fj%<;}K&%-W2S8q)}n@?&NW!caz z+z^V+|PERaP}ix<;xPUL%5H zoIH@JXLj=Rif1nIjsC6K0Se1=DG|}(o+rao9PNv0>ko7uZWVMM*dCB+WV5@pb#B&n z&Sy8o_4^AKOVAGI37jYn7P3uuAl}MO;j$$vp(2z7b!xKeZwM3?Go3dlmRfTIc&ZGe z`2ac@*TL1!=Bzc$yzA8<%6X zPv-nt^_E+e+|rYYcxIH?&4WWq@j1H2idbzq+RXOuTFRoP^OhOwgEn5=BPS&pRm}`& zPk2K8DzWI!!)y`ok{eS`j*ifl=+z5Q(l%C4&{+XH7u31^=qaT^%Rn3IoLsr`rX0)- z;JETUfvmGoV?5*9F~duU+PFn^y~1Q+kXn#^j9G!%yW#oKerW3Zv{q(`x645ylkt1c zDQ>G$s?DWUxTq?cHfjUDwnNa3%Uhma<9A7|mPvCZ-Ve!_R&~dSTa4$Uy5T%O`wqxu zH>gAX>jGV)6n4}oMv_h#O%Bz;ekz1K5B7hgS9_L_{AGP+ zN0<}B zNrNTTcw~^MxA&XqnhTcDsy)h)(>Y8C~ERYD5A);1HJ;4`}&D-V>QeaMu6L*PekuX-#tVd!h90w_la(G4P<}sTOd6QnU z$X)RpDjrEsbGVbeBf$8vNjsFGyfoNQHY@9Sg>q21GR|L{?1cdS4%eHn6l*z&`VEq{?(oH72}@OW8Rk&TEkW5j ziU67nVDa$ZlC1ohpE`_|lI{6}EH`v0^2N|L%@=OJg`;_c56LAxw@V;YOy(?;@#S<_ zL$gMvRX3s%wBB#if<{^Sb3TzLhL=AaCho*JLmNg5G4V&>wg)guUdUl|y;3BSuYMMj?kEw}%#p0fdF;jZ=~( zAH4sEmn~_|F`KDhr~5O`a@l77HGfxhwu>j~l}`br66`Z=tkPIn(yoziwwQ>c`A~r! z+^Xa-8<*A5R9CQBv9zqLc~!`^i>F^JgoqjW9YqM<_`3!o4?vA;u7{cd8i-JU%>vLs z_|HKAln7(q=JD%YmZTz;K%@a~akGi0uAKN%KcBcMj{o%>!yjpqa#)HCR9$7szo*Q< z_)Z5Ss)8YDA?KTXM!e2d{HL0A6+;unf1N-Vr?uM=EW?mDkLJ?G&FpaDW07&_>00RqmP zx97_Ox_kHOpZ~10-@=bS<6^K(Rw)r(Mbsh_HU%u@zMft#t4>irG$GFr*mc|!vl!Y0 z!!GzvA_Ae~oX6|{>(^8-m!-^%vMrZ9ZkL@70;c>mKNtOS2(8I>*E&-z>x+E9nAdU+L_z2L zX;IhnBH6VT)Wm7;&{wU&H7q8ZkrjUab(mN`M-xbxu|(i#+Q(+0~WnX>M)Iver=&gDp8J&u~SG^4pt68b=}%patA75S6+< z5mVQz_9_+`^;r!fHIO`lQJ->5@8%+Y#jQx<@FQo?z5D6+N6t{*Wbx?x?6VuP% zh0kT{h;Ar4(ypQ^6S+(jAhfJ$n<1*6O~P2@ErSbS)*!t^L=f}wThZOR&^Ya{TV`N@ zf`PYRC~3-?jLf8F=&%KD2vxFHYM;J z5@U%mf8m#3wHDB?@2Y*Rq3+H=zZ%jsqIi+P8Z8lg`ktkzeK=0Thd99m@^=So{eXkD zcN9(R_ESXX@fV;>GM8J|sDm%(yOwC}6IzVofM?2Kcr>Q9e1Rr{M#_nW{AgUS@p~Jy zPFN}Z+idDg2oruIz*AgKPp{oUY0h5!b-{lV4se?Ujd2Yg^jzy}vjBfE*ALQ|f zz5;Ge1qTdnTRJh6?7D3PpT&VnKmX{4*X)`dQ~C4|BI1DXAa~%|^FF>a{x z_`+oHSAa<$boiG^Uq=>)Rr_-;($++z$l?uD>E^mbL_i~iLyJIBDSa=J9Y<)piw-Is z6yldPv>XDv?hfSVUxfzOavT^R9dBp^{HYOFB?EhJ=5Ch8smSzl36q&bwNE#6-R_gx*uv69m1XQBV;SoFo-4OOXgXLrKPA~H9siukn(_8ztWsL}_UE36cU z2J*#VE#Zy1L-mu*+oYk+Pl(d|snJ~-M4P(NQ?2Z1Opv~mwq9&rS{bvYKcloQ)m?cN{i7@)6DEBy1h>V` zV`}ia=fU&CA$(ys9L!W%zunn5Jp#Q=iE2GYBH9H@B$TF#Yl*G zLb+gw|KK2GL$$87>X1pj7^diK;$I|r7Qt&UU=RDm|07Dqf2{N{o7Btzn*tj8wQTVX z8-_={5q%niBYUpSF-$V$A1kQ5{;xOM|3+Q>dpB&GJ9_hR$F}IEK)~Bjru7lDnnBc@ z)GGjMae0Ux#iWh`{-96o$P@M&SJ&8UxxHST&S)6RR8hev8AKWAJG#2+#x1vAp}87_ z>APst9`xGbvq~ojx47-f9LRhEfzmz#gK(h8^GjJErKL8|%DaG8{-kko2k;Q1m2?wv zD=uI{z#s6-9Uw|U-R8Lqt&>#yH%pmvszdr)*L$BGW35w}apU(0Z8u@T zj_F*%aBHo~ZXJNIx1aS&23gu(h9F@aOKW#1FdW`=+OLQJb^#_;3={i8iy#uP*TTJH zoHK6_Xc-K*i(imT0T>60Xzt+`H`Y=2hv+A%8uYDHGjXbgSwF%J{(rMsw$JUK2Xx4f zyjWub9GJGE)yyZiX}v#uZ?VYjVLPs?W^Xhnex4tL`i89A6t*G zh5gW;J zgdtV2oNCl$o|YJY^V5=gx_{nzt-8r4oYp=qTUM_0LOvhMT4#cEwe#@&{T^NEPH%{~ z1VmN2I-T=qJcdk?N2;H zaNM3XK}oZMa!>BM!E8zMK`whZ)-XQeC4VeSbx<2J@tW0Xm#gb^ctW14YAG&Ne)YV8 z$V`fY3MSO^$%KcEHD0mK=kYjVGd2GKtwKVMQX)t{Kk14r;o9Om4^Q;P}^ya5_I z<5+_@EuXhBq)luJr*dMlkU6~Ww(TPefirGu$$Y_%R!nB`(6|BSf&P?5kB-%049JYd z6fH&YXT7ZXoNOtrG;Dy_ZX8huK2qz$QwPOsy~LZw3?=@?y9b##q3f zq&Xs<`8dNG-C!-LAzB$(PO?UYWhMh^=Gp<8Gjd8Xq**oo3^d$W1#dKSji#ud>-kKu zCDt$#7TRSBQnfJt+`NK7zo|-?JHz|R`oV_g>LCK*$K5L>!x1N64f+Awdy(k{;Y+*H zuY6Nk0fAL_YB463_LHRY@;C?Zr zNTZ-86V1~LQ2GU^{jo+s4${mv@Q1-^waYLXskwO|){nHXTj1LsS$=Xjn{Ao7_yfkq z5D+vD{wePL;Ul9!we%Ph&wxsIP)qa@Qn}d+O1^$V|ZJ~N?kycl)cFc$a>!RW5 zPu-HD(IB>ULh>~KiE<7!&aQBbE3 zN1IMj;YyZ&)jHsIr0?c3WtXvXP?!NCXiANjT#IuHmXWx0@&{KYDMW~FsD8WEbxtSI z&+BJzZT8gT|;79D;#4q#ju-;rOT8p%cRMuh1hk5!Y zm1fn;G!gGBoKn~J)P zmkspOCP!DJsiE-}oeuN%4wM$mBdFlX3y_SrlmmUpJX==IA7pM?o*Ew$!it0=c-%zj zZxap*IX77p&v1}E2rEx^qCF42@}b3MoPa1pW$;MI2lwmIl71b{VvBCef|-OFyu#x1 z-eghZo}2nfFYcr7>AHz}zn${3LB-PwuRssGKNeF`yMF|WYsR#Vd!)Tcdf!Al@hAi$ zXCp4({i)!7r?@H01?UYNYhjW45#k7uZofn>%tg;j1S?+XXl}`&82h_MIsBgMxVC|P zB$z%%z}(JitW6s2v##-OxlW0=F_P?K+uL#!#=SIGtm@90 ze+zo+4J6QL=jmJ_KV^8QTPgyojb|<#T1KBUL|{jq?r=vz$vZU$fn6wOM3rlp<+r!s zT3lfCl=CuM|HRup)GAh;-^;(cslWB?W%-MF#Zf~iRSj=t(gdQ|;jKW?yu7YnIn=6t zifs#}4MM-AK?jop-#=p|o1w1T2+V0zGM-}Z)5Xy!C2wDO#NV5!U~q1V`6l7U0#bD8 zL&I0y{z&(B*N4vk`ovidtzyTx@?~cEi3>RRIE@O|o*_)*0u=jrmVH&GGc)yf9KTm;LSio zo_S&p0F5(i*$NE+dr%}YE8nw=3lV6PgBr+0UwHrmVFrpKUTV3|AKB(*8-QSKX7!es_+! zpzw~-HUrv5R=++y{q}uVW|&`g;VYvfiMeKy0>R`_E5eN^z-h-3OWJ^(%^L|b8Ews| zJF;u}lPF682Buef|B55Pz{z%?!(sHUj;>w5XOasSG<2(qZM^E+@FITtmM(EJvTRwl z2#wZ=jxQ_>DbTMdN|}xH4&XMhA0iSzrMG)K9hI@kYZ7P^}W5d zKhI|>WUB#PQz)b6P1BPGsM31^&c<)6Tm^uqVYPBJII5?`{OWm6t6Y5=QA&M1IdK&;B@Q#XvA(Q2}be9TekNRg@nSOkH z*;X!}F!0nF{M$QkzAo%7EDd)$^Y(hm&tHx{9+(T=!06-rEHOTQ>^S&38m}$htLyGJ z>*UjzmeYPbtgwNwI9-mJUorpw)~v0ot)2PO@8rDs{V_HrMMX>ZmfunT`qsO))wZsJ z=-*NQ{#K$Lf74V6daZNBK=8~k{>KzSS|l;u;=ks{;teupMBj25=#7G zC}Ibuj;DMHWN~vCvppznJ33Qg4;D2=sATJ4Vd`xDYq($IfIWm@+JrMUhI7nPK=H$9 zhCrizL(;DqdxRN@_&GX4 zBYXHXO;2K_i#6|I>__Wz={8pj`L=#_yFUD)%IZ%zzJWo(%{iI~d#EmHLIM+Sn#plL zXMaC>T-HqEu+eBH;lygi<89v2$VNE!5lB_IPh=Uf`~Xy$a{;=C*J`~E!i(I5q^mSl zbC&Vlh4;(RR+Bf=uO*v5<1k}mpNw6swwo`>Rb{@LN4`OqZUS!#R(j_yinuA>B?tka znOQ~Er!dT%FJ-TPju|i5zmFN-LKHu*-u!xo@$2IEqyI&L^ou6$_LA}4p`Lhw5^Z8N z(unV+iKaYAnJr66%`CmPJoS0Xiwnm>YajtWgqd18L=D2*wEF`U-P2l|#W2Qc zb-6UF4w%-n{jN>Eor_eJo_jFQ886VTnyE}|s2}@{1HaJp?XzDOWU}9M^AlumE(MnGv;TcBGGrr2gZQ3^x7Naf$+sgsP zb0V>y=X5fX#3=s24P&H$2{t#`8_Fs808Of`yR5H@y}*F-ak^j zyZDgwea*;d!GK}^)1pYGN{DJ|gdinmGAQ&xI()mexi;9Ww9D2iNMcP*O%-zu^T{#Y z+cQbLBFrM6kp5_t@>ptSp+hd0r_9G3LTjtlqV=VZJ#?9$$OIYn&zaH@Dn!1C&T(v= zkV!4S85$^`XoL3cy@CO7h{uwE^gy3M$c;z_EZ?tcvLh4jME4{nY5n~MA&0|OLl}|T zSZ~`~C~TvvoV3Zao5ybz8JZQCWePC*?_+#)5B=(yiaCnGdPzO5?qFdhlE+81b|?j- zZ)~#7Xs_nDwNWn%M;Ae>p%k~I<3t!Wf(`1ppWY4pQb(41#<`lsdvrg7VkOnTS(M~` zBg9hPl9cIl()z4ENBxX9dvD8h7CH*?#SC-3z~{yIee?+so-k3!hA13~?1W+j1 z*gVQ=v^H#-ku zo6HSSS`^q?xK-b3QnsJiEmLe%TNDuV@tPZF2N+--os@!9t33R=3r%>Py)F#|@8!Z3 zU{8cggVi4_@@=}avr&4?MD3NI5q}V2K08T8lY8(|^8x!;VV(kQ z=D`ivC4BDrFg4}iwB3qF0Rx+bZ9;sztbr6kK(!G@sI*DR$#K!kQ$;j=#^juv(vbED zU`58V*;A^2_>s5%v|o4gEX@)>BlwqUJ8b98PpI6IUth7}X7HaD4X=0Jt{r9d+oa#Q zD*S9Id*`E~Ra)K`%JmOD#zoq@0<4EBneBBlhIT`DDnCA}Wh?!*s^+1*mLXekU#5!| zq7Yp!CTS{A3|d~Flg&GxSvHDH*#wi1Cn;8z%!w^2;;VP3oBiz7Jv-VnEa(z^k9p$6{?cMg zHY`q7BRjV0s_V~VC-no+yS_nGBypLzSyyN{>L9d+x@$NQ*W5 z0V#xYk{YR4xfTkZT|?ZIWP(`cC>18y8(!@JxPyw zNqQ6xTJf%)a_j*9bK^E5@*@VKZDxH(DyCheCs+guJ_v{UaVm0{PL21mK8sbhoOa&Zc3R^}Cz!qIr@P-fhMog7_vs$>)VkmxgSa-T%6DRq6>Ul$r<9UC&AP3j&+RIE#Wvz!oRP%0}mm`lR>EJRMim?;pWqHsOe%%A5W`4S%*g{u} z+)0O=f=^iuH&)94&F9;CFp^^hbbnc`#+AU!Vj#U^)wGYFUDM^X_DqtG zd!xc2lc1$fVSnpt(`6y0ZhN5!5_hpg4IjZkSDtKpBsnv_3}p`ys}1J-0?ZF!XUcg{94mP_0-H?<;EN`Sfl6 zbzS@!+4NGPyy`R#ExNPLq}X^jG>ylM!3oPtoYp9fsp?_B%Db;`ek&rSF;p((F(4f@ z*Gk->0XT1N)v7j}v1aSL8rrO{4)IuKl^J$jfa0&2UN_hJmhJqpy^YOWKBw<}@wl+6 zK8K5iqbddTT%-f=%gvt}lE{tmDQ$Neg_Y*szccr>3w)~_6|2hFMclAY zx!+@@;{yj?@?~lKKCVdE)cObIWq|JsLkmY=gcK-IPzLOyuYUnTT*WFliVEz8czwnN(X!v3bBTA-)14bfy9r58)Z%P}|1H?QtC&6^JykF!fQ zN;tJUnZ1eu*|?D&R2G==XU8!ySp~e)M(~NH1FrwFIjV_1nC(iLCFPEuqYFV_^pD=; zaI+q92y6tJy$x*B2H&>cO+NwrTz?__Cfi06;&&k3Bn1X0*noh9tmN-RYgcfG1ZX#x zGW#ogs)X?-Twv#KQkGXMhe*opN20w!FI(qrM#(w*TOk2WE{?1BBC-wyBb3{!*lV(E zU;k(ocZ6M7&; z;JN@7zHl*N^`(x}HYO()_4bSeRj!}J~`O%Qwk zhAyzA4b`5YsE90p=PMC>z>|>?J1-=5|9ou>M95336{Dlxupw|>RXpi_Kr!bkDc=3P z-zeh~7-9lh)R!Bft@v5*37XQY^(8>{*l<0EgWsNo5=2>(#`%1PcT z2BmTRB9e1Fuz#0+a+?7wr!&3Px|_S)yoA#2WU9|VZXjb@3MQPcCyxwGRW!{E)@4z) zsj>d#OeH1%l2-t!?9|+2dnUEReC7rUG8Hu&^XiK5L|`9}7C>ve!BqoOS$w6V>F!%( z!bq)lpUC9U_5^TcxusU#fr{XI>_G|i*rc*hwhIv;*yll!YD=K2ql6gFXjd5i18d#v z)822YY(?2lz14ZH-24_xJNuV@{S>!xBuD;P@c;P53|Bu~v;NRCV$P1gXb!$gbF|iS zqIO)jc05@lY%#UlzeRn|VshjxSTRCAkhGX9pBtsJRKPPB3fZu85=kl=E`y1z_&{Ni z+`1dlXr{&pmuh~sLv4nxiL+r>)}+=-x7ejj!w4WJ6L7 z=XvBMHSDz}uX?q5F{O<8r4!5^g@W&769wPNCh%Q9oQ!4mPATuN@$RRn-r~)j$qx9) zUQM>aR}E05znT_Dj@Kvw_M>ih6j~PZMh|jfdkp;4ORDZNC(^~r3b;zniMn}7hQhpy zn1D)glzD4(3@))7#IuPIvvKI^U^e?$bc6KAAQ66#MEN{B>&+>_7u+ykQwrQ%0!_atdN7232 z{noDcAD^`lGeX-nmA>2$9H#K3S-0cLu5OeJYVS|68g^t{j^L-86eN6wIR)3vW`Q_N@ z$sRa=G*Zbw_?=<_t^mb!?Jj*YWki}{c7aSuv#;+RoUN(j(8O+(;3(XUZu0)s889jQYb~nzF1&!J5Svw?J$pvA&o*A% z1kR;f+QV)j>)c^XXiZo<;H-|eJC8S7ctq$a3h*XS=~=J8$CDy-PCKOtQ#2jiir-I& zK@lX?0=@+;d{f_@vIfgFR=R&pkKagBm!XYHN?zU&*QFS4p5Zvfya4Si_wD(q9o*BiU1}X+|2gY0t^bK}e14immHv{(Sk)HU35q z5(krD=xVwKSGcfiD#3B5<%U_L)oY}hLu7nj_$&NSH>`&jAPA>p$qmxWuiQkz3nT%R zntK2MP5+uV@C?>RdWp2uUs(N#v_uup!^zX5H;7_}V6t5)+{UNi6}-dMbG=3FCMufM zi8Fa7efm(->}i>yVE<08GTAQUd+$tNl(0+!OgT_nxg4JCq{54)%${Ju#Ep5v8cjaacJ&3YjONdyDw8fCMa% zuLo}r7ZT6~K$-8uSF4{50YV3r*O{zA_-m~nHL1=RqvH1Yy+`&g-6&6z6w0a!pMR>7 z>g2otp>%zi|#}$L^5j9*&;%2W>Tw1eRm@RML46ie;XS{M4|FwTo}`?L;P|Xw`-T&U(Q2jMZsl|tv(K5hTKJ8~eSS)xf6fbkM%En! zXA2FPuMIAL*|O8&LUYv>Hg9$emAl5*=m}i!PrfqOv&>OtjpfAZsi?s$neSA$xGu>% z3wa<+omoV_%oaNik*JY7%%s|^m{X{fAri)L29n6T2B>L5wS}vvL-J zvRn#JVYW1?U?O{9`4E{wDv~HBc^egGic<4sWT@XM@qUzb5 z`Tp>Fb;?KWQeB2o)7!I4ui}VT?T!!ga?d8GxqnvRi=P!}@wW=B7U3n#h-zEl)62fs zf2_?OcO}F%pA9f+9}!j`;luI&4_39;7$!+JXepK}MDic8e!HxYzm8k^v?pSNQ&!Qf zFOw#g<<4Jw%LA;=T*_*I~J%*z@t@Q;joEX1p`l_2VDdioaEI^vD1*2?jnb| zm4at4CB}wdxqAZ(qql`pEY?9#@VXl$-yqE1mR%(!CHDA3p3~ddcU*vfF8Y{yPR=ut zsRP#oVQ-5BIfs16>mgHZ76Qppnvk3-{(ObLr$EA}%+bC$LG{5OZTD=nsunFZqC6JW zHpGSR{*~w-S63_NT{pgNNNqy_$}agCK13?wc_EmgoKx=F1DcTAB>COn*-&(QFF;#i zrw!fVmV7CN1Ajw~ZDR4_l5DCKDX?m{aTfMfzKcrbwmdG#&m2Sz7}29%UpoM6K?ulf zJ6qIKpsRe_;!4^X#C=cacGRr44+$eOWL&W=dS&z0E#QVhY78%qMQcXUfKNZ8FDd^V zYyO$=-=-rtFNZb!cLZX2Boj)beY>5cdT{E80caA>w;ygnVpap>Tafl#4EbtY#6>I- z+St0iYpQD%^Kt6Ir4EtojO08eR@*tPk+K~qi2eJG$y*0*BubZ}vz3P~1iTI*bc2x- z-4Jo1)wt5FdrPO7{7dwkGMCr116yIV4BNHRfpg4l#j851+Ix9?EDe81H}gZZ61m#j zh&AuK!NIN2%um@(=59}eOf($?aYpwvK!==hdgG zZM7|&xt6!?P)-tAZhdJe5^t2$7Mw73$lm{iAJ$;BKGM5zdI3`F*uP|BK23JfHa=l- zJ8*?M7LQsu?v|2uS6evI!-#^y?)n9Glq=5%vSsHli*vpk*+ywb#Xf7SIpN3~>6~8| zG2w6xkuH~)V=2wkuVPpafaLbLw+{JtS?BY`xFQ|DAzdOi8Q?)1V)(A~_i+~$!m|Y| zu~_?V2h2PSst-p=KxxrkabLy5_54HjX`gP)uKKyGpOF;lADJ#BYy2PX-ZHAKZv7Wc zjT*G26u06I1&Ui++=~R4;ts_fYB+%cMS~S5Aq1yLphbc^!QI{6dKP{E@4oNeW1ln5 z{dn*Cz!(WxbIm1l&G|gfPjVsfYP-6&s;8)ijh8EnCaIy?mVN3j4KWxU+4nJ3yzbI< ztyvUOQqaSEUqP5d^MD3_wmVk2x=R?a3Q(M2b$1F(*P2zpK0nR1j4JUYh6Z{f6 zmVv@3VB(n}@Rw4>JdF3k}%; z?umuFJjLQ*%AXV?c0k?|OLNklg*Wm?EAw;Poggw5f(#r5y7lhEbz_>f z*2kQK{!l>2%lM@|3nBg20S7w<3-qRCCdTpZlr(d2BVPw&}VA4(JKBaY%Z=uc817gcg+RqFwCK77Mf*#zI%khBYY zfB9*Zontlhap{0EJ-yYke^{2cnWPld9w;-j%(;H}BVs|M?@;T-pbp1ZQF{#J<0O9w zuuM0K@FTt8aRPN!Gk@wp|Bf$WtfTfu?(FY#Crrcbm%!_qDEnKlt3jZ924%xq`joF@ zrkv`m++|p}M_cn=F(NSRWVlh_)u6a)FilIj6sXVGcgo6 z32b~;(6eb=j{sQNq;_^@j$UlPeE`R(H$LhXN8{a5vKG1N{bQ0x~19>8Ethw$sk4R)WT=pV@)V&Wj(I+ z0%qqmR=pF~bu-}S&FJ~zvJ3yz=hqWHkWjH+TK`b~mTalK&^ZZ2@ za6k;PNnP-yvusz<0L_{GC&_h_4_-E@ub7LR%au?{;#rwKd77(Baohb?N-ypq*MIgp0TJM6L@vG#A-+*WD79hMms;c?+6R|D* zL{t}bQ}g=zWS0#_unxpTuHTGO&a!L7(v)rp;bLxNbqDXZrt!W=1`my{>7R%7J~V#h zY9VM~?V5}3IIL#!-jnV58Mk)T?MJH?>dsFsG6jw_ZoId! zw}uriH5si=LZ~NIQF@k4)>jI;8&3C7_lhNO*UlW_fuyTahZR8#3R~&(qMRa%h z_ACKz&WH!nkuuojxz7>5l@X1x{i0c6%nIkL?)N<*%EMx=eJ5o(WoM7tstx?&p&N1O z*JDd1rPr?y?%7Y7zW)yRh~qIZeI+_)uBqCyzU!fRT*!k$xa41Qon8O-2F`Gj5{_n$ z{k%cQfL%Y>>o@3)x!aB9nJsvliWSj(EvyRLB}{2ds2rRUq1tl2%L9wxL$;2lLZS|` zr>H2ML^E$4Sope;GYBMg5|m?SjiCJ2fJtREFYVoHqrb@SYahiPBKVfD!Os2#X%mOw zH*$uKEQSPQ)Nc)5HwO?jJeoIKzFhe`3^nNge4ty+F>xj)@aw?fPGQfxo{&eKcPp6x~fRIx8=WoL!ADi_SgN_reoszE0c=5-EhW(h&hiYmJD=frJL3CYiJ6a0=Cz zG1q>oIK6se$JPF^ff3s;R^#LXY0v5})$S+E7m^ebY)i^vdR z4Tr|$n$zx=6;inmcWyK>)ayDD;$>$m%U+K%9#gG8ax<=1XPfbSn(KJJr8 z?||kggaaNVGV@yLhR}ZUNFfV1B-Ml7P<`)rSxaE}c138PxlmNH`1ShO*|yN^zj!PU zID)U%eFRVOu3tS1!wB;A_arzoAkdq`oa%;^Gwjs;*nRox*V?Jwl5cLvtM5kgJ+wC| zLvv+o=Iqi{)}*wpzAPz2@}+q83*nCq4a@yvz;~rcPU0_*`xaM9QNueaDNC(tlzuqC z^JR1t-cXRME$twJtVtu7$!F z7bd@sk%H-&2%d@e7U~u}>@;jPY>pOD@m1REI!!j<;2P?em5Ex-yu;ey?#eHHPPpV& z1;t6ykJgX*yeDeEzQT<5*A^BRG_fZ3)4%n0;jfQv&XKBfixahIik5~%+%^m#S~Z;> z{))PR#=b~Ac|j8Q%qK>9YE_VvCQhJ+*ZXZXi@e5|V}P2?5N7yGh2=AL(6e|-k?g_! z0?OA*4NwokG;d8ly*uq6uxeI&euF3-IoR?o4o^B?rmTls*bqouCHj5L&(d5B%fcN} zXU?iXl|QfBQ`f1~w^3wy?J&{rvXnPWEvm% z+s6Jf=LX}UL(_Cy=5LUp5mDD}F8CcdB(|3eBVE|(yn&>?kc;~0sM%?@2jup@HTNQD zk6B&|{u15_3AZiYhtZ6+Y~`tq#X}M=Ib+;}m@+d%O^ro(YibOdH=MylSSG|x2>;m3 zJ8Qy1E|?s}`6!MThi!!?@uc`2jRN)6>-y&;SVFS9(MQgjg9DW>`+MWSu@uNe@NR)) z0h9$l*kA@YqZ<$A#L7ezmsZ1r?q7Dh)UPn!1_ElEtqSn4o*a5r0<5P&9s<~8GqlYa zd6pw=R1kk`F9oA;0qbC3_nCGRrp4l+-D7H5mXVOY*6EKw!O6sJ`px=1D+=N5tHr7X z-zHGVg1v3v*Z;1e0DSRpo5qE;F#|TwU$!-L*~JONtKkpRq^heN%VWVN)-V`4(0faX zW7pRq$!Y{)(}k3H@3?8>oSWkTvu7H}H5@GYrNfAE>dSx=K{%Z9P8HIew)l51*Q5>B z$A>uR~A;SxFCEv^y=+~gjx`O zYD)}MAX63%S&LOWmm;>ZH(g6omctaGX-r1(iki_^k}1CP@gk6N|D;5-B=KL_V+cR- z#iEG?KzR{&Md&JmbHva`q2JfB9)XcmrDszB)cA7!&W2CY4%CPLPT%VPoxY{lUL41>nRDoWQ*OG)SKRK*cQ@4Z3@{ZN za02KbKn~Ytd-QNX4uw=pQWS!gh`0$Maq&n5_4{3O)?97#s5NF&UmR+h=xvp#ghgh= zK&Pk~<~ci>xj*x^)Mxm$6{jN2C2D#3p15 zMF5(}Rg%X^qbbtM*<-MWP~k;Ki)Kn>1XEONj)tkERBP&PWuzj>(A?bM+Qg2?6zXk{ z2Hvl<<`R6p#%M7o^=DO><#1KrVZ&YUw4}lsv=+@(?E{&F{Cbc*LvWtMA z#;28bNlQ{C%r$)gH|6HH`E2Q&xXgslFf<1p_HOONp%k=Uw(6i?g9kB-A47(~^?ZEW zThE%ftS+|AJfom(Qia+eq#YrhnPu}V+_H|-a`J7|t2l4;8es5Q0S1u0#XsG5b0=E< zqERgM84S4eBr#pe%NOm&YJ0py1wB(^&%fu&fw?miBVƚJ56p|_g1Lq z+;5QCc5$eJhi7BE;G0jVwb4Db%1ip5D=D8ljsteyRHg@-s4xFO4}j15Oa4X=kU8^@ z?2Y)u@ye2+_40Y#MNkM?tpcz(PWH=9U!L4NquG>waTTPL%z=b+)fDp9wJ6aIY2c7U z3?fSeeytyB?`c%b5im%KJ*i^euSSj18g~%17Z-T5ul3T7kV}5*R*_FA2q(xez(6ML z3_ZIxt-lTb_na|S($b#m-h~~6t6MT3OC?sP<@~lSO9?|0l8>>HLqZJ$j?8hG+A-t& z#x=;Qg1Tjn6v9WEgns_~L*NBLc`DNyd}@@Qv+#Jdqvjd9be9b~M^wbj{T9}{n+5ao zDjtrBCzQJDBjV>Nmxbm&Gie$D#d`#YvC8c&&hNTJ=X2i3CoNM(f(-)8Eu%Mn$A4v*e~c7!oRtoJ1S;yeq^6NQpggLFjHd85Gr#|29)*Va8rzOIau^_ zR{-COLz1fM*}%cGz?`I?lbt%D+Rx#A1tApehteowOg&mBe;~U&CvI?u_D6U!Hww?a zp{_*3$5xHFSs}BSOnA`E#GPd*qoF6lRpvwU`+;Yj8eax5eKjokCGHnhzd;aH7ZB*h zNiD`UHn|K}nnwT4w6|R0xP?$uiTz})E>n#rXpCdE3Lyj{sH9vA%cp{s`I89pq&VYLVmvol(!LERp9cZt z|DH7;DqOv-T$SW}kl3aSLVfoiUcR~UTxVG($M=Wsdx`F=7E`!Z$oe3h)KoI+gR*kZ zrgd=g+m6%aquR&4Pi{ZFk)?LOfQMeHK*3BnnJ2A%`q_r&eoXK6nEbaaJZII#A~qJq zl#e%HR(LqU{O+F6F!zkJCL)I~*fYrJrRev1lD#@+ z4w>O8pSt37aS!<*dxlp=#2uv@;?t@=EMmwvz|w+fO>jPo+1DiFkO`1 z93CH>;d%~?@qV%MTaaof(G={|i^(x$yFJCBDUuC5ccxnl5{)XN_3T+<`CVa2X<;A&FN-25!sK3p@yp#*7sBR)cA5fUwRgo=>RsDcCObO`=` z4R4-Idc)zX_e9QPVXa4r`WgVIxMqmZ3)6;<95v{=hVx`N7JR1Sp__PwM zlnvs&Clgt~VhY$c-b`lriShDl5Sh#(lbMuWTFQ-f-ahr}r8c#cKu!vo1)%JW6O1eQ z=*(g>b=tf7OI^+kZ}A-_jP)F3N^+y|&}w9$H?RJj8R*8-;i+dm3SQ~?#Orl%X0`;BTYB32BAKu_`(ZjswK2uX@L@7*h!aqK zUARRJ0x@p7{stADd{5`zJ)}#F3rLGTTvRUL0*Y+~l4&n)xJ4=I7rr0kOc>qkOnF|( zogr)DASRIq-CDRQ8_+%L>;iSb|N3c=GKjR#?Sw2PFPpB(-?fh1C*-=&jGwo)igRL<;R=ThiH*=0=|_puc~(hQc~Pgr3VgZ!jz2Sne~yPB{q=i_Taqh0O{2<~6;aiaCgf)iu{5+dB zXrP?f-gM?(zI|547WuWZ0OqG51jVvCCg*ypH_U(G1v}t@O9W2`TG%fV0im~JeBQ)U z-b>0H!xVGJkvu}`skXD@=~7pvW=B1RmOC-H!m6*{yt+fz(qrW+yneUWvnXfkQSKC` zYZ%+ypHE!b2(gGP2+^o#kSlQDt^l%p8HCKv17nfO%lXGc40Lz$0C(5|cU{rDJYHkQ zCt(SGgyPmLyERrcSx4w?&%NvV^-ZbQj^e9;4rfb8Y!`XQn-`u}J^*yK8E7j0+&b?6EXv?76W|gC zc|{(Wo>{&{bsLP7`Z*{U+;UK$&-|s@GwWcySxinI9GeG&?g^c7C6 zJ+vH5Eob~5MN*TqVUI38mO3|X@I?6)J=_kUSoU&0Fgvr>diSdIOp@JtlOxZ%sVVha zKL8WQ-;u?O62WIk2L{J~i*=M)i~_aw!b*t>^=sTDChiTz3|A|wzlmrzqw%S+KifWx z;hr~+mNhd~k);_nFNQ61)CNQIo)O+Q_FdD7QSDA^Su80Lx-t>`4YKKI0)b3oSR|b~ z!d5lw`eE8yr<$j9?)2wJZXcrnrjuJCP{GpE@2$#l<)%U1zxw2`8V@|kH|E;jy-&mr z6uoR$NSY*Mfwk9cKg|ckF8*Wtr>mzx)+OBPH;5N}mc_knj?y{iA6i%TN}n4xkKL~| z`Deh>K{n8G)iG^CxXJ{Nn;1782w#0N-fZB?x!%B&C|^eKDKndKt48r_V*WIs!^k~l z75TS#ni_Wb(rmCr=z#42TL>1^_D^?MCcT?3g))p~HejQ5M-KRmu>y4hglG3|%M=GZu=8uKw2 zh+Zn(BKMvB3zHqeB(-eL1Y50*`;>|h4iZ{`$nQNdW>t0%NnoD}|G3C*x@vPt~dX*B|Sn4e-AwF0( zdcqLcWjSvQ`wljD)z<3kQ-ZZ0sBegik+|5a^Rf(>6PwNZT*P60{m|!8}ZQX5+5LpxqUH$9}0>gk%UpraKrRPRIM0yFn%CG#CevI3%h$Qu zb~$$#cc#DlBdYeX{sFcLynJ|R*OS2PeU?U-cM^I|B9>xN`I@;xo3yo<;b6)vhRqHa ze;xqaK+HWXM^Jbfj$_JXMGYvPstVT{r6Z^Mjp&+q=6*O2T#bv7+PF@ZqJ{^dOcEy{ zY4fDe+#GBOVrz8sbrmKpP&11*7n|{_t%nfXF-|dg=jUKddW7FjJpQSe&>wR_h3Ut6-I%_6+l@-W+zV0()V?!8eKj zod=)p!98rDUp3X`+`s?P6XU`g?OxZ-JMH<)i z#A7v7@0E^`s9F7w8*A>>lpP8Tot4M82De?r;u4e-9kxILj*lOrL&vx|*uB+_5hgmV zXC6in+-;>OA$%$$7-oG0;J|7I%9?ps0A^8D-G-jHpplRrVGZ>AO5v*U)=tCMkLgHR z`bVy`Q$PG#_hZ*+n48JW%1A0upEZ4o(8d_zX$244M{;M4(Ypi^hV~Gfi5TwYw*K%} zPdond(7^f;)iTT6y7`VmdWa)IS$Lof*i3yC&%H`v(>4YVp*2(%0%08~q^ybT6U3+PI#>R8}x(HT_{8N+fOh5%U@wM|MmNyk84u; zDCFLWh+WP2PHYA5soend)s8`;TkaB`OdkM=weyevIK++lR_i)3lRQ0?jrvq9(%NsQdGEkq}38-Yu|3oKG|z{-6%hESaKq89u$^ z*zR!~_EtSD88S&ADl;=v*nQDmFjmNci^)Zoz7 zoO0sORX@~3t0LSETa6c!<}%}^XRIMFil^dB2iSUVZ z+JM-oxq_KG{T#H`mq*eby1w|qD^uKJWh`qus&NEYnN$>KZnEM{eS#hk!M&GSn%Ojo ztIz$c+$y9pfW?@Q>S5vPgc5rkEW?&y~oaTpq9l}HH<8#y8#W3hH#C$VaV znD_bFb{D2{pF@{L1%dT2>A%*)xO=>ZA*YsGx=9&v`IR~~?};rL+oPG=PmN_&y2guf zRTR1B*z_)a->*2064@Dy)$O;CaH1@eQBqp;)jO+w5n7dLnHt?gzjn>y-npFBQ=9;f zPT_J8NoU`uLri7;Ud0(>%{3Np0lng9L6rDvQhocilp6b7$%=Dw83#VhwQ20Gmt&*} zUJ$)WNB*GZ%0uZ!28I@aM{`*Db2-w|<}79}CDijo>0R~@T47Dn8F4Uchz+lG57}XE zG_i^PD6PGQ0DL>+yT{`GgAg( zXf_=bdKwqpbwOkAuBSJW;Yq4vxYAWDXX=dP+FMHlS;U(b_lrwqujd32Hkzvw?4r%% zo`S6-BZX452u+k)ae*>P{_O0Q>XxpQ5pp8dG}R;d80Dw)k^AK>J!XbyTvzUZ!VK1l zFQ)l7sD9{lD0V7uSlzg1RgayU;aQxWneq0wfR)R13H`Ahwinl3cxK^3S!O-u60{-= zI1F`&owAcI0~0YY5{ho{*Tcbh8M8{helO#XCBzMP3AN2%`WZtCNf#_N;-1Ocbz(g6Yk-DTH;rqvG_Cc<) zY6G|IT+WFp?mj_#6ZCz=o0Fo72Rttea|dK#VaY9irIZ&kyitQh_e#e4`pp@OX}g@! zg(<<24zEe1!!y;{Dqy&EKZ z@v-M(WM@_Es{lY7yjeITEEmsGaG;^$Qm0xypBvR?pkYUQ4)@Arqx|np+~a%df#mg< zG%L2rayp3i`+RabT4uYD7)k5#=%upsoshgFN&)8Aa`&n=q-zXYn-qJoEC1uDH)i|& zpF0Dw#{ausk1m!^-ol~hJSsCV#;R&X7X*h(zNMDp*Nz+f)YNsm|53XLry>@rxW&Nr&(FQQ_VIh7!L}Qc`k5 zT@6ctCr~kAffWo|6`;G7x8 z-X2;KKaVlsv0emN(a@;4|iHJX0nH|RU0zKLurXK-&7MN}m9)xKh{q%z9} z7hY0eCz2>~7dc&UqYI~ z^|OO9xMyC)Z6OvX*n5|T z^Yw=6c5@3Tg{Rk+WtXE|&2pm+yno!<{_CWvQ!~SeAisE?>m#?Ebf^GJ+@JtOfXR3i z^@1^V@soD*xE8C>CIw5E<_sR)neRz-NL1!pB#+KnIn(8hX1zd4>Dk>POCQX_Cwz%i z0%r8(g#89e*FC=jjrSPa3-KG&x>G8;4T<(!;UI9G$jj?~x>#Q)eYG&A-Zv!6ipR5y z+HGyIAEt&0z*6_V7wM3OOej~1Q@t)*=o~bYRn=Zt^TAoz)c#P#Pf1GLn5>g)vs6-e zig_~u0yXP&&oXtcu$fskvbaqNFqn`*FCJw3JHMSF4F2Icz$s~LPcToQl#&dn=+Q~s z^nUdHr&jFO#)M4TNc{w~Qj0*xHZ?U+9eO|$>EJ^oB4o^u*=vS|Xs-y!jC0k>89eWR zYd6;99rtb<96As8FOA$E{_^OB*vR{V{^Z;r&B?>33cQ@vVe*pUNb+(mQ(4@#V>l@5 zeUrGay|U!{0J@PoVc$D!X}&y&Z=~7xBd-+ZUktr^B(NgNH{H~NgMCh0rBcHar&j23 zHAbp8(iSSbGc;rs?bAPIOP}6ok)K_wzf#bIf;F#OpLa{HTc^TK+QEfWRoHr^dM-9X zmEdKq@z&`baorrAz!r&B4oN7_nr2dSh%Ol?XHV3YeiXvm9+oW-*|kpYbB;G_pF@b% zM!>(xD$53Cp2AoQ<>#4cLFfk>h{h~3sPKkdJfImYImU4XrBOL88A_ugp~c(V8C#+K z?X$=yGeaJuUNTUevvIoMm|_)gibf$xJ0)KeMgHP~F^;Z8%mCWo*=M=sx`5&RcLC#* zsMJ)k7#n$qv$`sVcU{x|7TXh{E;pV4t!j5%41%N z4}lk-*KvE;fLxHXgJA!_dUR|=l(nSuQJUub^ivs3RESDmp+kmFGjEcz#!FqTO53$t z1g`*G`F0ts?s`V-1ZKqg|Ctd1OQIsXnKj?d>&4G6c*;fm1Luz2tl%_t@gJX?6ipKP zMnic|9HYMLXeIWK$4SE^f)DgD1oZLD)@VMa=w{uBDbWQ?u0?Si1DyPdQNPO9zllw< zQ6X8tM^pZ_;Z5Jc@$ZwW>IXl^^{{Qj?u0;6o4=*o_|X~@zXM^DtognS&8zBkU+4S{ zYWi$N{`x~08-kqPIP9syu(0*&Nbu1CZqCY`5M_>#x?`?XDMU~?pR;jnG&4h89!8BC zbfa)=pZ8J(&Bh%YMkzAPjZ4jDiRVcX*8e?tPNSk{-HtyYsrs?d&ErmImxzzsN0fN4 zQX88m-$hBao~Wg|bxBOBYUE;`P%zqT>L>RNTTRa%Ug(3P@qMdO7o}`}g93SPAbund z={~fJ+l^m_3wcxGVc@$Qbpb6f*TR~?v73>T6M}S7i|@3Yyj?`NqaxQ9ZUMy|VAHJC zI2IoC&&VARH}p=m9sc}wDb6YrLhuRT+cN-HgVvsX?VS3W}H5nCIwBBZU)^wyH%G!=|Kx{%ZX#VP(L!NOMr&|g zm$F;+FuxBLv`wiq+wVeW+e~JZMxVNUVH6g8yY!B5#fyaeMXx$prdP*R-r#Oy=KR>} z9^TW$z5KrBz^4Efqzoe_Q;KSsPw%aQ=if0kR>s5E+M!D&DhF~*(X$4=)1-8leFGMu z)q5WFkXnWgYLQTAHSku$p$@c4kBx4mnU4w$9WDkJTIWLWAN~f(IC6&!?<9sklv6;_ zzfGYlXHZ+O%-qy|Rm;zcE)D!ar%W;mK0S_00AW$1{cmYlG@5WoF=^EZ@^z!DL|=lR zGv&Rfc$$^A)l3mIxK7;fvaztV9iaaqZK!7kq0q{*m@m(A8SdEKQsqZ0$@LTT%$+2b zMv+jEy(d!SQM=*fx%}lCr6S0EljAtg_a*s%|CoPRSJ*c1I`rwgI6YboC5)(Uk`kZ@ zMzBR(W?vmX&iNp86n{P_n)K-ZOCODR4CL;)RQSmqC$yQMbzJOCgCnG~06!LMbof?C z*Zi5_SZKZfud7{(&J@uRQpN1UxOfbHwXLWe69yN5*G;bHTQl{9-7&|YKYjgh5srP6 z4F%KdkQQ+@`DQlDwWd~6odFJEJ8K$E7tg3ehvpty<4XABw-1qKLt~`BsZ4>Gn;+6H{zK}_=AhOQa8igcfecD-Knl4b<0U8W}Xa zwADhVV%?>_ZhYI61$^AP`ef}iLz%b}Ld}XH5~iW4JmMBHembMxCEg|L_DIr}bAu*- zO^$uaGv$kWnS^;Y|C4;iS)EVx+cp)**z#H*b}8YT5_r1Qxl&U`D@#D40VID0u0n~} z+zkYr@G;X{Z3o{@bGA=6%K-(^NN{=2)zq&^3DPZBPc3w&TE4e(jPdhbYD$Tv+FXGU zKRS~r@T0({{ILMdAu3mTx``-qn+MOVD@?MNU1`A+ZALH`PolIPp`&MZaaXxzzWS($ zxCP{eh1(^Z5OrS^oI-bhd^^K$@cF(9*t|6`a4v;cFYQ{D)xc1yX59Ro&39eoWDRM< z>qM;n5!d@sZI688%7#qctsHAaYz>Q~NqJnBYc!VEP(tFe2CCIiiMgalf-}Mi9t&1# zU2yU^5*Njgzxd(dHDVUxroOzsMtiW%N-%ZIa4e6Be zVHv5e23IMoRs1fcEK!=T)tyu$5uB&tw%UUBkaK%z{x{uBiPeDzp>?9QM}`qev?ds^ z%9=5o)wxy;1s}SmaGt;_JnCuqh-+3SXMd>$b3%xkdwy@Fi4+x8%l4F`zyBsL{9nY+ zuFoCb4v2WbGUFe-hS95skg8+@8lZS?&n%7 z|LVpSs<-M9{3nYnTIh1oxj56{le#iaq=VC1r9FbVRHvMqI*^`aA!@x;XK*(kp}D=x zU!JVzCSVHBwU!BcIVFJ2rBtmUq^30P8TZ)vP{vA3Z!pL&0=q<_Cj|Q4FUe`dW}BGn zn`83p+47F)gSQ%~fZ5_7K7-iKXyaoh;(a_onw6B_Xpu(L*uThT%0_3hzBwE+`6S<~ zTb!&NoGr!d#GL6lUHWsxk78#pH8PiV;7>BT4oF7BV25xbm=>NQj-^?W6)~AgvB5D) zg=DCl4-A0KUoobFv;v!S`n3!@Tb7cKIO>Mui>^oUgNLu}g6I5kiz3&wH+T=2E`wkw zzj6?u6ma{lhPPR-pG(@+I+ZApZm1`>uS(()cF}h>b@+9QcwRp~`C$#YZmr`tD^6cK zuid7!XLeF4GawKahlbVIS*G$-qq5LkV82vk6qF^3=)hlNiACO0kuCe3l(Az$$Qo3% z@e(kKok}-48Q3z(7vsA-T(obAw5qn~dUlnaJ>t<`P)kM6#LnZ-*PrSYUU8`nRx0Co z69V|0a)|C(DKbW+F$q;O<%C^E)}c9OiN;(4!4NZ&Ua!|K@q`D>_gM?}J3^8?r{`8F zMpaaDCP?;4*-aC0*$!uqWu=UEC7~f*QZ4OnZ+I2FcNN22xwcnSFTX9}@<2 zDCMnWyjZ1p`O_+TQl{!GE^_m=9q}gpgVGZJa09uJI;d;>GVo4!?ZQ@TI?0A0A)6Wi zXu?@9Y%Da?rn`Jbge8ovOMu0?YpiVHO49GmQBf)RQp^Dhin@2$0VrErCIdnd_z@_| zW15&1RL;cv`jFzav%P^IXz`ej!ONMxUpg;%3C#+0YFEIa0|ZMQlr7=4WiGnRO^cE! zyC|^*9{e>IOQ@bWJgdmjq=+bfP|mSH**BM`TRN9_4T|5S`Ae#FtlNC*sY?V}T7(i0 za1ssVvc)0~FTXXXnO(_TnGn2F8JN^4xAq0)%0g#>iDvSoTT&s&QpJR%M%#Fz%#8){ zw*EJ1&=?>T==%H;8E&w;eaeINsP>rHt9WYim1-IAh{aQ&a_290{}lU|C^7-u@{6(I zJE9r=qdpQ(xdhnWM3rwJvFZ28gP$j{eDVU+YBg-75=l4LSC=IgjDLfSx%rwWbW8O3 z2j6o-t74=56TIZ^N2ffD+X!7mI0#^L55-DER&k07rUJ zW6r-HQxaCIW&1|F>#}n5Xs`~1A4KrO_v`SBcWan(9FO(UsQoSmgHH9`=fvdr4f<4B z(E{o4X-&0?#V^t#qHo+@1nkmGS_DoX%;=?GKP)`RYVmE2luKq4+y;AMD{{pw0O81_ zq0O*eWinYR5>omq#q7u3j185PZ%?~)g~R$}`{82|Cn`-H0mY(hH$H@{mL}dBTR2gb zyy@rXyFQZ0_jKP;gwJLVHtZue!yEt8;?Zvq;mD=&8=9C|4I2411XX-vooRs6PUFj` zLTP4hYZ2BTi~n?mv3o_#5>vbS7(h~hunB(r(&@Is9%4>oP`0jaAwjv>&w$TZJ_y>} z@6+jonXxm3w3WO0DA3g!G}CnxuaWF>;-wd~B|wy#KCV)K98Y&*ZL>rOSrPiKylL_I zF%UI<^qn09iYWZrZw-UzI{L#Tk7Pun2S}`K3rkIZ2s@7x*hJI1T4IQ%KJkpEYYo2e z(6fz<4rEW!tf~>{ZD_@)O@Xbcz5(}v#edUag@8qHJh*{bY(9$cxEZ)RAQ0}xRYbG7 z^AEDFeA@~%v$SZbiL)vnODw);pNi}~(2XveD3#XG)V(hNNJ!#(2Y?)M**i5{Hl6nU z?D5SV<_PYUIz`JWjdG{27lUznQW?vEkrvx8=3pwW);NTK8$(sw7R)ev9I&%4IUA+%7Rzb0Ejudj*Fk4}?W-XRqm>)AmS3ln zg4yO3sHBmt&$9f?u9wo@z@jFjC#4Jzm6j#fBE>Vv3H`o=+ehNL1@V$h@5mxl#B&zZ8wg-ESlXY2?e^bzvwYZ^6D{ts6jiInEKeTq9Ij$;9yd<91ns+w? zbik3az?Cyxg@a1HTJ*kCM}t&wCHsXHDt__&O_C zWzEr3RKMG8dstLSu@YV86S@#bE2)0d8!LW*Cnv(=h1N>8NmOT4Y^F_8{=B3FcRCQV z5dcV`2=AZ~de`c1qY{+Jl2l&61?Z!r*sHEW#~-!c>?Lrj+L$it43sBs=w#9dZx;vb zvbe7hcw|Ooc|~`XXiuVFoBl54q(hJ)wKM6V-9*x1<~8*Wn4TY1eGm0WdsbY&i-HXo zw?^1Rd8?=>dJfgz&2}`a%B8c@uF*(NYelvi3=x>r_WK36XX%h^{szr;;Q@udx#Ho> zs|6=t!m{xQ!d+ewLjxUmqE@H(oMZ><08TUiOal7 z1YoNhn1ND);MoMby2L$Vb-VTSEJTSXWV>vss`X%LQqA~B=n3PHw|3`vgHqH;>vHCt zpUuOsXvGhW#c}CAat9TQ8vD2u1Umy-R{C$_3$slN&PpUXXlvW42Sas=x%(9z_zF6u z-}wG-Qb{EiiO6;r(0JYhjSnDzbhlhqo$C5|MR-Kb;jY>? zz`6^(rT-3L;Sy+pr$`ht_BoozYh&_Qsee(fQtCaa!n-pe0au0Wrd<)5(T{NT-`(oY8KH`(ocIq_tsW%=wYyGcfzwQ zL(|WzY@9hc?d!bqpIbzV&8w)TH6SXG6&n4dI}q`^0qE9KM&nhCH?%Ybn-u~US=7iN zeg%4f_pdKv4$ z?;&`J%eACeRAx3Wnv>+vLZ=qzbIeC!VQ>7WDLFn&M$3i|T&h>~a!Q(LJ6Iic3RbcV zFedXrYn*w$eRU)6{nLe3KeG1<)Ak774xlPL5JGV-qW9YJf#Cba)n(8ZmO;vB6U+j~j-`?C(L?bO`QTwjSD}_X(#tUR$TM2-2x-2>9MjD+tdlwcPak-O0Q8OR|0L zWj@K5JNaEjcxTl%N@^)_%(I+33Oz=&uJb12yUbKj5a_!{jVkL^POpv`*saoYPYJRq z6LNRQb#}Wx`t;z&1-Xa!_ClTS|KXdPJaJo2O*I@rdU{*tA{6i_(|x1@V%J6JMf7jb zT?2~qQ;xJ(l)r|9>aOIMd;S0a_05a1un$Ri3Wg!I$!psYgh7XZ8b@d*#}a& z7!5&SSuC3rR=Q-U{ZLeG5C_?td>6gCj$fi}`=tWel()onx0=YrFl)}{|5A?1{TU#9 zvv3a3UUyB-2jft~m1fmRCJNMDQHhge1JVGH&qhJ{@k`jMAN%Usr_vh+O3)8l?{TJ;nfpPDZ>DV68&K&7OjZ8l&2hhGFWk6%| z>f+{o0vhK z6L?d3boW#B<{2So+ET&0V-|-IQq=h}g(e|nF9+}6KH6wx;q?M^o7g-@uy$7UrKNSO zWVm=zXxYji&c_lR+J0@@FbH1QP~T-jOjs)_XA~OKHCtb7drz8oA&?~@FM63EUOKPM+6?IYG( zit??l?%)VFHi%8Rqq))5_hgKf=l-r%QWzo&se}lp|CWnvBFp~xX zO6%T4IL4cCL6;m?s#MPcebV>Pm~)B+(w-J)zn&6eohX-RJ{NA)y*FJ6P$o2{qhm=Y z>HgXUYiRwp@)@JVk4UTDMQa}N5;pZSIiGR&KbXq87Wi15z3t_oIMcK>A41t$ca_jZ zZDm*FF$4QTqGs8yJRaCx96Jg@JXf}V@Lr0ljau^M5@}KuEz>r|mYT-S1KW&Brjce(G_pg4V9c^-zX*}$Gr({pJ55d^wT<|cl>th9h1-uFoH zYtnc&K_I=YBC-8(3yrFTL&*z5yVoMxfLl@QM7F1~@x_W^kZe{{@$rH4GJ>oeBjW5n z`k=d;9^?Jk6rnz<6wwo(hKt8WHd0l}jh$ld^a_1Gel=jlufHi(zr1uN9XsMvRp~YA zdpvObPirAVfWCFTJrg7Gt=fTi$M~AwNw(XV)OFA6K;QftJ*lH-U1>eo8A?(!J;b(ezRXR^Rs4W75-}j270cR7iSm_` zwPc_I8rKLVJXr!SHupp9=^cgZ7L++QuiRkH6`;@u975+{*r0x#^o ze7I%x9f@74$U+m!3r{^}rqLP7S%bj=D8N+KVB!_K#jv7Pwhs|MMVQlEMuvzO&Tr5u z(QRbaQYMoM215ixe11{vbq9`l6Y3 zK&Evb61t3PhuG*0PJ+SSxQ}@3)GPJlm`6fA3_=T{`7%c~tn=EOTJg28@KzbBWwfxHRm=IyUz$mdvjW$uW7AHykxRApiqBc11W`K1=De#o>0r?w(t5lY&j4XoxcQz(i&E z%hUF+%T`85&Ko00)^k3)%m_M(CI6qW8Y%@TxcOVZ|B@tPkBq7;DD>00=8hN=p$4Y+ ztg*>G0gjsqs`^8=vl_6>E|Eu4`cijRo_V>4AVn03a`1zA^}>OMLz@s;&ve+HGx%^lP0mJ8^p@ z^6Eywpmw_lHr)lHe2lplezc(}1OT@J+HX^SBGLsH5grmqpB{PjkGU_^Xz9*?ZbIgy zs{xs&)&E7^TSv9kZSA8}sX$vwDaBi~c##B`7WX1Sf)$q_#T}}+6qg_^4k4sSpb(^` zxLbm|ySwza>HD6ZbH498_ug^G9e0dx{{cyM*4lH;z1GT}b3XH#-z&r~ZVAlLIba+q zbgWUI@y4LLc4gxcQ^Et8-~=qxT5u$S>xrFfa$v+L;pypR*p7h;{C3UH%Y7~?_(up=|E(Q2{eG*r> zm3YHU9RPoKg{w3=uM()D!WTdkgNkp&)#^u;kqsAyWZKguckIj9ZU~_X`DY9(fuJvs z#^BJ5VhN2Tq6KT$qO#Q)T6XAXyOn*mW_~ke_g0a2me7zc;wipUxU#5{L>Oe~-_bE^@PCW~@KrK(;!>LZPHFa>VenUX>u?06&0Vqa1UeMWuwZraxBGNyyuu|5%lV z*}^WPM^Hg;W(8VueT9wC$E(P*MKwaH8!O`QlThl*oVh$YHZ_`FS>e-q%M-Vdt*&9Z;P!EL$WeD42msuC-S0raIm}EReps??>ORIzm{&Zed01`T)CsgdF*I{ z*XnYM(uV$Mr}AJ&2$Xv&aqXi6v2Ih6?h8VO=Kp)vJruU~l&sMvg;~UmP`*C(+7?69 zE|9i3#KoK0rqwDCGVyF6>bj825E*SoGThtt3gCKwIz~T>J=JTozPfjKN_krueyWn_ zCZ)l-dBM5#KRP;YFvj*fnerUhk{a8&b=P!$1WOLzq*IFSm0~g-=8XfT7?EQ=)rs>R zcwW}t;#fowjCdXzPJ1cwPkFmJj4IdTq|QNcIqx)-8zfC4=LU184Macc(9QJv#*T~; zYRd6;va~X;1)l@;by6GXg-QBfo0XbD^=aKYh1?k|BBIgZv?h97yYlYlsdH6EIh`5- zLJD_F(##t?EQT<)lnyLRJm=RSL${1Uy4>%#(nco)ws6P$_r@3eV2Nq`OxQ2Z-eJ{d zPTM`BgFNzw4eP1D@SUU+}N4(6Ve33^~5 zT5j7Jd@AdfWRLMSXSYFI?kjFyG;k@EuQ|R}`MFZ5+@WUfTUm`}2-W(6u4JDJ!uaw*T|}*`*1^ z+YA59GUI^hVPTOM+p$5Z#!Et&HV%!Hv}SA$>JJQbyx1H5AT&(*Ro0yVw%q69jN*7# z3?0PI38v3sM4SZ~xkNxwD^jJ({L*T1yz3w?-soo)22vYYxv{Y$g=6={(rjy z(z}C}e+E~Ksg6hxle+i#1$C`LUYr5fcqS00A6PfP=^kRm=WRwkuQ_YJHin_y$xFSE z;9rg_VM&`hNRY>jO{4-vA|2J>d<9rT6^ew$&LA~!OugeBifrsRA-~{+Z!SC>TK~IB zEl~&4ny$)GF$()6O$f9+P|;Wp^}^ym#PFh0X2=f=_2ZVU!nmr2$WnSbc>PwZ_luPD z%92JHlPzTZaV$G|5e7sBt-7B}N!Xd?4EAh>$l#|w5vQStcdyyIwUZ{Ao<${?q4)@1 zv}?l66K!O9M_ygXOenvel)Z1ZJBC^m&Ft~K%T(Qgo6#qV3}~W;s%&~IUOvQ=aaj$0 z|H3+t>M(WKQDG)N4 zTU392JDpAyps^YK6Q0<4l%#XR!T8z?8LBbxC3UC4Mb*^1&s zxP~KVx1dn2SY)pl$Hr>l?VJxa3*;udJlC4ioG6(=owcKUjc9E`P)N`DPyxocVuS@!|~7{LzYG(X?1oU(kgwgiriy z{Smn$Hxg`oq9^u7cwVfDF{_rD`U+&#{p6XTQtL@ywU;K%>B}x_$qDM;d2Xw&{0XB1 zCW|GQlB3&vF5S-7X&%%bhe10(TVYJBEe{j5%AY(uKk}D7Ea{^p>(h51*)A_-wb1+Q_Y1xV%6*O?B7cAVA8DzTlJ7* z^285K{Nyo(XEJ2^m2f9=wFi27{&Z;f$?03^`gX(FM!n_CAW{ z_TeR#)2TMlVXFrHS>}V-7DPs5bAwW>5Ljb$h?>+(6?Jq}6InOr3b*&k?`!mCsyb$3 zrh_-1VW}+vpRIr5`~Dh(ivlXI-UNNgbQ_g=N3$~(n1k2 z`J--EH=eXu@|5ov-mFp4u%W_KX&g0y0mmIpKIDc$dFwAcit*17O8kGLp$32X@v{Gs z?jWLl{$0C#lh>L-zoYll&o7hk#r)*WCO)It;0kxg5wQN2DT7ARZx`Bwc*_lNPrV8t z^tbLxZMLP-*1;9g9al;*c=PvsS{xIglJb=EbaNiaSzCw||I1m2avxYX$`h5OG5Be) z>A4a@^7>olZs~hQ1PQ;+)#??&okkojXe6V32^^I2=p7Q6hNK<8p$bt|=2iC* zcjNPd_B&2zUf;ipU!23c95T%Wq*~k+;}N(34x1UEy>OGf6rb_%`n^94_YpS`$|ZlF zwiykk&?kv226<6qjaiCZIViQrob}5b4T?Cxy)JSF?RHYurpCScbh4W7|M;J6X=6#&}g?GnMdqiznan{akC~5s2=?R!V zrC2yjAYs7(tXy@{nEb@i+Ez?Ob#lWPX%{&dVe08xw1`2sY%ym(WPx!|EOR*Puqrob zm^Qozna7(kKt-5YgJSmzwRmIA~`I~<_0!oP5wEx!rdwJ&brG%F_sH?ZwYSy&iCY~-0ex(FcxlCn-NH?4^VjV!ncD@Do_ARUZ4dLyFivDB|-bUC0iDnQi16xC9aS} z`f4*Fw&s*jONu8CRDV8jkZvpjSoFfIZNemhFN*@ucD^9$6=Co%yqwE(7xQOpqFJ@E zKW=PJ_VO?H%l`wTHXv?|rF>aC&+cK{ z)gAic*JY0(rt%)5p!{_SsisT9_m{r){4C>fFYZ;|TMcIEV;$(VreX^aLooz7@-j9o z&5O{CN9{qY@^C>celKWKO=k{qGeZPNJ$8XQZrl9rQcOA+b8>XX5Xpvy2}P~ilk{Zu ze)rY&Zxx?_yv6|&BU8uK+z9RN67zzcY%gAuC=(@$_$(J39DA?v+sam@OF(8+eqnFZ zU`XYC-F}2&j(JLMuPjqgPb)yQX8>m*-$&-zGKI1Q1^Fk8)LARvUwPyhDSq;3))O7g z^fifezj1F%c(673)zn__Od~OsUl5pa@(x1S&V>hzf5OLZKq{yj$t}Rv$lxvW~xA9-4&09_v*q~XSNRWzp+cP zBAnTWQ*zR`1v!Z+mG~o3ywyl{KL9axU3vcn$Gd9l;OP8{bI>ez8LPCdu^?zblgoXc zJNa=>Id5Tt6DOL&adu3&=og+Q+y+PDa_7nMr)g27|EXgk%edU>Hi*QNO#RrHu5N-O zrKsD*7*W_oT>9Vvjf)EjQxQd_3}xU_{G{xOe8zjI*yD=RY-EKT_HkPF##3szfSSd6 zA#IVhNv$C7KsvD?6x&n$3LyQ4ZCD;>bZFKfmA$DCKz(8vLMxPia&A7`vf$klWm+`D zdOAg=Z<{WO)jU-i%%gSvTDLi&SFUbC16_6&2o`HFD4d;hU0Ys|yxelJ#&Y>n3T$yU zNgQ`$ChRnDZ&iyW!Qk%Z0z(*@BePR_UO%yL@rhrF zvD^qeHPpjRp7o%4vabkqnCC+>A7)s@l;Lvf>c!L zDnwc_sEViQg!02wLPC-}cYS@s7t#%-!)^GsF z1T|DQ0f~~=sL(KOHN!>HdV`jRn@*oz{Q%65%WrW4x4mEVe_c$bM~pv{jWW06DIyA4 z#T!g|U~Q78nFC~--P!^_O-KndAvTrx1aQ}$q@QJ)cpupsg5D_ir7iusY++H4)PV6sBi-CVZS*O=W+vJCO3Xt$!apBS_P&O|q znBn|k{jgfQSaR2sE|SPqzsid_`U*vC`}Ehns80D^AYCfszFcakb%m;yDTKEq3|c~* z2Lf3bFD$KKb0gVug98r`v0AS<+(HfCx}{as!a7Doql5UgRM zx~9Z31Z&b{lwk5w1f0XF3K_Y?wC;NG0lps)tzMT!2`H_6u~j zm)?9v!$6zS{cv1#^?0P*tpY@U5C*I83r~EJ1*#SW(CroT@ExAe9Fc9_l0wDVy7!tC z@g?;=(2Oonv94nw@Rdf^h)qQm6vRGhQo6c4-Y7yB+xlURhUp-~;z*^U1h2>L&4utR z;~>il7ml!4t9lgX`Pt5GZ+l6l)VfdW_WeDoU9(@ET8d3+Wquqj;*H+BCepN~aEDbw zB|;`$r=@vNdt#MKbwdq*bS{JOBgb-xgH2S?tE@^-`^)_^!d{!TYTkjO)gK*3TlX1B z)qLacc3&}N&u7r3Ndo*3@fCezmsP+jy9w9V$p6g`(Mh+O?3~u=zJUlNoMBZ4cahjV zvWPq@-X$uZ(Mh^WT#-S{(97ro0#b5@GgAcMO^_k1@^F zvOe70?Uvh41pY}jw&mP>$EeenZPrY5mLRR;ejqYz8^N9vEwC|?H8z%7bOdMfc_=X= ztgkO`W`yYFgXeDbpiMmho1tb?2?KR}SGp-XMC(QLGH95StB(yK8z7{E_=<3g#6%wD z_{59KK}a;x9j#mGvLd{OJ2M$7A*Wlqhm?KDmRk1JGazEzGnOh8Q90g}@e*otTMK@y z2f){CWJnnvj`{0-NjvPkh#b>-xbNEorhkkZ`K)?IN?%KBup=|qL#GuQGUvsomBMU5 z#{EIg*@GwE3Vs|0X^TO-{YN!F?a$WA6g_neN?_Y}sFzvC!wfo2g>Tbll!-9;FQgyx8C0%jyJ^Y1Nf;+tr)MqX1I$8OJXThmn z`U?*UKUF`+2Txjq+~gR5loo)sU(-OrhRk7ta;_^2!6gD(JAqR$G48X+TmL9k=ddbc zi>)>`=f#zv5(ih=&7_p0RJJ4nM#$t_Q0e@7m_M)Y&H~FBSF8rYBEII9+ZtWi6OYreq z_4$c7Gp(HtX!rp+FEa!NbNA0!iZ&J8&M{&esI!Fj@H_mhHI}B1+xr>8GYmil-N7A9 z*Q>&xrHz76TV3@O@+}XM1B?}>EPpy{z8_7CO`LAC3S;tPcjO35EpeW31~-u^r=~R) zBXmk4@7U(ZI>N|{h&v-9A(+(TudAJIN#9IT;D+2Qr%=lo;d6T7l&mAp!NM4iRi&Or z4AJ?KmI$QPYJ7}~FlkpJ-z!XVMU4#J>8$8P=ceN-Fh8J4+}B-bR(2rb5s%d9e3jc_ z-KGh89?Zm{9h>O4`&IGmCSgOnNx7uP>{jXxr~SbpGP<*P^9DG?7Mf$)M%+)=UG={osV&}h$tebJd=E#DN@1v5jjM7BxF zd$=OhFthOH9JrgXNIzS*^@#_w-)_I>$6Lqu~H@GRnaX3LM*!xGGDh2Vbc+x z?)xH5@%?L_9$jKu#>v+~hiikpB~x!&7qWEZ7bt{kcoD^Bj3A+#d@>4Fdf=F!KSLXT zwh#-B@9jH@H6G+8CRx7nm&=YUYNCL2dBqYTtr>sE&aV-FCz~pIh>m00;X29@2|pO# z-v2%N%giWVYEtcKwXAyWoDi2htJF@?*z00?*7`T?*f=Ut^s(i$h{Pb0RZpg=w{A1V z+NUYnYcIBj7toE7H`x0+qtS`JW6mO}Yc zYf*2gl?ao%yhFUg%;(tcir0M#!kueTTA8_p;M{>yE1uiaE`h)U5%9ab!ysQKLE{(} zhxlA_^;2uVeyk21q)@4Qd-7|->UHWuP8#A2gg#{?SEp7!U2>s5R($WxZ&9xi*EcNk zBY%e6cac6%rB|nL`3que{};qI0jBlVT-zud6ekrWWujWOp&>c%I)*5w7CN*MRj>nS zqx;e27D~72k7YA-b~0pNe|h_3M!!RXSRPS*_{!8z>RxM5Vk);5?VP z(sqF0P(?s4wr${&^=CY0AJ|VDwD3=~L0~Ma>4~k{s{bHWrAqLjP1S^xW4oTR0cbW3 znWh~K_}y7UwDIs>l?=yU%gJXiD15=vzC{=BRy%2;j{L6HtB)PkinTT|h7)BAl%c}z+1y7d-DoLa9y9ywoZ-+| zoUu|U(9=y*JV?zsc$#UeG)q6*V)%tK5T_sD(8z%5f>&j(5|z8P&_#w>d$W92_0Wgq zg63LYlan(Qmw&BD@ML||==Z9ES2N`2VinKRm-fQpL;Gw(Lmf;tpXJ_M2;^>SUi1K^ znq_zTZ1CHht!46B3QLalvjadYF5|hr(Y=XmGu8~Ti0-vpwUJc4@`LT;G*1p>?uIoB ziiX+r`@5K*Z0{Dx*x*0A=UeP z`HY|}o$=FW++yj0W*^$S$nai>h*^-`)g%`xdIJCfATryIz@R50WK z7RpEuI*E5>Iv+#=0jk!zKrSa@Db;lBe17zN^#K5C7*E+V=X>5PnTUX70rLa}GR3Ua z^RBTKw5{h6-Ll^*h%1lQP=J>WyOz_HPgO^7tKr1qyVjmpgYwVjw=$9`;dK7Cr&AoM_sF=)S*2IKBWHUg6};tT|0}6#yf#ZyH}I( z@Bpe_<_P_pYLZI5Y*lytKJQ*03YJ!&pf`!s-p@#?Ip%~?e!md@&vE&7eM{nD%11eUK%xs6t2T3ggxA8S#~# zQMHD7^ON|vAP1e?fhcJ7dc7Aw(oo1Qj(4fyuPo#+$uWp0P&pwR#QA`+!sRtNW9^u5 z0X)64e*DHw)E$2NJF(&3ACFEB1e^-TntZNbdIenWN+>$NjZLlPs*1OdjaO1|!4XS! zz~etc|NY{U{#MCot?DVTHyV7WOHvswOWQFs%$E>#g<(-nVRoel`FGe|tqg@4^&7-a zD@5mj3e4K)dlHB}Uxg=8`s`WpsV4^tn$d$0qktYkXfD*bhCyP_gi=ifnO+k zcpefyBZV9y&IG?c)j@ihgW?9>LyCW2pZuWS?&eTQ^qy2EgZx0RM&h2^tW+$iuX}v` zBsae=??{7qYtn@BY_6`cO_|Q+0`AlkZ;!q{>4NXZ+!Zj`DsAvJ>9+)puQ6IHRVO^h zYhJiwn@E#Ej98=)*_l6_9a=>KYcRNx_<)OxjM7Fa35NztN?!Sqzv-|{C=gd)BWbK# zaM-glIDjfLTvwObt?|-*ohIg4b!bt+G!zZsRzu^Xi?&4Y1}S|~yz1qPYuTT_*1l?t zn6$278#}M7aMcJLSSK=e+4GSyp~{j~SQE?2 z8pko7Xk(%_T zRwoc&5(6}A4cIc|xI{tbKv2}H)Ldseq=}UT>~-(f6>$eG9V-^O98#q=zYMR+8zjXt z!t#4#P{qT@U>sT4rO2-30D5AsO$Gu*ipL!FJAEj=~yGI#IAmYxgHDNas+pCiZfjpEW(L)+@notb(Ut#-uO8BpR5A4k^{XQsz&njDN{lM%(_got#lxPNUv*=< zMhY2JPF_Y2=Uw~OE0C1m?|O!z5g*kCEaLH7TS!}2!;7FPHW&`&2dl|xqNKxWo44D zCX-2^xs%UYiS;53i+5OGixzR3YT)u*$=u0yM>c$29keISGH0hztmJl&sMNm;U>D|p z;c*4JMh~eFwvw;Zfb;f5nmx|u#0E+L*fZ7LzBpQ`ISGQ;Y$VH$d(3n@ua@*%KhcMP zwQ*s)J|0y**;Ck+1ZZR(Fy|g?24ymN!>4ZpDg#~oX0@j2cGjc59gBGKdIb7>RMi;F zr)g*LHA<7*qf9>1M_>_yj$(8B6Ka7SVG8D~JoT|N8sRb^QkN}`rU(7|T0qD+;#8xF z>FG`%!^$Lck5xE#7bnYpl39z&;@~-ZHh)`^wE_U>``L8P;wF>JCbAfEMXW)|HX~M7 znvKj+x+C1YG;u~GENr4x??iK~cy)hhNxLm6CNU^A+<9@`*UlUbd1BO?cmyzFSsdo& zVLVLGw(hoGHs26gyNih~9&*)=c46fY3EWrQ%O2H~Z6r9J)Qas906E1pZaL>)0wkt5 zdd|kvY7D%leEZG@Xf~qPPk-U{KidFI5-%>v#iYQwwHK?DU%KK&DV^R$l2*eSz07<& zT9FyxzV{DS2d6os(1|!Nv#3^|jgsvf`&VR!I!3G!V!={Ukf(D@VN=JbdF-59`06h_ z_0xf~Tl{N6`d>ATo%qm@yX-4%W+z`}cMOZEEvUQ(UR6P~6pQ z_QES%cdR#X?rX!g?uN97pYnI8*p}{pU_|7xb-`lYq=ZRJ&?H*14WK|YTAz#SMnl!i zfGb)y1;Mhwnp|xywNbERWX|S5R8ZA35a(M<1gh&C^+c;Gb;MyC+_47MohrdyXT2pc zZk`~df!~`pG4Uc;Mzn9_OWa6VihU-}(54fu>g@+<(iS*B8`z0wNBkd3TuP-r!43V4 z%JZE?z3iheKwP&C`2FL-&-ecJ*Ft@HYPG*mhZ0oU`}EMEqQu2^JcIMmEolXpysi8( zI1l$z;~y6U1I3_wU7WX$+pR`MH(MCBf{!4<3`yB>d3-28lXa1-J?SuEAAaV(x1WC8 zCW>sVA0iJlQckhGKyzC`9KP2soW9tZsfD$*_os3XexzvNbRH);PXwmxKU!y7NXd3?t)Zm9BURzxMVsAK-~ z1nugpoS%P>48J|>7TK*RY&Deio-ACHnLGkk!~2riCg95FJgb9Jf8X`{E^8kti^a1= zbem4Wt5A3EKH_6rjj)wHaS?5YkAm?5HxQf3_kZD0Z>JU$3IsRu91ms5p!!}@^}L3O zr9?)<^o`6{_@tl^zmN~*Uj?16(Gy(=1H9ssB$GODs%U8rM}ds*p`a-Q@AhHX+8WlHP5xnY>K6#BL&oPt@D?d ze@lyZYaygkbUm}LNGbGWDKv$>Zv@>sj(hxQ4Ikw&jH2$>q8E6oi>|a=_A*Gx53EKq zEr?>1hK(>f3zD)oO#kZR_g{R#w^52vFz?^`NnoobDsw?s=Z@95C3gRaKM}QLnjW;2 z^O}2ucofb6XKbd}1!|NNvUZjwUW&d46tvqj|Ik}mOjUe1FT0=JUfiX_h6}x;1pd|l9p#aGM*S_mSAYhd-QOwGVr()0QS=6Ac*ziU z=f0J`3ip__sK0_w@0y|Zo?FfCV?Rg!nXGCvU0yVm^gexC3;*3)!`bl#t(y}(x1-TL zbeR_2)MQApe8QWlk>>}JSFl=-N_WwAVM;X((VWl(eDN5vify z8|I=tDb6u!$43#EK>{pbgjN{XlnYHB=z1Z@1q_sLOyXl&R>y9F{%5><)Z8ztBc~cY zv&f@|zMk*T`aiW}<{`HANz%U`g|hfw{S#y?*y}ORSP& zcptvAEB1WS0Pj*THpw^pOJAa}$j70$&zf{VjCl#yr{ocbOy`}g=2cEawT@cosmhSU z$)saFA6WMlcFONfnX2-Pn_9I7I)Ckjg@a(@E+q-z`hK)pgzE+*%oSNFZd(wCJ1^=U( zj{7=HHTS;~*u5Y4b_RJ~;1orW%-m)N zuZ9ip@3EkPDfY$^TB57S2eWgQeA58L!Tuan50Tg$1Ko!fKsi|Jifxf*9GbW>|Fi11 zrndSqAL&yr5#IId2JW+tHgPrptf;qYxSkg!yrTG~B@meD9Y%FZoO#*0%`IF<><6~P z7_O6(jTg5PQu&wsbf69FQmQFIEZZtyfr*#?-zJ_M_WFk|*6exfp&qq5f!*zm zMSdfOe!4!Q5(5hF1#HF=)?w8OU@28M`IjtBmoK@Q6nz_&dYWgFF9of#DlrMm#eEqB z#zFut?t`Ei6IzYtG^}+3%;gU6gH+pJli(BVWqdZaJd-*PN^J7G*Vxt{-PoD!$n5gw zj{<-ffT2#ylD`^wy=Fxia~P70NH5(@VFxP?ISwQaP&{f_l>!SQ-B}>-N(rQ;2yDZ_ zOeWC!p?i14fwmzlcc_+ZjqVJ|%k`nb;r#~1<|JCVsT@n9NRhPnT)=F|TF?1>_on9# z#R`uwxOqLL7^@}Ql|D6<4q3Jf=t7*Wf51nqGnx6AlLn~&s5KoJFR(upp`|8pSyGXGeAasJ;9z*g{zUlxlrTp7ITBzL0OcW#5&leOsV@IroE50Dt95zNhzsFRc)3`rYYdv;y;%+{)(O&D{Lf$U9t61Gh2)~2H zxY~HYkRGwAg0BO@c$HIB&dF6$JX(0o(^SsMe&GpjZ+tvCIqC50%dc(sD&ONd%o=$Z z^`n7S5-cI2J=K(DdFO3F9eDy%ja`f6zqF)=|Va_lf2BUe$xU6lZG3@?xhMZ7N&j zeqTb*BO=;;>KcJE-Wzf&h_pzzG z7_=KRcR(Gn2U}up>1s1jwP^FMaIGBPPciqVBG($DxUMjK(8OakBfRA8NHE<^Q^|S?33lR&}u`y?fFc3cop9_$vBf z)#;8&<>tF@2gdQg@GJ)o2lgq1NW)ia^;KX_ZXOaQ$EUvAp0N*S;JU{HbbbvA(gh@& zr5Pw<^%P<=^`cg*KI@-*h8>F@x-%HDet_?o+apEjzyPZb3ad}2RT5L9R%7j|L2>b9 zitHD!z7=>PIT)^2P8HxpyO%g_Ql_en(`E~Qaf@V(7d1pgphS=qlJc4OlMw{08cp$ z`8sxXr2EC$pQm?nij3C86V!&t1apn$de=5!gDL)4p5XXvg^_PHH2_4|{wonMDhG}s zZKCrZrV9ZkO%urur@9|@YtJ1|MTzC+>se}fvx()xKuDsN*dQZH)8`Xd3dr0bDxr8kCyuH_}*9R z?>m9~sPLz|#{wIUtAT-$Z}0UF!$lejj+ero6$aeA9cibhp&^qqCG-oAjxeydQj0L_ zS@-nT>+SmS3Nu3so>P0zw~$SMVYw|$O!jZ6GeDt*-qH;LzfL;9SN@Ry)i_JezDS5c zN8K!<8?2U6Ibf|-4wZ}|o3@D*5u$7Y*t7oLr>;3PW()A^=LvXuz|OpG1t^>I|D;a+ z$sL~5(571YSbkenLq?R|n2tm}i>(Qa^Y~hKIv!@0cv225d~g;h&Oit!^&<^n73!7!5aq9g^facpg95o0IX3UGBsg$(4I zEUVa!t^zu~2>mtpOYnz13Egee+l~7gpy|*?;#HSG;F2aHWA5F(2ff1 zef$?|n2`ERfno2_F(@It2u5N3C=xkQ0<)qy zI`vrsI^#p??ZccSIlqpln}!WytqSK>vYXDkM)9KOk(0H-elhWI@M1Eh|=4s$4zz$KCJi=s|LV~%CV}fwBM0}-N zu>%rR5e+^5VYi9~5Vu`T9kM$lg%lIs{5{Jq{#~xX=`r<@$4!he3fj0Tkvd}_9+Yz` z6x{)sk5vk91s+ekd<+wMD4SB-`;wZ#9dU8dr<*1XhOO%7R^skl)2KP-UwFJ3KO#u{ zm{pFlE=@KC*T&=kTE8v~In5HAkHLRW&2p}mwj(e6XiIB5<`-Vl|NkAO#gw1+CeKX9 z9NnCW)`U@p>xAm}7TZ4%*B@MdG^+H-(!{#Kpi~LST!Mq{bpMNbfR{~kyz6%vq8exI z)NRTZ1@tu`PLal1-ZyfC0jp9}r$jUJZfg6M1TlC%N`{VnY`}C(4;3HbB=y5sz3Hw5-~@@t-G z4|r0KgR@c}>y&_Ew5lU{%z`;OUD#Tag{I9~yR}jEdr{08Hj&k|`mqszcjuPzpZZJv z)4x%J%EgS5dE2!3ysJEU^B?cu-(m6nJ$deky}093weaWtKNaC{e9viZ;g4oso)a!7 z2RlOp0^L@zh89=ZHRZ{p&5i z=2q}cF5872RjCNbTktUSt0nx8k1~brewVVA9inDYns^_OnTPS1PQOU)NO5^0bu7d|JeKg zx&y&0qEuF4Cbx;1AZkB}Es^!PEeV>iI$IAZK{osQ>zOxK$$0G?(IU_O_wGf%cgKKD znA&V4UBCB>W+-9?c^eYA$$rL{wMni^8S{EK@v1(`4L0jmbHHhx`1)Vx=4Gn9T{R1k zXnaSZ#1}nbl%GV8Q)bn2$VjD)qWHgl@#6J0r^d5WQfsl`@`10n_^!oHmpn_zR__}e zVrAgJk6)=FN7Y)8qHsuqB5THsH`v3O1`2`AFhMj*BTWsLD@G1ya$Em z$vs?V@OTpKvP3%V1^8=amI& z-+4#g{Od@X=IJ}N3Y&}_u*mj80Hj?v08^hBdvw;J-L>fyEPzykiScX^4aNO5mo!}; z3jPj$udIVDZY=0BUL5e!hATkZvgD|?DI2%{U6WVz-GAnD8i9Pk;<-h&B*d{HM?<$N zPNh5Sf^k0+Uy!Tqeh(2vl(1aO~|BU#cc$~f1EOKdz zLg?_G=k=WL9c}X+hXcO8<1}Y}A2x$hPH%`@iTGf%e_nT2+?!yu#%3b)^sM}7>zv3v zC|6kNeYc1UdfiTPZw&vl=yf|ouzjQSFT9b{-BV8am4Zt5sogsPu=Zj1$zA-vy6!tV zvES+2tR}!6#hNdFJ;s;VTeqrtDsmZUZ4b2e^}T4!k#<7|_2Ydv^}QX#_xS0e*Vlia z^Z9+w;9iLs@S%_L<9V^`G?Lzgf23ME)T?@I?gxB0pLD>rA+Z?o78BnA-%3jZ9=1fp zZuL){_AP9=+KoE`_4yKTOe{9I^r#i6jFN7AR!IS-lu8vF8?pJ>{muF>&j<++2!6lH+qMsbszp)sC{d4N{f8m`om->Qh z-~2qTO#MMhw3VKDVO$6BShtBO;YEK?Ft?3@;$5NtkN1te6Gxnbj5h>!`tkT zZgD+C6wRbl*}8fp)4z zKbsT}``y%{F?Q$bmZ2jeLBU)#R)i>1=I)%dwGwuOpEQZ&`P5n<(`_4>=)taR{24Yw z*~~PoD%PAIPgYcR0--AJ$VlCaNPLMG{o<%rs$AI*FnJ}m9uAtWBIc|sZ6&ROY9pFt zN%g<*mVV*!J;WAb$7roN>pd$b+S=MNG_>Jr_KQS4WfQXnAOy ze@UBU0+i^!!Je^|dYkmiU4~?8+F@JcP;REG+0mu<%C8WBjTHVijR8Zv8SUp6v_=!U z$zOfLt7f5R#BM*rK5$zpXA{WYWeqFb2n#e4{yd@Hv^O5)kI>8*eZQQh9zC z*(+#Po#0vX?#A5plW>YAv1#SVc~{1`zPzDehE+NRZ zDf=~%A5-YtT4GlLxtwQFAm}pJ-pL3+I}>~Z&|Of_=~UVl9e~;NZ&%$Q$ZE}_OO0&J z9S}l7s`Dy}ygBwW40m>X=vI4nWnBt!L z6*!6nMEvx-@BJov-Ue@TqCV|0{So|{5gR$?0xo$i{~H3lTvYz3YE@b@#1^YM!t%D_ zSWAAYG@G2#oP$~$K0qI`I`0>@q0Yxv^!!0o8=VdXd7;-RPyy=QkA zOGUhMn~r89qiy!@E!YU{n*oGGaSEKGBU(X_uu}ojywLN#D?6fRD>}E!3Wp<2Q|nQ2 zTBxuteXLUBx+~7m`B|CsT%KF1|F+iQE5E^)V1mE<(i*io;Ob9jDAg(*p;L2T2QU9O`=j|Z)Mfv>lY!U zOLc3l-Qu@pOQ}g>FocD z7XcP~1^ny^+0VK@FQp{JuA1haMR7<;j=lUhOY`@YZf%66&K*b0Y8~%iqU`m>mI5FI zmNPZ7oe~42S2D~cr~0J>tVI53oO>4InsqCBq+E#h9V2UIWLq)q6BU9)(YdW{_w#+s zU`f(+nUytG$J**R%I@HByw)aR(m~_dsZF1T<@ddetj~zq6#JAOhre_W(5!C0+98@)jNq5s6i8sRyfj^-FQEiZ|y5s7V91HWa za#O?Y(uS=o>}grC`R%@Pdo$4HLTeBLQxQKQB`Q#2L%}i8)`77YE|OMs@-6|b2S^W~ z!Sc|O%%vOL);=cX2G)kp(@dB3<&Y$vC4HU*CrsVG*SGCY1=E5%#Fhlk_~`qSI)t1q z&d{ebwa}~9$T*2~NI6yS8DK@qX%84z#}dm%Y`j1YN=ve3(L99_`Yn_d+~y+!xB+>{ zN4T1`B}3Fx$0#sZ0NR+L)8DTjDRjj}Ie%*|b64jV-j(Jxy-&02N;x8Hm?dzTf?}88 zSn)PS8Jh|Y-Rc6&a`dMri~mF1TgSDPeQV>DsY6?e6lWT&1S{^)7WX0{I4wbgmEc~P z;?h8I0>vT00tAY;#l1K|iv)M~`JJ}YnR)Nr@4fH6_x;>o{@@&TVDFukb=KZ{$@A>W zhP-X4P$Z{aEh9Lb+_?Khj3?rwElvgR2t+pHR+J=LkG2mW_3f8o&$)0Bk=?Mc+p3o8sIS9mrRNtvwip7n> z`)-+$e@t9!HjTtx4nez=r=t&M@Ya;m&tIOm4%wV{82p) zCpX6sQc)&Tr6)S>xg)ND$<@i|UDg>935qBFg>=(B2u10R``!rp#0U7+kVl4W~qXFsm(j;PpM2~2T%xh#jc&ALH6X{H<4 z`>LCWLxr2g=7a(BUVcIary;dWNB2#qUPLx?F79cs-$7l1ysOeKXw(df)Z@lOxw`SU z80-}h2*%d_6CDrz`A(^<1>eWLphA_5D-xJ3tW+AtQNQ*GL^(Yhz*^|Q+PA4EA{dG* z)T7*mp{1x>7mB`CwrigWMD;wwZHPtrefx|Wr|eSMY!qZ<5sI#}7_Eovkg;>1+2D5PHD^%+Ev1fe{uU?(Y3Rv!hrgOSPNY4AR1j0=wD@yh)?F4Gygng9#asT3uay(4>9*U|H7;)1=FZg zR$O?;j#e+5Cxuk`XHe^TO?b0$!CT`@;|}kZ*>I~-MvtiGk{3f0%wHb}xcq*O=-1b` zUP1g>v`z)Ww4!i7NGpZouU>0IMKj$2ai{It6w zKEgg?AHOXtm22%v&x_%{su(O5ey5q3j)xT1}+!uZKO48#y@fajiF|SQ*S-g>0hZpjj3jt38>J(0`E$@*hGz{0Bz* zKYfdQu%wsDyQ!o{B)I*$PNO>GO8R8U9-@3W=Ki@W0a{NTD?_e1kRoly^_y8mdB#GP z=_6T9ngYt++~?VW_rD|u72W_@ELUzd^~28tP+h<){W!Iz(7H}adWZhf*Y7RBkMmQV zU+>R5QeexE)b>#%%yJUC=zgQU@pA)RtQ+O>i>*6RJOdV{#_h$|4zry0!<5_AH&W!R zLo+-$v?)xBn?$Pe))DMLJvjJ2>Bi4a|6g3st2$D=Y5fpY5AZJVP5bw^vky1wl;oo_ z+y2q}e;Tb7YPy#GikOc-93R>y&}H)S>S>}#ExW~FzIkL;N-oQ~U)b_qe}6llh@HYI zCrw;wD5QE1KFp!U$`H`}NiLK)_-bN_ehSWKSM$8V`1YkaJt?zvaT(JupF{}**kAuB znEc1IzBgQWF>{e*Bq1d+{ir$maZ{jeuOtV1{NnRjF&b)H^K`0MZ;kZiz#4zJzV_(N zMMg-23WzoJ%`D#Q0#+sSW>b^XtTdJR6?UJgmfky_4K!FUvRjED?q}W6WH-fYylfE* zd)EvTE(v~CJkmM~Ch@(Ob*NvLmffp0>y>^iH(Fym(Jap$q_y@?M9rFNw_Ko6uYxP! z@cx++eMD3pTd60`OH%Hn48Np7Rn)&8R42SX;$jNY%X?vzK|K8g1Z^ z9lfd+Hk!u9IYiaM_00U0fXr^kkvyx1dsg4B&YozMztr?``KoTsf<|mYA$l*(p(-wA zPWtJN1#h1FRge=2EkfvSu-P{#p?<`ETh(_cq2v^GSf*+Ed+T3XtZ8lg*zc9o7#*;} z%j07GH?(|jR?EHe^A)i0-As#2I@$f&q^L##0>Ct!56C9)4yR6LM@wFvD7@auRI!hb z#4H2Nolf6U>r*z88wnK9rh!pR3aqeA_pIub-;rP9m#wO%JwNoBRc5=gcsQa^Bf=?y zJ*JySDcxf%%I4u5&c{(lr2x6+6zc*JboK0jLiFs*Ox3-p5yy{ok+oo-)m zbAKhI`c0NdNihtcKcOw0s1mR4Vt65~VFM=&~%lem@s0>HD_&=TmWAIXHRYc(t~v0I&pm3((_t6d%vy6qVLRli=LA|wd1wd6pA`*{ zu?W^mF&GsMy{Qt&J%)J0v(c*~@4$bH4tB!tSzd)U7E~H&^XWKtUqQaQQm-TG3VLZW z%3&LBVr&=mEjU;P;{T%vt*OzS6PjETn8?K2G^a=tt!9n(q4f}~DlMx9w>n(`H`#kv;BX3z0&=Q1B`*4dj9w_21!17=N}!OH*zfhvsgo*isZ9l=8Br= zjDQMO&=aeK%uq9y@zNOe;$!@3$xZOC;|G%O?3>!H15>W{qR?1Adg7Y{V-mPmhEhJ3 z03?^2!R4{w0gE&K@h-s<(xB$4&5bAN7foNW{IRgd!j=NH-buq9O~@@z9Y`(^-2)v@w4dB780H}T^*mb>8v;c6r7*2QX5ivXZ1g>f8}d z$AfAdZtfLo+7O_I%`~Ov6YVOXqo%_^TxFTV3+UDo(_(JkoYASe=TtfJMsEfs`@wRi zWUGcGJHjbCHlMC@X~VglxIs4bxvS3ET$MDFm#JGRlPQ^k)!a{&8`?3FQRqK1dp$(*8E4$nwh(^1Crj^nf356 zzeXGF;v%tY9R0fGQ?PyGy@+?j&GpZ)xXpW(ikOw1m(5X^;YQq>M zlUI~l_>SMtky2iIG`4Iv{^S zB0S?{3;=8EOsmOSOySo$iiv$T11{e9EiU83z{#YN#}e=8t~o^XOJ%{&0L#-i!9QTm z@5`tZQt(z4s%j}|C(hQ+j6AbP-9ve=jA<|R&eTR@bSPwh(Y;bEJeIS&F*iB z*^Tf~(w<(=-~nq2k*|#gF>Iu`2s3~&RQ`iOY>TVwmzIhzK6Gw3!&NaH_mv3p2 z|FhS>P3Bz#KSdxYk4u)tupDh;c_>-)pYW2 zfc!Z~N16nHnl(jLTa@ss$7*u;T$f~MbcUsDgt|ah3rAN$JLf4zqVHpyBbOEN zSfNpRUF_WSB4yavqMg6Pp#L4KmBqN4mmY4*T6B8bCLS}=F)M_z=FhaPBpn3E*p|HM zHTrzn<+1LRJJxA4Ulz;g)q72jZ%5cXg`B(@UeF;LvBL%UsaCZ)ALiX(+h!35MA1xKf)32%==+s$?f1i=W(5y|3- z{#rCgaR#w*c0t>3uf?)AhdWddjr4sITJdVeo0}5pOZh9#ZdYR5zOgE_yt#|vY?w8Q z1&-0UYkPuLQxLzMy;TvKfX4ORl9hlw!u-1;p9G0z#zPOsvbD?*aPTZ%O!@AW@Q;Mt z^gQ{&L;)eA2wCgidtYr7<573?^_@sUZDJDBO)K1%5%ipo7#W*%VHvRGS=F+GcES6F zJFBtFxvLKL!=u{)DsIx4w%k6OPv6r9Aq;~XK7jo#STlMXxWrUK zy^>R%o%qSfp?9moa{3-Rn=WiG_KJSC-b8DZUD7LIB=nNA*B;#${RJkQgw@J+7A*fe zowAV3{Eh+V`|}~LnJ``fRV)=imW84o9xM0#elAtcdw&Z_ZQmKN%CKs?!dIYtgwBR=9R)BS_MZHi z97t0g{eAPDB2N(Nq{E+?99YLmv=cu5rGfw$d~Bt|_218_`SpcXzaaCKIyH5of99|f zdP5nyOe7foxfR)}ydizVC#lyxuHM8sIjYN1N-&`c3MwJ%p5yn{6k0jY`ITLdMxcs1 za8O7L1iBmdNB4j^(HduXL+_Zl*kJVjk47Ms&XrO4SuVLB28cU6?=kLk?oMDkRaP$A zjMj61tfDdI5F>207-VJ?*HJ}FI3T(4qvGXrQ1o;Y)MfD-)P;BKn3wV%O0*PSuan0S zd^PS{?jZn)(Ex;aO2j;(`TvE3W6N?AVa~q?qah1}tvyM;T;mg>qr*<3U2kP4WvRSK zt3%;26QiGpWMCAMD-;jjV0!V#r2qSun}YXyldV`2@_3c6uaiZXifo=l8`r-*cO`^7 zuIcG!$t*%g^ZHwab1@p=NF57R%StQM&t1-i*Hq$GhCC&y(rl4+CGioeJ)R$bgn<4J zdcZ&D1&F%&aPY->G=AO?(+`@ z8ys%@z-7L8KI0T)4dmEDjxzY>Eo$`lg$I|_Jries!PZ}nkCC@+GS{Eyt`2k-sF1vG z*0Zh%UrZ?86~gXK^l7n_#Po-c3#{4AEHoM%;`(YlAQ*j=m5t6Dq%URdroCfj=k zgZti}JYv-G_eUg^*B^MNn?6dL_{W_7YPg)gIcK0UNOp(2k;ljO4R5=M`kho;n~Vo2 z0Z%>4`|H$ni!e~g$TPu;FUP%x@a@#GG1OXy$*`(}9ms61Dx+l_z7kmX)%moy9$5Kb7d@c52l?@Yt#`WqPO6!rt%Jht6#Q{WtLM|3nho% zjd)v14H}HZncgcJeb6x|R_+XWEStk!3y+iqAFcMqo~T)DqFn#5)4%vNeW$cIkSpZedjc48GGEhijPd`-Scrj zfP3j(o6`k(=Z1ZkUz?cTyDtS^UB~MQpYHl@W`JJNLU2eKoxk zu|C zBX&63I@292L)$DI+co~)i$CDDa}4_B-MdKB&a(A=>x1Si6D?G=vOko^T1iAVteP!*PqXfrlIz}{FHo>#gXIPKd_3v)ghsojPEYV zoHhCtK6_%_Me?OCyJ*NLr-Uaz__j)(SxcyCOM`1CN3*eW_cu2O$(N1;ie?XW)Zt!q zGl{#K@_CADtuZa5%6b%!_jHl8fx(uxjwF0}Mfr0Uj0$#kq5#{ff4;6Bum$@81dttc zHMwkXc$_&*F-t9BY_u$2=UBvKPi4;ZUZ`Ag!^Vt^h3}G-?ozyAMFrRypVW_|$36cmM zmvqR())zko@ZY|&t^I1w>+Bv>?}1I zwmm!y^j+Soe56u$awsalIB(QFFKeAF1rXS8jEV=&%4k`E2lBtMCO&RS4GJ8df!hw^ z#1+^SGS{g)HdNPzGFcVE@0S%1YV+6p5~IDC-?sLbC7bjwd1UNGAP`?OpTkocODyt!L0>Mm*9ze!5zhm4rLwkb;<=mbW_9RYTV0(6RO8eZPYet&x2=TdD|MJ|v9b@DjTJie zX8W`NR@y%F$z7d`I;33iXU3--DMa1$IP z2cuVGm4;%Ea6mjcz)P(LS$IIToh~h1!%=uF0JLT4yHYoE%hf2ZljQOA=n_N*XAa+P z+7;!{3f_4;L3e4D@B9&&WoI3mq{^^w-`U_^{ZJhudq8p+VUo|67Fa0nc~kI^eXxE{_e?!~D{;<$`S)|&CxT9ECeF~Ale_OzDj5*RX>|-=YkoQH zSu^31B<#!jN`3OYcAUD;UCF+AcH@is__^cQadH3^@X`Y7TEJNCLSgd(_aiCii~n9H zUv5?S^q53!7{Bse@M&KQKJk0CL%RR)|7pLuA3muEzkPaNOvh}q49q#J5U)o<8@VNR zOmrcjp}%dk1sZafILHozG_}1-h;rU6e>WLU**|U0;W`4)08!?ty<$B&Ry}mEBA=(j z4i$UWKh(%{w9!jeobh*t$c8QL(W_qsGUih|k2>GeUk38sr4x4(Wt3Gn#`oMmoUr7! z)Un{KGRtd>_nF2MdD=FFD1NMF)Ju?|!f&2$*cR-?}j- zxRp?NAntBFRsL?k@6%Ut?&qUz@Vc*?Q(pcPS`dI8^~FX_y#pH>KoM#FuXLn6(harl zRBSM&EMQpJK}{liv*wePF;ZK!|s?N31E?ftkxp0Fi=AgdUlulYWOe>e2| zjYK(OJOBLrLC8NHK>z)j1OJH;m_v1%0T$y{G5bDa82qmW_Enp##aYC2b#TE7dXP1P0OIJ`p_Z3?CO%XtK@~b) zAaS$TeNj*r@P1sKSMn$buo|;p9?TwGSb09*+UaiDNAii~l-k+&thtz&^KplFCu53e zc6<*e!K%`0wG!pfQ<{*T91cr#b|Fp4y?!x;^}#NUtT)`{C^!wXk|eU_#Bf)pXA=Lx zGc0r3DR92Eo}F77GC(MTvA`WnY;&B2d_2g>-Mslg^5uT z91ggevu7J|?KmI^d{Y6cGHQ~mEv47%&juD`1y@>Wtf{}za{3feCzoN}tJuz{mGjjS zSpI5i3Rz1Jx*YP#ajdBpZs?AQQsbHq!ADGy9g&5#PC%wL>4-0SqS*l90dqagzFWPJ z&!8x7yWfA9t5Gb|LejqSmaxW6Q8x$#1VUnKweV-{CZ4Uvkv%T8^BE;+YiM(XD1Ety zqPW2}WB9z7{OdV1F%*8iQwdwJ++G~1dd(lUImA_kmn}}M!glZ0L;+4H+qI>|YC-kF z=5gTY%qGu__QrshsPkNSfOqZ^!vFG4dwMW6a6-4z7837ThRP(ao_K*C*q-h zxSoE28ZW5$;!+<+A8Scp@(OykNHNxp5AHJRC(26cO>k1~U$>PMq;APANeZftI56H_ z*5bJIf1tvgyMN+$)G#Ohc@j@aQC2u`V8`$#mJu0ik*)op{D-$++fz#3!X?Xs(=s1~ zM}5fCJiZz~65)CNx2c=4eImI68F*U>v4{u}cHI)02<^v98#eF7qZ`i(KGG9AUy1xs zH!v(ty>N@!(GOE}<+aiI*#hoe*ISn>FSZ(*EUldXI`q(kJ>=Br^{0XtThO~lRp;CV zzR)X81C-`Z|4Thx?2(sUII)Ytr)OiosC2dJT_gbrOHzJ5(SKVEz%_i0oy5|rQ*63* zU{wDnc9X(ghVR=v0|sybqE$rG5+(iXV`nR`IC%Oyja2Ys#LyT!!u zx>$)fF|B%e4G;1;Dc;oAuQ8ahv>agv8Jf+rmN5{(&fLfwivIdzWKv``HXH2XA|P7% zAMK-mPV+yxUbvSf&}e&>(Le0jZ;YHaw^c9i3dYmdDE$V|D_&`|El|EzKQ^IEXTD(& zP;(?=Ff!pW+{k=n(Xh`yai=q$LeZjmFjAh|Ci$QTqJ8W8f%LrmzOJ|6=#%pU2ie-})cjoS(%d*3Yaz$wcft!2^(-x`tJv5Ow$67yq``zh~8| zd=<`2w|?qK&RPXlzC;N^-nm=Q1tTGAcQ^VnxxRJM5TI10c0X{@$!s|K_=q6g-q2UG$Y31!c(35Cc8$My6|@8A^nn?H*yK_Te9oo_yrV=yJIIC?HUhy z>C)dM@NXvs{0>7bbRC$2S+}f@wHxj32Iatbh;U9a&ssYz#uXlQ#i~4K6?gu%HtCz~5w8Y>$g+A%SVns9SqV5&Tv*ES-{I;KFr15Fi(Pl$!Tp z*Q=Rsbe-fuT)ekYxSb@VvaXc4e6VxH!KBe4rIhn33mv-nAyi6GM?B)s{kl zp9)vBhvv<&4YzmENoFrcTKG7dx)eGzIF)^~*^+?GR)5}o0&_dfW~g2)WR+((S$8s$V=|PMO?d0`7tv|)8%(_8S{1E zIQhhwsq02h^PIvWkPA&LOIceBEes|~&p@8GbA4fChX7V-YvQdDzmbVyUroSV>w8(W zcsrgNDEIx%Qh$%o`Y^nk{6~V$cTKu36l}a$&3-BZgb8@J&e_O$K}<^{X!s5dUlq#I zUkZcKj4}lyR~jxzr|RXr&dUkQfj|Mk#7X_hm_EWvi%m1Si|4$WVV0lF~4X{PF zGP)JRdT2v7(@-YQpPK|*d)J@y`Si(`EpwYlxPX!P#_+=CyOBLKSfmI`j!xc6JHPNp z9~XkBHHUkS_SKdHdzO{7m8kgDNy%;uvcVGW6M|N~9pEAnvg5SDiD~cu9Q40=b3UX$ zW=nTf;$1~}CNl;*=sENFcM z6B5t~Z<~Uvl4;==JQ%nZ8Wsq){tmY^4J~ewKbFV!4rUW^J9&}AOa&n4cE3;Uf7=H{ z7E9{WFRXWDIH~yNpAiJoM?{;VY-wi5DqMt&6Ld!QV)ok28=I=!aZit5zdF%Uo};`X zyb))WJXAuZS_qIZ2~vR4Qak=*Arsl_OWLRam1g+tRV`}dN)~S$?I8)-Z2rAlIph9q z=P9cg$k7+0T}9IFt6UGz&ruFWdF9Z$xmA;E-(>G_)V$KgarSvHFN8mrxDAW$E}=W-tpiPtuP23pU0C|s`cLeH^;FG+a+x#)Zc=Z zlK^6&?Ki~2rf0k4Oav8YbRfbfY1MTNsYn_jglQ##p9~1>4w^Y8XEtJZZ1KNBsjM&P zDTk0Vnro;uu1!@D8&0O8X_4mW4&0(xINqgYyLk27s{MI{L zdBq~jBWMqDU(vF*X=?cz6bb;lrhyoQPcl4&OpoP{ z4UUPdz1;4jtXC87Z8q=PajX4=%&%%SF8h`>;zRSVY!b2Sj0LY@Oyv|op`cw;nYW$P zG6dg%-t*kytG(=Ummu}T6XCmYXYKO;=kgyWdZK3&&{bUhNUK=HJ zp(mr@{6%fLyUJcwhdF6~b z+FOU90;d zP1OscR2FvBBhfyJOnWUa=}f!ul3H^6jps%A6x7M$I)e#sw&~m0D=M?rO(G*_1@@)X z;j?OXd#;~oPE&Ibg|*&l`b(E zIqWDgs;^={hIqNiqP0Z7G^;0kFgZLhgjHiYJ3#@@0Uu-^DS+%F-MB`mXeAWzG#otC zLF;U(75kSDK9v|(xQY#0GIdjKOMgK$n*yFa!fUSfV1%ZidtQgfVeh)HCB@<4F`>$C zo$ZMui+8xGwtpyhBt1}swj?({;dy|oTiD|JuwTBfe0Bg)?~Shw6UuZmMCZD3|r?cDkH)$vyhBc^x~?gY0Y#5yMVgzmMtfWJbO+S;B9_J zR`WT}_Bc<1*39{{DG7TuUA&O)D*x+QVY?(W8Y{AP@HVcPg(TiFp^hG%AMPx7u*dT) zX0IPw&X<%st$AX^ThO6`DT%(lsA(s`dewU@W|3He%Z8Q6z_Za9_aMHPL}z^F`R0tZ zESVTfr;MiTbvsf?#<{w84y?8&R==OK$;9sFkEQNrm)QoK-W@2d5&l^>BOO`&+$TmV zYQa0}?{zaDekwF@)u`6oSDi9h>^^dgeS0ASplZ!igLQD8B45T&3z6jalvBj^Zn{mp zFJJa?(1*yLC3Z~SJaAK&?~Q2;@>=J17#goU!$FL3Fv!E?zjgbM+Y17PrN4Prsket4 z^l3*I3tF+jqDhKNN>HS@>&Mc|C%Z%)qGEde(eY|+vHAnvt_d#;KmMbW|5`&1?ks*V zpp(6sN?unDa>CZGXU6xUXIf5=wgy(Nc6c_V-U{6qpk&iQYYZUVey zw0zO*7bxOhGl5r1`Ic8p@Ti%Wxn3IlrIEQw;ijZgP--Q=C=+fDM!0E1{F_rQi zJHPt)8#_=f&OJeP>!j`viJUt?;%dAYLYQFrmXK{e zjzEsr$PXl_M7ppvT1S1Fg;j{f8*TA)Lj^ijkD}eg#!^N$Y>(xv+TAmZkr~~@&lW`l z%2tlKdXo9ff@K$}_1$fVv=+uJOI|K+riNev4W$;+TYnY`X&qei8ZD#3*3Y+J4L`FJKAS-9z^l}#?iWwQ;}ABKP-5$4Pa{`pL-A%*XQ0G3HrP%i zH@MHplv`Bob!JdeUKcW#VG6xVuYV}+lN(9NDa#Zx+*IYlc|s`i^K3jwm_+Sc?vF_Y zQ_VKa4v=UjjdJg4H;(~kLUlFmbMkSYsg%gt@8{Ot>qrhqUH!c44^l%g7#?y2*`Syz z@)T?vx;#3gd6W&ODJru{3Jwx@i z&;F76{&+244ujadugo`xL3$Q2(xx2Sy@4w{?-vU#@h3F)kv6Mcjr!c1hf0sV-7#oFNdnz@~(cd z-*}1;htL0}mG6aHk=da`C;T-Cv*2n9^GH33x%6T66aMGTZ3?v(4n-I1<%zsALI9qG z%QY6b%+WEumhm1j;}SpWHkV8}G!r~$!g6IpkUhXqG&{q(U3_bQ!gR8r%H7>I5~vN4d-9_%lSnoB8E>>=?Ev~1kn6**dNN>QSlsN0aMC(M@nb_*%gTZ)gSiu)%D+eE)tAccXL|l#aJWS`oiZV zX9aC)pfS2h5vE%O?ZxjYy~7K8-4pF|k=?MRu6|iv3sk5lXg3Ybh7TE}R%f@y@l-5P zWNSkdQc)3}Y2H__R53<2n=aOlAHN;UYn1Wst)Xmyat=-&2Frm(3SoWD2(W8f>Tl%*J4Wn$5L_-#Q8(#8q}(x)&G76d_&ybpTy|f4 z1#`sRxNOHw9PQpGb~Bx{gP~{dx}H7nk>}E4)KB8}V`mT#q81si0(~Ij6ApSWMhY?Y z(+`b&Nn;FJV1mdF=w_Jort^KsTe~YnR6JemlHW{IVSK=(NP-zHMxXDC?LTt5#-u^u zBU`MM!^GhUw(wbXYI}FEO(!{`Ljb*`V%FETW9C>~Pdi`XMRJ>HJmfZ{P#}n4_N0*} z<=g(YNk9k+E;I#^bYDHX(pIg$tnu<*dQ7u9Tk(YGH_1Y1_|@gH5KM6zm=beTrDV#< zX&B@A#^7o{B-nbmIgR_{d#5HFPS)tMk+_pH$^JE2okpS+uL070sF?(wD2&Klgma>t z7``%^Y;7D0^8io@w2AKi`7JUVK=KFuJFVk6c(Dw$q1sNyLo5q9xg)`suMbxAXmfPx zbfq`di>vD>(yZ_l+C(+vWLMqFEgdvfG;OqCGxWBTmnc%glao#wy*Q5UoGRLA7F7Q( zSVj~6urS)SaipQCH{${WcvC80?w7aWQh^EA3Y-|wjtV(dV9uEHaSfJ59`0tbccUU8Mcmc%>eSYtBrpDI0WKFaxUCm{);4J8u(g~rp0j`nQ|k| zhOau^z-h`idR8G0n!E2QJ7NlY-EG*l7*VM@&TtXa1XrG8Zqt=?n!KGDoiP`l!xNvx zTe3R!WGh|(2AiwQMfSHXUz)UH2+JDZNT0WrW??iPIsN0>VFy3Qx<;-Hu>&NlhyN*A z!+{lT?FT~wgQ4kgc-bz!;I(Nt4y59-E_-r>-_8{A~Y(_1d~8SpUAPauUb$R^iEkFA6V^(oYKb435*P7VfmBw_P% zd#fED7@Jj6`v!8%lA&3NDkCgdP|vht-E$@Ig{4Kp{dBDGLZY5=Ccz*APM2Xd?t|z%dAQ4c41L2G3W$>h(6 zBU022Jb@ubmo)R=!O_v?Aw&w`2R%=Sh&%1lt@L9uJ7vROxuZ(owmu8>f=tfDpJ6cplbrF z=nb<{KR!cUFh>lqIaiM}e3LI>l`wa){c_!xJK?gzyC@Gn*Bo<_t~=WBl#uaHhgQ4Rfs*{bRnB71oqYg>tT`B7aO zxx83X{iA<1^lj({R3P}94;QyMj641 z!_BydyQVJ2@ttZ%2h{^Kk=WpU`CmGbE&WF2+R}B*(G63}f`gi;o^RQE0uPgsIKvvN z#2iZXpTdn4l!$;CH!~*8%F3!W%3hG-zFo?}t4NlRPS%WV8xpX^l9LlPF={+@vWp+C z&Q}h)4hUEYPm(mAxN?Q4;Qhy^k(b&CaIM9PO6Fvp=I++f_6}2an^j_dtn|(7##W99 zIi9fMM#wk$UXbNLHqy)@S8XT z)84TdOz(lgH51mvbWImi3C3iLCIV0nz}0qQ{+ri*M?W^g&MUra)%_8LI7Gj)X(v%@3 zv86Zdb#S*tE1@_oXZ*vh3d{vi+TLDr+?-dGGs5=HnkgpMMe9rW!c0OyjN%_F`Y44QrIgBfw!#KCvLfiBdfA4kf-F68rp(-m1TO3R!~C0;)!p zez~g*-%9Z+#DA3H1tch`S2qst(t7pdrYO4*hbU$6A)ncG9JAoLlOOmSK6`= z)wgKPLGQD;SYm|ga5Z{eH%EnEvIMIs*O#oWH{5rB@uj!mvT;)leel5kyH&g4yGP)M z>vl0{`#{9{yNeOt0^&+8dHIMvw$RMNB#kbn5_u!c3T;j%u^7_0iUzXKgdQ_?k)({U z5Gii84hs@Oa1r+9bc_4rhEc7ag6X;iMpf6>x{~gk$YK1$^h{gC+?xCKgsYGf%@7S3-5vUwUOYKl_#I;7hylQW zwy*nv0S)An{Y`V|Rcdctiq?%~49;}yNlT|_Ad1%w87&v(8cn|Sq>{1KP-yAN>0A0# zXZgqv4*jXethN5heQ`lXf>OkfGYTBQwbwTr29Kb9Arjd0m$aIkfPgC7Y+Hg;I~k*tE7fpq~tG~Wu<8#xMi6F-Dz2f+%~$xCo+fR zWd0_)mdBUKnslQ&u?sQJOPoR*wZ~h9vKP!mCS1p6T2$$n<%72E)`L*EUzPHcmVA8EzKolyMz(ScaSnD!lT+n!YW;o*^lAmEzp(Ywl9@Mg#3x z?;+pFcT1Hvs7=(ZJ^P00pP1{;MWI#!@s8u`3LqCdbNSWY^+$dZw7E-{{khQ=5N{Re zBwCtS`AxNC<9NPn@&(GgPhiCxy^0Y1y*+M&7YRnW`m5{$91Rh#+)c>xjQZju)S#f% z^|ZHtiTM13`lp-1;@;Xk(KN;xaDtfvIwyE{+$Sc!vRE9!p`<-iFhWX6}kd&vu&5lrB)uH553F7K_cI&%e(8+(i$wAFR+iPvmOT z*80p-5saa49KW02FhYRuEe&H#`2pl{Zv@s8+2LuM^UIAtl}e_Y__qe@6)&CxbXcKY zxKkvcRkDb++E#5MdcS}KZYduGl`X%kErO9k)~m*@0u~#@*?bnFmNHj#`8Y zBq^&cpVdC{UV)F&+t`E(MiZGA!>-=m+knmqX`4m3;559Ryg8trN{M>ny;5g&Z2-)_ z`OZco6A@fJNffEop<$i_i)OZ=ofjZk2u?g2cnI?SQ^c$}f&DUS z4XSUi@>kPOl@?3NFSbC|5$Hli?h$5cH%04lKBA2&>KWY2_<2TRK0fdymLk+l=XQxWPC z>iW1?UEf(Ik{Kdn6iu6mfskDl`b&|!B*Qq2uJ7FcjMOQl=4An))Af^?#;#^5zZtp3 z?c4SE>;ju@*O@gFv?Z9Z6JdNsdXZDW-bUC^+r=W1D{S3nVijL|>`8dSI})MxbeEh$ ztG&lOhYGATJT@<6nZ~Z+1J4dp2>5fn7s=Q#7Xsw%XJ)cgV}#WYNmE4;;BEfpg9k=>r$VDD(@ z*5Ii!|4(L5ziV~*EunAH*4f7C)}Vh$WDjm4UR3d^SZ45WkygKIfS}@lV#R`r=IgtbRF#8+fT9*x#QK5?6b1KzQXCn?$c7P zw|zPUoPR^Jjx|U(d9P-^KBY{ZMfAt&xp4=CcltLV8OQe1(n%m167ctwCnm(sEHsRlPVBGW=lG1qwgFu0 zLbC9^4{?OexRo-|Mxx)Y3FwWMV!>!ebbH9#%X!+L~%0hODQcL(i5O!cOiy^Rfz6e zqQGzai~sE`FNQXFwIGAQn?^%D%`P>0UbH|W4pQaC%=GVUcK)*_SaFqGyYFMl@$!3I zOTR|mZg56pVsn_p3U%o;Ws+JeCc^Ant{7s|FpqIz_DkIRYcY z&zh<<(LN_Y{KiZvmpb>sCo3j=u}W|O$ShD#_k>w=WfUnS&2*3+Vy}aI|7OWh+O~1h zQRtwBG(JO`Q;b(wA;V33*L6{9o~x+<|AVAuaZR0LZ4zg?pZfZv09{PMz-F2H`wRw& z>(DyhEi^UF>z_GQKN(v>>h+cHr0XZ5Kg5po9va#@;v<}Qb86%feHqpKtVeuC>`dlj zYyTf}ZynZFyX}k8(o(@nac|KUEfU;X+%*KJB}fTY+@a9o)16Fj&C+TyOkrC4z- z!Qs1UzxA!X&fRORbM`&^+0UJSg!%A3GgC6>nByJeCpuhb876O2IMn7z1$T+pRbDhS ziyNgWGI)!iniw!u`L6?^wWT^vfHE}vBak50|E2)(vxvdQ2h5*7H5N6UT{Gw1E6%Ndw#*K#U`(iIj4;dL&K zyI<&&&Oe6RAu>QqAj!y$#tIP=qylCCH&e3y>B1BWK3|}m_@iya?(#e79g{*DP8)h=3*B5Gm3SeI{NS^T&?lz+Y#>$4sE$ufIB3AL*5nVahXKz{UdV)uYi;%gH)P z?~IFY4EgP@Z*&e53{q*QC=ebIZq|B=OIvQU#58_vH(q19K<~OUoo_#LL=Z^>_!6{l zD!YrR1l_55a!sj^2WdwxN#O%AseYOT#?!xfwDZ57GWilDV%1N;_YSxHU%;T&zkosH z{|XrNU%r$6zifbgugq1xNSqTo=$_&0UM_njFDW}P+BYo! zqJ88r&^KC1Tl?Mj_m2LDOHD%RTB&2w{54szdI=u6-b90D<=^2a|1V$k|D!9=X-R&~ zoROkFlh5jVW%+oJUO+l}Rq4&4dZcQT4q@}IFxSEpk-1};d7UHSp)^l}G!p=Jt;m3t z0Xn@JVmFQrRZn}1?K*W4o=K|4mgyMPy=oe{(In<-(}n_N+wU=)!6x_}jZvHHN#zua* zxVxBkzoj-%vYxN0vy0C`5#PA3{UionS2;eH{-?dzvlMt#FylG?FF~0 zB^;j^W=sMw0ihLVxmlAld~_z7W`In*OxDYl?Qyz2f^)>$f-qW>svt7@1wc6*rcVYm z8J|s;WeXBur+-Xh7*W!_@|Lrq0aQusIQ20<{doe$p!q?Qk7fI2s<@GbZi`$2kLrbm zD>y3-Y0O(#(a5jMt@*4?TS>3cRa0OiM=nF;D*K%8H~KDx)nvX`S+6NY%iK?!_=+Ob z z*XJ5X9?Xus4{LkC-cKw8#M^aFi`H@FTq77i1smi??W?@@|D{>vHP`J)Ok`7xD zB&pH0PT{?@Zz*&2bemClqL=IumV{u=OhtRBTV zQP^gH-NB^R0!oCNJh2%n(0?tW0A;8NErInjzyGy!30}QiP!5v~ELyk8LUzr`BVj>9 z;`hj}(f_W6eSvql#yPm*VtlJlY=7TGA=?bOH`7LvWUY`Fq5Pd8UXW*}6NqWlPIs?{ z5ov1Rz5li6|LNDZ+957uY*D*CH82nznpp)gPk*-NjG@n0O=puo`gfN?XsXhrqP33L{MoS9l#68ST3tuQt&@rP!gjZlUV*552xH(1||Y2@kzzVXhOiqE)}F zW_-LQZ5%Pl`}_!ry3X8fsdWH5vxbBAWUB>8b)Wt4c6W{|sZY8+IZZhcGQJX_x~mA) zw#xrXN<9(L_?TuucAmVjvqg4O;PlHWUlElIIu164yNI2A@fNcCw}c`8uY}>{8ufsj z<>jT{xHtw zH3m3xS&e1MRz?(>W%A!Roh0@jsgx<4!-REaY%NX7LT9U?wEdN8fWSq3!c8Byv(!H+Pf_HVAs=f1C8|5NhDkNv>^)e_HPsL*0Qu$+p|H?H5p9!P_ycYq9x)Ss z$6a>+cF6v0C@M7H6>5OaW#P#wy~(&AXH4|1h9? z$#PxQXx8vUx>s52Kg;WToZ+ zriEJgAD8EjyQT10=U!qKuv4m?6H`~JHXb#5`Gl5wtkXwV@l5F+f&Bi&%2r7f_Db4QbeSi1nIWN> z8^z~jtJbM!l)b&hAqw?!omT3?A(dNVFUXw$d3$OYABWJ(qy! z(S{V(_El{|yF>8;jI5liAZxs!14dS^7t&5pN&jkLvH>H!0--FbD9d8v1m48?dJfG= zJdfM9nkJO*+h@03v?y=2m=95)R1B|Tyof6~?QPV`Zc(T>=-8P=#7%P z3r2>0XN!8HxJlJLVOuH-WHyrMR<@z(7EW+`5ss)ne%0opbwibE`1pjqj`s~)Mrbl^ z{C3NlE`zw3!pnMZxOs=EitE5gIC|UhMb~i_V&qs$riT)d6Xzn#=q|g(lQd!$wt_1V z3@yJqbpbE$=uqTJ%vwwUl8PiRI2`a*Z|Z|--@0w7gb#fViZo7K(A_)Zb&+VDvprnYmX8u+a>_N7!+1~p>uz7mrjDWT=KR3_?JSxM=Et32r9>Hul34MXa)Y|Xrj(oGMF%IUB0DMR}QV#;Wyi$WU6U% zb&@|}oqgTmV>E%MgWVPS1hv2Z{K|^RwE*AMydGa2 zVibiVF7AEm6Gq&`64r||H(7NTelQg%*F#cbv*HbMYbaA4(U{z^SK23WP4^01+tvEA z45Wa;6)Gi0i($<}9*;TpALM66rw_&qS!|ewh5wMLT{yGSx%mM7p&T$#Haq-6-Obvf z;<7d#)D15OalF_6LP%F|Qj_{S!b^K_Cw45=?6NnoLv_ohSY_j7rwhwoQ>Qiqmu48? zTj6v`1S1A}`DM!{7LH@$XBUZWmZSx3^Xih;Y^y(Ka>PHyk{Rhe&AZcg=UL&n zS6M$cb_c$A(o$NgrKzgbktQ%57&Y7@(j5tvh|`(oqK70vN@*kdRjGLO^;VOLrefvR z%yw{E4i{fdl}7pHHWb<$C$voCX(&wOY9Q2!i3p! zLUBgk6sm`}qp}3#yb<3}@-u1fv8pJL`}3Xu2u8owZiS4`6yMF`iSAz8Wzuw$Yu}EQ z_rs-L(Rki((zqPm;uE`RYayn~Nmr^oDG1%;bGB9kw0Fc4Jk?G%-O9_MX1sWK$FGi} zh9$nAkEr~h=$>RnD^Sp^U@766+Lc=;a9)2U`9We#GzJz~203A4yM;S(g5DL;H6PQd zEK`6tf`%;ROH%SL5zj{O{l2GHj9)~AB3d~Usw~K>=$k{0FW60A1L6S_hlmDxx*lxooQ+ibbgF1-HYWEs?+xL%&UX%zZhKt8x?2>jBzr~r1 z2(J4n>E5z=8X~>zeVO5x;lJUM-~JwRkuVDrD2J{DBHWF_Ao!r5g*lY{3mYzu*Q1~L z`qZhlwj{Ha2tlnAZZ${9N4QlVeoi;FMTv?RttEqaU7c5*kbTudteJQ$Widj@hP?GU zrj5p1H)xsN^-C!&vAZTuORt9J_ZxkehA$Rw))xyFFKC}vKEdpD$&G!#fX`Z;M1WK> zJXUW)7vOseph1})Vw-wE%Y||>7iE7ceylUVfGOG8l{)14+JBUU=tfV{yqyqga1yHL z-+D-gEN#BvYP70H!e%0THOAFlQ-#`||c+b37t0(k3 zk(TNbpLL!}a!_xMMVcH<{I%i#-*m=^ROQQ_eBDyz`{>HryVG6{wv^9iG7A4#^7jM& zCoA;HjgwEg4E3#ck?#5kzN4$pUO5_sqsc&Uky_2$5u#7+-7^{3#O9dkxO|O0p|&M^ ziBOtKef}{dymC^J?$oD7M>i|SNOI7UXYOU-GQ#E@upi0*UMSbMRp`iD7CO)4xjG`J{83TxDC;7Lh+BMkI>8+K; zWCJeB4b(ZO@{%@Rg@_=UC_1K%qNKUYh3E@*m-CzQi%RkG+)6&xlEZm{4#!wg@6;po zsZW#goqsm!!$_a&`&R)SY_-VNNop*jSaY~n+3Xg%oS&4f?I<$ce%@9_yNQ<{-(`vT zJRRQ(4PdQ;jF$A4``ai864is%RacKsOz)1iy;AAWHO!9zM#bn8$I4(^?iqwfv4fZw zbINho{CmvUKTpz;4|h58E;ws$eN-N6vwK=9y9W%cR!Yh5zb)NvLhwB!00|v)C8v!o z#m<$}1%g?@-eBE$dR*iXe_cxUZG*r6CbxMCx0xoF{jNVT_$R46TmgF11rv9qz9Xjj zfgwr~*eID2chh(ex@>O};F^3Bb@forw$g(fWfvV$*j9SNfI2LcI4y(oaH12z!$2e5 zj$D&!r__o5tDk<_Q|d<-G%DJNdyNpn zr4_(SYF$s~;wDYXa<$~G#MnCKyK^C{?|!o9=*ZuvMIS|Nz+$%6)M~V{Q2H)4@qsv! zZZ(6~;Wl>Zf2+{1{aK;+q7~9h5zs28ohmcEB+{wcFzf(fT9W&ipf|xGHcB_oJT@#Z=cL8t5*%SvK+eP~36!%vQX+j`7T^ zGgrG|b35rFgstrB#P3L&)>-M`%`-XamAM|O5~u9!tgmt}1QkMdvWJXrQ^Pxnl+|GZEXE@z!&vLtRkk52v%w@W^HUVqE3rc!2~6 zuuIX-$JSY)PFKkhRHcG2a-O_cyi#o%Ap@(3l3J*BLcisCjlx>4RsX>H40)bja_mFY zK<=WOot+_cm2>_#*6y#mA*24K#V;WN1M>2J3Pal)`e!gS%!~d0@_s{Y!8MUYGF9Fe zBp*m)9Wku#b`1;<7mpoFdkrN|dxRg@`%W$zNR%WFF0z+OgQ{CIU8u5nsfQTID-`E; zXw0~(ra^$4EIQbX<$S!~tL?p+Z|`!D=Uc5GXO7mwyGzUw z-4)2qX0lA@x@oDY0qQO!Yr0;s`Dw?TDM@iNfob5Y`F1nMdao*3#pX&?OB=S7z0{u# zCc7>8;{*ryVze&}20a^xP(@jDn?F2wK-278ol1^$o;*}%*?u=Q3^O=louHf`YDPm2xD%;A>{|8}%-XeNP-`Z2}b@xO)H^(GfNn7+aN)BF!C#rO78e>*JU z0kF}!FcFa;&?Hwz+IYie17JmcFszY~K^s%&%ST|IHvQXCaTe;a+(h6o9THCL6VHoJzU#e@rVdgFe&U)clAtK zcBXspaDatHRXQ)$u2>z@-EaxMeZA!0@FV`GiwuVFWK&M-@j*XATS+60Uaa`3s) z=Hy{DdZYmVA>esiqpS&zG~IS%4kHAB^=DfDKiu?sp3XR4$Njh-%<)d?L>6KJ+l9dS zN`w0UeiHv=g|L5KQ`g8IJ}~KJd&a9>Hv*Nq<}Ec71!PuI?e%~$7-Z}?7Yv46-a{sA zDW0ILXaCyp%L1cLQ~psE=dNgktWH>kvW51d^CT3jRY(^q-!QBSAQCDa!uiv)?+IdE^LlUouMy@vjkh4O*W!CU`jLeotL2Mp@#nRc?t6oI((CM`0@$T})arK-7$anZ(0lh+!irm; z?#R7bJ$d4_B*%zp0j9ue3uTNo2~(!g(U?yCA6S3t4B0Ps6rZwC^uCjbRFg|xEiz4V zw?^wmY(Hx+8kbnB@j_`3aCae*F|yKhH93kRE5^f6B2>8nf3A;BZ<)2cW}59P80s~b zI@fpI*)qLssE#t6+O#+2wEqovksRAu@7Hl?DZ5}ZL=~3{%%%>sEjLKi`hGf|Lmt%* zoEThAnCljkil7Lrfm90xPiH*bb)XGmR%D%MwXDKO2*pA#!kODfDM7xZrO=KiV&$d& za*7-GX7P;7$y?9KmgGf?2>``;nr|68JpZ2NLOh;Pk@al$*+-XNs)JI4)3i60P1!$y zTDsc@9W>ekRhNQuwWNKdaIn5Ls&W+IIRZ-P4DxJ+E}7XP#<057l^|8{MPOcSx% z1F7q}q^aIO{WfMM);o&Fp5ocbsueuvLp@DTdK2UyiNr;Tuao`2-}R%5NSUgr17i}p z&;EZSi@JliXB!l^{=l;Jq>o?B4*L0RAA-?Jl@uHm#2{_I{*pG$cKS=&l)s;D?dbd8 z?u9@9Jc{PJOxf}uA>Pa?o9|zk;Y+tuFO?F+x<$H7N99u+G_I3n)@c1)9BdUe7&Dno zl{dGW93EVk{<&O6`8b1GQkW|Dy?sOZ{1`D0e>xc<^-<^_Sg%+E*uM*q*@6h4OPK4n z93Rcje)sElILlE)3cKtzjYY+G+}{~XI;wn=RtTSLM@RWF%Sr)?!;EgL7_)o&0@xSt zaOkt-FzlW1i0ZOFHd)6vjBV9s4X9&Zi%y~B1e#3nfGl29(_tNM`P=9Z3F=?e@PBwO zv8ne5)=0^<9`yv)xf|Rd$G+^+WmrP|#(<5A@$0-MUCgBV2bR4++-F;Ezr^1;0+^A1 zFQ$=o4&z2N`hu2ruuNv@c)=xO;{`6}~xbqe+tAfsQFxI>V*44;3o*%W|+O)nWy)vhWOwqEQh4K4q&gee7Wqk``^JKU} z*wJ*u`SmwWX)bJ?0SNMe{cP;V3Y}sqdqh9lzWZ)YVZB`!{;QZ?(_SgW3F4z$>Y#c+ zXjr%OF*lj3Zqz`&peUedqt-rr!24lYJYi6HrUUj5pXzvs4GsLv^E=QfhYp@n=PF~I zoS)^Gqv5DdWSOTDZ+4%}C~PlU4ALTSF*t$jtNJoENIrP_?o>s1A0+&?&Q+VTJ<6Rc zF@!$46J(dVpe7Uk8FwnBY;GSL_j}N}Qkp;^$}=TlRdu0a5c-lDxsj+jotodcgi{h; z%tyS~r4YuUskY|gEYtdO183>G^KsRgn4dvhLT1fFh51XvW!>^`;o6O}p~5L%`7L3i z1ZL*BeiQp__ONDqxq#TJuPVUg@H$rkFYm21cUqeZf9!|yFApVktW}K~-37gD#$zwW z67qrGOJ0(3Y2lf6I1MB5HFg4;=$B4Gva*Ux0i`nz@%eG@#%@W-hiWWut?dqbapu}J zZdVsVYozAw*mHReyjD**x8}-#2-)sWv;KT~P5b>H>#?xqyfcv_yM7@?^QvqoRf^wf zP8k@?w)Tq8b8pA2u#&!WeIZyO6j5o2bOHpsw>R$uvbu@|9Z-i*GWWG%=BF9>Zz+lu zV^rfoNQ@qiU8w1!nhtTw0K=h}{3`YfHrLkDgq_Ss7pVdR5H?$637SrIV%~6Z zT%Wu$<*AyD64lU0Pq;wEMd0e$EE|%V)eG5FRV%EAGrGa~Mq6(ap5#F}g6ypvB6N`F z6E(0|biH>pBfn~VU3ts*UF+OWWzR+i^3F1536# z^LZNVq*MA9ke`ZaVgA0z(l8boFUCiON68<62Pr08!bvIam7$frpVZJ^ir6<1U*>3# zxWYy%u);OV%=2WFvL>JmN`2r=b*r70Wq_xQslCI`oA#Z!t>O-9%yvyfkeIx!`G+W@ z+G?}Z;EXwM&0Z;gQx_X+SXWFqSv@qSZH@c;&PuUX@k533>7m*m{;HZZsZqBp_gwnc zoEomxC0<=eXqCPNn(~MLeCuFZ5S<0tZAc-79EvEKo3z^yyJ`Ryv6E3^`n{{>mTD99 zs$*U}lt51owY5X`{)O!C$+Aj@2BwnnMR8H^jggauQkcg5#Rv60aWyc@Boy~TFVZY< zYz(6^;E(v+TD5~>*NpnYB_7uRq-7p-wV|$q2uPr?6@3jL-5zl-0di*U`GQJ^AUKqGdI-^J5L^S6A<~D6XYT>Vaj%^w3rgR1PfdC!t6W z-mv6_tvvPMBw#=u;4Z5=A~iYpFU#hh41)*B5Q@#6jjI!rptHm>b7i~zU|01GVLF_D zS~d^zofkgMP`ybI9L65XT8_y9qFiP|B}7fz=>sU-;5!rU!8>0k&!;6*YxXj zmu{Xe3DmSI!><1(Ia^qm>MnS0Gog3Q`6AsX=#nhYU>M$FO}e&ON^Z*gc{^V}Le|37 ztECq?S{zpctt9@8ue!AU_@@f$#Ob_0s_%&TLCRe_l&*eyA;xRw>5Fl%d(k3!u@>Bh z;gXvjQ5Jbo;4`A4ZLE)P)z~3v)Tv6sNxSMVt6X%q2H~e)mw}c)j-@%YWoatK>AGrY zx>yZ_;lJeQaFC}em-=5ld;CgOtZo6KU0IDv5KK$ll zt9rXf=suB@RgGbit6+T*$~)uQ{c%W7PI<$$7y$p6ik_rLuRP0{J!_4m;(y=4ks!5FkFMPp_4aXt4CoH6G@zS20nY z3lIPS4-yQS#q90p;e=svU3UvgB7M|=vXFp!5r1KjU_h6J4Pd~ccLzp=ESVP>QTh;S zq|nz&>i<(*;AM(NwG(Hd&elLjYenZ z^wFv0Co72D!l0{^SfV|VdRz|e064wZxJPZ=d&L44{fyK^H;Q$lMwR2+k624pwxqUAd5;v>oW%VzM8Q}fxr-!2q$|Twi#xx_`!(O*)A6e~L zRhM$qrtZte&e{1GpOne*wXeF%H^QsO=$N+Do_>Y@Xt(+aDY{eHlmlN2M^s*llM-Ro z0Cu@pZTf8+BH3Cv*y)h#uY}2jcV(}=vR=Ov$ge8O=jbS$whoQ=AD*`HOR5a8`<*w0 zI#O$x^S6ZeSr*o^>X!~XF%_0s9meqj=7}_7-|+G+el_D#ic+*43A7gWg3>Y)^;OXQ z&N*(!>;<17^RKQ6hds#O|2+qJs2nXhSbZ$nNP@wnKSL8m{;Gi z2}lNkYJx>rq`H+@6Q(D1TMeV&bYFZr^9ub3RIc6;3~j*khpfJJ z6&R{MSuV%38W!9%%5qYCHpFoJ*i9KPa(fzxH7Cwl#AUcNC@E^4dZJBI)t#2Q9~N<3 z%Ui!)U#g#1Xq24GhiV1Jiaa1*uPvKm9&e0MN>K#`* zo-mKU)#}s=jCw|i|JJaNM0E#3oB6ljcH?SP$6! zaZf9=a;>6u3s>MS_E`7v!}Tnx7+<>!iB_KyfKO_2erw7sX0oXNmnQA+xgrOx7<|WD z@HD12+e@M8Ia)tGIA&7>te;Q>_oA9*OK<1~!_DQo!yRG3QjjQ^QzHOkWE@%JUX#wW z#a7Q>T4G~66=#=(@DJayv5UXStHxjqF=+1izk##7DAOd$R65j5=I4)f>Rv0q-K$ZJ zc3V1B$uN}Y_OdrQH?+Lu^xfv=0MGA4uglj}it=$k+JqeeL}*hX`#d$K#HH%^cxMbQwUDQqq#e$h zO$02-a2{>$P8|gZhzW~T*?*VH{F8s`XQtA#FTQ%1&X~RQfkEQqHHLX3ho`C@1gHr#?$& zfi$Ymz8f#wa>{vJ#H8G4Uj2#2+KrYFZ~B~%&@onq&MsOF|074@3by_Fm&nkF@V^on z!oqqe8QzGv0V^)Hy{WFvssyyiNdP;Cs2Rz2P#;Unbe4tF5 zR+#~U@zsS*dz~Ircf&rc?2#!sb|km2!dDYhl=0rNkL=xi4Q$GHWhHY3-~}5}gGmdX z?GL|{EUA>hOo#C!!=Gga3uT+}3IM1&YXp9K6i_9!U4t;1ea*4v)f2it_)T$2v50%{ zZ`UWFYFtykPO-al#_&eJd}-F$baHuoT^*Q!_n6%(=xJNvm2h1SkHGGlfJ)Sjng`T< z18fDxs>dSD$Fw&oKMt%odO3?btyT;HBgjPjf=IhdKtGG!bO1IX9s@SHHEyCj0+EA# zct$(0$Q*&K9*Pod_k)?}mN#)CY?N$$RDy1E1)Mj&7N@wr zmHxGFukDG?jWFZwg0U;*Wxu4TMEKE^Gb6gB*zzv4|BSlPncOXWOk^AM!y3Zr%u!mb zxXDT5Jq2~(4%?<6wg+t3TvREyCr&W#X{K35b8qVvRZRumZHsbk=VZi7q^Qm--q8n| z1sp2%JoSE`a44^MNH3bJ&6fx>HKXS6esMhJQT(m2u-pXY^WlT$A=Mxq2a_v^I>e$T z*=9dVUvzq=qOG1>oy?_lUOU@~mQ;JQ!4#DIi^m}-()Q53FsD*@82uy z^A2u=Sd+Ud?MGZ@LPwsexN9)j<&qz-USx6;kX)u&`w#2c1m`v-^t($TsK}NAJ3O_J ziY#W`-4{ZnQY|ejhaY;IxwqJ)&@uO22NCg~lYCVpK|q^PKy-1^_yLD1c|qA*PFzS@ zapjDGwPl-0yN;T1!?~*ep7X*3}2H(4)<7nUa-mT`xy;Iq-AT%Hf#N8g&pow`0(qUpyi_x|I z7s-!=_a+iObrq;85_ao%af~ZC(&+=-6 zGn1L)-6fczdeOS@XC&?qt31-UOD-oh&5Ai3;xc{W8DxEhlbaX^PGx0Phi?@bIdmH~ zb#U@T1zYm4GwXzzr0ytMrPOHPw(dz%h!uak$<9^Qd)F1}^)1TF(^oyhMPp#JDyQ%V z*LP(P?Y7wqa{9OeD`wLCskGQCD101}ZJQ@K|MIF9dQR?dD`hIieJVD(U>|B@Pi>8a zR2##T?_a>1V*)lhben358=p}WU&)W^-7ODnmzU65B@&=`3!bWK2%}=uq}gi+2u3AT zouVz2g;S+@##*V#W%XGUVGQejy=W=HvB#~KtaQaU+vFqC@@I>Cm8{+QI0r;~PcNlW zepU@=C7jw*p6k=`)@&)YrX0S|fk% z#|d?8r9Pz&b1#4wKb}&s=h1FQ@0u#&ua7m{TYEv_^P$OZxLoAQsge#Sb{fL%@{r+N zm41S{9VhWrvAxsA)-2m?Y2{)QzvCN5M5N--5Dgy<01Zp=ga3ISOf_<(*?ZJ;ExcCU z$GGEAsp-qw8Pn~5*$12S+Z<}XaKQNTz9+M~OGF$@<-Dqn-WR)i7ZmYr=MSu}3-Any zFHgk#%YkF(C*G6O$GAD6e#2`c{CKrdX!zBo1T!VLNh9i);v&}Y&PO7?H_x@|;f!08DjfTzP806~ z)#0x)_Mk!OSv-4(bv)#lCW_&E!r2f~f7J>t{q0t_%ginLH}1hLCN&<@2U0jcp!2{? z{`68;s~?Us4|v-f?Aclw z9j{L4?>0T$_fy204}|VTw97h3xA{A)yq%93PMCN`!Sy4xeeAXK)FcuJ?5kCsTlG%?)@o^n$wi!TedkQDUEixefakXe@p%lNmTo5>cRc@|NXbD+>Tv# zXbi!XR8HA^(IoY6zVcQVb3|5rmSxmMe{%JXt|IYRk){+1?-bkBFQqTvP-fAmh`M3f z49So#sceaSpvD(2W|Z0aHF0TRc1Ugdb+BIDu|A{J=aUWz}$HN21 zRnj3ENXCsh+tRtM=RxX{bcwczHqnK%i8XWi+`_`+lh^hvjLfkZxtZ4B6lz{c`(S-D z7*Ig=hiuTGYOptU)Zm*kvm+iR*iL7Om&aIVG~^6Mf)C2ds!#fMEzfy{59nK2MaS|= zOoLC;2gf}js_g^}7F(RQ+=SfY4GeZ;3r_NQ63MJD=x(Ur4bn37fy(ujlOj?|QzV$? z-F08=t#BpxK{)C9-B_}REKa{0*U3y1`~H&n+Ke)8FBN`8V_z`hDSC~q37yO75nX7% z;+N&|%DGouU*JrxdZtVvMysf+Z`pv*hb`WDZ9-nj0~{6P(c2)Uq~lUTEvpAIgEc1Y z>G>jzBnx5o)`~|IMdDtvcqKl=!rzr^)cp0fX*g2}2xV6{PFK9x1$ph1cEWkvmu@2R zviVnqy7M+IoF@$V++1SoAN;*@iiL;cSr`wNxhcYkv?gAP^sYx%g>8@*z1kgrWz_I# zHGEW_#EBj_d-6!)`NW!aB=bbH&`!!p5`jYN1N8pZmKHzVJ$R$^csDeZ$<1plyrIxc z95(6Z@2O$4*+{D%ZThVcBV#Wb6Tg?Si*5T$U%KuPa^16+ia-+UMNEAR>?1NN)80ai z?s_ul%4MeLmB*NTc67FwTe8VBj9sLj7(?5A6YX62u@^2!*pF?5>5f4Cs_dUx6Tha4Y?Pa^V@DtG$ zS$=AH87K%U(~lc5+p z<7j?rg7w=Pf6Rv&rHPV>LlZ=YtCDtJlM^>Mqh;qc^$PgBdGYCD#i-i4x(oR^!aw(m zpe<@|b9RKg$m@PnciK;{K$G{$#2!Jk$P$Q8;SAY|FgzXD(_-E+(qy zl6*mDM@MuO3a#At9x1T_4my=}ho-^dcMS34Rl&JO+8y^~4Ho@CaJ1>#8(F;+wyPW! zA<-hrJiyfhPcI5v#*JwZ+KGK~HzWm~FrxL^03%lvq(JgN;s#M4L^tI6Zfu-q~qLeu}8qdB%!U5j<_wZ}FLDGJWZ1{Nc9i%`U$POx~#=^jF?VLGT2VccxqZ$~$ZR z$~#p>b1RO9@*MT=C&NGQ!IcjLTyD)xCx;KhdX{0EW7>@f)*tJyFm9R3qM2adJtnO@ zVEB0gy}=zbx8jA9S?lRR`t$9Hk$bi z4wZeHQ@OXmA;0m$_YO9`nz3lv@rp&|D^dO*J|%iq5Vo2_aj5Bs8??u9lDYe7Ea82j_Px13@*Bz-c|v0*RlSt)o9Ku=JZr zPhX7rGx(Q#wnZ(h&BQF1qIYiB@0nkA6t=2HY>+!V^3B>B;xoF)!#21~yt&*m9eE-! zaPxKXaX>EZ;LV2hkMn^=Y>;=sPFD-8bmn-!K_UR7mQ2J&FJ;}U&11-*qe!<%#j&Rk zBp9@h$)fisw* z|8#`VVD#{bY;2X&t73Y9&HG};9RKi`5X39kwB+ojtt2E5H5P5((dNqTXg(TEBX2(^ z0$;p#SOOmIyUnzU zr-m0UxHA%0`JuFH^Fldw1N#Z(JNWHl1q;B5$MYnFnZGu><8JWQN1(EuMU-*0rX(&e zTw%T{iIuH}=B{R5O}c}$#nPzAjPZAq5=&#>57Wyw<1JFZHp48pXeX%p9^^O@w#Ft* za&6wh$5@aa7~ECPtM+p@u*{sG8lmgt(L8H5V5@oRU}Lg;_2*{)U7w88*R+gh9 zw5IJMb)tPOJAYs_irCnGW@pDf`mgWhMbcog?U@;d8Afhj^@}Yl1g972zAG!_b-ab4 zbKg{dpR~<%`r(v8gL=xqKx;#`-AFO?Xk&!DddnZN#pDcjGie*$O(;}YsnZ;XR`wP* zgr0l}jl4NZUrw`rJSZ?rJSAv>DbM;><5yuU!|Z=4O%BdKO`3-`^zHQN%KAh~Zyk+# zCQ)mb0@d1qCiw-L*n#&$%S#UL39IB1Xc!`u2*KkO=OW5` zNbvSYgMi5x?I+!@*lV&U&!jS4gH=a`!4Pz>{*`cNsfw5u+_wBp#ztj&3I~3jV~^Y1 z*SPfFV0yt$0 z+;$^a3U6l&5W{cyR5WZ;7K(I9m=7Z$yR=a%)veJ^=#*W_)>PwGExy0b#zXA7%_PF7 z;oH+AJH|D#-h5 zP5uAbom1kG@y*AwrX>E2H{049%aN(siSCSmXQX=x6OD@@H96n5u7&*u?i0r%F9tRn zC{)N}4DcCrN|M;MJu<`LKLc*}uLwaqQr_{R?tQPm(sIsWooDj@R9xo{S+lRf>m-;iSm#>@$;$J%=4|I8hPd=qJvdc@^BhX(5t(#hpkxtc zfZGS9U8SP-EvR>%4dot1ytVtFL|;H#$xR`urKG0%!NVGZ>Pp|{(4p5cx32&n8bhY6 z38PPMK9T!BL=INBZs0MBy2$BG1tL8=OFSB^Q4-aRNX&QrRnI+;T1W^Ug zdD-9w_E_)Cv%{A4g!6Owp6Xbc9fzkp)jPZl@obvbbEV@-jnC1eCN0dC^`0u(@t~dI z4r>N>4;P0N=RHQ%Cqh|Xxx@-3hOHz99L7mQE|QGgjxPb8Xbj>)mAkoN;}bhLE)zz zOt01X{c@Hqt5sY;hY_Lw4|8uB5XZ8u4U-Tjgb;!RO>ig3;FjR7Gr(Yh!8N#Bgy3Xw z8(e2_w}B+M2X}XOmwc1#Wbb|M-S@ojJ!jwhz5S!LYOSj3>gv_iYdufPu)B4Y1J7LC zSkn-+F>oaGDwcN6(X|t7$K{*m)2Ur$6`ySE))yxUrIw2XiOPAWax<-kiZ5H&Go;(C zf+h|0TRYviK>`7m$O$7p#aBx?wzsSE*2V>S2#Sz|2h(}P@nvmSw8^IH9Njguc)8~} z((kH>&9++VD1jD*ECF&AAnqEy`8rqykgowVw#>zi4)w4tp<|^~x+Eas8|%T^MEXd^ zghNxu13_v;M+r;x2P|MYRhH`)o{1b1={^lFqf5(R{8t>FVDPAOP&Hye$0=2|T}!g2 z+477dJRV}dViw?N6IDRX&WiI&e7^y3ca(SE6CJmv{>p?fw3+ZMO?|C$Yu?`13;8Ou zmEia*4ELNz2X1lYGf-?$L@>s=Q)Pz#G_^67NhONlI2bz+1v`x`JUm6VhSW~kTnMMi zEH0U`oQPA6{3rmz-!4XG6vWPxUf*IK)D}{C7T%_Sk>-n_myQ=$E#G7K_<$IHwyBHA zuf~qrFP5ma$z5vrYjbxqZUs^PG9W;oUhg0(#T}^4un>E)IucND{qxL{?G~S`e(8~l z+%i4*jTAvx+e?G@tCRr;K<9aC+bso|MUEo|d1I#0M3a%zCodh|=REd1RP`L^zC(t- zWpoh8c2L5rCiEDT^cX7JX}>Ls3eQMQ-#8lnOTl zLUCTvhG4j+^;V5u7z!kWlE0rg^mJ*mMF)NOS}=GU?X`xPhRFM8o5f`BNnkW zQi^pWvpl(wun_78s#)SzIg|$GHQ;OuKlJ>~>{XSx0jT_ucs$~nf4^isRgf4E9dt4r z-m^|n6!H8_NHG^T!@XF6TF}w1O2t4trS}U=3UMGvy2ZAPRdzFGvAmq%1TGyAkSk74 z&wy5X!Mx-}?TR>*_?V~LxJ7NS+pqk2<^JN$NZf+bvl2no(>p=e$7YxC2SU7CUhv~E zZse9*7D21_4vNo?xuDtbd=X9V4${maa8&L|7gZ8C(u|VHZFj#&XLGPU*+Tb08RgK9 zvn+4|Pd~VoPU3!!QBdy}1Uly=rRoUcWHPdSEeOCUNF-ytyigt9#WtS>!GQ)eicTiG zolr!z&AgWK=`_vVGzIPDvNPR1qmuF`BRJ98G?XSfnG$Uu&`-m54RfwX!_@w?HbwP% z%f&ddoO7gwAP(?4~*y ze?rc>ijjB#+*AmM)*KUQn7E14;i76!m2-f&XM8 z{muz&sO3nI<|MIcmrlxbQz>CK6O74%$j~Ty#5~ej;qO0Z;BA4SHC&c&4(+lfPfgtl z2>5Ack)N&W=#_$K>hf9m_pBn#SUuc(7_^B}*iJ7>08v%)ZZgGRn+pnadmbT#YrTJF z`g=ClZVpzDU16!4Y%t;`yYslD8}xWyY>3mlT}OpL{CAgISOOPYLR~bQDp+$r@adP8 ziJe5B@`Gqq;w<5DU=|mg&wYj z$fvzkM@@*nKF8Xo7|X)z^yM|8w^}N4QHZiC_&FUK%MYrzebX@E$$7roS(_hPh%6k9 z0_Q{FSzw$4PxBlFhlIay-^E_Fi(o zp@t#;gc`=6&3?QW-TVzLIpebi7tk!FdqRid2V{g)$29%Mv|*@g4@9G+Tui_cil5W+ zeLA#vKfhJDOh>y=w6JVEbDY~p3X&YnmE`Xi-|`q~)YA`dkA(QZf<^f3MOh1;xMF7q z)-5weKUsPqKDn#?2r8XmV5Dr zxJ{^Yz!LdAHYy!i2?IJnAYEo}X5m;{iCt-Qu#G8(OFB@bab<{xqE<7$q(6kJCuV6e zW}kAPNVcr~s1+`=vMLDeQWaCKEcXvM$Y&IcLEt*(=8CQDrs%2qLqWcr)LrIE*jNdk z%KWe&I6Ir6c~*e`>@c2>Kl|1#0{ljuuoI3b;ialYaRMqqSwy8mU!qAicm<_n!W^uxZj*lFYeoW-0p z5b(e#Hrm;5VhA?Hyc^c;%LN(9ml(fMy*lD+Tw?KWy@~7JZR{);@%Co6Vc1kW>>RSE zjktY8YQe!S!q3`uKRAqs5M>-^=h>;{;o|cBR)cGp!xt|aS90xuo7+L}E>&0AN87iH zZ_vElIj3`DSZmQ_QHTsT24-*TDP?^9FKh9yUBCs^_E#l@)X#37+3PZaISn0hR4miP zKvDnHvF@*R5ooP#TEcWU!jE!%>>|E67k|*r-{hguxa&UZ1e};zF3|bR!pkYad=4kc zsU+DrkvBWa(tpU?E8qksHt=Fzd9;^`h=ztFlzBMHBXNTKk?`0w`5@L$^CNxv+Ji;iQa4vA0aVtnZP zE(xo{4O$Y3n5cpLH&kkM%CBNI2r=ZT1*_F#>*`d+m^gIwE&J3`Z<-$7H{fx1Bf}GC zALY{vCE4c3HXq8~GUt*`z?w4Jw;XuSV9XzGNFHrDHcVncMAKNo*V)%mx@xw%Yz)9` zwNiy2A^G_3=&W{(y=IaMoZ{GYl}2Grlp{m@&fB(CV>dq>Cr%7X0$9%JgmSD$lkj4~KOp`^4^-(ovRjB_y5_`j?%z2zg%9Qn#UU9G(o?Rry z*4?P^Xdj_|`_&&(|8eh*8lD{wKN;4(3!`P^n+bvK-gdu$oE|=cbcF0!;!?sFaN(hQ zKV)5s;v3rC?Ok zI|y`z_J~qc-~Pa5KIMoQEY7ICxT@yl%Zq%7;rx%~|F@5B`wp37OCJ(~wi_@5LqJ3R z-J^YQ$26?h|D|M7+42+=SrJxsEI6Xi$|~3xMP#;jv{~i*2j>zu;|u*eS#Lm6k8>qj zC~%2~0d3XvC~mtXVYD9ve_g!}Iy7E=X>87xDw%dTb#*KN@r5p9zR~Y|OO4tUqiAj3 zs9_#Iejw)lP5UIKhyrCHe96s(?Xam~?7SyU#J7q&3{2)y%E~0e#1cViga^R!9JB;) zv^z_wLt|>Pjrkph$v)#&Q0pG{I%+NlUJmBECUdjsO$cHnfPRRVWJy%a3v`}Jp8Y5% zGvJOvoLU-+2<$tb9f}7`x?g-ZP--oABgU0gGr&+HmnGIOIF{gM)bA~Vj-I5QFL4@i z3D+ZB)IJNM38W=l0FInx`@PiC3dn{TW9dJlKTOgrE>K3Bbh=kp#rg8Ku|9oHNSKcC z;k8LZ=OY2)@uy_CK%%OzTZX&uDiI-#YvZ~1-+Sp@Er&%^m@zv`v^LY8bQBv`}{tFzBkMq zwwPpz*iOy(vLIn$SAid}@PpE4@3_6@JEM=rc7S-gXH}ycUE-T??Or%Ntim_ZUm5&X zxl14Q(}FRKrD!C~#FB)v`1o>Zt7acStZ>6IUkvi31UwlB^K*mP+NSvC1`osNx1i&?)`~&w6pbbhoutU0O^&4DzYTd{)z7uu?YB5S%$UC~(9Mrsm3@?&bsRsM zS)>rq--5%rYFlEZUM?RVTStLI)fTB{s45a}nOGhJBLQ1#Fc6v@%;oS=n5Wtu0e533 z%DrpkT;|0hId`9l58`IzU@y^`>%!QVUvAEAey*;g0VsCq0Z8KV#Sa38Y4qa|&kX0v zF1nyWg)IzTcV+g8^rB3Fv5C!+V+&XN%eub@VY)$J)>exozed^r5v%L>zV|QeTxV~z zWOq_snaIaW_}V0p_!hbcPs2%zEoMF^54%tDgf)f{_#|OiFpu_`Tl`3Vu-~96dgV_& z4|1c}$-+|fLtfUPMmgLt@qCnU z)FY2?Z+mS54eh2wqZO4%<|ASqwg*=|nxkc=CBwFe8nUMZQ)I8_Eadzh@Y0g%9dMCT zr#dwm2D&&HTK_`*{%MwbGX8aI#Eh5B*6Ck1{~{%(?GSjI31QpQPbkxFqZy^X&JW8-?_jI}`1b7qMqaZ#cW`=60lZzi(G9U9CTkYvrvuoV`$>vZ}n$O>$hK==r9_~i!%l{O6;>i0$xX5zO_Mps^WaU zv5TO7AI7R;Xg(Gq_6-e3n7bx(Z4R*ISjUzs0faW_L5vNg=mCOpK1BjOg!NmQaY?uN z<42n5?RD=z!Y{mDGIs^n=1bkAGSsi~+DfH3O4gaqv7?#3d~tT)OBS;l4ehZm35REVpr~Q-j=ouwOZjZNF>K4k=VBvUn|lLID4@sOF5 zl;BUjys1lL`>wS^5lI}l5m8(MuL`Cz)OSee48|Vf89$-i3V|cY$r;Uj+3%-nsIx^H z+Lc4IUzNQp61okETtRg9r2A%$lMNgtp}?*OClb{s>&=?BUsm<&dahnrpWV|?Su`Iu z+vj=UIUBn6Ti|*Eaxec!;QGtGv1abx*0ykKH`~gJ$Tq{wm9-`s-2>=@rBDo0b+Cdk zrGjC{N~n`nQw@;@24ORoQL2TXo~fa_P2?2&!XxXd0G-H>>&p`4$^A+X+h-}m4C4T6 zSTPW1fhs$dG5Vuc5*8Ul`b`;CpvY09ti#U?9uTstJkV4SyRN^-#wQ@&hWKAS< zp)H@Xf!evO!jAqkx1FB^$iZ9EhlM{l*K7Yv&h@`|!2kYJ1TdLwn={BwJ5~=W2iHk5 z6dX0rH?(E}3OOi{--^w?_~CR*f6a9sijo_4tRw>q7QqtKGj_kqal0d8O)CeWsmNwR zrH?|I^;@b!j0N~sk7dDTmVn15oaSTF12sLG<^3k`Q1x%jnYOcg8R3)t_24K6I7{BRq%L3gzQ$e8*GA6%+2MD z(j#-h@a6ARtq=Fh%Zhc3T#foxM8zOg$^$tOb5SY=8fL2jsLA6a!SOLk=M3>gfeXGY z`CB44c(OH!EjC2OOux2du+KD*r41csKGw{Pu21@4XK-!M%o_#q(zPMRKTKT4%r<9z zGmooR=zQ!GEeTBovLS?`h}miYVS31%<^Q$H5vY*Hgf3n$|e ze-vG(l%=|ksK@KH=j&_Uy;xZ*@^{0w_~dzD%E`Uf8gg7a?kYjC&?Nbl+9BmMpb`_s zBZBEYE_C0-dC$l;1HnauR}~+sLl+h*2h3eA7i}!i=+_gp8u{p$85H!bjq_SA3S7zU z8BZ1zDdM67f&JL=4aNsY5*Suf%;OAjM9F72Mmcz3iW4i4+ySeh-@4;87+A> z$BT%czie_ZMxuCx69M!A3EJoR>vuCbqBL3Vvy2{rgWpi;`smz_Cuz~6nILWwARPa0V1}XkM?syla3k_Px6bQ!2!J*dOOR%{`B>p1&_YqozfVwN z6NNHGU0|xl|gBT_7hFOP-Uze3R|7yY^LZq%nuH{0GkE zHnNhJ1krG0ODgBy#_hcPiMawD5ATy=7JZU?H6Nn|>r~=#(OL(!N9t*}>AcWje9 zXL5(}CS4o5k>GO9$u!RCs47Q9x6DxONbJ5xr}>Fj+Eh(F7Ebi6U1u2y*IQJC9!JV! zUA)CH_TgS-ATx7O^ICVPvi#6?k(a`1Y46 zB`?3}oq-(!BtG*SvQL6erDWc3XRT@LzgN-uzRuh27g#f-ngqEX5o2bX&VOpc#W=e3dAcBh29|-4oLofriCTKRWRL;wrClaZ5A1el6)Z zrmZgYC?ku1P-X4o{g1zjaJy~Bw~?)ZUp9<~AkU2y^%l%uQ&m+2EYtaR?}Rf5TBARX z=IGqB_97cAt1tg@**|(Ap3IK|rzH+WW&;&JGGvp8*{zPD^G}TU9QX<~7xWK|sdu7VqD?+-m@@1x-`=(T{Cy^sEI6MC5Gh7_Ru=Uw_0y zthv^j3FWCv&?zn+C!QMj#R@JGI@1t#R?Hyck)i(Qo%NtqBS5#-6_iAe6Mi zBG_~e=G2HCGi6^6XKCxudogK>D!bBv*AN5K66Gb-2W=~~_wr^tVO*(au?Ir)VqV3O zyNnwfakafNvZ_FR%<87*Raf{8tyqzYUwlxuuUdpEOoCjY?dyPorwecVO3W3ODVHho ziAo(|j_cX+WQNY!pm(gho!?ULO*l5> zgl>%K3iJpvCKo^4bjL#PXt}f$#KF=<7Y7>V2I3mKIu=)Z6gSQqMdf51fY_)-{f!3h zmzZ90$&g!LDe~+ZUsK>pE-C6E0!fz{l%Zc(gxye!JDq4d{)VRL`IfBz`tbY`k@6A; zjQjbZaq&QHXvvK``L1OTBVMOrxS=;}FkuXyVVI2Svh$l1w=ZYTD8l*`Rb-oCuz;rG zv;JahFC%&e%;4F@HVYFH7KSMV`TSEc;K`Fh>^eTT=w(lB-2%PaTSiA^u>AlJb^Jsx z*%>>)rqdO3`ko*nMasZG-xc7bDr%d#TAY(9&_lWswC_G+eJyR@E0@r#9pv$x%X=_n zWfF&2Mn$#%gOw8{e!0R+VP0c&Z6ddLXKbyem>+tz{NCk`6Y-911uMV;BxE zdfXJjHwP=0%sq}aYxNKM|ULH7By*kOj5Kje>tPshultrv-e?!wJARfxHVrH5`bnOy)UFYGt z?4Dkmj6O)13M;EOq;bJjP&y z9_cY|a~uN!KzW+Z_&7}+RNS6VV-0p`q;~PjZ`L9{JRI1xGaT^>x!H#n0C_#=Pi@3K|F4AMRoqTyZ|Z^}Jzn(bYz_voXD<`3;R?26TRP zTzqxzHM+X!j+NN{h3&OVN$PN=#rI`*9W6-Xv)*4G&7*Hp9v=tL~<&$I0f*q z?l=hL6s;6`8ZAVhcv>GaeM5upFnD;mje2?1Z#awOT*`%tz5POhx?9j+$sB(rbN`0s zHd_B9UDo$>J`*B{3f1pYJ14@vp<#7hWz_Ep-n35IcH`QvxQq$p7jC6TIh)4Q9L4o_ z$DOlRmNwy0hGtA{eo@U*0D`+?icpX}OuFPwx@_$| zjSmx=9jzazxRUskcX+vNczN^spc{jA}o?XZ048!}(Zvr&- zoaUya5~oztzkK>i%-?LX#^S^~+NQo<<#1^Yoi?~T^7mIWzbx@1ii$JQzZ0acTu$#o zW=MLOR1#U&SS0)(l;PgSN6~7DP{Ybze3Wimg3< zXra4TrUr9j&3wi5kug0Y2~)CKI~{>7GNkRpK>tqFjnOVKVb<#LF{7+~v7Q_oghywR z*Wv9I*30LdcB*T&oDq#u5$&CM{)i4@%XRmHT~luru^O`UbxZrhp;j0xQ!{dvWic2MwA0j6{sXth6ox!izkTiGOggLRWI zYdU8i&i~jf$up}v`=Y7BW%p`-c530kG4%88H^C~~1E+((WO(Y2*PeVsbJ^Lpb>)+o zdHY#B=XlM1E%c`lAy&&NcdyzLrz!!Gw4E0-X{zUkf({Ztg^Ajy^jDjw_MuhRaKvBXr<6MCHV;Fu<@)bq}-(*BaMk2ks8 za^Ca7Exh;NtnmXmRI93yGpuHPDW-EU?PPx)V^;b|$05|g?VQUl+JQG7;b=5!r%SC- zpcapf#nPUrvfATSuAqB z6k4UeFx7F_Gb(m(RSSLU&@L$mUSONQ1}}R0Cr6*_p+j-oOmQ7T;wHhFb(f>AXI?Zw zScvFrYy>~wKmzl&wn&Dh4AerA+;9~s6lLa!^n1Ub4oJiY&8D?Mj3)W0-+M*8|Ayv$ zfX@B**X_B69PmTK((4gK0hDoJY6)n@3dGf{FpK^Ao33m5)nie1iZ-74iHICqf>!*8 zVcR+qsaBK2MYzdE{P)aQ+8v$x!jIEFWu{(a8GFzPxuXBE`^WXDC98}wD<1e95ZilD4`@!OopVZD|K7cDgtx2}KTPs|nLm7u3xvp%gtJy^j?0sYE zK}#uR9v~EGx7QxF#Z)WQ>GeaMTW%tO2W=gGd*^CFONWd7O?&oRYQ&nyd_7lIWchj@ z`$vNu@jr0i_kG!*SMY!|wQBt|ow*6eGOmc&nDPAC`(@eiwPbZhQq6!-lMvJdx$ls1U#@NDk|S7f(#^vw16cb;zP?|!vj24)E!8n z1^c+A6?ZNx)UM(mr&~DVk7jCrJrmi|$#~0i!%eL%v+n|@`R4GaNp?;KX(1LHor--n zr_KT|%^{W*{6}zdt7K276-~txh0pQ@qQR$`7WK>`2f~;H^LUZ<=?>$2gW$Y8+a6C~ zbYok^vKwY+Z*>+?&cPvL9#@4aL_Ur~dx%G6lvVhH4MJ(4VTHI<0s6>%H&6Y^ylZ{V z5{9=cz5A`Wcq&KSkSQhg1O21oTW)9!iL012h7~Jj>J(G>Un29o^ft9e<{UmhpqjU> zq<8gtW(qYhZT|QrRJ+<)bF}?tXk+IU1y-7!zw3gHU>C;27lZ``VNwuKDgaY&%eDiu zvgTgPxhZyAA@nei0_q2E@rl}sD4`0B14|l7(bnZ|z2DFtDVRWV>mlqnFYmrrFBtMS zRSLjh(+XU-Qr?7fkF}BeM4v9dc+B^WqD4Nc&YIw zkS3@_%(OgDTaD3X*v?=fI8TkS*a9ibE^KCOZ}buDLQj~Q#%XZ=!c?V+9mkO$6={?x zL0g41k9LxBB0tExA9$%s>$iCQ^6wuF{vL@00yddxgJ9if?>taZR@E%?(1f+A5b?Ir zYR|iIO?_4@*rCZ`_ONvjE%v!(S1*95?vw5Bny-nN!Hi=EM_ufF{k%Shb5|Mx*e>!=2jj-*xPbHZ>`G{A2_XyXOJBVbjaX>9O}cXCxn$tHY3Nt544`2B)u!1EdtNtuFoBth zTz}m(26n){o8D+aDA`qpeN1hq$4;7{WLX{K5obAVsxR%_cW+V@iWq>?91N56^Lr5Z z-Amz?5c}IC>Q+rWJBrYSr6sWWBZgDtVH8)Sh1QM!kBc}Aq^Sd@E>s$PtBg&3`xjsF zx`}WE*`7e6Sv}9HTC>m1A5|q?x0emtS(~Kt8?20MHY7ZJn;D|2K8U?(hu;Xl1*V-3 z;FKW3&V?xDnNg#{^CkVn%SZz~lQm`IRhH?8VA;ci2ujwL@^O0dTt68Q#n$KXUp=x9~s zscR&|z;c#t4RrxKZkXu7WIhUI1k<(S81Ra5+`e*RGERwp`)eHsjPORq-A|VQ{}ia% zXyVk%E;gruAypv`Z*|@@k=03scd}wUfVOC$DFPu%$FRahigMH36L&>#tL{K01i4(qDVE&WvX|34GTA;Q_s+tpcTf8W0U8 z8JqO|Fq@yy%PKR4={}Spmr=#pL7p{JVb$z2oa<~T0;11UQCu2&5S?x)x%mELk*7)1 z>ejv4gMqy+-+^l1v_(-V9%>&Hb`~MyQ@Jw1@Iw&|bD5;}{`?gXoqje7%3R3}xOOs; zVHv#T-WOBzRVO8(=d%&+6psc!ZN2KvrkACqt71AbnZaY2XYK5 z3ZF5kYT{}zT|0GsRu)|hO*fb<@NkF%3oat1H~jH;BKfp0*KVOc)c)UGUI8-Gt-|_F ziPzVd5)W7E{U!FSf6%e(eqlSAh@Rw^=iQU&&43pvwAEc8DU`;oZYTe$2^p?ZHXQEf z7Y~oX%NVSxBR5>`i{k`_k~WRgMPGfO2@bkG-4`c=kzT|KvH7@H#=8*dkx;OUTthLr z(OKFxY?QMwR=1#H4spVbwyLD+zKq|U@|k-;1BQUqh5Tx61KSmjZ1^^On13n!+qO1E z;t~5V_y_gE)YZO|yZ3}Hpt#+=f0KWcOz5n!fBz2t&tp_ndY5_J_sNvZ_Q0WlfLr(j z%zk@nm}e0}2CnaWNu_7&l|O8FA7Z5PM>cq7V2!eWH>22S)66L#N~jS6JI~C4oSa7g zDVgwZ3jT48t`J=<0eTUYl-ZyJ&Y@2Qsk%`aH={7<&{Y)IJyh_jdj_Nj8AOZb=sia< z4*oIk|3gOwm0i+IQPs+R{Cs&K)YOq?$YUqK&RST z4i&gSe&33%TI_}gV(gS`-fg%UeoGiN5As&`_0pL^IA~?{QKlp2G$YfRVdHg~hJ5n* zLl3SmC+GbPYVbw;L!JPbbRG>V!{588ObE9dtnA|W9-1vMHr$M7r;169>@pZ<;7#!C@P`lhGvFh2bsb)#dLij9wM>7gEs5l^thSzn z!>5J|G4B7(Ep&rj#YAlxARp1icJ?yHCGqe*WJ$wi@NUdcJ^#t}`kVFt?})o;*}`Hw ziSmY2{f1WCEO+2l#>BG(Pg3%nrBt)qwz9*U}d9rXpdn@zY(IXcB^ zFbRhDh)52`pRCqURxicl1~$#Vu3Q_S1sUKb8#CGzK6v$A#!cf!c|vMe^%7^E^9DuL z(XZ;pc~UNJ1UtlhbGnnORG*ihZexn5IHFj7+QasaknexB@Q(ug4J&L*v3!PCWu)<6 z+T^9kq9Zp9vKK@{p^6dSzX`u5=ym9ywe;|>(eQ4EKTG&DrzE&)n4`Lzs=!BnCjH?9 zS#GJv=p(|oP<#%=NHaKC>(*a*Vf_t7{|6BS)*vt^epQ&6yNJ%9Dk!g2Ny+1B+_V^T zYg`)E*@BSC{SK{R6g4Nxd#G2XNb%a9%S_(=k{)*Q0BVLLiiu!zLYP@Ta6^Q+3zDw8 z;Xq2~$zbD=j5T;BmDr=CA3k4!KgOoItz54WWeWCf#F~y$PYh!nXYMG8M|qBJUSDod%v5RE%{QF2A(5E2=h)VmT)DAh|4Z{pv>Ug9`Ru*SHW@R)S>1 zPv5J0qhIU>n;b(gw7JP{0x3CE#sySl1?wse4iMbaFze;z&P^K(aq?lDyKvp&m+o7aP|(x1og~NRNHER#>9rQX4o`est;*yaFBUq{Z|y#f?v}t z>sJ)lLA*{n{{*xQsKhgouQRAi=(G0kwE?yj`5Irfir8Wa2f5D?gFSI@EUr3|vNPC8 z93e$|?_iXXl=`5xF|3Q08k#F6o|dl9qr~+y&S0g>s0i$ZZ%hF#9$|PnIdypHmk3Y< zY?B0b?K;unsW7)?uS)lAD@Q{%;a+k}O=<}6cXtpBa$jl+ieSIs-@eOlfBjj5rxPzx z$olI2At0%#ShGp{ZFYM*Tz>iM?8)4LP}#UuCOtCO(t`aUb z@Yl(I`bNGcw|fqfd)vE_0v0nk)xaYQz@y&7t+$iEhnvI$i;Sf+&u|7-2!zV}7*Ee0g9H5ai})dr>OPd-$NDpU_HNBD$_DvQr{8 zmkn7%4|k&mDDJLak((eTTuk)YHcIr!HF!Bf#Lt>9%99C@UX7i%{8JC&Ks``)9F|kPXV{)W{*@j=EBImN~+9e<-6$K3;Rvv%7 zjM|oUV^Ph$R0KU@&dRUJfB~}HApxk$GLU+@`Mp>W*iiofgu8$Lw&up>@G@o=(oJe_}*mRIMu4J|G?r z5HQCEN?zX$^!IH|&=f}HzaXbXWMN{Wm@u?rBDk~bQl^H>H zu=|OeQeek|uh(_obnCM_4;ivnJQ0vU^iq$iCl)#C8u^)HJ#lUjU!*%&N+=F%VF8Zk zZrHioh~KF3#lSAtcK};}%3nXy5eQXl>^nyoVE|thq4(~}5g7ze#+R*D zv#9_29Q@T`{-*8qgjq|%*gGfD@)Jx^cMk`?lK<~!1*G2;W^nVJ0e zBBBS^*slPi25Yci+>l!e(;yCls7wP@B3))mZm4`_pb8MDNP~t*0J?NaRaG~|P=xB( zi>y|y`F!<=<2kXuJS!-YUShN}Z*%0q&c9aWhxv?pwy~4f;+r81j7Lk`o*|T~Ru|z# z>qB$EOt^xAydHfI8Rggc^%e$!wC z?Kj6CD>V>N1P4P12Fy`m zU*x(bWy3Wo@;)h}>mWuu`L{&>^gVnl%}GxB8(K}ZE7l5IQ0T#w`efGzgadc^WHLH_ zE$#X~ngn3sF7x|A3#U0Z^OZ)pN!DFzcYewK2gYXO3kM^4id+~Wk*|_`H7eQ&8B)O8 z!R{7LvO|%_c4Sj>5@!ZN!kGST{cpON4{(~pB$!+Xj#$79S}0?V1H~gdd0iLKb1-si zdTqjr*Zr2QOOEiV3JJuNj7V!?3;^J&`U2x~&0^~Sq{ik1O+FH*=*7=4A##BBxq4AZ z9b_dB4ZHR3iDAIBDOJTpB@U#g4AwVPL;#MDFxi`Uc1PfAC+h}g)us2Y23^@ijn;1U z_01&WQ=jg+)@j&kn49 zH2>~85pR-&cr=yTL<)xm8Qt`Q$Uu9fz(b`oV-HbE)o0_tHa6h$6GL3=ZdfsMY=>(j}gaLnr`*v^nZlp$+Wb_fSe_Bo&7vbtKDtGvN= zh&@&x3A)}h`+x#l2=id0WFC%OzaNj>Wd(W#+ZdPSLB9W<|Lz}yn#J}0FYf-odHQp6Ihjm(hd*X#jEK53ZjL;+{O|l`h*u;e2g~ ztqTi@?8Xvm$a-iFWLlKvijI=Ix#8yX6PApXC+`~;z!k|So*r}d3#AtHE5H6;L{hCO zETZ@Z?PdeLRR)dFDDfkfx`3p6ba^oeb#8B6mh|kbqo!gj=<{T}27F4&a)&E)h;Ww- z3fmjqZCRDN>`>!uYV!5mIb$4#Ro;yjNKDjzR453$m!k_BH02EsZN;M5f`i#h0~0y1Zu=Hf1!Un zZ7z50G_baKF-#-Ns|0Bcy?cSX$ zYcc&2mx)_OokJRe)Dm?y6rk^TkZv5HoTop9JVv?2t>Y+kKK)aH{*Vv&P;HG(-;`zB znplJYmB;^|E24O~04fe-(s*rPAY{qQGPVFvFjR=yz+vY%wH1EaLvmB_rX#sL?XNqNfXlYGYr*{euyysTTu&V%74si1qlnlJw*~y!$^o$P%6W+1N z+Egg7jjOzP`&Ei>R-X}G^%$bDD(PD5rEW%gfENRmRSb?(f(HRm4z~I^bMX0$6 zQWB}RjVNrB1ReQexf@;U3Pfs+M+N?^w<`46pKM_+$G`6q)O0l(fR|Z1&z^XWV=(yD zf>XRxz~#8T(;d3O2x;#6AW2>Y4xpH93Gco=4^t|586unk?59j%EIxfiT8Nll}?I~JU^8+quA z_3a-@!?vxb7J!rCrI?E$l8eD%;z!6(GoF02t$@Ih>}<3lW}$WG?c@BAM*0!he$ntF z7TvvPjEN6-PVF79ty|)3MJ}3edga0*~=QVVv5<3=1vL(43ASq>RYus1+N6T z)nf`5abM%NeLWV40$&Iij1uZsdl<6X2y8UXat<4q(I6(QSNLpo4+!&XC5KS~wV?@N zl|?qA_gwDb7673JZyHT0Sr$S2tdk}+JlMiNXLtTXkNIyL3T!}tJ+^@CcsP8+?;0*H zwvrRd@cp?kLeu&HZZ45r;@iW8zfOMN=2Qw?Ec@y16|dm6+JTo4T7KC%#i@s>sXi$e z`m^H)jxJ2f`kBn@gNKoP&%DeUb*eG^+7R4MNhz)d=LiWLL#pHDz*JQ!;lh&_Tb7w# zV!D~>Gxw=$jzZxV9ZqXn<{#{C?<3Vt;YnpPmWn8abXJ^Z8@RF*2R+UuR1tN4AAy_d zKsT!Mn=~1;TC;+A&Dw*N;=IUKZ1VE>wyAtu4-Rw(28}$cQciQ(KhR66xWvRnR{mU( z|Hl6Patfl@mY&eoLBoI&+XJXj(L{&@%?bUz!5IXY@GI++$E+ylAif_S0l$e&J5NCt znAQu49FX3e`CZfXu=&vaylcxRfu+Bn`&)M44oB#$ALp>JSo+f#%W^a#hcc`UB;j9% zeoFu6Z{r(IsE30Ku~J?R%3@CwTd`nup$=TAyw^h15JGJV%Clt8MWD=UTw84B&NJiu zrI>jg9jdp_a#__+$OaBmaE26iZCQoGS3Gf6JxH(4-iKCv)EO^1L<+uiK+yy>D zDCjg=hS5RnZCO#Db^V570CTE|#RNpOCa;}h!wvrP#^PLP7k=Z)r*aP6drv>rS%fH1 zF>A7&0R62TfRa#^Vh-CJ?N!ocJe5QSm(KAaapp}Fa--=!{8l4mSCv(jJVl;VdP9Oj zeVON~%Sn514Gzz&F{nwvWjD9Gn>`ePL5~blA2fVB1BuJNIiJoE!Y(~G1mlunSa5AK z>g=d3X**F+k*my7CQ2}E1x9;@xDAkdHJ+pib1Fwg%|*rMM?9~N#&>*YT4vwYL?eg8 zFFQriq4u83)@8*mZk%l*FUE0&uCZIX=$g&Lsb@R;Px5v;v-lA=V~7h7hByxT1!nZl zW64S}@@VE@JHgxQk8@sZO{r#{R(O4V`Duus@a?m7Rzf!gUL^PWwA6Bx$Kal8lRgrN zEI=Pmb7sjZEf(XbV45l(C8FMPZMrVm-9jl#?Yb@o0hHf8>97@~l6oXbqiBg0BlsYCw*ufRdFglHSe3`B*#8(3Iam;v(4D8T;3-+#RROFfT!tbp>+*q85- z@NsjVMQ<8#7oJsY%|>sPbnhC1AJn-k^KvqC1>k;UqA6joELeNzNC1}W*-2ST+Geh; z7g`q8w$30$VxDsk7(jrN6p6jw4gbU(Eb*pm!MtJ%lvWZ53v%or{{N_Z>$tYIZGRL> zffgwRN^xk>;za_*D>wv-BzR~EQWB)N7OCJ;ptu$%!68s2P>OqzU`2|%yY82E@1DKy zedq0W&pp4-`MkIO$XdyonKIUxV~siH7~i3(N{{grA5cIf>L<-cP~T60kJgU+E9fDG z+F01`U-*iV$(uTU)y($c$=@(DPfapfj(q_2D~h(Xol;;8GQ#3CbA(09a7u^6lzOV; z5lO7k2~=Va0ck{%SJok)^} zwQiB02s+svF3iX#?S-(6234h7}sv%tdtdrziwQTj9Y#%^-8$AZ_xH z);xT6FgV?l z;XZBXKjyFyOdMbDXIl?`k^8y~!Ft9J-2U~N3F)2@cqFuo*I*7!5!~1ytz^0B4j`9(yL##(zC#csGVkEHgZ1 zB`T*YPWAL>cA2{dOVovhUn|xLyh%Jw^Nwt$&i9mAY!O3|x)yf}l5x@4?L`X9a)+V8 z?sK!VE*{Jlq!Rb9`xm%c+wPjy_QZDFo7l{*f{Ep7P?S2sBFmmnNJ47q<%4+*)5G(7 z-p=z}tE8m)`~#k`^iSE|S8I=U-6{m13c&o(=YKBJ2q=xu6fxfBwFmrwrs_8*Q^<3S z#`v~q9(MN{8RVrFpz^reR#c=ar801Q8z6ov}Zq+9xPc!1g#T!-~|~O1;z{0*cAw`d-?}Oz>On8V-UNKuGv$TEIrh zIMFctTQOFF7aPFLgy5_eTWP9dIW5LAlj~W!YS@=v*B<7f1CXpr?Nrd6*|jH#~tSr_AZwf7wC1702L{@`6mZ z`Im9!f7z@5{x`xXuU3|nah;8+*R&heS``8wtP46D3b~;NzP}g!Tcz?>Z6X?hhw18~ zMrm2<;k+qKuHIIW8Gh(LYWP*?Upn*D3tKhRo)X?`6?R|AFnTi=fReuAw`n_MrJ-q7 z64hW~;>RAcSAzRVA@xZ4J2XFUM?dfo8v3d8gMMogxhYvT6*N3EU(Eh4xZ}rcfoPJYYMg<$y9?5~aJe@MH;`1w{ zZFajBtdb>>eLYGn^V{V;oZXAb#unL!=cwBn^&OPCrm{XEjv8m#uc~K%<|-lmGC1va z9}EB4xPNmWhud4$GjU#4x$wYOm$7TAN%!2z=Kz?7%Z3_1o{8Ma%$}ratsc9xUh z5KM&PngGA&&wh*Q?hz2>aYPW@uJ0fMv9Y@02@PapLGm zSt)WjNHx|j3Jc{=~vhdI2nX?GMj}1Z~=3&c9@|okQCu5AcvtMP#k~M78TwycM zUx&mZH(Js}W8S|+9kgtzjehNRgh|_}UOK!_B9xpa^({Nn$XN#}4N{S!n*N&Cmqf1YXwd>Mlr=KiMRrQT&+$)U z{YzINJIp``MnahHT_!L2X!%zM8FL*|Cr2UqNrPa^l8VLF@1-K_j<;2Y()|?$K?XnY zvcA2Pe)yve{Qs=v!DnxtcD`}jJ5Frixr^_Q4q%1+-?|R`UIur*?>SR)Cxx>!Qd&P}LrjHo2nvyhS^)Nije+Dp%KVvv(40!ox$4Ch1CPA~kIF=S(@j zi^}U*CH!97KR&OZs#`A^wW=+$PL!iwuzP87daBflhm=JbGZ#6v7O5irD^7+HZG$`+ zUY<@#yJsRAZZ|gciz=qxn~NQPt#O6q~5h{6pR?o6Y-&m)U6XTkXM&@zLP1#4?PHj7o@ zZnX-Hmw8>v5t4g2B2||a^o_IG7d*xE$^3ZS4r9T)&ID&nZPDq3F6eWQ!fH5?py0gV z!u6+jCEkQ2(yK3Qn3n0&npP2VDmA}W%y2AaN_i3B9W!eihM^^dmQrUWetBGmG$tNR z4%1N|xkHPV&wn{?LRZaDu^C*WGK$%Nrsgdja`fD(|JCA^fqYYX(6e&ls_+-zSgCEQ zu&G*kJSS{sCW(jhM?2NVkln}GFmQ~EC$;&NM_C7pu^9AmJUk)+mIW6R(TK2(euYk0 z!AN{le{l>`O{_RJH1D;BjZSl{_PxIs{$@-1K?x%A< za_0_}Wk3GL+?hs&kldT0TU7g^NnIMD zoo4Le6Zcv?DXudGM!=*c#(h0RswgWUIUW3xl-|nL`|8o=@750rkB#1jJYAKNiEpXp zj7{kWTZfre(-g1ySZY7?b@P~so3m;F#aF-#Ur$O5r@L(TQL=}*jDd>P&92n6?-U1o zng|=eUFJ~fQC@%DAZEchjRL9DXGQ9sje>6qliKXnl7_+??HR0HpwjI7hQzt-EL14+Q zLHmTAXMNN|h=Z3x%$d_&Ekz0}x^QL(1{YFhCoBgqD4@iQBXJ7Oe>JA>Msg_pbe4O; zzN{d|a%p)AOWs{^J+%7q(`)`8XIldX6Rb+z)!s%L1kn`39HqWM2HXhq!wXMT2ht)hEA3FCH?u^UwH)~k>Y&W4eEd?}yeI*cQVh2*cT_plCI z7Ke4tKLvc#85~Nc#k|ybix%DiA9-o2)V5xrleOqg-VCW>YSc+`mB4OS1tM5;a z<34{98NOmY#lvaZ1xN7Z<&K$@kJ!2()uu@RQ#HLSK~9T;w^g@Q0uz)^=a8Ujb&bz> zg7KRUsxiF!#+nYp`pZT7xiB6!ce2e_m;@DxL}LL9-9k7?jA_tX!xO1HHWv|#dtL$F zO?yXg^+`4uwZ$f+d|u64e&hv=qcvNL!CSlyqt9+%8dk4?7AdU56?I!IxmaxHqd1Y$ zLfL~7Nbt)B9c7gPwb6_OdY3nGF47HeKZ~tquO8(OR~g88iQY9Um6o#hunqoiDhCNc zwH%NXf$^c3JdLAUm}t(6MVTxjyhr>{?;Kmu?hymyru$sxAyL(MGsr9aTk1|@=EsRw zi=SXrBm{t;^UKUmP2~A!QtO<{5 zqO(0d6gbjWF24?DnHw;SUoCGBTYzT=73-)^Ryp7<1`xj~6t;Lnslr8vO%LW5=HR6g z=&|^ko?M12!z{8|)Da7ngz&nT?*2+`rJ>nv3zLZrlV5pFbTDJ~^l^J8%(_)DZ#cI( zkPukMf1|rEv6uV9proy32Oh)1m?#kQbisr&;G=-5Hoew$(J_--yr%2*eB-_@ z%`t|KkodR`G=e(ZIk5CJER)fTKae|$Gzw_HPs6=i_Cinu8W~axX{Q^+keJUt&x*=dBxg}9l)MA(QUswNl85~brGmI#EegwTj=IzGa?%t{B>gg6Q0SZow|KZ5xSF~- z@4s@~YOw)YPoLBc8qBfPiDitTN!L}mE0oQJ6p5Lt=2rdr%0gg?8{yqOays)dMaENs z$kDgu`0Ca!#FQ$Q0JB1sYmh?Yeq(xaWC_zhHbjV`LH3C`oD%1v^n{jTu-k}nl|CIM z^zAyeW)d?04ULogU8i;z?LG_uR=Oe|lCr!Zc@PMj0N3I94A;tI_sTss2GKbZgEe70 zh6&dE0>Eu==d|o;%S>6^>@o{q9Dr_(ArQ$sb3mfD8hJbGHyog^`msjQcXi6qYMZkL ze3DY-?J|+2z|NP##8PxBt){DXcH*|Gj@-wi1o?|=qgYL5L60tP!dt%lbJ{z+xD@(j z{?@LYq#Xg4h~)j|c|i~QY^8{k?Oe~n7ec1neHxpkTMR=lZsG!6pUjz7r88P-j||k3gqg&m z$tgCW4I=Sq)kh%k1Kvhq8SzL>Z(nlAO0l&bnOMUbX=ZPPSaD2(x*;o*loJIo5r~^JhG5&VFoQZ2;*v+Sf?gq*JYpXxsK1l z!*OkeV|z@SlGIYlN%TQQLYl6=?y*67aX{?76@0M>7C%-29 zs@qwf;=aZVbk%Iv^U z3ooV#rp78BqW8WmoTdw0yBmIuU+exq@BLd>R5?|cggW(_uki>+ zRE&dmXO|k71=)`n==!7D|J{3-Bt`@sUJEqon5LQ!NwYa|A1|D^cIZD4+J9-EZoW9n$B_x$EFL3Li?Bq z9~$0fTwic6tT4&oE@yDb*6`FDk=|55VM7p+Wh{4rhE=h%U;`tj8@i2e{FW0)0uCBR zpDR3ANzx!HuS)9e+g>7$XtdAOfeLQ#JEu2(3Dt-fy*GsgZS4D+vG6B_0x3mZgc?tg4^B*?sMyFoQ%0BePE(UBLJb=FsTN^R$d6BS^ zPawb5j<&w26~^9#cTphl2Ld2zLGFmP?7XC8QG~(={gFQdxUQPM7?u z+z2yPCd8T|-mHE-^!8PcPMZWnGa_aD`3Z!&FA!@E z0wt!0*3IfA+)OLQbu?sy^QZXl8mZ)FT#}CqE<8W?>?ls`>73MP1#FredeqKhRU`z# zK%#h4n(I#?L*ZBFBAe8+%Xmi0w&`^A$R~;OR1MFiM}IS}U9@FGFyM`AZ;(i5UoS$? z*ZOwg7v@T8f==3s7vQd^lvza`E0-V>i@_Yc=a}mlXi0lt1BF<-xW>13Z`il z#Hw6j*!)i9lvt?4pRN3-&5N!D``%FD!U_dQ?vts}>iJixuIn4gg@^2ce+>HG2{wrx zM_T;Dzlf&)cX~(^kE`gd#Ze`16km-(x_(Q_gw;N~Z@{Ws4z@xhXxfBd{+(+B?5G7=CAmnAMf#;%*jW-+S+k zTIc#N$ZH9KK?zKStXcNsP=**Jr&QnAFt6lT=dRWgAj~m&9&QkfK-99ZFsvEaSjBcL zywBMf@3QCN<0m8J`QG^XqKuYo&8MX4`Aa=s3qo2G`tYLnBkx4z)G;RX$@KuOu5*JH znUm*?!a_)=+}gwPJrP-jD+gj+L&Z+qsX};cIHaEab;5CEEm#cKSBmUJ?JIfGhUHX; z=V~VeUvQ~s`I8QSnZgf-EcRsZqZ*b>57t?{s0Q{{(BxMP3XW|jV-7i?{RW3_@t7+^ zIGS}v-v?InM}*&bX_}poa|B(em4#hINJ(fL+l%2n<^}saHZQIxU}ZsJftMpC$yWpO z(jD8EFRYrov#&cd3VixR7ol4gtP-V{pxSfSLXs}>>BrvjZPsTbAeD6z#zm+~QoLdU3=WN=KHI^K8Ki57$iM1_89+;gsVjU#@rXJF$Tal<6`mOKrCz%v4zuTfn7=z#<*>oQzttNfqi~&~aSB%kE}qg&JzD zpgUIIqEe61`-km2MC_&x+-p-siSi(${SSjMHRyY;e>)&xzBG(8+OqxR&9? ze)xut=WT)ThoaO1aeV{X5H;JFau4_Ci%!ye5H>7+k{@d-c9W$#f7~v>%kaw@cTJ>I znJOz^y=%*~b7Hu%t72|Bp1pna1UlSt6Uy14^$ILj>3XaQ*<6lZ?%FP zlNTmIU^WowsnE}=|HIOgfRYpO@{?CRJ++H&qRQ>wU2`6v~Z}Q#Sj)`lAY}IC^I$L)?#+rJ*U$6Fmh-^tDk1 z9{j6ck_MB-W{a*=gtksJmgnerML1M%`LrUwBoAlF$)i^GgtOGPl`b^wn|6wbFQ-c$ z9{~7S8u)w%4FUu-WA;SdsfGrNBZHGNHp95Gz9%`!t@fIpI(3V_!GF{G%syskbajQ) z4U6@=0owT)iCx7H%EX4H!KnBel+?iGjWL*7Gn2?BJs9#14;*cx=^8Ewx@=$Dt+pvx6< zw5yHBmp`(s;pr%#&Pi+FpU@d2oVD+5Z)0#Z2D4YRkGP1M!yHj)_z>3qmDIU2Ak}hc z`jW=|1>2TNam;drlZnLJxW|*T=1F4O;MvlHyGr9Fe3mT8w3RxN*iYp}@P0&%2D3Wm zU6NITxeH8jrKB(+elu`VY*0@YDC>p)U307Z=t<^8=%Cxr?~xgNYsg!+vbs-m!C%yZ z@ZkK*1^ZK`2k0UWopNQ7SRQ6u_OPz!*45+&Ki1(2+tuC@vV0L1hPeL0Z*oobqr@`M zEIUY_aINXi+Ph5LXQicz?5)Ewu+j@Pr*;gOwPrA1he&d>&q+60PXH+EZAfR(o0~+4 zcfe?HP^NOUU;4SWlrCh=WvuTCGjdnz*&C5Vt7K9BUdc?;GUbWD1E1cK87zsPCy9|5 zL|+ds35lJNsay}T0iGK2S!r8bJphVc)AS!pz$*$&1VLk!3Zn?7PSq4McmLeS>>GjZYF)R@NPk#F<@he2h3sgJ4s7=)wX^EikL29}ulEO!z8iYT*l5d8aIPA} zAk9EqiY-^R=m6BihvM^MdS(yMCPD>+G}gw+K5{)-DeVzUC#|^s*+NFpI59i`2PKsu zU7-&U%g@nk5cLBd6bTJa*>|2tGQhf&Et0MZc#dvf~>ctf5N@3rE+sqA%v-_hAa-a4Suz|4BUO} z;AX7fjwti6!{aODg&V}@F=s97pzN(PRJw@EDVRW$+OGEtF8>@JL~U`wPRzEWTB#>) z^NsSNlE@;dsUkuUv~u>H7f5Oc8ACcd+see~M=4}G73Ad=>}-`!xjCbY*mPn8@)^vD zZ5++Evdi+4gOeOGT|IGvXN4x*y$zYVV^f{U|=A9|(VF#XA6W}6L zU!!2n^m(+3WgfAuJlEH>R)}S4%cw5pK0VNw^ZiuJSE*UK z8VWBEB`hFuE!8KNhDnwq9U}f@J3p%x9*;^dAxUlRLc_ZQXomLR>F$1enEDe+mv`98 zNK~|k$`7%^i*;cV=)F%S8q;B);dOBD;075bfuQ@T`!f&TpmHkx%7IU5tDH;FFUHaM z2DH1RFxLug{(kgCNBxD(m9#%wj1wg?YW@VHuy`I!zedCtpRoot+sSbLMAWBgcKnUN zcHfQnHjA}`wB)#QYT%LBl7WSj{3Z8TM{kdQU@ImkCdTxYB}=tAHX-)7qF>6eHk{~F z{QSTWV!XbREo>~QEOQ@Q^adekpYcUg1-2s>x4d- z<__u@>*%T-y(-2WH@8+iXg(?V&K#5yBssc2$$B!o@TpQ?{R;+aEFpIgRJ5*};W6~| z-qt@}m?!Y%Z*(=S%xOB>NbK7|eSdkaXnpxybv^Z}w!ZN!+Ni*K;FFg*f=3^5f~EV4 z=vs|)dsAbMkQXdTQU!~pOGy9#P44gt$a}B|?-gBX9^y#}AoMU|yV(;7Cx^EPZP2IG zc=A_%IQok)kHdwEobypHrU&yzXHIo%^6!-XhC^r=Jfdax0AnKNB1J$b}*HrkjR+g z`8i0{eWF?nwG*k+lrgh^iu7`hoa+d+(~Shoj8nHlXDJFYw616~9C+0SuXx|zdkAGe zmbj}Qt|<5n1Xe<{jDhkls)MfmP*a`(D&s8j2C7NuH(jelTulle!cP=&Y->~FH*`V zc++G@+V`JTshbQ%!2(Df53R;xug`5k^i4fVH>R4%U356;AnxV^qu1GE`XY}zi##W4 zt3jsN?FBrJKkDJKU2bq(K3^j2cZWSX4>h?<3PkS99DJ%Z&Zcknaq_11P7oM)uE5_g z$s>R<9{DGFYEGef+1M2Sk}h?KKKIIa6ibiTK^jr+*XJ%oa2u~~zD1Q|OaG5==0tN_ zzlKEB(L9I@+9PPA56%tsz4Olo^OxVCdo^!JbM2$e9Ad}tU!23okK7W6cJc|nN-Sx? zz8!7J;!U|xMZmN>Yq6jh4@)*pcVe@Ve2Du%5r!_}c_?H#<^s_IX)zA}{+CMs^-H2N z(91XSt1|AfQJS^mv8y(eM@Lxa4vPv~O0hRzME!LoIkdnkm&#K~-14Ktew>N`*D*~O zM=+jbo*cv45Hg|jlhS2~qBMc!I&F+TAC zXIA22W8bjAO#k5^G7KzSrYHE2BNw8d4$@=yvT>XPo#X3F-uQQT9Cv<*d#-SO1B{l7P%;#RrsV$ogih_h73i zNC$(D)L*&R5$v80xq{?cXn%o(XDG2xHZ;CpE4 zSGK2eX;WBK7%VE61kGAc!4snWHSR5iGc=r8_{rHFflPMrob>mfLh|>1{?ku#K8>b2 zru^<%A#q&|o_m>=^_bnCA(Ux)~FyQqY#yfYlKwEn6W#1^h_$YVt4hJ#b=HVE#)qTGEiyc(^tc`%L* z5^>mg{eVSr&q#t(WBW!pk)snf8@0 zo*X37`dig!dv5FJ$F0It$X4nmfl_c1RT_yMT&m@2kA))_pFZSpxF}{e%nYt;8`&9%r?X}hfsjblRxjgR z;8;s_*LGVix3yEgxz)amemQQN>R(%b{#wk*y0rPd^wbeIM-LuplQ)-P_dNUOr5S5l zrI-z9q&NjA0)e>AuS?}gyeOgj_- z@pBDsCP@UCG)Csl-z&IL)VD*ZZT4bLsJO)xA!yP=Ojq8Wv>sCXc^94>TQxITXR|sU z>=i>y29}`@gT^pSQy{3brRoy+WOcbBipH2N3LKx1M9c%{Vd8X4Ec~D0EuYtZ`dP|k1KnZwhd#*i|qHIh?v(c zh+;Db5z?a6cAkhB@$^bZn2JZ3sub#&&~ZN|boFO6IpOR(rMR4Tdye@HCkh+d|50G` z;a|Tf{#l0b&&$7-_*aV$rM!x%iX9dYeV1%~e6pBOc56HO=n>(&4d}_eKA*Q~HB8i~i1|$EOA7Dh|G)i81}Fl+h;H&FoeOtkkW7=j zyU6iY&K_!8PIhhfkj%)!nkLw_#dk(Y0&HQ6z^^yH{}Qzfi={=5$$>*ajBY;p&5>lR z|G@Hsey)F0KjT%+tiV=D%bcDk z@UVB!esls?VSe|-#gLD+HA$%0DSN90<1&UQUU)TJN3VM`>&pHVJU_&xrG+D zRY?lK%JhRS)kVM{uEAL#O(AuNvYC47{#j*;=$W|5A>#+f1n8Y%LQ zp=I=WuzQ-2`1_1Za+(9nFrC=nYX)q_W>tFziW@5uq;{CCYQyuMI$N;D_i9+|YI3z) z*t~DoLaDcrm2dgGrQ8Zz{x&?4)A`ise%JVb!9|5_8!!1dWIt6rI`*T{)DL!q}6{S8=&>5ms8kV0@<$Jdy*v~OQmu9;OSBH}@ zl~$3-b!QdVMWQ6{My`fOeAUD=KaPxvbZXYIA7B~l>7n9Aq`i?X^GOGm4#7?Bo$rm< ze#Q7jR?U}+-huVu3rO-bd-XmZ*b><<=(*TR@E@XN1CRx+8&Th;^~f#o%tG3Tp0k>{ zh0$@m9ZE~DbB~Z{*z`Ee|FD-?pYQJ(R$%w$+cbWlO#Fr<FrLW|LhRl$BJdzAZuXxq6GBg$MFW*ElPt+~9-ZC^ z??nTZaqiy1J{F!{fNp9VN%?jo@qDD`h=m|kXRUr{+=`BqfwqR=hK{4Xvt|z@`-gg( zjxCOb3o~}~B0?hoi*^E0_2Zvba^=G!Jx=E3Og-@XbDS*V21QYN&5-785BzD1XlJx~_f~c`EC{(W)w(f5K7!-WM z=CMAEk_71tho)+XwxIrf83ybEiG>dsoZa8~3u0IO2e zNR_E=x+02$bgtj|arfUjaG&lBl3Ex8G@fdLY}Oeo^@BQ&iSmN+v)}YSN@$!DdT-NZ zTj0pzO(tPmh&8-WRWYEuoT)FgFZXE0`Y(qnmJ$)Kv~Z(-cH!RTvK@TfuJ7vwp&xd! z^Xtg^GQx}|reu@b4o`A@0y|tB8BKgaifgDk z5{1c^ToZs7O`=Xo1krsyZU?ey7N?{Vo=h6o;{df4n{e}D(TaDoH(vYxk4G<6jVRY^ zTzDQ7yked$$!b}@B&d1wn&q|o?nXD>w9b{qfYEvLDJB#M{`M2l)6}08;}Xu=W&_2u zHaaK|tF3OM6mspPQi?9;g*_xST5?5tg=v!PNbZ#j%&1&e>(>9HrWeRLy?bc|;apgcIC;)tg_b|EBG&u;3%@2p1jg%PO z+I$L74LB?t^XN)Xq5pPuj4kzBo&88ik=}1OXiegV&_oNVqM$_B4yS6UPO5AVO4NgL z{4M93y?KE21Sq6%sxl>g`Hsn#M4xinl369IO?Kv{RPHQYvG82AwJ0Ay5+d`8&1}v{Wh!7g+>#{I?JVkSFlK#WjTeFIdBGty0;az>O9)rA?b}fW1zVI=_-+!EBpSpeZFI;+! zkoKY*C9r6Ex=6dpFV1*iu|E^V)Hl@Yq8ud9B`mlK~9P0KN4xT`h0 zqp!|wPVX>hj?7-|I@d9ds60&K0vEz!KK>M$Y6^<WuS%HDj-4{4xsD0FhB@8Kxn=EaqS_2BQgv&Q@njdhX8tF} z^dG(Z5Bs~|3pD3(_)xk*4&@Hue8AqkG-BeP{kSR?m@L=wB2>{!qoRsKUL{7;W8QW2 zApohiP$DK21q;*8v+9dYE4mkRS%#ES3e<-bi`dPrdMC7w+i%c1^wcp{%#e1wv>C`b zeq`v|A)t*Sv50kapj8d&AX8jRqCOldsUYwz&R*+)4W%5!p+;=SF9@i{%LB+np{mmn z{WnwGk4DPsF`+t>|B$$%uC5FX4(QB%-9U-3XFsU*KRRT9BA}UAD+m_Nn`dfa9UD~% z!b1{&XtB5_??Z1Z379HUP-!_p6dHfn{8`_X${{a~W*9LSoGZ1b_i(EpW$})t*@Tju zh2z*-ZZy!Ccu$z&ets52+jI+PtPro&*dJVNf>W#AR^4J749GoVQlNDD|R@R55(&RTDM zQ6J8V*+dG@c&Q-LlH{MXA3Zs8dQ3FOr)jR}$l?-ow*0!4dn>g4e0Ma%g?*NH@~ra^8VQlOQJ)i+J%}NL*RS zWiO$FUwru>Dd!zyt@pi<>-Vel!wf?5c43>|wx>FUgRCy!qEGmZ8){$j)J#wh?mL8a zHm39nt*Vq|t%z_Rc-Dek15w;fY*cf>szTT}oi5(MX3tCPny%{n=3Cz%EX89h?isr( zaGHCuC}fv;k#E%p&^gV*I+gx@a6y)P&Y7E*lw{s?HYP&DRa!UlpeOrhxG5}y#iB+R zsyl<7_by+_G+rAr&^7orlnA9pIAGIz_bz{rELyoOipY@q-8NH6j34?L`iE^s7Tq=H z7}tt0<1;AKs{8LMcTBM4yM)!mfY8%iE@kGafmu=0T9lU9mnMsASUU{P!#}$A5n4#uB}v2{IsjOsMQf(LC$PpO_emY?I$YpPMdV+ z2_)rgRgbUFD=mi6M8t|XY^)JEnkC9v-RzACQ7ccy*^ii0o9=5nB>EP3F7$cDZJR9L z?yiJkN88ncQf_fX%L$(qP#Bv%Gv41NOU|WoybyN4(*(iB=Fr6dfz6*!PEKNxK15R} z8Jxx*mBR8`KzVaVFlf%V#=6Olk_l{zByKPId5Wp|7|c+Z*lmp z+@<)B6U$69BLl;$$^FYvG#@BT&6s5W`Ggd#jkY+q6Tlm4_EO2s_GQ zBHibV{_qO-`}AXXamBAvOZ({DgGwi_+p>?T79(2bO1)r1sC$_3YphMB`t+1$z*EX+ z#Czj54;C%;lbsK_v*a8zq`5R1lxKmJh6e)KM`=6~`;I{7Gv-&Jmt!9`ZJ~36;)}A_ zT$0OMNzXB(+s9G{ee_PKJwwA+E3^K1RJgpQPbe2+Hq5MH)mskhVRCC&XHtM2 zXicrCp;MOJx17(2(TcLZSlFe$DZ06d^!q`o=AyS(ynin*Ic!Z>fboJ}kx>ioa-a@* z00m+Z+5(qPgze$RjCS=7*CT2Z!eVONW58fVR?RPZ3oQmO6JpO^|F?MhKYH@(!~_Ix z*oiHk$%4~$8SsyUn!PoQHR*3Ss&Om#W8zkelEBEJ zVnB$n&^=SrdS#|sK%qJYbdNTtbH{`m>uQY=K`A$c#g!gTef=_J;S(^#HRtvXVKwOY zzVXJ5+lus(kkDP4Md<`Fi9jskS^Zbi=6p8f?Cv19)KKk)8z~N8l?M z7JjU}fv*ko{4&JXac8Sjjk(7Mo7{^&zSkakJv{Z|Lf0?&TC_7L1m$f;l8gT!qyV7q z!>J)=6{SdZHBm%KpBnyt^x+?Lx`Ny4%&szjL?eLuv%dkLenKi>-Y?%j7Q}7I%fsVMCv` zveS#P2@QDh#@m0c?xCin*)D5Jct7k8x8kUz`;Rr3XzP>Z$N%Wuzj4AfdfBA3=rn&? zdO|Y?GwG6PW{6PSqKkcibCoCLfpw~5z)790Xjz}F?a}hAJQe~pB|;071MB;Nsy=F3 z_QL)kGcxVZk`$+f9cA31g~=jJtSWo(s|ahI5xmgOmPyv5P$h|?@eBj@1J<`1RxyhH`A zKjxd?ouR7v zgSZa?PgiM72(A!rg}42NV{%dDm)2ik=H)yRip|++D{&Q@*I3S4e*V|9c=(`i(>&it zn0n-X-^;!6+tGW{8i@Z|C=zA1nU2H~CrpRjG zU(fyRi*5!O0!@ zLzRwn2*m^f(p7rzp-At&_vT&bt9$S7+vnVK&$;)z=gc3;O4eGlj5+39bB+0oXV50k zEV%F;J4XVD{ot6@^iwH+#NLEWmH|2s6M%lMPr}PfNn@jyfXRU}(&FlJ=)Hk-12NwL zQn*`*yELgNz~smbuIvLf09D43m%$2XHEJHOkY_8<^sFr?S`TVR<(HrR&&G84EvCzF zd^>FL)IQ!0XOq4&LiyWQQ&*8B6l=&3#DF#Gcr zr$TWuYZ>TG!fh35li!r?zg$oLtBsBo>%`$*FaoKBl+FOPAb7WbsLq^ch4#qCo50J~ zU9Z@V;!+@N@VV(b7u<3u9MT5R2D2Dr#ENW>LBm`98CGz zjn>>a-79I;$811w zzaOBT`x`b#712e=?Fw|z=||lMixajZ#G;&hlw;e6@3fA0Ow#&}g;Gd?NiFTNh9J;Q z^YD9h0+rM}uo}w=N^_BlArf|57 zR2-(fI?BVV+*P@4f|R4-Mcw;=;SB?V^{K&gFXQck6DGeO;#IHJ>EDPFZq-M9X-vDU z;D0kYTUaEd+-8H~>~xZfWSW-AjdVcsx!bPtzllGq{n+z)Ek$<-pfO`rXl=pRg~APz zRb?MJU*?8QEAx1YXU=9ee{NOHj^IR(Y7Zi8J|f-nQgmY?L%wRw5KZ7t`TcIOyzrUM zVra|bR;_?}%I|_6n&8NCWJ|~=I>WN`FH@)E72?UBHGJzt%6qWS0GD$*J3IS740YIV z$KK-Gy6M;0QT3FgcWNjmw;zkl?vU&VOjoE|v>WU0DlmD`yu=Er$OXhqiag|tNkdDH zlK{Mu$(3s_&2JNz9Npu451ouGp6uIf)u#(#EtGTI2*fGzxzlG-~U{O2hliCl=Lvdl;gc08YpHh(elBV@w4+ogM2p^EKdVZpJZnQT#R5AJXGN z9UH<^sq$bmUGjm^a$SN(<4)RcRx%Khw z*3&z_M`wQfKS19-Q23@yrg}{o1_QFznnJR&YqD3~_t9+vPv5;8lA&@_KZOf?{NGIC z6|-W*Uwt!<|MKy|)xi2n%M%5i>22ui+|1=+-FrU<&_^cohW1JY18N zp-=9by-(4)BSP7Ck5OaO@d&_t=0|qFHrl(DUz2|{7}%zP^N?x))dnnfj$;)V%I5?Z zT;5;3Y)L-);$tUNu>AHt&B`Uj9Cus0Lpoz-p@O_Y+U4p0wL}t!U6*YA10>aEl=i>X zX3(9Jk|*E4u8cFh4bIeRj7yMLVT#f0-C=hnetFqqs`(}un0uT2k!}f|d9dER0#b(E zd7@_P@Z@E0YGD2e)4=u6GdV~vUvJ;mgPI$UN@Ilr#6mnyN>}+POdAFDnhsdb2|WTK z;f##EbkTW*KE;K6eO5I>A(C1yOLLKuweP0v4_XzRUWw9jrC6UQm`{Z4sxlf>tap|7 z(&Fv?_4j)xzTKyTtlYCf`;}+{gXyOC{rX(=;pJ|mSNm)FofPL*jWTsuCDQ~fXV`0l zqB0`LM6xWbX&A6))ZzVlolLHSoH~DXjWEc!G_0yk1R8!Ok^MRYCnO|_rhj;a5nZGo z7$(Un3TS_m)tXuPW_vreXTrdNqf|o^Ly_KDlb@yop6mkO<4Sg?1*kbiU0PVMuKlN& zCvue{$VslH>s->8lKO8Y>;Qx4*sWvs_xa9Ig9$8%Dv^{%nB7&W08fjHnBHh!5%O=# z(~;(m4fzW^;=wZ(m-$>l_ItB6S*?RD#jeYHcvOM|NM&Usvew5Z`>u3>9@Dt>lQ4X% zPq1FA9G#SxFyWu*Qb0p2kZrkrZPPpz6RBqdbp|tDgrhsw8x*~m*j)W6CP6C>v|c`j zsdo*`y3gLIo(x0U_kJa2vRJL$K?6IqH5oj%2m%Kq?;@=B98zA0#&y6jqr*%siMS#XZfuZ;T2ZchiA=+h-x<`uq!! zi`5_5YKqh`uX&HAg~1*7B3r5iMUukQhGw$OTPS7 z^?$#HR@ufG#|!uY%4*;H0N`nLZsm+QS!PPP06hsw5#X*TZ@%0KaxMBSe(D z5vngUuY@$U3ZFF_N-RN*a+uXSj-?;L@_C;KfDyZBRA)U148!4X% z%1kM#fAL5CqDv3LCo3L2vZW1(V~&iQ3LrUf0`w27IEkd&c)(k2P(S2-Cf{f?6Ncr0 z$zwQF=*{2Vb=N7O8F9JV%NO=3H*qI;JVQ>yE3(d?@P&o!+>76qs35=d!_u6bj<(20 z_r>54>g>|F2OPWK%}Ddok@_VQfuY!DUw-9v&8I2ViB{?Ovt-S?2)R@Be#z%K`&@Vq z4$;w>NBw7$${q`Rx|0YFH?icVr@T>fj{KRwpvM3|6j*SQI<(8d`a!&Ex)MZV5UzED z&?ogv$le6SlWb^SwIGM~Y;DZ1&3f{`H|tHIH^6qKFL{B|u|SwnVdkX41XeBs&yLrG zFeOWy;5o0}w#Jb}#*3R$e>e7j8t1^zkF2aoZz!I(8W&5>}}sq2L1^=vsE0o5(U- zp>aiC;Vvh*{8jdah|$`_^`V!SCH%jYa1%|Yb`C$kuCE$=-pK{iHGiTlSXFix{LJR% zWf4PNTwd8*KR|PBl-k9{M`{lf3-a$9uT{o%@?Nya*A8Lgtpp|#~FuU_B}+v-`;hO5`DUi(kWv zo&0h-w)z$Z^J1ebQ2Wq;yUt6=#lQ3|>)X>^8pRG4H_`BSgv2rIgK|0wXY^JL^?HpY zqKj`GWSdYz=A;Ftz&cCen*uZ_XbV!hU1H%+@E8OP503CI5tKd}?!O)ZFG4uNPFWkN z;rVILbw9POtV`tcQv|FeHH7AwcbH~*T9+|Lt1+*9X*y^}%52<2FEN+mTsUt#Ar9~Q6Dky!Rm@VOmxoam zO`-N(AP1e%=1>8FnLSb@xn=VAjJpZKx^G<%W0tETSU%2DBi&UE_qI7;0|N*RTW#lJ zuq9N2Vv#sqP^om1aK&sdNu**MGdv%^Rp_}bS)a^A?4rBF@V0l%z(_xTJh(7qiG$hW zt`f8BPX4Fourj&>`DP_1c%e1eaNY=@U<2zEEU$JneXN4fG6#eYdTz``G1!3vuK86r z$bz-5J!>2XC&%6RAbvo=Hu{9O`|4I}-&cN$;9$cfR!8A?4{&&(GCY{iDfQ@EgJ(8~ zH*ofz`X`ZKi*=veasA-bHv^tdZ1x5n^(w@qr-U7IP82mXgp!tsl;kjLLLJVVfe&~t#gF#vxy+SDSk8kJ5jT6O3M7~j$k7C9 zN9`QXtN!cORSVoYnsxYbLxt|fXaXYY0LLB%_< zOn40mqfRo*)jsURW+-cwWNWz*YB0ncu1T`;)nPxPB}?^yzJ{WBUOxkD95IhBw9)iWY=VYKqvRPUqkKz z<_jjkk1%zG-TLyaZu-`o!?_7H`O!<9+cg`fG7T4Z;<#1K8Bd6sg5~1M=fNy zq@$&LZ}7$rK%%m4Vyjn&p}>*4X@K2UV6uu&M^uUvbNLtcH{F*z2=bG9oHvD7^SV6y z0idmzH1Yh#i%(ytIix+nA?MFtiZegI&5radxbLpqLk9+|ae7a0|Ci>3^_`qhfTmlX z9}IOnXBW78zx(Po5bOhBCKa|G&Rxj&TFU(shWVE;Joxmhh1deu0@SnQc}#QS;r823 z`4oaP>MwVSf!2cl(URZI{qjd9f13M8CVyIs^;;&df6MK^I;X2Nk9xEus5-$x5O1Lj zE0$O8oU*MSV*sAf@nTbWr_K>6=;XIP{QACh;86TEgi6B1dfZ_1`f2c9nbGIwY z;@V-Nl-^|av|`2VGPVWDDJc1UAxI)^mJ1N90h7`+y|PjOVb;^k-$+iWIWKl6`qQg_ zdj5ZQG+YJr%5g2BnDnImCW{zTSWA6>PmRHV-@B_}F?BP8k40YobC+OuzwdbVVoixp zUo?hLJI^v6&Uj|H^2XyqSp7nJPJRJo6(2k9_i+4E$^ZXV?|*1G%$o!n+}r?0>WrI- z@4$1VJ1NwJe}?Lu6Ik-vpO?vBGQoaZ6@Bkf$$0rS_b9+Fi9}1)G}yYs(f@zOTnG_6z_gbLpQLigPa6 zH}QEly#Ce|aAwNdp=e$p41h|ZyLn^bh&p>C^EE4T@4mT}W-amzQo3Z-af6T@yng8= zs!HShZqdwR5x`P^>f!h4E3U(t*XTH$KfQs$ryDoq)xd;T+!6zktvGuP*6*D?UeCss z1Qjipblc$ay3Tu4acK`;XT2EJeTz3_+&ZfdURO-nG-v!A@2=l78ESSMgpaoGm%Nt+ zwS2>1_;BWWY^RlV^m9Y+sG4mK1EfT4<5Etd@^V2_4O)XO8NyGjCDf=}X&1!KD>Ndt z`+x(T!L7zouQ-Szlq2C)AkhTk^W|cA^H74s0BQ!e#IS<8*xhK+ee zKu)`}A+^huQF~=R*HhD`Kv$&VhNz^ldE}zIw2OtqS(U1Twvko2v=4k4mJCbRiz5aE zGUHcEOd&fGlSEQbRUB$Jw!WfMvNgG@_-d1Y(VbUPP&)%J_fy5FeR)U??FUMZbbZR& zv9)z+Chct=X|2y_)`8L|eB5n;=e>xn3Sn91o=*I&W&u8wV#{5pff;z70vYxjHs* z$T^@b=1YSy9SUV4i)QM&2X=w;4>h!`R(G9AN_05YQ27c3fpebe-UN@uc+4DzkDhR? z4z7GFmcJXNI6PXUGie54clmJSVjP{II1`}>oj9zZY7!byUKH^nB!0oy9L1A+pMrv# zTborkk#TXN=99Q3GK*J@ zruolRXq;?mB>Vt%G@>eQF5W`EfbL0Np%@M7Ru-Aus&!T( zaiz^=rC*g{3CrUvA8Le0z$dxFnX8!(#ek4Y33022Vz&lG#>pB~P8==-Upbj?*Lmgn zM$q?4yP_YZ@V&hd9|llK?bthH#!Ev;FoSczuo8)u?2&O(fgKdxVcZDIZhNH&+XR81 z4pJTw`arc#h@xtjC~f%_d(D*dfbB}98XWdp{Yc?yoA<5t=i;C8ZDL}`=b2(?G`)Qy z(gr_^fy-@UvXgDs$0K>n z+qV^XO56w_5cmPQB`|JitRT>=)OC0OYQw zl+zj#F7rx9`4Qvf;$XSC5x}MB{R;>e*Z4Ix7S9S;H<$A<*GP*_wJwyEY|9T<^Q3pw zRp`UO2+A30=v?JHS3~ocLGq5&l=8}*7*t_37oH!Wikyuv zIos7DL*V^0s<3-t1Rlyq9Ld)v6)9r0+Dq{MCo+E9;{NAq3}2HNtA5E~Vq9hrSd$#E zXsyE14?eCU3~*hZB8&oC04rEi*Mu&wknAeZ|MM-db`iqX5ad^zr2u9GuwI)vTNqZ6*)-3 z>M7W}7Q#W;7F2~Ts>{XKegx{P3*AXcGI{#ax{KmzY30*71Z?blA8tk@WGQSXcYMP? z8QCD)ZVS@^e8L0~j{u*rz?NC44d4?-_}7*t|A&rSeBR;m&Hyy*bgY&IEe^{pnHs90 zOl^g!3qIu=vMSU|m3PUaI$FK!n9f!%E4Y~b(bb9>s+Q{i>ahyPg12)w91aa!@_-BO?@4!06*K^LsAE$R(lX@YGkp;%J6D;P7ro+kk^`hqGqKy4L zO-G0vS)M|AghOSC2DrU{oRX2lJj@FkQ&XTVV2geUJYP(AsUom)5N=H-%*Gky!+Q}U zUB_JOTG{C4wW}qS_+-Rfad7b$-wL18>CD6tS*WdSee0+E0b0z~F-hy4ey6X!+Pc-w z(&RqpzX6wgNjXMO<9uCN=T&FkcF6&!o9Uc}n(!kG@J1&?)1%tT=xNuCY+0y~*weXu z@L_3f(;Ln64n1G?;U`i{u1(fhSMI%_@_*g%0&+FK+(i%}l62YkoIu|zc@w|bY;C)U z`r^(qJsA~q&W7{Kj`mo@r0@NogWqRBzYqSL`E&65%zxb_Qx}9=%WJya$*)gS8F~8d zR7l|EI$vo|>N&nSe$}`Z!(y&a$wZ7cxVD(SZzl#>KB<5iqv@;6gCK5l9A0|aHkXnT zr!185{{qB+zWCNp z{dX_*Bt6AApX_+^wjxtR?s#Qtq+NHyX;#CA$Dr4Sk)f$RF>y>kyJ?^pTJym);bulG zYxbHKO}KT`q@1+KW$_@;oyyyC27Qq$3Tm!wwaaFyR@H%dK)^#bQ4lD|fm(HPV3d{X zT^drTGvzhh9BfUT5g$p!Nh*RI=2T-#8jM8WWAH?~$vNA|7k%;ka+x8nR3)(RO5EL( zdN^ea!n!fT^0_$Ula)E+@`j$DjC^^xzGNE~WadcUPfM|*sitMP+)lmhvFYIZFRA{k zO6e~W-}wTgxaGNt#KI(*^V|JQt3G#DKrzslis4IpECwh>eZBc!R7dfLEsgp*R_hb3 z+gH!Tc3kVS?re?tFvXkjPNf2w4DDso-8^!eTlZ6bIE^|{;Hx$LDs}5Dxp-MooOo*! zm~Nt91NnrTdpG~i_fJ6q?aMT6#}R+B{ap@0T%SnH?w@S`ltayv>BO<{C)+>e2$?7@ zKsx?p`==b6TKNA!^%*^kZ#Rd*9TQlYOo`zUh49S$CG(4KyOi1zGra|?HxhK8NlfSb zMW4?$mJ?MJ1GwEfyFL8@DoM_I8t3hqFhg znW>_>9h$9BmhG%SutAFd3uuX6Xrui1W|LyrwTmPTg>-Mca;Sd?paoYlX5=f|W;eC^ zPGEQAL_Ea2v`YN~GF6wa%A+D?-1%QaxW<(%x-6M^4Ex$SdJ`jge$M7~nY&s}#+K>M zwoUA^FaG09zaaU4(3JlF^m#OMLHuHSO}8lJ-OuTSfs%kU*BLvjb18ARw)*E2;Ma6w zsft;2?v|mRDN6pU*4enA8P+MG&83o8JRaFlgH)ctuq6nMUXDa@$}qW?{rgpmJdwqk zAtQZt=5DR_uo%H2DXV2mz59BGWyZ4G^|7b$Qc%>KchGXVGuH#oAtWBlD+?9kepiWb1;N(M z^fY>DpzJ%Ulg5wxWS3}}wa0X-(R;|L7sKL@|h{BL;)A%5`<2r=WlvygHZ4+D|id(QkN z3t*u7KildzDJpCpdA0Am-|;vqz3xeg`I!J~8XzL9elLcIW?dh|5JMrhPc$$%2QCgt z=@cTUxC>TIs7cq*;SI{4&Oh-^kfHaBDrxT5hHi^`&~%SB(2+fO3o);%$7}1R`pS`d zN-@Q!w;{Q7%&I&Bi>Fb&5GcV#X!mf_7@LsC<$u>(?J@B8y8CwN;0I_JC>L;h{7*qq zhv@jV@&F1TGV}*6Me}gonqW%p?EOtIZ@Q(r2JpMrS<)<0H+e2(RRJbloPA4`^ntu#(9exrr>P1iq7Qg2?I9} zRsHu5*v~XegRCt}M|}gAOkTeIStAe#>f%I|AfuUn!4BAJ9uOE#cZK(hNHF{Z)U$qg z3F@J{?QAl`-lQJJYnq^If&BKD`rV~-{?I;%ZC|4d{#qo7_4nH4s$RU*D+6}7Ny21h zl^@##?fk{aVKDb0@SMU0M({AS|ESe|GA>fcMKEtI(=8}egDS5WX*l4K?33ms^1Kpv zh@a5yt^Jo^HodAKa7urUkl1|H%SK>Te!DxI`!st+s&8bE%J=n4ko0F{YO7Vv&FH%D z@iHGX)wq+)n3h2gy+g6(G4qaF+9IA&tTo(n+jn=<6|2Kcb?|wdE#UP%IYC{$IiyyZ zrRc?_gis}6SWUiRHGS7u$JQZ%-5!E-O^LxfWOO5FW#CH6>_VWeIE~0)r`dYve_U){Iz7w9j7u~{=!Acg+e*IE-Aq`33AS0TAKZft8vbrSkBh#nkyL} zXrf@!O7q%F`>^?q}5fe&eg%-FolZ8H#e!B z!T1~RTjsP%h%Q7)Q`mZiJW-P6$fn8{QOdeQ#B# z6``!iS{}N<x?%mY)y;}V!TM^RH&4j+Q}V*NWXtGp@x4BodL@J#TKM{aaTe>9+FBT(qwyy9p$in_Y;N~0P%Oz;x;zON#;a{{CU?%5I{r8(Zj$pBM z!a_P4pv6<`)s#NI2HSt8h^VL0%s#!u5(^*wb-5Yyc)VB2jp|tN{tj%@Ldykvw(`RJ9SrLDoz~0q5w|@6*H1{+eupL;{b&Q-B2tlY1 zE8~{7&q;8sAprVh6;F6M_6KOATj-4vf$@}ZIoHDz>Wd13w?p{-eO94~{B1!p=@St5 z(Yjt;KYQP=;)+uR?hgJ#pH(^N!=vcAMMn#^z}1JS=N~Q_Cs-gtcHH zpPrQ8W(VhCgYUeHDy*?zeL?n=;hqAC3%HOibG}@Okh5Ei?)3!O0x!5s8*Fg>+nqp83u+0@bC`36}Yj_i6?6=w8xstTue-Nk8)$614>ZaOf zqjhZci;y3nc*P$eDsCAKX)~x-RLa07Ir{8~^Dx>R)2cTQg0`uMUa2;km8G1rrwAyC z;YNhet?J#5Qdo^vAV!v2ARb=j-#&0>4J!^-(WL+<KxIuZ+DH#CX2hTHaEjiHIUK zZwrGRNJaeK-Y~H*C zU7>AMJFDI7ck zI(YeOg8v@__$GG*WJ zOpp$paxu%gy8imk*z8yRvk^&lK2aj`vh)N zMZOE^Gj@oM??InxF(PzGRw%0&VL;EXW_k@biEKIw7Ro_HAZD@}RMEO_LC5DRQK{GJ zSJq?n2Q7J60j{2xVlTX@zsZTt36kdRdaxGlsF=4QPngSfla*hT;i6|=v7A!OLV5GH z8L^f?a{MRs$}ln!KKJ0A8o%$1jw;LL=IqVOwm$WZPdrr^xV1?pDjbA7C&DhikX^i; zp}Vc$tIt;PtenW!B08iL$;SD!1OC}{raApQ$13%5Sik-JxlLKi+v>tTOMI7-!ya(Z zOX@Pc|KL9WrlX6f31Vh+bvZA5Ss%Xk|316DI>K`>L7}1Az~61E-#jkR#K_2kD=VuJ zZ2C0t4~TC^d4Jy9Q3>DP+q#NoAsqpk0gC%`xN4#lqyU;-=R65IZ% zZ7HU-oxWOvc*uk>`r4vi8+5WXs|09n!TpO*YAq_0@ipRL27J>ma z1F^&jmS#p;*UoNyxVH2!jRReYnT{gHX(aHj;(K3=q;}9B;UO`5=8UBS;fAi~@f!=9WKDSz8t5o4;a(mB5J8+Ix62Ys&=V?K; zO%H}cVcv>85_j;)tW}rK);P$t$qo8!Q5?fb>&M*dk1~+;o!|^sUzLaES$gFQafId` zv+wbkq#cAQ5n2ZP-%9!avWtDLGO$rJ_)PL=jj!-#3Kj`#OYya#bAlX(jC8?q_QnTC z-;t~!(AJ9h1;6SuuKCS^AcHzq#R7Z&8m~UUCXa4Ly&XM#u2XfTSPpt!g>D&zFtg{w zYWbPtzwXPE&>01#%yz;`_P@Svyt~~zvrJnKaQBLumOHWUe{{u;9r+i*VJR-?AL4~o zYBM~^<0qX%Sk(m-^33K0xHRgbPx`Jj-2s6UR)ha~x?Y0wI=e!3Z(OyYt4wCA?W%kP z9^YtiJ72;nkKppD{v%ezop-(5_`0`mzw^J-G>aIFm!;t2g;zx}HJ#MX23}vxUV^}~a$qAku9&8S3{QM(dyc!Gepx|zewW(#3u$be zuVY$(R^)B`trcxZ0JisUv;y=?EA~|Wp%u$JMx$>POy5-&%d4J+ziwAvDqsAm6;X5k zrY}x|+-r0r`(jByet4bVdDAns$_xDVvi7({{lsF;t+ORVx9~`szcsEY-IcG9EXDi$ zFD(PeR$TJ@XN#_M+Us=;QZkedY*z%&GVD{95|oP9kV zeWO`nJF!E^vw~pz1z6ZZ&A`n z5b^%WQ=k%9p>4|rWDvHH*72Oa(Pj}NEo1ol(%>XEm{9e@Wm5>eXND^6Z(pc>j1k@6 zd^12<*_Lr_qdm{NpMLGl!78n(uh(+6z&Ga#5zm}ti7?&wgNdrTfU|7JBh8FbS1FYq zy>#Q)ff`qvsu9Ia)Qb)G)A~H`W1b!>B7}rTX(rEo8NJJ5E)Rc%&QKH+!^K0|l5(`hBc==4bCC9BY&<5KQPuW||+A^c2{@n|XcL{<1 zYA0!KMkEfemV8U#0*CkWIpe}$_?T+?sddiYEvyQW>2>FO?=@e~I%4T;mlGC3`%3w! zBBby)eU(DZaX;a0*PbKHcTQPDnn^};1}={}x{ zp_lTYq-E7Ux8UoFsLhS^b6lS6Yl}^x1l60jcV6^EYu#zuxx&PZ;XDSU8css4C~(&_ z-Ja5k#rL4PfZ)Jpf6r~)eM5^k1O3CEOb}{utF2mH4yu0R(}ervS{s(4M}z3tt9e8h z)PpUDSZMk1t=r>5JiWG9)$-x1#cpxOi}a^V*zUpIk5_AGGaWUWq1&qE320U$8gDKo z6UL(jC+SdFo1uD#71u*A{`ZJ?S#*3Ibx!PpEU`K^u<9t~Ozet~&9ba>G7$HKz*E~r zcfY8BMAF?a3;H%zqkBP~J^5boCFsZ^w;PA)8vR0JYG(pE^5S?yfsp)J0ha8P`d+Hf zBeNQBcaVCRgu}7kmz8l&B127u1)rD|6h7mrMDVvh38)}Lx-KskQ`ZPMT$%a- zy1lpYv1e~%7#*q9=P?UssJN>tr{HhtPTQBd8J{{dFUl)U(epvIkTZtrAhJ_mOL5TB z5%6fU{H19)A&*72md_eO0gnHVVjps=*x7kqAKx(~uJU52F93(cE~HqhSf(f$qa<8o zRPF&crE=x^gJId77HL) zIuGs@ct3yh@m@=jv1rvS2RPQnsH}wlqp7yt_;7Q6@t4O5rUBv)zrf-wBfE$^?m8m` zR0KK{GABu-jVeM3BxL*(sbh0HY;CKL_BJobgO_AZm=Xt^^UCbh8OE@eAJ%~ zGA_&`u~AGYzW5%MEd9>m_Pc~l{fRsag=tu$0kfF{b5N$ySXF-h-aUg40jr{)V1?+sMCdDl4s?_uijJae)RM-&C2tr zxKn6?`#3y zVO_o)qmrl&<(_3%tiX?sD^|~4<*%|pg8{5dn0hwydWNL@eph&5!3#epbJWRMl+g1m z7~|SOI$BtR%H@5GM-H!{e&1ihlmv#Z25(?_*LR&Ypp**Fy}L5nby`+$A~;DsSD^^@ zN|xd}j=0ijF1A?Rhm2pTX&HGvg+$t#5O7gAnD9GmJu3Qo{`9-Q1aD<6P!=iTFIF1 zz~vWRTCbtQLbedHc!@I4DYy7FqC3p!;T`H-3SEaH%hXa>!8^?icR--vZ60+qLCN6u zXG@$)7Tph9Q2FQ-l!c<@mJ4h1QwI_Y-92>fSN!0?aghwDper`!*I)+yQ~ZsXkSEUv zf(49Jf)Ch!fEs1VWTw?1I5Cy=}t$!!UZ1uv3t7shNX9vzi-?jXdv%CBjf3z8)cxJnd)esE!Lt^d=Su$1Z%n7N+ZSwpWdKWnDMgcDC`_LrY_-f2Rggd$o>zEIawLcIN*|gLPZ*hIRy=-r>;o$juc?TM8jZ!}ixA}! z7iJ6Y%C+l>KerWu+kd5Nav2ii=;j2(`3;qU$sDbkp1wMR@hLti=$?^ZE^4${Q|ysd zE9M^MP<-`76Mr+I<#LT+V87mHhvBwm8r4;&Difw^d(Wdm>24syEBP~IFMXduyPnQ% zs-APDUR^UbPK-iEFw+POg<>c?#AXCub10n?oSX*_T@c+NP6L7Nao_Y4#?bjC;eGj4 zqu-y`crwo?(W&ttPwtcF0v-0ydR-=Vu=1@lcTzGG%5of%H$vk3pj}4IDpWqcQ6{`e zIy9`t-pzW>FKp9<%Kr|K&o%u9I{G5}^v-8yKdWOI41i5|wL zUj&@nH7ZAl1)38tD^Ac)?2K(;q>mRjJS z_Z^$OOr}J730G7-Dic1w8&k4cWaRcFWqwr`4QaS``R;JTx3pI;%p}-FtYAK-s?du| z!vwFT4&Nok7(XVAIG-MPxv~;IC;N6_;18;h20XWFasBuPeSMP(sNnK4$sWRDCgP06iXiIN)Jxk_&lcRfR{RrnW1bI6^J%KxUWF{~(M% z_#JJiKT}Y$-5>F%0@b?)Y2cZy7TW5-H^?6YS^QJNmC-vzzF3_3PzU^pn-NSk%MfhfX|0~4 zlG?hjP!240hz?r3oatUQTT$Vm_Kqq0z2ayw$Fv91DK7!dBrn{PWQELN(Aj&grg8>l z2`RPY=TTVS)~#cnU`p+>jYm#Lk61$WrGBCvDgcZUk7&o%#prVcNy65AVR9M| zGgSz3_8?yv@-Hh1RFhbr1KK;9pjIwF!_9N_Xi88++5ubnS!ot7d2yu(fN@<@%ydJ+ z4U+>^&p4J$nj(&ad|P7+Pb?cr3e-N!GKfDKI)Mmg6={C-=u2 z?sF*`1|i!(+^_Ho{MeM~i`#hDT9YT)Te#GV!QKS{l69)lTV(iVG!HTabu2Xmb_>zl z1-bhgX;-Z)bvt#JIpx&jgGA@`8RhDVb%j{dKf_-Txj&nZQDTM{m~!baIG%kH^-7oM zChxjWOWS4K!@67W1N88vLBi~)aqh0fWS#@!yd%{QeS3-b{Ju~XZ|ujnjYh2#aTy3* z=12wC{B-xIX>B6`i+7FItHS%4WES64n46g&AdBTCxz&d{0YV!ylW94dsKDL0)Uu0(lB6wv7tW+0CF*U{1DYvr}R3sPz#2H$W^10MG#ZF|OJb5JGd4u94 z6G)z^8Xwq$p<4^N%LW}Q*0EsbmdwWm_MRITN~F4A>}L-ZoB*xJPldS=dDb|N-fs3ndMz-r3efzFw0#ZLlNJIen;6HMlOD% zEWJX@gc_X8=&8y_xNT1k_Z;Zgt2$OGT7xO{%1#C-R3S5Dbov#hIG^R}`HaMKKiZG` zUd2Rg!HM#dOC7OFVz_fz)StwA5+WAX@BT#JJ1S)qeJrXjzbiIWLy>Gkm2v_}tC25k zA=Tyh3#E6c78fh4z%un4f=`m;3}xK9pcj3(M<;JOWgP5m1CCrX=59pTa|vj)L{HmWE>lu zE3E=W5uZ+iYkq)SmUUCcbC1OgaUi$$-iT_e4rbKvZXBw32e*I=|e zW>UDPACBY~E%CkX?sHURf??+1zPm)gm6OrmXZjpL)}#hk>ExIQ{-!?~6POavhYU6S z7Y}~i?!aofyexKRSWDErrN+#i^|?vp=1lwgm-ZCT+Vd6SF;A(&wMDZq zS%=S=_kOXDyjv+GrE`01Wmvyiz+FG=T^Pi)pyUyQKou>QBUye51n?g)K4Lm7);>P* zOYbWfSJDqDs=~_t4#J*I!k`W3f!-a2ApgRZ`~f0EQL0vdP9}eInkV4f)e%yP;~ak> z_!!!4tuj+2n`|G97-T+y8Fge@k(DbK`7CB5dm8m-Yb6bkLxmye>)-?E+o#F7$mXhE zIvcFF%N@NjJY56V@#~x4qq&rwG)P4XC4yF;hQuYWAj(?13Z(-IRyVk-(QZgZU(t!DgNti;o%|PgioEM8*_a<-jTpFle-AaJg8Kz1~1)U!}zZ z=9UcbL7`e@JveBK)jE{T$TH`RJgTcdljGdpI%HImVy8Bbu}&kI$SA*|A!1$td1PMP zZvb7Zg|%L53pcg2mUy*$Ad(eZ$VxAYOs^d6^|j3b$KpajxFPahRn`%LkdwdS^8Q`| zp)Z#}3}ecJR+$2>3XILHJ|aX^btzG6{?ya1@O|Oz^d~nE@EUX?N^Ar%i3v4{)c@H7 z_tc&@_EhfPE?8kK3tQI8)(_hIGVEGGGvEX!B^8#F0qp!~pPA1l^*k4^gaEmkB*PWBcm%HQkq9`@&wtg0azWALx9?`{04qGbX6rym?X760hna{!#R~)1eW4h_bat+FD}P;<86=3!Uwg zB6N04TKR^RsFRL0oSu_>u%&azY(YWxjRJO4r&4ZC0={~n3ZHD38@k-L9~A9bK4U)z zENY5Bg<2f-=YL{vXj`I$Weiz}96;c{%!*%XUg%3z4y)>_WGtQTB=PUq*Rd)$_vCpBX z^{bdpCc7?5p|L-Wp18z@$P;SAXK1v5ZhtwuPCHVJloog%GlW}@4haIlQbJ8lz^~$O zy{Bg_H16OIaM$Ow`yL+w6G5OhF~Jl~@=Zu!A3OKg7T(O?u3G?SlQ4Q+cyY=1^_ObU zm3Pgjbt~O`Kp;F|vk9_cu3pOh(uw-c+LgosLyK>{wI8Z!MM^}jxiRw9eOfqH4Eg+; zwdu4_N&n3kElZf(vmFtD7s12tbrzKMZdb_&?s30_9_!F(zj}Grpn;lLr$q9ab210z zHd4a^JJ`KbsUWQZN}icBLDYV?8;WdaXScy@u#%{%Aw-8{cdJ-Lb-&#E$?$U7uq)9+ zzANIq=Pb}uQ9ViJLhJp^YXven44y@NZ>!2o+6kW13lss~<03KAAKkSh&?znR`y1fI zkrvgXWkD|n|F;W2b4mWcsC)0Irn+@sIDqIYMMQcNrAn6)dJ&}~ASDSMr9&vvJ17c5 zARr(pC4h8704V|KO7BPu9i;c(5%62M_df4F_nz~8XME%SbJrgk8A;ZfbFHlT%xA9W zSLTjVOZbHCi@J0Ek!0VRs# z?^;WS#eIYE)@y24Wr_k-sG7J(?W<0ont zeqOIS!W6_~(8*cufyx{$nrC_|_Wr^u*UUnyRo%Ce!jH6JtU+)wOE05SAe!iOUj1)) zkKvyDp(Ek}aBcLs7n@q5r&x*hMKpmq{Zd`wKjMjI3oroi!B?+Mw}dmh*M{`OfE;rsm)5n-Q{Vda?Xese6%jgK9i(X zINb0zh)0Qd$lG(Z;AwIji|O;bRp6OB^X66FHe$yn@CJ*go^kW+8%xFcxlVT3p8=6? z88e}RdHs0r9lmU#}O4sL}ap*Dhr2cm|VEQX_13jx_~M|#RUez!|r z0bK#XD{0b$P1$X!WjSP`Ec5d6{z#>$Ue3J;q8dC+Ap82sti}D(DpWI9s6LV>KQTe!|7EqOMIWL!z%pHqojcqZp>*SkZo|ZX0cR;DIX9gI zcBvax^@_^p9Fo|Lc6SY)v%VmePMVlMUH)mJ)_L9BuD()bdQLgTzEO0lm_RsD z14CA+KA%m~M3p*!-2K?@tT-$zP*ata#qc!8v3DOfJ`DnjTO>*zXVpw`gt`>0P}$1K zRQ+M*J#*wlI(^y4pEt<&>C~%u5*=8VvBKH$ItC5y`X5A!-I?g$<04?QOpH&J%CZ!F z&(uLod#jf#iS&gUW^Zg-n9rX-9Cv+;7~g4`${A)322J?)5yJDz(FcV5oCrGG~mlatlS-5=ViA3;HaUNI5!lzq;| zsL}7QggazyZ9bX@sWnoJ>0=n;y12Sb0%^u_&OXY34tHy%^u%8`S^KXXeJ+yCuRNmMG{S) zBuj1&o*NwFNK)~(12QB!pUJRRa>NnddGv(MK6(nBhd5BapLZflv`e%lM3Ter{UWrJ z=d*J63Wt9nCmJtqN2_%-D7H-UJTiRdx*4)oGJ)YV!8DD2jGV)AKRd5Z1Xci=M0?KG zsA2OUz7n&R7_*H!iTA_P_jjIXH|X@C2As_?`+TP;r{kstzfi$4lZixfPdL)JN@Qn5Ywk# zOQ8-8&0HANVvd z`=wT{I>QMMMc8_8u*C)*=gz{_rU1RrM<%8@v52$ z&n_jzFQV>U?(`Qj!kPCo1E#UjiJvdt)ASQLw)zcuxqmE|l=XhMmLTKN{&9`cF@rC~j2yd?4($2zO)oEc9ah*rI&$Jm@(I*u+zN~Q>6F0O_ z23;6R)4rfNti;P~O70?YUDB~Kz)r?;^JMfo4N`VVGH(B6fFIf2h$B>v zf4|n6A@$rm7nka8?rHsp{cg*`2B?k`s|$V32XuS^i&i|Up;>-B5?Ele)=JFwe040! z9a<|}TsDmrUtC`aqzQ81nRi-JGCO=d7-#_$8q5}W052cfS3tkQ?NKkM0-E5Z+)Z=xvVx(_3kHy zU`J-IKO<_d(lp`Y3Qf3#WLN1-29>fMp^gBaQQYgDLV07K-g01k^0qL!mru$kiI-;Z4OS6hP9; zCuOL1l0=S+Qr@qlM=!dbb1y=8aPFTr46@(_l7QJ%Ub#(8#n*jE(C3@J$p>q&YI@kZ zQSC!9aKfh+n|n=`LfX(8u%{M<^zJHxHjXNC}eNJZ(eUeQvx^z$nd#J$*Ra0A1q@8?|0? z*YwS(^vYK5qUm~f()xO1mfz|$_3r8~0na_5>hGLw_!!GcypR^SqdbYeteGWu zuvEH%%0Y^&JJ?4n1LEtS>IB|?cBGHPWm5{V!rZ5jqcK-4{-+8M0ck9*+J-8nIJDhz zuE5Frj%}>7ejBzI>JV*Oc>GBC!5=d(E`?pQ-ykO!f=6~yqrR}|(msl$p3BD9?yo>( z?7t2LhcL#KLW>_0_V?;H;tG#g#6uL`H(v6Gg9e{2v=AE*fFPPS) zd%k=lw9!WPP`JIOi;@@^|BOT>pQWyPN2C?3Q-=fp21yNT3lDsJ4kGkvm4fEI9Fem| zB|cA;JzPxS1e|3z1Zk`uzsO!rFyVU@YhZlJ9Eo18>#B)|vH?yQLeH;`^q*{|2>NcN zH})k|hDF@RpTQsv;5-#E~C(iN5z>Hk6Ky^;PWR7`vOaNi5@ z&0JaO!w-hyT1iH?i4tF3_5W(i2G3MSlLV=Ls_#u8i!Cs`up@EG82-crX5w-^8YqX_ ztXRZrL&7fN16*@oz^Rq6ofYeC=9pa#lJFWfWc$$-GnAWOdLbt>J>dgUov7vpSaaq^nyyk5pgZv>o&NIfR~^bIkgUDJ0!ke z)lhLEaP$0{S1Exse1=di(Y`@W6Ox?IBj_>fNuW^y0xY-;;NnOcRGmTRSR%xTVGvss zqa8K}ryrNDdMs^5|JdPss@aDG+OzjL;=@_oiwDi8`m+jkQ8p^k znj?*=JqSZ_vReJIdGJ z2;4Z?QB(!8)ghR%=29hz?d?rxCE9_Os@n?!CmCxY&A*w;q`>=i5r?D&q#CDu~)4ibIOp zj()8-_zsAhiFs=mbKZBO|5Y9G%!6l&l;?1Bn${r!J1CA>pfwKFUXpg5OS0a913rJH|%IKyBBFOL0Qlq zt$gGv;T!*`_k$9{)V7IFY4Px5!lA2sli-FEZ2GyeyGkVRUiz~D1pA;Po) zE;pE6z!NA3+d)oDg2`xcuu4k!N*=}F{x5|LF$C>f&F1c)WrBliCrj8(H{&5F+99vb zHZNV?T`A5)kRzjzf_B`~6>goW?MKJvIMpzjwQ-jruNT#C|9O)gcEU>xDBM})|9TUY z4b{paL?YVcBU-gT<2zE>?>6>@BS(*cU`8#)s?M&-@L}2Eiy8tGBSm91N@fdkH!LKr z9>k%-PI8d_?ZX?;I#E6Ryr1mh1?E_e6x6QpJbtXeu&8`uxQWjU56ELaG6-^R>yA&ApZjiD6Sp+))* z$40Ce>-Yko(nVk6G=&gI^1l(*#CCE{XQN8*n|o91`Eo;-N=nkATRO^d$xkLxS* ze_$bP6QmRrpzmwH1G=17TQGc@#o~Z>6lRty^25Zx+iy;9DI#;CIbF?tt6;D?Dc65a z=-HUf!T9jl#;Joz{Pq7Ka&p&R^PVu?>*jfvytu{XZ2Ui~+1}4hb0Nk~D@cfpDAvFGN4e>(i< z#eT*;)kc77gIX*zXc8Np8x$V*G-^ygyC+AyGQI4>tp48Ns(NP_)=}o3o9s211L9iA zS(zw@J%s}M8_I7#c$>pl0e)4zd^*Y8Hsn$dMwSsG4 z?(c@wdu=2d$nb4&cq!v6plt%^udICu@8ahrbqtq_m;kBsr%9Hcoh|&1f-QA~DY-;# zAB)vY^%M*2RtXY!SZXYvu}2S`nLapE6&}S2QeYjL!!NnB)eFF2jR1xNr;kAa*&o}E z)mfUSiN1hW7RRQ=@@OFX_w%7dKPnT%%^e)Hd@vgC|5RJ9&YbL4dU)?(RZb?AaXBxb zV_c2K*k8Nx8jFF>)^l^s`s=V&cv+8F;W&A&1Z`PcqjcUQ0u{<5!Wuw(kNZ@M=m`;@b^GuDi?O0+J81j_#fe$rrC z6lFbZ5A0Xnm4nT`7&;T?Qrd)XX0q+c#OGM10>Y4hT*)Tm^uv#~n@sUNi$(=lH=n{9 zt4zg!5Nq%FXVu^JSM((LRnpo}mW6&03Nm_MF1(=I8IxAZ$11-bXD^2Zq95=4MI;xN zx?U@E>V{`)-vfI$;BDvXBjKZu(pE_AwK@6J%+17;Ei5@NNL`8(bj0@ne3&;_ZNj%7 z$&5dlnl}r4eWO3@9~s%RuUhk8_X`)i9P~$7Z8HC&u?4-1)t^@|xQoketidzg745OH z#UzP(5BsXsMQI&}QcVBKzucwCdF6%eab>}3Yx_}=rIT=EhnLxPPik%i$~M8~^oK}& zXN$0NvQkR(`Ew#V3wDpo*gUI-#ikWq;~M@-F*{EZ=*=Vh*PEU2mfxU`jPtcu|M4#R zU*Fv&sc4w%9AZ-_RNdw4(;R#QV>1y=H?>8*%@WL zl>jsf5ck~v4T>N-4l{n1iAK+TZu?P2GVH|GUh41}RjSb38e#TMTB4E^J9LJE`j@W{ zC^Oc<$r`seKfdNIpnqbDjmpi>5$}52Ia(;a+Y}1LJY(TauCsbyYHX zTYB~Pf!h9Vli=?I{ac%4v*fw{DZS_2K@NYn`TXzx{o4ur+hKcDhUi26=$izR+f=QS zml{NN(A;gjw#VEX=pX!bR8zz#^a+>EYLObdp1lSU3t-^)rlW!2ZtROg)_yATv5t>c z8u4Nl|HPVYUt!Ju3alMiMZ4FX8%od#$1oF&l(3s`y~Vkgt1~E-JRC5Vb<S`j=_?8xj7 z+3Zp=?(b!~VPYN{!%@nWg))mzChaS37djVm+dcIc(1-R+=(y{x36H7m@h0hpjU`R( z$Kjk?dk?0#bwVV?0|`A>ig=4fOQ=7MnBH#mC|{+>DE?@+e#vaF&}z>$$gZ2ef)1~l zGh1!lw{;mKN|dB>){AEgZq|qu%S9Gxbad#SR21J-lG4(!i3C$r6HgQ}i?ke;GI;$f zb0bOPjLbD%YtoinfTUyaojvU#BlvdWPDG~h&b1t)MHvr*~OkrS{WQeT}6}yWR z<0`#V92+j%-uWy8a$sg(;JLIOGE*9Pwo}+wRS35SqLiVX+!8CXGCqUuc5&2FO1n=C zSHthWVfMiBY=wQuRZNfz4<8@c9rTIJ35tAg);Y)4I_5-x#za)7GcbY}nb}-&w*3rK z&x*dui$`0Rm2+e(_0gl}7strk)z^*kr+)-xDV`?><)rT1#Z?vCOB5+8Zs?i(SOCqk zImU{$E=(2KV*}X<^iDKde0`17ii8|Ebmj*ZN?F&HqRd&l1jRS{jwG3-e}m5M-DN24 zy;OfkcXcLSqX4T2V##Yf?1Oo;)}v-u+#~euVnD_CxS$GF;`N9x>h5ktSbPz*Rp_Q0 z#_LxLL`8w13z7n6d$FdiB7uv`%HDFK zZ9O60XboG1~) zo27N_X@dj~Ry{91BL~#u@>;=SFiC5c`!|1#W5J?un5t=K*JErROa8V)rTSh zt1b{2IKsKBpCFUa2PeN028`WrXL%$6%+ogF@V0m8?Mrnfn<>ChhM*}_ex~E){?+9G z|Kg}|d8cPOG&UK~Wf-`{WuVI90M*tu4LdZooC1soly{{J?Yvs9OYSchSF`A#sP*eP za^dP-TGp=B?;_m=7QgISIOO1Y7P7o91>t?)Q>J%1#^!IMH?C2H9l@m@+v_>(qKjlr zVi@1nW2;e{H_jByIx5fXw(&OMNk^GU3wow4ic_~9-4d`cZ3A~yOHfhC5)$Zn>$aqr zz8};zFPtm&O!=4{6J69=6*j^`ZT5dKoV_OMKi3|Rgg%{Ee{smz6;r*g!}h*qopJk) z!yNqoE0|u>GTsf+Tvvgg+j5=)ymx!*%Bs?`h0vtiqm%( zaVs!bWNGCSa>nVUklg<~-sAni<{as~*{^pB_7D#SrXXDwJ z_(qj=cpd_W)wLYcBbNGm8UVY6`DQ0xxW)ALM-3~)Ki417G7zSIkkogZnmS_sG6!I8 zlxbfK6^!+bSv5G69OM`>w$UHndQ8jP*s*iFiZvCC(7RN|q5&(w;>k{wt=X$3QDgS_ z_bqJ2CZ7N=tNJ|_r$Bdkw(3m7N57;B_I#{a2lw}qjl@>_eiU%7G#%IhIB;@Nh3n`a zyJKyGSQ)OR5GFnYs0uR<(Iyc=kM_4+eEh6HJQ{De)#BeexYS zqL;s5;<&G27j}kd*0)5$isRmoJ8I+$Cc(*UxZcMYk*Jn}Wj;ikwu;T_2o0pHDN5-Y zp!m?}ZyL>{0T9>XZhm4a)_5Rg7V+Q9;GgU~{={TF&ACFGoRFW-)@5OMpgLwR0(l-y z*tg?9lZiUWoVV*8*)lYi9XZ*wi}2qS{#<#z!@jiW98nkLTt6(aOJVdRv;5cgfhu33 zX$&;nk6KaV*y~$J;4Oav;Z>X7lj<*e?w;6oJ;V~s*erMfql^L5G2Dv?T5-H!VLyh2 zodn*u!(6v`n2^^el}45Dcr?Ki9I)L7kwP zW>-vcj;%~fS2c)ZT>1x;@_&(w7fmLc} zJ2rcYEgT|6&<*gZV$SR zY#(Y3u4acGr_+vgv3VO`r+BW%C`LK_8h02qELlI|DZKcv7n0-d;r+0#V!-^149j8x zrd*B9A3A_2OzjFB%=?s!Z-hKW5mlnqV&%;_Iv569M>rN1F#blcx0y`2Ic2T%;Zua0 zxacJFvoX?|){Cv|GwC43>soQGy2TzZFfP(cvAhe1U_SAZ0#hrsE(5C7JHmyv6ZD6T zM82uh&Me*6p+v&jG7J7CY|9p5qQM)ty-fPcTy078V{36ao{Cob^n!^Qg#TObEwN5Sy^KwL#FY9D& zN3}>t1YPFWv_kiqxi?Hc`y*52Tu&o-gD5Dl+4H$>bQ(`u)nI$0K7thnuht4#w(_pG zy}EIugVC;L8kbvN@a489>lkM9X1;b=EBfoHr-JL9Gu)b2IrA!c5Xa1FZtV?h5qvxf z9s{i3Xxt@0;&n(r#x{aU9Evg5re7s+~XdSXE3&*T1uF@rNOZ@B_O%QuV zSs#!Af>NB?gGy#`KEFZxG#m7NEdfuQpRHlr4I2$xWrkVIVWo)Zv6_v`@<&n!Fe)q# zl~kt&HS;BO1p{ecqU?inTa)n{tU;VD7ckFD{-+*^F5m>USpKbO>&Mrdu96neknGBg zgSgxerh~4^kVHm*W4XcZ*aZ?-m`T5-f|+%?9~M{D^iyP7&C`nKcy7dxTd9M7l^Vv; zqxaY#IMs;usoq|l8QX_+zsWlEvLKZnN&e}t%<1-%v(9};fnTG{6K zwh%cT-*G1UFms*|)hBlu0`hjP0Ki*cgMblFAAf@uJFToc>+(e_%T+&b_G_^!2HW0- zF42tF$}jj?IYj``qS+&DAU>%Vb7HX6U%Fm?KTWyQ?>zMZs)36!9NYCM zi6(G(^#3n94WFBqrYY}ont4^=Z)sFu6pE}YAy96o%^}o@OUI>VS^b;(Z)CS5S3ykZ41_EMH zodaO)BQfaV!*Xfyh2J1fgBkgm=YQ|<_mS~t+c-}pd(UqDB6Mys<(L|();>9P57oUp zTt;!`)32ReP9*mfZlaNmIT(%n@P(ylJ7P@XH|Rr)m{v%Nb_ZNMK`P0&natc{DqJjY zTZPydCo65zK8QOLZ6S(~`SdgXM_XwP$|o3(F%;R)2rL%QUI09)(Y-RJF6!-9hKZ0A zlvUMiMfC7*(7>~#QRI^%a7(*D+p(-oW7`y<6UQrR-D>#>9l76Btq_s z{&1MWm+>G){+Vi?org~XSBk%VzdbCrg*Yd<{pX`c|Ks=MYP<2?vDlr|g4kf;)WE~- z=iDUt%Y!^dTlACLK;0U#ZR+&#R~6}OivW2o)0*ONcV1}@%gE!%wX{?AOV~#oUSX0l z6yEKLpKWZKyp4`DklbIXMF{7~zOXGEZ!b$VxWUPaJclcUz@0-2t6eN;_g;QE5K-ajROU*Ovp6_Y3AK`Wy76*ulX}p16xN zxaqlVMJj(Jn5XiLJZGY~esL2*@b&88H=s78;obcSsXCFO1yU6f3=PM*wb}PSY`Ol} z?<|E>#^hdvw#N#}h!7M-yQPK~aLW!)A{#b%CKzabJl<=1{$-S&FE=8SRBiL}t=*Mc zk{{&J)<3%q0&!=sp!2TF#DW>ySOJ~OsXt#Rc)zXAAmy_=b161zKMLXqTfHN<>dJD` zb_-+wZqb*5DK)8ke!l8+NV2=VviVT;G+gW#BiVVCNqqg@kdrkp`w9teuH`Y!4s>&lABLf?%x9%zd;;b(OFuZ zLf$3BiLsb2De2m&BA>u3q6YWYYjT)>kzlwD2~VV@W1|XqGse+{JNC#gRp`9;$NTP@ za+o^$-T*Ewm;SIe+Gfu-Du(s)Ba1-f%eB_Lz=Wbrm9aa=Qb(&4c9H^j+Fz-^0@?#W z3CFZfCao`LWSo#y>ju$vRU9Gf3=?&sEE|O!zEHc|fs>m#F?*^Uq6L6uXjQLUrXo

Y6a_7Rryy&{ja09OED0tE%d;}&($S8X*&)nN}gqYB^ zW0D9S8ON8lmUu7ZS7e~;CBuku6Fma7gRwm1-%CTdnw?k;a;>iX!b%dqc-+&??d^jRnc7xUK>k(klUZ#ZQNRK(DfS(QC#$wT|Du!^1S%M;s3nA zFpa2e2O2E?!tx=|TwW{ojwfvo6QZR-Wwo+C#71HijX;-xArSL_kX7gBhl98lqkeY8$Q8B=Z zC?6ST4jUivrO=9Og?d=agkeexx}*7==zh9M`l;et7JX%t=&~0Uyd+ug);8jD-yj;{ zMs7<3BTg#3`u{g#1&5P=2+((c?$(17c=qPSLTnpLN$Mz>U=Y183>f zu6&+t3xq&Ex~0z^#$EGS>=B8N(Sdo4Ecd*Q!eE{b>D^l8nG0aI0y8i9WdTmYIML&$HC?SVc-H z-T`djE1D$&b-N2e4el2u{ZRW1w>k>1hFQOnLN%LUv!26uY0LOKS9dDj>2a~S<)nH~ zGvG$G?uEXiR6dMRzUm&S_180OSMf^ z*9T@^$v^lp+)KK0PF>J#xo(y2AJe4GW)5EK#dM>BI zVHKvDpg)H-=vS#jx3v~M;(vbC56b$u<3L=X^}CgCrs+h=h+3X5c!0*cB;wn{*@R%r z{f0tq$gho7JdytMhXaN(kpbLx;6A3YBmahd|EKqS3uPB~2h}FM^dv|u)`R+-#c5PP z7DA)OP-`>J9^R| zj|*~2*3hlLL8Xs4L>}n0>4s)YZUon=mb(aw!gVY2j(~r^V54<*50+XZiZ!r?oKHZ$ z$$R6;ABwwa!aOIi(yND7f3k?%g*S?9hK1}!dsN?Zh_Rd%LaR)AksRqD@(Zhc)`2AJ zOINibRJ=fyJ+g zSheF!;Q>}Jh)Eo3LD=Y}o6h=OcI)EIxm;wBAB$mJp;KA1mp&KEY?Eugx6 ze)6MKNCuvB_p-Tj_-sh@C@@FSKz#U2h*U8hzhN%JxoJxIj5|Ptt7w@BY{HRQUBc2w zFHga|E%*zaOu6fiT3arL4kU$45vL~kU&@iR70c^XKRq~J{D4dacU{*kuq!`Q$$#zw zHdj!3qDCz?2PY?ks)aZt$WAfvDnBu&X<-SnS}l!-?_$pk%DqA*h<)9{_3R7c@2ER} zmI}D^SU6K$oFaJJ5Tr4`4Mh$JMQ$7POFxZ~NvsTDm6h0U%v&=gzaVI{*B&fD4n_Jg zQQQ$VHRtKqmUjK@tD#PupBCH&tzzuSw6h7eHI7zxYB4X-%jK77bF_F{TA1-;b!0Tc z2gQ)mHl(qffIe_M=M>al8H9?%Tm0z3(M^k7U3`>)TgqC}jl?;O8P)nkQTB4OZ9<H|F&-ZLKR(gvU@(Nl{TQfcA}Zp@#JXEnt zR~3k(jMwD{bk>Lyk8X~ z7Ed+dSd{=U?l0~XH6+d7wToCMkHv-j1|?f`%n;bboP(R|VO84(4i|2(%8_n8FZqx1 zM#gIEmGC(AO~6Z+5r;d$q%IdKfm-^Z;wZe7Mg`n zrVNFpx_f?dELOlKL#$jA`Nal9KinQoc$3JaZ|z$Oc4INns|sA*g>_7dtw&QM>@=)> zv#rM`RHC>C&1?f%;i@Ydvchx*!=H{=N;`{%q_JOZUU#ozUkadfvRZVSb<;A+n||Ok!sr5Bm|v<4Ae=n$ z8}f&rDW_U2)KFbTsjWYEU}B_F}c@a*WTTa2na%MF5|C( zNUV*B5#7UTRou_UL+(baEIJFhaq2vv@~rz|C=?ahH(u~6n{rntX15*43DK(SJM;3@ ze?i=FG7$ZlBb95dAw>FXFNbnX-owI0qc}Kv>^%?fe!DIp@!ndDG_L+|42xD5OCNrD zI||QYM(Z7=m9Im^MxUgw@w<1#(p^)8==%nyB{_JA$kSK+=Am-}e+i9Gm^1~|&*ev1 z{bdpBY{OD_cTvv*P#=7Et2`$rM0V+iW}0J~y-Au>=RE1waZ&9UZ0$2}9>9i|7tmin z`Z6e8YBhF~pizS&ssF?we$7p`d8Z&t54A~H%42lmUgiflo`+~ z9Y6@0GR3*F+&dMmyC^sVL`{=CbCrO>Z7mRebDAr|t;V<8(+M2ZyqWeJltb8zK!oMh zd_KTis?sc6D}!K(*@Lfuyd8gly!fm?C**yEqffK$`%C{wlDtGVjaR^ucG2zAw!c*VznEHhlk4e`2n%l{CyXGg-)HEGf z&Oy4JgnE)H9JC9;2mpmMpfDoKZpOfvU!(a01Yk0p)fPRJVt>U)AS0x<8aJmGEVLsx zU0>houj{LZ--;RURXx@mIShzTKI-bEeGyEpNCJRbSiq6ytcF8ccUbTa{$K5v)qRK2 zhdf>8&6q2V2*1f1R>M^AeMLDfTtcm-5MAnnB^Uz|hWifn%+v5bQTY zjtngmJ={{5R+TjxDb(D2v8k?oik=yMK>w>T+V*Qhyd3UpNhHcRh*JwnRY_f!xgaNX z19$($&&xgYh9~bYtLK8s%d@9 z=+;MoKg1RXO#grQLu0w9Km4KF{}RW$x{dye?Z5NcYB|dvC_!UAuJhfED9sp;xV&e{ z(miXYq+YVHCYc2<+hFU~Pii1O9wVbcZ*neD3mkr}|3ODoX=j9)02e+HAp#^+>+ZIU zd72rA3Nb;lQx+8Z=BYaKkP29u^G6+HSZE4a>d@{JVhS%=Wmft$`uq_@mwD-j*lF28 zUKor=M~Qz0Jv{bk#7WTQPV!oV=G?MFk>6m0wfPn<6BZaw8k(P1ela#d__XKdq8>zL zu7qK-#-CK@87^=otwCSl$HJL)ms#)Z&M*)Z@0()aLlHdvHRq2Wo<-b93E7+5duRx! zu{m&>GTMv7l4;*0OW>6^r0J3FZ=s;@lG#$as_N#EEA2AKS2!U(*!i75t_QF{2$PPU zN7?Ty^J-~~vHpi2$d%Y=K`EzwOw|eUbNUzTOkUnWSPlPxQ)(<;aZ;?U&3qAyt)*p5 z{j`kX5jT% zW9z$M6?IYn67xD+Gm(8|fCD?jI*Ekq&On~AYVY1qV4a}1r#GtdROUG{KiAw~>+fKU zw3zoZZe!sw9-gA2X+^vOG4qZWXx`|v_x>CbUX#WegVaLmt+!eWCaP&vMq!x{Ys=tL z;Sxwe>Fedr3RucTK6-(@ zbwik7v1Q?TIgY7!_27!K(%r1;Akgj2N^NS>d8FBW{0)<$#*cGk)ik8P zLFyk@AI5)NtNDs-hEr~EBoEqF2#eChKxOOS1f4J00N|(M>;k?nM1s%}h0ijeGk}dG z5ud190#v#T+q!>TwiB=XQRxz>mJdt16@6u4{Iqz-f41*e-3*TM-X&QIKy3D^Ictp1 z@p~s}q~RjGQN)ygtw;LkYCJekn(hZG7Fax0+EA!j*t=u*2*vV*yR`_)&Ds#*D zgVDKoDCNPl5>USS86PW{LuMw3P9~~iKVCd{pZT|$;Q;Z7 zKdT=>!@+US?0t0A-;V!v#e}fCwycyCW~fy5&$NZ&GZo zp$u_caGCzccb|F_A=LvX4ytV5zr zJ0xj%s5U0B=sewAh0ZC%&;LcSkGRx773{t~H9r(MQf8K}e})BnRn1R`n)NiCAt_I# zLoRiYHH18mKzQ1=eO`_(OK3nbY;-9*QT>E)#8 z1HCibXQ*D&(Xv2iEewIPCTWyt-8P&)DsZw_Vcn!gjNsGNtx9C3`l|sTkv{=Q zF9ti)_v0b`Di{+~S>2-Uy;fG*R!$3Ta=EX!p5tk_WC{vD6gfC#HVG25ik~+Ry#CYp zEALZU>AS)-MB7Y~C5;g$YRzNDevg3e;Aj0jV&ID?DNMMyxy~Lr8T+lY#GtnYvTo=2 z3It2BFP4d-vzZO1>0mi7>Ne+z_bj6m7|2^G&q=iy&s&;X%LJ;4h?skHSH}X_FYMpt z>d#g7&(ky($3dTW|Eu+Pd!XMshOm&ekYApjhqmjcb@Ii@WNo)-h204*TAI6~k-cu* zO+r`hP1_=m1hv$|%%gJ}Pl<>f?9={2!K`jx5>0imGLr?9sg+-D&5QfN9Gt?Fbwyqs z<{niFN~J85y2r9ZK^Z|ykF&O%F5&qArw4WzGcmu&h5(e6L!I&?9OlJCjVT3+`i_Hkqf2aHKbUk@7MWxw_$3T5Ave zmRyn)>?cQYFWyQ_2dDIwhD$qDLDlJ?Fa&|fyF}LZYV8R04T=DzoCL~QnKfsFmjZa7 zRvllYe42odw1?MuUA!?Zsp2zgywL5}Y-=ubgZfTvQ+~OOE=Q!NO1mq17~HaKC^}+A4?~hG|duF(cpZzj`@#Lb3S_T z6Fu7=gQk>1cx35{((#xlbK!;(6onow37Q`=>Qja62Y_-74rq9^O# zFEq>t^@ivp^m1yBwP&MOg=xkHzVAWcW`+0o%*hJ_asqU0yL-9vB1FFMq@N$ad%xW` zW*bWL60xH0ESPbpsN*g1(>7nT&$oUBBGDux)xCizK<2Gd(~@gPuw>K}c5Y!69tnNJ zyE7sg%O{*&>LVs`M@gaZwH%Snp&%#{z>#QlyqYC#Mj;Z4wXv>2QMUCR+yCL8`pxhW zUa>#rr4{=8LIUcPg=rZH`9nD*H*txniWrcn8CqP~kk{T0|HZPgz)|~xqR}8lNsTdw zJM=M?VpNa!qI2`XWP1!7xGd_7hjG#k%&uZY^%hk!F;S`)WYJyhHiTo&-Ycntm|H9Q z$b!vjy4T-S>owaY*Hg(BH}EIU^F_vu*M6A^^*8|>T8*6oj38Uk;2s|%Hesca)}x}8 zr7AJS$3kr_ZE}xYc54Ip_eq0$^?LOZrWCv$PY6G5E4D-t$0Lp$$A|T}1~aUj3ZHES zAFV6*2nINWPR$1%>s;KG%Lo~h)uONV*xFr_gO3&e;0-WLs+ru~Ocq$0xlls*r#}7- z;!>Ilj!y@H^4x=JTFdsOnU(O5{uw?;D!qa5DFcMhe32umXFs$Kx~XsmT0O62zgu%# z?;HqunAD1Xc0itpZQ7qds8a(bSiwY$hH)?Vmgqp;eJpWCjDC%jjbCSue58_Kq-JXd zOVCTkZ}>^`N~Z4Lbq7BL1e{Oh$TLvvkav8g;UJ^BPC<-pmB)3)TZ9zsF_hNK>tr+r zVPt{P7T13{3abA0ze!hs?3vx2D;SwqZk<`}H%7bFNt_8?fbqY(87>tVhvns$W*lZ@ zgkmvpLtF+Jkjn&wp$KBq=>?|LDVEdDe>$Ij?{8f$*|LCi&%}44>>8Ya@D6NvDsW~` ze#UcMnxBA}s*8Yzl&&qNS5$OcvT0xh(82Ew1C%&?Vp<2E`IURVs%akG?-CH%PR;eR0| z{*K1JPLqGFfSTR%S<5?%)MG&YPvLF!pGBSZe_GW2JD+Ahq;dYsrxn|0uUSJyE{_HN z&}m-%w{#kiV0F`YLgqfC0sRbo9CEkcC5Sty(8YpI^6RB$Du&02&ri8M(31@Fq%B|k z_1&~@<}RZPs(XV2uUU~4$r(%LEE2ja>AJC#6XQXbFIHv&U{F@8N>%q9=G5mnsMyr$ zUGh=wChl_PtI4v}S-Y|YF)TUQ|DrJAI^V;q^9IRmS{N@^(c?1E=%V(pSMqx)CbO+V z%8IgCMa#K}YyQ7MdQsvQ;w?@7AMzz3?rW^=RPNkyqFT%{&%jA4atFn2vO zdhS0*{9j*gbj7n&sSO)I+B;_bo%PmCQd0`P&}syY8&S*a7DJyDiUJ6{m=pT-4GO+fn0r8lRATZw zqoZ*$q&7g^mnvI4Ci-h|0@zIlbaxU`GwQc-sqt zD{ga7kJ8D!zPk3YphiAN`D1aYdm1iLlkVppJ7rF(hN=)iT2sy#SCA{T0V%iJgC5Xi zQdl%!Gzc#`)G4sS8cx}sSmX)FS*`VW)>JGzbav%QBU_^MMy>x z4=j1K`_fEo!-|hgcei0`cus7M)8Ey%LPuBigM!z}rHcVt=HIgC*VAD%m5a zxeLx1)R$dd(@1Y>2+p0VNyx=&4i!`I8b8W;X5n~n0chr)Y*-uXy^qx@_&8%!EkI<= zSva7$3h+|23uG_)#)|?7luJ1?3T?Z1R6KQ3!MrYTxdT@gj56Fv;y|gnruxO9y~|6* z`u8QVDUi=b&$u$x8v~jE39w4i!c9K1P;^TV-Uw6yzP-F?GX_i}y-`XR!!PeGtxBts zgGLane3Qq_$O>(0!RPt_5t7{|@%`BTlbRu-W?J`Nmy$U(R?O@~qRkRW(q8ERGroC< zyoZvJU8VQw)HAY^C_+d)qk?$l9ju9VBtR-zYj#>m5m;0n>!KSIPrLR&I(*(Qx}{of ztczg-k)0<}&av%eHVkJ~%7cueZQq5FHpD^I>8Ub{;{k}UEnN{QoE@rV;fJC4Q!6x4 zc~IUB&s?89a?>Fb2pN(4Bu_Z!uqQ+T$35lS%Q_4TeSm#dq17axejD1*uZ!CpQ{-gq z^G5DCZWI*^ZXFPQJ7)e1QJkkdZ75|%Y_0-(-Z4~uS4+ihyE_;@aYCWI|H22F^eTo?V%FUj$6F&0~z!@XGS6! zIsL8<&HdtXu^KB?3BGg~l|s64Q7oPjtH${TR3j=^j9sJFVgPXQe0lu(*Cf6jxfP;Y zdPmUd;|f25aPq!Xo~4(3*VzxR&)v~WYd#OO^lttNN6F>(_0U*4*xwmh&DDh;7vGUc z4|NKMTNfHz#WlxqqW&_65gqV<)x#D${+N0U+i6qn1?? zJch!i>7$13_1MOf5sinF1bQ<-Kiu*iAiS_iYSe05pk4TYx4lz4MGNM)$gurk^;iZR zU0<;Oc~8L^H)cdAC;#r$b;sSVnb%6SYf$zK5UlmZeK+riLinNaXEeTBPpvZXz% zTx`sgL?+x=;~JsW`X(OoJ>2OhCn;frV0$IQ>@h!!a%Cq?rQbLz6@Px-vJOi)_*Io> z>DH=2klu(-DS)Gb0XV8x@kyfcD@-qvu84rxFUEE&ibvMJ4d55Clf(htCNFrjZ4&7k zJ6A2K06j2udiwn{+8aLV|8slzQ8e^(m*A%)0z=1z@I@0=sN7(;daO!;-VxW@c;)@x z3$boWAO))+txv+Hbma&W{cBqz>ks+Lt zqg?by#K7=AzfQMu%Et^wjW}C9ONVlkt*znezd-lmc|JDuf1i6cPfwW+P7&ww9z33? zf0RSW%b?{}@nsd6m(AVQ5U&;Vz~Lmja(gH&TvDaXtgc@gUuu_2&1}HyUYc=UOa37v z(kDotoW-EDge#q@H;*F$oInO&W1VWciqyLn>9dd6IO`$E=^tQK4RSQB(n1`*Lq~+^Up~-gq5D z!to#m`;uXi@pNoNqu&uK!TnX4=4>kI;7wI00s14>i`In+<%k(XZ)0uR5A_ur8wMV~ zPBtl4;FUd2lq&EBMh1oKGB+y@$HeGjV}U;ZXT$x&pc&Ip5;9;M*b=WS7*>-E1Iz>W zDpxHXcc#@8()Nm3CgO|pMm%gc#C#5L>c0U48HF&W({QI^T$AL}Dy}PG=P}EMf|n|s z+c==rqD#&)fG_}$yG+CGC*K+7TZV773s-Ug(m0>-5$F2A9oPfu2;k=%h-da2&v7&bW&&)sEC++B0g+rD zOnl?5OfthTmWp%;&}udoL(y^S%Be8V z&Fa&X!hLbPs9A(JJu>KWfGhVNXe9{EMxQCH!O+^swi~AQKIoLJzDVFuIc2tv?I?c` z4|%0`H}qa=gC78{Uy&H?zI&~>142nnk|1`;+Ljs2V`N!~ERr1MAxrM{q0eu3c?Tjp zVub^{G$7Kx`l*X9EpON5juBHzYM<5t|MC?7zd&qXx;#lu=nz5&eoFkSXHTwJ)&=;~rI!kU*i7s_ zcGe<*HGkUzp1QS{JY!P&eRbU}$W2>HGa3tDjQ#QhFVnFYa|RPP6On!5n~*V{oeAEd zr${|w+(oO;9mM)1P~EX59xe&W^LAhBCV|^=E%zD`?GsQj^+AUtKJSr%;f%`YzpStLJG1gu%SESaFm)g^W1ukjCr;7xgaC1Z@V$AMH#|k!zsSj z1GnjvRj$~QRC9IE*EYjx)Bocq9YD%srX;E(^He*hxCHqoY?HfXx1R2EV@KK^4yIa{ zI|PtPTVpU`o6BQXOg)ITTBQ~-IMb-Gz0uQw z)x|PyVGCac9wjCQpFxwL@*Xun&j z=-PRr>DoYk1?rY@z{O!fz8&6Y(7)#jp$gRdc}TayLR)LLZD^Bs{H*qVp0m7PieU{W*Fo=K2LTJA|wlQ4B9lN(Yyr#bS&$Mj;M1zgN_Q zrkoSYT+==tDO^_kUn^?R_R90Z>s9U+v}1Ai#L7AWdOD77Mv(6@d_-3VSDj#QTmZk# zDk<21!f2Pu$ZfHG^~o#vDgJlqH$p??ssR28sDu?r_w_B&H?>oT!}0W&s!w%=fdUC8 zfFoM}yeJ3tuepzE3BqL$3?c17)s)#pn#~B!z=629k|M{0!?gj)C{;00kHPlbB1gOA zfHP@yQfRY&`*%QQ)q`ww7zkro%SI@&?A+l}T)(8rs$PvnD@89tad9>&;An@(r&2xX zCFMy7BhD>~4MvBAl6<6#Bp&bRsV+nD-qC82wYcwKP<*)2*6S3Ety>zI!T7rXBMNp=`U+_ZCIaoZ&Nd-U$ zVAnu>*2?bZUeUP$%xQyG*m>proIZm-eTqRRi6Z&VO|8tL6|3pKVO3oT-5Q4CbcJq6 zS1V4whoK*QA7F7CDr`HPOf2rg1OCPgjF$Yy3^Wm6V+Kr+Ib-m2zRGUZXq4=Up4XMp zmH6`p58sqt1)bah-_#KVYFft~2f|xLr~d*Ojh|(pNR@^b40BK1m5!<9Sa;Sac98!p zdwJ{*KP(JCtDkd$6l6$Ft!-j zFCZ4<8|2AbLi|XU|B~P1Y*NTYZ8M_Sx)i0PmLf!rS&xDj9o`h&% zV~89O>kI4b@hvbTDvYgF`tGGukz*QlgauhO47?2Uo$e3Yp%$@YCA6~9##ftBybsBM00rUN+5aaS1IA^N@GNCLt2zd96AF@ zo`{|rxu3)}PO74jK?lMHs`QCtJfA-rvkG15brDY$(DHM-&T*6OCqFknuy;eXN0OY- zcQ7bs^#-;czF`0v9?IjPZ6&qMEskO}J~qVl zNj+FMtgjSKslZ6q(swDuPwr)M)@G6bM4%&vq+P+x@$qV_Od@&7*i~+f=>vo(I1DyF zJQO=unLLl|qQ=sLDjyLmzE9#3w8=(|mwuDhe`ZSp{OGj zs+af<+R0yL{tSG|;APv$Qo&_DyBmqMWH5)M@O6*rb{>WA@I|~*@J`!YtI0d%dq|uV z2~(rD=bM36J9^sChb}AB5e4r*xmEJ?6Out%Tg`sVWpwkcItBSj+E~A})E5zTq-pgO z%R#44uerednS2#nc*zK@GPBRi`A)adZd9EhJ<_?6;Nd%ER4(1Q9r$#291ONwC@GPL z8J2Z47OA1VxwSo*vQ;0!^gh&M4~9K$+@j}_V%uIjI}2EcLTNtMJ@wq!P0t&xpV=LI z%f#}?U1UpZ^us{-=)ztpQUdn`Jcdn*xFBPlMmx{_*yhUU3E;)Q9X-4f*vL$L70uH3hJj^JPfQGeIb8vS zc2RwstNZMSGV~sgLU!SPoQV{qv-j>gZR3H-IAvrEB<&EICVInl5I>1{!B>Iv^n6t^ zW2EHQYFnf}Kh$4cF?L;alGi>E`jqc`p&5S~Bjm0!Av_nNm8EK);1`wS7X6$W))>*) zMvQhm@l}ttjijhpa%#A! zHE0+Nu(t1}rgVZpVqiUF|Cn;zNV~Rpt&-X^;$e<@ny+dEa;(X7;%a3+kZ)@-+@w7s zxbyMn4#ba#kWfpl0r6`!=3>gHH~ouyOyt$+Ot&<1c0@o*>ench?leCQ&0em?Rnl4X z4T9%l-39cFC5|yrPx6yH3$wVthoz{IGOiuW_+CREnj3*W$Lxx{QrFtoYsj1pihNeR z&1!2hkUfQ*oRmCHy#t6|l#9OJ=T2gu!3W#cIdHM|k0<)v+BOPV;GFQqZoG9}4Ted> zJO@{F%~V#%*Sr&lqx2vb$x!toqCC0L;;dP`o1tKU%qrcKS&C0qox}e&?Jx<@OKATE zs%Zrn9{{p-$jZ9KSixEpFI2zR|hrr!^$NHXp3-@zArJR$1jkq8o+RQPgeLIiniGVXTgt-&^GpSMfqt{ zv?Mx?^j+Q-p|8wMS;%U#v!oxPZvqMoN4NP=$1|4b`iI{iPp9;8_~FglfvuJ{OOTkEb>||N5TQe_C2BJQO$GaZH^QU^gr20 zrKn)pcX4eQa$E~6WYunk|Lli)53?!Lb|>;%y(#2%Q7|Z7qAcSgzb$s>`9KmE{ZVDd zyu4DbReusrXxgwcM#yn(DDkljaCk$BG8kj7I9uSBk z+17dWdm1V|Z{yrN3^0_3$Oe$)afnOh_XRc2j)H?vRW2=q3EBo*_M8siXxm$k35x6& zdKuu6VQkYf9B_MoxQD%0vi5@zaadir6^tDiTvwu2X7!@HAPr!yX3#kJa^|CDx8yN2 zZAjz87OoQsumw{G8@*)moF-ygDnM2}ZTh8sH`u+Zt3WS=ne`^ZgaL0dIfamU%cY@7 zLbM0AaK%Sq{}+f0BTtw`@hu&4hqjHfvXKNgPt{aRxD6s4p%Ygw?b0TPme|ssxW$&TXi$h(*NEX!#0#nID2RO-4mY;3gpWy}Jo5S#uA*mFH@Q4oV*x zOcHX)!K#lo4ko!6(3T19h#Vy6I$oCyl2Xz)7Y`XwUC}Jxw#{%BXHkA_(A6syuX#yH zDLo*OB@Pd&2CELrv)hXwS=?zbG#^Isp3-&A^H!Udy8sa4?D4Hm7(!{BIJe<_c}e$Bvh% zxO+0vZ&`EZ9b>Ws?ht<*(aFSkX=rA&CRm%jn5d6ktwc%>$DRS9YY%I&Q%*hqu1|2> zKxPlm_u!XxXRwMFLnm@B+Eon2%P^af)W#2n5qA_lv*`BwIb^Z>>Wb6Y$8|1i8!5Ko z!?N$JFFW?0`gthl@=oEZOpH@4%611o@vvpDkpX3TeC2%sBjG%$X@Wa>S;YQ^|MEXMFcC2Sjg2~W?fBPi`5XCSVn<(8G&@k zb+t*MbOc2Gxx~&##qn<@#igl@n&l1yQaFkwMhoFwK1j$BBZ^(hsyo4%o?Zlu=hQ2V zbJD|h?hZ}vgl=!Fr(o)#dZ6JDzI5B7BewI0w?1-GODLgZHdBJsZ7)rT&nX#MMT@D! zuD(B0iHm3d5)m}8*Eeg`!cLdP_bk=1yQm0L2^YI)&97(C{^vR=-CJloUUo**CYFY! zvRY*|Pq1n-NH0@3l-X1UOao>X zW#1pFkXZN3zUhm8vFd(*I?9zRe>k4L;;jbRO1@0^b zliBGAKczYi2%8^N>*R=>vA0p$GGZ%aBOK zirYDs+~qr>oE&FyA{$n<&IWVpw7X-T{pASGK%y{J&U5mJ79)C^1g}=cAMot!Sayg? zQ?=;l4RCH)OhN)rf}lG=!dmaQ5y)i@Nr-zb*Ar>Ck8|hjhcmf;{l|3eJb^t}8)LgH z(4Od>M>5h&%&Xqd78dHmlDe{bEbzZWs&f}$=cz@cbBbK3#7x7A`R9ca=vgIk8Zlc& z1_k=eS74FCPoa)lLHVkp#{-n9sr1i&Pv(ENsW*gLoA#an_VmVc@2BS}cGCxy-VqNq zy>sb)o6*s~%xEdf8#6jw7b9iGT>7!7i-Nb0uFZJWG}Tq>o94;ZE$FfPkf4eGk+8n# z^K!y%rOQ6SpE+*Vd;{f|lB+^=_CD~YqgNQpyb5g-`fEjA;jp2RJUzZ(9ZqW^++3=K zNWGg~<`b5^#JMmWFL~>a-?O|-^ER4U1ABftUJYmCZr@?$u~Nr!7Lctz-9v9?JtE0p z)^kf}lS}{AF4vMc%ZJ&l+i6j0xY?w04$ergBNmHiIA3a*hk50;qeU&m!n5W&CarmE zXx0)8;elX4=6a=^1p*$xCM{J*?qnmzpxjty+J%CR}PxwxIGwGxdgtBalYQ-5rSI|2(5~SJbN?JG|F^6){npdfyeW0Q zQF0qmMY%BW9wp{@_B*a6ygDJ?($4Z(_c@LQ@W?1S`|B*|Ctt&|dQ=>bi;h=@`nRk|=N6no%iiTu zv@#f<=~sZ*@yJ&4)`aVKb+PS9zQp$`NW6Q_?h3*QmNJ25bYjc-0v_0@=Oi;&!z5s# zyqtiXEO>R9`EZhfO2!H6hh`nwOq+qxnp8M_k0m0Y9t8(g6%=vGbrJd8W?8{;m48{N z_ceg`wroyyn&6xh3t69YpJ87MwBD3ELj?i8vB63$2C!+UnXv^gK*{T<6Fe^_^7EfN z1@yqLWXKgt6&STSs${IHTA+(R+`022fvBI5mtZ65YiE&mM7P2;7ORSJDDx172zW%$(J-6lkelDfUlz<*xb zvjp?>GSLB{ki(rO(Usm;`qiDr>-9)<7tw=PduyrXB9%O?1xlYcgtqi$JgPKA+_7uU z*PZMjR?4X$`F2w%PV}9t^IfW*Juj|kv)iz_Eh@Uej#=j|+PiVG7EbE37rgCaX&N#6 z-?7v4m;|zSiNXjUix3(YHy*4jtTt~GlO@rJa64qu(i&(BQ#`#X^4G<1uhQx**{9)R zyfgq^U%dd(-;Mx^GOR@10ji!{xgMI9mScO9E)C)jhB9fwHdQy&qh|*)+>inLB3h=J zvA4oOASn_D@go7KBgr6O|5eem1S&pq6#A~kRCQ4n(JuF)_Jq}@kpKJT_`F)e7=M2B zM=n&K@jl&jmy{!rt6he6E=_vLp#HqPFaByS3ioINy#k6wUHX|_RiazxoQadUUi`V8nBg$ zT8Nbp7IcA_7+>AMT*#W~_>D`g$d+4PokiqH$iwd2eIRt_0bMR&CYy3m+v+^_QeqOw zsd{PDe7LOkCelmkva8sPw)aZw02gzcX$hm+!v0&nIW>B#yf9>d*77SN;Cmo( z7@06p{!+RJrYzAC9qpAIaCnAe-Tipk4v(ht>mhY1tfT5@G5iE)575QT#27LKW^yl| zl_JF5(}O)7w8j7W>%YuoE91s->@@arvf99(m!Wo~=gpV0L0@)pIWB>%y7e2kCF&~6 zkFJcm(^awK%FAr<>b#+)p5tbky8fvLd{T~61FpCl1`?-Ky%OXdwz|iwEJojQp>qQ z%~S7zk@OgSzKpi7HfEDG5lN+PZ18H}?9mQSX68Iu%3+K(QKzarke8(zQ5W~3-GH0E z+@>XBXtZ*lGol!MFf!(DzaG-PQ0%U&VbHTNC?M$s=mGOf2Ha4}JW>y|Z{4D^t5gcg zj>??wd*gi5;(ufBX4|X1&TX26@b49Kr!^Ga@PJXia-)f+Yut$Ye!D|m4iY`wHgc|D zj?R} zkC(GCk8haa5#u^AGqwf&=xz@$VYNY;N8&2urPd})6Dy*(HrquA&lgQ@T##DYr~&4l zNr$p#y2DYWPG*LEn7`dh-lm)V#A;|&&W@Dk10FktrmogQWeJur?_PoshcEmyk3hz2Gh!fOo+yRjBZ zQ!V2`T`sw>@aAeH04q)W3cBBbnh~ll+7!;=z`7QYZ^4Je^wiz}aXuhqjNqi@l)}tD zPE#UTur;7RIhp_#nxR_wFc=pePY*~CQH65Hgg^nzYY{^3MgS>}zyxX-#VHUxHcZKb zfBxl;k{Bu`OneBystt&W3Ic^vhqKXwdyZ_xQyJ>OC@Ub(j~D?6-ky7$=_)K%A-+(b zlT?s2lB$R%YB-P%hDlD(in%4aA?9L{EIezH!DsKJW+}$u^N|X3^Dod{K}gGdc3Ql2 zr2D4cR1jxNed>L~sgaT6>LLV0pIjNo^fB@+m)crQ?M1?FaibZ`04|vw&`Mg*TZE5X zpQk~svSP+PuhP-wy2#eZP-vWZ92Si?p(`s#UhQIJ_^kLrOdAZ+Ry~ooVek}h(KU)_ zr*Szq5{85??ry}}cu_nBI{em#nfSf?OE4oTRqH8UIq{YM+p+_iCVYIS%fneEt5rNjT9?9!YWciYebN zeiZ}c(C6p!b*T*8-*tPAUwzXI1-L-jmS^=@7VbGNO&>k#&|0&)${28mJb%44S0KSK z3b6uIwAvAJ{Z8^fG`03w2cy2PcwvZ0D>_hBdF-F9znvU9B|nmij0dj`M>Pd%;;^#n zQ-|rs4P{xt{gjAarOjFuo1Vnxg5{+9GHJD}b8xohVB4k=N9Z3YM0X>}C#g0cMPGw$ zQNKL}@WUSYI~|Li_hf<+R1f*eFzzMpCzX@{Fg>A{J?DB(JN3RDSuWkCq|-dVTyn?t zLYwX<0~lnVk6nlV0)2gXV8HUb-5``(2yhgG#GFB^K4c+$KoJbx)N@`}J!P7dJ*N(p z961soX#wNrivh^#qJNx>ASwo{x%IP$&vCD6eTA}IYeQRRUJN9^>u5y+)Tp&Eu*M<) zN&d;6Q5tK)51zY-@4D!a9uFjW8LSLxXp+JFT)=@JuErG5i`g#2r$A4r@*-X3X#8amF1g=~+^gr4Hx4tyZ?+uuyeagO`2 zbhin*OUQE9k=t!CLSmiR^+az`?;sXWBa|$~b$oFVBSvA8K%Q52{Y5v6E0j_}vBqX^ z&%cvy%RgI>%gs$>3Hw;omC-wJmStJs)oyi1JNb*`Dj@6RudEn%!vG1Wi{*d(v&aAo zaZOFi{=@T>nW8hgPxT3hsLk6f&pdy4uekfpyF`Bxd?Vz=#-w~>8Hz|I~teRKlHo*l=DwjLM2ail{xMCQ_i0%eK$N%c<96Pr<^}k z`fr*>#@rMxO6iI)MHL{8d9whc=JZai=M5cE z4#k%+@Apn=)uUmYKl;tr3xJI0{fMervzG>~T6t-t3Dt)`!V=rp?g3{A>x8 z$*+HmteOv5v@@B_(q(&dyLntsLXiO>68_rjd6h-lrzPXXTy4&=RRmjB%NM~NvjtnPGzQ;vUbtdZZ0qw6V0P>Y>x%k$^P zAw~-Zi`n&bc16<*>?u$9bHm;L*O&w5_dOQT8#=~r)(#Z>RL9Nbf4cFf z9sf;IjCN<5Yv&X+RgQOP(@{{2e%H*X1)=0Go_v8yzVT6in=21Q~& zKS{&fo?ORLF($_8$+Jp;gMTF`2MYbjYkD}5K@t}Ab}GxAb{lhs6aTgxCZM3$)`~!nk3|hB(cd| zvFfiX-`B{YZ3B~D62Me^Eg))tGnl6*F_cJdW**nE@c*}OitN^UI3JP&i%C+A%N^y( z729=mF-I;&%j&1PqqlE4HK4r^Bf0h?8!F`i%n1PLFC`_N!PD!x1iB4I9P$&X%PbV{ zZR7_DY7o@-$6m^cC{(Z%&^1^_${aynceeG8dej%VZ?%D@Uf*Q;bs5s*mW%Qa&|;Iu zjpg6WJN!ULPbqLQ3m#6?zU}WCHDj5+qvp ztZ}LAc?ebF^LQJ7Ge7e0*nLzMn}J8C071&(>4loMLm`vv^_D-Q{?wUm)mC}*G_s$> zJe4B9c;aPACPFN_e-PfU-qgcRBSB9uz`5ZbW%}3&qQfjmlIVZjEWLo{8BvYU7|vX2 zz9XtP>Re;aiptt;Nwmz<3KgA%xWh2fUE9{_dBcKuDeD9ul?J0!sXwG93d4>TXAuKTp6%{e8PyYqUyQrSnjoa>D-%iNa_*{UuXThU zxr7}XWHiLTpwANM_$;zgAYCsE=Zo737h;d?stxXasmZ`T>j@82WYOb1k6sgk>93Ed zO?LXYfyq}tUxGL}hxY+(SkCPcb1ui^sylU-$*YCvU4vvpC3Q9>hZYtRjzVRf!*Rpw z8uw=7_%gkw0dlfu1zue5Ys7FTY3?YS1!4aMl4mS6Uv1vuIp!=@Tj6WJYFS)T^Bgc% zm}J`Q$6k8t<@C-5FX{Zd|K&Z5ufEumESn-2%SLBe@9AggDw7!YkGzqR7*5+$2=+78 z8Lb4OWA=XinCNvKg?GQcLE!r5#O=L~9sWcI=IOe=&fKrFgMa)J@C&Y9=1f#%muC2+ zdRJ)Tdi5D;6pjApuWu}NWT@~uy{p?8h29gZ35`=maYpwN`N)<@^uFINqO@kJoa~d` za~2Q$bDG&~0c90YSv`-@?vo?Ia~Fi|z^VGv+$G}~U~Oal6rpL4`N*XcEDnPZuCq|R zY(SN1z=EM{^f&VfP?!HkTHIN)L=W*dvHPCn;phz^_ZLgT=(#Q3!Cb~cjzpe_tmL9& zvY+|;>70h7(3ZJp>S^NDPZ21Xu;#{5%qLGW#E+#sMADW4k-nD~)Cc6}ex^%rE>|#S z>AwC2nqU`V$Jmy34B%JtS6PFy!}<Av0a^JfB%)dZp zbEh+pOU}zP>J~JjJzNp7J9T%DKsP7>AlvP_d41UN$yK#AHUkD`xSWdt)QW#ek4nrjh9M* zK8ja|)Apt9)&5gF$={9nmmlPGu-=ypquA+=1o>9h<1TPI1KVPouV#;{dlSU93B44O z8w4LLCMTKWX(WU!JFr4UEd#S2mHJ?nE~PgvrGArL!y_2#&fyTOEB$7p0i0(4&mg{k zdE)OsiFg_j6paaWrd&A=8ia4MCnthRL~C(f&-uvQPMp(DBB1SWz=Cvl84t@PQqy}W zBFLk+t95C}VvVQ-k2 z5+a4{kH^=ZamSC&b9CW#@uIABrplK}nlWXx1)FK?IL{(8neifKrL3cbINaho< zT2n3j>(;+s(b?zfKF`Ug4!BuOj1X#u<+V^hDiK*{5EkKJ4xZyx@EQl0F#906HVugs zsb6K6L#5_G7|`zZkdvE7;Ur^C9L)Kk!bqx_D(fOpZ{A<`>g+8ixd6N$tzvtk`|%G8 zv`J&7568v>b>?A5ujG;51%Tc3aL8~|)<117VZT;&*y}pqyR``jgbOaZqgT9OH zoJYUsA-d&-`_yFuMfvMrFY4QT1y0sXhFMGoS*nJ^ro-`PIq@U&D%`fotvlt~+6`;Z z^z{&kv_ucBCWSYAUPrg8U~*`?5{$}hl2ZeYWPq=zwKgayF`F)lXs3;%4>%#WsN8l> z(;5H*!SotY98(uwrusqswgzD^(eOzx0*5~OQ3r+#G6l3FK97lc01287OPe%yFdc@7RAn9 zD*`j3&B$B{X2nD+(ceSX?Mn`r)lF0|_o;yRoOJGqNF99=lS&KGP&^B5$>Rb;rG1`v zwy_A^Ubp-!oqJzWg9X~GskeQTK1tr4{Cd#gln-k%z*RC15;PwKBB+Bq_qD1s&854< z-dAPUg#UDs+HIn&^o}7W{facfU`pG?OKnw`5BkDbI0(ZufQ5)U@PBnZ&9fi90P|L4 zGQ~x*Nk-_K?xC_ZZ>EOrd@JOBnAAx7E=yk4*F*t#G*u&+5`5l%IVRzeMd?24v`|M~ z$>uJ5Tl)u-ZWg^=HNm;2^iP*yTH`OC0YObtxH2BZ5hfDE^7|BVcq6{yWjruqDy7vb z$5i?afQ|6)bSmJQHyrmcwhO1IGR(Gan6~|$Yhi_~-hE;`K0-}~L8wk16c^y|FJRlq zho}HY*Z(<*pZMn>%yr4XbU{8m&qVy(YoWc!?UNqq+#;r;6}cZ=Pen95mW9@B>49lv zz@r4({$13+r#SpPCl^D%UPq?D4@4#A$sX6P7lX6&XpGPDAH>=)Ehg%fWagx=b|&&) zhE*KJvxGD&HYyfKM*{Tm_oyfX1B<~={E%6ij<+5ORY{EDvRq>#@RqYDDHf*a4khba zF_fxA+G-TmU|!)7Llq%v>_8BQjS#sSTu=(KdPgo0|0l}pN6`TZB1J(-qOcPY;V`kY++qt2e}uAzMA%&3?^zy@}8V#YoY3j``so|H|gW_X5G+FyA9Sm z0-vQ1AXc^xR-vj2uITu~7bNW~6ZFQLOHWI~N?Rd$hOqHkT$wEY3%r3iJ{3rZL~;(b zWH4!fQkZ-=-TT7Vzsmf7*#lT-c;4&*qNubjt@07=`Jek96-aE2M-x?pwdX}$g4jlT zdFk-e?akHTRLkBSCheKsk<-Tfk^~ZRnM%)nhVILqhAND*`h12@nu830XQH%x`S!+v z%f39n*J#MBR$p1dwnknz5+d~NI`|cIk3q{r9#}{#59KL5(T|z>>ksSCJKt$Oz5eju zq`ZpFE&aq(XYbliz>W*je`S?>D0xp91fu>ofc<}yAGq}(3;VLiSs~tpms?@WF#|Xe z6Q%_UxK1eKp9Z-}_7+qK9kaAl^6*sBJb0+(ht3Y=~ow3>SFPe(f4k zxR=w$WvQlCaakeh>8%EBw>q_SJ-YZP``PMYoD_*YJ^HIl8H0i@SrSFf-RT6aeSy6QwbreANal?&h&?zAvE_%jZ;}vvw4Eyb7j*8 zj*g8*pSpA|?Q`#0m&E9G3heJZ1`K`%#EfWKPp_YQ%YTqpLT)X{T$?@2XeIu@_J~bl zkK~;i*p(;x$2M8q@TiWLGeq@+IQ53Y*AK(@Sr!Kaa}JlZ>25P$qF_D^>z$m@9P^NU z%`wS*&}cY^z_2V^_ZXh`h<0{}eTQL4*2208yfw44fs>Qx1(n@P6L&X4qF6@*S6&_4 zAcv@9iTCi|zBgLwCRnW{mToXB+5;$=jml|9`+$R%2PUyw9=#R*)Bx(Ny*=qUcB=nR zGkl12D)c%9oUM>t=A=$bJ!S2k$#8|q2F zK-(K#_?3&^hSObPgVLY5u+36dgzNN-CEd|I!+Ql|$iKQ5{ zaU?H$i4Z9LncV8B9B>I5enXS8sSY(CX!I324N;cN?4dlFm7W@nCx)s6gL>p{EgxCa z?lW9wly^j2=&;a({jpXj+o@-!i;{I*R5HJhBY^MqegrGWpk1ttVs(_Hc^9>1F>|*e zu99$|UA&?(63@TIXfGmtC?nj-qZewj(uSKe@WbwUH~^U1#|_JrQJ_-P80YGks>bCg`E!QZ9zg zryli|N9t{ypO(AjeZ2-e9)lLOP;nMZ9e^qxbo@Kx15<1#NTPY>c+iIh=cK6l_{2?h z+D;uL)rqF$Qg&NA(7TAv4_A> zaL1e7kuw4~?uy}h@uQ!tgv12#l$4|?0id+=bgvZU{YD`Og)8Wf1JWblLLwQPpVv`d zWIE1osWX(*Im5~2V9LnVz9E1k9b2!K%iEOMW<8p;I@-lKmjHf8fF|6Mb`d`p z{@7jUbw3)g;BL|_o!vxu0?U98B}T!U0$0)b@(CUNiRG3l-DM0Q#);!f0>;B9!s}Qs z_GP?^d}kieHluu)pKCAPNKgTm8U5K|PpE(TI7j#NbGs*PM6PM6d|=Zmnlc0)NWDF3 zHL5z=SG|!H_%N1nj>Gq^h(K7^?q8q>7gnnrkQD_jP3$6_)OI9>{twr@cpThfp3(C@hQ@i%Bzlan&N#RS_EU{>xK+UwsYu^uji@!}eaXlU zr&Ee^&VD%l|0JDf!?{j6jcLveg@%$ICoKAQJ-LFT|0Sh=q(50UK0MP{#lu5?oL$q; zu31Xnk7=$B+snuj$7(LUyY}m?e;bA6pOk{j`n@;SApScSL_G1h_~CJf_B|i8;gid|{?PC{fd-N4tNroXWCWJHHioTMPa z=lu4Cen8P9wy$C89>zRrtRoDUcPZgpM||FeT(rsE>qiaD3JcKIy5Ns(b<4h$V8Mp9 zmGHAdGYI!_s#Cn=?I@aUlW`yC)ba|@v~xAUPS420xLn(xa-e0Ke)MaePf{PhgrN59 z*ba?!&M+6M&Lev8HtN?3UF6Ya%6-=t7>vnf{~q&RCEeX4w*4u)abETek_>sbD^<#3 zt4e>$k;M1{B)H|!n_6;6RB-v_wqED!>B|lIlWUe&{0n_LQX~3oy{z&PQhBjWNkVM- zqRi()zF05u_P#ZbqneGAS@sI^bQ{x((rRpP7VRiZ)td*PwMEp4NnL;|(tv3mWDZ!B z)yn-IQ+X9#+r20+jk2Bs!xolKT(pqHGQ9d9mc_rye4)4Yf?AZdF3y%|6bE3JopgkV zBg*1;A3W%2L#`AQ&2;L9-vtgOL`sYZ6CK(Neh2k-0t(5vt&m}4>*E9S%q>x4*{(V>`m3GSB02h5IbxS*v`L@-N_cLw-6jt0$dvj$ z>J#ckQqlcQj*AO}FkLX#>5N<F|ST&TU9*MQj@JA#GLn`J{n7aw zTPvqd!PjuC0?Yvs{DCxXDq6ARW(u>FL70^w{6yL-zkY-KUe@y(U&f#eu93aeJR^y6 zg$zdk4oMnpk6cG2d`P0bDI?AY*p#pHWWbKm;rs`)Km1}%_MPy1B2#mIi|W!6-@eh; z0`Yxqk3(>I-gZ*wYkO$})>X4cog5Rk3cp@HC52cuO<%PZjp{kIcmH6!SwL?7o(MYG zc>Pw-I~B;z*kM9hMXrJCn&DVK$|UEh66f=VLl#e+27tkSnX7Cn4m~iNX1CUMTuP+D z+u>bf69yO;EGi&ymW<~=t_0r&TLsWK$UANwzdMRbj9O5FBUd5Tol}^ z93&D7>GcBl#Eg1*O1H5<=Xq?ywBp@94sDU?3B=+(z!i1%SZ`sp#1pg-QmnvHa!n?y z^%;v}8kP=ath*3(`MWHC@Mw;oq^RkR38kOBH^YzR7!I!XnDX3nbPyXQfKO_^_0f8= z(-@#if2jVr_uPNa|5P96G1C{TyURiu=I%`wVmWogvDk1Ne$*;IS%pA!v7yvlOO2MF z8LppIo0-_s>hHL??0x4|w_48ob}UkK=B}?CbBQ6GhM@VsYW`C-|9vw&PhZEtY-Rf< zs84GN%&w^hxS~sD=c}KPUh(>DUetJp>^x5Aza;UGC#tM8C5f`H$t$LS5%xiW_2J*} zUawYY?c(9N7g8A**-)F(N1}$7+FN_15!t~_GZ2?zb6G{?trP^T>Cq)1v~~7l~o6!xnn6%2KnbI&$Rqu zd}_Hni8*?$cW@qn6PU~`gum6nf7SgTHMczBygGVv_~aP`cI{MTrKKrCIOY^PqKM^! zU0G#T^oWb@E4tQ&3+A*Jd%{f^78lpAcLKQObN$}%p|+WJ-lGqC)#So%vobf7e>C*_ z4F5g- zerEW(uxxt$ddBul&BJshV=fC0-6Vc~zRJ&6fApUJuAw|3;nlg)OCe5hVv_glh??8@ zR}OV|pQ&3oiAeZHMxXP8o7B}+7*{8TseO7jmhizZ){nA(`D3`&0Pl^h_jmjlS4u&q1+#!LwuIiVzINa}lgmFxbKxyfR=&G% z##?e&o`wj0+uoNt6A`rsf8>CNcU^E*Ml~atZ!Nr*bjGK-e5*j{i8zgurb#h3&9z*S zEiP;Ls!3I4=vXKy=NF^y@8mdds~e$2zJB4$h18GA%Y4|zRSF9?UumSRPTo3>4qMG5 z9Uok2n;=aryC<__j0^I^FCN3Cm+NH6_XyKcXO=JFYoqeIuWaeaL2aUS5N0&InVN1r zJTIb-cGyR}B)150MGXk&Yzw(aBzO^uO}|7_@CNo-!VG4BnuMBq;PyLpm!11Y%iFd8&l)J*TP`82IMOU0N;6{GCGWV))Xwfq!H@ zfuM8~lWkCiZecj^tFyIOc@MpJ#FzOk1@JvyB`#}fS`!7mMVcU{G*tt~J%7uckLLz= zpPP?7pC|r4ZEqT$-5vMU9?Qzu#rK)fcI_RFDB=la6MrFHSIfu?;UbOiKrGEthmN%jgx=!FEhu6~;I>fY6gHI_GKH-E0gL8FfQTz$s7 zYR7G66LQBIy-DC4+#a_4ADhM>oJK40ehV6XemzQO61LT{R}ca&>u_9wNiPF|nxI|T ztd31JQQYht(HI1Ct8gpGa~Q3$ZZk=R|` zb?ZA8_w0{_^ZyKNl=myJ(PU1sptjzuc~3dWNDsZ#JkvV}ND{nXywed|Z;jFtUU3#j zIp~j9cW57s?_Oihw0LsmpM4uR7mJPYx@$}jQqk+y0MeEL(zK6M07(aqWWBx%DXB!} zB*&srfQ!rzstQ%jJjXKcArlbDLOl0-lbsPe-sB8d zg`H-q7-b&(H{-AB-c$!%dNhDT9ob~$iTu0}GX9vHPsYB8)5`(Gytlg$+&9gdj_I9sPQt$} z9=NzNjcnx)Cl(5ClX*`4{1>Jdl$dK_mt05pP>U@ZgRx78MyOS%3ZQBp0iDQ}QCRAS zpRZEbe>46=eMGc*U!`ifuaO24-F>E#`bJ;~t+J2*5f6`{K_b+2IMgOw9p{v!r9o49 zZ?Y$^7$O;X|qPG&n!n!hd$<)dX$c@DW+4|pYi^k8x?L+K8A+_rK6wI^)^0U7DJ$F~za@PH$e(lIS#vI2i{I<3#>Y`e(*t9 zeU*}OE9+>0f-MpmHnX_&rDSTL$m{L=)1x{j8+fZt_M-JD^Tb8It6hb6JU2?fOa&>j zy2)1np@OJKt6!e+4vISqd|x6krq6ET;;aEVP90W{1IAsAQLokTIi;V)tl8%CTKWB2 z{6GKwee2+ZHXXq}U!4zRU_iL&Ck>I&v_rXI!s+$Wfy!Z$C;i+DwpusMHaw^XgYLwb zLyh023~4ULpEF7nKMP4Q=Acv|YIAW(gd*Y*1q>H>D07~FY>HH~LP?hi4FKmCl8de2 zV@Q^DHapkYi7Uh2L-3plRXwFvUApjsV7`^H@msbHWxdz?%C+x6p;~4=#koPPgJCfB z1*nZAexPXQTc^9EyrEvjCPvwJ#==ZIH|1|d)i2U(X8@d=BO1kQHU@OeplbS@FUr(g zpo!42uCN!a!vx7aUoV*i1nZh@9`PtYoq1Dl3yfA?2Bl~zSO#Y~N(%RlR7qXGUVZ)5 z{k9<9WkN&av=v7lqbYf(PJM5>izcH=g~;wCP+l!9c|+B6W74>$#nD#_OTlD;dq5x!=w`M#=(?#{rh*&{OT+)u(qm$OjgB%jlB}6a`4lpcJq?{J6hq{ zo0`SElv$$An6cbK3|9bF(BDde%ZKM?f&YF^%spgCP{$Q!vA10(^J@3aHUI@3*p3xN{jcp2p@O%5M z^bUvNC#Oekicr}#R3Yk@EkDiW>8rVu`8tOcu^YL+T}tP{BcQB{WqJiY5sZ*aV>aoS z7tuD8esKFJP0P|fE76cG187>;kTX&0Nwaf&vDRAR&YK=8D_r!p64jwhV@r+`2Z%pt z@rE1B_viwnEYrqMsn}E(Cai9H^xycfwfr-WBNR~h4R6ymU6u!0GmxLO?{cSq z%vlTj5+>ZfGsWHYvm$@}|FYDL9hE9J?(~TICJW_Ig}vA&XhPsRNu(d%H6!E~DgKOY z_ohRLzLo_PU{vQBSD6+3_CPu5lH9XRUDih>h`Kom6UVZ6Cq${a)&rJXe_fD&hw|Pr z;l6e`Lz-qG@hhOQznRSGru#2+{r18pk<4}U)W+53{o+xm72|(#VpYkEy0#h-2pe};O-UVA}!Tz zDkOO@qz4HwLbe#S@GDx&Iv?sDWhGXBL5hPB21l1XnFJA?4fvpy9RES!@*DDTb#+X_Xukj28 zwd`7RZQ!hYH$lep*W&v=qj4csoWI=8ExIvkCj{OfTY)9X)g_X3js>^vJlg9KZV_%5 z34WQ*1q;>_s<0oBSe4Pf$DwMWuo!!gr~09fs$D81>h#cq*-}$4sYT8`rfw7Q?(|ZI z6Qn{!O?QdKi`=G)rS8dT#Jwn5nFCAfU5u&C+)(tx6qke4sXZSuQ+NK%~7o^)_5CW*Z|;6L~Z4663gi5JNrO z-gA*h$fo6SkO<1?F;vG>V6{=;rf7l#99bb`)`G-@_7>zHRcM1gPuUxy#T~0*VdvJJAwCr+y4%!pVo6Ws5Z{H?GpDokSVzOyGj!Jel&H!OY`a zHS)#py^z|{b?WsUJokT}?`Og(it2$fL)z$!h%O2%EqFnC$@o6ITv6Z$8wL*E_qC(u zZFWP5j*ey~NU=)SBqi}~N2ct<_n#yo6EK}1Lf%Z)dcC1Fn( zT`F)klw`FFSPnZ~*pyK=7Gg<`n1*J=D=r{vtoCHtk9M^Ji5hzJ)QHkHx=nfCtpp8l zB3I^oVe6VAKY$%fseS-Epn+OMu9FSmCF@R_JlRKnnntin3VhLkAi=lk|ghqiw& z&Y1pT<+u<5EcteLBGXHjWn-GWAha4yj+QkV0H1ON(aU6~cu`$utQMm`_1zmMH%n@0 z;KfDNz(ugO!TIb-J$7qNpl%p#h0%!4-u3C%g-uM5y~iDspk3K1G(y0jLVMi)pZO2ZI-0 z|L+R@pQtZNCPjVj=``bqFF6`l(CV7lyYYw$eFbVgzV)3H-&nYKExr~9#(K|OwaRl6 zaM2!<{n-<D zpk@F~-=2nuPCRcQ`LGF}o@MuIPuYdGq_UQWh+?g)ZJwf=G4(5;gQRUUVqCgepJJne z=D&6A!S@RNt8|0Gqvx?-n-cA4BPL|cG_HaJNQHnx{T_VejjRcu?B~WFM-C@$4UX&1 z=o(chRw1UEf3jj8Mm}7u0+P~3=2W6AIbE5J6m(uIG1hWc^D*pX3%>L2KUG+;3%7ed+%4a%x>t6j&pa2-M*@HVRdM6Fgv>!p!7#UJ4~}e-u+}Duov7g zX|LB*?y)w5`i92?X-RF$S;t}L+Ni>SwJneo@-p!`eb2M~eGSLlqm zrWc2mNJQG`E|UL)7dTg~o%vnZl0ETP00YQ2N{J2byuKuDGijsKNvbZ&o0F=W<=nEk zV48Q`iOr(u#TFl-8dSD$N(+|D3+a3lhr@ZYC~5rsBB#*qF-AnQE=*?iJVD6U>xo&` zg+|Ro9~*KPZ?H>44L53u+)cj+KUXjKJ)%2a9^mceowc0iyNbFfASOGAV{*irIK5ES z8l@WFc!$neaqtk1^8uWT&^{yWyKE}_@Rfb>u3|LnnQKW;ZgXodV%wq2{N1}B4^>(i zp1DO6CT+fPa28gLN~cR$Zkf>KXVH**g%lJ!7nmkE27ugre7CinIn4y%`qQE7JGHFv z+bnx9J#}<#w@$o7dJxqjr9~De+7aPzY&Cw)+JI5clC#EzD+?+;e9=pp`wee+0m>BT zHp~Rb#@S!Gzx#tn0;S*iuqoE^^=s7zpNvQp=+vtf82Wi(+9Dl>;l!d?pw>V_zYTwV zLff#fTHDdEePK7B2;(wt2>Nt;u$sc!pphJ)dR}(uWvFDrb^lus=F*w4Vjvl&=2fWQ zH$0D4uVcwiH@;4PkQ#TKknGHt`u{eYp_~qHa+mPkFlZbT@(s^a9OdkA&`lLk_wzl| z9<|=*?Mse-wLJLL+0~MmA-p%lM={N46e#TkvS3P0m#3JTW@8*{o~2IY@5Y2$Tk$f zWQ(%{`@cg-Og*C67HvBJivVQvqZYnoW}|IJ18ub1;@ln)QG5YdWO zDERQ!mO(Ulm$I>^b??a)?)S>I0{yq0GK>9K1u-`Nq_PaDZ* z3K=hWmdwn&q{`5x$7Gx9_F$7UYQIzmoLGIsi-}$dyLv>qnRaGkj4?-vGWPI76pPCe z8r%ZQ%)6TkRU)!mRuc743p#-v9-_yBvx2P;t>{2O^{AV65{G*(t)Xhh%xBHPa(52< zabK_F@K8Uh@yGE)qv+C2flOM(Cn=EQv?KuBs17RU$wuBh_`36VF=D|JnO^FObtA7I zk=AkfW;p5T&C*aNvaEY4_H$|o6)aTcK6+NWdLB7A~(gT6-KKxCCgw`3^E55W6XXj2MvC%CKh1744| zIb-NyP|?|W%l=-MEz>UJSYZA@~lnZrh9q&nDnOhlghyT>9jy2d8-sdI0HoB?C2M_eOTd zDb1<8o3Ydccm4ljdg$WgAFa8>#p_tVmnlZg*LvrU@dYPQ*>p86$1K^|XNmuU$uHR) z_9C89OFTzer*nJ`X{R~XodB()bbOXS(%c(gl(KUE#TM6T-F_i?e+BRQ-WqN_=ktkN zu(fSh6Ev{Q7$thinw z=-EyZjct;uT}-07?1ZI2(%t)4TLau-58cEP21MMc)ei&Xn%G&v(($&tf3|UTBg&W) zU>QLL(bc6*Xxg8hcf7kWcA_7Coo+eCCFqS+$gc~K+^9I0{n})ogPgga1z(*43boh zK&)Eotc9^>cX_YAQnXg?C~K=nxrFFHNOVVO9AD(G+y6j$@CS~6e@M&j&u z14-jF*;8HiF*MnRl)Za8SR!}GO{H5kA=8Ek2eLK8qn+KHVRnBfRF*kut>FXx*MFK@U(AYRO;@a2P3D_;*6`IJ^_Gir3X9?mA6x5D z#UbLwk+|r`#!nmElXiH%{&N|B zx+&FsRa1X4IS}Y7Eq&gv6^xp{=^`LZMDGY5Y|-RXeq+Sr9Xv$sTR@HmbHH1IhO3Yp zxu3pszF*!bH(BgMVPG%I)eV`9t?r;aNczMP$}r>7K*)DWpKaY~PxaJkVOA%;;F)F6 zQpT76S)RXo#|1Z#%{7LI`sy={FT7$*0acD|>z_@CeQ4)g`d=KE)si z8iY2m4={dzsifcCsUG}};q=1#&v+46y~r@bLNxCL*rlVE4X^hgMY^l#(C^w!*tms= zVIMHe{b>28+NpLm(slMn8Zq+XeATN7|E7!(uDCzBLIW}^5n!>B= zSD%RH$8bs}HOht7^_oN4mS-Knh=WmNKf>%+p5%K~*rQ|xl}D>85)%inO`~sff{_AG z<%^<4GZX<)P!G-q-Vg{aZo)kDZ+Z5|iWJCgvFL7a5Lat-^ZL@+iZX zcfuDL^#x^y)ENg6elOY=5_#h~O}B*>()V&?QNf9BVWrO6LJeq^4}VP#Niz-^M>rya zBRulUK>~VJ7B?p5Q`vE3796P&7!!rYdE5%<;V|Ya?i`on7;pg#wEtN-R4Rr!w|XG< zXO580B+)U7Oo!=&kV5=)X@?`Xz=5kRK*rk(jUjBw>g4xRB{)Tt)tNQ8w82Ld3!6NGj zS0Xcajw^}Kk~>lk_pI9nMTeTF&NHKIH|=7bRk$>%xurf>8Y~5#MfN;y&)|F!dpHd1 z)7n&Susw_IV&_cy?C~H8EZukQ^O(<7Va5pA(yCaYeBhkQVTHKN;|cF>Mov3=q7!kXaIYU zEq>7rlotxHVdsc4Y}+e|)NOFT0c7 zP)*2H5Vm?jpior1agu@|e2fH4B9D>@4iJo-Ud&!sN0%%gv=B zr9FIc+yi725s3hE16$|gArLt#J6G4^-4*VlNS(r)^*#C6h;Mja0J>&S%M60Jg*CNiU~o@wAd7A5NBw1uxP zm+a2A+Hsa7Chf|iPDzhg+$<%>OP}9cJ6?mH7{1(zT#ZV47q2aL4T)evGCa=2jUvyw z5Bx%w+1v}RgIIdtyqIpicJ1;nsZyC^%04lL4O#PScOf{HVx1~*Ds@_Is<2!3E7-ki z$^6C8^}4Wegem*A+bsPNHkUV6U8CNhqK{nH zMi^>ShuqF1etYYO+y``z$FC<6-(48#VRB{0sr0JFX|B|rWotFY0KGA>BLYa~GGnHy z0O#oA@G?f=?m}I5fm@y@PgXky*HDK>AH5NWEYkFuLr{sk6Ot@TGb1jK$is>oXRFk? zrZ;l*vrJS`T4T&RpUPHt7L9!NrFAugoUN)7j$Q24e{VkLd%6pk_MeTHXVMf*ERGf0 z9A6JM8w^|aI6yLJT;`Z<3c}Cc}8= z?K6`zf_#Krov+7gCJ){{KAOIrnmN_F)oGuT69RqfS*BME7dbNQx*}WPiVnUOoRNaQ za~7Kjl6WgO&7SmItMGT%(pOR~^=8)a2}l&;F_CgGEKr%Td$C(am?Qg!9L%!ie05jV zJlI#aChxp?3ThMgJe{kw2U19~9*-xk)G$g3wq*H~M+fpYKJJ4-G^z|)z4lv<#9Thy z6m_lm0Gc%`4*SAF$RTPD&RpnJE@xfTpoXx>M2341dy14Soe%x}xmqkN?nZ9%<>tIW zxgF0g^yO9*rVa_>6Qbv*(mmK0)0d_*a>f@hXt=baQhAKGkR>dyy<2&f^D%GB?t1j2 z>^$imKTIL4t&qrL&{g{6r8H6(r_spzHci9d+vj9&q2&O9f>Xh#wz%T4jX}t?%$#8f z$Emgy^+;k#93+bEpl;!aAQ7iiR>8B6H&nw4>$A+U02O63XVFV5#aNLeH3bFxf=Cem zwb@=?P{fg9dxXnSLehQq(BG6A?M+@%gh2P1=q%{=rg?`cl;)0n9lh$wY8j@|Jg)3= zjjvaUB*Pq%#aG+DEVMgQ`>4Gx$k-R_0wxBKNbati?;Mu*U0=07^(BHxrrY#H$#@>m zS8I3U+9?5O!kK2spVM?B=!X0_9B0mInMp9fDkXcSQdXuNJ$-NdH!1urt;%-e3;1)9 zZ_?c&dLsn;4Trk6O3e-7%6E$+q4#Gv?nWR(qf$RtwwkNc_pDuHR#RwLn{MF=FP0g(d$iU8TA|t57<2Ani(!kvoxbGki#IV zTxIaj`Ar-7wIP`dBDxc`?ca=0+}SA~)V=kS=l{|FXItf^ieKYqo97m_wa#A6I!Jv; z1PkP1K7m{MQIlxSuFs-h_BqIdtD7oPg)iHePR_Z3W>cXmni9$+5Bj3sUn!h<2VbmW z%-kuKLB9Raq^>Ti>-`?%Rhmt`@fQUpkc#g*c~T5-(#xnLK!Ws1Brl5sjDnz5$}GdI zs>{lB`{|ua@D>aRaNDFT@?+^lD>OK61SEk8l?u=QZ+L7r+lofFG;U0;QyMa3D;167H0rwD4b=nE>^aXq7^H70he8eE zl_4YfHz>+_A-w>T$PJo|7s{|Og&3H&GZZ1Sal{BE!xV8DO<$GX?BDR9=hP3xroQ3La`{**C6f~je+==~??0Yv z15TUmNuk4YFqKzpBvKBN-S~a_rHr{0BieG`)G+1A)=H&1_m_MzC>8VtacfY1fkE$> zl!hPg`A7GElu(Lp0KGbwXkf6C9~NN;A6=Q%n&IHNC!I-#;vk(}}o z%rO;NxsO>*gj6nc3(A;Og@Pr-^bqm(e(H-~BiZ^^dW)kyEiRY?{V?nJSQysa(I zv=m&)NJ?B+;szIckds>Y`}db?8qD$K4}P|xG+7ob)HxuLw40G*N46QocN@qxS}f|f z*ffr3(ZVed1?j!0rD!y&eF7lbs z46FI*^zd+AjKYDv^pN4{SIt-={Cc)LgSZ;vPzK!;IDpp*L51gAyjUkf%2oKtF?vQob$*!<{n6q%5z23g4@Wt(|x@H}W z9BxWnOhy=0%i=}KC$uHeT}HM38K|1U#(2rg;3TtV?m=)vn>^@*_FRpgL8qQC zrO_7bOw5<}ihANEc*D_>m$WO$b>ZP*{><7=5_E_>eBvx;Ig)!g6*n(88s(7g$6@MA zRUV^`P6(qA%U z26W86Nd!!D4+`%XtAGxuHTFbCrXs=}nhS{$p3e3W12G{LsJY!p3>;d#y)fY4dT zVs-j;_JdF9Ui6zzTXEm;VmZg?1N(Grm9d4f*cLsPXo*6khleRBl^O4gjUgKvb9lA) zsf$g)6Ta&TO!9|i6XkilZ+efIgS7Ys?95!Y*ZQC;bExkPm>@AJ$7h`8)W92?1(r7$ zv}JPW7OR_z*>J1xY_T~rBXm>kZJEBWl{3qy1X;0jRI%1H_gMozx`YrgzPY7a0otUQ z9XxW7WpUBqIutZv@?Y#8mNS}iqTi0|cWA~g7>)1i9C}0cgrmVHASbSXh)i1Uk0UnB z2!MNK^XI{r!Fc19Z+=BAZ}v2WS&cX|84XK3)8$ZqGM$e}pN# zY0yC@baI1gS2%`5OQRzIV5a*GHg-JdLLo%^ODqx^@NTP_gx9#^^!P#8*Bxz=7l%+m zjOgh;dp#c~ww$Y6HY&;jwj6`o9T;8L*Z4H|P;z8PO>4=Wtnp`!OEa@)3u4%vw}t>U zj*>o6J(;a{*MVvq4{p=s$-N+Ci!X2>&ZQ5{+$*34GEczO(-f>#JOkS@P#wjtstD;l zv^h+FIlACMEVxqAhBY7}my!G9GIWo_Hr2^7m3t{b^T#{VL@6uD@x(OeA-GXZ95<%* z+TnVdV&cxTUG{{A*Hhk==wpG8uaB@9h0I@HrQ#+E$nyJFWZ(AcHC_SC&+lr?>M~Qk zE7oo9MliHH`1cKg0ZgBk$m=%r@5S|0RSk$5+A|uQ?F*43Ug`vr0^cx&`2I< zj#H|SVyf(pUMo$hi-tAwMJy_l1QO$gna&l#K~PK2%}Tp25h(oZc3h{Qm1>edB6Rhf zY6GU4^&;$gn)*{kzTu63&?-X{){Vq^?pbXT?1EdxCWw53v{Oy?1l}#Qi=Z~5>}$jj zV{o&AVkVOU-R^8^2^+t@GAQubCon<13SG;hxvY~@sr{D<%NxQ)VJ3N`4B5d$UH-0Q zkq2(|8IFh-yx&Ka@YMTy@kFwHB4ZEAbd!Yy9C7bIb;)QZkKUrUc}#)I0X^tr_B5=S zpBQ=;-g3c>Fy&*s63-)?N~^ItS*G*;CPOSHu0uE}F z=&dTNN>+_{Z(Fc;V?3~ei$?t`^EeX02-T?}L-l z$EzAf44dXX61sJz#pJCR=R5wX`Ekiasx!J{(q+14vGin!aaGw=*K;!iG&Z^xs2-n7 zKrxc0X^3*2rIb8E_WluVP|;Q&=lB%u3igewq&o#yIf!XDpU{bw04Y4urB&Fa6IB`V z#(o?s2T;#DoBe%YEOsst6M}xJSL2B`fR$7&9V;eOq(wVFx`Pr3t!kMA_0#HAP^+7m zYLv_74mDjO!l^UhyE=K%W8c-u2s8z4v!@964YpOwopVfLz5DSP?cj5vqk0KW+=le$ins$!~a81p7n(+fFL(A^H* zMePe!0QB={v{E7#2@l%XY9uMpZNhbL?KWQA>8;Z?wXZt4*Q+VatbeVIvsDn;>NVL8kZqrePeBTq=ARTsMsbIL3Z z1%q#VT7uV=+?BWs#`Vv@^Bixx64Y>WrJ=Gqp&>MROtLsV#TciZ>(ZC;rq31)k~4s; z1=ffw)MR5-w4+$risScJAO9GeBL;h@P;;z=^vWHEj-Vbn#1Lp}4ZHGyiKbS5d8PW9 z-7}Z80=OuJP*nuLEmw7b!J8}XAOPClUlucPqe;_5DjiX<+JV4IpLfgQjWr0pKC28; zurds%3fGgK(S+;oKXW)tPRvDGWuVaId}+#ONxHtmJ8w-W`a;8v zE4M^ecL*smu9Drdg8*?z5T48dLA1V~Ke+xcr`P```8oQ|8J9$I&-}rw+LN|c*#o0G zcY&L!08@~EUfxarqe}(Arf87$U^NRU z-KvWPhhGvTE{h!o!Gj8p)K6EMA$OXFJSijw3iQcr%DO*|nI~g~Pfk(T*~HBe7}GAy zgqOT35O#hMC+bBp&QqBS&ufSuC1xS=9q+K(z<37h>=ByX|Jac(1$$K3t0f$ryT}iC-&gqT-k>VjIlF*nH>_@7;(_0$5%=vy0|J!GNzdH# zSZ?_UkJzFO#m4k(sC>n5fW10LXVD}d&~A4!`4=jKI%4Lvf|k(I%ktyd2*i^(LTLFn zJdLGFABMebbK!YyR#8oVpixlyeWKS>9hMyaJ(05oLq#cEE-Eqk7_Zn_swhn}65ugz0w2PSHm zS_D>Ssd@jBYNo84?C#heH6b(Tc%jN3h_CHzNmVSVW;{LDIY&|r2*DJBOg(O^qAsIKyf@Z+HodQthLs1xce8<^2>zZ}8O?l=WBj^I-m^ z4z|H$H$a{jQStl6p?D?~Yy$;8$+3HRUoo=1!ZAK4RT&qy)8ykv zOJhAx@9xcw&)u#bgp0)8wotg`;!NSG{tb`xQ&&XY`#2q*Zs8^qd4HHpu#I{9^kX{sA52aZNn#9Ui&T0ptGmfp>-pzmE!sES0y{%h zJ-~ZXtRK53C^ZtGP!IIzHP=ReMM=!X%J`b2m`Yt@=Ta$6h1*o%np+G9|`ivXmDsE z&Ri&1j-J{N{);EatoKT~tE0SbIigjofLTOCuQW4=7GnzyPb(uL&WY;ZbK#^Cz=~+a z83bEz?~jf!C$h@bSkbn=>WUfVU+O5cZEv3ZhG%VYvMyIK85T>WMlfljZR&%4&k>usdNPKGhjK3l<=fnH+0ZLG1bn5!!$(CZR1V71O1xe9kXKA6uMU z>pXmjWkmbWlzsL>eA>yWGn8&F*$w7g^Snersjg{g4DA?8YHy*Ow%Ar)^wb^3x>L&k zlz^G0Q(>adh2;&ALU?202S*klFvMnNlQX!KFC|+XZ{mxJZ#SoZ<%gg5|2zK#H|jzK zGAXvc3e*vmz1L1xi#GRU3iqVyJB$CcShwLRk`xfVyOzF$Gcg=MPxslEI01t1WecH|} zyBtNWuSMRWYQ4PWTv;ucTJ`_5_trsewe9*Sg+c{ri)-O6T7ne`Zf$Wb5`xp>1T7M@ zP};W?3pCI`aknH$a0}4lF2UW4yIX&G_davxn{#H*d~@dP-<~sb);~!mD@*RI=Xq|q z?(1R>Bl)gHEG((O{QB4;xfb@{DtmVi!9g20H3+0*2P44l*cBQn zEh@cI>NX_j9^E!vXu znu9T<>4h>UZ#O?VoEm59G#j{6HnP2E+$|fNtoZrPS>8J5^s*AHM#4Ri>93^4@fUGK>Qna{N($I`iIk!9(t|v<g_6~f?7zg&Iq1xXF_N^(|D#$`h8p2m&;Am1LNZqH?JglC%R{onS|tdZqLfrh)qNx zWACw>RCVQ(!G6Kb6uW}4u@Y+=V?f#cjA)ROoBUr)MA^PuS8bv?&onRBM=zr>*8YO1RojyFp zOQFtmwvc}pbSX7d`@wUXNLS2?+kuh4Wk)}z?Y&qi^k@u#a{bl0JEvIMQSN;VJBkEv z*e}k<+d-0+mxI~2bHaxzZx0I?dD~qyB1V#y`g(0{fDQ{0)CsL8@9ItT%{N>$3_06Z z7GrJDv3(?08kA5$5rwf^`7IbQHT!Fam4}d7pdqVh$=4#7Ru#7y$=H(yp0vj__YB#JeBbI@4j*=7 z3v_nf^jbXsn(2Y7YLtI79BW_P2ea%v3sk-N826NcxtNy79r%_AVU%HK5AW0Fiy{Izfvt!8W8JxA0(zC@$*RkP4 z!gy^;$4_aowFpxR4Bn~M8tT3W6&Ly3iQUdb1D=Y8_1m5$ zMNX*{9H@L9-xhl~l@!~FDr29)xNT(J5g;G|@0tH07|Oi_kbK@3oK?AVU~>62dH-I| z+4?e%md8q8#C-$Ldl(BKcKsqE$`Wx3IjK{i^uXo0r-=L(ss$ z>&?-#6xh6#Ml@FCnKrLM(QB&*R8S2n__j}L!95)E<~m3UwLMo$4B0-_>pg}Q-s}>x zv(idJ+|@=S!{hzrD*3Yj3s}qZk$pt1`Ng({H3Jk}!K{KgOfTGPiQ3ySjD`S*FPz-FeSd&B->qd!)E`cYUbAU>Vp9Zrf&pCYwq7YB?2aTFxiWhq}jngn_zO(Y-Ymk^DXOvb^^yb z(QPVIU)EQqy`1+ZyY?jszS(&jCgHjmZ9PU&a!QKUEPn_r_QJ!*0{b@2NC-)E61skB zix>B9JjLtf#muM>bj~ObWy-M#MP*aE-9C*7oos`N9 z3!qXW<%DBoS<6|FVaOuczX+aa3szEvCM6%+-B>YtmQ=*d%+SIHUa}0@V1UDJpL~C$ zs}W4^Qf#prTmrryY4mwy`4p9WIXIQS8Z2#Y-g>G=l7N%cY)BRi_=RNxV-D*JVWKgH z8-)jTq&jwjuC-*dSF~#Z^`M9_^p%2hiDz{!zV?@6FYrdjPqUGi!r0lyZM^WYz05F& z;{n%-6$`KXVQ<%6leAb#aOJKcch0dQp*=IEhlvBW6Tlo+9VE^8>gyKiTE>Hean$Ag5gvoKtgtfH=F`)|?A^w5$kA+1aRPq|^jz{;IQs|K4 z@M9j&Z`E2Pc_%x`a`qy4m3%Mf$hUdl)>9&oA5{6h6O5t^RtINO1l^sl)PagW)8J9A z+|(B{!C_d*!Xm=fN@CFDAC`4!9IXzjj(1sxHG+C*rS;nDb0|ujf=pu5C^un1ov$YT zSoE&cgG92h^ObTYmf2NzT3eWY9;6dK(Z8JKfhkdp?R5q@;k*o#EZU?#7_Z}TeDCFp zl})NZ@a;6ek+#vMyR&WG_RDD(N_3BhLDY0a0&GGNZ(u5n6FBNM)$O2_EQlQ+QT~p zJFk8tM6VT`EmBoK&X86pbNd`VQ(_tZv}!Hz4rNm-j5xe>T>zraBLcP;D%qfE3|&xl zPfECx7CyY>QR!0I*puZiX?hq}`S>j>UIF+(&Nvs(Yb@?LhfkwXwIg%*uR)8wlxTsG zW6T!m8viLr&e^if(B^3$}7sf?~CE}%Js#&c)m%7iZ#1#8=X+>kxUr0RVC`s?`? zDRHWX8V9}sD%u!&EKaEMWyFYYn^caM+hm$*l21p4H%Jefxj{tXWUzq_QqL2cGbtA| zLSR}<)BR`$Hnpabkxv2GDj{ey)OOV?CJl@>3nxLX2o5DE^5DY42Yk^j>Go;5YYX_5@~=rEo}ULybxcO z#nya2wLZ#NpXE`aJ4vnknu#5cn-ODUHyXhcRUEMW^)E*wI9S=2Ap1Nt{+^nzX+JL{ zR`3F@aAJ_H9K}PUUiy;~D$K>sbW8p(ESUfLWBmK;^Oh^j8+^JIp9#X=xqL+MM1ar! z(8TSPt>ALCOS&-?=WQXU2hf7pdS);GeZmV#&DBlKwBuU3*0}y~wnHFHpLCWX$NiXR z$+N<-&BT$P-|vzun^J441QPPzJx<(5vL(vkj1N1e8>dhX#rlh*7?&+1&ax&yC6In| zHD&ic9dlSUjm5L=^&x07v?@rVd$-INBTlhc9|&YUv8 zzpdnaC!&hmwvBBp3`5l2NuTsyU(A{(g6hBRRea*HJK1bg&YmgI@tw{pu=fg8VMF{# zI2c49Xor`N!cVZ#MCKA|xx7wzv|q9PAtJwsi4*oLq3|GCzDU%L>Sw=RZc=YKKeK2m zd96uwdRm1XW^?~}+L}(&<>zDi+r(-HJ_}ym1`g&)gq@S=2HBAo$N);^Pj}so^1^n41Gzo3cHvu1Iusp&P;q*8n!l-+t=La z9cV`t5w_y^c56p>Q$x+>=zeS90fxama-yuWp@{JN0`0TJi=(=FVU*9adB?kYeOsdu zFqPUjX$RFxZ~8aLqLp}s1eY_7mr|+hQS4g*%p`U0aq5QSR4W^11FO6`CSn?luLbp& z#g=6g3p2a`lksYNd2%^#UX|7RalD#2_j&D-Zq-(}vQVD??wCBO#>w162X z(}>?K*bJ;Be;v_gy7#cWi*T-!+YD{RUKK%oEgs@zFP&ELX`v`j@=X7)PP23Sz=rEX zyd=xRt=ei`)6eqopLI$mRS@*HJ(f?wCC!vagFFn2sM|F;dG}8qIyHR1u-Hs4+dg9Q zIy!Hp3uo=9H8|7(ZH^%vU1EJ)D*1smA+(q>+x4}f8V~wOWh@6gZ+#4<&Qyz&PyH(I zLxSb|aZ-F?(YF1`FvRm9(y}Z21%-;z#+mkEEn@sKkGuT)`l98d1Me5qN5&Co?2j(x zcsqkPXHvn1Pmf4?D|Gr=00g=cbyUe9}<}uT+?wl(5 zqg@D!M|hM=d~PwL&HXv0>hO|8nI3Vxqu2`DV3!&~7VSWi%4cqN;ikvZf4QET^5~O> zW8-QXNq_u`F|JRnNjN6*OU>~(4zWkaMOv9b&k(ooP zGeh=@WGFbC`deoicP+*W750hD=QGnIwTx3Ol{e)xoa&z?Qdr#0k4bulc%f?#2F6ti zTi^kIBJ4}b-I3djVPQC;U;Mye;|=5ec>A>Chor+=j{C&}l_FCkHkOez0~t`XtduOrf*FLOYnb z);>RzilDKXC?5zjOT9OGmFbZrtM3QFvM1Oaup{VPny)XHT+<>oou(_Y%dL7{yhOy* zW(NS-Q_+*%&K7Q;6Q90+rMYI!3v`|>O27NGa9Fux|| z2EIg$D81HsZdX3TIzEOePBX?7iWoWs>MPuKQOM{J`2vheAVmQ$RAV9>ra9cSJAfN? z0LG3M&3Hl6_u~4N{W(y+SCMA|4YSVCT(Y`;j2RY5O<(Km4Xm6E%3n2j@;MEfSW}d} z`a?jOmC3O1Me^YCkyN~C577BbXReOlHbPgS*cYBs7{7Fyf|K-v{u&SD6$R7_8|Nx{ zg4SB5hcE5=h;yw;)lO<9{W-rh6r>YS|0WalcYMU=B;#ya1U4v=1zB$9ct&;-5R$e( z_B7YJ6bZuIZxPM*w!Hh|yj9WocI73%hVo0^)j>C^>2)Zb4sv>M}T|2FL z?#1&*9H^6MEUu2WeiYJtT#qmidOCbi#U~bTtxLgV<)!nI;K!MCKw9vnvPb`@e{_5T zu9mqiXHII!bqvZEE2xi;WI`17v+v&iwIXcN0m`yiNqw7Zs9nQ;P~0ov0&an^7aZst zq|=!^07VW?hH{k|r7js55E=S<7M07$j8QqFTQ_S~sw#iOz|iKKQ?+by^_}jK!pUkY z#pwHNUS-$-#&)jF3`lW;hK$v?xWH<>Vzbq$muU4QsvjAp!o(gsl%#$OJfEhQ;x&y! zpq=@x$``GwZ5O{jcmnF59<_Bu_18TSrjEwGIG5TpCtJ4rrO&4ppLP2A{_`g z;}@!Yy3HP4Zczlute7f)wb8qlhJB@JZisBYJ%hu6#b4IbDvb(A%6!D_cH0U|7RNv) zak#))x^T|$H!}*NJ{ng|(zm)6mpbC1zUDx`Ea!pnVdb?P^mJ-OVYP^llGY$O=W=^Y zjP{z4m~mW_XHzD^;wtudK&3TrWdD~~S4*mJN8U~l5Ko0V&w3o1gqye~`w^u{^HlVR zBa#(H@uai{PxeI-H%QMkzE8tlDXPc2>MyWs0DtqKv2ii&v4P_TPn-IE*Pn!9u)iVf z6PUbStNML%;{_8dr}qzzMMSNDdA!AXnqzuO)wv;EU zV%ycBLKHVdA15N{XQ0A4UI!=7icCZ|qq(vfeF%Wkbv}D}i8qYyBFw@|<|$lXTb2{E zJCJ$=2@QO#Ft$7lEu0#Yt(~5UaQJxA!&!lG1sKJ;f;)IFo@i^r=x! zMyHiu<`Y_rbyzOLzg6p`>L)=;R97cuR9BUz-R0$d;rWSqz9G=RMJCGHyY8X`2Yw@F z*>$fZwUFsw4mYPCAjZb+Vr`Yyg%GPNpOWb?$*0%_rxMy@4rcigiNgIcdmCEDa)n~( zUZ|15EbLigkf+9?a4Z`~e4!FYSZdgbTQTA1C5zJTO&5EOuM@mZclEmAD}3d+068wv zy7wkT%I`3EK6_%dp=?QujdwONaf|*n76GG6m5yy)k@&(UPOaiP9%(J?=HrI8^I#Gz0l?v;F$-((l3} zYGW)A={Q60uA5#lQu54-GvZ^;v8R6tTDV3`h%Z*f>e|zSP{pe6PH*#`yu9$ybl|La3hIW$IT=GwjBP36 zl!FZ;+>d57sDOuR?9+8Zv`X`Kl>blr4*$zF?f>;J?pz#vJ+sRELjah>BbtehnF-{K zL(wN|5hJ=?L_Z4IZ4^Ce8bv(>4Vw+3mu%^#&E+%P$-4#~J7p?7Ym+zRiZooidY9?{h7X0 zG{Un+1xp!}SheJ~n3W~8y4A`%`aX$DU+3LA%*^>9|JQ{P+DT!&aV?5?q&~49uQ+WR z0#IxUJ@Jrj?{bQ9^v~`+3(9SV?0Is}-@Lpe3)Y;@5cN?Gz~-k5vtEWCNEUg5iWLnE zCUyOXbQO@o%!)W)jjepIX!2UVBrFK>#`;-E%mKRO6>%FM_(}etq;q^`@8~D`F>;J( z_ME(>2)^so>|T9s!qiwmKhKHn=k_S_hP}s!?l`MPU*;Zun6w2P&lX^CCSvczX|_wT zrh{P!Q$kfAH|c%@(`4J*ZH!=$GtdamrJPk^y%-1)k`Z1(vJskc6@SAS8naI~D`g z?5vEe){*s{-N+83yLTR(@BQd&KC}%Ts~tDM98TWFW(ANTm2Tesjwh1ny=~$d3SVgd z`wfy#z8`l(atHeH-yN(#(>gaMHl4hZ zEdRD>U%HJ)xKL@J3VvshhSI)-oKHI zQ_0^KO+rScZLI`%P!S;9?+oRWBX1Mlv2M`rGfK`rP?l;?j5p-!fkaVBZ#G$z=7-nT zJ1@8bZ2fa;8GYARE#q4e&3@8}iHwdZc*5MgT56^9Sdn)ML$~|^l{%N&(wocTAMdlE z$D~-5W^qvo7j=XVlNL`D09N^|Pk+OmdDUM8~i19OVClq~1sRDlG17dKfc);+!)ulL}XZBAe#n1FA3hDYa*5HKdw$*CQRNIjVSJaem|vGDj@x7HmbCFztAQCuQa0*o({;3f(1`Z=_{ugIVpz(e{g5yBMz<2}ZW4Pl7K#i;}+EwZ9 zdxI_wLkHdT&@q6z2L7%k15j!gb$0uA&`)vOieFW)eS&S2CVvHcrV3tC-a*N5)X&Q} zct`SHyM&jqB*QmdJBh8Ci$D-SrH_)qHXE~k6aOQYO~*y(z#g5*ZxB0?MZk~Ibf`D$ z;#rEk=JXpNt_~8(`mx!<;N-v}P+qEUe4S;Ncr_pOE2=MdJJW8n?Le5em-zL}IrgB{ zk1N%0#e~+f?tBgzx)Xx_Fv8mDD45`Rs%*~8c_x8l%?v>?K7OiVaNs0Mq8-4eA#4Fh zk@bJsJWlvf?O4`$&#UBW&FSRgV~5#-#iz34_#R;@c^CMwX~0OiyDvn7SxM8DBf!bf zs*DRe(;eP;10ng2uMQe<{UPXf2*Qvr{<{;#|KNW2epfx)*>8H;vuY50mLdO2foAUs zkWQfn3w+gG&YM))$uv|v{L&nJK8dBIo25!=iW(MuZV<3+dj;qkl{S1n-SihjG2zds z&tIwSkWm|JY2Hqli=yO7HG6#mHa_w6l(qJEZ`j4w;N_Ax9Z7 z$`&QlZyEnOAV27MqhvUruyLh>ZR}B2I;;Fx)>PnsrEF6n(OH8B|60+ooF>o>& z*#Uuw6puNl#%ksbX9VXMj>K81>cyQKu9!618&0mdA0LY7!iV}~;NxnhQ|+C1@6~2i z0Y%p!JrvmM>Y+G6*fV9e(u9M#J_ldN_z1Nh9sQSmp+Ex_Sv{!CeK*h76G={QTyNll z`J)M^Izq6P#UeWaZCK&mr@U!%*y*L|dCJ+ZV^fUAvsvl5 zoz`xrd)WREJmj^Vy@!<5_|fy4s@m3(rSl!=YzEGD$9gDJgH@05`&I|BZxSr}`4r4a ze?bxOXkCWvLW^|XUC8X(J5T!$K_7sZnb^D8m3z6s;6&(0Str9VYQ6ttGOekKQw){Z z&@i1h?N;-I*|ij!IqossYNE1)0kO{G31Nhtgpus*kUs?PDAe|eLbVEJ3}1q8#w@ck z9KeJ(Bkrs;B?W#>1ov42K)pA%ABSrqnT!KdPhtafpAL?Oi?3LQDlnT_iHA-l9~)dc z7<}lJ@2v#?Mh10+l#|5llgI7VGFGOm~@$z z)-%;=ldoE6LxW;2F;8aRA@agY71BIBsl$ur9>&v=x}_Bhl^OTwQ0}&QWOY3Ka0{GO z$dt~hm^nE*-#(&85!X;Ypqoo#yII)H{VJbFRDiyDa@NSWH@>hB?rRx(wSR^V02Wt?iHWN zs$zW)XGQ)GMbsDiclPI6yY9 z_@NufmO!#X&@onCVWn4FL))nV&(gPR$Z-)yd9c_vc8iWIUW-wse`@BLmX@QaU{fp6 zwMWwym2+2;j@gsj5wt+vXXkUPgtD`bccFg>BoutFQV>)MCPx+XFF%?u{JT@r{RGMi zCt&FJ`Y3IKvScmYt);}!RQG&vPlk8Oplx;_P5xZbnsgN%&_9jj;!ng0fibD5uy7bBuir{>ALGOSgBs{B&<+|UwhPxw+~Zn-p?2dV2ooA zO}*k$rZx(eC18qz_;~%j^=%}{k^&UT|2am`lw0&oVATQ=-pRuXQ)(C?MWdx1SICs!DNI?pSepFxjyEO(+`RoKs$b0tSCs4 zA8vH|yng5FG7J1fC#$&Z`B|;b4WOhE*+0f;nuUk3$|eb?Y)&OQ4XRl$sQQ51S+gn% zM>}*;>S%{f-d~+rR#0G+7M$F9H)T@D#QHd~IsTu--L)}82Wv&|h53^NUW}!Hm{%&E zKfqws_sQH^s>Z{SUdIi~ig3Z8m;3l2Z>={ZufeKN47GHl$^Y0=_@zmH=MLfnwO_!& zXHBVj(j8n*8>mB=A$LyVJ?|3$06;26A z^6?(-2~Iv+n|Rsb+Bf-!;3$17ejJ@r>?0>xBUDH}0^ z-^c|<(47tfJf>+}&XKd|6=S)r5Hqd7G~pK8P{X{9o9DC3*0|Oc zLfK;PRDSO8fHUV>-Umb08~J8&nzwl;PcNQ|2k^99?2Dt(<92Oc!gy!*X|#_0Mo4}j zZNEUm5+6t~t2sVaohtheK_*+t^y>hEaf=P~-&)NJLRa}3{J;Cd_Z8dhcl@l}r~cQXOPk>*&G|oLxfJ>Qy!csMCVPI|2hY^Skq)^Cc~Xb7dmB)2tL=DI3ap-B3))Cw=~*Q+A&U|G?t;X z9s6JuZ0+lnSNapuU<1ZLr;ZJ@V`&W3-uWzGGkh!aFl~-}qT~GPqlqSkhGl{WaXm{; zZ+bSMwv+i7wFHDsW95%60RzP;6|T+sT_W|}KLlth`BhPQqtN}2|L&aoKer!6pYlx* z_JDG=)eyMsiD^I7iIvC*=F z!EaOdYm-YxNEzMg*r2?0MvDxGOl3C;EJ08A)AbilzE8#I*+MK*|AMmzaK<}Iflpjl z;V{12XY1J@l~Q)A+yJj3x$ZARM~ukI&%ZViCW0Ohqz#Bczz zhIRf@&+g^74?7AQ%W0ZlI*N}x*s@wv8={C>T3jDPG)?J~BDt~8ojJqa3#p?>e`|O2 zdM=5RpC=UOW#TkG77{B4tB~?(pMce-kf{4q0?dy6Y>t}deEfSgP>0vNRq>L)@eUdi zt4m=u+3|>dBaIEx0BYc*`8k%!*NNhY>n~SrmrkY7Wt+@U@^x!zMLSn@erA^sfKSHzvLZ;@43dT9J^M8Ng1dkhPe;c@WN|#b5`ulNZnu#y}*7ed% z1ReWQ8>VXzg~Qk@vjG)8f)^~xn$U-S#sY2OpLcU`b+Zs!hiYvOjve%lpA_$n~>jLrKpxDvjGv*qS zgxEO)WdYN>oQgbNwRg|jtc?oVOGL}+CwtgVu`7SyAs!u-NQ}#ecPuf56y4nk)G8M} zLeDb&At-}8kKa*vdrsQg@W>=j=V{_EB}n-h?1pc?j(;Sx;}1dmC3qel$6ls2O(O6^_XQF*ako06(R%?aUF zHBPnk+v23imh%OA0N|xpvcS%6Ve{I;E0|a!E*D5F(Z9o0q>=9EN<32%T>lhE@b`N`<@dZMBbv@`&7&DTvH&-k-FR*Dp=7*bp|!7P#(8dL zR<+@SCz|_-t#qb9C0pE`kGXq&9LYBiEbf-vY3Wgm{R~8w?e?rJ zZiO@yJ~ct@wK2$a73(qT$up1}yRhZ6JFBi|pZj<+o)8&&|MRpV%f~&%%Oq$u#Jake z%eUJxVv1TzVpM7>B~CAC%+=1=3?l9-R_YlWs$lEcJTcplJKVq{i+us7MDP9$X>_i) z-9{=A!l*x?I*mJnVLQk)Mibc`j~8${%5qMyDIIa^_M~5)&yjd1AT2GIH-94Vy4bl9`P z{Vy+4+Ohhm1>ja#ipWxEeYLg*w3KE68Ke4FwBNXGFa(KKu=VxM#6Md|lWgib|5dfT zjXH7s@)olVQc~4u7mYWGa;eGs9jF?IL6*cM>W>3yI+LrH6@cY$_C+Ma21KMYJ@`Vl z5jM8Qba3l)`rS51(HPziC!k(=#VPIg&1C-Zf&SKWJu%4zJ1@y2OrCHNKBW-&OKC(o;|fm2*5 z;EiQ>_qi$4tZmm&7Wcvna+d*jQn1z04IVbU+g_S8%V}#I&#=JH{FuD*>Dfys%irnW zjSx>C0i*FX6eS(zg0K<)5YW;&MTwA9u3ctgD8f$lIO1AQP-94X?p>MYuT`%#!YXNv zpX%W9RNI7SGDTqQjT3LYn5@OwbH<&R1s^I0h-&$PZK;NL0&V{m#}%GlqQpjdXxVlI z^~%Q6B9mbuKrsVX0SyVvcye_cV*mgX|3lE@!1n&GQKn)=qVEU%cw#6 z*H)iMWM*hjF{3|dXvbz>OI#baqqyqj5OJ?Qg7HDpyml@}*#zHDVJ#Cd8%lTDxyVm* z;}OMsfhawzGRA}k_ZWZlJ}A-RLN`G7qw}f>q7d|{$YGlKMeD?pbpU-kS7(?9k;!rV zk5-$Sd8F_$p7Zi$TR&=_e^TF`={dojp}4E^r`pa2G@O{;WZ~YDP&rG+-F8_YV6~-t ze)#1$@6+oq+jn`W&8eR#@m&cuiIUiBI0T71I_nGlxSqqyq6MTfaT=$0^*1GPm@jth zbugO#vp~MA6esoVLRpgeVRgbk*vSWla(B1;^H4HTUM6799h}~wKwgN&Xu$pe{?VrI zOCDPTZcUJ2Hv1pE=Qe;a)uQ~Eskl(u1V66j$SbX?8-uOMACNRVO|fs--!uE!@*`84at-k#M^8 z8q%h(GK^5tzh-W8u=5v&C#}h-$%j{0O+|YGgFJHF``X3Gs_RC!Uav#fTM$d+kj|-@ z`_I|sF+h$N|68G2;&4g!TFyLi++=gW{)|x93XsD;+syLQ&U)3J$s4v@2T6^O4=p27 z=t@xeX2k%9%~i1}gJ1)F#BThs`JjnkI4l!#pJ%V!&@VHI>x1}TEHIy+E|v&aMyL&# z_dDY7W~g4KkXlz91&5cn{u){wCo8MdD>B-JB$t=bp(fR7R*<8?H^oFet6bza6`tI+mkSUG5I|78~$CEeZ4UeURpPUS(lztB;OPE zF~jyfs<|_|M*C4R@1z=|=Nj+K%W}JH!eLtV`q|d_aNW4Nxu0u#MEG~=kDs}}XZISR zJz{+b)`8`)N%Y-1%v*TkImQK1(69C!kNUo8n9f1FJ!km#Z?Aa&(>oFwUgaD~`j))K zV3}@L(I2F*HIr!RmMkD`#gN zKeX1jM+*4+W~`p&PV8w5Acn2apYdC7%rDvC*sBbm(~139n)}qR|HZmzJXa9FHd=8U z(tA!}UdF>0ls|l+GOU)?jtNUe*>UI`fCzI(lalfpVVB|>O^w4n%U8K@D>q^z#$%7| zJ*JU5FTTH4-U7@J_mQ5>?d5)bgn@`|V_OEtq!Q-Z-${!?|R>EUkee~k10 z-Obz1eIlig!EyC@3#7c>8eWe_i3D^LKl{GxF?iHn2C9bV--LF?3dxE}t)EG*CzWOI zn0HR!{=#A~AMO$!w!AH!VC2t_%qCagx&7O*YHP1H0HnR}k3F6mr-*TjD4Y1HznDVr zPOP}%`Fqk1LsV_0Qz~Q*(Fm6nM||qPj3Y`%>rtI-zdskc;0o*^hSz2-adx(RNzhO_ z!6!x%>A1AsqE{cB$>>sdwQDh|U-RE+YeMeg8oy6tBYzdaH?ltXpfb`p6$5D6iIn!O mRUM_Bu}pTg-l^{gY<2zRzl{*~{5M|7|9|Lz%L2iFCjS@nv{R7) diff --git a/model_cards/unideeplearning/polibert_sa/README.md b/model_cards/unideeplearning/polibert_sa/README.md deleted file mode 100644 index 7a71dc9361..0000000000 --- a/model_cards/unideeplearning/polibert_sa/README.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -language: it -tags: -- sentiment -- Italian -license: MIT -widget: -- text: 'Giuseppe Rossi è un ottimo politico' ---- - -# 🤗 + polibert_SA - POLItic BERT based Sentiment Analysis - -## Model description - -This model performs sentiment analysis on Italian political twitter sentences. It was trained starting from an instance of "bert-base-italian-uncased-xxl" and fine-tuned on an Italian dataset of tweets. You can try it out at https://www.unideeplearning.com/twitter_sa/ (in italian!) - -#### Hands-on - -```python -import torch -from torch import nn -from transformers import AutoTokenizer, AutoModelForSequenceClassification - -tokenizer = AutoTokenizer.from_pretrained("unideeplearning/polibert_sa") -model = AutoModelForSequenceClassification.from_pretrained("unideeplearning/polibert_sa") - - - - -text = "Giuseppe Rossi è un pessimo politico" -input_ids = tokenizer.encode(text, add_special_tokens=True, return_tensors= 'pt') - -logits, = model(input_ids) -logits = logits.squeeze(0) - -prob = nn.functional.softmax(logits, dim=0) - -# 0 Negative, 1 Neutral, 2 Positive -print(prob.argmax().tolist()) -``` - -#### Hyperparameters - -- Optimizer: **AdamW** with learning rate of **2e-5**, epsilon of **1e-8** -- Max epochs: **2** -- Batch size: **16** - -## Acknowledgments - -Thanks to the support from: -the [Hugging Face](https://huggingface.co/), https://www.unioneprofessionisti.com - -https://www.unideeplearning.com/ diff --git a/model_cards/urduhack/roberta-urdu-small/README.md b/model_cards/urduhack/roberta-urdu-small/README.md deleted file mode 100644 index b0d6cfbf40..0000000000 --- a/model_cards/urduhack/roberta-urdu-small/README.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -language: ur -thumbnail: https://raw.githubusercontent.com/urduhack/urduhack/master/docs/_static/urduhack.png -tags: -- roberta-urdu-small -- urdu -- transformers -license: mit ---- -## roberta-urdu-small - -[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/urduhack/urduhack/blob/master/LICENSE) -### Overview -**Language model:** roberta-urdu-small -**Model size:** 125M -**Language:** Urdu -**Training data:** News data from urdu news resources in Pakistan -### About roberta-urdu-small -roberta-urdu-small is a language model for urdu language. -``` -from transformers import pipeline -fill_mask = pipeline("fill-mask", model="urduhack/roberta-urdu-small", tokenizer="urduhack/roberta-urdu-small") -``` -## Training procedure -roberta-urdu-small was trained on urdu news corpus. Training data was normalized using normalization module from -urduhack to eliminate characters from other languages like arabic. - -### About Urduhack -Urduhack is a Natural Language Processing (NLP) library for urdu language. -Github: https://github.com/urduhack/urduhack diff --git a/model_cards/valhalla/bart-large-finetuned-squadv1/README.md b/model_cards/valhalla/bart-large-finetuned-squadv1/README.md deleted file mode 100644 index e53087c1ba..0000000000 --- a/model_cards/valhalla/bart-large-finetuned-squadv1/README.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -datasets: -- squad ---- - -# BART-LARGE finetuned on SQuADv1 - -This is bart-large model finetuned on SQuADv1 dataset for question answering task - -## Model details -BART was propsed in the [paper](https://arxiv.org/abs/1910.13461) **BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension**. -BART is a seq2seq model intended for both NLG and NLU tasks. - -To use BART for question answering tasks, we feed the complete document into the encoder and decoder, and use the top -hidden state of the decoder as a representation for each -word. This representation is used to classify the token. As given in the paper bart-large achives comparable to ROBERTa on SQuAD. -Another notable thing about BART is that it can handle sequences with upto 1024 tokens. - -| Param | #Value | -|---------------------|--------| -| encoder layers | 12 | -| decoder layers | 12 | -| hidden size | 4096 | -| num attetion heads | 16 | -| on disk size | 1.63GB | - - -## Model training -This model was trained on google colab v100 GPU. -You can find the fine-tuning colab here -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1I5cK1M_0dLaf5xoewh6swcm5nAInfwHy?usp=sharing). - - -## Results -The results are actually slightly worse than given in the paper. -In the paper the authors mentioned that bart-large achieves 88.8 EM and 94.6 F1 - -| Metric | #Value | -|--------|--------| -| EM | 86.8022| -| F1 | 92.7342| - - -## Model in Action 🚀 -```python3 -from transformers import BartTokenizer, BartForQuestionAnswering -import torch - -tokenizer = BartTokenizer.from_pretrained('valhalla/bart-large-finetuned-squadv1') -model = BartForQuestionAnswering.from_pretrained('valhalla/bart-large-finetuned-squadv1') - -question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" -encoding = tokenizer(question, text, return_tensors='pt') -input_ids = encoding['input_ids'] -attention_mask = encoding['attention_mask'] - -start_scores, end_scores = model(input_ids, attention_mask=attention_mask, output_attentions=False)[:2] - -all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0]) -answer = ' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]) -answer = tokenizer.convert_tokens_to_ids(answer.split()) -answer = tokenizer.decode(answer) -#answer => 'a nice puppet' -``` - -> Created with ❤️ by Suraj Patil [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/patil-suraj/) -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/psuraj28) diff --git a/model_cards/valhalla/distilbart-mnli-12-1/README.md b/model_cards/valhalla/distilbart-mnli-12-1/README.md deleted file mode 100644 index fd41fa7191..0000000000 --- a/model_cards/valhalla/distilbart-mnli-12-1/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -datasets: -- mnli -tags: -- distilbart -- distilbart-mnli -pipeline_tag: zero-shot-classification ---- - -# DistilBart-MNLI - -distilbart-mnli is the distilled version of bart-large-mnli created using the **No Teacher Distillation** technique proposed for BART summarisation by Huggingface, [here](https://github.com/huggingface/transformers/tree/master/examples/seq2seq#distilbart). - -We just copy alternating layers from `bart-large-mnli` and finetune more on the same data. - - -| | matched acc | mismatched acc | -| ------------------------------------------------------------------------------------ | ----------- | -------------- | -| [bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) (baseline, 12-12) | 89.9 | 90.01 | -| [distilbart-mnli-12-1](https://huggingface.co/valhalla/distilbart-mnli-12-1) | 87.08 | 87.5 | -| [distilbart-mnli-12-3](https://huggingface.co/valhalla/distilbart-mnli-12-3) | 88.1 | 88.19 | -| [distilbart-mnli-12-6](https://huggingface.co/valhalla/distilbart-mnli-12-6) | 89.19 | 89.01 | -| [distilbart-mnli-12-9](https://huggingface.co/valhalla/distilbart-mnli-12-9) | 89.56 | 89.52 | - - -This is a very simple and effective technique, as we can see the performance drop is very little. - -Detailed performace trade-offs will be posted in this [sheet](https://docs.google.com/spreadsheets/d/1dQeUvAKpScLuhDV1afaPJRRAE55s2LpIzDVA5xfqxvk/edit?usp=sharing). - - -## Fine-tuning -If you want to train these models yourself, clone the [distillbart-mnli repo](https://github.com/patil-suraj/distillbart-mnli) and follow the steps below - -Clone and install transformers from source -```bash -git clone https://github.com/huggingface/transformers.git -pip install -qqq -U ./transformers -``` - -Download MNLI data -```bash -python transformers/utils/download_glue_data.py --data_dir glue_data --tasks MNLI -``` - -Create student model -```bash -python create_student.py \ - --teacher_model_name_or_path facebook/bart-large-mnli \ - --student_encoder_layers 12 \ - --student_decoder_layers 6 \ - --save_path student-bart-mnli-12-6 \ -``` - -Start fine-tuning -```bash -python run_glue.py args.json -``` - -You can find the logs of these trained models in this [wandb project](https://wandb.ai/psuraj/distilbart-mnli). \ No newline at end of file diff --git a/model_cards/valhalla/distilbart-mnli-12-3/README.md b/model_cards/valhalla/distilbart-mnli-12-3/README.md deleted file mode 100644 index fd41fa7191..0000000000 --- a/model_cards/valhalla/distilbart-mnli-12-3/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -datasets: -- mnli -tags: -- distilbart -- distilbart-mnli -pipeline_tag: zero-shot-classification ---- - -# DistilBart-MNLI - -distilbart-mnli is the distilled version of bart-large-mnli created using the **No Teacher Distillation** technique proposed for BART summarisation by Huggingface, [here](https://github.com/huggingface/transformers/tree/master/examples/seq2seq#distilbart). - -We just copy alternating layers from `bart-large-mnli` and finetune more on the same data. - - -| | matched acc | mismatched acc | -| ------------------------------------------------------------------------------------ | ----------- | -------------- | -| [bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) (baseline, 12-12) | 89.9 | 90.01 | -| [distilbart-mnli-12-1](https://huggingface.co/valhalla/distilbart-mnli-12-1) | 87.08 | 87.5 | -| [distilbart-mnli-12-3](https://huggingface.co/valhalla/distilbart-mnli-12-3) | 88.1 | 88.19 | -| [distilbart-mnli-12-6](https://huggingface.co/valhalla/distilbart-mnli-12-6) | 89.19 | 89.01 | -| [distilbart-mnli-12-9](https://huggingface.co/valhalla/distilbart-mnli-12-9) | 89.56 | 89.52 | - - -This is a very simple and effective technique, as we can see the performance drop is very little. - -Detailed performace trade-offs will be posted in this [sheet](https://docs.google.com/spreadsheets/d/1dQeUvAKpScLuhDV1afaPJRRAE55s2LpIzDVA5xfqxvk/edit?usp=sharing). - - -## Fine-tuning -If you want to train these models yourself, clone the [distillbart-mnli repo](https://github.com/patil-suraj/distillbart-mnli) and follow the steps below - -Clone and install transformers from source -```bash -git clone https://github.com/huggingface/transformers.git -pip install -qqq -U ./transformers -``` - -Download MNLI data -```bash -python transformers/utils/download_glue_data.py --data_dir glue_data --tasks MNLI -``` - -Create student model -```bash -python create_student.py \ - --teacher_model_name_or_path facebook/bart-large-mnli \ - --student_encoder_layers 12 \ - --student_decoder_layers 6 \ - --save_path student-bart-mnli-12-6 \ -``` - -Start fine-tuning -```bash -python run_glue.py args.json -``` - -You can find the logs of these trained models in this [wandb project](https://wandb.ai/psuraj/distilbart-mnli). \ No newline at end of file diff --git a/model_cards/valhalla/distilbart-mnli-12-6/README.md b/model_cards/valhalla/distilbart-mnli-12-6/README.md deleted file mode 100644 index fd41fa7191..0000000000 --- a/model_cards/valhalla/distilbart-mnli-12-6/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -datasets: -- mnli -tags: -- distilbart -- distilbart-mnli -pipeline_tag: zero-shot-classification ---- - -# DistilBart-MNLI - -distilbart-mnli is the distilled version of bart-large-mnli created using the **No Teacher Distillation** technique proposed for BART summarisation by Huggingface, [here](https://github.com/huggingface/transformers/tree/master/examples/seq2seq#distilbart). - -We just copy alternating layers from `bart-large-mnli` and finetune more on the same data. - - -| | matched acc | mismatched acc | -| ------------------------------------------------------------------------------------ | ----------- | -------------- | -| [bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) (baseline, 12-12) | 89.9 | 90.01 | -| [distilbart-mnli-12-1](https://huggingface.co/valhalla/distilbart-mnli-12-1) | 87.08 | 87.5 | -| [distilbart-mnli-12-3](https://huggingface.co/valhalla/distilbart-mnli-12-3) | 88.1 | 88.19 | -| [distilbart-mnli-12-6](https://huggingface.co/valhalla/distilbart-mnli-12-6) | 89.19 | 89.01 | -| [distilbart-mnli-12-9](https://huggingface.co/valhalla/distilbart-mnli-12-9) | 89.56 | 89.52 | - - -This is a very simple and effective technique, as we can see the performance drop is very little. - -Detailed performace trade-offs will be posted in this [sheet](https://docs.google.com/spreadsheets/d/1dQeUvAKpScLuhDV1afaPJRRAE55s2LpIzDVA5xfqxvk/edit?usp=sharing). - - -## Fine-tuning -If you want to train these models yourself, clone the [distillbart-mnli repo](https://github.com/patil-suraj/distillbart-mnli) and follow the steps below - -Clone and install transformers from source -```bash -git clone https://github.com/huggingface/transformers.git -pip install -qqq -U ./transformers -``` - -Download MNLI data -```bash -python transformers/utils/download_glue_data.py --data_dir glue_data --tasks MNLI -``` - -Create student model -```bash -python create_student.py \ - --teacher_model_name_or_path facebook/bart-large-mnli \ - --student_encoder_layers 12 \ - --student_decoder_layers 6 \ - --save_path student-bart-mnli-12-6 \ -``` - -Start fine-tuning -```bash -python run_glue.py args.json -``` - -You can find the logs of these trained models in this [wandb project](https://wandb.ai/psuraj/distilbart-mnli). \ No newline at end of file diff --git a/model_cards/valhalla/distilbart-mnli-12-9/README.md b/model_cards/valhalla/distilbart-mnli-12-9/README.md deleted file mode 100644 index fd41fa7191..0000000000 --- a/model_cards/valhalla/distilbart-mnli-12-9/README.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -datasets: -- mnli -tags: -- distilbart -- distilbart-mnli -pipeline_tag: zero-shot-classification ---- - -# DistilBart-MNLI - -distilbart-mnli is the distilled version of bart-large-mnli created using the **No Teacher Distillation** technique proposed for BART summarisation by Huggingface, [here](https://github.com/huggingface/transformers/tree/master/examples/seq2seq#distilbart). - -We just copy alternating layers from `bart-large-mnli` and finetune more on the same data. - - -| | matched acc | mismatched acc | -| ------------------------------------------------------------------------------------ | ----------- | -------------- | -| [bart-large-mnli](https://huggingface.co/facebook/bart-large-mnli) (baseline, 12-12) | 89.9 | 90.01 | -| [distilbart-mnli-12-1](https://huggingface.co/valhalla/distilbart-mnli-12-1) | 87.08 | 87.5 | -| [distilbart-mnli-12-3](https://huggingface.co/valhalla/distilbart-mnli-12-3) | 88.1 | 88.19 | -| [distilbart-mnli-12-6](https://huggingface.co/valhalla/distilbart-mnli-12-6) | 89.19 | 89.01 | -| [distilbart-mnli-12-9](https://huggingface.co/valhalla/distilbart-mnli-12-9) | 89.56 | 89.52 | - - -This is a very simple and effective technique, as we can see the performance drop is very little. - -Detailed performace trade-offs will be posted in this [sheet](https://docs.google.com/spreadsheets/d/1dQeUvAKpScLuhDV1afaPJRRAE55s2LpIzDVA5xfqxvk/edit?usp=sharing). - - -## Fine-tuning -If you want to train these models yourself, clone the [distillbart-mnli repo](https://github.com/patil-suraj/distillbart-mnli) and follow the steps below - -Clone and install transformers from source -```bash -git clone https://github.com/huggingface/transformers.git -pip install -qqq -U ./transformers -``` - -Download MNLI data -```bash -python transformers/utils/download_glue_data.py --data_dir glue_data --tasks MNLI -``` - -Create student model -```bash -python create_student.py \ - --teacher_model_name_or_path facebook/bart-large-mnli \ - --student_encoder_layers 12 \ - --student_decoder_layers 6 \ - --save_path student-bart-mnli-12-6 \ -``` - -Start fine-tuning -```bash -python run_glue.py args.json -``` - -You can find the logs of these trained models in this [wandb project](https://wandb.ai/psuraj/distilbart-mnli). \ No newline at end of file diff --git a/model_cards/valhalla/electra-base-discriminator-finetuned_squadv1/README.md b/model_cards/valhalla/electra-base-discriminator-finetuned_squadv1/README.md deleted file mode 100644 index e51eebe8ae..0000000000 --- a/model_cards/valhalla/electra-base-discriminator-finetuned_squadv1/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# ELECTRA-BASE-DISCRIMINATOR finetuned on SQuADv1 - -This is electra-base-discriminator model finetuned on SQuADv1 dataset for for question answering task. - -## Model details -As mentioned in the original paper: ELECTRA is a new method for self-supervised language representation learning. -It can be used to pre-train transformer networks using relatively little compute. -ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, -similar to the discriminator of a GAN. At small scale, ELECTRA achieves strong results even when trained on a single GPU. -At large scale, ELECTRA achieves state-of-the-art results on the SQuAD 2.0 dataset. - -| Param | #Value | -|---------------------|--------| -| layers | 12 | -| hidden size | 768 | -| num attetion heads | 12 | -| on disk size | 436MB | - -## Model training -This model was trained on google colab v100 GPU. -You can find the fine-tuning colab here -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/11yo-LaFsgggwmDSy2P8zD3tzf5cCb-DU?usp=sharing). - -## Results -The results are actually slightly better than given in the paper. -In the paper the authors mentioned that electra-base achieves 84.5 EM and 90.8 F1 - -| Metric | #Value | -|--------|--------| -| EM | 85.0520| -| F1 | 91.6050| - - -## Model in Action 🚀 -```python3 -from transformers import pipeline - -nlp = pipeline('question-answering', model='valhalla/electra-base-discriminator-finetuned_squadv1') -nlp({ - 'question': 'What is the answer to everything ?', - 'context': '42 is the answer to life the universe and everything' -}) -=> {'answer': '42', 'end': 2, 'score': 0.981274963050339, 'start': 0} -``` - -> Created with ❤️ by Suraj Patil [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/patil-suraj/) -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/psuraj28) diff --git a/model_cards/valhalla/longformer-base-4096-finetuned-squadv1/README.md b/model_cards/valhalla/longformer-base-4096-finetuned-squadv1/README.md deleted file mode 100644 index b6b096bf47..0000000000 --- a/model_cards/valhalla/longformer-base-4096-finetuned-squadv1/README.md +++ /dev/null @@ -1,54 +0,0 @@ -# LONGFORMER-BASE-4096 fine-tuned on SQuAD v1 -This is longformer-base-4096 model fine-tuned on SQuAD v1 dataset for question answering task. - -[Longformer](https://arxiv.org/abs/2004.05150) model created by Iz Beltagy, Matthew E. Peters, Arman Coha from AllenAI. As the paper explains it - -> `Longformer` is a BERT-like model for long documents. - -The pre-trained model can handle sequences with upto 4096 tokens. - - -## Model Training -This model was trained on google colab v100 GPU. You can find the fine-tuning colab here [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1zEl5D-DdkBKva-DdreVOmN0hrAfzKG1o?usp=sharing). - -Few things to keep in mind while training longformer for QA task, -by default longformer uses sliding-window local attention on all tokens. But For QA, all question tokens should have global attention. For more details on this please refer the paper. The `LongformerForQuestionAnswering` model automatically does that for you. To allow it to do that -1. The input sequence must have three sep tokens, i.e the sequence should be encoded like this - ` question context`. If you encode the question and answer as a input pair, then the tokenizer already takes care of that, you shouldn't worry about it. -2. `input_ids` should always be a batch of examples. - -## Results -|Metric | # Value | -|-------------|---------| -| Exact Match | 85.1466 | -| F1 | 91.5415 | - -## Model in Action 🚀 -```python -import torch -from transformers import AutoTokenizer, AutoModelForQuestionAnswering, - -tokenizer = AutoTokenizer.from_pretrained("valhalla/longformer-base-4096-finetuned-squadv1") -model = AutoModelForQuestionAnswering.from_pretrained("valhalla/longformer-base-4096-finetuned-squadv1") - -text = "Huggingface has democratized NLP. Huge thanks to Huggingface for this." -question = "What has Huggingface done ?" -encoding = tokenizer(question, text, return_tensors="pt") -input_ids = encoding["input_ids"] - -# default is local attention everywhere -# the forward method will automatically set global attention on question tokens -attention_mask = encoding["attention_mask"] - -start_scores, end_scores = model(input_ids, attention_mask=attention_mask) -all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist()) - -answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1] -answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens)) -# output => democratized NLP -``` - -The `LongformerForQuestionAnswering` isn't yet supported in `pipeline` . I'll update this card once the support has been added. - -> Created with ❤️ by Suraj Patil [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/patil-suraj/) -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/psuraj28) diff --git a/model_cards/valhalla/t5-base-e2e-qg/README.md b/model_cards/valhalla/t5-base-e2e-qg/README.md deleted file mode 100644 index f9536523a7..0000000000 --- a/model_cards/valhalla/t5-base-e2e-qg/README.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: "Python is a programming language. It is developed by Guido Van Rossum and released in 1991. " -license: mit ---- - -## T5 for question-generation -This is [t5-base](https://arxiv.org/abs/1910.10683) model trained for end-to-end question generation task. Simply input the text and the model will generate multile questions. - -You can play with the model using the inference API, just put the text and see the results! - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline - -text = "Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum \ -and first released in 1991, Python's design philosophy emphasizes code \ -readability with its notable use of significant whitespace." - -nlp = pipeline("e2e-qg", model="valhalla/t5-base-e2e-qg") -nlp(text) -=> [ - 'Who created Python?', - 'When was Python first released?', - "What is Python's design philosophy?" -] -``` \ No newline at end of file diff --git a/model_cards/valhalla/t5-base-qa-qg-hl/README.md b/model_cards/valhalla/t5-base-qa-qg-hl/README.md deleted file mode 100644 index 5e2ef575bc..0000000000 --- a/model_cards/valhalla/t5-base-qa-qg-hl/README.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: "generate question: 42 is the answer to life, the universe and everything. " -- text: "question: What is 42 context: 42 is the answer to life, the universe and everything. " -license: mit ---- - -## T5 for multi-task QA and QG -This is multi-task [t5-base](https://arxiv.org/abs/1910.10683) model trained for question answering and answer aware question generation tasks. - -For question generation the answer spans are highlighted within the text with special highlight tokens (``) and prefixed with 'generate question: '. For QA the input is processed like this `question: question_text context: context_text ` - -You can play with the model using the inference API. Here's how you can use it - -For QG - -`generate question: 42 is the answer to life, the universe and everything. ` - -For QA - -`question: What is 42 context: 42 is the answer to life, the universe and everything. ` - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline -nlp = pipeline("multitask-qa-qg", model="valhalla/t5-base-qa-qg-hl") - -# to generate questions simply pass the text -nlp("42 is the answer to life, the universe and everything.") -=> [{'answer': '42', 'question': 'What is the answer to life, the universe and everything?'}] - -# for qa pass a dict with "question" and "context" -nlp({ - "question": "What is 42 ?", - "context": "42 is the answer to life, the universe and everything." -}) -=> 'the answer to life, the universe and everything' -``` \ No newline at end of file diff --git a/model_cards/valhalla/t5-base-qg-hl/README.md b/model_cards/valhalla/t5-base-qg-hl/README.md deleted file mode 100644 index 40d603dda5..0000000000 --- a/model_cards/valhalla/t5-base-qg-hl/README.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: " 42 is the answer to life, the universe and everything. " -- text: "Python is a programming language. It is developed by Guido Van Rossum . " -- text: "Although practicality beats purity " -license: mit ---- - -## T5 for question-generation -This is [t5-base](https://arxiv.org/abs/1910.10683) model trained for answer aware question generation task. The answer spans are highlighted within the text with special highlight tokens. - -You can play with the model using the inference API, just highlight the answer spans with `` tokens and end the text with ``. For example - -` 42 is the answer to life, the universe and everything. ` - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline -nlp = pipeline("question-generation", model="valhalla/t5-base-qg-hl") -nlp("42 is the answer to life, universe and everything.") -=> [{'answer': '42', 'question': 'What is the answer to life, universe and everything?'}] -``` \ No newline at end of file diff --git a/model_cards/valhalla/t5-base-squad/README.md b/model_cards/valhalla/t5-base-squad/README.md deleted file mode 100644 index 18f314759a..0000000000 --- a/model_cards/valhalla/t5-base-squad/README.md +++ /dev/null @@ -1,38 +0,0 @@ -# T5 for question-answering -This is T5-base model fine-tuned on SQuAD1.1 for QA using text-to-text approach - -## Model training -This model was trained on colab TPU with 35GB RAM for 4 epochs - -## Results: -| Metric | #Value | -|-------------|---------| -| Exact Match | 81.5610 | -| F1 | 89.9601 | - -## Model in Action 🚀 -``` -from transformers import AutoModelWithLMHead, AutoTokenizer - -tokenizer = AutoTokenizer.from_pretrained("valhalla/t5-base-squad") -model = AutoModelWithLMHead.from_pretrained("valhalla/t5-base-squad") - -def get_answer(question, context): - input_text = "question: %s context: %s " % (question, context) - features = tokenizer([input_text], return_tensors='pt') - - out = model.generate(input_ids=features['input_ids'], - attention_mask=features['attention_mask']) - - return tokenizer.decode(out[0]) - -context = "In Norse mythology, Valhalla is a majestic, enormous hall located in Asgard, ruled over by the god Odin." -question = "What is Valhalla ?" - -get_answer(question, context) -# output: 'a majestic, enormous hall located in Asgard, ruled over by the god Odin' -``` -Play with this model [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1a5xpJiUjZybfU9Mi-aDkOp116PZ9-wni?usp=sharing) - -> Created by Suraj Patil [![Github icon](https://cdn0.iconfinder.com/data/icons/octicons/1024/mark-github-32.png)](https://github.com/patil-suraj/) -[![Twitter icon](https://cdn0.iconfinder.com/data/icons/shift-logotypes/32/Twitter-32.png)](https://twitter.com/psuraj28) diff --git a/model_cards/valhalla/t5-samll-qg-prepend/README.md b/model_cards/valhalla/t5-samll-qg-prepend/README.md deleted file mode 100644 index c4f35c3da8..0000000000 --- a/model_cards/valhalla/t5-samll-qg-prepend/README.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: "answer: 42 context: 42 is the answer to life, the universe and everything. " -- text: "answer: Guido Van Rossum context: Python is a programming language. It is developed by Guido Van Rossum. " -- text: "answer: Explicit context: Explicit is better than implicit " -license: mit ---- - -## T5 for question-generation -This is [t5-small](https://arxiv.org/abs/1910.10683) model trained for answer aware question generation task. The answer text is prepended before the context text. - -You can play with the model using the inference API, just get the input text in this format and see the results! -`answer: answer_text context: context_text ` - -For example - -`answer: 42 context: 42 is the answer to life, the universe and everything. ` - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline -nlp = pipeline("question-generation", qg_format="prepend") -nlp("42 is the answer to life, universe and everything.") -=> [{'answer': '42', 'question': 'What is the answer to life, universe and everything?'}] -``` \ No newline at end of file diff --git a/model_cards/valhalla/t5-small-e2e-qg/README.md b/model_cards/valhalla/t5-small-e2e-qg/README.md deleted file mode 100644 index 743b0334e9..0000000000 --- a/model_cards/valhalla/t5-small-e2e-qg/README.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: "Python is developed by Guido Van Rossum and released in 1991. " -license: mit ---- - -## T5 for question-generation -This is [t5-small](https://arxiv.org/abs/1910.10683) model trained for end-to-end question generation task. Simply input the text and the model will generate multile questions. - -You can play with the model using the inference API, just put the text and see the results! - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline - -text = "Python is an interpreted, high-level, general-purpose programming language. Created by Guido van Rossum \ -and first released in 1991, Python's design philosophy emphasizes code \ -readability with its notable use of significant whitespace." - -nlp = pipeline("e2e-qg") -nlp(text) -=> [ - 'Who created Python?', - 'When was Python first released?', - "What is Python's design philosophy?" -] -``` \ No newline at end of file diff --git a/model_cards/valhalla/t5-small-qa-qg-hl/README.md b/model_cards/valhalla/t5-small-qa-qg-hl/README.md deleted file mode 100644 index 7f3a24c346..0000000000 --- a/model_cards/valhalla/t5-small-qa-qg-hl/README.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: "generate question: 42 is the answer to life, the universe and everything. " -- text: "question: What is 42 context: 42 is the answer to life, the universe and everything. " -license: mit ---- - -## T5 for multi-task QA and QG -This is multi-task [t5-small](https://arxiv.org/abs/1910.10683) model trained for question answering and answer aware question generation tasks. - -For question generation the answer spans are highlighted within the text with special highlight tokens (``) and prefixed with 'generate question: '. For QA the input is processed like this `question: question_text context: context_text ` - -You can play with the model using the inference API. Here's how you can use it - -For QG - -`generate question: 42 is the answer to life, the universe and everything. ` - -For QA - -`question: What is 42 context: 42 is the answer to life, the universe and everything. ` - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline -nlp = pipeline("multitask-qa-qg") - -# to generate questions simply pass the text -nlp("42 is the answer to life, the universe and everything.") -=> [{'answer': '42', 'question': 'What is the answer to life, the universe and everything?'}] - -# for qa pass a dict with "question" and "context" -nlp({ - "question": "What is 42 ?", - "context": "42 is the answer to life, the universe and everything." -}) -=> 'the answer to life, the universe and everything' -``` \ No newline at end of file diff --git a/model_cards/valhalla/t5-small-qg-hl/README.md b/model_cards/valhalla/t5-small-qg-hl/README.md deleted file mode 100644 index ca295f1dfa..0000000000 --- a/model_cards/valhalla/t5-small-qg-hl/README.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -datasets: -- squad -tags: -- question-generation -widget: -- text: " 42 is the answer to life, the universe and everything. " -- text: "Python is a programming language. It is developed by Guido Van Rossum . " -- text: "Simple is better than complex . " -license: mit ---- - -## T5 for question-generation -This is [t5-small](https://arxiv.org/abs/1910.10683) model trained for answer aware question generation task. The answer spans are highlighted within the text with special highlight tokens. - -You can play with the model using the inference API, just highlight the answer spans with `` tokens and end the text with ``. For example - -` 42 is the answer to life, the universe and everything. ` - -For more deatils see [this](https://github.com/patil-suraj/question_generation) repo. - -### Model in action 🚀 - -You'll need to clone the [repo](https://github.com/patil-suraj/question_generation). - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/patil-suraj/question_generation/blob/master/question_generation.ipynb) - -```python3 -from pipelines import pipeline -nlp = pipeline("question-generation") -nlp("42 is the answer to life, universe and everything.") -=> [{'answer': '42', 'question': 'What is the answer to life, universe and everything?'}] -``` \ No newline at end of file diff --git a/model_cards/vinai/bertweet-base/README.md b/model_cards/vinai/bertweet-base/README.md deleted file mode 100644 index 4d6b041f5d..0000000000 --- a/model_cards/vinai/bertweet-base/README.md +++ /dev/null @@ -1,80 +0,0 @@ -# BERTweet: A pre-trained language model for English Tweets - - - BERTweet is the first public large-scale language model pre-trained for English Tweets. BERTweet is trained based on the [RoBERTa](https://github.com/pytorch/fairseq/blob/master/examples/roberta/README.md) pre-training procedure, using the same model configuration as [BERT-base](https://github.com/google-research/bert). - - The corpus used to pre-train BERTweet consists of 850M English Tweets (16B word tokens ~ 80GB), containing 845M Tweets streamed from 01/2012 to 08/2019 and 5M Tweets related to the **COVID-19** pandemic. - - BERTweet does better than its competitors RoBERTa-base and [XLM-R-base](https://arxiv.org/abs/1911.02116) and outperforms previous state-of-the-art models on three downstream Tweet NLP tasks of Part-of-speech tagging, Named entity recognition and text classification. - -The general architecture and experimental results of BERTweet can be found in our [paper](https://arxiv.org/abs/2005.10200): - - @inproceedings{bertweet, - title = {{BERTweet: A pre-trained language model for English Tweets}}, - author = {Dat Quoc Nguyen and Thanh Vu and Anh Tuan Nguyen}, - booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations}, - year = {2020} - } - -**Please CITE** our paper when BERTweet is used to help produce published results or is incorporated into other software. - -For further information or requests, please go to [BERTweet's homepage](https://github.com/VinAIResearch/BERTweet)! - -### Installation - - - Python 3.6+, and PyTorch 1.1.0+ (or TensorFlow 2.0+) - - Install `transformers`: - - `git clone https://github.com/huggingface/transformers.git` - - `cd transformers` - - `pip3 install --upgrade .` - - Install `emoji`: `pip3 install emoji` - -### Pre-trained models - - -Model | #params | Arch. | Pre-training data ----|---|---|--- -`vinai/bertweet-base` | 135M | base | 845M English Tweets (cased) -`vinai/bertweet-covid19-base-cased` | 135M | base | 23M COVID-19 English Tweets (cased) -`vinai/bertweet-covid19-base-uncased` | 135M | base | 23M COVID-19 English Tweets (uncased) - -Two pre-trained models `vinai/bertweet-covid19-base-cased` and `vinai/bertweet-covid19-base-uncased` are resulted by further pre-training the pre-trained model `vinai/bertweet-base` on a corpus of 23M COVID-19 English Tweets for 40 epochs. - -### Example usage - - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -bertweet = AutoModel.from_pretrained("vinai/bertweet-base") -tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base") - -# INPUT TWEET IS ALREADY NORMALIZED! -line = "SC has first two presumptive cases of coronavirus , DHEC confirms HTTPURL via @USER :cry:" - -input_ids = torch.tensor([tokenizer.encode(line)]) - -with torch.no_grad(): - features = bertweet(input_ids) # Models outputs are now tuples - -## With TensorFlow 2.0+: -# from transformers import TFAutoModel -# bertweet = TFAutoModel.from_pretrained("vinai/bertweet-base") -``` - -### Normalize raw input Tweets - -Before applying `fastBPE` to the pre-training corpus of 850M English Tweets, we tokenized these Tweets using `TweetTokenizer` from the NLTK toolkit and used the `emoji` package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens `@USER` and `HTTPURL`, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets. BERTweet provides this pre-processing step by enabling the `normalization` argument. - -```python -import torch -from transformers import AutoTokenizer - -# Load the AutoTokenizer with a normalization mode if the input Tweet is raw -tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", normalization=True) - -# from transformers import BertweetTokenizer -# tokenizer = BertweetTokenizer.from_pretrained("vinai/bertweet-base", normalization=True) - -line = "SC has first two presumptive cases of coronavirus, DHEC confirms https://postandcourier.com/health/covid19/sc-has-first-two-presumptive-cases-of-coronavirus-dhec-confirms/article_bddfe4ae-5fd3-11ea-9ce4-5f495366cee6.html?utm_medium=social&utm_source=twitter&utm_campaign=user-share… via @postandcourier" - -input_ids = torch.tensor([tokenizer.encode(line)]) -``` diff --git a/model_cards/vinai/bertweet-covid19-base-cased/README.md b/model_cards/vinai/bertweet-covid19-base-cased/README.md deleted file mode 100644 index e09c71e4b7..0000000000 --- a/model_cards/vinai/bertweet-covid19-base-cased/README.md +++ /dev/null @@ -1,80 +0,0 @@ -# BERTweet: A pre-trained language model for English Tweets - - - BERTweet is the first public large-scale language model pre-trained for English Tweets. BERTweet is trained based on the [RoBERTa](https://github.com/pytorch/fairseq/blob/master/examples/roberta/README.md) pre-training procedure, using the same model configuration as [BERT-base](https://github.com/google-research/bert). - - The corpus used to pre-train BERTweet consists of 850M English Tweets (16B word tokens ~ 80GB), containing 845M Tweets streamed from 01/2012 to 08/2019 and 5M Tweets related to the **COVID-19** pandemic. - - BERTweet does better than its competitors RoBERTa-base and [XLM-R-base](https://arxiv.org/abs/1911.02116) and outperforms previous state-of-the-art models on three downstream Tweet NLP tasks of Part-of-speech tagging, Named entity recognition and text classification. - -The general architecture and experimental results of BERTweet can be found in our [paper](https://arxiv.org/abs/2005.10200): - - @inproceedings{bertweet, - title = {{BERTweet: A pre-trained language model for English Tweets}}, - author = {Dat Quoc Nguyen and Thanh Vu and Anh Tuan Nguyen}, - booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations}, - year = {2020} - } - -**Please CITE** our paper when BERTweet is used to help produce published results or is incorporated into other software. - -For further information or requests, please go to [BERTweet's homepage](https://github.com/VinAIResearch/BERTweet)! - -### Installation - - - Python 3.6+, and PyTorch 1.1.0+ (or TensorFlow 2.0+) - - Install `transformers`: - - `git clone https://github.com/huggingface/transformers.git` - - `cd transformers` - - `pip3 install --upgrade .` - - Install `emoji`: `pip3 install emoji` - -### Pre-trained models - - -Model | #params | Arch. | Pre-training data ----|---|---|--- -`vinai/bertweet-base` | 135M | base | 845M English Tweets (cased) -`vinai/bertweet-covid19-base-cased` | 135M | base | 23M COVID-19 English Tweets (cased) -`vinai/bertweet-covid19-base-uncased` | 135M | base | 23M COVID-19 English Tweets (uncased) - -Two pre-trained models `vinai/bertweet-covid19-base-cased` and `vinai/bertweet-covid19-base-uncased` are resulted by further pre-training the pre-trained model `vinai/bertweet-base` on a corpus of 23M COVID-19 English Tweets for 40 epochs. - -### Example usage - - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -bertweet = AutoModel.from_pretrained("vinai/bertweet-covid19-base-cased") -tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-covid19-base-cased") - -# INPUT TWEET IS ALREADY NORMALIZED! -line = "SC has first two presumptive cases of coronavirus , DHEC confirms HTTPURL via @USER :cry:" - -input_ids = torch.tensor([tokenizer.encode(line)]) - -with torch.no_grad(): - features = bertweet(input_ids) # Models outputs are now tuples - -## With TensorFlow 2.0+: -# from transformers import TFAutoModel -# bertweet = TFAutoModel.from_pretrained("vinai/bertweet-covid19-base-cased") -``` - -### Normalize raw input Tweets - -Before applying `fastBPE` to the pre-training corpus of 850M English Tweets, we tokenized these Tweets using `TweetTokenizer` from the NLTK toolkit and used the `emoji` package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens `@USER` and `HTTPURL`, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets. BERTweet provides this pre-processing step by enabling the `normalization` argument. - -```python -import torch -from transformers import AutoTokenizer - -# Load the AutoTokenizer with a normalization mode if the input Tweet is raw -tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-covid19-base-cased", normalization=True) - -# from transformers import BertweetTokenizer -# tokenizer = BertweetTokenizer.from_pretrained("vinai/bertweet-covid19-base-cased", normalization=True) - -line = "SC has first two presumptive cases of coronavirus, DHEC confirms https://postandcourier.com/health/covid19/sc-has-first-two-presumptive-cases-of-coronavirus-dhec-confirms/article_bddfe4ae-5fd3-11ea-9ce4-5f495366cee6.html?utm_medium=social&utm_source=twitter&utm_campaign=user-share… via @postandcourier" - -input_ids = torch.tensor([tokenizer.encode(line)]) -``` diff --git a/model_cards/vinai/bertweet-covid19-base-uncased/README.md b/model_cards/vinai/bertweet-covid19-base-uncased/README.md deleted file mode 100644 index 4f807de06a..0000000000 --- a/model_cards/vinai/bertweet-covid19-base-uncased/README.md +++ /dev/null @@ -1,80 +0,0 @@ -# BERTweet: A pre-trained language model for English Tweets - - - BERTweet is the first public large-scale language model pre-trained for English Tweets. BERTweet is trained based on the [RoBERTa](https://github.com/pytorch/fairseq/blob/master/examples/roberta/README.md) pre-training procedure, using the same model configuration as [BERT-base](https://github.com/google-research/bert). - - The corpus used to pre-train BERTweet consists of 850M English Tweets (16B word tokens ~ 80GB), containing 845M Tweets streamed from 01/2012 to 08/2019 and 5M Tweets related to the **COVID-19** pandemic. - - BERTweet does better than its competitors RoBERTa-base and [XLM-R-base](https://arxiv.org/abs/1911.02116) and outperforms previous state-of-the-art models on three downstream Tweet NLP tasks of Part-of-speech tagging, Named entity recognition and text classification. - -The general architecture and experimental results of BERTweet can be found in our [paper](https://arxiv.org/abs/2005.10200): - - @inproceedings{bertweet, - title = {{BERTweet: A pre-trained language model for English Tweets}}, - author = {Dat Quoc Nguyen and Thanh Vu and Anh Tuan Nguyen}, - booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations}, - year = {2020} - } - -**Please CITE** our paper when BERTweet is used to help produce published results or is incorporated into other software. - -For further information or requests, please go to [BERTweet's homepage](https://github.com/VinAIResearch/BERTweet)! - -### Installation - - - Python 3.6+, and PyTorch 1.1.0+ (or TensorFlow 2.0+) - - Install `transformers`: - - `git clone https://github.com/huggingface/transformers.git` - - `cd transformers` - - `pip3 install --upgrade .` - - Install `emoji`: `pip3 install emoji` - -### Pre-trained models - - -Model | #params | Arch. | Pre-training data ----|---|---|--- -`vinai/bertweet-base` | 135M | base | 845M English Tweets (cased) -`vinai/bertweet-covid19-base-cased` | 135M | base | 23M COVID-19 English Tweets (cased) -`vinai/bertweet-covid19-base-uncased` | 135M | base | 23M COVID-19 English Tweets (uncased) - -Two pre-trained models `vinai/bertweet-covid19-base-cased` and `vinai/bertweet-covid19-base-uncased` are resulted by further pre-training the pre-trained model `vinai/bertweet-base` on a corpus of 23M COVID-19 English Tweets for 40 epochs. - -### Example usage - - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -bertweet = AutoModel.from_pretrained("vinai/bertweet-covid19-base-uncased") -tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-covid19-base-uncased") - -# INPUT TWEET IS ALREADY NORMALIZED! -line = "SC has first two presumptive cases of coronavirus , DHEC confirms HTTPURL via @USER :cry:" - -input_ids = torch.tensor([tokenizer.encode(line)]) - -with torch.no_grad(): - features = bertweet(input_ids) # Models outputs are now tuples - -## With TensorFlow 2.0+: -# from transformers import TFAutoModel -# bertweet = TFAutoModel.from_pretrained("vinai/bertweet-covid19-base-uncased") -``` - -### Normalize raw input Tweets - -Before applying `fastBPE` to the pre-training corpus of 850M English Tweets, we tokenized these Tweets using `TweetTokenizer` from the NLTK toolkit and used the `emoji` package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens `@USER` and `HTTPURL`, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets. BERTweet provides this pre-processing step by enabling the `normalization` argument. - -```python -import torch -from transformers import AutoTokenizer - -# Load the AutoTokenizer with a normalization mode if the input Tweet is raw -tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-covid19-base-uncased", normalization=True) - -# from transformers import BertweetTokenizer -# tokenizer = BertweetTokenizer.from_pretrained("vinai/bertweet-covid19-base-uncased", normalization=True) - -line = "SC has first two presumptive cases of coronavirus, DHEC confirms https://postandcourier.com/health/covid19/sc-has-first-two-presumptive-cases-of-coronavirus-dhec-confirms/article_bddfe4ae-5fd3-11ea-9ce4-5f495366cee6.html?utm_medium=social&utm_source=twitter&utm_campaign=user-share… via @postandcourier" - -input_ids = torch.tensor([tokenizer.encode(line)]) -``` diff --git a/model_cards/vinai/phobert-base/README.md b/model_cards/vinai/phobert-base/README.md deleted file mode 100644 index afae1177c7..0000000000 --- a/model_cards/vinai/phobert-base/README.md +++ /dev/null @@ -1,55 +0,0 @@ -# PhoBERT: Pre-trained language models for Vietnamese - -Pre-trained PhoBERT models are the state-of-the-art language models for Vietnamese ([Pho](https://en.wikipedia.org/wiki/Pho), i.e. "Phở", is a popular food in Vietnam): - - - Two PhoBERT versions of "base" and "large" are the first public large-scale monolingual language models pre-trained for Vietnamese. PhoBERT pre-training approach is based on [RoBERTa](https://github.com/pytorch/fairseq/blob/master/examples/roberta/README.md) which optimizes the [BERT](https://github.com/google-research/bert) pre-training procedure for more robust performance. - - PhoBERT outperforms previous monolingual and multilingual approaches, obtaining new state-of-the-art performances on four downstream Vietnamese NLP tasks of Part-of-speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. - -The general architecture and experimental results of PhoBERT can be found in our EMNLP-2020 Findings [paper](https://arxiv.org/abs/2003.00744): - - @article{phobert, - title = {{PhoBERT: Pre-trained language models for Vietnamese}}, - author = {Dat Quoc Nguyen and Anh Tuan Nguyen}, - journal = {Findings of EMNLP}, - year = {2020} - } - -**Please CITE** our paper when PhoBERT is used to help produce published results or is incorporated into other software. - -For further information or requests, please go to [PhoBERT's homepage](https://github.com/VinAIResearch/PhoBERT)! - -### Installation - - Python 3.6+, and PyTorch 1.1.0+ (or TensorFlow 2.0+) - - Install `transformers`: - - `git clone https://github.com/huggingface/transformers.git` - - `cd transformers` - - `pip3 install --upgrade .` - -### Pre-trained models - -Model | #params | Arch. | Pre-training data ----|---|---|--- -`vinai/phobert-base` | 135M | base | 20GB of texts -`vinai/phobert-large` | 370M | large | 20GB of texts - -### Example usage - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -phobert = AutoModel.from_pretrained("vinai/phobert-base") -tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-base") - -# INPUT TEXT MUST BE ALREADY WORD-SEGMENTED! -line = "Tôi là sinh_viên trường đại_học Công_nghệ ." - -input_ids = torch.tensor([tokenizer.encode(line)]) - -with torch.no_grad(): - features = phobert(input_ids) # Models outputs are now tuples - -## With TensorFlow 2.0+: -# from transformers import TFAutoModel -# phobert = TFAutoModel.from_pretrained("vinai/phobert-base") -``` diff --git a/model_cards/vinai/phobert-large/README.md b/model_cards/vinai/phobert-large/README.md deleted file mode 100644 index 7bbf4521ef..0000000000 --- a/model_cards/vinai/phobert-large/README.md +++ /dev/null @@ -1,55 +0,0 @@ -# PhoBERT: Pre-trained language models for Vietnamese - -Pre-trained PhoBERT models are the state-of-the-art language models for Vietnamese ([Pho](https://en.wikipedia.org/wiki/Pho), i.e. "Phở", is a popular food in Vietnam): - - - Two PhoBERT versions of "base" and "large" are the first public large-scale monolingual language models pre-trained for Vietnamese. PhoBERT pre-training approach is based on [RoBERTa](https://github.com/pytorch/fairseq/blob/master/examples/roberta/README.md) which optimizes the [BERT](https://github.com/google-research/bert) pre-training procedure for more robust performance. - - PhoBERT outperforms previous monolingual and multilingual approaches, obtaining new state-of-the-art performances on four downstream Vietnamese NLP tasks of Part-of-speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. - -The general architecture and experimental results of PhoBERT can be found in our EMNLP-2020 Findings [paper](https://arxiv.org/abs/2003.00744): - - @article{phobert, - title = {{PhoBERT: Pre-trained language models for Vietnamese}}, - author = {Dat Quoc Nguyen and Anh Tuan Nguyen}, - journal = {Findings of EMNLP}, - year = {2020} - } - -**Please CITE** our paper when PhoBERT is used to help produce published results or is incorporated into other software. - -For further information or requests, please go to [PhoBERT's homepage](https://github.com/VinAIResearch/PhoBERT)! - -### Installation - - Python 3.6+, and PyTorch 1.1.0+ (or TensorFlow 2.0+) - - Install `transformers`: - - `git clone https://github.com/huggingface/transformers.git` - - `cd transformers` - - `pip3 install --upgrade .` - -### Pre-trained models - -Model | #params | Arch. | Pre-training data ----|---|---|--- -`vinai/phobert-base` | 135M | base | 20GB of texts -`vinai/phobert-large` | 370M | large | 20GB of texts - -### Example usage - -```python -import torch -from transformers import AutoModel, AutoTokenizer - -phobert = AutoModel.from_pretrained("vinai/phobert-large") -tokenizer = AutoTokenizer.from_pretrained("vinai/phobert-large") - -# INPUT TEXT MUST BE ALREADY WORD-SEGMENTED! -line = "Tôi là sinh_viên trường đại_học Công_nghệ ." - -input_ids = torch.tensor([tokenizer.encode(line)]) - -with torch.no_grad(): - features = phobert(input_ids) # Models outputs are now tuples - -## With TensorFlow 2.0+: -# from transformers import TFAutoModel -# phobert = TFAutoModel.from_pretrained("vinai/phobert-large") -``` diff --git a/model_cards/voidful/albert_chinese_base/README.md b/model_cards/voidful/albert_chinese_base/README.md deleted file mode 100644 index d86eec8871..0000000000 --- a/model_cards/voidful/albert_chinese_base/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# albert_chinese_base - -This a albert_chinese_base model from [Google's github](https://github.com/google-research/ALBERT) -converted by huggingface's [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py) - -## Attention (注意) - -Since sentencepiece is not used in albert_chinese_base model -you have to call BertTokenizer instead of AlbertTokenizer !!! -we can eval it using an example on MaskedLM - -由於 albert_chinese_base 模型沒有用 sentencepiece -用AlbertTokenizer會載不進詞表，因此需要改用BertTokenizer !!! -我們可以跑MaskedLM預測來驗證這個做法是否正確 - -## Justify (驗證有效性) -[colab trial](https://colab.research.google.com/drive/1Wjz48Uws6-VuSHv_-DcWLilv77-AaYgj) -```python -from transformers import * -import torch -from torch.nn.functional import softmax - -pretrained = 'voidful/albert_chinese_base' -tokenizer = BertTokenizer.from_pretrained(pretrained) -model = AlbertForMaskedLM.from_pretrained(pretrained) - -inputtext = "今天[MASK]情很好" - -maskpos = tokenizer.encode(inputtext, add_special_tokens=True).index(103) - -input_ids = torch.tensor(tokenizer.encode(inputtext, add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids, masked_lm_labels=input_ids) -loss, prediction_scores = outputs[:2] -logit_prob = softmax(prediction_scores[0, maskpos]).data.tolist() -predicted_index = torch.argmax(prediction_scores[0, maskpos]).item() -predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0] -print(predicted_token,logit_prob[predicted_index]) -``` -Result: `感 0.36333346366882324` diff --git a/model_cards/voidful/albert_chinese_large/README.md b/model_cards/voidful/albert_chinese_large/README.md deleted file mode 100644 index 4311563c04..0000000000 --- a/model_cards/voidful/albert_chinese_large/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# albert_chinese_large - -This a albert_chinese_large model from [Google's github](https://github.com/google-research/ALBERT) -converted by huggingface's [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py) - -## Attention (注意) - -Since sentencepiece is not used in albert_chinese_large model -you have to call BertTokenizer instead of AlbertTokenizer !!! -we can eval it using an example on MaskedLM - -由於 albert_chinese_large 模型沒有用 sentencepiece -用AlbertTokenizer會載不進詞表，因此需要改用BertTokenizer !!! -我們可以跑MaskedLM預測來驗證這個做法是否正確 - -## Justify (驗證有效性) -[colab trial](https://colab.research.google.com/drive/1Wjz48Uws6-VuSHv_-DcWLilv77-AaYgj) -```python -from transformers import * -import torch -from torch.nn.functional import softmax - -pretrained = 'voidful/albert_chinese_large' -tokenizer = BertTokenizer.from_pretrained(pretrained) -model = AlbertForMaskedLM.from_pretrained(pretrained) - -inputtext = "今天[MASK]情很好" - -maskpos = tokenizer.encode(inputtext, add_special_tokens=True).index(103) - -input_ids = torch.tensor(tokenizer.encode(inputtext, add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids, masked_lm_labels=input_ids) -loss, prediction_scores = outputs[:2] -logit_prob = softmax(prediction_scores[0, maskpos]).data.tolist() -predicted_index = torch.argmax(prediction_scores[0, maskpos]).item() -predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0] -print(predicted_token,logit_prob[predicted_index]) -``` -Result: `心 0.9422469735145569` diff --git a/model_cards/voidful/albert_chinese_small/README.md b/model_cards/voidful/albert_chinese_small/README.md deleted file mode 100644 index 00a212752f..0000000000 --- a/model_cards/voidful/albert_chinese_small/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# albert_chinese_small - -This a albert_chinese_small model from [brightmart/albert_zh project](https://github.com/brightmart/albert_zh), albert_small_google_zh model -converted by huggingface's [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py) - -## Attention (注意) - -Since sentencepiece is not used in albert_chinese_small model -you have to call BertTokenizer instead of AlbertTokenizer !!! -we can eval it using an example on MaskedLM - -由於 albert_chinese_small 模型沒有用 sentencepiece -用AlbertTokenizer會載不進詞表，因此需要改用BertTokenizer !!! -我們可以跑MaskedLM預測來驗證這個做法是否正確 - -## Justify (驗證有效性) -[colab trial](https://colab.research.google.com/drive/1Wjz48Uws6-VuSHv_-DcWLilv77-AaYgj) -```python -from transformers import * -import torch -from torch.nn.functional import softmax - -pretrained = 'voidful/albert_chinese_small' -tokenizer = BertTokenizer.from_pretrained(pretrained) -model = AlbertForMaskedLM.from_pretrained(pretrained) - -inputtext = "今天[MASK]情很好" - -maskpos = tokenizer.encode(inputtext, add_special_tokens=True).index(103) - -input_ids = torch.tensor(tokenizer.encode(inputtext, add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids, masked_lm_labels=input_ids) -loss, prediction_scores = outputs[:2] -logit_prob = softmax(prediction_scores[0, maskpos]).data.tolist() -predicted_index = torch.argmax(prediction_scores[0, maskpos]).item() -predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0] -print(predicted_token,logit_prob[predicted_index]) -``` -Result: `感 0.6390823125839233` diff --git a/model_cards/voidful/albert_chinese_tiny/README.md b/model_cards/voidful/albert_chinese_tiny/README.md deleted file mode 100644 index 6d495bb49c..0000000000 --- a/model_cards/voidful/albert_chinese_tiny/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# albert_chinese_tiny - -This a albert_chinese_tiny model from [brightmart/albert_zh project](https://github.com/brightmart/albert_zh), albert_tiny_google_zh model -converted by huggingface's [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py) - -## Attention (注意) - -Since sentencepiece is not used in albert_chinese_tiny model -you have to call BertTokenizer instead of AlbertTokenizer !!! -we can eval it using an example on MaskedLM - -由於 albert_chinese_tiny 模型沒有用 sentencepiece -用AlbertTokenizer會載不進詞表，因此需要改用BertTokenizer !!! -我們可以跑MaskedLM預測來驗證這個做法是否正確 - -## Justify (驗證有效性) -[colab trial](https://colab.research.google.com/drive/1Wjz48Uws6-VuSHv_-DcWLilv77-AaYgj) -```python -from transformers import * -import torch -from torch.nn.functional import softmax - -pretrained = 'voidful/albert_chinese_tiny' -tokenizer = BertTokenizer.from_pretrained(pretrained) -model = AlbertForMaskedLM.from_pretrained(pretrained) - -inputtext = "今天[MASK]情很好" - -maskpos = tokenizer.encode(inputtext, add_special_tokens=True).index(103) - -input_ids = torch.tensor(tokenizer.encode(inputtext, add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids, masked_lm_labels=input_ids) -loss, prediction_scores = outputs[:2] -logit_prob = softmax(prediction_scores[0, maskpos]).data.tolist() -predicted_index = torch.argmax(prediction_scores[0, maskpos]).item() -predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0] -print(predicted_token,logit_prob[predicted_index]) -``` -Result: `感 0.40312355756759644` diff --git a/model_cards/voidful/albert_chinese_xlarge/README.md b/model_cards/voidful/albert_chinese_xlarge/README.md deleted file mode 100644 index f5ddba938d..0000000000 --- a/model_cards/voidful/albert_chinese_xlarge/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# albert_chinese_xlarge - -This a albert_chinese_xlarge model from [Google's github](https://github.com/google-research/ALBERT) -converted by huggingface's [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py) - -## Attention (注意) - -Since sentencepiece is not used in albert_chinese_xlarge model -you have to call BertTokenizer instead of AlbertTokenizer !!! -we can eval it using an example on MaskedLM - -由於 albert_chinese_xlarge 模型沒有用 sentencepiece -用AlbertTokenizer會載不進詞表，因此需要改用BertTokenizer !!! -我們可以跑MaskedLM預測來驗證這個做法是否正確 - -## Justify (驗證有效性) -[colab trial](https://colab.research.google.com/drive/1Wjz48Uws6-VuSHv_-DcWLilv77-AaYgj) -```python -from transformers import * -import torch -from torch.nn.functional import softmax - -pretrained = 'voidful/albert_chinese_xlarge' -tokenizer = BertTokenizer.from_pretrained(pretrained) -model = AlbertForMaskedLM.from_pretrained(pretrained) - -inputtext = "今天[MASK]情很好" - -maskpos = tokenizer.encode(inputtext, add_special_tokens=True).index(103) - -input_ids = torch.tensor(tokenizer.encode(inputtext, add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids, masked_lm_labels=input_ids) -loss, prediction_scores = outputs[:2] -logit_prob = softmax(prediction_scores[0, maskpos]).data.tolist() -predicted_index = torch.argmax(prediction_scores[0, maskpos]).item() -predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0] -print(predicted_token,logit_prob[predicted_index]) -``` -Result: `心 0.9942440390586853` diff --git a/model_cards/voidful/albert_chinese_xxlarge/README.md b/model_cards/voidful/albert_chinese_xxlarge/README.md deleted file mode 100644 index 800cfe12a3..0000000000 --- a/model_cards/voidful/albert_chinese_xxlarge/README.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -language: zh ---- - -# albert_chinese_xxlarge - -This a albert_chinese_xxlarge model from [Google's github](https://github.com/google-research/ALBERT) -converted by huggingface's [script](https://github.com/huggingface/transformers/blob/master/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py) - -## Attention (注意) - -Since sentencepiece is not used in albert_chinese_xxlarge model -you have to call BertTokenizer instead of AlbertTokenizer !!! -we can eval it using an example on MaskedLM - -由於 albert_chinese_xxlarge 模型沒有用 sentencepiece -用AlbertTokenizer會載不進詞表，因此需要改用BertTokenizer !!! -我們可以跑MaskedLM預測來驗證這個做法是否正確 - -## Justify (驗證有效性) -[colab trial](https://colab.research.google.com/drive/1Wjz48Uws6-VuSHv_-DcWLilv77-AaYgj) -```python -from transformers import * -import torch -from torch.nn.functional import softmax - -pretrained = 'voidful/albert_chinese_xxlarge' -tokenizer = BertTokenizer.from_pretrained(pretrained) -model = AlbertForMaskedLM.from_pretrained(pretrained) - -inputtext = "今天[MASK]情很好" - -maskpos = tokenizer.encode(inputtext, add_special_tokens=True).index(103) - -input_ids = torch.tensor(tokenizer.encode(inputtext, add_special_tokens=True)).unsqueeze(0) # Batch size 1 -outputs = model(input_ids, masked_lm_labels=input_ids) -loss, prediction_scores = outputs[:2] -logit_prob = softmax(prediction_scores[0, maskpos]).data.tolist() -predicted_index = torch.argmax(prediction_scores[0, maskpos]).item() -predicted_token = tokenizer.convert_ids_to_tokens([predicted_index])[0] -print(predicted_token,logit_prob[predicted_index]) -``` -Result: `心 0.995713472366333` diff --git a/model_cards/wietsedv/bert-base-dutch-cased/README.md b/model_cards/wietsedv/bert-base-dutch-cased/README.md deleted file mode 100644 index ec4c18dab9..0000000000 --- a/model_cards/wietsedv/bert-base-dutch-cased/README.md +++ /dev/null @@ -1,12 +0,0 @@ -# BERTje: A Dutch BERT model - -BERTje is a Dutch pre-trained BERT model developed at the University of Groningen. - -⚠️ **The new home of this model is the [GroNLP](https://huggingface.co/GroNLP) organization.** - -BERTje now lives at: [`GroNLP/bert-base-dutch-cased`](https://huggingface.co/GroNLP/bert-base-dutch-cased) - -The model weights of the versions at `wietsedv/` and `GroNLP/` are the same, so do not worry if you use(d) `wietsedv/bert-base-dutch-cased`. - - - diff --git a/model_cards/wptoux/albert-chinese-large-qa/README.md b/model_cards/wptoux/albert-chinese-large-qa/README.md deleted file mode 100644 index cf43f31714..0000000000 --- a/model_cards/wptoux/albert-chinese-large-qa/README.md +++ /dev/null @@ -1,34 +0,0 @@ -# albert-chinese-large-qa -Albert large QA model pretrained from baidu webqa and baidu dureader datasets. - -## Data source -+ baidu webqa 1.0 -+ baidu dureader - -## Traing Method -We combined the two datasets together and created a new dataset in squad format, including 705139 samples for training and 69638 samples for validation. -We finetune the model based on the albert chinese large model. - -## Hyperparams -+ learning_rate 1e-5 -+ max_seq_length 512 -+ max_query_length 50 -+ max_answer_length 300 -+ doc_stride 256 -+ num_train_epochs 2 -+ warmup_steps 1000 -+ per_gpu_train_batch_size 8 -+ gradient_accumulation_steps 3 -+ n_gpu 2 (Nvidia Tesla P100) - -## Usage -``` -from transformers import AutoModelForQuestionAnswering, BertTokenizer - -model = AutoModelForQuestionAnswering.from_pretrained('wptoux/albert-chinese-large-qa') -tokenizer = BertTokenizer.from_pretrained('wptoux/albert-chinese-large-qa') -``` -***Important: use BertTokenizer*** - -## MoreInfo -Please visit https://github.com/wptoux/albert-chinese-large-webqa for details. diff --git a/model_cards/xlm-mlm-en-2048-README.md b/model_cards/xlm-mlm-en-2048-README.md deleted file mode 100644 index a3a2b5e002..0000000000 --- a/model_cards/xlm-mlm-en-2048-README.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -tags: -- exbert - -license: cc-by-nc-4.0 ---- - - - - diff --git a/model_cards/xlm-roberta-base-README.md b/model_cards/xlm-roberta-base-README.md deleted file mode 100644 index 8230171a7d..0000000000 --- a/model_cards/xlm-roberta-base-README.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -tags: -- exbert - -license: mit ---- - - - - diff --git a/model_cards/xlm-roberta-large-finetuned-conll03-german-README.md b/model_cards/xlm-roberta-large-finetuned-conll03-german-README.md deleted file mode 100644 index e05bb660a3..0000000000 --- a/model_cards/xlm-roberta-large-finetuned-conll03-german-README.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -language: de ---- - -## xlm-roberta-large-finetuned-conll03-german diff --git a/model_cards/yjernite/bart_eli5/README.md b/model_cards/yjernite/bart_eli5/README.md deleted file mode 100644 index 168be620f8..0000000000 --- a/model_cards/yjernite/bart_eli5/README.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -language: en -license: apache-2.0 -datasets: -- eli5 ---- - -## BART ELI5 - -Read the article at https://yjernite.github.io/lfqa.html and try the demo at https://huggingface.co/qa/ diff --git a/model_cards/ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli/README.md b/model_cards/ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli/README.md deleted file mode 100644 index 15679d0e5b..0000000000 --- a/model_cards/ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli/README.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -datasets: -- snli -- anli -- multi_nli -- multi_nli_mismatch -- fever -license: mit ---- -This is a strong pre-trained RoBERTa-Large NLI model. - -The training data is a combination of well-known NLI datasets: [`SNLI`](https://nlp.stanford.edu/projects/snli/), [`MNLI`](https://cims.nyu.edu/~sbowman/multinli/), [`FEVER-NLI`](https://github.com/easonnie/combine-FEVER-NSMN/blob/master/other_resources/nli_fever.md), [`ANLI (R1, R2, R3)`](https://github.com/facebookresearch/anli). -Other pre-trained NLI models including `RoBERTa`, `ALBert`, `BART`, `ELECTRA`, `XLNet` are also available. - -Trained by [Yixin Nie](https://easonnie.github.io), [original source](https://github.com/facebookresearch/anli). - -Try the code snippet below. -``` -from transformers import AutoTokenizer, AutoModelForSequenceClassification -import torch - -if __name__ == '__main__': - max_length = 256 - - premise = "Two women are embracing while holding to go packages." - hypothesis = "The men are fighting outside a deli." - - hg_model_hub_name = "ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli" - # hg_model_hub_name = "ynie/albert-xxlarge-v2-snli_mnli_fever_anli_R1_R2_R3-nli" - # hg_model_hub_name = "ynie/bart-large-snli_mnli_fever_anli_R1_R2_R3-nli" - # hg_model_hub_name = "ynie/electra-large-discriminator-snli_mnli_fever_anli_R1_R2_R3-nli" - # hg_model_hub_name = "ynie/xlnet-large-cased-snli_mnli_fever_anli_R1_R2_R3-nli" - - tokenizer = AutoTokenizer.from_pretrained(hg_model_hub_name) - model = AutoModelForSequenceClassification.from_pretrained(hg_model_hub_name) - - tokenized_input_seq_pair = tokenizer.encode_plus(premise, hypothesis, - max_length=max_length, - return_token_type_ids=True, truncation=True) - - input_ids = torch.Tensor(tokenized_input_seq_pair['input_ids']).long().unsqueeze(0) - # remember bart doesn't have 'token_type_ids', remove the line below if you are using bart. - token_type_ids = torch.Tensor(tokenized_input_seq_pair['token_type_ids']).long().unsqueeze(0) - attention_mask = torch.Tensor(tokenized_input_seq_pair['attention_mask']).long().unsqueeze(0) - - outputs = model(input_ids, - attention_mask=attention_mask, - token_type_ids=token_type_ids, - labels=None) - # Note: - # "id2label": { - # "0": "entailment", - # "1": "neutral", - # "2": "contradiction" - # }, - - predicted_probability = torch.softmax(outputs[0], dim=1)[0].tolist() # batch_size only one - - print("Premise:", premise) - print("Hypothesis:", hypothesis) - print("Entailment:", predicted_probability[0]) - print("Neutral:", predicted_probability[1]) - print("Contradiction:", predicted_probability[2]) -``` - -More in [here](https://github.com/facebookresearch/anli/blob/master/src/hg_api/interactive_eval.py). - -Citation: -``` -@inproceedings{nie-etal-2020-adversarial, - title = "Adversarial {NLI}: A New Benchmark for Natural Language Understanding", - author = "Nie, Yixin and - Williams, Adina and - Dinan, Emily and - Bansal, Mohit and - Weston, Jason and - Kiela, Douwe", - booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics", - year = "2020", - publisher = "Association for Computational Linguistics", -} -``` diff --git a/model_cards/youscan/ukr-roberta-base/README.md b/model_cards/youscan/ukr-roberta-base/README.md deleted file mode 100644 index c698a26d98..0000000000 --- a/model_cards/youscan/ukr-roberta-base/README.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -language: -- uk ---- - -# ukr-roberta-base - -## Pre-training corpora -Below is the list of corpora used along with the output of wc command (counting lines, words and characters). These corpora were concatenated and tokenized with HuggingFace Roberta Tokenizer. - -| Tables | Lines | Words | Characters | -| ------------- |--------------:| -----:| -----:| -| [Ukrainian Wikipedia - May 2020](https://dumps.wikimedia.org/ukwiki/latest/ukwiki-latest-pages-articles.xml.bz2) | 18 001 466| 201 207 739 | 2 647 891 947 | -| [Ukrainian OSCAR deduplicated dataset](https://oscar-public.huma-num.fr/shuffled/uk_dedup.txt.gz) | 56 560 011 | 2 250 210 650 | 29 705 050 592 | -| Sampled mentions from social networks | 11 245 710 | 128 461 796 | 1 632 567 763 | -| Total | 85 807 187 | 2 579 880 185 | 33 985 510 302 | - -## Pre-training details - -* Ukrainian Roberta was trained with code provided in [HuggingFace tutorial](https://huggingface.co/blog/how-to-train) -* Currently released model follows roberta-base-cased model architecture (12-layer, 768-hidden, 12-heads, 125M parameters) -* The model was trained on 4xV100 (85 hours) -* Training configuration you can find in the [original repository](https://github.com/youscan/language-models) - -## Author -Vitalii Radchenko - contact me on Twitter [@vitaliradchenko](https://twitter.com/vitaliradchenko) diff --git a/model_cards/yuvraj/summarizer-cnndm/README.md b/model_cards/yuvraj/summarizer-cnndm/README.md deleted file mode 100644 index f124c031a3..0000000000 --- a/model_cards/yuvraj/summarizer-cnndm/README.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -language: "en" -tags: -- summarization ---- -​ -# Summarization -​ -## Model description -​ -BartForConditionalGeneration model fine tuned for summarization on 10000 samples from the cnn-dailymail dataset -​ -## How to use -​ -PyTorch model available -​ -```python -from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline -​ -tokenizer = AutoTokenizer.from_pretrained("yuvraj/summarizer-cnndm") -AutoModelWithLMHead.from_pretrained("yuvraj/summarizer-cnndm") -​ -summarizer = pipeline('summarization', model=model, tokenizer=tokenizer) -summarizer("") -​ -## Limitations and bias -Trained on a small dataset diff --git a/model_cards/yuvraj/xSumm/README.md b/model_cards/yuvraj/xSumm/README.md deleted file mode 100644 index 12c1141984..0000000000 --- a/model_cards/yuvraj/xSumm/README.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -language: "en" -tags: -- summarization -- extreme summarization ---- -​ -## Model description -​ -BartForConditionalGenerationModel for extreme summarization- creates a one line abstractive summary of a given article -​ -## How to use -​ -PyTorch model available -​ -```python -from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline -​ -tokenizer = AutoTokenizer.from_pretrained("yuvraj/xSumm") -model = AutoModelWithLMHead.from_pretrained("yuvraj/xSumm") -​ -xsumm = pipeline('summarization', model=model, tokenizer=tokenizer) -xsumm("") -​ -## Limitations and bias -Trained on a small fraction of the xsumm training dataset diff --git a/model_cards/zanelim/singbert-large-sg/README.md b/model_cards/zanelim/singbert-large-sg/README.md deleted file mode 100644 index e3be8882d1..0000000000 --- a/model_cards/zanelim/singbert-large-sg/README.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -language: en -tags: -- singapore -- sg -- singlish -- malaysia -- ms -- manglish -- bert-large-uncased -license: mit -datasets: -- reddit singapore, malaysia -- hardwarezone -widget: -- text: "kopi c siew [MASK]" -- text: "die [MASK] must try" ---- - -# Model name - -SingBert Large - Bert for Singlish (SG) and Manglish (MY). - -## Model description - -Similar to [SingBert](https://huggingface.co/zanelim/singbert) but the large version, which was initialized from [BERT large uncased (whole word masking)](https://github.com/google-research/bert#pre-trained-models), with pre-training finetuned on -[singlish](https://en.wikipedia.org/wiki/Singlish) and [manglish](https://en.wikipedia.org/wiki/Manglish) data. - -## Intended uses & limitations - -#### How to use - -```python ->>> from transformers import pipeline ->>> nlp = pipeline('fill-mask', model='zanelim/singbert-large-sg') ->>> nlp("kopi c siew [MASK]") - -[{'sequence': '[CLS] kopi c siew dai [SEP]', - 'score': 0.9003700017929077, - 'token': 18765, - 'token_str': 'dai'}, - {'sequence': '[CLS] kopi c siew mai [SEP]', - 'score': 0.0779474675655365, - 'token': 14736, - 'token_str': 'mai'}, - {'sequence': '[CLS] kopi c siew. [SEP]', - 'score': 0.0032227332703769207, - 'token': 1012, - 'token_str': '.'}, - {'sequence': '[CLS] kopi c siew bao [SEP]', - 'score': 0.0017727474914863706, - 'token': 25945, - 'token_str': 'bao'}, - {'sequence': '[CLS] kopi c siew peng [SEP]', - 'score': 0.0012526646023616195, - 'token': 26473, - 'token_str': 'peng'}] - ->>> nlp("one teh c siew dai, and one kopi [MASK]") - -[{'sequence': '[CLS] one teh c siew dai, and one kopi. [SEP]', - 'score': 0.5249741077423096, - 'token': 1012, - 'token_str': '.'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi o [SEP]', - 'score': 0.27349168062210083, - 'token': 1051, - 'token_str': 'o'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi peng [SEP]', - 'score': 0.057190295308828354, - 'token': 26473, - 'token_str': 'peng'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi c [SEP]', - 'score': 0.04022320732474327, - 'token': 1039, - 'token_str': 'c'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi? [SEP]', - 'score': 0.01191170234233141, - 'token': 1029, - 'token_str': '?'}] - ->>> nlp("die [MASK] must try") - -[{'sequence': '[CLS] die die must try [SEP]', - 'score': 0.9921030402183533, - 'token': 3280, - 'token_str': 'die'}, - {'sequence': '[CLS] die also must try [SEP]', - 'score': 0.004993876442313194, - 'token': 2036, - 'token_str': 'also'}, - {'sequence': '[CLS] die liao must try [SEP]', - 'score': 0.000317625846946612, - 'token': 727, - 'token_str': 'liao'}, - {'sequence': '[CLS] die still must try [SEP]', - 'score': 0.0002260878391098231, - 'token': 2145, - 'token_str': 'still'}, - {'sequence': '[CLS] die i must try [SEP]', - 'score': 0.00016935862367972732, - 'token': 1045, - 'token_str': 'i'}] - ->>> nlp("dont play [MASK] leh") - -[{'sequence': '[CLS] dont play play leh [SEP]', - 'score': 0.9079819321632385, - 'token': 2377, - 'token_str': 'play'}, - {'sequence': '[CLS] dont play punk leh [SEP]', - 'score': 0.006846973206847906, - 'token': 7196, - 'token_str': 'punk'}, - {'sequence': '[CLS] dont play games leh [SEP]', - 'score': 0.004041737411171198, - 'token': 2399, - 'token_str': 'games'}, - {'sequence': '[CLS] dont play politics leh [SEP]', - 'score': 0.003728888463228941, - 'token': 4331, - 'token_str': 'politics'}, - {'sequence': '[CLS] dont play cheat leh [SEP]', - 'score': 0.0032805048394948244, - 'token': 21910, - 'token_str': 'cheat'}] - ->>> nlp("confirm plus [MASK]") - -{'sequence': '[CLS] confirm plus chop [SEP]', - 'score': 0.9749826192855835, - 'token': 24494, - 'token_str': 'chop'}, - {'sequence': '[CLS] confirm plus chopped [SEP]', - 'score': 0.017554156482219696, - 'token': 24881, - 'token_str': 'chopped'}, - {'sequence': '[CLS] confirm plus minus [SEP]', - 'score': 0.002725469646975398, - 'token': 15718, - 'token_str': 'minus'}, - {'sequence': '[CLS] confirm plus guarantee [SEP]', - 'score': 0.000900257145985961, - 'token': 11302, - 'token_str': 'guarantee'}, - {'sequence': '[CLS] confirm plus one [SEP]', - 'score': 0.0004384620988275856, - 'token': 2028, - 'token_str': 'one'}] - ->>> nlp("catch no [MASK]") - -[{'sequence': '[CLS] catch no ball [SEP]', - 'score': 0.9381157159805298, - 'token': 3608, - 'token_str': 'ball'}, - {'sequence': '[CLS] catch no balls [SEP]', - 'score': 0.060842301696538925, - 'token': 7395, - 'token_str': 'balls'}, - {'sequence': '[CLS] catch no fish [SEP]', - 'score': 0.00030917322146706283, - 'token': 3869, - 'token_str': 'fish'}, - {'sequence': '[CLS] catch no breath [SEP]', - 'score': 7.552534952992573e-05, - 'token': 3052, - 'token_str': 'breath'}, - {'sequence': '[CLS] catch no tail [SEP]', - 'score': 4.208395694149658e-05, - 'token': 5725, - 'token_str': 'tail'}] - -``` - -Here is how to use this model to get the features of a given text in PyTorch: -```python -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained('zanelim/singbert-large-sg') -model = BertModel.from_pretrained("zanelim/singbert-large-sg") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained("zanelim/singbert-large-sg") -model = TFBertModel.from_pretrained("zanelim/singbert-large-sg") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -#### Limitations and bias -This model was finetuned on colloquial Singlish and Manglish corpus, hence it is best applied on downstream tasks involving the main -constituent languages- english, mandarin, malay. Also, as the training data is mainly from forums, beware of existing inherent bias. - -## Training data -Colloquial singlish and manglish (both are a mixture of English, Mandarin, Tamil, Malay, and other local dialects like Hokkien, Cantonese or Teochew) -corpus. The corpus is collected from subreddits- `r/singapore` and `r/malaysia`, and forums such as `hardwarezone`. - -## Training procedure - -Initialized with [bert large uncased (whole word masking)](https://github.com/google-research/bert#pre-trained-models) vocab and checkpoints (pre-trained weights). -Top 1000 custom vocab tokens (non-overlapped with original bert vocab) were further extracted from training data and filled into unused tokens in original bert vocab. - -Pre-training was further finetuned on training data with the following hyperparameters -* train_batch_size: 512 -* max_seq_length: 128 -* num_train_steps: 300000 -* num_warmup_steps: 5000 -* learning_rate: 2e-5 -* hardware: TPU v3-8 diff --git a/model_cards/zanelim/singbert-lite-sg/README.md b/model_cards/zanelim/singbert-lite-sg/README.md deleted file mode 100644 index 13819e064b..0000000000 --- a/model_cards/zanelim/singbert-lite-sg/README.md +++ /dev/null @@ -1,168 +0,0 @@ ---- -language: en -tags: -- singapore -- sg -- singlish -- malaysia -- ms -- manglish -- albert-base-v2 -license: mit -datasets: -- reddit singapore, malaysia -- hardwarezone -widget: -- text: "dont play [MASK] leh" -- text: "die [MASK] must try" ---- - -# Model name - -SingBert Lite - Bert for Singlish (SG) and Manglish (MY). - -## Model description - -Similar to [SingBert](https://huggingface.co/zanelim/singbert) but the lite-version, which was initialized from [Albert base v2](https://github.com/google-research/albert#albert), with pre-training finetuned on -[singlish](https://en.wikipedia.org/wiki/Singlish) and [manglish](https://en.wikipedia.org/wiki/Manglish) data. - -## Intended uses & limitations - -#### How to use - -```python ->>> from transformers import pipeline ->>> nlp = pipeline('fill-mask', model='zanelim/singbert-lite-sg') ->>> nlp("die [MASK] must try") - -[{'sequence': '[CLS] die die must try[SEP]', - 'score': 0.7731555700302124, - 'token': 1327, - 'token_str': '▁die'}, - {'sequence': '[CLS] die also must try[SEP]', - 'score': 0.04763784259557724, - 'token': 67, - 'token_str': '▁also'}, - {'sequence': '[CLS] die still must try[SEP]', - 'score': 0.01859409362077713, - 'token': 174, - 'token_str': '▁still'}, - {'sequence': '[CLS] die u must try[SEP]', - 'score': 0.015824034810066223, - 'token': 287, - 'token_str': '▁u'}, - {'sequence': '[CLS] die is must try[SEP]', - 'score': 0.011271446943283081, - 'token': 25, - 'token_str': '▁is'}] - ->>> nlp("dont play [MASK] leh") - -[{'sequence': '[CLS] dont play play leh[SEP]', - 'score': 0.4365769624710083, - 'token': 418, - 'token_str': '▁play'}, - {'sequence': '[CLS] dont play punk leh[SEP]', - 'score': 0.06880936771631241, - 'token': 6769, - 'token_str': '▁punk'}, - {'sequence': '[CLS] dont play game leh[SEP]', - 'score': 0.051739856600761414, - 'token': 250, - 'token_str': '▁game'}, - {'sequence': '[CLS] dont play games leh[SEP]', - 'score': 0.045703962445259094, - 'token': 466, - 'token_str': '▁games'}, - {'sequence': '[CLS] dont play around leh[SEP]', - 'score': 0.013458190485835075, - 'token': 140, - 'token_str': '▁around'}] - ->>> nlp("catch no [MASK]") - -[{'sequence': '[CLS] catch no ball[SEP]', - 'score': 0.6197211146354675, - 'token': 1592, - 'token_str': '▁ball'}, - {'sequence': '[CLS] catch no balls[SEP]', - 'score': 0.08441998809576035, - 'token': 7152, - 'token_str': '▁balls'}, - {'sequence': '[CLS] catch no joke[SEP]', - 'score': 0.0676785409450531, - 'token': 8186, - 'token_str': '▁joke'}, - {'sequence': '[CLS] catch no?[SEP]', - 'score': 0.040638409554958344, - 'token': 60, - 'token_str': '?'}, - {'sequence': '[CLS] catch no one[SEP]', - 'score': 0.03546864539384842, - 'token': 53, - 'token_str': '▁one'}] - ->>> nlp("confirm plus [MASK]") - -[{'sequence': '[CLS] confirm plus chop[SEP]', - 'score': 0.9608421921730042, - 'token': 17144, - 'token_str': '▁chop'}, - {'sequence': '[CLS] confirm plus guarantee[SEP]', - 'score': 0.011784233152866364, - 'token': 9120, - 'token_str': '▁guarantee'}, - {'sequence': '[CLS] confirm plus confirm[SEP]', - 'score': 0.010571340098977089, - 'token': 10265, - 'token_str': '▁confirm'}, - {'sequence': '[CLS] confirm plus egg[SEP]', - 'score': 0.0033525123726576567, - 'token': 6387, - 'token_str': '▁egg'}, - {'sequence': '[CLS] confirm plus bet[SEP]', - 'score': 0.0008760977652855217, - 'token': 5676, - 'token_str': '▁bet'}] - -``` - -Here is how to use this model to get the features of a given text in PyTorch: -```python -from transformers import AlbertTokenizer, AlbertModel -tokenizer = AlbertTokenizer.from_pretrained('zanelim/singbert-lite-sg') -model = AlbertModel.from_pretrained("zanelim/singbert-lite-sg") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: -```python -from transformers import AlbertTokenizer, TFAlbertModel -tokenizer = AlbertTokenizer.from_pretrained("zanelim/singbert-lite-sg") -model = TFAlbertModel.from_pretrained("zanelim/singbert-lite-sg") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -#### Limitations and bias -This model was finetuned on colloquial Singlish and Manglish corpus, hence it is best applied on downstream tasks involving the main -constituent languages- english, mandarin, malay. Also, as the training data is mainly from forums, beware of existing inherent bias. - -## Training data -Colloquial singlish and manglish (both are a mixture of English, Mandarin, Tamil, Malay, and other local dialects like Hokkien, Cantonese or Teochew) -corpus. The corpus is collected from subreddits- `r/singapore` and `r/malaysia`, and forums such as `hardwarezone`. - -## Training procedure - -Initialized with [albert base v2](https://github.com/google-research/albert#albert) vocab and checkpoints (pre-trained weights). - -Pre-training was further finetuned on training data with the following hyperparameters -* train_batch_size: 4096 -* max_seq_length: 128 -* num_train_steps: 125000 -* num_warmup_steps: 5000 -* learning_rate: 0.00176 -* hardware: TPU v3-8 diff --git a/model_cards/zanelim/singbert/README.md b/model_cards/zanelim/singbert/README.md deleted file mode 100644 index bd5a0f96f2..0000000000 --- a/model_cards/zanelim/singbert/README.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -language: en -tags: -- singapore -- sg -- singlish -- malaysia -- ms -- manglish -- bert-base-uncased -license: mit -datasets: -- reddit singapore, malaysia -- hardwarezone -widget: -- text: "kopi c siew [MASK]" -- text: "die [MASK] must try" ---- - -# Model name - -SingBert - Bert for Singlish (SG) and Manglish (MY). - -## Model description - -[BERT base uncased](https://github.com/google-research/bert#pre-trained-models), with pre-training finetuned on -[singlish](https://en.wikipedia.org/wiki/Singlish) and [manglish](https://en.wikipedia.org/wiki/Manglish) data. - -## Intended uses & limitations - -#### How to use - -```python ->>> from transformers import pipeline ->>> nlp = pipeline('fill-mask', model='zanelim/singbert') ->>> nlp("kopi c siew [MASK]") - -[{'sequence': '[CLS] kopi c siew dai [SEP]', - 'score': 0.5092713236808777, - 'token': 18765, - 'token_str': 'dai'}, - {'sequence': '[CLS] kopi c siew mai [SEP]', - 'score': 0.3515934646129608, - 'token': 14736, - 'token_str': 'mai'}, - {'sequence': '[CLS] kopi c siew bao [SEP]', - 'score': 0.05576375499367714, - 'token': 25945, - 'token_str': 'bao'}, - {'sequence': '[CLS] kopi c siew. [SEP]', - 'score': 0.006019321270287037, - 'token': 1012, - 'token_str': '.'}, - {'sequence': '[CLS] kopi c siew sai [SEP]', - 'score': 0.0038361591286957264, - 'token': 18952, - 'token_str': 'sai'}] - ->>> nlp("one teh c siew dai, and one kopi [MASK].") - -[{'sequence': '[CLS] one teh c siew dai, and one kopi c [SEP]', - 'score': 0.6176503300666809, - 'token': 1039, - 'token_str': 'c'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi o [SEP]', - 'score': 0.21094971895217896, - 'token': 1051, - 'token_str': 'o'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi. [SEP]', - 'score': 0.13027705252170563, - 'token': 1012, - 'token_str': '.'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi! [SEP]', - 'score': 0.004680239595472813, - 'token': 999, - 'token_str': '!'}, - {'sequence': '[CLS] one teh c siew dai, and one kopi w [SEP]', - 'score': 0.002034128177911043, - 'token': 1059, - 'token_str': 'w'}] - ->>> nlp("dont play [MASK] leh") - -[{'sequence': '[CLS] dont play play leh [SEP]', - 'score': 0.9281464219093323, - 'token': 2377, - 'token_str': 'play'}, - {'sequence': '[CLS] dont play politics leh [SEP]', - 'score': 0.010990909300744534, - 'token': 4331, - 'token_str': 'politics'}, - {'sequence': '[CLS] dont play punk leh [SEP]', - 'score': 0.005583590362221003, - 'token': 7196, - 'token_str': 'punk'}, - {'sequence': '[CLS] dont play dirty leh [SEP]', - 'score': 0.0025784350000321865, - 'token': 6530, - 'token_str': 'dirty'}, - {'sequence': '[CLS] dont play cheat leh [SEP]', - 'score': 0.0025066907983273268, - 'token': 21910, - 'token_str': 'cheat'}] - ->>> nlp("catch no [MASK]") - -[{'sequence': '[CLS] catch no ball [SEP]', - 'score': 0.7922210693359375, - 'token': 3608, - 'token_str': 'ball'}, - {'sequence': '[CLS] catch no balls [SEP]', - 'score': 0.20503675937652588, - 'token': 7395, - 'token_str': 'balls'}, - {'sequence': '[CLS] catch no tail [SEP]', - 'score': 0.0006608376861549914, - 'token': 5725, - 'token_str': 'tail'}, - {'sequence': '[CLS] catch no talent [SEP]', - 'score': 0.0002158183924620971, - 'token': 5848, - 'token_str': 'talent'}, - {'sequence': '[CLS] catch no prisoners [SEP]', - 'score': 5.3481446229852736e-05, - 'token': 5895, - 'token_str': 'prisoners'}] - ->>> nlp("confirm plus [MASK]") - -[{'sequence': '[CLS] confirm plus chop [SEP]', - 'score': 0.992355227470398, - 'token': 24494, - 'token_str': 'chop'}, - {'sequence': '[CLS] confirm plus one [SEP]', - 'score': 0.0037301010452210903, - 'token': 2028, - 'token_str': 'one'}, - {'sequence': '[CLS] confirm plus minus [SEP]', - 'score': 0.0014284878270700574, - 'token': 15718, - 'token_str': 'minus'}, - {'sequence': '[CLS] confirm plus 1 [SEP]', - 'score': 0.0011354683665558696, - 'token': 1015, - 'token_str': '1'}, - {'sequence': '[CLS] confirm plus chopped [SEP]', - 'score': 0.0003804611915256828, - 'token': 24881, - 'token_str': 'chopped'}] - ->>> nlp("die [MASK] must try") - -[{'sequence': '[CLS] die die must try [SEP]', - 'score': 0.9552758932113647, - 'token': 3280, - 'token_str': 'die'}, - {'sequence': '[CLS] die also must try [SEP]', - 'score': 0.03644804656505585, - 'token': 2036, - 'token_str': 'also'}, - {'sequence': '[CLS] die liao must try [SEP]', - 'score': 0.003282855963334441, - 'token': 727, - 'token_str': 'liao'}, - {'sequence': '[CLS] die already must try [SEP]', - 'score': 0.0004937972989864647, - 'token': 2525, - 'token_str': 'already'}, - {'sequence': '[CLS] die hard must try [SEP]', - 'score': 0.0003659659414552152, - 'token': 2524, - 'token_str': 'hard'}] - -``` - -Here is how to use this model to get the features of a given text in PyTorch: -```python -from transformers import BertTokenizer, BertModel -tokenizer = BertTokenizer.from_pretrained('zanelim/singbert') -model = BertModel.from_pretrained("zanelim/singbert") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='pt') -output = model(**encoded_input) -``` - -and in TensorFlow: -```python -from transformers import BertTokenizer, TFBertModel -tokenizer = BertTokenizer.from_pretrained("zanelim/singbert") -model = TFBertModel.from_pretrained("zanelim/singbert") -text = "Replace me by any text you'd like." -encoded_input = tokenizer(text, return_tensors='tf') -output = model(encoded_input) -``` - -#### Limitations and bias -This model was finetuned on colloquial Singlish and Manglish corpus, hence it is best applied on downstream tasks involving the main -constituent languages- english, mandarin, malay. Also, as the training data is mainly from forums, beware of existing inherent bias. - -## Training data -Colloquial singlish and manglish (both are a mixture of English, Mandarin, Tamil, Malay, and other local dialects like Hokkien, Cantonese or Teochew) -corpus. The corpus is collected from subreddits- `r/singapore` and `r/malaysia`, and forums such as `hardwarezone`. - -## Training procedure - -Initialized with [bert base uncased](https://github.com/google-research/bert#pre-trained-models) vocab and checkpoints (pre-trained weights). -Top 1000 custom vocab tokens (non-overlapped with original bert vocab) were further extracted from training data and filled into unused tokens in original bert vocab. - -Pre-training was further finetuned on training data with the following hyperparameters -* train_batch_size: 512 -* max_seq_length: 128 -* num_train_steps: 300000 -* num_warmup_steps: 5000 -* learning_rate: 2e-5 -* hardware: TPU v3-8