From 7bc6d76396f4a603161539aefaa6207d61260f60 Mon Sep 17 00:00:00 2001 From: Matt Date: Thu, 1 Feb 2024 14:44:58 +0000 Subject: [PATCH] Add tip on setting tokenizer attributes (#28764) * Add tip on setting tokenizer attributes * Grammar * Remove the bit that was causing doc builds to fail --- docs/source/en/chat_templating.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/source/en/chat_templating.md b/docs/source/en/chat_templating.md index a478c32e6f..e0ffd9ad15 100644 --- a/docs/source/en/chat_templating.md +++ b/docs/source/en/chat_templating.md @@ -343,6 +343,15 @@ tokenizer.push_to_hub("model_name") # Upload your new template to the Hub! The method [`~PreTrainedTokenizer.apply_chat_template`] which uses your chat template is called by the [`ConversationalPipeline`] class, so once you set the correct chat template, your model will automatically become compatible with [`ConversationalPipeline`]. + +If you're fine-tuning a model for chat, in addition to setting a chat template, you should probably add any new chat +control tokens as special tokens in the tokenizer. Special tokens are never split, +ensuring that your control tokens are always handled as single tokens rather than being tokenized in pieces. You +should also set the tokenizer's `eos_token` attribute to the token that marks the end of assistant generations in your +template. This will ensure that text generation tools can correctly figure out when to stop generating text. + + + ### What are "default" templates? Before the introduction of chat templates, chat handling was hardcoded at the model class level. For backwards