SUMIN/HuggingFace_transformer
1
0
Fork 0
Code Issues Pull Requests Actions 7 Packages Projects Releases Wiki Activity

[research_projects] deal with security alerts (#15594)

Browse Source 
* [research_projects] deal with security alerts

* add a note of the original PL ver and warning
This commit is contained in:
Stas Bekman
2022-02-11 11:31:09 -08:00
committed by GitHub
parent f15c99fabf
commit fcb0f74397
8 changed files with 55 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
examples/research_projects/pplm/README.md
View File

@@ -10,6 +10,9 @@ Blog link: https://eng.uber.com/pplm
Please check out the repo under uber-research for more information: https://github.com/uber-research/PPLM
# Note
⚠️ This project should be run with pytorch-lightning==1.0.4 which has a potential security vulnerability
## Setup
@@ -20,7 +23,7 @@ pip install nltk torchtext # additional requirements.
cd examples/research_projects/pplm
```
## PPLM-BoW
## PPLM-BoW
### Example command for bag-of-words control
@@ -30,7 +33,7 @@ python run_pplm.py -B military --cond_text "The potato" --length 50 --gamma 1.5
### Tuning hyperparameters for bag-of-words control
1. Increase `--stepsize` to intensify topic control, and decrease its value to soften the control. `--stepsize 0` recovers the original uncontrolled GPT-2 model.
1. Increase `--stepsize` to intensify topic control, and decrease its value to soften the control. `--stepsize 0` recovers the original uncontrolled GPT-2 model.
2. If the language being generated is repetitive (For e.g. "science science experiment experiment"), there are several options to consider: </br>
a) Reduce the `--stepsize` </br>
@@ -48,7 +51,6 @@ python run_pplm.py -D sentiment --class_label 2 --cond_text "My dog died" --leng
### Tuning hyperparameters for discriminator control
1. Increase `--stepsize` to intensify topic control, and decrease its value to soften the control. `--stepsize 0` recovers the original uncontrolled GPT-2 model.
1. Increase `--stepsize` to intensify topic control, and decrease its value to soften the control. `--stepsize 0` recovers the original uncontrolled GPT-2 model.
2. Use `--class_label 3` for negative, and `--class_label 2` for positive

2
examples/research_projects/pplm/requirements.txt
View File

@@ -5,7 +5,7 @@ psutil
sacrebleu
rouge-score
tensorflow_datasets
pytorch-lightning==1.0.4
pytorch-lightning
matplotlib
git-python==1.0.3
faiss-cpu