|
|
|
# EXPERIMENTS
|
|
|
|
|
|
|
|
| MODEL | TRAIN PPL | VALID PPL |
|
|
|
|
| --------- | --------- | --------- |
|
|
|
|
| DE-unfrozen | 8 | 5000 |
|
|
|
|
| ES-unfrozen | 33 | 11,420 |
|
|
|
|
| DE-freeze only last layer | 11 | 2254 |
|
|
|
|
| ES-freeze only last layer | 290 | 9900 |
|
|
|
|
| DE-train only wte | 51 | 1980 |
|
|
|
|
| ES-train only wte | 4120 | 25,745 |
|
|
|
|
| DE-train wte + lm head | 40 | 10,971 |
|
|
|
|
| ES-train wte + lm head | 94 | 32,649 |
|
|
|
|
|
|
|
|
|
|
|
|
# LINKS
|
|
|
|
|
|
|
|
https://wandb.ai/susannb/huggingface?workspace=user-susannb
|
| ... | ... | |
