mirror of
https://github.com/acon96/home-llm.git
synced 2026-01-08 21:28:05 -05:00
39 lines
750 B
Markdown
39 lines
750 B
Markdown
## TinyLlama/TinyLlama-1.1B-Chat-v1.0
|
|
|
|
# tinyhome-rev1
|
|
- full fine tune
|
|
- epochs: 1
|
|
- 2048 train ctx
|
|
- batch size 32
|
|
- learning rate 1e-5
|
|
- weight decay 0.1
|
|
- gradient clipping 1.0
|
|
- dataset size: medium
|
|
+ evaluation results:
|
|
- 100: 0.5814777327935222
|
|
- 200: 0.7120445344129555
|
|
- 300: 0.7616396761133604
|
|
- 400: 0.7899797570850202
|
|
- 500: 0.8643724696356275
|
|
- 600: 0.8674089068825911
|
|
- 700: 0.8780364372469636
|
|
- 800: 0.8755060728744939
|
|
- 900: 0.8709514170040485
|
|
- 1000: 0.8775303643724697
|
|
- Final: 0.8724696356275303
|
|
|
|
# tinyhome-rev2
|
|
- dataset size: medium
|
|
- learning rate 2e-5
|
|
+ evaluation results:
|
|
- 100:
|
|
- 200:
|
|
- 300:
|
|
- 400:
|
|
- 500:
|
|
- 600:
|
|
- 700:
|
|
- 800:
|
|
- 900:
|
|
- 1000:
|
|
- Final: 0.9817813765182186 |