github/home-llm
1
1
Fork 0
mirror of https://github.com/acon96/home-llm.git synced 2026-01-09 13:48:05 -05:00
Code Issues Packages Projects Releases Wiki Activity
323 Commits 9 Branches 39 Tags
adae87addd69ff17b8a92f7c5fdca8814d3c36da
Commit Graph

62 Commits

Author SHA1 Message Date
Alex O'Connell
adae87addd training fixes, default values + other fixes 2024-04-21 23:40:28 -04:00
Alex O'Connell
3cd9aaf5d7 working dpo + llama 3 training 2024-04-21 20:55:13 -04:00
Alex O'Connell
d82b6996b0 fix training script 2024-04-16 21:14:05 -04:00
Alex O'Connell
74d2c290fd typo in error handler 2024-04-15 22:42:31 -04:00
Alex O'Connell
1ff12f9fb6 fix fsdp saves 2024-04-15 20:57:48 -04:00
Alex O'Connell
644a326c0f add mfu callback 2024-04-15 17:39:41 -04:00
Alex O'Connell
b41742b9fb more eval fixes + tinyllama tests 2024-04-15 17:39:41 -04:00
Alex O'Connell
90ca37c6ed start working on dpo trainer + add s3 upload callback 2024-04-15 17:39:41 -04:00
Alex O'Connell
8144267e69 fix templated action quotes + add fsdp config 2024-04-14 18:08:11 -04:00
Alex O'Connell
2e92adbd9e fixes for multi-gpu training 2024-04-14 16:43:42 -04:00
Alex O'Connell
f5bab7b119 better dpo parameters 2024-04-14 15:42:51 -04:00
Alex O'Connell
0a6b41d5bc support many checkpoints in evaluate 2024-03-23 10:09:42 -04:00
Alex O'Connell
c67759e16f experiment notes 2024-03-20 23:05:22 -04:00
Alex O'Connell
5dec71eae2 finalize new model version 2024-02-22 21:12:38 -05:00
Alex O'Connell
c285e3c6a9 instructions for adding personas 2024-02-17 23:13:45 -05:00
Alex O'Connell
411276408b train new models based on stablelm + properly add new response types 2024-02-13 20:21:51 -05:00
Alex O'Connell
1da51c9fb6 remove broken training tweaks 2024-02-10 10:04:58 -05:00
Alex O'Connell
631daebd97 try different way of pre-allocating the cuda buffers 2024-02-05 21:59:10 -05:00
Alex O'Connell
7b01251f5d fixes for training zephyr base 2024-02-04 11:40:03 -05:00
Alex O'Connell
cecf9bc53e move to jsonl, finish sharegpt dataset format, and add flag to add chatml prompt template 2024-01-31 23:00:32 -05:00
Alex O'Connell
d901eaffdf start working on other base models 2024-01-30 22:12:46 -05:00
Alex O'Connell
8b1cc5a587 add random sampler with largest first to optimize pytorch memory usage 2024-01-30 22:12:12 -05:00
Alex O'Connell
2bdfb79f89 clear cuda cache after evaluation to clear out the irregularly sized blocks from the allocation cache 2024-01-30 22:12:12 -05:00
Alex O'Connell
da7b7b4d95 try to checkpoint saved lora modules 2024-01-30 22:12:12 -05:00
Alex O'Connell
30361809ae count tokens faster 2024-01-27 14:54:46 -05:00
Alex O'Connell
9723a98139 more dataset + model experiments using the evaluation script 2024-01-27 14:54:14 -05:00
Alex O'Connell
e6fae06133 wizardlm merge + fix eval 2024-01-25 20:46:59 -05:00
Alex O'Connell
57634519ca move to eval script instead of during training 2024-01-25 20:46:59 -05:00
Alex O'Connell
c3cb5c5354 try writing an accuracy metric 2024-01-25 20:46:59 -05:00
Alex O'Connell
0b9beb85ca finish trianing new models 2024-01-17 18:24:25 -05:00
Alex O'Connell
2a0dbd8806 fix training script for Phi-2 lora 2024-01-16 21:47:15 -05:00
Alex O'Connell
8f99025507 scale dataset halfway back to where it was 2024-01-13 23:36:56 -05:00
Alex O'Connell
e479b5ffac more fixes 2024-01-13 15:29:37 -05:00
Alex O'Connell
23dae4faa9 fix lora train error with new phi model 2024-01-12 23:24:16 -05:00
Alex O'Connell
74bd516148 resize dataset + add state attributes to status requests 2024-01-12 12:44:26 -05:00
Alex O'Connell
35e609108c update x86 wheel 2024-01-06 19:08:19 -05:00
Alex O'Connell
ce0ec6b48f fix chatml again + gbnf + phi1.5 2023-12-31 16:00:24 -05:00
Alex O'Connell
b1575f2512 chatml prompt tweaks, support multi-turn examples in the SFT masking step, and pre-tokenize dataset to avoid CPU bottleneck 2023-12-26 23:52:27 -05:00
Alex O'Connell
03b3bf0b20 add shebangs 2023-12-23 09:13:52 -05:00
Alex O'Connell
206463a2bd cleanup + license + readme updates 2023-12-22 23:55:14 -05:00
Alex O'Connell
8614152751 more cleanup 2023-12-20 23:18:39 -05:00
Alex O'Connell
d429e37378 add quantization scripts 2023-12-19 21:49:09 -05:00
Alex O'Connell
b921a3234b lots of changes from phi-2 fine-tuning 2023-12-18 00:57:07 -05:00
Alex O'Connell
5f59b0d84c support saving more frequently 2023-12-14 00:44:26 -05:00
Alex O'Connell
2384b75583 add merge with alpaca dataset 2023-12-13 21:53:48 -05:00
Alex O'Connell
46b6ee4f65 updates with new Phi model release 2023-12-13 18:29:47 -05:00
Alex O'Connell
cb7cc1ed53 more plugin work. it passes info around properly 2023-10-29 23:22:59 -04:00
Alex O'Connell
2246586a08 turn on resize so it doesn't break 2023-10-28 00:18:18 -04:00
Alex O'Connell
ad16cfa41d more cmd line flags + gptq support 2023-10-28 00:15:03 -04:00
Alex O'Connell
82b4ea4c65 add load in 8bit + enable lora 2023-10-27 00:23:13 -04:00