github/AMD-SHARK-Studio
1
1
Fork 0
mirror of https://github.com/nod-ai/AMD-SHARK-Studio.git synced 2026-04-03 03:00:17 -04:00
Code Issues Packages Projects Releases Wiki Activity
1,385 Commits 66 Branches 705 Tags
20230624.787
Commit Graph

49 Commits

Author SHA1 Message Date
powderluv
726d73d6ba Revert "[vicuna] Add streaming of tokens (#1587)" (#1588) 
This reverts commit 4d55e51d46.
 2023-06-23 10:29:00 -07:00
Gaurav Shukla
4d55e51d46 [vicuna] Add streaming of tokens (#1587) 
Signed-off-by: Gaurav Shukla <gaurav@nod-labs.com>
 2023-06-23 08:20:46 -07:00
jinchen62
4002da7161 Add int4/int8 options to chatbot webui (#1586) 2023-06-23 07:18:34 -07:00
Eliasj42
8822b9acd7 added ability to use config file to shard vicuna (#1565) 
Co-authored-by: Elias Joseph <elias@nod-labs.com>
 2023-06-22 17:40:35 -05:00
Daniel Garvey
0ca3b9fce3 fix some mmap and vicuna bugs (#1576) 2023-06-22 17:39:55 -05:00
Daniel Garvey
a202bb466a fp16 fixes for webui (#1571) 2023-06-21 20:24:02 -07:00
Phaneesh Barwaria
88cc2423cc Enable Vicuna fp16 cpu (#1562) 
* fix second vic mlir gen

* fp16 mlir/vmfb download from shark_tank
 2023-06-20 13:43:21 -05:00
Vivek Khandelwal
855435ee24 Fix for the user input for Falcon pipeline 2023-06-20 18:09:32 +05:30
Elias Joseph
6f9f868fc0 fixed a bug where designating device for vicuna didn't work 2023-06-20 17:09:32 +05:30
Vivek Khandelwal
fafd713141 Minor change to falcon pipeline 2023-06-19 22:36:32 +05:30
Vivek Khandelwal
015d0132c3 Modify falcon pipeline to add fp16 support (#1551) 2023-06-19 09:57:13 -07:00
Vivek Khandelwal
46184a81ac Add Falcon pipeline (#1534) 2023-06-14 09:39:16 -07:00
PhaneeshB
149165a2f0 add multi-device mutli-precision vmfb names 2023-06-14 22:08:24 +05:30
dan
bec82a665f mega vicuna merge 
single endpoint in apps/language/models/scripts/vicuna.py
removed main functions from pipelines
replaced divergent utils compile with shark_importer
adds support for different precisions
 2023-06-14 19:06:29 +05:30
Nithin Meganathan
34f1295349 Add a model config generator (#1511) 
Model config generator takes a PyTorch model as input and generates a JSON file with model layers and other propperties that define sharding on a particular hardware.
 2023-06-09 15:32:00 -07:00
Phaneesh Barwaria
1980d7b2c3 Cpu device map (#1515) 
* update cpu iree device

* fix vmfb paths vic unsharded
 2023-06-09 11:27:02 -05:00
Phaneesh Barwaria
436f58ddc4 cli using generate and mem fixes (#1509) 2023-06-08 13:13:32 -05:00
Phaneesh Barwaria
6b29bd17c8 Enable compilation vicuna (#1507) 
* add cli for unsharded vic

* enable mlir download and compile
 2023-06-07 13:08:22 -07:00
Daniel Garvey
f206ecc635 reenable compilation in vicuna pipeline, add flags (#1505) 
* replace vicuna.py backend with pipeline

* add some memory management to fist vicuna compile

reenable compilation
 2023-06-07 09:49:27 -07:00
PhaneeshB
f23b778a6c remove old vicuna scripts 2023-06-06 21:35:58 +05:30
PhaneeshB
436edf900d add vic sharded pipeline 2023-06-06 21:35:58 +05:30
Phaneesh Barwaria
a83808ddc5 Vicuna cuda on A100 40G (#1496) 
* vic chat with memory management (precompiled vmfb)

* fix vmfb path and download
 2023-06-06 15:10:33 +05:30
Phaneesh Barwaria
f0a4e59758 LLM Pipeline Wrapper (#1477) 
* [LLM] Add LLM pipeline

Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>

* add base pipeline and stableLM

* StableLM on UI - full block

* add SLM default model name

* add vicuna with pipeline

* add one token gen api for vic

* Fix stableLM bugs

* debug vic memory

* lint fix

---------

Signed-off-by: Gaurav Shukla <gaurav@nod-labs.com>
Co-authored-by: Gaurav Shukla <gaurav@nod-labs.com>
 2023-05-31 10:17:20 -07:00
Elias Joseph
73cd7e8320 added full vicuna to vicuna.py 2023-05-26 22:06:40 +05:30
PhaneeshB
6d64b8e273 vic and slm common generation base 2023-05-25 20:29:41 +05:30
PhaneeshB
a8ea0326f5 correct SLM saved vmfb naming 2023-05-25 20:29:41 +05:30
PhaneeshB
58e9194553 add Lists import 2023-05-25 20:29:41 +05:30
PhaneeshB
eb360e255d remove unused imports 2023-05-25 20:29:41 +05:30
PhaneeshB
a6f88d7f72 refactor mlir compile 2023-05-25 20:29:41 +05:30
Phaneesh Barwaria
f5ce121988 SLM on Sharkstudio (#1454) 
* localize import, fix file reading, device cpu

* extract out model args
 2023-05-19 11:21:08 -07:00
PhaneeshB
09bea17e59 fix #2 SLM in SharkStudio 2023-05-18 00:56:22 +05:30
Daniel Garvey
aefcf80b48 swap to cpu an remove hardcoded paths (#1448) 
Co-authored-by: powderluv <powderluv@users.noreply.github.com>
 2023-05-17 10:53:34 -07:00
PhaneeshB
6602a2f5ba add continuous output for CLI 2023-05-17 18:33:46 +05:30
powderluv
8ee2ac89f8 Rename sharded_vicuna_fp32_web.py to vicuna_web.py 2023-05-16 09:41:35 -07:00
powderluv
60cb48be2e Rename sharded_vicuna_fp32.py to vicuna.py 2023-05-16 09:40:51 -07:00
powderluv
86a215b063 Delete sharded_vicunia.py 2023-05-16 09:37:39 -07:00
powderluv
d6e3a9a236 Delete standalone_vicuna.py 2023-05-16 09:37:26 -07:00
Daniel Garvey
4731c1a835 prevent loading tokenizer on import (#1432) 
also adds sentencepiece dep for exe
moved vicuna imports to after an if statement
in general we should avoid importing files that load whole models as
global variables
 2023-05-12 19:11:45 -07:00
Gaurav Shukla
e0cc2871bb [SD] Yield 2 tokens at a time in vicuna 
Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>
 2023-05-11 23:49:01 +05:30
Gaurav Shukla
649f39408b [SD] Fix vicuna response 
Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>
 2023-05-11 18:06:21 +05:30
Gaurav Shukla
9e07360b00 [SD] Standalone vicuna with web 
Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com>
 2023-05-11 17:23:44 +05:30
Eliasj42
fa833f8366 fixed spacing issue with chat-bot (#1417) 
Co-authored-by: Elias Joseph <elias@nod-labs.com>
 2023-05-10 16:07:50 -07:00
Gaurav Shukla
fcb059aa38 [SD] Integrate vicuna in the web (#1410) 2023-05-10 11:30:22 -07:00
PhaneeshB
517c670f82 vicuna chat cli 2023-05-10 22:55:06 +05:30
Eliasj42
59df14f18b added vicuna demo (#1408) 
Co-authored-by: Elias Joseph <elias@nod-labs.com>
 2023-05-09 21:18:20 -07:00
Daniel Garvey
7a4a51ae73 vulkan vic f16 (#1404) 
Co-authored-by: dan <dan@nod-labs.com>
 2023-05-08 16:46:53 -07:00
Eliasj42
54ce3d48ca added standalone vicuna script (#1401) 
Co-authored-by: Elias Joseph <elias@nod-labs.com>
 2023-05-05 18:05:52 -05:00
Gaurav Shukla
fed63dfd4b [SD] Add stableLM chatbot (#1383) 
Signed-off-by: Gaurav Shukla <gaurav@nod-labs.com>
Co-authored-by: powderluv <powderluv@users.noreply.github.com>
 2023-05-03 15:37:20 -05:00
Vivek Khandelwal
d2f7e03b7e Add StableLM model (#1331) 2023-04-21 09:51:02 -07:00