Vivek Khandelwal
c4723f469f
Update langchain_requirements.txt
2023-08-14 14:32:19 +05:30
Vivek Khandelwal
d804f45a61
Update langchain_requirements.txt
2023-08-14 14:32:19 +05:30
Vivek Khandelwal
d22177f936
Update requirements.txt
2023-08-14 14:32:19 +05:30
George Petterson
75e68f02f4
Remove CUDNN
2023-08-14 14:32:19 +05:30
Gaurav Shukla
4dc9c59611
[chatbot] Add tokens generated per second ( #1753 )
2023-08-13 11:25:41 -07:00
Gaurav Shukla
18801dcabc
[chat] Update chatbot ui
...
Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com >
2023-08-13 18:39:22 +05:30
Gaurav Shukla
3c577f7168
[vicuna] fix shard config generator script ( #1747 )
...
Signed-off-by: Gaurav Shukla <gaurav@nod-labs.com >
2023-08-10 11:26:03 -07:00
Stefan Kapusniak
f5e4fa6ffe
UI/Web - Revert tab order ( #1724 )
...
* Revert ui tab order
* Reverts the tab order, so that SD, LLM, and Experimental are grouped
together again as far as is possible.
* Labelled "Generate Sharding Config" as experimental as pressing the
'Get Model Config' errors for me.
* Fix formatting in index.py
2023-08-10 11:25:36 -07:00
powderluv
48de445325
Enable caching and disable vma ( #1746 )
...
* Enable caching allocator by default
Going to toggle VMA off too and this is required for performance. Will have to monitor in the wild reports.
* Disable VMA
Disable VMA
2023-08-10 10:49:44 -07:00
Gaurav Shukla
8e90f1b81a
[vicuna] add default config in case of sharded vicuna
...
Signed-Off-by: Gaurav Shukla<gaurav@nod-labs.com >
2023-08-10 21:28:08 +05:30
Vivek Khandelwal
e8c1203be2
Fix vicuna script ( #1745 )
2023-08-10 06:11:14 -07:00
Vivek Khandelwal
e4d7abb519
Final patch for fixing Langchain token streaming issue ( #1744 )
2023-08-09 10:09:41 -07:00
powderluv
96185c9dc1
pin safetensors to 0.3.1 ( #1740 )
2023-08-08 19:24:44 -07:00
powderluv
bc22a81925
re-enable constant folding ( #1739 )
...
Tested and works well. (modulo unrelated driver issue)
2023-08-08 17:17:38 -07:00
Eliasj42
5203679f1f
Bandaid fix 2 ( #1728 )
...
* download all mlirs
* fixed install method
* download all mlirs (#1727 )
Co-authored-by: Elias Joseph <elias@nod-labs.com >
* added taggs
* fix name check for file existence
* Remove SD from all_models.csv (#1706 )
Removes SD from pytests as it has its own test suite.
* gpt_langchain.py fixes for pydantic (#1722 )
* removed dead code
---------
Co-authored-by: Elias Joseph <elias@nod-labs.com >
Co-authored-by: PhaneeshB <b.phaneesh@gmail.com >
Co-authored-by: Ean Garvey <87458719+monorimet@users.noreply.github.com >
Co-authored-by: Stefan Kapusniak <121311569+one-lithe-rune@users.noreply.github.com >
2023-08-08 12:14:57 -05:00
Vivek Khandelwal
bf073f8f37
[Langchain] Expand pipelines to fix token streaming issue
2023-08-08 10:27:23 +05:30
Stella Laurenzo
cec6eda6b4
Optimize device enumeration overhead and log details on long operations. ( #1734 )
...
* Optimize device enumeration overhead and log details on long operations.
* Various fixes to add `@functools.cache` to what should be one time, expensive, device enumeration and setup activities. Cuts several seconds off of initialization on my machine.
* Add detailed tracing to actual invocations if they exceed a certain timeout or have an exception.
* Add detailed tracing to loading status.
* By default detail logging is only printed if an operation takes an excessive amount of time. All logging/timing can be printed by setting the variable `$env:SHARK_DETAIL_TRACE = "1"`
* Remove cache from unhashable functions
2023-08-07 17:20:53 -07:00
Stella Laurenzo
9e37e03741
Clearly differentiate phases of loading modules to better understand if things are taking a long time. ( #1733 )
2023-08-07 14:03:12 -07:00
Stefan Kapusniak
9b8c4401b5
gpt_langchain.py fixes for pydantic ( #1722 )
2023-08-07 00:55:38 -07:00
Ean Garvey
a9f95a218b
Remove SD from all_models.csv ( #1706 )
...
Removes SD from pytests as it has its own test suite.
2023-08-05 15:55:52 -05:00
PhaneeshB
872bd72d0b
fix name check for file existence
2023-08-05 21:33:53 +05:30
Eliasj42
fd1c4db5d0
download all mlirs ( #1727 )
...
Co-authored-by: Elias Joseph <elias@nod-labs.com >
2023-08-04 18:22:06 -05:00
Daniel Garvey
759664bb48
add py files to pyinstaller for shark ( #1723 )
2023-08-04 14:10:43 -07:00
Daniel Garvey
14fd0cdd87
add missing subprocess import ( #1721 )
2023-08-04 15:15:22 -05:00
Daniel Garvey
a57eccc997
fix lint ( #1720 )
2023-08-04 14:54:33 -05:00
Daniel Garvey
a686d7d89f
temporarily disable langchain stuff in webui ( #1719 )
...
its breaking the exe
2023-08-04 12:48:06 -07:00
Eliasj42
ed484b8253
added functionality for int8 vicuna and 4 shards ( #1712 )
...
combined vicuna_4_shards.py and vicuna.py to reduce code duplication
Co-authored-by: Elias Joseph <elias@nod-labs.com >
2023-08-04 14:05:05 -05:00
gpetters94
7fe57ebaaf
Add vector database and add support on the web UI ( #1699 )
2023-08-04 13:47:19 -04:00
Nithin Meganathan
c287fd2be8
Add GPU ID's in model_confg.json by default for manual annotation ( #1718 )
2023-08-04 12:46:27 -05:00
Gaurav Shukla
51ec1a1360
[vicuna] Integrate sharded vicuna in web ( #1717 )
...
Signed-off-by: Gaurav Shukla <gaurav@nod-labs.com >
2023-08-04 11:46:53 -05:00
Gaurav Shukla
bd30044c0b
[Shard] Add sharding generation in shark studio
...
Signed-Off-by: Gaurav Shukla <gaurav@nod-labs.com >
2023-08-04 21:51:14 +05:30
Ean Garvey
c9de2729b2
Add flag for toggling constant folding. ( #1714 )
2023-08-04 04:55:52 -07:00
Vivek Khandelwal
a5b13fcc2f
[Langchain] Patch for fixing streaming of tokens ( #1709 )
2023-08-03 10:06:49 -07:00
Stefan Kapusniak
6bb329c4af
Unsharded Vicuna: Fix Memory Error compiling mlir for lmsys/vicuna-7b-v1.3 fp16 with 64 GiB ( #1702 )
2023-08-01 06:07:56 -07:00
Vivek Khandelwal
98fb6c52df
Expand pipelines to fix streaming of tokens
2023-07-31 22:11:01 +05:30
Stefan Kapusniak
206c1b70f4
UI/Web: Reorder tabs to separate SD and LLM ( #1701 )
...
Shuffle the tabs around so that:
* All the SD tabs are together
* All the LLM tabs are together
* All the experimental tabs are together
2023-07-29 22:25:30 -04:00
PhaneeshB
cdb037ee54
use shark_args for vulkan debug utils flag
2023-07-30 07:54:26 +05:30
PhaneeshB
ce2fd84538
fix cpu device name for SharkStudio
2023-07-30 07:54:26 +05:30
PhaneeshB
4684afad34
update upscalar example
2023-07-28 21:06:28 +05:30
PhaneeshB
8d65456b7a
Move vulkan runtime flags to shark_args
2023-07-28 21:06:28 +05:30
PhaneeshB
d6759a852b
add vulkan vma alloc flag
2023-07-28 21:06:28 +05:30
Daniel Garvey
ab57af43c1
Couple of fixes for vicuna.py ( #1696 )
...
* mega vicuna merge pt 2
* add fallback to ensure compile is called
2023-07-27 15:53:05 -07:00
jinchen62
4d5c55dd9f
Fix vicuna script ( #1697 )
2023-07-27 17:24:26 -05:00
Vivek Khandelwal
07399ad65c
[Langchain] Remove unused code ( #1698 )
2023-07-27 11:59:54 -05:00
Vivek Khandelwal
776a9c2293
Fix for Langchain ( #1694 )
...
For CPU, remove max time stopping criteria
Fix web UI issue
2023-07-26 09:00:23 -07:00
Eliasj42
9d399eb988
fixed bug where device_idx was hardcoded ( #1693 )
...
Co-authored-by: Elias Joseph <elias@nod-labs.com >
2023-07-25 19:00:13 -05:00
Vivek Khandelwal
927b662aa7
Add Langchain SHARK Compilation support for all paths
2023-07-25 22:15:42 +05:30
Abhishek Varma
47f8a79c75
[MiniGPT4] Add MiniGPT4 to SHARK ( #1554 )
...
* [MiniGPT4] Add MiniGPT4 to SHARK
-- This is the first installment of MiniGPT4 in SHARK.
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com >
* Add int8 support for MiniGPT4
-- This commit adds int8 support for MiniGPT4.
Signed-off-by: Abhishek Varma <abhishek@nod-lab.com >
* Update .spec for MiniGPT4's config files
* black format MiniGPT4
---------
Signed-off-by: Abhishek Varma <abhishek@nod-labs.com >
Signed-off-by: Abhishek Varma <abhishek@nod-lab.com >
2023-07-25 09:42:27 -07:00
Stefan Kapusniak
289f983f41
SD - Implement seed arrays for batch runs ( #1690 )
...
* SD Scripts and UI tabs that support batch_count can now take a
string containing a JSON array, or a list of integers, as their seed
input.
* Each batch in a run will now take the seed specified at the
corresponding array index if one exists. If there is no seed at
that index, the seed value will be treated as -1 and a random
seed will be assigned at that position. If an integer rather than
a list or json array has been, everything works as before.
* UI seed input controls are now Textboxes with info lines about
the seed formats allowed.
* UI error handling updated to be more helpful if the seed input is
invalid.
2023-07-24 19:22:34 -07:00
Daniel Garvey
453e46562f
mega vicuna merge pt 2 ( #1685 )
2023-07-24 12:42:20 -05:00
