12 Commits

Author	SHA1	Message	Date
Ryan Dick	a83a999b79	Reduce peak memory used for unit tests.	2024-12-24 14:32:11 +00:00
Ryan Dick	f8a6accf8a	Fix bitsandbytes imports to avoid ImportErrors on MacOS.	2024-12-24 14:32:11 +00:00
Ryan Dick	3f990393a1	Simplify the state management in InvokeLinear8bitLt and add unit tests. This is in preparation for wrapping it to support streaming of weights from cpu to gpu.	2024-12-24 14:32:11 +00:00
Ryan Dick	26832c1a0e	Add unit test to confirm that GGMLTensor sizes (bytes) are being calculated correctly.	2024-10-02 18:33:05 -04:00
Ryan Dick	ec7e771942	Add a compute_dtype field to GGMLTensor.	2024-10-02 18:33:05 -04:00
Ryan Dick	fe84013392	Add unit tests for GGMLTensor.	2024-10-02 18:33:05 -04:00
Brandon Rising	446e2884bc	Remove no longer used code paths, general cleanup of new dequantization code, update probe	2024-10-02 18:33:05 -04:00
Brandon Rising	7d9f125232	Run ruff and update imports	2024-10-02 18:33:05 -04:00
Brandon Rising	0875e861f5	Various updates to gguf performance	2024-10-02 18:33:05 -04:00
Ryan Dick	f347b26999	Initial experimentation with Tensor-like extension for GGUF.	2024-10-02 18:33:05 -04:00
Brandon Rising	8cf19c4124	Run Ruff	2024-10-02 18:33:05 -04:00
Brandon Rising	f7112ae57b	Add unit tests for torch patcher	2024-10-02 18:33:05 -04:00