Ryan Dick d7ab464176 Offload the current model when locking if it is already partially loaded and we have insufficient VRAM.

2025-01-07 02:53:44 +00:00

..

cached_model

Offload the current model when locking if it is already partially loaded and we have insufficient VRAM.

2025-01-07 02:53:44 +00:00

torch_module_autocast

Fix bug in CustomConv1d and CustomConv2d patch calculations.

2024-12-29 19:10:19 +00:00

__init__.py

Remove ModelCacheBase.

2024-12-24 14:23:18 +00:00

cache_record.py

First pass at adding partial loading support to the ModelCache.

2025-01-07 00:30:58 +00:00

cache_stats.py

Move CacheStats to its own file.

2024-12-24 14:23:18 +00:00

dev_utils.py

Add a utility to help with determining the working memory required for expensive operations.

2025-01-07 01:20:15 +00:00

model_cache.py

Offload the current model when locking if it is already partially loaded and we have insufficient VRAM.

2025-01-07 02:53:44 +00:00

utils.py

Add get_effective_device(...) utility to aid in determining the effective device of models that are partially loaded.

2025-01-07 00:31:00 +00:00