Ryan Dick 0cf51cefe8 Revise the logic for calculating the RAM model cache limit.

2025-01-16 23:46:07 +00:00

..

cached_model

Memory optimization to load state dicts one module at a time in CachedModelWithPartialLoad when we are not storing a CPU copy of the state dict (i.e. when keep_ram_copy_of_weights=False).

2025-01-16 17:00:33 +00:00

torch_module_autocast

Fix bug in CustomConv1d and CustomConv2d patch calculations.

2024-12-29 19:10:19 +00:00

__init__.py

Remove ModelCacheBase.

2024-12-24 14:23:18 +00:00

cache_record.py

First pass at adding partial loading support to the ModelCache.

2025-01-07 00:30:58 +00:00

cache_stats.py

Move CacheStats to its own file.

2024-12-24 14:23:18 +00:00

dev_utils.py

Add a utility to help with determining the working memory required for expensive operations.

2025-01-07 01:20:15 +00:00

model_cache.py

Revise the logic for calculating the RAM model cache limit.

2025-01-16 23:46:07 +00:00

utils.py

Add get_effective_device(...) utility to aid in determining the effective device of models that are partially loaded.

2025-01-07 00:31:00 +00:00