Evan Reichard
88308602c8
Allow one CUDA0 and one CUDA1 model to run simultaneously. Dual-GPU models (using -ts splits) are excluded from the matrix so they evict everything when loaded. vLLM docker models get evict_cost=50 to discourage eviction due to slow cold starts.