This might be useful if enough people use it. Sort of like down detector, but for model performance throttling or deterioration.