When we reach for it
Custom models, fine-tuning, and computer vision where an off-the-shelf API does not fit the domain. That normally means the data is specialised and belongs to the client — a defect pattern, a document layout, a signal nobody else holds a corpus of. Where that is true, a small model trained on it beats a large general one and costs less to serve.
When we would argue against it
Problems a hosted model already solves. Training something to do what an API does is a research budget spent on a solved problem. We would rather lose the more interesting project than bill a research budget for an outcome an API already delivers, and we will put the comparison in front of you before either of us commits to it.
What it looks like in delivery
A held-out evaluation set defined before training, and the metric agreed with whoever will act on the output. Runs are reproducible from a recorded data version and a seed, and a model that cannot beat the simple baseline does not ship — which happens more often than the field likes to admit.