Confidence Is Not Reliability: Trust Signals for Automated Plant Diagnosis
The Short Version
Most plant diagnosis tools show you a confidence number. Confidence tells you how strongly the model picked an answer – not whether you should act on it. Those are different questions, and on a hard photo they get different answers: a model can be very confident and flat wrong. If you're feeding a diagnosis into an automation – a Home Assistant flow, a grow-room controller, a dashboard alert – the signal you actually want is reliability: how trustworthy is this answer, on this specific image? PlantLab returns both. Here's how to use each, and why automation should gate on reliability, not confidence.
The failure mode nobody warns you about
Here's the trap. You wire a plant diagnosis API into an automation. When the model says “magnesium deficiency” with 0.9 confidence, you trigger an action – a notification, a nutrient-dosing routine, a log entry. It works in testing. The clear photos you tested with all return high confidence and correct answers.
Then a real photo arrives: backlit, half in shadow, two plants in frame, taken at midnight because something looked off. The model returns “magnesium deficiency” at 0.9 confidence again. Except this time it's wrong, because that confidence number was never a promise about correctness. It was a statement about how strongly the model leaned toward one class. On an ambiguous image, the model can lean hard in the wrong direction and report a high number while doing it.
A confidence score that's only trustworthy on easy photos isn't a trust signal. It's a confidence display. And the cases where you most need to know whether to trust an answer are exactly the cases where raw confidence is least informative.
This isn't hypothetical. A wave of consumer photo-diagnosis apps has shipped in the last year, and most of them do the same thing: wrap a general-purpose vision model and print a “confidence” percentage next to the answer. The problem is that a general-purpose model handed a plant photo will produce a confident-looking number whether or not it has any real basis for the call – the percentage reflects how hard the model leaned, not how often it's right when it leans that hard. A confidence figure with nothing calibrating it against actual outcomes is decoration. It looks like a trust signal and behaves like one right up until the photo gets hard, which is the moment you needed it most.
Three different questions
The fix is to stop treating “confidence” as one signal and recognize that an automated diagnosis is really answering three separate questions.
| Signal | The question it answers | Where it lives |
|---|---|---|
| Per-class confidence | “How strongly did the model pick each condition?” | confidence on each item in conditions[] |
| Reliability | “Should this whole diagnosis be acted on, on this image?” | reliability_score on the response |
| Policy threshold | “What does my system do at each trust level?” | Your code |
Per-class confidence is the model's strength of preference for a specific condition. It's useful when you want to know what the model saw and how it ranked the possibilities. It is not a verdict on whether the overall answer is safe to act on.
Reliability is a single number from 0 to 1 that estimates how trustworthy the entire diagnosis is on this particular image. It's the signal that stays informative on the hard cases – the ambiguous, badly-lit, multi-plant photos where you most need to know whether to trust the result. Higher is better. Low means “something about this image makes the answer shaky, double-check before acting.”
The policy threshold is yours. The API gives you a number; your system decides what to do at each level. That decision is a product choice, not a model output, and it's where the actual safety lives.
Wiring automation to the right signal
The practical rule for any automation: branch on reliability, then use per-class confidence for the details.
A reasonable starting policy, which you should tune to your own tolerance for false actions:
- High reliability (say, above 0.7): safe to act automatically. Update the dashboard, log the plant state, trigger the routine. This is the band where automating saves you work without costing you trust.
- Middle reliability (roughly 0.3 to 0.7): don't act blind. This is the band to ask for a second photo, queue the case for a human glance, or surface a “check this one” flag instead of firing an action.
- Low reliability (below 0.3): do not automate. Show the result as informational at most. Acting on a low-reliability diagnosis is how an automation does the wrong thing confidently.
The reason this ordering matters: per-class confidence will happily hand you a high number in all three bands. Reliability is what separates them. If you gate automation on confidence alone, you'll take action on the 2 AM backlit photo. If you gate on reliability, that photo lands in the “ask for another shot” band where it belongs.
For an integrator, this also means you don't have to build your own uncertainty handling from scratch. The hard part – estimating whether an answer holds up on a given image – is in the response. Your job is to pick the cutoffs that match what your automation does when it's wrong.
Why PlantLab returns a single reliability number
PlantLab used to return rule-based trust fields derived from the model's confidences. Those were removed in favor of a single learned reliability signal, and the API schema was bumped to make the change loud rather than silent. The full migration writeup is linked below.
The short version of why: a trust signal assembled from a handful of rules works on easy cases and falls apart on hard ones, which is the opposite of what you want. A trust signal is only earning its keep on the ambiguous cases – the ones where the answer genuinely might be wrong. So PlantLab consolidated to one number, on one contract: a 0-to-1 reliability score, present whenever the API returns a condition diagnosis, designed to stay honest on the cases that matter.
One implementation note worth knowing: reliability is omitted when there's no condition diagnosis to score – for a non-cannabis photo, or a healthy plant. Treat a missing score as “no score available,” not as a low score. Your automation logic should handle absence explicitly rather than defaulting it to zero.
The product principle underneath
A diagnosis API that always answers, always confidently, is easy to build and dangerous to automate against. The harder thing – and the one I think actually matters – is an API that knows when to slow you down, that can say, in effect, “I have an answer, but this image is the kind where I'm often wrong, so don't wire a pump to this one.”
That's what the confidence-versus-reliability distinction is really about. Confidence is the model talking about itself. Reliability is the system telling you whether to listen. For anything beyond a human reading a single result on a screen – and especially for automation that acts without a person in the loop – reliability is the number to build on.
PlantLab is free to try at plantlab.ai. Three diagnoses a day, results in milliseconds. Every diagnosis returns a reliability score; the full API documentation lives at plantlab.ai/docs.
FAQ
What's the difference between confidence and reliability?
Per-class confidence is how strongly the model picked a specific condition. Reliability is how trustworthy the whole diagnosis is on this particular image. A model can be highly confident in a wrong answer on an ambiguous photo – which is exactly why you want a separate reliability signal for automation decisions.
Which signal should I use to trigger an automation?
Reliability. Gate the automation on the reliability score (act high, verify middle, don't automate low), then use per-class confidence for the details of what to display or log. Gating on confidence alone will take action on hard photos where confidence is high but the answer isn't trustworthy.
What thresholds should I use?
A reasonable starting point is above 0.7 for automatic action, 0.3 to 0.7 for “ask for another photo or a human check,” and below 0.3 for “don't automate.” These are starting values – tune them to how costly a wrong action is in your setup.
What if the reliability score is missing?
It's omitted when there's no condition diagnosis to score – a non-cannabis photo or a healthy plant. Treat absence as “no score available,” not as a low score, and handle it explicitly in your code.
Does a high confidence number mean the diagnosis is correct?
No. Confidence reflects how strongly the model leaned toward an answer, not whether the answer is right. On clear photos the two usually agree; on ambiguous ones they can diverge, which is the whole reason reliability exists as a separate signal.
Related reading: – How PlantLab Knows When It Might Be Wrong: The reliability_score Field – The schema change and one-line migration – How PlantLab's AI Diagnoses 31 Cannabis Plant Problems in 18 Milliseconds – The pipeline behind the API – Build an Autonomous Plant Health Monitor with AI + Home Assistant – Where reliability thresholds earn their keep