AI screening in the pilot · every result says “show your vet”
How our AI scores on photos it has never seen, where it goes wrong, and what it hasn’t been tested on yet. A model only gets a number here once it has been tested honestly.
Reads a photo of fresh droppings and screens for coccidiosis, salmonellosis and Newcastle disease.
Listens to 10 seconds of a poultry house and screens for coughing, rattling and distress.
Any model looks brilliant on photos like the ones it learned from. The real test is photos from somewhere else, with answers a lab confirmed. We always show both numbers.
Balanced accuracy: the average of how often it’s right on each of the four classes, so the rarest disease counts as much as the commonest. Across three training runs the lab-confirmed score stayed between 79% and 82%: it’s the model, not luck.
Researchers trained well-known models on the same farm photos and tested them on the same lab-confirmed photos. Here they are next to N-Anya.
Share of the 1,201 lab-confirmed photos each model got right. The published models are the average of three training runs each.
“Catches it”: of the photos the lab confirmed for each class, how many the model named correctly. Healthy is lowest: 107 of 312 healthy photos were called salmonellosis.
Each row is what a lab test confirmed, on the 1,201 photos. Green is agreement; the rest shows exactly which diseases get mixed up.
107 of 312 · 34%
A healthy flock can be told “worth checking for salmonellosis”. It happened in all three training runs (97 to 107 photos), so it’s a pattern, not bad luck.
30 of 357 · 8%
The dangerous direction: 30 to 44 photos across the runs. It’s why the app always says to call your vet if birds look sick, whatever the photo shows.
294 of 1,201 · 24%
Safe, because those go straight to the vet, but frequent. Clearer photos and Nigerian training data should bring it down.
Pilot
Every test photo so far comes from Tanzania. Nigerian feed, breeds, floors and phones may look different. Proving it here is what the pilot with founding vets is for.
N-Anya only stops saying “show your vet” on every result once it passes all three, on lab-confirmed photos, in every training run. One rule is still just short, so it stays in the pilot.
| Rule | Needed | Run 1 (in use) | Run 2 | Run 3 | |
|---|---|---|---|---|---|
| Sick flocks told to check | 80% or more | 81.2% | 79.6% | 78.3% | Just short |
| Healthy flocks wrongly alarmed | 25% or less | 20.8% | 23.7% | 16.7% | Passes |
| Newcastle cases caught | 70% or more | 89.2% | 93.5% | 88.7% | Passes |
On the test photos, a result counts as correct only when it matches a lab (PCR) result, not a guess from the photo.
The test photos are a separate collection the model never learned from. We checked: not one is a copy or near-copy of a training photo.
Every new version is scored on the same 1,201 photos, so an improvement is real, not luck.
Photos: Machuve et al. (2021), NM-AIST poultry diseases datasets, farm-labelled and PCR-annotated, Tanzania, CC BY 4.0 (doi:10.5281/zenodo.4628934, doi:10.5281/zenodo.5801834). Published models and duplicate lists: Pratama et al. (2026), lab-to-farm audit, CC BY 4.0 (doi:10.5281/zenodo.23374400). Not yet tested on Nigerian farms: that’s what the pilot is for.