Device Accreditation
A Bayesian Approach to R&D Validation
"The demand by decision-makers for strategic intelligence on new and emerging technologies has arguably never been higher..."
OECD Science, Technology and Industry Policy Papers, No. 146.
67% of estimates
must be within ±1 IMF%
95% of estimates
must be within ±2 IMF%
The Challenge: Determine an Accreditation Process that is Fair and Cost Effective
The Australian meat industry relies on measuring intramuscular fat (IMF%) to predict the eating quality of lamb. However, the traditional laboratory-based test is destructive and too slow for abattoirs. New, non-destructive sensor technologies are needed, but they must first be accredited against strict industry accuracy standards.
The problem was that even a highly accurate device could easily fail accreditation due to statistical chance when tested with a small sample size. Our simulations showed that a device with an acceptable accuracy (a standard deviation of 0.95 IMF%) had only a 40% chance of passing the test with 800 samples. This created a significant barrier to innovation.
Our Approach: A Smarter, Fairer Model
To solve this, we moved away from a simple pass/fail based on sample counts. Instead, we developed a regression model using a Bayesian framework. This approach provides a transparent, robust, and reproducible method for assessing a device's true accuracy.
Crucially, the Bayesian model allows us to incorporate prior knowledge, based on industry consultation. This acknowledges the reality that manufacturers would only seek accreditation if their device was already reasonably accurate. By including this information, we significantly reduced the sample size required to fairly test a new device.
For a detailed technical overview of this methodology, the full study is published in the peer-reviewed journal PLoS ONE.
Interactive Comparison: Two Accreditation Models
Traditional Rules-Based Model
Relies solely on a large test sample to meet the strict pass/fail criteria.
Large Sample Size Required
Our Bayesian Model
Combines a smaller test sample with approved prior industry knowledge.
Smaller, Fairer Sample Size
The chart above illustrates the probability of three different devices passing accreditation under the traditional rules-based model. It shows that a high-accuracy device (A) passes quickly, but an acceptable device (B) has a very low chance of passing, even with a large sample size. This highlights the key flaw that our Bayesian model solves.
The Impact: Accelerating Innovation
Our Bayesian model has been implemented for all new devices seeking accreditation via an R Shiny App. This tool allows assessors and manufacturers to work together transparently. Since its deployment, several devices have successfully obtained accreditation, providing clear, understandable metrics for assessment personnel.
One notable success is a handheld microwave system that has already achieved its first commercial installation at WA's Dardanup Butchering Company. As Professor Graham Gardner of Murdoch University noted, this technology helps deliver "fairer and more accurate grading of carcase eating quality," creating opportunities for the sheepmeat supply chain to develop premium lamb brands.
Ask Our AI Assistant
Hello! I am here to answer your questions about our device accreditation model. Please select one of the common questions to get started.
Common Questions:
Ready to Build Your Architecture?
Stop relying on generic APIs. Let our Forward Deployed Engineers build a proprietary AI system that belongs to you.