Indefensible Data: The Thread That Can Quickly Unravel

By Jeanne Mensingh, NELAP Assessor and President, Labtopia Solutions, and Thermo Fisher Scientific Partner

Not all data is good data. Businesses know this — or at least should — and it effects how they make decisions about sales, marketing and R&D. And when compliance with industry regulation hangs in the balance, as it does in the water industry, the stakes are even higher. An indefensible laboratory result is like a thread that can quickly unravel a project and/or lead to onerous financial penalty.   

Running a laboratory with sufficient rigor to literally engineer discipline into the process, from sample collection to report generation is critical — we call this engineering defensibility. While this was nearly impossible in the days of paper-based workflows, today’s labs can track, manage and report with precision and timeliness. But not all do, and that must change. Make no mistake, the penalties for poor defensibility may start with irritating delays, but exposure to business and legal risk, including civil suits, can rapidly escalate costs in short order.

Mitigating Risk with Unquestioned Data

Water industry labs rely on many instruments, from gas chromatographs to mass spectrometry, and a single test has multiple fail points, not to mention the other factors that go into a results: consumables, staff training, equipment maintenance records and much more. Average labs run thousands of tests month, so it’s not hard to imagine the sizable margin for error.

In most labs, data defensibility isn’t a coordinated or continuous process. In fact, is often reactionary and episodic. It takes time and resources that could be applied elsewhere — so labs tend to address defensibly on an as-required basis. And, when the moment arrives, what confronts them isn’t good — handwritten notes, files in different locations and no consolidated system for analysis and reporting. This task is daunting and often doesn’t even produce sufficient detail to defend the result.

Rigor is strong antidote for risk. This is especially true where analytical instruments and complex workflows are concerned. Risk goes down the more control we have over the process.

The first step for the water industry, like others where labs are central to operations, was to take the long journey from paper to digital. Major strides have been made there. Equally important has been the advent of software and other technology, including laboratory information management systems (LIMS), which are now more like comprehensive enterprise-level data management platforms that gather, monitor and manage all laboratory data and records.

Simply put, the technology exists. The means are there. The industry is heading in the right direction. This is fortunate since regulation will only get more onerous, not less.

Real-World Example

Few water laboratories are without a gas chromatograph. But it’s not just a lone workstation in a lab. Cascading from a single instrument are so many steps, inputs, dependencies and other variables that influence defensibility. Fortunately, some smart software engineers conceived of a way to capture all this in software, making the necessary outputs available to the right people when, how and in whatever form it’s needed. Today we simply call this system a LIMS.

A LIMS is a workhorse in a lab, but its role in defensibility is may be less understood. Let’s change that. Considering the chromatography example, there are two areas where the LIMS really shows its worth to the average water industry laboratory: staff performance and technical quality assurance.

Staff Performance

Even with the best intentions, humans make mistakes. More often than not, these mistakes stem from poor training, failure to follow processes and errors during data entry. If there’s no way to mitigate the frequency and risk of these mistakes, it is likely that data produced by staff will not be defensible.

• Poor Training

Instruments are updated and swapped out. Workflows change and new ones are added. Staff are redirected to unfamiliar processes. As this all happens, every detail must be documented, and this must inform refresher training. A LIMS is set up to simplify this by collecting and organizing all this in one place. Staff records can be quickly accessed, including whether they are up-to-date on certification and are therefore qualified to run a test in the first place.  

• Process inattention

Multistep processes have multiple failure points. Corners are cut and distractions can cause staff to miss a small detail. But a LIMS can store, and in some cases automate, SOPs so that staff can follow analytical and operational process. As staff progress through steps of an SOP, the system prompts them to confirm completion — in order, creating an audit trail as they go. Some labs enable their LIMS to report deviations from processes to a lab manager in real-time so that problems can be fixed before they escalate.

• Data entry errors

Many labs are moving toward automatic data capture, avoiding the critical error point of manual data entry.  A LIMS can automatically collect and aggregate lab instrument data for use by others or elsewhere in the workflow. A vendor-agnostic LIMS is particularly important here because it’s capable of interfacing with a broad range of instruments from multiple vendors.

Technical Quality Assurance

The quality of a result starts with vendors outside the lab: suppliers can be a first point of failure. The quality of instruments and consumables isn’t a given, especially as instruments are used and consumables inventories ebb and flow. Smart labs will subject instruments and consumable inventories to constant quality rigor, and a LIMS can help here too.

• Suppliers

Instruments are undergoing constant flux. A chromatograph has syringes, carrier gases, columns and other parts and consumables that wear and change under constant use. Not every supplier product can be tested — it makes more sense to establish a roster of trusted suppliers who can be periodically audited to ensure ongoing quality.

The LIMS can then be set up to track supplies as they enter a facility, matching them to preferred suppliers. As long as technicians use supplies from trusted suppliers — and the LIMS will make it hard to do otherwise — consumables can largely be ruled out as a future fail point. In other words, it’s easier to defend a result from LIMS-approved vendor than something acquired out of the process.

• Consumable Quality

Even with trusted vendors, consumables can eventually go out of spec. A LIMS can help here too. It can be configured to warn staff about aging stock and other changes in inventory that could render a result indefensible in the future.

• Instrument Maintenance and Calibration

Chromatographs require regular maintenance and calibration. Poor maintenance is a major fail point for labs. The LIMS provides a defense against lax maintenance and calibration practices: to complete a test it will require certified reference materials. If configured properly, it can almost stand in the way of staff prepared to test on instruments that could potentially deliver an indefensible result.

Tracing Errors at the Source, No Matter How Small

If you work in or manage a water industry lab, you know firsthand that it can be frenetic and fast-paced with many distractions. This is especially true today as shrinking budgets force many lab do more with less. But you can’t let frenzy and distraction compromise quality — you still need to defend results. Otherwise compliance reporting is meaningless and risk exposure is unacceptably high.

Not matter how small something may seem, it probably plays or will play a pivotal role in future defensibility. By engineering defensibility into end-to-end laboratory processes, water industry labs protect themselves from the high costs and disruptions of being caught off-guard if their data is challenged. You can’t know when that will happen, but, when it does, it’s good to know that your LIMS-enabled lab can provide exactly what’s needed to defend your data, all the way to the source of the error. Assuming, of course, that you haven’t already engineered those errors out.