How to Assess the Effectiveness of Your Compliance Program
Compliance officers everywhere want to believe the compliance program they oversee is effective. Some believe it is effective, some hope it will be found effective and some know the program is not effective because of significant gaps in one or more of the seven elements of an effective compliance program.
If you are a believer, ask yourself, “What methods have I established to demonstrate effectiveness?”
If you are filled with hope – well hope is not a strategy.
If you know your program has gaps, what are you doing to address those gaps?
An additional resource now exists to help evaluate effectiveness. The OIG/HCCA Measuring Compliance Program Effectiveness: A Resource Guide released March 27, 2017, provides recommendations on what to measure and how to measure it under each of the seven elements.
Resource Guide Background
First, I’d like to provide some basic information about the Resource Guide. It is the compilation of work by compliance professionals and staff from the DHHS Office of Inspector General in a roundtable discussion. The outline used to identify which areas might be evaluated for effectiveness comes from the Detailed Content Outline (DCO) of the Certified in Healthcare Compliance (CHC) Candidate Handbook. The DCO comes from work of the Compliance Certification Board, with a psychometrics company, based on a wide variety of compliance professionals’ responses to a job analysis survey.
Over the course of the 17 plus years the CHC has been in existence, multiple job analysis surveys have been performed regularly and the DCO updated accordingly. The job analysis survey asks professionals what percentage of their time is spent in different tasks. As you can imagine, there is variation between what a compliance officer for a nursing home might do compared to one working for a large health system or a research facility. But the basis of the outline comes from what a variety of compliance professionals say they do on a regular basis. Because of this variation, not every element in the Resource Guide is going to be relevant to all compliance programs.
With that in mind, the tool can still be useful as a starting point for assessing compliance program effectiveness. But because the tool is quite expansive, it might seem daunting. The seven elements are broken down into a total of 75 potential areas to assess or measure. Those categories are further tied to a total of 401 methods for measurement or evaluation.
How to Use the Resource Guide
So how do you tackle this? First, the DHHS Inspector General made it very clear in his keynote address given to an audience of compliance professionals at the HCCA Annual Institute on April 27, 2017, the Resource Guide is not intended to be an exact recipe for assessing effectiveness. Some of the categories of what to measure might not be applicable to every compliance program. Just as the ways to measure or assess a particular item might not be all-inclusive. One needs to carefully consider the Resource Guide and pick the items that most closely match the compliance program that is being evaluated.
Correlate Current Program Elements Against Recommendations
One could take each of the seven elements and correlate what is currently being performed, tracked, measured, assessed, audited or monitored in the program against the recommendations in the Guide.
- If a current process exists how does it compare to what is being recommended?
- Are there things being measured in your program that are not on the Guide? If so document this and why the measures being used have and do demonstrate the effectiveness of your program.
Compare How You Measure Each Item
Of the items that do have a correlation, do a comparison of how your compliance program measures each item to what the Guide recommends as a possible way to measure the same item. If you are not currently documenting or gathering the metrics to be able to measure the component of your program, think about how you would begin to do so.
It might simply be a matter of reviewing something to determine if you engage in the activity.
For example, under “Standards, Policies and Procedures,” the first category is “Access” and the first subcategory is “Accessibility”. A suggested way to measure that is a review of the link to an employee accessible website or intranet that includes the Code of Conduct.
- Do you have that?
- Are there ways to evaluate how often the site is visited, ideally by unique users? If you discover it is seldom or never visited you might wonder just how accessible it is.
Other measures might be a little more complex. Also under “Standards, Policies and Procedures” is a category titled “Quality” and a subcategory “Are policies (and procedures) as good as industry practice?” The recommended method to measure this is “Peer reviews.” This might require having several colleagues review your policies or engaging a reputable third party to evaluate based on their industry experience.
Determine If the Elements are Being Covered by a Business Unit Other than Compliance
Another useful way to use the Resource Guide is to review the various items to measure and ask if that element is being covered by someone in your organization. For example, if the compliance officer does not have oversight for conflicts of interest, who does? If the answer is no one, there might be an issue for the organization and could potentially lead to a finding the organization has not effectively crafted an oversight structure for this important area. If it is being addressed by another business unit, is it being audited by compliance or someone else to assure policies and procedures are being followed and identified conflicts are managed?
Benefits of Using the Resource Guide
The Resource Guide has a few possible ways in which it can be used to assess all or some of the components of a compliance program. The benefit of the tool is that there is now a much stronger starting point to begin the assessment. Also, the tool was developed in conjunction with the enforcement body that is likely to be determining if a program is effective for purposes of enforcement actions. The more closely you can match your program to the relevant portions of the Resource Guide the better off you are likely to be should your organization ever find itself in that unfortunate situation.
It might even make those that hope their program is effective feel better about it once several of the categories and subcategories have been evaluated. It also gets the industry a little further away from the typical response of “I can’t define it but I know when I see it” when responding to the question, “What does an effective compliance program look like?”