Super Models

Victoria Ceh, left,  oversaw the effort
to make the International Society of
Hair Restoration Surgery one of the most
exemplary CME providers in the United States.
Diane Alberson, right, of the Society of Critical
Care Medicine created an ACCME-friendly
tool kit for developing new programs.

Most companies that provide continuing medical education undergo a rigorous tire-kicking every four years by the Chicago-based Accreditation Council for Continuing Medical Education. The process involves a written report, often of several hundred pages, and a comprehensive site visit. To prepare for accreditation, CME providers are given a set of general guidelines that must be translated into specific actions -- not an easy task. To an organization that has to undergo this process for the first time, the experience can be harrowing.

Despite appearances, the ACCME is not out to make the lives of CME providers difficult. As an aid, the council publishes a list of those few providers who have achieved “exemplary compliance” on one or more of the 10 required elements for accreditation. The best of these companies have been granted a special “accreditation with commendation” status, a category that gives providers added prestige and an extra two years before having to renew. These stars of the industry -- those who received commendation and those who are on track to -- were only too happy to share their best practices with M&C.

The wrench in the gears is that the ACCME has announced a major change in the way accreditation will be granted (see “New Criteria on the Horizon”), which will require CME providers to step up their efforts in the coming years. What became apparent when talking to these meeting planners, however, is that the best companies already are doing most of what the ACCME will be requiring. What’s equally clear is that groundbreaking methods for planning continuing medical education -- good record keeping, constant tweaking and active collection of feedback -- are applicable not just to medical meetings, but to meetings in all industries.

Benchmark Medical Consultants
Exemplary in: Needs Assessment, Activity Evaluation, Program Evaluation

Benchmark Medical Consultants is a Sacramento, Calif.-based company that provides administrative support for doctors who serve as medical experts in courtrooms. After the company received standard accreditation in 2000, Craig Vreeken, CME director, wanted to do even better. Vreeken went to an accreditation workshop put on by the ACCME in Chicago to figure out how to improve the company’s score. He says it was very helpful to see what documents the site surveyors use to evaluate a program. Now that he knows what they’re looking for, he approaches every activity thinking about how to achieve an exemplary score.

Benchmark takes special care in collecting attendee needs-assessment data. The company surveys attendees once a year, including some verbal interviews, and keeps up on the hot topics in the industry by reviewing journal articles and glancing at competitors’ agendas.

The company also compiles report cards on the doctors’ courtroom testimony and gives individual feedback based on their performance. Those reports make great fodder for deciding which topics to cover in future education sessions. “If 35 percent of our doctors are not addressing one topic, we really should address it in an educational activity,” says Vreeken.

Another source of data comes from the doctors’ legal clients, who may or may not be happy with aspects of doctor testimony. Benchmark keeps a spreadsheet of all the doctors’ performance ratings and then targets the weakest areas.

Constant benchmarking also allows Vreeken to show that his programs were effective. If the doctors who attended a particular session improve in their ratings on that subject, the improvement can be linked pretty convincingly to the education.

“We know our CME activities have made an impact on the performance of doctors,” says Vreeken. “That’s exactly what the ACCME is looking for in terms of outcomes.”

To education providers looking to improve measurements of success, Vreeken suggests looking closely at what measurements already are taken, and brainstorming ways to use that information.

International Society of Hair Restoration Surgery
Exemplary in: Mission, Planning Processes, Needs Assessment, Activity Evaluation, Program Evaluation

When the International Society of Hair Restoration Surgery was formed in 1993, one of the founders’ goals was to receive accreditation from the ACCME. In the late ’90s, management at the ISHRS applied for the association’s first accreditation, and the application was rejected.

“When we applied, we didn’t think it was a hard thing to do,” says Victoria Ceh, MPA, executive director of the society, based in Geneva, Ill. “We turned in the application and were caught with our pants down.”

Over the next five or six years, ISHRS management worked diligently to restructure the entire operation, and they applied again in 2005. In March 2006, they heard that not only had the ACCME granted them provisional accreditation (the best they grant to new applicants), but also that the society was found exemplary in five elements, making it one of the most compliant CME providers in the country.

As to how they did it, Ceh just says it took a lot of work. “You really have to buy into the whole process,” she says. “There’s no easy way.”

The first step, she says, is to sit down with the accreditation criteria to figure out what the ACCME
is looking for with each of the elements. She called other organizations that had achieved exemplary compliance and bought a number of editions of the Best Practices in Accreditation Handbook from the Birmingham, Ala.-based Alliance for Continuing Medical Education, which highlights selections from the best applications over the years. That gave her an abundance of ideas for both what to put into the application and what practices to adopt.

The next step is to hire a competent statistician who has a good understanding of the CME world to process all the data that comes out of needs assessments and post-event surveys. The right person knows when the statistics show a change is needed, and when things are working as they are. Ceh has invited her statistician to participate on conference calls with stakeholders to help communicate to them exactly how much attendees are learning.

The last step in the accreditation process is to fully document every idea that’s been tried. Ceh documents even quick conversations, if any changes were discussed. Every idea goes into a “best practices” file.

Among the notable changes the ISHRS has made:

* The post-event evaluations have been put online and offered three days after the event, to give attendees time to reflect on their experience. An open-ended question was added to allow attendees to write in what they wanted to change in their practice that year. In a follow-up communication three months later, each attendee was shown her response and asked if she was successful at making the change.

* Speakers have been chosen by the strength of their research and vetted by their scores from past years (a panel of physicians rates every talk). Those with a poor command of English have been welcomed but asked to submit their speeches for editing, and to have the speech recorded by a native English speaker, to be played while the presenter stands at the podium.

For last year’s annual meeting of 600 attendees, Ceh ensured every ACCME policy was followed to the letter, and the results were stunning. “By buying into the policies, it came off as the best meeting we ever had,” she notes.

Ohio State Medical Association
Exemplary in: Mission, Needs Assessment, Activity Evaluation, Program Evaluation, Organizational Framework

Mary Whitaker, director of education at the Ohio State Medical Association, based in Hilliard, Ohio, realizes receiving exemplary compliance is as much about what she’s doing as it is about how well she’s reporting it.

“It’s kind of a two-part thing,” says Whitaker. “One, you have to have the systems in place. Two, when you’re surveyed, you have to be able to communicate it. I think a lot of people do things they don’t consider.”

She keeps an electronic diary of projects and insights, and it’s backed up in case the computer crashes. “It’s for whenever I have an ‘aha!’ moment, or when we make a change in something, or when we respond to an article.”

All these bits of information take a long time to compile; Whitaker suggests allotting nine months to a year before the reaccreditation for this purpose. And she recommends considering everything for inclusion. “What you might take for granted as a kind of routine thing might actually be a change,” she says. “Any change is an improvement, and they emphasize improvements.”

Whitaker also takes great pains to make the application as readable as possible. She puts all the improvements into a convenient timeline format, pulling out each year’s gains in text boxes. She includes an executive summary at the beginning of the application to help orient surveyors. “When you send in your application, it can be 300 pages long,” she says. “You want to make it as easy as possible for them to find the information you want them to glean.”

Next, she shares the application with the company stakeholders. Whitaker says the accreditation process becomes less onerous if the focus is on sharing improvements internally, to show the stakeholders how well the CME program is performing. She points out that a supportive education committee is crucial to creating the best education.

In addition, Whitaker offers the following take-home tips.

* Require all presenters to teach at least one thing that learners can implement immediately into their practices. Then follow up by asking physicians if they implemented it.

* Keep at physicians to complete evaluation forms and follow-ups. “We’re like a dog with a bone,” Whitaker says. “We don’t let go.”

* On the needs-assessment form, instead of asking attendees, “What other programs are you interested in?” write, “I could do my job more effectively if I knew more about _____.” It gets at more precisely what learners need.

In regard to the newly announced changes to the accreditation criteria, Whitaker’s first reaction was, “Oh, no, not again!” Upon further examination, however, she came to the realization that because she had been striving for exemplary compliance, her program was halfway there.

PRIME
Exemplary in:Program Evaluation, Organizational Framework

“The process of accrediting and reaccrediting is an art and a science,” notes Kathleen Moreo, RN, BSN, BHSA, president and COO of PRIME, a full-service medical education company based in Tamarac, Fla. “The science is how to quantify it, and the art is understanding how to go through the land mines of how you plan to go through the data.”

Due to an error in compiling its report, PRIME didn’t technically earn accreditation with commendation (which requires three “exemplary” ratings); however, the company did receive exemplary compliance in two crucial areas, for its cutting-edge practices in developing and evaluating programs.

PRIME is focused on discovering the best way to measure the performance improvement of attendees. The company is piloting a program called Transformational Learning, a method of education in which the follow-up after the conference is just as important as the education itself. As Moreo explains, the ACCME requires companies to be able to determine whether the activity was relevant to attendees. PRIME attempts to go one step further, to determine whether doctors use in their practices what they learned at meetings.

One element Moreo was especially proud of, which wasn’t included in the reaccreditation report, related to the management of grant funding. When PRIME receives commercial support, the money is immediately put into an escrow account and is retrieved only when an invoice is written for an expense. When the program is done, the company presents all the receipts and returns any unused funds.

The procedure takes very little extra work and makes a great impression on funders. “Did that take hiring a lot of staff or developing committees?” Moreo poses. “No! It took sitting down with finance managers and accountants and setting things up much the way they do in an attorney’s office. And it helps the supporters to see we’re very serious about the money they give us.”

For companies looking to improve their accreditation scores, Moreo advises seeing what resources are available within the organization and examining current processes for better methods. As she puts it, “If we were on the other side of the table, giving the funding, what would we want to see? Those are the kinds of innovations the ACCME wants to see.”

ProScan Imaging/MRI Education Foundation
Exemplary in: Needs Assessment, Ac-tivity Evaluation, Organizational Framework

ProScan Imaging is an imaging center (providing MRIs, CT scans, etc.) not affiliated with a university or hospital. The MRI Education Foundation is ProScan’s program to educate radiologists on new imaging technology. Two accreditation cycles ago, the company was put on probation by the ACCME for substandard practices. This year, it earned accreditation with commendation -- a bit of a surprise to Mary Rider, director of CME and physician services. Looking at the company’s intricate processes, though, it’s easy to trace the path to excellence.

Rider and her colleagues take the following measures to provide the very best education.

* During multiday programs, the company brings a focus group of attendees out to dinner to ask them how the meeting is going and what topics and techniques they would like to see in the future.

* Before each meeting, Rider sends out surveys to attendees, and they fill out an online needs assessment as well. To get physicians to complete the surveys, the needs assessment pops up right when people register. The owner, who is also the primary instructor, uses that data to craft the perfect syllabus.

* Attendees are surveyed immediately after each course and are asked questions about what specific actions they will take to improve their practice. By comparing pre- and post-course assessments, the company can determine how much attendees learned.

Because the main educator is the owner of the company, there was a worry that the education could be biased. To circumvent this concern, it was decided that the owner would not be a voting member of the CME committee. “It’s not the easiest thing to do, to tell the CEO that he can’t vote,” Rider says.

Rider developed an accounting code system to make it easier to track spending. Every course has a separate code.

She documents every process improvement that is attempted. “With a lot of things, it isn’t that you don’t do it, it’s that you didn’t document it,” she says. If the organizational system isn’t up to par, ask the person in charge for permission to make necessary changes.

To providers looking to improve, Rider recommends starting with the easiest things to implement and going from there. She says to start now, because a good practice takes a long time to create. “Everyone is so rushed, it’s hard to do,” she says. “But it’s well worth it.”

Society of Critical Care Medicine
Exemplary in: Needs Assessment, Activity Evaluation, Program Evaluation

In a large organization such as the Des Plaines, Ill.-based Society of Critical Care Medicine, which plans an annual meeting for 5,000 critical-care providers, it’s a huge challenge to ensure that every department participates in compliance with the often-abstruse regulations. Diane Alberson, manager, continuing education, created a tool kit for developing new programs that incorporates both the ACCME regulations and the society’s internal regulations, which often are stricter.

In the tool kit are checklists and step-by-step instructions that give everyone, regardless of where they are in the organization, the means of following all the guidelines. Extensive communication is required to keep every department abreast of changing rules.

Another way to ensure consistent compliance throughout the organization is through quarterly audits. The CME department collects feedback about each program from the volunteers in charge of running them and spends two or three hours going through the data, deciding what’s not up to standard and how all the programs could be improved.

This focus on continual improvement is what the ACCME will be looking for when the compliance rules change, says Alberson. “The new guidelines are encouraging providers to think about the steps,” she notes. “They’re no longer focusing on the end result, but questioning how you’re developing the programs, based on our ultimate goal: impacting care at the bedside.”

Adds Nancy Stonis, RN, MJ (Master of Jurisprudence in Health Law), director, program development and professional affairs for SCCM, the goal is to create education that improves patient care and encourages lifelong learning. “When developing the programs,” she says, “it helps to be able to put the pieces together and incorporate the bigger picture into the program.”

Specific actions the SCCM is taking:

* Encouraging speakers to bring in more case studies and to use audience-response technology to better involve the attendees;

* Sharing printed and online material with attendees before the course
or on-site;

* Developing a new website that will give members the opportunity to learn individually, by watching webcasts of programs or testing themselves before and after programs. The new site debuts in February.

All the extra paperwork, analysis and communication can seem like useless effort. But when that accreditation with commendation rating comes in, “it validates all those ‘extra things’ that are being required of our volunteers,” says Alberson. Those extra things are “actually necessary for the good quality programming we do.”

For an organization looking to improve its accreditation score, Alberson suggests sitting down with the ACCME’s criteria, figuring out what everything means, and determining all the strengths and weaknesses within the organization. Then, spend a few months on each of the problem areas, working to improve processes.

“If you know what the criteria are, you can then just develop good quality programming to meet the needs of your attendees,” says Alberson. “The policies will fall right into place.”

New Criteria on the Horizon

In 2003, the Chicago-based Accreditation Council for Continuing Medical Education made the decision to explore ways providers of continuing medical education could facilitate physician performance improvement. Up to that point, there was no requirement for CME providers to make sure physicians actually got better at their jobs through the education they received.

In September 2006, the ACCME announced updated compliance criteria in an attempt to make CME more effective for learners. Providers should start adapting to the new criteria now. The new criteria are complex, but here are the basics of what CME providers need to know.

Constant improvement. Learning and change are emphasized, both for providers and learners. Providers are expected to improve their practice over time, in ways that explore how to improve physician practice.

A focus on experimentation. Trying to do better will be rewarded. Because there are no standardized ways of offering the most effective education, providers who seek ways of improving physician performance will be commended, even if they don’t succeed.

Newly defined grades. The ACCME has redefined its three levels of performance for providers: Level one, provisional accreditation, suggests a beginning attempt to analyze how doctors learn. Level two, accreditation, represents an effort to determine what is needed to facilitate physician learning, and to begin to implement those changes. Providers who have reached level three, accreditation with commendation, will be on the frontier of medical education. They will identify barriers to physician change and strive to improve physician performance.

Providers are being given two years to adjust to the changes; beginning in November 2008, providers going up for accreditation or reaccreditation will be evaluated under the new standards. By July 2012, all providers must achieve level two, or they will lose their accreditation. -- J.V.

New Requirements

September 2006
Accreditation Council for Continuing Medical Education announces updated accreditation criteria.

August 2007
The first providers to undergo the new process will be notified of their impending re-accreditation evaluation.

November 2008
The first round of accreditation decisions, under the new criteria, will be handed down.

July 2012
Those accredited in 2008 must reach level two accreditation.