Telemedicine Evaluation: Making the Case for Value in Healthcare

When you make a major purchase, you don't expect to invest in something that doesn’t provide value to you. However, the concept of value is complex, and not everyone defines it similarly.

The same idea applies to telemedicine and telehealth. The healthcare industry at large won’t fully embrace telemedicine and medical electronic communication until there’s demonstrated value and return on investment for all stakeholders—something that’s only possible with a standardized framework and methodology for consistent and comprehensive telemedicine evaluation.

If there’s ever been a more appropriate time to create this type of evaluation, it’s now. When COVID-19 struck, providers adopted telemedicine in a matter of weeks. Telehealth visits spiked. Payers instituted regulatory flexibilities that enabled physicians to render telemedicine services with ease. With this explosion in telemedicine services comes the ability to engage in important conversations about a validated structure and universal practices for telemedicine evaluation. Without this framework, payers may roll back regulatory flexibilities (note that some already have), and the industry could take two steps back instead of moving forward.

This article addresses why telemedicine evaluation—both of the program itself and of the practitioner delivering the services during a telehealth visit—is critical, and how the healthcare industry can capitalize on increased adoption rates and current regulatory flexibilities to make it happen.

What is telemedicine?

Although telemedicine and telehealth are often used interchangeably, there are important differences. Telemedicine refers to the clinical application of remote technology. Telehealth is a broader term that can include remote non-clinical services. It’s important to make this distinction to avoid confusion in the health policy realm—specifically in terms of how remote technology is used and whether (and how) payers will cover it. It’s also important in terms of identifying the scope for telemedicine evaluation.

Why is standardized telemedicine evaluation needed?

The justification for payers, provider organizations, physicians, and patients to invest in telemedicine isn't just about proving a financial return (e.g., avoidance of high-cost emergency department visits), but also about the value of time, resources, and more. 

A formal and consistent telemedicine evaluation framework can certainly help make the case for spend and cost containment, but it can also reflect either positively or negatively on other factors, such as patient outcomes, provider satisfaction, and patient experience, for example.

Standard and universal measures for telemedicine evaluation can show whether it enhances providers’ ability to treat more patients more effectively, or not. And it can show whether clinical services delivered electronically instill confidence among consumers, or not. However, the first step is for industry stakeholders to agree on the scope of these evaluations and determine precisely what to measure and evaluate to reach valid conclusions.

More specifically, the industry needs to reach a consensus on these two important questions:

• What are the focus areas of telemedicine evaluation? Quality of care and health outcomes? Access to care? Costs and cost-benefits? Patient perceptions? Clinician perceptions?
• What specific questions should evaluators ask to ensure effective telemedicine evaluations of both the program and the caregiver?

The idea of creating a broad framework for telemedicine evaluation is not new. In fact, in the mid-1990s, the National Library of Medicine requested that the Institute of Medicine author such a framework. The result, a report titled “Telemedicine: A Guide to Assessing Telecommunications in Health Care,” focuses on an evaluation framework that assesses quality of care and health outcomes, access to care, healthcare costs and cost-effectiveness, patient perceptions, and clinician perceptions. The goal? To guide policymakers, reassure patients and clinicians, inform health plan managers, and help improve telemedicine programs overall. 

However, since the publication of this report, reliable data on costs, effectiveness, and the impact of telemedicine remains limited. Progress may remain stalled until there is a validated and well-demonstrated approach to telemedicine evaluations. The lack of such assessment is in stark contrast to in-person care evaluations focused on clinical outcomes and quality.

What are current approaches to telemedicine program evaluation?

Current approaches to telemedicine evaluation focus on specific clinical specialties, or they focus on specific outcomes (e.g., cost effectiveness, patient satisfaction, or clinical functional outcomes). There’s no consistent and comprehensive approach. Instead, several limited models are currently in use:

• Structure-process-outcome. This model evaluates user acceptance but doesn’t consider clinical or organizational implications.
• Unified theory of acceptance and use of technology (UTAUT) model. This model considers performance expectancy, effort expectancy, and social influence.
• Clinical, human and organizational, educational, administrative, technical, and social (CHEATS) model. This model includes multi-dimensional perspectives, especially human and organizational factors.

More specific models have also been proposed, including the following:

• A five-dimensional assessment model that includes performance measures (time, quality, and cost), outcomes measures (safety, efficacy, and effectiveness), summary measures (cost-effectiveness comparisons), operational considerations (access and acceptability), and other issues (confidentiality and legal).
• A three-dimensional assessment model that includes functionality (consultation, diagnosis, monitoring, and mentoring), technology (modes, network design, and connectivity), and applications (treatment modalities, medical specialty, disease types, and sites).
• The Institute for a Broadband-Enabled Society model that uses patient control, clinician quality of care, organization sustainability, and technology capability with no consideration of human factors.
• The Telemedicine maturity model (TMMM) that emphasizes multiple maturity levels of a telehealth service.
• A six-component integrated model that includes health domains, health services, technologies, communication infrastructure, environmental settings, and socioeconomic analysis.

In addition, several studies have applied the Model for Assessment of Telemedicine (MAST) that includes a pre-implementation assessment followed by a multidisciplinary assessment that includes safety assessments as well as evaluations of clinical effectiveness, patient perspectives, economic implications, and more. Each of these models has value, but none is widely accepted as a national framework.

What provider types deliver telehealth visits?

Many types of providers render remote clinical care depending on their licensure and credentialing.This includes physicians, certain advanced practice professionals (APPs) (e.g., nurse practitioners, physician assistants, and clinical psychologists), and licensed clinical social workers. During the COVID-19 public health emergency, Medicare and many state Medicaid programs expanded the list of providers to include occupational and physical therapists as well as speech-language pathologists.

However, not all providers possess the experience and skill to effectively deliver telehealth services. Some providers themselves hesitate to provide telemedicine or have outright declined to offer it because they are concerned about quality, safety, or efficiency. As a result, telemedicine program evaluation must include robust individual provider evaluation.

How are individual providers evaluated for telehealth?

Telehealth evaluations of individual providers are tied to performance and payment systems, and differ depending on the provider type and/or their organizational affiliation. 

Hospitals and health systems: Physicians and APPs working in or affiliated with hospitals and health systems undergo credentialing, privileging, and reappointment processes to assess their current competence and to continually evaluate their performance—including for the telemedicine services they deliver. A provider need not be privileged specifically in telemedicine, but the services they deliver using the technology must be evaluated just as their services for in-person care would be. 

Likewise, providers working in federally qualified health centers and rural health centers abide by Centers for Medicare & Medicaid and state laws regarding the care and services they deliver, and their telemedicine services would be evaluated as a part of the holistic care and services they provide. The Department of Health & Human Services also provides best practices for providers delivering telehealth services.

Reimbursement by private or government payers increasingly depends on high quality of care at reduced cost—giving hospitals, health systems, and other facilities strong incentive to conduct evaluations on the telemedicine services their clinicians provide.

Physician practices and group practices: Physicians and others who are unaffiliated with a hospital or health system are increasingly rare, but there are standalone practices. These providers aren’t subject to the credentialing and performance monitoring protocols of hospitals to gauge their telemedicine performance—or any of the care they deliver. However, they are dependent on reimbursement for services from insurers (unless they practice concierge medicine), so they are evaluated according to the payers’ criteria, in addition to or in lieu of their own facility’s evaluation of telemedicine services delivery.  

Telemedicine companies/telemedicine vendors: Companies that provide only telemedicine services often contract with or employ physicians and other provider types who are affiliated with a health system, ensuring that they have the proper credentials and current competency. Some even have a chief medical officer (CMO) and/or a physician-led committee for hiring and evaluations, as you’d find in a hospital. However, not all do, prompting the need for a standardized or national evaluation framework for individual provider performance.  

The CMO of one telemedicine company that advocates for the stringent evaluation of telehealth provider performance lists recommended evaluation criteria, including:

• Medical experience
• History and background check
• Chart review
• Clinical guideline adherence
• Patient satisfaction scores and direct feedback
• Medical resolution rates
• Antibiotic prescription rates
• Quality assurance programs

As a result of the expanding number and type of clinicians and healthcare organizations offer telemedicine visits, the need for stringent, standard evaluations grows. 

Data from all types of providers as well as patients and other personnel can also inform a standardized or national framework moving forward, whether or not it involves tenets of the MAST.

What are general best practices for telemedicine program evaluations?

There are many considerations. First, an effective telemedicine evaluation defines the purpose, target audience, and scope. Second, evaluations should look beyond providers to include patients and payers for a more holistic approach. Third, evaluations should be flexible to include new and evolving technologies. They should also take into consideration the maturity of the telemedicine program being evaluated. Ideally, evaluations would also be randomized and include a control group (i.e., one not using telemedicine). Future telemedicine evaluations should also recognize barriers, including those that are behavioral, organizational, technical, economical, and legal, that might affect outcomes.

Technology’s role

Patients and healthcare providers must be comfortable using telemedicine technology for it to be effective. The success of telemedicine visits is often more about changing clinician and patient expectations than about the technology itself. Training, education, and technical assistance become paramount. 

Evaluations must also consider whether telemedicine has strong leadership support and is closely linked with explicit strategic goals. From a technical perspective, it’s important to consider whether telemedicine technology can consistently and reliably review patient data to enable cost-effective care. Adequate third-party reimbursement is also an important consideration as is the ability to comply with multiple sets of state laws and ensure the security of electronic health information.

These barriers and challenges can be visualized using the fishbone diagram/tool for root-cause analysis. This tool summarizes potential causes into three main categories: Human (service provider and patients/clients), system (organization and technology), and environment (society and rules/regulations).

What evaluation strategies can hospitals, health systems, and telemedicine companies use?

Hospitals and health systems can collect critical data necessary for building a national framework for telemedicine evaluation. These organizations often have teams of individuals who monitor care quality and drive process improvement. These teams, which often have a big-picture view into enterprise wide operations, can spearhead efforts to assess telemedicine effectiveness. For example, they can use patient telemedicine satisfaction questionnaires as part of measuring the overall patient experience. They can also launch telemedicine surveys for doctors to gauge overall clinician experience as well.

Telemedicine companies have direct access to providers using telemedicine technology, and are thus well-positioned to collect data that can help shape a standardized framework for telemedicine evaluations. This is particularly true for telemedicine companies that saw a huge uptick in customer usage during the pandemic. For example, telemedicine companies can conduct studies, site visits, surveys, or moderate telephone discussions to assess views and collect data.

Evaluation data will determine telemedicine's value

Although the use of telemedicine expanded exponentially during the COVID-19 pandemic, there’s still controversy about its value and return on investment. Although there’s plenty of anecdotal information and a number of studies available to support the assertion that telehealth visits garner high patient satisfaction rates, there is no nationwide consistent approach to telemedicine evaluations. If a MAST or other type of evaluation were widely available, it might speed up the process for regulatory change and even more widespread adoption by clinicians, payers, and patients.

An evaluation framework is necessary to verify the effectiveness and stability of telemedicine and to inform policy going forward.

Learn more about automating telehealth provider data.