Beclometasone

Objective. To compare different indicators for assessing the quality of drug prescribing and establish their agreement in identifying doctors who may not adhere to treatment guidelines.

Data Sources/Study Setting. Data from 181 general practitioners (GPs) from The Netherlands. The case of asthma is used as an example because, in this area, different quality indicators exist whose validity is questioned. The study is part of the European Drug Education Project.

Study Design. Spearman rank correlations were assessed among the GPs' scores on self-report instruments, aggregated prescribing indicators, and individualized prescribing indicators. Kappa values were calculated as agreement measures for identifying low adherence to the guidelines.

Data Collection. Prescribing data from GPs were collected through pharmacies, public health insurance companies, or computerized GP databases. Two self-report instruments were mailed to the GPs. The GPs first received a questionnaire assessing their competence regarding the treatment of asthma patients. Three months later they received a series of 16 written asthma cases asking for their intended treatment for each case.

Principal Findings. Correlations between scores based on self-report instruments and indicators based on actual prescribing data were mostly nonsignificant and varied between 0 and 0.21. GPs identified as not adhering to the guidelines by the prescribing indicators often had high scores on the self-report instruments. Correlations between 0,20 and 0.55 were observed among indicators based on aggregated prescribing data and those based on individualized data. The agreement for identifying low adherence was small, with kappa values ranging from 0.19 to 0.30.

Conclusions. Indicators based on self-report instruments seem to overestimate guideline adherence. Indicators assessing prescribing quality at an aggregated level give clearly different results, as compared to indicators evaluating prescribing data on an individual patient level. Caution is needed when using such prescribing indicators to identify low adherence to guidelines. Further validation studies using a gold standard comparison are needed to define the best possible indicator.

Key Words. Asthma, drug utilization, quality indicators, physician practice patterns, guideline adherence

The need for cost-effective health care has stimulated intense interest in systematic quality assessment and efficient quality improvement in general practice (Grol et al. 1994; Palmer 1997). Many professional societies have developed and distributed guidelines to improve the quality of medical practice, but it has become clear that additional activities are needed to implement such recommendations (Grol 1992; Granados, Jonsson, Banta, et al. 1997). Targeting physicians likely to benefit most from interventions will improve the cost-effectiveness of such programs. Valid performance indicators are essential to identify physicians who do not provide optimal care and who might be in need of (educational) intervention strategies. Such indicators are also needed for the evaluation of interventions (Headrick, Crain, Evans, et al. 1996).

Good performance indicators should measure aspects of care controlled by the health care provider (Giuffrida, Gravelle, and Roland 1999). They should be based on the process (i.e., the care provided) and not on the outcome of care (e.g., health status of patients) (Brook, McGlynn, and Cleary 1996). Evidence-based guidelines are often used for defining explicit performance criteria (Fang, Mittman, and Weingarten 1996). Adherence to such guidelines is seen as good quality of care. Currently, many different performance indicators are in use, all with (theoretical) advantages and disadvantages. Little is known about the validity and agreement among these indicators. The objective of this study is to evaluate the concurrent validity of different quality indicators for drug prescribing. The case of asthma treatment is used as an example.

The most accurate measure of performance would be based on a thorough review of medical records giving information about the full clinical and treatment history of individual patients. Due to privacy regulations and lack of databases that include detailed, anonymous medical records, this is not a realistic option. Therefore, alternative methods are needed to assess the quality of drug prescribing.

On the one hand, self-report instruments can be used, asking physicians to record their prescribing for actual or written simulated cases or to complete competence or performance questionnaires. One advantage of such methods is that an accurate assessment can be made of the appropriateness of the answers using explicit criteria. Moreover, these instruments are easy to apply and can cover a wide range of subjects. The obvious disadvantage is that physicians may try to do their best, resulting in discrepancies between self-reported and actual performance (Jones, Gerrity, and Earp 1990; Rethans, van Leeuwen, Drop, et al. 1990; Adams et al. 1999). In the case of asthma, both competence questionnaires and self-recording of actual prescribing behavior have been used to assess prescribing quality (Smeele, Grol, van den Bosch, et al. 1996; Tomson et al. 1997).

Alternatively, available databases of actual performance data can be used. Automated databases containing information on the drugs prescribed or dispensed to patients are available in most developed countries. These databases lack information about the patients' clinical history but may include information on the age and gender of the patients and often include some kind of patient identification number. Most performance indicators based on actual prescribing data used so far do not incorporate any of this patient information (Bateman, Eccles, Campbell, et al. 1996; De Vries et al. 1999). Instead, prescribing is summarized per physician at an aggregated level. In the case of asthma, commonly used indicators are the ratio of prophylactic to bronchodilator drugs prescribed or the percentage of specific drugs prescribed (Naish, Sturdy, and Toon 1995; Feder, Griffiths, Highton, et al. 1995; Shelley et al. 1996; Griffiths, Sturdy, Naish, et al. 1997; Lang, Sherman, and Polansky 1997; Aveyard 1997). These aggregat ed measures are easy to develop but give an ambiguous assessment of prescribing performance. The extent to which individual patients are truly treated in agreement with the guideline recommendations is not clear (Bateman, Eccles, Campbell, et al. 1996).

Evaluating the treatment prescribed to individual patients would give a more precise measure (Hallas and Hansen 1993; Hallas and Nissen 1994). For an accurate assessment of prescribing for asthma, it is relevant to evaluate the dosing schedules of the drugs prescribed and whether bronchodilators and prophylactic drugs are combined for individual patients (Roberts 1997). Using the data available in most prescribing databases, it is possible, although more complicated, to conduct evaluations at an individual instead of an aggregated level. The European Drug Education Project group jointly developed indicators describing prescribing for asthma at the individual level (Veninga, Lagerlov, Wahlstrom, et al. 1999). These indicators describe guideline adherence for maintenance treatment at the patient level and for asthma exacerbations at the episode level.

None of the indicators--either based on self-report data, on aggregated prescribing data, or on individualized prescribing data--can be seen as the true measure of the quality of health care provided by a physician. They all have limitations and are all indicators of performance. It is not clear, however, to what extent they measure the same thing nor whether it makes a difference which indicator is used when trying to identify physicians who may not provide adequate health care. We have used the example of asthma treatment to examine the concurrence among the three types of performance indicators. In particular, we have evaluated the extent to which these indicators agree on which physicians adhere the least to guidelines and thus may be providing suboptimal care.

MATERIAL AND METHODS

Guideline Recommendations

The Dutch guidelines for general practice available at the time of the data collection were used to define the quality of prescribing performance (Bottema, Fabels, van Grunsven, et al. 1992; Van der Waart, Dekker, Nijhoff, et al. 1992). These guidelines emphasize a stepwise approach. They recommend initiating anti-inflammatory therapy if the patient uses bronchodilators continuously. If adequate control is not achieved (as indicated by a high daily use of bronchodilators), the dosage of inhaled corticosteroids should be increased. The guidelines explicitly state that the definitions of "continuous use" and "high daily use" are somewhat arbitrary because the literature is ambiguous on this matter. In the update of the guidelines in 1997, the recommendations are more specific: asthma patients who need more than one inhalation of a bronchodilator per day during two to four weeks should receive inhaled corticosteroids (Geijer, Van Hensbergen, Bottema, et al. 1997). In cases of severe exacerbations, oral corticost eroid courses are recommended, and the routine use of antibiotics is discouraged.

Indicators for Maintenance Treatment

Two indicators for maintenance treatment were developed using self-reports from general practitioners (GPs). The first was based on a questionnaire assessing the competence of the GPs regarding asthma treatment, including five questions about maintenance treatment (Table 1, competence-1). The proportion of the GPs' responses in agreement with the guidelines was expressed in this indicator. The second indicator was based on a series of seven written cases and expressed the proportion of cases for which the GP recommended a change in inhaled corticosteroid treatment in agreement with the guidelines (Table 1, written cases-1). Furthermore, two indicators based on aggregated prescribing data were included in the analysis. One was the commonly used ratio of prophylactic to bronchodilator drugs prescribed (Table 1, ratio). The other was the proportion of patients on asthma medication using inhaled corticosteroids (Table 1, inhaled corticosteroids). This latter indicator focuses on the patient level but disregards a ny information on dosage or how drugs are combined. Finally, two indicators were included describing the prescribing quality on an individual patient level. These two indicators each measure a different aspect of asthma treatment. One focuses on the continuous use of bronchodilators without inhaled corticosteroids (Table 1, continuous use). Thus, it addresses the matter of starting inhaled corticosteroids in time, the second step of asthma treatment. The other focuses on the use of inadequate levels of inhaled corticosteroids (Table 1, low corticosteroids). This deals with increasing the dosage of inhaled corticosteroids in time, the third step of asthma treatment. To express the level of drug use for these indicators, the number of defined daily dosages (DDD) (WHO Collaborating Centre for Drug Statistics Methodology 1996) prescribed was calculated. The defined daily dosage methodology standardizes drug quantities, thus allowing direct comparisons among amounts of drugs that differ in potencies, as is the cas e for different bronchodilators. Based on the treatment guidelines for asthma, "continuous use" of bronchodilators was defined as using, on average, more than 0.25 DDD per day during a six-month period and "high daily use" as more than 0.5 DDD per day. For the bronchodilators commonly prescribed in The Netherlands, salbutamol, terbutaline, and ipratropium, more than 0.25 DDD per day equals an average of more than one-half to two inhalations a day, and more than 0.5 DDD equals more than one to four inhalations a day. Based on the treatment guidelines, the dose of inhaled corticosteroids should be increased, not maintained at the lowest level, when adequate control is not achieved. For the commonly prescribed inhaled corticosteroids, beclometasone and budesonide, the lowest level advised is 400 [micro]g per day, which equals 0.5 DDD per day.

Indicators for Exacerbation Treatment

For exacerbation treatment, two indicators were developed using self-report instruments. The first one was based on the competence questionnaire, including six questions about treatment of exacerbations, and again expressed the proportion of responses in agreement with the guidelines (Table 1, competence-2). The second indicator concerned responses to nine written cases and expressed the proportion of oral corticosteroid courses recommended for cases where such treatment was advised according to the guidelines (Table 1, written cases-2). Finally, one indicator was included for assessing the quality of exacerbation treatment using actual prescribing data (Table 1, oral corticosteroids). For this indicator, exacerbations were defined as all episodes for which oral glucocorticosteroid courses or specified antibiotics (Table 2) were dispensed to patients who also receive anti-asthmatic drugs as defined in the Anatomical Therapeutic Chemical classification index (ATC code R03) (WHO Collaborating Centre for Drug St atistics Methodology 1996). As most asthma exacerbations in The Netherlands are treated with either an oral glucocorticosteroid or a course of antibiotics, prescriptions for both are needed as markers for asthma exacerbations (Smeele, van Schayck, van den Bosch, et al. 1998; Gerrits et al. 1999).

Sample and Data Collection

A total of 181 GPs were included in the study. This was the total number of GPs in 24 Dutch local counseling groups participating in the European Drug Education Project. Within the framework of this project, prescribing data were collected, and all GPs received a competence questionnaire by mail in 1995. Three months later, written simulated cases regarding the treatment of asthma were sent to the GPs of 12 randomly selected groups (90 GPs). As part of the Project, the other 12 groups received written cases on another subject, the treatment of urinary tract infections. The average age of the 181 GPs was 46.7 years, and their average practice size was 2,492 patients. These figures are comparable to the national averages in 1995, the national average age being 45.1 years and average practice size being 2,274 patients.

Prescribing data were available for 90.6 percent of the GPs. The response rate after two reminders was 87.8 percent for the written simulated cases and 87.3 percent for the competence questionnaire.

Data on drugs dispensed (Table 2) between June 1995 and November 1995 to patients who received at least one anti-asthmatic drug (ATC code R03) in this period were collected from pharmacies, from public health insurance companies, or directly from computerized databases of dispensing GPs. These automated databases are commonly used for these purposes in The Netherlands. They contain information on all reimbursed drugs dispensed to patients. All drugs included in the analyses are reimbursed. Pharmacy databases contain information on drugs dispensed to both privately and publicly insured patients. In The Netherlands, patients usually register with one pharmacy. In some areas in The Netherlands, no pharmacy is present and GPs dispense drugs themselves. In these cases, automated databases of the GPs containing similar information on dispensed drugs were used. For five of the 24 GP groups, the patient population of the pharmacy database did not satisfactorily overlap with that of the GPs. In these cases, databases of public health insurance companies were used, which combine data from individual pharmacies regarding drugs dispensed to publicly insured patients.

Data were restricted to people aged 18 to 49 to exclude children and minimize inclusion of chronic obstructive pulmonary disease (COPD) patients. For all prescriptions, the number of DDDs dispensed was available. Antibiotic and oral glucocorticosteroid prescriptions for chronic treatment were excluded (according to dosage schedule [greater than] 14 days). The final data material consisted of 18,177 prescriptions dispensed to 4,975 patients.

Validity of Inclusion Criteria

The prescribing data did not contain information on indication. The inclusion criteria were validated in a pilot study using an automated database of 16 GPs containing information on indications and prescribed drugs between July 1994 and December 1994. The indications were coded by the GPs according to the International Classification of Primary Care (ICPC) (Lamberts and Wood 1987). In this pilot study, all patients aged 18--49 years using one or more anti-asthmatic drugs (ATC code R03) were included. Of these patients, 66 percent had diagnosed asthma according to the ICPC coding of the GPs. The other patients had diagnoses such as dyspnoea (about 5 percent of the patients), acute bronchitis (7 percent), COPD (5 percent), and hayfever (4 percent), or no diagnosis was recorded (4 percent). If considering only patients using inhaled bronchodilators, 69 percent had diagnosed asthma, and when the inclusion of patients was restricted to using inhaled corticosteroids, 79 percent had diagnosed asthma. Of all patient s using a combination of inhaled bronchodilators and inhaled corticosteroids, 85 percent had diagnosed asthma. Oral corticosteroid or specified antibiotic courses (Table 2) prescribed to patients aged 18-49 years using at least one anti-asthmatic drug were used as markers of asthma exacerbations. The pilot study showed that 76 percent of the prescriptions of these drugs were for exacerbations, according to the ICPC coding of the GPs. In approximately 12 percent of the cases, asthma patients received oral corticosteroids for other respiratory or upper airways infections (not classified as asthma exacerbation), and in approximately 40 percent of the cases, they received one of the specified antibiotics for sinusitis, nonrespiratory infections, or other respiratory infections (not classified as asthma exacerbation).

Analysis

To assess the overall concurrence among the different types of indicators, Spearman rank correlations were calculated. Cohen's kappa was calculated to evaluate the extent to which the indicators agreed on physicians who seem to adhere the least to the guidelines. For the self-report instruments, explicit criteria were used to assess whether the responses were in agreement with the guidelines. GPs answering more than one-third of the questions or cases not in agreement with the guideline recommendations were considered to have low adherence. For the indicators based on actual prescribing data, there are no explicit values for assessing agreement or nonagreement with the guidelines. Therefore, low adherence to the guidelines was defined in two ways. First, indicator scores below 1 s.d. from the mean value were defined as indicating low adherence. Second, a fixed percentage of GPs having the lowest scores on an indicator were defined as having a low adherence. An arbitrary cutoff point of 25 percent was used. An alyses were conducted for both definitions of low adherence.

RESULTS

The mean scores on both self-report instruments concerning maintenance treatment were above 80 percent agreement with the guidelines (competence-1 and written cases-1, Table 3). Many GPs had maximum scores, but 14.3 percent had a competence score that was considered suboptimal, and 20.5 percent had a suboptimal score on the written cases. Analysis of prescribing data on an aggregated level showed a mean value of the ratio of prophylactic to bronchodilator drugs of 0.68 (ratio, Table 3). Of all patients using anti-asthmatic drugs, 58 percent were being prescribed inhaled corticosteroids (inhaled corticosteroids, Table 3). On average, 28 percent of the patients using inhaled bronchodilators received more than 0.25 DDD per day without additional anti-inflammatory treatment (continuous use, Table 3). Of the patients using inhaled bronchodilators in combination with inhaled corticosteroids, 17 percent used an inadequate inhaled corticosteroids level (low corticosteroids, Table 3). Regarding exacerbation treatment, 79 percent of the competence answers and 88 percent of the responses to the written simulated cases were in agreement with the guidelines. Suboptimal scores on these two self-report instruments were observed for 13.4 and 11.4 percent of the GPs, respectively. Looking at the actual prescribing data, however, only 30 percent of the exacerbations appeared to be treated with oral corticosteroid courses (oral corticosteroids, Table 3).

Regarding asthma maintenance treatment, the competence scores of the GPs correlated only slightly with their scores on the prescribing data indicators (Table 4). The highest correlation (0.211) was found between the competence-1 scores and the ratio of prophylactic to bronchodilator drugs. No significant correlations were seen between the responses to written cases and any of the other indicators. Among the indicators based on aggregated prescribing data versus indicators based on individualized prescribing data, some significant correlations were found. The ratio correlated significantly with the other prescribing indicators, but the strength of these correlations never reached a moderate level (Table 4). The inhaled corticosteroids indicator did correlate moderately with one of the individualized indicators, but not at all with the other (Table 4). Finally, there was no correlation between the two indicators based on individualized prescribing data. Indicators for assessing the treatment of asthma exacerba tions also showed only slight correlations between the self-report instruments and actual prescribing (Table 4).

Looking at the identification of GPs who may not adhere to the guide-lines, there was again little agreement between the indicators based on self-report data and on prescribing data (Table 5). At best, of 33 GPs classified by either one of the two indicators as showing low adherence, only six GPs were classified by both, resulting in kappa values from less than 0 to 0.29. Agreement in the identification of low adherence was somewhat higher when comparing indicators based on aggregated data with those based on individualized prescribing data. Using a deviance of 1 s.d. as the cutoff point, kappa values between 0.20 and 0.30 were observed, identifying with two different indicators up to ten out of 30 GPs having low adherence (Table 5). Quite similar values of agreement were found when using the 25 percent lowest scoring GPs as the cutoff point (Table 6), identifying up to 24 out of 41 GPs with low adherence. Remarkable were the agreements between the identification of low adherence using the inkaled corticoste roids and the low corticosteroids indicators and using the written cases-1 and continuous use indicators (Tables 5 and 6), as these pairs of indicators did not show a significant correlation along their entire scales (Table 4).

DISCUSSION

Self-report instruments showed a high adherence of the GPs to the guidelines for both the asthma maintenance and the exacerbation treatment. Looking at the scores on indicators based on actual prescribing data, however, it seems that asthma maintenance treatment was more in agreement with the guideline recommendations than treatment of asthma exacerbations. More than 70 percent of the patients using bronchodilators use these drugs either in small amounts or in combination with inhaled corticosteroids. On the other hand, only 30 percent of the exacerbations are treated with corticosteroids. These findings are in line with previous research in The Netherlands. Smeele and colleagues observed that about three-quarters of the patients using bronchodilators more than twice daily also used anti-inflammatory treatment in agreement with the guidelines, whereas only 24 percent of the asthma exacerbations were treated with oral corticosteroid courses (Smeele, Grol, van den Bosch, et al. 1996; Smeele, van Schayck, van de n Bosch, et al. 1998). Qualitative interviews with GPs in The Netherlands also showed discrepancies between the GPs' views on exacerbation treatment and the recommendations in the guidelines, whereas their views on maintenance treatment were fully in line with the guideline recommendations (Veninga et al. 1998).

In general, correlations between scores on self-report instruments and indicators based on prescribing data were low and often nonsignificant. GPs identified as not adhering to the guidelines by their prescribing indicators often did have high scores on the self-report instruments. These results demonstrate once more the gap between knowing what should be done (competence) and actually doing it (performance) (Rethans, van Leeuwen, Drop, et al. 1990). The value of indicators based on written simulated cases to assess treatment quality is affected by the discrepancy between intended and real behavior (Jones, Gerrity, and Earp 1990). When assessing adherence to guidelines, self-report instruments usually result in overestimation (Jones, Gerrity, and Earp 1990; McPhee and Bird 1990; Adams et al. 1999).

Several significant, but moderate, correlations existed between the indicators based on aggregated prescribing data and those based on individualized prescribing data. When used to identify physicians who may not adhere to the guidelines, the agreement between both types of indicators was small. Most physicians identified as having low adherence to the guidelines by one indicator were not identified as such by the other indicator. In other words, the indicators based on aggregated prescribing data measure an aspect of performance that does not resemble prescribing performance as assessed at the individual patient level. Summarizing data of all patients without taking into account whether bronchodilators and inhaled corticosteroids are combined or the amounts of drugs prescribed may limit the accuracy of these indicators. The indicators based on individualized prescribing data do take the quantities and combination of drugs prescribed to individual patients into account, resulting in the theoretical advantage of having a higher content validity. These indicators were based on asthma guidelines, which emphasize a stepwise approach guided by the severity of the disease. Although the severity of asthma cannot be directly measured without detailed clinical data, prescribing data allow judgment as to whether a treatment is in accordance with a given step in the guidelines (Hallas and Hansen 1993; Gaist et al. 1996). The treatment of a patient using (almost) daily bronchodilators without anti-inflammatory treatment is not in agreement with the guidelines (continuous use indicator). In this study, continuous use was defined as use of 0.25 DDD per day, on average, during a six-month period. For the most commonly used bronchodilator, salbutamol, this amount equals two or more multidose aerosol packages, or 180 capsules of 200 [micro]g, or 90 capsules of 400 [micro]g. These amounts largely exceed the amounts that can be prescribed for occasional use, for instance, for treating persisting cough or hayfever. In The Netherland s, prescriptions for such use are only allowed for a maximum of two weeks.

When bronchodilator use is needed several times daily, the guidelines recommend increasing the inhaled corticosteroids. This is expressed by the low corticosteroids indicator, which gives the proportion of patients treated with insufficient levels of inhaled corticosteroids (100-400 [micro]g/day) as indicated by a relatively high use of bronchodilators (on average, more than 0.5 DDD per day, equaling more than 1-4 inhalations per day depending on the strength of the drug). The definition of these threshold values may be questioned, as the guidelines state that there is no clear evidence at precisely which level of bronchodilator use inhaled corticosteroids should be started or increased. Further validation studies are needed in which a detailed review of the clinical and treatment history of patients is used as the gold standard to determine which thresholds are most accurate in identifying suboptimal prescribing performance.

The two indicators at the individual patient level focus on different steps of asthma treatment. One looks at the second step of asthma treatment (continuous use) and the other at the third step (low corticosteroids). The low correlation and agreement between these two indicators shows that they indeed measure different aspects of performance. Apparently, the GPs who are reluctant to start prescribing inhaled corticosteroids (as expressed in the continuous use indicator) are not the same GPs who are treating the more severe asthma patients with too low levels of corticosteroids (as expressed in the low corticosteroids indicator).

Guidelines recommend treating asthma exacerbations with oral corticosteroid courses or, in mild cases, by increasing the dosage of inhaled corticosteroids. The guidelines recommend against the routine use of antibiotics. The oral corticosteroids indicator was limited to orally treated exacerbations. The number of orally treated exacerbations was estimated using the total number of prescriptions for antibiotics and oral corticosteroid courses because these are the drugs prescribed for exacerbations in The Netherlands (Smeele, Grol, van den Bosch, et al. 1996). Using both the oral corticosteroids and the specified antibiotics as markers for an exacerbation may lead to overestimating the number of exacerbations. It is estimated that in approximately 15 percent of the asthma exacerbations in The Netherlands, GPs prescribe both an oral corticosteroid and an antibiotic (Smeele, van Schayck, van den Bosch, et al. 1998). Such combinations will result in an overestimation of the denominator by counting one exacerbati on twice, and thus results in a lower overall estimate of this indicator. Consequently, a GP who always combines both drugs for asthma exacerbations will have the same score on this indicator as a GP who prescribes oral corticosteroids only in half of the cases. In both situations, the GPs are only partly adhering to the guidelines, and having a lower score on this indicator is therefore justified.

It is important to keep in mind that it will never be possible to develop an error-free measure of quality of care (Brook, McGlynn, and Cleary 1996). Several confounding factors appear when using prescribing databases. First, there is the inclusion of prescriptions for nonasthmatics when using databases that have no information on the indications. It is possible to reduce this inclusion by using additional selection criteria regarding age and drug use of the patient. Our pilot study showed that up to 34 percent of nonasthmatics might have been included in the denominators of the various prescribing indicators for maintenance treatment. The number of nonasthmatics in the numerators of these indicators is expected to be somewhat lower (between 15 and 31 percent). This implies that all prescribing indicators, either using aggregated or individualized data, may slightly overestimate low adherence due to the unequal inclusion of nonasthmatic patients in their numerator and denominator. For the exacerbation treatm ent indicator, only short courses for corticosteroids and antibiotics were included. In addition, only those antibiotics were included that are typically used for respiratory infections and asthma exacerbations so as to minimize the inclusion of prescriptions for infections unrelated to asthma. The pilot study showed that 24 percent of the prescriptions included in the denominator may not have been intended for the treatment of an asthma exacerbation. Again, the number of prescriptions for other indications is expected to be lower in the numerator, resulting in a possible overestimation of low adherence using the exacerbation treatment indicator. In the most extreme situation, when all oral corticosteroids were indeed prescribed for an asthma exacerbation but half of the antibiotic prescriptions were intended for other infections, the average score on this indicator would increase from 0.30 to 0.40.

For the indicators based on individualized prescribing data, a second confounding factor is that the use of drugs is estimated from the total number of dosages delivered to the patient during the study period. If only one prescription is dispensed during the study period intended to be used "when needed," it is difficult to make an accurate estimation of its use. In The Netherlands, prescriptions are of limited duration, with a maximum of three months. Therefore, the six-month study period will provide good estimates of the actual use. For other countries, a six-month period could be insufficient.

Finally, the indicators are based on prescriptions collected by the patients, thus representing an amalgam of physician and patient behavior. It is conceivable that selective redemption of medication by patients may conceal greater accordance to the guidelines among physicians than our study results indicate (Nilsson, Johansson, and Wennberg 1995).

In summary, indicators based on self-report instruments seem to be affected by the discrepancy between competence and performance and therefore do not provide good measures of performance quality. Furthermore, this study shows that indicators based on aggregated prescribing data and on individualized prescribing data measure different aspects of prescribing performance and differ considerably in their identification of physicians adhering the least to asthma guidelines. On theoretical grounds, indicators evaluating prescribing data on the individual patient level are preferred to indicators assessing prescribing quality on an aggregated level. Further research to test the validity of the prescribing indicators will be necessary to confirm this theoretical preference.

ACKNOWLEDGMENTS

The authors thank the other core members of the European Drug Education Project for their input: V. Diwan, G. Tomson, R. Wahlstrom T. Oke, and C. Stalsby Lundborg(Sweden); M. Andrew, M. Loeb, P. Lagerlov, and the late I. Matheson (Norway); M. M. Kochen, and E. Hummers-Pradier (Germany); and M. Muskova and Z. Kopernicka (Slovak Republic). For her contribution to this manuscript, the authors thank D. Postma. C. C. M. Veninga, M.Sc.Pharm., Ph.D.; P. Denig, M.Sc.Pharm,, Ph.D., Assistant Professor of Drug Utilization Studies; L. G. Pont, B.Sc., B.Pharm.; and F. M. Haaijer-Ruskamp, M.Sc., Ph.D., Professor of Drug Utilization Studies are from Northern Centre for Healthcare Research, Department of Clinical Pharmacology, University of Groningen, The Netherlands. Address correspondence to P. Denig, Ph.D., Assistant Professor of Drug Utilization Studies, Northern Centre for Healthcare Research, Department of Clinical Pharmacology, University of Groningen, Antonius Deusinglaan 1,9713 AV Groningen, The Netherlands. This a rticle, submitted to Health Services Research on April 14, 1999, was revised and accepted for publication on January 24, 2000.

REFERENCES

Adams, A. S., S. B. Soumerai, J. Lomas, and D. Ross-Degnan. 1999. "Evidence of Self-Report Bias in Assessing Adherence to Guidelines." International Journal for Quality in Health Care 11 (3): 187-92.

Aveyard, P. 1997. "Assessing the Performance of General Practices Caring for Patients with Asthma." British Journal of General Practice 47 (420): 423-26.

Bateman, D. N., M. Eccles, M. Campbell, J. Soutter, S.J. Roberts, and J. M. Smith. 1996. "Setting Standards of Prescribing Performance in Primary Care: Use of a Consensus Group of General Practitioners and Application of Standards to Practices in the North of England." British Journal of General Practice 46 (402): 20-25.

Bottema, B. J. A. M., E. J. Fabels, P. M. van Grunsven, W. van Henbergen, J. W. M. Muris, C. P. van Schayck, F. Schreuder, R. Starmans, and C. F. H. Rosmalen. 1992. "NHG-Standaard Cara bij Voiwassenen: Diagnostiek" [in Dutch]. Huisarts en Wetenschap 35 (9): 430-36.

Brook, R. H., E. A. McGlynn, and P. D. Cleary. 1996. "Quality of Health Care. Part 2: Measuring Quality of Care." New England Journal of Medicine 335 (13): 966-70.

De Vries, C. S., Th. F. J. Tromp, W. Blijleven, and L. T. W. Dejong-van den Berg. 1999. "Prescription Data as a Tool in Pharmacotherapy Audit. (I) General Considerations." Pharmacy World and Science 21 (2): 80-84.

Fang, E., B. S. Mittman, and S. Weingarten. 1996. "Use of Clinical Practice Guidelines in Managed Care Physician Groups." Archives of Family Medicine 5 (October): 528-31.

Feder, G., C. Griffiths, C. Highton, S. Eldridge, M. Spence, and L. Southgate. 1995. "Do Clinical Guidelines Introduced with Practice Based Education Improve Care of Asthmatic and Diabetic Patients? A Randomised Controlled Trial in General Practices in East London." British Medical Journal 311 (7018): 1473-78.

Gaist, D.,J. Hallas, N. C. Hansen, and L. F. Gram. 1996. "Are Young Adults with Asthma Treated Sufficiently with Inhaled Steroids? A Population-based Study of Prescription Data from 1991 and 1994." British Journal of Clinical Pharmacology 41 (4): 285-89.

Geijer, R. M. M., W Van Hensbergen, B. M. A. M. Bottema, C. P. Van Schayck, A. P. E. Sachs, I. J. M. Smeele, H. A. Thiadens, C. Van Weel, A. A. C. Van der Zwan, and C. F. H. Rosmalen. 1997. "NHG-Standaard Astma bij Volwassenen: Behandeling" [in Dutch]. Huisarts en Wetenschap 40 (9): 443-54.

Gerrits, C. M. J. M., R. M. C. Herings, H. G. M. Leufkens, and J.-W.J. Lammers. 1999. "Asthma Exacerbations During First Therapy with Long Acting Beta-2-Agonists." Pharmacy World and Science 21 (3): 116-19.

Giuffrida, A., H. Gravelle, and M. Roland. 1999. "Measuring Quality of Care with Routine Data: Avoiding Confusion Between Performance Indicators and Health Outcomes." British Medical Journal 319 (7202): 94-98.

Granados, A., E. Jonsson, H. D. Banta, L. Bero, A. Bonair, C. Cochet, N. Freemantle, R. Grilli, J. Grimshaw, E. Harvey, R. Levi, D. Marshall, A. Oxman, L. Pasart, V. Raisanen, E. Rius, and J. A. Espinas. 1997. "EUR-ASSESS Project Subgroup Report on Dissemination and Impact." International Journal of Technology Assessment in Health Care 13 (2): 220-86.

Griffiths, C., P. Sturdy, J. Naish, R. Omar, S. Dolan, and G. Feder. 1997. "Hospital Admissions for Asthma in East London: Associations with Characteristics of Local General Practices, Prescribing, and Population." British Medical Journal 314 (7079): 482-86.

Grol, R. 1992. "Implementing Guidelines in General Practice Care." Quality in Health Care 1 (3): 184-91.

Grol, R., R. Baker, M. Wensing, and A. Jacobs. 1994. "Quality Assurance in General Practice: The State of the Art in Europe." Family Practice 11(4): 460-67.

Hallas, J., and N. C. Hansen. 1993. "Individual Utilization of Anti-Asthma Medication by Young Adults: A Prescription Database Analysis." Journal of Internal Medicine 234 (1): 65-70.

Hallas, J., and A. Nissen. 1994. "Individualized Drug Utilization Statistics. Analysing a Population's Drug Use from the Perspective of Individual Users." European Journal of Clinical Pharmacology 47 (4): 367-72.

Headrick, L., E. Crain, D. Evans, M. N. Jackson, B. H. Layman, R. M. Bogin, M. Young, and K. B. Weiss. 1996. "National Asthma Education and Prevention Program Working Group Report on the Quality of Asthma Care." American Journal of Respiratory and Critical Care Medicine 154 (Suppl.): S96-S118.

Jones, T. V., M. S. Gerrity, and J. Earp. 1990. "Written Case Simulations: Do They Predict Physicians' Behavior?" Journal of Clinical Epidemiology 43 (8): 805-15.

Lamberts, H., and M. Wood. 1987. ICPC--International Classification of Primary Care. Oxford, U.K.: Oxford University Press.

Lang, D. M., M. S. Sherman, and M. Polansky. 1997. "Guidelines and Realities of Asthma Management. The Philadelphia Story." Archives of Internal Medicine 157

McPhee, S. J., and J. A. Bird. 1990. "Implementation of Cancer Prevention Guidelines in Clinical Practice." Journal of General Internal Medicine 5 (5 Suppl): S116-S122.

Naish, J., P. Sturdy, and P. Toon. 1995. "Appropriate Prescribing in Asthma and its Related Cost in East London." British Medical Journal 310 (6972): 97-100.

Nilsson, J. L. G., H. Johansson, and M. Wennberg. 1995. "Large Differences Between Prescribed and Dispensed Medicines Could Indicate Undertreatment." Drug Information Journal 29 (4): 1243-46.

Palmer, R. H. 1997. "Process-based Measures of Quality: The Need for Detailed Clinical Data in Large Health Care Databases." Annals of Internal Medicine 127 (8, Part 2): 733-38.

Rethans, J. J., Y. van Leeuwen, R. Drop, C. van der Vleuten, and F. Sturmans. 1990. "Competence and Performance: Two Different Concepts in the Assessment of Quality of Medical Care." Family Practice 7 (3): 168-74.

Roberts, S. J. 1997. "Ratio of Inhaled Corticosteroid to Bronchodilator as Indicator of Quality of Asthma Prescribing: Authors Discuss a Result that Was Not Shown" [Letter]. British Medical Journal 314 (7081): 680-81.

Shelley, M., P. Croft, S. Chapman, and C. Pantin. 1996. "Is the Ratio of Inhaled Corticosteroid to Bronchodilator a Good Indicator of the Quality of Asthma Prescribing? Cross Sectional Study Linking Prescribing Data to Data on Admissions." British Medical Journal 313 (7065): 1124-26.

Smeele, I. J. M., R. P. T. M. Grol, W. van den Bosch, C. P. van Schayck, A. F. M. Haverkort, and J. Molema. 1996. "Knelpunten in de Zorg voor Volwassen Carapatienten" [in Dutch]. Huisarts en Wetenschap 39 (12): 548-53.

Smeele, I. J. M., C. P. van Schayck, W. J. H. M. van den Bosch, H. J. M. van den Hoogen, J. W. M. Muris, and R. P. T. M. Grol. 1998. "Discrepanties tussen de Richtlijnen en het Handelen van Huisartsen bij Volwassenen met een Exacerbatie van Cara" [in Dutch]. Nederlands Tijdschrift voor Geneeskunde 142 (42): 2304-08.

Tomson, Y, J. Hasselstrom, G. Tomson, and H. Aberg. 1997. "Asthma Education for Swedish Primary Care Physicians--A Study on the Effects of 'Academic Detailing' on Practice and Patient Knowledge." European Journal of Clinical Pharmacology 53 (3-4): 191-96.

Van der Waart, M. A. C., F. W Dekker, S. Nijhoff, H. A. Thiadens, C. van Weel, M. Helder, R. Starmans, and C. F. H. Rosmalen. 1992. "NHG-Standaard Cara bij Volwassenen: Behandeling" [in Dutch]. Huisarts en Wetenschap 35(9): 437-43.

Veninga, C. C. M., P. Denig, J. W. Heyink, and F. M. Haaijer-Ruskamp. 1998. "Denkbeelden van Huisartsen over de Behandeling van Astma" [in Dutch]. Huisarts en wetenschap 41(5): 236-40.

Veninga, C. C. M., P. Lagerlov, R. Wahlstrom, M. Muskova, P. Denig, H. Berkhof, F. M. Haaijer-Ruskamp, and the European Drug Education Project Group. 1999. "Evaluating an Educational Intervention to Improve the Treatment of Asthma in Four European Countries." American Journal of Respiratory and Critical Care Medicine 160 (4): 1254-62.

World Health Organization (WHO) Collaborating Centre for Drug Statistics Methodology. 1996. Anatomical Therapeutic Chemical (ATC) Classification Index. Oslo, Norway: WHO Collaborating Centre.

Indicators for Maintenance and Exacerbation Treatment

Self-Report Instruments

Competence-1: Proportion of the GP's responses, in agreement with the guidelines, to five questions about maintenance treatment

Written cases-1: Proportion of seven maintenance cases for which the GP increased the level of inhaled corticosteroids in agreement with the guidelines

Competence-2: Proportion of the GP's responses, in agreement with the guidelines, to six questions about exacerbation treatment

Written cases-2: Proportion of nine exacerbation cases for which the GP prescribed oral corticosteroids in agreement with the guidelines

Aggregated Prescribing Data

Ratio: Ratio of inhaled corticosteroids (prophylactic) to bronchodilator drugs ([[beta].sub.2] agonists and anticholinergics) dispensed to the GP's patients (in DDDs)

Inhaled corticosteroids: Proportion of the GP's patients using inhaled corticosteroids of all patients prescribed anti-asthmatic drugs

Individualized Prescribing Data

Continuous use: Proportion of the GP's patients using on average per day [greater than] 0.25 DDD inhaled bronchodilators ([[beta].sub.2] agonists and anticholinergics) without anti-inflammatory treatment (inhaled corticosteroids and cromoglycates) of all the GP's patients receiving inhaled bronchodilators

Low corticosteroids: Proportion of the GP's patients using on average per day [greater than] 0.5 DDD inhaled bronchodilators and [less than or equal to]0.5 DDD inhaled corticosteroids of all the GP's patients receiving a combination of inhaled bronchodilators and corticosteroids

Oral corticosteroids: Proportion of oral corticosteroid courses of all antibiotic and oral corticosteroid courses dispensed to the GP's patients receiving anti-asthmatic drugs

COPYRIGHT 2001 American College of Healthcare Executives
COPYRIGHT 2001 Gale Group