Types and levels of measurement, survey methodology

Types and levels of measurement, survey methodology summarize, evaluate, and critique the research hypotheses, literature review, methodology, results discussion, and findings. Need introduction of approximately 150 words from the attached and below : Use the information below: Proceedings Methodologies for data collection Sheri Happel Lewis† and Richard Wojcik† • • † Equal contributors Author Affiliations The Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA The electronic version of this article is the complete one and can be found online at:http://www(dot)biomedcentral(dot)com/1753-6561/2/S3/S5 Published: 14 November 2008 © 2008 Lewis and Wojcik; licensee BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons(dot)org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Background Electronic disease surveillance systems can be extremely valuable tools; however, a critical step in system implementation is collecting data. Without accurate and complete data, statistical anomalies that are detected hold little meaning. Many people who have established successful surveillance systems acknowledge the initial data collection process to be one of the most challenging aspects of system implementation. Methods This discussion will describe the various methods for collecting data as well as describe some of the more common data feeds used in surveillance systems today. Given that every city/region/country looking to establish a surveillance capability has varying degrees of automated data, alternative data collection methods must be considered. Results While it would be ideal to collect automated electronic data in a real-time fashion without human intervention, data may also be effectively collected via telephone (both mobile and land lines), fax, and email. Another consideration is what type of data will be used in a surveillance system. If one data source is of high value to one locality, it should not be assumed that it will be as useful in another area. Determining what data sources work best for a particular area is a critical step in system implementation. Conclusion Regardless of data type and how they are collected, surveillance systems can be successful if the implementers and end users understand the limitations of both the data and the collection methodology and incorporate that knowledge into their interpretation procedures. Background Data are the cornerstone of any electronic disease surveillance system. For the purposes of this discussion, data are defined as any information that would be of value in a disease surveillance system. Data comes in a variety of formats, from raw text line listings of patient encounters up through information gathered via analysis or end user interpretation (Figure 1) [1]. Privacy regulations in a given situation may determine the level of data that are to be shared within or between systems. For example, in a local health department in the United States, data may be collected at the individual hospital level, transmitted to the health department and then reviewed on a "per patient" basis for the purposes of assessing the health of the community. However, since the regional or state health department may not have the same perspective on the particular nuances of a community, they might only have a need to review aggregated counts or an assessment/interpretation of the data. Similarly, at the national level, again, lacking the local knowledge of recent events, they may only benefit from the analytical results or epidemiologist's interpretations of the data. Figure 1. Pyramid of data formats. There is a distinction that can and should be made between indicator surveillance and event-based surveillance. Indicator surveillance, which is the basis for this paper, refers more to syndromic surveillance such as Electronic Surveillance System for Early Notification of Community-based Epidemics (ESSENCE), while event-based surveillance, such as the World Health Organization Early Warning and Response Network (WHO EWARN) relies more on the capture of information about events that pose a potential health risk to a particular population. Data for event-based systems can be both formal and informal with examples of formal being routine reporting systems and informal being news media and rumors [2]. Additionally, not all data sources are equal for surveillance purposes. One must be very careful to evaluate each data source thoroughly before deciding to include it in an electronic surveillance system. For example, if the data are collected and transmitted in a timely fashion but really provide little value to the end user in assessing the health of the community, it may be better not to include that data source, so as not to deplete precious resources for data acquisition, analysis, and interpretation. And although a data source can be used for a system in one locale does not mean that it can be used in a system in another locale. Methods Data source consideration When assessing the value of data sources, the following considerations should be made: • Availability - Consider if data are already being collected for another purpose. If so, will the system developers be able to access it? • Privacy regulations - Research any privacy laws that may be applicable in the area of interest. • Early indicator - Consider if a particular source has the potential to provide early notification of a problem if early detection is a system requirement. • Coverage - Determine what population is covered by a particular data source. • Timeliness - Understand the frequency with which data are collected and transmitted. • Digitalization - Establish how the data are being collected and transmitted. • Automaticity - Verify if the data are sent to a server automatically or if cued by a person. • Reliability - Ascertain how many times the data source "drops" or is unavailable over the course of several months. • Centralization - Determine if the data are being collected from one central point or from multiple sources. • Cost - Understand the start-up or recurring costs associated with the data. Characteristics of a good data source What makes a good data source for one type of system may not hold true for other similar systems operating in different environments. For example, if the community under surveillance is a rural area with limited access to large chain stores, spending large amounts of time and money to acquire over-the-counter pharmaceutical data will be difficult. Unlike an urban area, which might be able to acquire data from chain stores that have multiple outlets in an area, rural areas will have independently owned and operated stores that likely do not have an automated inventory system. Therefore, the time and effort needed to acquire the data will likely outweigh the benefit of the data since it is limited in its scope. Identifying potential data sources When contemplating the acquisition and addition of particular data for inclusion in an electronic disease surveillance system, the developers and implementers should consider the WHO, WHAT, WHEN and HOW of each data source. Who The WHO of an electronic disease surveillance system will vary depending on the requirements of each system. For example, if the system is intended to pick up diseases in remote parts of a country where people have limited access to hospitals, more effort should be placed on remote data collection utilizing a laptop, personal digital assistant (PDA), integrated voice response (IVR), etc. Many of the considerations under this particular category pertain to demographic coverage. • Will the data source cover human health, human behavior, animal health, plant health, etc.? • Will the data source include urban or rural populations? • What age groups will make up the majority of the data? • What are the primary occupations of the individuals covered by this data source? • Is the covered population subject to privacy regulations? What It is critical to assess what is trying to be gained from a particular data source and how the data will provide that information for the community in question. Questions to consider include: • Will the data source provide traditional or non-traditional indictors? • Is the data source an early indicator of disease? • Is the data already being collected for another purpose? When The frequency of data transmission will vary between data sources and systems. Similarly, the frequency needs will vary by system. While one system may require that data be collected and transmitted in real-time, other systems may only need data on a weekly basis. Again, the implementers and developers need to set forth the requirements early in the system development process and clearly identify their surveillance goals. How In many locales data can be transmitted electronically via a computer; however, it is likely that in more remote environments data collection may be done via a telephone, fax machine, or the like. There is a great amount of work/research going on in this field and it is likely that in the near future more robust options for remote data collection will likely be available. Data in use today There are currently many data sources that are being utilized by electronic disease surveillance systems in both countries with a high level of internet connectivity as well as those settings with very limited or no internet connectivity. In countries with a high level of internet connectivity, the focus is on the use of pre-diagnostic data for the purposes of early detection of a disease outbreak. To aid in this detection, data sources frequently used include physician office visit data, ambulance 911 calls (a.k.a. EMS), hospital emergency department visits, hospital admissions, school absentee data, pharmacy sales, nurse-hotline data, and laboratory test requests [3]. Table1 provides examples of some data sources commonly considered to be traditional and non-traditional [4]. Important properties of these pre-diagnostic data include sensitivity, specificity, latency, and completeness. Table 1. Traditional and non-traditional data sources. Achieving sustainable data collection One of the most challenging aspects of electronic surveillance systems is sustaining data acquisition. While much time and energy goes into the physical data acquisition during the implementation stages, it is equally important to consider issues surrounding sustainability in the long-term. This is a consideration that must be well thought out in the planning stages as it has been the experience of many public health entities in the United States that once a data feed has been established it is more difficult to get people to dedicate time and resources into modifying either the data elements collected, the type of transmission, or the mode of transmission [4]. If data are not currently being collected at the desired level of detail or frequency, the developers need to consider implementing electronic data collection. It is important to get the data collectors in the habit of data entry early in the process because once the task of data entry is embedded in the daily routine, sustainability is more attainable. Likewise, it is recommended that as many variables as possible be collected at the initiation of the project. It is always better to have more information than is actually needed then to try to go back and add additional pieces later. As mentioned previously, it can be challenging to have people allocate time and resources on modifying a data feed that is functioning properly. Data transmission methods play a large role in the success of a system. Data are only reliable as the method of data transfer. If the mode of transmission is not reliable, consider building in redundancy until the preferred method is improved upon. For example, if the goal is to transmit data electronically via the internet but the internet connection is not as robust as desired, consider implementing the internet transfer but also ask for data via flash drives until the internet connectivity improves in future years. Additional challenges Once data are collected, there are inherent challenges that must be addressed. While data cleansing and processing are outside the scope of this paper, issues for consideration include workdays vs. weekends, holidays, data dropouts, data timeliness, incomplete data, and regional and cultural medical seeking behavioural differences. Conclusion Selecting and maintaining appropriate data sources is a challenging aspect to the planning and implementation of electronic disease surveillance systems. Similarly, obtaining the data feeds can be a time-consuming task, so sufficient evaluation and planning must be made upfront to reduce wasting precious resources. Data sources are not valuable unless they are complete, timely, and cover the desired population. Data does not need to be automated in order to provide high value to a system as long as the timeliness of receiving the data meets the surveillance goals. Although it would be ideal to always receive the data as quick as possible, the surveillance activity cannot place unrealistic requirements on data transmission - accept the best that a data provider can offer and aim to improve it in the future. In summary/conclusion, careful data source evaluation and data collection planning will save time in both system implementation and day-to-day system monitoring. Competing interests The authors declare that they have no competing interests. Authors' contributions Both authors contributed equally to this paper. Acknowledgements The authors would like to acknowledge their colleagues at both the Johns Hopkins University Applied Physics Laboratory (JHU/APL) and those in state and local health departments who have contributed their knowledge to the topic of data collection. This article has been published as part of BMC Proceedings Volume 2 Supplement 3, 2008: Proceedings of the 2007 Disease Surveillance Workshop. Disease Surveillance: Role of Public Health Informatics. The full contents of the supplement are available online athttp://www(dot)biomedcentral(dot)com/1753-6561/2?issue=S3. References 1. Loschen W, Sniegoski C, Coberly J, Lombardo J: Moving From Data to Information Sharing in Disease Surveillance Systems. Poster presented at the American Medical Informatics Association (AMIA) 2007 Spring Congress, May 22-24, 2007, Orlando, FL 2. World Health Organization: The World Health Report 2007 - A Safer Future: Global Public Health Security in the 21st Century.[http://www(dot)who(dot)int/whr/2007/whr07_en.pdf] webcite World Health Organization, Geneva, Switzerland; 2007. 3. Babin S, Magruder S, Hakre S, Coberly J, Lombardo J: Understanding the Data: Health Indicators in Disease Surveillance. In Disease Surveillance: A Public Health Informatics Approach. Edited by Lombardo JS, Buckeridge DL. Hoboken (NJ): John Wiley & Sons, Inc.; 2007:43-90. 4. Wojcik R, Hauenstein L, Sniegoski C, Holtry R: Obtaining the Data. In Disease Surveillance: A Public Health Informatics Approach. Edited by Lombardo JS, Buckeridge DL. Hoboken (NJ): John Wiley & Sons, Inc.; 2007:91-142. Predictors of On-Duty Coronary Events in Male Firefighters in the United States Coronary heart disease (CHD) accounts for 39% of “on-duty” deaths in firefighters in the United States. No studies have examined the factors that distinguish fatal from nonfatal work-associated CHD events. Male firefighters experiencing on-duty CHD events were retrospectively investigated to identify cardiovascular risk factors predictive of case fatality; 87 fatalities (death within 24 hours of the event) were compared with 113 survivors who retired with disability pensions for heart disease after on-duty nonfatal events. Cardiovascular risk factors were then examined for associations with case fatality. Predictors of CHD death in multivariate analyses were a previous diagnosis of CHD (or peripheral/cerebrovascular disease) (odds ratio [OR] 4.09, 95% confidence intervals [CI] 1.58 to 10.58), current smoking (OR 3.68, 95% CI 1.61 to 8.45), and hypertension (OR 4.15, 95% CI 1.83 to 9.44). Age ≤45 years, diabetes mellitus, and serum cholesterol level were not significant predictors of case fatality. In conclusion, previous CHD, current smoking, and hypertension are strong predictors of fatality in male firefighters experiencing on-duty CHD events. Accordingly, prevention efforts should include early detection and control of hypertension, smoking cessation/prohibition, and the restriction of most firefighters with significant CHD from strenuous duties.  The study was supported in part by a pilot project research training grant from the Harvard Education and Research Center for Occupational Safety and Health (Boston, Massachusetts), supported by Training Grant No. T42 OH008416-02 from the Centers for Disease Control and Prevention and the National Institute for Occupational Safety and Health (Cincinnati, Ohio). Additionally, the investigation was supported in part by a grant from the Massachusetts Public Employees Retirement Administration Commission (Somerville, Massachusetts). The funding agencies had no involvement in study design, data analysis, writing of the report, and/or the decision to submit the report for publication. The contents are solely the responsibility of the authors and do not necessarily reflect the views of the Public Employee Retirement Administration Commission or the National Institute for Occupational Safety and Health. Copyright © 2014 Elsevier Inc. All rights reserved. | Privacy Policy | Terms & Conditions | Feedback | About Us | Help | Contact Us The content on this site is intended for health professionals. Advertisements on this site do not constitute a guarantee or endorsement by the journal, Association, or publisher of the quality or value of such product or of the claims made for it by its manufacturer.

Comparison of US accredited and non-accredited rural critical access hospitals

M. NAWAL LUTFIYYA, AMRITA SIKKA, SONA MEHTA AND MARTIN S. LIPSKY

Department of Family and Community Medicine, University of Illinois — Chicago, College of Medicine at Rockford, Rockford, IL 61107, USA

Abstract

Background. US critical access hospitals play an integral role in rural healthcare. Accreditation may be helpful in assuring that these hospitals provide high-quality care.

Objective. To determine whether quality measures used in the US Centers for Medicare and Medicaid Services Hospital Compare database differed for critical access hospitals based on Joint Commission on Accreditation of Healthcare Organizations accreditation status.

Research design. Cross-sectional with /-test statistics computed on weighted data to ascertain statistically significant differences (P < 0.01).

Main outcome measure. Differences between accredited and non-accredited rural critical access hospitals on quality care indicators related to acute myocardial infarction, heart failure, pneumonia and surgical infection.

Subjects. US critical access hospitals.

Results. The differences between accredited and non-accredited rural critical access hospitals for 4 out of 16 hospital quality indicators were statistically significant (P < 0.01) and favored accredited hospitals. Also, accredited hospitals were more likely to rank in the top half of hospitals for 6 of the 16 quality measures.

Conclusions. The results indicate that in the setting of critical access hospitals, external accreditation appears to result in modestly better performance.

Keywords: hospital accreditation and quality care, quality indicators, critical access hospitals, US rural hospital care, disparities in hospital care
I0.I093/intqhc/mzp003
Introduction

The quality and access to healthcare varies widely in the USA [1, 2]. For the 20% of US residents living in rural settings, access to quality health care often depends on a critical access hospital [3]. These small hospitals have less than 25 beds, often function as the primary source of health care for a region and may even be the sole provider for a community's Medicare and Medicaid beneficiaries and uninsured individuals [4].

Their integral role in rural healthcare delivery makes it important for these hospitals to provide quality care. External monitoring, which offers an unbiased assessment of internal mechanisms and provides benchmarks for an organization, is one way to help assure quality care. Accreditation is one important external measure, and among the hospital accrediting organizations in the USA, the Joint Commission

on Accreditation of Healthcare Organizations (JCAHO) is the most widely recognized [5]. JCAHO is an independent, non-profit organization that conducts quality assessments in ^80% of hospitals in the USA [6, 7]. Its stated mission is to improve the safety and quality of care through evaluation and accreditation of healthcare organizations [6, 7]. In seeking JCAHO accreditation, a hospital agrees to be measured against a consistent and objective set of standards in areas such as patient assessment and care, patients' rights, human resources, organizational leadership, clinical ethics, management and information management.

Factors such as size, case mix, ownership and cost can influence a hospital's decision to seek JCAHO accreditation. Cost is perhaps the key factor for a critical access hospital [8]. Given their limited resources, it is not surprising that there is a substantial difference in accreditation rates between urban and rural hospitals. More than 95% of urban hospitals are JCAHO accredited compared with ~35% of critical access hospitals.

Studies examining the link between accreditation status and quality reveal mixed results. One study concluded that while JCAHO accreditation correlated with a higher quality of care for acute myocardial infarction (AMI) and lower 30-day mortality rates, accreditation levels 'were of limited value in differentiating quality' among surveyed hospitals [6]. Other studies fail to establish a correlation between accreditation scores and outcome measures such as mortality index, cost per case and length of stay [7] or demonstrate only a weak correlation between JCAHO scores and inpatient quality indicators [6, 9, 10]. In contrast, Longo et al. identified JCAHO accreditation as a key predictor for implementing patient safety initiatives [11]. However, many studies are limited because they examined only a single disease state or focused on larger non-rural hospitals, providing little insight into the association of JCAHO accreditation with the quality of healthcare at rural hospitals.

This study sought to determine whether the process measures used in the US Centers for Medicare and Medicaid Services (CMS) Hospital Compare database differed for critical access hospitals based on accreditation status. The Hospital Compare website is a tool used by the Hospital Quality Alliance to convey information about quality to the public. The results should provide one objective assessment of whether there is a link between JCAHO accreditation and the quality of care provided by a critical access hospital and insight into how well the JCAHO process meets its mission to improve safety and quality of care in the setting of smaller rural hospitals.

Methods

Context

This study used the Hospital Compare database to examine the critical access hospital outcome measures by accreditation status (accredited versus non-accredited). These data are collected by the CMS along with the Hospital Quality Alliance. The Hospital Quality Alliance is a public—private collaboration of several organizations including: the American Medical Association, Blue Cross and Blue Shield Association, National Business Coalition on Health and the JCAHO. By recording and making certain hospital quality measures publicly available, the Hospital Quality Alliance hopes to encourage hospitals to improve their quality of care [12].

Study design

This was a cross-sectional study examining secondary data from the 45 US states with at least one critical access hospital that submitted data to the Hospital Compare database. The study was approved by the University of Illinois — Chicago, College of Medicine at Rockford's Institutional Review Board.

Population and sample (hospitals/patients)

In March 2006, there were roughly 1300 critical access hospitals in the USA. The study analyzed data from all 730 critical

access hospitals that reported to Hospital Compare—a 56% participation rate [13]. All critical access hospitals coded as accredited in the Hospital Compare database were contacted to confirm their JCAHO accreditation. The 16 quality measure variables included in this analysis encompassed data from a total of 218 290 patients.

Variables

Table 1 displays the quality indicators available from the website. The indicators include eight AMI/heart attack care measures, four measures related to heart failure care, six pneumonia care measures and two measures related to surgical infection prevention. Because of inadequate amounts of data, four quality measures with less than 2000 cases reported for each measure were not included in the study. The discarded measures were: (i) the use of an angiotensin-converting enzyme (ACE) inhibitor for left ventricular systolic dysfunction (LVSD), (ii) whether patients received percutaneous coronary intervention (PCI) within 2 h of hospital arrival for an AMI, (iii) the delivery of smoking cessation counseling for patients experiencing an AMI and (iv) whether patients received a thrombolytic agent within 30 min of hospital arrival for a heart attack.

Data collection methods

Both large, urban 'acute care hospitals' [13] and small, remote 'critical access hospitals' voluntarily provide data to Hospital Compare [14, 15]. Starting with 2004 discharges, eligible urban acute care hospitals received an incentive payment if they reported on a 'starter set' of 10 initial quality performance measures and agreed to make this information publicly available [16]. In contrast, critical access hospitals were not eligible for a financial incentive but voluntarily choose to report on one or more of the 20 performance measures and whether to make their data publicly available.

Statistical analysis

Data from the Hospital Compare website were exported into a customized database for analysis using Statistical Package for Social Scientists (version 16.0, Chicago, IL, USA). The data were aggregated by accreditation status—accredited versus non-accredited—and then aggregated by each hospital quality indicator combining all states. All observations were weighted in proportion to the total number of eligible patients for a specific indicator to account for the differences in eligible patients for each indicator. The weighting variable was determined by dividing the total number of eligible patients for a specific indicator in an individual hospital by the mean sample size for the indicator. After applying the weighting variable, a two-tailed t-test for independent samples was computed for each hospital quality indicator to compare the accredited versus non-accredited critical access hospitals. Statistical significance was set at an a of 0.01.