Grand Canyon University Investigation Measurement and Planning Precision Responses

pls answer each DQ with 200-250 words

Nicole:

Distinguish between reliability and validity in research design. Using a translational research article from your graphic organizer, analyze the methods and results sections to discuss reliability and validity as it relates to the translational research. Include the permalink to the article in your reference.

When doing research it is important to ensure that measures used are both reliable and valid. Reliability and validity, are both important to research. They both refer to how well a method measures something and are concepts used to evaluate the quality of research. A test must be reliable to produce a valid result.

Reliability in research design refers to consistency across time (time-retest reliability), across items (internal consistency), and across researchers/ interrater reliability, the consistency in measurement over repeated measures (Nelson, 1980). It is understood that a reliable measure is one with a low random chance of error and is assessed by one of four methods. Retest, alternative-form test, split-halves test, or internal consistency test (Nelson, 1980). If you apply the same method to the same sample under the same conditions, this should render the same results, if not then the measurement may be unreliable.

Validity in research design is measuring what is intended to be measured, a valid measure are those with low nonrandom systemic errors (Nelson, 1980). Validity is measured by one of three methods which include content validation, criterion-related validation, and construct validation.

The translational research study I have chosen from my graphic organizer is 3076 Findings from the first year of California’s Workplace Violence Prevention in Healthcare standard (Title 8, Section 3342) (Odes, Hong, Chapman, 2019). Methods/study population: Reports submitted to CalOSHA pursuant to the WORKPLACE VIOLENT INCIDENT REPORTING SYSTEM FOR HOSPITALS are considered public record and available through the state Public Records Act (PRA) mandate. Records from 7/1/17-9/30/18 were obtained through the CalOSHA PRA request process. Descriptive statistics and correlations were calculated using Stata.

Records reporting 11,116 individual events of violence were analyzed. These results do not include reports submitted by the five California State Hospitals, a group of facilities which treated nearly 13,000 patients in 2017, many of whom have a psychiatric diagnosis and are undergoing treatment mandated by a judicial system. For each record 111 variable were reported, characteristics of workers involved, factors which may have triggered the event, and what measures were take to mitigate the situation during and after the incident.

Due to the underreporting of incidents from hospitals and California facilities that were not included in the study, this leads to challenges in defining the scope of the problem of WPV and the evaluation of the efficacy of interventions to address (Odes et al, 2019). Selection of the subjects threatens the validity.

References

Odes, R., Hong, O., Chapman, S. (2019). 3076 Findings from the first year of California’s Workplace Violence Prevention in Healthcare standard (Title 8, Section 3342). Journal of Clinical and Translational Science, 3(S1), 117-118. Retrieved from: https://www.cambridge.org/core/journals/journal-of-clinical-and-translational-science/article3076-findings-from-the-first-year-of-californias-workplace-violence-prevention-in-healthcare-standard-title-8-section-3342/0529617E4AFA746503670178A2F67515

Nelson, A. A. (1980). Research design: Measurement, reliability, and validity. American Journal of hospital pharmacy, 37(6), 851-857. Retrieved from: https://pubmed.ncbi.nlm.nih.gov/7395882/

Holly:

Reliability and validity are concepts used to evaluate the quality of research. They indicate how well a method, technique or test measures something. Reliability is about the consistency of a measure, and validity is about the accuracy of a measure. It’s important to consider reliability and validity when either creating research design or evaluating one. They are closely related, but they mean different things. A measurement can be reliable without being valid. However, if a measurement is valid, it is usually also reliable. Reliability refers to how consistently a method measures something. If the same result can be consistently achieved by using the same methods under the same circumstances, the measurement is considered reliable. Validity refers to how accurately a method measures what it is intended to measure. If research has high validity, that means it produces results that correspond to real properties, characteristics, and variations in the physical or social world. Reliability can be estimated by comparing different versions of the same measurement. Validity is harder to assess, but it can be estimated by comparing the results to other relevant data or theory (Nelson, 1980). In my article, the trial was a cluster-randomized community-based diabetes prevention study conducted in predominantly African American, Christian churches. Clusters were defined as churches and selected from the North Texas region based on recommendations from the Community Advisory Board. Eligible churches were those with parishioners who were primarily African American and agreed to provide one or two peers to facilitate the intervention. Six churches were randomized to a faith-enhanced diabetes prevention program (F-DPP) while the remaining five churches were randomized to the standard diabetes prevention program (S-DPP). The prevalence of metabolic syndrome was 42.08% before receiving the DPP intervention and 31.22% after the intervention that represented a 10.86% absolute reduction and a 25.81% relative reduction from baseline. The adjusted odds ratio (OR) of being free from metabolic syndrome at post-intervention in contrast to baseline was 2.14 (p = 0.02) (Dodgen, 2020).

References:

Dodgen, L., Kitzman, H., & Mamun, A. (2020). Reducing metabolic syndrome through a community-based lifestyle intervention in African American women. Nutrition, Metabolism and Cardiovascular Diseases, 30(10), 1785–1794. https://doi-org.lopes.idm.oclc.org/10.1016/j.numecd.2020.06.005

Nelson, A. Research design: measurement, reliability, and validity. American J Hosp Pharm. 1980 Jun;37(6):851-7. PMID: 7395882.

James:

Reliability and validity both have their value in terms of research. Validity is defined as the extent to which a concept is accurately measured in a quantitative study and reliability relates to the consistency of a measure. Validity is a judgment based on various types of evidence. The relevant evidence includes the measure’s reliability, whether it covers the construct of interest, and whether the scores it produces are correlated with other variables they are expected to be correlated with and not correlated with variables that are conceptually distinct. The reliability and validity of a measure is not established by any single study but by the pattern of results across multiple studies. The assessment of reliability and validity is an ongoing process. (opentextbc.ca 2020)

In 2016 there was a study that aimed to investigate the social and electronic media attention received by psychiatry research using the Altmetric attention scores (AAS) and the predictors of this public engagement. The researchers analyzed all research articles published in 2016 in the top 5 impact factor-based psychiatry journals. They extracted the AAS, various media (news, Twitter, Facebook, etc.) engagements, and citations received by each article using online database. Univariate and multivariate linear regression analysis was performed. A total of 360 research articles published in JAMA Psychiatry, The Lancet Psychiatry, World Psychiatry, American Journal of Psychiatry, Psychotherapy and Psychosomatics were included. The median AAS was 44 [IQR = 15 – 146] and median citations were 26 (14-47) with a significant but weak correlation (rs = 0.43; p = 0.001) between the two metrics. (sciencedirect.com. 2016) Given that this study is the first of its kind, we can not say there is solid evidence for the reliability of the study, however the validity of the study shows the social media highlights the interest of the public regarding mental health research. “Future studies should help understand the effect of social media engagement with psychiatry research, the means for harnessing it in disseminating new information, and the de-stigmatization of mental illness”.

Reference

Chiang, I.-C. A., Jhangiani, R. S., & Price, P. C. (2015, October 13). Reliability and Validity of Measurement. Research Methods in Psychology 2nd Canadian Edition. https://opentextbc.ca/researchmethods/chapter/reliability-and-validity-of-measurement/. https://www-sciencedirect-com.lopes.idm.oclc.org/science/article/pii/S0959438820301616