The Use of Statistics in Medical Research: A Comparison of Wiener Klinische Wochenschrift andWiener Medizinische Wochenschrift

Alexander M. Strasak; Qamruz Zaman; Gerhard Marinell; Karl P. Pfeiffer; Hanno Ulmer

doi:10.17713/ajs.v36i2.327

Authors

Alexander M. Strasak Dept. of Medical Statistics, Informatics and Health Economics, Innsbruck Medical University, Austria
Qamruz Zaman Dept. of Medical Statistics, Informatics and Health Economics, Innsbruck Medical University, Austria
Gerhard Marinell Inst. of Statistics, University of Innsbruck, Austria
Karl P. Pfeiffer Dept. of Medical Statistics, Informatics and Health Economics, Innsbruck Medical University, Austria
Hanno Ulmer Dept. of Medical Statistics, Informatics and Health Economics, Innsbruck Medical University, Austria

DOI:

https://doi.org/10.17713/ajs.v36i2.327

Abstract

To evaluate the quantity and quality of the use of statistics in Austrian medical journals, all “original research” papers in No. 116/1-12 of Wiener Klinische Wochenschrift (WKW) and 153/1-24, 154/1-24 of Wiener Medizinische Wochenschrift(WMW) were screened for their statistical content. Types, frequencies and complexity of statistical methods applied were
systematically recorded. A 46-item checklist was used to evaluate statistical quality for a subgroup of papers. 74.3% of WKW papers contained inferential methods beyond descriptive statistics. Only 43.7% of WMW papers employed methods of inferential statistics. There was a statistical significant difference regarding the use of statistical methods between the two journals (p = 0:009). In addition, complexity and sophistication of statistical analysis was considerable higher for WKW papers (p = 0:02). Statistical errors
and deficiencies were identified in a large proportion of papers. Although inferential statistics were frequently identified in papers from WKW, only a minority of WMW research had analytical character. Types and frequencies of statistical errors identified, did not vary meaningful from findings of similar studies for a wide range of medical journals. There is reason to assume, that the journal impact-factor does not seem to be a powerful predictor for the statistical quality of published research.

References

Altman, D. G. (1981). Statistics and ethics in medical research. Improving the quality of statistics in medical journals. British Medical Journal, 282, 44-47.

Altman, D. G. (1982). Statistics in medical journals. Statistics in Medicine, 1, 59-71.

Altman, D. G. (1991). Statistics in medical journals: Developments in the 1980s. Statistics in Medicine, 10, 1897-1913.

Altman, D. G. (1998). Statistical reviewing for medical journals. Statistics in Medicine, 17, 2661-2674.

Altman, D. G. (2000). Statistics in medical journals: Some recent trends. Statistics in Medicine, 19, 3275-3289.

Altman, D. G., Gore, S. M., Gardner, M. J., and Pocock, S. J. (1983). Statistical guidelines for contributors to medical journals. British Medical Journal, 286, 1489-1493.

Armitage, P. (1955). Tests for linear trends in proportions and frequencies. Biometrics, 11, 375-386.

Bezeau, S., and Graves, R. (2001). Statistical power and effect sizes of clinical neuropsychology research. Journal of Clinical and Experimental Neuropsychology, 23, 399-406.

Cardiel, M. H., and Goldsmith, C. H. (1995). Type of statistical techniques in rheumatology and internal medicine journals. Revista de Investigación Clinica, 47, 197-201.

Clopper, C. J., and Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26, 404-413.

Colditz, G. A., and Emerson, J. D. (1985). The statistical content of published medical research: Some implications for biomedical education. Medical Education, 19, 248-255.

Cooper, R. J., Schriger, D. L., and Close, R. J. H. (2002). Graphical literacy: The quality of graphs in a large-circulation journal. Annals of Emergency Medicine, 40, 317-322.

Davies, H. T. (1998). Describing and estimating: Use and abuse of standard deviations and standard errors. Hospital Medicine, 59, 327-328.

Emerson, J. D., and Colditz, G. A. (1983). Use of statistical analysis in the New England Journal of Medicine. New England Journal of Medicine, 309, 709-713.

Freedman, K. B., Back, S., and Bernstein, J. (2001). Sample size and statistical power of randomised, controlled trials in orthopaedics. Journal of Bone and Joint Surgery, 83, 397-402.

García-Berthou, E., and Alcaraz, C. (2004). Incongruence between test statistics and P values in medical papers. BMC Medical Research Methodology, 4, 13-17.

Gardenier, J. S., and Resnik, D. B. (2002). The misuse of statistics: concepts, tools, and a research agenda. Accountability in Research, 9, 65-74.

Gardner, M. J., Altman, D. G., Jones, D. R., and Machin, D. (1983). Is the statistical assessment of papers submitted to the “british medical journal” effective? British Medical Journal, 286, 1485-1488.

Gardner, M. J., and Bond, J. (1990). An exploratory study of statistical assessment of papers published in the british medical journal. Journal of the American Medical Association, 263, 1355-1357.

Goodman, S. N., Altman, D. G., and George, S. L. (1998). Statistical reviewing policies of medical journals. Journal of General Internal Medicine, 13, 753-756.

Gore, S. M., Jones, I. G., and Rytter, E. C. (1976). Misuse of statistical methods: critical assessment of articles in bmj from january to march 1976. British Medical Journal, 1, 85-87.

Gore, S. M., Jones, I. G., and Thompson, S. G. (1992). The Lancet’s statistical review process: areas for improvement by authors. Lancet, 340, 100-102.

Hoffmann, O. (1984). Application of statistics and frequency of statistical errors in articles in Acta Neurochirurgica. Acta Neurochirurgica, 71, 307-315.

Huang, W., LaBerge, J. M., Lu, Y., and Glidden, D. V. (2002). Research publications in vascular and interventional radiology: research topics, study designs, and statistical methods. Journal of Vascular and Interventional Radiology, 13, 247-255.

Kanter, M. H., and Taylor, J. R. (1994). Accuracy of statistical methods in Transfusion: A review of articles from July/August 1992 through June 1993. Transfusion, 34, 697-701.

MacArthur, R. D., and Jackson, G. G. (1984). An evaluation of the use of statistical methodology in the Journal of Infectious Diseases. Journal of Infectious Diseases, 149, 349-354.

Marshall, S. W. (2004). Testing with confidence: The use (and misuse) of confidence intervals in biomedical research. Journal of Science and Medicine in Sports, 7, 135-137.

McKinney, W. P., Young, M. J., Hartz, A., and Bi-Fong Lee, M. (1989). The inexact use of Fisher’s exact test in six major medical journals. Journal of the American Medical Association, 261, 3430-3433.

Menegazzi, J., Yealy, D., and Harris, J. (1991). Methods of data analysis in the emergency medicine literature. American Journal of Emergency Medicine, 9, 225-227.

Murray, G. D. (1991). Statistical guidelines for the British Journal of Surgery. British Journal of Surgery, 78, 782-784.

Nagele, P. (2001). Misuse of standard error of the mean (sem) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. British Journal of Anaesthesia, 90, 514-516.

Olsen, C. H. (2003). Review of the use of statistics in Infection and Immunity. Infection and Immunity, 71, 6689-6692.

Pocock, S. J., Hughes, M. D., and Lee, R. J. (1987). Statistical problems in the reporting of clinical trials – a survey of three medical journals. New England Journal of Medicine, 317, 426-432.

Reed III, J. F., Salen, P., and Bagher, P. (2003). Methodological and statistical techniques: what do residents really need to know about statistics? Journal of Medical Systems, 27, 233-238.

Schor, S., and Karten, I. (1966). Statistical evaluation of medical manuscripts. Journal of the American Medical Association, 195, 1123-1128.

Sheehan, T. J. (1980). The medical literature – Let the reader beware. Archives of Internal Medicine, 140, 472-474.

Welch II, G. E., and Gabbe, S. G. (1996). Review of statistics usage in the american journal of obstetrics and gynecology. American Journal of Obstetrics and Gynecology, 175, 1138-1141.