Reviewer's report Title: The external validity of published randomized controlled trials in primary care Version: 1 Date: 13 November 2008 Reviewer: Chris Butler Reviewer's report: Review for Bio Med Central This is a well written useful contribution to research methodology (in the true sense of that word!) in primary care. The authors describe the reporting of eligibility fraction, enrolment fraction and recruitment fraction and numbers needed to be screened in RCTs in four primary care journals. Predictably, they find that a significant minority (70%) of trials reported the number of individuals assessed by investigators for eligibility. About 10% more studies reported the number eligible for participation, while all reported the actual number recruited. A problem with primary care research is that patients are often recruited opportunistically into studies during busy periods of consultation. It is something that busy clinicians do in addition to all the other demands made of them. Keeping logs of all potentially eligible patients, those approached for participation and the reasons why they are not approached or declined participation is a lower priority than actually recruiting patients. Gathering this kind of information usually requires researchers in the practice with dedicated time for supporting practitioners in this work. However, as in most primary care studies, a few patients are recruited from each site. This becomes very inefficient unless practitioners are involved in numerous studies at the same time. Given the complexity of the delivery of primary care, perhaps this problem is something that we have to learn to live with, especially when one is dealing with opportunistic recruitment for acute conditions (where searching databases is not helpful to promoting recruitment). It does not necessarily follow that a low enrolment fraction leads to studies with limited external validity. A low enrolment fraction can still result in a study population that is representation of the eligible population; it is just that we are unable to make this judgement properly, given a lack of knowledge about the eligible population. However, this does not always matter, in that for some conditions, as long as the patients are properly described, clinicians can compare their individual patients with the patients that were enrolled in the study. This problem therefore does not necessarily lead to bias and poor external validity. Perhaps this point could be brought out further in the discussion. All the points made about RCTs would apply to most non-randomised designs, especially prospective cohort studies.
Otherwise this is a very clearly written report which raises an important issue. Completeness of reporting the population at risk and the eligibility fraction however does not necessarily mean poor quality research. Sometimes excellent and highly useful research is derived from situations where it is simply not practical to get these data, and this should also be recognised.