Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial, open data platform and infrastructure that supports research on pragmatic trials. It gathers and distributes clean trial data, ratings, and evaluations using PRECIS-2. This permits a variety of meta-epidemiological studies to evaluate the effects of treatment across trials of different levels of pragmatism.
Background
Pragmatic studies are increasingly recognized as providing real-world evidence for clinical decision-making. The term "pragmatic", however, is used inconsistently and its definition and evaluation need further clarification. Pragmatic trials must be designed to guide clinical practice and policy decisions, rather than confirm the validity of a clinical or physiological hypothesis. A pragmatic trial should also try to be as similar to actual clinical practice as is possible, including its recruitment of participants, setting up and design as well as the implementation of the intervention, and the determination and analysis of the outcomes, and primary analyses. This is a major distinction between explanation-based trials, as defined by Schwartz and Lellouch1, which are designed to confirm the hypothesis in a more thorough manner.
Truely pragmatic trials should not blind participants or clinicians. This can result in a bias in the estimates of the effect of treatment. Pragmatic trials should also seek to recruit patients from a wide range of health care settings to ensure that the results can be compared to the real world.
Furthermore studies that are pragmatic should focus on outcomes that are crucial to patients, like quality of life or functional recovery. This is especially important in trials that require surgical procedures that are invasive or may have harmful adverse impacts. The CRASH trial29, for example, focused on functional outcomes to evaluate a two-page case report with an electronic system to monitor the health of patients in hospitals suffering from chronic heart failure. In addition, the catheter trial28 focused on urinary tract infections that are symptomatic of catheters as the primary outcome.
In addition to these characteristics the pragmatic trial should also reduce the trial's procedures and data collection requirements in order to reduce costs. Finaly the aim of pragmatic trials is to make their results as relevant to real-world clinical practices as possible. This can be achieved by ensuring that their analysis is based on an intention-to treat method (as described within CONSORT extensions).
Many RCTs that do not meet the criteria for pragmatism but contain features in opposition to pragmatism, have been published in journals of varying types and incorrectly labeled as pragmatic. This can lead to misleading claims of pragmatism, and the use of the term should be made more uniform. The creation of a PRECIS-2 tool that can provide a standardized objective evaluation of the pragmatic characteristics is the first step.
Methods
In a practical study it is the intention to inform clinical or policy decisions by demonstrating how an intervention can be integrated into routine care in real-world situations. This is different from explanatory trials that test hypotheses regarding the cause-effect connection in idealized conditions. In this way, pragmatic trials may have a lower internal validity than explanatory studies and be more prone to biases in their design as well as analysis and conduct. Despite these limitations, pragmatic trials can contribute valuable information to decision-making in healthcare.
The PRECIS-2 tool assesses the level of pragmatism that is present in an RCT by assessing it across 9 domains, ranging from 1 (very explanatory) to 5 (very pragmatic). In this study, the recruit-ment organization, flexibility in delivery and follow-up domains were awarded high scores, but the primary outcome and the procedure for missing data were not at the practical limit. This suggests that a trial could be designed with good practical features, yet not harming the quality of the trial.
It is, however, difficult to judge how practical a particular trial is since pragmatism is not a binary characteristic; certain aspects of a study can be more pragmatic than others. A trial's pragmatism can be affected by modifications to the protocol or logistics during the trial. Koppenaal and colleagues found that 36% of 89 pragmatic studies were placebo-controlled or conducted prior to the licensing. Most were also single-center. Therefore, they aren't very close to usual practice and are only pragmatic in the event that their sponsors are supportive of the lack of blinding in these trials.
Another common aspect of pragmatic trials is that researchers try to make their results more valuable by studying subgroups of the trial. This can result in unbalanced analyses with less statistical power. This increases the risk of missing or misdetecting differences in the primary outcomes. In the case of the pragmatic trials that were included in this meta-analysis this was a major issue since the secondary outcomes were not adjusted to account for the differences in the baseline covariates.
Furthermore practical trials can present challenges in the gathering and interpretation of safety data. This is due to the fact that adverse events are usually self-reported and are susceptible to reporting errors, delays or coding deviations. It is therefore important to enhance the quality of outcomes ascertainment in these trials, and ideally by using national registry databases instead of relying on participants to report adverse events on the trial's own database.
Results
Although the definition of pragmatism does not require that all trials be 100 100% pragmatic, there are some advantages to incorporating pragmatic components into clinical trials. These include:
Increased sensitivity to real-world issues as well as reducing the size of studies and their costs, and enabling the trial results to be faster translated into actual clinical practice (by including patients from routine care). However, pragmatic studies can also have disadvantages. The right amount of heterogeneity, like, can help a study extend its findings to different patients or settings. However the wrong kind of heterogeneity can reduce the assay sensitivity, and therefore reduce a trial's power to detect minor treatment effects.
Several studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 created an approach to distinguish between explanation-based trials that support a clinical or physiological hypothesis as well as pragmatic trials that help in the choice of appropriate therapies in clinical practice. Their framework comprised nine domains, each scoring on a scale of 1 to 5 with 1 being more informative and 5 suggesting more pragmatic. The domains included recruitment and setting, delivery of intervention, flexible adherence, follow-up and primary analysis.
The original PRECIS tool3 included similar domains and a scale of 1 to 5. 프라그마틱 무료슬롯 et al10 created an adaptation of this assessment called the Pragmascope that was simpler to use in systematic reviews. They discovered that pragmatic systematic reviews had higher average scores across all domains but lower scores in the primary analysis domain.
This difference in primary analysis domains can be explained by the way most pragmatic trials analyse data. Some explanatory trials, however do not. The overall score was lower for pragmatic systematic reviews when the domains on organisation, flexible delivery and follow-up were combined.
It is important to remember that a pragmatic study should not mean that a trial is of poor quality. In fact, there are an increasing number of clinical trials which use the term "pragmatic" either in their title or abstract (as defined by MEDLINE, but that is neither sensitive nor precise). The use of these terms in abstracts and titles could suggest a greater awareness of the importance of pragmatism however, it is not clear if this is evident in the contents of the articles.
Conclusions
As the importance of real-world evidence grows commonplace the pragmatic trial has gained popularity in research. They are randomized clinical trials which compare real-world treatment options instead of experimental treatments under development. They include patient populations that more closely mirror the ones who are treated in routine medical care, they utilize comparators that are used in routine practice (e.g., existing medications), and they depend on participants' self-reports of outcomes. This method can help overcome the limitations of observational research such as the biases that are associated with the use of volunteers as well as the insufficient availability and the coding differences in national registry.
Pragmatic trials also have advantages, such as the ability to draw on existing data sources and a higher chance of detecting significant differences than traditional trials. However, they may have some limitations that limit their reliability and generalizability. For example the rates of participation in some trials may be lower than anticipated due to the healthy-volunteer influence and incentives to pay or compete for participants from other research studies (e.g. industry trials). Many pragmatic trials are also restricted by the need to recruit participants quickly. Some pragmatic trials also lack controls to ensure that any observed differences aren't due to biases that occur during the trial.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatic and that were published from 2022. The PRECIS-2 tool was employed to assess the degree of pragmatism. It includes domains such as eligibility criteria as well as recruitment flexibility, adherence to intervention, and follow-up. They found that 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or more) in at least one of these domains.
Trials that have a high pragmatism score tend to have higher eligibility criteria than traditional RCTs that have specific criteria that are unlikely to be found in the clinical setting, and include populations from a wide variety of hospitals. According to the authors, can make pragmatic trials more relevant and applicable in the daily clinical. However, they don't guarantee that a trial will be free of bias. The pragmatism principle is not a definite characteristic; a pragmatic test that does not possess all the characteristics of an explicative study can still produce valid and useful outcomes.