An international panel including patients, clinicians, researchers, acupuncture and surgery trialists, statisticians, and experts in clinical epidemiology and methodology have developed new guidance for randomised controlled trials in acupuncture. It addresses the most prevalent and critical concerns of current acupuncture trials and will help funding agencies, trial registers, and journal editors to evaluate the relevance, importance, and quality of submitted trial proposals and completed trials

Acupuncture therapies are used in 183 countries according to a 2013 survey by the World Federation of Acupuncture-Moxibustion Societies.1 Despite over 14 000 published randomised controlled trials in acupuncture,23 scepticism remains about the number of studies to support the effectiveness and safety of interventions, as is the case for other complementary therapies. Challenges in designing and conducting high quality acupuncture trials may partly explain the scepticism. Randomised trials provide the most rigorous support for therapeutic and health system decisions for patients, clinicians, payers, and policy makers.45 Multiple meta-epidemiological studies have identified shortcomings in acupuncture trials. Common problems include limitations in study design, applicability, and reporting.6789

In contrast to pharmacological interventions, acupuncture therapies rely on procedural expertise and are complex and multifaceted.89 These characteristics present unique methodological challenges for randomised trials, including standardisation of interventions and controls, dealing with non-specific effects of interventions, and patients’ expectations.

Despite over 14 000 published randomised controlled trials in acupuncture, shortcomings persist, including research corresponding poorly to clinical practice, poor reporting, and high risk of bias

In contrast to pharmacological interventions, acupuncture therapies rely on procedural expertise and are often complex and multifaceted

An international panel including patients, clinicians, researchers, trialists in acupuncture and surgery, and experts in statistics, patient engagement, and clinical epidemiology and methodology developed guidance for trials

This guidance covers methodological issues, including selection of control, blinding, and the challenges particular to acupuncture trials, including four essential design elements for acupuncture based on whether trials are primarily explanatory/mechanistic or pragmatic/practical

The original and revised Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA)1011 focus on reporting not optimal trial conduct. Other acupuncture guidance covers type of trial (efficacy,12 effectiveness13), design, and lack of representativeness (include an international panel14), but is not informed by comprehensive and systematic evidence synthesis.

To address the most prevalent design and methodological concerns in current acupuncture randomised trials, an international panel including patients, clinicians, researchers, and trialists in acupuncture, surgery, statistics, patient engagement, and clinical epidemiology developed guidance for research teams planning acupuncture trials. To inform the guidance, we conducted a systematic survey15 exploring characteristics associated with acupuncture treatment effects.

The issues covered in this guidance fall into two categories: those unique to acupuncture trials, and, based on our exhaustive review of acupuncture trials, those that are not specific but still applicable to acupuncture trials. We frame our discussion as a series of questions for readers to consider when designing trials, focusing on choices of the research question, patients, interventions, and outcomes and presenting the implications of these choices using examples from existing trials (box 1).

Consideration 1: Is the question important?

Trialists should establish the rationale for the research question informed by a systematic review of the relevant literature.

Consideration 2: Who should the participants be?

Trialists with a primary explanatory or mechanistic objective could include people who are most responsive to the intervention. With a primary pragmatic or practical objective, trialists could include a broad population varying in age, severity, comorbidity, and exposure to other interventions.

Consideration 3: How can trialists address possible differences in effect across patient groups?

For pragmatically oriented trials include heterogeneous populations, with prior specification of subgroup analyses to consider hypothesised effect modification.

Consideration 4: Who should perform the intervention?

Trialists should report the expertise of the acupuncturists. A trial that aims to show whether an acupuncture treatment can work under ideal conditions will choose the most expert practitioners available. Trialists aiming to establish the effect of treatment in ordinary practice will select clinicians with typical levels of expertise.

Consideration 5: What specific acupuncture technical features should be considered if aiming to design trials for maximum treatment effect?

If aiming for maximum treatment effect, trialists should select a high frequency of acupuncture treatment sessions and penetrating type of acupuncture (manual and electroacupuncture) over lower frequency or non-penetrating (transcutaneous electrical acupoint stimulation (TEAS), laser, acupressure) acupuncture.

Consideration 6: Are trialists interested in the specific effect or the overall (specific and non-specific) effect of acupuncture?

Trialists should blind data collectors, outcome assessors, and data analysts and carefully consider the desirability of a sham that leads to underestimation of acupuncture’s treatment effects in clinical practice.

Consideration 7: To what extent should the trialist choose a flexible intervention and comparator?

Trialists with a primary explanatory or mechanistic focus might specify that practitioners administering both the intervention and the control use a highly standardised approach. Trialists with a primary pragmatic or practical focus might include clinicians with varied techniques reflecting practice in the community and instruct them to use their usual treatment approaches.

Consideration 8: How should trialists deal with adherence?

When designing a trial that primarily takes the individual patient’s perspective, trialists might consider implementing strategies to achieve optimal adherence. When trialists take a public health perspective, there is no need to implement strategies to increase adherence.

Consideration 9: What type of outcomes should trialists choose?

Trialists with mainly mechanistic objectives can focus on surrogate outcomes (eg, blood pressure, heart rate). If trialists have a primary pragmatic objective, they should focus on patient important outcomes such as pain. Trialists should continue using symptoms and functional outcomes and increase the use of quality of life and major events (eg, cardiovascular events, hospital admission, infections) as outcomes.

Consideration 10: How should trialists decide the duration of follow-up?

When investigators have primarily explanatory objectives, short follow-up times are likely to be optimal. Acupuncture trials with primarily pragmatic objectives might choose follow-up times of six months or longer.

Consideration 11: How should triaists engage patients during design and conduct of trials?

To empower patients and conduct more patient centric research, acupuncture trials should consider informing, educating, discussing, engaging, and partnering with patient organisations and with individual patients.

We established a steering committee including frontline acupuncture clinicians, acupuncture trialists, clinical trial methodologists, and a statistician (XHJ, JPL, LXL, CMW, LT, YQZ, RMJ, and GHG) with extensive experience in acupuncture and trial methodology. The steering committee recruited an international expert panel of 27 experts (from Asia, Europe, America, and Australia), including patients, frontline acupuncture clinicians, acupuncture and surgical trialists, clinical trial methodologists, and statisticians. Acupuncture trialists and methodologists were appointed to the panel on the basis of their h index and assessment of their research expertise, and frontline clinicians on the basis of their clinical experience and reputation.

Using the results of our systematic survey,15 the steering committee drafted the first version of the guidance. The purpose of the survey was to identify the most prevalent trial design and execution issues and factors associated with the magnitude of acupuncture treatment effects.15 In an iterative process, the expert panel provided feedback on the structure and content of the acupuncture trials guidance draft and on piloting of the guidance and user feedback (web appendix 1).

In the early stages of designing a randomised controlled trial of acupuncture, researchers should specify their objective. Studies may fall along a continuum from explanatory or mechanistic to pragmatic or practical (box 2).45 Trials with primary explanatory or mechanistic objectives will evaluate the specific effect of acupuncture whereas those with mainly pragmatic or practical objectives will aim to inform clinical or policy decision making. Trialists can refer to the pragmatic-explanatory continuum indicator summary (PRECIS)18 or PRECIS-219 to assess where their trials are positioned.

Schwartz and Lellouch’s explanatory-pragmatic framework classified trials as either aiming to achieve the maximum effects of interventions (eg, measure tumour size or cancer related mortality as outcomes) under ideal conditions or to inform decisions in clinical or health system contexts by resembling current practice settings.4 Witt and colleagues have provided guidance to apply this framework specific to acupuncture trials.1316

Karanicolas and colleagues provide an alternative framework emphasising the purpose and suggest that trial purpose depends on the decision makers’ perspectives (eg, patients and clinicians versus policy makers).5 They proposed a revised mechanistic-practical framework to design and interpret randomised trials. Mechanistic trials have proof-of-concept objectives and often include surrogate outcomes such as biological measures (eg, forced exhaled volume in 1 second (FEV1) in patients with asthma17); practical trials provide comprehensive information that bears directly on healthcare decisions.

Once trialists situate their study in the explanatory-pragmatic continuum, they can use the PICOT framework20 to determine critical design elements. For instance, planning a randomised trial to study the effect of acupuncture on chronic low back pain requires selection of participants (P), intervention (I) (eg, acupoints, stimulation methods, depth of the insertion), comparator (C) (eg, sham acupuncture or medication), outcomes (O), and duration of follow-up (T). If their primary objective is to evaluate the specific effect of acupuncture, trialists will likely choose sham as the comparator. If the primary objective is to explore the effectiveness and safety of acupuncture therapies and inform clinical or policy decision making, commonly used alternatives (such as non-steroidal anti-inflammatory drugs) might be suitable comparators.

In defining their objectives and consequent methods in relation to the explanatory-pragmatic continuum (box 3), trialists should consider a systematic review to establish the rationale for their research question (box 1, consideration 1).

Population—Patients with chronic low back pain

Intervention—Clinicians gave true acupuncture treatment twice a week for five weeks (10 sessions). If patients experienced a 10-50% reduction in pain (Von Korff chronic pain grade scale), they received five additional sessions

Comparators—Sham acupuncture (same frequency and session as true acupuncture) or German guideline based multimodal treatment programme (10 sessions of physician or physiotherapist administered physiotherapy or exercise)

Outcomes and follow-up—The primary outcome was pain or functional status of low back pain six months after randomisation. Secondary outcomes were quality of life and patient global assessment of therapy effectiveness, medication use, and adverse events at each session after six months. Trialists defined pain status as 33% improvement on three pain related items on the Von Korff chronic pain grade scale and functional status as 12% improvement on the Hanover functional ability questionnaire

Use of guideline based conventional therapy and patient important outcomes with long term follow-up indicates a pragmatic design. Having sham acupuncture as the comparator shows a more explanatory feature

Although the guidance we offer to determine the objective of pragmatic versus explanatory trials generally applies, there will be exceptions. For instance, although pragmatic objectives can generally be achieved by enrolling heterogeneous populations, if the desired focus is a narrower population, it does not preclude a primarily pragmatic objective. Once the objectives are defined trialists should consider the 11 questions below in designing their trial.

Published acupuncture randomised trials rarely cite previous systematic reviews in describing the rationale for conducting a trial, suggesting a high risk of wasteful use of research resources22 (web appendix 2, 2.1.1). Deciding on a trial’s objective requires an understanding of the available body of evidence on related questions, which requires the availability or conduct of systematic reviews.22 Are there already sufficient trials exploring the specific effect of acupuncture therapies? Is there already moderate or high certainty evidence establishing treatment effects in particular settings to inform decision making? If existing systematic reviews show little or no evidence on acupuncture therapies for a target condition, investigators can conclude that an optimal randomised trial will provide proof-of-concept information. If sufficient evidence establishes a therapeutic effect on a surrogate outcome or patient important outcome under narrowly defined circumstances, moving to a pragmatic trial is likely to be appropriate (box 1, consideration 1).

Trialists need to determine the population that might be the most responsive to and are interested in the intervention and what are the necessary enrolment restrictions (box 1, consideration 2). Restricting participants to those who are most responsive reflects a proof-of-concept approach. Such a trial will aim to establish the effect of acupuncture therapies in people most likely to benefit, leaving open the question of its value in routine clinical settings serving a diverse population.

Trialists with a primarily explanatory or mechanistic objective might include people who are most responsive, those at highest risk of adverse outcomes, or the most highly compliant and may exclude those with comorbidities that might affect the outcomes or likely to drop out because of severe comorbidity.

Trials with a primarily pragmatic aim would produce widely generalisable results that optimally inform clinical practice. The trial should include a broad population varying in age, severity, comorbidity, and exposure to other interventions and, depending on their target audience, either highly compliant or more general populations (box 4).

Inclusion criteria (explanatory or mechanistic)—Patients were eligible if they fulfilled all five criteria: age ≥50 years; history of knee pain for >3 months; reported knee pain on most days of the past month; average knee pain severity  ≥ 4 on an 11 point scale over the past month; and) reported morning knee stiffness of <30 minutes

Exclusion criteria—The study excluded patients if they fulfilled any of the following 13 criteria: history of any systemic arthritic condition; knee arthroplasty on most painful knee; on waiting list for any knee surgery; any knee surgery in past six months; presence of any other condition affecting lower limb function (eg, trauma, malignancy, neurological condition); any knee injection in past six months (eg, cortisone, hyaluronic acid);) current use of oral or injectable anticoagulant medication; use of acupuncture in past 12 months; any bleeding disorder; allergy to light; referral to a pain clinic or use of morphine or pethidine within the past six months; any other medical condition precluding participation in the trial (eg, kidney or liver disease, deep vein thrombosis); knee pain subject to compensation claim; unable to give written informed consent

Inclusion criteria (pragmatic or practical)—Patients were eligible if they fulfilled all four criteria: diagnosis of knee osteoarthritis by American College of Rheumatology criteria; age ≥45 years; pain in one or both knees for three months or longer; confirmed knee osteoarthritis with radiological evidence (at least grade 1 according to the Ahlbäck classification)

Inclusion criteria (pragmatic or practical)—Any patients aged 18-65 with non-specific low back pain for 4-52 weeks. Patients with spinal disease were excluded

A limitation of pragmatic trials is that they include very heterogeneous and rarified populations and run the risk of producing an overall estimate of effect that is not applicable to all enrolled patients. In the extreme, they report an intermediate effect when some patients experience large benefit and some none at all. Trialists can overcome this limitation by prespecifying subgroup analyses with statistical tests of interaction to investigate hypothesised effect modification (box 1, consideration 3).

Participants’ expectation serves as an example of subgroup effect in the context of acupuncture trials. Trialists hypothesised that acupuncture would show larger effects in patients who expect a benefit from acupuncture than in those who do not.26 Across therapeutic areas and health outcomes (eg, pain relief, symptom improvement, return to work), results have consistently confirmed this hypothesis: participants who have positive expectations of interventions consistently show a larger effect of acupuncture than the more pessimistic.272829

The repeated finding that patient expectations influence magnitude of effect suggests trialists should assess participants’ expectations at the time of randomisation. Trialists can use validated instruments such as the acupuncture expectancy scale,30 expectation for treatment scale,31 and the expectations for complementary and alternative medicine treatments questionnaire.32

Published acupuncture trials have rarely explored effect modification (web appendix 2, 2.1.2). Trialists can consider exploring effect modifiers when designing acupuncture trials with sufficiently large sample sizes and, if they identify possible subgroup effects, assess their credibility criteria using the rigorously constructed and user tested instrument to assess the credibility of effect modification analyses (ICEMAN).33 Choosing a small number of hypotheses, ideally identified as promising (ie, with moderate credibility on ICEMAN) in previous trials, represents an optimal approach (box 5).33

Study of acupuncture versus sham acupuncture for patients with chronic low back pain.34 The study used the expectations for relief scale to measure patients’ expectations at baseline and conducted a subgroup analysis in patients with high versus low expectations. Expectations were a significant predictor of only true acupuncture response (P=0.002), such that those with greater expectations had more substantial pain relief.

We used ICEMAN33 to assess the credibility of potentially relevant effect modification:

Definitely yes: subgroups (patients’ expectation) were prespecified in published protocol, and publications included an explicit statement that they had correctly predicted the direction of effect modification

Strong support: the main result was consistent with prior solid evidence directly applicable to the clinical scenario

Chance may not explain: P=0.002 (interaction P values ≤0.01 and >0.005)

Probably yes: three effect modifiers were prespecificed but applied to other outcomes than specified in the protocol

Definitely yes: The analysis was based on a full continuum—eg, assuming linear or logarithmic relationships. This study used the Expectation for Relief Scale (ERS) to measure patients’ expectations at baseline and analysed the effect of expectations as a continuous variable using ANCOVA.

Yes, probably increase credibility: the effect modification persisted after adjustment for other potential effect modifiers (interventions and psychopathology)

High credibility very likely: none of the response options definitely or probably reduced credibility

We judged the credibility of the potential effect modification as high considering a small number of clearly a priori hypotheses with a specified direction; support from previous evidence; a low P value in the test of interaction; and adjustment for other potential subgroup effects

The effect of acupuncture may differ depending on various characteristics of the acupuncturists (box 1, consideration 4). These include education and training (whether they had systematic training such as undergraduate, graduate, diploma training, or short term training), length of practice, and how well the acupuncturists perform a particular complex technique (eg, fire needles and intradermal needle).

Differential expertise can create a systematic difference353637 across interventions and between trials.38 A bibliometric study investigated 7085 Chinese randomised trials of acupuncture and found that over 20% of the trials compared one acupuncture treatment with another or with an alternative treatment that required expertise (box 6).3

Consider a randomised controlled trial in which 80 participants with post-stroke pseudobulbar palsy were randomised to an acupuncture intervention—a quick needle insertion at the Aqiang point—or a comparator of routine acupuncture in combination with western medicine.39 Clinicians implementing the intervention needed to perform a quick needle insertion with strong stimulation at the Aqiang point (located in the anterior midline of the neck, between the thyroid cartilage and cricoid cartilage) that may induce the patient to cough violently. In the comparator group, clinicians needed only to stimulate CV 23 (Lian quan) point with the standard acupuncture manipulation. If practitioners in the trial had a greater skill or more experience in performing the special acupuncture techniques, the results might not apply to those with less expertise in the Aqiang point approach.

This trial failed to find a difference in swallow function between the two groups, but it did not establish the expertise of those administering the more challenging intervention. The effects of expert administration therefore remain uncertain, although the results may be representative of suboptimal expertise in the community. The trial’s small sample size might also explain the failure to detect a difference.

In a patient level analysis with 9990 patients treated by 2781 physicians in Germany, neither training duration nor experience affected the extent of the acupuncture effect.40 Unfortunately, exploration of the effect of expertise is limited because acupuncture trials often fail to report this information (web appendix 2, 2.1.3 and 2.2.1).

Whether a trial has explanatory or pragmatic objectives affects trialists’ decisions about the extent of expertise. A trial that primarily aims to show whether an acupuncture treatment can work under ideal conditions will choose the most expert practitioners. On the other hand, trialists primarily aiming to establish the effect of treatment in everyday practice will select clinicians with typical levels of expertise (box 7).

In this explanatory mechanistic trial all participating acupuncturists had at least 10 years of experience. Having highly skilled acupuncturists perform the intervention may enhance treatment effects but raises questions for a patient consulting a practitioner who has just completed acupuncture training

In this pragmatic trial, acupuncture was delivered by 67 physiotherapists trained by the Acupuncture Association of Chartered Physiotherapists (35 hours of training), with three to over 10 years of practice. A positive result of such a trial would suggest a widely applicable intervention; a negative result would leave the question of the effect with more expert practitioners uncertain

One way to deal with the issue of expertise is by using a trial that randomises by expertise. For example, investigators interested in the relative effect of Japanese meridian therapy versus Korean constitutional acupuncture might randomise patients to clinicians who specialised in meridian therapy or those who specialised in Korean constitutional acupuncture. Such expertise based designs may enhance applicability, encourage practitioners’ participation in trials,43 result in less crossover,35 and provide patients the best quality care. To date,44 no acupuncture trial has used an expertise based design, but it is an attractive option for future trials of alternative acupuncture approaches that take a practical perspective.4546

In any case, trialists should report the expertise of the acupuncturists who administered intervention and comparator treatments. More transparent reporting will enable further investigation into how expertise affects the effectiveness of acupuncture treatment.

The effect of acupuncture may differ depending on the choice of techniques, including underlying acupuncture theory (eg, western medical acupuncture, traditional Chinese medicine acupuncture, dry needling) and type of acupuncture (manual versus electroacupuncture).

The multivariate analysis in our systematic survey15 found that penetrating types of acupuncture (manual and electroacupuncture) showed a larger treatment effect than non-penetrating (transcutaneous electrical acupoint stimulation, laser, acupressure) acupuncture. In addition, a higher frequency of acupuncture treatment sessions was associated with larger effects than a lower frequency. Trialists can establish an expert committee to define the optimal strategy for acupuncture related details (eg, acupoints selection, stimulation technique) (box 1, consideration 5).

Blinding refers to those involved in randomised trials being unaware of treatment assignment after randomisation. Blinding is relevant to participants, clinicians, data collectors, outcome assessors, and data analysts. Blinding of clinicians and patients avoids non-specific effects and bias arising from co-intervention; blinding of others minimises bias in the assessment of outcome. Blinding clinicians and patients raises both theoretical and practical challenges, but to minimise bias trialists should blind data collectors, outcome assessors, and data analysts (box 1, consideration 6, web appendix 2, 2.2.2(1)).

Although legitimate scepticism exists about tests of blinding, in seven of 10 studies we identified in which authors tested for blinding (one to four times during the trial), study participants were unable to distinguish active and sham treatment, providing potentially useful reassurance of the success of blinding.15 Thus, the choice of whether to conduct tests for blinding remains, in our view, a matter of investigator judgment in which both options have merit.

In drug trials, which are closer to a explanatory or mechanistic design, trialists often use a placebo to achieve blinding. Acupuncture trials, however, face multiple challenges when using placebo or sham acupuncture for blinding.15 It is nearly impossible to blind clinicians who deliver acupuncture, although one device exists that allows practitioner blinding but limits the acupuncture to a rather superficial version (maximum 5 mm insertion).45 Participants who have had previous acupuncture experience may also be hard to blind.4748

Whether to attempt blinding in acupuncture trials depends on the hypotheses and objectives of a trial. Here, we will refer to a treatment’s biological effects as specific effects and placebo effects as acupuncture’s non-specific effects. Trialists examining explanatory questions should include sham acupuncture control with adequately blinded participants to differentiate acupuncture’s specific and non-specific effects.45

The results of such explanatory studies require careful interpretation, however. The clinical and basic science literature support the possibility of specific effects generated by sham acupuncture4950— that is, a failure to show a difference between real and sham acupuncture may be because the sham has effects closely related to that of the intervention. Therefore, when using sham control, trialists must consider the possibility of a specific effect generated by the sham and thus underestimating the effects of acupuncture in clinical practice compared with no intervention or other interventions such as drugs.

Previous studies of shams have focused on pain and chronic pain and conducted univariable analyses (web appendix 2, 2.2.2 (2)). We used multivariable analyses in our systematic survey, adjusting for other potential factors such as treatment frequency, flexibility of the acupuncture regimen, and sample size. Like a previous systematic review,51 we found the type of sham did not influence acupuncture’s effect.15 This is perhaps unsurprising considering the finding of meta-epidemiological studies in areas beyond acupuncture: results of such studies have proved inconsistent, with one recent thorough review showing no systematic effect of blinding.52

Considering all the evidence, the effect of the type of sham remains uncertain and might vary with the type of medical condition. Over 90% of acupuncture trials focus on pain, quality of life, function, and other symptoms. These are all subjective outcomes in which blinding may be particularly important.15 Trialists should therefore carefully consider the desirability of a sham in randomised trials investigating efficacy and, if desirable, the nature of the sham in relation to their trial’s specific objectives. Trialists should also be aware that the use of non-penetrating needle shams mandates the use of the same device in the real acupuncture group. These devices may impede real acupuncture treatment effects (box 1, consideration 6; box 8).55

Trial 1—Patients with knee osteoarthritis were assigned to true acupuncture plus education and attentional therapy, sham acupuncture plus education and attentional therapy, or education and attentional therapy alone. Sham acupuncture (combination of non-insertion and needle-insertion sham acupuncture) was received with two needles at sham points and tapping at nine real points.53

Trial 2—Patients with knee osteoarthritis were assigned to drug treatment alone, drug treatment plus true acupuncture, or drug treatment plus sham acupuncture.24 Sham acupuncture was administered with the same duration and frequency and by the same specialist who performed the true acupuncture. Retractable needles were placed into small adhesive cylinders to support the needles but did not perforate the skin. The acupuncturist placed the needles at the same points as the non-sham group and used the same pairs of electrodes to simulate the electrical connection.

Trial 3—Patients with knee osteoarthritis were randomised to intensive acupuncture or sham acupuncture.54 Sham acupuncture (combined non-insertion and needle insertion sham acupuncture) was achieved by choosing acupoints that were away from the conventional acupoints or meridians, superficially penetrated (2-3 mm in depth), and without manipulation of the needles after insertion for “de qi” responses.

Trial 4—Patients with persistent non-specific low back pain received either a short course of traditional acupuncture or usual care only. Patients in the usual care group received NHS treatment according to their general practitioner’s assessment of need.25

Acupuncture therapies are multifaceted interventions that incorporate patient-practitioner interactions, treatment theories, tools to deliver the stimulations, manipulation techniques, and points selection. The trial objectives should determine the flexibility of the intervention or control.

Suppose trialists aim to assess the maximum treatment effect of a particular acupuncture technique or regimen (closer to explanatory approach). In that case, they should specify that practitioners use a highly standardised approach to administer both the intervention and the control. In-person or video training for all participating practitioners regarding types of needles, the number of needles, needling details (eg, direction, depth of insertions, needle manipulation), needle retention time, and responses (eg, de qi or muscle twitches) may facilitate standardisation and may be particularly important in multicentre studies. Trialists can monitor standardisation in participating centres by visiting sites regularly or reviewing the recorded video reports. In addition, standardisation should include the number, frequency, and duration of treatment sessions (box 1, consideration 7; box 9, example 1).

Example 1: Postmenopausal women with early stage breast cancer taking an aromatase inhibitor randomised to acupuncture, sham acupuncture, or waiting list control56

Acupuncture intervention—A group of acupuncturists developed a standard operating protocol based on previous acupuncture studies for aromatase inhibitor related arthralgias adhering to the STRICTA recommendation. Trialists described the frequency of treatment, acupoint prescription, whether “de-qi” occurred, depth of needle insertion, and acupoint manipulation in detail

Comparison (sham acupuncture)—The sham acupuncture consisted of a standardised prescription of minimally invasive, shallow needle insertion using thin and short needles at the regimen of non-acupuncture points. The sham acupuncture protocol also included joint specific treatments and an auricular sham based on application of adhesives to non-acupuncture points on the ear

Example 2: A pragmatic acupuncture randomised controlled for the management of chronic low back in older adults57

Acupuncture intervention—An acupuncture advisory panel produced a consensus acupuncture intervention protocol. Acupuncture point sites must include both local and distal points with 6–20 insertions per session, but selection of specific acupoints was at the practitioner’s discretion. The needle retention time and length of the session ranged between 0 and 40 minutes and 45-60 minutes, respectively

Example 3: Patients visiting emergency departments with an ankle sprain, migraine, or lower back pain were randomised to acupuncture alone or combined with pharmacotherapy58

Comparison (pharmacotherapy)—Pharmacotherapy can be any first or second line analgesia drug.

The example of the highly standardised intervention and comparator reflects an explanatory goal for the study. The studies with flexible intervention and comparator reflect a pragmatic goal, and inferences from these studies might be highly applicable to clinical practice.

Trialists with primarily pragmatic objectives will take a different approach. They will probably recruit a wide variety of clinicians representing the variability in technique in the community and instruct practitioners to use their usual treatment approaches (box 9, examples 2 and 3).

The multivariable analysis in our systematic survey found that single centred randomised trials of acupuncture have larger treatment effects than multicentre trials.15 The greater standardisation in single centres might be responsible for the larger effects.

Reporting all technical details of the intervention and controls transparently according to the STRICTA recommendations would enhance the interpretation and applicability of acupuncture trials (web appendix 2, 2.1.4).1059

Although it is often inevitable that some trial patients do not attend all treatments, how this affects inferences from the evidence may vary depending on your perspective. When patients face the decision to use acupuncture therapies, they will be interested in the effectiveness and safety of the treatment. Thus, substantial non-adherence limits the applicability of the trial from the patients’ point of view.

From a public health perspective, however, decision makers might be more interested in the treatment effect in a population with the suboptimal adherence that will actually occur in typical clinical practice. The non-adherence, if indeed typical of what happens in population implementation, will actually strengthen applicability from the public health point of view.

Thus, when designing a trial that primarily takes the individual patient’s perspective, trialists should consider implementing strategies to achieve optimal adherence. One strategy is to engage patients to help design the trial. Patients can identify and improve onerous trial design, therefore increasing participation.60 Another strategy is to include a run-in period before randomisation and only randomise patients who are highly compliant in the run-in.

Including a run-in period is highly feasible in drug trials when patients may be offered a placebo during the run-in or when the trial is focused on morbid or fatal events and short term exposure will have no influence on the outcome. The situation is more complicated in acupuncture trials focusing on symptoms in which initial exposure to treatment during a run-in may complicate inferences about what occurs after randomisation. This is probably why acupuncture trials rarely use run-in periods to enhance adherence.

As an alternative to run-ins, acupuncture trials can assign tasks to patients before randomisation to assess their adherence (web appendix 2, 2.2.3). Trialists can select patients based on completeness and timeliness of fulfilling these tasks. Trials could also provide additional support and adequate incentives to improve adherence ( box 1, consideration 8; box 10).4562

Patients in the acupuncture group could have up to 20 acupuncture treatments by one of four acupuncturists over six months. The acupuncturist and patients decided treatment dates together, respecting the treatment frequency to increase compliance. Patients were paid $35 (£26; €31) for their baseline visit and $25 to complete each follow-up visit. Those who completed all six visits were paid an additional $50.

If trialists take a public health perspective when considering patients’ adherence, there is no need to implement strategies to increase adherence. The minimal enforcement approach closely resembles standard practice and provides a reasonable estimation of how treatment effects might be in the setting in which trials are conducted.

Trialists with mainly mechanistic objectives can usually focus on surrogate outcomes (eg, blood pressure, heart rate) to estimate the effect on patient important outcomes. If trialists’ have a pragmatic objective primarily of guiding healthcare decision making, they should usually select patient important outcomes.63 Trialists can consider taking advantage of minimal sets of agreed and standardised collection of outcomes in their clinical areas, known as core outcome set, on which investigators have agreed (https://www.comet-initiative.org/, web appendix 2, 2.2.4 (1)).646566 Including all outcomes in a core outcome set will ensure maximum applicability of trial results to patients, the public, healthcare professionals, and other decision makers. In addition, trialists can consider outcomes recommended by research organisations and medical societies’ guidelines on conducting trials for specific diseases (web appendix 2, 2.2.4(2)). 646567

As well as using symptoms and functional outcomes trialists should increase the use of quality-of-life outcomes (web appendix 2, 2.1.5). Including major morbid events (eg, serious cardiovascular events, hospital admission, infections) might greatly enhance the importance and applicability of acupuncture trials when relevant but would likely require far larger sample sizes than is typical of current trials (web appendix 2, 2.2.4 (3)). Trials should also report adverse events outcomes (box 1, consideration 9; box 11).

This trial measured forced expiratory flow in first second (FEV1), forced vital capacity (FVC), and peak expiratory flow rate (PEFR) at 0, 5, 10, 15, 20 minutes after treatment. As the trial had an explanatory function trialists used biological or surrogate outcomes

This trial used symptoms and dysfunction as the primary outcomes and disability, utilisation (provider visits, radiological procedures, operations, and hospital admissions), and cost as secondary outcomes. On a 0 to 10 scale, patients rated how “bothersome” their symptoms (back pain, leg pain, and numbness or tingling) were. Patients reported their dysfunction using the modified Roland disability scale. Blinded interviewers measured all outcomes at 4, 10, and 52 weeks after randomisation.

The aim of the study was to inform clinical or policy decision making. The follow-up selection informs the long term impact of acupuncture intervention. It also sheds light on the possible need for “booster treatments” or “maintenance treatment,” therefore helping to achieve practical objectives.

Acupuncture effects often emerge immediately or shortly after treatment sessions, although specific timing depends on the disease or conditions acupuncture treats and the outcome measures. For example, acupuncture shows a 20% reduction in chronic low back pain after 4-5 sessions,69 and it might take up to three months to show effects on post-stroke motor functions.70 When investigators with explanatory objectives primarily, short follow-up times are likely to be optimal. Such design choices do not answer the question of interest to patients—can I expect a longer term benefit from acupuncture? Thus, acupuncture trials with mainly pragmatic objectives are likely to choose the longer follow-up times most important for clinical or policy decision making.71

Acupuncture trials published in the past five years mainly used short to medium term follow-up times (web appendix 2, 2.1.6). Trialists should take into account the course of disease or condition for optimal follow-up times. In many instances, when designing pragmatic or practical trials, trialists need to consider measuring outcomes after six months or longer to optimally inform clinical decision making (box 1, consideration 10; box 11).

Patients are among the most critical stakeholders in clinical care and health research. Involving patients when designing trials can enhance the research question’s relevance and importance, which helps to improve adherence (box 1, consideration 11).727374 Furthermore, having patients involved as members of the trial team can facilitate the understanding of research findings and therefore facilitate dissemination and application.7374

As complex interventions, acupuncture therapies focus on patient engagement, though this is rarely included in randomised trials. Previous guidance on practical acupuncture trials also recommended engagement of relevant stakeholders (eg, practitioners, clinicians, patients, payers, researchers).13 Our systematic survey15 showed that over half of the acupuncture trials did not mention even the minimal extent of patient involvement—signing the informed consent form.

To empower patients and conduct more patient centric research, acupuncture trials should consider informing, educating, discussing, engaging, and partnering with patient engagement organisations such as Clinical Trials Ontario, which helps trial teams engage with patients.7576

We have produced guidance to consider before and during the design of acupuncture trials to deal with the unique challenges and the most prevalent concerns in current acupuncture trials (box 1). With the rapid growth of published acupuncture trials and upcoming funding opportunities worldwide in integrative medicine, failure to enhance the rigour of acupuncture trials will result in additional research waste.77 This guidance highlights the importance of granting agencies and researchers carefully considering the need for research in the trial planning and funding processes. Our guidance will help both researchers and granting agencies consider the critical decisions linking trial design and its inferences and application to produce trials that facilitate healthcare decision making.

We thank Qi Zhou, McMaster University, Canada, for statistical and data interpretation support from the survey studies; Klaus Linde, Technical University of Munich, Andrew J Vickers, Memorial Sloan-Kettering Cancer Center, and Kim Bennell, University of Melbourne, for feedback on the manuscript; Yu-Tong Fei, Hui-Juan Cao, Ru-Yu Xia, Qian-Yun Chai, Chang-Hao Liang, YuTing Feng, and Yi-Ran Du, from Centre for Evidence-Based Chinese Medicine and Dongzhimen Hospital Beijing for several key references manusript; and Jie Wang, Guang’anmen Hospital, for methodological feedback.

Contributors: Y-QZ and R-MJ contributed equally to this work. Y-QZ and X-HJ are the initiators of this series of articles, defined the scope, content, structure, drafted the manuscript, and coordinated the research team. R-MJ identified the acupuncture trial examples, discussed the manuscript’s structure, revised tables and main text, organised references, and coordinated the research team. GG was involved in planning this series of articles and designing the content and structure of this article. X-HJ, Y-QZ, GG, LT, CW, J-PL, R-MJ, and LL are steering committee members. They discussed the structure and content of the paper in four teleconferences and revised versions of the manuscript. JJM, LL, X-HJ, and DPR discussed and debated the paper’s content at the Society of Acupuncture Research 2021 international research conference and contributed further to the content. W-JG provided statistical and data interpretation support for the survey studies. All authors reviewed, revised, and contributed to the creation of the manuscript. All authors approved the final version of the manuscript. X-HJ is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: The China Academy of Chinese Medical Sciences (CACMS) Innovation Fund (CI 2021A03503 and CI2021B011) and “ The Belt and Road” Traditional Chinese Medicine Cooperation Project of China Academy of Chinese Medical Sciences (GH201901) supported this work. The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Competing interests: All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare: CW received a grant at the University of Zurich from the Kelm Foundation for a randomized controlled trial on acupuncture for dysgeusia during chemotherapy. CW is an unpaid board Member of the Society of Acupuncture Research. KJS received a grant from NIH (National Center for Complementary and Integrative Health). KJS seats on a Data Safety Monitoring Board or Advisory Board from NIH-DoD (Department of Defence)-VA(Veterans Affairs) Collaboratory DSMB, and Advisory Board of the National Center for Complementary and Integrative Health of the US NIH with honorarium. MC receives royalties from non commercially used Medical Acupuncture textbooks.MC works as the Medical Director of the British Medical Acupuncture Society (BMAS) full time. DPR is a volunteer vice president of the Canadian Arthritis Patient Alliance, a patient-led and run arthritis organization that accepts arms-length funding from pharma companies, and an employee of Five02 Labs Inc, under Clinical Trials Ontario to support its patient and public engagement efforts. DPR received an honorarium attend meetings and travel support from Eli Lilly Canada for her speech about living with arthritis and arthritis patient organizations in Canada. MW received grants from the NIH, Sanofi, Astrazeneca Teva, and Jin Hua Foundation Cohero. MW received consulting fees from AstraZeneca, Amgen, Glaxosmithkline, Sanofi, Genzyme, Regeneron, Boehringer Ingelheim, Novartis, genentech, Pulmatrix, Teva, Equillium, cytoreason , Restorbio, Cohero Health, cerecor, incyte, sound biologics, and kinaset. MW received grants for payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing or educational events from AstraZeneca, Glaxosmithkline, Sanofi, and Regeneron. PJD received grants from Abbott Diagnostics, Roche Diagnostics, and Siemens, the monitoring device of CloudDX, and the medical device of Philips Healthcare. PJD reports as a Member of the Scientific Advisory Board of Bayer and Quidel. The study was supported by the China Academy of Chinese Medical Sciences Innovation Fund (CI 2021A03503 & CI2021B011) and “The Belt And Road” Traditional Chinese Medicine Cooperation Project of China Academy of Chinese Medical Sciences; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work of all authors.

Patient and public involvement: Dawn Richards, a patient with rheumatoid arthritis and osteoarthritis and director of patient and public engagement of Clinical Trials Ontario, and Eun-Kyung Anna Kim, a patient representative from Virginia University of Integrative Medicine, reviewed, revised, and provided feedback on the manuscript.

Provenance and peer review: Commissioned; externally peer reviewed.

This article is part of a collection funded by the special purpose funds for the belt and road, China Academy of Chinese Medical Sciences, National Natural Science Foundation of China, the National Center for Complementary and Integrative Health, the Innovation Team and Talents Cultivation Program of the National Administration of Traditional Chinese Medicine, the Special Project of “Lingnan Modernization of Traditional Chinese Medicine” of the 2019 Guangdong Key Research and Development Program, and the Project of First Class Universities and High-level Dual Discipline for Guangzhou University of Chinese Medicine. The BMJ commissioned, peer reviewed, edited, and made the decision to publish. Kamran Abbasi was the lead editor for The BMJ. Yu-Qing Zhang advised on commissioning for the collection, designed the topic of the series, and coordinated the author teams. Gordon Guyatt provided valuable advice and guidance.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Register for alerts If you have registered for alerts, you should use your registered email address as your username

If you are unable to import citations, please contact technical support for your product directly (links go to external sites):

Thank you for your interest in spreading the word about The BMJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.