Exercise science research has been instrumental in advancing our understanding of how strength affects health, performance, and well-being. However, like any scientific field, it has its shortcomings that can impact the applicability and reliability of its findings. Understanding these limitations is crucial for refining and improving the quality of any coaching practice. To base your coaching solely on scientific research (i.e. "Evidence Based" training) would be a mistake. Through a marriage of the practical and demonstrably effective methods observed in the real world and a healthy respect for research and it's ability to lead us in a positive direction, a coach can engender the best results. Here, I'll explore the specific shortcomings of exercise science research, and why it cannot be used to inform coaching practice in a vacuum.
Small Sample Sizes and Lack of Diversity
Limited Generalizability
One of the most significant issues in exercise science research is the frequent use of small, homogenous sample sizes. Many studies rely on small groups of participants, often drawn from specific populations such as college students, athletes, or individuals from similar backgrounds. This lack of diversity can limit the generalizability of the findings. For example, results derived from a study on young, fit college students may not be applicable to older adults, or individuals with chronic conditions.
Impact on Statistical Power
Small sample sizes also affect the statistical power of a study, reducing the likelihood that it can detect a true effect. This can lead to a higher rate of Type II errors (false negatives), where the study fails to identify a significant effect that actually exists. Consequently, potentially valuable insights may be overlooked, and the reliability of exercise recommendations based on these studies can be compromised.
Variability in Study Designs
Lack of Standardization
Exercise science research encompasses a wide variety of study designs, ranging from RCTs to observational studies and case reports. While this diversity can provide a broad range of insights, it also introduces variability that can make it difficult to compare and synthesize findings across studies. Differences in intervention protocols, measurement techniques, and outcome variables can lead to inconsistent results and make it challenging to draw definitive conclusions.
Short Duration of Studies
Many exercise studies are conducted over relatively short periods, often ranging from a few weeks to a few months. While these short-term studies can provide valuable information about immediate effects, they may not capture the long-term impacts of exercise interventions. Understanding the chronic adaptations and long-term health benefits of exercise requires longitudinal studies, which are more challenging and resource-intensive to conduct.
Measurement Challenges
Subjective Assessments
Exercise science often relies on self-reported data, such as physical activity questionnaires or dietary logs. These subjective assessments can be prone to biases and inaccuracies, as participants may overestimate or underestimate their activity levels or dietary intake. The reliance on self-reported data can compromise the accuracy of the findings and lead to erroneous conclusions about the relationship between exercise and health outcomes.
Objective Measures
While objective measures like accelerometers, heart rate monitors, EMGs and biochemical markers provide more accurate data, they also come with their own set of challenges. These tools can be expensive, require technical expertise, and may not be feasible for large-scale studies. Additionally, there can be variability in how these measurements are implemented and interpreted across different studies, further complicating comparisons and synthesis of research findings.
Further Complications
When considering the results of studies, particularly studies that deal with the efficacy of different movement patterns, there is one exceptionally confounding variable: the complications of the movements themselves. As you'll see in a later paragraph, many studies comparing two movements, such as the Squat (more complex) and the Leg Extension (extremely simple) tend to produce a result a result that favors the more simple exercise. This is not necessarily because the simpler exercises are actually more effective at building muscle, but because none of the (already very short) study window is spent learning the movement pattern. Comparing the effectiveness of two movements across 6 weeks, one of which is still being learned in the second week, is not really as great a comparison as someone saying "the evidence suggests" would like you to believe.
Publication Bias and Research Transparency
Positive Results Bias
Publication bias is a well-documented issue in scientific research, including exercise science. Studies with positive or significant findings are more likely to be published than those with null or negative results. This bias can skew the body of evidence, creating a distorted view of the efficacy of certain exercise interventions. As a result, researchers, practitioners, and the public may develop unrealistic expectations about the benefits of specific exercises or training programs.
Lack of Open Data and Replication
The transparency and reproducibility of exercise science research are critical for verifying results and building a robust evidence base. However, there is often a lack of open data and insufficient replication of studies in this field. Without access to raw data and methodological details, it is challenging for other researchers to validate findings or explore alternative explanations. Promoting data sharing and replication studies can help address this issue and improve the reliability of exercise science research.
Ethical and Practical Constraints
Ethical Considerations
Exercise science research involving human participants must adhere to ethical guidelines, which can limit the types of studies that can be conducted. For example, it would be unethical to expose participants to potentially harmful levels of exercise or to withhold beneficial exercise from a control group over a long period. These ethical constraints can restrict the scope of research and necessitate careful design of experiments to balance scientific rigor with participant safety.
Practical Limitations
Practical limitations such as funding, access to facilities, and participant recruitment can also impact the quality and scope of exercise science research. Studies may be limited by the availability of resources, leading to compromises in study design, sample size, or duration. Additionally, recruiting and retaining participants, especially for long-term studies, can be challenging and affect the feasibility of certain research projects.
Example
One study conducted in 2021 published data that suggested the Leg Extension machine as a better choice than the Smith Machine squat for developing the major muscles of the quadriceps. As you dive in to the methods the study used, any critical thinking mind would discover a myriad of issues with this conclusion. First of all, this experiment featured just 27 participants. This is actually a better number than I was expecting, as exercise science studies typically feature very few participants as I discussed above. 27, as good as that is, is still not a large enough sample size to draw any meaningful conclusions. We have no real background on any of these people, their training history, or other confounding factors. Worse yet, the duration of the study lasted 5 weeks, not nearly enough time to see any meaningful change (yet the authors felt confident saying so). Even more woeful, the measurements were taken with an ultrasound. While this is probably just as accurate as the old cloth tape measure method, it's reliability and accuracy should be questioned.
Now we get to the exercises themselves: a Smith Squat (even to "parallel" as the study claims) and a Leg Extension. What do we know about these exercises? For one, the leg extension is preferred by many trainers because it requires absolutely no coaching. People certainly would feel, upon first use, more comfortable that they could accurately perform a Leg Extension than a Smith Squat. Although many gym newbies choose Smith Squats because they are perceived as easier to perform than barbell squats, which is precisely why they so often are not included in training programs, Smith Squats do require more coaching than Leg Extensions. In practical terms, this means trainees in the study will not go nearly as close to "failure" on the Smith exercise, as they are still learning it in the first few weeks.
But perhaps the most damning part of this study are the things it completely omits: the recovery side of the equation. This is not the fault of the authors, but it is nearly impossible to control this side of the equation. In this study, there are no parameters on what the participants were eating, how much they were sleeping, hydrating, or what other physical activity they were engaging in. This is part of the limiting scope of exercise science research.
The interpretations of this research are also not without issue. Widely across the internet, if you search "squats vs. leg extension" this paper will be cited in reference to why you shouldn't squat if trying to build your quads. This is a heinous misinterpretation, as (noted above) the study did not even use the barbell squat. This is a grossly inaccurate extrapolation of the data and does not purport with the common experience of many strength coaches.
All this to say, Leg Extensions may be better for quad growth, but they also may not be. In any system of training that is not specifically for a bodybuilding contest, it doesn't make much sense to include exercises just for the sake the quads when you can use compound exercises like squats to build the quads, glutes, hamstrings, calves, spinal erectors, etc. at the same time. It's more bang for your workout buck to just squat. That said, if a client ever came to me with bodybuilding goals, Leg Extensions would be something I consider, and would observe its efficacy as we increased the load week to week.
Conclusion
While exercise science research has significantly advanced our understanding of strength training and its benefits, it is important to recognize its limitations. While I have a generally positive outlook on the field, small sample sizes, lack of diversity, variability in study designs, measurement challenges, publication bias, and ethical and practical constraints all still pose challenges to its legitimacy. By acknowledging and understanding these limitations, strength coaches can continue to evolve and provide more reliable and actionable insights for improving health and performance. When paired with coaching practice, exercise science can inform, but should not dictate decision making. Any coach basing their exercise selection and programming based solely on the research is likely misinformed. In my opinion, it is the partnership of lived coaching experience with an eye on the research that leads to the best coaching practices.
Comments