(%)
Of the 329 identified as feasibility studies:
Therefore, from the 160 records with known feasibility outcomes, 133 (83%) concluded that the RCT was feasible and 27 (17%) that the RCT was not feasible. The interrater reliability across the three rating points was 84%.
The breakdown of the results by topic and setting are presented in Tables Tables2 2 and and3, 3 , albeit based on small numbers in Table Table2, 2 , where there were at least 10 studies reported for a specific research topic, the feasibility rate ranged between 79% and 100% for individual research topics, and in Table Table3, 3 , the feasibility rate ranged between 73% and 93% for research setting.
Breakdown of research topic area and whether RCT was feasible or not feasible for studies with known outcomes
Research topic | Total ( = 160) (% of total) | RCT feasible ( = 133) (% of topic) | RCT not feasible ( = 27) (% of topic) |
---|---|---|---|
Cancer | 19 (12) | 15 (79) | 4 (21) |
Circulatory System | 19 (12) | 19 (100) | 0 (0) |
Digestive System | 4 (3) | 3 (75) | 1 (25) |
Ear, Nose and Throat | 0 (0) | 0 (0) | 0 (0) |
Eye Diseases | 2 (1) | 2 (100) | 0 (0) |
Genetic Diseases | 0 (0) | 0 (0) | 0 (0) |
Infections and Infestations | 4 (3) | 1 (25) | 3 (75) |
Injury, Occupational Diseases, Poisoning | 2 (1) | 0 (0) | 2 (100) |
Mental and Behavioural Disorders | 39 (24) | 33 (85) | 6 (15) |
Musculoskeletal Diseases | 7 (4) | 5 (71) | 2 (29) |
Neonatal Diseases | 1 (1) | 1 (100) | 0 (0) |
Nervous System Diseases | 9 (6) | 8 (89) | 1 (11) |
Not Applicable | 8 (5) | 5 (63) | 3 (38) |
Nutritional, Metabolic, Endocrine | 15 (9) | 13 (87) | 2 (13) |
Oral Health | 1 (1) | 1 (100) | 0 (0) |
Pregnancy and Childbirth | 5 (3) | 5 (100) | 0 (0) |
Respiratory | 4 (3) | 3 (75) | 1 (25) |
Signs and Symptoms | 10 (6) | 9 (90) | 1 (10) |
Skin and Connective Tissue Diseases | 1 (1) | 1 (100) | 0 (0) |
Surgery | 6 (4) | 5 (83) | 1 (17) |
Urological and Genital Diseases | 4 (3) | 4 (100) | 0 (0) |
Breakdown of research setting and whether RCT was feasible or not feasible for studies with known outcomes
Research Setting | Total ( = 160) (% of total) | RCT feasible ( = 133) (% of setting) | RCT not feasible ( = 27) (% of setting) |
---|---|---|---|
Community | 3 (2) | 2 (67) | 1 (33) |
GP practices | 15 (9) | 11 (73) | 4 (27) |
Home | 1 (1) | 1 (100) | 0 (0) |
Hospitals | 80 (50) | 69 (86) | 11 (14) |
Internet | 0 (0) | 0 (0) | 0 (0) |
Not specified | 14 (9) | 13 (93) | 1 (7) |
Other | 37 (23) | 30 (81) | 7 (19) |
Schools | 10 (6) | 7 (70) | 3 (30) |
Figure Figure1 1 shows the specific reasons RCTs were deemed not to be feasible, irrespective of research topic area or research setting, with patient recruitment clearly being the most common reason and reported for 18 (60%) of the 27 feasibility studies reporting the RCT to be not feasible.
Reasons why RCTs were not feasible
The largest outcome group was ‘unknown if RCT feasible’ ( n = 160) which was where the feasibility study had completed but the published results could not be found. The trial end date of these studies ranged from 1999 to 2019. Sixty three of the 160 studies had trial end dates at least 2 years before our data access and therefore would have had ample time to publish. This suggests that there is a large amount of under-reporting of results of feasibility studies which is consistent with the general under-reporting of trials [ 14 – 18 ].
It was not possible to determine whether certain research topics and/or research settings are more associated with feasibility because we found too few studies reporting that the RCT was not feasible. Irrespective of research topic or research setting, there were consistent reasons why RCTs were considered not feasible: patient recruitment, trial design/methods, intervention and outcome measures. Of these, patient recruitment was the most commonly reported reason in the published results paper. This will not be a controversial finding for the research and funding community.
There are several plausible reasons why studies showing that RCTs are not feasible do not often appear in the literature. First, whilst feasibility studies are typically smaller than RCTs, it is clear from RfPB data [ 12 ] that feasibility studies are still a large investment of resource costing on average £219,048 and taking 31 months. Research teams may be confident (and persuasive) enough to apply for the full RCT, perhaps including an internal pilot. Second, it is becoming more routine to include feasibility progression criteria in feasibility designs with ‘red, amber, green’ pre-specified assessments to indicate feasibility. Many of the studies we reviewed did not use progression criteria or used unclear criteria. Therefore, it is very plausible that some studies were optimistic in reporting the RCT to be feasible. We also noted examples where researchers had reported the RCT feasible subject to such substantial changes that it was debatable what relation the feasibility study could claim to any eventual RCT. Third, the bar to demonstrate feasibility may be artificially low if studies are not identifying and addressing the specific challenges in the clinical field of interest. This may be accentuated if feasibility studies rely on a formulaic design [ 8 ]. Fourth, the team may be reluctant to publish the conclusion that a trial is not feasible. Historically, there have been challenges publishing ‘negative results’, but recent advances such as better reporting guidance (CONSORT) and specialist journals such as Pilot and Feasibility Studies have helped to address these challenges. Looking at the 160 studies that apparently had no results published, 63 had finished more than 2 years before our data access and thus would have had ample time to publish. The remaining 97 studies may well go on to publish non-feasible conclusions in future. Therefore, future reviews of feasibility studies might show a more complete publication picture.
There were some challenges in reviewing the ISRCTN records. Firstly, although we used a clear definition of what constitutes a feasibility study, to some extent, we were retro-fitting a definition to previously funded research. Nevertheless, we took the view that any study that explicitly stated it was preparing for a RCT, and had typical feasibility outcomes, was eligible for inclusion. One particular challenge was the number of studies that identified as pilot studies yet had no explicit plan to conduct a further RCT. Some included a mix of typical feasibility study outcomes and typical pilot study outcomes. These were typically excluded if they were primarily exploratory studies or even underpowered trials. Another challenge from the ISRCTN records was that we found self-identified feasibility studies which were examining the feasibility of implementation of a particular treatment/service rather than examining whether a future RCT was feasible. Studies that were not planning to progress to a RCT were excluded but were often challenging to identify.
It is clear from the data that more feasibility studies, or at least more studies which resemble current definitions of feasibility studies, are being conducted, with over 97 feasibility studies completing in 2018 compared with 21 10 years earlier and two 10 years before that. This follows the trend of more focus on feasibility studies with activity such as more research, specialised journals and better guidance on designing, conducting and reporting feasibility studies. It would be interesting to understand why more feasibility studies are being conducted. Could it be that funders of RCTs require more evidence that expensive RCTs will be successfully delivered? Perhaps there are also more funding opportunities for feasibility studies with NIHR programmes like RfPB becoming well known for supporting such studies. It is likely that more feasibility studies are now included on trial registries and were therefore discoverable.
Whilst the majority of the feasibility studies with reported outcomes are feasible, it would have been useful to have known how many studies progressed to RCT and how long they took to progress to have a better-informed view of the utility of feasibility studies. Whilst we noticed that some ISRCTN records contained both the feasibility study and subsequent RCT in a single ISRCTN record, it was not clear how systematically these records were updated. It would be useful if trial registries collected more data including links between feasibility studies and RCTs and ultimately outcomes from RCTs. Knowing the full picture from feasibility study progression to RCT competition and study outcome would allow a more informed view on how feasibility studies contribute to the trials pathway.
It is reasonable to assume that if the feasibility rate is too high, then researchers and research funders are contributing to the waste in the system by conducting studies which will often show the RCT is feasible. In such cases, it may be more appropriate to go straight to the RCT and accept that a relatively small percent will inevitably fail, whilst noting that feasibility studies do help address other uncertainties which may not have been foreseen. Conversely, if the feasibility rate is too low, it may indicate that researchers and funders are too optimistic and commit to studies which are not likely to progress down the trials pathway, although based on the current analysis, there is insufficient evidence to suggest that this scenario is occurring. However, it is acknowledged that many useful aspects and insights are gained via feasibility studies and that those demonstrating the RCT to be feasible often do once addressing challenges encountered via the feasibility study.
There must be a point where if the feasibility rate is too high it would be more cost and time effective to fund studies as RCTs and accept that a certain percentage will inevitably fail. However, for those that do fail, a greater amount of time and/or funding will have been saved by successfully completed RCTs which avoided the feasibility and/or pilot step. The cost of the ISRCTN feasibility studies was not reported, but based on the previous RfPB review [ 12 ], the average feasibility study cost £219,048 and took 31 months. If the 83% feasibility rate of ISRCTN studies is correct, then that is a substantial amount of funding and time taken up by the feasibility studies when only approximately 1 in 5 will show the RCT to not be feasible. Using the average cost of feasibility studies (£219,048) and RCTs (£1,163,996) from our previous review of RfPB studies [ 12 ], we can begin to estimate what an appropriate feasibility rate may be. RfPB feasibility studies were shown to be feasible in approximately two out of three studies, and therefore approximately £657,144 was spent on feasibility studies to save up to approximately £1,163,996 for the RCT which was not feasible. If we apply the same estimates based on the ISRCTN feasibility rate of 83% (approximately four out of five studies), then approximately £1,095,240 needs to be spent on feasibility studies to save up to approximately £1,163,996 for the RCT which was not feasible. In addition to cost, there is also a time addition as each feasibility study takes an average 31 months to complete. A feasibility rate of 83% appears too high and would suggest feasibility studies are wasteful, whereas 64% may be considered more reasonable.
How many of these RCTs genuinely needed a feasibility study and then focussed on the actual uncertainties instead of adopting a generic design? Perhaps even more important is how many of those 83% which demonstrate feasibility actually progress to RCT? Feasibility studies which show the RCT feasible but which do not progress to RCT, for various reasons, could also be considered to be adding to research waste, especially if these feasibility studies are not really needed in the first place. The answer to this question is made challenging by historic poor reporting and publication rates of feasibility studies.
This study raises, but cannot address, the question of what would be an acceptable ‘success’ rate for feasibility studies: should most demonstrate feasibility of the RCT or should more demonstrate the RCT not feasible? How much risk do funders and researchers want to take? Might shorter and more cost effective feasibility studies be more informative? Perhaps the view that feasibility studies are essential before conducting a RCT is leading to the design of studies which are likely to ‘succeed’ and therefore lack equipoise or do not focus on the most important uncertainties which need to be addressed in relation to the specific trial. As shown by the existing literature, it is often the case that feasibility studies in certain topic areas do not maximise their potential benefit and focus on the key uncertainties [ 8 , 11 ] and instead adopt a generic design.
Although we were unable to answer our initial question, a potentially more important and interesting question is what the rate of feasibility studies demonstrating the RCT is feasible should be. The previous review of the RfPB portfolio [ 12 ] showed that 64% of studies demonstrated the RCT to be feasible and this review of ISRCTN registered studies showed that 83% of studies with known/published outcomes demonstrated the RCT feasible. Are these feasibility rates appropriate and what do they mean for the wider trajectory along the trials pathway?
It is likely that there are insufficient published studies demonstrating that the RCT is not feasible to be able to assess whether some studies may or may not be more feasible than average based on research topic and/or research setting. More discussion is required between researchers, methodologists and research funders on exactly what feasibility studies are aiming to achieve and what proportion of studies should demonstrate feasibility or not and how this relates to the wider research, funder and patient benefit pathway. This will help ensure that feasibility studies maximise the potential to reduce waste in research instead of potentially adding to it.
The sample included a range of studies spanning over 20 years. During this time, research design and conduct has changed including definitions of ‘feasibility study’. To that end, we applied a relatively recent definition of ‘feasibility study’ to historical studies which made reviewing studies challenging. The potential under-reporting of non-feasible studies possibly biased the sample leading to an artificially high feasibility rate.
SZ is an NIHR Senior Investigator Emerita.
BM, JH, DA and SZ conceived the study. The work was undertaken by BM, JH and KK and supported by DA and SZ. Analysis and quality assurance were conducted by BM, JH and KK. All authors read and approved the final manuscript.
This study was funded by the NIHR Central Commissioning Facility (CCF).
Declarations.
Not applicable
BM, JH and KK are employed through NIHR to manage RfPB and its databases. SZ is the current Programme Director of RfPB, and DA was the Programme Director of RfPB until May 2017. The views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health and social Care.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Elad I. Levy, MD, MBA, a principal investigator on the Synchron clinical trial, is holding the engineered precision 3-D printed model of a patient’s anatomy and vascular structures that was created by bioengineers at the Jacobs Institute.
By Ellen Goldbaum
Release Date: September 26, 2024
BUFFALO, N.Y. – Results of the clinical trial that assessed the safety of Synchron’s endovascular brain-computer interface in people with severe paralysis will be presented Sept. 30 at the Congress of Neurological Surgeons (CNS).
The results will be presented by Elad I. Levy, MD, MBA , a principal investigator, SUNY Distinguished Professor and the L. Nelson Hopkins, MD, Professor and Chair of the Department of Neurosurgery in the Jacobs School of Medicine and Biomedical Sciences at the University at Buffalo. Levy also is co-director of the Gates Stroke Center and Cerebrovascular Surgery at Kaleida Health’s Buffalo General Medical Center/Gates Vascular Institute (GVI) and president of UB Neurosurgery (UBNS).
For this abstract, Levy is being awarded the Duke Samson Award, which the CNS gives to recognize the best clinical paper addressing a topic of cerebrovascular surgery.
Now that the initial follow-up of the feasibility study has been completed, patients will begin to be enrolled next year at additional sites in the U.S., one of which will be the GVI in Buffalo.
The COMMAND trial was conducted under the first investigational device exemption awarded by the Food and Drug Administration to a company assessing a permanently implanted brain-computer interface. The early feasibility study assessed safety while evaluating quantified efficacy measures of Synchron’s brain-computer interface device, which allows people with limited to no mobility to operate technology such as mobile devices and computers using their thoughts. The Synchron technology uses digital motor output, or DMO, to use peripheral assist devices to restore lost function.
Through a minimally-invasive endovascular procedure, the device is implanted in the blood vessel on the surface of the motor cortex of the brain via the jugular vein. Once implanted, it is designed to detect and wirelessly transmit motor intent out of the brain, intended to restore the capability for severely paralyzed people to have hands-free control over personal devices.
In the study, two patients were enrolled in Buffalo at the Gates Vascular Institute, two were enrolled at the University of Pittsburgh and two at Mount Sinai Health System.
Levy explains that the technology is designed to give functional independence back to patients who have become paralyzed.
“It takes a person who has to have everything done by a caretaker and it gives them back some independence and functionality through this brain-computer interface,” he says.
“Our group at UB and the GVI has been in the forefront of learning how to navigate complex devices in the venous system of the brain,” says Levy, “so it was a very natural choice for Synchron to come to Buffalo and have us do these initial cases. All of the research that we’ve done pioneering in strokes and, more specifically, in venous anatomy at the Jacobs School, as well as the clinical experience we have at Gates, was the reason we were selected to be one of the first places where this kind of procedure would be done.”
Allison Brashear, MD, MBA , UB’s vice president for health sciences and dean of the Jacobs School, says: “That our UB and Gates Vascular Institute neurosurgeons were selected to do this points to the fact that for the last 20 years, they have been at the forefront of neuroendovascular surgery, delivering paradigm-changing care to patients not only in this region but globally. This continued pursuit of advanced technologies highlights our commitment to innovation and excellence in patient care.”
The strong partnership between the Department of Neurosurgery, the GVI and the Jacobs Institute allowed the Buffalo group to optimize the procedures before touching the patient. The Jacobs Institute is a vascular and neurologic medical device innovation center on the Buffalo Niagara Medical Campus.
Through this partnership, the bioengineers at the Jacobs Institute created engineered precision 3D printed models to scale of a patient’s anatomy and vascular structures.
“This allows us to continue to optimize and practice our procedures,” says Levy, “becoming very familiar with the patient’s brain anatomy so that when we do implant these incredibly expensive devices into these very fragile patients, we can do it flawlessly.”
Rosalind Lai, MD , assistant professor of neurosurgery at UB, who was at the time a UB neurosurgery fellow, says she felt privileged to be part of the team. “It was very exciting to be part of this cutting-edge neurosurgery. It was almost like science fiction,” she says. “It’s so remarkable because they implanted the device endovascularly through a catheter in the neck, through the venous system and up into the superior sagittal sinus at the top of the head. Then they threaded the electrode to the motor cortex in the venous sinus system and landed it there precisely.”
The electrode connects to a telemetry device in the patient’s chest.
Levy notes that this trial has provided invaluable lessons for the team that planned and developed the procedure, including neurosurgical residents and fellows at UB.
“Using computer software planning, we pinpointed with submillimeter accuracy precisely where the implantable stent—what we call the neuroprosthetic—will go in the brain,” says Levy. “This is what our residents, fellows and even our students at the Jacobs School get to see in action: medical technology that really doesn’t exist in mainstream medicine yet. It’s part of this whole revolution in AI that’s happening right now.”
Ellen Goldbaum News Content Manager Medicine Tel: 716-645-4605 [email protected]
We use some essential cookies to make this website work.
We’d like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.
We also use cookies set by other sites to help us deliver content from their services.
You have accepted additional cookies. You can change your cookie settings at any time.
You have rejected additional cookies. You can change your cookie settings at any time.
Research report for the impact evaluation feasibility study of the Taking Teaching Further financial incentive.
Taking teaching further financial incentive: impact evaluation feasibility study.
Ref: ISBN 978-1-83870-570-1, RR1454
PDF , 320 KB , 35 pages
The Department for Education (DfE) commissioned this research to assess the feasibility of an impact evaluation of the financial incentive.
This report includes recommendations for the:
Sign up for emails or print this page, is this page useful.
Don’t include personal or financial information like your National Insurance number or credit card details.
To help us improve GOV.UK, we’d like to know more about your visit today. Please fill in this survey (opens in a new tab) .
COMMENTS
A Feasibility Study is a project assessment that determines its viability and potential success based on technical, financial, legal, and environmental factors. Learn the importance, types, and steps of conducting a Feasibility Study with examples and a report template.
These definitions propose that any type of study relating to the preparation for a main study may be classified as a "feasibility study", and that the term "pilot" study represents a subset of feasibility studies that specifically look at a design feature proposed for the main trial, whether in part of in full, that is being conducted ...
It is the authors' opinion that the experiment is a vastly underutilized research design for feasibility studies. Small-scale experiments that more closely approximate the clinical or community context of an RCT can also be used to test other aspects of intervention feasibility. Questions about safety; optimal dose (treatment intensity ...
Pilot studies are a necessary first step to assess the feasibility of methods and procedures to be used in a larger study. Some consider pilot studies to be a subset of feasibility studies (), while others regard feasibility studies as a subset of pilot studies.As a result, the terms have been used interchangeably ().Pilot studies have been used to estimate effect sizes to determine the sample ...
What is a Feasibility Study in Research? A feasibility study is an in-depth assessment conducted to determine the practicality and viability of a proposed project or idea. It involves evaluating various factors such as technical, economic, legal, operational, and scheduling aspects to ascertain whether the project can be successfully implemented.
A feasibility study is a research that evaluates the viability of a project before approval. It considers technical, financial, market, and organizational factors to determine the project's success, cost, and ROI. Learn how to conduct a feasibility study and its benefits for project management.
A feasibility study is a systematic analysis of a project or business idea to assess its viability and potential for success. Learn about the four types of feasibility studies (market, technical, financial, and operational) and the steps to prepare and conduct them.
This paper provides broad guidance for the conduct of feasibility and pilot studies for implementation trials, which aim to test the effects of implementation strategies on the adoption, integration or uptake of an evidence-based intervention. It covers the aims, methods, design, measures, progression criteria and reporting of such studies, and uses a framework based on the hybrid trial design ...
A pilot or feasibility study is a small study to test or evaluate a research idea, procedure or intervention before a main study. This article reviews the current practice and editorial policy of such studies in medical journals and research proposals.
Rigorous assessment of feasibility for studies being put forward for funding is one way of increasing the likelihood that the studies run in the NHS are well designed and likely to deliver to time and to target. Equally, a record of running successful studies that answer the key research question is a strong metric of success for funders.
A feasibility study is a detailed analysis that considers the critical aspects of a proposed project to determine its likelihood of success. Learn how to conduct a feasibility study, what ...
Nursing clinical research is a growing field, and as more nurses become engaged in conducting clinical research, feasibility studies may be their first encounter. Understanding what they are, how to conduct them, and the importance of properly reporting their outcomes is vital to the continued advancement of nursing science.
This article reviews the literature and theory of feasibility studies for projects that are novel, complex, or unfamiliar. It identifies seven general principles to inform feasibility studies and explores the challenges and implications for project management and research.
Nursing clinical research is a growing field, and as more nurses become engaged in conducting clinical research, feasibility studies may be their first encounter. Understanding what they are, how to conduct them, and the importance of properly reporting their outcomes is vital to the continued advancement of nursing science.
The feasibility study includes: Market research to gauge consumer interest, assess competitor offerings, and estimate potential market share for the target market. Technological considerations, including R&D requirements, production processes, and any necessary patents or certifications.
Feasibility Studies. Feasibility Studies are pieces of research done before a main study. They are used to estimate important parameters that are needed to design the main study. For instance: • standard deviation of the outcome measure, which is needed in some cases to estimate sample size, • willingness of participants to be randomised,
A feasibility study is a crucial step to take before diving into any project and is generally performed during the project initiation phase of project management. It helps identify potential roadblocks, assess risks, and estimate resource allocation; skipping this step can lead to project failure, wasted resources, and financial losses.
In this article, we highlight the distinctive features of a feasibility study, identify the main objectives and guiding questions of a feasibility study, and illustrate the use of these objectives. We synthesized the research methods literature related to feasibility studies to identify five overarching objectives of feasibility studies that ...
Given the significance of a feasibility study in decision making and implementation of the project, many people especially potential investors, financiers or even management lack the practical ...
A feasibility study is an assessment tool that helps determine if a proposed project is successful. It considers technical, economic, legal and operational factors, and provides useful information for the next steps after the study.
A feasibility study is an assessment of the practicality of a proposed project plan or method. Learn what to include in a feasibility report, the types of feasibility studies and the steps to do one for your project.
A feasibility study is an assessment of the practicality of a project or system based on technical, economic, legal, operational and scheduling aspects. Learn the formal definition, common factors and methodology of feasibility studies in project management and engineering.
Feasibility studies are often conducted before committing to a randomised controlled trial (RCT), yet there is little published evidence to inform how useful feasibility studies are, especially in terms of adding or reducing waste in research. This study ...
Research design: A prospective, longitudinal, correlational, single-cohort feasibility study using an investigator-developed survey and intervention field data. Participants and research context: Registered nurses were recruited from an academic quaternary-care medical center and 9 small- to mid-sized regional hospitals within one health system ...
The early feasibility study assessed safety while evaluating quantified efficacy measures of Synchron's brain-computer interface device, which allows people with limited to no mobility to operate technology such as mobile devices and computers using their thoughts. ... All of the research that we've done pioneering in strokes and, more ...
The Department for Education (DfE) commissioned this research to assess the feasibility of an impact evaluation of the financial incentive. This report includes recommendations for the: most ...