In the third of our series of articles on Managing Data and eClinical solutions for Medical Device Companies, the CRFWEB team discuss the business case for migrating from paper to EDC for Medtech SMEs:
Assessing the true costs of paper v EDC has many variables, and will vary case by case and business by business – we help you look at the whole picture. Sometimes a direct like for like isn’t truly possible and its about assessing all the factors and making an educated decision.
ARTICLE 2 – THE BUSINESS CASE FOR EDC FOR MEDTECH SMEs
Whilst the regulatory and process improvement arguments in favour of EDC for Medtech are largely unquestionable for the larger medical device manufacturer with multiple devices on the market, it’s not necessarily clear for the smaller player with perhaps one or two devices – particularly those with products in the lower clinical risk classifications or products based on relatively minor modifications to well documented technology.
The true cost of EDC for the smaller MedTech SME – higher or lower?
The cost of a clinical investigation will vary dramatically depending on the number of participants, the complexity of the data gathered, the number of sites, where an investigation is being conducted and how it is being supported. So, there will never be a simple, definitive answer to this question, but even where costs appear to be higher for a specific investigation when using EDC for Medtech, there is still a strong case to be made for factoring in the value of longer-term benefits such as automated audit trails, data security and integrity. Undoubtedly too, the many ways in which digitized processes naturally support compliance with regulation only adds to the argument. The reality is that it is hard to read much of the newly specified requirements of MDR (and supporting QMS and risk management standards) and not conclude that EDC is pretty much expected, without that expectation being explicitly stated.
Opportunity cost matters, too with EDC for Medtech
For new products, the available evidence (albeit from pharma) suggests EDC has a significant role to play in reducing time to market and, whilst this opportunity for earlier revenue generation might well be difficult to predict or quantify accurately, it should demand serious attention.
An excellent review by Medidata (05/15 White Paper: Reducing the time to market with an eclinical system) estimates that sponsors can:
- Accelerate study start-up through a 27 percent to 41 percent reduction of study build time
- Reduce study close-out time by 41 percent to 65 percent
In combination, taking time out of a study start-up and close-out can bring forward the clinical study report and regulatory submission date by up to 45 days.
However, it’s fair to say that these less immediately tangible benefits will only receive a respectful hearing if the upfront cost of using EDC for Medtech is not too eye-wateringly high.
Cost savings using EDC – what does the literature say?
The amount of independent research attempting to provide a unbiased analysis of this issue is somewhat thin on the ground and, surprisingly, none particularly recent – most of the papers regularly cited by vendors are over a decade old. This matters of course, as the real cost of any digital solution naturally declines over time (Moore’s Law, again) as the benefits of scale of uptake and later-entrant advantage play in. On the other hand, most of the available data concerns larger scale pharma trials and so needs to be treated with a little caution when trying to tease out conclusions about the validity to med tech device investigations. So, reasons to be both cautiously optimistic…and to reach for the salt spoon.
Where the rubber hits the road
Data Capture – paper (PDC) vs EDC for Medtech
Manual data capture and re-entry is notoriously error prone and is the main driver for ballooning costs of data capture using traditional paper and spreadsheets. This of course has always been a major headache for pharma – a problem in desperate need of a digital solution. As early as 1998, a paper by N. Banik, “Evaluation of EDC versus paper in a multinational asthma trial”, which was presented at the DIA European Data Management Meeting in Berlin, suggested that using EDC could shorten study duration by as much as 30 per cent, decrease time to database lock by 43 per cent, and reduce queries by 86 per cent. This was for a 19-site, 5 country trial with N=226.
- Brown, “Clinical Trials’ EDC Endgame, Grading EDC Vendors and the Road to eClinical Suites,” Forrester Research, September 29, 2004 further suggested projected savings of nearly $350,000 when comparing use of EDC against paper in a Phase II study. These findings were based on a 20-site study, 200 subjects (10 patients/site), and a 12-month study duration plus time for data cleaning. For a phase III trial with 200 sites, 2000 subjects (10 patients/site), and a 24-month duration plus time for data cleaning, the paper also suggested savings of some 67% on operational expenses in excess of $11.4m
A widely cited, closely argued paper by Pavlović, Kern and Miklavčič (Contemporary Clinical Trials, Volume 30, Issue 4, July 2009, Pages 300-316) concludes that:
“For the sample clinical trial considered in our simulation study and our parameters estimations the EDC process decreased data collection costs for 55%. For different scenarios of parameters variations we show that the EDC process may bring from 49% to 62% of savings when compared to paper data collection (PDC) processes.”
For the record, the sample clinical trial referred to in Pavlović et al. had the following characteristics:
- 10 research centres (RC)
- 100 patients per centre (all together 1000 patients)
- CRF contains 10 sections
- Entire CRF contains 1000 data points
- 24 months study duration
- Monitor visits each RC once a month
The following values were then used in the calculations
- Total number of CRF forms (equal to the number of patients) =1000
- Total number of CRF sections = 10,000
- Total number of collected data points = 1,000,000
- Total number of monitor visits = 240
Perhaps unsurprisingly, eager MedTech vendors have been somewhat reticent about the detail when enthusiastically quoting this paper, given the size of the hypothetical study!
However, the paper’s analysis focused on exploring in depth how staff costs declined as a result of efficiency gains in key actions and sub-processes achieved when deploying EDC in comparison to PDC. It goes into considerable detail providing itemising and providing time saving estimates for all of the sub-processes for each of the key actions associated with data capture and management:
- Data gathering at site
- Monitoring
- Data management (data entry, analysis, query resolving, query closing)
Although a little simplistic, a case could be made for sketching out a simple linear function of data points collected vs. staff cost saved, with 0 savings for 0 data points collected up to say 55% savings for a million data points. This idea has some legs because the analysis is focused solely on savings in staff TIME and is therefore independent of actual local staff costs or any other considerations that may have varied significantly over the last decade. Crude? Yes, but a place to start. A better place to start, given the financial outlay involved in clinical trials, would be to purchase the paper!
To repeat, the stated savings above DO NOT include the cost of licensing or other costs associated with EDC deployment such as form design etc. We’ll look at fee structures in the following companion article to this, but any EDC system will require a set of actions and processes that won’t be needed for PDC. These costs will also vary from vendor to vendor and will also be impacted by the decision to use a CRO, but we can review these in broad outline.
PDC vs EDC for Medtech: a deeper dive
Of course, as the amount of data collected increases so will the cost of data entry, monitoring and the cost of management. The amount of data gathered depends on:
- Number of patients in study
- Number of visits of each patient over the course of the study
- Number of CRF pages completed per visit.
As we have seen, Pavlovic et al suggest headline cost improvements when using EDC rather than PDC in the region of 55%. Diving deeper into the paper offers some more relevant pointers for smaller clinical investigations. Specifically, it offers a cost saving analysis for:
- A 12 month trial
- 5 sites
- 50 CRF (with 500 data points)
The analysis of this smaller scale trial suggests the following cost savings when using EDC:
- Data gathering at sites: 52%
- Data monitoring: 40%
- Data management (including entry, analysis, query resolving and query closing): 97.5%
The cost breakdown (as a % of the overall figure) for each of the key activities was as follows:
% Cost Savings
Activity |
% cost of total cost of using PDC |
% cost of total cost of using EDC |
| Data gathering
|
9.7% | 10.4% |
| Data monitoring | 66.2% | 88.3% |
| Data management | 24.1% | 1.3% |
The dramatic difference in both the relative and absolute cost of data management assumes that for PDC, the expected query rate would be 2%, whilst for EDC it was taken to be 0.2% – a 90% reduction.
Overall the savings, when using EDC for Medtech investigations rather than PDC for a study of this size was 55%.
Further evidence
These cost reductions when using EDC rather than PDC broadly reflect with those discussed in a 2010 paper in Applied Clinical Trials, 1 April 2010, “The upfront cost hurdle of EDC”. The trial used to develop costings and indicated savings in this paper is significantly larger and more complex than that used above:
- 30 sites per trial
- 350 subjects per trial
- 25 unique case report forms per study
- Study duration of 14 months
- Costs of $1.50 to $1.75/page for double data entry for paper-based systems, and $1.50 to $1.75/page for printing and mailing costs of paper documents
The breakdown for costings is slightly different in this paper:
- Cost of data entry at sites not directly specified
- Monitoring costs refer only to visits
- Query rate of 1% cited for PDC
The cost of data entry at site is only some 10% of the overall cost and so, while not unimportant, is not a major component of the cost-benefit analysis of moving to EDC. In any event, the cost saving for data entry at sites of c.50% for EDC over PDC is widely accepted elsewhere in the available literature – although it’s important to be aware that there is a strong echo chamber effect at work here.
Monitoring costs refer only to visits in this paper, and suggests that savings of 20% are achievable, where as Pavlovic et al suggest overall savings of 40% are possible when using EDC instead of PDC, when all monitoring functions are included.
Citing data provided from StudyManager, the paper suggests a cost saving of some 90% for cost of managing and resolving queries. This analysis was based on the following parameters:
- A 1% error rate for PDC – all resolved by data management
This error rate of c.1% is broadly in line with that suggested in a research paper from 2008, “Analysis of data query as parameter of quality”, Sonia Khatawkar et al (Perspect Clin Res. 2014 Jul-Sep; 5(3): 121–124. doi: 10.4103/2229-3485.134312
- A 0.5% error rate for EDC – BUT 75% resolved by the software with no need for further intervention, so an effective error rate of 0.125%.
So although the absolute error rate for PDC is half that quoted in Pavlovic et al, the relative error rate for EDC over PDC is very similar – a reduction of 87.5% , rather than 97.5%, so still very significant.
So, returning to the Pavlovic et al model for a small scale trial, we can summarise the savings as follows, but we’ll use the lower figure quoted above for % saving for data management:
% Cost Savings
Activity |
% cost of total cost of using PDC |
% saving when of using EDC |
| Data gathering | 9.7% | 52% |
| Data monitoring | 66.2% | 40% |
| Data management | 24.1% | 87.5% |
In reality, the actual % savings will be impacted by the size and complexity of the study, where the trial sites are situated and how the trial will be supported for monitoring and data management.
EDC deployment costs
Having got some sort of a handle on assessing the cost saving of data capture offered by EDC, this would need to be offset by looking at the costs associated with using an EDC for Medtech SMEs.
- Data Management
Whether PDC or EDC is the route of choice, data management will be a fundamental. A data management plan (DMP) and a data validation plan (DVP) will need to be written as part of the clinical evaluation plan (CEP). They will need to be written with the context of the chosen EDC in mind and this may lead to an additional cost, particularly if plans are being adapted from existing documentation developed for PDC.
There are important data management activities which are clearly common to both EDC and PDC investigations but for which it would be reasonable to assume that EDC will have a natural cost advantage and so aren’t reviewed here. These are:
- Data review and cleansing
- Medical coding
- Project management
An annotated CRF (aCRF) will need to be created. The annotated CRF is used to document questions and clinical measurements, and their storage in the study data tabulation model (SDTM) datasets.
Site staff will need to be trained but this is widely overestimated in terms of both cost and committed time. From our experience, virtually all site staff can get the training they need remotely, easily and quickly.
- eCRF Build & Implementation
This is the creation of the electronic data forms and fields in which the data will be entered. This work will also include the configuration of automatic queries and edit checks to detect data discrepancies.
After the eCRF has been developed it will need to be subject to user acceptance testing (UAT). The key purpose of this testing is to validate the software against the system requirements and is carried out by the end-users who understand the eCRF requirements.
Finally, there are Data export and import tasks (initial set up and subsequent export or import executions) to factor into the costings.
- eCRF Software License and Hosting – an exercise in String Theory?
As you move from vendor to vendor, you quickly realise that there is a wide range of differing software modules to choose from as well as a bewildering multiplicity of billing schemes and cost plans – usually depending on study size (number of sites and patients) and duration (months of recruitment and follow-up). It’s like trying to compare the costs and benefits of an exciting collection of bits of string. The fragments are all of slightly different length and come in a range of quality and probable utility – all the way from simple cotton parcel string to 3-stranded, 16 inch hawser rope and there’s no simple, consistent way these strings are priced.
Summary
The more you scratch beneath the surface the more complex the challenge of assessing paper v EDC seems to become. There are numerous variables that apply to a larger or lesser degree to every business and every investigation. There is no magic formula, but watch this space – in future articles we’ll introduce a calculator to help you out.
For now, the message is knowledge is everything and whilst the financial case may seem multi-faceted, the risk mitigation and compliance case is clear. Shrewd planners will also have one eye on the future. Today’s pre-market study will be your fore-runner and likely benchmark for future ongoing PMS requirements… consistency of approach and centralised data will be of great future advantage.
Having looked at the business case of whether to go EDC or paper, the next article in this series will explore the full range of factors you’ll need to take into consideration when choosing your EDC vendor.
How CRFWEB can help you
Many companies will need to re-engineer core business processes such as QMS, Risk Management and Documentation in order to be fully MDR-compliant. There will never be a better time to bite the bullet and migrate to an integrated EDC platform for all future clinical investigations as part of this systems overhaul.
Optimised for the Med Tech sector, CRFWEB will help your company capture data from trials more easily, efficiently and cost effectively. Of course, many in the EDC and wider eClinical sector can make similar claims, but we are confident that CRFWEB delivers all the key functionality you need in an agile and simple to use package…at a fraction of the cost of the big industry names.
