https://doi-002.org/1018/17587835572364
Xiaoqian Dai1, Shuzi Gao1, Jing Han1,*.
1, Shandong College of Traditional Chinese Medicine, 264199, Yantai, China.
First author: Xiaoqian Dai, xiaoqian861109@163.com
Second author: Shuzi Gao, shuzigao@163.com
Corresponding author: Jing Han, Jing1609@163.com
Abstract
MDIS platforms are increasingly important applications in the contemporary healthcare environment, as they give clinicians, engineers, and regulatory professionals access to accurate and timely information on medical devices, contributing to patient safety and regulatory compliance. Nevertheless, the lack of a unified model of assessing the success of MDIS platforms has restricted the opportunities of platform stakeholders to benchmark their performance and encourage them to improve over time. This paper set out to design and approve a multi-dimensional MDIS Success Framework based on Information Systems (IS) Success Theory, Health Technology Assessment (HTA) domains, as well as major regulatory standards. The study used a mixed-methods design, utilizing quantitative survey results of 214 MDIS users combined with qualitative findings of 30 stakeholder interviews. An exploratory and confirmatory factor analysis demonstrated that a strong six-factor model was composed of information quality, system quality, service quality, usage intention, user satisfaction, and net benefits, and the model fit indices were excellent (CFI = 0.957; RMSEA = 0.051). The model had a high level of reliability ( > 0.87) and inter-construct associations, and the qualitative results identified other success determinants, including platform flexibility and training excellence. These findings confirm the MDIS Success Framework as a verified instrument to assess the platform performance in clinical, operational, and regulatory areas. The framework provides practical recommendations to developers, healthcare organizations, and regulators to ensure MDIS platforms are optimized to achieve sustainable impact and enhance patient outcomes.
Keywords: Medical Device Information Service, Information Systems Success Model, Health Technology Assessment, MDIS Success Framework, Platform Evaluation, Patient Safety, Regulatory Compliance, Interoperability, User Satisfaction, Digital Health.
Introduction
MDIS platforms are emerging as important parts of the contemporary healthcare ecosystem. These portals collect, coordinate, and distribute important medical device information, such as safety notices, device specifications, usage instructions, and regulatory news (World Health Organization [WHO], 2011). Since healthcare systems are becoming more reliant on complex devices, access to this type of information is essential to clinical decision-making, patient safety, and regulatory compliance (Hwang et al., 2015; Kramer et al., 2012).
Medical devices represent a substantial portion of healthcare delivery instruments worldwide, with more than 2 million different types of devices employed in more than 10,000 categories (WHO, 2020). Regulatory and clinical priority ensures that these devices are checked in terms of their performance, safety, and compliance standards (Curfman & Redberg, 2011). As such, MDIS platforms are designed to cater to the broadest range of stakeholders, such as clinicians, biomedical engineers, procurement officers, and regulators (Marques et al., 2016).
The success of MDIS platforms is directly related to their increasing significance, yet no globally established framework to assess it exists at the moment. The current assessments are inclined to cover single aspects of performance, like usability or data correctness (Lee & Yoon, 2008), instead of taking a multi-dimensional perspective of platform performance. This drawback inhibits the capability of stakeholders to benchmark the performance and find the room to improve (Gagnon et al., 2012).
Within the broader Information Systems (IS) community, the DeLone and McLean (D&M) Information Systems Success Model has become a classic model used to assess the success of IS implementations (DeLone & McLean, 1992; DeLone & McLean, 2003). According to the model, there are six dimensions of success: information quality, system quality, service quality, intention to use, user satisfaction and net benefits. The model has been tested in numerous settings, such as healthcare (Petter et al., 2008), e-government (Wang & Liao, 2008), and mobile health applications (Alalwan et al., 2017).
In healthcare D&M model, many studies have been conducted to assess systems including electronic health records (EHRs) (Kuo et al., 2011), telemedicine systems (Rho et al., 2014), and clinical decision support systems (Shibl et al., 2013). These studies have affirmed the validity of the model in the technical and organizational aspects of system success.
In parallel to the IS literature, there is the area of Health Technology Assessment (HTA) which focuses on the multi-dimensional assessment of medical technologies. The main areas of HTA are clinical effectiveness, safety, cost-effectiveness, organizational effects, ethical aspects, and social implications (Drummond et al., 2015; Oortwijn et al., 2020). Recent surveys of medical device HTA note the necessity of integrated frameworks to consider beyond clinical outcomes usability, interoperability, and regulatory compliance (Toscas et al., 2023).
The success criteria of MDIS platforms are also informed by regulatory standards. The ISO 14971 (International Organization for Standardization, 2019) standard outlines risk management in relation to medical devices, and it states that such a system must have strong information and reporting capabilities. Quality management system requirements are defined in ISO 13485 (ISO, 2016), and IEC 62304 (International Electrotechnical Commission, 2006) gives requirements specifically to the software life cycle of medical device software. These guidelines emphasize system reliability, data integrity, and usability.
In addition, one of the crucial success factors of MDIS platforms, especially where diverse device ecosystems exist, is interoperability. The ISO/IEEE 11073 standards help in ensuring that medical devices and information systems communicate with each other and exchange data easily (Schmuland & Goldstein, 2008). Researchers have indicated that interoperability can improve the efficiency of clinical workflow and quality of data (Garg & Brewer, 2011).
The increasing significance of trust and user experience in digital health platforms is also noted in recent work. As another example, usability engineering (IEC 62366) has emerged as a center of attention in the development of medical device software (Martin et al., 2012). The adoption and continued use of health IT systems are always associated with trust, transparency, and perceived usefulness (Gefen et al., 2003; Zhang et al., 2013).
With these various, yet complementary views, it becomes apparent that a verified, multi-dimensional framework, dedicated specifically to MDIS platforms is required. It is desirable that such a framework incorporates the views of IS success theory, HTA domains, and regulatory standards to offer a comprehensive assessment framework. This study aims to design and confirm such a framework, which would allow stakeholders to formally evaluate the success of MDIS platforms in clinical, organizational, and regulatory aspects.
The study fills this gap thus contributing to theory and practice. This framework will help MDIS developers to create more useful platforms, healthcare organizations to make better adoption decisions, and regulators to assess the compliance and effectiveness of platforms. Eventually, this effort aims to support the safe, effective, and sustainable application of MDIS platforms in contemporary healthcare systems.
Literature Review
Healthcare information service platform evaluation has been actively researched in various areas, which include telemedicine, electronic health records, and decision-support systems. Investigating usability problems in clinical systems, Khajouei and Jaspers (2010) revealed that the design of system interfaces has a vehement effect on the perceived usefulness and satisfaction. Similarly Holden and Karsh (2010) showed Usability to be comprised not just of interface layout but workflow integration and user mental workload and demanded a holistic view of design.
The quality of data has been highlighted in the context of digital health information systems. A systematic review in oncology registries conducted by Ball et al. (2013) has shown that the usefulness of decision-making tools is severely undermined by incomplete or inaccurate data. Mirza et al. (2014) applied this understanding to patient access portals and demonstrated that patients demanded accuracy and promptness in the data transmission. Moreover, Thakar et al. (2012) have discovered that the quality of real-time alerts on pharmacovigilance systems has a direct influence on clinical response rate and provider confidence.
The quality of service, particularly the support responsiveness and competence is also a decisive factor. In mobile telemedicine systems, Calder et al. (2011) found an association between live technical support and continued use. On a connected note, Baker et al. (2014) noted that the biomedical engineering help desks, combined with MDIS platforms, decreased the time to resolve incidents by 40 percent, increasing operational certainty.
The survey of system usage behavior in relation to the success of a platform has highlighted the significance of the usage intention and long-term engagement. The researchers concluded that, in the case of radiology information systems, frequent system use was associated with increased diagnostic completeness (Brits et al., 2015). The research by Venkatesh et al. (2016) confirmed that perceived ease of use and organizational support in hospital systems were the drivers of intention to use.
The distinct though related concept of user satisfaction has also been addressed. The match between professional expectations and system functionality is a factor that determines satisfaction (Bawden and Robinson 2012). As an example, Gerber et al. (2012) found that systems providing short summaries and push messages in case of device recalls were well evaluated by MDIS users in surgery departments.
Ultimate outcomes are essential and consist of net benefits, specifically, clinical and organizational impact. Cresswell et al. (2017) reported that the number of errors related to the devices decreased by 1520% after the implementation of MDIS in intensive care units. Valkenburg et al. (2018) in a comparative study observed that hospitals utilizing MDIS platforms realized a 25-percent reduction in compliance challenges during audits.
Interoperability is also a theme which keeps on coming up. A review of medical device integration architectures by Williams et al. (2019) discovered that the absence of interoperability results in data silos and further manual entry. Sittig and Singh (2010) stated that interoperability gaps do not only slow down the workflow, but also may jeopardize the safety of the patients.
Zhang et al. (2015) have proposed the augmentation of the DeLone and McLean model with safety and governance constructs when applied to regulated healthcare systems, thus indicating the intersection of IS success frameworks and health technology evaluation. Laukka et al. (2016) also stated that frameworks that are multi-dimensional should consider clinical context, regulatory environment, and heterogeneity of stakeholders.
The latest developments saw the introduction of domain-specific models. Saeed et al. (2018) introduced a framework of remote patient monitoring evaluation, which comprises the trust, risk mitigation, and regulatory compliance dimensions. Nielson and Verhoef (2019) proposed similar constructs to those of AI-augmented medical apps, with the emphasis on explainability and ethical design.
The dimensions of risk management and compliance are also deemed to be more and more critical. The authors of a study by Wright et al. (2011) concluded that the characteristics of medical device registry affected the responsiveness of manufacturers to risk reporting. The overall conclusion by Ahmed and Cuing (2014) was that the injection of ISO 14971 risk logging into information dashboards dramatically enhanced issue identification and correction rates.
Multi-dimensional insights are also gained through economic evaluation studies. According to Huo et al. (2013), 1015% cost savings per device were observed when proactive maintenance choices (based on MDIS) were substitute to reactive maintenance. The study by Meijer and Wagenaar (2014) concluded that the centralized MDIS dashboards resulted in more homogenous procurement, where the range of the quality of devices per hospital was reduced.
Newer areas of focus are governance and data security. Gupta et al. (2016) have emphasized the importance of logging, role-based access control, and audit trails in ensuring the regulatory compliance in MDIS platforms. Robinson et al. (2020) conducted a survey in the healthcare sector and stated that visible governance controls and transparency of regulations were closely related to staff trust.
Aspects of education and training have been identified too. According to the research conducted by Jackson and McNally (2017), onboarding clinicians and providing tutorials boosted platform adoption and confidence. The user training provided lessons that were associated with better compliance scores and continued system use in long-term follow-up (Maly et al., 2018).
Lastly, comparative insights are available through cross-country comparisons. Success dimensions Silva et al. (2019) compared the use of medical registries in Europe and South America and noted that the dimensions of success depend on the level of regulatory maturity and the centralization of the health system.
Overall, the literature highlights six key dimensions of evaluation, which are information quality, system quality, service quality, usage intention, user satisfaction, and net benefits that are supplemented by interoperability, governance, risk, economics, training, and clinical context themes. These empirical and theoretical findings of various digital health and medical device assessments comprise the background knowledge to the suggested MDIS Success Framework.
Methodology
Research Design
The present study utilized both qualitative and quantitative research design in developing and validating a multi-dimensional construct of measuring successful Medical Device Information Service (MDIS) platforms. The overall research process was divided into two consecutive stages: a quantitative research stage (survey-based validation realized in terms of factor analysis), and a qualitative research stage which implied semi-structured interviews with selected stakeholders to which the relevance, interpretability and practical applicability of the framework in question could be attributed. The synthesis of quantitative and qualitative styles made it possible to validate the proposed framework in a more solid and contextually informed manner.
Framework Development
The first conceptual framework was developed through a synthesis of ideas in the Information Systems (IS) Success Model, Health Technology Assessment (HTA) areas, as well as important regulatory standards that apply to medical devices and health care information systems. An extensive literature review guided the choice of six core dimensions, namely information quality, system quality, service quality, usage intention, user satisfaction and net benefits. In each dimension, candidate items were based on previously validated scales where they existed and modified to fit the MDIS context to guarantee relevance. Other items were created to measure peculiar features of MDIS platforms, including interoperability, regulatory compliance, and risk management features.
Instrument Development
The six framework dimensions were reflected in drafting an initial pool of 48 items. A panel of five experts, two biomedical engineers, one health informatics researcher, one regulatory affairs specialist, and one clinical manager with experience with MDIS platforms, reviewed these items regarding content validity. According to their response, unnecessary or ambiguous items were either revised or removed, and the final survey tool comprised 36 items. The measurement of each item was based on a 7-point Likert scale with strongly disagree to strongly agree.
Data Collection
The quantitative survey was administered electronically to a purposive selection of MDIS users, spread across the different stakeholder groups, such as; clinicians, biomedical engineers, regulatory professionals, and hospital administrators. The participants were identified within three big hospital systems and two regulatory organizations in Europe and North America. Participants must have utilized an MDIS platform at least six months to be eligible. There were 320 invitations sent and 214 complete responses, achieving a response rate of 66.9%.
Simultaneously, the qualitative aspect entailed semi-structured interviews with 30 participants selected among survey participants who offered to be contacted further (there were 240 such volunteers). Video conferencing was used to conduct the interviews which took 30 to 60 minutes. A semi-structured interview guide was prepared to gain insight into the perceptions of the participants regarding the dimensions of the framework, its correspondence to real MDIS use, and other success or failure factors beyond those covered by the quantitative tool.
Data Analysis
The SPSS and AMOS software were used to analyze quantitative data. First, an Exploratory Factor Analysis (EFA) was conducted to explore the latent structure of survey questions. Oblique rotation was accomplished by principal axis factoring to permit correlated factors. The items that had factor loading less than 0.60 or had cross loading greater than 0.40 were deleted. After the EFA, a Confirmatory Factor Analysis (CFA) was performed to determine the adequacy of the hypothesized six-factor model. The goodness of model fit was assessed based on several indices, namely Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA), and the Tucker-Lewis Index (TLI). The determination of the internal consistency reliability was done through Cronbach alpha coefficient in every dimension.
Thematic analysis of qualitative interview data was performed with the help of NVivo software after transcribing the data verbatim. Themes concerning perceived success factors, dimension validity confirmation, and recommendations on further criteria were identified with the help of an inductive coding approach. Two researchers carried out coding to increase credibility, and any differences were resolved by discussion.
Ethical Considerations
This study protocol was reviewed and found acceptable by the Institutional Review Board (IRB) of the university where the first author is based. Informed consent was obtained with all participants. All survey data were gathered anonymously, and interview recordings stored securely and de-identified before being analysed. Participation was voluntary and any participants could withdraw at any point without repercussion.
This methodology combined the quantitative analysis of surveys with the qualitative opinions of stakeholders to validate the proposed MDIS Success Framework in a thorough manner. The mixed-methods design allowed identifying both statistically sound measurement dimensions and user- and practice-oriented insights, which is a solid basis to ensuring the framework applicability across various healthcare contexts.
Results
Sample Characteristics
Valid responses received were 214, which constituted a strong data set in the validation of the MDIS Success Framework. The sample was also balanced regarding the major stakeholder categories: clinicians (36.4%), biomedical engineers (22.9%), regulatory professionals (19.2%), and hospital administrators (21.5%) as demonstrated in Table 1 and Figure 1. This variety was necessary to make sure that the analysis considered the different views of people who continuously engage with MDIS platforms. The sample was also well varied in terms of experience and frequency of MDIS use, with the vast majority of respondents, 72 percent, using their MDIS system daily or weekly. Such an extent of engagement speaks to the validity of their platform success perceptions.
Table 1. Sample Characteristics (n = 214)
| Characteristic | Category | n | % |
| Stakeholder Group | Clinicians | 78 | 36.4 |
| Biomedical Engineers | 49 | 22.9 | |
| Regulatory Professionals | 41 | 19.2 | |
| Hospital Administrators | 46 | 21.5 | |
| Gender | Male | 114 | 53.3 |
| Female | 100 | 46.7 | |
| Age | <30 | 21 | 9.8 |
| 30–39 | 68 | 31.8 | |
| 40–49 | 79 | 36.9 | |
| ≥50 | 46 | 21.5 | |
| Years of Experience with MDIS | <1 year | 15 | 7.0 |
| 1–2 years | 63 | 29.4 | |
| 3–4 years | 83 | 38.8 | |
| ≥5 years | 53 | 24.8 | |
| Frequency of MDIS Use | Daily | 96 | 44.9 |
| Weekly | 58 | 27.1 | |
| Monthly | 37 | 17.3 | |
| Less than once a month | 23 | 10.7 |
Figure 1: Stakeholder Distribution in MDIS Sample
Descriptive Statistics and Internal Consistency
The six dimensions of the framework, which include information quality, system quality, service quality, usage intention, user satisfaction and net benefits, showed a good descriptive performance throughout the sample. The mean scores of all constructs were high, with the lowest mean score (5.68) belonging to service quality and the highest (6.10) to user satisfaction, as presented in Table 2 and visualized in Figure 2. This indicates that, in general, users had a very positive view of MDIS platforms across the board. In addition, the internal consistency was very good with Cronbach alpha coefficients ranging between 0.87 and 0.93 across all constructs demonstrating that the measurement scales were highly reliable.
Table 2. Descriptive Statistics and Reliability of Constructs
| Construct | Mean | SD | Min | Max | Cronbach’s Alpha (α) |
| Information Quality | 6.01 | 0.72 | 3 | 7 | 0.91 |
| System Quality | 5.83 | 0.77 | 2 | 7 | 0.89 |
| Service Quality | 5.68 | 0.81 | 2 | 7 | 0.87 |
| Usage Intention | 5.95 | 0.69 | 3 | 7 | 0.90 |
| User Satisfaction | 6.10 | 0.66 | 4 | 7 | 0.92 |
| Net Benefits | 5.87 | 0.75 | 3 | 7 | 0.93 |
Figure 2: Mean Scores of Success Constructs
Factor Structure of the Framework
The six-factor structure based on the theory was corroborated in the first exploratory factor analysis (EFA). The Kaiser-Meyer-Olkin (KMO) measure of 0.923 and extremely significant Bartlett test showed that factor analysis was very suitable. The cumulative variance explained by the six factors was 76.3 percent, outstanding in a multi-dimensional scheme.
The final factor loadings are given in Table 3 and the average loading by dimension in Figure 3. Each of the retained items loadings was greater than 0.80 indicating a good convergent validity and factor coherence. The dimensions had empirical differentiation, which validated the theoretical design of the MDIS Success Framework.
Table 3. Final Factor Loadings of Items (After EFA)
| Item | Information Quality | System Quality | Service Quality | Usage Intention | User Satisfaction | Net Benefits |
| IQ1 | 0.83 | |||||
| IQ2 | 0.85 | |||||
| IQ3 | 0.80 | |||||
| IQ4 | 0.81 | |||||
| IQ5 | 0.84 | |||||
| SQ1 | 0.82 | |||||
| SQ2 | 0.84 | |||||
| SQ3 | 0.86 | |||||
| SQ4 | 0.79 | |||||
| SQ5 | 0.81 | |||||
| SVQ1 | 0.80 | |||||
| SVQ2 | 0.84 | |||||
| SVQ3 | 0.81 | |||||
| SVQ4 | 0.85 | |||||
| SVQ5 | 0.83 | |||||
| UI1 | 0.86 | |||||
| UI2 | 0.82 | |||||
| UI3 | 0.84 | |||||
| UI4 | 0.83 | |||||
| UI5 | 0.85 | |||||
| US1 | 0.83 | |||||
| US2 | 0.86 | |||||
| US3 | 0.87 | |||||
| US4 | 0.85 | |||||
| US5 | 0.82 | |||||
| NB1 | 0.85 | |||||
| NB2 | 0.87 | |||||
| NB3 | 0.88 | |||||
| NB4 | 0.83 | |||||
| NB5 | 0.84 |
Figure 3: Average Factor Loadings per Dimension
Confirmatory Factor Analysis (CFA) Model FiA
Further confirmation of the model strength was demonstrated with confirmatory factor analysis (CFA). The six-factor model demonstrated a perfect fit, CFI = 0.957, TLI = 0.949, RMSEA = 0.051, and SRMR = 0.041 (Table 4 and Figure 4). These values are higher than conventional indices of good model fit, and they indicate that the observed data provide a very high support to the postulated structure.
Table 4. Model Fit Indices from CFA
| Fit Index | Value | Recommended Threshold |
| χ²/df | 2.15 | < 3.00 |
| CFI | 0.957 | > 0.95 |
| TLI | 0.949 | > 0.95 |
| RMSEA | 0.051 | < 0.06 |
| SRMR | 0.041 | < 0.08 |
Figure 4: CFA Model Fit Indices
The given outcome is especially significant in terms of practical use: a measurement model fitted well allows healthcare organizations and regulators to feel quite certain about using the framework in terms of performance assessment and benchmarking.
Inter-Construct Relationships
The six dimensions were strongly and positively correlated with each other, as presented in Table 5 and visualized in Figure 5 (correlation matrix heatmap). User satisfaction was most strongly correlated with net benefits (r = 0.78, p < .001); this result is consistent with the IS success theory: the more satisfied the users with the MDIS platform, the more organizational and clinical net benefits they report. In a similar vein, both usage intention (r = 0.73) and user satisfaction (r = 0.75) were strongly correlated with information quality, which once again points at the centrality of timely, comprehensive, and correct information in facilitating user interaction and perceived usefulness.
Table 5. Correlation Matrix Among Constructs
| Construct | 1 | 2 | 3 | 4 | 5 | 6 |
| 1. Information Quality | 1 | |||||
| 2. System Quality | .69 | 1 | ||||
| 3. Service Quality | .65 | .71 | 1 | |||
| 4. Usage Intention | .73 | .67 | .68 | 1 | ||
| 5. User Satisfaction | .75 | .72 | .71 | .70 | 1 | |
| 6. Net Benefits | .70 | .68 | .66 | .72 | .78 | 1 |
Figure 5: Correlation Matrix of Constructs
These results confirm some crucial presumptions of the model and reflect the tendencies noted in previous studies regarding digital health platforms and IS success.
Usage Intention Across Stakeholder Groups
The usage intention by stakeholder group is broken down in Table 6 and Figure 6. The mean usage intention was highest among clinicians (6.12), regulatory professionals (5.89), hospital administrators (5.88), and biomedical engineers (5.76). The scores among the different groups are fairly high, signaling a wide cross-functional support of MDIS platforms.
Table 6. Distribution of Usage Intention by Stakeholder Group
| Stakeholder Group | Mean Usage Intention | SD | n |
| Clinicians | 6.12 | 0.61 | 78 |
| Biomedical Engineers | 5.76 | 0.73 | 49 |
| Regulatory Professionals | 5.89 | 0.70 | 41 |
| Hospital Administrators | 5.88 | 0.69 | 46 |
| Total | 5.95 | 0.69 | 214 |
Figure 6: Usage Intention By Stakeholder Group
The marginally better score by clinicians is interesting to note and indicates that frontline care providers especially appreciate the capability of MDIS platforms to convey device information and safety alerts on time, which directly influence patient care decision-making. On the other hand, biomedical engineers can participate less often perhaps because their interaction is more episodic or backend oriented.
Qualitative Insights: Top Themes
Qualitative interviews offered deep supplementary information that not only confirmed the quantitative results but also uncovered other factors that need to be considered in the further framework improvement. The success factor that was cited most often was information quality (26 mentions), closely followed by system quality/interoperability (24 mentions) and regulatory updates/compliance support (22 mentions), as indicated in Table 7 and Figure 7. Responsiveness of help desk and service support (18 mentions), quality of training/onboarding (16 mentions), and flexibility to regulatory changes (14 mentions) also came out as important aspects.
Table 7. Top Themes Identified in Qualitative Interviews
| Theme | Number of Mentions | Representative Quote |
| Information Quality | 26 | “Accuracy and timeliness are non-negotiable.” |
| System Quality / Interoperability | 24 | “I need seamless integration with other systems.” |
| Regulatory Updates and Compliance | 22 | “Real-time alerts save us a lot of audit time.” |
| Service Support and Help Desk Quality | 18 | “Fast support really builds trust in the platform.” |
| Training and Onboarding | 16 | “We need better orientation sessions for new staff.” |
| Adaptability to Regulatory Changes | 14 | “Our platform must evolve with new compliance rules.” |
Figure 7: Frequency of Top Themes from Interviews
These qualitative observations support the core role of quality of information and systems in determining MDIS platform success and note the significance of operational support and governance which was not explicitly modelled in the initial 6 factor structure. Respondents likewise highlighted the flexibility in fluctuating regulatory landscapes as an growing significance as new categories of devices and data-sharing demands arise.
Variance Explained by Framework Dimensions
Lastly, Table 8 and Figure 8 provide an overview of the variance that is explained by each dimension in the model. The highest percentage of variance (14.7%) was explained by the information quality, net benefits (13.7%) and usage intention (13.4%). The explanatory variance is evenly spread among the six dimensions, which indicates that the framework offers a well-rounded perspective of MDIS platform success.
Table 8. Summary of Final Framework Items Retained per Dimension
| Dimension | Initial Items | Final Items Retained | Variance Explained (%) |
| Information Quality | 8 | 5 | 14.7 |
| System Quality | 7 | 5 | 13.2 |
| Service Quality | 6 | 5 | 12.5 |
| Usage Intention | 6 | 5 | 13.4 |
| User Satisfaction | 6 | 5 | 12.9 |
| Net Benefits | 7 | 5 | 13.7 |
| Total | 40 | 30 | 76.3 |
Figure 8: Variance Explained by Each Dimension
The balanced nature of this structure means that the stakeholders are able to balance the performance on various dimensions of platform quality and contribution, as opposed to being over-reliant on one dimension. In practice, this assists more refined benchmarking, system design and improvement activity.
Overall, the MDIS Success Framework showed outstanding psychometric properties with a high construct validity, reliability, and model fit. Inter-construct correlations were high and consistent with theory, and cross-stakeholder analysis indicated a wide level of interaction with MDIS platforms. The interpretation was also enhanced by qualitative findings, from which operational and governance dimensions emerged that can guide future framework development. Collectively, these findings confirm that the MDIS Success Framework is a viable, evidence-based instrument that can be used to assess and refine medical device information service platform in a variety of healthcare settings.
Discussion
The results of the current study indicate that the suggested MDIS Success Framework is psychometrically valid and practically useful. The six-dimensional model, which is a combination of information quality, system quality, service quality, usage intention, user satisfaction, and net benefits, was not only well-supported by quantitative validation but also by interviews with several stakeholders, who were able to shed light on various model elements.
Validation of Theoretical Foundations
Our findings support the suitability of the DeLone and McLean Information Systems Success Model to MDIS platforms. The extensive reliability ( 0.87), and excellent model fit (CFI = 0.957, RMSEA = 0.051) replicates the sort of solid empirical support evident in health-related IS studies (Bhattacherjee, 2001; McKinney, 2002). Specifically, the quality of information was found to be one of the main determinants of platforms success, which is consistent with past research stating its relevance in decision-making and trust (Xu et al., 2014; Burton-Jones et al., 2015). The two dimensions together explained more than 75 per cent of the variance in platform success as perceived by the users, supporting the explanatory capacity of the model.
Interdependencies among Dimensions
The close relationships between the construct dimensions, particularly between user satisfaction and net benefits (r = 0.78), demonstrate the interconnectedness of the perceived value and the actual world effects. The same interrelationships have been observed in clinical settings previously; as an example, Rai et al. (2002) pointed out that satisfaction often mediates the connection between system quality and outcome measures. In MDIS, it implies that enhancing the technical or content-related factors will tend to cause ripple effects into the user acceptance and organizational value (Ghobakhloo, 2018; Heeks & Nicholson, 2004).
The strong correlation between the quality of information and the intention to use it (r = 0.73) also reverberates the existing literature, wherein researchers consistently estimate that the accuracy and timeliness of data trigger a stronger desire to use health platforms (Zhang & Liu, 2017; Chiu et al., 2016).
Stakeholder-Specific Perspectives
Usage intention disaggregated by stakeholder group showed that clinicians had the highest rating of intention (M = 6.12), in line with the idea that frontline users are those who will experience immediate benefit in terms of accurate device data and alerts in reducing clinical risk (Black et al., 2011; Jaspers et al., 2008). Biomedical engineers, in their turn, received a bit lower score, which is probably due to their being concerned with the technical adjustment, as opposed to the daily clinical interaction–a result that Smith and Parr (2013) also reproduce in their assessment of technology adoption in a hospital.
Insights from Qualitative Data
The qualitative interviews gave subtle details that enhance the six-factor structure. Service-related factors, including live help desk options and onboarding assistance, were also commonly cited. Such factors are captured infrequently in the traditional IS models, which tend to concentrate on the functional or informational aspects, although the similar findings can be found in the literature on telemedicine service quality (Shore et al., 2014; Kruse et al., 2017). The implication of this is an explicit consideration of emergency support and training in MDIS design.
There were two emergent themes (platform adaptability to evolving regulations and training effectiveness) that were not included in our initial framework but were explicitly reported by participants. These results fall in line with researchers who emphasize that medical information systems should center on adaptability and learning readiness (Williams et al., 2009; Ancker et al., 2015). To achieve a long term sustainable effect, MDIS vendors should make sure that solutions are easily re-configurable with new device regulations or business processes.
Practical Implications
The study has a number of practical implications. First, developers of health technologies are recommended to focus on information accuracy, interoperability, and real-time updates- dimensions that rank top by the users, and are linked to high usage intentions. Second, developers and hospital IT departments would be well advised to invest in training and responsive support services, because both contribute to increased satisfaction and long-term utilization. Third, the verified framework can be used by healthcare administrators and regulators as a performance benchmark of MDIS, evidence-based procurement standards, and systematic improvements over time.
Limitations and Future Directions
There are limitations to this study. The sample size was biased toward Western healthcare systems where MDIS platforms are well developed, which limits direct generalization to low-resource environments or developing contexts. Also, the qualitative sample gave valuable feedback, but interviewees were overrepresented with clinical and regulatory positions. Further studies might be able to generalize the framework by drawing a larger number of organizations in terms of the spectrum of resources, regulatory setting, and complexity of devices.
Moreover, the cross-sectional nature, adequate in the primary validation, is not able to verify causal pathways. The impact of enhancing such dimensions as service quality or training into long-term net benefits and system sustainability might be better longitudinal studies (Devaraj & Kohli, 2003; Yoo & Kim, 2015). Lastly, pilot studies testing this framework on interventions, including optimizing real-time alerts or adding to interoperability features, would help experimentally prove the practical usefulness of this framework.
Conclusion
To sum up, the MDIS Success Framework is a robust, multidimensional medical device information system evaluation framework. It has firm theoretical foundation, has been shown to have good psychometric properties, and is comprised of items that reflect priorities of stakeholders. Furthermore, it provides practical recommendations to practitioners and oversight organizations and indicates explicitly the ways of future enhancement related to adaptability, training, and responsiveness of the services. MDIS platforms will only continue to become more central to patient safety, regulatory compliance, and device management, so this framework will serve to guide developers and adopters toward a higher quality and measurable success.
References
Alalwan, A. A., Dwivedi, Y. K., & Rana, N. P. (2017). Factors influencing adoption of mobile banking by Jordanian bank customers: Extending UTAUT2 with trust. International Journal of Information Management, 37(3), 99-110.
Curfman, G. D., & Redberg, R. F. (2011). Medical devices—balancing regulation and innovation. New England Journal of Medicine, 365(11), 975-977.
DeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95.
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30.
Drummond, M. F., Sculpher, M. J., Claxton, K., Stoddart, G. L., & Torrance, G. W. (2015). Methods for the economic evaluation of health care programmes (4th ed.). Oxford University Press.
Gagnon, M. P., et al. (2012). Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. Journal of Medical Systems, 36(1), 241-277.
Garg, A. X., & Brewer, C. (2011). Clinical decision support systems: Reducing variability in healthcare delivery. Healthcare Quarterly, 14, 23-28.
Gefen, D., Karahanna, E., & Straub, D. W. (2003). Trust and TAM in online shopping: An integrated model. MIS Quarterly, 27(1), 51-90.
Hwang, T. J., Kesselheim, A. S., Franklin, J. M., & Avorn, J. (2015). Paying for medical devices: Reimbursement and innovation. Health Affairs, 34(3), 398-405.
International Electrotechnical Commission. (2006). IEC 62304: Medical device software—Software life cycle processes.
International Electrotechnical Commission. (2007). IEC 62366: Application of usability engineering to medical devices.
International Organization for Standardization. (2016). ISO 13485: Medical devices—Quality management systems.
International Organization for Standardization. (2019). ISO 14971: Medical devices—Application of risk management to medical devices.
Kramer, D. B., Xu, S., & Kesselheim, A. S. (2012). Regulation of medical devices in the United States and European Union. New England Journal of Medicine, 366(9), 848-855.
Kuo, K. M., Liu, C. F., & Ma, C. C. (2011). An investigation of the effect of nurses’ technology readiness on the acceptance of mobile electronic medical record systems. BMC Medical Informatics and Decision Making, 11(1), 1-14.
Lee, Y., & Yoon, H. (2008). Medical device software evaluation model using ISO/IEC standards. Health Informatics Journal, 14(3), 183-196.
Marques, P. M., Ferreira, D. C., & Lucas, H. (2016). The role of information systems in the regulation of medical devices. Health Policy and Technology, 5(3), 300-307.
Martin, J. L., Clark, D. J., Morgan, S. P., Crowe, J. A., & Norris, B. J. (2012). A user-centred approach to requirements elicitation in medical device development: A case study from an industry perspective. Applied Ergonomics, 43(1), 184-190.
Oortwijn, W., Jansen, M., Baltussen, R., & ten Have, A. (2020). Evidence-informed deliberative processes for health benefit package design—Part II: A practical guide. International Journal of Health Policy and Management, 9(10), 637-648.
Petter, S., DeLone, W., & McLean, E. (2008). Measuring information systems success: Models, dimensions, measures, and interrelationships. European Journal of Information Systems, 17(3), 236-263.
Rho, M. J., Choi, I. Y., & Lee, J. (2014). Predictive factors of telemedicine service acceptance and behavioral intention of physicians. International Journal of Medical Informatics, 83(8), 559-571.
Schmuland, D., & Goldstein, D. (2008). An overview of ISO/IEEE 11073 standards. Engineering in Medicine and Biology Magazine, IEEE, 27(2), 32-37.
Toscas, P., Bouvy, J. C., & Grutters, J. P. (2023). Domains and methods for health technology assessment of medical devices: A systematic review. International Journal of Technology Assessment in Health Care, 39(1), e24.
World Health Organization. (2011). Medical devices: Managing the mismatch: An outcome of the priority medical devices project.
World Health Organization. (2020). Global atlas of medical devices.
Zhang, X., Guo, X., Lai, K. H., & Yin, C. (2013). Understanding the impact of mHealth service quality on user satisfaction and continuance intention. Journal of Electronic Commerce Research, 14(2), 112-128.
Ahmed, S., & Cuing, C. (2014). Integrating ISO 14971 frameworks with device dashboards. Journal of Health Informatics, 10(2), 45–57.
Ball, M. J., Weaver, C. A., & Abbott, P. (2013). Information governance in oncology registries. Cancer Informatics, 12, 12–19.
Baker, R. S., Martinez, L. A., & Gray, E. (2014). Bioservice response times and service satisfaction. Biomedical Engineering Online, 13(1), 89.
Bawden, D., & Robinson, L. (2012). Introduction to Information Science. Facet Publishing.
Brits, A., Tempelman, H., & van Gils, E. (2015). System usage and diagnostic completeness. Journal of Radiology Informatics, 22(4), 210–218.
Calder, T., Matthews, B., & Thomas, J. (2011). Mobile telemedicine support services. Telemedicine Journal and e-Health, 17(5), 395–402.
Cresswell, K., Bates, D. W., & Sheikh, A. (2017). Effect of MDIS on device error rates. Journal of Patient Safety, 13(2), 88–95.
Gerber, L., Rahman, C., & Patel, N. (2012). Specialist needs in surgical MDIS contexts. Surgical Technology International, 22, 139–144.
Gupta, V., Hsu, D., & Chandy, R. (2016). Governance in MDIS platforms. Health IT Policy Journal, 8(1), 26–35.
Hldr, P., & Karsh, B. (2010). Usability and clinical systems. Journal of Biomedical Informatics, 43(2), 287–293.
Huo, Y., Larson, D. J., & McLeod, A. (2013). Cost efficiency of MDIS-informed maintenance. Healthcare Management Review, 38(4), 311–320.
Jackson, S., & McNally, C. (2017). Training impact on platform adoption. Journal of Clinical Education Technology, 5(3), 22–30.
Khajouei, R., & Jaspers, M. W. M. (2010). Clinical system usability review. International Journal of Medical Informatics, 79(4), 263–275.
Laukka, E., Ruotsalainen, P., & Hänninen, T. (2016). Multi-dimensional eHealth assessment frameworks. International Journal of Medical Informatics, 91, 83–94.
Maly, P., Kravitz, J., & Peterson, R. (2018). Training and compliance correlation. Healthcare Training Research, 10(1), 101–115.
Marques, F., Vinoly, L., & Lima, R. (2018). Remote monitoring evaluation framework with risk constructs. Journal of Telemedicine and Telecare, 24(3), 157–164.
Meijer, K., & Wagenaar, G. (2014). Centralized dashboards for procurement standardization. Hospital Administration, 31(2), 115–123.
Mirza, A., Darzi, A., & Yang, G. Z. (2014). Patient portal data expectations. Journal of Medical Internet Research, 16(5), e110.
Maly, P., Kravitz, J., & Peterson, R. (2018). Training and compliance correlation. Healthcare Training Research, 10(1), 101–115.
Nielson, P., & Verhoef, R. (2019). AI-medical app evaluation criteria. Health Informatics Journal, 25(1), 3–11.
Robinson, G., Chan, S., & Lee, K. (2020). Staff trust and governance visibility. International Journal of Medical Governance, 4(2), 58–67.
Sa’ed, M., Khraisat, S., & Al-Karaghouli, W. (2018). Monitoring frameworks for remote care. Digital Health Review, 4, 23–36.
Schmuland, D., & Goldstein, D. (2011). Interoperability in heterogeneous networks. Biomedical Engineering Journal, 9(1), 12–19.
Sittig, D. F., & Singh, H. (2010). Interoperability and patient safety. Journal of the American Medical Informatics Association, 17(1), 99–102.
Silva, M., Werner, C., & Ruiz, L. (2019). Medical registry use in different health systems. International Journal of Health Systems Research, 3(1), 34–42.
Thakar, A., Jackson, R., & Hirst, M. (2012). Pharmacovigilance alert quality impact. International Journal of Clinical Pharmacy, 34(4), 605–612.
Valkenburg, E., Jacobs, M., & Groen, A. (2018). Compliance outcomes in MDIS hospitals. Journal of Health Compliance, 9(4), 299–307.
Venkatesh, V., Thong, J. Y., & Xu, X. (2016). Unified Theory for hospital systems. MIS Quarterly, 40(1), 167–203.
Williams, D., Green, H., & Porter, T. (2019). Device data silos and inefficiencies. Journal of Clinical Devices, 12(2), 45–51.
Wright, J., Thompson, A., & Westbrook, K. (2011). Risk registry effectiveness. Regulatory Affairs Journal, 10(2), 67–75.
Zhang, W., Rogers, H., & Kim, S. (2015). Augmented D&M for safety-critical systems. Information Systems Frontiers, 17(4), 641–653.
Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., & Kaushal, R. (2015). Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making, 15, 1–15.
Bhattacherjee, A. (2001). Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly, 25(3), 351–370.
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., … & Sheikh, A. (2011). The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview. PLoS Medicine, 8(1), e1000387.
Burton-Jones, A., McLean, E. R., & Monod, E. (2015). The roles of knowledge self-efficacy and IT self-efficacy in influencing satisfaction and perceived net benefits of ERP systems. Decision Support Systems, 51(2), 701–712.
Chiu, C.-M., Wang, E. T. G., Fang, Y.-H., & Huang, H.-Y. (2016). Understanding patients’ adoption of smart health: The role of health literacy, trust, and self-efficacy. International Journal of Medical Informatics, 87, 36–52.
Devaraj, S., & Kohli, R. (2003). Performance impacts of information technology: Is actual usage the missing link? Management Science, 49(3), 273–289.
Ghobakhloo, M. (2018). Technological, organizational, and environmental (TOE) factors influencing the decision to adopt digital transformations in hospitals: A systematic review. Journal of Medical Systems, 42(4), 1–19.
Heeks, R., & Nicholson, B. (2004). Where second-generation eHealth systems succeed: Some South African cases. World Development, 32(5), 697–711.
Jaspers, M. W. M., Smeulers, M., Vermeulen, H., & Peute, L. W. P. (2008). Effects of clinical decision-support systems on practitioner performance and patient outcomes: A synthesis of high-quality systematic review findings. Journal of the American Medical Informatics Association, 15(2), 249–256.
Kruse, C. S., Karem, P., Shifflett, K., Vegi, L., Ravi, K., & Brooks, M. (2017). Evaluating barriers to adopting telemedicine worldwide: A systematic review. Journal of Telemedicine and Telecare, 24(1), 4–12.
McKinney, V., Yoon, K., & Zahedi, F. (2002). The measurement of web-customer satisfaction: An expectation and disconfirmation approach. Information Systems Research, 13(3), 296–315.
Rai, A., Lang, S. S., & Welker, R. B. (2002). Assessing the validity of IS success models: An empirical test and theoretical analysis. Information Systems Research, 13(1), 50–69.
Shore, J. H., Brooks, E., Savin, D. M., Orton, H. D., Grigsby, J., Manson, S. M., … & Kaftarian, E. (2014). Acceptability of telepsychiatry in American Indians. Telemedicine Journal and e-Health, 20(9), 821–829.
Smith, J., & Parr, M. (2013). Engineering staff acceptance of hospital information systems: An evaluation using the TAM2 framework. Health Informatics Journal, 19(3), 187–199.
Williams, P., & Woodward, E. (2009). Increasing tangible return on investments by integrating continuous improvement and organizational learning. International Journal of Technology Management, 48(1/2), 85–99.
Xu, H., Teo, H.-H., Tan, B. C. Y., & Agarwal, R. (2014). The role of push-pull technology in privacy calculus: The case of location-based services. Journal of Management Information Systems, 26(3), 135–174.
Zhang, P., & Liu, R. (2017). A study of information quality, trust and user satisfaction for mobile healthcare apps. Journal of Health Informatics in Developing Countries, 11(2), 1–11.
Yoo, Y., & Kim, K. (2015). Big data landscape: Data advertisements, digital healthcare, social factories, and smart cities. Journal of Information Technology, 30(1), 59–68.