Quantitative Research

1 3

Med Biol Eng Comput (2015) 53:1295–1303 DOI 10.1007/s11517-015-1245-3

SPECIAL ISSUE - ORIGINAL ARTICLE

Performance assessment of a closed‑loop system for diabetes management

A. Martinez‑Millana · G. Fico · C. Fernández‑Llatas · V. Traver

Received: 31 March 2014 / Accepted: 23 January 2015 / Published online: 11 February 2015 © International Federation for Medical and Biological Engineering 2015

1 Introduction

Diabetes is defined as a set of syndromes where high blood glucose level is the common factor. Inefficiency or the absence of endogen insulin leads to metabolic disor- ders which cause micro- and macrovascular complications in both short and long term [9]. A good quality of life, together with an adequate adherence to the medical treat- ment, can delay the rise of complications [22].

The management of this disease has improved with the arrival of new monitoring systems, insulin pumps and oral drugs. From the handmade logbook reporting to stem- cells research, a wide variety of medical technologies are applied to maintain patient health. The healthcare industry and academics are fostering improvements in monitoring technologies: Recently, the accuracy and reliability of a fiber-coupled optical continuous glucose sensor was tested for 14 consecutive days in a clinical trial, showing promis- ing results [14]; closed-loop artificial pancreas [8] can sig- nificantly reduce glucose variability, regardless of patient adherence to treatment. Nevertheless, the patient should not be taken out of the equation. A recent study evaluated the capability of 25 diabetic patients to adjust the subcuta- neous insulin infusion therapy and avoid glycemic excur- sions, proving that an information management system can improve glycemic control by performing accurate adjust- ments to the insulin delivery [18].

While the literature is progressively focusing on the cost-effectiveness of applying new technologies to diabe- tes management [16, 19], the focus should be on promot- ing patient education and consciousness to deal with the disease. In the case of non-communicable diseases, where long duration and slow progression require concrete actions on patient lifestyle, the information and communication technologies (ICT) may contribute to provide the individual

Abstract Telemedicine systems can play an important role in the management of diabetes, a chronic condition that is increasing worldwide. Evaluations on the consistency of information across these systems and on their performance in a real situation are still missing. This paper presents a remote monitoring system for diabetes management based on physiological sensors, mobile technologies and patient/ doctor applications over a service-oriented architecture that has been evaluated in an international trial (83,905 opera- tion records). The proposed system integrates three types of running environments and data engines in a single service- oriented architecture. This feature is used to assess key performance indicators comparing them with other type of architectures. Data sustainability across the applications has been evaluated showing better outcomes for full inte- grated sensors. At the same time, runtime performance of clients has been assessed spotting no differences regarding the operative environment.

Keywords mHealth · SOA · Diabetes · Sensors · Telemonitoring · Performance · KPI

A. Martinez-Millana (*) · C. Fernández-Llatas · V. Traver ITACA, Health and Wellbeing Technologies, Universidad Politécnica de Valencia, Camino de Vera s/n, 46022 Valencia, Spain e-mail: [email protected]

G. Fico Life Supporting Technologies, Universidad Politécnica de Madrid, Madrid, Spain

1296 Med Biol Eng Comput (2015) 53:1295–1303

1 3

and supporting careers with solutions to manage therapies, increase awareness and promote healthy self-management habits [19, 20].

One of the purposes of remote monitoring in diabetes is to prevent a worsening condition and hospitalization due to long-period hyperglycemias and decompensation by providing patient data to care providers [25], which may also lead to reducing the high economic health costs [16]. Besides educational and telephone aid, patient self- management support may be augmented by using home- based technologies that generate data points which provid- ers can potentially use to make more timely changes in the patients’ care. The effectiveness of a secure e-mail atten- tion program in diabetic population has been demonstrated [27]; nevertheless, authors point out the need for embed- ding such communication channels within a continuous self-management system.

ICT solutions are presented as key solutions for man- aging diabetes [11]. First interventions used the Internet and public switched telephone network in urban and rural environments [1, 23]. Later on, mobile networks based on GPRS [7, 8, 15] and Internet portals [12, 21] were used as new paradigms of the application of ICT in diabetes man- agement. More recently, comparative frameworks [17] have described the benefits of using mobile technologies in the management of chronic conditions and other authors have suggested the implementation of new communication architectures based on services (SOA) [4].

The impact of telemedicine interventions is highly dependent on the architecture and performance of the system. The variety of results achieved in the literature [3] could be better understood by contextualizing each intervention in terms of system implementation and per- formance. There are qualitative and quantitative tools to measure the user response to a single tool or system [24]; however, the technical implementation and evaluation should be presented to add significance to the results of a telemedicine intervention. The performance should be assessed using specific benchmarks, known as key perfor- mance indicators (KPIs), that describe the quality of ser- vice. Typical examples of KPIs include cost, security, reli- ability and delays.

The rationale of this paper is to present a new system for diabetes remote monitoring oriented to patient self- management based on a SOA and to introduce a technical assessment of the system performance in a trial with 30 type 1 and 2 diabetes patients and up to 83,905 interac- tion records. The system uses home and portable devices for monitoring blood glucose, physical activity, weight, blood pressure and applications to record data (food intake, medication, educational level and lifestyle). Unlike the related work, the insulin pump device is discarded as the patient himself is the booster of the treatment (according to

doctor’s guidelines). The presented SOA is based on Inter- net protocol communication which merges different opera- tive systems and software environments (Windows Mobile, iOS, Apache Tomcat and .NET) and is evaluated using existing KPIs under specific domains [3]. Clinical and usability assessments are out of the scope of this research work. Records of data, synchronization logs and services invocations have been analyzed to evaluate the effective- ness of the proposed solution from a performance and data aggregation perspective.

2 Methods

The architecture of the telemonitoring system consists of three main blocks as shown in Fig. 1. In the bottom square: patient devices with mobile and desktop applications to retrieve data from sensors and to self-manage the disease by recording events, reading prescriptions from the doc- tor side and showing historic trends. Top right the control panel (CP): from which healthcare professionals are able to supervise patient evolution, prescribe treatments and mod- ify therapies. And top left the system central server: where

Fig. 1 System schema combining the components devoted for patients, doctors and server. Sensors are connected to the Patient Panel through the interfaces depicted in Sect. 2.2. All the communica- tions are held by the Internet Protocol

1297Med Biol Eng Comput (2015) 53:1295–1303

1 3

all the communicating functionalities and intelligent mod- ules are hosted to provide a user-blinded closed-loop solu- tion to all users: both patients and care providers.

Single features for each type of user were translated into requirements and then deployed as tools to run over multiple platforms (Fig. 1). This process was made under user centered design (UCD) techniques [5]. These tools were developed within Application Program Interfaces (APIs) and scaled to provide multiple levels of interactions, depending on the type of diabetes and the type of user. The information generated was sent via web services to health- care professionals involved in the follow-up process. Two patient mobile devices (PMD) contained native applica- tions to record and display prescriptions fixed by the doctor in the CP, trends from sensor and lifestyle variables (food intake and medication) and educational material. T1DM patients were provided with a HTC (WM 6.1) and T2DM an iPhone 3G (iOS 4.2.1). Patient Panel (PP) was a table/ desktop application that both type of users were able to use to download sensor data (sensor platform is further detailed in point 2.4) from each device and upload it to the system server so it was accessible to both the physicians applica- tion and the patient mobile application.

2.1 Service-oriented architecture

The main technical challenge consisted in storing this kind of information in a common personal health record (PHR), enabling different interaction from all the modules within the system. Client applications (patient and doctor) are connected through a distributed SOA which contains a cen- tral database and surrounding modules to build up a secure and scalable platform of functionalities for data analysis and exchange.

Up to seven modules were in charge of administrating the central database, performing data analysis, controlling patient compliance and treatment follow-up awareness in terms of the uploaded data. This approach was designed to provide consistent security features (access and integrity) at the same time to all data entities.

To meet system requirements in the architecture drawn in Fig. 2, web services were developed to be invoked from different technologies through a combination of SOAP/ XML and WSLD protocols. A total of forty-five methods were designed, implemented and tested. Information was embedded into XML files and sent as serialized or compact objects depending on the connection speed rate.

Different client applications (iOS, .NET, java) are inte- grated in this platform using several data engines (SQLite, SQLCE, MySQL). Despite the similar data type defini- tions for these technologies, many troubles were found in the integration of some data types (mostly related to the number of bytes contained integer and float values). Thus,

all services were developed to work with the common data type across mentioned platform (32-byte string) and then convert the value locally depending on the type of client implementation.

2.2 Sustainable data management and storage

Several approaches to support easy information acquisition have been developed to guarantee sustainable data storage. Giving the user the right information at the right moment may enable patients to achieve better results on health indicators.

To maintain data integrity and keep a stable PHR syn- chronization among components, an ad hoc acknowl- edgment protocol based on global unique identifiers was developed. To be able to use communication services, each application had to be registered prior to normal ser- vice requests. The registration process consisted in pro- viding the patient with a username and a password to log into the application. If credentials matched, the applica- tion was registered into the server and services enabled to start exchanging sensitive information only to that unique patient device. Subsequent access to the application was made possible by inputting a personal identification num- ber (configurable by patients).

Within the mentioned services, a set of three services were developed to deal with sensor data. Sensor synchro- nization was designed to occur in a wide variety of time intervals. Against plain-text information wrapped into XMLs, sensor data were sorted into a custom format vec- tor to avoid service overloads (headers and identifiers repeated for each measurement). For instance, continuous glucose monitoring system (CGMS) data recorded during 2 weeks could be uploaded at once containing more than

Fig. 2 System architecture is built up by three layers connecting dif- ferent technologies through a web service implementation

1298 Med Biol Eng Comput (2015) 53:1295–1303

1 3

four thousand measurements as well as pedometer data cor- responding to 1 day containing twenty-four values.

Different types of information acquired from several data sources had to be stored in a homogeneous data- base. ISO 11073 data structure standard was used to clas- sify values read from the sensors through an application program interface (API) [6]. The deployed API merged functionalities to hold communications through physical interfaces (Group 1: USB, Group 2: virtual sensors, using proprietary protocol and providing CSV reports, Group 3: Bluetooth) and logical channels, deploying the communi- cation protocols for each sensor. Table 1 shows the corre- spondence between each sensor and the physical channel implemented.

Automated scripts were implemented to manage the sensor data in a transparent approach for the user. Host application (PP) was deployed to download, store and syn- chronize sensor data automatically as soon as the patient connected the sensor.

2.3 Data heterogeneity: management and synchronization

The dynamic nature of the medical parameters gathered in the PP and the distributed heterogeneous data from each single sensor and manual record required new approaches to data warehousing and management.

The central database, contained in the system server, was located in a Windows Server 2008 virtual server in Prage, CZ, separate to hospital networks, due to the multi- center study design and in order to maintain data integrity and avoid integration issues. The database was complied with the security standards on electronic medical data stor- age (ISO 27001) and privacy of data was ensured through dynamic encryption.

Multi-parametric data coming from devices and sensors (dynamic data) were integrated in a centralized database. The distributed approach enabled dynamic information to be added, modified and displayed at the same time, indi- vidually or as a whole in each client application (Fig. 2). Therefore, to avoid persistent misleading, each value was assigned with a unique identifier allowing them to be

distinguished in an efficient and rapid way. Good practices were developed to maximize communication data exchange and memory allocation and an online/offline management system was developed to ensure data synchronization.

During the communication processes between medical devices, data were only transmitted through wireless inter- net connection (mobile network or Wi-Fi) availability. In case of offline sessions, data were locally stored and sent whenever the connection was available. Only new data (no downloaded or sent previously) were synchronized. Integ- rity of data and synchronization has been also considered. Furthermore, unique identifiers for patients and parameters were assigned to provide a high integrity of data allow- ing re-synchronization in case it was needed. Automatic backup routines were also implemented in the central server to guarantee data loss or corruption recovery.

2.4 Evaluation

All screened patients who fulfilled inclusion criteria entered a Baseline Phase of at least 3 days (up to 6 days, based on local monitoring practice). The inclusion criteria for patients were to be categorized for all types of diabe- tes and to have the ability to use mobile devices and sen- sors. During the Baseline Phase, patients recorded meta- bolic data on food and drug intake (through a paper diary), physical activity (through the combined use of OMRON HJ720-IT pedometer and Sensewear Armband), fingerprint glycemic values (Bayer™ Contour Link glucometer) and continuous subcutaneous glucose concentration (Medtronic GuardianRT® continuous glucose monitoring system).

Thirty patients from three EU countries (Spain, Italy and Czech Republic) were planned to be enrolled for the trial. Up to 33 patients were enrolled finally, from which three dropped out, meaning 30 patients completed the trial phase. All patients signed an agreement and were fully informed about the objective of the trial. After the initial baseline, which consisted in a paper track of the monitoring parame- ters, patients were trained during a 2-h session to use PMD and PP to achieve the recommended goals for four consec- utive weeks. Trial duration was 35 ± 6 days. All patients were asked to monitor the complete set of metabolic data during the final 3 (6) days of the intervention phase (includ- ing the paper diary for food and drug intake).

Four follow-up visits (V0–V3) were established for all subjects. The number of days between the chosen visits depended on clinical practice in each country, V1–V0 was 6 ± 2, V2–V1 was 20 ± 4 and V3–V2 was 6 ± 2.

During trials, PMDs, PPs and services usage were evalu- ated trough the follow-up of data increase, services con- sumption, exceptions rise and memory load (virtual and physical). Daemon services were enabled to record locally and remotely the interaction flows within modules and

Table 1 Medical sensors within the platform sorted by the group to which they belong depending on the data access protocol

ID Sensor Group

1 Medtronic GuardianRT® CGMS 2

2 Bayer™ Contour Link glucometer 2

3 Sensewear armband—physical activity sensor 2

4 OMRON HJ-720IT Pedometer 1

5 OMRON MT-10T blood pressure 1

6 AANDD weight scale 3

1299Med Biol Eng Comput (2015) 53:1295–1303

1 3

communications. To perform an automatized analysis, the information was recorded in txt files following a structured format.

The ethical committee of each involved hospital approved the study protocol. No ethical issues were found prior and after the realization of the study.

3 Results

Prior to trial evaluation, the system was tested in laboratory facilities with real devices, servers and switching 3G/Wi-Fi connections obtaining good results on time delays and per- formance for a set of benchmarked operations designated as critical. After these tests, the platform was evaluated in an international multi-center open-label randomized study consisting of two groups, one intervention and other con- trol. Table 2 shows a description of the group which com- pleted the trial. System usefulness and innovative features as data aggregation, PHR tailoring and comprehensive view on health data have been evaluated through a transpar- ent analysis of the components interaction, based on events triggered by patients.

The system was used during 35 ± 6 days between Feb- ruary 2012 and September 2012 by a total set of 30 patients aged 45 ± 11. Table 2 depicts the dataset description on gender distribution, studies level, marital status, comor- bidities and lifestyle. All the patients were able to perform measurements with the sensors and use mobile and desk- top applications, and 90 % of the study population had

secondary or higher education. Patients were assisted by technical staff and their medical practitioner during the study duration.

The main purpose for the system administration, mainte- nance and performance was measured and evaluated under a set of key performance indicators (KPIs). A KPI is intro- duced as significant assessing criteria for system perfor- mance under a specific domain [2, 10] that may be referred by an absolute or relative value. For the purpose of this evaluation, the following KPIs were addressed: total meas- urements and single entries for data aggregation; computa- tional load, time delay and success rate for system applica- tions performance.

The assessment consisted in the analysis of service invo- cation logs, application logs and database mining to evalu- ate the following two features:

3.1 Data aggregation from multiple sensors to support health decisions

This analysis evaluates platform capability of aggregating data from different sensors, displaying them and making them available to health professionals preserving data integ- rity. Data have been analyzed for each patient in two levels; in the client application and the server core. Table 3 shows the number of patients which used each device. Through the PP application, data were downloaded and aggregated in a local database (row 3) and then synchronized to the server a total of 124,875 measurements. Physicians had straight access to server data through the CP, so that was the total

Table 2 Study population dataset description in absolute and relative values divided for total population, type 1 diabetes patients (T1DM) and type 2 diabetes patients (T2DM)

Total (n = 30)

T1DM (n = 20)

T2DM-IT (n = 10)

N % N % N %

Gender Male 19 63 14 70 10 100

Female 11 37 6 30 0 0

Studies Undergraduate 3 10 3 15 1 10

Secondary 10 33 8 40 3 30

University 16 53 17 85 4 40

PhD 1 3 1 5 0 0

Marital status Single 14 47 13 65 3 30

Married 13 43 12 60 7 70

Divorced 2 7 2 10 0 0

Widowed 1 3 0 0 0 0

Comorbidities Infraction 0 0 0 0 0 0

Stroke 2 7 2 10 0 0

Renal failure 0 0 0 0 0 0

Liver failure 0 0 0 0 0 0

Lifestyle Smoking 9 30 4 20 5 50

Alcohol 12 40 8 40 4 40

Restrictive diet 8 27 6 20 2 20

1300 Med Biol Eng Comput (2015) 53:1295–1303

1 3

percentage of data available to him (row 3 shows the per- centage of the patient data visible to health professionals). Outliers have been removed prior to data statistical analy- sis, meaning that only patients who used sensors regularly have been included. The group (n = 30) was told to use the system to retrieve data from the different sensors (glucose, continuous monitor glucose, physical activity).

Two main families of sensors were defined, F1 contain- ing sensors #1,#2 and #3 for T1DM patients and F2 con- taining #2, and #4 for T2DM.

One limitation of the study is that not all the patients used the given sensors, as the second row states. The third row shows the mean and standard deviation of total meas- urements for each sensor and patient device. As expected, continuous monitors had large quantities of data. The variations found in sensor number 2 are due to the fact that T1DM patients only used it to calibrate sensor num- ber 1. Forth row shows the percentage of duplicated data (mean ± SD) and the fifth row the percentage of original data that were finally transmitted to the server, and thus was available to the doctor.

3.2 Uniform access to operational tools

Services wrapping complex tasks in an automatic and user- transparent way (e.g., sensor data synchronization) were been evaluated and analyzed. This evaluation was tackled from two approaches. The first focused on the access to remote functionalities such as web services invocation suc- cess and the second on the performance of the applications in terms of memory (computational) load and time delay on loading the modules in which the applications consisted of.

During the study, 83,905 calls to web services were per- formed, from which, 63 % (n = 53,004) were performed by T1DM and 37 % (n = 30,901) were performed by T2DM patients. Global service invocations distribution was 3,356 ± 2,426, from which, 4,077 ± 2,853 were made by T1DM and 2,575 ± 1,639 were done by T2DM.

From the total of calls, a 97 % (n = 81,294) were identi- fied as 3G or Wi-Fi connection, 64 % (n = 51,814) of them were performed by T1DM and 36 % (n = 29,480) were made by T2DM. Analysis on the type of connection showed that 69 % (n = 56,064) of the total of calls were carried

out through Wi-Fi connections and 31 % (n = 25,230) by 3G. Drilling down on the type of connection and type of diabetes, for T1DM patients 71 % (n = 36,999) were via Wi-Fi and 29 % (n = 14,815) via 3G. In T2DM, 65 % (n = 19,065) used Wi-Fi and 35 % (n = 10,415) 3G con- nection, showing no significant differences.

Regarding the response time, on average, the request delay per call was 1,059 ms. For the iPhone, it took 641 ms, and for the HTC, it was 1,303 ms, the largest requests being the ones related to sensor data synchronization.

To value the reported results, the comparative Table 5 shows the most important features of the referenced stud- ies, SMARTDIAB [21], PFTH [26] and METABO, the system assessed in this manuscript. First of all, it is clear that there is a lack of parameters in [21] on the technical assessment and report of KPI performance. Second, a bet- ter behavior is found in [26] with respect to the data man- agement, even the amount of the dataset is not comparable (4,002 single entries vs. 124,875). The unique KPI depicted in the three studies is the system availability, in which the SOA approach shows the best performance rate.

Due to the system specifications, it is not possible to compare some of the KPI straightforward. For instance, it is senseless to perform a mean of the measurements per patient and compare the result with the value from PFTH. However, it can be compared to each sensor, showing that the diabetes system gathers around forty times the data of each patient for sensor 1.

4 Discussion

Telemonitoring solutions for chronic conditions manage- ment are being assessed currently through multiple clinical [14, 27] and economic studies [16, 19]. Some of the pre- liminary results conclude that the use of these technologies is not related to an improvement in health indicators [3]. Nonetheless, other authors [13] point out that the use of remote monitoring systems has to be selective, prioritizing it for empowered patients or who are undergoing in a treat- ment modification.

The self-management of chronic diseases such as diabe- tes can be boosted by applying user-friendly systems with

Table 3 Data integrity evaluation in terms on mean and standard deviation for the whole trial evaluation

Sensor ID refers to sensors in Table 1

Sensor ID 1 2 3 4

Number of patients 18 (69 %) 19 (73 %) 12 (46 %) 23 (88 %)

Total measurements in patient device 4106,3 ± 3647.2 94.1 ± 74.1 4,027 ± 395 33 ± 7.7 Single entries in patient device (%) 85.9 ± 32.0 83.6 ± 61 43.5 ± 5.8 95 ± 22.36 Measurements in server (%) 79.6 ± 27.7 87.6 ± 16.3 18.4 ± 7.4 93.7 ± 14.9

1301Med Biol Eng Comput (2015) 53:1295–1303

1 3

a low error rate in communications and excellent perfor- mance. Moreover, these technologies would provide large amounts of temporary information to detect patient changes and worsening periods.

In this paper, a novel system for diabetes monitor- ing and management has been presented. The system has introduced two new concepts: the integration of different software environments (enabling patient specific inter- faces to be built) through a service-oriented architecture based and the creation of a platform of sensors that allows for the sharing of data, making it available for patients and professionals.

In this system, patients have been equipped with mobile devices (T1DM: HTC; T2DM: iPhone 3G), desk- top applications and sensors to follow up their medi- cal parameters and normal daily lifestyle. A set of four sensors (Medtronic GuardianRT, Bayer Conotur Link, Sensewear Armband and Omron HJ720 pedometer) has been integrated into the system through the development of a communication API.

Furthermore, the system has been evaluated in a multi- center trial (n = 30) during 35 ± 6 days within a 7-month period. The population sample was categorized by gender, marital status, academic level, the presence of comorbidi- ties and lifestyle. No analysis within these groups has been performed during this study.

The evaluation strategy consisted on performing a tech- nical evaluation of the logs registered during trials and

databases. The data aggregation from different sensors has been evaluated analyzing the integrity of data across the platform. Despite the low performance of sensor #3 (Table 1), a high percentage of non-replied data from dif- ferent sensors was found on both patient and doctor appli- cations, pointing out the success on data aggregation.

The integrity of these values on their exchange through the system has been assessed by comparing the number of entries for each patient and sensor. Results show that the vast majority of these values were correctly transmitted and therefore contribute to highlight the quality of data in terms of integrity and reliability. The best performance has been achieved by sensor #4, revealing that a better level of integration (Group 1 in Table 1) guarantees better level of data aggregation and integrity. The strategy of parsing val- ues to string type and wrap them in ad hoc vector packets stands as a good strategy to avoid data mismatches. Despite being from Group 2 (Table 1), sensor #3 resulted in an extremely low percentage of successful transmissions to the server (Table 3). The reason could be that in the high number of duplicated entries caused by this type of sensor, and errors in the particular logic of the upstream methods that should be revised after the evaluation (the API share methods within the same sensor group). In comparison with the results achieved by [26], the system shows a bad performance in data KPIs, leading to conclude that more methods to guarantee a correct data acquisition should be implemented. Since the availability of the services and their successful transactions rate achieves a 98 %, the problems underneath the incomplete data transfer are localized in the communication protocols with the sensors at a device level. Therefore, better integrity and reliability features (CRC checksum, logical channel control) should be addressed.

Subsequently, server and application logs have been analyzed to evaluate uniform access to technical compo- nents to hide operational complexity. Applications had a good performance on each operative system and device, as memory footprint and delays between modules parameters reveal (Table 4). However, problems were found on T1DM application (running on Windows Mobile 6.5) as the dura- tion of the session was increasing, thus, authors conclude that this operative system is not suitable for complex graph- ical resources management and XML exchange through web services. Communications used within the trial via web services were handled correctly with a 98 % of suc- cessful transactions, even via 3G or Wi-Fi connections

Table 4 Application performance parameters of user applications: memory footprint and time delay

CP T1DM T2DM PP

Computational load (kB) 120,000 (2.9 %) 30,000 (7 %) 15,000 (3 %) 110,000 (2.8 %)

Time delay (s) 5.83 1.63 2.93 2.6

Table 5 Comparison of the study features and reported KPIs

The kindness of the designed system over the referenced architectures is achieved by improving some of the KPI analyzed Time Delay, Sys- tem Availability. Other aspects related to data integrity do not achieve the rates reported by [26]

KPI\intervention SMARTDIAB PFTH METABO

Number of patients – 30 30

Timespan (days) – 24.3 35 ± 6 Total measurements – 4,002 124,875

Measurements per patient – 133 ± 37 Table 3 Duplicated data – – 22,3 %

Data lost – 0 % 30,5 %

Maximum time delay (s) 93.33 – 2

System availability 95 % 96,6 % 98 %

Computational load of the applications

– – Table 4

1302 Med Biol Eng Comput (2015) 53:1295–1303

1 3

from all the system components. The average time delay is acceptable (<2 s). Therefore, masking complex operations with web services in a SOA system has been demonstrated to have a good performance with respect to other architec- tures [21, 26], as Table 5 shows.

The study does not refer to usability and user satisfac- tion metrics, which could enhance the quality of the result by categorizing the system performance toward system acceptance. The main limitation of the study is found in the sensor assignation for each type of user. Due to logis- tic issues, some sensors had to be replaced between the ini- tial sets (F1 and F2) and therefore it has not been possible to report comparisons between these two groups. Another limitation is found on the way to measure security and reli- ability of the system. Other mechanisms should be added to assess in-depth the integrity of data, such as quality vector methodologies. The next step toward a full evaluation is to include a user satisfaction evaluation and clinical assess- ment of the proposed solution.

5 Conclusion

This paper has demonstrated that web services are suit- able to build complex systems which integrate desktop and mobile devices and physiological sensors. The literature is currently focused on determining the cost-effectiveness of telemonitoring systems without approaching a techni- cal evaluation, as it done is this paper. It also presents a novel SOA that merges different types of technologies and monitoring sensors. This paper will help other researchers to perform similar evaluations, focusing on the same KPI, and also to enhance techniques and methods to evaluate the performance of remote monitoring systems in order to set a quantitative framework for assessing the performance of remote monitoring systems. Future work claims to per- form a clinical evaluation of the proposed system in a large study. This study should evaluate the impact of this system in health indicators such HbA1C, hospitalizations, body mass gain and QALY. Another field for future work will be adding qualitative results on user satisfaction and useful- ness of the system.

Acknowledgments The authors wish to acknowledge the consor- tium of the METABO project (funded by the European Commission, Grant nr. 216270) for their commitment during concept development and trial execution.

References

1. Bellazzi R, Larizza C, Montani A et al (2002) A telemedicine support dor diabetes management: the T-IDDM project. Comput Methods Programs Biomed 69:147–161

2. Boloor K, Chirkova R, Salo T, Viniotis Y (2011) Analysis of response time percentile service level agreements in soa-based applications. IEEE global telecommunications conference (GLOBECOM 2011), dec. 2011, pp 1–6

3. Cartwright M et al (2013) Effect of telehealth on quality of life and psychological outcomes over 12 months: nested study of patient reported outcomes in a pragmatic, cluster randomised controlled trial. BMJ 346:f653

4. Chen I-Y et al (2008) Pervasive digital monitoring and trans- mission of pre-care patient biostatics with an OSGi, MOM and SOA based remote health care system. In: Proceedings of the 6th annual IEEE international conference on PerCom. Hong Kong

5. Fico G, Fioravanti A, Arredondo MT, Leuteritz JP, Guillén A, Fernandez D (2011) A user centered design approach for patient interfaces to a diabetes IT platform. Conf Proc IEEE Eng Med Biol Soc 2011:1169–1172

6. Fioravanti A, Fico G, Arredondo MT, Salvi D, Villalar JL (2010) Integration of heterogeneous biomedical sensors into an ISO/ IEEE 11073 compliant application. In: Engineering in medicine and biology society (EMBC), 2010 Annual international confer- ence of the IEEE, pp 1049–1052

7. García Saez G et al (2009) Architecture of a wireless per- sonal assistant for telemedical diabetes care. Int J Med Inform 9(78):391–403

8. Gómez EJ, Hernando ME et al (2008) The INCA system: a fur- ther step towards a telemedical artificial pancreas. IEEE Trans Inf Technol Biomed 12(4):470–479

9. Harrison’s Principles of Internal Medicine (2011) McGraw-Hill. ISBN:978-0071748896. Ed. July 2011

10. Ke X, Li W et al (2010) WCDMA KPI framework definition methods and applications. ICCET proceedings V4-471–V4-475

11. Klonof D (2013) Twelve modern digital technologies that are transforming decision making for diabetes and all areas of health care. J Diabetes Sci Technol 7(2):291–295

12. Lanzola G et al (2007) Going mobile with a multiaccess service for the management of diabetic patients. J Diabetes Sci Technol 1(5):730–737

13. Ma C et al (2006) Empowering patients with essential informa- tion and communication support in the context of diabetes. Int J Med Inform 75(8):577–596

14. Müller AJ, Knuth M, Nikolaus KS, Krivánek R, Küster F, Has- slacher C (2013) First clinical evaluation of a new percutaneous optical fiber glucose sensor for continuous glucose monitoring in diabetes. J Diabetes Sci Technol 7:13

15. Nundy S et al (2012) Using mobile health to support chronic care model: developing an institutional model. Int J Telemed Appl 2012, Art Id 871925. doi:10.1155/2012/871925

16. Obstfelder A, Engeseth KH, Wynn R (2007) Characteristic of succesfully implemented telemedical applications. Implement Sci 2:25

17. Pravin P et al (2012) A framework for the comparison of mobile patient monitoring systems. J Biomed Inf 45:544–556

18. Reichel A, Rietzsch H, Ludwig B, Röthig K, Moritz A, Born- stein S (2013) Self-adjustment of insulin dose using graphi- cally depicted self-monitoring of blood glucose measurements in patients with type 1 diabetes mellitus. J Diabetes Sci Technol 7(1):156–162

19. Ryan D et al (2012) Clinical and cost effectiveness of mobile phone supported self-monitoring of asthma: multicenter rand- omized controlled trial. BMJ 344:e1756

20. Schade DS et al (2005) To pump or not to pump. Diabetes Tech- nol Therapeutics 7:845–848

21. Stravroula G, Bartsocas CS et al (2010) SMARTDIAB: a com- munication and information technology approach for the intelli- gent monitoring, management and follow-up of type 1 diabetes patients. IEEE Trans Inf Technol Biomed 14(3):622–633

1303Med Biol Eng Comput (2015) 53:1295–1303

1 3

22. The Diabetes Control and Complications Trial Research Group (1993) The effect of intensive treatment of diabetes on the devel- opment and progression of long-term complications in insulin- dependent diabetes mellitus. N Engl J Med 329(14):977–986

23. Trief PM, Morin PC, Izquierdo R, Teresi JA, Eimicke JP, Goland R, Starren J, Shea S, Winstock RS (2006) Depression and glycae- mic control in elderly etchnically diverse patients with diabetes: the IDEATel project. Diabetes Care 29(4):830–835

24. van der Weegentres S et al (2013) The development of a mobile monitoring and feedback tool to stimulate physical activity of people with a chronic disease in primary care: a user-centered design. JMIR 1(2):e8

25. Wakefield BJ et al (2014) Effect of home telemonitoring on gly- cemic and blood pressure control in primary care clinic patients with diabetes. Telemed e-Health 20(3):199–205. doi:10.1089/ tmj.2013.0151

26. Winkler S et al (2011) A new telemonitoring system intended for chronic heart failure patients using mobile technology—Feasibil- ity Study. Int J Cardiol 153:55–58

27. Zhou YY, Kanter MH, Wang JJ, Garrido T (2010) Improved qual- ity at kaiser permanente through e-mail between physicians and patients. Health Aff 29(7):1370–1375

Copyright of Medical & Biological Engineering & Computing is the property of Springer Science & Business Media B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.