 | Expert Health Data Programming We care about your health data | Contact Us |
Comparison - Health Data Query Systems |
Print-Friendly | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Table of Contents: Introduction · Methods · Results · Discussion Summary Table of Results · Detailed Table of Results Introduction: Data collection, analysis, and dissemination are core public health functions. Historically, data were analyzed with generic statistical software. However, that process is unreliable and tedious, and only available to a few expert users. Published reports are valuable, but give the user no choice for customizing the analyses. Web-based query systems can make much better use of public health data sets. Therefore, many agencies have tried to develop web-based query systems.
Vitalnet is a commercial data analysis solution, with both
desktop and web-based front ends.
The purpose of this evaluation is to objectively compare the capabilities
of Vitalnet with the two most prominent and well-funded state-developed
web-based query systems.
Systems included - For completeness, we included both Vitalnet desktop and web-based platforms in the comparison. It should be noted that VitalPro for Windows is under development, and VitalPro for Unix is mostly used for testing. For comparison, we included the two most well-funded and long-established State-developed web systems. We selected mortality data as a convenient data set for comparing systems. Systems excluded - We did not include generic statistical software (such as SAS or SPSS), because they require an expert to run and need extensive programming to be able to produce population-based rates. The only other commercial software for this purpose we are aware of merely displays pre-tabulated spreadsheet-type displays, so we excluded from the analysis as too limited. Positive Capabilities - To compare the systems, we identified 65 "positive capabilities" that can be clearly marked "YES" or "NO". We used criteria in the 2005 report "Current Status and Desirable Functionalities of State Web-based Data Query Systems", by Friedman and Parrish, as a starting point. We only included capabilities that: 1) are not trivial, 2) can be objectively scored "YES/NO", and 3) are in at least one, but not all, of the systems compared. For example, 1) only the most important chart customizations were included, as others might be considered trivial, 2) subjective criteria such as "ease of use" cannot be objectively scored "YES/NO" and were excluded, and 3) "counts" was excluded because all systems scored "YES". Negative characteristics - We also identified 3 significant "negative characteristics", obvious faults that should not be present. We only included negatives that: 1) are significant (not trivial), 2) can be objectively scored "YES/NO", and 3) are in at least one of the systems compared.
Scoring the criteria -
We made a concerted effort to find every positive capability in the systems.
For example, we wanted to include "SVG maps" as a capability promised
by one of the State systems, but were unable to produce that capability.
For each system,
we added a point for each "positive capability" provided,
and subtracted a point for each "negative characteristic" present.
Please refer to Summary Table of Results and Detailed Table of Results below. 97% - VitalPro for Windows (VFW) (desktop) under development, scored highest. 92% - VitalWeb Ajax (AJX) (browser) scored second highest. The ICD system is under development. 89% - VitalPro for DOS (VPD) (desktop) was close behind. It does not make maps. 89% - VitalWeb Wizard (WZD) (browser) makes maps and charts, and runs step-by-step. 85% - VitalWeb Standard (STD) (browser) does not make multi-tables. 62% - VitalPro for UNIX (VPU) (desktop) is mostly used for testing. 31% - State system #1 (browser) scored much lower. It lacks many needed capabilities.
9% -
State system #2
(browser)
scored very low.
It only includes a few needed capabilities.
In an objective comparison of capabilities, the Vitalnet systems score much higher than the two most prominent and well-funded state systems. Vitalnet offers the opportunity for any agency to avoid or get out of get out of the expensive, frustrating, and failure-prone software development business. Vitalnet allows the agency to get best value, provide better services, make operations smoother, and lower costs. There are possible limitations or room for expansion of the comparisons, including: 1) Ease of use is not measured: Ease of use is a top priority with Vitalnet, including extensive user testing. However, controlled usability comparisons between the systems have not been carried out. This kind of user analysis is subjective and fraught with great difficulty. Readers are encouraged to try Vitalnet and come to their own conclusions. 2) Output quality is not measured: The Vitalnet output is clean, documented, publication-ready, and correct. The output from both the other systems have obvious deficiencies. However, output quality is difficult to quantify. An objective comparison would require great expertise and care. Readers are encouraged to look at Vitalnet output and come to their own conclusions.
3) Criteria are not prioritized:
Priorities could be assigned to the different capabilities.
For example, "makes YPLL" might be worth 2 points, and "does mean age of death" 1 point.
Or perhaps vice versa.
In any case, the general trend would remain that the Vitalnet systems
far outperform the others.
Detailed Table of Results:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
