Comparison - Health Data Query Systems
Introduction:
Importance of data analysis -
Data collection, analysis, and dissemination are core public health functions.
Historically, data were analyzed with generic statistical software.
However, that process is unreliable and tedious, and only available to a few expert users.
Static published reports are helpful, but give the user no choice for customizing the analyses, or an easy way to incorporate the results into reports.
Static online electronic reports are more helpful, but again give the user no choice for customization.
Web-based query systems make best use of public health data sets.
Therefore, many agencies have tried to develop web-based query systems.
Purpose of comparison -
Vitalnet is a professional data analysis solution,
with both desktop and web-based front ends.
The purpose of this evaluation is to objectively compare the capabilities
of different Vitalnet platforms with each other,
and possibly with one or more prominent state-developed web-based query systems.
Methods:
Systems included -
We included several fully functioning Vitalnet desktop and web-based platforms in the comparison.
For comparison, we might include one or more long-established state-developed web systems.
We selected mortality data as a convenient data set for comparing systems.
Capabilities and Functions -
To compare the systems, we identified 59 "positive capabilities" that can be
clearly marked "YES" or "NO".
We used criteria in the 2005 report
"Current Status and Desirable Functionalities of State Web-based Data Query Systems",
by Friedman and Parrish, as a starting point.
We tried to only include positives that:
1) are not trivial
(eg, only include the most important chart customizations),
and
2) can be objectively scored "YES/NO"
(eg, important yet subjective criteria such as "ease of use" were excluded).
Negative characteristics -
We also identified 3 significant "negative characteristics",
obvious faults that should not be present.
We tried to only include negatives that:
1) are not trivial,
and
2) can be objectively scored "YES/NO".
Scoring the criteria -
We made a concerted effort to find every positive capability in the systems.
For example, we wanted to include "SVG maps" as a capability promised
by one of the State systems, but were unable to produce that capability.
For each system,
we added a point for each "positive capability" provided,
no action (zero change) if the "positive capability" was missing,
and subtracted a point for each "negative characteristic" present.
Summary Table of Results:
|
System |
Overall Score |
System Environment |
VPC |
VitalPro for Win32 |
 |
97% |
For Windows PC
|
STD |
VitalWeb Standard |
 |
88% |
For web browser
|
WZD |
VitalWeb Wizard |
 |
88% |
For web browser
|
AJX |
VitalWeb Ajax |
 |
97% |
For web browser
|
OTH |
Other system |
|
|
|
Detailed Table of Results:
Discussion:
In an objective comparison of capabilities,
the Vitalnet systems score very high.
Vitalnet offers the opportunity for any agency to avoid or get
out of the expensive, frustrating, and failure-prone software development business.
Vitalnet allows the agency to get best value, provide better services,
make operations smoother, and lower costs.
There are possible limitations or room for expansion of the comparisons, including:
1) Ease of use is not measured:
Ease of use is a top priority with Vitalnet, including extensive user testing.
However, controlled usability comparisons between the systems have not been carried out.
This kind of user analysis is subjective and fraught with great difficulty.
Readers are encouraged to
try Vitalnet
and come to their own conclusions.
2) Output quality is not measured:
Vitalnet output is clean, documented, publication-ready, and correct.
However, output quality is difficult to quantify.
An objective comparison would require great expertise and care.
Readers are encouraged to look at
Vitalnet output
and come to their own conclusions.
3) Criteria are not prioritized:
Priorities could possibly be assigned to the different capabilities.
For example, "makes YPLL" might be worth 2 points, and "does mean age of death" 1 point.
Or perhaps vice versa.
4) Always possible we missed something:
We took care to use basic capabilities, and verify that capabilities were present (or absent).
But some may choose to include other criteria, or exclude various capabilities.
Please
let us know if any suggestions or comments.
Regardless of the possible improvements to this kind of analysis, the basic point is clear,
that Vitalnet provides the needed capabilities for analyzing / disseminating large, complex data sets.