Abstract
This chapter focuses on the fundamental principles of conformance testing. Although the concepts of conformance testing are the key focal points addressed here, they can’t be discussed in isolation; thus, we include related concepts when appropriate. A testing life cycle and process are presented along with their relationship to the standards development life cycle. An important principle in standards development is to integrate testing early in the process in order to obtain feedback for the authors of the standard. A testing methodology framework is introduced that provides a process for developing, organizing, and managing tests, as well as conducting testing and analyzing the results. A detailed description and example of a Test Plan is given. Since our focus is on communication between distributed applications, we discuss how sending and receiving applications can be tested, using a laboratory test results case study as the context. Next, we offer a set of basic principles for developing Test Plans and Test Cases. Finally, a comparison between capability testing and site-testing is presented. The conformance testing principles discussed in this chapter are applied in Chap. 13 and are examined in relation to test tool implementations in Chap. 14.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Both coded and un-coded.
- 2.
Meaning that there is only one system tested in isolation. Conformance testing may proceed multiple times until the system passes all Test Cases.
- 3.
For all specifications except the most rudimentary; in the healthcare space most specifications are complex.
- 4.
If this is the case, interoperability initiatives may be impaired.
- 5.
In the context of this discussion; interfaces engines serve other purposes as well.
- 6.
This is the goal; however, partial functionally is also desirable and useful.
- 7.
Office of the National Coordinator.
- 8.
Some messaging implementation guides include functional requirements; IHE integration profiles combine interoperability and functional requirements. In some cases, however, an interoperability specification is devoid of functional requirements; they are determined by the implementer’s business operations.
- 9.
We explain concepts in this section in the context of HL7 v2 messages; however, the concepts apply equally to documents and other message protocols.
- 10.
Usage is R-Required when the condition predicate is true and the usage is X-Not Supported when the condition is false.
- 11.
Not be confused with the “RE” usage construct.
- 12.
The case study is for ONC 2014 Edition Certification in which EHRs are being certified. The case study is showing an EHR-S with an LIS module. The testing can equally apply to independent LISs.
- 13.
In some cases, it may be necessary for the test tool to allow for variances, e.g., LOINC equivalencies. These exceptions are decided by the test administrator.
- 14.
This is not desirable, but it may be useful if a vendor already supports a proprietary interface and not a standard interface.
- 15.
Even without this assumption, the response only will deliver the correct responses if this interface is functioning correctly.
- 16.
Although this scenario is not typical it is valid. In most cases the EHR-S lab module would directly create the ELR message.
- 17.
Testing of the Acknowledgements is not part of ONC 2014 Edition certification testing; however, it is planned for future releases of the test tool (and should be included in testing).
- 18.
However, “Implied Verification” elements can have an impact on the juror document. For example, results related to parent-child linking have a certain organization that must be met.
- 19.
Use of test data and the associated test data categories can be used to assess all of the requirements imposed by the various conformance constructs (including usage, cardinality, vocabulary, and length).
- 20.
Testing of content should also include the handling of encoding characters.
- 21.
It is important to note that the set of requirements are drawn from the specification and any specification it references or is derived from. For example, in HL7 v2 a conformance profile is defined that is based on the underlying HL7 v2 standard. Requirements from the base standard are brought forward and must be supported unless they are replaced (profiled) by the conformance profile definition.
- 22.
The recommendation is not to determine core (or crucial) requirements arbitrarily, but to test a functionality without being completely exhaustive for every possible instance. A simple example is cardinality, for which it is sufficient to test to 10 instances of an element if the requirement is to support 10 instances. It is of little value to test that it can support exactly 8 instances or exactly 9 instances. (Please see boundary testing later in this chapter.)
- 23.
The term “site” in this context can mean a single site, multiple sites, or a group of sites that define the same (implementation) profile requirements.
- 24.
For simplicity of the example, required precision for the date of birth is to the day.
- 25.
Additional requirements would also be specified to account for the number of days that can occur for a given month.
- 26.
For example, a calendar dialog could be used to select a date, or a series of drop down boxes that restrict the user’s input to valid entries could be used.
- 27.
In conformance to another set or sets of requirements for handling and use of the date.
- 28.
A contrived example might be that the LIS was configured so the Preliminary result required verification by a pathologist before it could be released for transmission to the EHR-S, and the pathologist did not verify it, so it was not transmitted; then, the Final result was produced, the pathologist verified it, and this Final version of the result was transmitted to the EHR-S. The pathologist then realizes s/he hadn’t verified the Preliminary version of the result, and now they verify it, and it is transmitted to the EHR-S AFTER the Final version of the result was transmitted.
- 29.
In HL7 v2, this is expressed as the conformance length in a constrainable profile.
- 30.
This is a grey area, and it is not always clear where the delineation should occur. Often such requirements do belong not in the interoperability specification but in functional requirement guides or defined as business rules. Where such requirements are defined is not of concern; that such requirements are specified and tested is important.
References
Gebase L, Snelick R. Testing Environments for Assessing Conformance and Interoperability. 2010 Software Engineering Research and Practice (SERP10), WORLDCOMP’10 July 12-15, 2010, Las Vegas, NV.
Monkewich O: Tutorial on Conformance and Interoperability Testing. International Telecommunication Union (ITU), December 8th, 2006.
Hogan MD, Fang Liu, Sokol AW, Tong Jin. NIST-SP 500-291, NIST Cloud Computing Standards Roadmap. August 10, 2011; http://www.nist.gov/manuscript-publication-search.cfm?pub_id=909024
HL7 Version 2.5.1 Implementation Guide: S&I Framework Lab Results Interface interoperability standards (DSTU), July 2012; http://www.hl7.org
National Institute of Standards and Technology (NIST): Lab Results Interface (LRI) DSTU Release 1 HL7 v2 Conformance Test Tool. http://hl7v2-lab-testing.nist.gov
National Institute of Standards and Technology (NIST): Immunization Messaging, Release 1.4 HL7 v2 Conformance Test Tool ONC 2014 Edition Certification Testing. http://hl7v2-iz-testing.nist.gov
National Institute of Standards and Technology (NIST): Immunization Messaging, Release 1.5 HL7 v2 Conformance Test Tool. ONC 2015 Edition Certification Testing http://hl7v2-iz-r1.5-testing.nist.gov
National Institute of Standards and Technology (NIST): PHIN Messaging Guide For Syndromic Surveillance: Emergency Department and urgent Care Data HL7 v2 Conformance Test Tool. Contact: rsnelick@nist.gov; http://hl7v2-ss-testing.nist.gov
National Institute of Standards and Technology (NIST): Electronic Lab Reporting (ELR), Release 1 HL7 v2 Conformance Test Tool. http://hl7v2-elr-testing.nist.gov
Meaningful Use Immunization On-Boarding Instructions; State of Arkansas. http://www.healthy.arkansas.gov/programsServices/MeaningfulUse/Documents/ImmunizationOn-BoardingInstructions.pdf
HL7 Version 2.5.1. Implementation Guide for Immunization Messaging. March 2014. http://www.cdc.gov/vaccines/programs/iis/technical-guidance/downloads/hl7guide-1-5-Mar2014.pdf
National Institute of Standards and Technology (NIST): Meaningful Use Stage 2, 2014 Edition, Approved Test Procedures http://healthcare.nist.gov/
HL7 EHR-S Functional Requirements: S&I Framework Laboratory Results Messages, Release 1, US Realm Draft Standard for Trial Use March 2016; http://hl7.org
Smoke Testing (Software) https://en.wikipedia.org/wiki/Smoke_testing_(software)
DIN ISO 7498. Informationsverarbeitung Kommunikation Offener Systeme, Basis-Referenzmodell. DIN EN ISO 7498-1 ISO/OSI-Modell, Beuth Verlag, 1982
HL7 EHR-S Functional Model Specification. http://www.hl7.org
Bunker N. Data Quality Assurance (DQA) Test tool for Immunization. https://openimmunizationsoftware.net/dataQuality/dataQuality.html
ISTQB Exam Certification. http://istqbexamcertification.com/what-is-usability-testing-in-software-and-its-benifits-to-end-user/
Lowry SZ, Quinn MT, Mala Ramaiah, et al. Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records. February 12th, 2012. NIST Interagency/Internal Report (NISTIR) – 7804
HL7 Version 2.5.1 Implementation Guide: Electronic Laboratory Reporting to Public Health, Release 1 (US Realm); February 2010. http://www.hl7.org
Test Procedure for §170.314(b)(5)(A) Incorporate laboratory tests and values/results. http://www.healthit.gov/sites/default/files/170.314b5aincorporatelabtests_2014_tp_approved_v1.4_onc.pdf
Test Procedure for §170.314(f)(4) Inpatient setting only – transmission of reportable laboratory tests and values/results. http://www.healthit.gov/sites/default/files/170.314f4transmissionreportablelabs_tp_2014_approved_v1.3.pdf
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Oemig, F., Snelick, R. (2016). Principles of Conformance Testing. In: Healthcare Interoperability Standards Compliance Handbook. Springer, Cham. https://doi.org/10.1007/978-3-319-44839-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-44839-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44837-4
Online ISBN: 978-3-319-44839-8
eBook Packages: Computer ScienceComputer Science (R0)