Active – Passive: On Preconceptions of Testing
DOI:
https://doi.org/10.26636/jtit.2011.3.1159Keywords:
behavior, development, metrology, monitoring, passive testing, reactive systems, Scientific MethodAbstract
In telecommunications and software engineering, testing is normally understood to be essentially active: a tester is said to stimulate, control, and enforce. Passive testing does not fit this paradigm and thus remains the niche research subject, which bears on the scope and depth of the obtained results. It is argued that such limited understanding of testing is one of its many community-bound preconceptions. It may be acceptable in the current engineering approach to testing, but can and should be challenged in order to converge on the core concepts of the proposed science of testing (“testology”). This methodological work aims at establishing that there are no fundamental reasons for admitting the dominant role of the active element in testing. To show this, external (also extratechnical) areas are consulted for insight, direct observations, and metaphors. The troublesome distinction between (passive) testing and monitoring, as well as unclear relations between testing and measurements, are also addressed.
Downloads
References
[1] K. M. Brzeziński, „On common meta-linguistic aspects of intrusion detection and testing”, J. Inform. Assur. Secur., vol. 2, no. 3, pp. 167-178, 2007.
View in Google Scholar
[2] K. M. Brzeziński, „A joint meta-linguistic taxonomy of intrusion detection and testing/verification”, in Proc. 2nd Int. Worksh. Secure Inform. Syst. SIS’07, Wisła, Poland, 2007.
View in Google Scholar
[3] K. M. Brzeziński, „Towards the methodological harmonization of passive testing across ICT communities”, in Engineering the Computer Science and IT, S. Soomro, Ed. InTech, 2009, pp. 143-168.
View in Google Scholar
[4] K. M. Brzeziński, „On conceptual struggles over ‘testing’”, in Proc. XIV Poznańskie Warsztaty Telekomunikacyjne PWT 2010, Poznań, Poland, 2010.
View in Google Scholar
[5] K. M. Brzeziński, „Standards are signs”, in Proc. 15th EURAS Ann. Standardization Conf., Lausanne, Switzerland, 2010, pp. 43-60.
View in Google Scholar
[6] Software testing. Vocabulary. BS 7925-1. British Standards Institution, 1998.
View in Google Scholar
[7] Standard Glossary of Terms Used in Software Testing, version 2.0. ISTQB (Glossary Working Party), 2007.
View in Google Scholar
[8] Certified Tester. Foundation Level Syllabus. ISTQB, 2007.
View in Google Scholar
[9] Certified Tester. Advanced Level Syllabus. ISTQB, 2007.
View in Google Scholar
[10] A. Spillner, T. Linz, and H. Schaefer, Software Testing Foundations. Rocky Nook, 2007.
View in Google Scholar
[11] G. Bath and J. McKay, The Software Test Engineer’s Handbook. Rocky Nook, 2008.
View in Google Scholar
[12] C. F. Tschudin, „On the structuring of computer communications”, Ph.D. dissertation, University of Geneva, 1993.
View in Google Scholar
[13] VIM, International vocabulary of metrology - Basic and general concepts and associated terms. Joint Committee for Guides in Metrology (JCGM), 2008, vol. JCGM 200.
View in Google Scholar
[14] R. R. Hoffman, Metaphor in Science. Lawrence Erlbaum Associates, Inc., 1980, pp. 393-423.
View in Google Scholar
[15] S. Purao et al., „The sciences of design: observations on an emerging field”, Harvard Business School, Working Paper 09-056, 2008.
View in Google Scholar
[16] K. Naik and P. Tripathy, Software Testing and Quality Assurance: Theory and Practice. Wiley, 2008.
View in Google Scholar
[17] L. Cacciari and O. Rafiq, „Controllability and observability in distributed testing”, Inform. Software Technol., vol. 41, no. 11-12, pp. 767-780, 1999.
View in Google Scholar
[18] C. Sunshine, „Formal techniques for protocol specification and verification”, Computer, vol. 12, no. 9, pp. 20-27, 1979.
View in Google Scholar
[19] IEEE Standard Glossary of Software Engineering Terminology. IEEE Std 610-12. IEEE, 1990.
View in Google Scholar
[20] Guide to the Software Engineering Body of Knowledge. SWEBOK, IEEE, 2004.
View in Google Scholar
[21] L. Frantzen and J. Tretmans, „Model-based testing of environmental conformance of components”, in Formal Methods of Components and Objects (FMCO’06), LNCS 4709. Springer, 2007, pp. 1-25.
View in Google Scholar
[22] E. Brinksma and J. Tretmans, „Testing transition systems: an annotated bibliography”, in Proc. MOVEP 2000, Nantes, France, 2000, pp. 187-195.
View in Google Scholar
[23] E. Brinksma, W. Grieskamp, and J. Tretmans, „Summary”, in Perspectives of Model-Based Testing of Dagstuhl Seminar Proceedings, E. Brinksma, W. Grieskamp, and J. Tretmans, Eds., no. 04371. IBFI, 2005.
View in Google Scholar
[24] M. Utting, A. Pretschner, and B. Legeard, „A taxonomy of model-based testing”, Univ. of Waikato, Hamilton, New Zealand, Working Paper 04/2006, 2006.
View in Google Scholar
[25] J. Ryser, S. Berner, and M. Glinz, „On the state of the art in requirements-based validation and test of software”, Tech. Rep. IFI-98.12, Univ. of Zurich, May 1998.
View in Google Scholar
[26] Conformance Testing Methodology and Framework. ISO/IEC 9646. ISO/IEC, n.d., vol. 1-7.
View in Google Scholar
[27] MTS; The Testing and Test Control Notation version 3. ETSI ES 201 873, ETSI, n.d.
View in Google Scholar
[28] K. M. Brzeziński, „Intrusion detection as passive testing: linguistic support with TTCN-3”, in DIMVA, LNCS 4579. Lucerne: Springer, 2007, pp. 79-88.
View in Google Scholar
[29] K. M. Brzeziński, D. Mastalerz, and R. Artych, „Practical support of testing activities: the PMM Family”. COST 247 WG3 Internal Report, IT P.W., LTiV Tech. Rep. 965, 1996.
View in Google Scholar
[30] K. M. Brzeziński, „Weryfikacja i testowanie”, Przegląd Telekomunikacyjny i Wiadomości Telekomunikacyjne, no. 4, pp. 139-140, 2010 (in Polish).
View in Google Scholar
[31] K. M. Brzeziński, „Testowanie w cyklu życia systemu: nieregularności meta-standaryzacji”, in Krajowe Sympozjum Telekomunikacji i Teleinformatyki (KSTiT), Warszawa, 2009 (in Polish).
View in Google Scholar
[32] M.-C. Gaudel, „Formal methods and testing: hypotheses, and correctness approximation”, Formal Methods, pp. 2-8, 2005.
View in Google Scholar
[33] K. M. Brzeziński, A. Gumieniak, and P. Jankowski, „Passive testing for reverse engineering: specification recovery”, in Proc. IASTED Int. Conf. Paral. Distrib. Comput. Netw. PDCN 2008, Innsbruck, Austria, 2008, pp. 27-32.
View in Google Scholar
[34] K. Popper, Conjectures and Refutations: The Growth of Scientific Knowledge. London: Routledge, 1963.
View in Google Scholar
[35] L. Heerink and E. Brinksma, „Validation in context”, in Proc. 15th IFIP Int. Symp. Protocol Specification, Testing and Verification PSTV 1995, Warsaw, Poland, 1995, pp. 221-236.
View in Google Scholar
[36] Framework on Formal Methods in Conformance Testing. ITU-T Z500. ITU-T, May 1997.
View in Google Scholar
[37] M. Heidelberger, „Experimentation and instrumentation”, in Encyclopedia of Philosophy, D. M. Borchert, Ed. Thomson Gale, 2006, vol. 10, pp. 12-20.
View in Google Scholar
[38] J. S. Mill, Of Observation and Experiment. Routledge and Kegan Paul, 1974.
View in Google Scholar
[39] J. A. Arnedo, A. R. Cavalli, and M. Núñez, „Fast testing of critical properties through passive testing”, in Proc. IFIP Int. Conf. Testing Commun. Syst. TestCom 2003, Sophia Antipolis, France, 2003, pp. 295-310.
View in Google Scholar
[40] R. Dssouli and R. Fournier, „Communication software testability”, in Protocol Test Systems III. North-Holland, 1991, pp. 45-55.
View in Google Scholar
[41] K. Svozil, „Extrinsic-Intrinsic Concept and Complementarity”, in Inside Versus Outside, H. Atmanspacher and G. J. Dalenoort, Eds. Berlin: Springer, 1994, pp. 273-288.
View in Google Scholar
[42] P. G. Neumann, „Cause of AT&T network failure”, Risks Dig., vol. 9, no. 62, 1990.
View in Google Scholar
[43] M. Boahene, „Information systems development methodologies: are you being served?” in Proc. 10th Australasian Conf. Information Syst., Wellington, New Zealand, 1999, pp. 88-99.
View in Google Scholar
[44] Webster’s Encyclopedic Unabridged Dictionary of the English Language. Gramercy Books, 1996.
View in Google Scholar
[45] R. Wvong, „A new methodology for OSI conformance testing based on trace analysis”, Master’s thesis, University of British Columbia, 1990.
View in Google Scholar
[46] G. von Bochmann and O. B. Bellal, „Test result analysis with respect to formal specifications”, in Proc. 2nd. Int. Worksh. Protocol Test Syst., Berlin, Germany, 1989, pp. 272-294.
View in Google Scholar
[47] D. Lee, A. N. Netravali, K. K. Sabnani, B. Sugla, and A. John, „Passive testing and applications to network management”, in Int. Conf. Netw. Protoc. ICNP’97, Atlanta, USA, 1997, pp. 113-122.
View in Google Scholar
[48] J. Tretmans, „Testing concurrent systems: a formal approach”, in 10th Int. Conf. CONCUR’99, Eindhoven, The Netherlands, 1999, pp. 46-65.
View in Google Scholar
[49] K. M. Brzeziński, „Towards Practical Passive Testing”, in Proc. IASTED Int. Conf. Paral. Distrib. Comput. Netw. PDCN 2005, Innsbruck, Austria, 2005, pp. 177-183.
View in Google Scholar
[50] M. Dilman and D. Raz, „Efficient reactive monitoring”, IEEE J. Select. Areas Commun., vol. 20, no. 4, pp. 668-676, 2002.
View in Google Scholar
[51] L. Mari, „The role of determination and assignment in measurement”, Measurement, vol. 21, no. 3, pp. 79-90, 1997.
View in Google Scholar
[52] Evaluation of Measurement Data - Guide to the Expression of Uncertainty in Measurement. GUM. Joint Committee for Guides in Metrology (JCGM), 2008, vol. JCGM 100.
View in Google Scholar
[53] „Metrology for Information Technology (IT)”. NISTIR 6025. NIST, White paper, 1997.
View in Google Scholar
[54] Conformity Assessment - Vocabulary and General Principles. ISO/IEC 17000, ISO/IEC, 2004.
View in Google Scholar
[55] A. C. Baratto, „Measurand: a cornerstone concept in metrology”, Metrologia, vol. 45, pp. 299-307, 2008.
View in Google Scholar
[56] R. M. Olejnik, „Kod pocztowy jako przykład metrologicznej skali nominalnej”, Pomiary Automatyka Robotyka, no. 7-8, pp. 186-188, 2004 (in Polish).
View in Google Scholar
[57] T. S. E. Maibaum, „The epistemology of validation and verification testing”, in Proc. 17th IFIP Int. Conf. TestCom 2005, Montreal, Canada, 2005, pp. 1-8.
View in Google Scholar
Downloads
Submitted
Published
Issue
Section
License
Copyright (c) 2023 Journal of Telecommunications and Information Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
