SYSTEST - Project Status
Return to Project 01-05 description
SYSTEST is a three-year project that started in March 2002. The project
successfully completed its second of three years in March 2004. The project
is on schedule and has been successful in developing usable methods and
tools for system verification, validation, and testing (VVT). The primary
products of SYSTEST are:
- VVT Methodology Guidelines (VVTMG) - a 400+ page document containing
general and specific guidelines for VVT.
- VVT Process Model (VVTPM) - a software set that allows evaluating
a VVT strategy in terms of its impact on cost, schedule, and risk.
- Dissemination papers - a series of professional papers and presentations
describing the SYSTEST results.
The VVTMG and VVTPM will be released for public use following the completion
of the project. They are currently being evaluated in pilot projects as
part of Work Package 6 (WP6) within each of the commercial partners. The
figure below shows the project plan and status. All project tasks are
Milestones and Accomplishments
Major milestones and accomplishments are:
Task or Deliverable
||Report on VVT state of the art
||SYSTEST Project Technical Requirements
||Evaluation of COTS tools
||Methodology Assessment Criteria
||VVT Methodology Guidelines (First Draft)
||VVTMG Assimilation Kit (for pilot projects)
||VVT Process Model
||Start use of VVTMG and VVTPM on pilot projects
||Integrated platform for VVT Process Model
||First Internal SYSTEST Users Group Meeting
||Assessment/Evaluation of the Assimilation Process (on pilot projects)
||VVT Methodology Guidelines (Second Draft)
||European SYSTEST Users Group Meeting (at INCOSE symposium, Toulouse)
||Complete pilot projects evaluation
||VVT Methodology Guidelines (final)
||VVT Process Model (final)
Papers and Presentations
More technical information on SYSTEST and its results can be found in
the following extensive list of papers and presentations that have resulted
from the project:
- SYSTEST Project was presented at the 3rd European Systems Engineering
Conference in Toulouse, France on 22-24, May 2002. Same presentation
has been given at the INCOSE 2002, the 12th International Symposium
in Las Vegas, Nevada, USA, on 28 Jul - 1 Aug. 2002.
- VOLLERTHUN A., ENGEL A., HASKINS C. “Methodology Development
for Systems Verification, Validation & Testing”, Conference
paper presented at the 12th Annual International Symposium of INCOSE,
Las Vegas, USA, 2002.
- ENGEL A. "A Methodology for Modeling VVT Risk & Cost”.
Conference paper presented at the 13th Annual International Symposium
of INCOSE in Washington, DC, USA. June 29 - July 3, 2003.
- HOPPE M., LÉVÁRDY V., VOLLERTHUN A., WENZEL S. "Interfacing
the Verification, Validation, and Testing Process Model with Product
Development Methods". Conference paper presented at the 13th Annual
International Symposium of INCOSE in Washington, USA. July 1-3, 2003.
- BUEHLER O., WEGENER J. “Evolutionary Functional Testing of
an Automated Parking System”. International Conference on Computer,
Communication and Control Technologies: CCCT '03 and The 9th. International
Conference on Information Systems Analysis and Synthesis: ISAS '03,
Orlando, Florida, USA. July 31- August 2, 2003.
- HOPPE M., LÉVÁRDY V., IGENBERGS, E., "Das VVT
Process Model für die Verifikations- und Validierungsplanung".
Paper published in the Conference proceedings of the 15th Annual Spring
Academy of the TU Munich and TU Budapest Balatonfüred, Hungary.
May 17-21, 2003.
- BARESEL A., POHLHEIM H., SADEGHIPOUR S., “Structural and Functional
Sequence Test of Dynamic and State-Based Software with Evolutionary
Algorithms”. Proceedings of the Genetic and Evolutionary Computation
Conference - GECCO 2003, Chicago, Illinois, USA; pp. 2428 - 2441. July,
- BARESEL A., STHAMER H. “Evolutionary Testing of Flag Conditions”.
Proceedings of the Genetic and Evolutionary Computation Conference -
GECCO 2003, Chicago, Illinois, USA; pp. 2442 - 2454. July, 2003.
- ENGEL A., BARAD M. "A Methodology for Modeling VVT Risks and
Costs”. Systems Engineering, Volume 6, issue 3, pp. 135 –
151, Wiley InterScience, Online ISSN: 1520-6858, Print ISSN: 1098-1241.
- LÉVÁRDY V., HOPPE M., WENZEL S., VOLLERTHUN A. “Process
Modeling Procedure for Verification, Validation and Testing Planning”.
Conference paper was presented at the ASME 2003 Design Engineering Technical
Conferences and Computers and Information in Engineering Conference
(DETC2003/DTM-48682) Chicago. September 2-6, 2003.
- LÉVÁRDY V., HOPPE M., “Systemtechnische Methodik
zur Verifikations- und Validierungsplanung - Zwischenergebnisse des
EU-Projekts SysTest”. Presented at the 3rd German Day of Systems
Engineering, Munich, Germany. November 28, 2003.
- BARESEL A., CONRAD M., SADEGHIPOUR S., WEGENER J. “The Interplay
between Model Coverage and Code Coverage”. Paper to appear in
the proceedings of the International European Conference on Software
Testing, Analysis and Review. EuroSTAR 03.
- SYSTEST project was presented by Avner Engel from IAI at the 4th
Israeli System Engineering Conference in Haifa, Israel on 7 January
2004, as part of “The Modeling Systems’ VVT Risks and Costs”
- BUEHLER, J., WEGENER, J. “Automatic Testing of an Autonomous
Parking System using Evolutionary Computation”. SAE World Congress,
Detroit, USA, March 8 – 11, 2004.
- WEGENER, J.: “Test Case design by means of the Classification-Tree
Method and the Classification-Tree Editor”. Proceedings of Test
and Analysis of Component-based Systems (TACoS 2004), Barcelona, Spain,
March 27 – April 04, 2004.
- LÉVÁRDY V., HOPPE M. “Verification, Validation
and Testing Strategy Planning Supported by a Process Model”. Conference
paper published in the proceedings of the Tools and Methods for Competitive
Engineering (TMCE) 2004 conference, Lausanne, Switzerland. April 12-16,
- ENGEL A., BOGOMOLNI I., SHACHAR S., GRINMAN A. “Using historical
cost and risk data to optimize the VVT process” (The 14th Annual
International Symposium of INCOSE, Toulouse, France. June 20-24, 2004).
- HOPPE M., LÉVÁRDY V., LEARDI C., MENDIKOA I., DE ABAJO
N., GONZALEZ J., LOBATO V., PEREGRINA S. ”Application Experiences
of the VVT Process Modeling Procedure at the Verification and Validation
Planning”. Conference published in the proceedings of the 14th
Annual International Symposium of INCOSE, Toulouse, France. June 20-24,
- HOPPE M., LÉVÁRDY V., HASKINS C., BOGOMOLNI I., IGENBERGS
E. “Generic Verification, Validation and Testing Methodology”.
Tutorial presented at the 14th Annual International Symposium of INCOSE
in Toulouse, France. June 20-24, 2004.
- LÉVÁRDY V., HOPPE M., HONOUR E. ”Verification,
Validation & Testing Strategy and Planning Procedure”. Conference
paper published in the proceedings of the 14th Annual International
Symposium of INCOSE, Toulouse, France. June 20-24, 2004.
- LAMMERMANN F., BARESEL A., WEGENER J. “Evaluating Evolutionary
Testability with Software Measurements”. Proceedings of the Genetic
and Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle,
Washington, USA, June 26-30, 2004.
- WEGENER J., BÜHLER O. “Evaluation of Different Fitness
Functions for the Evolutionary Testing of an Autonomous Parking System”.
Proceedings of the Genetic and Evolutionary Computation Conference,
Part 2 - GECCO 2004, Seattle, Washington, USA, June 26-30, 2004.
- BARESEL A., STHAMER H., WEGENER J. “Applying Evolutionary Testing
to Search for Critical Defects“. Proceedings of the Genetic and
Evolutionary Computation Conference, Part 2 - GECCO 2004, Seattle, Washington,
USA, June 26-30, 2004.