Test methods for software
There is one thing that should be clearly stated at the beginning with regard to software testing, even though it may sound obvious. This is that testing a program does not mean conducting a wholesale search for errors in the program, but instead performing tests according to a test specification, with the objective of discovering whether there are errors in the program being tested. Testing thus has nothing to do with debugging, since tests are performed by testers and debugging is performed by software developers. As we have repeatedly emphasized elsewhere, programs for smart card microcontrollers are not very large compared with other types of software. Nevertheless, they have their own special features. This can be illustrated by considering some of the general specifications of a typical operating system. In a microcontroller having 16 kB of ROM and 8 kB of EEPROM, the software requires around 20 kB of memory. This leaves 4 kB of EEPROM available for any applications. If this software is programmed in assembler, it will amount to around 30,000 lines of source code, which would fill 500 sheets of paper if printed at 60 lines to the page. The number of conditional branches will be around 2000, and even an experienced programmer needs around nine months to generate 20 kB of assembler code. Presently, very high-performance smart card operating systems occupy up to 12,000 lines of source code in C, which corresponds to approximately 2000 sheets of paper. This numerical example clearly demonstrates that smart card operating system software is rather complex. On top of this, the software must be used almost exclusively in areas where security is a consideration. This means that the demand for a low level of errors cannot be met by simply employing a few homemade tests during or following software development. Instead, a suitable testing strategy is necessary.

With smart cards, as in other areas, the trend is shifting from assembly language programming to high-level programming languages, such as the C language, which is relatively close to the hardware level, and Java, which is an object-oriented language. Using a high-level programming language is necessary for large smart card operating systems with more than 30 kB of code, not only due to the resulting decrease in implementation time, but also because of the need to minimize the number of errors. This can be illustrated as follows. It can be assumed that the number of errors per line of source code is nearly the same for almost all programming languages. Since the functional level of a high-level language is significantly higher than that of assembler, this means that the error density is lower for the same amount of executable program code. For example, if we assume the entirely realistic value of 1.5 errors for every 100 lines of program code, and in addition we assume that only half of all errors can be found with an acceptable amount of effort, then a tested program will still have 0.75 errors for every 100 lines of program code. With Java, the relationship between lines of source code and machine code(bytecode) is around 1:6. This means that the functionality of one line of Java code roughly corresponds to that of six lines of machine code. If we assume that all other conditions are the same and that the compilation process is largely error-free, the number of errors in a program can be reduced by a factor of six by using a more powerful language. Even if the actual value is lower than this in practice, this is still an exceptionally strong justification for using powerful programming languages.

Fundamentals of smart card software testing
It is necessary to consider the life cycle of smart card software before even starting to define a test strategy. The waterfall model proposed by W. W. Royce, which has been known since 1970 and has been published in many forms, can be used for this purpose. It is relatively well suited to mask-programmed smart card operating systems. However, since it is also designed to be used with very extensive software projects for PCs and mainframe computers, here we use a simplified version specially adapted to smart cards. The five steps described here are normally performed in sequence. However, it is certainly possible for a problem encountered in a certain step to make it necessary to go back and repeat one or more steps. This should be avoided as much as possible, since each iteration costs time and money. In order to meet economic demands, such as time-to-market and short software development time, it is often necessary to overlap the steps to a certain extent, instead of performing them in strict sequence. With this method, which is known as simultaneous engineering, sections of the software are split into individual modules as early as possible. These modules are then launched down the waterfall concurrently. It thus can happen that smart card software containing only a data transmission protocol may already be at the system integration level while the cryptographic algorithm for the same application is still being specified.

The analysis stage includes establishing the basic definition of the objective and compiling the requirements in the form of a formal requirements specification document that defines all the requirements that must be satisfied by the smart card software to be developed. The analysis stage also allows for the generation of proposed solutions in the form of preliminary designs. Put simply, this stage defines what the finished software has to do.

Analysis is followed by the design stage, which establishes how the software will do its job. For this purpose, it is necessary to generate precise specifications that are not subject to interpretation and that completely define one of the various possible solutions to the requirements produced in the analysis stage. Formally constructed specifications are best, since they allow the features, functions and processes of the software to be clearly and unequivocally defined. Specifications written in pseudocode, which can be tested for consistency and freedom from errors using computer programs, are well suited to this purpose. In a computer-aided software engineering (CASE) environment, such specifications can be used directly to generate the source code and test programs. They are sometimes referred to as ‘executable specifications’, which means specifications produced in a form that can be interpreted and further processed by computers.

Implementation and test
Once the specifications have been finalized and accepted, the program flowcharts for assembler or C programming may be generated. This is followed by programming and associated testing. The outcome of this stage is a fully programmed and tested smart card operating system.

System integration
Since smart cards can only function as part of a larger system, the various system components must be integrated in this stage. The results of system integration are the complete and error-free interaction of all parts of the system and the final documentation for the entire system.

This final stage of software development can only be used to modify any general parameters located in issued cards. Large-scale software upgrades or modifications are no longer possible at this stage. In the future, the ability to easily and quickly program smart cards using a high-level language such as Java will make it possible to use evolutionary life-cycle models5 in addition to the traditional waterfall model. With an evolutionary model, the analysis, design, and implementation and test stages, and in part even the system integration stage, are iterated several times with increasingly improved results. The objective is to quickly arrive at an optimum solution by expending minimal effort on generating specifications and working with fully functional prototypes. It is a well-known fact of long standing that in all kinds of projects, and particularly in the case of software development, the cost of correcting an error increases as the project progresses. This fact should lead to the expenditure of an appropriate amount of time and effort in the initial stages of the project, as represented by the waterfall model. If the design is incomplete or the specification is faulty, the cost of remedying the problem rises exponentially in the subsequent stages of the project.