A common problem in econometric research may be characterized as follows: we have several methods to do something, some criteria for performance evaluation, and some circumstances/models under which would like to compare performance of the methods. Estimation and testing are obvious applications in which asymptotic theory may provide some guidance for performance evaluation, but one often wishes to explore finite sample performance via Monte-Carlo experimentation if only as a way to validate the relevance of the theory for empirical applications.
Unfortunately, such experiments are often carried out over a long period in which hardware and software may evolve and memory affords an imperfect index to the welter of data files that remain behind. In such circumstances it is common to insert a table or a figure into a paper-in-progress and later find that it is difficult or even impossible to recreate the process which produced the object in the first instance.
In view of the difficulties of reproducing our own work, it is hardly surprising that others have difficulties reproducing published work in econometrics. It has been an established part of the folklore in applied econometrics that even ordinary regression results are rarely reproducible and as statistical methods become more sophisticated this has not improved. In the following pages we attempt to assemble some tools and strategies designed to counter this tendency. The methods are illustrated by their application to some recent work on improving algorithms for quantile regression. Section 2 introduces some SPLUS ideas for improving the documentation and organization of simulation results. Section 3 considers some aspects of dealing with FORTRAN components of simulation experiments. Section 4 discusses some tools for organizing SPLUS graphics and introducing them into LaTeX documents. Section 5 deals with packaging research results and making them accessible over the internet.
As with all aspects of S and SPLUS, we rely heavily on Becker, Chambers, and Wilks (1988) Spector (1994) and Venables, and Ripley (1994) for guidance on the language and its use. Throughout, we will also often rely on code contributed to STATLIB and via SNEWS to the SPLUS community. These repositories constitute a rich source of enhancements for the efficient use of the language.