# Numerical accuracy

Gretl Manual: Gnu Regression, Econometrics and Time-series Library
Prev |  | Next

# Appendix C. Numerical accuracy

`gretl` uses double-precision arithmetic throughout — except for the multiple-precision plugin invoked by the menu item "Model/High precision OLS" which represents floating-point values using a number of bits given by the environment variable `GRETL_MP_BITS` (default value 256). The normal equations of Least Squares are by default solved via Cholesky decomposition, which is accurate enough for most purposes (with the option of using QR decomposition instead). The program has been tested rather thoroughly on the statistical reference datasets provided by NIST (the U.S. National Institute of Standards and Technology) and a full account of the results may be found on the gretl website (follow the link "Numerical accuracy").

In October 2002 I had a useful exchange with Giovanni Baiocchi and Walter Distaso, who were writing a review of `gretl` for the Journal of Applied Econometrics, and James MacKinnon, software review editor for the journal.[1] and I am grateful to Baiocchi and Distaso for their careful examination of the program, which prompted the following modifications.

1. The reviewers pointed out that there was a bug in `gretl`'s "p-value finder", whereby the program printed the complement of the correct probability for negative values of z. This was fixed in version 0.998 of the program (released July 9, 2002).

2. They also noted that the p-value finder produced inaccurate results for extreme values of x (e.g. values of around 8 to 10 in the t distribution with 100 degrees of freedom). This too was fixed in `gretl` version 0.998, with a switch to more accurate probability distribution code.

3. The reviewers noted a flaw in the presentation of regression coefficients in `gretl`, whereby some coefficients could be printed to an unacceptably small number of significant figures. This was fixed in version 0.999 (released August 25, 2002): now all the statistics associated with a regression are printed to 6 significant figures.

4. It transpired from the reviewer's tests that the numerical accuracy of `gretl` on MS Windows was less than on Linux, where I had done my testing. For example, on the Longley data — a well-known "ill-conditioned" dataset often used for testing econometrics programs — the Windows version of gretl was getting some coefficients wrong at the 7th digit while the same coefficients were correct on Linux. This anomaly was fixed in `gretl` version 1.0pre3 (released October 10, 2002).

The current version of `gretl` includes a "plugin" that runs the NIST linear regression test suite. You can find this under the "Utilities" menu in the main window. When you run this test, the introductory text explains the expected result. If you run this test and see anything other than the expected result, please send a bug report to `cottrell@wfu.edu`.

As mentioned above, all regression statistics are printed to 6 significant figures in the current version of `gretl` (except when the multiple-precision plugin is used, then results are given to 12 figures). If you want to examine a particular value more closely, first save it (see genr) then print it using `print --ten` (see Chapter 13). This will show the value to 10 digits.

### Notes

[1] | This review has since been published; see Baiocchi and Distaso (2003).

Prev | Home | Next
Technical notes |  | Advanced econometric analysis with free software