Wednesday, December 10, 2008

Vectors, Likelihoods and Partial Derivatives

The basic architecture for the open source application Cronos contains TimeSeries, TimeSeriesModel and ARMAModel classes to get started. These classes, i.e. 46 pages, require an understanding of linear algebra, minimization and partial differentiation from the gsl perspective. The current matrix and vector classes supplied would not compile so in an attempt to have a better understanding, I am rewriting the code in a reduced version using just gsl_vector and gsl_matrix for .NET. References can be found in this book

GNU Scientific Library Reference Manual - Revised Second Edition (v1.8)by M. Galassi, J. Davies, J. Theiler, B. Gough, G. Jungman, M. Booth, F. RossiPaperback (6"x9"), 636 pages, 60 figures. ISBN 0954161734RRP £24.99 ($39.99)

gsl_vector samples:
http://www.network-theory.co.uk/docs/gslref/Exampleprogramsforvectors.html

gsl_matrix samples:
http://www.network-theory.co.uk/docs/gslref/Exampleprogramsformatrices.html

I want to be able to compute the loglikelihood from Brockwell and Davis (1991) and be able to use different criterions such as AIC, BIC, etc. as shown in my time series books. Basically, it is a minimization on -(log likelihood) that uses the gsl_multimin_function, gsl_multimin_fminimizer

Here is an example :
http://aldebaran.devinci.fr/~cagnol/promotion2007/cs302/gsl/multimin/gsl_multimin.h.html

Futhermore, I have to be able to calculate the partial derivatives of the log-likelihood to obtain
the information matrix for the parameters. For this I can use gsl_deriv_central

An example:
http://www.gnu.org/software/gsl/manual/html_node/Numerical-Differentiation-functions.html

A good example on non-linear least squares can be found at:

http://www.physics.brocku.ca/~tharroun/parratt/group__lstsq.html

and multidimensional minimization

http://www.physics.brocku.ca/~tharroun/gsl_fit/group__mdmin.html

Currently, I am working on some examples for each of these areas to post and making modifications to create a forecastable ARMA model class that I can extend to GARCH and SVM. This will enable me to use additional research algorithms in these components.

References

Time Series: Theory and Methods, second edition (1991) P.J. Brockwell and R.A. Davis, Springer-Verlag, New York.

B. Stroustrup, The C++ Programming Language (3rd Ed), Section 22.4 Vector Arithmetic.
Addison-Wesley 1997, ISBN 0-201-88954-4.

No comments: