Genetic Algorithms:Loss Functions

Steinwart's paper on "How to compare different loss functions and Risks" at http://www.c3.lanl.gov/ml/pubs/2005_loss/paper.pdf , sets the stage for modifying the Genetic Algorithm component of Kirillov mentioned in a previous post. He uses for the Evaluate function the Absolute value loss function, given by

error += Math.Abs( y - data[i + windowSize] )

The entire method is presented below:

public double Evaluate( IChromosome chromosome )
{
// get function in polish notation
string function = chromosome.ToString( );

// go through all the data
double error = 0.0;
for ( int i = 0, n = data.Length - windowSize - predictionSize; i < n; i++ )
{
// put values from current window as variables
for ( int j = 0, b = i + windowSize - 1; j < windowSize; j++ )
{
variables[j] = data[b - j];
}

// avoid evaluation errors
try
{
// evaluate the function
double y = PolishExpression.Evaluate( function, variables );
// check for correct numeric value
if ( double.IsNaN( y ) )
return 0;
// get the difference between evaluated value and
// next value after the window, and sum error

error += Math.Abs( y - data[i + windowSize] );

}
catch
{
return 0;
}
}

// return optimization function value
return 100.0 / ( error + 1 );
}


In a presentation by Wang and Zhang on "Risk and Loss Functions" at http://www.ee.columbia.edu/~dpwe/mlsp/yong-chap3.pdf , the authors review squared error Loss, Huber's Robust Loss, Absolute Loss, and e-sensitive Loss. They provide some applied examples that work well with Steinwart's work. Clearly, here is an example where one can add value by creating a class of loss functions that are tied to the empirical probability density function (pdf) of the observed data. Of course, this class can be reused in other estimation problems as well.