TOMLAB OPTIMIZATION ENVIRONMENT: clsSolve

clsSolve

Purpose

For solving nonlinear least squares problems with linear inequality and equality constraints and bound constraints.

Syntax

   Result = clsSolve(Prob, varargin);

Description

clsSolve implements algorithms for nonlinear least squares problems with linear inequality and equality constraints and bound constraints.

Problem

clsSolve solves the nonlinear least squares problem of the form:

      min       0.5 * sum {r(x).^2}      <==>   min       0.5 *  r(x)^T * r(x)
       x                                         x
                x_L <=   x  <= x_U                        x_L <=   x  <= x_U
                b_L <=  Ax  <= b_U                        b_L <=  Ax  <= b_U 
                c_L <= c(x) <= c_U.                       c_L <= c(x) <= c_U.

where r is the m-dimensional residual vector and x is a n-dimensional unknown parameter vector

(The nonlinear constraints c(x) are currently not treated)

Algorithms

• 1 - The Fletcher-Xu hybrid method; Gauss-Newton / BFGS
• 2 - The Al-Baali - Fletcher hybrid method; Gauss-Newton/BFGS
• 3 - Huschens method. SIAM J. Optimization. Vol 4, No 1, pp 108-129 jan 1994.
• 4 - The Gauss-Newton method
• 5 - Wang, Li, Qi Structured MBFGS method
• 6 - Wang, Li, Qi Structured BFGS method
• 0 - empty or other number, gives the Fletcher-Xu algorithm

Search method technique

• 0 - QR with pivoting (both sparse and dense)
• 1 - SVD (dense)
• 2 MATLABs inversion routine (Uses QR)
• 3 Explicit computation of pseudoinverse, pinv(J_k)

Bound constraints treated as described in Gill, Murray, Wright: Practical Optimization, Academic Press, 1981. Null space method for the equality constraints.

clsAssign.m may be used to define the Prob structure

Input Parameters

Use tomMenu.m for interactive run/ tomRun as driver routine

 Prob    Structure, where the following variables are used:

   Solver.Alg Described above
   LargeScale If = 1, then SQR2 = a sparse QR method, is used.
              The Q matrix is then stored efficiently
              Otherwise the standard Matlab sparse QR leads to a very big
              Q matrix, that might blow up the available memory
              The size of Q will be m x m, where m = # of residuals.
              The above only applies to Prob.Solver.Method = 0 (default).

   x_0       Starting point
   x_L       Lower bounds for x
   x_U       Upper bounds for x
   b_L       Lower bounds for linear constraints
   b_U       Upper bounds for linear constraints
   A         Linear constraint matrix
   PriLevOpt Print Level
   optParam Optimization parameters. Fields used in optParam are:
      bTol, eps_absf, eps_g, eps_Rank, eps_x, IterPrint, MaxIter, 
      PreSolve size_f, size_x, xTol, wait

      optParam.QN_InitMatrix:  Initial Quasi-Newton matrix, if not empty,
                               otherwise use identity matrix
   LineParam Line search parameters, special field used:

   LineAlg  = 1 Standard Fletcher line search (default)
   LineAlg  = 2 Armijo-Goldstein line search
   If LineAlg == 2, then the following parameters are used
   agFac        Armijo Goldsten reduction factor, default 0.1
   sigma        Line search accuracy tolerance, default 0.9

   USER.r:  The routine to compute the residual, given as a string
   USER.J:  The routine to compute the Jacobian, given as a string

Extra parameters

 VARARGIN: User defined parameters passed to f,g,H, r and J, and c and dc.

Output Parameters

 Result      Structure with results from optimization
    x_k      Optimal point
    v_k      Lagrange multipliers NOT USED
    f_k      Function value at optimum
    g_k      Gradient vector at optimum
    x_0      Starting value vector
    r_k      Residual at optimum
    J_k      Jacobian matrix at optimum
    xState   Variable: Free==0; On lower == 1; On upper == 2; Fixed == 3;
    bState   Linear constraint: Inactive==0; On lower bound == 1; 
             On upper bound == 2; Equality == 3;
    Iter     Number of iterations
    ExitFlag Flag giving exit status
       0     Convergence
       1     Error, see Inform
    ExitTest Text string giving ExitFlag and Inform information
    Inform   Code telling type of convergence
             1   Iteration points are close.
             2   Projected gradient small.
             3   Iteration points are close and projected gradient small.
             4   Function value close to 0.
             5   Iteration points are close and function value close to 0.
             6   Projected gradient small and function value close to 0.
             7   Iteration points are close, projected gradient small and
                 function value close to 0.
             8   Relative function value reduction low for 10 iterations.
            32   Local min with all variables on bounds.
           101   Max no of iterations reached.
           102   Function value below given estimate.
           104   x_k not feasible, constraint violated.
           105   The residual is empty. No NLLS problem

conSolve