Derivative free optimization

• Derivative free optimization is repeated evaluation of objective function
• The concept are based on natures wisdom, such as evolution and thermodynamics
• The analytic opacity knowledge about them are based on empirical studies.

• Stopping condition in derivative free optimization:

  Let k denote an iteration count and fk denote the best objective function obtained at count k. stopping condition depends on

  • Computation time
  • Optimization goal;
  • Minimal Improvement
  • Minimal relative improvement

Derivative based optimization

• Derivative based optimization deals with gradient-based optimization techniques, capable of determining search directions according to an objective function’s derivative information

• It is used in optimizing non-linear neuro-fuzzy models,

  – Steepest descent

  – Conjugate gradient