Iterative Improvement Algorithms

• Rather than searching for a solution path and then executing the steps associated with the solution path, iteratively pick a next best move, make that move, and then repeat. Hence, irrevocably make a decision about one step at each iteration.
• Best used in problems where all the information for a solution is contained in the node itself. For example, cryptarithmetic problems.

• Hill-Climbing Search
  
  • Look at all immediate successors of current state n
  • If there exists a successor s such that h(s) < h(n), and h(s) <= h(t) for all the successors t of n, then move from n to s. Otherwise, halt at n.
  • Looks one step ahead to determine if any successor is better than the current state; if there is, move to the best successor.
  • Similar to Greedy search in that it uses h, but does not allow backtracking or jumping to an alternative path since there is no nodes list of other candidate frontier nodes from which the search could be continued. Corresponds to Beam search with a beam width of 1 (i.e., the maximum size of the nodes list is 1).
  • Not complete since the search will terminate at "local minima," "plateaus," and "ridges."
  • Algorithm visualized in terms of a surface in 3D:

    Consider the state space to be the set of points in the x,y plane, and for each such point the height h is the value of the heuristic function for that state. This height function, h = h(x,y), defines a surface. The initial state corresponds to a point on this surface and the goal is to find a state where the height is 0 (or is a global minimum).

    Hill-climbing (should be called Valley-Finding in this context where we are minimizing instead of maximizing a value) moves in the direction of steepest descent since it moves to the successor (i.e., adjacent) node that decreases h the most.

    Notice that by considering the state space as a continuous space of points in the x,y plane, if the height surface is continuous (i.e., smooth so that derivatives are well-defined everywhere), then the direction of steepest descent corresponds to the gradient direction = [dh(x,y)/dx, dh(x,y)/dy], and the search is called gradient descent.