Beam Search Algorithm (Draft by Andrew Jungwirth)

**还未翻译完，先保存

Objectives目标

• To show how the Beam Search Algorithm uses a heuristic function and a given beam width in an attempt to simulate the Breadth-First Search in a memory-efficient way.
• 阐述BeamSearch算法如何使用启发式函数和给定的搜索宽度去近似高效率的宽度优先搜索算法变种。

• To emphasize the importance of limiting a graph-search's memory consumption when finding solutions to large problem spaces.
• 在大尺度的解空间搜索最优解过程中，限制图搜索算法的内存消耗的重要性

• To demonstrate the strengths and weaknesses of the Beam Search Algorithm.
• 解释BeamSearch算法的优势与劣势

Preparation准备

In order to understand this algorithm, you must be familiar with the concept of a graph as a group of nodes/vertices and edges connecting these nodes.

为了理解算法，你必须熟悉图的概念，了解图的结构：节点和链接节点的边。

It is also helpful to understand the how a search tree can be used to show the progress of a graph search. 

理解搜索树如何对图进行一步一步的搜索过程也很有帮助。

Additionally,knowledge of the Breadth-FirstSearch Algorithm is required because Beam Search is a modification of this algorithm.

另外，了解宽度优先搜索算法对于理解Beam Search算法很必要。

Beam Search Algorithm

Even though the Breadth-First Search Algorithm is guaranteed tofind the shortest path from a start node to a goal node in anunweighted graph, it is infeasible to use this algorithm on largesearch spaces because its memory consumption is exponential. Thiscauses the algorithm run out of main memory before a solution can befound to most large, nontrivial problems. For this reason, Beam Searchwas developed in an attempt to achieve the optimal solution found bythe Breadth-First Search Algorithm without consuming too muchmemory.

In order to accomplish this goal, Beam Search utilizes a heuristicfunction, h, to estimate the cost to reach the goal from a givennode. It also uses a beam width, B, which specifies the numberof nodes that are stored at each level of the Breadth-FirstSearch. Thus, while the Breadth-First Search stores all the frontier nodes (the nodes connected to the closing vertices) in memory, the Beam Search Algorithm only stores the B nodes with the best heuristic values at each level of the search. The idea is that the heuristic function will allow the algorithm to select nodes that will lead it to the goal node, and the beam width will cause the algorithm to store only these important nodes in memory and avoid running out of memory before finding the goal state.

Instead of the open list used by the Breadth-First Search Algorithm, the Beam Search Algorithm uses the BEAM to store thenodes that are to be expanded in the next loop of the algorithm. Ahash table is used to store nodes that have been visited, similar to the closed list used in the Breadth-First Search. Beam Searchinitially adds the starting node to the BEAM and the hashtable. Then, each time through the main loop of the algorithm, BeamSearch adds all of the nodes connected to the nodes in the BEAM to its SET of successor nodes and then adds the B nodes withthe best heuristic values from the SET to the BEAM andthe hash table. Note that a node that is already in the hash table is not added to the BEAM because a shorter path to that node has already been found. This process continues until thegoal node is found, the hash table becomes full (indicatingthat the memory available has been exhausted), or the BEAM isempty after the main loop has completed (indicating a dead end in thesearch).

The Beam Search Algorithm is shown by the pseudocode below. This pseudocode assumes that the Beam Search is used on an unweighted graphso the variable g is used to keep track of the depth of thesearch, which is the cost of reaching a node at that level.

Pseudocode for the Beam Search Algorithm

/* initialization */
g = 0;
hash_table = { start };
BEAM = { start };

/* main loop */
while(BEAM ≠ ∅){                             // loop until the BEAM contains no nodes
  SET = ∅;                                   // the empty set

  /* generate the SET nodes */
  for(each state in BEAM){
    for(each successor of state){
      if(successor == goal) return g + 1;
      SET = SET ∪ { successor };             // add successor to SET
    }
  }

  BEAM = ∅;                                  // the empty set
  g = g + 1;

  /* fill the BEAM for the next loop */
  while((SET ≠ ∅) AND (B > |BEAM|)){         // set is not empty and the number of nodes in BEAM is less than B
    state = successor in SET with smallest h value;
    SET = SET \ { state };                   // remove state from SET
    if(state ∉ hash_table){                  // state is not in the hash_table
      if(hash_table is full) return ∞;
      hash_table = hash_table ∪ { state };   // add state to hash_table
      BEAM = BEAM ∪ { state };               // add state to BEAM
    }
  }
}

// goal was not found, and BEAM is empty - Beam Search failed to find the goal
return ∞;

Example Trace of the Beam Search Algorithm

The following traces of the Beam Search Algorithm use two rows torepresent each main loop of the algorithm's execution. The first row of each numbered loop displays the nodes added to the SET. These nodes are ordered by their heuristic values, withalphabetical ordering used to sort nodes with identical hvalues. Since the SET is a mathematical set, if a node isinserted into the SET more than once from multiple parents, itonly appears in the SET once. The second row of each numbered loop lists the nodes from the SET that are added to the BEAM in the second part of the main loop. Both rows alsodisplay the hash table to show its current state. Notice thatthe hash table has only seven slots, indicating that the memorysize for this example trace is seven. A simple linear hashing scheme with key values determined by the node names' ASCII values mod 7 is used for simplicity. In all three of these lists, nodes are listed in the format node_name(predecessor_name). The algorithm is traced four times with different values of B to demonstrate the strengths and weaknesses of the algorithm. Each trace includes a search tree that shows the BEAM at each level of the search. In the graph, the numbers under the node names are the h values for the nodes. These traces show how Beam Search attempts to find the shortest path from node I to node B in the graph shown in Figure 1. (Figure 1 is included above each trace for convenience.)

Figure 1

Trace 1, B = 1

loop number	SET (first row per numbered loop) BEAM (second row per numbered loop)	hash_table
	BEAM = { I(null) }	hash_table = { _, I(null), _, _, _, _, _ }
1	SET = { G(I), J(I), E(I), H(I) }	hash_table = { _, I(null), _, _, _, _, _ }
1	BEAM = { G(I) }	hash_table = { _, I(null), _, _, _, _, G(I) }
2	SET = { D(G), J(G), I(G) }	hash_table = { _, I(null), _, _, _, _, G(I) }
2	BEAM = { D(G) }	hash_table = { _, I(null), _, D(G), _, _, G(I) }
3	SET = { G(D) }	hash_table = { _, I(null), _, D(G), _, _, G(I) }
3	BEAM = { }	hash_table = { _, I(null), _, D(G), _, _, G(I) }

Figure 2

At this point, the BEAM is empty, and the Beam SearchAlgorithm has reached a dead-end in its search. Since the nodeG in the SET was already in the hash table, itcould not be added to the BEAM, which left the BEAMempty. This trace illustrates the greatest weakness of the Beam SearchAlgorithm: An inaccurate heuristic function can lead the algorithminto a situation in which it cannot find a goal, even if a path to thegoal exists. While increasing the value of B may allow BeamSearch to find the goal, increasing B by too much may cause thealgorithm to run out of memory before it finds the goal. For thisreason, the choice of B has a large impact on Beam Search'sperformance. Figure 2 shows the BEAM nodes at each level inthis dead-end search.

Figure 1

Trace 2, B = 2

loop number	SET (first row per numbered loop) BEAM (second row per numbered loop)	hash_table
	BEAM = { I(null) }	hash_table = { _, I(null), _, _, _, _, _ }
1	SET = { G(I), J(I), E(I), H(I) }	hash_table = { _, I(null), _, _, _, _, _ }
1	BEAM = { G(I), J(I) }	hash_table = { _, I(null), J(I), _, _, _, G(I) }
2	SET = { A(J), D(G), G(J), J(G), E(J), I(G) }	hash_table = { _, I(null), J(I), _, _, _, G(I) }
2	BEAM = { A(J), D(G) }	hash_table = { A(J), I(null), J(I), D(G), _, _, G(I) }
3	SET = { C(A), G(D), J(A) }	hash_table = { A(J), I(null), J(I), D(G), _, _, G(I) }
3	BEAM = { C(A) }	hash_table = { A(J), I(null), J(I), D(G), C(A), _, G(I) }
4	SET = { B(C) [goal found - algorithm returns], A(C) }	hash_table = { A(J), I(null), J(I), D(G), C(A), _, G(I) }

Figure 3

In this trace, the Beam Search Algorithm successfully found thegoal via the path IJACB. Even though a solution was found, thissolution is not optimal because IECB is a shorter path to thegoal node. Once again, an inaccurate heuristic function reduced theeffectiveness of the Beam Search Algorithm. Figure 3 shows theBEAM nodes at each level of the search. Notice that only onenode appears in the BEAM at level three in the tree. Thisdemonstrates that Beam Search may not always be able to fill theBEAM at each level in the search. In the last level of thetree, node A was first added to the SET, and then nodeB (the goal node) was found and caused the search tocomplete.

Figure 1

Trace 3, B = 3

loop number	SET (first row per numbered loop) BEAM (second row per numbered loop)	hash_table
	BEAM = { I(null) }	hash_table = { _, I(null), _, _, _, _, _ }
1	SET = { G(I), J(I), E(I), H(I) }	hash_table = { _, I(null), _, _, _, _, _ }
1	BEAM = { G(I), J(I), E(I) }	hash_table = { _, I(null), J(I), _, E(I), _, G(I) }
2	SET = { A(J), C(E), D(G), F(E), G(J), J(E), E(J), H(E), I(E) }	hash_table = { _, I(null), J(I), _, E(I), _, G(I) }
2	BEAM = { A(J), C(E), D(G) }	hash_table = { A(J), I(null), J(I), C(E), E(I), D(G), G(I) }
3	SET = { B(C) [goal found - algorithm returns], A(C), C(A), J(A) }	hash_table = { A(J), I(null), J(I), C(E), E(I), D(G), G(I) }

Figure 4

With B = 3, the Beam Search Algorithm found the optimal pathto the goal. However, the larger beam width caused the algorithm tofill the entire memory available for the hash table. Figure 4 showsthe BEAM nodes at each level in the search. In the last levelof the tree, nodes A, C, and J were added to theSET, and then the goal node B was found, which caused tosearch to complete.

Figure 1

Trace 4, B = 4

loop number	SET (first row per numbered loop) BEAM (second row per numbered loop)	hash_table
	BEAM = { I(null) }	hash_table = { _, I(null), _, _, _, _, _ }
1	SET = { G(I), J(I), E(I), H(I) }	hash_table = { _, I(null), _, _, _, _, _ }
1	BEAM = { G(I), J(I), E(I), H(I) }	hash_table = { H(I), I(null), J(I), _, E(I), _, G(I) }
2	SET = { A(J), C(E), D(G), F(E), G(J), J(E), E(H), H(E), I(E) }	hash_table = { H(I), I(null), J(I), _, E(I), _, G(I) }
2	BEAM = { A(J), C(E), D(G) [not enough memory - algorithm returns] }	hash_table = { H(I), I(null), J(I), A(J), E(I), C(E), G(I) }

Figure 5

Using B = 4, the Beam Search Algorithm quickly ran out ofmemory. This shows the second major weakness of the Beam SearchAlgorithm: When B becomes large, the algorithm consumes memoryvery quickly like the Breadth-First Search Algorithm. Figure 5 showsthe BEAM at each level in the search. The last level in thetree shows the progress of the search when the algorithm ran out ofmemory.

Efficiency/Algorithm Analysis

It is generally effective to analyze graph-search algorithms byconsidering four traits:

• Completeness: A search algorithm is complete if it will find a solution (goal node) when a solution exists.

• Optimality: A search algorithm is optimal if it finds the optimal solution. In the case of the Beam Search Algorithm, this means that the algorithm must find the shortest path from the start node to the goal node.

• Time complexity: This is an order-of-magnitude estimate of the speed of the algorithm. The time complexity is determined by analyzing the number of nodes that are generated during the algorithm's execution.

• Space complexity: This is an order-of-magnitude estimate of the memory consumption of the algorithm. The space complexity is determined by the maximum number of nodes that must be stored at any one time during the algorithm's execution.

Completeness

In general, the Beam Search Algorithm is not complete. This isillustrated in Trace 1 above. Even though the memory was not depleted,the algorithm failed to find the goal because it could not add anynodes to the BEAM. Thus, even given unlimited time and memory,it is possible for the Beam Search Algorithm to miss the goal nodewhen there is a path from the start node to the goal node. A moreaccurate heuristic function and a larger beam width can improve BeamSearch's chances of finding the goal. However, this lack ofcompleteness is one of the foremost weaknesses of the Beam SearchAlgorithm.

Optimality

Just as the Beam Search Algorithm is not complete, it is also notguaranteed to be optimal. This is shown by Trace 2 above. In thisexample, Beam Search found the goal node but failed to find the optimalpath to the goal, even though the heuristic in Figure 1 is admissible(underestimates the cost to the goal from every node) and consistent(underestimates the cost between neighboring nodes). This happenedbecause the beam width and an inaccurate heuristic function caused thealgorithm to miss expanding the shortest path. A more preciseheuristic function and a larger beam width can make Beam Search morelikely to find the optimal path to the goal.

Time Complexity

The time for the Beam Search Algorithm to complete tends to dependon the accuracy of the heuristic function. An inaccurate heuristicfunction usually forces the algorithm to expand more nodes to find thegoal and may even cause it to fail to find the goal. In the worstcase, the heuristic function leads Beam Search all the way to thedeepest level in the search tree. Thus, the worst case time isO(Bm), where B is the beam width, and m is themaximum depth of any path in the search tree. This time complexity islinear because the Beam Search Algorithm only expands B nodesat each level; it does not branch out more widely at each level likemany search algorithms that have exponential time complexities. Thespeed with which this algorithm executes is one of its greateststrengths.

Space Complexity

Beam Search's memory consumption is its most desirabletrait. Because the algorithm only stores B nodes at each levelin the search tree, the worst-case space complexity isO(Bm), where B is the beam width, and m is themaximum depth of any path in the search tree. This linear memoryconsumption allows Beam Search to probe very deeply into large searchspaces and potentially find solutions that other algorithms cannotreach.

Compare with Your Textbook

Algorithms can look differently but still operate in almost thesame ways. Compare the pseudocode above with the description in yourtextbook (if available). Then consider these questions:

1. Does your textbook use a hash table to store the nodes that have been expanded? If not, how does it store these nodes?

2. Does your textbook explain what type of structure should be used to implement the SET? If so, what structure does it use?