Why sat solvers have become so efficient in the last 10 15 years?
1. Davis–Putnam–Logemann–Loveland: DPLL uses Depth First Search (DFS) to find a satisfying
assignment.
The architecture of a DPLL solver is:
1. Unit Propagation
2. Conflict Detection
3. Decide
4. Backtracking
Branching is essential to move forward in the search space, and backtracking is used to return from futile
portions of the space. Unit propagation speeds up the search by deducing appropriate consequences, i.e.
implications,of branching choices. Usually solver keeps a flag for every variable whether both values have
been tried or not. When a conflict arises, the decision variable with the highest decision level that has not
been tried with both values is flipped and all the assignments below that decision level including current
level undone. This method is called chronological backtracking. One of the drawbacks is it doesn’t learn
from the search space it visited. It uses heuristics but no learning.
The performance of backtrack style search methods can vary dramatically depending on the way one
selects the next variable to branch on (the “variable selection heuristics”) and in what order the possible
values are assigned to the variable (the “value selection heuristics”).
Given an initial formula, many search systems attempt to augment it with additional implicates to increase
the deductive power during the search process. This is usually referred to as learning and can be
performed either as a preprocessing step (static learning) or during the search (dynamic learning).
Conflict Driven Clause Learning:
Most prominent dynamic learning method is clause learning method which is implemented in all the modern
SAT solvers. The idea is to cache the “cause of conflict” in a succinct manner (as learned clauses) and
utilize this information to prune the search in different part of the search space encountered later. This
type learning is name “Conflict Directed Clause Learning” (CDCL). This conflict clauses are also used so
that in future the same conflict never arise again. This conflict also dictates on which branch the search
will backtrack to. This is called “Conflicted Driven Backtracking” in other words, non chronological
backtracking or backjumping which is first introduced in Constraint Satisfaction Problem (CSP) domains.
And it is first implemented in GRASP SAT solver.
After detection of each conflict a conflict analysis is performed which tries to find the reason i.e. the
assignment to some variables which leads to the conflict. Each of the variable assignments in conflicting
clause must be a decision variable or implied variable due to propagation of a constraint. The propagation
constraints are in turn asked for the set of variable assignments that forced the propagation to occur,
continuing the analysis backwards. The procedure is repeated until some condition is met, resulting in a set
of assignments that implies the conflict. This learnt clause must always, by construction be implied by
original problem constraints. And instead of backtracking to the lowest untried assignment of a variable, it
backtracks to the closest assignment which caused the conflict.
Usually steps of CDCL solvers are:
2. 1. Boolean Constraint Propagation
2. Conflict Detection
3. Decide: Choose Variable, Assign Variable
4. Conflict Analysis and Clause Learning
5. Conflict Driven Backjumping
The search procedure starts by selecting an unassigned variable x (called a decision variable). Assume a
value for it and the consequence of the assignment will then be propagated, possibly resulting in more
variable assignment (called implied variable). The unit clause is used for implying. All variables assigned as
a consequence of x is said to be from same decision level as x, counting from 1 for the first decision made
and so forth. Assignments made before the very first assignment is called top level assignment, decision
level 0.
All the assignments done by search are saved in a stack in the order they are made with their decision
level which is referred to as trail. The trail is usually divided into decision levels and used to undo
assignments during backtracking.
In recent years, modern SAT solvers based on CDCL improved dramatically. The key techniques behind
these improvements are:
● Learning new clauses from conflicts during backtrack search.
● Exploiting structure of conflicts during clause learning.
● Using lazy data structures for the representation of formulas.
● Branching heuristics must have low computational overhead, and must receive feedback from
backtrack search.
● Periodically restarting backtrack search.
● Additional techniques include deletion policies for learnt clauses, the actual implementation of lazy
data structures, the organization of unit propagation, among others.
Unsatisfied Formulas:
Moreover, for unsatisfiable subformulas, the clauses learnt by a CDCL SAT solver encode a resolution
refutation of the original formula. Given the way clauses are learnt in SAT solvers, each learnt clause can
be explained by a number of resolution steps, each of which is a trivial resolution step. As a result, the
resolution refutation can be obtained from the learnt clauses in linear time and space on the number of
learnt clauses. For unsatisfiable formulas, the resolution refutations obtained from the clauses learnt by a
SAT solver serve as a certificate for validating the correctness of the SAT solver. Moreover, resolution
refutations based on clause learning find key practical applications, including hardware model checking.
Besides allowing producing a resolution refutation, learnt clauses also allow identifying a subset of clauses
that is also unsatisfiable. For example, a minimally unsatisfiable subformula can be derived by iteratively
removing a single clause and checking unsatisfiability. Unnecessary clauses are discarded, and eventually
a minimally unsatisfiable subformula is obtained.
Preprocess:
Usually the assignments at decision level 0 are done by Pure Literal Rule. If a variable exists only in one
literal form, then that variable can be easily eliminated. For example, if a variable X has only literal X, then
3. we could simply assign it to True value without further analysis. But if we detect a conflict at decision
level 0, then the formula is UNSAT. Most SAT solvers use Variable Elimination. Hidden/asymmetric
tautology elimination discover redundant clauses through probing. Equivalent literal substitution find
strongly connected components in binary implication graph, replace equivalent literals by representatives.
Naive Data Structure:
Each clause keeps two counters: one for number of literals which evaluate to True and another for False
in the clause. Each variable has two lists which contain all the clauses where that variable appears as a
positive and negative literal respectively. When a variable assigned a value, all the clauses containing that
variable in both literal forms must update their counters. If the counter of False values becomes equal to
the number of literals in that clause, then that clause is declared unsatisfied. Hence a conflict occurs. If
True counter is 0 and False counter is one less than the number of literals in that clause, then that clause
becomes unit clause. It’s not efficient. If the instance has m clauses and n variables, and the average each
clause has l literals, then when a variable gets assigned, on the average l*m/n counters must be updated.
On backtracking l*m/n undo on counters on average. The situation becomes worse when learning
mechanism adds more clauses to clause database.
Occur Lists:
BCP has to detect only the clauses that are unit or conflicting clauses. There is no need to track all the
clauses are true. Instead of traversing whole clauses again and again, for each literal store the clauses
where it appears. Every time a literal is assigned only the clauses containing inverse literal needs to be
visited.
Watched Literals:
A Nliteral clause can be unit clause and conflicting clause when the (N1)th literal is assigned. Only then
that clause becomes important, otherwise we can simply ignore the satisfiability of the clause. We can
ignore first (N2) liteal assignments. We will pick two literals from each clause to watched literal and can
ignore any assignment to other literals because they will fall into first (N2) assignments. In fact one
watched literal is enough for correctness. If a clause can become newly implied via any sequence of
assignments, then this sequence will include an assignment of one of the watched literals. If a literal l is set
to False, then every clause C that has l as a watched literal, we try to find another literal which is
unassigned or True. For every previously active clause that has now become satisfied because of the
assignment l to False, we add ~l to its watched literal. Moreover, relative cost of extra bookkeeping
involved in maintaining watched literals pays off when one unassigns a literal in fact undoing a variable
assignment during backtracking takes constant time. If a clause is unsatisfied, then two watched variables
will be the last two variables to be assigned False. So any backtracking will make sure that the literals
being watched are either unassigned or set to True. Another great advantages of this scheme is very long
clauses doesn’t affect the propagation because they aren’t even looked at unless one of the literal is
assigned. So the number of literals in a clause doesn’t have any adverse effect. It also reduces number of
visiting clauses during BCP, useful for many learned clauses. This “watched literals” scheme is first
implemented in zChaff SAT solver.
Head and Tail Data Structure:
4. One of the most important improvements is using lazy data structure for SAT solvers. The first lazy data
structure for SAT solvers is Head and Tail (H/T) data structure. This data structure involves two
references with each clause, the Head (H) and the Tail(T) literal references. Initially the head reference
points to the first literal of the clause and tail reference points to the last literal of the clause. Each time
head or tail literal reference is assigned, a new unassigned literal is searched for forward and backward
direction respectively. So essentially, both pointers move to the center of the clause as H and T literals
become assigned. If an unassigned variable is found, it became the new head or tail, a new reference is
created and associated with the variable. In case no unassigned literal is found and the search reached
opposite end that means the clause becomes unit clause and appropriate boolean value is assigned to make
the clause satisfied. If a satisfied literal is discovered, the clause is declared satisfied. Previous head or tail
is useful when the search backtracks, hence are kept. When the search process backtracks the reference
with head or tail can be discarded and previous head and tail activated. Undoing a variables assignment
during backtracking has the same computational complexity as assigning the variable. In worst case, it
requires unassignments equal the total number of literals in the clause.
Implication Graph:
Usually after conflict analysis only one clause is learnt per conflict, but some strategies allow multiple
clauses to be learnt per conflict. The conflict analysis for learning and backtracking is done via implication
graph, a Directed Acyclic Graph (DAG) which captures the current state of the SAT solver. Each node
represents assignments of variables, each directed edge represents clause which cause implication due to
source nodes to sink nodes. Nodes which have no incoming edges represent decision nodes. A conflict
arises when implication graph has two nodes which assigns opposite values to the same variable. The
conflict analysis process starts with the conflict clause itself. is performed by following back the edges
from the conflicting nodes, up to any edge cutset which separates the conflicting nodes from the decision
nodes. When conflict analyzing the implication graph, we will only consider the connected component
which contains the conflict variable known as Conflict Graph. An implication graph may not contain
conflict but a conflict graph always contains conflict.
First Unique Implication Point:
Different strategies are used for termination condition for conflict analysis process. Among First Unique
Implication Point (firstUIP) is used in most leading SAT solvers and outperforms other strategies. It is
optimal in the backjump level. A node is a UIP (Unique Implication Point) of the current decision level if
all paths from the decision variable of current d level to the conflicting variable needs to go through this
node. In simple words, it is articulation point in the implication graph makes the original component into
biconnected components, a node which if removed would disconnect two parts of a graph. UIPs can be
found by graph traversal in the order of the made assignments. Count open paths initialize with the size
of clause with only false literals. Decrease counter if new reason or antecedent clause resolved. If all
paths converge at one node i.e. counter = 1, then UIP is found. The implication graph is bipartitioned; one
partition is called reason side which contains all the decision variables and conflicting side contains
conflicting variables. A simple cut always exists which separates the decision variables (which caused
conflict) from others. Multiple UIPs can be found in conflict graph which are proposed in GRASP. But
first UIP is the one which is closest to the conflict variable. First UIP also produce shorter clauses than
other UIPs. Using firstUIP (first proposed in Chaff) the analysis process terminated when the conflict
5. clause contain exactly one literal from the current decision level.
Local Conflict Clause Recording:
More recently, it has been shown that modern CDCL solvers implicitly build and prune a decision tree
whose nodes are associated with flipped variables. This motivates a practical useful enhancement named
local conflictclause recording. If a conflict clause has only one literal at highest decision level, after
backtracking, the clause will become unit and force the solver to explore a new search space. Such a
conflict clause is called asserting clause. A local conflict clause is a nonasserting conflict clause,
recorded in addition to the first UIP conflict clause if the last decision level contains some flipped variables
that have implied the conflict. The last variable in these circumstances is considered to be a decision
variable, defining a new decision level. A local conflict clause is the first UIP clause with respect to this
new decision level.
More conflict clause generation ideas are:
1. Include all the roots contributing to the conflict.
2. Find minimal cut in conflict graph
3. Cut off at first literal of lower decision level.
Conflict Clause Minimization:
Conflict clause minimization which is introduced by MiniSAT solver, tries to reduce the size of a learned
conflict clause C by repeatedly by identifying and removing any literals of C that are implied to be False
when the rest of the literals in C are set to False. There are two types of conflict minimizations: local and
recursive. Local minimization is achieved using selfsubsumption resolution rule is applied in reverse
assignment order, using antecedent marked in the implication graph. This rule can be generalized, at the
expense of extra computational cost which usually pays off, to a sequence of subsumption resolution
derivation such that the final clause subsumes the first antecedent clause. In recursive, the conflict clause
is minimized by deleting the literals whose antecedents are dominated by other literals of other clause in
the implication graph. Two types of self subsumption: backward self subsumption, forward self
subsumption. Forward subsumption is check old clauses being subsumed by newly learnt clause and
backward subsumption is check newly learnt clause to be subsumed by existing clauses.
MiniSAT employs four step Recursive Minimization Algorithm. First, mark all the first UIP clauses. For
each candidate literal, search the implication graph. Start at antecedents of candidate literals. If search
always terminates at marked literals remove that candidate.
Dynamic Largest Individual Sum:
One of the important factors in determining the speed of the choice of decision variables. Dynamic
Largest Individual Sum (DLIS) dictates assigning the decision variable which satisfies the maximum
number of unsatisfied clauses at current state. As it must set every clause satisfied that contains the
literal which has been set to True. A counter has to be maintained for each unsatisfied clause. When
clause counters transits from 0 to 1, update ranking of each variable present in that clause. Counters for all
unassigned variables need to be recalculated. This process also needs to be reversed when the search
backtracks. DLIS is state dependent, different variable assignments will give different count. Still based on
6. static statistics in the sense that it does not take search history into consideration. The next decision will be
determined by the current clause database and search tree, regardless of how you reach current state. A
good heuristics should also incorporate the learning so far to choose decision variable.
Variable State Independent Decaying Sum:
VSIDS selects the literal with most frequency over all the clauses with higher weight on most recently
learnt clauses. So one floating point counter is required for each literal. Initially all the counters are
assigned to 0. During the search, the metric is updated when a new clause is learned. Periodically, divide
each counters by a constant. Chaff divides all instances by two after 256 conflicts. So that recently
unsatisfied clauses are prioritized. Usually the value of the constant dictate how much newly learned
clause will prioritize. It is important for decisions to keep search strongly localized. Instead of developing
accurate heuristics it emphasizes on fast but dynamically adapted heuristics. This is independent of current
variable state but changes as new clauses are added to clause database.
It is observed recently added conflict clauses contain all the good variables from VSIDS. But the order of
the clause isn’t used in VSIDS. The basic idea of BerkMin’s Dynamic Second Order Heuristics is simply
try to satisfy recently learned clauses. Use VSIDS to choose the decision variable for one clause. If all
learned clauses are satisfied use other heuristics then. Intuitively achieve another order of localization (no
proofs yet).
Furthermore, MiniSAT takes advantage of literal phase saving to avoid solving independent subproblems
multiple times, when nonchronological backtracking occurs. Phase saving caches the literals that are
erased from the list of assignments during backtracking, and uses them to decide on the phase of the
variable that the branching heuristic suggests next. Using this strategy, SAT solvers maintain the
information of the variables that are not related to the current conflict, but forced to be erased from the list
of assignments by backtracking.
Clause Deletion:
Keeping as many clauses is good as they guarantee the completeness of the search procedure. But as the
search process backtracks each time, it learns a new clause and add it to the clause database. But
unrestricted clause keeping becomes unpractical in many cases. Many clause savings can lead to memory
exhaustion. The number of recorded clauses increases with number of conflicts and in the worst case,
number of learned clauses grows exponentially with the number of variables. In practice smaller clauses
are more effective. Moreover, large recorded clauses are not very useful for pruning search space.
Adding larger clauses leads to additional overhead for searching which overweighs the benefit of learning.
Some clause deletion techniques are:
1. We may consider norder learning, that records only clauses with n or fewer literals. So routinely
throw away large clauses. Logically the conflict clauses are implied. So there’s no loss in
soundness/completeness. But there is also danger of forgetting important clauses.
2. Clauses can be temporarily recorded while they either imply variable assignments or are unit
clauses, being discarded as soon as the number of unassigned literals is greater than an integer m.
This technique is named msize relevancebased learning.
3. Clauses with a size less than a threshold k are kept during the subsequent search, whereas larger
7. clauses are discarded as soon as the number of unassigned literals is greater than one. It is known
as kbounded learning.
MiniSAT reduces the number of clauses to half when limit is reached. It keeps most active, then shortest,
then youngest (FIFO) clauses. After reduction, maximum number of learned clauses is increased
geometrically.
A good heuristics is needed to determine which clause to throw away. GLUCOSE, a new heuristics is
now used determine the usefulness of a learnt clause. A learnt clause will link together blocks of
propagated literals. Usually stronger link are better. The score is the number of decision levels in the
clause. It allows to aggressively clean up clause database and arithmetic increase of number of kept
clauses.
Restart:
A sequence of short runs instead of a single long run may be a more effective search procedure. Useful
for satisfiable instances the solver may get stuck in the unsatisfiable part even if the search space contains
a large satisfiable part. Restarts add robustness to the solvers. They are commonly used in local search
algorithms to escape local minima. Restarting search after a number conflicts found. Usually all the
assignments and implications done by so far are undone. But the learned clauses are still kept in clause
database. So the learning so far are not lost. This learning reduces search space. So new search will have
less space to explore. Restarts are usually done after number of conflicts are found. Restarts are usually
costly as they involve undoing all the assignments and implications. They are also kept minimum to
maintain completeness of search dynamically increase restart limit. Variants of restart strategies include
randomized backtracking and randomized jump strategies. Randomized backtracking is when a conflict is
detected instead of jumping previous unassigned decision branch, the search proceeds to random previous
decision level.
One potential disadvantage of CDCL SAT solvers is all decision and implied variable assignments made
between the level of the conflict and the backtracking level are erased during backtracking. One possible
solution is to simply save the partial decision assignments. When the solver decides to branch on a
variable, it first checks whether there is information saved with respect to this variable. In case it exists,
the solver assigns the variable with the same value that has been used in the past. In case it does not exist,
the solver uses the default phase selection heuristic.
Stochastic Methods:
Using stochastic methods causes recent advancements in SAT solvers. One of the disadvantages of
stochastic methods that, it can’t prove unsatisfiability hence incomplete. Usually two types of stochastic
methods are seen: local search, hill climbing. Local search views the solution space as a set of points
connected to each other. It involves starting at some point in the solution space, and moving to the adjacent
points in an attempt to lower the cost function. The cost function for an assignment is usually the number
of unsatisfied clauses in CNF form. So the conclusion satisfied is achieved when local search finds a point
where cost function is zero i.e. every clause is satisfied. There are also long sequences of sideway moves,
i.e. moves that don’t increase or decrease number of unsatisfied clauses. It’s referred to as “Plateau”.
Local search are good for MAX SAT problems (where the goal is satisfy maximum number of clauses in
8. CNF form). When represented in CNF, SAT can be regarded as a discrete optimization problem with the
objective to maximize the number of satisfied clauses. Such incomplete local search may outperform
complete search algorithms. One of the main disadvantages of local search is being trapped in local
minima. They are escaped usually by adding random noise i.e. random walk by flipping a variable in an
unsatisfied clauses randomly in search space. It’s usually selects a variable flipping which doesn’t make
currently satisfied clause to unsatisfied. In practice, in a very high dimensional search space it is very rare
to encounter local minima. Although it may take substantial amount of time on each plateau before moving
on to the next one. Success depends on ability to move between successively lower plateaus. WalkSAT
SAT solver introduce random noise in the search in the form of uphill movement.
The focusing strategy of WalkSAT is based on selecting variables solely from unsatisfied clauses. It can
be shown that for any satisfiable formula and starting from any truth assignment, there exists a sequence
of flips using only variables from unsatisfied clauses such that one obtains a satisfying assignment. It
usually alternates between greedy steps and random steps. Recently, it was shown that the performance
of stochastic solvers on many structured problems can be further enhanced by using new SAT encodings
that are designed to be effective for local search. WalkSAT solver is faster on random kSAT.
The Survey Propagation (SP) method is another radical method that behaves like the usual DPLL based
methods, which also assign variable values incrementally in an attempt to find a satisfying assignment.
However, quite surprisingly, SP almost never has to backtrack. In other words, the heuristic guidancefrom
SP is almost always correct. Nonetheless, SP is able to efficiently approximate certain marginals on
random SAT instances and uses this information to successfully find a satisfying assignment.
Other Heuristics:
Usually CNF with larger number of variables and lesser number of constraints tend to be SAT and CNF
with smaller number of variables and larger number of constraints most likely to be UNSAT. If the
problem is SAT, find satisfying assignment quickly: prune search spaces where an assignment doesn’t
exist, try to satisfy as many clause as possible, zoom in on the space where the solution exists. And if the
problem is UNSAT try to prove unsatisfiability quickly: try and force a conflict (through implication).
Usually heuristics which does not depend on clauses are very cheap.
Beyond CDCL SAT Solvers:
Quantied Boolean Formula (QBF) reasoning and counting the number of models (solutions) of a problem
are two important problems that extend beyond propositional satisfiability testing and lie at the heart of the
next generation automated reasoning systems. These problems present fascinating challenges and expose
new research frontiers. Efficient algorithms for these will have a significant impact on many application
areas that are inherently beyond SAT, such as adversarial and contingency planning, unbounded model
checking, and probabilistic reasoning.
These problems can be solved, in principle and to some extent in practice, by extending the two most
successful frameworks for SAT algorithms: CDCL and local search. However, there are some interesting
issues and choices that arise when extending SATbased techniques to these harder problems. In general,
these problems require some techniques that reduces the effectiveness and relevance of commonly used
SAT heuristics designed for quickly exploiting a single solution.
9.
With the arrival of multicore processors, there is emerging interest in efficient multicore implementations
of parallel SAT solvers. Many discrete optimization techniques have been explored in the SAT context,
including simulated annealing, tabu search, artificial neural networks, and genetic algorithms. Another non
DPLL algorithm is Stålmarck's algorithm which uses breadthfirst search instead of depthfirst search as
in DPLL, and has been shown to be practically useful. Its performance relative to CDCL based solvers is
unclear as public versions of efficient implementations of Stålmarck's algorithm are not available due to its
proprietary nature.