NP completeness. Classes P and NP are two frequently studied classes of problems in computer science. Class P is the set of all problems that can be solved by a deterministic Turing machine in polynomial time.
2. Instructor
Prof. Amrinder Arora
amrinder@gwu.edu
Please copy TA on emails
Please feel free to call as well
Available for study sessions
Science and Engineering Hall
GWU
Algorithms NP-Completeness 2
LOGISTICS
5. “Any question can be made
immaterial by subsuming all its
answers under a common head.
The sovereign road to
indifference, whether to evils or
to goods, lies in the thought of
the higher genus.”
Algorithms NP-Completeness 5
THE HIGHER GENUS
William James
7. P = { L | L is accepted by a deterministic Turing
Machine in polynomial time}
The TM takes at most O(nc) steps to accept a string of length n
NP = { L | L is accepted by a non-deterministic Turing
Machine in polynomial time }
The TM takes at most O(nc) steps on each computation path to
accept a string of length n
They are sets of languages
Algorithms NP-Completeness 7
THE CLASS P AND THE CLASS NP
8. A language is defined as a set of strings
For example, if an alphabet is {a, b, c}, then a language can be:
{“a”, “ab”, “ac”}. Another language can be: {“ab”, “abab”,
“ababab”, “abababab”, “ababababab”, …}. Languages can be
finite or infinite.
Problem is typically defined as when we are asked to
find/solve, or at least confirm something.
For example, given a graph G does it have a Hamiltonian Cycle?
Algorithms NP-Completeness 8
LANGUAGES VS. PROBLEMS
9. A problem instance can also be encoded as a String.
For example, a graph G can be encoded where the
first line of the input file contains the number of
nodes n, then the edges, etc.
If the algorithm (Turing Machine) accepts that input,
that input is in the language of the Turing Machine.
All inputs that are valid graphs and have Hamiltonian
Cycles, can be accepted. All inputs that are either
malformed (not a graph) or don’t have Hamiltonian
cycle, can be rejected.
Algorithms NP-Completeness 9
ENCODING A PROBLEM
10. Thus, a language L is a mathematical way of
representing what we usually define as a problem.
Algorithms NP-Completeness 10
LANGUAGES ARE PROBLEMS
11. P = { L | L is accepted by a deterministic Turing
Machine in polynomial time}
The TM takes at most O(nc) steps to accept a string of length n
NP = { L | L is accepted by a non-deterministic Turing
Machine in polynomial time }
The TM takes at most O(nc) steps on each computation path to
accept a string of length n
They are sets of languages
Or, as we now know, P and NP are classes (sets) of
problems.
Algorithms NP-Completeness 11
THE CLASS P AND THE CLASS NP
12. Algorithms NP-Completeness 12
TURING MACHINE REFRESHER
. . .
Infinite tape: Γ*
Tape head: read current square on tape,
write into current square,
move one square left or right
FSM: like PDA, except:
transitions also include direction (left/right)
final accepting and rejecting states
FSM
13. Are non-deterministic Turing machines really more
powerful (efficient) than deterministic ones?
Essence of P vs. NP problem
Does non-determinism help?
In case of automata – there is no difference
In case of PDA – yes, there is a difference
In case of TM, we do not know
Algorithms NP-Completeness 13
D TM VS. N TM
14. One of the most important unanswered questions since
1960s
Many optimization problems appear amenable only to brute force,
i.e. (near-)exhaustive enumeration.
Edmonds 1966: “The classes of problems which are respectively
known and not known to have good algorithms are of great
theoretical interest […] I conjecture that there is no good
algorithm for the traveling salesman problem. My reasons are the
same as for any mathematical conjecture: (1) It’s a legitimate
mathematical possibility, and (2) I do not know.”
How can we make progress on this problem?
We could find an algorithm for every NP problem
We could use polynomial time reductions to find the “hardest”
problems and just work on those
Algorithms NP-Completeness 14
P = NP?
15. Input for
Problem B
Output for
Problem B
Algorithm for Problem B
Reduction
from
B to A
Algorithm
for A
x R(x)
Yes/No
Algorithms NP-Completeness 15
REDUCIBILITY
16. How would you define NP-Complete?
They are the “hardest” problems in NP
Algorithms NP-Completeness 16
NP-COMPLETENESS
P
NP
NP-Complete
17. Q is an NP-Complete problem if:
1) Q is in NP
2) Every other NP problem polynomial time reducible
to Q
Algorithms NP-Completeness 17
DEFINITION OF NP-COMPLETE
18. Algorithms NP-Completeness 18
Satisfiability problem: Given a boolean formula, find whether it
has a satisfying assignment or not.
For example, say the formula is:
(x1 or x2 or x3) and (x1 or n(x2) or n(x3)) and
(n(x1) or x2 or n(x3)) and (n(x1) or n(x2) or n(x3)) and
(n(x1) or x2 or x3) and (x1 or n(x2) or x3) and
(x1 or n(x2) or n(x3)) and (n(x1) or x2 or n(x3))
Here n(x1) represents the negation of x1. So, if x1 is true, then
n(x1) is false, and vice versa.
So, if we assign x1 = x2 = x3 = true, then overall clause
becomes:
(T or T or T) and (T or F or F) and (F or T or F) and (F or F or F)
and
(F or T or T) and (T or F or T) and (T or F or F) and (F or T or F),
which becomes: T and T and T and F and T and T and T and T = F
= false.
So, this assignment does not satisfy this clause.
Some other true/false assignment to variables may satisfy this
clause, or it is possible that this clause is not satisfiable.
SAT
19. Save your job
Well, I can’t solve it, but so can’t thousands of
other computer scientists working for past 40
years on this problem X (even if we just invented
X)
Approximation
Chances of coming with an optimal solution to X
are slim, perhaps we can focus on approximate
solution instead?
Computability
X may really be impossible to solve in
polynomial time, and perhaps we need to
simplify it to solve it in reasonable amount of
time.
Algorithms NP-Completeness 19
TOP 3 REASONS TO PROVE PROBLEM X IS
NP-COMPLETE
20. Input for
Problem B
Output for
Problem B
Reduction
from
B to A
Algorithm
for A
x R(x) Yes/No
Algorithm for Problem B
Algorithms NP-Completeness 20
REDUCIBILITY
Problem A is at least as hard as Problem B.
21. Satisfiability problem is that given a boolean formula, we have to
find whether it has a satisfying assignment or not.
For example, say the formula is:
(x1 or x2 or x3)
and (x1 or n(x2) or n(x3))
and (n(x1) or x2 or n(x3))
and (n(x1) or n(x2) or n(x3))
and (n(x1) or x2 or x3)
and (x1 or n(x2) or x3)
and (x1 or n(x2) or n(x3))
and (n(x1) or x2 or n(x3))
Here n(x1) represents the negation of x1. So, if x1 is true, then
n(x1) is false, and vice versa.
Algorithms NP-Completeness 21
SAT
22. 22
Suppose we are given a NTM N and a string w of length
n which is decided by N in f(n) or fewer
nondeterministic steps.
Then there is an explicit CNF formula f of length O
(f(n)3) which is satisfiable iff N accepts w.
In particular, when f(n) is a polynomial, f has
polynomial length in terms of n so that every language
in NP reduces to CSAT in polynomial time. Thus CSAT
is NP-hard.
Finally, as CSAT is in NP, CSAT is NP-complete.
Algorithms NP-Completeness
COOK - LEVIN
THEOREM
23. To show that X is NP-Complete:
1. Show that X is in NP, i.e., a polynomial time verifier
exists for X.
2. Pick a suitable known NP-complete problem, S (ex:
SAT)
3. Show a polynomial algorithm to transform an
instance of S into an instance of X
SOX RESTOX mnemonic can help.
SAT Outside the Box
Reduce SAT To X
Algorithms NP-Completeness 23
NP-COMPLETENESS PROOF METHOD
24. To show that X is NP-Complete:
1. Show that X is in NP, i.e., a polynomial time verifier
exists for X.
2. Pick a suitable known NP-complete problem, S (ex:
SAT)
3. Show a polynomial algorithm to transform an
instance of S into an instance of X
a) Draw the boxes, including inputs and outputs
b) Write relationship between the inputs
c) Describe the transformation
Algorithms NP-Completeness 24
NP-COMPLETENESS PROOF METHOD (CONT.)
25. Input for
Problem B
Output for
Problem B
Reduction
from
B to A
Algorithm
for A
x R(x) Yes/No
Algorithm for Problem B
Algorithms NP-Completeness 25
REDUCIBILITY
Problem A is at least as hard as Problem B.
26. CLIQUE = { <G,k> | G is a graph with a clique of size
k }
A clique is a subset of vertices that are all connected
Why is CLIQUE in NP?
Algorithms NP-Completeness 26
EXAMPLE: CLIQUE
27. Textbook Section 9.5.1
Pick an instance of 3-SAT, Φ, with k clauses
Make a vertex for each literal
Connect each vertex to the literals in other clauses
that are not the negation
Any k-clique in this graph corresponds to a satisfying
assignment
Algorithms NP-Completeness 27
REDUCING 3-SAT TO CLIQUE
29. INDEPENDENT SET = { <G,k> | where G has an
independent set of size k }
An independent set is a set of vertices, such that
there are no edge between any two vertices in that
set (two people are “independent” if they do not
know each other).
How can we reduce the clique problem to IS?
Algorithms NP-Completeness 29
EXAMPLE: INDEPENDENT SET (IS)
30. This is the dual problem!
Complement of a graph: Same vertices, but
“reversed” edges – remove the ones that exist, and
add the ones that don’t.
Algorithms NP-Completeness 30
CLIQUE TO INDEPENDENT SET
31. HAMILTONIAN PATH TO HAMILTONIAN CYCLE
Algorithms NP-Completeness 31
Given a graph G = (V,E), construct a graph G’ =
(V’,E’), such that:
V’ = V union {z}
E’ = E union {(z,v) | all v in V}
G’ has one more vertex, and n more edges
Claim:
G has a Hamiltonian Path if and only if G’ has a Hamiltonian
Cycle
32. HAMILTONIAN CYCLE TO HAMILTONIAN PATH
Algorithms NP-Completeness 32
Given a graph G = (V,E), construct a graph G’ =
(V’,E’), such that:
V’ = V union {z, w, x}
E’ = E union new edges.
Firstly, select a random edge {u,v} in E.
{w,u}
{x,v} for all v connected to u.
{z,x}
[Thus, G’ has 3 more vertices, and a few more edges]
Claim:
G has a Hamiltonian Cycle if and only if G’ has a Hamiltonian
Path
33. Algorithm for Independent Set
Algorithms NP-Completeness 33
INDEPENDENT SET TO VERTEX COVER
Input for
Independent
Set
Output for
Independent
Set
Transformation
G’ = G
Algorithm for
Vertex Cover
G,k G’,n-k Yes/No
34. Show the following polynomial time reductions
Independent Set P Vertex Cover
Vertex Cover P Dominating Set
3-SAT P Vertex Cover
Hamiltonian Cycle P Hamiltonian Path
Algorithms NP-Completeness 34
PRACTICE NP-COMPLETENESS REDUCTIONS
35. NP complete problems are the HARDEST problems in
NP
Reducibility – Art of reducing one problem to another
Any problem in NP can be reduced to any NP-
complete problem in polynomial time.
Is P = NP? This is one of the most fascinating
question in Computer Science today, with
phenomenal impacts if proven in affirmative.
Reducing problems can be hard, takes practice
SAT and 3-SAT are one of the earlier known NP-
complete problems, today there are thousands.
Algorithms NP-Completeness 35
SUMMARY
36. To prove a problem X is NP-complete, remember to
do two things:
Show X is in NP
Take a well known NP-complete problem, such as SAT, and
reduce SAT to X.
SOX RESTOX mnemonic may help
Algorithms NP-Completeness 36
SUMMARY