Decidability
Turing Machines
Closure Properties and Chomsky Hierarchy
Decidability / Undecidability
Concept of Reducibility
Computational History and Rice’s Theorem
Other Problems
Computational History
Rice’s Theorem
●
The computational history for a Turing Machine
M in an input string w is simply the sequence of
configurations that the machine goes through
as it processes the input. This corresponds to
the output generated by Turing’s Universal
Machine.
#C1#C2#...#Ck#
●
The use of computation histories is a technique
most often employed to show that ACCEPT TM
is reducible to another formal language.
Computational History
Rice’s Theorem
●
Example: ALL CFG is not decidable
–
Suppose ALL CFG is decidable, D is decider
–
Consider now the TM M running on input
string w. If there is an accepting
computational history
●
C1 is a starting configuration
●
Ck is an accepting configuration
●
For every configuration Cj the next
configuration Cj+1 follows the rules of M
Computational History
Rice’s Theorem
–
Rather than constructing a CFG we will
construct a PDA. Given an input string t (a
computational history), D performs the
following
●
D checks C1 is not starting configuration
– If
●
it is then ACCEPT
D checks Ck is not accepting configuration
– If
it is then ACCEPT
Computational History
Rice’s Theorem
●
D now compares each computational
history with the next
– push
Cj on stack
– then
one symbol at at a time compare
with Cj+1
If any comparison does not match a
valid transition, then ACCEPT
Observe M accepts w if and only if L = L(D)
= S*
●
●
Computational History
Rice’s Theorem
●
●
ACCEPT TM (<M>,w) ≤ ALL CFG (<D>)
If ALL CFG (<D>) is decidable, then so is
ACCEPT TM (<M>,w). A contradiction.
Computational History
Rice’s Theorem
●
●
●
Every time we have addressed a decidability
question regarding Turing Machines we have
found the question undecidable. Let us prove
this fact once and for all.
Def: A subset P of all possible languages is
called a property.
Def: A property P is said to be a nontrivial
property of Turing Machines provided
–
there exists a TM MY such that L(MY) ∈ P
–
there exists a TM MN such that L(MN) ∉ P
Computational History
Rice’s Theorem
●
●
Examples:
–
nontrivial:
P EMPTY = {∅}
–
trivial:
PREC = { L | L recognizable}
Theorem:
Rice’s Theorem
Suppose P is a nontrivial property of Turing
Machines. Then the language
LP = { <M> | L(<M>) ∈ P }
is undecidable.
Computational History
Rice’s Theorem
●
Proof:
–
We will show that ACCEPT
–
Consider input (<M>,w).
TM
≤ LP.
Computational History
Rice’s Theorem
–
Case One: ∅ ∉ P
●
Choose a TM MY such that L(MY) ∈ P.
●
Define TM Mw acting on input strings t
– simulate
M on w
– if M ACCEPTs
● simulate M on input t
Y
● If M ACCEPTs, then ACCEPT
Y
Computational History
Rice’s Theorem
●
●
Note: Mw ACCEPTs t iff M ACCEPTs w and
MY ACCEPTs t.
– L(Mw)
= L(MY)
if M ACCEPTs w
– L(Mw)
=∅
if M does not
But MY ∈ P and ∅ ∉ P implies that
L(Mw) ∈ P if and only if M ACCEPTs w
●
Hence ACCEPT
TM
≤ LP
Computational History
Rice’s Theorem
–
Case Two:
∅∈P
●
then ∅ ∉ PC
●
and LP = (LPC)C
Other Problems
●
●
Decidability Problems are not limited to formal
languages. Any question which can be
rephrased into a simple YES or NO response is
a decidability problem.
Two areas of particular note are:
–
Mathematical Foundations
–
Number Theory
Other Problems
Mathematical Foundations
●
●
Propositional logic focuses on sample
statements (propositions), their truth values
(either true or false), and the four operators
not, and, or, and if-then. Truth tables are an
essential tool for determining the truth value for
more complicated formulas built up from
simpler components.
Theorem: The propositional logic is decidable.
Other Problems
Mathematical Foundations
●
●
Predicate calculus builds on propositional
logic by incorporating the concepts of
variables, quantifiers ∃ (existential) and ∀
(universal), and well-formed formulas.
Theorem: The predicate calculus is not
decidable.
Other Problems
Number Theory
●
Number Theory is one of the oldest branches
of mathematics and also one of its most
difficult. Alonzo Church, building on the work of
Kurt Godel, showed that no algorithm can
decide in general whether statements in
number theory are true or false.
●
∀x∃y[x+x=y]
example true statement
●
∃y∀x[x+x=y]
example false statement
Other Problems
Number Theory
●
●
Theorem: Theory (ℕ,+), natural numbers with
addition, is decidable.
Theorem: Theory (ℕ,+,x), natural numbers with
addition and multiplication, is not decidable.
Complexity
Landau Notation
Deterministic Time and Space
Nondeterministic Time and Space
Important Theorems
Landau Notation
●
Time complexity focuses on the question: How
long will this problem take to solve?
–
●
Speed will be measured by the number of
steps (transitions) necessary for a single tape
Turing Machine.
Space complexity focuses on the question:
How much scratch pad (memory) will I need?
–
Space will be measured by the number of
cells (tape positions) necessary for a single
tape Turing Machine.
Landau Notation
●
Landau Notation is a mathematical shorthand
designed to compare the rate of growth for
functions f:ℕ→ℕ
–
Ο(g(n)) provided
there exists C2 > 0 and no ∈ ℕ such that
Def: We say that f(n) is
f(n) ≤ C2 g(n) for all n > n0.
–
Ω(g(n)) provided
there exists C1 > 0 and no ∈ ℕ such that
Def: We say that f(n) is
C1 g(n) ≤ f(n) for all n > n0.
Landau Notation
–
Θ(g(n)) provided
there exists C1,C2 > 0 and no ∈ ℕ such that
Def: We say that f(n) is
C1 g(n) ≤ f(n) ≤ C2g (n) for all n > n0.
●
Lemma:
–
f(n) is
Ο(g(n)) iff g(n) is Ω(f(n))
–
f(n) is
Θ(g(n)) iff f(n) is Ο(g(n)) and Ω(g(n))
–
All three properties
Ο, Ω, and Θ are transitive
Landau Notation
–
Def: We say that f(n) is ο(g(n)) provided
limn→∞ f(n) / g(n) = 0.
–
Def: We say that f(n) is
ω(g(n)) provided
limn→∞ f(n) / g(n) = ∞.
Landau Notation
●
Lemma:
–
f(n) is ο(g(n)) iff g(n) is
ω(f(n))
–
If f(n) is ο(g(n)) then f(n) is
–
If f(n) is
–
Both properties
Ο(g(n))
ω(g(n)) then f(n) is Ω(g(n))
ο and ω are transitive
Landau Notation
●
Hierarchy of Basic Functions
–
1, log2(n), √n
–
n, n log2(n), n √n
–
n2, n3, n4, …
–
2n, 2n^2, 2n^3, ...
Deterministic
Time and Space
●
Def: The class DTIME(t(n)) is the collection
{ deterministic TM M | M is a decider
and for an input tape of length n
terminates in at most t(n) steps }
●
Example: L = { 0k 1k | k ≥ 0 }
D ∈ DTIME(n2)
Deterministic
Time and Space
●
Def: The class DSPACE(s(n)) is the collection
{ deterministic TM M | M is a decider
and for an input tape of length n
terminates having used at most s(n)
●
tape cells }
Example: L = { 0k 1k | k ≥ 0 }
D ∈ DSPACE(n)
Deterministic
Time and Space
●
Lemma:
–
If f(n) is Ο(g(n)) then DTIME(f(n)) ⊆
DTIME(g(n))
–
If f(n) is Ο(g(n)) then DSPACE(f(n)) ⊆
DSPACE(g(n))
Deterministic
Time and Space
●
Def: P = Uk DTIME(nk)
●
Def: PSPACE = Uk DSPACE(nk)
●
Def: EXP = Uk DTIME (2^(nk))
●
Def: EXPSPACE = Uk DSPACE (2^(nk))
●
Theorem:
–
P ⊆ EXP
–
PSPACE ⊆ EXPSPACE
Deterministic
Time and Space
●
In English:
–
P represents problems that can be solved in
a reasonable time.
–
EXP represents problems that typically
exceed normal waiting times.
–
PSPACE represents problems that require a
reasonable amount of memory.
–
EXPSPACE represents problems that require
exorbitant memory.
Deterministic
Time and Space
●
Lemma:
For any given function f :
ℕ→ℕ
DTIME(f(n)) ⊆ DSPACE(f(n))
●
Proof:
If a TM M terminates in at most f(n) steps, then
it can affect only f(n) tape cells.
Deterministic
Time and Space
●
●
Theorem:
–
P ⊆ PSPACE
–
EXP ⊆ EXPSPACE
Proof:
Use lemma above!
Deterministic
Time and Space
●
Theorem:
Summary of Important Results
–
P ⊆ EXP
–
PSPACE ⊆ EXPSPACE
–
P ⊆ PSPACE
–
EXP ⊆ EXPSPACE
Nondeterministic
Time and Space
●
Def: The class NTIME(t(n)) is the collection
{ nondeterministic TM M | M is a decider
and for an input tape of length n
terminates in at most t(n) steps
along any of its branches }
Nondeterministic
Time and Space
●
Example: SAT = { bool expr ϕ | ϕ is satisfiable }
–
Define D as follows
●
Generate the set of truth values
for each truth value combo, evaluate ϕ
●
if ϕ is true, then ACCEPT
REJECT
– D
∈ DTIME(2n)
– D
∈ NTIME(nk)
Nondeterministic
Time and Space
●
Def: The class NSPACE(s(n)) is the collection
{ nondeterministic TM M | M is a decider
and for an input tape of length n
terminates having used at most s(n)
tape cells along any of its branches }
Nondeterministic
Time and Space
●
Example: SAT = { bool expr ϕ | ϕ is satisfiable }
–
Define D as in previous example
●
TM D requires storage for input (n), truth
values (n), and stack contents (n)
– D
∈ DSPACE(n)
– D
∈ NSPACE(n)
Space is reusable; time is not!
Nondeterministic
Time and Space
●
Lemma:
–
If f(n) is Ο(g(n)) then NTIME(f(n)) ⊆
NTIME(g(n))
–
If f(n) is Ο(g(n)) then NSPACE(f(n)) ⊆
NSPACE(g(n))
Nondeterministic
Time and Space
●
Def: NP = Uk NTIME(nk)
●
Def: NPSPACE = Uk NSPACE(nk)
●
Def: NEXP = Uk NTIME (2^(nk))
●
Def: NEXPSPACE = Uk NSPACE (2^(nk))
●
Theorem:
–
NP ⊆ NEXP
–
NPSPACE ⊆ NEXPSPACE
Nondeterministic
Time and Space
●
Lemma:
For any given function f :
ℕ→ℕ
NTIME(f(n)) ⊆ NSPACE(f(n))
●
Proof:
If a TM M terminates in at most f(n) steps, then
it can affect only f(n) tape cells.
Nondeterministic
Time and Space
●
●
Theorem:
–
NP ⊆ NPSPACE
–
NEXP ⊆ NEXPSPACE
Proof:
Use lemma above!
Nondeterministic
Time and Space
●
Theorem:
Summary of Important Results
–
NP ⊆ NEXP
–
NPSPACE ⊆ NEXPSPACE
–
NP ⊆ NPSPACE
–
NEXP ⊆ NEXPSPACE
Important Theorems
●
●
Theorem One: If M is a deterministic multitape Turing Machine and M ∈ DTIME(t(n)),
then M has an equivalent deterministic singletape Turing Machine M’ with M’ ∈ DTIME
(t(n)2).
Corollary: Pmulti-tape ⊆ Psingle-tape
Important Theorems
●
●
Theorem Two: If M is a nondeterministic
single-tape Turing Machine and M ∈
NTIME(t(n)), then M has an equivalent
deterministic single-tape Turing Machine M’
with M’ ∈ DTIME (2O(t(n))).
Corollary: NP ⊆ EXP
Important Theorems
●
●
Theorem Three: If M is a deterministic singletape Turing Machine and M ∈ DSPACE(s(n)),
then M ∈ DTIME (2O(s(n))).
Corollary: PSPACE ⊆ EXP
Important Theorems
●
Theorem Four: (Savitch’s Theorem)
If M is a nondeterministic single-tape Turing
Machine and M ∈ NSPACE(s(n)), then M has
an equivalent deterministic single-tape Turing
Machine M’ with M’ ∈ DSPACE (s(n)2).
●
Corollary:
–
PSPACE = NPSPACE
–
EXPSPACE = NEXPSPACE
Important Theorems
CONCLUSION
P ⊆ NP ⊆ NPSPACE = PSPACE ⊆
EXP ⊆ NEXP ⊆ NEXPSPACE = EXPSPACE