Greedy Algorithm贪心算法

Greedy Algorithms Greedy Algorithms

Dr. Zhenyu He

Autumn 2008

1

16

Greedy Algorithms z 16 Greedy Algorithms

Similar to dynamic programming. Used for optimization problems.

2z

z

Optimization problems typically go through a sequence of steps, with a set of

choices at each step.

For many optimization problems, using dynamic programming to determine

z

y p p ,g y p g g the best choices is overkill (过度的杀伤威力).

Greedy Algorithm: Simpler, more efficient

16

Greedy Algorithms z

16 Greedy Algorithms

Greedy algorithms (GA) do not always yield 3

y g ()y y

optimal solutions, but for many problems they do.

161the activity selection problem

16.1, the activity-selection problem （活动安排）16.2, basic elements of the GA; knapsack prob.

（贪婪算法的基本特征；背包问题）

16.3, an important application: the design of data compression (Huffman) codes. （哈夫曼编码）16.5, unit-time tasks scheduling

6.5,u t t e tas s sc edu g （有限期作业调度）

16

Greedy Algorithms z

16 Greedy Algorithms

The greedy method is quite powerful and works 4

g y q p

well for a wide range of problems:

minimum-spanning-tree algorithms (Chap 23)

minimum-spanning-tree algorithms (Chap 23)

（最小生成图）

h t t th f i l (Ch 24)

shortest paths from a single source (Chap 24)（最短路径）

set-covering heuristic (Chap 35).

（集合覆盖）

…

First example:Activity Selection First example: Activity Selection

Horseback Riding (骑马)

5

Canoeing (独木舟)

Surfing (冲浪)

Driving

Swimming Rock Climbing (攀岩)

z

9

1011121314151617

How to make an arrangement to have the more activities?

S1Sh i i fi 短优原

S1. Shortest activity first （最短活动优先原则）Swimming , Driving

S2First starting activity first

S2. First starting activity first

（最早开始活动优先原则）Horseback Riding , Driving

S3. First finishing activity first （最早结束活动优先原则）

Swimming , Rock Climbing , Surfing

161An activity selection problem 16.1 An activity-selection problem

6

i i i i

z

n activities require exclusive use of a common resource.Example, scheduling the use of a classroom.playground

p e,sc edu g e use o c ss oo

（n 个活动，1 项资源，任一活动进行时需唯一占用该资源）

Set of activities S = {a 1, a 2, . . . , a n }.

a i needs resource during period [s i , f i ), which is a half-open interval, where s i is start time and f i is finish time.

Goal:Select the largest possible set of nonoverlapping (mutually

Goal: Select the largest possible set of nonoverlapping (mutually compatible) activities. （安排一个活动计划，使得相容的活动数目最多）Other objectives: Maximize income rental fees , …j ,

161

An activity selection problem

z

16.1 An activity-selection problem

n activities require exclusive use of a common resource.

7

Set of activities S = {a1, a2, . . . , a n}

a i needs resource during period [s i , f i )

l S fi i i

z Example: S sorted by finish time:

Maximum-size mutually

Maximum size mutually

compatible set:

{a1 , a3 , a6 , a

,a3,a6,a8}.

Not unique: also

{a2 , a5 , a7 , a9}.

16.1.1 Optimal substructure of activity selection

Space of subproblems 8

z S ij = {a k ∈S : f i ≤ s k < f k ≤ s j }

= activities that start after a i finishes & finish before a j starts z

Activities in S ij are compatible with

all activities that finish by f i （完成时间早于f i 的活动）, and ll ti iti th t t t li th z

all activities that start no earlier than s j .

To represent the entire problem, add fictitious activities:a 0=[0);1=[“

z

0 = [﹣∞, 0); a n+1 = [∞, “∞+1”)We don’t care about ﹣∞ in a 0 or “∞+1” in a n+1.

Then S = S 0,n+1

. Range for S ij is 0 i, j n + 1.,g j ≤,j ≤

16.1.1 Optimal substructure of activity selection Space of subproblems 9

z z

p p

S ij = {a k ∈S : f i ≤ s k < f k ≤ s j }

Assume that activities are sorted by monotonically increasing finish time

Assume that activities are sorted by monotonically increasing finish time （以结束时间单调增的方式对活动进行排序）

f 0 ≤ f 1 ≤ f 2 ≤ · · · ≤ f n < f n+1 (if i ≤ j, then f i ≤ f j )

(16.1)11

10

98

7

6

5

4

3

2

1

123456

7891011121314

16.1.1 Optimal substructure of activity selection

if 012···n

Then i ≥ j S ij =

i k

if f 0 ≤ f 1 ≤ f 2 ≤ ≤ f n < f n+1 (if i ≤ j, then f i ≤ f j ) (16.1)Proof If there exists a k ∈S ij , then

f i ≤ s k < f k ≤ s j < f j f i < f j .

z

But i ≥ j f i ≥ f j . Contradiction.

So only need to worry about with 0 ≤i

are ij S ij

16.1.1 Optimal substructure of activity selection

11

S th t l ti t i l d H 2b b z

Suppose that a solution to S ij includes a k . Have 2 sub-prob S ik (start after a i finishes, finish before a k starts)S kj (start after a k finishes, finish before a j starts)z

Solution to S ij = (solution to S ik )∪{a k }∪(solution to S kj )

Since a k is in neither of the subproblems, and the subproblems are disjoint |solution to S |=|solution to ik |+1+|solution to kj |z

disjoint, | solution to S | = | solution to S ik | + 1 + | solution to S kj | .Optimal substructure: If an optimal solution to S ij includes a k , then the solutions to S ik and S kj used within this solution must be optimal ll (l t d t t)z

as well. (use usual cut-and-paste argument).Let A ij = optimal solution to S ij ,

so A ij = A ik ∪{a k ∪A kj (16.2)so j {}j ,

assuming: S ij is nonempty; and we know a k .

()

A i

l i

z

16.1.2 A recursive solution Let c[i, j ] = size of maximum-size subset of mutually compatible activities in

ij c[i j]ij 12

i ≥ j S ij = c[i, j ] = 0.

z S ij .（c[i,j] 表示S ij 相容的最大活动数）If S ij ≠ , suppose that a k is used in a maximum-size subsets of mutually S ij .Then c[i j]=c[i k]+1+c[k j ]z

Then c[i, j] = c[i, k] + 1 + c[k, j ] .

But of course we don’t know which k to use, and so

0if Sij = (16.3)

0 , if Sij ,

c [i , j ] =

max{c [i , k ] + c [k , j ] + 1} , if Sij ? .

　

i

k ij

a S ∈

16.1.3 Converting a DP solution to a greedy solution

　0 , if S ij = ,

] 13

　i < k < j c [i , j ]

= max{c [i , k ] + c [k , j ] + 1} , if S ij ? .

(16.3)

z

　a k S ij

It may be easy to design an algorithm to the problem based on recurrence

(16.3).

Di t i l ith (l it ?)

z

Direct recursion algorithm (complexity?)Dynamic programming algorithm (complexity?)Can we simplify our solution?

16.1.3 Converting a DP solution to a greedy solution

Theorem 16.1L t d l t b th ti it i ith th li t fi i h ti 14

Let S ij ≠ , and let a m be the activity in S ij with the earliest finish time: f m =min {f k : a k ∈S ij }. Then

1. a m is used in some maximum-size subset of mutually compatible activities of S i j .（a m 包含在某个最大相容活动子集中）

2. S im = , so that choosing a m leaves S mj as the only nonempty

subproblem.

（仅剩下一个非空子问题S mj ）p j 11

10

98

7

6

5

4

3

2

1

123456

7891011121314

16.1.3 Converting a DP solution to a greedy solution

Theorem 16.1L t d l t b th ti it i ith th li t fi i h ti 15

Let S ij ≠ , and let a m be the activity in S ij with the earliest finish time: f m =min {f k : a k ∈S ij }. Then

1. a m is used in some maximum-size subset of mutually

compatible activities of S i j .（a m 包含在某个最大相容活动子集中）2. S im = , so that choosing a m leaves S mj as the only

nonempty subproblem. （仅剩下一个非空子问题S mj ）

Proof

2. Suppose there is some .Then Then a S ∈i l l m m l m f s f s f f f <<≤

and it has an earlier finish time than , which contradicts our choice of

l im

m f l

im

im

a S S ∈?=?

16.1.3 Converting a DP solution to a greedy solution

Theorem 16.1 Let S ij ≠ , and let a m be the activity in S ij with the li t fi i h ti i {f }Th

16

earliest finish time: f m = min {f k : a k ∈S ij }. Then 1. a m is used in some maximum-size subset of mutually

compatible activities of S i j .（a m 包含在某个最大相容活动子集中）p j Proof 1. Let A ij be a maximum-size subset of mutually Compatible

activities in ij Order activities in ij in monotonically increasing activities in S ij . Order activities in A ij in monotonically increasing order of finish time. Let a k be the first activity in A ij .If a k = a m , done (a m is used in a maximum-size subset).

a k a m

Otherwise construct =A ij }(replace k by )

first activity in A ij to finish. f m ≤f k a m doesn’t overlap anything Otherwise, construct B ij A ij -{a k }∪{a m } (replace a k by a m ).Activities in B ij are disjoint. (Activities in A ij are disjoint, a k is the else in ij ).Since |B ij |=ij |and ij is a maximum-size subset,else in B ij ). Since |B ij | |A ij | and A ij is a maximum size subset,so is B ij .

16.1.3 Converting a DP solution to a greedy solution

　0 , if S ij = ,

,j ]= 17

　i k k S

c [i , j ]

max{c [i , k ] + c [k , j ] + 1} , if S ij ? .

　a < < ij

(16.3)

Th 161L t d l t b th ti it i

Theorem 16.1 Let S ij ≠ , and let a m be the activity in S ij with the earliest finish time: f m = min {f k : a k ∈S ij }. Then 1. a m is used in some maximum-size subset of mutually y

compatible activities of S i j .（a m 包含在某个最大相容活动子集中）2. S im = , so that choosing a m leaves S mj as the only nonempty b bl 仅剩下个非空子问题z

subproblem. （仅剩下一个非空子问题S mj ）This theorem is great:

before theorem

after theorem # of sub-prob in optimal solution #of choices to consider 2O (j i 1)

11

# of choices to consider

O (j –i –1)

16.1.3 Converting a DP solution to a greedy solution

Theorem 16.1 Let S ij ≠ , and let a m be the activity in S ij with the earliest finish time:=min {f k :k ij }Then

18

earliest finish time: f m = min {f k : a k ∈S ij }. Then 1. a m is used in some maximum-size subset of mutually compatible activities

of S i j .（a m 包含在某个最大相容活动子集中）

2i = so that choosing leaves j as the only nonempty subproblem

z

2. S im = , so that choosing a m leaves S mj as the only nonempty subproblem.（仅剩下一个非空子问题S mj ）

Now we can solve a problem ij in a top-down fashion Now we can solve a problem S ij in a top down fashion Choose a m ∈S ij with earliest finish time: the greedy choice. ( it leaves as

much opportunity as possible for the remaining activities to be scheduled )

（留下尽可能的时间来安排活动，贪心选择）

Then solve S mj .

z

16.1.3 Converting a DP solution to a greedy solution

What are the subproblems?

19

Original problem is S 0,n+1 〔a 0 = [﹣∞, 0); a n+1 = [∞, “∞+1”) 〕Suppose our first choice is a m1 (in fact, it is a 1)Then ne t s bproblem is Then next subproblem is

S m1,n+1Suppose next choice is a m2 (it must be a 2?)

Next subproblem is

S m2,n+1p ,

And so on

11

10

7

8

93

5

4

6

1

2

123456

7891011121314

z

16.1.3 Converting a DP solution to a greedy solution

What are the subproblems?

20

Original problem is S 0,n+1

Suppose our first choice is a m1Then next subproblem is m1n+1 Then next subproblem is S m1,n+1Suppose next choice is a m2Next subproblem is S m2,n+1z

And so on

Each subproblem is S mi ,n+1.

And the subproblems chosen have finish times that increase z

z

And the subproblems chosen have finish times that increase. （所选的子问题，其完成时间是增序排列）

Therefore, we can consider each activity just once, in monotonically increasing order of finish time increasing order of finish time.