ch10-图的分析1

Note to other teachers and users of these slides: We would be delighted if you found this our material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them to fit your own needs. If you make use of a significant portion of these slides in your own lecture, please include this message, or a link to our web site:
8
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
[Girvan-Newman ‘02]

Divisive hierarchical clustering based on the notion of edge betweenness:
1. 2.
How to compute betweenness? How to select the number of clusters?
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,

arg minA,B cut(A,B) Degenerate case:
3

Find micro-markets by partitioning the query-to-advertiser graph:
query
advertiser
[Andersen, Lang: Communities from seed sets, 2006]
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 4
1 2 3 4 5 6
A
B
26
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,

Express partitioning objectives as a function of the “edge cut” of the partition Cut: Set of edges with only one vertex in a group:
16

Compute betweenness by working up the tree: If there are multiple paths count them fractionally
1+1 paths to H Split evenly 1+0.5 paths to J Split 1:2 1 path to K. Split evenly
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
2
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 17

Compute betweenness by working up the tree: If there are multiple paths count them fractionally

Edge betweenness: Number of shortest paths passing over the edge Intuition:
b=16
b=7.5
Edge strengths (call volume) in a real network
Edge betweenness in a real network
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datte club: Hierarchical decomposition
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
Q
Next time: Why not optimize Modularity directly?
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 23

1
5 4
2
3
6
A
2
1
A
2
1 4
5
6
B
cut(A,B) = 2
3
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
27

Criterion: Minimum-cut
Minimize weight of connections between groups
Connected components are communities Gives a hierarchical decomposition of the network
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 9
How to find communities?
We will work with undirected (unweighted) networks
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 7
Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman
Stanford University


We often think of networks being organized into modules, cluster, communities:
13
1. 2.
How to compute betweenness? How to select the number of clusters?
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
14


0
1
2
3
4
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
15

J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
Number of shortest paths passing through the edge

Girvan-Newman Algorithm:
Undirected unweighted networks
Repeat until no edges are left:
Calculate betweenness of edges Remove edges with highest betweenness
Step 3:
Hierarchical network decomposition:
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
11
Communities in physics collaborations
21

Normalizing cost.: -1<Q<1
Aij = 1 if ij, 0 else
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
22

Modularity is useful for selecting the number of clusters:

Clusters in Movies-to-Actors graph:
[Andersen, Lang: Communities from seed sets, 2006]
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 5
1
12 33 49
Need to re-compute betweenness at every step
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,
10
Step 1:
Step 2:
19

Need a null model!
J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets, 20

i
j
Note: