CS 361A

CS 361A - Autumn 2003-04
(Advanced Data Structures and Algorithms)

News Flash Administrivia Signup Overview Handouts/Homeworks Lecture Schedule Readings

News Flash

Homework 4 has just been released..

Administrivia
        Instructor:   Rajeev Motwani
        Teaching Assistant: Krishnaram Kenthapadi (kngk@stanford.edu)
        Class Schedule: Mon/Wed, 3:15-4:30, Room 380-380W
        Office Hours:
                                Krishnaram Kenthapadi              Tue 12:45-2:30 and Thu 1:15--2:30, Gates 510 (5-3903)
                                Rajeev Motwani                        Tue, 1:30-2:30, Gates 474 (3-6045)

Course URL: http://theory.stanford.edu/~rajeev/cs361-2003.html

Class Sign-up To sign up for this course, please send email to Krishnaram Kenthapadi with the following information: name, department, status (Phd/MS/UG, year), area (Databases, Systems, Theory, etc), and email address. Please also include registration status (credit, pass/fail/audit).

Class Mailing List We have set up a class mailing list to help you get the latest information regarding the class. The email lists are auto-populated using current course enrolment information. The main list will be: cs361a-aut0304-all@lists.stanford.edu . Those who audit the course can subscribe by sending an email to majordomo@lists.stanford.edu with the following text in the body of the mail: subscribe cs361a-aut0304-guests

Grading Since this course will be treated as a graduate research seminar, we expect that most students will register pass/fail (and not for a letter grade). If you do choose to sign up for a letter grade, be sure to mention this in your sign-up email to the TA. We will give out 3-4 homeworks, one of which will serve as a take-home midterm exam. There will no final exam. The scores on these homeworks as well as class participation will determine your final grade.

Course Overview

Efficient strategies for complex data-structuring problems are essential in the design of fast algorithms for a variety of applications, including combinatorial optimization, databases and data mining, information retrieval and web search, and geometric applications. We will give a systematic exposition of the central ideas in the design of such data structures. The second main theme of this course will be the design and analysis of online algorithms and data stream algorithms. The field of competitive analysis of online algorithms got its start in the amortized analysis for data structures and forms a natural extension of some of the ideas we will discuss in the earlier part of the course. We will present some of the main ideas and motivating applications for this class of algorithms. Time permitting, we will also cover some topics in the related area of algorithms and data structures in the stream model of computation. The material to be covered will be drawn from the following list:

Advanced Data Structures: hash tables (universal hashing, perfect hashing, locality-sensitive hashing, Bloom filters); data structures for combinatorial optimization (union-find, Fibonacci heaps, dynamic trees, dynamic graph structures); self-adjusting data structures (lists, splay trees); search trees (red-black trees, self-adjusting trees, treaps, skip lists, finger search trees, biased search trees); fault-tolerant and persistent data structures; suffix trees and string searching; databases/data-mining/data-stream (histograms, indexes, hashing, synopses and sketches, sliding windows); geometric and kinetic data structures.

Online and Data Stream Algorithms: paging/caching problems; abstractions (k-server problem, request-answer games, and metrical task systems); scheduling and load balancing; network algorithms; data migration/replication in distributed computing; stream algorithms and data structures for database problem.

This course should be of interest to graduate students in computer science and related fields, especially those with a mathematical bent of mind. We will assume familiarity with basic material in algorithms, combinatorics, and probability theory (at the level of the core undergraduate courses on these topics).

Handouts and Homeworks

Handout	Date	Topic	Download
1	Wed, Sep 24	Reading List	ps or pdf
2	Wed, Sep 24	Notes for Lectures 1/2/3	ps or pdf
3	Wed, Oct 1	Notes for Lecture 3/4	hard-copy only
4	Mon, Oct 6	Notes for Lecture 5	hard-copy only
5	Wed, Oct 8	Homework 1	ps or pdf
6	Mon, Oct 13	Notes for Lecture 6	hard-copy only
7	Wed, Oct 15	Notes for Lectures 7/8	hard-copy only
8	Mon, Oct 20	Homework 2	ps or pdf
9	Wed, Oct 22	Notes for Lectures 9/10	hard-copy only
10	Wed, Oct 29	Notes for Lecture 11	hard-copy only
11	Mon, Nov 3	Homework 3	ps or pdf
12	Mon, Nov 3	Notes for Lecture 12	hard-copy only
13	Wed, Nov 5	Notes for Lecture 13	hard-copy only
14	Mon, Nov 10	Notes for Lecture 14	hard-copy only
15	Wed, Nov 12	Slides for Lecture 15	ppt
16	Mon, Nov 17	Slides for Lecture 16/17	ppt
17	Wed, Nov 19	Homework 4	ps or pdf
18	Mon, Nov 24	Slides for Lecture 18	ppt
19	Wed, Nov 26	Slides for Lecture 19	ppt
20	Mon, Dec 1	Slides for Lecture 20	ppt
21	Wed, Dec 3	Slides for Lecture 21	ppt

Lecture Schedule

Lecture	Date	Topic	Lecture Notes
1 2	Wed, Sep 24 Mon, Sep 29	Should Tables be Sorted?	Handout 2 (ps, pdf)
3 4	Wed, Oct 1 Mon, Oct 6	Hashing: Universal and Perfect	Handout 2 (ps, pdf) Handout 3 (hard-copy only)
5	Wed, Oct 8	Amortization and List Update	Handout 4 (hard-copy only)
6	Mon, Oct 13	Disjoint Sets and Union-Find	Handout 6 (hard-copy only)
7 8	Wed, Oct 15 Mon, Oct 20	Competitive Analysis and Paging	Handout 7 (hard-copy only)
9 10	Wed, Oct 22 Mon, Oct 27	Randomized Online Algorithms	Handout 9 (hard-copy only)
11	Wed, Oct 29	Self-Adjusting Search Trees	Handout 10 (hard-copy only)
12	Mon, Nov 3	Treaps: Randomized Search Trees	Handout 12 (hard-copy only)
13	Wed, Nov 5	Skip Lists	Handout 13 (hard-copy only)
14	Mon, Nov 10	Self-Adjusting and Fibonacci Heaps	Handout 14 (hard-copy only)
15	Wed, Nov 12	Hashing for Massive/Streaming Data	Handout 15 (ppt)
16 17	Mon, Nov 17 Wed, Nov 19	Synopses, Samples, and Sketches	Handout 16 (ppt)
18	Mon, Nov 25	Fingerprints, Min-Hashing and Document Similarity	Handout 18 (ppt)
19	Wed, Nov 26	Data Mining: Association Rules	Handout 19 (ppt)
20	Mon, Dec 1	Bloom Filters	Handout 20 (ppt)
21	Wed, Dec 3	Near Neighbors	Handout 21 (ppt)

Reading List

Text-books: There is no required text-book for this course but the following may be useful for some of the we plan to cover. At least the first three are recommended (but not required), the rest are useful reading if you are interested in delving further into the topics.

Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Introduction to Algorithms, T.H. Cormen, C.E.Leiserson, R.L. Rivest, and C. Stein, McGraw-Hill, 2002.
Managing Gigabytes: Compressing and Indexing Documents and Images, I.H. Witten, A. Moffat and T. C. Bell, Morgan Kauffman, 1999.
Modern Information Retrieval, Baeza-Yates and Ribeiro-Neto, Addison-Wesley, 1999.

Lectures 1 and 2 - Should tables be sorted?

Lecture Notes (Handout 2: ps, pdf)
Should tables be sorted? A. Yao. JACM 28 (1981).
Tables should be sorted (on random access machines), F. Fich and P.B. Miltersen. BRICS Report RS-95-26, 1995.

Lectures 3 and 4 - Hashing: Universal and Perfect

Lecture Notes (Handout 2: ps, pdf)
Lecture Notes (Handout 3: hard-copy only).
Chapter 8, Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Universal classes of hash functions. L. Carter and M. Wegman. STOC 1977. Full Version: Journal of Computer and System Sciences, 18 (1977).
Storing a Sparse Table with O(1) Worst Case Access Time. M.L. Fredman, J. Komlos, and E. Szemeredi, JACM 31 (1984).
Denial of Service via Algorithmic Complexity Attacks. S.A. Crosby and D.S. Wallach. Usenix Security 2003.
(A cool application of some of the hashing ideas we have been discussing.)

Lecture 5 - Amortization and List Update Problem

Lecture Notes (Handout 4: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Introduction to Algorithms, T.H. Cormen, C.E.Leiserson, R.L. Rivest, and C. Stein, McGraw-Hill, 2002.
Amortized Efficiency of List Update and Paging Rules. D. Sleator and R.E. Tarjan. CACM 28 (1985).

Lecture 6 - Disjoint Sets and Union-Find

Lecture Notes (Handout 6: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.
Introduction to Algorithms, T.H. Cormen, C.E.Leiserson, R.L. Rivest, and C. Stein, McGraw-Hill, 2002.

Lectures 7 and 8 - Competitive Analysis and Paging

Lecture Notes (Handout 7: hard-copy only).
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Amortized Efficiency of List Update and Paging Rules. D. Sleator and R.E. Tarjan. CACM 28 (1985).

Lectures 9 and 10 - Randomized Online Algorithms

Lecture Notes (Handout 9: hard-copy only).
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Competitive Paging Algorithms. A. Fiat, R.M. Karp, M. Luby, L.A. McGeoch, D. Sleator, and N. Young. Journal of Algorithms 12(4): 685-699 (1991).
Randomized Competitive Algorithms for the List Update Problem. N. Reingold, J. Westbrook, D. Sleator, Algorithmica 11(1): 15-32 (1994).

Lecture 11 - Self-Adjusting Search Trees

Lecture Notes (Handout 10: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.

Lecture 12 - Treaps: Randomized Search Trees

Lecture Notes (Handout 12: hard-copy only).
Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Randomized Search Trees. C. Aragon and R. Seidel. FOCS 1989.

Lecture 13 - Skip Lists

Lecture Notes (Handout 13: hard-copy only).
Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Skip Lists: A Probabilistic Alternative to Balanced Trees. W. Pugh. WADS 1990.

Lecture 14 - Self-Adjusting and Fibonacci Heaps

Lecture Notes (Handout 14: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.

Lecture 15 -- Hashing for Massive/Streaming Data

Lecture Slides (Handout 15: ppt)
Models and Issues in Data Stream Systems. Talk by Rajeev Motwani at SIGMOD/PODS 2002.
- Slides in PowerPoint
- Slides in pdf
Models and Issues in Data Stream Systems, B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom.
PODS 2002.
STREAM: STanford stREam datA Manager. Web page of the Stanford Stream Project (check out the relevant papers there).
Towards Estimation Error Guarantees for Distinct Values. M. Charikar, S. Chaudhuri, R. Motwani, and V. Narasayya. PODS 2000.
Probabilistic counting algorithms for data base applications. Flajolet and Martin. JCSS (1985).
The space complexity of approximating the frequency moments. N. Alon, Y. Matias, and M. Szegedy. STOC 1996.
Distinct Sampling for Highly-Accurate Answers to Distinct Value Queries and Event Reports. P. Gibbons. VLDB 2001.

Lecture 16 and 17 -- Synopses, Samples, and Sketches

Lecture Slides (Handout 16: ppt)
Synopses
- Synopsis data structures for massive data sets. Gibbons and Matias, DIMACS 1999.
- Tracking Join and Self-Join Sizes in Limited Storage. Alon, Gibbons, Matias and Szegedy. PODS 1999.
- Join Synopses for Approximate Query Answering, Acharya, Gibbons, Poosala, and Ramaswamy. SIGMOD 1999.
- Random Sampling for Histogram Construction: How much is enough? Chaudhuri, Motwani, and Narasayya. SIGMOD 1998.
- Random Sampling Techniques for Space Efficient Online Computation of Order Statistics of Large Datasets, Manku, Rajagopalan, and Lindsay. SIGMOD 1999.
- Space-efficient online computation of quantile summaries, Greenwald and Khanna. SIGMOD 2001.
Samples
- qRandom Sampling with a Reservoir, Vitter. Transactions on Mathematical Software 11(1):37-57 (1985).
- q
- qOn Sampling and Relational Operators. Chaudhuri and Motwani. Bulletin of the Technical Committee on Data Engineering (1999).
- qOn Random Sampling over Joins. Chaudhuri, Motwani, and Narasayya. SIGMOD 1999.
- qCongressional Samples for Approximate Answering of Group-By Queries, Acharya, Gibbons, and Poosala. SIGMOD 2000.
- qOvercoming Limitations of Sampling for Aggregation Queries, Chaudhuri, Das, Datar, Motwani and Narasayya. ICDE 2001.
- qA Robust Optimization-Based Approach for Approximate Answering of Aggregate Queries, Chaudhuri, Das and Narasayya. SIGMOD 01.
- qSampling From a Moving Window Over Streaming Data. Babcock, Datar, and Motwani. SODA 2002.
Sketches
- Probabilistic counting algorithms for data base applications. Flajolet and Martin. JCSS (1985).
- The space complexity of approximating the frequency moments. Alon, Matias, and Szegedy. STOC 1996.
- Approximate Frequency Counts over Streaming Data. Manku and Motwani. VLDB 2002.
- Finding Frequent Items in Data Streams. Charikar, Chen, and Farach-Colton. ICALP 2002.
- An Approximate L1-Difference Algorithm for Massive Data Streams, Feigenbaum, Kannan, Strauss, and Viswanathan. FOCS 1999.
- Stable Distributions, Pseudorandom Generators, Embeddings and Data Stream Computation, Indyk. FOCS 2000.

Lecture 18 -- Fingerprints, Min-Hashing, and Document Similarity

Lecture Slides (Handout 18: ppt)
Fingerprinting by random polynomials. M. Rabin. Technical Report TR-15-81, Harvard University (1981).
qSome applications of Rabin's fingerprinting method. A. Broder. Sequence II (1993).
qOn the Resemblance and Containment of Documents, A. Broder. SEQUENCES 1997.
Syntactic Clustering of the Web, A. Broder, S. Glassman, M. Manasse, and G. Zweig, WWW 1997.
Computing iceberg queries efficiently, M. Fang, N. Shivakumar, H. Garcia-Molica, R. Motwani and J.D. Ullman. VLDB 1998.
Finding near-replicas of documents on the web. N. Shivakumar and H. Garcia-Molina. WebDB 1998.
qIdentifying and Filtering Near-Duplicate Documents, Andrei Broder. CPM 2000.
qApproximate String Matching with q-grams and Maximal Matches. E. Ukkonen. Theoretical Computer Science (1992).
qCompleteness and Robustness Properties of Min-Wise Independent Permutations. A. Broder and M. Mitzenmacher.
qMin-Wise Independent Permutations, A. Broder, M. Charikar, A. Frieze and M. Mitzenmacher, JCSS (2000).
qA Small Approximately min-wise Independent Family of Hash Functions. P. Indyk. SODA 1999.
qApproximate Nearest Neighbors: Towards Removing the Curse of Dimensionality, P. Indyk and R. Motwani. STOC 1998.
qSimilarity Search in High Dimensions via Hashing, A. Gionis, P. Indyk, and R. Motwani. VLDB 1999.
qSimilarity Estimation Techniques from Rounding Algorithms, M. Charikar, STOC 2002.

Lecture 19 -- Data Mining: Association Rules

Lecture Slides (Handout 19: ppt)
Mining Associations between Sets of Items in Massive Databases, R. Agrawal, T. Imielinski, and A. Swami. SIGMOD 1993.
Fast Algorithms for Mining Association Rules, R. Agrawal and R. Srikant. VLDB 1994.
An Effective Hash-Based Algorithm for Mining Association Rules, J. S. Park, M.-S. Chen, and P. S. Yu. SIGMOD 1995.
An Efficient Algorithm for Mining Association Rules in Large Databases , A. Savasere, E. Omiecinski, and S. Navathe. The VLDB Journal 1995.
Sampling Large Databases for Association Rules, H. Toivonen. VLDB 1996.
Dynamic Itemset Counting and Implication Rules for Market Basket Data, S. Brin, R. Motwani, S. Tsur, and J.D. Ullman. SIGMOD 1997.
Computing iceberg queries efficiently, M. Fang, N. Shivakumar, H. Garcia-Molica, R. Motwani and J.D. Ullman. VLDB 1998.
Query Flocks: A Generalization of Association-Rule Mining, D. Tsur, J.D. Ullman, S. Abiteboul, C. Clifton, R. Motwani, S. Nestorov and A. Rosenthal. SIGMOD 1998.
Finding Interesting Associations without Support Pruning, E. Cohen, M. Datar, S. Fujiwara, A. Gionis, P. Indyk, R. Motwani, J.D. Ullman, and C. Yang. ICDE 2000.
Dynamic Miss-Counting Algorithms: Finding Implication and Similarity Rules with Confidence Pruning, S. Fujiwara, R. Motwani, and J.D. Ullman. ICDE 2000.

Lecture 20 -- Bloom Filters

Space/time tradeoffs in hash coding with allowable errors. B. Bloom. CACM 13 (1970).
Computing iceberg queries efficiently, M. Fang, N. Shivakumar, H. Garcia-Molica, R. Motwani and J.D. Ullman. VLDB 1998.
Summary cache: a scalable wide-area web cache sharing protocol. L. Fan, P. Cao, J. Almeida, and A. Broder. IEEE/ACM Transactions on Networking 2000.
Network Applications of Bloom Filters: A Survey. A. Broder and M. Mitzenmacher. Allerton Conference 2002.
Spectral Bloom Filters. S. Cohen and Y. Matias. SIGMOD 2003.
The Bloomier Filter: An Efficient Data Structure for Static Support Lookup Tables. B. Chazelle, J. Kilian, R. Rubinfeld, and A. Tal. SODA 2004.

Lecture 21 -- Near Neighbors

Nearest Neighbors

Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality, P. Indyk and R. Motwani. STOC 1998.
A Replacement for Voronoi Diagrams of Near Linear Size, S. Har-Peled. FOCS 2001.
Similarity Search in High Dimensions via Hashing, A. Gionis, P. Indyk, and R. Motwani. VLDB 1999.

Random Projections

Extensions of Lipschitz maps into a Hilbert space, W. Johnson and J. Lindenstrauss. Contemp. Math., 26 (1984), 189-206.
The Johnson-Lindenstrauss lemma and the sphericity of some graphs, P. Frankl and H. Maehara. Journal of Combinatorial Theory Series B, 44:355-362, 1988.
An elementary proof of the Johnson Lindenstrauss lemma, S. Dasgupta and Anupam Gupta. ICSI TR-99-006.
Fast Monte-Carlo Algorithms for Finding Low-Rank Approximations, A. Frieze, R. Kannan, and S. Vempala. FOCS 1998.
Database-friendly Random Projections, D. Achlioptas, PODS 2001.

Similarity Search in High Dimensions via Hashing, A. Gionis, P. Indyk, and R. Motwani. VLDB 1999.