CS 361A - Autumn Quarter 2005-06
(Advanced Data Structures and Algorithms)

News Flash Administrivia Signup Overview Handouts/Homeworks Lecture Schedule Readings

News Flash

Please note the change in office hours for Dilys Thomas.

Administrivia
        Instructor:   Rajeev Motwani
        Teaching Assistant: Dilys Thomas (dilys@stanford.edu)
        Class Schedule: Mon/Wed, 3:15-4:30, Gates B08

        Office Hours:
                                Dilys Thomas              Tue/Thu 2-4pm (Location: Gates 482, Phone: 723-4532)
                                Rajeev Motwani          Mon/Wed, 4:30pm (right after the lectures)

Course URL: http://theory.stanford.edu/~rajeev/cs361.html

Class Sign-up To sign up for this course, please send email to Dilys Thomas with the following information: name, department, status (Phd/MS/UG, year), area (Databases, Systems, Theory, etc), and email address. Please also include registration status (credit, pass/fail/audit).

Mailing Lists and Newsgroup We have set up a class mailing list to help you get the latest information regarding the class. The email lists are auto-populated using current course enrolment information. The main list will be cs361a-aut0506-all@lists.stanford.edu. Those who audit the course can subscribe by sending an email to majordomo@lists.stanford.edu with the following text in the body of the mail: subscribe cs361a-aut0506-guests. Those who are auditing the course and have filled the signup sheet in class on September 28 have already been added onto the guest list. We also have a newsgroup su.class.cs361a for the class.

Grading Since this course will be treated as a graduate research seminar, we expect that most students will register pass/fail (and not for a letter grade). If you do choose to sign up for a letter grade, be sure to mention this in your sign-up email to the TA. We will give out 3-4 homeworks, one of which will serve as a take-home midterm exam. There will no final exam. The scores on these homeworks as well as class participation will determine your final grade.

Course Overview

Efficient strategies for complex data-structuring problems are essential in the design of fast algorithms for a variety of applications, including combinatorial optimization, databases and data mining, information retrieval and web search, and geometric applications. We will give a systematic exposition of the central ideas in the design of such data structures. The second main theme of this course will be the design and analysis of online algorithms and data stream algorithms. The field of competitive analysis of online algorithms got its start in the amortized analysis for data structures and forms a natural extension of some of the ideas we will discuss in the earlier part of the course. We will present some of the main ideas and motivating applications for this class of algorithms. Time permitting, we will also cover some topics in the related area of algorithms and data structures in the stream model of computation. The material to be covered will be drawn from the following list:

Advanced Data Structures: hash tables (universal hashing, perfect hashing, locality-sensitive hashing, Bloom filters); data structures for combinatorial optimization (union-find, Fibonacci heaps, dynamic trees, dynamic graph structures); self-adjusting data structures (lists, splay trees); search trees (red-black trees, self-adjusting trees, treaps, skip lists, finger search trees, biased search trees); fault-tolerant and persistent data structures; suffix trees and string searching; databases/data-mining/data-stream (histograms, indexes, hashing, synopses and sketches, sliding windows); geometric and kinetic data structures.

Online and Data Stream Algorithms: paging/caching problems; abstractions (k-server problem, request-answer games, and metrical task systems); scheduling and load balancing; network algorithms; data migration/replication in distributed computing; stream algorithms and data structures for database problem.

This course should be of interest to graduate students in computer science and related fields, especially those with a mathematical bent of mind. We will assume familiarity with basic material in algorithms, combinatorics, and probability theory (at the level of the core undergraduate courses on these topics).

Handouts and Homeworks

Handout	Date	Topic	Download
1	Mon, Sep 26	Reading List	ps or pdf
2	Mon, Sep 26	Notes for Lectures 1 & 2	ps or pdf
3	Mon, Oct 3	Notes for Lectures 3 & 4	hard-copy only
4	Mon, Oct 10	Notes for Lecture 5	hard-copy only
5	Mon, Oct 10	Notes for Lecture 6	hard-copy only
6	Mon, Oct 10	Homework 1	ps or pdf
7	Wed, Oct 12	Notes for Lectures 7 & 8	hard-copy only
9	Wed, Oct 19	Notes for Lectures 9 & 10	hard-copy only
10	Mon, Oct 24	Solutions for Homework 1	hard-copy only
11	Mon, Oct 24	Homework 2	ps or pdf
12	Wed, Oct 26	Notes for Lecture 11	hard-copy only
13	Wed, Oct 26	Notes for Lecture 12	hard-copy only
14	Wed, Nov 2	Notes for Lecture 13	hard-copy only
15	Wed, Nov 2	Notes for Lecture 14	hard-copy only
16	Wed, Nov 9	Additional slides for Lecture 14	ppt
17	Wed, Nov 9	Notes for Lecture 15	ppt
18	Wed, Nov 9	Notes for Lecture 16/17	ppt
19	Mon, Nov 14	Solutions for Homework 2	hard-copy only
20	Mon, Nov 14	Homework 3	ps or pdf
21	Mon, Nov 28	Notes for Lecture 18	ppt
22	Wed, Nov 30	Notes for Lecture 19	ppt
23	Wed, Nov 30	Notes for Lecture 20	ppt
24	Wed, Nov 30	Homework 4	ps or pdf

Lecture Schedule

Lecture	Date	Topic	Lecture Notes
1 2	Mon, Sep 26 Wed, Sep 28	Should Tables be Sorted?	Handout 2 (ps, pdf)
3 4	Mon, Oct 3 Wed, Oct 5	Hashing: Universal and Perfect	Handout 2 (ps, pdf) Handout 3 (hard-copy only)
5	Mon, Oct 10	Amortization and List Update	Handout 4 (hard-copy only)
6	Wed, Oct 12	Disjoint Sets and Union-Find	Handout 5 (hard-copy only)
7 8	Mon, Oct 17 Wed, Oct 19	Competitive Analysis and Paging	Handout 7 (hard-copy only)
9 10	Mon, Oct 24 Wed, Oct 26	Randomized Online Algorithms	Handout 9 (hard-copy only)
11	Mon, Oct 31	Self-Adjusting Search Trees	Handout 12 (hard-copy only)
12	Wed, Nov 2	Treaps: Randomized Search Trees	Handout 13 (hard-copy only)
13	Mon, Nov 7	Skip Lists	Handout 14 (hard-copy only)
14	Wed, Nov 9	Caching Queues Self-Adjusting & Fibonacci Heaps	Handout 16 (ppt) Handout 15 (hard-copy only)
15	Mon, Nov 14	Hashing for Massive/Streaming Data	Handout 17 (ppt)
16 17	Wed, Nov 16 Mon, Nov 28	Synopses, Samples, and Sketches	Handout 18 (ppt)
18	Wed, Nov 30	Fingerprints, Min-Hashing and Document Similarity	Handout 21 (ppt)
19	Mon, Dec 5	Near Neighbors	Handout 22 (ppt)
20	Wed, Dec 7	Data Mining: Association Rules	Handout 23 (ppt)

Reading List

Text-books: There is no required text-book for this course but the following may be useful for some of the we plan to cover. At least the first three are recommended (but not required), the rest are useful reading if you are interested in delving further into the topics.

Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.

Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.

Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.

Introduction to Algorithms, T.H. Cormen, C.E.Leiserson, R.L. Rivest, and C. Stein, McGraw-Hill, 2002.

Managing Gigabytes: Compressing and Indexing Documents and Images, I.H. Witten, A. Moffat and T. C. Bell, Morgan Kauffman, 1999.

Modern Information Retrieval, Baeza-Yates and Ribeiro-Neto, Addison-Wesley, 1999.

Lectures 1 and 2 - Should tables be sorted?

Lecture Notes (Handout 2: ps, pdf)
Should tables be sorted? A. Yao. JACM 28 (1981).
Tables should be sorted (on random access machines), F. Fich and P.B. Miltersen. BRICS Report RS-95-26, 1995.

Lectures 3 and 4 - Hashing: Universal and Perfect

Lecture Notes (Handout 2: ps, pdf)
Lecture Notes (Handout 3: hard-copy only).
Chapter 8, Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Universal classes of hash functions. L. Carter and M. Wegman. STOC 1977. Full Version: Journal of Computer and System Sciences, 18 (1977).
Storing a Sparse Table with O(1) Worst Case Access Time. M.L. Fredman, J. Komlos, and E. Szemeredi, JACM 31 (1984).
Denial of Service via Algorithmic Complexity Attacks. S.A. Crosby and D.S. Wallach. Usenix Security 2003.
(A cool application of some of the hashing ideas we have been discussing.)

Lecture 5 - Amortization and List Update Problem

Lecture Notes (Handout 4: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Introduction to Algorithms, T.H. Cormen, C.E.Leiserson, R.L. Rivest, and C. Stein, McGraw-Hill, 2002.
Amortized Efficiency of List Update and Paging Rules. D. Sleator and R.E. Tarjan. CACM 28 (1985).

Lecture 6 - Disjoint Sets and Union-Find

Lecture Notes (Handout 6: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.
Introduction to Algorithms, T.H. Cormen, C.E.Leiserson, R.L. Rivest, and C. Stein, McGraw-Hill, 2002.

Lectures 7 and 8 - Competitive Analysis and Paging

Lecture Notes (Handout 7: hard-copy only).
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Amortized Efficiency of List Update and Paging Rules. D. Sleator and R.E. Tarjan. CACM 28 (1985).

Lectures 9 and 10 - Randomized Online Algorithms

Lecture Notes (Handout 9: hard-copy only).
Online Computation and Competitive Analysis, A. Borodin and R. El-Yaniv, Combridge University Press, 1998.
Competitive Paging Algorithms. A. Fiat, R.M. Karp, M. Luby, L.A. McGeoch, D. Sleator, and N. Young. Journal of Algorithms 12(4): 685-699 (1991).
Randomized Competitive Algorithms for the List Update Problem. N. Reingold, J. Westbrook, D. Sleator, Algorithmica 11(1): 15-32 (1994).

Lecture 11 - Self-Adjusting Search Trees

Lecture Notes (Handout 10: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.

Lecture 12 - Treaps: Randomized Search Trees

Lecture Notes (Handout 12: hard-copy only).
Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Randomized Search Trees. C. Aragon and R. Seidel. FOCS 1989.

Lecture 13 - Skip Lists

Lecture Notes (Handout 13: hard-copy only).
Randomized Algorithms, R. Motwani and P. Raghavan, Cambridge University Press, 1995.
Skip Lists: A Probabilistic Alternative to Balanced Trees. W. Pugh. WADS 1990.

Lecture 14 (part 1) - Caching Queues

Caching Queues in Memory Buffers, R. Motwani and D. Thomas, SODA 2004.

Lecture 14 (part 2) - Self-Adjusting and Fibonacci Heaps

Lecture Notes (Handout 14: hard-copy only).
Data Structures and Network Algorithms, R.E. Tarjan, SIAM 1983.

Lecture 15 -- Hashing for Massive/Streaming Data

Lecture Slides (Handout 15: ppt)
Models and Issues in Data Stream Systems. Talk by Rajeev Motwani at SIGMOD/PODS 2002.

Models and Issues in Data Stream Systems, B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom.
PODS 2002.
STREAM: STanford stREam datA Manager. Web page of the Stanford Stream Project (check out the relevant papers there).
Towards Estimation Error Guarantees for Distinct Values. M. Charikar, S. Chaudhuri, R. Motwani, and V. Narasayya. PODS 2000.
Probabilistic counting algorithms for data base applications. Flajolet and Martin. JCSS (1985).
The space complexity of approximating the frequency moments. N. Alon, Y. Matias, and M. Szegedy. STOC 1996.
Distinct Sampling for Highly-Accurate Answers to Distinct Value Queries and Event Reports. P. Gibbons. VLDB 2001.

Lecture 16 and 17 -- Synopses, Samples, and Sketches

Lecture Slides (Handout 18: ppt)

Synopses

Synopsis data structures for massive data sets. Gibbons and Matias, DIMACS 1999.
Tracking Join and Self-Join Sizes in Limited Storage. Alon, Gibbons, Matias and Szegedy. PODS 1999.
Join Synopses for Approximate Query Answering, Acharya, Gibbons, Poosala, and Ramaswamy. SIGMOD 1999.
Random Sampling for Histogram Construction: How much is enough? Chaudhuri, Motwani, and Narasayya. SIGMOD 1998.
Random Sampling Techniques for Space Efficient Online Computation of Order Statistics of Large Datasets, Manku, Rajagopalan, and Lindsay. SIGMOD 1999.
Space-efficient online computation of quantile summaries, Greenwald and Khanna. SIGMOD 2001.

Samples

qRandom Sampling with a Reservoir, Vitter. Transactions on Mathematical Software 11(1):37-57 (1985).
q
qOn Sampling and Relational Operators. Chaudhuri and Motwani. Bulletin of the Technical Committee on Data Engineering (1999).
qOn Random Sampling over Joins. Chaudhuri, Motwani, and Narasayya. SIGMOD 1999.
qCongressional Samples for Approximate Answering of Group-By Queries, Acharya, Gibbons, and Poosala. SIGMOD 2000.
qOvercoming Limitations of Sampling for Aggregation Queries, Chaudhuri, Das, Datar, Motwani and Narasayya. ICDE 2001.
qA Robust Optimization-Based Approach for Approximate Answering of Aggregate Queries, Chaudhuri, Das and Narasayya. SIGMOD 01.
qSampling From a Moving Window Over Streaming Data. Babcock, Datar, and Motwani. SODA 2002.

Sketches

Probabilistic counting algorithms for data base applications. Flajolet and Martin. JCSS (1985).
The space complexity of approximating the frequency moments. Alon, Matias, and Szegedy. STOC 1996.
Approximate Frequency Counts over Streaming Data. Manku and Motwani. VLDB 2002.
Finding Frequent Items in Data Streams. Charikar, Chen, and Farach-Colton. ICALP 2002.
An Approximate L1-Difference Algorithm for Massive Data Streams, Feigenbaum, Kannan, Strauss, and Viswanathan. FOCS 1999.
Stable Distributions, Pseudorandom Generators, Embeddings and Data Stream Computation, Indyk. FOCS 2000.

Lecture 18 -- Fingerprints, Min-Hashing, and Document Similarity

Lecture Slides (Handout 21: ppt)

Fingerprinting by random polynomials. M. Rabin. Technical Report TR-15-81, Harvard University (1981).

qSome applications of Rabin's fingerprinting method. A. Broder. Sequence II (1993).

qOn the Resemblance and Containment of Documents, A. Broder. SEQUENCES 1997.

Syntactic Clustering of the Web, A. Broder, S. Glassman, M. Manasse, and G. Zweig, WWW 1997.

Computing iceberg queries efficiently, M. Fang, N. Shivakumar, H. Garcia-Molica, R. Motwani and J.D. Ullman. VLDB 1998.

Finding near-replicas of documents on the web. N. Shivakumar and H. Garcia-Molina. WebDB 1998.

qIdentifying and Filtering Near-Duplicate Documents, Andrei Broder. CPM 2000.

qApproximate String Matching with q-grams and Maximal Matches. E. Ukkonen. Theoretical Computer Science (1992).

qCompleteness and Robustness Properties of Min-Wise Independent Permutations. A. Broder and M. Mitzenmacher.

qMin-Wise Independent Permutations, A. Broder, M. Charikar, A. Frieze and M. Mitzenmacher, JCSS (2000).

qA Small Approximately min-wise Independent Family of Hash Functions. P. Indyk. SODA 1999.

qApproximate Nearest Neighbors: Towards Removing the Curse of Dimensionality, P. Indyk and R. Motwani. STOC 1998.

qSimilarity Search in High Dimensions via Hashing, A. Gionis, P. Indyk, and R. Motwani. VLDB 1999.

qSimilarity Estimation Techniques from Rounding Algorithms, M. Charikar, STOC 2002.

Lecture 19 -- Near Neighbors

Nearest Neighbors

Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality, P. Indyk and R. Motwani. STOC 1998.

A Replacement for Voronoi Diagrams of Near Linear Size, S. Har-Peled. FOCS 2001.

Similarity Search in High Dimensions via Hashing, A. Gionis, P. Indyk, and R. Motwani. VLDB 1999.

Random Projections

Extensions of Lipschitz maps into a Hilbert space, W. Johnson and J. Lindenstrauss. Contemp. Math., 26 (1984), 189-206.

The Johnson-Lindenstrauss lemma and the sphericity of some graphs, P. Frankl and H. Maehara. Journal of Combinatorial Theory Series B, 44:355-362, 1988.

An elementary proof of the Johnson Lindenstrauss lemma, S. Dasgupta and Anupam Gupta. ICSI TR-99-006.

Fast Monte-Carlo Algorithms for Finding Low-Rank Approximations, A. Frieze, R. Kannan, and S. Vempala. FOCS 1998.

Database-friendly Random Projections, D. Achlioptas, PODS 2001.

Lecture 20 -- Data Mining: Association Rules

Mining Associations between Sets of Items in Massive Databases, R. Agrawal, T. Imielinski, and A. Swami. SIGMOD 1993.

Fast Algorithms for Mining Association Rules, R. Agrawal and R. Srikant. VLDB 1994.

An Effective Hash-Based Algorithm for Mining Association Rules, J. S. Park, M.-S. Chen, and P. S. Yu. SIGMOD 1995.

An Efficient Algorithm for Mining Association Rules in Large Databases , A. Savasere, E. Omiecinski, and S. Navathe. The VLDB Journal 1995.

Sampling Large Databases for Association Rules, H. Toivonen. VLDB 1996.

Dynamic Itemset Counting and Implication Rules for Market Basket Data, S. Brin, R. Motwani, S. Tsur, and J.D. Ullman. SIGMOD 1997.

Computing iceberg queries efficiently, M. Fang, N. Shivakumar, H. Garcia-Molica, R. Motwani and J.D. Ullman. VLDB 1998.

Query Flocks: A Generalization of Association-Rule Mining, D. Tsur, J.D. Ullman, S. Abiteboul, C. Clifton, R. Motwani, S. Nestorov and A. Rosenthal. SIGMOD 1998.

Finding Interesting Associations without Support Pruning, E. Cohen, M. Datar, S. Fujiwara, A. Gionis, P. Indyk, R. Motwani, J.D. Ullman, and C. Yang. ICDE 2000.

Dynamic Miss-Counting Algorithms: Finding Implication and Similarity Rules with Confidence Pruning, S. Fujiwara, R. Motwani, and J.D. Ullman. ICDE 2000.