CS265/CME309: Randomized Algorithms and Probabilistic Analysis, Fall 2019
Announcements:
12/4: Final lecture slides: Today's lecture will be on some research of my students and me which leverages some of the tools that we've seen during class.
12/2: Today's lecture will be on "Expander Codes". We'll spend the first half discussing expander graphs (sparse graphs that behave like dense ones in the sense that they are well-connected), and the second half discussing how to use expander graphs to make good error-correcting codes.
11/16: Lecture 17 notes: The Martingale Stopping Theorem (aka the Optional Stopping Theorem). [This material will be covered monday, and the last question on ps8 will require this material---just a direct application of Theorem 1---so I wanted to post the notes early.]
11/13: Lecture 16 notes: Martingales, the Doob Martingale, and Azuma-Hoeffding Tail Bounds.
11/12: Midterm exam from 2017: This was very hard, (and I used some of these problems on this year's psets) but grades were curved generously. Here are solutions: 2017 solutions.
11/12: I'll hold a special midterm/final prep office hour Tuesday (11/19), 5-7pm (in my office or in the room across the hall from my office). I'll be happy to go through the practice midterms, or talk about any of the concepts we've covered. I will *NOT* discuss problems from Problem Set 8 at this office hour (unless/until everyone's midterm/final questions have already been answered).
11/11: Last year's midterm exam: This was quite a hard exam, and I really liked question 3...maybe a bit too much. The material on last year's exam did not include Markov chains and Martingales, which your exam will cover. [Last year they also had a take-home final...]
11/11: Lecture 15 notes: Mixing Times, Strong Stationary Times, and Coupling.
11/11: The eighth and final problem set is here: Problem Set 8. It is due at 11:59pm on Tuesday, 11/19. Solution template for ps8.
11/10: We will be having the in-class "final" exam on Wednesday, November 20th. The exam will during our normal class time, and we will likely split between our current classroom and the one across the breezeway (waiting to confirm that). The exam will be closed books/laptop, though you will be allowed to bring one double-sided page of notes, that you must have prepared yourself. I will post some midterm exams from previous years, though those exams only cover the material through October.
11/10: For the next few weeks, Shivam will be holding the remote student office hours Tuesdays, 10-10:30am.
11/6: Lecture 14 notes: The Fundamental Theorem of Markov Chains, and stationary distributions.
11/5: Lecture 13 notes: Introduction to Markov Chains and a randomized 2-SAT algorithm.
11/5: The seventh problem set is here: Problem Set 7. It is due at 11:59pm on Tuesday, 11/12. Solution template for ps7.
10/30: Lecture 12 notes: Moser's ``Entropic Proof'' of the Constructive/Algorithmic Lovasz Local Lemma.
10/30: Selected solutions for problem set 5.
10/28: The sixth problem set is here: Problem Set 6. It is due at 11:59pm on Tuesday, 11/5. Solution template for ps6.
10/28: Lecture 11 notes: The Second-Moment Method, and Lovasz Local Lemma and applications.
10/26: Shashwat's Monday office hours are being shifted earlier, from 9am-10:30am in Huang Basement, and Shivam will now be having Monday office hours from 10:30-11:30 also in Huang Basement.
10/26: I'll be out of town Monday (10/28). We'll still have lecture (Vatsal has kindly offered to give another guest lecture), though I won't be having my usual Monday 4-5:30pm office hours. I'll still be active on Piazza.
10/24: I updated the lecture notes, though they are a little more rough then usual (I'm still feeling pretty sick, and will tidy them up when I'm feeling a bit better).
10/23: Lecture 10 notes: Introduction to the Probabilistic Method, Ramsey Numbers, and de-randomization via conditional probabilities.
10/23: Selected solutions for problem set 4.
10/21: For the off-campus/remote students, Shivam will hold a special dial-in office hours (its been hard juggling the skype/hangouts students with those in person during the OH). If you are a remote student who would like to join, please email me with "CS265: REMOTE STUDENT" in the email title, and we will ping you to figure out a convenient time to hold this office hour.
10/21: The fifth problem set is here: Problem Set 5. It is due at 11:59pm on Tuesday, 10/29. Solution template for ps5.
10/21: Lecture 9 notes: Johnson-Lindenstrauss dimension reduction, and introduction to nearest neighbor search.
10/16: Selected solutions for problem set 3.: In general, I will post solutions/solution sketches for the one or two questions from the pset that students have the most questions on....
10/16: Lecture 8 notes: metric embeddings, and Bourgain's embedding of any finite metric into R^d under L1.
10/14: The fourth problem set is here: Problem Set 4. It is due at 11:59pm on Tuesday, 10/22. Solution template for ps4.
10/14: Lecture 7 notes are now posted.
10/13: My 4-5:30pm Monday office hours will be cancelled this week (I'll be out of town Mon/Tues).
10/9: Lecture 6 notes are now posted.
10/7: The third problem set is here: Problem Set 3. It is due at 11:59pm on Tuesday, 10/15. Feel free to use this solution template for ps3.
10/7: Lecture 5 notes are now posted.
10/7: Starting this week, Ray's office hours will be in Gates B26.
10/1: Lecture notes for this week have been updated to include the "fingerprinting" application we discussed at the end of class.
9/30: The second homework is here: Problem Set 2. It is due at 11:59pm on Tuesday, 10/8. Feel free to use this solution template for ps2.
9/30: Lecture notes for this week: Lecture 3 and 4 Notes (combined). [These will be updated after Wednesday's class to include Lecture 4 material.]
9/27: For those of you who wanted the latex of the problem set, or wanted the problem statements in your solution, feel free to use this solution template for ps1.
9/26: I'll be giving an open lecture today in the ISL colloquium (4:30pm in Packard 101), which might be of interest to some of you.
9/25: Lecture notes for today's class are Lecture 2 Notes.
9/25: The first homework is here: Problem Set 1. It is due at 11:59pm on Tuesday. In future, homeworks will be posted Monday night, and will be due 8 days later.
9/25: Beginning Monday, we will be moving across the breezeway to our new classroom, Thornton 110!!
9/25: For those of you who tried to enroll but were not able to, the enrollment cap has now been removed (so try again).
9/23: First day of class! Lecture notes for today's class are Lecture 1 Notes.
8/23: We have a Piazza page for class discussions HERE . This is for your benefit, though you do not need to join it. The TAs and I will do our best to monitor it and will try to answer any questions asked on the forum, though please refer to this course website for all announcements/corrections, etc. Also, please feel free to discuss/answer questions that your classmates post.
Instructor: Gregory Valiant (Office hours: Monday 4-5:30pm, Gates 470. Contact info: send email to my last name at stanford dot edu)
Teaching Assistants:
Ray Li (rayyli@stanford), Shashwat Silas (silas@stanford), Shivam Garg (shivam13@stanford)
Office Hours:
Greg: Monday 4-5:30pm, Gates 470
Ray: Friday 10:30-noon, Gates B26
Shashwat: Monday 9:00-10:30am, Huang Basement
Shivam: Monday 10:30-11:30, Huang Basement, and remote student dial-in Tuesdays, 10-10:30am
Class Time/location: 1:30-2:50 Mon/Wed. Location: Thornton Center, 110 (starting Monday, 9/30).
Course description:
Randomness pervades the natural processes around us, from the formation of networks, to genetic recombination, to quantum physics. Randomness is also a powerful tool that can be leveraged to create algorithms and data structures which, in many cases, are more efficient and simpler than their deterministic counterparts. This course covers the key tools of probabilistic analysis, and applications of these tools to understand the behaviors of random processes and algorithms. Emphasis is on theoretical foundations, though we will apply this theory broadly, discussing applications in machine learning and data analysis, networking, and systems.
Topics: Markov and Chebyshev inequalities, Chernoff bounds, random graphs, and expanders, moment generating functions, metric embeddings, the probabilistic method, Lovasz Local Lemma, Markov chains and random walks, MCMC, martingales, stopping times, Azuma-Hoeffding inequality, and many powerful and elegant randomized algorithms whose analyses rely on the above tools.
Prerequisites: CS 161 and STAT 116, or equivalents and instructor consent.
Grading: 65% problem sets, 35% in-class exam.
Problem Set Policies: Late problem sets will not be accepted, though your TWO lowest problem set scores will be dropped. We strongly encourage using LaTex to typeset your problem sets, and have a LaTex template that you may use. We will be using the Gradescope online submission system. You should receive an email saying that you've been enrolled in CS265/CME309 from Gradescope. If not, create an account on Gradescope using your Stanford ID and join CS265/CME309 using entry code 92EK55.
You are encouraged to find a partner, and hand in 1 problem set with both names. As always, collaboration and discussion of the problems is encouraged, though you must understand everything that you hand in, and the writeup that your partner and you hand in must be your own group's writing.
Schedule:
Week 1 (9/23, 9/25): Introduction, models of computation, randomized polynomial identity testing algorithm (Schwartz-Zippel), Karger's randomized min-cut algorithm.
Lecture notes Lecture 1. Models of randomized computation, and the Schwartz-Zippel randomized polynomial identity test.
Lecture notes Lecture 2. Karger's randomized min-cut algorithm, and analysis of Quicksort with random pivots.
Week 2 (9/30, 10/2): Randomized Primality Testing. This is what originally put randomized algorithms on the map back in the late 1970's, and modern cryptography as we know it would not be possible without the existence of fast randomized primality checking algorithms. We also discussed some non-crypto applications of the ability to efficiently choose random prime numbers, including succinct "fingerprints".
Lecture notes for this week: Lecture 3 and 4 Notes (combined). [These will be updated after Wednesday's class to include Lecture 4 material.]
Week 3 (10/7, 10/9): Markov's inequality, Chebyshev's inequality, moment generating functions and Chernoff bounds. Randomized routing.
Lecture notes Lecture 5. Markov's and Chebyshev's inequalities, and sampling-based median algorithm.
Lecture notes Lecture 6. Moment-generating functions, Chernoff bounds, and randomized routing on the hypercube.
Week 4 (10/14, 10/16): Balls in bins, the "Poissonization" technique, "power of two choices". Metric embeddings.
Lecture notes Lecture 7. Balls in bins, the "Poissonization" technique, "power of two choices".
Lecture notes Lecture 8. Introduction to Metric Embeddings, and Bourgain's low-distortion randomized embedding of any finite metric into L1.
Week 5 (10/21, 10/23): Metric embeddings continued, and one lecture on the "probabilistic method".
Lecture notes Lecture 9: Johnson-Lindenstrauss dimension reduction, and introduction to nearest neighbor search.
Lecture notes Lecture 10. The Probabilistic Method: bounding Ramsey numbers, derandomization via conditional expectations.
Week 6 (10/28, 10/30):Second-Moment Method and Lovasz Local Lemma (and algorithmic version), with applications.
Lecture notes Lecture 11: Probabilistic Method Continued: Second-moment method and the Lovasz Local Lemma (leveraging local dependencies to avoid the union bound)!
Lecture notes Lecture 12: Moser's ``Entropic Proof'' of the Constructive/Algorithmic Lovasz Local Lemma.
Week 7 (11/4, 11/6):Intro to Markov Chains, Mixing Times.
Lecture notes Lecture 13: Introduction to Markov chains, and a randomized algorithm for 2-SAT.
Lecture notes Lecture 14: The Fundamental Theorem of Markov Chains, and stationary distributions.
Week 8 (11/11, 11/13):Strong Stationary Times, and Coupling. Martingales, stopping times.
Lecture notes Lecture 15: Mixing Times, Strong Stationary Times, and Coupling
Lecture notes Lecture 16: Martingales, the Doob martingale, and Azuma-Hoeffding tail bounds.
Week 9 (11/18):The Martingale Stopping Theorem, and applications.
Lecture notes Lecture 17: The Martingale Stopping Theorem (aka the Optional Stopping Theorem), and applications.
11/20: In-class Exam (in place of formal final exam)
Week 10 (12/2, 12/4):Expander codes, and final lecture on current research threads.
Lecture 18: Expander Codes. We'll spend the first half discussing expander graphs (sparse graphs that behave like dense ones in the sense that they are well-connected), and the second half discussing how to use expander graphs to make good error-correcting codes.
Lecture 19: This final lecture talked about some recent research of my students and me that builds on some of the things we saw in class. Slides here.