Patricia Hoffman PhD

Essay on how to start solving a machine learning problem (click here)

Research Papers:

Recommender Systems:

Collaborative Filtering with Temporal Dynamics: Yehuda Koren's paper, describes recommendation algorithm incorporating time dependent infothe paper  

Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model: Yehuda Koren's paper describes how to add the implicit information (which movies were rented) to the model based on explicit data of movies which were rated.

Koren's paper describes in part his algorithm for Netfix's contest, .  It is very interesting algorithm and all comments on the paper are welcome. A news article describing the team is here:

The BellKor Solution to the Netflix Grand Prize by Yehuda Koren: this paper describes Koren's teams contribution to the "Bell-Kor's Pragmatic Chaos" final solution, which won the Netflix Grand Prize.  

The Ensemble Team wrote Feature-Weighted Linear Stacking    by Joseph Sill, Gabor Takacs, Lester Mackey, David Lin

Another contestant also wrote papers:  Chris Volinsky's Publications   including Collaborative Filtering for Implicit Feedback Datasets  paper

Recommender Problems for Web Applications 

Jure Leskovec CS246 Mining Massive Data Sets: Recommender Systems Lecture

Support Vector Machines: 

Support Vector Machines by Patricia Hoffman, Ph.D. DRAFT
Fast Training of SVM Using Sequential Minimal Optimization
John C. Platt      Paper  
Statistical Learning Theory by Vladimir Vapnik, Wiley-
Interscience; 1998
Burges Tutorial:   q87856173126771q.pdf
Applet to visualize SVM  using various Kernels (It takes a bit of time for it to initialize)
Kernel Machines Papers, Software, and Tutorials

SVD for Large Data Sets:

Finding Structure with Randomness:  Stochastic Algorithms for Constructing Approximate Matrix Decompositions by N. Halko, P.G. Martinsson, and J.A. Tropp  Paper
Python Code (Mike Bowles PhD)  

Decision Trees:

Multivariate Adaptive Regression  by Jerome H. Friedman
MART Tutorial

John Herbach Lecture on PLANET

Convex Optimization:

Convex Optimization by Stephen Boyd and Lieven Vanderberghe
Matlab Code
by Michael Grant and Stephen Boyd
Python Code by Vandenberghe
Andrew Ng's Convex Optimization One & Two (Lagrange Duality)


From Trees to Forests and Rule Sets - A Unified Overview of Ensemble Methods video                                                                                              by Giovanni Seni, PhD Santa Clara University

Elder Research Web Page

Lectures and Labs from Samy Bengio PhD


Beginning Data Mining Tutorial

Tutorials on Collaborative Filtering include: 1) Programming Collective Intelligence: Building Smart Web 2.0 Applications by Toby Segaran (Second Chapter).   Koren did a fantastic job of adding time dependence to the algorithms described by Segaran.  2) Yehuda Koren, Robert Bell, and Chris Volinsky wrote an article titled, "Matrix Factorization Techniques for Recommender Systems" for IEEE Aug. 2009

Tutorial on Support Vector Machines for Pattern Recognition Paper

Google Talks

Data Mining Tutorials  Basic Math

Andrew Ng's Review notes: Linear Algebra, Probability,  Hidden Markov Models

Great Linear Algebra Text: "Introduction to Linear Algebra" by Gilbert Strang. His lectures are also available online at: 

or at:

 Course materials can be found at: 

Computational Science lectures can be found at: 

For Matrix Computations Golub and Van Loan

For distributed Trefethen's Matrix Computations


Video Course at Stanford Lecture by Professor Andrew Ng for Machine Learning (uses MatLab/Octave) (CS 229) All Lectures      Web Page Course Info         Another course

Machine Learning Course based on Hastie, Tibshirani, & Friedman's text Web Page First Semester Second Semester 

                 Papers and software by Jerome Friedman

Machine Learning Course based on Bishop's Pattern Recognition and Machine Learning text  Web Page

Machine Learning Course at CMU Web Page

Association for Computing Machinery Data Mining Special Interest Group Web Page

Statistical Aspects of Data Mining by Professor Rajan Patel (Stanford Statistics 202)  Web Page Prof David Mease Video Lecture

Advanced Machine Learning (Focus: Learning with matrix parameters) Professor Manfred K. Warmuth (UCSC) Web Page

Machine Learning & Data Mining UCSC CMPS 142 Course Web Page

MIT 6.867 Machine Learning  Web Page Open Course Lecture Notes

Generalized Linear Models UCSC AMS 274 Course  Course information GLMs  Notes on GLM definitions & maximum likelihood estimation

Stanford MS&E 237 The Social Data Revolution: Data Mining and Electronic Business Wiki

For the Beginner: Linear Algebra at MIT by Prof. Gilbert Strang

Mining Massive Data Sets Text by Jeff Ullman  Lectures by Jure Leskovec

MIT courses in General

Stanford Courses in General

Recommended Texts:

Free Texts

Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman (ISBN 0387952845)

Pattern Recognition and Machine Learning by  Springer, Christopher Bishop, 2006.
Pattern Classification,  Richard Duda, Peter Hart and David Stork, 2nd ed. John Wiley & Sons, 2001.
Machine Learning.  Tom Mitchell,  McGraw-Hill, 1997.
Reinforcement Learning: An introduction, Richard Sutton and Andrew Barto  MIT Press, 1998

For the beginner

Introduction to Data Mining, Pang-Ning Tan, Michael Steinbach, Vipin Kumar, Pearson, Addison Wesley

       Check out my class at the  Hacker Dojo Dr. Mike Bowles & Dr. Patricia Hoffman

       Pang-Ning Tan Web Page

        Course Web Pages: David Mease 

       Sample Chapters:

Introduction to Machine Learning, Ethem Alpaydin,  MIT Press 2004 Short booklet to download
Two Crows Introduction
Short booklet to download

For Data Mining and Electronic Business: The Social Data Revolution (Andreas Weingend,PhD) recommends
T. Segaran: Collective Intelligence (2007) Hands on, hacker mentality, includes python code, useful for the recommendation engine homework

M.J.A. Berry and G.S. Linoff: Data Mining Techniques (2004) Applications of data mining in broad marketing and business in general (not just web)
P. Baldi, P. Frasconi, and P. Smyth: Modeling the Internet and the Web (2003) Background on web technology, solid statistical modeling of behavior, information retrieval
C. Shapiro, and H.R. Varian: Information Rules (1998) Short book with insights about the networked economy (network effects, economics of digital goods, pricing, etc.)

More References

Software and Data Sets

UCI Repository:
UCI KDD Archive:

More Data Sets


Journal of Machine Learning Research
 Machine Learning
 Neural Computation
 Neural Networks
 IEEE Transactions on Neural Networks
 IEEE Transactions on Pattern Analysis and Machine
 Annals of Statistics
 Journal of the American Statistical Association


International Conference on Machine Learning (ICML)
 European Conference on Machine Learning (ECML)
 Neural Information Processing Systems (NIPS)
 Uncertainty in Artificial Intelligence (UAI)
 Computational Learning Theory (COLT)
 International Joint Conference on Artificial Intelligence (IJCAI)
 International Conference on Neural Networks (Europe)

Math Review: 

Review Matrices:

Make a free website with Yola