# Essay on how to start solving a machine learning problem (click here).

# Research Papers:

# Recommender Systems:

# Collaborative Filtering with Temporal Dynamics: Yehuda Koren's paper, describes recommendation algorithm incorporating time dependent info. the paper

Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model: Yehuda Koren's paper describes how to add the implicit information (which movies were rented) to the model based on explicit data of movies which were rated.Koren's paper describes in part his algorithm for Netfix's contest, http://www.netflixprize.com//index . It is very interesting algorithm and all comments on the paper are welcome. A news article describing the team is here: http://cacm.acm.org/news/32450-award-winning-paper-reveals-key-to-netflix-prize/fulltext

The BellKor Solution to the Netflix Grand Prize by Yehuda Koren: this paper describes Koren's teams contribution to the "Bell-Kor's Pragmatic Chaos" final solution, which won the Netflix Grand Prize.

The Ensemble Team wrote Feature-Weighted Linear Stacking by Joseph Sill, Gabor Takacs, Lester Mackey, David Lin

Another contestant also wrote papers: Chris Volinsky's Publications including Collaborative Filtering for Implicit Feedback Datasets paper

Jure Leskovec CS246 Mining Massive Data Sets: Recommender Systems Lecture

Fast Training of SVM Using Sequential Minimal Optimization

# SVD for Large Data Sets:

Finding Structure with Randomness: Stochastic Algorithms for Constructing Approximate Matrix Decompositions by N. Halko, P.G. Martinsson, and J.A. Tropp Paper# Decision Trees:

Multivariate Adaptive Regression
by Jerome H. Friedman MART Tutorial

Code

John Herbach Lecture on PLANET

# Convex Optimization:

Convex Optimization by Stephen Boyd and Lieven Vanderberghe Matlab Code by Michael Grant and Stephen Boyd

# Lectures:

From Trees to Forests and Rule Sets - A Unified Overview of Ensemble Methods video by Giovanni Seni, PhD Santa Clara University

Lectures and Labs from Samy Bengio PhD

# Tutorials:

# Beginning Data Mining Tutorial

# Tutorials on Collaborative Filtering include: 1) Programming Collective Intelligence: Building Smart Web 2.0 Applications by Toby Segaran (Second Chapter). Koren did a fantastic job of adding time dependence to the algorithms described by Segaran. 2) Yehuda Koren, Robert Bell, and Chris Volinsky wrote an article titled, "Matrix Factorization Techniques for Recommender Systems" for IEEE Aug. 2009

# Tutorial on Support Vector Machines for Pattern Recognition Paper

Google TalksAndrew Ng's Review notes: Linear Algebra, Probability, Hidden Markov Models

Great Linear Algebra Text: "Introduction to Linear Algebra" by Gilbert Strang. His lectures are also available online at:

http://ocw.mit.edu/courses/

or at: http://www.youtube.com/watch?v=ZK3O402wf1c

Course materials can be found at: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/

Computational Science lectures can be found at: http://www.youtube.com/watch?v=CgfkEUOFAj0.

For Matrix Computations Golub and Van Loan

For distributed Trefethen's Matrix Computations

# Courses:

Video Course at Stanford Lecture by Professor Andrew Ng for Machine Learning (uses MatLab/Octave) (CS 229) All Lectures Web Page Course
Info Another course

Machine Learning Course based on Hastie, Tibshirani, & Friedman's text Web Page First Semester Second Semester

Papers and software by Jerome Friedman

Machine Learning Course based on Bishop's Pattern Recognition and Machine Learning text Web Page

Machine Learning Course at CMU Web Page

Association for Computing Machinery Data Mining Special Interest Group Web Page

Statistical Aspects of Data Mining by Professor Rajan Patel (Stanford Statistics 202) Web Page Prof David Mease Video Lecture

Advanced Machine Learning (Focus: Learning with matrix parameters) Professor Manfred K. Warmuth (UCSC) Web Page

Machine Learning & Data Mining UCSC CMPS 142 Course Web Page

MIT 6.867 Machine Learning Web Page Open Course Lecture Notes

Generalized Linear Models UCSC AMS 274 Course Course
information GLMs Notes
on GLM definitions & maximum likelihood estimation

Stanford MS&E 237 The Social Data Revolution: Data Mining and Electronic Business Wiki

For the Beginner: Linear Algebra at MIT by Prof. Gilbert Strang

Mining Massive Data Sets Text by Jeff Ullman Lectures by Jure Leskovec

# Recommended Texts:

Free TextsElements of Statistical Learning by Trevor Hastie, Robert Tibshirani and Jerome Friedman (ISBN 0387952845)

Pattern Recognition and Machine Learning by Springer, Christopher Bishop, 2006.

Pattern Classification, Richard Duda, Peter Hart and David Stork, 2nd ed. John Wiley & Sons, 2001.

Machine Learning. Tom Mitchell, McGraw-Hill, 1997.

Reinforcement Learning: An introduction, Richard Sutton and Andrew Barto
MIT Press, 1998

For the beginner

Introduction to Data Mining, Pang-Ning Tan,
Michael Steinbach, Vipin Kumar, Pearson, Addison Wesley

Check out my class at the Hacker Dojo Dr. Mike Bowles & Dr. Patricia Hoffman

Pang-Ning Tan Web Page

Course Web Pages: David Mease

Sample Chapters:

- Chapter 4. Classification:
*Basic Concepts, Decision Trees, and Model Evaluation*(444KB) - Chapter 6. Association Analysis:
*Basic Concepts and Algorithms*(612KB) - Chapter 8. Cluster Analysis:
*Basic Concepts and Algorithms*(1.3MB)

Introduction to Machine Learning, Ethem
Alpaydin, MIT Press 2004 Short booklet to download

Two Crows Introduction, Short booklet to download

For Data Mining and Electronic Business: The Social Data
Revolution (Andreas Weingend,PhD) recommends

T. Segaran: Collective Intelligence (2007) Hands on, hacker mentality,
includes python code, useful for the del.icio.us recommendation engine homework

M.J.A. Berry and G.S. Linoff: Data
Mining Techniques (2004) Applications of data
mining in broad marketing and business in general (not just web)

P.
Baldi, P. Frasconi, and P. Smyth: Modeling the Internet and the Web (2003)
Background on web technology, solid statistical modeling of behavior,
information retrieval

C. Shapiro, and H.R. Varian:
Information Rules (1998) Short
book with insights about the networked economy (network effects, economics of
digital goods, pricing, etc.)

http://www.ics.uci.edu/~mlearn/MLRepository.html

UCI KDD Archive:

http://kdd.ics.uci.edu/summary.data.application.html

Statlib: http://lib.stat.cmu.edu/

Delve: http://www.cs.utoronto.ca/~delve/

More Data Sets

Journals

Journal of Machine Learning Research www.jmlr.org

Machine Learning

Neural Computation

Neural Networks

IEEE Transactions on Neural Networks

IEEE Transactions on Pattern Analysis and Machine

Intelligence

Annals of Statistics

Journal of the American Statistical Association

Conferences

International Conference on Machine Learning (ICML)

ICML05: http://icml.ais.fraunhofer.de/

European Conference on Machine Learning (ECML)

ECML05: http://ecmlpkdd05.liacc.up.pt/

Neural Information Processing Systems (NIPS)

NIPS05: http://nips.cc/

Uncertainty in Artificial Intelligence (UAI)

UAI05: http://www.cs.toronto.edu/uai2005/

Computational Learning Theory (COLT)

COLT05: http://learningtheory.org/colt2005/

International Joint Conference on Artificial Intelligence (IJCAI)

IJCAI05: http://ijcai05.csd.abdn.ac.uk/

International Conference on Neural Networks (Europe)

ICANN05: http://www.ibspan.waw.pl/ICANN-2005/

Math Review: http://www.mathopenref.com/

Review Matrices: https://www.khanacademy.org/math/precalculus/precalc-matrices/Basic_matrix_operations/v/introduction-to-the-matrix