# Reading

I have never believed it possible to have too many books. You should not think of these as required reading, but rather as references that you might find helpful along the way. All the texts linked on this page are either open or can be accessed via the Cornell library, though you may need to use Passkey. We note abbreviations for the books where we actually assign readings.

If you want to know my take on much of the background for this class, I recommend the notes from my own course. I also have some background notes that might be summarized as “things I might’ve assumed you knew” (correctly or incorrectly).

I have taught variants of this course before, and you might find some of them useful as well.

## General Numerics

If you want to refresh your general numerical analysis chops and have fun doing
it, I recommend the *Afternotes* books by Pete Stewart. If you would like a
more standard text that covers most of the background relevant to this class,
you may like Heath’s book (expanded for the “SIAM Classics” edition).
I was involved in a book on many of the same topics, together with Jonathan
Goodman at NYU. O’Leary’s book on *Scientific Computing with Case Studies*
is probably the closest of the lot to the topics of this course, with
particularly relevant case studies. And Higham’s *Accuracy and Stability of
Numerical Methods* is a magesterial treatment of all manner of error analysis
(highly recommended, but perhaps not as a starting point).

- Afternotes on Numerical Analysis and Afternotes Goes to Graduate School, Stewart
- Scientific Computing: An Introductory Survey, Heath
- Principles of Scientific Computing, Bindel and Goodman
- Scientific Computing with Case Studies, O’Leary
- Accuracy and Stability of Numerical Algorithms, Higham

## Numerical Linear Algebra

I learned numerical linear algebra from Demmel’s book, and still tend to go to
it as a reference when I think about how to teach. Trefethen and Bau is
another popular take, created from when Trefethen taught at Cornell CS.
Golub and Van Loan’s book on Matrix Computations ought to be on your shelf
if you decide to do this stuff professionally, but I also like the depth
of coverage in Stewart’s *Matrix Algorithms* (in two volumes). And Elden’s
*Matrix Methods in Data Mining and Pattern Recognition* is one of the closest
books I’ve found to the spirit of this course (or at least part of it).

- ALA: Applied Numerical Linear Algebra, Demmel
- Numerical Linear Algebra, Trefethen and Bau
- Matrix Algorithms, Vol 1 and Matrix Algorithms, Vol 2, Stewart
- Matrix Methods in Data Mining and Pattern Recognition, Elden

## Numerical Optimization

My go-to book on numerical optimization is Nocedal and Wright, with the book by Gill, Murray, and Wright as a close second (the two Wrights are unrelated). For the particular case of convex optimization, the standard reference is Boyd and Vandeberghe. And given how much of data fitting revolves around linear and nonlinear least squares problems, we also mention an old favorite by Bjorck.

- NO: Numerical Optimization, Nocedal and Wright
- Practical Optimization, Gill, Murray, and Wright
- Convex Optimization, Boyd and Vandenberghe
- Numerical Methods for Least Squares Problems, Bjorck

## Machine Learning and Statistics

This class is primarily about numerical methods, but the application (to tasks in statistics, data science, and machine learning) is important to the shape of the methods. My favorite book for background in this direction is Hastie, Tribshirani, and Friedman, but the first book I picked up (and one I still think is good) was Bishop. And while you may decide not to read the entirety of Wasserman’s book, I highly recommend at least reading the preface, and specifically the “statistics/data mining dictionary”.

- ESL: Elements of Statistical Learning, Hastie, Tribshirani, and Friedman
- Pattern Recognition and Machine Learning, Bishop
- All of Statistics, Wasserman

## Math Background

If you want a quick refresher of “math I thought you knew” and prefer something beyond my own notes, Garrett Thomas’s notes on “Mathematics for Machine Learning” are a good start. If you want much, much more math for ML (and CS beyond ML), the book(?) by Gallier and Quaintance will keep you busy for some time.

- Mathematics for Machine Learning, Thomas
- Much more math for CS and ML, Gallier and Quaintance