Python Scientific Lecture Notes

Download

• PDF, 2 pages per side
• PDF, 1 page per side
• HTML and example files
• Source code (github)

This document

Teaching material on the scientific Python ecosystem, a quick introduction to central tools and techniques. The different chapters each correspond to a 1 to 2 hours course with increasing level of expertise, from beginner to expert.

• Authors
• What’s new
• Scipy-Lecture-Notes
• License

---

• 1. Getting started with Python for science
  • 1.1. Scientific computing with tools and workflow
    • 1.1.1. Why Python?
      • 1.1.1.1. The scientist’s needs
      • 1.1.1.2. Specifications
      • 1.1.1.3. Existing solutions
    • 1.1.2. Scientific Python building blocks
    • 1.1.3. The interactive workflow: IPython and a text editor
      • 1.1.3.1. Command line interaction
      • 1.1.3.2. Elaboration of the algorithm in an editor
      • 1.1.3.3. IPython Tips and Tricks
  • 1.2. The Python language
    • 1.2.1. First steps
    • 1.2.2. Basic types
      • 1.2.2.1. Numerical types
      • 1.2.2.2. Containers
        • 1.2.2.2.1. Lists
        • 1.2.2.2.2. Strings
        • 1.2.2.2.3. Dictionaries
        • 1.2.2.2.4. More container types
      • 1.2.2.3. Assignment operator
    • 1.2.3. Control Flow
      • 1.2.3.1. if/elif/else
      • 1.2.3.2. for/range
      • 1.2.3.3. while/break/continue
      • 1.2.3.4. Conditional Expressions
      • 1.2.3.5. Advanced iteration
        • 1.2.3.5.1. Iterate over any sequence
        • 1.2.3.5.2. Keeping track of enumeration number
        • 1.2.3.5.3. Looping over a dictionary
      • 1.2.3.6. List Comprehensions
    • 1.2.4. Defining functions
      • 1.2.4.1. Function definition
      • 1.2.4.2. Return statement
      • 1.2.4.3. Parameters
      • 1.2.4.4. Passing by value
      • 1.2.4.5. Global variables
      • 1.2.4.6. Variable number of parameters
      • 1.2.4.7. Docstrings
      • 1.2.4.8. Functions are objects
      • 1.2.4.9. Methods
      • 1.2.4.10. Exercises
    • 1.2.5. Reusing code: scripts and modules
      • 1.2.5.1. Scripts
      • 1.2.5.2. Importing objects from modules
      • 1.2.5.3. Creating modules
      • 1.2.5.4. ‘__main__’ and module loading
      • 1.2.5.5. Scripts or modules? How to organize your code
      • 1.2.5.6. Packages
      • 1.2.5.7. Good practices
    • 1.2.6. Input and Output
      • 1.2.6.1. Iterating over a file
        • 1.2.6.1.1. File modes
    • 1.2.7. Standard Library
      • 1.2.7.1. os module: operating system functionality
        • 1.2.7.1.1. Directory and file manipulation
        • 1.2.7.1.2. os.path: path manipulations
        • 1.2.7.1.3. Running an external command
        • 1.2.7.1.4. Walking a directory
        • 1.2.7.1.5. Environment variables:
      • 1.2.7.2. shutil: high-level file operations
      • 1.2.7.3. glob: Pattern matching on files
      • 1.2.7.4. sys module: system-specific information
      • 1.2.7.5. pickle: easy persistence
    • 1.2.8. Exception handling in Python
      • 1.2.8.1. Exceptions
      • 1.2.8.2. Catching exceptions
        • 1.2.8.2.1. try/except
        • 1.2.8.2.2. try/finally
        • 1.2.8.2.3. Easier to ask for forgiveness than for permission
      • 1.2.8.3. Raising exceptions
    • 1.2.9. Object-oriented programming (OOP)
  • 1.3. NumPy: creating and manipulating numerical data
    • 1.3.1. The numpy array object
      • 1.3.1.1. What are Numpy and numpy arrays
      • 1.3.1.2. Reference documentation
      • 1.3.1.3. Creating arrays
      • 1.3.1.4. Basic data types
      • 1.3.1.5. Basic visualization
      • 1.3.1.6. Indexing and slicing
      • 1.3.1.7. Copies and views
      • 1.3.1.8. Adding Axes
      • 1.3.1.9. Fancy indexing
        • 1.3.1.9.1. Using boolean masks
        • 1.3.1.9.2. Indexing with an array of integers
    • 1.3.2. Numerical operations on arrays
      • 1.3.2.1. Elementwise operations
      • 1.3.2.2. Basic reductions
      • 1.3.2.3. Broadcasting
      • 1.3.2.4. Array shape manipulation
        • 1.3.2.4.1. Flattening
        • 1.3.2.4.2. Reshaping
        • 1.3.2.4.3. Views and copies
        • 1.3.2.4.4. Dimension shuffling
        • 1.3.2.4.5. Resizing
        • 1.3.2.4.6. Some examples of real-world use cases
      • 1.3.2.5. Sorting data
      • 1.3.2.6. Some exercises
      • 1.3.2.7. Summary
    • 1.3.3. More elaborate arrays
      • 1.3.3.1. More data types
        • 1.3.3.1.1. Casting
        • 1.3.3.1.2. Different data type sizes
      • 1.3.3.2. Structured data types
      • 1.3.3.3. maskedarray: dealing with (propagation of) missing data
    • 1.3.4. Advanced operations
      • 1.3.4.1. Polynomials
        • 1.3.4.1.1. More polynomials (with more bases)
      • 1.3.4.2. Loading data files
        • 1.3.4.2.1. Text files
        • 1.3.4.2.2. Images
        • 1.3.4.2.3. Numpy’s own format
        • 1.3.4.2.4. Well-known (& more obscure) file formats
  • 1.4. Matplotlib: plotting
    • 1.4.1. Introduction
      • 1.4.1.1. IPython and the pylab mode
      • 1.4.1.2. pylab
    • 1.4.2. Simple plot
      • 1.4.2.1. Using defaults
      • 1.4.2.2. Instantiating defaults
      • 1.4.2.3. Changing colors and line widths
      • 1.4.2.4. Setting limits
      • 1.4.2.5. Setting ticks
      • 1.4.2.6. Setting tick labels
      • 1.4.2.7. Moving spines
      • 1.4.2.8. Adding a legend
      • 1.4.2.9. Annotate some points
      • 1.4.2.10. Devil is in the details
    • 1.4.3. Figures, Subplots, Axes and Ticks
      • 1.4.3.1. Figures
      • 1.4.3.2. Subplots
      • 1.4.3.3. Axes
      • 1.4.3.4. Ticks
        • 1.4.3.4.1. Tick Locators
    • 1.4.4. Other Types of Plots: examples and exercises
      • 1.4.4.1. Regular Plots
      • 1.4.4.2. Scatter Plots
      • 1.4.4.3. Bar Plots
      • 1.4.4.4. Contour Plots
      • 1.4.4.5. Imshow
      • 1.4.4.6. Pie Charts
      • 1.4.4.7. Quiver Plots
      • 1.4.4.8. Grids
      • 1.4.4.9. Multi Plots
      • 1.4.4.10. Polar Axis
      • 1.4.4.11. 3D Plots
      • 1.4.4.12. Text
    • 1.4.5. Beyond this tutorial
      • 1.4.5.1. Tutorials
      • 1.4.5.2. Matplotlib documentation
      • 1.4.5.3. Code documentation
      • 1.4.5.4. Galleries
      • 1.4.5.5. Mailing lists
    • 1.4.6. Quick references
      • 1.4.6.1. Line properties
      • 1.4.6.2. Line styles
      • 1.4.6.3. Markers
      • 1.4.6.4. Colormaps
  • 1.5. Scipy : high-level scientific computing
    • 1.5.1. File input/output: scipy.io
    • 1.5.2. Special functions: scipy.special
    • 1.5.3. Linear algebra operations: scipy.linalg
    • 1.5.4. Fast Fourier transforms: scipy.fftpack
    • 1.5.5. Optimization and fit: scipy.optimize
    • 1.5.6. Statistics and random numbers: scipy.stats
      • 1.5.6.1. Histogram and probability density function
      • 1.5.6.2. Percentiles
      • 1.5.6.3. Statistical tests
    • 1.5.7. Interpolation: scipy.interpolate
    • 1.5.8. Numerical integration: scipy.integrate
    • 1.5.9. Signal processing: scipy.signal
    • 1.5.10. Image processing: scipy.ndimage
      • 1.5.10.1. Geometrical transformations on images
      • 1.5.10.2. Image filtering
      • 1.5.10.3. Mathematical morphology
      • 1.5.10.4. Measurements on images
    • 1.5.11. Summary exercises on scientific computing
      • 1.5.11.13. Maximum wind speed prediction at the Sprogø station
        • 1.5.11.13.1. Statistical approach
        • 1.5.11.13.2. Computing the cumulative probabilities
        • 1.5.11.13.3. Prediction with UnivariateSpline
        • 1.5.11.13.4. Exercise with the Gumbell distribution
      • 1.5.11.14. Non linear least squares curve fitting: application to point extraction in topographical lidar data
        • 1.5.11.14.1. Introduction
        • 1.5.11.14.2. Loading and visualization
        • 1.5.11.14.3. Fitting a waveform with a simple Gaussian model
          • 1.5.11.14.3.1. Model
          • 1.5.11.14.3.2. Initial solution
          • 1.5.11.14.3.3. Fit
        • 1.5.11.14.4. Going further
      • 1.5.11.15. Image processing application: counting bubbles and unmolten grains
        • 1.5.11.15.1. Statement of the problem
        • 1.5.11.15.2. Proposed solution
      • 1.5.11.16. Example of solution for the image processing exercise: unmolten grains in glass
      • 1.5.11.13. Maximum wind speed prediction at the Sprogø station
        • 1.5.11.13.1. Statistical approach
        • 1.5.11.13.2. Computing the cumulative probabilities
        • 1.5.11.13.3. Prediction with UnivariateSpline
        • 1.5.11.13.4. Exercise with the Gumbell distribution
      • 1.5.11.14. Non linear least squares curve fitting: application to point extraction in topographical lidar data
        • 1.5.11.14.1. Introduction
        • 1.5.11.14.2. Loading and visualization
        • 1.5.11.14.3. Fitting a waveform with a simple Gaussian model
          • 1.5.11.14.3.1. Model
          • 1.5.11.14.3.2. Initial solution
          • 1.5.11.14.3.3. Fit
        • 1.5.11.14.4. Going further
      • 1.5.11.15. Image processing application: counting bubbles and unmolten grains
        • 1.5.11.15.1. Statement of the problem
        • 1.5.11.15.2. Proposed solution
      • 1.5.11.16. Example of solution for the image processing exercise: unmolten grains in glass
      • 1.5.11.13. Maximum wind speed prediction at the Sprogø station
        • 1.5.11.13.1. Statistical approach
        • 1.5.11.13.2. Computing the cumulative probabilities
        • 1.5.11.13.3. Prediction with UnivariateSpline
        • 1.5.11.13.4. Exercise with the Gumbell distribution
      • 1.5.11.14. Non linear least squares curve fitting: application to point extraction in topographical lidar data
        • 1.5.11.14.1. Introduction
        • 1.5.11.14.2. Loading and visualization
        • 1.5.11.14.3. Fitting a waveform with a simple Gaussian model
          • 1.5.11.14.3.1. Model
          • 1.5.11.14.3.2. Initial solution
          • 1.5.11.14.3.3. Fit
        • 1.5.11.14.4. Going further
      • 1.5.11.15. Image processing application: counting bubbles and unmolten grains
        • 1.5.11.15.1. Statement of the problem
        • 1.5.11.15.2. Proposed solution
      • 1.5.11.16. Example of solution for the image processing exercise: unmolten grains in glass
  • 1.6. Getting help and finding documentation
• 2. Advanced topics
  • 2.1. Advanced Python Constructs
    • 2.1.1. Iterators, generator expressions and generators
      • 2.1.1.1. Iterators
      • 2.1.1.2. Generator expressions
      • 2.1.1.3. Generators
      • 2.1.1.4. Bidirectional communication
      • 2.1.1.5. Chaining generators
    • 2.1.2. Decorators
      • 2.1.2.1. Replacing or tweaking the original object
      • 2.1.2.2. Decorators implemented as classes and as functions
      • 2.1.2.3. Copying the docstring and other attributes of the original function
      • 2.1.2.4. Examples in the standard library
      • 2.1.2.5. Deprecation of functions
      • 2.1.2.6. A while-loop removing decorator
      • 2.1.2.7. A plugin registration system
      • 2.1.2.8. More examples and reading
    • 2.1.3. Context managers
      • 2.1.3.1. Catching exceptions
      • 2.1.3.2. Using generators to define context managers
  • 2.2. Advanced Numpy
    • 2.2.1. Life of ndarray
      • 2.2.1.1. It’s...
      • 2.2.1.2. Block of memory
      • 2.2.1.3. Data types
        • 2.2.1.3.1. The descriptor
        • 2.2.1.3.2. Example: reading .wav files
        • 2.2.1.3.3. Casting and re-interpretation/views
          • 2.2.1.3.3.1. Casting
          • 2.2.1.3.3.2. Re-interpretation / viewing
      • 2.2.1.4. Indexing scheme: strides
        • 2.2.1.4.1. Main point
          • 2.2.1.4.1.1. C and Fortran order
          • 2.2.1.4.1.2. Slicing with integers
        • 2.2.1.4.2. Example: fake dimensions with strides
        • 2.2.1.4.3. Broadcasting
        • 2.2.1.4.4. More tricks: diagonals
        • 2.2.1.4.5. CPU cache effects
        • 2.2.1.4.6. Example: inplace operations (caveat emptor)
      • 2.2.1.5. Findings in dissection
    • 2.2.2. Universal functions
      • 2.2.2.1. What they are?
        • 2.2.2.1.1. Parts of an Ufunc
        • 2.2.2.1.2. Making it easier
      • 2.2.2.2. Exercise: building an ufunc from scratch
      • 2.2.2.3. Solution: building an ufunc from scratch
      • 2.2.2.4. Generalized ufuncs
    • 2.2.3. Interoperability features
      • 2.2.3.1. Sharing multidimensional, typed data
      • 2.2.3.2. The old buffer protocol
      • 2.2.3.3. The old buffer protocol
      • 2.2.3.4. Array interface protocol
    • 2.2.4. Array siblings: chararray, maskedarray, matrix
      • 2.2.4.1. chararray: vectorized string operations
      • 2.2.4.2. masked_array missing data
        • 2.2.4.2.1. The mask
        • 2.2.4.2.2. Domain-aware functions
      • 2.2.4.3. recarray: purely convenience
      • 2.2.4.4. matrix: convenience?
    • 2.2.5. Summary
    • 2.2.6. Contributing to Numpy/Scipy
      • 2.2.6.1. Why
      • 2.2.6.2. Reporting bugs
        • 2.2.6.2.1. Good bug report
      • 2.2.6.3. Contributing to documentation
      • 2.2.6.4. Contributing features
      • 2.2.6.5. How to help, in general
  • 2.3. Debugging code
    • 2.3.1. Avoiding bugs
      • 2.3.1.1. Coding best practices to avoid getting in trouble
      • 2.3.1.2. pyflakes: fast static analysis
        • 2.3.1.2.1. Running pyflakes on the current edited file
        • 2.3.1.2.2. A type-as-go spell-checker like integration
    • 2.3.2. Debugging workflow
    • 2.3.3. Using the Python debugger
      • 2.3.3.1. Invoking the debugger
        • 2.3.3.1.1. Postmortem
        • 2.3.3.1.2. Step-by-step execution
        • 2.3.3.1.3. Other ways of starting a debugger
      • 2.3.3.2. Debugger commands and interaction
        • 2.3.3.2.1. Getting help when in the debugger
    • 2.3.4. Debugging segmentation faults using gdb
  • 2.4. Optimizing code
    • 2.4.1. Optimization workflow
    • 2.4.2. Profiling Python code
      • 2.4.2.1. Timeit
      • 2.4.2.2. Profiler
      • 2.4.2.3. Line-profiler
      • 2.4.2.4. Running cProfile
      • 2.4.2.5. Using gprof2dot
    • 2.4.3. Making code go faster
      • 2.4.3.1. Algorithmic optimization
        • 2.4.3.1.1. Example of the SVD
    • 2.4.4. Writing faster numerical code
      • 2.4.4.1. Additional Links
  • 2.5. Sparse Matrices in SciPy
    • 2.5.1. Introduction
      • 2.5.1.1. Why Sparse Matrices?
      • 2.5.1.2. Sparse Matrices vs. Sparse Matrix Storage Schemes
      • 2.5.1.3. Typical Applications
      • 2.5.1.4. Prerequisites
      • 2.5.1.5. Sparsity Structure Visualization
    • 2.5.2. Storage Schemes
      • 2.5.2.1. Common Methods
      • 2.5.2.2. Sparse Matrix Classes
        • 2.5.2.2.1. Diagonal Format (DIA)
          • 2.5.2.2.1.1. Examples
        • 2.5.2.2.2. List of Lists Format (LIL)
          • 2.5.2.2.2.1. Examples
        • 2.5.2.2.3. Dictionary of Keys Format (DOK)
          • 2.5.2.2.3.1. Examples
        • 2.5.2.2.4. Coordinate Format (COO)
          • 2.5.2.2.4.1. Examples
        • 2.5.2.2.5. Compressed Sparse Row Format (CSR)
          • 2.5.2.2.5.1. Examples
        • 2.5.2.2.6. Compressed Sparse Column Format (CSC)
          • 2.5.2.2.6.1. Examples
        • 2.5.2.2.7. Block Compressed Row Format (BSR)
          • 2.5.2.2.7.1. Examples
      • 2.5.2.3. Summary
    • 2.5.3. Linear System Solvers
      • 2.5.3.1. Sparse Direct Solvers
        • 2.5.3.1.1. Examples
      • 2.5.3.2. Iterative Solvers
        • 2.5.3.2.1. Common Parameters
        • 2.5.3.2.2. LinearOperator Class
        • 2.5.3.2.3. A Few Notes on Preconditioning
      • 2.5.3.3. Eigenvalue Problem Solvers
        • 2.5.3.3.1. The eigen module
    • 2.5.4. Other Interesting Packages
  • 2.6. Image manipulation and processing using Numpy and Scipy
    • 2.6.1. Opening and writing to image files
    • 2.6.2. Displaying images
    • 2.6.3. Basic manipulations
      • 2.6.3.1. Statistical information
      • 2.6.3.2. Geometrical transformations
    • 2.6.4. Image filtering
      • 2.6.4.1. Blurring/smoothing
      • 2.6.4.2. Sharpening
      • 2.6.4.3. Denoising
      • 2.6.4.4. Mathematical morphology
    • 2.6.5. Feature extraction
      • 2.6.5.1. Edge detection
      • 2.6.5.2. Segmentation
    • 2.6.6. Measuring objects properties: ndimage.measurements
  • 2.7. Mathematical optimization: finding minima of functions
    • 2.7.1. Knowing your problem
      • 2.7.1.1. Convex versus non-convex optimization
      • 2.7.1.2. Smooth and non-smooth problems
      • 2.7.1.3. Noisy versus exact cost functions
      • 2.7.1.4. Constraints
    • 2.7.2. A review of the different optimizers
      • 2.7.2.1. Getting started: 1D optimization
      • 2.7.2.2. Gradient based methods
        • 2.7.2.2.1. Some intuitions about gradient descent
        • 2.7.2.2.2. Conjugate gradient descent
      • 2.7.2.3. Newton and quasi-newton methods
        • 2.7.2.3.1. Newton methods: using the Hessian (2nd differential)
        • 2.7.2.3.2. Quasi-Newton methods: approximating the Hessian on the fly
      • 2.7.2.4. Gradient-less methods
        • 2.7.2.4.1. A shooting method: the Powell algorithm
        • 2.7.2.4.2. Simplex method: the Nelder-Mead
      • 2.7.2.5. Global optimizers
        • 2.7.2.5.1. Brute force: a grid search
        • 2.7.2.5.2. Simulated annealing
    • 2.7.3. Practical guide to optimization with scipy
      • 2.7.3.1. Choosing a method
      • 2.7.3.2. Making your optimizer faster
      • 2.7.3.3. Computing gradients
      • 2.7.3.4. Synthetic exercices
    • 2.7.4. Special case: non-linear least-squares
      • 2.7.4.1. Minimizing the norm of a vector function
      • 2.7.4.2. Curve fitting
    • 2.7.5. Optimization with constraints
      • 2.7.5.1. Box bounds
      • 2.7.5.2. General constraints
  • 2.8. Traits
    • 2.8.1. Introduction
    • 2.8.2. Example
    • 2.8.3. What are Traits
      • 2.8.3.1. Initialisation
      • 2.8.3.2. Validation
      • 2.8.3.3. Documentation
      • 2.8.3.4. Visualisation
      • 2.8.3.5. Deferral
      • 2.8.3.6. Notification
      • 2.8.3.7. Some more advanced traits
    • 2.8.4. References
  • 2.9. 3D plotting with Mayavi
    • 2.9.1. Mlab: the scripting interface
      • 2.9.1.1. 3D plotting functions
        • 2.9.1.1.1. Points
        • 2.9.1.1.2. Lines
        • 2.9.1.1.3. Elevation surface
        • 2.9.1.1.4. Arbitrary regular mesh
        • 2.9.1.1.5. Volumetric data
      • 2.9.1.2. Figures and decorations
        • 2.9.1.2.1. Figure management
        • 2.9.1.2.2. Changing plot properties
        • 2.9.1.2.3. Decorations
    • 2.9.2. Interactive work
      • 2.9.2.1. The “pipeline dialog”
      • 2.9.2.2. The script recording button
  • 2.10. Sympy : Symbolic Mathematics in Python
    • 2.10.1. First Steps with SymPy
      • 2.10.1.1. Using SymPy as a calculator
      • 2.10.1.2. Exercises
      • 2.10.1.3. Symbols
    • 2.10.2. Algebraic manipulations
      • 2.10.2.1. Expand
      • 2.10.2.2. Simplify
      • 2.10.2.3. Exercises
    • 2.10.3. Calculus
      • 2.10.3.1. Limits
      • 2.10.3.2. Differentiation
      • 2.10.3.3. Series expansion
      • 2.10.3.4. Exercises
      • 2.10.3.5. Integration
      • 2.10.3.6. Exercises
    • 2.10.4. Equation solving
      • 2.10.4.1. Exercises
    • 2.10.5. Linear Algebra
      • 2.10.5.1. Matrices
      • 2.10.5.2. Differential Equations
      • 2.10.5.3. Exercises
  • 2.11. scikit-learn: machine learning in Python
    • 2.11.1. Loading an example dataset
      • 2.11.1.1. Learning and Predicting
    • 2.11.2. Classification
      • 2.11.2.1. k-Nearest neighbors classifier
      • 2.11.2.2. Support vector machines (SVMs) for classification
        • 2.11.2.2.1. Linear Support Vector Machines
        • 2.11.2.2.2. Using kernels
    • 2.11.3. Clustering: grouping observations together
      • 2.11.3.1. K-means clustering
    • 2.11.4. Dimension Reduction with Principal Component Analysis
    • 2.11.5. Putting it all together: face recognition
    • 2.11.6. Linear model: from regression to sparsity
      • 2.11.6.1. Sparse models
    • 2.11.7. Model selection: choosing estimators and their parameters
      • 2.11.7.1. Grid-search and cross-validated estimators
        • 2.11.7.1.1. Grid-search
        • 2.11.7.1.2. Cross-validated estimators
  • 2.12. Interfacing with C
    • 2.12.1. Introduction
    • 2.12.2. Python-C-Api
      • 2.12.2.1. Example
      • 2.12.2.2. Numpy Support
    • 2.12.3. Ctypes
      • 2.12.3.1. Example
      • 2.12.3.2. Numpy Support
    • 2.12.4. SWIG
      • 2.12.4.1. Example
      • 2.12.4.2. Numpy Support
    • 2.12.5. Cython
      • 2.12.5.1. Example
      • 2.12.5.2. Numpy Support
    • 2.12.6. Summary
    • 2.12.7. Further Reading and References
    • 2.12.8. Exercises
      • 2.12.8.1. Python-C-API
      • 2.12.8.2. Ctypes
      • 2.12.8.3. SWIG
      • 2.12.8.4. Cython

---

Version: 2013.1-2-g9a33667 (output of git describe for project repository)

PythonScientific.pdf PythonScientific-simple.pdf