Serious Programming - small courses

Date:

Aug 03, 2025

Author:

Mark Galassi <mark@galassi.org>

Author:

Leina Gries <leinagries@gmail.com>

Author:

Sophia Mulholland <smulholland505@gmail.com>

Author:

Almond Heil <almondheil@gmail.com>

Contents:

• 1. Motivation and plan
  • 1.1. Notes for teachers
  • 1.2. Acknowledgements
  • 1.3. Status of the book
  • 1.4. Footnotes
• 2. Starting out: data files and first plots
  • 2.1. Motivation, prerequisites, plan
  • 2.2. Very first data plots with gnuplot
  • 2.3. Plotting functions with gnuplot
  • 2.4. Reading and writing files, in brief
  • 2.5. Generating our own data to plot
  • 2.6. The broad landscape of plotting software
  • 2.7. Data formats
  • 2.8. Simple surface plots
  • 2.9. Topics we have covered
• 3. Intermediate plotting
  • 3.1. A worked example
  • 3.2. Histograms
  • 3.3. Matplotlib
  • 3.4. A histogram snippet to conclude
• 4. A tour of functions
  • 4.1. Motivation, Prerequisites, Plan
  • 4.2. Linear Functions
  • 4.3. Polynomials
    • 4.3.1. Derivatives
  • 4.4. Higher order polynomials
  • 4.5. Inverse functions
  • 4.6. Elementary transcendental functions
    • 4.6.1. Exponentials
    • 4.6.2. Trigonometric function
  • 4.7. Gaussian distribution
• 5. Growth – checked and not
  • 5.1. Motivation, prerequisites, plan
    • 5.1.1. Motivation
    • 5.1.2. Prerequisites
    • 5.1.3. Plan
  • 5.2. Pure exponential growth
  • 5.3. Checked growth
    • 5.3.1. Checked by lack of resources
    • 5.3.2. Checked by competition with other species
  • 5.4. Simple predator-prey interactions
    • 5.4.1. The Lotka-Volterra equations
• 6. Advanced plotting
  • 6.1. Our setup
  • 6.2. A first line plot
  • 6.3. Polishing line plots
  • 6.4. Increasing the challenge: parametrized solutions
  • 6.5. Adding a dimension: surface plots
  • 6.6. Adding a dimension: color
    • 6.6.1. Spetrograms
    • 6.6.2. Spectrograms for standard acoustic files
    • 6.6.3. Ideas for further exploration
  • 6.7. Loops in gnuplot
  • 6.8. Adding a dimension: animation
    • 6.8.1. Animation in gnuplot
    • 6.8.2. Animation in matplotlib
  • 6.9. Further reading
• 7. Fitting functions to data
  • 7.1. Motivation, Prerequisites, Plan
  • 7.2. Examples to get started
  • 7.3. Straight line fits
    • 7.3.1. Our goal
    • 7.3.2. Stepping back: just two points
    • 7.3.3. Let’s plot that line with our data
    • 7.3.4. Physical interpretation of the line fit
  • 7.4. Proper line fitting
  • 7.5. Using Python’s scientific libraries to fit lines
  • 7.6. When to not try a linear fit
  • 7.7. Fitting curves
    • 7.7.1. Polynomial fits
    • 7.7.2. Overfitting
    • 7.7.3. Fitting arbitrary functions
  • 7.8. Topics for further study
    • 7.8.1. Interpolation and extrapolation
    • 7.8.2. How high should the degree of the polynomial be?
• 8. Case studies in data
  • 8.1. Population data from the web
    • 8.1.1. Exercises
• 9. Special numbers: \(\pi\)
  • 9.1. Motivation, prerequisites, plan
    • 9.1.1. Motivation
    • 9.1.2. Prerequisites
    • 9.1.3. Plan
  • 9.2. A collection of factoids
  • 9.3. Calculating \(\pi\): ancient history
  • 9.4. Calculating \(\pi\): monte carlo method
  • 9.5. Calculating \(\pi\): series that converge to \(\pi\)
    • 9.5.1. Madhava-Leibniz series
    • 9.5.2. “Efficient” infinite series
    • 9.5.3. Formulae based on the Riemann zeta function
  • 9.6. Relationships between special numbers
• 10. A workshop on programming by yourself (!)
• 11. Random number basics
  • 11.1. Prerequisites
  • 11.2. Motivation
  • 11.3. Types of distributions
  • 11.4. Further reading
• 12. Randomness and Disorder
  • 12.1. Experiment: burn a match
  • 12.2. Experiment: ink in water
  • 12.3. Discussion on “ink in water” experiment
  • 12.4. Flipping a single coin
  • 12.5. Review: random numbers in Pythyon
  • 12.6. Experiment: flipping a single virtual coin
    • 12.6.1. Just the flips
    • 12.6.2. Long-running average of single coin flips
  • 12.7. Flipping multiple coins
  • 12.8. Experiment: flipping virtual coins
  • 12.9. Experiment: back to physical coins - disorder
  • 12.10. The drunk fencer
  • 12.11. The drunkard’s walk
  • 12.12. Matplotlib animation of a random walk
  • 12.13. Making a movie from walk frames
    • 12.13.1. Reviewing graphics and animation
    • 12.13.2. Making individual frames of the random walk
  • 12.14. Discussion
  • 12.15. Further reading and videos
• 13. Random Processes
  • 13.1. Motivation, prerequisites, plan
  • 13.2. Reviewing random number generation
  • 13.3. Poisson processes
    • 13.3.1. A pure poisson process
    • 13.3.2. An angry lightning goddess
    • 13.3.3. Vicious glow-worms
  • 13.4. Brownian motion
  • 13.5. Further reading
  • 13.6. Progression of record peaks
  • 13.7. The gambler’s fallacy
  • 13.8. The gambler’s ruin
  • 13.9. Link to other chapters
• 14. Power laws, Zipf, Benford, …
  • 14.1. Motivation, prerequisites, plan
  • 14.2. A brief refresher on log-log plots
  • 14.3. Zipf’s law
  • 14.4. What are “power laws”?
  • 14.5. Deadly conflicts
  • 14.6. Benford’s law
  • 14.7. Pareto’s principle
  • 14.8. Olber’s paradox
• 15. Pushing toward calculus
  • 15.1. Motivation and plan
  • 15.2. Prerequisites
  • 15.3. Limits, the infinitely big, and the infinitesimally small
  • 15.4. Continuous functions
  • 15.5. Convergence and divergence
  • 15.6. Weird mixes
  • 15.7. Limits of some functions
  • 15.8. The limit of a series
  • 15.9. The most important application: derivatives
  • 15.10. Visualizing derivatives with an animation
• 16. Numerical integration
  • 16.1. The integral
  • 16.2. Calculating the integral numerically
  • 16.3. Improving the numerical approximation
  • 16.4. Stepping back from numerical integration back to analytical work
• 17. Differential Equations
  • 17.1. Motivation, Prerequisites, Plan
  • 17.2. Derivatives
    • 17.2.1. Why?
  • 17.3. Definitions
  • 17.4. An Example
  • 17.5. Population Growth
    • 17.5.1. But what does the derivative tell us?
  • 17.6. Euler’s Method
  • 17.7. Compiling and running C programs
  • 17.8. A program to use Euler’s method
  • 17.9. Second Order Differential equations
  • 17.10. Falling Body
    • 17.10.1. Adding Air Resistance to Falling Body
  • 17.11. The harmonic oscillator
    • 17.11.1. The simple harmonic oscillator
    • 17.11.2. The damped harmonic oscillator
    • 17.11.3. The non-linear pendulum
• 18. Simulating Differential Equations
  • 18.1. Motivation, Prerequisites, and Plan
  • 18.2. What is a Differential Equation?
  • 18.3. Programming and Solving Differential Equations
  • 18.4. Conclusion
• 19. The Lotka Volterra System
  • 19.1. Motivation, Prerequisites, and Plan
  • 19.2. A Brief Recap of Differential Equations
  • 19.3. The Lotka-Volterra System
  • 19.4. Fractional Deer
  • 19.5. Further Exploration
  • 19.6. Conclusion
• 20. Ecology
  • 20.1. Motivation, Prerequisites, Plan
  • 20.2. Factors that come up in modeling population ecology
  • 20.3. Exponential growth
    • 20.3.1. Unlimited
    • 20.3.2. Resource-constrained - r-selection versus K-selection
  • 20.4. History of the human population on earth
  • 20.5. The logistic function
  • 20.6. The Lotka-Volterra differential equations
  • 20.7. Further reading
• 21. Biology – phylogeny
  • 21.1. Motivation, prerequisites, plan
    • 21.1.1. Motivation
    • 21.1.2. Prerequisites
    • 21.1.3. Plan
  • 21.2. Start with a video and then make a simple table
  • 21.3. Terminology
  • 21.4. First steps with biopython
  • 21.5. Downloading the datasets
  • 21.6. Preparing a tree by hand
  • 21.7. Inferring a tree
  • 21.8. Other sequence analysis resources
  • 21.9. Linguistics datasets
    • 21.9.1. lingpy.org
    • 21.9.2. elinguistics.com
    • 21.9.3. Others
  • 21.10. Evolution of programming languages
• 22. Recursion
  • 22.1. Motivation, prerequisites, plan
  • 22.2. Visual examples
  • 22.3. Word examples
  • 22.4. Components of a recursive definition
    • 22.4.1. Simple math
    • 22.4.2. Programming simple math recursion
    • 22.4.3. Recursion with data structures
  • 22.5. Visualizing what the recursion is doing
  • 22.6. Towers of Hanoi
  • 22.7. Should we really use recursion in programming?
• 23. Programming topics: sorting
  • 23.1. Motivation, prerequisites, plan
  • 23.2. Experiment: a game of cards
  • 23.3. Intuition to algorithm on card sorting
  • 23.4. Writing up the algorithm in Python
  • 23.5. Profiling the algorithm
    • 23.5.1. Modify the program to print information
    • 23.5.2. Run the program and make plots
    • 23.5.3. How do we understand these plots?
    • 23.5.4. Exercises
  • 23.6. Computational complexity
  • 23.7. Further reading
• 24. Introduction to NetworkX Graphs
  • 24.1. Motivation, Prerequisites, Plan
  • 24.2. First Steps
  • 24.3. Directed and Weighted Networks
  • 24.4. Using Larger Data Sets
  • 24.5. Analyzing with Python
  • 24.6. Conclusion
• 25. Generating Mazes
  • 25.1. Motivation, Prerequisites, Plan
  • 25.2. What is a Maze?
  • 25.3. Representing Mazes with Python
  • 25.4. Introductory Example: A Snaking Path
  • 25.5. Generation Algorithms
  • 25.6. Interactive Mazes
  • 25.7. Conclusion
• 26. Birthday paradox
  • 26.1. To get started
  • 26.2. A practical demonstration
  • 26.3. The theory
  • 26.4. Take-home
• 27. Optimal Stopping
  • 27.1. Motivation, Prerequisites, and Plan
  • 27.2. What is Optimal Stopping?
  • 27.3. The Employee Problem
  • 27.4. A (marginally) More Useful Application: Parking
  • 27.5. Conclusion
  • 27.6. Further Reading
• 28. Graphical user interfaces
  • 28.1. A chat about sources of input in a GUI
  • 28.2. Widgets and widget sets
  • 28.3. The simplest programs
    • 28.3.1. The programs
    • 28.3.2. Packers: more than just one button
    • 28.3.3. A tour of widgets
  • 28.4. Following a tutorial
  • 28.5. Cellular automata on a canvas
    • 28.5.1. A simply drawing of the CA
    • 28.5.2. Adding controls to the program
  • 28.6. Conway’s game of life
  • 28.7. Tic-tac-toe with buttons
  • 28.8. A glance at PySimpleGUI
  • 28.9. Other resources
• 29. Drawing on a canvas
  • 29.1. Simplest canvas
  • 29.2. Simplest animation
    • 29.2.1. Exercises
• 30. The Traveling Salesman
  • 30.1. Cities and path lengths
  • 30.2. Solving the Traveling Salesman Problem
    • 30.2.1. A digression on optimization
    • 30.2.2. Generating and visualizing lists of cities
    • 30.2.3. Animating the drawing of cities
  • 30.3. Improvements to the route
    • 30.3.1. Before you start
    • 30.3.2. Impossibile to compute the optimal solution
    • 30.3.3. Greedy algorithm
    • 30.3.4. A digression on hill climbing
    • 30.3.5. Hill climbing for the traveling salesman problem
    • 30.3.6. Further study
  • 30.4. Where do you go from search
• 31. Basic Agent-Based Modeling
  • 31.1. Motivation, Prerequisites, and Plan
  • 31.2. Conceptualizing the model
    • 31.2.1. Agent-Based Modeling Concepts
    • 31.2.2. Object-Oriented Programming Concepts
  • 31.3. Classes and steps
  • 31.4. Space and movement
  • 31.5. Visualization
  • 31.6. Interactions between agents
  • 31.7. Data collection & plotting
    • 31.7.1. Collecting data from the code
    • 31.7.2. Plotting from the command line
    • 31.7.3. Plotting as the live model runs
  • 31.8. Source code
  • 31.9. Making an SIR model
  • 31.10. Further reading
• 32. Emergent behavior
  • 32.1. Motivation, prerequisites, plan
  • 32.2. Before you start
  • 32.3. Write the simple_ca.py program which implements a cellular automaton
  • 32.4. Conway’s game of life
  • 32.5. Install and run the golly program
  • 32.6. Further study
    • 32.6.1. Play with the simple_ca.py program
  • 32.7. Further reading
• 33. Web scraping
  • 33.1. Motivation, prerequisites, plan
  • 33.2. What does a web page look like underneath? (HTML)
  • 33.3. Command line scraping with wget
  • 33.4. Scraping from a Python program
  • 33.5. Finding neat scientific data sets
  • 33.6. Beautiful Soup
• 34. Getting to philosophy
  • 34.1. Motivation, prerequisites, plan
  • 34.2. Parsing simple web pages
  • 34.3. Making vertex and edge graphs
  • 34.4. A program to get to philosophy
  • 34.5. When things go wrong
  • 34.6. When we simply don’t “get to philosophy”
• 35. Music basics
  • 35.1. Motivation, prerequisites, plan
  • 35.2. What is sound?
  • 35.3. How is sound generated?
  • 35.4. Measuring and recording
  • 35.5. What is music
  • 35.6. Understanding what we plot in an amplitude plot
  • 35.7. How does the GNU/Linux microphone work?
  • 35.8. Generating your own musical tone
    • 35.8.1. A single tone
    • 35.8.2. From notes to frequencies
  • 35.9. File formats
  • 35.10. Converting our ascii music .dat files to other formats
  • 35.11. Effects filters
• 36. Collecting mp3s
  • 36.1. Purpose: turn audio from youtube into mp3s
  • 36.2. preparation/prerequisites
  • 36.3. Get the video
  • 36.4. Verify that it’s a good video file
  • 36.5. Extracting the audio portion
  • 36.6. Tagging the mp3 file
  • 36.7. A shortcut to the mp3
• 37. Computer art
  • 37.1. Understanding photos and images
    • 37.1.1. Discussion of graphics formats
    • 37.1.2. Photo collection management
    • 37.1.3. Image manipulation: command line and GUI
  • 37.2. metapixel and photomosaics
  • 37.3. ASCII art
  • 37.4. Evolutionary art
  • 37.5. Image manipulation from your own Python program
    • 37.5.1. Geometric transformations
    • 37.5.2. Filters and enhancement
  • 37.6. Topics for further study
• 38. Image filtering
  • 38.1. Motivation, prerequisites, plan
    • 38.1.1. Motivation
    • 38.1.2. Prerequisites
    • 38.1.3. Plan
  • 38.2. Manipulating images with command line programs
  • 38.3. How computers store images, disk and memory
  • 38.4. First example: blurring and other effects with PIL
  • 38.5. The cycle of training and running an AI system
  • 38.6. Miscellaneous examples in various areas
    • 38.6.1. Astronomy example with scipy image kit
    • 38.6.2. Extracting the portion of a scan which has text
    • 38.6.3. Thresholding
  • 38.7. Learning OpenCV
    • 38.7.1. numpy and opencv
    • 38.7.2. Image manipulation with OpenCV
  • 38.8. Using tensorflow with ImageAI to find objects
    • 38.8.1. ImageAI + tensorflow from Fritz AI article
    • 38.8.2. ImageAI + tensorflow from towarddatascience
  • 38.9. Using tensorflow from their own tutorials
    • 38.9.1. The tutorial from tensorflow.org
    • 38.9.2. For more on training the network
    • 38.9.3. The most complete tutorial on preparing training sets and doing the training
• 39. Cryptography
  • 39.1. Preliminary: ASCII values
  • 39.2. Weak crypto
    • 39.2.1. A simple Caesar encryptor
    • 39.2.2. Substitution ciphers
    • 39.2.3. A “literary” substitution cypher
    • 39.2.4. Preparing to attack substitution cyphers: frequency analysis
    • 39.2.5. Applying the frequency analysis to a message
  • 39.3. Strong crypto
    • 39.3.1. Binary numbers, XOR, hiding the byte
    • 39.3.2. Manipulating bits in python
    • 39.3.3. Revisiting random number generators
    • 39.3.4. Implementing tougher encryption
    • 39.3.5. Decrypting this tougher encryption
  • 39.4. Further reading
    • 39.4.1. Technical details
    • 39.4.2. Historical
    • 39.4.3. Videos
• 40. Other languages - Go
  • 40.1. Hello World in Go
  • 40.2. Writing a Go program with command line arguments
  • 40.3. Goroutines and channels
  • 40.4. More complicated Go program
• 41. Appendix: An itinerary for guest lectures
  • 41.1. Motivation for linking computing and scholarship
  • 41.2. A tour of topics
• 42. Appendix: How to build the book
  • 42.1. Motivation, prerequisites, plan
  • 42.2. The tools needed
  • 42.3. Version control: cloning the repository (you only do this once)
  • 42.4. Building the book
  • 42.5. Making and committing changes (your day-to-day)
• 43. Appendix: How to add a chapter
  • 43.1. Anatomy of the chapter
  • 43.2. The chapter: Title
  • 43.3. The chapter: frontmatter
  • 43.4. The chapter: The problem
    • 43.4.1. A basic equation
    • 43.4.2. Some terminology
    • 43.4.3. But wait! Two solutions??
  • 43.5. The chapter: Plots
  • 43.6. The chapter: The quadratic formula
  • 43.7. The chapter: Numerical approximation
  • 43.8. The chapter: Applications
    • 43.8.1. Physics: falling bodies
    • 43.8.2. Geometry: areas and the Dido problem
  • 43.9. Exercises for the chapter
  • 43.10. Further study
• 44. Appendix: Project proposals
  • 44.1. Social sciences
    • 44.1.1. Optimal stopping and life/business
    • 44.1.2. Multi-armed bandits and exploration vs. exploitation
    • 44.1.3. Deadly conflicts
    • 44.1.4. Quantifying overfitting in personal decisions
    • 44.1.5. Just about anything from Gwern Branwen
  • 44.2. Sports
    • 44.2.1. Time series for improvement on records
    • 44.2.2. Optimal tournament structure
    • 44.2.3. Soccer analytics
  • 44.3. Physical sciences
    • 44.3.1. Brownian motion
  • 44.4. Life sciences
    • 44.4.1. Datasets for phylogenetic analysis
    • 44.4.2. Predator-prey ecology: equations and agents
    • 44.4.3. Infectious disease modeling
  • 44.5. Mathematics
    • 44.5.1. puzzles
    • 44.5.2. On the role of intuition in mathematics
    • 44.5.3. Monty Hall’s door problem
    • 44.5.4. Random walks
    • 44.5.5. Runge-kutta method
    • 44.5.6. Sync and the Kuramoto model
    • 44.5.7. Analysis of chess
  • 44.6. Computer science and information technology
    • 44.6.1. Situational awareness of your network
  • 44.7. Humanities and the arts
    • 44.7.1. Music generation: tone beyond sin waves
    • 44.7.2. Music generation: add stereo
    • 44.7.3. Further explorations into Zipf’s law
    • 44.7.4. Analyzing wordle
    • 44.7.5. Can generative AI make art or music with a simple project?
• 45. Appendix: Proposed chapters
  • 45.1. A tour of datasets
  • 45.2. Molecules in three dimensions
  • 45.3. Zeros of polynomials and other functions
  • 45.4. Artificial life
  • 45.5. Genetic algorithms
  • 45.6. Fourier Analysis
  • 45.7. Simpson’s paradox
• 46. Copying and legal matters
  • 46.1. Copyright
  • 46.2. License for the book
  • 46.3. License for code samples
  • 46.4. CC BY-SA 4.0 license
  • 46.5. GNU General Public License

Indices and tables

• Index

• Module Index

• Search Page

[Bac77]

J Backus. Musical note to frequency conversion chart. http://www.audiology.org/sites/default/files/ChasinConversionChart.pdf, 1977. Accessed: 2016-05-06.

[Bir15]

Alistair Bird. Apiological: mathematical speculations about bees (part 3: travelling salesman). http://aperiodical.com/2015/03/apiological-part-3/, 2015. Accessed: 2015-03-19.

[Gal15]

Mark Galassi. Hacking Camp Teacher's Manual. web, 2015.

[Gal16]

Mark Galassi. Scientific Computing for Kids. web, 2016.

[Gra12]

Christopher M Graney. Doubting, testing, and confirming galileo: a translation of giovanni battista riccioli's experiments regarding the motion of a falling body, as reported in his 1651 almagestum novum. arXiv preprint arXiv:1204.3267, 2012.

[Pin11]

Steven Pinker. The better angels of our nature: Why violence has declined. Penguin, 2011.

[RR10]

Carmen M Reinhart and Kenneth S Rogoff. Growth in a time of debt (digest summary). American Economic Review, 100(2):573–578, 2010.

[ZJZ+16]

Caiping Zhang, Jiuchun Jiang, Linjing Zhang, Sijia Liu, Leyi Wang, and Poh Chiang Loh. A generalized soc-ocv model for lithium-ion batteries and the soc estimation for lnmco battery. Energies, 9(11):900, 2016.

Copyright (C)  2017-2022  Mark Galassi, Leina Gries, Sophia
Mulholland, Almond Heil.

This work is licensed under the Creative Commons
Attribution-ShareAlike 4.0 International License. To view a copy
of this license, visit
http://creativecommons.org/licenses/by-sa/4.0/ or send a letter to
Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.