Serious Programming - small courses

Date:

Aug 17, 2025

Author:

Mark Galassi <mark@galassi.org>

Author:

Leina Gries <leinagries@gmail.com>

Author:

Sophia Mulholland <smulholland505@gmail.com>

Author:

Almond Heil <almondheil@gmail.com>

Contents:

• 1. Motivation and plan
  • 1.1. Notes for teachers
  • 1.2. Acknowledgements
  • 1.3. Status of the book
  • 1.4. Footnotes
• 2. Starting out - data files and first plots
  • 2.1. Motivation, prerequisites, plan
  • 2.2. Very first data plots with gnuplot
  • 2.3. Plotting functions with gnuplot
  • 2.4. Reading and writing files, in brief
  • 2.5. Generating our own data to plot
  • 2.6. The broad landscape of plotting software
  • 2.7. Data formats
  • 2.8. Simple surface plots
  • 2.9. Topics we have covered
• 3. Intermediate plotting
  • 3.1. A worked example
  • 3.2. Histograms
  • 3.3. Matplotlib
  • 3.4. A histogram snippet to conclude
• 4. A tour of functions
  • 4.1. Motivation, Prerequisites, Plan
  • 4.2. Linear Functions
  • 4.3. Polynomials
    • 4.3.1. Derivatives
  • 4.4. Higher order polynomials
  • 4.5. Inverse functions
  • 4.6. Elementary transcendental functions
    • 4.6.1. Exponentials
    • 4.6.2. Trigonometric function
  • 4.7. Gaussian distribution
• 5. Advanced plotting
  • 5.1. Our setup
  • 5.2. A first line plot
  • 5.3. Polishing line plots
  • 5.4. Increasing the challenge: parametrized solutions
  • 5.5. Adding a dimension: surface plots
  • 5.6. Adding a dimension: color
    • 5.6.1. Spetrograms
    • 5.6.2. Spectrograms for standard acoustic files
    • 5.6.3. Ideas for further exploration
  • 5.7. Loops in gnuplot
  • 5.8. Adding a dimension: animation
    • 5.8.1. Animation in gnuplot
    • 5.8.2. Animation in matplotlib
  • 5.9. Further reading
• 6. Fitting functions to data
  • 6.1. Motivation, Prerequisites, Plan
  • 6.2. Examples to get started
  • 6.3. Straight line fits
    • 6.3.1. Our goal
    • 6.3.2. Stepping back: just two points
    • 6.3.3. Let’s plot that line with our data
    • 6.3.4. Physical interpretation of the line fit
  • 6.4. Proper line fitting
  • 6.5. Using Python’s scientific libraries to fit lines
  • 6.6. When to not try a linear fit
  • 6.7. Fitting curves
    • 6.7.1. Polynomial fits
    • 6.7.2. Overfitting
    • 6.7.3. Fitting arbitrary functions
  • 6.8. Topics for further study
    • 6.8.1. Interpolation and extrapolation
    • 6.8.2. How high should the degree of the polynomial be?
• 7. Case studies in data
  • 7.1. Population data from the web
    • 7.1.1. Exercises
• 8. Special numbers: \(\pi\)
  • 8.1. Motivation, prerequisites, plan
    • 8.1.1. Motivation
    • 8.1.2. Prerequisites
    • 8.1.3. Plan
  • 8.2. A collection of factoids
  • 8.3. Calculating \(\pi\): ancient history
  • 8.4. Calculating \(\pi\): monte carlo method
  • 8.5. Calculating \(\pi\): series that converge to \(\pi\)
    • 8.5.1. Madhava-Leibniz series
    • 8.5.2. “Efficient” infinite series
    • 8.5.3. Formulae based on the Riemann zeta function
  • 8.6. Relationships between special numbers
• 9. A workshop on programming by yourself (!)
• 10. Random number basics
  • 10.1. Prerequisites
  • 10.2. Motivation
  • 10.3. Types of distributions
  • 10.4. Further reading
• 11. Randomness and Disorder
  • 11.1. Experiment: burn a match
  • 11.2. Experiment: ink in water
  • 11.3. Discussion on “ink in water” experiment
  • 11.4. Flipping a single coin
  • 11.5. Review: random numbers in Pythyon
  • 11.6. Experiment: flipping a single virtual coin
    • 11.6.1. Just the flips
    • 11.6.2. Long-running average of single coin flips
  • 11.7. Flipping multiple coins
  • 11.8. Experiment: flipping virtual coins
  • 11.9. Experiment: back to physical coins - disorder
  • 11.10. The drunk fencer
  • 11.11. The drunkard’s walk
  • 11.12. Matplotlib animation of a random walk
  • 11.13. Making a movie from walk frames
    • 11.13.1. Reviewing graphics and animation
    • 11.13.2. Making individual frames of the random walk
  • 11.14. Discussion
  • 11.15. Further reading and videos
• 12. Random Processes
  • 12.1. Motivation, prerequisites, plan
  • 12.2. Reviewing random number generation
  • 12.3. Poisson processes
    • 12.3.1. A pure poisson process
    • 12.3.2. An angry lightning goddess
    • 12.3.3. Vicious glow-worms
  • 12.4. Brownian motion
  • 12.5. Further reading
  • 12.6. Progression of record peaks
  • 12.7. The gambler’s fallacy
  • 12.8. The gambler’s ruin
  • 12.9. Link to other chapters
• 13. Power laws, Zipf, Benford, …
  • 13.1. Motivation, prerequisites, plan
  • 13.2. A brief refresher on log-log plots
  • 13.3. Zipf’s law
    • 13.3.1. Example from a paragraph of your own.
    • 13.3.2. Example from Project Gutenberg
    • 13.3.3. Explanations of Zipf’s law
  • 13.4. What are “power laws”?
    • 13.4.1. Calculating the power from the data
  • 13.5. Deadly conflicts
  • 13.6. Benford’s law
    • 13.6.1. Physical constants
    • 13.6.2. Stock quotes
    • 13.6.3. Further reading related to Benford’s law
  • 13.7. Pareto’s principle
    • 13.7.1. Further reading related to Pareto’s principle
  • 13.8. Olber’s paradox
• 14. Pushing toward calculus
  • 14.1. Motivation and plan
  • 14.2. Prerequisites
  • 14.3. Limits, the infinitely big, and the infinitesimally small
  • 14.4. Continuous functions
  • 14.5. Convergence and divergence
  • 14.6. Weird mixes
  • 14.7. Limits of some functions
  • 14.8. The limit of a series
  • 14.9. The most important application: derivatives
  • 14.10. Visualizing derivatives with an animation
• 15. Numerical integration
  • 15.1. The integral
  • 15.2. Calculating the integral numerically
  • 15.3. Improving the numerical approximation
  • 15.4. Stepping back from numerical integration back to analytical work
• 16. Differential Equations
  • 16.1. Motivation, Prerequisites, Plan
  • 16.2. Derivatives
    • 16.2.1. Why?
  • 16.3. Definitions
  • 16.4. An Example
  • 16.5. Population Growth
    • 16.5.1. But what does the derivative tell us?
  • 16.6. Euler’s Method
  • 16.7. Compiling and running C programs
  • 16.8. A program to use Euler’s method
  • 16.9. Second Order Differential equations
  • 16.10. Falling Body
    • 16.10.1. Adding Air Resistance to Falling Body
  • 16.11. The harmonic oscillator
    • 16.11.1. The simple harmonic oscillator
    • 16.11.2. The damped harmonic oscillator
    • 16.11.3. The non-linear pendulum
• 17. Ecology
  • 17.1. Motivation, Prerequisites, Plan
  • 17.2. Factors that come up in modeling population ecology
  • 17.3. Exponential growth
    • 17.3.1. Unlimited
    • 17.3.2. Resource-constrained - r-selection versus K-selection
  • 17.4. History of the human population on earth
  • 17.5. Predator-prey systems and the Lotka-Volterra differential equations
    • 17.5.1. The equations
    • 17.5.2. Solving the equations
  • 17.6. Further reading
• 18. Biology – phylogeny
  • 18.1. Motivation, prerequisites, plan
    • 18.1.1. Motivation
    • 18.1.2. Prerequisites
    • 18.1.3. Plan
  • 18.2. Start with a video and then make a simple table
  • 18.3. Terminology
  • 18.4. First steps with biopython
  • 18.5. Downloading the datasets
  • 18.6. Preparing a tree by hand
  • 18.7. Inferring a tree
  • 18.8. Other sequence analysis resources
  • 18.9. Linguistics datasets
    • 18.9.1. lingpy.org
    • 18.9.2. elinguistics.com
    • 18.9.3. Others
  • 18.10. Evolution of programming languages
• 19. Recursion
  • 19.1. Motivation, prerequisites, plan
  • 19.2. Visual examples
  • 19.3. Word examples
  • 19.4. Components of a recursive definition
    • 19.4.1. Simple math
    • 19.4.2. Programming simple math recursion
    • 19.4.3. Recursion with data structures
  • 19.5. Visualizing what the recursion is doing
  • 19.6. Towers of Hanoi
  • 19.7. Should we really use recursion in programming?
• 20. Programming topics: sorting
  • 20.1. Motivation, prerequisites, plan
  • 20.2. Experiment: a game of cards
  • 20.3. Intuition to algorithm on card sorting
  • 20.4. Writing up the algorithm in Python
  • 20.5. Profiling the algorithm
    • 20.5.1. Modify the program to print information
    • 20.5.2. Run the program and make plots
    • 20.5.3. How do we understand these plots?
    • 20.5.4. Exercises
  • 20.6. Computational complexity
  • 20.7. Further reading
• 21. Birthday paradox
  • 21.1. To get started
  • 21.2. A practical demonstration
  • 21.3. The theory
  • 21.4. Take-home
• 22. Optimal Stopping
  • 22.1. Motivation, Prerequisites, and Plan
  • 22.2. What is Optimal Stopping?
  • 22.3. The Employee Problem
  • 22.4. A (marginally) More Useful Application: Parking
  • 22.5. Conclusion
  • 22.6. Further Reading
• 23. Graphical user interfaces
  • 23.1. A chat about sources of input in a GUI
  • 23.2. Widgets and widget sets
  • 23.3. The simplest programs
    • 23.3.1. The programs
    • 23.3.2. Packers: more than just one button
    • 23.3.3. A tour of widgets
  • 23.4. Following a tutorial
  • 23.5. Cellular automata on a canvas
    • 23.5.1. A simply drawing of the CA
    • 23.5.2. Adding controls to the program
  • 23.6. Conway’s game of life
  • 23.7. Tic-tac-toe with buttons
  • 23.8. A glance at PySimpleGUI
  • 23.9. Other resources
• 24. Drawing on a canvas
  • 24.1. Simplest canvas
  • 24.2. Simplest animation
    • 24.2.1. Exercises
• 25. The Traveling Salesman
  • 25.1. Cities and path lengths
  • 25.2. Solving the Traveling Salesman Problem
    • 25.2.1. A digression on optimization
    • 25.2.2. Generating and visualizing lists of cities
    • 25.2.3. Animating the drawing of cities
  • 25.3. Improvements to the route
    • 25.3.1. Interesting video resources before you start
    • 25.3.2. Impossibile to compute the optimal solution
    • 25.3.3. Greedy algorithm
    • 25.3.4. A digression on hill climbing
    • 25.3.5. Hill climbing for the traveling salesman problem
    • 25.3.6. Further study
  • 25.4. Where do you go from search
• 26. Basic agent-based modeling
  • 26.1. Motivation, Prerequisites, and Plan
  • 26.2. Conceptualizing the model
    • 26.2.1. Agent-Based Modeling Concepts
    • 26.2.2. Object-Oriented Programming Concepts
  • 26.3. Classes and steps
  • 26.4. Space and movement
  • 26.5. Visualization
  • 26.6. Interactions between agents
  • 26.7. The final source code
  • 26.8. Making an SIR model
  • 26.9. Further reading
• 27. Emergent behavior
  • 27.1. Motivation, prerequisites, plan
  • 27.2. Before you start
  • 27.3. Write the simple_ca.py program which implements a cellular automaton
  • 27.4. Conway’s game of life
  • 27.5. Install and run the golly program
  • 27.6. Further study
    • 27.6.1. Play with the simple_ca.py program
  • 27.7. Further reading
• 28. Web scraping
  • 28.1. Motivation, prerequisites, plan
  • 28.2. What does a web page look like underneath? (HTML)
  • 28.3. Command line scraping with wget
    • 28.3.1. First download with wget
    • 28.3.2. Simple analysis of the drinks.csv file
    • 28.3.3. Looking at birth data
  • 28.4. Scraping from a Python program
    • 28.4.1. Brief interlude on string manipulation
    • 28.4.2. The birth data from Python
  • 28.5. Finding neat scientific data sets
    • 28.5.1. Time histories
    • 28.5.2. Images
  • 28.6. Beautiful Soup
• 29. Getting to philosophy
  • 29.1. Motivation, prerequisites, plan
  • 29.2. Parsing simple web pages
  • 29.3. Making vertex and edge graphs
  • 29.4. A program to get to philosophy
  • 29.5. When things go wrong
  • 29.6. When we simply don’t “get to philosophy”
• 30. Music basics
  • 30.1. Motivation, prerequisites, plan
  • 30.2. What is sound?
  • 30.3. How is sound generated?
  • 30.4. Measuring and recording
  • 30.5. What is music
  • 30.6. Understanding what we plot in an amplitude plot
  • 30.7. How does the GNU/Linux microphone work?
  • 30.8. Generating your own musical tone
    • 30.8.1. A single tone
    • 30.8.2. From notes to frequencies
  • 30.9. File formats
  • 30.10. Converting our ascii music .dat files to other formats
  • 30.11. Effects filters
• 31. Collecting mp3s
  • 31.1. Purpose: turn audio from youtube into mp3s
  • 31.2. preparation/prerequisites
  • 31.3. Get the video
  • 31.4. Verify that it’s a good video file
  • 31.5. Extracting the audio portion
  • 31.6. Tagging the mp3 file
  • 31.7. A shortcut to the mp3
• 32. Computer art
  • 32.1. Understanding photos and images
    • 32.1.1. Discussion of graphics formats
    • 32.1.2. Photo collection management
    • 32.1.3. Image manipulation: command line and GUI
  • 32.2. metapixel and photomosaics
  • 32.3. ASCII art
  • 32.4. Evolutionary art
  • 32.5. Image manipulation from your own Python program
    • 32.5.1. Geometric transformations
    • 32.5.2. Filters and enhancement
  • 32.6. Topics for further study
• 33. Image filtering
  • 33.1. Motivation, prerequisites, plan
    • 33.1.1. Motivation
    • 33.1.2. Prerequisites
    • 33.1.3. Plan
  • 33.2. Manipulating images with command line programs
  • 33.3. How computers store images, disk and memory
  • 33.4. First example: blurring and other effects with PIL
  • 33.5. The cycle of training and running an AI system
  • 33.6. Miscellaneous examples in various areas
    • 33.6.1. Astronomy example with scipy image kit
    • 33.6.2. Extracting the portion of a scan which has text
    • 33.6.3. Thresholding
  • 33.7. Learning OpenCV
    • 33.7.1. numpy and opencv
    • 33.7.2. Image manipulation with OpenCV
  • 33.8. Using tensorflow with ImageAI to find objects
    • 33.8.1. ImageAI + tensorflow from Fritz AI article
    • 33.8.2. ImageAI + tensorflow from towarddatascience
  • 33.9. Using tensorflow from their own tutorials
    • 33.9.1. The tutorial from tensorflow.org
    • 33.9.2. For more on training the network
    • 33.9.3. The most complete tutorial on preparing training sets and doing the training
• 34. Cryptography
  • 34.1. Preliminary: ASCII values
  • 34.2. Weak crypto
    • 34.2.1. A simple Caesar encryptor
    • 34.2.2. Substitution ciphers
    • 34.2.3. A “literary” substitution cypher
    • 34.2.4. Preparing to attack substitution cyphers: frequency analysis
    • 34.2.5. Applying the frequency analysis to a message
  • 34.3. Strong crypto
    • 34.3.1. Binary numbers, XOR, hiding the byte
    • 34.3.2. Manipulating bits in python
    • 34.3.3. Revisiting random number generators
    • 34.3.4. Implementing tougher encryption
    • 34.3.5. Decrypting this tougher encryption
  • 34.4. Further reading
    • 34.4.1. Technical details
    • 34.4.2. Historical
    • 34.4.3. Videos
• 35. Other languages - Go
  • 35.1. Hello World in Go
  • 35.2. Writing a Go program with command line arguments
  • 35.3. Goroutines and channels
  • 35.4. More complicated Go program
• 36. Appendix: An itinerary for guest lectures
  • 36.1. Motivation for linking computing and scholarship
  • 36.2. A tour of topics
• 37. Appendix: How to build the book
  • 37.1. Motivation, prerequisites, plan
  • 37.2. The tools needed
  • 37.3. Version control: cloning the repository (you only do this once)
  • 37.4. Building the book
  • 37.5. Making and committing changes (your day-to-day)
• 38. Appendix: How to add a chapter
  • 38.1. Anatomy of the chapter
  • 38.2. The chapter: Title
  • 38.3. The chapter: frontmatter
  • 38.4. The chapter: The problem
    • 38.4.1. A basic equation
    • 38.4.2. Some terminology
    • 38.4.3. But wait! Two solutions??
  • 38.5. The chapter: Plots
  • 38.6. The chapter: The quadratic formula
  • 38.7. The chapter: Numerical approximation
  • 38.8. The chapter: Applications
    • 38.8.1. Physics: falling bodies
    • 38.8.2. Geometry: areas and the Dido problem
  • 38.9. Exercises for the chapter
  • 38.10. Further study
• 39. Appendix: Project proposals
  • 39.1. Social sciences
    • 39.1.1. Optimal stopping and life/business
    • 39.1.2. Multi-armed bandits and exploration vs. exploitation
    • 39.1.3. Deadly conflicts
    • 39.1.4. Quantifying overfitting in personal decisions
    • 39.1.5. Just about anything from Gwern Branwen
  • 39.2. Sports
    • 39.2.1. Time series for improvement on records
    • 39.2.2. Optimal tournament structure
    • 39.2.3. Soccer analytics
  • 39.3. Physical sciences
    • 39.3.1. Brownian motion
  • 39.4. Life sciences
    • 39.4.1. Datasets for phylogenetic analysis
    • 39.4.2. Predator-prey ecology: equations and agents
    • 39.4.3. Infectious disease modeling
  • 39.5. Mathematics
    • 39.5.1. puzzles
    • 39.5.2. On the role of intuition in mathematics
    • 39.5.3. Monty Hall’s door problem
    • 39.5.4. Random walks
    • 39.5.5. Runge-kutta method
    • 39.5.6. Sync and the Kuramoto model
    • 39.5.7. Analysis of chess
  • 39.6. Computer science and information technology
    • 39.6.1. Situational awareness of your network
  • 39.7. Humanities and the arts
    • 39.7.1. Music generation: tone beyond sin waves
    • 39.7.2. Music generation: add stereo
    • 39.7.3. Further explorations into Zipf’s law
    • 39.7.4. Analyzing wordle
    • 39.7.5. Can generative AI make art or music with a simple project?
• 40. Appendix: Proposed chapters
  • 40.1. A tour of datasets
  • 40.2. Molecules in three dimensions
  • 40.3. Zeros of polynomials and other functions
  • 40.4. Artificial life
  • 40.5. Genetic algorithms
  • 40.6. Fourier Analysis
  • 40.7. Simpson’s paradox
• 41. Copying and legal matters
  • 41.1. Copyright
  • 41.2. License for the book
  • 41.3. License for code samples
  • 41.4. CC BY-SA 4.0 license
  • 41.5. GNU General Public License

Indices and tables

• Index

Glossary

population ecology

The study of how populations of a species or group change over time.

agent based models

Virtual computer organisms that follow a certain logic in interacting with others in a computer simulation.

Bibliography

[Bac77]

J Backus. Musical note to frequency conversion chart. http://www.audiology.org/sites/default/files/ChasinConversionChart.pdf, 1977. Accessed: 2016-05-06.

[Bir15]

Alistair Bird. Apiological: mathematical speculations about bees (part 3: travelling salesman). http://aperiodical.com/2015/03/apiological-part-3/, 2015. Accessed: 2015-03-19.

[Gal15]

Mark Galassi. Serious Programming Teacher's Manual. web, 2015.

[Gra12]

Christopher M Graney. Doubting, testing, and confirming galileo: a translation of giovanni battista riccioli's experiments regarding the motion of a falling body, as reported in his 1651 almagestum novum. arXiv preprint arXiv:1204.3267, 2012.

[Pin11]

Steven Pinker. The better angels of our nature: Why violence has declined. Penguin, 2011.

[RR10]

Carmen M Reinhart and Kenneth S Rogoff. Growth in a time of debt (digest summary). American Economic Review, 100(2):573–578, 2010.

[ZJZ+16]

Caiping Zhang, Jiuchun Jiang, Linjing Zhang, Sijia Liu, Leyi Wang, and Poh Chiang Loh. A generalized soc-ocv model for lithium-ion batteries and the soc estimation for lnmco battery. Energies, 9(11):900, 2016.

Copyright (C)  2017-2022  Mark Galassi, Leina Gries, Sophia
Mulholland, Almond Heil.

This work is licensed under the Creative Commons
Attribution-ShareAlike 4.0 International License. To view a copy
of this license, visit
http://creativecommons.org/licenses/by-sa/4.0/ or send a letter to
Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.