Preface
After enduring a two-hour livestream deep-dive into the chemistry of gluten networks covering protein cross-linking and hydration dynamics, the home baker finally closes their browser. They’ve nodded along through every molecular diagram, but by the end their eyes are glazing over. When the presenter wraps up, the baker leans forward and asks, “Great talk…but what’s the exact ratio of flour to water I should use so my sourdough doesn’t collapse?”
While this may fall short as a critical review, who hasn’t wanted to skip straight to the good part? Sometimes you don’t need to master biochemical pathways to get a perfect loaf, just as you don’t need to understand aerodynamics to ride a bicycle. In baking (and in life), having that one simple formula can be all you really need.
As a case in point, there is now a large literature on parallel random number generation. But, you don’t need to understand all this literature to be able to use the methodology it develops. This book is written from that perspective. Specifically, we try to provide some “formulas” that can be used in practice to speed up your simulation experiments.
Our intended audience consists of data scientists with limited computer resources: either by necessity or choice. These are people actively writing code to support their research, where the research, not the supporting code, is the endpoint they care about. The assumption is that they have a personal computer (desktop or laptop machine) running macOS, Linux, or Windows and that they know how to use it from the command line or a familiar IDE.
To get the most from this book you will want some Statistics in your back pocket. Comfort with the idea of a random sample, the normal and uniform distributions, and standard estimation methods such as least-squares is enough. If you are newer than that the examples in Chapter 2 still stand on their own. They show what simulation can do.
In terms of code, we restrict attention to two languages: R and Python. For both we recommend Positron, a modern IDE that handles R and Python equally well. RStudio remains an excellent choice for R and IDLE is a fine, lightweight option for Python. If you reach for the reticulate package to run Python in RStudio, be aware that at the time of this writing it experiences some friction pairing with Python’s multiprocessing module.
This book pulls together ideas and methods for parallel simulation that are otherwise scattered across the literature and online into one practical resource. Our hope is that it serves you well as a springboard for your own more creative work.
How To Read This Book
The book is short and meant to be read from start to finish; but, it is built so you can take a shorter path. Chapter 1 sets up the problem, and Chapter 2 covers the ideas both languages share: random number generation, the quantile transformation and why the use of parallel random number streams requires careful planning. Read these two since everything after them builds on these foundations. From there the book splits by language. Chapters 3 and 5 are the R track: the parallel formula followed by a full case study. Chapters 4 and 6 are the Python track.
The R and Python tracks stand on their own in the sense that each can be read without the other. For example, a reading path for an R user might go from Chapter 2 to Chapters 3 and 5. A Python coder might instead read Chapter 2 and then Chapters 4 and 6.
Chapter 7 collects operational details: counting cores, watching memory, and benchmarking. It is worth knowing, but you can return to it when those questions come up. Chapter 8 looks past your computer’s CPU to GPUs, the cloud, and clusters. Skip it until you outgrow the basic computing ability your laptop provides. Chapter 9 is a brief recap. The appendices are optional. Appendix B and Appendix C explain how random numbers are produced; you can skip both with no practical loss. Appendix A is the exception. There is a system for running our code that emerges here. So, skim it before Chapter 3 or 4, particularly if you are not yet comfortable running code from an IDE.
What you’ll need. Any reasonably modern laptop or desktop will suffice to run the programs in this book. All core examples run on an ordinary multicore machine. Only the optional GPU example requires specialized hardware.
On the software side you need R or Python, whichever you work in, plus a handful of packages we install once using the setup script described below under “Accompanying code.” The code runs on macOS, Linux, and Windows alike. Windows users in particular need not worry. R’s parallel package defaults to so-called PSOCK clusters, which spin up independent worker processes that behave the same way on every operating system, so none of our examples depend on Unix-only features.
Accompanying code. All of the code in this book, our two parallel templates sim_template.R and sim_template.py, and the worker scripts for each chapter’s examples live in an accompanying GitHub repository, https://github.com/ismayc/parallel-simulation-book. We recommend cloning it before reading Chapters 3 and 4 so that the listings in the text are sitting in front of you ready to run. The repository also contains a short setup script that installs the R and Python packages we depend on; do that once before you start so that you’re not waiting on installs mid-experiment.