[1] 1
[1] 2
[1] 3
[1] 4
[1] 5Speedy R:
loops, parallelization, and the cloud
For loops (II) – store results
hist(bootstrapped_means)
Functional loops (I)
Loop limitations (I)
- Loops are slower than native vectorized functions
v1 <- rnorm(1000000); v2 <- rnorm(1000000)system.time(v1 + v2) user system elapsed
0.002 0.000 0.002 system.time(lapply(seq_along(v1), function(i){sum(v1[i], v2[i])})) user system elapsed
0.792 0.016 0.809 lapply()always outputs a list. For more flexibility and other goodies, check out thepurrrpackage from thetidyverse.
The idea behind parallelization
Modern computers have multiple threads in which to compute operations (laptops usually have 8–16)
By default, R uses only one thread…
So multiple solutions have been created to run R in parallel
Any set of independent operations can be parallelized
Source: Athena on Pexels, 20194
Cloud providers
- The most well-known providers are Amazon Web Services (AWS) and Google Cloud Platform (GCP)
- Today we’ll use a smaller provider, 🌊DigitalOcean (signup for students via GitHub; for non-students)
- Pros: easy to use
- Cons: fewer available configs; no GPUs
Source: Mark Drake, 20196
Droplet setup (I)
- “Spin up a Droplet” or “Droplets” > “Create Droplet”
Droplet setup (II)
Droplet setup (III)
Droplet setup (IV)
- “Create Droplet” and wait 1-2mins.
In the console, type
adduser rstudio, choose a password, and type Enter and Y to confirmIn your web browser, go to
<ip>:8787(e.g.,165.227.82.108:8787) and log in with username “rstudio” and your newly-created password
RStudio server
We can run code, install packages, and do pretty much anything that we could do on the desktop
To upload or download files, use the bottom-right “Files” panel
- Uploading .zip files will uncompress the folder automatically!
To terminate the droplet, go to its control panel and “Destroy”>“Destroy Droplet”
Cloud conundrums
If you work with sensitive data, you might want to access a cloud provider via UT.
💰 Remember to terminate your virtual machines! 💰
- Activate billing alerts. On DO it’s under “Billing”
![]()
- Activate billing alerts. On DO it’s under “Billing”
You can host your own Shiny server on the cloud (see tutorial by Saskia A. Otto)
Footnotes
Allison Horst. 2019. Illustrations from Hadley Wickham’s ACM talk “The Joy of Functional Programming (for Data Science).”
Hadley Wickham, 2019. The Joy of Functional Programming (for Data Science)
Juggler on Wikimedia Commons, 2009. Passing 2jugglers 6balls PPSS side.gif
Athena on Pexels, 2019. Computer processor
NASA on Wikimedia Commons, 2017. PIA21474-CrabNebula-5Observatories-Animation.gif
Mark Drake, 2019. How To Probe the Depths of Nautically-Themed Open-Source Projects Using Moby Dick
