Posts | arnie seong

Intro to datagoboop: pitch & playback

Mon, 18 Jan 2021 00:00:00 +0000

Introducing `datagoboop`

datagoboop is an R package for data sonification, or the audio version of data visualization. This project started off while I was dreaming of representing data as a symphony — our ears can with some training pick out each instrument in a symphony orchestra (maybe 15-20 instruments), whereas representing more than 4 or 5 variables in one plot is usually quite chaotic.

I asked Sebastian Waz and Jaylen Lee to join me in putting together this first pass at an R package for data sonification. While we haven’t quite gotten to symphonic sound — turns out a library of high-quality audio files gets large fast, so most output from datagoboop sounds more like 8-bit video game music 😜 — we still think it sounds pretty neat!

Why `datagoboop`

There are a few other data sonification packages for R, such as

(audiolyzr in particular sounds pretty neat!)

The main difference between these packages and datagoboop is that datagoboop includes several built-in sonification versions of common plots, but also allows users to build audio plots piece by piece by providing a higher level of control and modular functions.

Setup

if (!requireNamespace("devtools")) install.packages('devtools')
devtools::install_github("akseong/datagoboop")

(You can also see the repo here)

Ground-level sonification

Pitch

Pitch in datagoboop is parameterized as idealized piano key numbers, where the pitch frequency (hz) is given as a function of the piano key number $n$ by $$f(n) = 2^{\big( \frac{n - 49}{12}\big)} \times 440$$

The Wikipedia page for piano key frequencies is a useful reference.

Note that datagoboop’s implementation uses the equation above, so the (seemingly arbitrary) numbering for the extended keys on the Wikipedia page will not correspond correctly.

functions: `cplay()`, `wplay()`

To actually play audio, you’ll need to use either

cplay(): audio playback with console controls (“control playback”)
wplay(): audio playback better suited for .Rmd documents knitted to HTML.
- in console, wplay() institutes a system timeout during playback.
  - use console controls instead: argument wplay_controls = TRUE
  - suppress playback completely: argument wplayback = FALSE
- when knitting to html, wplay() saves the audio output as a .wav (named with timestamp), in a folder with the same name as your .Rmd file.

Set the optional arguments wplay_controls and wplayback for your entire R session by creating new variables with the same names.

For example, you can just create a new variable by executing wplayback <- FALSE to suppress playback globally from calls to wplay(), for example when you are executing multiple code chunks.

Just set it to TRUE when you want to hear it again!

function: `note()`

library(datagoboop)

midC <- note(40)
wplay(
  midC, 
  file_path = "index.en.Rmark/midC.wav"
  )

note() is the most basic function in datagoboop — most other functions in datagoboop are built on top of it. The only required argument is a pkey (piano key); output is a .wav-format vector.

The optional arguments in note() are worth spending a moment on, since they appear frequently in datagoboop functions.

optional arguments and defaults:
- vol = 1, volume (values from 0 to 1);
- dur = 1, duration in seconds;
- fs = 44100, sampling rate (varies by speaker hardware/audio driver);
- inst_lab = "sine", instrument type
  - "piano", "string", "wind", "bass", are built in, though as mentioned they sound like an 8-bit piano, string, flute, or bass
  - You can also add sounds using add_inst() if you don’t like our 8-bit sounds, then specify its label here;
- decay = FALSE, exponential decay of sound;
- decay_rate = 6, sound decay rate if decay = TRUE

midC_bounce <- note(
  pkey = 40, 
  vol = 1, 
  dur = 2, 
  inst_lab = "piano",
  decay = TRUE,
  decay_rate = 6
  )

wplay(
  midC_bounce, 
  file_path = "index.en.Rmark/midC_bounce.wav"
  )

functions: `notes()` & `chords()`

There is also a vectorized version notes() ….

pkeys <- 52:64      # piano keys for a 1 octave chromatic scale
vols <- 1:13 / 13   # increasing volume
durs <- 12:24 / 96  # slowing down

chrom <- notes(
  pkey = pkeys, 
  vol = vols, 
  dur = durs,
  decay = TRUE,
  progbar = FALSE
  )

wplay(
  chrom,
  file_path = "index.en.Rmark/chrom.wav"
  )

…. and a single chord function chord()

chord_prog <- c(
  chord(pkey = c(50, 54, 57), decay = TRUE), 
  chord(pkey = c(50, 55, 59), decay = TRUE), 
  chord(pkey = c(50, 54, 57), decay = TRUE), 
  chord(pkey = c(49, 55, 57), decay = TRUE), 
  chord(pkey = c(50, 54, 57), decay = TRUE)
)

wplay(
  chord_prog,
  file_path = "index.en.Rmark/chords.wav"
)

Example: audio histogram / violinplot

One of the first things we did after creating some of the basic playback functions was to try making an audio version of a histogram. A little modification turns it into something like a violinplot, which I think may convey the general shape a little more clearly.

This is pretty different from the built-in function sonify_hist() below, but the idea is to walk through construction of an audioplot using datagoboop’s ground-level functions.

Overall, we’re going to

x-axis $\mapsto$ pitch: map histogram breakpoints to pitch;
y-axis $\mapsto$ duration: map counts/bar heights to duration;
y-axis $\mapsto$ volume: scale volume to counts/bar heights.

I think it’s a bit clearer when we start off hearing the highest count bars, and then allow the lower count bars to gradually emerge (so we’ll do that 👍).

Audio Histogram

First, we generate the data, and create a histogram. We’ll specify the breaks manually (16 breaks = 15 bins), and also store the histogram’s info.

set.seed(2)
y <- rnorm(1000, 40, 5)
breaks = seq(
  floor(min(y)), 
  floor(max(y)) + 1, 
  length.out = 16
  )
hist_info <- hist(y, breaks = breaks)

Next, we’ll select a C-major scale to map the breaks to.

major_scale() generates a vector of piano key numbers corresponding to a major scale starting at the specified tonic and number of octaves.
The second line selects every other note in order to avoid dissonance. (There will still be some!)

Cmajor <- major_scale(tonic_pkey = 40, n_octaves = 4)
pkeys <- Cmajor[2 * 1:length(breaks)]

Now we need to construct each “bar” of the audio histogram — each bar gets a note. Some calculations are required, since we need to fill in each note with silence to start, then the pitch of the note for a duration that expresses bar height.

We use the sampling rate and desired total duration to calculate total length of each note (each note is a .wav vector, playable on its own).

fs <- 44100
total_dur <- 8
wave_length <- total_dur*fs

# scale the durations and volume to reflect counts
# i.e. the longest note is the bin with the highest counts
scaling <- hist_info$counts / max(hist_info$counts)
vols <- scaling + .25
durs <- scaling * total_dur
fill_durs <- wave_length - durs*fs

Representing variables using volume can be a bit tricky, because volume is a combination of different sound wave characteristics — not just amplitude. For example, higher frequencies/pitches sound louder than lower frequencies with the same amplitude.

In the code above, I’ve also added a small nugget + 0.25 so that the volume doesn’t get too quiet.

Last, we construct each note in a loop, and sum each note’s sound wave together to construct the audio histogram’s sound wave.

audiohist <- rep(0, wave_length)
for (i in 1:length(durs)){
  wave_i <- c(
    # notes begin with silence
    rep(0, fill_durs[i]),
    # audible portion of note 
    note(pkey = pkeys[i], 
         dur = durs[i], 
         vol = vols[i])
    )
  audiohist <- audiohist + wave_i[1:wave_length]
}

Summing note vectors to construct the sound wave is OK even though the vectors will exceed [-1, 1] because wplay() normalizes the .wav vector prior to save/playback.

The same histogram is shown again below.

wplay(
  audiohist, 
  file_path = "index.en.Rmark/hist.wav"
  )

Audio Violinplot

I like the overall effect of the histogram — sort of like an island emerging out of the water. But the tails are very hard to hear. So, let’s try a violin plot, which we can do simply by reversing the sound vector and placing it back to back.

wplay(
  c(audiohist, rev(audiohist)), 
  file_path = "index.en.Rmark/violin.wav"
  )

(A small happy accident if you listen closely: around 5 seconds in, you can hear a progression in there that sounds — just for a moment — like the opening to Debussy’s La Fille Aux Cheveux De Lin)

The optional argument file_path is specified in calls to wplay() in this post in order to place the .wav file in the folder containing files for this post (see the previous note).

function: `sonify_hist()`

datagoboop also has a built-in audio histogram function:

counts/bar height $\mapsto$ pitch
x-axis $\mapsto$ time
static bursts mark quartiles

# requires dataframe as input
df <- data.frame(y)
wplay(
  sonify_hist(
    data = df, 
    var = "y", 
    breaks = 16, 
    duration = 5
    ),
  file_path = "index.en.Rmark/sonifyhist.wav"
  )

Next up: built-in audio plot analogues

Next post will cover some of the basic plot types built-into datagoboop, for example, a scatterplot, in which the lower register indicates rising values in the $x$-variable, and the higher register indicates the value of the $y$-variable.

library(ggplot2)
ggplot(
  mtcars, 
  aes(
    y=disp, 
    x = mpg, 
    size = factor(gear),
    color = factor(gear)
    )
  ) + 
  geom_point(alpha = 0.4) + 
  labs(title = "mtcars")

mtcars_audio <- sonify_scatter(
  data = mtcars,
  x_var = "mpg",
  y_var = "disp",
  factor_var = "gear",
  volume = "gear",
  vol_scaling = "binned 3 exp 2"
  )
# output suppressed; shows progress bar in console

wplay(
  mtcars_audio, 
  file_path = "index.en.Rmark/mtc.wav"
  )

A small parting gift

bp2 <- bach()

wplay(
  bp2,
  file_path = "index.en.Rmark/bach.wav"
  )

Making a website with blogdown & Academic Theme for Hugo 1: setup to basic customization

Mon, 28 Dec 2020 00:00:00 +0000

This is the first post in a series on building a website with R, blogdown, and the Academic Theme for Hugo. The next post will cover quality-of-life tweaks, and more advanced customization.

Update: Jan 3, 2021:

The main reason I thought this post was worth writing was that Alison Hill’s last published guide preceded major updates to the Academic theme for Hugo. But she has a new post on setting up your website using blogdown and Academic Theme for Hugo, i.e. the topic of this post.

You should use hers as your primary guide — there is like 90% overlap and she’s the expert — and maybe check here if you need a beginner’s perspective on particular topics.

Intro:

This is less a guide I wrote and more a mixture of guides/info from different sources (mostly by Alison Hill and Yihui Xie) annotated with my observations as a first-time user. If you want a more streamlined guide, Alison Hill’s 2019 slides on starting a website with blogdown and Hugo are the bees' knees, though not specific to the Academic Theme.

A few reasons you might want to use these guides anyways: it’s the most recent as of this writing 😎, I’ll detail some tricks / workarounds / failures, and I borrow heavily from Alison Hill’s guide anyways.

Regarding recentness: it's actually a little important if using Academic Theme for Hugo --- it seems to have had a breaking update last year (2019).

Who is this meant for?

Essentially, this guide is meant for people like me a week ago: I use R, rmarkdown, and GitHub daily, but I had roughly, oh, about exactly zero experience with HTML, Hugo, or the blogdown package.

I decided to write this in large part to document what I did so that I can work on my website in the future, particularly because blogdown + Hugo can be finicky. Hopefully these posts can save you some pain and frustration if you are also a first-time user.

Caveats

I am not an expert on any of this! I just started using these tools and, basically, if it worked, I moved on. So some of the suggestions here are likely sub-optimal. I tried to do my research, but please feel free to let me know if you have better ways to do any of this!

Preliminaries:

Note on changes to Academic Theme

As mentioned earlier, Academic Theme for Hugo had a big update last year: it became part of something called Wowchemy, and this appears to have broken things for some people, or required different settings, because

its folder structure changed, and
how it, um, handles some stuff is like, different. (I really have no idea 🤷)

On my first attempt building this site, I followed an older guide; the discrepancy in folder structure was confusing, and I ended up messing around with the actual theme files 😟. Then plotting broke 😳. Then the site didn’t deploy … and I ended up starting over 😖.

So follow a recent guide.

Resources

(ordered from most used to least used)

slides by Alison Hill
- These are for a different theme, but she literally wrote the book (along with Yihui Xie) on blogdown, and it’s easy to follow steps in this presentation format. Her slides are also purty.
- Her website and website repo were also frequently useful to me when troubleshooting.
blogdown documentation by Yihui Xie
- Yihui Xie is basically the Gandalf of the Fellowship of R Users.
Wowchemy customization guide
- It was a little difficult as a first-time user to find the info I needed here, but if/once I found it, it was helpful!

Starting up:

This section is almost all from Alison Hill’s slides.

Install latest blogdown and Hugo:

if (!requireNamespace("devtools")) install.packages('devtools')
devtools::install_github('rstudio/blogdown')
blogdown::install_hugo()

make a repo on GitHub.
create a new Rproject and connect it to the repo.
- in Rstudio: File –> New Project –> Version Control –> Git
if you don’t know how to do these things, follow the links in Alison’s slides.

If you plan to host your website from GitHub, you will probably want to host it from a repository with the name `yourGitHubusername.github.io`. So you can't use that for this step.

Initialize site with Academic Theme

library(blogdown)
new_site(theme="gcushen/hugo-academic",
         sample=TRUE,
         theme_example=TRUE,
         empty_dirs = TRUE,
         to_yaml = TRUE)

After that finishes, see if the site builds locally.

Use the “Addins” dropdown (end of the 2nd row of Rstudio’s menu bar) to select “Serve site” or use

blogdown::serve_site()

.gitignore

Add the resources/ and public/ folders to your .gitignore file so that it looks like this:

.Rproj.user
.Rhistory
.RData
.Ruserdata
public/
resources/

Check deployment

Don’t skip this step! It will help you detect problems as you move forward.

Two common options for hosting:

continuous deployment via Netlify (via GitHub): awesome. Auto-updates website when you push changes to your repo, and you can also preview/deploy branches to test changes you make to the website before putting them out into the world..
GitHub: simpler at first, but a few more steps required for deployment every time you update.

option 1: continuous deployment via Netlify (recommended)

Steps:

Push everything to your GitHub repo (you already added resources/ and public/ folders to your .gitignore, right?).

I love RStudio’s Git tab, but don’t use it for this unless you like clicking, and waiting, and clicking, and waiting, and….

staging many files in Rstudio’s Git tab

Instead, in the RStudio terminal (the tab next to your console), type:

git add -A               # stages everything
git commit -m "init"     # commit with message "init"
git push

While that’s going, set up an account at netlify.com.
Click “New site from Git” –> “Github” –> “Only select repositories” –> select your website repo.
- If you end up switching repos for some reason, you can always give Netlify access to another repo.
Your url will be something oddly delightful like drowsy-koala-d7ce09.netlify.app.
- Change it: “Site settings” link –> “Change site name”.
- You can always change the site name, or set up hosting on a new domain.
  - rbind.io is a recommended, and free, option.

Cool! Now whenever you push to your repo, netlify will build it and serve your website.

Use the branch deploy feature to preview changes (such as a new post)! All you have to do now is create a branch AND make a pull request, and netlify will let you preview it! More details + suggested workflow here.

option 2: Github deployment

Steps:

Build the site: either use blogdown::build_site() or the “Build” tab (next to Environment, History, Connections) –> Build Website
- This builds your website locally, in a new folder named public in your project root directory.
Push the contents of the public folder (not the folder itself, just what’s inside) to a GitHub repo named <yourGitHubusername>.github.io. For example, my repo would be named akseong.github.io, and that would also be the url of my website.

Done! Super easy, but unfortunately, whenever you want to update your website or add content, you will need to follow these same steps.

Folder structure: a short tour

Alison Hill gives you a short assignment to figure out your folder structure, and I found it helpful to go through. Your folders / files are going to look a little different, though (updates since 2019, different theme).

Your folder structure will look like this (unless it has changed yet again!):

├───archetypes
├───assets            # CSS, icons
├───config            
│   └───_default      # site customization
├───content           
│   ├───authors       # modify about page info in the 'admin' folder
│   ├───courses
│   ├───home          # home page
│   ├───post
│   ├───project
│   ├───publication
│   ├───slides
│   └───talk
├───data              
│   ├───fonts         
│   ├───themes        # "themes" = color scheme
├───layouts           # overrides theme templates
├───R
├───resources         # intermediary build files - delete often
├───static            # place for files that do not need to get "built" by Hugo
│   ├───admin
│   └───media         # images used with posts/projects
└───themes            # NO TOUCH!

The comments above should be fairly self-explanatory, but the content folder is worth discussing further.

The general idea is that the folders inside the content folder either

are a separate page in the website
- these folders contain an index.md file + widget .md files
or contain info used by widgets
- these folders usually contain a subfolder for each piece of content, e.g. each post or project

This is a little loose (e.g. the contents/home/gallery folder doesn’t fit so neatly in here) but I find it useful to think of it this way.

New page folders

The content/home folder is the only “separate page”-type folder here right now — the Academic Theme by default generates a 1-page website. Inside content/home, you’ll see a file index.md; open it up. It should say:

---
# Homepage
type: widget_page

# Homepage is headless, other widget pages are not.
headless: true
---

type: widget page indicates that the page initialized by this index.md file (the home page) is composed of widgets.
The other .md files inside this folder (content/home/) are the different widgets on the home page: an “about” widget, an “accomplishments” widget, etc.
You can add new folders like this to create new pages in your website, but you have to mimic this structure. More on this in a later post.

Folders containing info used by widgets will contain a file _index.md — note the underscore — and typically subfolders for entries.

For example, the folder content/post/ is where your blog posts go by default, and each post gets its own subfolder inside content/post/. The widget that uses the content in content/post/ is actually contents/home/posts.md (a widget on the home page) which contains the lines

widget: pages              
...<other options>... 
page_type: post

widget:pages indicate that this is a “pages” widget
page_type: post indicates that this widget draws its content from the folder content/post/. (Scroll down to the “Recent Posts” section on the homepage, and compare the posts to the folders inside content/post/.)

Basic customization

Now to make your website yours!

Use LiveReload

If it’s not already running, use blogdown::serve_site() to serve/host your website locally, i.e. see your website in RStudio’s viewer pane. You can also open the site in a browser tab using the “show in new window” icon in the viewer pane menu bar.

This is LiveReload — an awesome feature! The site refreshes on save, so you can see how your changes affect the website…. or if they broke something, which is unfortunately always a concern here.
If changes you make do not seem to be appearing, stop and restart the local server by running blogdown::stop_server(), then blogdown::serve_site().
- You might encounter an error message about not being able to stop the server when you run blogdown::stop_server(), but just executing it again always stops the server for me — so far at least 🤞
If your site blows up over nothing like a low-blood-sugar toddler, try stopping the local server, deleting the resources/ and public/ folders in your project’s root directory, and then starting the local server again.

resources/ contains intermediary build files, and will be auto-generated as needed.
public/ is generated when you build the website using the “Build” tab.

Site config

There are 3 main files you’ll want to edit:

config.yaml: general site configuration (next section).
config/_default/params.toml: personal info + site feature configuration, e.g. syntax highlighting.
config/_default/menus.toml: navbar configuration.

With LiveReload, you shouldn’t have too much trouble figuring out params.toml (heavily commented) and menus.toml (fairly straightforward) — and if in doubt about a particular setting in params.toml or menus.toml, leave it for now.

config.yaml requires a bit more care, however.

config.yaml

In your root directory, change the following fields in config.yaml:

title: site name (your name if this is a personal website)
baseurl: set it to the url (in quotes) your site has been deployed to.
- you didn’t skip that step, did you? If you did, you can also change it to ‘/’. For now. But it must be specified.
- include “https://” in the url. Also, see Yihui Xie’s post on redirecting HTTP to HTTPS
ignoreFiles: add the following to the default list under ignoreFiles

    - \.knit\.md$
    - \.utf8\.md$

I have seen some guides recommend including "_files$" in ignoreFiles, but this will break plotting in code chunks in .Rmd-generated posts.

Almost done. Add the lines:

relativeURLs: yes
canonifyURLs: yes

relativeURLs: yes is necessary for cloud setup.
cannonifyURLs: yes: consolidates duplicate URLs.

Last, beneath permalinks, I suggest adding post: /:year-:month-:day/:slug/ (follow indents; this modifies post urls, e.g. “…/2020-12-29/making-of-1/” rather than “…/post/making-of-1/").

Your info

For ease of editing, bring the about.md widget to the top of the page (deactivate the hero.md and demo.md widgets in content/home/ by changing set active: false, or make these widgets come after the about.md widget by setting weight: > 20). It will just be easier to see (using blogdown::serve_site()) the changes you’re making. You can always reactivate/reorder these widgets later.

The about.md widget in content/home/ uses info from the folder content/authors/admin/.

For the text, edit _index.md with your info. Quick note: the bio: field in the header is what appears next to your name at the end of posts, not in the section generated by the “about” widget.
Add your picture by placing a .jpg image into content/authors/admin/ and naming it avatar.jpg

Other widgets

Poke around content/home/ and modify the content of the different widgets with your info.

deactivate the widgets you don’t want by setting active: false. In general I would not delete these, as they are useful templates.
modify the order in which widgets appear using the weight: parameter (lower numbers = higher on page)

Hello World!

Tue, 15 Dec 2020 00:00:00 +0000

Hey! I made a web thingamabob! <polishes fingernails on shoulder> Spiffy!

So I just wanted to take a moment to put down what I have planned thus far for my very own web thingamabob. My plan for this blog is to be a mix of statistics, data vis appreciation, and practical / quality-of-life-type code snippets — and likely all three at the same time.

Subject to change, of course!

planned posts:

stuff I’ve picked up along the way

Building this site with HUGO + wowchemy Academic theme + lots of code-lifting + selectorGadget.
- Or "HUGO + wowchemy is a finicky #*!% GAHHH"
- Lots of posts on this basic topic, but not many people have written about this after Academc theme became part of Wowchemy and that made things hard to figure out for a first-timer.
- Also will help me remember what I did and where it goes.
Having babies while doing a PhD.
- tl;dr: don’t do it if you can help it. Otherwise: coffeeeeeeeeee!
Posts on datagoboop:
- Intro to datagoboop’s built-in audioplotting functions
- Intro to basic data $\mapsto$ sound functions in datagoboop
- Building an audioplot from scratch with datagoboop
- Development plans for datagoboop
Regression with penalized splines
Custom .rstheme + colored parens
Common penalty contours + P-spline penalty
Bayesian model selection basics
Parallelizing R scripts on a PC.
- don’t forget about using source as local job in RStudio!
Articles of interest

other stuff:

testing .Rmd features

i.e. making sure stuff ain’t broke

divtips

desiree de leon’s post

puzzle divtip

fyi divtip

demo divtip

testing <- TRUE
hey <- "it works!"

note divtip