Crash Course in I-O Technology: A Crash Course in Web Applications and Shiny

Richard N. Landers, Old Dominion University

The real power of technology for I-O becomes most apparent when you combine skillsets. This issue, we’ll be drawing connections between the ideas in my “Crash Course on the Internet” and my “Crash Course on R” to understand web applications, also known as “apps.” By the end of this Crash Course, you’ll be able to build a simple data visualization app and deploy it to the Internet for other people to see and play with.

 So to start, what exactly is a web application? Remember from my Internet crash course that all interactions between your web browser and the Internet can described as a series of data exchanges. Your web browser sends a request to a server, that server sends you something in return that is hopefully what you asked for, and your web browser interprets what it receives and displays it to you. But now we’re talking about apps! Wait a second, you ask yourself – if that’s true, how can I use something like Google Docs, which seems to record what I’m writing as I write it?

The answer is a little programming sleight of hand. Fundamentally, Google Docs is still a webpage. Your computer sent a request for the Google Docs page, Google sent it back to you, and then your web browser interpreted that and displayed it. But what’s different is that behind the scenes, in the code that your browser received from Google, there are extra instructions. Those instructions say that for everything you type in the webpage, send a copy of what you’ve done back to Google. Once your web browser gets verification from Google that each group of key presses was successfully received by Google, those data are now “real.” If your browser doesn’t get that verification, your words will actually disappear a few moments later. You can even see each request being sent and received if you work on a collaborative document with other people. Their text will appear a chunk at a time, but I guarantee you that they’re not actually writing that way. Instead, Google is updating roughly every half second to ensure what’s called client–server synchronization – that what you’re typing, what you see, and what everyone else sees are all roughly the same, give or take half a second of update requests.

This type of behind-the-scenes back and forth is what makes your web app appear interactive. You are still in fact bound by the regular rules of webpages I described in my Internet crash course, but programmers have learned to bend those rules to make the webpage appear response to your actions in real time. But the reality is that there are independent flows of data being passed back and forth between you and Google, triggered every half second or at whatever time interval Google has decided is appropriate.

Importantly, different apps handle this communication different ways. If you’ve ever used the survey platform Qualtrics, for example, you might have noticed it handles things differently. The only time a behind-the-scenes message is sent is after you click away from the question/item you’re working on. In other words, it waits until it thinks you’re done writing or updating a survey question before it sends a message about what you changed to the Qualtrics server. The unfortunate side effect of this is that if you don’t know that these messages are delayed, it’s very easy to exit your web browser thinking that changes to your survey have been saved when in reality, the update message was never sent. That happens, when you log back in later, your last change is missing.

Importantly, this sort of behind-the-scenes messaging only occurs when the web app needs to save information about what you’re doing locally, that is, if Google wants to keep a copy of your document. If everything stays local to your web browser, this is unnecessary. For example, I’ve written a couple of web applications to help with teaching undergraduate statistics in which all processing occurs inside your web browser: one that automatically generates new datasets and inferential hypothesis testing hand calculation walkthroughs and one that displays interactive correlation visualizations. Once you download the code to run these apps, the code runs entirely on your computer; my server will never receive any information about anything you do with it.

The differences between these approaches have big implications in terms of infrastructure. If you want to build an app that talks to a server, you need to build both the app and the server. Fortunately, many app building systems these days include relatively simple tools to just that and even to give you free server space. In the rest of this Crash Course, we’ll explore one such system that you can use to build interactive R-based apps. This system is called Shiny.

R Shiny

When people refer to a “Shiny app,” they usually are talking about an entire system of technologies that work together to convert R code into interactive web applications. These technologies are:

• R. Remember that R is fundamentally a processing engine; it takes R code, interprets it, and produces output. That’s the most fundamental aspect of an R app; you need the app user to send information to R, you need R to process and respond to that input, and then you need R to send information back that the person’s web browser can see. But you need them to do it without ever writing—or even seeing—any R code.

• Markdown. Markdown is a markup language much like HTML. Remember from my Crash Course on the Internet that webpages are primarily created by your web browser from HTML code. Under the hood, HTML is essentially plain text plus tags, indicated by <> symbols, that indicate a semantic meaning for the text they surround. For example, <p>This is a paragraph!</p> would render in a web browser as a paragraph (that’s what “p” stands for) containing the text “This is a paragraph!” Markdown, in contrast, is a markup language for R. Much like HTML can be used to mark up plain text to convert it into a webpage, Markdown can be used to mark up R code to convert it into a webpage. Thus, to make an app, you need to learn Markdown. Fortunately, it’s very easy, as we’ll see shortly.

• pandoc. pandoc is a software application that runs on your computer that converts between markup languages. It’s extremely flexible, but in the Shiny software ecosystem specifically, it’s used to convert Markdown into HTML. This is particularly important to you because Markdown is very easy to write, and HTML is not. Thus, you’ll be writing your app in Markdown, and pandoc will convert your Markdown into HTML automatically.

• knitr. knitr is a library in R that explains to pandoc what parts of your R script you want to be interactive and in what way. For example, say you want an app to automatically download data from Qualtrics and create some figures summarizing it as it is downloaded. That means you’d need R to download the data when the person opens the app, not when you write the code. knitr is what translates your need into something pandoc can understand and stitches everything together at the end.

So that’s a lot of systems. Fortunately, R Studio, the development environment I used to demonstrate R before, automates the relationships between these systems so that you don’t need to worry too much about the details. The main thing you need to worry about is installing all the software and R libraries that it wants to coordinate.­­

Let’s See It in Action

To make an app using R, you’re going to need to install a bunch of software, all of which is free. To make your app accessible on the Internet, you’re also going to need to create a free account with a service that hosts shiny apps. So to start, do the following in order:

1.       Download and install R and R Studio, if you don’t have them already.

2.       Within R Studio, download and install the rsconnect, shiny, and knitr packages.

3.       Download and install pandoc from its github page. As of the time of this writing, pandoc v2.0 was recently released, and as of pandoc v2.0.2, it doesn’t yet work with the rest of the shiny ecosystem. So if you install the newest version of pandoc and it doesn’t work, you might want to uninstall it, go to this webpage, scroll down, and try the last version of pandoc v1 instead, which was 1.19.2.1.

4.       Create an account on http://shinyapps.io, following the instructions precisely. At one point, you’ll need to copy some code into R Studio by clicking “Copy to Clipboard” and then pasting into the R Console. Do NOT manually copy/paste; use the button. If you do this correctly, you will see no errors in R. If you accidently click past the getting started screen and lose the R code you need to copy/paste, just log into your shinyapps account, click on Account > Tokens, and then the “Show” button to get it to reappear.

Once you complete these steps, you won’t need to do any of these things again; that was all initial setup. To create your first app using R Studio:

1.       In the menus, navigate to File > New File > R Markdown

2.       Click on Shiny.

3.       Click on the OK button.

4.       Save this file somewhere you’ll be able to find it later! I suggest naming it myfirstshiny.Rmd

5.       At this point, you can click the Run Document button and see a preview of your new app on your computer. It’s an app, but it’s not on the Internet yet. When you’re done looking at it, close it.

6.       Next, in the menus, navigate to Session > Set Working Directory > To Source File Location.

7.       To deploy it on the internet using your free shinyapps account, in the console, type:

a.       library(rsconnect)

b.      deployApp(“myfirstshiny.Rmd”)

That’s it! Your app now exists on the Internet, and anyone with the address can see it with a web browser.

To understand the relationship between R and Markdown, I suggest closely comparing the Rmd document you just created with the app. Here are a few things to get you started:

• The file starts with several lines of text between ---. This is called a YAML header, and it does two things: (a) is specifies the heading text that appears at the top of your app, and (b) it specifies what to turn the markdown code into when you click Run. Although we’re building apps, you can also use Markdown to convert R code into automatically created PDFs, Word documents, and slide decks, among other types.

• After that, you’ll see ```{r setup, include = FALSE}. This triggers Markdown to interpret the next code it sees, until the next ```, as R code to be executed. In this case, you’ll see some default options being set in the knitr library. Generally, you can just ignore this little code block when writing your own shiny apps. The word after “r” gives the name of this code block, and anything after that provides rendering options to knitr. In this case, the code block is named setup, and the code and output of this code block will not be included in the app, although they will still be executed behind the scenes (this is what include=FALSE does).

• Outside of ```, you’ll see some plain text, starting “This markdown document…” In the app, plain text will appear as plain text! So if you want to add text to your app, it is very easy indeed.

• Next, you’ll see a line with brackets and parentheses. This is your standard web link. For example, [best crash course on the Internet](http://0-www-siop-org.library.alliant.edu/tip/july16/crash.aspx) would render in the app as: best crash course on the Internet. If you’ve ever posted a link on Reddit, you’ll find this type of markup familiar.

• Afterward, you’ll see two hashes plus some text; this renders as a “Level 2” heading. You can generally think of higher numbers as subheadings of lower numbers, much like APA formatting.

• If we skip down a bit, you’ll come to a new R block named eruptions with a new function in it called inputPanel(), with functions inside it called selectInput() and sliderInput(). These functions create the HTML code that allows app users to choose options for things that will change in real time based upon what they select in the app. In this case, the selectInput() creates a dropdown selector named n breaks and labeled “Number of Bins” whereas sliderInput() creates a slider bar named bw adjust labeled “Bandwidth Adjustment.”

• The most important part of an app is the interactive component. When the interactive component is a figure or plot, you use the renderPlot() function, which appears a little below. The easiest way to think about renderPlot() is this: whatever “inputs” you specify elsewhere in the app become named elements of a vector called input. Thus, when this app runs, there’s a variable called input with two named elements: n breaks and bw adjust. Whenever the user changes the dropdown, input$n breaks gets a new value. Whenever the user changes the slider, input$bw adjust gets a new value. renderPlot() reruns its code based upon these new values any time either of them change. This is what makes the app appear interactive.

Given all of this, you might be able to guess what’s happening behind the scenes in a shiny app. First, the user downloads a web page containing your apps “start state.” When the user changes something, information about the thing they changed is sent to shinyapps.io. Shinyapps.io then reruns the renderPlot() code and sends information back to the web browser about what to update/change. In this way, the app becomes interactive.

To Learn More

Crash Course in I-O Technology: Web Apps with R Shiny from TNTLab on Vimeo.

To learn more about shiny apps, I strongly recommend the Data Camp lesson on R Markdown and Shiny apps. I use it to teach I-O graduate students in my data science class how to make their own apps, and with just one class period of instruction, everyone can create their own basic apps. Even with what we’ve walked through here, assuming you know a little R, you should be able to modify the starter Rmd file to your own needs. If you want to take a more self-study approach, I recommend checking out the R Markdown cheat sheet for a quick primer on the sorts of things you can do with Markdown. Then just make a few apps!

The main restriction of having a free account on shinyapps.io is that you can only have five active applications at a time, and you can only have 25 “active hours” per month. This is what it sounds like; if someone, including you, opens one of your apps and plays with it for 5 minutes, you’ve lost 5 minutes of your 25 monthly hours. This can add up quickly, so be careful if you want to share your apps with stakeholders beyond a couple of colleagues. But for your own testing and learning purposes, it’s plenty. If you decide you ultimately want to make shiny apps a major part of your job or business, you’ll eventually want to your organization to host its own shiny server; then you don’t need to pay for server space or worry about processing limits.

Finally, you can see an I-O relevant example of a data exploration app I created to rank I-O PhD programs in terms of interdisciplinarity for an article in the next issue of TIP! Basically anything you can create in R can become a shiny app. The only limits are your imagination and your skill with R.

Conclusion

That’s it for the seventh edition of Crash Course! If you have any questions, suggestions, or recommendations about web applications or Crash Course, I’d love to hear from you (rnlanders@odu.edu; @rnlanders).