Interactive dataviz on the web with R & plotly

class: center, middle, inverse, title-slide

# Interactive dataviz on the web with R & plotly
### Carson Sievert
### Slides: <a href="https://bit.ly/useR18">bit.ly/useR18</a> <br /> <br /> <i class="fab  fa-twitter "></i> <a href='https://twitter.com/cpsievert'><span class="citation">@cpsievert</span></a> <br /> <i class="fab  fa-github "></i> <a href='https://github.com/cpsievert'><span class="citation">@cpsievert</span></a> <br /> <i class="fas  fa-envelope "></i> <a href="mailto:cpsievert1@gmail.com">cpsievert1@gmail.com</a> <br /> <i class="fas  fa-globe "></i> <a href="https://cpsievert.me/" class="uri">https://cpsievert.me/</a> <br /> <br /> Slides released under <a href='https://github.com/cpsievert/talks/blob/gh-pages/LICENSE'>Creative Commons</a>

---

class: inverse, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

### Open the slides [bit.ly/useR18](https://bit.ly/useR18)

### Go to this address [rstudio.cloud/project/14090](https://rstudio.cloud/project/14090)

.footnote[
---
Time: 2 minutes
]

---
class: middle, principles

<h3 align="center"> About me </h3>

* PhD in statistics at Iowa State with Heike Hofmann & Di Cook (Dec 2016) 
  
  * Thesis: [Interfacing R with web technologies](http://lib.dr.iastate.edu/etd/15422/)
  
* CEO of [Sievert Consulting](https://consulting.cpsievert.me/) LLC (Jan 2017 - Present)
  * Services: product development, statistical modeling, and R training  
  * Clients: plotly, Sandia Labs, NOAA, LL Bean
  
* Expert at Library of Congress (June 2018 - Present)
  * Provide statistical expertise for DoD, BJS, and other US Federal Government projects
  
* I ❤️ interactive data visualization
  
  * Maintain/author R 📦s: plotly, LDAvis, animint

---
background-image: url(workflow.svg)
background-size: contain
class: inverse

# Data science workflow

---
background-image: url(workflow1.svg)
background-size: contain
class: inverse

## Web graphics are great for presentation!

### Sharable, portable, composable (i.e., reports, dashboards, etc)

---
background-image: url(workflow2.svg)
background-size: contain
class: inverse

## Web technologies aren't designed for this iteration!

### Follow-up questions (ignited through visualization) may rely on sophisticated computations

---
class: principles

## ...but interactivity augments exploration

* Identify structure that otherwise goes missing ([Tukey 1972](http://stat-graphics.org/movies/prim9.html)).
    
* Search for information quickly without fully specified questions<sup>1</sup> ([Unwin & Hofmann, 2000](https://www.researchgate.net/publication/2425912_GUI_and_Command-line_-_Conflict_or_Synergy))
    * Multiple linked views are the optimal framework for posing queries about data ([Buja, Cook, & Swayne 1996](https://www.jstor.org/stable/1390754))
    
* Diagnose, compare, and understand models ([Wickham, Cook, & Hofmann 2015](http://onlinelibrary.wiley.com/doi/10.1002/sam.11271/abstract)).

.footnote[
---

[1]: Worried about inference? See visual ([Majumder et al 2013](http://amstat.tandfonline.com/doi/abs/10.1080/01621459.2013.808157?journalCode=uasa20#.Wl01_ZM-dTY)) and post-selection ([Berk et al 2013](https://projecteuclid.org/euclid.aos/1369836961)) inference frameworks.
]

---
background-image: url(workflow2.svg)
background-size: 400px
background-position: 90% 8%
class: inverse, middle, center

## Interactive graphics can <font color="red">augment exploratory analysis</font>, but are only practical when we can <font color="red">iterate quickly</font>

---
background-image: url(workflow1.svg)
background-size: 400px
background-position: 90% 8%
class: inverse, middle, center

## Interactive graphics can <font color="red">enhance presentation</font>, but are only practical when <font color="red">easily distributed</font>

---
background-image: url(server-client.svg)
background-size: contain
class: middle, right

# When is a web application necessary?

---
background-image: url(server-client-dim.svg)
background-size: contain
class: middle, right

# Easier to share, scale, and maintain

---
background-image: url(server-client-dim.svg)
background-size: contain
class: middle, right

# Client-side technologies: HTML, JavaScript, CSS

---
background-image: url(server-client-dim.svg)
background-size: contain
class: middle, right

# DYK: Many R packages generate HTML/JavaScript...

.footnote[
...via **htmlwidgets** and **htmltools**
]

---
background-image: url(server-client-dim.svg)
background-size: contain
class: middle, right

# plotly can do a lot in a standalone page!

---
class: middle, principles, inverse

<h2 align="center"> plotly's client-side reactivity options </h2>

1. Graphical (database) queries
  * In _plotly for R_, I call this [linking views without shiny](https://plotly-book.cpsievert.me/linking-views-without-shiny.html).
  
2. Respond to plotly [sliders](https://plot.ly/r/sliders/), [buttons](https://plot.ly/r/custom-buttons/), and [dropdowns](https://plot.ly/r/dropdowns/) via [plotly.js functions](https://plot.ly/javascript/plotlyjs-function-reference/)

3. Custom JavaScript via `htmlwidgets::onRender()`
  * Respond to [plotly.js events](https://plot.ly/javascript/plotlyjs-events/) in a custom way
  
  
---
class: middle, principles, inverse

<h2 align="center"> plotly's client-side reactivity options </h2>

1. __Graphical (database) queries__
  * __In _plotly for R_, I call this [linking views without shiny](https://plotly-book.cpsievert.me/linking-views-without-shiny.html).__
  
2. Respond to plotly [sliders](https://plot.ly/r/sliders/), [buttons](https://plot.ly/r/custom-buttons/), and [dropdowns](https://plot.ly/r/dropdowns/) via [plotly.js functions](https://plot.ly/javascript/plotlyjs-function-reference/)

3. Custom JavaScript via `htmlwidgets::onRender()`
  * Respond to [plotly.js events](https://plot.ly/javascript/plotlyjs-events/) in a custom way

---
### Make ggplot2 interactive

```r
library(plotly)
p <- ggplot(txhousing) + geom_line(aes(date, median, group = city))
ggplotly(p)
```

---
### Customize tooltip

```r
library(plotly)
p <- ggplot(txhousing) + geom_line(aes(date, median, group = city, text = city))
ggplotly(p, tooltip = "text")
```

---
### Highlight a key (e.g. city) column

```r
library(plotly)
*tx <- highlight_key(txhousing, ~city)
p <- ggplot(tx) + geom_line(aes(date, median, group = city, text = city))
gg <- ggplotly(p, tooltip = "text")
*highlight(gg, on = "plotly_click")
```

---
### Direct/indirect manipulation & persistent highlighting!

```r
gg <- ggplotly(p, tooltip = "text")
*highlight(gg, on = "plotly_hover", selectize = TRUE, dynamic = TRUE)
```

---
### Link aggregates to raw data

---
background-image: url(pipeline.svg)
background-size: contain
class: bottom, left

## "Linking as a <br> database query"

---
background-image: url(pipeline.svg)
background-size: contain
class: bottom, left

.pull-left[
```sql
SELECT * FROM table 
  WHERE city == "South Padre Island"
```
]

---
background-image: url(../gifs/storms.gif)
background-size: contain
class: bottom, left

### Works with 3D!

---
background-image: url(../gifs/mapbox-aggregates.gif)
background-size: contain
class: bottom, left

### Works with aggregates (& mapbox)!

---
background-image: url(../gifs/us-arrests.gif)
background-size: contain
class: bottom, left

### Works with list-columns & animation!

---
background-image: url(../gifs/dt-trails.gif)
background-size: contain
class: bottom, left

### Works with other htmlwidgets

---
class: inverse, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

<h2 align="center"> <em>Choose</em> Your Turn </h2>

(A) Think of a question you'd like to ask of your data via a (linked) interactive graphic (bonus: draw it)!

(B) Study [the code](https://github.com/ropensci/plotly/blob/master/demo/crosstalk-highlight-pipeline.R) for generating [this visual](#21). How does it work?

.footnote[
---

Time: 5 minutes
]

---
## Texas housing prices

```r
tx <- txhousing %>%
  select(city, year, month, median) %>%
  filter(city %in% c("Galveston", "Midland", "Odessa", "South Padre Island"))
```

```r
#> # A tibble: 748 x 4
#>    city       year month median
#>    <chr>     <int> <int>  <dbl>
#>  1 Galveston  2000     1  95000
#>  2 Galveston  2000     2 100000
#>  3 Galveston  2000     3  98300
#>  4 Galveston  2000     4 111100
#>  5 Galveston  2000     5  89200
#>  6 Galveston  2000     6 108600
#>  7 Galveston  2000     7  99000
#>  8 Galveston  2000     8  96200
#>  9 Galveston  2000     9 104000
#> 10 Galveston  2000    10 118800
#> # ... with 738 more rows
```

#### How does price differ across cities?

---
### Price versus month, by city & year

```r
library(ggplot2)
ggplot(tx, aes(month, median, group = year)) +
  geom_line() +
  facet_wrap(~city, ncol = 2)
```

---
### Query specific years

```r
library(plotly)
*TX <- highlight_key(tx, ~year)
p <- ggplot(TX, aes(month, median, group = year)) + geom_line() +
  facet_wrap(~city, ncol = 2)
*ggplotly(p, tooltip = "year")
```

---
## Set selection mode and default selections

```r
highlight(.Last.value, on = "plotly_hover", defaultValues = 2006)
```

---
## Make comparisons with dynamic brush

```r
highlight(.Last.value, dynamic = TRUE, persistent = TRUE, selectize = TRUE)
```

---
## Customize the appearance of selections

```r
highlight(
  .Last.value, dynamic = TRUE, persistent = TRUE, 
*  selected = attrs_selected(mode = "markers+lines", marker = list(symbol = "x"))
)
```

---
## Automate queries via animation

```r
p <- ggplot(tx, aes(month, median)) + 
  geom_line(aes(group = year), alpha = 0.2) +
*  geom_line(aes(frame = year), color = "red") +
  facet_wrap(~city, ncol = 2)
ggplotly(p)
```

---
class: middle, inverse
background-image: url(../gifs/epl-compare.gif)
background-size: contain

# Generally useful for comparing within/across panels!

.footnote[
[interactive](epl.html)
]

---
class: inverse, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

Run the following R code to generate the soccer visualization:

```r
demo("crosstalk-highlight-epl-2", package = "plotly")
```

1. Compare the performance of 'Liverpool' with 'Chelsea'.
2. Run `plotly_json()`. What does this return?

<hr>

Time: 3 minutes

---
## Inspect the underlying data

`plotly_json()` returns the underlying JSON of any **plotly** graph -- nice way to learn how `ggplotly()` maps to [plotly.js](https://plot.ly/javascript/)!

---
class: inverse
background-image: url(printing.svg)
background-size: contain

---
## Want to work with plotly.js directly?

`schema()` returns the official plotly.js [figure reference](https://plot.ly/r/reference/) tied to the R package

---

<h2 align="center"> <b>plotly</b> is much more than <code>ggplotly()</code>! </h2>

#### Initiate a plotly graph (without ggplot2):

* `plot_ly()`: 'flexible' interface to [plotly.js](https://github.com/plotly/plotly.js)
* `plot_mapbox()`: `plot_ly()` wrapper/shortcut for [scattermapbox](https://plot.ly/r/scattermapbox/)
* `plot_geo()`: `plot_ly()` wrapper/shortcut for [scattergeo](https://plot.ly/r/choropleth-maps/)

#### Add data (i.e., traces) to a graph

* `add_markers()`, `add_bars()`, etc (see [`help(add_trace)`](https://www.rdocumentation.org/packages/plotly/versions/4.7.1/topics/add_trace) for full list)

#### Modify a graph (before printing)

* `style()` to modify traces of an existing plotly graph
* `layout()` to add/modify to a [layout component](https://plot.ly/r/reference/#layout)

#### Tools for talking with plotly cloud

* Send data and/or graphs with `api_create()`
* Retrieve data and/or graphs with `api_download_plot()`/`api_download_grid()`
* Do anything the [plotly's server API](https://api.plot.ly/v2/) supports with `api()`

---
### 'Smart' trace type defaults

```r
subplot(
  plot_ly(diamonds, x = ~cut, y = ~clarity),
  plot_ly(diamonds, x = ~cut, color = ~clarity),
  nrows = 2, shareX = TRUE
)
```

---
class: inverse, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

Use `plotly_json()` to study how the R code (on the [last slide](#42)) maps to JSON.

**NOTE**: We don't have time to cover `plot_ly()` in depth..learn more in the [plotly cookbook](https://plotly-book.cpsievert.me/the-plotly-cookbook.html) chapter of the [plotly for R book](https://plotly-book.cpsievert.me/)

.footnote[
---

Time: 3 minutes
]

---
## Aggregating selections

---

```r
library(plotly)
d <- highlight_key(mpg)

dots <- plot_ly(d, color = ~class, x = ~displ, y = ~cyl)
boxs <- plot_ly(d, color = ~class, x = ~class, y = ~cty) %>% 
  add_boxplot()
bars <- plot_ly(d, colors = "Set1", x = ~class, color = ~class)

subplot(dots, boxs, titleX = TRUE, titleY = TRUE) %>%
  subplot(bars, nrows = 2, titleX = TRUE, titleY = TRUE) %>%
  layout(
    barmode = "overlay",
    showlegend = FALSE
  ) %>%
  highlight("plotly_selected")
```

---
## Aggregating selections (continued)

---

```r
library(plotly)
d <- highlight_key(mtcars)
sp <- plot_ly(d, x = ~mpg, y = ~disp) %>% 
  add_markers(color = I("black"))

# 'statistical trace types'
hist <- plot_ly(d, x = ~factor(cyl)) %>% add_histogram(color = I("black"))
box <- plot_ly(d, y = ~disp, color = I("black")) %>% add_boxplot(name = " ")
violin <- plot_ly(d, y = ~disp, color = I("black")) %>% add_trace(type = "violin", name = " ")

subplot(sp, box, violin, shareY = TRUE, titleX = TRUE, titleY = TRUE) %>%
  subplot(hist, widths = c(.75, .25), titleX = TRUE, titleY = TRUE) %>%
  layout(
    barmode = "overlay", 
    title = "Click and drag scatterplot",
    showlegend = FALSE
  ) %>%
  highlight("plotly_selected")
```

---
## Aggregating selections (continued)

---

```r
library(plotly)
tx <- highlight_key(txhousing, ~city)

p1 <- ggplot(tx, aes(date, median, group = city)) + geom_line()
gg1 <- ggplotly(p1, tooltip = c("city", "date", "median"))
p2 <- plot_ly(tx, x = ~median, color = I("black")) %>% 
  add_histogram(histnorm = "probability density")

subplot(gg1, p2, titleX = TRUE, titleY = TRUE) %>% 
  layout(barmode = "overlay") %>%
  highlight(dynamic = TRUE, selected = attrs_selected(opacity = 0.3))
```

---
## Talk with other htmlwidgets

```r
library(leaflet)
sd <- highlight_key(quakes)
p <- plot_ly(sd, x = ~depth, y = ~mag) %>% add_markers(alpha = 0.5) %>% highlight("plotly_selected")
map <- leaflet(sd) %>% addTiles() %>% addCircles()
crosstalk::bscols(p, map)
```

---
### Linking plotly with DT

```r
library(plotly)
data(trails, package = "mapview")
tsd <- highlight_key(trails)
crosstalk::bscols(
  plot_mapbox(tsd, text = ~FKN, hoverinfo = "text"),
  DT::datatable(tsd)
)
```

---
class: center, middle, inverse

## Expectations vs reality

.pull-left[
  <img src="https://i.imgur.com/fZIenVE.jpg" height = "500" />
]

.pull-right[
<br />
<br />
<br />
<br />

**plotly** has advanced support for *highlight* events (e.g., `persistent`, `dynamic`, `selectize`)

Other [**crosstalk**-enabled htmlwidgets](https://rstudio.github.io/crosstalk/) likely won't respect (non-default) `highlight()` options.
  
  However, *filter* events should generally be supported.
]

---
class: middle, center, principles

# Filter vs highlight

*Highlight* events **dim the opacity** of existing marks.

*Filter* events **completely removes** existing marks and rescales axes.

At least currently, *filter* events must be fired from **crosstalk** widgets.

---
background-image: url(../gifs/filter-vs-highlight.gif)
background-size: contain

---
## Crosstalk's filtering widgets

```r
library(crosstalk)
tx <- highlight_key(txhousing)
widgets <- bscols(
  widths = c(12, 12, 12),
  filter_select("city", "Cities", tx, ~city),
  filter_slider("sales", "Sales", tx, ~sales),
  filter_checkbox("year", "Years", tx, ~year, inline = TRUE)
)
widgets
```

---
## Filtering

```r
bscols(
  widths = c(4, 8), widgets, 
  plot_ly(tx, x = ~date, y = ~median, showlegend = FALSE) %>% 
    add_lines(color = ~city, colors = "black")
)
```

---
class: inverse, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

1. Use `htmlwidgets::saveWidget()` to save a **plotly** graph (e.g. `plot_ly()`). What's the size of the HTML file it creates?
2. Use `htmltools::save_html()` to save the **plotly**+**leaflet** example. What's the size of the HTML file it creates?
3. What's the difference between using `saveWidget()` and `save_html()`? When is one preferred to the other?

.footnote[
---

Time: 5 minutes

]

---
class: inverse, middle, principles
background-image: url(htmltools.svg)
background-size: contain

---
class: middle, principles, inverse

<h2 align="center"> plotly's client-side reactivity options </h2>

1. Graphical (database) queries
  * In _plotly for R_, I call this [linking views without shiny](https://plotly-book.cpsievert.me/linking-views-without-shiny.html).
  
2. __Respond to plotly [sliders](https://plot.ly/r/sliders/), [buttons](https://plot.ly/r/custom-buttons/), and [dropdowns](https://plot.ly/r/dropdowns/) via [plotly.js functions](https://plot.ly/javascript/plotlyjs-function-reference/)__

3. Custom JavaScript via `htmlwidgets::onRender()`
  * Respond to [plotly.js events](https://plot.ly/javascript/plotlyjs-events/) in a custom way
  
  
---
## Dropdown example

---
## The implementation

```r
styles <- schema()$layout$layoutAttributes$mapbox$style$values
style_buttons <- lapply(styles, function(s) {
  list(label = s, method = "relayout", args = list("mapbox.style", s))
})

storms <- sf::st_read(system.file("shape/storms_xyz.shp", package = "sf"), quiet = TRUE)
plot_mapbox(storms, color = I("red")) %>%
  layout(
    title = "Changing the base layer",
    updatemenus = list(list(y = 0.8, buttons = style_buttons))
  )
```

---
class: middle, principles, inverse

<h2 align="center"> plotly's client-side reactivity options </h2>

3. __Custom JavaScript via `htmlwidgets::onRender()`__
  * __Respond to [plotly.js events](https://plot.ly/javascript/plotlyjs-events/) in a custom way__
  
---
### Google search on click

The `customdata` attribute provides a way to attach "meta-data" to visual attributes that you can access with JavaScript

```r
plot_ly(mtcars, x = ~wt, y = ~mpg) %>%
  add_markers(customdata = ~paste0("http://google.com/#q=", rownames(mtcars))) %>%
  htmlwidgets::onRender("function(el, x) {
    el.on('plotly_click', function(d) {
      var url = d.points[0].customdata;
      window.open(url);
    });
  }")
```

---
class: middle, inverse, principles

## Demo

In the [RStudio cloud project](https://rstudio.cloud/project/14090), open the 'customdata.R' script:

```r
file.edit("~/customdata.R")
```

---
class: center, middle

# Hello <img src="https://www.rstudio.com/wp-content/uploads/2014/04/shiny.png" width=100 /> 👋

```r
shiny::runApp("~/tutorials/20180711/shiny/01", display.mode = "showcase")
```

---
class: center, middle

# Accessing plotly user events

```r
shiny::runApp("~/tutorials/20180711/shiny/02", display.mode = "showcase")
```

---
class: inverse, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

1. Modify the last app to use `plot_ly()` instead of `ggplotly()`

2. Add output blocks that print out data from the following events:

* `"plotly_hover"`
* `"plotly_click"`
* `"plotly_relayout"`

.footnote[
---

Time: 5 minutes
]

---
class: center, middle

# Targetting events

```r
shiny::runApp("~/tutorials/20180711/shiny/03", display.mode = "showcase")
```

.footnote[
See also <https://plotly-book.cpsievert.me/linking-views-with-shiny.html#targeting-views>
]

---
class: center, middle

# plotly proxies

By default, **shiny updates require a full redraw**, but proxies allows us to leverage [the plotly.js API](https://plot.ly/javascript/plotlyjs-function-reference/) to modify/update graphs more efficiently

```r
shiny::runApp("~/tutorials/20180711/shiny/04", display.mode = "showcase")
```

---
class: center, middle

# Streaming data

```r
shiny::runApp("~/tutorials/20180711/shiny/05", display.mode = "showcase")
```

.footnote[
Inspired by <https://plot.ly/r/streaming/>
]

---
class: inverse, middle
background-image: url(your-turn.jpeg)
background-size: contain

Open the last example

```r
file.edit("~/tutorials/20180711/shiny/05/app.R")
```

Modify it to do the following:

1. Add `sliderInput()` for controlling the streaming interval.

2. Add a widgets to:
  * Change *only* the width of the line
  * Change *only* the color of the line
  * *Only* add/remove markers (e.g. points) for each data point

<hr>

Time: 10 minutes

Hint: some of these shiny example apps will be helpful (e.g. `proxy_restyle_economics`) -- <https://github.com/ropensci/plotly/tree/master/inst/examples/shiny>

---
class: inverse, center, middle, principles
background-image: url(your-turn.jpeg)
background-size: contain

## Your turn

Remember [this](#28) "your turn"? Let's try to implement it!

---
background-image: url(plotly.svg)
background-size: 100px
background-position: 90% 8%
class: center, middle

# Thanks!

Resources for more learning:

https://plotly-book.cpsievert.me <br />
https://blog.cpsievert.me <br />
https://talks.cpsievert.me <br />
https://github.com/cpsievert/apps <br />
https://github.com/cpsievert/pedestrians <br />
https://github.com/cpsievert/bcviz <br />
https://github.com/cpsievert/phd-thesis <br />

Find me online:

Web: <http://cpsievert.me/> <br />
Twitter: [@cpsievert](https://twitter.com/cpsievert) <br />
GitHub: [@cpsievert](https://github.com/cpsievert) <br />
Email: <cpsievert1@gmail.com>

---
background-image: url(plotly.svg)
background-size: contain

# Ask me anything!!