shiny.worker: Speed Up R Shiny Apps by Offloading Heavy Calculations

This article is originally published at https://appsilon.com

This article is part of a series on speeding up Shiny. Learn how to omit the server.r bottleneck and push actions to the browser in this article by Marcin Dubel. Learn how to diagnose poor Shiny performance, use faster functions, and take advantage of caching operations in this article by Krystian Igras. 

TL;DR

Because of the way R Shiny is designed, long running calculations freeze the UI of Shiny dashboards until the calculations are complete. This can result in a sluggish app and a negative user experience. Appsilon has created a package to offload long running calculations to an external machine so that the UI of Shiny dashboards can remain responsive.

• Long-Running Calculations Result in Frozen Shiny Dashboards
• shiny.worker: An R Shiny Package to Offload Heavy Calculations
• shiny.worker: How Does It Work?
• How Can I Start Using shiny.worker?
• More Ways to Speed Up Shiny
• Appsilon is Hiring!

Long-Running Calculations Result in Frozen Shiny Dashboards

One of the performance challenges in Shiny is that long running calculations can paralyze your UI. You will not be able to change input values and parts of your application will be frozen until the heavy task is completed. This is an intentional feature of Shiny to avoid race conditions, but is nonetheless very frustrating for the end user. 

Unfortunately, the Shiny promises library doesn’t help with this problem. Promises are a great improvement for Shiny performance, but the problems that Shiny promises solve are different from the problem of heavy calculations. Promises were designed for inter-session reactivity rather than intra-session. They simply don’t work like promises in JavaScript, and cannot be used to unfreeze Shiny apps during heavy calculations in a single session.

shiny.worker: An R Shiny Package to Offload Heavy Calculations

Here is how Appsilon has solved the problem in our projects. We have developed a proprietary R package called shiny.worker, which is an abstraction based on futures, for delegating jobs to an external worker. While the job is running and the heavy calculation is being processed by the external worker, you can still interact with the app and the UI remains responsive.

Here is an example app where you can see shiny.worker in action: https://demo.appsilon.ai/apps/shiny-worker/ 

As you can see in this demonstration, the UI elements (slider and button) remain responsive even while a heavy calculation is being performed.

The idea of our solution is very simple. If I have long running calculations that are likely to freeze the UI of my app, then I delegate them to the worker. The worker is an external machine that executes R code. 

shiny.worker unfreezes Shiny by offloading heavy calculations.

shiny.worker: How Does It Work?

Here is the code of the shiny.worker demo app:

library(shiny)
library(shiny.info)

ui <- fluidPage(theme = "style.css",

  titlePanel("shiny.worker"),
  shiny.info::powered_by("shiny.worker", link = "https://appsilon.com/shiny"),

  sidebarLayout(
    sidebarPanel(
      div("Play with the slider. Histogram will be still responsive, even if job is running:"), br(),
      sliderInput("bins", "Number of bins:", min = 1, max = 50, value = 30),
      div("Then try to run new job again:"), br(),
      actionButton("triggerButton", "Run job (5 sec.)")
    ),
    mainPanel(
      fluidRow(
        column(6, plotOutput("distPlot")),
        column(6, 
           uiOutput("loader"),
           plotOutput("futurePlot")
        )
      )
    )
  )
)

server <- function(input, output) {

  output$distPlot <- renderPlot({
    x    <- faithful[, 2]
    bins <- seq(min(x), max(x), length.out = input$bins + 1)
    hist(x, breaks = bins, col = 'darkgray', border = 'white')
  })

  plotValuesPromise <- shiny.worker::job(
    job = function(args) {
      Sys.sleep(5)
      cbind(rnorm(args$n), rnorm(args$n))
    },
    trigger_args = reactive({
      input$triggerButton
      list(n = 1000)
    }),
    cancel_active_job_on_input_change = FALSE, # ignore input change, wait until resolved
    value_until_not_resolved = NULL
  )

  output$loader <- renderUI({
    task <- plotValuesPromise()
    if (!task$resolved) {
      div(
        div(class = "loader-text", "Job is running..."),
        div(class = "loader")
      )
    }
  })

  output$futurePlot <- renderPlot({
    task <- plotValuesPromise()
    if (task$resolved) {
      plot(task$result, main = "There you go")
    }
  })

}

shiny.worker::init()
shinyApp(ui = ui, server = server)

The key fragment is the line with the shiny.worker::job() call where you schedule a job. The job is your long running calculation. When it was a regular reactive, it was blocking the UI, but now it is delegated to the worker. As the calculation is delegated to the worker, the UI is unfrozen while the calculation is being performed.

Arguments for the job are provided as a reactive (trigger_args). Its value will be passed to the job function as args. This means that every time the value of this reactive changes, shiny.worker will take action, depending on the strategy you choose. It can be triggering a new job and cancelling a running job or ignoring the change (no new job is scheduled until it is resolved). It is the developer’s responsibility to implement app logic to avoid potential race conditions.

To access the worker’s result, you call it like you do with a reactive (plotValuesPromise()). As a result you are able to read its state (task$resolved) and returned value (task$result). You decide what should be returned when the job is still running with the argument value_until_not_resolved.

How Can I Start Using shiny.worker?

shiny.worker has not yet been released to the public, but we have made it available to several of our clients. So far, we’ve seen dramatic improvements in UX after implementing shiny.worker. If you are experiencing problems with Shiny performance due to heavy calculations, reach out to us and we will be happy to help you make use of shiny.worker. 

More Ways to Make R Shiny Apps Faster

• 4 Tips to Make Your Shiny Dashboard Faster by Appsilon Data Scientist Krystian Igras
• Make R Shiny Dashboards Faster with updateInput, CSS, and JavaScript by Appsilon Software Engineer Marcin Dubel
• Use Appsilon’s Open Source R Shiny Packages

Appsilon is Hiring

• We are hiring a Data Science Writer to create and edit interesting Data Science and ML content.
• We are also hiring a WordPress Developer with an excellent visual sense to maintain and improve our website.

Article shiny.worker: Speed Up R Shiny Apps by Offloading Heavy Calculations comes from Appsilon Data Science | End­ to­ End Data Science Solutions.

Thanks for visiting r-craft.org
This article is originally published at https://appsilon.com
Please visit source website for post related comments.