Redesigning Dashboards with Shiny and Rhino: World Bank’s Carbon Pricing

This article is originally published at https://appsilon.com

The modern revolution of big data has led to a boom in business intelligence. But data is only as valuable as the insights it provides. That means the true value of data is in exposing hidden insights. But if the way you share your data is slow, unscalable, or poorly designed – it might be time to redesign your dashboard. 

Data scientists are insight seekers. They are modern-day explorers sifting through data to find meaningful connections. But their efforts would be in vain if not for sharing their work with those who might benefit.

This is where dashboards and applications come into play. But with so many solutions, where do you begin? 

R Shiny is a good option for enterprise app development. Shiny is a full web framework for building production-ready dashboards through R (and now Python). It’s not the best choice for every case, but it is a great open-source tool when handling large datasets and complex statistical computing. The freedom to customize and quickly build POCs highlights Shiny’s Proof of Value (POV).

Power BI or R Shiny – Which one is right for you?

Today I’ll show you how to take an existing dashboard using the World Bank Carbon Pricing as an example, and transform it into a Shiny dashboard fit for production. The World Bank’s dashboard is quite extensive, so we’ll limit it to one item. But we encourage you to practice using other elements/data after the tutorial. 

TOC:

• Why use R and Shiny?
• How hard is it to get started with Shiny?
• How to recreate a dashboard in Shiny with Rhino
• Data and pre-processing R script
• Installing Rhino and setting up Shiny
• Using semantic.dashboard to speed up the dashboard build
• Interactive mapping in Shiny with Leaflet
• Final output

Why use R and Shiny to build your dashboard?

No hidden costs to Shiny

Shiny provides everything you need to develop and deploy a dashboard for free. Nothing is hidden behind a subscription. Everything you need to develop and deploy is available through Posit and the Shiny community – free of charge. 

Should you need more scalability, customization, or team extensions, there are enterprise solutions. Posit provides a suite of products (at cost) for better tooling. And teams like ours provide end-to-end services as Posit Certified Partners and Shiny developers.

Shiny’s business value: quick start with a big impact

Change is inevitable, especially when building solutions for an organization. Developing and integrating new ideas is important when as projects tend to evolve. Having the ability to implement, test, and adjust quickly is key to maximizing proof-of-value.

The secret sauce is open source

Every component you need is open source and comes with extensive documentation. The Shiny ecosystem is an open-source one, filled with easy-to-understand docs and tutorials. You can take a few minutes to explore underneath the hood of a package or feature and see if it’s the right fit. 

Explore Appsilon’s open-source tools for a smooth Shiny development experience.

Was that a bug or a feature that you saw while building the dashboard? You can post an issue with any bugs discovered to find workarounds and raise issues with a feature request as an end user.

The Shiny community and accessing support

Throughout your R Shiny journey, if you come across issues, an entire community of open-source developers is ready to help out. From community forums to social media, folks in the R community are willing to help you resolve the issue that you are struggling with.

Many who discuss their R Shiny journey talk about how grateful they are for the inclusive and supportive nature of the Shiny community. It seems to be a common theme in the R programming world and its benefits show.

How hard is it to get started with Shiny?

Coding can seem hard. Especially something that is going to be used at a production level. The notorious spaghetti code looms on the far horizon as you keep on adding features. After a few hours, days, or weeks of coding, you come back to a confusing labyrinth of code that is hard to decipher. 

It used to be Tableau vs Shiny – now it’s Tableau with Shiny! Check out the pros and cons of each or combine the two.

But there is a solution to safeguard yourself along the way. When in doubt, take a framework to guide your journey in building production-ready dashboards. There are several options for developers to choose from. Rhino is our contribution to the community that started as a way to improve internal workflows. It’s an opinionated framework that is built with love from Appsilon to help developers at every expertise level build Shiny apps like a software engineer.

How to recreate a dashboard in Shiny with Rhino?

Instead of starting on creating an entirely new dashboard, why not try improving an existing dashboard using R Shiny and Rhino? There are so many datasets, visualizations, and public-facing dashboards that could use some extra attention. So, we thought, why not do just that?

Let’s take an attempt at recreating a dashboard available for the public. While it is difficult to create an exact replica of an existing dashboard in production, we can try to build a dashboard that is as similar as possible to the original. 

The dashboard we are using today is the World Bank Carbon Pricing Dashboard. Carbon pricing is an initiative to reduce unnecessary carbon emissions by either imposing a tax on excessive emissions (Carbon tax) or establishing a method for trading the allowed amounts of emissions to ensure that a country does not excessively produce greenhouse gases. You can find more information on the carbon pricing initiative here. A snapshot of the dashboard is shown below.

At first, creating a dashboard of this extent can seem daunting. But as we progress through this tutorial, you’ll see the relative ease at which we can recreate a dashboard with a similar (or maybe even better) set of functionalities.

Data and pre-processing with R

To start building the dashboard we need to first collect the data to visualize. For the sake of keeping it simple, we will be picking only the data for the year 2022. Thankfully, the creators of the original dashboard have given the option to download the data. 

Messy data can be a headache. Learn how to clean and validate your data using these two R packages. 

The downloaded data contains an excel file which requires a bit of fine-tuning and preprocessing. The exact procedure for the data processing is a bit beyond the scope of this tutorial. Instead, you can see the data preprocessing script here and the resulting file can be found here. Download them before proceeding as we shift focus to building the dashboard. 

Related files can be found on the World-Bank-Carbon-Pricing tutorial repo.

(Re)Designing a dashboard in Shiny

First things first, be sure to download R from here (preferably a version greater than 4.0) and RStudio from here. Once you are done, you will be greeted with a user interface (UI) with four panels.

Install Rhino and set up your Shiny project

In this UI, go to the bottom right pane and locate the Packages tab to click on the Install button. A dialog box should pop up, where you can enter Rhino as the package that you need to install.

Once it is installed, you can set up a Rhino application easily by clicking on File > New Project (or the blue box with a green plus icon). 

In the new dialog box that opens up, click on New Directory and then scroll down to select Shiny Application using Rhino. In the resulting dialog box put the directory where you want to set up your dashboard. 

Leave the checkmark for Github Actions CI on as you will need it later on when you put the dashboard into production.

Using Rhino

Now that we have Rhino set up, let’s take it for a spin to see if things are working. In the bottom right pane, click on the Files tab and locate the app.R file and open it. An editor should open up with the file contents where you can see a button called ‘Run App’. 

Click on the button and run the Shiny application written using the Rhino framework. 

The bottom right pane should pop up with a Viewer pane, that shows an empty site with a text saying Hello World.

Congrats! You’ve successfully run your first Shiny app in Rhino.

Using semantic.dashboard to speed up the dashboard-building process

While it is possible for us to manually recreate the dashboard by using CSS and additional HTML, there is an easier, faster way to build it. semantic.dashboard lets users generate a dashboard based on the look and feel of Fomantic UI. So in addition to making your life easier as a Shiny developer, it also gives your dashboards a professional look.

Does your team use Microsoft tooling? Try shiny.fluent to add a rich set of Fluent UI components!

Installing semantic.dashboard

Unlike the way you installed the Rhino package, we will install semantic.dashboard using the R console. In the default configuration of RStudio, the console is the pane on the bottom left with a prompt (‘>’). In the console, simply type the following command and press Enter.

renv::install("semantic.dashboard")

What just happened? You used the package renv to install the semantic.dashboard. The renv package helps with keeping track of which packages and what versions of them were installed in this project. At the same time, it also keeps the packages used in this project isolated from other projects. It’s a good practice to use renv when you build dashboards for the production level.

Using semantic.dashboard and box commands

Let’s add in semantic.dashboard into our project and see how easy it is to build a dashboard. First things first we will need to remove the existing code that was given by Rhino when the project was initialized. After removing the pieces the main.R file should look like the following:

box::use(
  shiny[bootstrapPage, moduleServer, NS],
)

#' @export
ui <- function(id) {
  ns <- NS(id)
  bootstrapPage(
  )
}

#' @export
server <- function(id) {
  moduleServer(id, function(input, output, session) {
  })
}

Time to sprinkle a bit of semantic.dashboard magic! Normally in R code, we use functions in a package by using library(package_name), but that would import all of the functions in the package, leading to conflicts and unnecessary weights. Since we are making this dashboard for production we have to make sure that we keep things isolated and lightweight as much as possible. 

That’s where the box commands at the top come into play. By using the ‘use’ function in the box package we can specify which function from which package we are ‘using’. When we are adding new functionalities to the dashboard, we need to add the functions we are using to the box::use command at the top. 

To start using semantic.dashboard let’s add the following commands to the box::use call.

box::use(
    shiny[moduleServer, NS, fluidRow, icon, h1],
    semantic.dashboard[
        dashboardPage,
        dashboardHeader, dashboardBody, dashboardSidebar,
        sidebarMenu, menuItem
    ]
)

Afterward, we can update the contents of the UI function as follows:

ui <- function(id) {
  ns <- NS(id)
  dashboardPage(
    dashboardHeader(left = h1("Carbon Pricing dashboard")),
    dashboardSidebar(sidebarMenu(
      menuItem(tabName = "map", text = "Map"),
      menuItem(tabName = "ghg", text = "GHG Emission Coverage"),
      menuItem(tabName = "price", text = "Price"),
      menuItem(tabName = "revenue", text = "Revenue")
    ), side = "top", visible = FALSE),
    dashboardBody(
      fluidRow(

      )
    )
  )
}

We added a new dashboard, which takes in three components: the header, the sidebar, and the body. For the header, we mentioned that a first-level heading should be on the left side. In the original dashboard, there are several tabs available for navigation. We will add these items to the sidebar and then position them to be on the top. For now, we have left the body of the dashboard empty as we will be adding these later on.

Working in production environments

While this will work on your laptop, it won’t in production environments like Posit Connect or shinyapps.io. For this, you’ll need to specify the libraries being used in the dependencies.R file in the project directory. Here you can enter the libraries required for the Shiny application to run:

# This file allows packrat (used by rsconnect during deployment) to pick up dependencies.
library(rhino)
library(semantic.dashboard)

In addition to specifying the added library in the dependencies.R file, we also need to update the renv library to make sure that anyone who uses this project can get the same set of packages that you are using.  was one of the main reasons for using renv in the beginning. To do so, you can run the following function call in the console.

renv::snapshot()

A prompt will pop up asking if the package versions should be added to the lockfile; provide yes and update the lockfile.

Interactive mapping in Shiny with Leaflet

There are many solutions for developing interactive maps in an R Shiny environment. However, the most comprehensible and versatile method would be to use Leaflet. Leaflet is a JavaScript library that can draw world maps relatively fast and it is available for use as a CRAN package. 

Leaflet vs. Tmap – which is the right choice for building interactive maps with R and Shiny?]

Installing Leaflet

Similar to the previous steps, run the following command in the console to install the Leaflet package into the renv library.

renv::install("leaflet")

Once installed, we need to place the map on the User Interface and tell from the server end how to render the map. For that, we have to use two functions: leafletOutput and renderLeaflet. The pattern of componentOutput and renderComponent is common in Shiny development; where we call the output function in the UI function to indicate where the component should be placed (or outputted) and the render function in the server function to indicate what should be rendered in the UI. 

Making stunning geomaps in R is straightforward. Follow our complete guide with Leaflet.

Using Leaflet and box commands

Let’s add the following functions to the box::use() function call at the top:

box::use(
    shiny[moduleServer, NS, fluidRow, icon, h1],
    semantic.dashboard[
        dashboardPage,
        dashboardHeader, dashboardBody, dashboardSidebar,
        sidebarMenu, menuItem
    ],
    leaflet[
        leafletOutput, renderLeaflet,
        leaflet, setView, addTiles
    ]
)

And in the UI and server function of the main.R file:

#' @export
ui <- function(id) {
  ns <- NS(id)
  dashboardPage(
    dashboardHeader(left = h1("Carbon Pricing dashboard")),
    dashboardSidebar(sidebarMenu(
      menuItem(tabName = "map", text = "Map"),
      menuItem(tabName = "ghg", text = "GHG Emission Coverage"),
      menuItem(tabName = "price", text = "Price"),
      menuItem(tabName = "revenue", text = "Revenue")
    ), side = "top", visible = FALSE),
    dashboardBody(
      fluidRow(
        leafletOutput(ns("main_map"))
      )
    )
  )
}

#' @export
server <- function(id) {
  moduleServer(id, function(input, output, session) {
    output$main_map <- renderLeaflet({ leaflet() |>
        setView(lng = -71.0589, lat = 42.3601, zoom = 12) |>
        addTiles()
    })
  })
}

Shiny modules and ns()

We added a leafletOutput call to the UI with the ID “main_map”. The ns() function stands for namespace which stems from the use of Shiny modules. Shiny modules enable us to break down a large web application into smaller components called modules. For example, in a weather dashboard, we might find a box showing the current weather forecast. As a Shiny module, this can then be reused with different data throughout the dashboard. 

Namespaces 7 enables us to define outputs and inputs with identifiers that do not collide with other Shiny modules. For more info check out Posit’s article on Shiny Modules. 

In addition, while rendering the Leaflet map in the renderLeaflet function, we generate a map by calling the leaflet() function and mentioning where we want to view the map. A set of tiles is added through the addTiles() function that draws the different features of the map. 

Now when you run the app a map should be visible in the dashboardBody area. Don’t forget, we will need to add Leaflet to the dependencies and renv library now that it’s being used in our Shiny application.

Update the dependencies.R file as follows:

# This file allows packrat (used by rsconnect during deployment) to pick up dependencies.
library(rhino)
library(semantic.dashboard)
library(leaflet)

And update the renv library by running the following command in the console:

renv::snapshot()

Adding pre-processed data to a Shiny dashboard

To use actual data in the project, we need to download the processed data files and put them in a directory. We can then use these datasets to plot the map.

For that let’s create an empty directory in our repo named data-raw and download the three .rds files (nationals.rds, subnationals.rds, and regionals.rds) into the folder. Once downloaded we can begin to plot our map. We can type the code to generate the plot in the main.R but that would lead to a really long file at the end of rendering all the outputs. 

Besides, the main.R file should tell from a glance, what the structure of the Shiny app is while the rest of the processing part or logic can reside outside in separate R files. 

Luckily, Rhino already has us covered by creating a separate folder in the app folder called logic. Here you can put all the R files related to the logic of your Shiny application.

Create a new R file in RStudio and save it in the app/logic folder as plot_map.R. Insert the following code segment into the file and save it again.

box::use(
    leaflet[
        leaflet, leafletOptions,
        providers, addProviderTiles, providerTileOptions,
        awesomeIcons, addAwesomeMarkers, addPolygons,
        colorFactor, highlightOptions, addLayersControl, layersControlOptions],
    dplyr[recode, pull, filter],
    sf[st_as_sf]
)
plot_leaflet_map <- function(nationals, regionals, subnationals) {
  ldf <- rbind(nationals, regionals) |>
    st_as_sf()
  lf_map <- leaflet(options = leafletOptions(minZoom = 3)) |>
    addProviderTiles(
      providers$Esri.WorldTopoMap,
      options = providerTileOptions(minZoom = 0, maxZoom = 18)
    )

  for (stype in unique(subnationals$Type)) {
    icons <- awesomeIcons( icon = recode(subnationals|>
                      filter(Type == stype) |>
                      pull(Status),
                    "Implemented" = "fa-check",
                    "Scheduled" = "fa-clock-o",
                    "Under consideration" = "fa-spinner"),
      iconColor = "black",
      library = "fa",
      markerColor = recode(subnationals |>
                             filter(Type == stype) |>
                             pull(Type),
                           "ETS" = "blue", "Carbon tax" = "green")
    )
    lf_map <- lf_map |>
      addAwesomeMarkers(data = subnationals |> filter(Type == stype),
                        lat = ~lat, lng =~lon, icon=icons,
                        label = ~paste(`Name of the initiative`),
                        group = paste0("", stype))
  }

  for (ntype in unique(ldf$Type)) {
    lf_map <- lf_map |>
      addPolygons(data = ldf |> filter(Type == ntype),
                  weight = 1, group = paste0("", ntype),
                  fillColor = ~colorFactor(c("green", "blue", "gray"),
                                           domain = ldf$Type)(Type),
                  color = "white", label = ~paste(`Name of the initiative`),
                  highlightOptions = highlightOptions(color = "white",
                                                      weight = 2,
                                                      bringToFront = TRUE))
  }

  lf_map |>
    addLayersControl(
      overlayGroups = c(paste0("", unique(subnationals$Type)),
                        paste0("", unique(ldf$Type))),
      options = layersControlOptions(collapsed = FALSE)
    )
}

Adding data to the Leaflet map

Both national and regional datasets contain data on carbon initiatives across the world. For this, we will need to plot an area. The subnational dataset contains data on initiatives taken by certain cities or organizations and therefore we’ll need to pinpoint an exact coordinate consisting of a latitude and longitude. 

Firstly, we combine and convert the nationals and regionals into a single data frame of simple features to be used with Leaflet for plotting. Now to set up the Leaflet map; we set the minimum level of zoom to 3 so that users won’t try to zoom too far back by accident. It’s a simple step that web map creators can make to avoid negative user experience.

Changing Leaflet tiles

Earlier we used the addTiles function to generate the tiles to be plotted on the Leaflet map. For this, we will be using a different provider to give a better look and feel for the map. In this case the Esri.WorldTopoMap provider will be used. If you want to change to a different provider, this gallery contains a preview of the existing options.

Icons, point features, and polygons in Leaflet

After that, we add the subnational markers to the plot by defining the icons to be used and other details necessary to plot the markers. The icons are going to indicate the type of initiative with the color and the status of the initiative with the icon. After that, we go through national and regional areas and add polygons to the areas along with the ability to highlight the regions. Both markers and polygons have the name of the initiative and the status as a label when a user hovers over the area.

For this code piece to work several new packages will have to be installed as seen in the box::use function call. For that run the following commands in the console:

renv::install(c("dplyr","sf"))
renv::snapshot()

And update the dependencies.R file as follows:

# This file allows packrat (used by rsconnect during deployment) to pick up dependencies.
library(rhino)
library(semantic.dashboard)
library(leaflet)
library(dplyr)
library(sf)

Now to use this function in our main.R file, we will need to first read the data in. So let’s add the following in the main.R:

    nationals <- readRDS("data-raw/nationals.rds")
    subnationals <- readRDS("data-raw/subnationals.rds")
    regionals <- readRDS("data-raw/regionals.rds")

    output$main_map <- renderLeaflet({
      plot_leaflet_map(nationals, regionals, subnationals)
    })

Final dashboard redesign built with Shiny and Rhino

At this point, the dashboard should look visually similar to what is available on the Carbon Pricing dashboard – with a few changes in styling and map aesthetics with how the icons in the maps are showcased.

If you’re looking to upgrade your enterprise dashboard or are curious about how to get started using Rhino for your projects, contact Appsilon. And if you’re just getting started and want to explore Shiny, download a free Shiny template and start sharing your data insights today!

The post Redesigning Dashboards with Shiny and Rhino: World Bank’s Carbon Pricing appeared first on Appsilon | Enterprise R Shiny Dashboards.

Thanks for visiting r-craft.org
This article is originally published at https://appsilon.com
Please visit source website for post related comments.