Generalized Shiny observer deduplication

A workaround for shiny issues #825 and #4045

Problem statement and current state of affairs

Dynamic creation of Shiny observers can lead to problems, as succinctly put by Winston Chang in Shiny issue #825:

if an observer is created in a reactive or observer, it will stay around forever, and keep responding to its inputs

The issue is also nicely explored by Kyle Husmann here.

One way to solve this problem is to collect the dynamically generated observers and destroy()them before launching their replacements. However, that is a finicky transformation that snowballs in complexity when those observers arise from repeat calls to Shiny modules (as described in this comment and also in Shiny issue #4045).

A solution that scales

We believe the way forward is Chang’s original proposal of collecting the observers under a “reactive subdomain”. The example code he provided (11 years ago!) does not work anymore, but its spirit can be adapted to modern Shiny releases.

Here’s the code, but please keep on reading before attempting to understand it.

The interface of the function that captures observers is simple:

observer_dedup(id, expr, session, verbose)

Here, id is a string that uniquely identifies the observer generator, expr is the expression that will generate the observers, session is the caller’s Shiny session and verbose is a boolean that enables observer removal diagnostic messages.

The core of that routine, unsurprisingly, runs expr inside a new reactive domain:

...
state[["subdomain"]] <- shiny:::createSessionProxy(
  session,
  makeScope = make_scope_that_captures_callbacks,
  onEnded = capture_callbacks,
  end = invoke_and_remove_callbacks
)
    
expr <- substitute(expr)
env <- parent.frame()
result <- shiny::withReactiveDomain(state[["subdomain"]], eval(expr, env))
...

As you can see by the use of the triple colon, the original drawback of using an unexported Shiny function is still present here, although one could argue this is less of a problem today¹.

When the observer_dedup is reinvoked with the same id, all the previously captured observers under that id are destroyed.

Examples of use

Applying this function to Shiny issue #825 is as simple as:

server <- function(input, output, session) {
  observe({
    foo <- input$foo
    
    observer_dedup(                                             # change
      id = 'dummy_id', session = session, verbose = TRUE,       # change
      expr =  observe({
        print(paste0("foo was: ", foo, ". bar: ", input$bar))
      })
    )                                                           # change
  })
}

Kyle Husmann’s example ( the “attempt 2” version) is similarly solved with:

observeEvent(input$column_select, {
  walk(input$column_select, function(i) {
    close_btn_id <- glue("{i}_close")
    observer_dedup(                                         # change
      id = close_btn_id, session = session, verbose = TRUE, # change
      expr = observeEvent(
        input[[close_btn_id]],
        {
          appendMessage(glue("Closing {i}"))
          updateSelectInput(
            inputId = "column_select",
            selected = discard(input$column_select, \(j) j == i)
          )
        },
        ignoreInit = TRUE
      )
    )                                                       # change
  })
})

And as for Shiny issue #4045, this tweak does the job²:

server <- function(input, output, session) {  
 observe({
    lapply(1:input$num, function(i) {
      observer_dedup(                                                 # change
        id = paste0(’server’, i), session = session, verbose = TRUE,  # change
        expr = selectInputServer(paste0(”select”, i))
      )                                                               # change
    })
  })
  ...
}

A proposal

We could write a wrapper around moduleServer to do automatic observer deduplication, using that function’s id parameter as the identifier for deduplication. But it would be even better for Shiny itself to provide that behavior, at the cost of one extra reactive domain per module. This would turn #4045 into a non-issue.

The case for stand-alone calls to observe, observeEvent, etc. is not so clear-cut. They don’t have an id parameter that can make deduplication transparent. However, if they had such a parameter, solving issue #825 could be as simple as providing id = “my_id” to the nested observer. That new parameter would be optional. Observers that omitted it would incur no deduplication cost.

1. This technique works unmodified since Shiny version 0.13.0 (2016) and we know of no scalable alternative to it. It feels like this function wants to become public and fill a void in the Shiny API. Or, to put it in less impersonal terms, we feel our use of this private function is justified. We will explore this (possibly misguided) sense of entitlement in a future post by the name of “Hyrum’s Usucaption”.

2. For this last example, there’s the unrelated issue of preventing actionButtons from firing when they are recreated. We can do that by appending ignoreInit = TRUE to the end of `bindEvent(input$show_popup, ignoreInit = TRUE)`.