Diagnostic Accuracy (sensitivity/specificity) Sample Size Calculator

Diagnostics studies are relatively simple to conduct but require careful planning and data collection. It is important to identify gold standard test and diagnostic criteria you are going to use (usually some sort of standard of care) and an experimental test you are going to study. You must have an idea of the prevalence of the disease you are studying and the estimated sensitivity and specificity of the experimental diagnostics. Sometimes, we want to correlate two diagnostic measures that are not binary, like Cd4 and Viral load. This is a different type of analysis and we will cover this in a future post. There is also finding a threshold of a continuous diagnostic, perhaps a viral load, that is most predictive of a binary outcome, like death. This is also a different type of analysis and we will cover this in a future post.

It is crucial to include enough subjects in both the diseased and non-diseased groups. Two key performance measures are sensitivity (the test’s ability to correctly identify those with the disease) and specificity (the test’s ability to correctly identify those without the disease).

To estimate either sensitivity or specificity with a desired level of precision, you can use the standard formula for proportions:

\[ n=z^2×p(1−p)d^2 \]

\[ n = \frac{z^2 \times p(1-p)}{d^2} \]

\[ n=d^2z^2×p(1−p)​ \]

Where:

• \(n\) is the required sample size within the subgroup (diseased or non-diseased),

• \(z\) is the critical value corresponding to your desired confidence level (for a 95% confidence interval, \(z≈1.96z\))

• \(p\) is the expected sensitivity (or specificity), and

• \(d\) is the desired margin of error (half-width of the confidence interval).

Practical Example:
Suppose you expect your diagnostic test to have a sensitivity of 90% (0.90) and you desire a 95% confidence interval with a margin of error of ±5% (0.05). The required number of diseased subjects is:

\[ n_{\text{disease}} =(1.96)^2×0.90(1−0.90)(0.05)^2≈139 \] \[ n_{\text{disease}} = \frac{(1.96)^2 \times 0.90(1-0.90)}{(0.05)^2} \approx 139 \] \[ n_{\text{disease}} =(0.05)^2(1.96)^2×0.90(1−0.90)​≈139 \]

Similarly, if you expect a specificity of 90%, you would also need approximately 139 non-diseased subjects to achieve the same precision.

If the prevalence of the disease in your study population is known, you can determine the overall total sample size required. For example, if the disease prevalence is 10% (0.10), then to obtain about 139 diseased subjects you would need approximately:

Total sample size (for disease)=1390.10≈1390 = 1390Total sample size (for disease)=0.10139​≈1390

Likewise, you can calculate the total number needed to yield the required number of non-diseased subjects.

Reference:
Buderer, N. M. F. (1996). Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Academic Emergency Medicine, 3(9), 895-900.

Interactive Example

Below is an interactive Shiny tool pre‐set to the example above. The default inputs assume:

• An expected sensitivity and specificity of 90%

• A desired margin of error of 5%

• A 95% confidence level (with z≈1.96z 1.96z≈1.96)

• A disease prevalence of 10%

Users can adjust these parameters to match the specifics of their own diagnostic study.

#| '!! shinylive warning !!': |
#|   shinylive does not work in self-contained HTML documents.
#|   Please set `embed-resources: false` in your metadata.
#| standalone: true
#| echo: false 
#| messages: false
#| viewerHeight: 800

library(shiny)
library(shinythemes)
library(shinyjs)

ui <- fluidPage(
  theme = shinytheme("cyborg"),
  useShinyjs(),
  tags$style(HTML("
    #explanationText {
      font-family: 'Times New Roman', Times, serif;
      white-space: pre-wrap;
    }
    .copy-button {
      margin-bottom: 10px;
    }
  ")),
  tags$script(HTML("
    function copyExplanation() {
      var explanation = document.getElementById('explanationText').innerText;
      var tempInput = document.createElement('textarea');
      tempInput.value = explanation;
      document.body.appendChild(tempInput);
      tempInput.select();
      document.execCommand('copy');
      document.body.removeChild(tempInput);
      alert('Complete calculation with your current inputs copied to clipboard!');
    }
  ")),
  
  titlePanel("Sample Size Calculator for Diagnostic Accuracy Studies"),
  sidebarLayout(
    sidebarPanel(
      numericInput("sens", "Expected Sensitivity:", value = 0.90, min = 0.5, max = 1, step = 0.01),
      numericInput("spec", "Expected Specificity:", value = 0.90, min = 0.5, max = 1, step = 0.01),
      numericInput("d", "Desired Margin of Error (d):", value = 0.05, min = 0.01, max = 0.2, step = 0.01),
      numericInput("alpha", "Significance Level (α):", value = 0.05, min = 0.001, max = 0.1, step = 0.005),
      numericInput("prev", "Disease Prevalence:", value = 0.10, min = 0.001, max = 1, step = 0.01)
    ),
    mainPanel(
      h3(textOutput("resultTitle")),
      uiOutput("resultsUI"),
      tags$button("Copy Full Calculation", class = "copy-button", onclick = "copyExplanation()"),
      uiOutput("explanationUI"),
      plotOutput("barPlot")
    )
  )
)

server <- function(input, output, session) {
  
  # Calculate critical value from alpha
  calc_z <- reactive({
    qnorm(1 - input$alpha/2)
  })
  
  # Calculate required sample sizes for sensitivity and specificity (in diseased and non-diseased groups)
  calc_sample_sizes <- reactive({
    z <- calc_z()
    sens <- input$sens
    spec <- input$spec
    d <- input$d
    prev <- input$prev
    
    # Sample size for diseased subjects (for sensitivity)
    n_disease <- (z^2 * sens * (1 - sens)) / (d^2)
    # Sample size for non-diseased subjects (for specificity)
    n_nondisease <- (z^2 * spec * (1 - spec)) / (d^2)
    
    # Overall total required given the prevalence
    total_disease <- n_disease / prev
    total_nondisease <- n_nondisease / (1 - prev)
    overall_total <- ceiling(max(total_disease, total_nondisease))
    
    list(n_disease = ceiling(n_disease),
         n_nondisease = ceiling(n_nondisease),
         total_disease = ceiling(total_disease),
         total_nondisease = ceiling(total_nondisease),
         overall_total = overall_total)
  })
  
  output$resultTitle <- renderText({
    "Estimated Sample Size Requirements"
  })
  
  output$resultsUI <- renderUI({
    sizes <- calc_sample_sizes()
    tagList(
      tags$p(paste("Required number of diseased subjects (for sensitivity):", sizes$n_disease)),
      tags$p(paste("Estimated total sample size needed to yield sufficient diseased subjects:", sizes$total_disease)),
      tags$p(paste("Required number of non-diseased subjects (for specificity):", sizes$n_nondisease)),
      tags$p(paste("Estimated total sample size needed to yield sufficient non-diseased subjects:", sizes$total_nondisease)),
      tags$p(paste("Overall total sample size required (largest estimate):", sizes$overall_total))
    )
  })
  
  output$explanationUI <- renderUI({
    sizes <- calc_sample_sizes()
    z <- calc_z()
    sens <- input$sens
    spec <- input$spec
    d <- input$d
    alpha <- input$alpha
    prev <- input$prev
    confidence_level <- (1 - alpha) * 100
    
    # Calculate intermediate values for explanation
    sens_numerator <- z^2 * sens * (1 - sens)
    spec_numerator <- z^2 * spec * (1 - spec)
    denominator <- d^2
    
    explanation <- paste0(
      "CALCULATION BASED ON YOUR INPUTS:\n\n",
      "Parameters:\n",
      "• Expected Sensitivity: ", round(sens, 3), " (", round(sens*100, 1), "%)\n",
      "• Expected Specificity: ", round(spec, 3), " (", round(spec*100, 1), "%)\n",
      "• Desired Margin of Error: ±", round(d, 3), " (±", round(d*100, 1), "%)\n",
      "• Confidence Level: ", round(confidence_level, 1), "% (α = ", alpha, ", z = ", round(z, 3), ")\n",
      "• Disease Prevalence: ", round(prev, 3), " (", round(prev*100, 1), "%)\n\n",
      "Formula: n = (z² × p(1-p)) / d²\n\n",
      "For SENSITIVITY (diseased subjects needed):\n",
      "    n_sens = (", round(z, 3), "² × ", round(sens, 3), " × (1-", round(sens, 3), ")) / ", round(d, 3), "²\n",
      "    n_sens = (", round(z^2, 3), " × ", round(sens, 3), " × ", round(1-sens, 3), ") / ", round(d^2, 4), "\n",
      "    n_sens = ", round(sens_numerator, 4), " / ", round(denominator, 4), " = ", round(sens_numerator/denominator, 1), "\n",
      "    Required diseased subjects: ", sizes$n_disease, "\n\n",
      "For SPECIFICITY (non-diseased subjects needed):\n",
      "    n_spec = (", round(z, 3), "² × ", round(spec, 3), " × (1-", round(spec, 3), ")) / ", round(d, 3), "²\n",
      "    n_spec = (", round(z^2, 3), " × ", round(spec, 3), " × ", round(1-spec, 3), ") / ", round(d^2, 4), "\n",
      "    n_spec = ", round(spec_numerator, 4), " / ", round(denominator, 4), " = ", round(spec_numerator/denominator, 1), "\n",
      "    Required non-diseased subjects: ", sizes$n_nondisease, "\n\n",
      "TOTAL SAMPLE SIZE CALCULATIONS:\n",
      "Given disease prevalence of ", round(prev*100, 1), "% (", round(prev, 3), "):\n\n",
      "To obtain ", sizes$n_disease, " diseased subjects:\n",
      "    Total needed = ", sizes$n_disease, " / ", round(prev, 3), " = ", sizes$total_disease, "\n\n",
      "To obtain ", sizes$n_nondisease, " non-diseased subjects:\n",
      "    Total needed = ", sizes$n_nondisease, " / ", round(1-prev, 3), " = ", sizes$total_nondisease, "\n\n",
      "FINAL RECOMMENDATION:\n",
      "Overall total sample size required: ", sizes$overall_total, " participants\n",
      "(This is the larger of the two total sample size estimates)\n\n",
      "With this sample size, you can expect:\n",
      "• Approximately ", round(sizes$overall_total * prev), " diseased subjects\n",
      "• Approximately ", round(sizes$overall_total * (1-prev)), " non-diseased subjects\n\n",
      "Reference: Buderer, N. M. F. (1996). Statistical methodology: I. Incorporating the prevalence of disease into the sample size calculation for sensitivity and specificity. Academic Emergency Medicine, 3(9), 895-900."
    )
    
    tags$pre(explanation, id = "explanationText")
  })
  
  output$barPlot <- renderPlot({
    sizes <- calc_sample_sizes()
    bar_data <- c(sizes$total_disease, sizes$total_nondisease)
    bar_labels <- c("Total for Diseased", "Total for Non-Diseased")
    
    barplot(bar_data, names.arg = bar_labels,
            col = c("tomato", "skyblue"),
            main = "Estimated Total Sample Sizes",
            ylab = "Number of Participants",
            ylim = c(0, max(bar_data) * 1.1))
    abline(h = sizes$overall_total, col = "darkgreen", lty = 2)
    legend("topright", legend = paste("Overall Required =", sizes$overall_total),
           bty = "n", col = "darkgreen", lty = 2)
  })
}

shinyApp(ui = ui, server = server)