Part 1 Elements of a design

1.1 Overview

The key idea of the DeclareDesign simulation workflow is that we simulate the design as a set of steps. When concatenated the steps produce an instance of a design. When replicated these instances let us assess the overall performance of a design.

Fully specified designs include information on:

  1. M: The background model—for instance, background beliefs about network structures.
  2. I: The inquiries, i.e. what exactly is the quantity we want to estimate. The estimands should be readable from the background model.
  3. D: What is the data strategy? For instance how will sampling be undertaken?
  4. A: What is the answer strategy? How will estimation be done?

Each of these MIDA steps can be defined using “handlers” from the hiddenmeta package and can be adjusted to reflect details of individual study designs.

1.2 Individual designs

The individual study design simulation involves:

  1. Selecting parameters
  2. Using these to declare a design
  3. Diagnosing the design

1.2.1 Study design features are provided as arguments

(#tab:example_parameters)Parameters
argument value
N 2000
K 2
prev_K 0.3, 0.1
rho_K 0.05
p_edge_within c(0.05, 0.05), c(0.05, 0.9)
p_edge_between 0.05, 0.01
p_visibility 0.99, 0.7
add_groups 0.3, 0.3, 0.1, 0.2, 0.1, 0.2
n_seed 20
n_coupons 3
target_type sample
target_n_rds 100
target_n_pps 400
target_n_tls 1

1.2.2 Declaration

These arguments are used to declare a complete design


# Simulate a network
population <-   
  do.call(what = declare_population,
          args = c(handler = get_study_population, study_1[1:8])) 

# Simulate RDS sampling
rds_sampling <-   
  do.call(declare_sampling,
          args = c(handler = sample_rds, 
            sampling_variable = "rds",
            drop_nonsampled = FALSE, study_1[9:12]))

# Simulate PPS sampling
pps_sampling <- 
  do.call(declare_sampling,
          c(handler = sample_pps, 
            sampling_variable = "pps",
            drop_nonsampled = FALSE, study_1[13]))

# Simulate TLS sampling
tls_sampling <- 
  do.call(declare_sampling,
          c(handler = sample_tls, 
            sampling_variable = "tls",
            drop_nonsampled = FALSE, study_1[14]))

# Declare estimands
estimands <- 
  declare_estimand(handler = get_study_estimands)

# Declare SSPSE estimator
estimator_sspse <- 
  declare_estimator(handler = get_study_est_sspse, label = "sspse")

# Declare HT estimator
estimator_ht <- 
  declare_estimator(handler = get_study_est_ht, label = "ht")

# Declare Chords estimator
estimator_chords <- 
  declare_estimator(type = "integrated",
                    handler = get_study_est_chords, label = "chords")

# Declare NSUM estimator
estimator_nsum <- declare_estimator(handler = get_study_est_nsum, label = "nsum")

# Declare the complete study level design
design <- 
  population + 
  estimands +
  rds_sampling + pps_sampling + tls_sampling +
  estimator_sspse + estimator_ht + estimator_chords + estimator_nsum

1.2.3 Diagnosis

Finally we can diagnose an individual study design. Diagnosis is done by running the whole design from beginning to end many times. Each time estimands and estimates are calculated and these are compared against each other.


study_diagnosands <-
  declare_diagnosands(
    mean_estimand = mean(estimand),
    mean_estimate = mean(estimate),
    sd_estimate = sd(estimate),
    mean_se = mean(se),
    bias = mean(estimate - estimand),
    rmse = sqrt(mean((estimate - estimand) ^ 2))
  )

diagnosis <- 
  diagnose_design(design, 
                  diagnosands = study_diagnosands,
                  sims = 100, bootstrap_sims = 100)
(#tab:report_example_diagnose)Design diagnosis
Estimand Label Estimator Label Mean Estimand Mean Estimate SD Estimate Bias RMSE
degree_average degree_average_nsum 5.12 5.13 0.19 0.01 0.18
(0.01) (0.02) (0.01) (0.02) (0.02)
degree_hidden_average degree_hidden_chords 0.99 6.54 0.64 5.55 5.58
(0.01) (0.06) (0.03) (0.06) (0.06)
hidden_prev hidden_prev_ht 0.10 0.10 0.03 -0.00 0.03
(0.00) (0.00) (0.00) (0.00) (0.00)
hidden_size hidden_size_chords 199.89 215.59 46.31 15.70 52.34
(1.35) (4.96) (3.70) (5.24) (5.14)
hidden_size hidden_size_nsum 199.89 253.47 35.41 53.58 61.67
(1.35) (3.75) (2.69) (3.26) (3.13)
hidden_size hidden_size_sspse 199.89 189.88 33.21 -10.02 35.84
(1.35) (3.33) (1.90) (3.65) (2.37)

1.3 Meta-design

Simulated study level results feed into the meta-analysis.