Part 1 Elements of a design
1.1 Overview
The key idea of the DeclareDesign
simulation workflow is that we simulate the design as a set of steps. When concatenated the steps produce an instance of a design. When replicated these instances let us assess the overall performance of a design.
Fully specified designs include information on:
- M: The background model—for instance, background beliefs about network structures.
- I: The inquiries, i.e. what exactly is the quantity we want to estimate. The estimands should be readable from the background model.
- D: What is the data strategy? For instance how will sampling be undertaken?
- A: What is the answer strategy? How will estimation be done?
Each of these MIDA steps can be defined using “handlers” from the hiddenmeta
package and can be adjusted to reflect details of individual study designs.
1.2 Individual designs
The individual study design simulation involves:
- Selecting parameters
- Using these to declare a design
- Diagnosing the design
1.2.1 Study design features are provided as arguments
argument | value |
---|---|
N | 2000 |
K | 2 |
prev_K | 0.3, 0.1 |
rho_K | 0.05 |
p_edge_within | c(0.05, 0.05), c(0.05, 0.9) |
p_edge_between | 0.05, 0.01 |
p_visibility | 0.99, 0.7 |
add_groups | 0.3, 0.3, 0.1, 0.2, 0.1, 0.2 |
n_seed | 20 |
n_coupons | 3 |
target_type | sample |
target_n_rds | 100 |
target_n_pps | 400 |
target_n_tls | 1 |
1.2.2 Declaration
These arguments are used to declare a complete design
# Simulate a network
population <-
do.call(what = declare_population,
args = c(handler = get_study_population, study_1[1:8]))
# Simulate RDS sampling
rds_sampling <-
do.call(declare_sampling,
args = c(handler = sample_rds,
sampling_variable = "rds",
drop_nonsampled = FALSE, study_1[9:12]))
# Simulate PPS sampling
pps_sampling <-
do.call(declare_sampling,
c(handler = sample_pps,
sampling_variable = "pps",
drop_nonsampled = FALSE, study_1[13]))
# Simulate TLS sampling
tls_sampling <-
do.call(declare_sampling,
c(handler = sample_tls,
sampling_variable = "tls",
drop_nonsampled = FALSE, study_1[14]))
# Declare estimands
estimands <-
declare_estimand(handler = get_study_estimands)
# Declare SSPSE estimator
estimator_sspse <-
declare_estimator(handler = get_study_est_sspse, label = "sspse")
# Declare HT estimator
estimator_ht <-
declare_estimator(handler = get_study_est_ht, label = "ht")
# Declare Chords estimator
estimator_chords <-
declare_estimator(type = "integrated",
handler = get_study_est_chords, label = "chords")
# Declare NSUM estimator
estimator_nsum <- declare_estimator(handler = get_study_est_nsum, label = "nsum")
# Declare the complete study level design
design <-
population +
estimands +
rds_sampling + pps_sampling + tls_sampling +
estimator_sspse + estimator_ht + estimator_chords + estimator_nsum
1.2.3 Diagnosis
Finally we can diagnose an individual study design. Diagnosis is done by running the whole design from beginning to end many times. Each time estimands and estimates are calculated and these are compared against each other.
study_diagnosands <-
declare_diagnosands(
mean_estimand = mean(estimand),
mean_estimate = mean(estimate),
sd_estimate = sd(estimate),
mean_se = mean(se),
bias = mean(estimate - estimand),
rmse = sqrt(mean((estimate - estimand) ^ 2))
)
diagnosis <-
diagnose_design(design,
diagnosands = study_diagnosands,
sims = 100, bootstrap_sims = 100)
Estimand Label | Estimator Label | Mean Estimand | Mean Estimate | SD Estimate | Bias | RMSE |
---|---|---|---|---|---|---|
degree_average | degree_average_nsum | 5.12 | 5.13 | 0.19 | 0.01 | 0.18 |
(0.01) | (0.02) | (0.01) | (0.02) | (0.02) | ||
degree_hidden_average | degree_hidden_chords | 0.99 | 6.54 | 0.64 | 5.55 | 5.58 |
(0.01) | (0.06) | (0.03) | (0.06) | (0.06) | ||
hidden_prev | hidden_prev_ht | 0.10 | 0.10 | 0.03 | -0.00 | 0.03 |
(0.00) | (0.00) | (0.00) | (0.00) | (0.00) | ||
hidden_size | hidden_size_chords | 199.89 | 215.59 | 46.31 | 15.70 | 52.34 |
(1.35) | (4.96) | (3.70) | (5.24) | (5.14) | ||
hidden_size | hidden_size_nsum | 199.89 | 253.47 | 35.41 | 53.58 | 61.67 |
(1.35) | (3.75) | (2.69) | (3.26) | (3.13) | ||
hidden_size | hidden_size_sspse | 199.89 | 189.88 | 33.21 | -10.02 | 35.84 |
(1.35) | (3.33) | (1.90) | (3.65) | (2.37) |
1.3 Meta-design
Simulated study level results feed into the meta-analysis.