BOOTSTRAPPING

Bootstrapping is the process of resampling *with replacement* (*all* values in the sample have an *equal* probability of being selected, including multiple times, so a value could have a duplicate). Resample, calculate a statistic (e.g. the mean), repeat this hundreds or thousands of times and you are able to estimate a precise/accurate uncertainty of the mean (confidence interval) of the data’s distribution. There are less assumptions about the underlying distribution using bootstrap compared to calculating the standard error directly.

PERMUTATION TESTING

Similar to bootstrapping, except permutation testing resamples *without replacement* (meaning when a value is selected, it can not be selected again, so no value can be a duplicate). This simply shuffles the values. In the case of a univariate statistic (e.g. mean), this will not change anything. However, when there are two or more variables, reshuffling one variable will change the test statistic (e.g. correlation or regression). Usually this is done on the response/outcome/y variable, and usually for tests that use discrete or categorical (“yes”, “no”) variables.

```
ds <- data.frame(y = runif(10), x = runif(10))
ds
```

```
## y x
## 1 0.26248353 0.60789022
## 2 0.27608331 0.81612978
## 3 0.35928985 0.47067467
## 4 0.34308694 0.86670014
## 5 0.61019344 0.23596764
## 6 0.31907683 0.53597891
## 7 0.51808777 0.77629172
## 8 0.39792031 0.09387856
## 9 0.04529551 0.75595895
## 10 0.56524394 0.44996910
```

```
# Using spearman correlation to be consistent with the next example
cor(ds$y, ds$x, method = "spearman")
```

```
## [1] -0.5393939
```

```
# Resampled the y only (reshuffled the order)
ds$resample_y <- sample(ds$y)
ds
```

```
## [1] -0.07878788
```

```
library(coin)
```

```
## Loading required package: methods
```

```
## Loading required package: survival
```

```
##
## Attaching package: 'survival'
```

```
## The following object is masked from 'package:boot':
##
## aml
```

```
ds <- data.frame(y, x)
```

```
## Error in data.frame(y, x): object 'y' not found
```

```
spearman_test(y ~ x, data = ds)
```

```
##
## Asymptotic Spearman Correlation Test
##
## data: y by x
## Z = -1.6182, p-value = 0.1056
## alternative hypothesis: true rho is not equal to 0
```

There are many more tests found within the `coin`

package. Check them out in the vignette:

`vignette("coin", package = "coin")`