Let be a stochastic process with mean , and variance . Let be an observed time series of .
A good estimator for is . We know that if the observations are IID, . However, if the observations are not IID, the variance will be larger – this is what I learned recently in Loucks (2005). The authors skipped some details of the book, so I worked them out. Below is a detailed proof.
Now, on one hand, the summands where can be grouped; on the other hand, note that when , there is one summand for and one summand for . Therefore,
Let , in other words, denotes the lag between the and timesteps. (1) becomes
where is the lag- autocorrelation and is defined as
Observe that compared to the IID case, the variance of the sample mean estimator is inflated by a factor bigger than 1. Furthermore, it can be checked that this factor does not decrease as increases. We conclude that the sample mean of an autocorrelated time series always has a bigger standard error than that of an IID time series with the same variance.
Loucks, D.P et al. (2005). Water Resources Systems Planning and Management (Chapter 7, p. 198-201). UNESCO Publication. (The book is publicly available on the UNESCO website)