Metrics for model accuracy
Mean bias error (MBE) and root mean square error (RMSE) are used to measure the model accuracy. The bias \(\Delta\) is the difference between SolarFarmer predictions and the actual AC power output for each time stamp:
$$\Delta P_\text{AC}= P_\text{AC, SolarFarmer} - P_\text{AC, Actual}$$
According to this definition, the bias is positive if SolarFarmer predicts more than actual power and negative otherwise. Aggregating the biases over all time stamps \(N\), we can calculate \(\mathit{MBE}\):
$$\mathit{MBE}=\frac{{ \sum_{n = 1}^{N} {\Delta P_\text{AC, n}}}}{N}$$
The bias tells the average difference between the modelled and actual power and the \(\mathit{RMSE}\) tells the spread of the difference. The \(MBE\) might contain a mix of positive and negative biases which cancel out, yielding a low average, whereas the \(\mathit{RMSE}\) will increase to show that there is a spread of positive and negative values:
$$\mathit{RMSE}=\sqrt{\frac{{ \sum_{n = 1}^{N} ({\Delta P_\text{AC, n}}})^2}{N}}$$
The absolute \(\mathit{MBE}\) and \(\mathit{RMSE}\) are more informative by giving them some context by scaling them by average expected value over the same time interval, here the measured AC power output. We can define the relative mean bias \(\mathit{rMBE}\) and the relative root mean square error \(\mathit{rRMSE}\) as:
$$\mathit{rMBE}=\frac{\mathit{MBE}}{\bar{P}_\text{AC, Actual}}$$
$$\mathit{rRMSE}=\frac{\mathit{RMSE}}{\bar{P}_\text{AC, Actual}}$$
$$\bar{P}_\text{AC, Actual}=\frac{{ \sum_{n = 1}^{N} {P_\text{AC, Actual, n}}}}{N}$$
It should be noted that if the scaling factor is relatively small compared to the bias, then the relative bias will be disproportionately larger than in other intervals. For example, relative errors in the early morning in winter for sites in the northern hemisphere are much larger than the average annual error because of the lower actual power output compared to other time of day or month in a year. Therefore relative errors can be aggregated over longer time periods to reduce this misrepresentation.