Goodness-of-fit evaluation

I have read some text on statistics to look for the most appropriate method for my case study data analysis. In this post, I am going to summarizes what I have learned on the goodness-of-fit analysis.

1. Student t-test
T-test can be used to test whether two samples are significantly different. If you want to test whether there is a significant difference between means of two samples, you can apply t-test.

2. Paired t-test
In the t-test, if two observations can be naturally paired together, you can use paired t-test. Note that paired t-test is one-tailed analysis. Some people also call it dependent t-test.

3. Chi-Square Goodness-of-fit test
The chi-square goodness-of-fit test can evaluate the discrepancy between the actual values and expected values. Note that the values can not be continuous real numbers but frequencies of events.

4. F-Test
The f-test can also be used for goodness-of-fit test. Given two models, you can use f-test to check whether one fits with the other.

In my analysis, I have chosen chi-square goodness-of-fit test. It is questionable because the episode number is not a frequency value.

The paired t-test seems right for my data analysis, which validate the video analysis results against Zorro's inference results, but t-test can not measure how close two data sets are.

Christian D. Schunn and Dieter Wallach survey the goodness-of-fit analysis methods. They said that there is no formal standard for goodness-of-fit evaluation.

"Unfortunately, there are no formal standards for how to evaluate the quantitative goodness-of-fit of models to data, either visually or numerically."

In the end, they recommended the combination of r-square and root mean squared deviation.