Test Statistics

The most common are:  z, t, f, x2

The probability distributions are standardized sampling distributions.

Standardized distributions have a mean = 0 and a s = 1.

Always calculate descriptive statistics on all your key research variables because your variables will have to approximate the same distribution as the test statistic that you want to use.  If it does not than you need to do a different statistical analysis, or transform your data. 

Means tests, correlations and regression use t test statistics which requires normality. Nominal and ordinal data (categorical) don’t form normal distributions so for your purposes you can’t do means tests, correlations or regression with categorical data.  The data must be interval or ratio (continuous). Even then sometimes you have an interval variable that does not approximate a normal distribution.  For example, in the Bible Belt, the number of alcoholic beverages consumed per week probably has a large percentage of people who do not drink any alcoholic beverages. Hence, the distribution would not approximate a normal distribution.

Crosstabs use a chi-square test statistic which does not assume normality.  So you can use nominal or ordinal data with this type of inferential analysis. 


How do we use these test statistics in hypothesis testing? 

1. With any inferential analysis, such as a means test, we will calculate the test statistic based on our sample.  We call this statistic the calculated test statistic.  The calculated test statistic represents the probability associated with our sample statistic, such as a mean or a percentage. 

For example, the formula for a one group means test is:  See board

t = (x-u) / (s / square root of n)

The numerator is “the difference” between two groups' means

The denominator is the standard error of the statistic.  The standard error is the standard deviation of the sampling distribution and it is a measure of the precision of the statistic (in this case, a mean) for your sample.

See diagram on board.  

2a.  If using a probability table (http://www.statsoftinc.com/textbook/sttable.html):  We then look up the critical test statistic.  This statistic is based  the sampling distribution for our sample statistic.  The critical test statistic is what we would expect if the null hypothesis was correct.  To find the correct critical test statistic in a probability table we need to know the sample size and the level of error we are willing to accept.  

We use the sample size to determine the degrees of freedom (df).  Loosely, degrees of freedom concern how many unique pieces of information we have in our data to calculate the statistic in question (the mean in this example).  We lose a degree for every statistic we calculate.  So when doing a simple mean test we lose one degree of freedom. 

df = n -1 for a one group means test (comparing a sample mean to a population parameter) 


2b. If using a software package, the computer
automatically "looks up" the critical test statistic for you and reports the exact probability of rejecting the null hypothesis when it is really correct. 

3a.  If using a probability table (http://www.statsoftinc.com/textbook/sttable.html): We then compare the calculated test statistic and the critical test statistic.  If the calculated test statistic is bigger in absolute value than the critical test statistic we reject the null hypothesis.

See diagrams on board.  

 

3b.  If using a software package, we compare the reported probability with the maximum probability of error we are willing to accept (the alpha level).  If the reported probability is smaller than the alpha level, we reject the null hypothesis because the error is lower than maximum amount we were willing to accept.  In this case the chances are small that the relationship we found between X and Y is due to random variation.  Rather it is due to there really being a relationship between the two variables.

Note: If  you are doing a one tailed test, you need to divide the reported probability in half.  See the diagram on the board for an explanation. 

 

The computer probability is the exact amount of error associated with your hypothesis test:

a.  The exact probability of rejecting the null when it is correct.

b.  If you collected 100 more samples and did the same test, you would reject the null when you shouldn’t have exactly X times (x=  p%).

c.  Exact probability of a mean difference of this size being due to random variation.

d.  Exact probability of getting a test stat of this size if the null was true.


If you set alpha at .05 and got a t=2.5, p=.025 for our mean difference of 3 between men and women’s drinking:  (See diagram on board)

a.  Low probability of error, you would make a mistake if you rejected the null 2.5 out of 100 times

b.  The probability of a difference of 3 being due to chance/random variation is .025.

c.  The likelihood of there not being a difference between men and women’s drinking = .025.

d. The probability of getting a t=2.5 if the null was correct is .025.  


If you set alpha at .05 and got a t=.97, p=.35: (See diagram on board)

a. High probability of error, you would make a mistake if you rejected the null 35 out of 100 times

b. The probability of a difference of 3 being due to chance/random variation is .35.

c.  The likelihood of there not being a difference between men and women’s drinking = .35.

d. The probability of getting a t=.97 if the null was correct is .35   

Relationship between Alpha and p

Alpha is the maximum amount of error you are willing to accept; set this before collecting data and conducting analysis. Alpha is also referred to as the significance level.

p is the exact amount of error associated with rejecting the null hypothesis for your sample statistic. It is calculated by the computer.  If your software package calculates probabilities based on two tailed test (as most do), you must divide the reported probability in half for a one tailed test.

See diagram on board for comparison of alpha and p.