FST calculation gives negative result

[argiedude]

Hi, I've been trying to understand the more complicated version of the FST calculation developed by Cockerham & Weir (1984), which is more accurate than the conventional FST estimate developed by Wright. I keep getting negative results. The result is only supposed to go from 0 to 1. Here's one example.



http://img147.imageshack.us/img147/511/ ... sulhg6.gif

Thanks for any help.

[stereologist]

The result is an estimate. The estimate is close to 0. Since the result should be in the range [0,1] you should restrict the result to 0. The issue begins with a. That value is 0 because s^2 is 0.

Similar issues happen with other estimators I have used.