Those counties were hit hard with EHD a few years back and have also been hit since. Do the neighbors farther away hunt? Thats a fairly populated area so Id bet that you have more pressure than you think.
I think some areas suffer from a few hunters not laying off the doe until numbers recover.I have seen numbers in this area continue to drop as well since the 2012 outbreak, whereas the place I hunt has turned the corner. I don't understand it either.
I think the lack of cover on surrounding farms is not helping your case on 40 acres.. There are corn, beans, and clover on my near by properties and but very limited cover.