Methodology

The methodology used to count the ballots was crucial to the reliability and credibility of the project findings. The fundamental problem facing any ballot counting exercise is that every method of ballot tabulation produces slightly different results with each pass through the ballots. The range of these differences varies from ballot system to ballot system. NORC used the variation from pass to pass to measure the reliability of the particular ballot system used. Under this measurement scheme, one ballot system can be judged superior to another system to the extent that its variation from pass to pass is smaller.

NORC made its assessment of reliability by calibrating the variation from pass to pass of the various ballot systems against the results of a careful hand examination. This examination of the appearance of every ballot was done by highly trained observers. These observers noted in detail all aspects of the ballot, including hand notations, that might help identify voter intent.

A hurdle to be overcome when using this approach to measuring reliability is that, although hand classification is widely recognized as having the least variation from pass to pass, even hand classifications have some variability from pass to pass, as some people will see some ballots differently. The NORC project minimized this variation by using three-person teams of highly trained observers, each team member working independently, to classify each ballot into categories based on the varying interpretations Florida canvassing boards have confronted in manual recounts of machine-readable ballots. On punch-card ballots, for example, NORC’s three coders recorded independently whether a ballot’s key chads (the pieces of the punch card that must be clearly punched through for the vote to be registered) were missing, hanging by one or more corners, or dimpled; whether light was visible around any dimple; and whether the rest of the ballot was consistently marked. Three independent codings guarded against partisan biases and helped judge the degree of difficulty canvassing boards encountered in trying to assess voter intent on machine ballots.

Return to Florida Ballots Page