Skip to content

Instantly share code, notes, and snippets.

@rstacruz

rstacruz/Comelec SD card failure analysis.md

Last active May 23, 2019 04:49
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save rstacruz/9a96070479c71a71b90ddca7f2109752 to your computer and use it in GitHub Desktop.
Save rstacruz/9a96070479c71a71b90ddca7f2109752 to your computer and use it in GitHub Desktop.
Download ZIP
Comelec SD card simulation
Raw
Comelec SD card failure analysis.md

Analyzing SD card failures in the Philippine elections

News reports say that 2019 Philippine elections are severely affected by SD card glitches. If we go by the facts, there should only be ~8 machines affected by SD card failures. Only 0.01% of the vote-counting machines should be affected by these issues. Let's do the math and see why.

What we know

Here's some data we can gather from news articles. (Comelec says there are 961 machine failures, but these failures may or may not be related to SD cards.)

Key Value
Total vote-counting machines (2) 85,769
Vote-counting machines affected (1) (2) 961
Defective SD cards (1) 1,665

2 cards per machine

News reports claim there are 2 SD cards per machine. We don't know the exact setup, but my best guess is it's probably a RAID 1-like setup where the redundancy protects you from data corruption. For a machine to fail, both cards have to fail together.

  • "Each machine has 2 SD cards." (3)

  • 'The poll body [Comelec] noted that protocols have been formulated as "there is a possibility that both the main and backup SD cards may be damaged while in transit."' (4)

  • "Jimenez, however, said there are 85,769 SD cards with the same number of backup cards." (5)

Cards that were defective = 0.97%

Only less than 1% of SD cards were defective. This defect rate isn't unheard of when you're ordering 170k+ units of an item by wholesale.

There are 1,665 defective cards. We don't know how many SD cards there are, but we can estimate that there are probably 171,538 SD cards in total. Each vote-counting machine has 2 SD cards.

Key Value
Total vote-counting machines (2) 85,769
Cards per machine (3) (4) 2
Total number of cards (5) 85,769 * 2 = 171,538
Defect rate 1,665 / 171,538 = 0.97%

Chance of VCM failure = 0.009%

There is less than 0.01% chance that a VCM will fail due to SD card issues. In other words, it's 99.99%+ likely that at least one of the cards will work.

The probability of getting a card is 1665 / 171538, or 0.97% (like above).

# Probability of getting a defective SD card
defective_rate = 1_665 / 171_538

# => 0.0097

The probably of 2 cards failing, however, is 0.009%.

# Probability of getting a pair of defective cards:
# Get the probability of getting 1 defect, and the
# probability another defect on the next draw, then
# multiply them.
pair_defect_rate =
  1_665 / 171_538 *
  1_664 / 171_537 

# => 0.00009415

Extrapolating from here, we can multiply that by the number of total VCM's (85,769). That yields roughly 8.076. That's how many machines are likely to be affected.

total_vcms = 85_769
total_vcms * pair_defect_rate
# => 8.07569

Proving our work

I did some simulations in Ruby (below) of how this would play out. Out of 500 simulations, 73 of them had 8 defective SD card pairs... so I suppose our math above checks out. Thanks to Bryan Bibat for the tip! The source code for this is in this Gist below.

   1 defective pairs -  1x
   2 defective pairs - **** 10x
   3 defective pairs - **** 12x
   4 defective pairs - *********** 2x
   5 defective pairs - ******************* 48x
   6 defective pairs - ******************* 48x
   7 defective pairs - ************************** 65x
   8 defective pairs - ****************************** 73x
   9 defective pairs - *********************** 58x
  10 defective pairs - ********************* 53x
  11 defective pairs - ****************** 44x
  12 defective pairs - ********** 25x
  13 defective pairs - ***** 14x
  14 defective pairs - *** 8x
  15 defective pairs - ** 7x
  16 defective pairs - ** 5x
  17 defective pairs -  2x

So what does this mean?

  • The 961 VCM failures had nothing to do with the 1,665 SD card failures.

    "That 1,000 SD cards didn't work is on top of the 450 VCMs that had to be replaced on Election Day." (6)

  • 1,665 SD card failures isn't significant. It translates to only 8 machines affected. (And for these 8, there could've been some other contingencies for them.)

  • If polling precincts were stalled due to SD card issues, it's more likely to be a failure in logistics and organizing rather than a failure in equipment. The backup cards weren't given out until they found out that the primary cards fail.

    “We respond by having replacement cards. This will be swapped with the corrupted SD Cards… so we do not issue out new SD cards unless we have the corrupted SD Cards.” (9)

  • Not all machines had 2 cards from the start, despite having enough cards on hand.

    "Jimenez, however, said there are 85,769 SD cards with the same number of backup cards." (8)

  • Why are SD cards the primary transmission medium, rather than the contingency? Transmitting encrypted results online is faster and more reliable.

References

  1. "The Commission on Elections (Comelec) said Tuesday that 961 vote-counting machines (VCMs) and 1,665 SD cards suffered glitches during the midterm polls on Monday." https://newsinfo.inquirer.net/1118605/961-vcms-1665-sd-cards-suffer-glitches-in-2019-polls

  2. "The Commission on Elections (Comelec) disclosed on Tuesday afternoon, May 14, that 961 vote-counting machines (VCMs) bogged down in this year's elections" https://www.rappler.com/nation/politics/elections/2019/230604-comelec-says-vote-counting-machines-defective

  3. "Each machine has 2 SD cards." https://www.rappler.com/thought-leaders/230839-how-to-fix-what-went-wrong-elections-2019

  4. 'The poll body [Comelec] noted that protocols have been formulated as "there is a possibility that both the main and backup SD cards may be damaged while in transit."' https://www.philstar.com/headlines/2019/05/09/1916345/comelec-issues-protocol-sd-card-replacement

  5. "Jimenez, however, said there are 85,769 SD cards with the same number of backup cards." https://tribune.net.ph/index.php/2019/05/12/defective-sd-cards-discovered/

  6. "That 1,000 SD cards didn't work is on top of the 450 VCMs that had to be replaced on Election Day." https://www.rappler.com/nation/politics/elections/2019/230543-number-sd-cards-that-did-not-work-may-13

  7. “We don’t issue out new SD cards unless we get the corrupted SD cards in return,” Jimenez explained. https://tribune.net.ph/index.php/2019/05/12/defective-sd-cards-discovered/

  8. "Jimenez, however, said there are 85,769 SD cards with the same number of backup cards." https://tribune.net.ph/index.php/2019/05/12/defective-sd-cards-discovered/

  9. “We respond by having replacement cards. This will be swapped with the corrupted SD Cards… so we do not issue out new SD cards unless we have the corrupted SD Cards.” https://www.bworldonline.com/comelec-assures-it-will-be-ready-for-the-polls-despite-hitches/

  10. "Comelec Chairman Sheriff Abas earlier said that out of 1,665 defective SD cards, the poll body was able to replace 1,253." https://businessmirror.com.ph/2019/05/17/comelec-told-come-clean-on-vcm-sd-card-glitches/

  11. "The Comelec has 85,769 back up SD Cards ready for scenarios like this." https://www.bworldonline.com/comelec-assures-it-will-be-ready-for-the-polls-despite-hitches/

  12. “Contingency SD cards allocated per regional technical hub amount to 125 per hub." https://tribune.net.ph/index.php/2019/05/12/defective-sd-cards-discovered/

Raw
monte_carlo_simulation.rb
# There are 85,769 vote-counting machines.
# (Source: https://www.rappler.com/nation/politics/elections/2019/230604-comelec-says-vote-counting-machines-defective)
vcms = 85769
# There are 1,665 defective SD cards.
# (Source: https://newsinfo.inquirer.net/1118605/961-vcms-1665-sd-cards-suffer-glitches-in-2019-polls)
defective_cards = 1665
# Each machine has 2 SD cards.
# (Sources: https://www.rappler.com/thought-leaders/230839-how-to-fix-what-went-wrong-elections-2019)
cards_per_machine = 2
# Given that each VCM machine has 2 SD cards, then there should be
# ~171k SD cards.
total_cards = vcms * cards_per_machine
# Let's keep a pile of SD cards. Some of them are defective (`0`) and
# some are working (`1`).
cards = []
defective_cards.times { cards << 0 }
(total_cards - defective_cards).times { cards << 1 }
result = Hash.new(0)
# Let's run a simulation for 500 times, keeping track of how many
# times we run into a defective pair of SD cards. (`count`)
500.times do |i|
cards = cards.shuffle
count = 0
cards.each_slice(2).each do |x, y|
count += 1 if x == 0 && y == 0
end
result[count] += 1
end
# Print the results
min = result.keys.min
max = result.keys.max
vmax = result.values.max
(min..max).each do |i|
value = result[i]
bar = '*' * (30 * value / vmax).to_i
puts "%3i defective pairs - %s %ix" % [i, bar, value]
end
# We'll get a histogram of how many VCM machines are likely to have
# both SD cards defective.
#
# It looks only 8 machines are most likely to be affected.
#
# 1 defective pairs - 1x
# 2 defective pairs - **** 10x
# 3 defective pairs - **** 12x
# 4 defective pairs - *********** 2x
# 5 defective pairs - ******************* 48x
# 6 defective pairs - ******************* 48x
# 7 defective pairs - ************************** 65x
# 8 defective pairs - ****************************** 73x
# 9 defective pairs - *********************** 58x
# 10 defective pairs - ********************* 53x
# 11 defective pairs - ****************** 44x
# 12 defective pairs - ********** 25x
# 13 defective pairs - ***** 14x
# 14 defective pairs - *** 8x
# 15 defective pairs - ** 7x
# 16 defective pairs - ** 5x
# 17 defective pairs - 2x
#
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment