Hacker News Reader

> I noticed if we plot the journalist killings against general population killings in Gaza, the graph deviates from an approximately straight line that might be expected if journalists were merely being killed as a result of "collateral damage" amidst indiscriminated bombings

I think this assumption is wrong. Journalists are known to try to get the most shocking photos and reports. I would expect their rates of injury compared to random Joes in similar situations to be similar to that of people doing extreme sports compared to people doing recreational sports.

There should be some level of occupational hazard, but surely not 3x as high when compared to people actually engaged in combat (Journalist:~75:1; Hamas militant:~25:1), and high enough to greatly deplete their numbers.

In those figures, "Hamas militant" actually means adult man. Nobody is identifying every person killed and checking against a list of Hamas members, they're just claiming every man killed belonged to Hamas.

Is there a way to verify this? I tend to agree, but I have no way to be rigorously convinced.

For example, iirc Palestine health authority reports the names of all the civilians[0] that have been killed. I think the number is something like 30k. I frequently wonder if that number is simply exaggerated, as it would be in Palestine's interest to do so. However, if the number is supported by a corresponding list of the names of individuals, I would think the Israeli intelligence services would be incentivised and capable of refuting a significant enough number of false claims[1] to cast serious doubt upon the claimed number of civilian deaths.

To my knowledge, Israel has not done so, and so I don't doubt the number too much.

[0] As noted, it is extremely challenging to pin down whether or not someone is 'Hamas'. I assume Palestine is reporting all deaths as civilian, in the same way Israel seems to report them all as combatant.

[1] whether it is an individual that does not exist, perished prior to Oct 7. '23, or is still alive.

Yes unfortunately those are the identified and confirmed deaths, there are for sure many still buried under the rubble which are not accounted for.

Unfortunately it seems to be in Palestine's (and even Hamas) best interest to be truthful, in this case

AFAIK, and I've seen news articles saying the same, Israel has not provided any evidence for its claim to have killed 12,000 Hamas fighters.

But the latest reported death toll is over 30k, 22k of whom are women and children. Another 10k or more are missing and presumed dead. If about one-third of the missing are adult men, matching the ratio of deaths, then about 12k adult men are dead or missing.

Is it plausible that Israel is so precise that all the adult men they've killed were Hamas fighters?

> Is it plausible that Israel is so precise that all the adult men they've killed were Hamas fighters?

We know from some of their previous actions in the region that they could be, if they wanted to - we've seen them take out individual aparment units with missiles, for example.

But given how many unguided munitions Israel are using in Gaza... I mean, almost the whole of Gaza has been reduced to rubble - the idea that every military-age male is Hamas is just not credible.

There is an article published in The Lanced where they investigated this. They found no evidence of inflated numbers. But if later they are found inflated anyway, this makes the high number of journalists killed more of an anomaly, not less.

[1]. https://www.thelancet.com/journals/lancet/article/PIIS0140-6...

Is there a way to calculate significance of the fit? I.e. something similar to the Student's t-test?

Surely there is, but I still didn't figure out how to. I'm not used to work with this Wallenius distribution (indeed, I discovered it existed a few days ago, while working on this problem). To get some semblance of error bars, I just took max/min overs a couple tens of runner ups. But I explained in detail the assumptions in my reasoning and posted the source code for my analysis, so people with better statistical shops than me can replicate and improve upon it.

But the uncertainty here probably is dominated by uncertainty in the inputs. I used the lower estimates for journalists deaths, about 90 on 01/03. But some organizations publish figures as high as 130. The uncertainty in total deaths also high, I would not be surprised if 1.5-2x higher, given all missing people.

SciPy stats includes a Wallenius’ discrete random variable[15], that takes four parameters on its creation: M is the total number of subjects, n is the number of Type I subjects (journalists), and odds is the odds ratio: the odds of killing a journalist rather than a non-journalist when there is only one individual of each. The random variate represents the number of journalists that end dead if Israeli forces kills pre-determined x people from Gaza Strip one by one. The total population in the strip (M) is approximately known. x is known from OCHA daily stats. For the effective population of journalists (n) and odds ratio (odds) we can use Wallenius' to test several value pairs and check which best fit the data. From the article we can estimate the membership of the Palestinian Journalists Syndicate at 1200. That is a upper bound on the effective number of journalists, taking into account the likely inclusion of supporting staff and the many professionals forced to flee the Strip[16], sometimes after receiving explicit threats[17]. About the odds ratio, the current body count is about 1.3% of the total population in the Strip. Meanwhile, US estimates about 20%-30% of the 30000 strong Hamas forces were killed[18]. This number is probably exaggerated, as it implies every dead male in the age bracket around 13-60 is being counted as a Hamas militant, but for the sake of iron-manning the argument, let's assume those figures are true. That means the average Hamas militant is about 20:1 to 30:1 times more likely to be killed than a random person. So we should expect jurnalists' odds ratio to sit between somewhere between random citizen and Hamas militant, taking into account occupational hazards but not deliberate targeting. In doubt I took 100:1 as an upper bound, and wrote a model (source bellow).

The best fit I got over the search grid had an effective journalist population of 155 and odds ratio of 75:1. As the data is noisy, I took the top 20 matches to have some idea of error intervals, and got ranges of 135 - 205 effective population and 51 - 96 odds ratio. The odds ratio is larger than our estimate for Hamas militants, indicating that in fact Israeli forces are busier targeting journalists than Hamas militants. Even if we assume the total kill counts are underestimates, given all people missing under the rubble and lack of proper rescue work due to targeting of rescue crews, say, adding a xs = [2*x for x in xs] line after the first definition of xs, to double the total counts, we still get killing odds for journalists equal or greater than for militants.

[1]. https://en.wikipedia.org/wiki/Decimation_(punishment) [2]. https://xkcd.com/356/ [3]. https://www.reuters.com/world/middle-east/global-media-outle... [4]. https://en.wikipedia.org/wiki/Urn_problem [5]. https://www.bbc.com/news/world-middle-east-68445973 [6]. https://euromedmonitor.org/en/article/6202/Israeli-tanks-hav... [7]. https://en.wikipedia.org/wiki/Noncentral_hypergeometric_dist... [8]. https://en.wikipedia.org/wiki/Wallenius%27_noncentral_hyperg... [9]. https://en.wikipedia.org/wiki/Fisher%27s_noncentral_hypergeo... [10]. https://cpj.org/2024/03/journalist-casualties-in-the-israel-... [11]. https://twitter.com/EuroMedHR/status/1761121386347516095 [12]. https://en.wikipedia.org/wiki/List_of_journalists_killed_in_... [13]. https://en.wikipedia.org/wiki/Al-Rashid_humanitarian_aid_inc... [14]. https://www.ochaopt.org/ [15]. https://docs.scipy.org/doc/scipy/reference/generated/scipy.s... [16]. https://www.instagram.com/byplestia/p/Cz9MfelKk3x/ [17]. https://english.ahram.org.eg/News/516373.aspx [18]. https://www.timesofisrael.com/us-said-to-believe-israel-kill...

2/3 (Continues)

3/3 (Source code used for analysis)

  import csv
  from collections import Counter
  from datetime import date, timedelta
  from itertools import accumulate, islice
  from matplotlib import pyplot as plt
  from scipy.stats import nchypergeom_wallenius
  from statistics import linear_regression
  
  def batched(iterable, n):
      # batched('ABCDEFG', 3) --> ABC DEF G
      if n < 1:
          raise ValueError('n must be at least one')
      it = iter(iterable)
      while batch := tuple(islice(it, n)):
          yield batch
  
  def dateab(a, b):
      # Lists dates from day a to day b.
      i = 0
      while (c := a + timedelta(days = i)) <= b:
          yield c.isoformat()
          i += 1
  
  def reader(file):
      with open(file, newline='', encoding='utf-8') as csvfile:
          for row in csv.reader(csvfile, delimiter=';', quotechar='"'):
              yield row[0:2]
  
  def sum_absdiff(iter1, iter2):
      # Compute absolute differences sum between vectors, as error measure.
      return sum(abs(x - y) for x, y in zip(iter1, iter2))
  
  days = list(dateab(date(2023, 10, 7), date(2024, 3, 1)))
  weeks = [e for e in batched(days, 7) if len(e) == 7]
  
  total_killings = [(d, int(n)) for d, n in islice(reader('gs_cummulative_total_killings.csv'), 1, None)]
  d = dict(total_killings)
  xs = list(max(d.setdefault(e, 0) for e in week) for week in weeks)
  
  journalist_killings = [t[0] for t in reader('gaza_strip_journalists_killed.csv')]
  c = Counter(journalist_killings)
  ys = list(accumulate(sum(c.setdefault(e, 0) for e in week) for week in weeks))
  
  slope, intercept = linear_regression(xs, ys, proportional=True)
  fitted = [slope * x + intercept for x in xs]
  print('Linear regression parameters: slope =', slope, ';', 'intercept =', intercept, '\n')
  
  
  M = 2375259    # 2022 population estimate (source: https://en.wikipedia.org/wiki/Gaza_Strip)
  ns = [i for i in range(1, 1200) if i % 5 == 0]
  odds = [i for i in range(1, 100) if i % 3 == 0]
  
  candidate_store = {}
  for n in ns:
      for odd in odds:
          L = []
          for x in xs:
              r = nchypergeom_wallenius(M, n, x, odd).mean()
              L.append(r)
          candidate_store[(n, odd)] = L[:]
  best_wallenius_fit = min(candidate_store, key= lambda e: sum_absdiff(candidate_store[e], ys))
  print('n, odds (best Wallenius fit):', best_wallenius_fit, '\n')
  top_20 = sorted(candidate_store, key= lambda e: sum_absdiff(candidate_store[e], ys))[:20]
  print('Top 20 (n, odds):', top_20, '\n')
  top_20_ns = [e[0] for e in top_20]
  top_20_odds = [e[1] for e in top_20]
  print('Top 20 n variation in range:', min(top_20_ns), '-', max(top_20_ns))
  print('Top 20 odds variation in range:', min(top_20_odds), '-', max(top_20_odds))
  
  fig, ax1 = plt.subplots()
  ax1.plot(xs, ys, label='Available data')
  ax1.plot(xs, fitted, label='Linear regression fit')
  ax1.plot(xs, candidate_store[best_wallenius_fit], label='Wallenius\' distribution fit')
  ax1.set_xlabel('General population (source: ochaopt.org)')
  ax1.set_ylabel('Journalists (source: wikipedia.org)')
  ax1.set_title('Cummulative killings by Israeli forces in Gaza Strip\n2023-10-07 to 2024-03-01')
  ax1.legend()
  plt.show()

Output (in addition to figure above):

  Python 3.11.8 (main, Feb  7 2024, 00:00:00) [GCC 13.2.1 20231011 (Red Hat 13.2.1-4)] on linux
  Type "help", "copyright", "credits" or "license()" for more information.
  
  ========== RESTART: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX ==========
  Linear regression parameters: slope = 0.00344597608865967 ; intercept = 0.0 
  
  n, odds (best Wallenius fit): (155, 75) 
  
  Top 20 (n, odds): [(155, 75), (160, 72), (145, 84), (165, 69), (170, 66), (150, 78), (175, 63), (140, 87), (135, 93), (150, 81), (185, 60), (140, 90), (180, 60), (190, 57), (195, 54), (200, 54), (135, 96), (180, 63), (205, 51), (185, 57)] 
  
  Top 20 n variation in range: 135 - 205
  Top 20 odds variation in range: 51 - 96