That's where we disagree. Can you give me an example of anything that doesn't exist within this physical universe? I'm not familiar with the concept of a "thing" that isn't physical.
Sorry about being confusing with the last sentence. I'll try and address your questions:
1. When we talk about the likelihood of something being true we are now discussing probabilities, which have a long and well proven track record of obeying the laws of statistics. Statistics are quite often counter-intuitive and our brains don't think in terms of the math that governs the actual outcomes
A good example is the birthday problem [1]. Imagine we had a room full of people and we wanted to know what the odds are that two of those people in the room had the same birthday. Obviously the more people we have, the more likely we are to have a match, but how many people do you need to get to that point?
It turns out that with just 23 people in the room there's a 50% chance of two people having the same birthday. Most people's intuition leads them to believe that for 50% you'd need 365/2 = 182.5 people, but once there's 70 people in a room, you're almost certain to have a match with a probability of 99.9%. This is a good example of a case where not knowing the math will lead your predictions of the world to be incorrect.
2. When we discuss evidence we are trying to figure out the probability of something, thus the rules of probabilities apply. One of the fundamental theorems of statistics is Bayes' Theorem [2], which at its core tells you how to calculate the probability of an event given another event.
To motivate that, here's a more realistic example: "A patient goes to see a doctor. The doctor performs a test with 99 percent reliability--that is, 99 percent of people who are sick test positive and 99 percent of the healthy people test negative. The doctor knows that only 1 percent of the people in the country are sick. Now the question is: if the patient tests positive, what are the chances the patient is sick?"
If you answered 99% then you'd not only be wrong, but you'd have misdiagnosed your patient. The correct answer here is 50% [3].
3. To bring it all together: to properly know things one must understand the laws of mathematics. This applies not just to problems that obviously involve probabilities, but everything in general.
At it's core, I think our disagreement revolves around this idea of non-physicality and the belief that there are things that aren't governed by mathematics. Everything is physical and obeys the laws of physics which themselves are governed by the laws of mathematics.
[1] https://en.wikipedia.org/wiki/Birthday_problem
[2] https://en.wikipedia.org/wiki/Bayes'_theorem
[3] https://sphweb.bumc.bu.edu/otlt/mph-modules/bs/bs704_probabi...