Recently I’ve browsed some specific UFO encounter reports, and I must admit they can feel quite compelling. But then I remember the huge selection effect. We all go about our lives looking at things, and only rarely do any of us officially report anything as so strange that authorities should know about it. And then when experts do look into such reports, they usually assign them to one of a few mundane explanation categories, such as “Venus” or “helicopter.” For only a small fraction do they label it “unidentified”.
I know the problem I'm raising is that the calculation asks you to compute a probability that you can't reasonably compute.
To give a more concrete analogy suppose a friend of mine seems really good at solving certain kinds of math problems in his head and I want to figure out if he is cheating or just got lucky and guessed solutions. That might be easy to compute if I know the fastest way to solve those math problems but maybe there is a much quicker way I haven't thought of which he might know. So computing the probability I want depends on knowing if there is a faster algorithm for some kind of problem (easy to cast in terms of decryption etc..).
I'm sure you'll be inclined to just say 'ok just use your prior for there being such a faster algorithm'. But you can't do that. There is no assignment of probabilities that obeys the probability axioms and doesn't assign the existance of such an algorithm either 0 or 1 (with a unimportant technical qualification that we can assume doesn't apply). The rules for probability functions force it to respect logical entailment . Thinking of new ways to arrange things you know about is essentially the same problem as putting together the symbols of a math proof in a novel way with the same problem.
If you are talking about assigning a probability to something like the existance of a faster algorithm for cracking AES maybe you can argue the deviations are so complex that they won't matter but that's not the kind of thing we are talking about here.
—
In other words a prior probability function tells you the probability that certain contingent events might occur but you can't use it to figure out if there is some way of arranging stuff that obeys newton's laws that produces a given effect since that's going to be logically entailed by Newton's laws etc.. and thus forced to 0 or 1 for any true probability assignment.
As did I. I was making a suggestion about what you might want to do to more effectively communicate.
I mean given all the posts you make about signaling shouldn’t you expect that people will presume that you are doing the same kind of signaling everyone else is doing via their public speech?
Besides what’s the harm? Heck, why not just create a standard disclaimer to go on the bottom of every post saying something like: Nothing in this post should be understood as trying to indirectly suggest any approval or disapproval for any position, person or issue even if you can’t imagine any other motivation for this post’s publication. I take no shortage of unpopular positions and if I mean to imply some value judgement I will do so.
Yes, and I explain why in effect you end up doing this. Point is that since you aren't logically omniscent you don't know all the probabilities you need to do that evaluation correctly.
I mean it's like saying I'm going to figure out what math problem to work on by first evaluating the probability that P=NP and trying to prove that only if it's large. Evaluating that probability *is* deciding the provability quesiton and similarly in this case your computation essentially demands you compute the probability that 'there is some arrangement (this case stuff previously proof steps) I haven't thought of that would be compelling if I did'.
It seems likely you could identify whether some unknown phenomena has been mixed in but it would be fallacious to try and match it only against some manageable finite list of known possible alternatives and infer something from that. After all, it might be some kind of event that you simply hadn't considered.
In an idealized case, what one would have to do is compare it against all possibilities your prior assigns non-zero to. But the idea of a prior here is meant to represent some survey of all logically possible events (probabilities assume logical omniscience) and in practice you would be discounting logically possible events that you hadn't thought of.
This is in fact what tends to happen with UFO videos. Someone comes along with some explanation that, once we consider that possibility, seems quite compelling but if we tried to evaluate the probabilities without including that in our list it would have fallaciously inferred that UFO was a likely culprit.
I agree that some manner of expectation-based noise must be dealt with in other (if not most) text-based historical inference domains, e.g. evolution of natural languages, Biblical scholarship.
I agree that expectations can color what people see, but that's true in most all areas of life. So it doesn't make this area much harder to analyze than others.
Apologies if this is addressed in the paper, but at least in the UFO domain, which is dominated by eyewitness testimony, it seems that the many prevalent UFO memes would muddy the waters considerably. (OTOH, simply rejecting anything which resembles such a meme might be foolish, since one of the ways to start a UFO meme is to be a legit UFO).
It's well known that images can move faster than light and without respect to inertia etc. Just look at a cat chasing a laser pointer. Presumably any visual/instrument reports of such from jet pilots are just targeted beams designed to confuse the viewer.
Put differently, I think making the assumption that there are aliens on earth does interesting things to your intuitions about what you would/wouldn't consider potential evidence of aliens.
If one day every earthling simultaneously received a psychic message stating simply 'We are aliens. We are on earth. Come find us. Over.' that would definitely lead people to broaden the search beyond 'weird, often grainy pictures and videos of things in the sky.'
Although this assumption obviously increases the risk of false positives, I don't think it's generally an obviously bad heuristic to assume the thing you're looking for is in fact there.
For context, I'm firmly in the 'measurement error' camp on the current evidence.
Well it's very hard to say what they would be. I think I'm gesturing towards something like this. If one attempted a behavioral analysis of the four groups (Shy/Prom. + Omni/Rare), one might find certain classes of data which, were they indeed evidence of alien activity, would be evidence of some but not all of the groups.
So, for instance, one might find that 'incidents in caves' or 'blue things' would be evidence of only the ruled-out groups, and not the 'behaviorally most plausible' groups. And so data on such would be more likely to merely add noise to any analysis. It may be that this is most relevant if one broadens the data set beyond UFOs, to a larger set of Unidentified Objects/Phenomena. If we're looking for aliens on earth it seems a bit strange to 'only' look for flying ones.
Yeah I don't know... Mostly just thinking out loud.
I linked to a paper on the stat method to use; it takes everything from the data, doesn't need more prob. estimates.
I linked to a paper on the stat method to use; it takes everything from the data, doesn't need more prob. estimates.
I know the problem I'm raising is that the calculation asks you to compute a probability that you can't reasonably compute.
To give a more concrete analogy suppose a friend of mine seems really good at solving certain kinds of math problems in his head and I want to figure out if he is cheating or just got lucky and guessed solutions. That might be easy to compute if I know the fastest way to solve those math problems but maybe there is a much quicker way I haven't thought of which he might know. So computing the probability I want depends on knowing if there is a faster algorithm for some kind of problem (easy to cast in terms of decryption etc..).
I'm sure you'll be inclined to just say 'ok just use your prior for there being such a faster algorithm'. But you can't do that. There is no assignment of probabilities that obeys the probability axioms and doesn't assign the existance of such an algorithm either 0 or 1 (with a unimportant technical qualification that we can assume doesn't apply). The rules for probability functions force it to respect logical entailment . Thinking of new ways to arrange things you know about is essentially the same problem as putting together the symbols of a math proof in a novel way with the same problem.
If you are talking about assigning a probability to something like the existance of a faster algorithm for cracking AES maybe you can argue the deviations are so complex that they won't matter but that's not the kind of thing we are talking about here.
—
In other words a prior probability function tells you the probability that certain contingent events might occur but you can't use it to figure out if there is some way of arranging stuff that obeys newton's laws that produces a given effect since that's going to be logically entailed by Newton's laws etc.. and thus forced to 0 or 1 for any true probability assignment.
As did I. I was making a suggestion about what you might want to do to more effectively communicate.
I mean given all the posts you make about signaling shouldn’t you expect that people will presume that you are doing the same kind of signaling everyone else is doing via their public speech?
Besides what’s the harm? Heck, why not just create a standard disclaimer to go on the bottom of every post saying something like: Nothing in this post should be understood as trying to indirectly suggest any approval or disapproval for any position, person or issue even if you can’t imagine any other motivation for this post’s publication. I take no shortage of unpopular positions and if I mean to imply some value judgement I will do so.
Yes, and I explain why in effect you end up doing this. Point is that since you aren't logically omniscent you don't know all the probabilities you need to do that evaluation correctly.
I mean it's like saying I'm going to figure out what math problem to work on by first evaluating the probability that P=NP and trying to prove that only if it's large. Evaluating that probability *is* deciding the provability quesiton and similarly in this case your computation essentially demands you compute the probability that 'there is some arrangement (this case stuff previously proof steps) I haven't thought of that would be compelling if I did'.
I didn't stay to only consider known possible alternatives.
It seems likely you could identify whether some unknown phenomena has been mixed in but it would be fallacious to try and match it only against some manageable finite list of known possible alternatives and infer something from that. After all, it might be some kind of event that you simply hadn't considered.
In an idealized case, what one would have to do is compare it against all possibilities your prior assigns non-zero to. But the idea of a prior here is meant to represent some survey of all logically possible events (probabilities assume logical omniscience) and in practice you would be discounting logically possible events that you hadn't thought of.
This is in fact what tends to happen with UFO videos. Someone comes along with some explanation that, once we consider that possibility, seems quite compelling but if we tried to evaluate the probabilities without including that in our list it would have fallaciously inferred that UFO was a likely culprit.
I agree that some manner of expectation-based noise must be dealt with in other (if not most) text-based historical inference domains, e.g. evolution of natural languages, Biblical scholarship.
I agree that expectations can color what people see, but that's true in most all areas of life. So it doesn't make this area much harder to analyze than others.
"a legit UFO" -> "of truly exotic origin"
Apologies if this is addressed in the paper, but at least in the UFO domain, which is dominated by eyewitness testimony, it seems that the many prevalent UFO memes would muddy the waters considerably. (OTOH, simply rejecting anything which resembles such a meme might be foolish, since one of the ways to start a UFO meme is to be a legit UFO).
Not 'Presumably any', but 'Perhaps some'.
No point in look at data if you are already sure of the answer.
It's well known that images can move faster than light and without respect to inertia etc. Just look at a cat chasing a laser pointer. Presumably any visual/instrument reports of such from jet pilots are just targeted beams designed to confuse the viewer.
Put differently, I think making the assumption that there are aliens on earth does interesting things to your intuitions about what you would/wouldn't consider potential evidence of aliens.
If one day every earthling simultaneously received a psychic message stating simply 'We are aliens. We are on earth. Come find us. Over.' that would definitely lead people to broaden the search beyond 'weird, often grainy pictures and videos of things in the sky.'
Although this assumption obviously increases the risk of false positives, I don't think it's generally an obviously bad heuristic to assume the thing you're looking for is in fact there.
For context, I'm firmly in the 'measurement error' camp on the current evidence.
Well it's very hard to say what they would be. I think I'm gesturing towards something like this. If one attempted a behavioral analysis of the four groups (Shy/Prom. + Omni/Rare), one might find certain classes of data which, were they indeed evidence of alien activity, would be evidence of some but not all of the groups.
So, for instance, one might find that 'incidents in caves' or 'blue things' would be evidence of only the ruled-out groups, and not the 'behaviorally most plausible' groups. And so data on such would be more likely to merely add noise to any analysis. It may be that this is most relevant if one broadens the data set beyond UFOs, to a larger set of Unidentified Objects/Phenomena. If we're looking for aliens on earth it seems a bit strange to 'only' look for flying ones.
Yeah I don't know... Mostly just thinking out loud.