Sources and Reliability, Part II

SourcesReliability.

I drop into a specific militia website from time to time.  I’m in no way affiliated with them – I’m not even in a militia – but I’m naturally curious and just want to see what they’re saying.  They have an ‘Intel’ forum where they share information.  A lot of the “intel” is probably pretty legitimate; they see a military or police helicopter active in the area, they see a military convoy driving up some road somewhere, or they read that the local law enforcement has been/is going to be downsized by x number of officers.

But there’s this one guy who always posts this totally outlandish information.  Most of us probably know his type – he’s just trying to be helpful and is probably a little too zealous in passing on the random scuttlebutt that he hears from his equally crazy friends… or the internet.  One of his posts was about a child indoctrination/re-education camp being built outside a local military base.  Another was about an increased military presence in the area, as if they were gearing up for a large scale invasion of the Walmart Supercenter.  (He could be an agent provocateur for all I know.  And I don’t.)

If you’re running sources or collecting information in a situation where your safety depends on it, it’s going to be absolutely critical that you compile your reports for cross-reference.  Equally critical is how to determine what weight you give to a certain piece of information.  For instance, who do you trust if you have three sources reporting different information?  Similarly, who do you trust when three sources are reporting one way but a fourth is reporting the opposite?

The military has reliability ratings that I’ll provide below (open source) but I have an abbreviated methodology to determining veracity that I’ll include after that.

Source Reliability

Rating Description
A Reliable No doubt about the source’s authenticity, trustworthiness, or competency. History of complete reliability.
B Usually reliable Minor doubts. History of mostly valid information.
C Fairly reliable Doubts. Provided valid information in the past.
D Not usually reliable Significant doubts. Provided valid information in the past.
E Unreliable Lacks authenticity, trustworthiness, and competency. History of invalid information.
F Cannot be judged Insufficient information to evaluate reliability. May or may not be reliable.

Information Reliability

Rating Description
1 Confirmed Logical, consistent with other relevant information, confirmed by independent sources.
2 Probably true Logical, consistent with other relevant information, not confirmed.
3 Possibly true Reasonably logical, agrees with some relevant information, not confirmed.
4 Doubtfully true Not logical but possible, no other information on the subject, not confirmed.
5 Improbable Not logical, contradicted by other relevant information.
6 Cannot be judged The validity of the information can not be determined.

In theory, reported information is going to get feedback from the end user.  For instance, your community defense leader says he needs to know what types of weapons the Leroy Jenkins Gang employs and it’s your job to find out.  You’d recruit and develop new sources or task existing ones to get that information.  A week later, three sources have reported back to you with the following conclusions:

Source A says they all have AK-47s.  Source B says they have AK-47s and RPGs.  Source C says they have no rifles, only pistols.

This should be pretty easy to confirm or deny after the gang’s next attack.  If there were shoulder launched grenades that went boom, they obviously have (or had) RPGs.  If nothing went boom then we can’t rule out the possibility that they do have RPGs.  If there are no 7.62×39 or 5.56 casings at the scene of the attack, then we can cast some doubt on the veracity of Sources A and B.  Similarly, if all we see are short pistol caliber casings at the scene and no one reports seeing a rifle, then we can say that Source C was likely the most accurate.  As your sources continue to report, cross-check their information against what other sources are saying and against actual events.  Get them to report on something you already know as fact.  Start rating your sources by the tables above.

The best sources are going to have an A rating, the worst are going to have a D rating, and almost always, new sources are going to be rated F.  These ratings can be upgraded (or downgraded) after an established pattern has emerged.  If they repeatedly report information that can’t be confirmed or denied, then they should continue to be rated as an F source, still unknown.

As for rating information reported by sources, use your best judgement if no other data can confirm or deny the information.  If we know that the Leroy Jenkins Gang uses RPGs in ambushes and Source A says they don’t, then his reliability rating will suffer along with that information because we know otherwise.

Abbreviated information reliability.

I have my own system of quickly rating information.  Let’s look at two examples.

First is the child re-education camp reportedly being built outside a military base.  This was reported in the forum as fact.  And speaking of fact, here’s the formula:

F – is it feasible?  A – is it apt? C – is it consistent with other information? T – is it timely? S – is it suitable? FACTS!

It isn’t feasible that the government will forcibly remove tens or hundreds of thousands of children from their homes and ship them off to a military base to be re-educated.  It isn’t apt to happen because most parents, along with the public, wouldn’t allow it.  It’s not consistent with any news reports or other reliable sources.  It’s not timely but it also isn’t untimely.  Lastly, it’s not even suitable because most children in public schools are already in a re-education camps.  Cumulatively, it’s probably not a FACT.

Next, let’s assess a military convoy rolling down the interstate.

It’s entirely feasible because military equipment moves in convoys and I’ve seen them myself.  It’s apt because equipment has to be moved sometimes.  It’s consistent with other information because there was a nearby military base and military convoys have occurred in the past.  It was timely because it was on the weekend, when national guard and reserve drill occurs.  Finally, it’s suitable because equipment is moved in convoys.  Cumulatively, it’s likely a FACT!

Questions or comments?  Please leave them below.

Comments are closed.

%d bloggers like this: