I would’ve thought that we’d all learned that finding people who agree with each other on the internet says nothing about their individual or collective sanity.

Nobody knows anything except their names and what they themselves claim. The primary author and editor shares a name with a prominent American lawyer, and that’s about all we know. I’ve tried digging into the guy (he claims to have a physiology degree and works in healthcare), but I can’t find a single source that indicates he even exist.

Edit: anyone who tries to tell me that a name on a mortgage is an indication that someone exists should look a little bit more closely at those documents. I’ll wait.

The only thing we know about the authors of Media Bias Fact Check is what they’ve posted on their website. There’s no corroborating source, no evidence of background, and no indication that these people actually exist other than a business registration for a sole prop and a sparsely-populated mortgage for the lead editor. You can feel free to dig deeper (I’ve focused on the primary author and editor), but this wouldn’t pass the sniff test.

Moreover, their methodology, frankly, doesn’t hold up to any type of scientific method. It’s a perversion of the scientific process. Their methodology is essentially surveying one of the authors and asking them to draw a point on a line. That’s not science.

Based on their methodology, Electronic Intifada has never failed a fact check and should be “very factual”… But it’s recorded as “mostly factual” because they have biased reporting. Reuters is recorded as “very factual,” but they’ve gotten a number of things wrong without correction… Sort of comes with the territory of being a news wire service. CBC, which has also not failed a fact check, only gets a “highly factual” rating because of their supposed left-leaning bias. Essentially, their subjective analysis conflates factuality with bias.

This also raises a bigger problem: the lead author and editor is clearly American and guides ratings towards an American Overton window. Thus, bias is viewed on the left-to-right scale commonly used in the US, with the center defined as the American center. Normally, this wouldn’t be a huge problem for American audiences, but as established MBFC conflates factuality with bias: essentially, any source that deviates from the “center of the American political spectrum” is seen as less factual. Their source for fact checks is a newspaper run out of a school that has received funding from the US state-funded Voice of America.

This is, of course, operating under the assumption that their methodology is actually valid… And that, in itself, is a dangerous assumption to make. There’s a lack of transparency in who’s doing what evaluation, and the end result of that is that the assessment itself is basically “what does one guy think about this site.” Despite them defining what they mean by left and right, they give scant evidence to justify their “quantitative” evaluation of sources - this, itself, makes their evaluation qualitative.

Some of their details also show a lack of understanding of the media landscape. Calling CBC News a “TV station” is a joke. Their website is also, frankly, a mess, which bothers me because it’s clear that they aren’t following basic modern web development principles and that’s really fucking annoying.

Their methodology is bunk, there’s very little transparency within their organization to establish credibility, and they conflate factuality with their perception of bias.