Perception, Wittgenstein, News & Data Science

Prelude

In the social media era, everybody publishes whatever they want. Fake news websites are widely common, publishing lies and fabricated news. Far-right politicians in the post-truth era appeal to emotions and impose personal views; they hide the truth and convince people of what is untrue.

Where do mainstream media organisations stand in the post-truth era? How do they maintain people’s trust, identity, credibility and originality?

Now i would like to analyse this mess we are in as a society, and may be try to put forth few ideas of what can we do to solve this, but before that i need you to get comfortable with few terms/ideas that will appear throughout this analysis.

1.Figures & ground

When a figure or “positive space” (e.g., a human form, or a letter, or a still life is drawn inside a frame, an unavoidable consequence is that its complementary shape-also called the “ground”, or “background”, or “negative space”-has also been drawn. In most drawings, however, this figure ground relationship plays little role. The artist is much less interested in ground than in the figure. But sometimes, an artist will take interest in ground as well.

There are beautiful alphabets which play with this figure-ground distinction. A message written in such an alphabet is shown below. At first it looks like a collection of somewhat random blobs, but if you step back ways and stare at it for a while, all of a sudden, you will see three letters appear in this ..

figure 1

now take few minutes and read this overview about figures & grounds.

2.Fake news

If you don’t read newspaper you are uninformed, but if you read you are misinformed - denzel washington

As a society have we failed on how to think about Data and information ?
read this

3.Signal to noise ratio

Signal to noise ratio (SNR) is a measure used in electrical engineering that compares the level of a desired signal to the level of background noise. It is defined as the ratio of signal power to the noise power, often expressed in decibels.

\(SNR = \frac {P_{Signal}}{P_{Noise}}\)

read more about it here

interlude : What follows below is a way of how i might approach this complex problem, and i will try to convey of how I as a data scientist working at KSF Media (Maybe ?) might try to solve this mess and on a short note I might take an initiative and work with my prospective colleagues towards fixing the democracy and maybe our awesome team might even get a nobel award while we’re at it as teased in this video.

Now that you know about these terms, let’s dive and analyse the situation.
This will be a bit lengthy but i promise to keep it interesting, so readon…
This analysis will be divided into two parts where i discuss the problem and some possible solutions.

Part I

The problem of fake news

Problem description and objectives

Fake news detection is an important and complex area for potential application of data mining techniques given the economic and social consequences that are usually associated with these illegal, not so democratic, unethical activities. From the perspective of data analysis, Fake news are usually associated with unusual observations as these are activities that are supposed to be deviations from the norm. These deviations from normal behavior are frequently known as outliers in several data analysis disciplines.

In effect, a standard definition of an outlier is that it is an observation which deviates so much from other observations as to arouse suspicions that it was generated by a different mechanism" (Hawkins, 1980).

But with the proliferation of more and more social media ,It has become rather hard to define what is an outlier (with respect to Fake news) and this gets even messier when you look closely into the actual data ,and realise how something like a facebook could be misused to ramify the effects of this problem ,as evident by the recent presidential election in USA.

Now how do we solve this serious issue ?

well now I’d like you to step back and see this issue from the perspective of a ‘Figure and ground’.

In a perfect ideal world we should have had the real news as a Figure & the Fake news as a Background but in reality those two have switched places and in effect our perception has been manipulated, and we have truly faild to seperate the ‘positive space’ (or the figure ) from the ‘Negative space’ (or the background).

KSF Media’s objective here should be to somehow seperate these spaces and provide a rich, true content to it’s customers, and as a team we can do this .by doing so we can possibly expand our reach as a company and inturn attract more readers.

Now that we have defined our problem and our objective, The data scientists at KSF (I and my awesome colleague’s) can just do dive in and solve this.

Part II

Possible solutions to counter the fake news.

My personal recommendation would be to look for spam posting behavior, build reputations for specific websites, and aggregate user feedback on the veracity of information. Wikipedia can probably teach us a lot about how to do the latter.

AI can’t evaluate a specific claim as being correct or not. For example, if someone posts on social media, “Call to participate in a survey organized by obama, modern AI is not able to figure out how to investigate that claim and figure out if it is true or false. A human would know to check obama’s website for an announcement of the survey, run a web search looking for an announcement, or do a reverse phone lookup to see if the number actually belongs to obama. A human could also speculate about reasons why someone would make the false claim: maybe someone wants to swamp obama’s phone line with calls responding to a survey that doesn’t exist, maybe the phone number doesn’t belong to obama and the poster wants to swamp a 3rd party’s phone line with calls, or maybe the poster owns the number and wants to collect a list of valid phone numbers to use for telemarketing. AI is not yet sophisticated enough to reason about motives in this way.

AI can be used in the fight against fake news, but mostly as a tool for picking up on other signals, the way that spam detectors work.

and we awesome KSF folks can build algorithms which can do some of the tasks mentioned above

Defining the data mining tasks

The main goal of this application is to use data mining to provide guidance in the task of deciding which news reports should be considered for inspection as a result of strong suspicion of being fake after giving it a first pass into our system for further evaluation before we publish it in anything associated with KSF. Given the limited and varying resources available for this inspection task, such guidance should take the form of a ranking of Fake probability.

Two ways of approaching this might be

Linguistic approach

Inspiration from Wittgenstein
How do human beings communicate ideas between one another ?
His answer - Language works by triggering within us pictures of how things are in the world, words enable us to make pictures of facts

Language is the key, to solve most of the problems and with various Natural language processing approaches.

Most liars use their language strategically to avoid being caught. In spite of the attempt to control what they are saying, language “leakage” occurs with certain verbal aspects that are hard to monitor such as frequencies and patterns of pronoun, conjunction, and negative emotion word usage (Feng & Hirst, 2013). The goal in the linguistic approach is to look for such instances of leakage or, so called “predictive deception cues” found in the content of a message.

1. Data representation : The most simplest approach would be to build a bag of words model and work from there
2. Deep syntax : Deep syntax analysis is implemented through Probability Context Free Grammars (PCFG). Sentences are transformed to a set of rewrite rules (a parse tree) to describe syntax structure, for example noun and verb phrases, which are in turn rewritten by their syntactic constituent parts (Feng, Banerjee & Choi, 2012). The final set of rewrites produces a parse tree with a certain probability assigned. This method is used to distinguish rule categories (lexicalized, unlexicalized, parent nodes, etc.) for deception detection with 85-91% accuracy (depending on the rule category used) (Feng et al., 2012). Third-party tools, such as the Stanford Parser (de Marneffe, MacCartney, Manning, 2006; Rahangdale & Agrawa, 2014), AutoSlog-TS syntax analyzer (Oraby, Reed, Compton, Riloff, Walker, & Whittaker, 2015) and others assist in the automation.
   
3. Sementic analysis : As an alternative to deception cues, signals of truthfulness can also be analyzed and achieved by characterizing the degree of compatibility between a personal experience (e.g., a hotel review) as compared to a content “profile” derived from a collection of analogous data. This approach extends the n-gram plus syntax model by incorporating profile compatibility features, showing the addition significantly improves classification performance. (Feng & Hirst, 2013). The intuition is that a deceptive writer with no experience with an event or object (e.g., never visited the hotel in question) may include contradictions or omission of facts present in profiles on similar topics. For product reviews, a writer of a truthful review is more likely to make similar comments about aspects of the product as other truthful reviewers. Extracted content from key words consists of attribute:descriptor pair. By aligning profiles and the description of the writer’s personal experience, veracity assessment is a function of the compatibility scores: 1.Compatibility with the existence of some distinct aspect (eg. an art museum near the hotel); 2. Compatibility with the description of some general aspect, such as location or service. Prediction of falsehood is shown to be approximately 91% accurate with this method.
4. Classifiers : Various classifiers such as SVM or naive bayes classifiers can be used to solve this.
5. Rhetorical structure and Discourse analysis : At the discourse level, deception cues present themselves both in CMC communication and in news content. A description of discourse can be achieved through the Rhetorical Structure Theory (RST) analytic framework, that identifies instances of rhetoric relations between linguistic elements. Systematic differences between deceptive and truthful messages in terms of their coherence and structure has been combined with a Vector Space Model (VSM) that assesses each message’s position in multi-dimensional RST space with respect to its distance to truth and deceptive centers (Rubin & Lukoianova, 2014). At this level of linguistic analysis, the prominent use of certain rhetorical relations can be indicative of deception. Tools to automate rhetorical classification are becoming available, although not yet employed in the context of veracity assessment.

Network approach

Innovative and varied, using network properties and behavior are ways to complement content-based approaches that rely on deceptive language and leakage cues to predict deception. As real-time content on current events is increasingly proliferated through micro-blogging applications such as Twitter, deception analysis tools are all the more important.

1. Linked data : The inherently structured data network of entities is connected through a predicate relationship. Fact checking can be effectively reduced to a simple network analysis problem: the computation of the simple shortest path. Queries based on extracted fact statements are assigned semantic proximity as a function of the transitive relationship between subject and predicate via other nodes. The closer the nodes, the higher the likelihood that a particular subject-predicate-object statement is true.
   
2. Social network behaviour

Closing notes

ufff…..that was long…
now i’d like to thank you for taking your time to read this quick conceptual level analysis of what i might offer , in a sense this is a representation of the way i approach a problem.and I tend to perform good when surrounded by talented induvidals in a work environment and the possibility of working at you’r organisation excites me deeply.

I belive that a company is defined by its people and i really like to be involved in your company so this was kind of my pitch, and i also have a view that when you hire people you must hire the best,as such I think I might be able to offer something new to your organisation and you must seriously consider my application.

So moving forward I hope to be a part of the formal interview, where you can assess my technical skills .