Frontiers of Computational Journalism week 1 - Introduction and High Dimensional Data

1. Frontiers of Computational Journalism Columbia Journalism School Week 1: High Dimensional Data September 12, 2018

2. “Stories will emerge from stacks of financial disclosure forms, court records, legislative hearings, officials' calendars or meeting notes, and regulators' email messages that no one today has time or money to mine. With a suite of reporting tools, a journalist will be able to scan, transcribe, analyze, and visualize the patterns in these documents.” - Cohen, Hamilton, Turner, Computational Journalism, 2011 Computational Journalism: Definitions

4. Surgeon Scorecard, ProPublica 2015

5. Computational Journalism: Definitions “Broadly defined, it can involve changing how stories are discovered, presented, aggregated, monetized, and archived. Computation can advance journalism by drawing on innovations in topic detection, video analysis, personalization, aggregation, visualization, and sensemaking.” - Cohen, Hamilton, Turner, Computational Journalism, 2011

6. Journalism & Technology: Big Data, Personalization & Automation Shailesh Prakash, The Washington Post

7. Kony 2012 early network, Gilad Lotan

8. We are now living in a world where algorithms, and the data that feed them, adjudicate a large array of decisions in our lives: not just search engines and personalized online news systems, but educational evaluations, the operation of markets and political campaigns, the design of urban public spaces, and even how social services like welfare and public safety are managed. … Journalists are beginning to adapt their traditional watchdogging and accountability functions to this new wellspring of power in society. They are investigating algorithms in order to characterize their power and delineate their mistakes and biases. - Nick Diakopoulos, Algorithmic Accountability, 2015 Computational Journalism: Definitions

9. Websites Vary Prices, Deals Based on Users' Information Valentino-Devries, Singer-Vine and Soltani, WSJ, 2012

10. Message Machine Jeff Larson, Al Shaw, ProPublica, 2012

11. “Computational Journalism” in this course Reporting on society, using computation and Reporting on computation in society

12. Natural Language Processing Visualization Sociology Artificial Intelligence Cognitive Science Statistics Graph Theory Text Analysis Filter Design Inference Algorithmic Accountability Network Analysis Disinformation Privacy and Security Information Retrieval Epistemology Course Topics

13. Administration Assignments Some assignments require programming, but your writing counts for more than your code! Final project Code, story, or research Course blog http://compjournalism.com Grading 40% assignements 40% final project 20% class participation

14. This class • Introduction • High dimensional data • Text analysis in journalism • The Document Vector Space model • The Overview document mining platform

15. High dimensional data

16. Vector representation of objects x1 x2 x3 xN é ë ê ê ê ê ê ê ê ù û ú ú ú ú ú ú ú Fundamental representation for (almost) all data mining, clustering, machine learning, visualization, NLP, etc. algorithms.

17. Interpreting High Dimensional Data UK House of Lords voting record, 2000-2012. N = 1043 lords by M = 1630 votes 2 = aye, 4 = nay, -9 = didn't vote

18. Dimensionality reduction Problem: vector space is high-dimensional. Up to thousands of dimensions. The screen is two-dimensional. We have to go from x ∈ RN to much lower dimensional points y ∈ RK<<N Probably K=2 or K=3.

19. Projection Projection from 3 to 2 dimensions

20. Which direction should we look from? Principal components analysis: find a linear projection that preserves greatest variance Take first K eigenvectors of covariance matrix corresponding to largest eigenvalues. This gives a K-dimensional sub-space for projection.

21. PCA on House of Lords data Classification is arguably one of the most central and generic of all our conceptual exercises. It is the foundation not only for conceptualization, language, and speech, but also for mathematics, statistics, and data analysis in general. Kenneth D. Bailey, Typologies and Taxonomies: An Introduction to Classification Techniques

22. UK House of Lords PCA notebook

23. Classification and Clustering Classification is arguably one of the most central and generic of all our conceptual exercises. It is the foundation not only for conceptualization, language, and speech, but also for mathematics, statistics, and data analysis in general. Kenneth D. Bailey, Typologies and Taxonomies: An Introduction to Classification Techniques

24. Distance metric d(x, y) ≥ 0 - distance is never negative d(x, x) = 0 - “reflexivity”: zero distance to self d(x, y) = d(y, x) - “symmetry”: x to y same as y to x d(x, z) ≤ d(x, y) + d(y, z) - “triangle inequality”: going direct is shorter

25. Distance matrix Data matrix for M objects of N dimensions Distance matrix X = x1 x2 xM é ë ê ê ê ê ù û ú ú ú ú = x1,1 x1,2 x1,N x2,1 x2,2 x1,M xM,N é ë ê ê ê ê ê ù û ú ú ú ú ú Dij = Dji = d(xi , xj ) = d1,1 d1,2 dM,M d2,1 d2,2 d1,M dM,M é ë ê ê ê ê ê ù û ú ú ú ú ú

26. Different clustering algorithms • Partitioning o keep adjusting clusters until convergence o e.g. K-means o Also LDA and many Bayesian models, from a certain perspective • Agglomerative hierarchical o start with leaves, repeatedly merge clusters o e.g. MIN and MAX approaches • Divisive hierarchical o start with root, repeatedly split clusters o e.g. binary split

27. K-means demo https://www.naftaliharris.com/blog/visualizing-k-means-clustering/

28. UK House of Lords voting clusters Algorithm instructed to separate MPs into five clusters. Output: 1 2 2 1 3 2 2 2 1 4 1 1 1 1 1 1 5 2 1 1 2 2 1 2 3 2 2 4 2 1 2 3 2 1 3 1 1 2 1 2 1 5 2 1 4 2 2 1 2 1 1 4 1 1 4 1 2 2 1 5 1 1 1 2 3 3 2 2 2 5 2 3 1 2 1 4 1 1 4 4 1 1 2 1 1 2 2 2 2 1 2 1 2 1 2 2 1 3 2 1 1 2 2 1 2 3 4 2 2 2 . .

29. Voting clusters with parties LDem XB Lab LDem XB Lab XB Lab Con XB 1 2 2 1 3 2 2 2 1 4 Con Con LDem Con Con Con LDem Lab Con LDem 1 1 1 1 1 1 5 2 1 1 Lab Lab Con Lab XB XB Lab XB Lab Con 2 2 1 2 3 2 2 4 2 1 Lab XB Lab Con XB XB LDem Lab XB Lab 2 3 2 1 3 1 1 2 1 2 Con Con Lab Con XB Lab Lab Con XB XB 1 5 2 1 4 2 2 1 2 1 Con XB Con Con XB Con Lab XB LDem Con 1 4 1 1 4 1 2 2 1 5 Con Con Con Lab Bp XB Lab Lab Lab LDem 1 1 1 2 3 3 2 2 2 5 Lab XB Con Lab Con XB Con Con XB XB 2 3 1 2 1 4 1 1 4 4 Con Con Lab Con Con XB Lab Lab Lab Con 1 1 2 1 1 2 2 2 2 1 Lab LDem Lab Con Lab Lab Con XB Lab Con 2 1 2 1 2 2 1 3 2 1 Con Lab XB Con XB XB XB Lab Lab Lab 1 2 2 1 2 3 4 2 2 2

30. No unique “right” clustering Different distance metrics and clustering algorithms give different results. Should we sort incident reports by location, time, actor, event type, author, cost, casualties…? There is only context-specific categorization. And the computer doesn’t understand your context.

31. Different libraries, different categories

32. Clustering Algorithm Input: data points (feature vectors). Output: a set of clusters, each of which is a set of points. Visualization Input: data points (feature vectors). Output: a picture of the points.

33. Linear projections (like PCA) Projects in a straight line to closest point on "screen.” y = Px where P is a K by N matrix. Projection from 2 to 1 dimensions

34. Nonlinear projections Still going from high- dimensional x to low- dimensional y, but now y = f(x) for some function f(), not linear. So, may not preserve relative distances, angles, etc. Fish-eye projection from 3 to 2 dimensions

35. Multidimensional scaling Idea: try to preserve distances between points "as much as possible." If we have the distances between all points in a distance matrix, D = |xi – xj| for all i,j We can recover the original {xi} coordinates exactly (up to rigid transformations.) Like working out a country map if you know how far away each city is from every other. But notice that the original dimension is not encoded in the matrix… we can re-project to any number of dimensions.

36. Multidimensional scaling Torgerson's "classical MDS" algorithm (1952)

37. MDS Stress minimization The formula actually minimizes “stress” Think of “springs” between every pair of points. Spring between xi, xj has rest length dij Stress is zero if all high-dimensional distances matched exactly in low dimension. stress(x) = xi - xj - dij( ) 2 i, j å

38. Multi-dimensional Scaling Like "flattening" a stretchy structure into 2D, so that distances between points are preserved (as much as possible")

39. House of Lords MDS plot

40. Robustness of results Regarding these analyses of legislative voting, we could still ask: • Are we modeling the right thing? (What about other legislative work, e.g. in committee?) • Are our underlying assumptions correct? (do representatives really have “ideal points” in a preference space?) • What are we trying to argue? What will be the effect of pointing out this result?

41. Text Analysis in Journalism

42. Count incident types by date. For Level 14, ProPublica, 2015

43. The Child Exchange, Reuters, 2014

44. USA Today/Twitter Political Issues Index

45. Politico analysis of GOP primary, 2012

46. The Post obtained draft versions of 12 audits by the inspector general’s office, covering projects from the Caribbean to Pakistan to the Republic of Georgia between 2011 and 2013. The drafts are confidential and rarely become public. The Post compared the drafts with the final reports published by the inspector general’s office and interviewed former and current employees. E- mails and other internal records also were reviewed. The Post tracked changes in the language that auditors used to describe USAID and its mission offices. The analysis found that more than 400 negative references were removed from the audits between the draft and final versions. Sentiment analysis used by Washington Post, 2014

47. LAPD Underreported Serious Assaults, Skewing Crime Stats for 8 Years Los Angeles Times

48. The Times analyzed Los Angeles Police Department violent crime data from 2005 to 2012. Our analysis found that the Los Angeles Police Department misclassified an estimated 14,000 serious assaults as minor offenses, artificially lowering the city’s crime levels. To conduct the analysis, The Times used an algorithm that combined two machine learning classifiers. Each classifier read in a brief description of the crime, which it used to determine if it was a minor or serious assault. An example of a minor assault reads: "VICTS AND SUSPS BECAME INV IN VERBA ARGUMENT SUSP THEN BEGAN HITTING VICTS IN THE FACE.”

49. We used a machine-learning method known as latent Dirichlet allocation to identify the topics in all 14,400 petitions and to then categorize the briefs. This enabled us to identify which lawyers did which kind of work for which sorts of petitioners. For example, in cases where workers sue their employers, the lawyers most successful getting cases before the court were far more likely to represent the employers rather than the employees. The Echo Chamber, Reuters

50. Document Vector Space Model

51. Message Machine clusters emails Using TF-IDF document vectors

52. Document vectors in journalism - Text clustering for stories, e.g. Message Machine - Find “key words” or “most important words” - Topic analysis, e.g. ProPublica’s legislative tracker - Key component of filtering algorithms, e.g. Google News - Standard representation for document classification. - Basis of all text search engines. A text analysis building block.

53. What is this document "about"? Most commonly occurring words a pretty good indicator. 30 the 23 to 19 and 19 a 18 animal 17 cruelty 15 of 15 crimes 14 in 14 for 11 that 8 crime 7 we

55. Features = words works fine Encode each document as the list of words it contains. Dimensions = vocabulary of document set. Value on each dimension = # of times word appears in document

56. Example D1 = “I like databases” D2 = “I hate hate databases” Each row = document vector All rows = term-document matrix Individual entry = tf(t,d) = “term frequency”

57. Aka “Bag of words” model Throws out word order. e.g. “soldiers shot civilians” and “civilians shot soldiers” encoded identically.

58. Tokenization The documents come to us as long strings, not individual words. Tokenization is the process of converting the string into individual words, or "tokens." For this course, we will assume a very simple strategy: o convert all letters to lowercase o remove all punctuation characters o separate words based on spaces Note that this won't work at all for Chinese. It will fail in ,many ways even for English. How?

59. Distance metric for text Useful for: • clustering documents • finding docs similar to example • matching a search query Basic idea: look for overlapping terms

60. Cosine similarity Given document vectors a,b define If each word occurs exactly once in each document, equivalent to counting overlapping words. Note: not a distance function, as similarity increases when documents are… similar. (What part of the definition of a distance function is violated here?) similarity(a,b) º a·b

61. Problem: long documents always win Let a = “This car runs fast.” Let b = “My car is old. I want a new car, a shiny car” Let query = “fast car” this car runs fast my is old I want a new shiny a 1 1 1 1 0 0 0 0 0 0 0 0 b 0 3 0 0 1 1 1 1 1 1 1 1 q 0 1 0 1 0 0 0 0 0 0 0 0

62. similarity(a,q) = 1*1 [car] + 1*1 [fast] = 2 similarity(b,q) = 3*1 [car] + 0*1 [fast] = 3 Longer document more “similar”, by virtue of repeating words. Problem: long documents always win

63. Normalize document vectors similarity(a,b) º a·b a b = cos(Θ) returns result in [0,1]

64. Normalized query example this car runs fast my is old I want a new shiny a 1 1 1 1 0 0 0 0 0 0 0 0 b 0 3 0 0 1 1 1 1 1 1 1 1 q 0 1 0 1 0 0 0 0 0 0 0 0 similarity(a,q) = 2 4 2 = 1 2 » 0.707 similarity(b,q) = 3 17 2 » 0.514

65. Cosine similarity cosq = similarity(a,b) º a·b a b

66. Cosine distance (finally) dist(a,b) º1- a·b a b

67. Problem: common words We want to look at words that “discriminate” among documents. Stopwords: if all documents contain “the,” are all documents similar? Common words: if most documents contain “car” then car doesn’t tell us much about (contextual) similarity.

68. Context matters Car ReviewsGeneral News = contains “car” = does not contain “car”

69. Document Frequency Idea: de-weight common words Common = appears in many documents “document frequency” = fraction of docs containing term df (t,D)= d Î D:t Î d D

70. Inverse Document Frequency Invert (so more common = smaller weight) and take log idf (t,D) = log D d Î D:t Î d( )

71. TF-IDF Multiply term frequency by inverse document frequency n(t,d) = number of times term t in doc d n(t,D) = number docs in D containing t tfidf (t,d,D)= tf (t,d)×idf (d,D) = n(t,d)×log D n(t,D)( )

72. TF-IDF depends on entire corpus The TF-IDF vector for a document changes if we add another document to the corpus. TF-IDF is sensitive to context. The context is all other documents tfidf (t,d,D)= tf (t,d)×idf (d,D) if we add a document, D changes!

73. What is this document "about"? Each document is now a vector of TF-IDF scores for every word in the document. We can look at which words have the top scores. crimes 0.0675591652263963 cruelty 0.0585772393867342 crime 0.0257614113616027 reporting 0.0208838148975406 animals 0.0179258756717422 michael 0.0156575858658684 category 0.0154564813388897 commit 0.0137447439653709 criminal 0.0134312894429112 societal 0.0124164973052386 trends 0.0119505837811614 conviction 0.0115699047136248 patterns 0.011248045148093

76. Salton’s description of tf-idf - from Salton et al, A Vector Space Model for Automatic Indexing, 1975

77. TF nj-sentator-menendez corpus, Overview sample files color = human tags generated from TF-IDF clusters TF-IDF

78. Cluster Hypothesis “documents in the same cluster behave similarly with respect to relevance to information needs” - Manning, Raghavan, Schütze, Introduction to Information Retrieval Not really a precise statement – but the crucial link between human semantics and mathematical properties. Articulated as early as 1971, has been shown to hold at web scale, widely assumed.

79. Bag of words + TF-IDF widely used Practical win: good precision-recall metrics in tests with human-tagged document sets. Still the dominant text indexing scheme used today. (Lucene, FAST, Google…) Many variants and extensions. Some, but not much, theory to explain why this works. (E.g. why that particular IDF formula? why doesn’t indexing bigrams improve performance?) Collectively: the vector space document model

Frontiers of Computational Journalism week 1 - Introduction and High Dimensional Data

Frontiers of Computational Journalism week 1 - Introduction and High Dimensional Data

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados(20)

Similar a Frontiers of Computational Journalism week 1 - Introduction and High Dimensional Data

Similar a Frontiers of Computational Journalism week 1 - Introduction and High Dimensional Data(20)

Mais de Jonathan Stray

Mais de Jonathan Stray(12)

Último

Último(20)

Frontiers of Computational Journalism week 1 - Introduction and High Dimensional Data

Notas do Editor