Stream text data analysis on twitter using apache spark streaming

Abstract

With today's developing technology, people's access to information and its production have reached a very fast level. These generated and obtained information are instantly created, entered into data systems and updated. Sources of streaming data can be transformed into valuable analysis results when they are handled with targeted methods. In this study, a text data field is determined to perform analysis on instantaneous generated data and Twitter, the richest platform for instant text data, is used. Twitter instantly generates a variety of data in large quantities and it presents it as open source using an API. A machine learning framework Apache Spark's stream analysis environment is used to analyze these resources. Situation analysis was performed using Support Vector Machine, Decision Trees and Logistic Regression algorithms presented under this environment. The results are presented in tables.