Real time Sentiment Analysis with Apache Storm and Python

Shaik Asifullah (~shaik2)


13

Votes

Description:

Data is everywhere. How much data can we process in real time? How twitter is managing this huge data? This talk will help in understanding Big Data better. Twitter uses Apache Storm. Take a dive into Apache storm and learn more about Twitter Sentiment Analysis in Real Time.

This will help you get started with Apache Storm with one use case of Sentiment Analysis. This talk will be very basic and intends to motivate the attendees towards Apache Storm and help them to understand Apache Storm better.

Apache Storm can process tens of thousands of messages in a second, and if properly configured it can process millions in a second. But we shall be using some dump of twitter tweets and use it for sentiment Analysis with simple Heuristics. And if time permits we will use tweepy library to get real time streaming from twitter.

I shall be using Petrel (a Python Library) to submit the Storm topologies that we together build in our talk session. If time permits, setting up of Apache Storm shall be demoed and run the sample Topology.

Prerequisites:

Basic Python Knowledge

Basic Awareness of NLP is a plus though not required

Content URLs:

https://drive.google.com/open?id=0B6nwjmOC5sCGZ3k1ekZvWmVTNHc

https://github.com/apache/storm

http://hortonworks.com/apache/storm/

Speaker Info:

Preetam Purbia:

Software Engineer at @WalmartLabs. With 6+ years experience and Currently working on BigData tech stack like Kafka, Spark, Storm. Also worked on retail fulfillment product with capabilities like order life cycle management, picking path optimization. Filed 3 Patents related to E-commerce Domain.

Shaik Asifullah:

Software Engineer at @WalmartLabs. Python Enthusiast with great interest in Big Data. Interned at Calfor Finance and worked under Dr. Sulkhan, CEO Calfor Finance, Teaching Assistant at University of Zurich. Also interned at GreyOrange Robotics. Worked extensively on Sentiment Analysis with Apache Storm. Loves Cycling, Meta Physics. Also working on a paper related to Psychology and Twitter Sentiment.

Speaker Links:

Preetam Purbia: https://www.linkedin.com/in/preetampurbia

Shaik Asifullah: www.linkedin.com/in/shaikasifullah

Section: Data Visualization and Analytics
Type: Talks
Target Audience: Intermediate
Last Updated:

Hi,

Please update the description with a proper outline of what the talk is going to be about.

Can you talk about:

  1. Scale at which Apache Storm is being used.
  2. Is the use-case mentioned by you a production use-case or "How to get started with Apache Storm".
  3. What libraries are you planning to share like StreamParse, Pyleus or any other. Why you would choose one over the other ?
  4. What kind of deployment strategies do you follow to deploy storm cluster like green/blue deployment strategy or something else ?
  5. Since you are mentioning Big Data, will you also cover how Storm cluster can be tuned and configured beyond basic deployment ?
konark modi (~konark)

Thanks for your comment.

This talk is intended for beginners who wants to know how to get started with Apache Storm for Real time Processing. I do use storm to process the huge data that we get from multiple sources (Spouts)

The use case that I mentioned (Sentiment Analysis) is just to motivate the attendees towards it. Processing the data (Tweets) that we get from Twitter will be one use case that I will be mentioning in my talk.

I will be using Petrel Library so as to make it simple and easy to build and help others to understand better.

And this will be a basic deployment.

Shaik Asifullah (~shaik2)

Apache storm ! How exactly can it help us Shaik asifullah

Riyazuddin shaik (~riyazuddin)

Apache storm ! How exactly can it help us Shaik asifullah

Riyazuddin shaik (~riyazuddin)

Thanks Riyaz. Say you want to analyze the twitter data flowing in real time, and to process such a large flow of data, you use Apache Storm. Attend the talk, to know more.

Shaik Asifullah (~shaik2)

Will you just dwell on the procedures and implementations of Apache Spark during the talk or the installation too?

Lakshya Pawa (~lakshya)

Login to add a new comment.