Real-Time stream computation on graphs using Storm, Neo4j and Python
by Sonal Raj (speaking)
Objective
This session will introduce the working principles of the Storm real-time computation system and the Neo4J graph database and how to use them in conjunction to perform stream computations on distributed graph data systems using Python APIs.
Description
Storm
We will briefly outline the installation and features of Twitter’s Storm framework and basic operations which make it an ideal tool for real-time stream processing of data. We will also look into it's Fault-Tolerance capabilities using programmatic implementations.
Neo4J
We will briefly introduce Neo4j, a No-SQL graph Database – how it works, its APIs, working with REST and Cypher.
Distributed Graph Operations
We shall then focus on how Storm and Neo4J can be made to work in co-ordination in a distributed environment to produce efficient and fast operations on significantly large graph data. The session will also include a live implementation of a scenario to demonstrate the use of Storm, Neo4J and their co-ordinated jobs in Python.
Finally the growing social applications of graph databases will not be a mystery anymore !
Requirements
- basic knowledge of Distributed Systems and Databases. Although idea of Storm or Neo4J would help.
Speaker bio
I am an artist at heart, a technocrat by profession and a python enthusiast. I am passionate about python as a futuristic technology and use it avidly for development. I am a developer for Enfoss Inc. , where Python is an integral part of data analytics and business intelligence products.
I have given talks on Storm Framework, whose link is as follows :
Storm Presentation at SERC
3
▼
This talk is a duplicate.
http://in.pycon.org/funnel/2013/1-introduction-to-graph-database-and-neo4j-using-python
1
▼
Hi Pallav, This is not a duplicate talk, it definitely includes the one you referenced. This talk will focus more on the Integration of Storm Framework with graph databases like Neo4J. The talk will cover more aspects distributing workloads on a Storm-Neo4j system, rather than the operations of Neo4J itself. So, the only reason it seems similar to the above talk is that in order to explain the integration process, the audience would require an introduction to Neo4J. We just can't mention it in the requirements, can we?
1
▼
Hi , are you doing integration using Python ?
1
▼
Hi Harshit! Yes, I am using the Python Bindings for both Storm and Neo4J so the integration is also in python. Although I have done it in java natively, but the python adapters do make it comfortable for end users.