Objective
Understand what is Elasticsearch, what problems can it solve, the Python ecosystem around it and how we are using all of this at Wingify for search and analytics.
Description
Elasticsearch is a flexible and power open-source distributed search engine based on Apache Lucene. Made for distributed environments from the ground up, Elasticsearch is a pragmatic system that tries to be easy to work with, maintain and scale where every other search system fails. Its REST APIs and support for JSON documents makes it extremely simple and flexible to use. While Elasticsearch was built for search, its querying, filtering and faceting capabilities have made Elasticsearch eligible to be one of the few simple solutions available for real-time analytics. We, at Wingify, have been using Elasticsearch for search and analytics both and it has proven to be an amazing solution for our needs.
This talk will be about:
- What is Elasticsearch? - In a gist, its querying and filtering capabilities and the flexibility its APIs have to offer.
- Python libraries to work with Elasticsearch.
- What kind of problems does it solve for us at Wingify and what features of Elasticsearch do we use for each problem.
- Search
- Analytics
- An architectural insight into our current setup to solve the above problems
- A glance of what Elasticsearch looks like in production in our infra
- What other tools are we using to work with Elasticsearch to make it of use to us
- Indexing and querying
- Maintenance and monitoring
- Scalability problems
- Sharding strategies and what worked for us
- The things about Elasticsearch we learned and what helped us the most
- Understanding mappings and how they can help
- The new Aggregations Framework
- The problems we faced with Elasticsearch
- Avoiding the common gotchas
- Problems with the Python library
- A few best practices and tips from our experience
Note: this talk is not a tutorial. It briefly introduces Elasticsearch just enough for the user to understand what it does and moves on to real-world usage scenario.
Speaker bio
I am a software engineer based out of New Delhi, working for Wingify, a Delhi based bootstrapped startup that develops the A/B testing tool - Visual Website Optimizer (VWO). At Wingify, I am primarily focused on services, scalability and infrastructure engineering, which also happens to involve working with data and analytics and all the problems that come with it. I am an open-source enthusiast. In my free time, I evangelize and try to organize local meetups and watch movies.