Deploying AirBnB’s Knowledge Repo at scale

Abhirath Batra (~abhirath54)


Description:

What :

Knowledge Repo, an open-source knowledge sharing platform by Airbnb, has rapidly become a tool of choice for Data Scientists to share analyses and insights while ensuring reproducibility, a key foundation of any scientific research. It allows for analyses done in Jupyter Notebooks and R-Markdowns to be curated in an immutable, presentable, searchable and shareable manner allowing seamless consumption of insights and knowledge.

The problem :

However, Knowledge Repo has been made to work comfortably for internal teams, with a selected set of “repositories” loaded on deployment. It would also need an internal server admin of sorts to manage the server.

What we did : The poster will talk about our fork of Knowledge Repo which is modified to :

  1. Use Database backend to remove the need to use KRs(Knowledge Repositories)
  2. Add KRs to a deployed server on the fly.
  3. Add posts to KRs directly
  4. Attach this as a microservice to a platform(case in point : Polly[HTML_REMOVED]TM[HTML_REMOVED])

How: It will also talk about how these features were implemented by:

  1. Leveraging the impeccable Object Oriented design of KR
  2. Modifying Database object and connection cycles
  3. URL blueprinting to allow reverse proxy

Prerequisites:

  1. Jupyter notebook
  2. Object Oriented Programming
  3. Some web development

Content URLs:

https://github.com/ElucidataInc/knowledge-repo

Speaker Info:

I'm Abhirath Batra, working at Elucidata as a Bioinformatics Engineer. I've been interested in using computing to solve problems in science and have worked on molecular simulations, ML and data to solve questions in biological systems

Speaker Links:

https://www.linkedin.com/in/abhirath-batra-10740587/

Id: 1636
Section: Data Science, Machine Learning and AI
Type: Poster
Target Audience: Intermediate
Last Updated: