Exploring Augmented Reality with Python
Srikar Mutnuri (~srikar) |
Description:
This is the main focus of the talk :
Build a simple AR app, using Python.
While it is possible to develop instant Augmented Reality (AR) applications using some out-of-the-box libraries, the use of Python for AR, while still a bit nascent, is a portal to a plethora of opportunities - literally. At its core, AR can be said to be basically a Computer Vision problem with some twists, and thats something Python has already well taken care of, with numerous libraries (OpenCV, I'm looking at you), which should make it a very obvious choice for developing AR apps - but that is not the current situation.
True, the existing libraries (like ARCore, written in other languages like C++ or C#) have everything taken care of on our behalf, and allow us to simply write down a few lines of code to pop a virtual character here and a 3D object there in to the real world. But with the current SOTA in Python, we might have to sweat out and dig deeper into the nitty gritty of projecting a virtual object into the real world - feature extraction, homography and projection (among others), and handle every aspect of the process ourselves.
Given the extensive libraries and tools at our disposal when taking the Pythonic approach, all this could become quite simple. The central goal is to take the audience through this process, and equip them with means to build their own primitive AR app with Python.
In addition, I would also be exploring, in theory, some of the major problems which hinder good AR experiences, like occlusion and performance optimisation, and how various Machine Learning models can help us solve them.
Outline:
- Vision - Human and Machine (3 min)
- AR and Computer Vision (2 min)
- AR - under the hood (7 min)
- Some issues, and potential solutions (5 min)
- Demo (~5 min)
- Conclusion
Prerequisites:
- Basic knowledge about Python
- General understanding of Computer Vision and some linear algebra (recommended, but not necessary)
Content URLs:
Link to slides (WIP)
For a general overview of the topic, the below two articles can be a great start.
Additionally, the slides contain a list of further references towards the end.
Speaker Info:
Currently working as a Unity Developer in TCS Interactive Labs at Chennai, Srikar has over two years of experience in AR/VR, game design and big data. His primary area of interest is in the application of algorithms from ML/DL towards enabling XR.
Speaker Links:
A previous talk given on the subject at FOSSASIA 2019, by me: