A Pythonista's Approach to DeepFakes
Navin Pai (~navin57) |
DeepFakes is “an AI-based human image synthesis technique used to combine and superimpose existing images and videos onto source images or videos”. It shot to prominence in late 2017 on forums like Reddit, and have been a topic of debate, both from a technical as well as an ethical standpoint since they first came out.
In this talk, we explore how DeepFakes works purely from a Python coder's perspective to answer Why and How and What we can do about it.
This talk is an "light-on-math, high on code" talk on how anyone can generate Deepfakes-like videos using off-the-shelf tools and frameworks such as
OpenCV among others. This technique borrows heavily from ideas originally described by the creator of the Deepfakes method on Reddit (and ideas borrowed from a paper by Nvidia).
This talk is not a deep-dive into Deep Learning. It's a specific problem statement that we are looking to solve using the tools we have available to us. We'll focus on similar problems and similar domains since we want to spend more time talking about code and the problem itself than about nitty-gritties of DL or arguing about which frameworks/tools to use.
- Introduction to DeepFakes (2-3 mins)
- Why Deep Learning? (3-4 mins) (Can skip this if we have some other speaker doing a basic intro to DL techniques)
- How traditional face-swapping CGI has worked (3 mins) - Why Deep Learning is a great fit for this problem (1.5 min)
- Breaking down the problem into components (2 mins)
- Data Generation w/ code (2 mins)
- Intro to
AutoEncodersw/ code (2-3 mins)
- Training + code (2-3 mins)
- Building a pipeline (4 mins) (If I don't have to concentrate on Introduction, I'll increase this part to dive a little deeper)
- Common problems + Problems I faced and getting past them (2 mins)
- Ethicality (1.5-2 mins)
- Current State of the Art + Where do we go from here (2 mins)
- Questions (~5 mins)
Should you attend?
You should attend if:
- You are interested in understanding what all the hype about Deepfakes is about
- You are interested in CGI
- You want to know how they created Hulk in Marvel movies (and brought back Grand Moff Tarkin in Star Wars)
- The words
pytorchpeek your interest
- Want to look at a specific use case of Python in the field of Deep Learning and clarify some ideas along the way
Do not attend if:
- You expect to learn DL from a 30 minute talk
- You are a DL practitioner expecting a discussion on research papers for 30 mins
- You don't like looking at code in talks
Basic working knowledge of Python, and an open mind.
Understanding of Deep Learning is helpful but not necessary. The talk isn't math heavy, or intended purely for DL folks, and I'll be abstracting concepts out to allow non-DL practitioners to follow along.
In the talk, I'll also touch upon a bunch of tools like OpenCV and ffmpeg, but no knowledge of those is required.
[WIP] Slide Deck: https://navinpai.github.io/pycon-deepfakes/
Also, I presented a more general, less Python-focussed talk on the same topic at ExpertTalks Pune. Video of the same is available on Youtube (though only parts of the talk will be common since the audience is pretty different)
M@dMAx” Pai is a code monkey from Goa. He graduated from Goa Engineering College, and completed his M.Tech at IIIT-Bangalore where he majored in AI and ML. He's currently part of Tact.AI, building an AI-powered platform for B2B sales workflows. He enjoys dabbling across tech stacks for a living and some of his more "fun" experiments have resulted in brilliant results such as shorted motherboards, burnt RAMs, and a $1000+ AWS bill from a unattended cluster.
He has previously spoken at a bunch of conferences (including NullCon, ExpertTalks, Anthill Inside) and won multiple hackathons (organized by MakeMyTrip, ICICI, IBM Watson, Wayray among others). He's also conducted introductory courses (mostly on Python/Machine Learning) at multiple colleges over the years.
In his free time, he enjoys talking about himself from a third person perspective in Speaker Info sections of conference CFP submissions.