Deep Hybrid Neural Models with Optimizers - Analysis for Multiclass classification in NLP

Harsh Sharma (~harsh444)


In Recent Advances in NLP, Multi-class classification has become an important task. It is based on textual data refers to a supervised machine learning task, where we try to predict a certain class for input data, given the input data in raw textual format. Well, thanks to TensorFlow, we have a variety of algorithms available to perform multi-class classification via natural language processing.

For any deep learning enthusiast, it's always a tedious task to know, which type of individual neural network along with a correct set of loss functions and optimizers can lead to the best performance of their models. Furthermore, a computational constraint might also impact their choices for selecting a neural network. Hence, I have been doing detailed research analysis and comparing various hybrid combinations of deep neural networks and choosing different optimizer's impact on their learning rates for models, choosing suitable callbacks in TensorFlow while training models for their performance with total time taken to train them.

Therefore, I would like to propose a talk on this topic, which could help many developers derive insightful results on these hybrid deep neural networks. I would be explaining why to go for hybrid models, instead of individual neural networks. We would be looking at a comparative analysis between :

  • 3 Standard Neural Networks
  • 2 Hybrid Neural Networks

Along with adaptive and non-adaptive optimizers selection :

  • Adam
  • SGD
  • AdaGrad
  • RMSProp
  • Adadelta

This analysis would be consisting of results and inferences of each individual model with its respective model's performance - Loss vs Accuracy visualized on plots. We will also observe the total time taken by a hybrid model combination to train on GPU and non-GPU environments. I would be explaining it with the results I obtained using the Tensorflow framework.

Talk Outline :

Origin [ 3 minutes ]:

  • Short Intro of Speaker
  • Introduction of the Talk

Basic Theory [ 5 minutes ] :

  • Intro To Hybrid Neural Networks - Recent Trend in NLP
  • Why Hybrid Neural Networks are better than Standard Neural Networks
  • Discussing One of the common Hybrid Neural Networks for Natural Language Processing Tasks - Multi-Class Classification

Breaking Down of Hybrid Neural Layers [ 7 minutes ] :

  • Brief Working of LSTMs
  • Brief Working of Bidirectional LSTMs
  • Brief Working of ConvNet 1D Networks - How they have improved feature extraction in spatial Text Data.
  • Pipeline for Hybrid Neural Network - How to stack up the neural network layers.

Analysis and Inferences [ 10 minutes ]:

  • Breakdown and Analysis of Standard Neural Network and Hybrid Neural Network
  • Comparing Inferences and Results obtained using Different Optimizers - Adaptive vs Non-Adaptive optimizers.
  • Observing the Model's Performance with the help of Plots and Graphs Obtained.
  • Comparing Time taken to train the model in GPU vs Non-GPU Environment in Summarised Evaluation.

Final Analysis and Conclusion [ 5 minutes ]:

  • Final Conclusion - Who won - Standard Neural Network or Hybrid Neural Network?
  • Final Tips for NLP practitioners could try to achieve better performances with Hybrid Neural Networks.
  • Q & A Round


  • General Knowledge of Deep Learning and its basics [Activation Functions, Basic Optimizers as per Keras Documentation]
  • Basic Knowledge of Neural Networks like Convolutional Neural Networks [CNN], Long Short Term Memory Networks [LSTMs]
  • Basic Knowledge of Natural Language Processing [NLP] and experience with Natural Language Toolkit [NLTK] in terms of Deep Learning.
  • General Knowledge of Tensorflow Framework and Python.

Video URL:

Speaker Info:

Harsh Sharma is currently a CSE UnderGrad Student at SRM Institute of Science and Technology, Chennai. He is a Data Science Enthusiast and a passionate deep learning developer and researcher, who loves to work on projects belonging to Data Science Domain, By using a wide range of sources Available from Kaggle and other data-sharing platforms, to make some accessible models, by Applying appropriate analytical skills and algorithms. He has been shortlisted as finalists in quite a few hackathons and part of student-led tech clubs, handling executive positions after getting reviewed for his hard work, passion, and skills for deep learning. His contribution as a co-author in one of the research papers also got selected for IEEE-INCET 2021. Currently, he works mentoring juniors in hackathons,Hosting and organizing webinar sessions on deep learning, writing blogs based on deep learning-based natural language processing content, and collaborating with fellow deep learning developers on state-of-the-art projects and research titles constantly. He is also currently working as ML Developer in Start-up called Insolvers tech, a Product Software with Seniors. He is also one of the Founding Members of Github Community SRM, the foremost student-led community spearheading the Open Source.

Mentored @ Hackathons:

  • Ignite3.0 Sparkathon - By NMIMS
  • Vihaan 4.0 - By IEEE- DTU
  • Tech Ingenium '21 - By Ahmedabad University
  • Hacowasp Hackathon - By Thapar Institute of Engineering And Technology
  • ACF Family Fitness Hackathon'21 - By Applied Computing Foundation Organisation

Public Talk Sessions:

  • Tensorflow User Group Chennai [TFUG] - May '21: About Siamese Neural Networks in NLP
  • ACF Family Fitness Hack'21 - April '21: Efficient Deep Learning Practices in NLP
  • NeuRes 1.0 Session - March'21 : Hybrid NLP Model Practices in Deep Learning

Section: Data Science, Machine Learning and AI
Type: Talks
Target Audience: Intermediate
Last Updated: