Description:

In the poster, the audience will understand how open-source ML tooling like Flyte can be used to provide a declarative specification for the infrastructure required for a wide array of ML workloads, including the fine-tuning of LLMs, even with limited resources. Thus the attendee will learn how to leverage open-source ML toolings like Flyte's capabilities to streamline their ML workflows, overcome infrastructure constraints, reduce cost, and unlock the full potential of LLMs in their specific use case. Thus making it easier for a larger audience to leverage and train LLMs.

LLMs used in tools like ChatGPT are everywhere; however, only a few organizations with massive computing resources can train such large models. While eager to fine-tune these models for specific applications, the broader ML community often grapples with significant infrastructure challenges. Also, another significant challenge is to keep these LLMs up to date thus requiring techniques like RAG and external data storage.

In the poster, the audience will understand how open-source ML tooling like Flyte can be used to provide a declarative specification for the infrastructure required for a wide array of ML workloads, including the fine-tuning of LLMs, even with limited resources. Thus the attendee will learn how to leverage open-source ML toolings like Flyte's capabilities to streamline their ML workflows, overcome infrastructure constraints, reduce cost, and unlock the full potential of LLMs in their specific use case. Thus making it easier for a larger audience to leverage and train LLMs.

Main Points Covered in the Poster: - The infrastructure requirements and challenges for fine-tuning LLM models - How to use Retrieval Augmented Generation to keep LLMs up to date. - Using modern techniques for fine-tuning LLMs like 8-bit quantization and LoRA - How open-source ML Orchestrator frameworks like Flyte's declarative specification and abstractions can automate and simplify infrastructure setup - Leveraging open-source tooling to specify ML workflows for fine-tuning large language models - How Flyte can reduce infrastructure costs and optimize resource usage

Through this poster, attendees will understand how open-source ML orchestration tooling can unlock the full potential of large language models by making their fine-tuning easier and more accessible, even with limited resources. This will enable a larger community of researchers and practitioners to leverage and train large language models for their specific use cases.

Prerequisites:

Basics of ML/AI

Content URLs:

https://docs.google.com/presentation/d/1in2iXAhHfZFTjbYV39SIzgll4W1pvdfNWN1Qu3fcZc8/edit?usp=sharing

Speaker Info:

Shivay

Biography

Shivay Lamba is a software developer specializing in DevOps, Machine Learning and Full Stack Development.

He is an Open Source Enthusiast and has been part of various programs like Google Code In and Google Summer of Code as a Mentor and has also been a MLH Fellow. He is actively involved in community work as well. He is a TensorflowJS SIG member, Mentor in OpenMined and CNCF Service Mesh Community, SODA Foundation and has given talks at various conferences like Github Satellite, Voice Global, Fossasia Tech Summit, TensorflowJS Show & Tell.

Speaker Links:

https://github.com/shivaylamba

https://twitter.com/howdevelop

KubeCon 2022: https://youtu.be/VCYdz_eKEaE

Pycon USA 2022 Lightning talk: https://www.youtube.com/watch?v=1IiL31tUEVk&ab_channel=PyConUS

Github Satellite  : https://www.youtube.com/watch?v=RmcMORdFMQA ( Introduction to TensorFlow.js )

Node Congress : https://www.youtube.com/watch?v=Riqbl6zFdak ( Machine Learning in Nodejs using TensorFlow.js )

React Native EU : https://www.youtube.com/watch?v=xKOkILSLs0Q&t=19750s ( Machine Learning in React Native apps using MediaPipe )

DevRelCon Tokyo 2021 : https://www.youtube.com/watch?v=IG5IziWonK8 ( Hackathons : A source to empower developer communities )

Belpy 2021 : https://www.youtube.com/watch?v=yh9KhbU7l_Y ( Converting Python based machine learning models to Javascript using TensorFlow.js Converter )

Voice Global Summit : https://www.modev.com/en/workshop-blog/shivay-lamba ( Voice in Networking )

CNCF Meetup : https://www.youtube.com/watch?v=ZgSZkZDarj8 ( Introduction to Service Meshes )

PromCon NA at KubeCon : https://youtu.be/LTzMobfHsIc

Cloud Native WASM Day : https://youtu.be/wjt11RbOcww