Blazingingly fast Data Science with GPU and rapids.ai
Ayush Kumar (~ayush26) |
If we look back last 10 years, the data size has grown exponentially whereas hardware specially the CPU compute power is almost the same as it was 10 years ago defeating moore's law. Which signifies that all the big data processing technologies, be it hadoop, spark etc. are no longer be useful in just few years looking at the rate of the growth of data. And to solve this, we simply need more compute power.
This is where GPU came in to rescue, as we all know that GPU had already proved its dominance in Deep Learning because of its huge parallel computer capability. And with the advent of rapids.ai and other GPU accelerated libraries, it is being used massively for the ETL and traditional ML also. The rapids.ai is a collection of open source python libraries that enables the execution of end-to-end data science and analytics pipelines entirely on GPUs. Rapids.ai is supported by NVIDIA and licensed under Apache 2.0.
At Walmart, we heavily use rapids.ai, XGBoost, TensorFlow for gpu acceleration as well as we have converted some of our custom algorithms to low level CUDA code. With all of this, any organisation's most important resources, Data Scientists are being more and more productive.
Slight understanding of Bigdata and GPU
Data Science libraries like Pandas, Scikit-learn and etc.
Rapids.ai Github: https://github.com/rapidsai
Rapids doc: https://docs.rapids.ai/
A gpu computing enthusiast and an open-source contributor to NVIDIA’s Rapids, Kubeflow and member of JupyterLab. And a founding member of GPU center of excellence at WalmartLabs. Currently, working as software developer in Machine Learning Platform in WalmartLabs, Bangalore. I am also conducting internal training to educate associates about the GPU computing along with the hands-on with various GPU accelerated libraries, one of participants, wrote a blog about his experience on using gpu: https://medium.com/walmartlabs/how-gpu-computing-literally-saved-me-at-work-fc1dc70f48b6