Automating testbed experiments, data collection and visualization with Python
Sunit Kumar Nandi (~sunitknandi) |
During my M.Tech. programme at IIT Guwahati, I observed that researchers in both industry and academia work with testbeds, both real and virtual, for making advancements in Computer Science, whether it is in algorithms, networking protocols or data science by running exhaustive experiments. I realised that Python, being a very versatile language, can be used to do everything related to experimental research and analysis, without requiring the usage of any other scripting language. Based on my experiences, I am presenting a talk to explain how to build automated testbed experiments, data collection and analysis with Python and a few libraries, avoiding big and bulky frameworks as much as possible.
My talk is structured as follows:
- Building a testbed for computing and networking experiments
- Using Python and
paramikoto provison entities (PCs, smartphones, Raspberry Pis, routers, switches, etc.) in the testbed
- Running tests on the entities with
- Collecting log files and other trace data from testbed entities
- Parsing log files and trace information to collect statistics with basic text processing and regex and storing them in appropriate Python data structures like lists, tuples and dictionaries for easy access
- Analysing collected statistics with Python
mathand generating reports
- Visualizing graphs from statistics with
I hope that after attending my talk, you will be able to automate your testbed experiments to the extent of spending less time on experimentation and data collection and more time on actual research and publishing papers.
There are minimal prerequisities for my talk. You need to have knowledge of the following:
- Basic algorithms and data structures
- Computer networks, especially IP addressing
- Python basics
Also you need to have:
- The willingness to learn and experiment
Contents related to the talk will be added later.
Hello everyone! I am Sunit Kumar Nandi, a Trainee Teacher at National Institute of Technology, Arunachal Pradesh. I have completed my M.Tech. at IIT Guwahati this year and am also enrolling for Ph.D.
I am deeply interested in computer networking, telecommunications, operating systems and distributed systems design. I use Python for most of my daily work involving a great deal of experimentation. Apart from that I contribute to SuperX OS, a Linux distribution with KDE, based out of Assam, India. I love BSD and Linux based systems and have been involved with them since my childhood. As a result, I have had 14 years of experience with managing Linux servers, networking equipment and designing automated systems in the simplest way possible. In my free time, I spend my efforts running Techno FAQ, an e-magazine for science, technology, education and business.