Data as API: Building a Type Manager with Python
Vaideeswaran Ganesan (~vaideeswaran) |
Ever wonder creating your own super-type-manager leveraging the python's own type constructs?
Ever explored alternatives to APIs for integration?
In this talk, we will cover our experience in building a new type manager (as part of developing open source OpenManage(tm) Software Development Kit) leveraging pythons own type constructs and explore how this new type manager provides a credible alternative to APIs, especially in those information-heavy environments like Device Management.
Devices (like Servers, Switches, Telecom Switches) are data-intensive systems. Their information model is so intensive, that practically all operations (health, inventory, metrics, configuration) on the device ends up in primarily as CRUD operations on the information model they expose. Only a paltry few operations are exposed as APIs.
When building an API for managing these devices, we realized that providing classic function-style APIs only degraded the user experience. What we realized was there was significant information available on the Servers, and providing an API for exposing traditional CRUD (Create, Retrieve, Update and Delete) for all information nuggets was just exploding the API sets. It was not necessarily covering all the scenarios that could be possible for management and did not seem to scale.
Our approach was to take this information model within the devices and expose them as a huge navigable data structure representing the entire spectrum of the device and provide a language native experience. We created a new type manager leveraging the python class special operators (getattr(), setattr(), le() etc.) to create a whole new type manager that provides additional controls and safeguards. Some of the safeguards include:
- Not allowing edits to read-only components
- Allowing only applicable changes only (ranges, enumerations)
- Providing native python experience for special types (IP Address Types etc.)
- Providing mechanisms to validate cross-attribute validations
- Providing custom indices for arrays (like Virtual Disks, Users)
- Providing mechanism for tracking changes to configuration
- Apply changes to the device optimally
- Provide mechanisms for identifying configuration drifts
Outline of the presentation:
- Device Configuration - Aspects & Peculiarities
- Pitfalls of API approach for Device Configuration
- Type Manager - introduction
- Super Types - Enumerations, Fields, Classes and Arrays
- Bringing in Native Type Experience
- Data as API - Enriched user experience
Key takeways to audience
Audience will get an exposure:
- How to create your own type manager by overloading python type constructs
- Exposure to alternative approach to creating APIs for data-heavy systems & explore benefits
- Learn how type manager simplifies your life as well as the life of your consumers.
- Secrets of the python inbuilt __ operators - and how you can leverage them to provide native type experience even for your own custom classes
- How you can create a better user experience for customers in a simple way
- How you can incorporate Object Oriented SOLID principles
- General familiarity with type concepts (fields, arrays, classes, enums) is needed
- Exposure to in-built operators like (getattr etc. will help)
- Exposure to Systems Management would be useful.
Vaideeswaran Ganesan, Senior Principal Technologist @ Dell EMC, has special focus in building systems management products for servers, networking and data center products. His passion is compiler design, analytics, systems management, networking protocols and automation.
Ajaya Senapati, Principal Technologist @ Dell EMC, has special focus in building systems management products for servers, networking and storage products.