Data Encoding Formats: A Comparison of Text-Based and Binary Encoding
Priyadarshan Patil (~priyadarshan) |
Description:
Text-Based Encoding (translation of in-memory data representation to a byte sequence) formats are ubiquitous and are known to everyone namely JSON, XML, and CSV. JSON and XML formats are widely used for API and message-based communication but have poor support for running different versions of applications at the same time apart from their average performance in storage and communication. Alternative binary encoding formats exist namely Avro, Thrift, and Protobuf but there is little awareness about them. These formats provide superior support on application forward and backward compatibility requirements and improved performance on storage and communication. But they also come with their own set of overheads and are not always the best fit for all our application implementation use cases. The talk aims to sensitize techies on these different types of encoding formats, their data encoding strategies, the impact on the performance of storage, and data communication speeds and application compatibility issues. To validate the above points the results of benchmarking these encoding formats for speed and storage metrics using a Python Based Application Performance Monitoring tool - Locust would be shared. Armed with this knowledge choosing the right encoding formats in the context of the use cases would be easy
Prerequisites:
- JSON and XML Formats
- What Schemas are?
Content URLs:
Speaker Info:
Priyadarshan works as Solution Consultant at Sahaj.ai. He has 16 years of Software Engineering experience focussed on Tech Consulting and implementation and building strong technology teams. He has worked on Distributed systems, and traditional monolith platforms and built products on top of Eclipse Platform. He has diverse experience in Software Engineering - building tech platforms, delivery management, customer relationship management, and presales. At Sahaj, he is learning the art of building technology platforms with extreme programming practice and working in high-trust and High accountability technology teams. He is also helping Sahaj bootstrap its delivery center in Hyderabad.