Internet Protocol Libraries - Python

Let us look at the some of Python Standard Library modules which formulate and make available various Internet Protocol and help us build good Internet Applications.

There are 3 versions of this presentation:

• S5 presentation
• Plain HTML handout
• reStructuredText source

©2010, licensed under a Creative Commons Attribution/Share-Alike (BY-SA) license.

About Me

• Python Core Developer.
• Maintainer of urllib2 and related modules.
• Software Engineer at Akamai, India.

Status of Python

• Python 3.2a2+ (This is the trunk)
• Python 3.1.2 (release31-maint)
• Python 2.7 (release27-maint)

Unsupported versions:

• Python 3.0 is not there!
• Python 2.6.5 (release26-maint)

http://www.python.org/dev/

Internet Protocols and Support

• Most of the modules are built on top of socket module.
• They are highlevel modules implmenting Specifications mostly from RFC.
• Provide commonly used helper functions.

Applications which are using these

• Mercurial
• Google Code (and google-cl)
• youtube-dl
• Bittorrent
• Mailman

Modules in 2 and 3

• webbrowser
• cgi
• wsgiref
• uuid

Modules in 2 and 3

• less commonly used protocol modules.
• ftplib
• poplib
• imaplib
• smtplib
• telnetlib

urllib package in python3

• urllib -> urllib.request, urllib.response, urllib.parse
• urllib2 -> urllib.request, urllib.response, urllib.parse
• urlparse -> urllib.parse

http://www.python.org/dev/peps/pep-3108/

http package in python3

• httplib -> http.client
• cookiekib -> http.cookiejar
• Cookie -> http.cookies
• BaseHTTPServer -> http.server
• SimpleHTTPServer -> http.server
• CGIHTTPServer -> http.server
• SocketServer -> socketserver

xmlrpc package in python3

• xmlrpclib -> xmlrpc.client
• SimpleXMLRPCServer -> xmlrpc.server
• DocXMLRPCServer -> xmlrpc.server

webbrowser

• Module for Handling all different kinds of browsers

url = 'http://www.python.org/'
webbrowser.open_new_tab(url + 'doc/')
webbrowser.open_new(url)

cgi

• Support for Common Gateway Interface Scripts

print "Content-Type: text/html"
print
print "<h1>Hello,World</h1>"

• FieldStorage class for handling forms.

import cgitb
cgitb.enable()

wsgiref

• The Web Server Gateway Interface (WSGI)
• PEP 333
• Django, TurboGears, App Engine, all many different Web Frameworks.
• Standardization helps. You speak one language and follow similar protocols.
• wsgiref is the reference implementation of WSGI Specification.

wsgiref

from wsgiref.simple_server import make_server
def hello_world_app(environ, start_response):
    status = '200 OK'
    headers = [('Content-type', 'text/plain')]
    start_response(status, headers)
    return ["Hello World"]
httpd = make_server('', 8000, hello_world_app)
httpd.serve_forever()

uuid

• Universally Unique Identifier (RFC 4122)

>>> import uuid
>>> print uuid.uuid1()
32fb97c6-c8d3-11df-a47f-00216bad94a4
>>> print uuid.uuid4()
f9e0a0cd-23dd-46c8-95d6-e0bb0198c121

urllib

• Higher Level Interface for Fetching Data from the Web.
• Clients/ Client libraries which handle URLs.
• urllib.urlopen

urllib2

• Next Version of urllib.
• Create a Request Object.
• Have different Handlers depending upon the Protocol.
• Get the Response by the corresponding Handler object.

urlparse

• Parse the URLs into its constituents.
• It is required when are building bigger applications.
• Referencing sections within pages.
• Parse IPv6 URLs too.

httplib

• It is the underlying layer to the urllib.
• It implements HTTP/1.1 spec.
• Implements the HTTP States on top of socket.

ftplib

• Defines the FTP class and related items. The FTP class implments the client side FTP protocol.
• Internally used by urllib module for handling ftp urls.
• Got support FTP TLS from Python 2.7 onwards.

SocketServer

• Handling synchronous network requests.
• Helper classes for Mixins.

+------------+
| BaseServer |
+------------+
      |
      v
+-----------+        +------------------+
| TCPServer |------->| UnixStreamServer |
+-----------+        +------------------+
      |
      v
+-----------+        +--------------------+
| UDPServer |------->| UnixDatagramServer |
+-----------+        +--------------------+

Other Servers

• BaseHTTPServer
• SimpleHTTPServer
• CGIHTTPServer

python -m SimpleHTTPServer 8000

xmlrpclib

• XML-RPC client code; it handles all the details of translating between conformable Python objects and XML on the wire.

import xmlrpclib
proxy = xmlrpclib.ServerProxy("http://localhost:8000/")
print "3 is even: %s" % str(proxy.is_even(3))
print "100 is even: %s" % str(proxy.is_even(100))

SimpleXMLRPCServer

import xmlrpclib
from SimpleXMLRPCServer import SimpleXMLRPCServer

def is_even(n):
    return n%2 == 0

server = SimpleXMLRPCServer(("localhost", 8000))
print "Listening on port 8000..."
server.register_function(is_even, "is_even")
server.serve_forever()

Good Resources

• http://docs.python.org/py3k/
• http://diveintopython3.org/
• http://www.python.org/dev/peps/pep-3108/
• http://stackoverflow.com/questions/1453952/most-useful-python-modules-from-the-standard-library

Thank you. :)

Senthil Kumaran <orsenthil@gmail.com>