Patterns for Twisted

Author: Dave Kuhlman
Address:
dkuhlman@rexx.com
http://www.rexx.com/~dkuhlman
Revision: 1.0c
Date: Sept. 22, 2003
Copyright: Copyright (c) 2003 Dave Kuhlman. This documentation is under The Artistic License: http://www.opensource.org/licenses/artistic-license.

Abstract

This document describes and discusses several design patterns for the development of Twisted Web applications.

Contents

1   Introduction

This document contains notes and examples that describe several programming patterns for Twisted, including patterns for deferred operations, for database access, for a client of a text search engine, and for an XML-RPC client.

I've called these "design patterns", although they are also "implementation patterns" in an attempt to suggest that they are a bit more abstract. My intent is to encourage you to use these patterns both in the design of your Twisted Web application as well as in implementing it (possibly by copying and editing the examples providing).

These patterns may seem to have a REST (REpresentational State Transfer) cast, and that is intentional. My aims are to help those attempting to treat the Web as a resource, in contrast to dealing with the Web as an extended computer using some style of RPC (e.g. XML-RPC or SOAP). However, I'm also hoping that the patterns described in this document are generic enough so that they will be helpful in other styles of Web application as well.

I've read the Twisted documentation on deferreds as closely as I could, but I do not feel that I am experienced enough to have good intuitions here, so any suggestions and corrections would be appreciated.

2   Application Goals

Here are some goals for applications developed with the use of these patterns:

3   The Design Patterns

These are the patterns that we describe in this document. We hope to also provide some support for implementing with these patterns.

We are interested in the following development patterns, which are described in more detail below:

4   The Dispatcher Pattern

This pattern takes the URL and translates it into a call to the Python method intended to implement the request. It does this by inspecting the path on the URL.

4.1   Example

4.1.1   The brute-force pattern

Here is an example of a class that dispatches requests:

class ResourceDispatcher(resource.Resource):

    isLeaf = True

    def render(self, request):
        done = False
        if len(request.postpath) == 1 and request.postpath[0]:
            if request.postpath[0] == 'show_plants':
                dbaccess = dbmod.DBAccess(registry)
                dbaccess.show_plants(request)
                done = True
            elif request.postpath[0] == 'show_plants_by_rating':
                dbaccess = dbmod.DBAccess(registry)
                dbaccess.show_plants_by_rating(request)
                done = True
            elif request.postpath[0] == 'get_add_form':
                dbaccess = dbmod.DBAccess(registry)
                dbaccess.show_add_form(request)
                done = True
            elif request.postpath[0] == 'get_delete_form':
                dbaccess = dbmod.DBAccess(registry)
                dbaccess.show_delete_form(request)
                done = True
            elif request.postpath[0] == 'add_plant':
                dbaccess = dbmod.DBAccess(registry)
                dbaccess.add_plant(request)
                done = True
            elif request.postpath[0] == 'delete_plant':
                dbaccess = dbmod.DBAccess(registry)
                dbaccess.delete_plant(request)
                done = True
        if done:
            return NOT_DONE_YET
        else:
            content = Content_Dispatch % (
                request.postpath,
                request.args,
                time.ctime(),
                )
            request.write(content)
            request.finish()

Comments and explanation:

  • Setting the class variable isLeaf to True tells Twisted not to perform further processing on the URL path.
  • The list request.postpath contains un-digested parts of the URL path.

4.1.2   The name-prefix pattern

For some purposes, this pattern is a bit simpler, although it may not give you as much control over individual cases:

class ResourceDispatcher(resource.Resource):
    isLeaf = True
    def render(self, request):
        if len(request.postpath) == 1 and request.postpath[0]:
            name = request.postpath[0]
            dbaccess = dbmod.DBAccess(registry)
            try:
                method = getattr(dbaccess, 'webrequest_' + name)
            except AttributeError:
                pass
            else:
                method(request)
                return NOT_DONE_YET
        content = Content_Dispatch % (
            request.postpath,
            request.args,
            time.ctime(),
            )
        request.write(content)
        request.finish()

Explanation:

  • This dispatcher retrieves an attribute (the method) from a class given a request name on the right end of the URL.
  • If your implementation is spread across several modules and/or classes, then you might still have to do a bit of if-else stuff. Or, you might consider using a dictionary that maps names to modules and classes.

5   The Database Update Pattern

This pattern uses Twisted deferred objects to request database operations. Here is a description:

5.1   Strategy

  • Separate the deferring operations (for example, requests to the PostgreSQL database) from the code that calls them and that also contains application logic.
  • Execute operations in sequence. Operation n+1 must not begin operation n until is complete. Implement this by chaining callbacks.
  • Provide simple Web page and form definitions. Enable editing and delivery of simple HTML documents.

5.2   Example

5.2.1   The application logic class

And, here is the resource class that requests the slow operations and provides a callback to be called when the slow operation is complete. This class also provides application logic. For each operation that requires multiple sequential operations, it provides multiple methods that each calls a database operation and provides a callback (except, of course, for the last step):

class DBAccess:
    def __init__(self, registry):
        self.registry = registry
        get_globals(registry)

    o
    o
    o

    #
    # Delete a row.
    #
    def delete_plant(self, request):
        self.request = request
        self.db = PlantDatabase("pyPgSQL.PgSQL", CONNECTION_ARGS)
        self.content = Content_ShowPlants
        self.plant_name = request.args.get('plant_name')
        if self.plant_name and len(self.plant_name) > 0:
            self.plant_name = self.plant_name[0]
        self.plant_desc = request.args.get('plant_desc')
        if self.plant_desc and len(self.plant_desc) > 0:
            self.plant_desc = self.plant_desc[0]
        self.plant_rating = request.args.get('plant_rating')
        if self.plant_rating and len(self.plant_rating) > 0:
            self.plant_rating = self.plant_rating[0]
        self.delete_plant_step_1()
        return NOT_DONE_YET

    def delete_plant_step_1(self):
        self.db.checkPlant(self.plant_name).addCallbacks(
            self.delete_plant_step_2,
            self.db.operationError)

    def delete_plant_step_2(self, resultlist):
        if resultlist:
            deferred = self.db.deletePlant(
                self.plant_name)
            deferred.addCallbacks(self.delete_plant_step_3,
                self.db.operationError)
        else:
            self.content = Content_Missing
            self.content = self.content % self.plant_name
            self.request.write(self.content)
            self.request.finish()
            

    def delete_plant_step_3(self, resultlist):
        self.db.getPlants().addCallbacks(self.gotPlants,
            self.db.operationError)

    #
    # Show the plants in the database.
    #
    def gotPlants(self, resultlist):
        plantlist = []
        plantlist.append('<table border="5" width="80%">')
        plantlist.append('<caption>Plant DB</caption>')
        plantlist.append('<tr><th width="25%">Name</th><th width="65%">Description</th><th width="10%">Rating</th></tr>')
        for plant in resultlist:
            plantlist.append('<tr>')
            s1 = '<td>%s</td><td>%s</td><td align="right">%s</td>' % \
                (plant[0], plant[1], plant[2])
            plantlist.append(s1)
            plantlist.append('</tr>')
        plantlist.append('</table>')
        plantstr = '\n'.join(plantlist)
        content = self.content % (self.request.args, plantstr)
        self.request.write(content)
        self.request.finish()

The following second pattern uses a single method implemented as a state machine, rather than using separate methods for each step (as in the previous example). Whereas the previous example, deleted a plant, this example adds a plant if it is not already in the database, or updates the plant if it is in the database:

STEP_1 = 1
STEP_2 = 2
STEP_3 = 3

class DBAccess:

    o
    o
    o

    #
    # Add or update a row.
    # This uses a state machine pattern.
    #
    def add_plant(self, request):
        self.request = request
        self.state = STEP_1
        self.add_plant_machine()

    def add_plant_machine(self, *args):
        if self.state == STEP_1:
            self.db = PlantDatabase("pyPgSQL.PgSQL", CONNECTION_ARGS)
            self.content = Content_ShowPlants
            self.plant_name = self.request.args.get('plant_name')
            if self.plant_name and len(self.plant_name) > 0:
                self.plant_name = self.plant_name[0]
            self.plant_desc = self.request.args.get('plant_desc')
            if self.plant_desc and len(self.plant_desc) > 0:
                self.plant_desc = self.plant_desc[0]
            self.plant_rating = self.request.args.get('plant_rating')
            if self.plant_rating and len(self.plant_rating) > 0:
                self.plant_rating = self.plant_rating[0]
            self.state = STEP_2
            self.db.checkPlant(self.plant_name).addCallbacks(
                self.add_plant_machine,
                self.db.operationError)
            return NOT_DONE_YET
        elif self.state == STEP_2:
            resultlist = args[0]
            # Is the plant already in the database?
            if resultlist:
                self.state = STEP_3
                deferred = self.db.updatePlant(
                    self.plant_name, self.plant_desc, self.plant_rating)
                deferred.addCallbacks(self.add_plant_machine,
                    self.db.operationError)
            else:
                self.state = STEP_3
                deferred = self.db.addPlant(
                    self.plant_name, self.plant_desc, self.plant_rating)
                deferred.addCallbacks(self.add_plant_machine,
                    self.db.operationError)
            return NOT_DONE_YET
        elif self.state == STEP_3:
            self.db.getPlants().addCallbacks(self.gotPlants,
                self.db.operationError)
            return NOT_DONE_YET

Explanation:

  • This state machine is sequential: it executes step 1, 2, and 3 in order.
  • Each state is implemented by a clause in the if-elif statement.
  • Each state sets the variable self.state to indicate the next state to be executed.
  • Each state sets this method itself as the next callback.

Comparison and analysis:

  • The difference between the above two patterns is that the first uses separate methods for each step whereas the second pattern uses a single method with separate states (each implemented by a clause in an if-elif statement.
  • Once you learn to recognize each pattern and its structure, either is understandable and usable. The choice between them may be a personal preference.
  • A good recommendation is to pick one of the above two patterns and to use it consistently. What you want is code that is recognizable and clear, and that you can understand and debug.

And, here is yet one more pattern for the same task. This one attempts to simplify things by replacing the if-elif-... statement. It implements each state as a separate method, and provides a dispatcher to these methods:

class DBAccess:

    o
    o
    o

    #
    # Add or update a row.
    # This method uses a state machine pattern.
    # It also has a dispatcher that retrieves and calls the
    #   method that implements the current state.
    #
    def add_plant(self, request):
        self.request = request
        self.state = 'step_1'
        self.add_plant_machine()

    #
    # This method is the dispatcher.
    #
    def add_plant_machine(self, *args):
        method = getattr(self, 'add_plant_machine_' + self.state)
        method(*args)
    
    def add_plant_machine_step_1(self, *args):
        self.db = PlantDatabase("pyPgSQL.PgSQL", CONNECTION_ARGS)
        self.content = Content_ShowPlants
        self.plant_name = self.request.args.get('plant_name')
        if self.plant_name and len(self.plant_name) > 0:
            self.plant_name = self.plant_name[0]
        self.plant_desc = self.request.args.get('plant_desc')
        if self.plant_desc and len(self.plant_desc) > 0:
            self.plant_desc = self.plant_desc[0]
        self.plant_rating = self.request.args.get('plant_rating')
        if self.plant_rating and len(self.plant_rating) > 0:
            self.plant_rating = self.plant_rating[0]
        self.state = 'step_2'
        self.db.checkPlant(self.plant_name).addCallbacks(
            self.add_plant_machine,
            self.db.operationError)
        return NOT_DONE_YET

    def add_plant_machine_step_2(self, *args):
        resultlist = args[0]
        if resultlist:
            self.state = 'step_3'
            deferred = self.db.updatePlant(
                self.plant_name, self.plant_desc, self.plant_rating)
            deferred.addCallbacks(self.add_plant_machine,
                self.db.operationError)
        else:
            self.state = 'step_3'
            deferred = self.db.addPlant(
                self.plant_name, self.plant_desc, self.plant_rating)
            deferred.addCallbacks(self.add_plant_machine,
                self.db.operationError)
        return NOT_DONE_YET

    def add_plant_machine_step_3(self, *args):
        self.db.getPlants().addCallbacks(self.gotPlants,
            self.db.operationError)
        return NOT_DONE_YET

Explanation:

  • Note the method add_plant_machine. It acts as a dispatcher for our state machine. It takes the current state (a string: "step_1", "step_2, "step_3"), concatenates it to "add_plant_machine_", then retrieves and calls the method by that name.
  • Each method that implements a state (except the last), sets self.state to the next state, and adds add_plant_machine as a callback.

5.2.2   Repository class

Here is an example of the repository access class or database class -- Each method implements a single operation that may cause a delay, then returns a deferred:

class PlantDatabase(adbapi.ConnectionPool): 
    """Update and retrieve from the Plant_DB database.
    """ 

    def checkPlant(self, name):
        deferred = self.runQuery("select * from Plant_DB where p_name = '%s'" % name)
        return deferred

    def updatePlant(self, plant_name, plant_desc, plant_rating):
        sql = "update Plant_DB set p_desc='%s', p_rating='%s' where p_name='%s'" % \
            (plant_desc, plant_rating, plant_name)
        deferred = self.runOperation(sql)
        return deferred

    def addPlant(self, plant_name, plant_desc, plant_rating):
        sql = "insert into Plant_DB values ('%s', '%s', '%s')" % \
            (plant_name, plant_desc, plant_rating)
        deferred = self.runOperation(sql)
        return deferred

    def deletePlant(self, plant_name):
        sql = "delete from Plant_DB where p_name='%s'" % \
            (plant_name,)
        deferred = self.runOperation(sql)
        return deferred

    def getPlants(self): 
        sql = "SELECT * FROM Plant_DB order by p_name"
        deferred = self.runQuery(sql) 
        return deferred

    def operationError(self, error):
        log.msg("%s Operation Failed: %s" % (reflect.qual(self.__class__), error))
        log.err(error)

Comments and explanation:

  • There is one function in this class for each type of request. In our example these are check for existence, update, add, delete, and get all.
  • Each of these operations is a "slow" operation. To prevent blocking, it requests the operation and immediately returns a deferred. (Note: We will learn how to implement a slow operation without blocking the in the text repository pattern.
  • By inheriting from twisted.enterprise.adbapi.ConnectionPool, we get get the ability to run database queries that use the connection pool. In order to do so, we pass to the constructor for class PlantDatabase the name of the database API and the connection args. See the application logic class.

5.2.3   Connection pooling

This section provides an example that shows both:

  1. How to share values across sessions and
  2. How to create and uses a database connection pool.

ConnectionPool saves the time needed to create a database connection for each session.

Here is code that shares a single instance of the database connection pool across all sessions:

class Globals:
    def __init__(self):
        self.dbpool = None
    def getDbpool(self):
        return self.dbpool
    def setDbpool(self, dbpool):
        self.dbpool = dbpool

def get_globals(registry):
    globalContainer = registry.getComponent(globalvalues.Globals)
    if not globalContainer:
        globalContainer = globalvalues.Globals()
        dbpool = PlantDatabase("pyPgSQL.PgSQL",
            "localhost:5432:test:postgres:flicker")
        globalContainer.setDbpool(dbpool)
        registry.setComponent(globalvalues.Globals, globalContainer)
    else:
        dbpool = globalContainer.getDbpool()
    return dbpool

Explanation:

  • Class Globals is our container for global values. In this example, the only value stored in it is the database connection pool.
  • The function get_globals attempts to get the existing, shared globals container. If the globals container exists, it returns the DB connection pool (an instance of PlantDatabase) from that container. If it does not exist, it creates a container and a DB connection pool, puts the DB connection pool in the container, saves it, and returns the new DB connection pool.
  • In order to use the database connection pool, we create an instance of a class that subclasses twisted.enterprise.adbapi.ConnectionPool (in our example code this class is PlantDatabase) passing to the constructor the database connection parameters. The query functions provided by that superclass know how to use the database connection pool. See the "Repository class" example.

5.3   Questions and discussion

Here are a few questions:

  • We've presented two patterns for using deferreds, specifically (1) chaining separate methods and (2) a state machine in a single method. Which is better? Is there an approved or commonly accepted Twisted pattern for using deferreds? Putting multiple steps in a single method seems to avoid some code clutter at the expense of code density.
  • I've separated the slow operations into a separate class. Is that a good idea? In my example, the operations are very simple, in part because I've kept the logic in the resource class. Perhaps, instead of putting them in a separate class, I should just code them in-line in the resource class in the state machine method. One reason for separate methods is to make them reusable.
  • The state-machine-pattern returns multiple times. It returns NOT_DONE_YET each time. Is that the correct thing to do?
  • Twisted already has an API that creates and returns deferreds for database operations. What if I need to wrap some other long lasting operation up in the same way? How do I do that? Answer: See The Text Repository Pattern.

6   The Text Repository Pattern

Assumptions:

Requirements:

Definitions:

Here is a description of the pattern:

The implementation -- Here is a conceptual view of the implementation pattern:

You may want to compare the structure of the text repository pattern with that of the database update pattern. I'm trying to re-work these patterns so that they are as isomorphic (structurally similar) as possible. Hopefully, that will make knowledge and understanding of one pattern transferable to the other.

6.1   Example

6.1.1   Application logic class

Here is an example of the application logic class:

#
# Application logic class
#
class GlimpseTextRepositoryAccess:

    #
    # Produce a form to be used to search the text repository.
    #
    def show_search_form(self, request):
        self.request = request
        self.content = Content_SearchForm
        self.request.write(self.content)
        self.request.finish()

    #
    # Produce a list of the search results.
    #
    def show_search_results(self, request):
        self.request = request
        self.repository = GlimpseTextRepository()
        self.content = Content_ShowSearchResults
        self.querystr = request.args.get('querystr')
        if self.querystr and len(self.querystr) > 0:
            self.querystr = self.querystr[0]
        self.casesensitive = request.args.get('casesensitive')
        if self.casesensitive and len(self.casesensitive) > 0:
            self.casesensitive = self.casesensitive[0]
        self.state = STEP_1
        self.show_search_results_machine()
        return NOT_DONE_YET

    def show_search_results_machine(self, *args, **kw):
        if self.state == STEP_1:
            #dbglogmsg('*** (show_search_results Step_1)')
            self.repository.search_repository(
                self.show_search_results_machine,
                self.querystr,
                self.casesensitive)
            self.state = STEP_2
        elif self.state == STEP_2:
            resultlist = args[0]
            formatter = Formatter()
            content = formatter.format_search_results(self.content, resultlist)
            self.request.write(content)
            self.request.finish()

    #
    # Fetch and show one file from the repository.
    #
    def get_text_file(self, request, path):
        self.request = request
        infile = file(path, 'r')
        filecontent = infile.read()
        infile.close()
        formatter = Formatter()
        content = formatter.format_file(filecontent, path)
        self.request.write(content)
        self.request.finish()

Explanation:

  • Method show_search_form provides a form that can be used to request a query.
  • Method show_search_results_machine implements a small FSM (finite state machine). This FSM contains two steps (states) which (1) get the deferred that starts the query and (2) format the results for delivery to the client. The variable self.state is used to keep track of state and to transfer from one state to another.
  • In step (state) 1, method show_search_results_machine (1) calls the search_repository method in the repository class, (2) adds itself as a callback, and (3) sets the (next) state to STEP_2 so that when the callback is called, step 2 will be executed.
  • In step (state) 2, which is executed when the method is called as a callback, method show_search_results_machine formats the results and delivers them to the client.
  • Method get_text_file responds to a request for one of the files listed in the search results. The right-end of the URL contains the path to the requested file. Note that the list of search results contains the URLs needed to make these requests. In effect, this enables the user to "drill down" into one of the results (a file) returned by the text search engine.

6.1.2   Repository class

Here is an example of the repository access class:

#
# Repository class
#
class GlimpseTextRepository:

    def search_repository(self, callback, querystr, casesensitive): 
        flags = ''
        if not casesensitive:
            flags = '-i'
        port = 2001
        queryobj = glimpselib.Query()
        deferred = queryobj.query_server(querystr, flags, port)
        deferred.addCallback(callback)
        return deferred

Explanation:

  • Method search_repository calls the method in the Glimpse to initiate the search and create a deferred object. It then adds a callback to the deferred.
  • This class is intended a "insulation" from the Glimpse library that actually does the work. It would, hopefully, enable user to easily switch to another implementation of text searching.

6.1.3   The Glimpse library

This example from the Glimpse library directs its requests to the Glimpse text search engine:

#
# This version uses glimpseserver.
# Doing so saves loading the index into memory for each query.
# It uses twisted.utils.getProcessOutput() to run the query, 
#   captures the results, then creates a list of tuples to return
#   to the callback.
#
class Query:
    def query_server(self, querystr, flags='-i', port=2001):
        executable = '/usr/local/bin/glimpse'
        args = ['-C', '-K', '%d' % port]
        if flags:
            args.append(flags)
        if querystr:
            args.append(querystr)
        self.deferred = utils.getProcessOutput(executable, args)
        self.deferred.addCallbacks(self._query_server_2, self.error_back)
        return self.deferred

    def _query_server_2(self, result):
        lines = result.split('\n')
        result = []
        for line in lines:
            pos = line.find(':')
            if pos > -1:
                val = (line[:pos], line[pos+2:])
                result.append(val)
        return result
    
    def error_back(self, result):
        return [('***error***', result.getErrorMessage()),]

Explanation:

  • The method query_server uses twisted.utils.getProcessOutput() to run the Glimpse search engine (as if through the command line) and capture the results. (Note to self: We really should wrap Glimpse as a Python extension module.)
  • The -C flag tells Glimpse to use the Glimpse server to perform the search. Doing so saves time that would have been used to load the search engine's index for each search.
  • getProcessOutput() returns a deferred. We add _query_server_2 as a callback to that deferred. getProcessOutput() returns the output of the executable (in our case glimpse) as a string to the callback.
  • The callback _query_server_2 receives the results of the query, then creates and returns a list of tuples, one tuple for each match. Each tuple has (1) the file name of the match and (2) the line containing the match.

6.1.4   Representation formatting class

And, finally, when the results have been returned, the formatting class is used to produce a representation of the results. Putting this formatting code in a separate class has at least two benefits:

  • In a project of any size, there may be specialists working on the user interface, the formatting of representations, etc. Separating formatting code would provide a way to provide a separate set of modules for those specialists to work on.

  • It might be useful to do formatting with the Quixote PTL (Python templating language), which is also available in Twisted. By putting formatting code in a separate class, we would be able to put formatting code in .ptl modules (for example) for processing with PTL, then instructing Twisted to process them as resource templates with something like the following:

    $ mktap web --path=/var/www \
          --processor=.rtl=twisted.web.script.ResourceTemplate
    

Our example, however, uses simple Python code to perform formatting. Here is an example of the representation formatting class:

#
# Representation Formatting class
#
class Formatter:

    def format_search_results(self, pagecontent, resultlist):
        contentlist = []
        contentlist.append('<table border="1">')
        contentlist.append('<tr><th>File</th><th>Match</th></tr>')
        idx = 0
        prevfilename = ''
        for result in resultlist:
            filename = result[0]
            line = result[1].replace('<', '&lt;')
            if filename != prevfilename:
                idx = 0
            target = '<a href="/dispatch.rpy/get_text_file/%d%s">%s</a>' % \
                (idx, filename, filename)
            linecontent = '<tr><td>%s</td><td>%s</td></tr>' % \
                (target, line)
            contentlist.append(linecontent)
            idx += 1
            prevfilename = filename
        contentlist.append('</table>')
        contentstr = '\n'.join(contentlist)
        contentstr = pagecontent % contentstr
        return contentstr

    def format_file(self, incontent, path):
        stem, ext = os.path.splitext(path)
        if ext == '.html':
            return incontent
        else:
            content1 = incontent.replace('<', '&lt;')
            content2 = TextWrapper % (path, path, content1)
            return content2

Explanation:

  • Method format_search_results formats a list of search results, one for each hit. Note that it generates URLs that can be used to request the file in which the pattern is found.
  • Method format_file formats one file requested through a URL generated by format_search_results. If the file contains HTML, the file's contents are returned unchanged. If the file does not contain HTML, then left-corner-brackets are replaced and a bit of HTML header and footer is wrapped around the content.

6.2   Questions and discussion

And, a few questions:

  • How slow does a slow method have to be before it is worth putting it into a deferred? After all, the purpose of using a search engine such as Glimpse is to speed up searches. Perhaps using deferred methods is over-kill. Is there any down-side or cost or over-head to using deferreds?

7   The XML-RPC Pattern

This simple pattern can be used to make remote calls to an XML-RPC server and then to deliver content that contains the results. The Twisted Web application is acting as an XML-RPC client . The Web application uses XML-RPC to request information that it will return in its responses.

Requirements:

Definitions:

7.1   Example

7.1.1   The application logic class

This class contains any application logic. A typical example would be the need to make more than one XML-RPC request, but to include logic between the requests so that one request is dependent on the results of a previous request.

Here is an example:

#
# XML-RPC applicationLogic class
#
class TemperatureAccess:

    #
    # Produce a form to be used to search the text repository.
    #
    def show_temperature_form(self, request):
        self.request = request
        self.content = Content_ConvertTemperatureForm
        self.request.write(self.content)
        self.request.finish()
        return NOT_DONE_YET

    #
    # Convert the temperature and produce the result.
    #
    def convert_temperature(self, request, direction):
        self.request = request
        intemp = None
        arg = request.args.get('intemp')
        if arg and len(arg) > 0:
            intemp = arg[0]
        self.intemp = intemp
        self.converter = Conversion()
        self.direction = direction
        if direction == 'c2f':
            self.content = Content_ConvertC2FResults
            deferred = self.converter.c2f(intemp)
            deferred.addCallbacks(self.receiveTemperature, self.receiveError)
            return NOT_DONE_YET
        elif direction == 'f2c':
            self.content = Content_ConvertF2CResults
            deferred = self.converter.f2c(intemp)
            deferred.addCallbacks(self.receiveTemperature, self.receiveError)
        return NOT_DONE_YET
        
    def receiveTemperature(self, value):
        valuestr = '%.2f' % value
        self.outtemp = valuestr
        if self.direction == 'c2f':
            scale = 'F'
        else:
            scale = 'C'
        deferred = self.converter.category(scale, value)
        deferred.addCallbacks(self.receiveCategory, self.receiveError)
        return NOT_DONE_YET

    def receiveCategory(self, category):
        self.category = category
        deferred = self.converter.ranges()
        deferred.addCallbacks(self.receiveRanges, self.receiveError)
        return NOT_DONE_YET
        
    def receiveRanges(self, ranges):
        self.ranges = ranges
        rangelist = ['<ul>']
        for range in ranges:
            rangelist.append('<li>Less than %0.1f C is %s</li>' % \
                (range[0], range[1]))
        rangelist.append('</ul>')
        rangestr = '\n'.join(rangelist)
        content = self.content % \
            (self.intemp, self.outtemp, self.category, rangestr)
        self.request.write(content)
        self.request.finish()
        return NOT_DONE_YET
        
    def receiveError(self, error):
        raise RuntimeError, error

Explanation:

  • Method show_temperature_form delivers a form that enables the client to provide a temperature and request a temperature conversion.
  • Method convert_temperature uses the XML-RPC class. It (1) creates and instance of the XML-RPC class, (2) calls the conversion method in that class, which returns a deferred, then (3) adds a callback function (receiveTemperature) to that deferred.
  • Method receiveTemperature is called when the first XML-RPC request has been completed. It also uses the XML-RPC class. It (1) saves the converted temperature, (2) calls the category method in the XML-RPC class, which returns a deferred, then (3) adds a callback function (receiveCategory) to this (new) deferred.
  • Method receiveCategory (1) receives the next result (the category) and (2) saves that value in an instance variable (self.category). This method also (3) calls the ranges method in the XML-RPC class, which returns a deferred, then (4) adds a callback function (receiveRanges) to this (new) deferred.
  • Method receiveRanges (1) receives the final result (the category), (2) formats the content, and (3) returns it to the client.
  • The methods described above show how to chain methods together when each method in a sequence is dependent on the results returned by the previous method. We could have alternatively used a single method (in place of receiveTemperature, receiveCategory, and receiveRanges) which implemented a finite state machine with separate states (for receiveTemperature, receiveCategory, and receiveRanges).
  • Method receiveError reports any errors raised by the XML-RPC server.

7.1.2   The XML-RPC requester class

This class makes the XML-RPC requests. It acts as the XML-RPC client. It uses Twisted's XML-RPC support, which returns a deferred object.

Here is a sample class:

from twisted.web.xmlrpc import Proxy

#
# XML-RPC back-end resource class.
# This class makes the XML-RPC requests and returns deferreds.
#
class Conversion:
    def c2f(self, intempstr):
        intemp = -1
        try:
            intemp = float(intempstr)
        except:
            pass
        proxy = Proxy('http://localhost:8081')
        deferred = proxy.callRemote('c2f', intemp)
        return deferred

    def f2c(self, intempstr):
        intemp = -1
        try:
            intemp = float(intempstr)
        except:
            pass
        proxy = Proxy('http://localhost:8081')
        deferred = proxy.callRemote('f2c', intemp)
        return deferred

    def category(self, scale, intemp):
        proxy = Proxy('http://localhost:8081')
        deferred = proxy.callRemote('temperature_category', scale, intemp)
        return deferred

    def ranges(self):
        proxy = Proxy('http://localhost:8081')
        deferred = proxy.callRemote('temperature_ranges')
        return deferred

Explanation:

  • Each method in this class makes an XML-RPC request and returns a deferred.
  • These methods use Twisted's XML-RPC support (as opposed to using module xmlrpclib in the Python standard library, for example).

7.1.3   The XML-RPC server

This is the simple XML-RPC server written in Python that I used while testing my examples:

import SimpleXMLRPCServer

def c2f(ctemp):
    #ftemp = ((212.0 - 32.0)/100.0 * ctemp) + 32.0
    ftemp = ((9.0 / 5.0) * ctemp) + 32
    return ftemp

def f2c(ftemp):
    #ctemp = 100.0/(212.0 - 32.0) * (ftemp - 32.0)
    ctemp = (5.0 / 9.0) * (ftemp - 32)
    return ctemp

Ranges = [
    (0.0, 'cold'),
    (20.0, 'cool'),
    (25.0, 'moderate'),
    (30.0, 'warm'),
    (35.0, 'hot'),
    (40.0, 'very hot'),
    (1000.0, 'extremely hot'),
    ]

def temperature_category(scale, intemp):
    if scale.upper() == 'F':
        temp = f2c(intemp)
    else:
        temp = intemp
    category = 'Unknown'
    for range in Ranges:
        if temp <= range[0]:
            category = range[1]
            break
    return category

def temperature_ranges():
    return Ranges

def startup():
    server = SimpleXMLRPCServer.SimpleXMLRPCServer(("localhost", 8081))
    server.register_function(c2f)
    server.register_function(f2c)
    server.register_function(temperature_category)
    server.register_function(temperature_ranges)
    server.serve_forever()

Explanation:

  • The functions c2f, f2c, temperature_category, and temperature_ranges are exported and are available to be called through XML-RPC requests.
  • Function startup creates a server, registers the exported functions, and starts the server.

7.1.4   An XML-RPC server implemented with Twisted

And here is the same XML-RPC server implemented as a Twisted applications:

from twisted.web import xmlrpc, server

Ranges = [
    (0.0, 'cold'),
    (20.0, 'cool'),
    (25.0, 'moderate'),
    (30.0, 'warm'),
    (35.0, 'hot'),
    (40.0, 'very hot'),
    (1000.0, 'extremely hot'),
    ]

class Converter(xmlrpc.XMLRPC):
    def xmlrpc_c2f(self, ctemp):
        #ftemp = ((212.0 - 32.0)/100.0 * ctemp) + 32.0
        ftemp = ((9.0 / 5.0) * ctemp) + 32
        return ftemp
    def xmlrpc_f2c(self, ftemp):
        #ctemp = 100.0/(212.0 - 32.0) * (ftemp - 32.0)
        ctemp = (5.0 / 9.0) * (ftemp - 32)
        return ctemp
    def xmlrpc_temperature_category(self, scale, intemp):
        if scale.upper() == 'F':
            temp = self.xmlrpc_f2c(intemp)
        else:
            temp = intemp
        category = 'Unknown'
        for range in Ranges:
            if temp <= range[0]:
                category = range[1]
                break
        return category
    def xmlrpc_temperature_ranges(self):
        return Ranges

def main():
    from twisted.internet.app import Application
    app = Application("xmlrpc")
    r = Converter()
    app.listenTCP(8081, server.Site(r))
    return app

application = main()

if __name__ == '__main__':
    application.run(save=0)

Explanation:

  • This example is basically a copy of the previous example with the functions encapsulated in a class.
  • To learn how to use Twisted to expose the methods in a class as XML-RPC services, read Creating XML-RPC Servers and Clients with Twisted. I followed the instructions and example in that document. It's simple enough so that I won't add much.
  • By default, methods prefixed with "xmlrpc_" are exposed to clients as XML-RPC services. See twisted.web.xmlrpc.py.

8   Miscellaneous Hints and Howto's

Here are a few clues and examples that were not covered in the patterns themselves.

8.1   Body data/content

In order to extract the data from the body of a request, use request.content. If there is content, then request.content will contain an instance of cStringIO.

Here is an example that shows how to use it:

data = ''
if request.content:
    data = request.content.getvalue()

8.2   Debugging

It is often helpful to produce some debugging information. This example shows how to print messages to a log file:

from twisted.python import log

# Set to 0 (debugging off) or 1 (debugging on).
_DEBUG = 0

def dbglogmsg(mesg):
    if _DEBUG:
        log.msg(mesg)

l Sharing values across sessions ------------------------------

Twisted provides a mechanism for sharing values across sessions. In order to do this, define a class, store the values you wish to share in an instance of the class, then request the shared values by asking Twisted for the shared value of that data type. The Connection pooling example above shows how to share values across sessions.

8.3   A test harness

You may want to be able to create a testing harness that acts as a client. It can be used to send requests to your Twisted Web application and to display or process the responses. If expanded, it can serve as a simple client that drives your application. You can use the following example as a starting point for the implementation of a Python client that talks to your Twisted Web application.

Here is the example:

import httplib, urllib

#
# Send a request to my Twisted Web application.
# Add a few arguments to the request and pack some content into
#   the body of the request.
# Get the response, display the status of the response, and
#   display the content received with the response.
#
def test():
    # Create some sample content to send as the body of the request.
    datalist = []
    datalist.append('<testcontent>')
    datalist.append('Here is stuff for a test.')
    datalist.append('</testcontent>')
    data = '\n'.join(datalist)
    # Create a few dummy parameters.
    paramdict = {'aaa': 111, 'bbb': 222}
    params = urllib.urlencode(paramdict)
    # Here are headers for the request.  Note the "Content-length".
    headers = {"Content-type": "application/x-www-form-urlencoded",
        "Accept": "text/html",
        "Content-length": len(data),
        }
    conn = httplib.HTTPConnection("localhost:8080")
    conn.set_debuglevel(1)
    selector = '/dispatch.rpy/?%s' % params
    # Pick one: GET or POST?
    conn.putrequest("GET", selector)
    ## conn.putrequest("POST", selector)
    for key, val in headers.iteritems():
        conn.putheader(key, val)
    conn.endheaders()
    conn.send(data)
    response = conn.getresponse()
    print 'status/reason', response.status, response.reason
    data1 = response.read()
    print '=' * 50
    print data1

Explanation:

  • Each call to function test makes one request to the Web application.
  • Local variables datalist/data are used to create content to be sent in the body of the request.
  • Note the commented lines that would enable you to choose between making a GET or a POST request.
  • This example prints out the response from the Web application. However, your own client implementation could process the response in other ways.

9   Summary and Conclusions

This document has tried to help Twisted developers of Web applications in two ways:

Thanks in advance for suggestions and guidance.

10   Acknowledgements

Thanks to Andrew Bennetts for many suggestions and criticisms on an earlier draft.

Thanks to the developers and supporters of Twisted for a powerful product.

11   See Also

Twisted: The Twisted network framework at TwistedMatrix

Python home: The Python Web site.

RestFaq: The REST FAQ -- Information and links for REpresentational State Transfer.

Glimpse: The Glimpse search engine Web site.

Docutils: Python Documentation Utilities -- This document was formatted with Docutils.