Python Notes

Monday, January 24, 2005

Storing persistent classes with SQLObject

I've been playing with extensions to store persistent classes and custom methods using SQLObject. The need arised as I was trying to write a relational representation of the Petri net model for a business framework. The standard model boasts few basic concepts (basically speaking, transitions, places, tokens and arcs). The relationship between the entities is simple, and easy to map with SQLObject. But the details are harder to model correctly, specially when transitions are concerned.

In a Petri net, a transition is the entity that tells what happens when one action is executed. In a workflow model it represents the status changes as the work is done. In a object-oriented system, the obvious way to do it is to represent each transition as a custom descendant of the base Transition class.

At this point things get complicated. A real-life workflow application has typically hundreds, or even thousands, of transitions. Implementing each transition as a standard Python class turns out to be a problem. It makes customization harder; simple customizations may require a server restart to have any effect. Python modules are known to be hard to reload properly (even for Zope wizards). What is really required is a solution to have persistent classes: class definitions that can be saved and loaded from the same database which holds the relational representation of the Petri Net model. Note the difference; it's not a persistent instance, so pickle, or even more advanced solutions as the ZODB do not apply here. But if we want to store it in a database, we really want to have only one Transition table. Having one table for each custom transition makes things way too complicated to handle.

The simplest solution is to store the class definition itself into the database. Code can be stored into a string, and later read and executed on demand. It's important to make sure that class instances generated this way are short lived, to avoid problems with obsolete instances in memory. For a web application, this can be achieved by working with instances that are valid only for one request and discarded later. For example (untested!)

class Transition(SQLObject):
name = StringCol()
transition_class = StringCol()

def _init(self, *args, **kw):
SQLObject._init(self, *args, **kw)
exec self.transition_class
self.transition = locals()[self.name]()
...
t = Transition(
name='ChargeCreditCard',
transition_class='class ChargeCreditCard(CustomTransition): pass'
)
...
t.transition.execute()

While this solution works, it involves an intermediate object. Another solution is to dynamically attach custom methods to the transition instances. The idea is to implement a special MethodCol column that would store the code for a single method in the database, either as text or in compiled format. On read, the MethodCol automatically executes the function definition and binds it to the instance, as if it was a method of the class. For example:

class Transition(SQLObject):
name = StringCol()
execute = MethodCol()
...
t = Transition(
name='ChargeCreditCard',
execute='def execute(self, fromPlace, toPlace): pass'
)
...
t.execute(fromPlace, toPlace)

To work as presented above, MethodCol needs to do a few tricks. On get, it must return an special object: a callable, which calls the custom method, but that on repr returns the source code of the object (which is necessary to allow customization using a web interface). On set, it would take the source code.

Right now, I don't know which approach is better. I'm willing towards the MethodCol solution, but I don't really know if it's going to work in practice. There are also some questions about security; however, any customizable system is subject to security problems anyway. The code in the database should only be modified by someone with the proper qualifications and credentials; the same individual could just as easily make a much greater damage with access to the file system to modify the system's code.

82 Comments:

Post a Comment

<< Home