Flow Based Programming & the new Python coding style

Since being introduced in the language, generators (and their lesser cousins, list comprehensions) are finally making it into the community mindset. In the long term, I think that this change will have a lasting effect on the way we write Python programs.

In the beginning, Python was a conventional scripting language that happened to have a exceptionally clean syntax and a solid object model. So most Python programs would resemble conventional sequential scripts. As the language evolved, the applications grew more complex, but their essence was still the same. The network-enabled modules pushed the envelope early on. Two architectures to handle multiple simultaneous requests were implemented, following (again) the conventional wisdom: async modules (based on the standard Posix select() call) and threaded execution.

Stackless Python was a interesting event in Python's history. It's still one of the most amazing pieces of Python-related software. I personally believe that the most important effect of Stackless was to break out the paradigm box. Once Stackless became available, people could see other ways of doing things. I personally believe that Stackless was a very strong driving force, even if only as a reference of what could be done once the conventional restrictions of the sequential processing model were lifted.

List comprehensions were added first to the language. The iterator protocol followed it, and soon after, generators were introduced. People started using then, at first slowly. List comprehensions in particular have lead a number of calls for help from people trying to grasp their syntax. But now, more and more systems rely on these new style constructs to implement complex programming patterns. While checking libraries, I can often see list-comps being used in places where loops would be used just a couple of years ago. More recently, the Web SIG defined a generator-based interface in the WSGI spec, that will allow async-style calls between the web server and the application engine. And new modules and projects seem to be getting to grips with the async nature of generators, and are using them in highly interesting and innovative ways.

After following these changes over the past few years, I was surprised to be a late "rediscoverer" of Flow Based Programming. Just a quick reading on the topic shows how much can be done in terms of application modelling with Python. There are strong parallels between FBP and other paradigms, such as functional programming; a good discussion can be found on the C2 Wiki (Flow Based Programming). The basic premise is that business applications are data-driven by nature, and thus, poorly suited to the strictly sequential Von Neumann model. This may sound like old talk today, but it's interesting to note that this stuff was developed over 30 years ago, in a time when concepts like Object Oriented Programming were still an academic novelty. Critical applications written using FBP are still in use today, which proves its suitability for an extremely demanding task.

I personally believe that right now, more and more people are "rediscovering" how to think and write data centric code in Python. The newest features (specially generator expressions) will encourage this style of programming. This will lead to a change as the Python community incorporates this data-centric paradigm shift into new applications. The trend is already stablished; it's one more great application for Python, with a huge potential for success.