web2py

The Python language

About Python

Python is a general-purpose high-level programming language. Its design philosophy emphasizes programmer productivity and code readability. It has a minimalist core syntax with very few basic commands and simple semantics, but it also has a large and comprehensive standard library, including an Application Programming Interface (API)

Python supports multiple programming paradigms, including object-oriented (class), imperative (def), and functional (lambda) programming. Python has a dynamic type system and automatic memory management using reference counting (similar to Perl, Ruby, and Scheme).

Python was first released by Guido van Rossum in 1991. The language has an open, community-based development model managed by the non-profit Python Software Foundation. There are many interpreters and compilers that implement the Python language, including one in Java (Jython) but, in this brief review, we refer to the reference C implementation created by Guido.

You can find many tutorials, the official documentation and library references of the language on the official Python website.^[python]

For additional Python references, we can recommend the books in ref.^[guido] and ref.^[lutz] .

You may skip this chapter if you are already familiar with the Python language.

Starting up

The binary distributions of web2py for Microsoft Windows or Apple OS X come packaged with the Python interpreter built into the distribution file itself.

You can start it on Windows with the following command (type at the DOS prompt):

web2py.exe -S welcome

On Apple OS X, enter the following command type in a Terminal window (assuming you're in the same folder as web2py.app):

./web2py.app/Contents/MacOS/web2py -S welcome

On a Linux or other Unix box, chances are that you have Python already installed. If so, at a shell prompt type:

python web2py.py -S welcome

If you do not have Python 2.5 (or later 2.x) already installed, you will have to download and install it before running web2py.

The -S welcome command line option instructs web2py to run the interactive shell as if the commands were executed in a controller for the welcome application, the web2py scaffolding application. This exposes almost all web2py classes, objects and functions to you. This is the only difference between the web2py interactive command line and the normal Python command line.

The admin interface also provides a web-based shell for each application. You can access the one for the "welcome" application at.

http://127.0.0.1:8000/admin/shell/index/welcome

You can try all the examples in this chapter using the normal shell or the web-based shell.

help, dir

The Python language provides two commands to obtain documentation about objects defined in the current scope, both built-in and user-defined.

We can ask for help about an object, for example "1":

>>> help(1)
Help on int object:

class int(object)
 |  int(x[, base]) -> integer
 |
 |  Convert a string or number to an integer, if possible.  A floating point
 |  argument will be truncated towards zero (this does not include a string
 |  representation of a floating point number!)  When converting a string, use
 |  the optional base.  It is an error to supply a base when converting a
 |  non-string. If the argument is outside the integer range a long object
 |  will be returned instead.
 |
 |  Methods defined here:
 |
 |  __abs__(...)
 |      x.__abs__() <==> abs(x)
...

and, since "1" is an integer, we get a description about the int class and all its methods. Here the output has been truncated because it is very long and detailed.

Similarly, we can obtain a list of methods of the object "1" with the command dir:

>>> dir(1)
['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__',
'__delattr__', '__div__', '__divmod__', '__doc__', '__float__',
'__floordiv__', '__getattribute__', '__getnewargs__', '__hash__', '__hex__',
'__index__', '__init__', '__int__', '__invert__', '__long__', '__lshift__',
'__mod__', '__mul__', '__neg__', '__new__', '__nonzero__', '__oct__',
'__or__', '__pos__', '__pow__', '__radd__', '__rand__', '__rdiv__',
'__rdivmod__', '__reduce__', '__reduce_ex__', '__repr__', '__rfloordiv__',
'__rlshift__', '__rmod__', '__rmul__', '__ror__', '__rpow__', '__rrshift__',
'__rshift__', '__rsub__', '__rtruediv__', '__rxor__', '__setattr__',
'__str__', '__sub__', '__truediv__', '__xor__']

Types

Python is a dynamically typed language, meaning that variables do not have a type and therefore do not have to be declared. Values, on the other hand, do have a type. You can query a variable for the type of value it contains:

>>> a = 3
>>> print type(a)
<type 'int'>
>>> a = 3.14
>>> print type(a)
<type 'float'>
>>> a = 'hello python'
>>> print type(a)
<type 'str'>

Python also includes, natively, data structures such as lists and dictionaries.

str

Python supports the use of two different types of strings: ASCII strings and Unicode strings. ASCII strings are delimited by '...', "..." or by '..' or """...""". Triple quotes delimit multiline strings. Unicode strings start with a u followed by the string containing Unicode characters. A Unicode string can be converted into an ASCII string by choosing an encoding for example:

>>> a = 'this is an ASCII string'
>>> b = u'This is a Unicode string'
>>> a = b.encode('utf8')

After executing these three commands, the resulting a is an ASCII string storing UTF8 encoded characters. By design, web2py uses UTF8 encoded strings internally.

It is also possible to write variables into strings in various ways:

>>> print 'number is ' + str(3)
number is 3
>>> print 'number is %s' % (3)
number is 3
>>> print 'number is %(number)s' % dict(number=3)
number is 3

The last notation is more explicit and less error prone, and is to be preferred.

Many Python objects, for example numbers, can be serialized into strings using str or repr. These two commands are very similar but produce slightly different output. For example:

>>> for i in [3, 'hello']:
        print str(i), repr(i)
3 3
hello 'hello'

For user-defined classes, str and repr can be defined/redefined using the special operators __str__ and __repr__. These are briefly described later on; for more, refer to the official Python documentation^[pydocs] . repr always has a default value.

Another important characteristic of a Python string is that, like a list, it is an iterable object.

>>> for i in 'hello':
        print i
h
e
l
l
o

list

The main methods of a Python list are append, insert, and delete:

>>> a = [1, 2, 3]
>>> print type(a)
<type 'list'>
>>> a.append(8)
>>> a.insert(2, 7)
>>> del a[0]
>>> print a
[2, 7, 3, 8]
>>> print len(a)
4

Lists can be sliced:

>>> print a[:3]
[2, 7, 3]
>>> print a[1:]
[7, 3, 8]
>>> print a[-2:]
[3, 8]

and concatenated:

>>> a = [2, 3]
>>> b = [5, 6]
>>> print a + b
[2, 3, 5, 6]

A list is iterable; you can loop over it:

>>> a = [1, 2, 3]
>>> for i in a:
        print i
1
2
3

The elements of a list do not have to be of the same type; they can be any type of Python object.

There is a very common situation for which a list comprehension can be used. Consider the following code:

>>> a = [1,2,3,4,5]
>>> b = []
>>> for x in a:
        if x % 2 == 0:
            b.append(x * 3)
>>> b
[6, 12]

This code clearly processes a list of items, selects and modifies a subset of the input list, and creates a new result list, and this code can be entirely replaced with the following list comprehension:

>>> a = [1,2,3,4,5]
>>> b = [x * 3 for x in a if x % 2 == 0]
>>> b
[6, 12]

tuple

A tuple is like a list, but its size and elements are immutable, while in a list they are mutable. If a tuple element is an object, the object attributes are mutable. A tuple is delimited by round brackets.

>>> a = (1, 2, 3)

So while this works for a list:

>>> a = [1, 2, 3]
>>> a[1] = 5
>>> print a
[1, 5, 3]

the element assignment does not work for a tuple:

>>> a = (1, 2, 3)
>>> print a[1]
2
>>> a[1] = 5
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment

A tuple, like a list, is an iterable object. Notice that a tuple consisting of a single element must include a trailing comma, as shown below:

>>> a = (1)
>>> print type(a)
<type 'int'>
>>> a = (1,)
>>> print type(a)
<type 'tuple'>

Tuples are very useful for efficient packing of objects because of their immutability, and the brackets are often optional:

>>> a = 2, 3, 'hello'
>>> x, y, z = a
>>> print x
2
>>> print z
hello

dict

A Python dict-ionary is a hash table that maps a key object to a value object. For example:

>>> a = {'k':'v', 'k2':3}
>>> a['k']
v
>>> a['k2']
3
>>> a.has_key('k')
True
>>> a.has_key('v')
False

Keys can be of any hashable type (int, string, or any object whose class implements the __hash__ method). Values can be of any type. Different keys and values in the same dictionary do not have to be of the same type. If the keys are alphanumeric characters, a dictionary can also be declared with the alternative syntax:

>>> a = dict(k='v', h2=3)
>>> a['k']
v
>>> print a
{'k':'v', 'h2':3}

Useful methods are has_key, keys, values and items:

>>> a = dict(k='v', k2=3)
>>> print a.keys()
['k', 'k2']
>>> print a.values()
['v', 3]
>>> print a.items()
[('k', 'v'), ('k2', 3)]

The items method produces a list of tuples, each containing a key and its associated value.

Dictionary elements and list elements can be deleted with the command del:

>>> a = [1, 2, 3]
>>> del a[1]
>>> print a
[1, 3]
>>> a = dict(k='v', h2=3)
>>> del a['h2']
>>> print a
{'k':'v'}

Internally, Python uses the hash operator to convert objects into integers, and uses that integer to determine where to store the value.

>>> hash("hello world")
-1500746465

About indentation

Python uses indentation to delimit blocks of code. A block starts with a line ending in colon, and continues for all lines that have a similar or higher indentation as the next line. For example:

>>> i = 0
>>> while i < 3:
>>>    print i
>>>    i = i + 1
>>>
0
1
2

It is common to use four spaces for each level of indentation. It is a good policy not to mix tabs with spaces, which can result in (invisible) confusion.

for...in

In Python, you can loop over iterable objects:

>>> a = [0, 1, 'hello', 'python']
>>> for i in a:
        print i
0
1
hello
python

One common shortcut is xrange, which generates an iterable range without storing the entire list of elements.

>>> for i in xrange(0, 4):
        print i
0
1
2
3

This is equivalent to the C/C++/C#/Java syntax:

for(int i=0; i<4; i=i+1) { print(i); }

Another useful command is enumerate, which counts while looping:

>>> a = [0, 1, 'hello', 'python']
>>> for i, j in enumerate(a):
        print i, j
0 0
1 1
2 hello
3 python

There is also a keyword range(a, b, c) that returns a list of integers starting with the value a, incrementing by c, and ending with the last value smaller than b, a defaults to 0 and c defaults to 1. xrange is similar but does not actually generate the list, only an iterator over the list; thus it is better for looping.

You can jump out of a loop using break

>>> for i in [1, 2, 3]:
         print i
         break
1

You can jump to the next loop iteration without executing the entire code block with continue

>>> for i in [1, 2, 3]:
         print i
         continue
         print 'test'
1
2
3

while

The while loop in Python works much as it does in many other programming languages, by looping an indefinite number of times and testing a condition before each iteration. If the condition is False, the loop ends.

>>> i = 0
>>> while i < 10:
        i = i + 1
>>> print i
10

There is no loop...until construct in Python.

if...elif...else

>>> for i in range(3):
>>>     if i == 0:
>>>         print 'zero'
>>>     elif i == 1:
>>>         print 'one'
>>>     else:
>>>         print 'other'
zero
one
other

"elif" means "else if". Both elif and else clauses are optional. There can be more than one elif but only one else statement. Complex conditions can be created using the not, and and or operators.

>>> for i in range(3):
>>>     if i == 0 or (i == 1 and i + 1 == 2):
>>>         print '0 or 1'

try...except...else...finally

>>> try:
>>>     a = 1 / 0
>>> except Exception, e:
>>>     print 'oops: %s' % e
>>> else:
>>>     print 'no problem here'
>>> finally:
>>>     print 'done'
oops: integer division or modulo by zero
done

If the exception is raised, it is caught by the except clause, which is executed, while the else clause is not. If no exception is raised, the except clause is not executed, but the else one is. The finally clause is always executed.

There can be multiple except clauses for different possible exceptions:

>>> try:
>>>     raise SyntaxError
>>> except ValueError:
>>>     print 'value error'
>>> except SyntaxError:
>>>     print 'syntax error'
syntax error

The else and finally clauses are optional.

Here is a list of built-in Python exceptions + HTTP (defined by web2py)

BaseException
 +-- HTTP (defined by web2py)
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- Exception
      +-- GeneratorExit
      +-- StopIteration
      +-- StandardError
      |    +-- ArithmeticError
      |    |    +-- FloatingPointError
      |    |    +-- OverflowError
      |    |    +-- ZeroDivisionError
      |    +-- AssertionError
      |    +-- AttributeError
      |    +-- EnvironmentError
      |    |    +-- IOError
      |    |    +-- OSError
      |    |         +-- WindowsError (Windows)
      |    |         +-- VMSError (VMS)
      |    +-- EOFError
      |    +-- ImportError
      |    +-- LookupError
      |    |    +-- IndexError
      |    |    +-- KeyError
      |    +-- MemoryError
      |    +-- NameError
      |    |    +-- UnboundLocalError
      |    +-- ReferenceError
      |    +-- RuntimeError
      |    |    +-- NotImplementedError
      |    +-- SyntaxError
      |    |    +-- IndentationError
      |    |         +-- TabError
      |    +-- SystemError
      |    +-- TypeError
      |    +-- ValueError
      |    |    +-- UnicodeError
      |    |         +-- UnicodeDecodeError
      |    |         +-- UnicodeEncodeError
      |    |         +-- UnicodeTranslateError
      +-- Warning
           +-- DeprecationWarning
           +-- PendingDeprecationWarning
           +-- RuntimeWarning
           +-- SyntaxWarning
           +-- UserWarning
           +-- FutureWarning
	   +-- ImportWarning
	   +-- UnicodeWarning

For a detailed description of each of them, refer to the official Python documentation.

web2py exposes only one new exception, called HTTP. When raised, it causes the program to return an HTTP error page (for more on this refer to Chapter 4).

Any object can be raised as an exception, but it is good practice to raise objects that extend one of the built-in exception classes.

def...return

Functions are declared using def. Here is a typical Python function:

>>> def f(a, b):
        return a + b
>>> print f(4, 2)
6

There is no need (or way) to specify types of the arguments or the return type(s). In this example, a function f is defined that can take two arguments.

Functions are the first code syntax feature described in this chapter to introduce the concept of scope, or namespace. In the above example, the identifiers a and b are undefined outside of the scope of function f:

>>> def f(a):
        return a + 1
>>> print f(1)
2
>>> print a
Traceback (most recent call last):
  File "<pyshell#22>", line 1, in <module>
    print a
NameError: name 'a' is not defined

Identifiers defined outside of function scope are accessible within the function; observe how the identifier a is handled in the following code:

>>> a = 1
>>> def f(b):
        return a + b
>>> print f(1)
2
>>> a = 2
>>> print f(1) # new value of a is used
3
>>> a = 1 # reset a
>>> def g(b):
        a = 2 # creates a new local a
        return a + b
>>> print g(2)
4
>>> print a # global a is unchanged
1

If a is modified, subsequent function calls will use the new value of the global a because the function definition binds the storage location of the identifier a, not the value of a itself at the time of function declaration; however, if a is assigned-to inside function g, the global a is unaffected because the new local a hides the global value. The external-scope reference can be used in the creation of closures:

>>> def f(x):
        def g(y):
            return x * y
        return g
>>> doubler = f(2) # doubler is a new function
>>> tripler = f(3) # tripler is a new function
>>> quadrupler = f(4) # quadrupler is a new function
>>> print doubler(5)
10
>>> print tripler(5)
15
>>> print quadrupler(5)
20

Function f creates new functions; and note that the scope of the name g is entirely internal to f. Closures are extremely powerful.

Function arguments can have default values, and can return multiple results:

>>> def f(a, b=2):
        return a + b, a - b
>>> x, y = f(5)
>>> print x
7
>>> print y
3

Function arguments can be passed explicitly by name, and this means that the order of arguments specified in the caller can be different than the order of arguments with which the function was defined:

>>> def f(a, b=2):
        return a + b, a - b
>>> x, y = f(b=5, a=2)
>>> print x
7
>>> print y
-3

Functions can also take a runtime-variable number of arguments:

>>> def f(*a, **b):
        return a, b
>>> x, y = f(3, 'hello', c=4, test='world')
>>> print x
(3, 'hello')
>>> print y
{'c':4, 'test':'world'}

Here arguments not passed by name (3, 'hello') are stored in the tuple a, and arguments passed by name (c and test) are stored in the dictionary b.

In the opposite case, a list or tuple can be passed to a function that requires individual positional arguments by unpacking them:

>>> def f(a, b):
        return a + b
>>> c = (1, 2)
>>> print f(*c)
3

and a dictionary can be unpacked to deliver keyword arguments:

>>> def f(a, b):
        return a + b
>>> c = {'a':1, 'b':2}
>>> print f(**c)
3

lambda

lambda provides a way to create a very short unnamed function very easily:

>>> a = lambda b: b + 2
>>> print a(3)
5

The expression "lambda [a]:[b]" literally reads as "a function with arguments [a] that returns [b]". The lambda expression is itself unnamed, but the function acquires a name by being assigned to identifier a. The scoping rules for def apply to lambda equally, and in fact the code above, with respect to a, is identical to the function declaration using def:

>>> def a(b):
        return b + 2
>>> print a(3)
5

The only benefit of lambda is brevity; however, brevity can be very convenient in certain situations. Consider a function called map that applies a function to all items in a list, creating a new list:

>>> a = [1, 7, 2, 5, 4, 8]
>>> map(lambda x: x + 2, a)
[3, 9, 4, 7, 6, 10]

This code would have doubled in size had def been used instead of lambda. The main drawback of lambda is that (in the Python implementation) the syntax allows only for a single expression; however, for longer functions, def can be used and the extra cost of providing a function name decreases as the length of the function grows. Just like def, lambda can be used to curry functions: new functions can be created by wrapping existing functions such that the new function carries a different set of arguments:

>>> def f(a, b): return a + b
>>> g = lambda a: f(a, 3)
>>> g(2)
5

There are many situations where currying is useful, but one of those is directly useful in web2py: caching. Suppose you have an expensive function that checks whether its argument is prime:

def isprime(number):
    for p in range(2, number):
        if (number % p) == 0:
            return False
    return True

This function is obviously time consuming.

Suppose you have a caching function cache.ram that takes three arguments: a key, a function and a number of seconds.

value = cache.ram('key', f, 60)

The first time it is called, it calls the function f(), stores the output in a dictionary in memory (let's say "d"), and returns it so that value is:

value = d['key']=f()

The second time it is called, if the key is in the dictionary and not older than the number of seconds specified (60), it returns the corresponding value without performing the function call.

value = d['key']

How would you cache the output of the function isprime for any input? Here is how:

>>> number = 7
>>> seconds = 60
>>> print cache.ram(str(number), lambda: isprime(number), seconds)
True
>>> print cache.ram(str(number), lambda: isprime(number), seconds)
True

The output is always the same, but the first time cache.ram is called, isprime is called; the second time it is not.

Python functions, created with either def or lambda allow re-factoring existing functions in terms of a different set of arguments. cache.ram and cache.disk are web2py caching functions.

class

Because Python is dynamically typed, Python classes and objects may seem odd. In fact, you do not need to define the member variables (attributes) when declaring a class, and different instances of the same class can have different attributes. Attributes are generally associated with the instance, not the class (except when declared as "class attributes", which is the same as "static member variables" in C++/Java).

Here is an example:

>>> class MyClass(object): pass
>>> myinstance = MyClass()
>>> myinstance.myvariable = 3
>>> print myinstance.myvariable
3

Notice that pass is a do-nothing command. In this case it is used to define a class MyClass that contains nothing. MyClass() calls the constructor of the class (in this case the default constructor) and returns an object, an instance of the class. The (object) in the class definition indicates that our class extends the built-in object class. This is not required, but it is good practice.

Here is a more complex class:

>>> class MyClass(object):
>>>    z = 2
>>>    def __init__(self, a, b):
>>>        self.x = a, self.y = b
>>>    def add(self):
>>>        return self.x + self.y + self.z
>>> myinstance = MyClass(3, 4)
>>> print myinstance.add()
9

Functions declared inside the class are methods. Some methods have special reserved names. For example, __init__ is the constructor. All variables are local variables of the method except variables declared outside methods. For example, z is a class variable, equivalent to a C++ static member variable that holds the same value for all instances of the class.

Notice that __init__ takes 3 arguments and add takes one, and yet we call them with 2 and 0 arguments respectively. The first argument represents, by convention, the local name used inside the method to refer to the current object. Here we use self to refer to the current object, but we could have used any other name. self plays the same role as *this in C++ or this in Java, but self is not a reserved keyword.

This syntax is necessary to avoid ambiguity when declaring nested classes, such as a class that is local to a method inside another class.

Special attributes, methods and operators

Class attributes, methods, and operators starting with a double underscore are usually intended to be private (i.e. to be used internally but not exposed outside the class) although this is a convention that is not enforced by the interpreter.

Some of them are reserved keywords and have a special meaning.

Here, as an example, are three of them:

• __len__
• __getitem__
• __setitem__

They can be used, for example, to create a container object that acts like a list:

>>> class MyList(object):
>>>     def __init__(self, *a): self.a = list(a)
>>>     def __len__(self): return len(self.a)
>>>     def __getitem__(self, i): return self.a[i]
>>>     def __setitem__(self, i, j): self.a[i] = j
>>> b = MyList(3, 4, 5)
>>> print b[1]
4
>>> b.a[1] = 7
>>> print b.a
[3, 7, 5]

Other special operators include __getattr__ and __setattr__, which define the get and set attributes for the class, and __sum__ and __sub__, which overload arithmetic operators. For the use of these operators we refer the reader to more advanced books on this topic. We have already mentioned the special operators __str__ and __repr__.

File input/output

In Python you can open and write in a file with:

>>> file = open('myfile.txt', 'w')
>>> file.write('hello world')
>>> file.close()

Similarly, you can read back from the file with:

>>> file = open('myfile.txt', 'r')
>>> print file.read()
hello world

Alternatively, you can read in binary mode with "rb", write in binary mode with "wb", and open the file in append mode "a", using standard C notation.

The read command takes an optional argument, which is the number of bytes. You can also jump to any location in a file using seek.

You can read back from the file with read

>>> print file.seek(6)
>>> print file.read()
world

and you can close the file with:

>>> file.close()

In the standard distribution of Python, which is known as CPython, variables are reference-counted, including those holding file handles, so CPython knows that when the reference count of an open file handle decreases to zero, the file may be closed and the variable disposed. However, in other implementations of Python such as PyPy, garbage collection is used instead of reference counting, and this means that it is possible that there may accumulate too many open file handles at one time, resulting in an error before the gc has a chance to close and dispose of them all. Therefore it is best to explicitly close file handles when they are no longer needed. web2py provides two helper functions, read_file() and write_file() inside the gluon.fileutils namespace that encapsulate the file access and ensure that the file handles being used are properly closed.

When using web2py, you do not know where the current directory is, because it depends on how web2py is configured. The variable request.folder contains the path to the current application. Paths can be concatenated with the command os.path.join, discussed below.

exec, eval

Unlike Java, Python is a truly interpreted language. This means it has the ability to execute Python statements stored in strings. For example:

>>> a = "print 'hello world'"
>>> exec(a)
'hello world'

What just happened? The function exec tells the interpreter to call itself and execute the content of the string passed as argument. It is also possible to execute the content of a string within a context defined by the symbols in a dictionary:

>>> a = "print b"
>>> c = dict(b=3)
>>> exec(a, {}, c)
3

Here the interpreter, when executing the string a, sees the symbols defined in c (b in the example), but does not see c or a themselves. This is different than a restricted environment, since exec does not limit what the inner code can do; it just defines the set of variables visible to the code.

A related function is eval, which works very much like exec except that it expects the argument to evaluate to a value, and it returns that value.

>>> a = "3*4"
>>> b = eval(a)
>>> print b
12

import

For example, if you need to use a random number generator, you can do:

>>> import random
>>> print random.randint(0, 9)
5

This prints a random integer between 0 and 9 (including 9), 5 in the example. The function randint is defined in the module random. It is also possible to import an object from a module into the current namespace:

>>> from random import randint
>>> print randint(0, 9)

or import all objects from a module into the current namespace:

>>> from random import *
>>> print randint(0, 9)

or import everything in a newly defined namespace:

>>> import random as myrand
>>> print myrand.randint(0, 9)

In the rest of this book, we will mainly use objects defined in modules os, sys, datetime, time and cPickle.

All of the web2py objects are accessible via a module called gluon, and that is the subject of later chapters. Internally, web2py uses many Python modules (for example thread), but you rarely need to access them directly.

In the following subsections we consider those modules that are most useful.

os

This module provides an interface to the operating system API. For example:

>>> import os
>>> os.chdir('..')
>>> os.unlink('filename_to_be_deleted')

Some of the os functions, such as chdir, MUST NOT be used in web2py because they are not thread-safe.

os.path.join is very useful; it allows the concatenation of paths in an OS-independent way:

>>> import os
>>> a = os.path.join('path', 'sub_path')
>>> print a
path/sub_path

System environment variables can be accessed via:

>>> print os.environ

which is a read-only dictionary.

sys

The sys module contains many variables and functions, but the one we use the most is sys.path. It contains a list of paths where Python searches for modules. When we try to import a module, Python looks for it in all the folders listed in sys.path. If you install additional modules in some location and want Python to find them, you need to append the path to that location to sys.path.

>>> import sys
>>> sys.path.append('path/to/my/modules')

When running web2py, Python stays resident in memory, and there is only one sys.path, while there are many threads servicing the HTTP requests. To avoid a memory leak, it is best to check if a path is already present before appending:

>>> path = 'path/to/my/modules'
>>> if not path in sys.path:
        sys.path.append(path)

datetime

The use of the datetime module is best illustrated by some examples:

>>> import datetime
>>> print datetime.datetime.today()
2008-07-04 14:03:90
>>> print datetime.date.today()
2008-07-04

Occasionally you may need to time-stamp data based on the UTC time as opposed to local time. In this case you can use the following function:

>>> import datetime
>>> print datetime.datetime.utcnow()
2008-07-04 14:03:90

The datetime module contains various classes: date, datetime, time and timedelta. The difference between two date or two datetime or two time objects is a timedelta:

>>> a = datetime.datetime(2008, 1, 1, 20, 30)
>>> b = datetime.datetime(2008, 1, 2, 20, 30)
>>> c = b - a
>>> print c.days
1

In web2py, date and datetime are used to store the corresponding SQL types when passed to or returned from the database.

time

The time module differs from date and datetime because it represents time as seconds from the epoch (beginning of 1970).

>>> import time
>>> t = time.time()
1215138737.571

Refer to the Python documentation for conversion functions between time in seconds and time as a datetime.

cPickle

This is a very powerful module. It provides functions that can serialize almost any Python object, including self-referential objects. For example, let's build a weird object:

>>> class MyClass(object): pass
>>> myinstance = MyClass()
>>> myinstance.x = 'something'
>>> a = [1 ,2, {'hello':'world'}, [3, 4, [myinstance]]]

and now:

>>> import cPickle
>>> b = cPickle.dumps(a)
>>> c = cPickle.loads(b)

In this example, b is a string representation of a, and c is a copy of a generated by de-serializing b.

cPickle can also serialize to and de-serialize from a file:

>>> cPickle.dump(a, open('myfile.pickle', 'wb'))
>>> c = cPickle.load(open('myfile.pickle', 'rb'))