web2py

Forms and validators

There are four distinct ways to build forms in web2py:

• FORM provides a low-level implementation in terms of HTML helpers. A FORM object can be serialized into HTML and is aware of the fields it contains. A FORM object knows how to validate submitted form values.
• SQLFORM provides a high-level API for building create, update and delete forms from an existing database table.
• SQLFORM.factory is an abstraction layer on top of SQLFORM in order to take advantage of the form generation features even if there is no database present. It generates a form very similar to SQLFORM from the description of a table but without the need to create the database table.
• CRUD methods. These are functionally equivalent to SQLFORM and are based on SQLFORM, but provide a more compact notation.

All these forms are self-aware and, if the input does not pass validation, they can modify themselves and add error messages. The forms can be queried for the validated variables and for error messages that have been generated by validation.

Arbitrary HTML code can be inserted into or extracted from the form using helpers.

FORM and SQLFORM are helpers and they can be manipulated in as similar way as the DIV. For example you can set a form style:

form = SQLFORM(..)
form['_style']='border:1px solid black'

FORM

Consider as an example a test application with the following "default.py" controller:

def display_form():
    return dict()

and the associated "default/display_form.html" view:

{{extend 'layout.html'}}
<h2>Input form</h2>
<form enctype="multipart/form-data"
      action="{{=URL()}}" method="post">
Your name:
<input name="name" />
<input type="submit" />
</form>
<h2>Submitted variables</h2>
{{=BEAUTIFY(request.vars)}}

This is a regular HTML form that asks for the user's name. When you fill the form and click the submit button, the form self-submits, and the variable request.vars.name and its value is displayed at the bottom.

You can generate the same form using helpers. This can be done in the view or in the action. Since web2py processed the form in the action, it is OK to define the form in the action.

Here is the new controller:

def display_form():
   form=FORM('Your name:', INPUT(_name='name'), INPUT(_type='submit'))
   return dict(form=form)

and the associated "default/display_form.html" view:

{{extend 'layout.html'}}
<h2>Input form</h2>
{{=form}}
<h2>Submitted variables</h2>
{{=BEAUTIFY(request.vars)}}

The code so far is equivalent to the previous code, but the form is generated by the statement {{=form}} which serializes the FORM object.

Now we add one level of complexity by adding form validation and processing.

Change the controller as follows:

def display_form():
    form=FORM('Your name:',
              INPUT(_name='name', requires=IS_NOT_EMPTY()),
              INPUT(_type='submit'))
    if form.accepts(request,session):
        response.flash = 'form accepted'
    elif form.errors:
        response.flash = 'form has errors'
    else:
        response.flash = 'please fill the form'
    return dict(form=form)

and the associated "default/display_form.html" view:

{{extend 'layout.html'}}
<h2>Input form</h2>
{{=form}}
<h2>Submitted variables</h2>
{{=BEAUTIFY(request.vars)}}
<h2>Accepted variables</h2>
{{=BEAUTIFY(form.vars)}}
<h2>Errors in form</h2>
{{=BEAUTIFY(form.errors)}}

Notice that:

• In the action, we added the requires=IS_NOT_EMPTY() validator for the input field "name".
• In the action, we added a call to form.accepts(..)
• In the view, we are printing form.vars and form.errors as well as the form and request.vars.

All the work is done by the accepts method of the form object. It filters the request.vars according to the declared requirements (expressed by validators). accepts stores those variables that pass validation into form.vars. If a field value does not meet a requirement, the failing validator returns an error and the error is stored in form.errors. Both form.vars and form.errors are gluon.storage.Storage objects similar to request.vars. The former contains the values that passed validation, for example:

form.vars.name = "Max"

The latter contains the errors, for example:

form.errors.name = "Cannot be empty!"

The full signature of the accepts method is the following:

form.accepts(vars, session=None, formname='default',
             keepvalues=False, onvalidation=None,
             dbio=True, hideerror=False):

The meaning of the optional parameters is explained in the next sub-sections.

The first argument can be request.vars or request.get_vars or request.post_vars or simply request. The latter is equivalent to accepting as input the request.post_vars.

The accepts function returns True if the form is accepted and False otherwise. A form is not accepted if it has errors or when it has not been submitted (for example, the first time it is shown).

Here is how this page looks the first time it is displayed:

Here is how it looks upon invalid submission:

Here is how it looks upon a valid submission:

The process and validate methods

A shortcut for

form.accepts(request.post_vars,session,...)

is

form.process(...).accepted

the latter does not need the request and session arguments (although you can specify them optionally). it also differs from accepts because it returns the form itself. Internally process calls accepts and passes its arguments to it. The value returned by accepts is stored in form.accepted.

The process function takes some extra argument that accepts does not take:

• message_onsuccess
• onsuccess: if equal to 'flash' (default) and the form is accepted it will flash the above `message_onsuccess- message_onfailure- onfailure: if equal to 'flash' (default) and the form fails validation, it will flash the above `message_onfailure
• next the user to redirect after the form is accepted.

onsuccess and onfailure can be functions like lambda form: do_something(form).

form.validate(...)

is a shortcut for

form.process(...,dbio=False).accepted

Hidden fields

When the above form object is serialized by {{=form}}, and because of the previous call to the accepts method, it now looks like this:

<form enctype="multipart/form-data" action="" method="post">
your name:
<input name="name" />
<input type="submit" />
<input value="783531473471" type="hidden" name="_formkey" />
<input value="default" type="hidden" name="_formname" />
</form>

Notice the presence of two hidden fields: "_formkey" and "_formname". Their presence is triggered by the call to accepts and they play two different and important roles:

• The hidden field called "_formkey" is a one-time token that web2py uses to prevent double submission of forms. The value of this key is generated when the form is serialized and stored in the session. When the form is submitted this value must match, or else accepts returns False without errors as if the form was not submitted at all. This is because web2py cannot determine whether the form was submitted correctly.
• The hidden field called "_formname" is generated by web2py as a name for the form, but the name can be overridden. This field is necessary to allow pages that contain and process multiple forms. web2py distinguishes the different submitted forms by their names.
• Optional hidden fields specified as FORM(..,hidden=dict(...)).

The role of these hidden fields and their usage in custom forms and pages with multiple forms is discussed in more detail later in the chapter.

If the form above is submitted with an empty "name" field, the form does not pass validation. When the form is serialized again it appears as:

<form enctype="multipart/form-data" action="" method="post">
your name:
<input value="" name="name" />
<div class="error">cannot be empty!</div>
<input type="submit" />
<input value="783531473471" type="hidden" name="_formkey" />
<input value="default" type="hidden" name="_formname" />
</form>

Notice the presence of a DIV of class "error" in the serialized form. web2py inserts this error message in the form to notify the visitor about the field that did not pass validation. The accepts method, upon submission, determines that the form is submitted, checks whether the field "name" is empty and whether it is required, and eventually inserts the error message from the validator into the form.

The base "layout.html" view is expected to handle DIVs of class "error". The default layout uses jQuery effects to make errors appear and slide down with a red background. See Chapter 11 for more details.

keepvalues

The optional argument keepvalues tells web2py what to do when a form is accepted and there is no redirection, so the same form is displayed again. By default the form is cleared. If keepvalues is set to True, the form is pre-populated with the previously inserted values. This is useful when you have a form that is supposed to be used repeatedly to insert multiple similar records. If the dbio argument is set to False, web2py will not perform any DB insert/update after accepting form. If hideerror is set to True and the form contains errors, these will not be displayed when the form is rendered (it will be up to you to display them from form.errors somehow. The onvalidation argument is explained below.

onvalidation

The onvalidation argument can be None or can be a function that takes the form and returns nothing. Such a function would be called and passed the form, immediately after validation (if validation passes) and before anything else happens. The purpose of this function is multifold. It can be used, for example, to perform additional checks on the form and eventually add errors to the form. It can also be used to compute the values of some fields based on the values of other fields. It can be used to trigger some action (like sending an email) before a record is created/updated.

Here is an example:

db.define_table('numbers',
    Field('a', 'integer'),
    Field('b', 'integer'),
    Field('c', 'integer', readable=False, writable=False))

def my_form_processing(form):
    c = form.vars.a * form.vars.b
    if c < 0:
       form.errors.b = 'a*b cannot be negative'
    else:
       form.vars.c = c

def insert_numbers():
   form = SQLFORM(db.numbers)
   if form.process(onvalidation=my_form_processing).accepted:
       session.flash = 'record inserted'
       redirect(URL())
   return dict(form=form)

Detect record change

When filling a form to edit a record there is a small probability that another user may concurrently be editing the same record. So when we save the record we want to check for possible conflicts. This can be done:

db.define_table('dog',Field('name'))

def edit_dog():
    dog = db.dog(request.args(0)) or redirect(URL('error'))
    form=SQLFORM(db.dog,dog)
    form.process(detect_record_change=True)
    if form.record_changed:
        # do something
    elif form.accepted:
        # do something else
    else:
        # do nothing
    return dict(form=form)

Forms and redirection

The most common way to use forms is via self-submission, so that the submitted field variables are processed by the same action that generated the form. Once the form is accepted, it is unusual to display the current page again (something we are doing here only to keep things simple). It is more common to redirect the visitor to a "next" page.

Here is the new example controller:

def display_form():
    form = FORM('Your name:',
              INPUT(_name='name', requires=IS_NOT_EMPTY()),
              INPUT(_type='submit'))
    if form.process().accepted:
        session.flash = 'form accepted'
        redirect(URL('next'))
    elif form.errors:
        response.flash = 'form has errors'
    else:
        response.flash = 'please fill the form'
    return dict(form=form)

def next():
    return dict()

In order to set a flash on the next page instead of the current page you must use session.flash instead of response.flash. web2py moves the former into the latter after redirection. Note that using session.flash requires that you do not session.forget().

Multiple forms per page

The content of this section applies to both FORM and SQLFORM objects. It is possible to have multiple forms per page, but you must allow web2py to distinguish them. If these are derived by SQLFORM from different tables, then web2py gives them different names automatically; otherwise you need to explicitly give them different form names. Here is an example:

def two_forms():
    form1 = FORM(INPUT(_name='name', requires=IS_NOT_EMPTY()),
               INPUT(_type='submit'))
    form2 = FORM(INPUT(_name='name', requires=IS_NOT_EMPTY()),
               INPUT(_type='submit'))
    if form1.process(formname='form_one').accepted:
        response.flash = 'form one accepted'
    if form2.process(formname='form_two').accepted:
        response.flash = 'form two accepted'
    return dict(form1=form1, form2=form2)

and here is the output it produces:

When the visitor submits an empty form1, only form1 displays an error; if the visitor submits an empty form2, only form2 displays an error message.

Sharing forms

The content of this section applies to both FORM and SQLFORM objects. What we discuss here is possible but not recommended, since it is always good practice to have forms that self-submit. Sometimes, though, you don't have a choice, because the action that sends the form and the action that receives it belong to different applications.

It is possible to generate a form that submits to a different action. This is done by specifying the URL of the processing action in the attributes of the FORM or SQLFORM object. For example:

form = FORM(INPUT(_name='name', requires=IS_NOT_EMPTY()),
        INPUT(_type='submit'), _action=URL('page_two'))

def page_one():
    return dict(form=form)

def page_two():
    if form.process(session=None, formname=None).accepted:
         response.flash = 'form accepted'
    else:
         response.flash = 'there was an error in the form'
    return dict()

Notice that since both "page_one" and "page_two" use the same form, we have defined it only once by placing it outside of all the actions, in order not to repeat ourselves. The common portion of code at the beginning of a controller gets executed every time before giving control to the called action.

Since "page_one" does not call process (nor accepts), the form has no name and no key, so you must pass session=None and set formname=None in process, or the form will not validate when "page_two" receives it.

SQLFORM

We now move to the next level by providing the application with a model file:

db = DAL('sqlite://storage.sqlite')
db.define_table('person', Field('name', requires=IS_NOT_EMPTY()))

Modify the controller as follows:

def display_form():
   form = SQLFORM(db.person)
   if form.process().accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   else:
       response.flash = 'please fill out the form'
   return dict(form=form)

The view does not need to be changed.

In the new controller, you do not need to build a FORM, since the SQLFORM constructor built one from the table db.person defined in the model. This new form, when serialized, appears as:

<form enctype="multipart/form-data" action="" method="post">
  <table>
    <tr id="person_name__row">
       <td><label id="person_name__label"
                  for="person_name">Your name: </label></td>
       <td><input type="text" class="string"
                  name="name" value="" id="person_name" /></td>
       <td></td>
    </tr>
    <tr id="submit_record__row">
       <td></td>
       <td><input value="Submit" type="submit" /></td>
       <td></td>
    </tr>
  </table>
  <input value="9038845529" type="hidden" name="_formkey" />
  <input value="person" type="hidden" name="_formname" />
</form>

The automatically generated form is more complex than the previous low-level form. First of all, it contains a table of rows, and each row has three columns. The first column contains the field labels (as determined from the db.person), the second column contains the input fields (and eventually error messages), and the third column is optional and therefore empty (it can be populated with the fields in the SQLFORM constructor).

All tags in the form have names derived from the table and field name. This allows easy customization of the form using CSS and JavaScript. This capability is discussed in more detail in Chapter 11.

More important is that now the accepts method does a lot more work for you. As in the previous case, it performs validation of the input, but additionally, if the input passes validation, it also performs a database insert of the new record and stores in form.vars.id the unique "id" of the new record.

A SQLFORM object also deals automatically with "upload" fields by saving uploaded files in the "uploads" folder (after having them renamed safely to avoid conflicts and prevent directory traversal attacks) and stores their names (their new names) into the appropriate field in the database. After the form has been processed, the new filename is available in form.vars.fieldname (i.e., it replaces the cgi.FieldStorage object in request.vars.fieldname), so you can easily reference the new name right after upload.

A SQLFORM displays "boolean" values with checkboxes, "text" values with textareas, values required to be in a definite set or a database with drop-boxes, and "upload" fields with links that allow users to download the uploaded files. It hides "blob" fields, since they are supposed to be handled differently, as discussed later.

For example, consider the following model:

db.define_table('person',
    Field('name', requires=IS_NOT_EMPTY()),
    Field('married', 'boolean'),
    Field('gender', requires=IS_IN_SET(['Male', 'Female', 'Other'])),
    Field('profile', 'text'),
    Field('image', 'upload'))

In this case, SQLFORM(db.person) generates the form shown below:

The SQLFORM constructor allows various customizations, such as displaying only a subset of the fields, changing the labels, adding values to the optional third column, or creating UPDATE and DELETE forms, as opposed to INSERT forms like the current one. SQLFORM is the single biggest time-saver object in web2py.

The class SQLFORM is defined in "gluon/sqlhtml.py". It can be easily extended by overriding its xml method, the method that serializes the objects, to change its output.

SQLFORM(table, record=None,
        deletable=False, linkto=None,
        upload=None, fields=None, labels=None,
        col3={}, submit_button='Submit',
        delete_label='Check to delete:',
        showid=True, readonly=False,
        comments=True, keepopts=[],
        ignore_rw=False, record_id=None,
        formstyle='table3cols',
	buttons=['submit'], separator=': ',
        **attributes)

• The optional second argument turns the INSERT form into an UPDATE form for the specified record (see next subsection).
  showid
  delete_label
  id_label
  submit_button
• If deletable is set to True, the UPDATE form displays a "Check to delete" checkbox. The value of the label if this field is set via the delete_label argument.
• submit_button sets the value of the submit button.
• id_label sets the label of the record "id"
• The "id" of the record is not shown if showid is set to False.
• fields is an optional list of field names that you want to display. If a list is provided, only fields in the list are displayed. For example:

fields = ['name']

• labels is a dictionary of field labels. The dictionary key is a field name and the corresponding value is what gets displayed as its label. If a label is not provided, web2py derives the label from the field name (it capitalizes the field name and replaces underscores with spaces). For example:

labels = {'name':'Your Full Name:'}

• col3 is a dictionary of values for the third column. For example:

col3 = {'name':A('what is this?',
      _href='http://www.google.com/search?q=define:name')}

• linkto and upload are optional URLs to user-defined controllers that allow the form to deal with reference fields. This is discussed in more detail later in the section.
• readonly. If set to True, displays the form as readonly
• comments. If set to False, does not display the col3 comments
• ignore_rw. Normally, for a create/update form, only fields marked as writable=True are shown, and for readonly forms, only fields marked as readable=True are shown. Setting ignore_rw=True causes those constraints to be ignored, and all fields are displayed. This is mostly used in the appadmin interface to display all fields for each table, overriding what the model indicates.
• formstyle
  formstyle determines the style to be used when serializing the form in html. It can be "table3cols" (default), "table2cols" (one row for label and comment, and one row for input), "ul" (makes an unordered list of input fields), "divs" (represents the form using css friendly divs, for arbitrary customization). formstyle can also be a function that takes (record_id, field_label, field_widget, field_comment) as attributes and returns a TR() object.
• buttons
  is a list of INPUTs or TAG.BUTTONs (though technically could be any combination of helpers) that will be added to a DIV where the submit button would go.
• separator
  separator sets the string that separates form labels from form input fields.
• Optional attributes are arguments starting with underscore that you want to pass to the FORM tag that renders the SQLFORM object. Examples are:

_action = '.'
_method = 'POST'

There is a special hidden attribute. When a dictionary is passed as hidden, its items are translated into "hidden" INPUT fields (see the example for the FORM helper in Chapter 5).

form = SQLFORM(....,hidden=...)

causes the hidden fields to be passed with the submission, no more, no less. form.accepts(...) is not intended to read the received hidden fields and move them into form.vars. The reason is security. hidden fields can be tampered with. So you have to do explicitly move hidden fields from the request to the form:

form.vars.a = request.vars.a
form = SQLFORM(..., hidden=dict(a='b'))

SQLFORM and insert/update/delete

SQLFORM creates a new db record when the form is accepted. Assuming

form=SQLFORM(db.test)

If you pass a record as the optional second argument to the SQLFORM constructor, the form becomes an UPDATE form for that record. This means that when the form is submitted the existing record is updated and no new record is inserted. If you set the argument deletable=True, the UPDATE form displays a "check to delete" checkbox. If checked, the record is deleted.

If a form is submitted and the delete checkbox is checked the attribute form.deleted is set to True.

You can modify the controller of the previous example so that when we pass an additional integer argument in the URL path, as in:

/test/default/display_form/2

and if there is a record with the corresponding id, the SQLFORM generates an UPDATE/DELETE form for the record:

def display_form():
   record = db.person(request.args(0)) or redirect(URL('index'))
   form = SQLFORM(db.person, record)
   if form.process().accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   return dict(form=form)

Line 2 finds the record and line 3 makes an UPDATE/DELETE form. Line 4 does all the corresponding form processing.

An update form is very similar to a create form except that it is pre-populated with the current record and it previews images. By default deletable = True which means the update form will display a "delete record" option.

Edit forms also contain a hidden INPUT field with name="id" which is used to identify the record. This id is also stored server-side for additional security and, if the visitor tampers with the value of this field, the UPDATE is not performed and web2py raises a SyntaxError, "user is tampering with form".

When a Field is marked with writable=False, the field is not shown in create forms, and it is shown readonly in update forms. If a field is marked as writable=False and readable=False, then the field is not shown at all, not even in update forms.

Forms created with

form = SQLFORM(...,ignore_rw=True)

ignore the readable and writable attributes and always show all fields. Forms in appadmin ignore them by default.

Forms created with

form = SQLFORM(table,record_id,readonly=True)

always show all fields in readonly mode, and they cannot be accepted.

SQLFORM in HTML

There are times when you want to use SQLFORM to benefit from its form generation and processing, but you need a level of customization of the form in HTML that you cannot achieve with the parameters of the SQLFORM object, so you have to design the form using HTML.

Now, edit the previous controller and add a new action:

def display_manual_form():
   form = SQLFORM(db.person)
   if form.process(session=None, formname='test').accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   else:
       response.flash = 'please fill the form'
   # Note: no form instance is passed to the view
   return dict()

and insert the form in the associated "default/display_manual_form.html" view:

{{extend 'layout.html'}}
<form>
<ul>
  <li>Your name is <input name="name" /></li>
</ul>
  <input type="submit" />
  <input type="hidden" name="_formname" value="test" />
</form>

Notice that the action does not return the form because it does not need to pass it to the view. The view contains a form created manually in HTML. The form contains a hidden field "_formname" that must be the same formname specified as an argument of accepts in the action. web2py uses the form name in case there are multiple forms on the same page, to determine which one was submitted. If the page contains a single form, you can set formname=None and omit the hidden field in the view.

form.accepts will look inside response.vars for data that matches fields in the database table db.person. These fields are declared in the HTML in the format

<input name="field_name_goes_here" />

Note that in the example given, the form variables will be passed on the URL as arguments. If this is not desired, the POST protocol will have to be specified. Note furthermore, that if upload fields are specified, the form will have to be set up to allow this. Here, both options are shown:

<form enctype="multipart/form-data" method="post">

SQLFORM and uploads

Fields of type "upload" are special. They are rendered as INPUT fields of type="file". Unless otherwise specified, the uploaded file is streamed in using a buffer, and stored under the "uploads" folder of the application using a new safe name, assigned automatically. The name of this file is then saved into the field of type uploads.

As an example, consider the following model:

db.define_table('person',
    Field('name', requires=IS_NOT_EMPTY()),
    Field('image', 'upload'))

You can use the same controller action "display_form" shown above.

When you insert a new record, the form allows you to browse for a file. Choose, for example, a jpg image. The file is uploaded and stored as:

applications/test/uploads/person.image.XXXXX.jpg

"XXXXXX" is a random identifier for the file assigned by web2py.

Notice that, by default, the original filename of an uploaded file is b16encoded and used to build the new name for the file. This name is retrieved by the default "download" action and used to set the content disposition header to the original filename.

Only its extension is preserved. This is a security requirement since the filename may contain special characters that could allow a visitor to perform directory traversal attacks or other malicious operations.

The new filename is also stored in form.vars.image.

When editing the record using an UPDATE form, it would be nice to display a link to the existing uploaded file, and web2py provides a way to do it.

If you pass a URL to the SQLFORM constructor via the upload argument, web2py uses the action at that URL to download the file. Consider the following actions:

def display_form():
   record = db.person(request.args(0)) or redirect(URL('index'))
   form = SQLFORM(db.person, record, deletable=True,
                  upload=URL('download'))
   if form.process().accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   return dict(form=form)

def download():
    return response.download(request, db)

Now, insert a new record at the URL:

http://127.0.0.1:8000/test/default/display_form

Upload an image, submit the form, and then edit the newly created record by visiting:

http://127.0.0.1:8000/test/default/display_form/3

(here we assume the latest record has id=3). The form will display an image preview as shown below:

This form, when serialized, generates the following HTML:

<td><label id="person_image__label" for="person_image">Image: </label></td>
<td><div><input type="file" id="person_image" class="upload" name="image"
/>[<a href="/test/default/download/person.image.0246683463831.jpg">file</a>|
<input type="checkbox" name="image__delete" />delete]</div></td><td></td></tr>
<tr id="delete_record__row"><td><label id="delete_record__label" for="delete_record"
>Check to delete:</label></td><td><input type="checkbox" id="delete_record"
class="delete" name="delete_this_record" /></td>

which contains a link to allow downloading of the uploaded file, and a checkbox to remove the file from the database record, thus storing NULL in the "image" field.

Why is this mechanism exposed? Why do you need to write the download function? Because you may want to enforce some authorization mechanism in the download function. See Chapter 9 for an example.

Normally uploaded files are stored into "app/uploads" but you can specify an alternate location:

Field('image', 'upload', uploadfolder='...')

In most operating system, accessig the file system can become slow when there are many files in the same folder. If you plan to upload more than 1000 files you can ask web2py to organize the uploads in subfolders:

Field('image', 'upload', uploadseparate=True)

Storing the original filename

web2py automatically stores the original filename inside the new UUID filename and retrieves it when the file is downloaded. Upon download, the original filename is stored in the content-disposition header of the HTTP response. This is all done transparently without the need for programming.

Occasionally you may want to store the original filename in a database field. In this case, you need to modify the model and add a field to store it in:

db.define_table('person',
    Field('name', requires=IS_NOT_EMPTY()),
    Field('image_filename'),
    Field('image', 'upload'))

then you need to modify the controller to handle it:

def display_form():
    record = db.person(request.args(0)) or redirect(URL('index'))
    url = URL('download')
    form = SQLFORM(db.person, record, deletable=True,
                   upload=url, fields=['name', 'image'])
    if request.vars.image!=None:
        form.vars.image_filename = request.vars.image.filename
    if form.process().accepted:
        response.flash = 'form accepted'
    elif form.errors:
        response.flash = 'form has errors'
    return dict(form=form)

Notice that the SQLFORM does not display the "image_filename" field. The "display_form" action moves the filename of the request.vars.image into the form.vars.image_filename, so that it gets processed by accepts and stored in the database. The download function, before serving the file, checks in the database for the original filename and uses it in the content-disposition header.

autodelete

The SQLFORM, upon deleting a record, does not delete the physical uploaded file(s) referenced by the record. The reason is that web2py does not know whether the same file is used/linked by other tables or used for other purpose. If you know it is safe to delete the actual file when the corresponding record is deleted, you can do the following:

db.define_table('image',
    Field('name', requires=IS_NOT_EMPTY()),
    Field('file','upload',autodelete=True))

The autodelete attribute is False by default. When set to True is makes sure the file is deleted when the record is deleted.

Links to referencing records

Now consider the case of two tables linked by a reference field. For example:

db.define_table('person',
    Field('name', requires=IS_NOT_EMPTY()))
db.define_table('dog',
    Field('owner', db.person),
    Field('name', requires=IS_NOT_EMPTY()))
db.dog.owner.requires = IS_IN_DB(db,db.person.id,'%(name)s')

A person has dogs, and each dog belongs to an owner, which is a person. The dog owner is required to reference a valid db.person.id by '%(name)s'.

Let's use the appadmin interface for this application to add a few persons and their dogs.

When editing an existing person, the appadmin UPDATE form shows a link to a page that lists the dogs that belong to the person. This behavior can be replicated using the linkto argument of the SQLFORM. linkto has to point to the URL of a new action that receives a query string from the SQLFORM and lists the corresponding records. Here is an example:

def display_form():
   record = db.person(request.args(0)) or redirect(URL('index'))
   url = URL('download')
   link = URL('list_records', args='db')
   form = SQLFORM(db.person, record, deletable=True,
                  upload=url, linkto=link)
   if form.process().accepted:
       response.flash = 'form accepted'
   elif form.errors:
       response.flash = 'form has errors'
   return dict(form=form)

Here is the page:

There is a link called "dog.owner". The name of this link can be changed via the labels argument of the SQLFORM, for example:

labels = {'dog.owner':"This person's dogs"}

If you click on the link you get directed to:

/test/default/list_records/dog?query=dog.owner%3D5

"list_records" is the specified action, with request.args(0) set to the name of the referencing table and request.vars.query set to the SQL query string. The query string in the URL contains the value "dog.owner=5" appropriately url-encoded (web2py decodes this automatically when the URL is parsed).

You can easily implement a very general "list_records" action as follows:

def list_records():
    table = request.args(0)
    query = request.vars.query
    records = db(query).select(db[table].ALL)
    return dict(records=records)

with the associated "default/list_records.html" view:

{{extend 'layout.html'}}
{{=records}}

When a set of records is returned by a select and serialized in a view, it is first converted into a SQLTABLE object (not the same as a Table) and then serialized into an HTML table, where each field corresponds to a table column.

Pre-populating the form

It is always possible to pre-populate a form using the syntax:

form.vars.name = 'fieldvalue'

Statements like the one above must be inserted after the form declaration and before the form is accepted, whether or not the field ("name" in the example) is explicitly visualized in the form.

Adding extra form elements to SQLFORM

Sometimes you may wish to add an extra element to your form after it has been created. For example, you may wish to add a checkbox which confirms the user agrees with the terms and conditions of your website:

form = SQLFORM(db.yourtable)
my_extra_element = TR(LABEL('I agree to the terms and conditions'),                       INPUT(_name='agree',value=True,_type='checkbox'))
form[0].insert(-1,my_extra_element)

The variable my_extra_element should be adapted to the formstyle. In this example, the default formstyle='table3cols' has been assumed.

After submission, form.vars.agree will contain the status of the checkbox, which could then be used in an onvalidation function, for instance.

SQLFORM without database IO

There are times when you want to generate a form from a database table using SQLFORM and you want to validate a submitted form accordingly, but you do not want any automatic INSERT/UPDATE/DELETE in the database. This is the case, for example, when one of the fields needs to be computed from the value of other input fields. This is also the case when you need to perform additional validation on the inserted data that cannot be achieved via standard validators.

This can be done easily by breaking:

form = SQLFORM(db.person)
if form.process().accepted:
    response.flash = 'record inserted'

into:

form = SQLFORM(db.person)
if form.validate():
    ### deal with uploads explicitly
    form.vars.id = db.person.insert(**dict(form.vars))
    response.flash = 'record inserted'

The same can be done for UPDATE/DELETE forms by breaking:

form = SQLFORM(db.person,record)
if form.process().accepted:
    response.flash = 'record updated'

into:

form = SQLFORM(db.person,record)
if form.validate():
    if form.deleted:
        db(db.person.id==record.id).delete()
    else:
        record.update_record(**dict(form.vars))
    response.flash = 'record updated'

In the case of a table including an "upload"-type field ("fieldname"), both process(dbio=False) and validate() deal with the storage of the uploaded file as if process(dbio=True), the default behavior.

The name assigned by web2py to the uploaded file can be found in:

form.vars.fieldname

SQLFORM.factory

There are cases when you want to generate forms as if you had a database table but you do not want the database table. You simply want to take advantage of the SQLFORM capability to generate a nice looking CSS-friendly form and perhaps perform file upload and renaming.

This can be done via a form_factory. Here is an example where you generate the form, perform validation, upload a file and store everything in the session :

def form_from_factory():
    form = SQLFORM.factory(
        Field('your_name', requires=IS_NOT_EMPTY()),
        Field('your_image', 'upload'))
    if form.process().accepted:
        response.flash = 'form accepted'
        session.your_name = form.vars.your_name
        session.filename = form.vars.your_image
    elif form.errors:
        response.flash = 'form has errors'
    return dict(form=form)

Here is the "default/form_from_factory.html" view:

{{extend 'layout.html'}}
{{=form}}

You need to use an underscore instead of a space for field labels, or explicitly pass a dictionary of labels to form_factory, as you would for a SQLFORM. By default SQLFORM.factory generates the form using html "id" attributes generated as if the form was generated from a table called "no_table". To change this dummy table name, use the table_name attribute for the factory:

form = SQLFORM.factory(...,table_name='other_dummy_name')

Changing the table_name is necessary if you need to place two factory generated forms in the same table and want to avoid CSS conflicts.

One form for multiple tables

It often happens that you have two tables (for example 'client' and 'address' which are linked together by a reference and you want to create a single form that allows to insert info about one client and its default address. Here is how:

model:

db.define_table('client',
     Field('name'))
db.define_table('address',
    Field('client',db.client,writable=False,readable=False),
    Field('street'),Field('city'))

controller:

def register():
    form=SQLFORM.factory(db.client,db.address)
    if form.process().accepted:
        id = db.client.insert(**db.client._filter_fields(form.vars))
        form.vars.client=id
        id = db.address.insert(**db.address._filter_fields(form.vars))
        response.flash='Thanks for filling the form'
    return dict(form=form)

Notice the SQLFORM.factory (it makes ONE form using public fields from both tables and inherits their validators too). On form accepts this does two inserts, some data in one table and some data in the other.

This only works when the tables don't have field names in common.

CRUD

One of the recent additions to web2py is the Create/Read/Update/Delete (CRUD) API on top of SQLFORM. CRUD creates an SQLFORM, but it simplifies the coding because it incorporates the creation of the form, the processing of the form, the notification, and the redirection, all in one single function.

The first thing to notice is that CRUD differs from the other web2py APIs we have used so far because it is not already exposed. It must be imported. It also must be linked to a specific database. For example:

from gluon.tools import Crud
crud = Crud(db)

The crud object defined above provides the following API:

• crud.tables() returns a list of tables defined in the database.
• crud.create(db.tablename) returns a create form for table tablename.
• crud.read(db.tablename, id) returns a readonly form for tablename and record id.
• crud.update(db.tablename, id) returns an update form for tablename and record id.
• crud.delete(db.tablename, id) deletes the record.
• crud.select(db.tablename, query) returns a list of records selected from the table.
• crud.search(db.tablename) returns a tuple (form, records) where form is a search form and records is a list of records based on the submitted search form.
• crud() returns one of the above based on the request.args().

For example, the following action:

def data(): return dict(form=crud())

would expose the following URLs:

http://.../[app]/[controller]/data/tables
http://.../[app]/[controller]/data/create/[tablename]
http://.../[app]/[controller]/data/read/[tablename]/[id]
http://.../[app]/[controller]/data/update/[tablename]/[id]
http://.../[app]/[controller]/data/delete/[tablename]/[id]
http://.../[app]/[controller]/data/select/[tablename]
http://.../[app]/[controller]/data/search/[tablename]

However, the following action:

def create_tablename():
    return dict(form=crud.create(db.tablename))

would only expose the create method

http://.../[app]/[controller]/create_tablename

While the following action:

def update_tablename():
    return dict(form=crud.update(db.tablename, request.args(0)))

would only expose the update method

http://.../[app]/[controller]/update_tablename/[id]

and so on.

The behavior of CRUD can be customized in two ways: by setting some attributes of the crud object or by passing extra parameters to each of its methods.

Settings

Here is a complete list of current CRUD attributes, their default values, and meaning:

To enforce authentication on all crud forms:

crud.settings.auth = auth

The use is explained in chapter 9.

To specify the controller that defines the data function which returns the crud object

crud.settings.controller = 'default'

To specify the URL to redirect to after a successful "create" record:

crud.settings.create_next = URL('index')

To specify the URL to redirect to after a successful "update" record:

crud.settings.update_next = URL('index')

To specify the URL to redirect to after a successful "delete" record:

crud.settings.delete_next = URL('index')

To specify the URL to be used for linking uploaded files:

crud.settings.download_url = URL('download')

To specify extra functions to be executed after standard validation procedures for crud.create forms:

crud.settings.create_onvalidation = StorageList()

StorageList is the same as a Storage object, they are both defined in the file "gluon/storage.py", but it defaults to [] as opposed to None. It allows the following syntax:

crud.settings.create_onvalidation.mytablename.append(lambda form:....)

To specify extra functions to be executed after standard validation procedures for crud.update forms:

crud.settings.update_onvalidation = StorageList()

To specify extra functions to be executed after completion of crud.create forms:

crud.settings.create_onaccept = StorageList()

To specify extra functions to be executed after completion of crud.update forms:

crud.settings.update_onaccept = StorageList()

To specify extra functions to be executed after completion of crud.update if record is deleted:

crud.settings.update_ondelete = StorageList()

To specify extra functions to be executed after completion of crud.delete:

crud.settings.delete_onaccept = StorageList()

To determine whether the "update" forms should have a "delete" button:

crud.settings.update_deletable = True

To determine whether the "update" forms should show the id of the edited record:

crud.settings.showid = False

To determine whether forms should keep the previously inserted values or reset to default after successful submission:

crud.settings.keepvalues = False

Crud always detects whether a record being edited has been modified by a third party in the time between the time when the form is displayed and the time when it is submitted. This behavior is equivalent to

and it is set in:

crud.settings.detect_record_change = True

and it can be changed/disabled by setting the variable to False.

You can change the form style by

crud.settings.formstyle = 'table3cols' or 'table2cols' or 'divs' or 'ul'

You can set the separator in all crud forms:

crud.settings.label_separator = ':'

You can add captcha to forms, using the same convention explained for auth, with:

crud.settings.create_captcha = None
crud.settings.update_captcha = None
crud.settings.captcha = None

Messages

Here is a list of customizable messages:

crud.messages.submit_button = 'Submit'

sets the text of the "submit" button for both create and update forms.

crud.messages.delete_label = 'Check to delete:'

sets the label of the "delete" button in "update" forms.

crud.messages.record_created = 'Record Created'

sets the flash message on successful record creation.

crud.messages.record_updated = 'Record Updated'

sets the flash message on successful record update.

crud.messages.record_deleted = 'Record Deleted'

sets the flash message on successful record deletion.

crud.messages.update_log = 'Record %(id)s updated'

sets the log message on successful record update.

crud.messages.create_log = 'Record %(id)s created'

sets the log message on successful record creation.

crud.messages.read_log = 'Record %(id)s read'

sets the log message on successful record read access.

crud.messages.delete_log = 'Record %(id)s deleted'

sets the log message on successful record deletion.

Notice that crud.messages belongs to the class gluon.storage.Message which is similar to gluon.storage.Storage but it automatically translates its values, without need for the T operator.

Log messages are used if and only if CRUD is connected to Auth as discussed in Chapter 9. The events are logged in the Auth table "auth_events".

Methods

The behavior of CRUD methods can also be customized on a per call basis. Here are their signatures:

crud.tables()
crud.create(table, next, onvalidation, onaccept, log, message)
crud.read(table, record)
crud.update(table, record, next, onvalidation, onaccept, ondelete, log, message, deletable)
crud.delete(table, record_id, next, message)
crud.select(table, query, fields, orderby, limitby, headers, **attr)
crud.search(table, query, queries, query_labels, fields, field_labels, zero, showall, chkall)

• table is a DAL table or a tablename the method should act on.
• record and record_id are the id of the record the method should act on.
• next is the URL to redirect to after success. If the URL contains the substring "[id]" this will be replaced by the id of the record currently created/updated.
• onvalidation has the same function as SQLFORM(..., onvalidation)
• onaccept is a function to be called after the form submission is accepted and acted upon, but before redirection.
• log is the log message. Log messages in CRUD see variables in the form.vars dictionary such as "%(id)s".
• message is the flash message upon form acceptance.
• ondelete is called in place of onaccept when a record is deleted via an "update" form.
• deletable determines whether the "update" form should have a delete option.
• query is the query to be used to select records.
• fields is a list of fields to be selected.
• orderby determines the order in which records should be selected (see Chapter 6).
• limitby determines the range of selected records that should be displayed (see Chapter 6).
• headers is a dictionary with the table header names.
• queries a list like ['equals', 'not equal', 'contains'] containing the allowed methods in the search form.
• query_labels a dictionary like query_labels=dict(equals='Equals') giving names to search methods.
• fields a list of fields to be listed in the search widget.
• field_labels a dictionary mapping field names into labels.
• zero defaults to "choose one" is used as default option for the drop-down in the search widget.
• showall set it to True if you want rows returned as per the query in the first call (added after 1.98.2).
• chkall set it to True to turn on all the checkboxes in the search form (added after 1.98.2).

Here is an example of usage in a single controller function:

## assuming db.define_table('person', Field('name'))
def people():
    form = crud.create(db.person, next=URL('index'),
           message=T("record created"))
    persons = crud.select(db.person, fields=['name'],
           headers={'person.name': 'Name'})
    return dict(form=form, persons=persons)

Here is another very generic controller function that lets you search, create and edit any records from any table where the tablename is passed request.args(0):

def manage():
    table=db[request.args(0)]
    form = crud.update(table,request.args(1))
    table.id.represent = lambda id, row:        A('edit:',id,_href=URL(args=(request.args(0),id)))
    search, rows = crud.search(table)
    return dict(form=form,search=search,rows=rows)

Notice the line table.id.represent=... that tells web2py to change the representation of the id field and display a link instead to the page itself and passes the id as request.args(1) which turns the create page into an update page.

Record versioning

Both SQLFORM and CRUD provides a utility to version database records:

If you have a table (db.mytable) that needs full revision history you can just do:

form = SQLFORM(db.mytable, myrecord).process(onsuccess=auth.archive)

form = crud.update(db.mytable, myrecord, onaccept=auth.archive)

auth.archive defines a new table called db.mytable_archive (the name is derived from the name of the table to which it refers) and on updating, it stores a copy of the record (as it was before the update) in the created archive table, including a reference to the current record.

Because the record is actually updated (only its previous state is archived), references are never broken.

This is all done under the hood. Should you wish to access the archive table you should define it in a model:

db.define_table('mytable_archive',
   Field('current_record',db.mytable),
   db.mytable)

Notice the table extends db.mytable (including all its fields), and adds a reference to the current_record.

auth.archive does not timestamp the stored record unless your original table has timestamp fields, for example:

db.define_table('mytable',
    Field('created_on','datetime',
          default=request.now,update=request.now,writable=False),
    Field('created_by',auth.user,
          default=auth.user_id,update=auth.user_id,writable=False),

There is nothing special about these fields and you may give them any name you like. They are filled before the record is archived and are archived with each copy of the record. The archive table name and/or reference field name can be changed like this:

db.define_table('myhistory',
    Field('parent_record',db.mytable),
    db.mytable)
## ...
form = SQLFORM(db.mytable,myrecord)
form.process(onsuccess = lambda form:auth.archive(form,
             archive_table=db.myhistory,
             current_record='parent_record'))

Custom forms

If a form is created with SQLFORM, SQLFORM.factory or CRUD, there are multiple ways it can be embedded in a view allowing multiple degrees of customization. Consider for example the following model:

db.define_table('image',
    Field('name'),
    Field('file', 'upload'))

and upload action

def upload_image():
    return dict(form=crud.create(db.image))

The simplest way to embed the form in the view for upload_image is

{{=form}}

This results in a standard table layout. If you wish to use a different layout, you can break the form into components

{{=form.custom.begin}}
Image name: <div>{{=form.custom.widget.name}}</div>
Image file: <div>{{=form.custom.widget.file}}</div>
Click here to upload: {{=form.custom.submit}}
{{=form.custom.end}}

where form.custom.widget[fieldname] gets serialized into the proper widget for the field. If the form is submitted and it contains errors, they are appended below the widgets, as usual.

The above sample form is show in the image below.

Notice that a similar result could have been obtained with:

crud.settings.formstyle='table2cols'

without using a custom form. Other possible formstyles are "table3cols" (the default), "divs" and "ul".

If you do not wish to use the widgets serialized by web2py, you can replace them with HTML. There are some variables that will be useful for this:

• form.custom.label[fieldname] contains the label for the field.
• form.custom.comment[fieldname] contains the comment for the field.
• form.custom.dspval[fieldname] form-type and field-type dependent display representation of the field.
• form.custom.inpval[fieldname] form-type and field-type dependent values to be used in field code.

It is important to follow the conventions described below.

CSS conventions

Tags in forms generated by SQLFORM, SQLFORM.factory and CRUD follow a strict CSS naming convention that can be used to further customize the forms.

Given a table "mytable", and a field "myfield" of type "string", it is rendered by default by a

SQLFORM.widgets.string.widget

that looks like this:

<input type="text" name="myfield" id="mytable_myfield"
       class="string" />

Notice that:

• the class of the INPUT tag is the same as the type of the field. This is very important for the jQuery code in "web2py_ajax.html" to work. It makes sure that you can only have numbers in "integer" and "double" fields, and that "time", "date" and "datetime" fields display the popup calendar/datepicker.
• the id is the name of the class plus the name of the field, joined by one underscore. This allows you to uniquely refer to the field via, for example, jQuery('#mytable_myfield') and manipulate the stylesheet of the field or bind actions associated to the field events (focus, blur, keyup, etc.).
• the name is, as you would expect, the field name.

Hide errors

Occasionally, you may want to disable the automatic error placement and display form error messages in some place other than the default. That can be done easily.

• In the case of FORM or SQLFORM, pass hideerror=True to the accepts method.
• In the case of CRUD, set crud.settings.hideerror=True

You may also want to modify the views to display the error (since they are no longer displayed automatically).

Here is an example where the errors are displayed above the form and not in the form.

{{if form.errors:}}
  Your submitted form contains the following errors:
  <ul>
  {{for fieldname in form.errors:}}
    <li>{{=fieldname}} error: {{=form.errors[fieldname]}}</li>
  {{pass}}
  </ul>
  {{form.errors.clear()}}
{{pass}}
{{=form}}

The errors will displayed as in the image shown below.

This mechanism also works for custom forms.

Validators

Validators are classes used to validate input fields (including forms generated from database tables).

Here is an example of using a validator with a FORM:

INPUT(_name='a', requires=IS_INT_IN_RANGE(0, 10))

Here is an example of how to require a validator for a table field:

db.define_table('person', Field('name'))
db.person.name.requires = IS_NOT_EMPTY()

Validators are always assigned using the requires attribute of a field. A field can have a single validator or multiple validators. Multiple validators are made part of a list:

db.person.name.requires = [IS_NOT_EMPTY(),
                           IS_NOT_IN_DB(db, 'person.name')]

Normally validators are called automatically by the function accepts and process of a FORM or other HTML helper object that contains a form. They are called in the order in which they are listed.

One can also call validators explicitly for a field:

db.person.name.validate(value)

which returns a tuple (value,error) and error is None if no the value validates.

Built-in validators have constructors that take an optional argument:

IS_NOT_EMPTY(error_message='cannot be empty')

error_message allows you to override the default error message for any validator.

Here is an example of a validator on a database table:

db.person.name.requires = IS_NOT_EMPTY(error_message='fill this!')

where we have used the translation operator T to allow for internationalization. Notice that default error messages are not translated.

Mind that the only validators that can be used with list: type fields are:

• IS_IN_DB(...,multiple=True)
• IS_IN_SET(...,multiple=True)
• IS_NOT_EMPTY()
• IS_LIST_OF(...)

The latter can be used to apply any validator to the individual items in the list.

Validators

IS_ALPHANUMERIC

This validator checks that a field value contains only characters in the ranges a-z, A-Z, or 0-9.

requires = IS_ALPHANUMERIC(error_message='must be alphanumeric!')

IS_DATE

This validator checks that a field value contains a valid date in the specified format. It is good practice to specify the format using the translation operator, in order to support different formats in different locales.

requires = IS_DATE(format=T('%Y-%m-%d'),
                   error_message='must be YYYY-MM-DD!')

For the full description on % directives look under the IS_DATETIME validator.

IS_DATE_IN_RANGE

Works very much like the previous validator but allows to specify a range:

requires = IS_DATE_IN_RANGE(format=T('%Y-%m-%d'),
                   minimum=datetime.date(2008,1,1),
                   maximum=datetime.date(2009,12,31),
                   error_message='must be YYYY-MM-DD!')

For the full description on % directives look under the IS_DATETIME validator.

IS_DATETIME

This validator checks that a field value contains a valid datetime in the specified format. It is good practice to specify the format using the translation operator, in order to support different formats in different locales.

requires = IS_DATETIME(format=T('%Y-%m-%d %H:%M:%S'),
                       error_message='must be YYYY-MM-DD HH:MM:SS!')

The following symbols can be used for the format string (this shows the symbol and an example string):

%Y  '1963'
%y  '63'
%d  '28'
%m  '08'
%b  'Aug'
%b  'August'
%H  '14'
%I  '02'
%p  'PM'
%M  '30'
%S  '59'

IS_DATETIME_IN_RANGE

Works very much like the previous validator but allows to specify a range:

requires = IS_DATETIME_IN_RANGE(format=T('%Y-%m-%d %H:%M:%S'),
                       minimum=datetime.datetime(2008,1,1,10,30),
                       maximum=datetime.datetime(2009,12,31,11,45),
                       error_message='must be YYYY-MM-DD HH:MM::SS!')

For the full description on % directives look under the IS_DATETIME validator.

IS_DECIMAL_IN_RANGE

INPUT(_type='text', _name='name', requires=IS_DECIMAL_IN_RANGE(0, 10, dot="."))

It converts the input into a Python Decimal or generates an error if the decimal does not fall within the specified inclusive range. The comparison is made with Python Decimal arithmetic.

The minimum and maximum limits can be None, meaning no lower or upper limit, respectively.

The dot argument is optional and allows you to internationalize the symbol used to separate the decimals.

IS_EMAIL

It checks that the field value looks like an email address. It does not try to send email to confirm.

requires = IS_EMAIL(error_message='invalid email!')

IS_EQUAL_TO

Checks whether the validated value is equal to a given value (which can be a variable):

requires = IS_EQUAL_TO(request.vars.password,
                       error_message='passwords do not match')

IS_EXPR

Its first argument is a string containing a logical expression in terms of a variable value. It validates a field value if the expression evaluates to True. For example:

requires = IS_EXPR('int(value)%3==0',
                   error_message='not divisible by 3')

One should first check that the value is an integer so that an exception will not occur.

requires = [IS_INT_IN_RANGE(0, 100), IS_EXPR('value%3==0')]

IS_FLOAT_IN_RANGE

Checks that the field value is a floating point number within a definite range, 0 <= value <= 100 in the following example:

requires = IS_FLOAT_IN_RANGE(0, 100, dot=".",
         error_message='too small or too large!')

The dot argument is optional and allows you to internationalize the symbol used to separate the decimals.

IS_INT_IN_RANGE

Checks that the field value is an integer number within a definite range, 0 <= value < 100 in the following example:

requires = IS_INT_IN_RANGE(0, 100,
         error_message='too small or too large!')

IS_IN_SET

Checks that the field values are in a set:

requires = IS_IN_SET(['a', 'b', 'c'],zero=T('choose one'),
         error_message='must be a or b or c')

The zero argument is optional and it determines the text of the option selected by default, an option which is not accepted by the IS_IN_SET validator itself. If you do not want a "choose one" option, set zero=None.

The zero option was introduced in revision (1.67.1). It did not break backward compatibility in the sense that it did not break applications but it did change their behavior since, before, there was no zero option.

The elements of the set must always be strings unless this validator is preceded by IS_INT_IN_RANGE (which converts the value to int) or IS_FLOAT_IN_RANGE (which converts the value to float). For example:

requires = [IS_INT_IN_RANGE(0, 8), IS_IN_SET([2, 3, 5, 7],
          error_message='must be prime and less than 10')]

You may also use a dictionary or a list of tuples to make the drop down list more descriptive:

#### Dictionary example:
requires = IS_IN_SET({'A':'Apple','B':'Banana','C':'Cherry'},zero=None)
#### List of tuples example:
requires = IS_IN_SET([('A','Apple'),('B','Banana'),('C','Cherry')])

IS_IN_SET and Tagging

The IS_IN_SET validator has an optional attribute multiple=False. If set to True, multiple values can be stored in one field. The field should be of type list:integer or list:string. multiple references are handled automatically in create and update forms, but they are transparent to the DAL. We strongly suggest using the jQuery multiselect plugin to render multiple fields.

Note that when multiple=True, IS_IN_SET will accept zero or more values, i.e. it will accept the field when nothing has been selected. multiple can also be a tuple of the form (a,b) where a and b are the minimum and (exclusive) maximum number of items that can be selected respectively.

IS_LENGTH

Checks if length of field's value fits between given boundaries. Works for both text and file inputs.

Its arguments are:

• maxsize: the maximum allowed length / size (has default = 255)
• minsize: the minimum allowed length / size

Examples: Check if text string is shorter than 33 characters:

INPUT(_type='text', _name='name', requires=IS_LENGTH(32))

Check if password string is longer than 5 characters:

INPUT(_type='password', _name='name', requires=IS_LENGTH(minsize=6))

Check if uploaded file has size between 1KB and 1MB:

INPUT(_type='file', _name='name', requires=IS_LENGTH(1048576, 1024))

For all field types except for files, it checks the length of the value. In the case of files, the value is a cookie.FieldStorage, so it validates the length of the data in the file, which is the behavior one might intuitively expect.

IS_LIST_OF

This is not properly a validator. Its intended use is to allow validations of fields that return multiple values. It is used in those rare cases when a form contains multiple fields with the same name or a multiple selection box. Its only argument is another validator, and all it does is to apply the other validator to each element of the list. For example, the following expression checks that every item in a list is an integer in the range 0-10:

requires = IS_LIST_OF(IS_INT_IN_RANGE(0, 10))

It never returns an error and does not contain an error message. The inner validator controls the error generation.

IS_LOWER

This validator never returns an error. It just converts the value to lower case.

requires = IS_LOWER()

IS_MATCH

This validator matches the value against a regular expression and returns an error if it does not match. Here is an example of usage to validate a US zip code:

requires = IS_MATCH('^\d{5}(-\d{4})?$',
         error_message='not a zip code')

Here is an example of usage to validate an IPv4 address (note: the IS_IPV4 validator is more appropriate for this purpose):

requires = IS_MATCH('^\d{1,3}(.\d{1,3}){3}$',
         error_message='not an IP address')

Here is an example of usage to validate a US phone number:

requires = IS_MATCH('^1?((-)\d{3}-?|\(\d{3}\))\d{3}-?\d{4}$',
         error_message='not a phone number')

For more information on Python regular expressions, refer to the official Python documentation.

IS_MATCH takes an optional argument strict which defaults to False. When set to True it only matches the beginning of the string:

>>> IS_MATCH('a')('ba')
('ba', <lazyT 'invalid expression'>) # no pass
>>> IS_MATCH('a',strict=False)('ab')
('a', None)                          # pass!

IS_MATCH takes an other optional argument search which defaults to False. When set to True, it uses regex method search instead of method match to validate the string.

IS_NOT_EMPTY

This validator checks that the content of the field value is not an empty string.

requires = IS_NOT_EMPTY(error_message='cannot be empty!')

IS_TIME

This validator checks that a field value contains a valid time in the specified format.

requires = IS_TIME(error_message='must be HH:MM:SS!')

IS_URL

Rejects a URL string if any of the following is true:

• The string is empty or None
• The string uses characters that are not allowed in a URL
• The string breaks any of the HTTP syntactic rules
• The URL scheme specified (if one is specified) is not 'http' or 'https'
• The top-level domain (if a host name is specified) does not exist

(These rules are based on RFC 2616^[RFC2616] )

This function only checks the URL's syntax. It does not check that the URL points to a real document, for example, or that it otherwise makes semantic sense. This function does automatically prepend 'http://' in front of a URL in the case of an abbreviated URL (e.g. 'google.ca').

If the parameter mode='generic' is used, then this function's behavior changes. It then rejects a URL string if any of the following is true:

• The string is empty or None
• The string uses characters that are not allowed in a URL
• The URL scheme specified (if one is specified) is not valid

(These rules are based on RFC 2396^[RFC2396] )

The list of allowed schemes is customizable with the allowed_schemes parameter. If you exclude None from the list, then abbreviated URLs (lacking a scheme such as 'http') will be rejected.

The default prepended scheme is customizable with the prepend_scheme parameter. If you set prepend_scheme to None, then prepending will be disabled. URLs that require prepending to parse will still be accepted, but the return value will not be modified.

IS_URL is compatible with the Internationalized Domain Name (IDN) standard specified in RFC 3490^[RFC3490] ). As a result, URLs can be regular strings or unicode strings. If the URL's domain component (e.g. google.ca) contains non-US-ASCII letters, then the domain will be converted into Punycode (defined in RFC 3492^[RFC3492] ). IS_URL goes a bit beyond the standards, and allows non-US-ASCII characters to be present in the path and query components of the URL as well. These non-US-ASCII characters will be encoded. For example, space will be encoded as'%20'. The unicode character with hex code 0x4e86 will become '%4e%86'.

Examples:

requires = IS_URL())
requires = IS_URL(mode='generic')
requires = IS_URL(allowed_schemes=['https'])
requires = IS_URL(prepend_scheme='https')
requires = IS_URL(mode='generic',
                  allowed_schemes=['ftps', 'https'],
                  prepend_scheme='https')

IS_SLUG

requires = IS_SLUG(maxlen=80, check=False, error_message='must be slug')

If check is set to True it check whether the validated value is a slug (allowing only alphanumeric characters and non-repeated dashes).

If check is set to False (default) it converts the input value to a slug.

IS_STRONG

Enforces complexity requirements on a field (usually a password field)

Example:

requires = IS_STRONG(min=10, special=2, upper=2)

where

• min is minimum length of the value
• special is the minimum number of required special characters special characters are any of the following !@#$%^&*(){}[]-+
• upper is the minimum number of upper case characters

IS_IMAGE

This validator checks if a file uploaded through the file input was saved in one of the selected image formats and has dimensions (width and height) within given limits.

It does not check for maximum file size (use IS_LENGTH for that). It returns a validation failure if no data was uploaded. It supports the file formats BMP, GIF, JPEG, PNG, and it does not require the Python Imaging Library.

Code parts taken from ref.^[source1]

It takes the following arguments:

• extensions: iterable containing allowed image file extensions in lowercase
• maxsize: iterable containing maximum width and height of the image
• minsize: iterable containing minimum width and height of the image

Use (-1, -1) as minsize to bypass the image-size check.

Here are some Examples:

• Check if uploaded file is in any of supported image formats:

requires = IS_IMAGE()

• Check if uploaded file is either JPEG or PNG:

requires = IS_IMAGE(extensions=('jpeg', 'png'))

• Check if uploaded file is PNG with maximum size of 200x200 pixels:

requires = IS_IMAGE(extensions=('png'), maxsize=(200, 200))

• Note: on displaying an edit form for a table including requires = IS_IMAGE(), a delete checkbox will NOT appear because to delete the file would cause the validation to fail. To display the delete checkbox use this validation:

requires = IS_EMPTY_OR(IS_IMAGE())

IS_UPLOAD_FILENAME

This validator checks if the name and extension of a file uploaded through the file input matches the given criteria.

It does not ensure the file type in any way. Returns validation failure if no data was uploaded.

Its arguments are:

• filename: filename (before dot) regex.
• extension: extension (after dot) regex.
• lastdot: which dot should be used as a filename / extension separator: True indicates last dot (e.g., "file.tar.gz" will be broken in "file.tar" + "gz") while False means first dot (e.g., "file.tar.gz" will be broken into "file" + "tar.gz").
• case: 0 means keep the case; 1 means transform the string into lowercase (default); 2 means transform the string into uppercase.

If there is no dot present, extension checks will be done against an empty string and filename checks will be done against the whole value.

Examples:

Check if file has a pdf extension (case insensitive):

requires = IS_UPLOAD_FILENAME(extension='pdf')

Check if file has a tar.gz extension and name starting with backup:

requires = IS_UPLOAD_FILENAME(filename='backup.*', extension='tar.gz', lastdot=False)

Check if file has no extension and name matching README (case sensitive):

requires = IS_UPLOAD_FILENAME(filename='^README$', extension='^$', case=0)

IS_IPV4

This validator checks if a field's value is an IP version 4 address in decimal form. Can be set to force addresses from a certain range.

IPv4 regex taken from ref.^[regexlib] Its arguments are:

• minip lowest allowed address; accepts: str, e.g., 192.168.0.1; iterable of numbers, e.g., [192, 168, 0, 1]; int, e.g., 3232235521
• maxip highest allowed address; same as above

All three example values are equal, since addresses are converted to integers for inclusion check with following function:

number = 16777216 * IP[0] + 65536 * IP[1] + 256 * IP[2] + IP[3]

Examples:

Check for valid IPv4 address:

requires = IS_IPV4()

Check for valid private network IPv4 address:

requires = IS_IPV4(minip='192.168.0.1', maxip='192.168.255.255')

IS_LOWER

This validator never returns an error. It converts the value to lower case.

requires = IS_LOWER()

IS_UPPER

This validator never returns an error. It converts the value to upper case.

requires = IS_UPPER()

IS_NULL_OR

Deprecated, an alias for IS_EMPTY_OR described below.

IS_EMPTY_OR

Sometimes you need to allow empty values on a field along with other requirements. For example a field may be a date but it can also be empty. The IS_EMPTY_OR validator allows this:

requires = IS_EMPTY_OR(IS_DATE())

CLEANUP

This is a filter. It never fails. It just removes all characters whose decimal ASCII codes are not in the list [10, 13, 32-127].

requires = CLEANUP()

CRYPT

This is also a filter. It performs a secure hash on the input and it is used to prevent passwords from being passed in the clear to the database.

requires = CRYPT()

If a key is not specified, it uses the MD5 algorithm. If a key is specified CRYPT uses the HMAC algorithm. The key may contain a prefix that determines the algorithm to use with HMAC, for example SHA512:

requires = CRYPT(key='sha512:thisisthekey')

This is the recommended syntax. The key has to be a unique string associated to the database used. The key can never be changed. If you lose the key the previously hashed values become useless.

The CRYPT validator hashed the input and this makes it somewhat special. If you need to validate a password field, before it is hash, you can use CRYPT in a list of validators but must make sure it is the last of the list so that it is called last. For example:

requires = [IS_STRONG(),CRYPT(key='sha512:thisisthekey')]

CRYPT also takes a min_length argument which defaults to zero.

Database validators

IS_NOT_IN_DB

Consider the following example:

db.define_table('person', Field('name'))
db.person.name.requires = IS_NOT_IN_DB(db, 'person.name')

It requires that when you insert a new person, his/her name is not already in the database, db, in the field person.name. As with all other validators this requirement is enforced at the form processing level, not at the database level. This means that there is a small probability that, if two visitors try to concurrently insert records with the same person.name, this results in a race condition and both records are accepted. It is therefore safer to also inform the database that this field should have a unique value:

db.define_table('person', Field('name', unique=True))
db.person.name.requires = IS_NOT_IN_DB(db, 'person.name')

Now if a race condition occurs, the database raises an OperationalError and one of the two inserts is rejected.

The first argument of IS_NOT_IN_DB can be a database connection or a Set. In the latter case, you would be checking only the set defined by the Set.

The following code, for example, does not allow registration of two persons with the same name within 10 days of each other:

import datetime
now = datetime.datetime.today()
db.define_table('person',
    Field('name'),
    Field('registration_stamp', 'datetime', default=now))
recent = db(db.person.registration_stamp>now-datetime.timedelta(10))
db.person.name.requires = IS_NOT_IN_DB(recent, 'person.name')

IS_IN_DB

Consider the following tables and requirement:

db.define_table('person', Field('name', unique=True))
db.define_table('dog', Field('name'), Field('owner', db.person)
db.dog.owner.requires = IS_IN_DB(db, 'person.id', '%(name)s',
                                 zero=T('choose one'))

It is enforced at the level of dog INSERT/UPDATE/DELETE forms. It requires that a dog.owner be a valid id in the field person.id in the database db. Because of this validator, the dog.owner field is represented as a dropbox. The third argument of the validator is a string that describes the elements in the dropbox. In the example you want to see the person %(name)s instead of the person %(id)s. %(...)s is replaced by the value of the field in brackets for each record.

The zero option works very much like for the IS_IN_SET validator.

The first argument of the validator can be a database connection or a DAL Set, as in IS_NOT_IN_DB. This can be useful for example when wishing to limit the records in the drop-down box. In this example, we use IS_IN_DB in a controller to limit the records dynamically each time the controller is called:

def index():
    (...)
    query = (db.table.field == 'xyz') #in practice 'xyz' would be a variable
    db.table.field.requires=IS_IN_DB(db(query),....)
    form=SQLFORM(...)
    if form.process().accepted: ...
    (...)

If you want the field validated, but you do not want a dropbox, you must put the validator in a list.

db.dog.owner.requires = [IS_IN_DB(db, 'person.id', '%(name)s')]

Occasionally you want the drop-box (so you do not want to use the list syntax above) yet you want to use additional validators. For this purpose the IS_IN_DB validator takes an extra argument _and that can point to a list of other validators applied if the validated value passes the IS_IN_DB validation. For example to validate all dog owners in db that are not in a subset:

subset=db(db.person.id>100)
db.dog.owner.requires = IS_IN_DB(db, 'person.id', '%(name)s',
                                 _and=IS_NOT_IN_DB(subset,'person.id'))

IS_IN_DB also takes a cache argument that works like the cache argument of select.

IS_IN_DB and Tagging

The IS_IN_DB validator has an optional attribute multiple=False. If set to True multiple values can be stored in one field. This field should be of type list:reference as discussed in Chapter 6. An explicit example of tagging is discussed there. multiple references are handled automatically in create and update forms, but they are transparent to the DAL. We strongly suggest using the jQuery multiselect plugin to render multiple fields.

Custom validators

All validators follow the prototype below:

class sample_validator:
    def __init__(self, *a, error_message='error'):
        self.a = a
        self.e = error_message
    def __call__(self, value):
        if validate(value):
            return (parsed(value), None)
        return (value, self.e)
    def formatter(self, value):
        return format(value)

i.e., when called to validate a value, a validator returns a tuple (x, y). If y is None, then the value passed validation and x contains a parsed value. For example, if the validator requires the value to be an integer, x is converted to int(value). If the value did not pass validation, then x contains the input value and y contains an error message that explains the failed validation. This error message is used to report the error in forms that do not validate.

The validator may also contain a formatter method. It must perform the opposite conversion to the one the __call__ does. For example, consider the source code for IS_DATE:

class IS_DATE(object):
    def __init__(self, format='%Y-%m-%d', error_message='must be YYYY-MM-DD!'):
        self.format = format
        self.error_message = error_message
    def __call__(self, value):
        try:
            y, m, d, hh, mm, ss, t0, t1, t2 = time.strptime(value, str(self.format))
            value = datetime.date(y, m, d)
            return (value, None)
        except:
            return (value, self.error_message)
    def formatter(self, value):
        return value.strftime(str(self.format))

On success, the __call__ method reads a date string from the form and converts it into a datetime.date object using the format string specified in the constructor. The formatter object takes a datetime.date object and converts it to a string representation using the same format. The formatter is called automatically in forms, but you can also call it explicitly to convert objects into their proper representation. For example:

>>> db = DAL()
>>> db.define_table('atable',
       Field('birth', 'date', requires=IS_DATE('%m/%d/%Y')))
>>> id = db.atable.insert(birth=datetime.date(2008, 1, 1))
>>> row = db.atable[id]
>>> print db.atable.formatter(row.birth)
01/01/2008

When multiple validators are required (and stored in a list), they are executed in order and the output of one is passed as input to the next. The chain breaks when one of the validators fails.

Conversely, when we call the formatter method of a field, the formatters of the associated validators are also chained, but in reverse order.

Notice that as alternative to custom validators, you can also use the onvalidate argument of form.accepts(...), form.process(...) and form.validate(...).

Validators with dependencies

Usually validators are set once for all in models.

Occasionally, you need to validate a field and the validator depends on the value of another field. This can be done in various ways. It can be done in the model or in the controller.

For example, here is a page that generates a registration form that asks for username and password twice. None of the fields can be empty, and both passwords must match:

def index():
    form = SQLFORM.factory(
        Field('username', requires=IS_NOT_EMPTY()),
        Field('password', requires=IS_NOT_EMPTY()),
        Field('password_again',
              requires=IS_EQUAL_TO(request.vars.password)))
    if form.process().accepted:
        pass # or take some action
    return dict(form=form)

The same mechanism can be applied to FORM and SQLFORM objects.

Widgets

Here is a list of available web2py widgets:

SQLFORM.widgets.string.widget
SQLFORM.widgets.text.widget
SQLFORM.widgets.password.widget
SQLFORM.widgets.integer.widget
SQLFORM.widgets.double.widget
SQLFORM.widgets.time.widget
SQLFORM.widgets.date.widget
SQLFORM.widgets.datetime.widget
SQLFORM.widgets.upload.widget
SQLFORM.widgets.boolean.widget
SQLFORM.widgets.options.widget
SQLFORM.widgets.multiple.widget
SQLFORM.widgets.radio.widget
SQLFORM.widgets.checkboxes.widget
SQLFORM.widgets.autocomplete

The first ten of them are the defaults for the corresponding field types. The "options" widget is used when a field's requires is IS_IN_SET or IS_IN_DB with multiple=False (default behavior). The "multiple" widget is used when a field's requires is IS_IN_SET or IS_IN_DB with multiple=True. The "radio" and "checkboxes" widgets are never used by default, but can be set manually. The autocomplete widget is special and discussed in its own section.

For example, to have a "string" field represented by a textarea:

Field('comment', 'string', widget=SQLFORM.widgets.text.widget)

Widgets can also be assigned to fields a posteriori:

db.mytable.myfield.widget = SQLFORM.widgets.string.widget

Sometimes widgets take additional arguments and one needs to specify their values. In this case one can use lambda

db.mytable.myfield.widget = lambda field,value:     SQLFORM.widgets.string.widget(field,value,_style='color:blue')

Widgets are helper factories and their first two arguments are always field and value. The other arguments can include normal helper attributes such as _style, _class, etc. Some widgets also take special arguments. In particular SQLFORM.widgets.radio and SQLFORM.widgets.checkboxes take a style argument (not to be confused with _style) which can be set to "table", "ul", or "divs" in order to match the formstyle of the containing form.

You can create new widgets or extend existing widgets.

SQLFORM.widgets[type] is a class and SQLFORM.widgets[type].widget is a static member function of the corresponding class. Each widget function takes two arguments: the field object, and the current value of that field. It returns a representation of the widget. As an example, the string widget could be recoded as follows:

def my_string_widget(field, value):
    return INPUT(_name=field.name,
                 _id="%s_%s" % (field._tablename, field.name),
                 _class=field.type,
                 _value=value,
                 requires=field.requires)

Field('comment', 'string', widget=my_string_widget)

The id and class values must follow the convention described later in this chapter. A widget may contain its own validators, but it is good practice to associate the validators to the "requires" attribute of the field and have the widget get them from there.

Autocomplete widget

There are two possible uses for the autocomplete widget: to autocomplete a field that takes a value from a list or to autocomplete a reference field (where the string to be autocompleted is a representation of the reference which is implemented as an id).

The first case is easy:

db.define_table('category',Field('name'))
db.define_table('product',Field('name'),Field('category'))
db.product.category.widget = SQLFORM.widgets.autocomplete(
     request, db.category.name, limitby=(0,10), min_length=2)

Where limitby instructs the widget to display no more than 10 suggestions at the time, and min_length instructs the widget to perform an Ajax callback to fetch suggestions only after the user has typed at least 2 characters in the search box.

The second case is more complex:

db.define_table('category',Field('name'))
db.define_table('product',Field('name'),Field('category'))
db.product.category.widget = SQLFORM.widgets.autocomplete(
     request, db.category.name, id_field=db.category.id)

In this case the value of id_field tells the widget that even if the value to be autocompleted is a db.category.name, the value to be stored is the corresponding db.category.id. An optional parameter is orderby that instructs the widget on how to sort the suggestions (alphabetical by default).

This widget works via Ajax. Where is the Ajax callback? Some magic is going on in this widget. The callback is a method of the widget object itself. How is it exposed? In web2py any piece of code can generate a response by raising an HTTP exception. This widget exploits this possibility in the following way: the widget sends the Ajax call to the same URL that generated the widget in the first place and puts a special token in the request.vars. Should the widget get instantiated again, it finds the token and raises an HTTP exception that responds to the request. All of this is done under the hood and hidden to the developer.

SQLFORM.grid and SQLFORM.smartgrid (experimental)

These are two high level gadgets that create complex CRUD controls. They provide pagination, the ability to browser, search, sort, create, update and delete records from a single gadgets.

The simplest of the two is SQLFORM.grid. Here is an example of usage:

@auth.requires_login()
def manage_users():
    grid = SQLFORM.grid(db.auth_user)
    return locals()

which produces the following page:

The first argument of SQLFORM.grid can be a table or a query. The grid gadget will provide access to records matching the query.

Before we dive into the long list of arguments of the grid gadget we need to understand how it works. The gadget looks at request.args in order to decide what to do (browse, search, create, update, delete, etc.). Each button created by the gadget links the same function (manage_users in the above case) but passes different request.args. By default all the URL generated by the grid are digitally signed and verified. This means one cannot perform certain actions (create, update, delete) without being logged-in. These restrictions can be relaxed:

def manage_users():
    grid = SQLFORM.grid(db.auth_user,user_signature=False)
    return locals()

but we do not recommend it.

Because of the way grid works one can only have one grid per controller function, unless they are embedded as components via LOAD.

Because the function that contains the grid may itself manipulate the command line arguments, the grid needs to know which args should be handled by the grid and which not. For example here is an example of code that allows one to manage any table:

@auth.requires_login()
def manage():
    table = request.args(0)
    if not table in db.tables(): redirect(URL('error'))
    grid = SQLFORM.grid(db[table],args=request.args[:1])
    return locals()

the args argument of the grid specifies which request.args should be passed along and ignored by the gadget. In our case request.args[:1] is the name of the table we want to manage and it is handled by the manage function itself, not by the gadget.

The complete signature for the grid is the following:

SQLFORM.grid(query,
             fields=None,
             field_id=None,
             left=None,
             headers={},
             orderby=None,
             searchable=True,
             sortable=True,
             deletable=True,
             editable=True,
             details=True,
             create=True,
             csv=True,
             paginate=20,
             selectable=None,
             links=None,
             upload = '<default>',
             args=[],
             user_signature = True,
             maxtextlengths={},
             maxtextlength=20,
             onvalidation=None,
             oncreate=None,
             onupdate=None,
             ondelete=None,
             sorter_icons=('[^]','[v]'),
             ui = 'web2py',
             showbuttontext=True,
             search_widget='default',
             _class="web2py_grid",
             formname='web2py_grid',
             ignore_rw = False,
             formstyle = 'table3cols'):

• fields is a list of fields to be fetched from the database. It is also used to determine which fields to be shown in the grid view.
• field_id must be the field of the table to be used as ID, for example db.mytable.id.
• headers is a dictionary that maps 'tablename.fieldname' into the corresponding header label.
• left is an optional left join expressions used to build ...select(left=...).
• orderby is used as default ordering for the rows.
• searchable, sortable, deletable, details, create determine whether one can search, sort, delete, view details, and create new records respectively.
• csv if set to true allows to download the grid in CSV.
• paginate sets the max number of rows per page.
• links is used to display new columns which can be links to other pages. The links argument must be a list of dict(header='name',body=lambda row: A(...)) where header is the header of the new column and body is a function that takes a row and returns a value. In the example, the value is a A(...) helper.
• maxtextlength sets the maximum length of text to be displayed for each field value, in the grid view. This value can be overwritten for each field using maxtextlengths, a dictionary of 'tablename.fieldname':length.
• onvalidation, oncreate, onupdate and ondelete are callback functions. All but ondelete take a form object as input.
• sorter_icons is a list of two strings (or helpers) that will be used to represent the up and down sorting options for each field.
• ui can be set equal to 'web2py' and will generate web2py friendly class names, can be set equal to jquery-ui and will generate jquery UI friendly class names, but it can also be its own set of class names for the various grid components:

ui = dict(widget='',
          header='',
          content='',
          default='',
          cornerall='',
          cornertop='',
          cornerbottom='',
          button='button',
          buttontext='buttontext button',
          buttonadd='icon plus',
          buttonback='icon leftarrow',
          buttonexport='icon downarrow',
          buttondelete='icon trash',
          buttonedit='icon pen',
          buttontable='icon rightarrow',
          buttonview='icon magnifier')

• search_widget allows to override the default search widget and we refer the reader the source code in "gluon/sqlhtml.py" for details.
• showbutton allows to turn off all buttons.
• _class is the class for the grid container.
• formname, ignore_rw and formstyle are passed to the SQLFORM objects used by the grid for create/update forms.

deletable, editable and details are usually boolean values but they can be functions which take the row object and decide whether to display the corrsponding button or not.

A SQLFORM.smartgrid looks a lot like a grid, in fact it contains a grid but it is designed to take as input not a query but only one table and to browse said table and selected referencing tables.

For example consider the following table structure:

db.define_table('parent',Field('name'))
db.define_table('child',Field('name'),Field('parent','reference parent'))

With SQLFORM.grid you can list all parents:

SQLFORM.grid(db.parent)

all children:

SQLFORM.grid(db.child)

and all parents and children in one table:

SQLFORM.grid(db.parent,left=db.child.on(db.child.parent=db.parent.id))

With SQLFORM.smartgrid you can put all the data in one gadget that spawns both tables:

@auth.requires_login():
def manage():
    grid = SQLFORM.smartgrid(db.parent,linked_tables=['child'])
    return locals()

which looks like this:

Notice the extra "children" links. One could create the extra links using a regular grid but they would point to a different action. With a smartgrid they are created automatically and handled by the same gadget.

Also notice that when clicking on the "children" link for a given parent one only gets the list of children for that parent (and that is obvious) but also notice that if one now tried to add a new child, the parent value for the new child is automatically set to the selected parent (displayed in the breadcrumbs associated to the gadget). The value of this field can be overwritten. We can prevent this by making it readonly:

@auth.requires_login():
def manage():
    db.child.parent.writable = False
    grid = SQLFORM.smartgrid(db.parent,linked_tables=['child'])
    return locals()

If the linked_tables argument is not specified all referencing tables are automatically linked. Anyway, to avoid accidentally exposing data we recommend explicitly listing tables that should be linked.

The following code creates a very powerful management interface for all tables in the system:

@auth.requires_membership('managers'):
def manage():
    table = request.args(0) or 'auth_user'
    if not table in db.tables(): redirect(URL('error'))
    grid = SQLFORM.smartgrid(db[table],args=request.args[:1])
    return locals()

The smartgrid takes the same arguments as a grid and some more with some caveats:

• The first argument is a table, not a query
• There is a extra argument constraints which is a dictionary of 'tablename':query which can be used to further restrict access to the records displayed in the 'tablename' grid.
• There is a extra argument linked_tables which is a list of tablenames of tables that should be accessible via the smartgrid.
• All the arguments but the table, args, linked_tables and user_signatures can be dictionaries as explained below.

Consider the previous grid:

grid = SQLFORM.smartgrid(db.parent,linked_tables=['child'])

It allows one to access both a db.parent and a db.child. Apart for navigation controls, for each one table, a smarttable is nothing but a grid. This means that, in this case, one smartgrid can create a grid for parent and one grid for child. We may want to pass different sets of parameters to these grids. For example different sets of searchable parameters.

While for a grid we would pass a boolean:

grid = SQLFORM.grid(db.parent,searchable=True)

for a smartgrid we would pass a dictionary of booleans:

grid = SQLFORM.smartgrid(db.parent,linked_tables=['child'],
     searchable= dict(parent=True, child=False))

In this way we made parents searchable but children for each parent not searchable (there should not be that many to need the search widget).

The grid and smartgrid gadgets are here to stay but they are marked experimental because the actual html layout of what they return and the exact set of parameters one can pass to them may be subject to change as new functionalities are added.

grid and smartgrid do not automatically enforce access control like crud does but you can integrate it with auth using explicit permission checking:

grid = SQLFORM.grid(db.auth_user,
     editable = auth.has_membership('managers'),
     deletable = auth.has_membership('managers'))

or

grid = SQLFORM.grid(db.auth_user,
     editable = auth.has_permission('edit','auth_user'),
     deletable = auth.has_permission('delete','auth_user'))

The smartgrid is the only gadget in web2py that displays the table name and it need both the singular and the plural. For example one parent can have one "Child" or many "Children". Therefore a table object needs to know its own singular and plural names. web2py normally guesses them but you can set the explicitly:

db.define_table('child', ..., singular="Child", plural="Children")

or with:

db.define_table('child', ...)
db.child._singular = "Child"
db.child._plural = "Children"

They should also be internationalized using the T operator.

The plural and singular values are then used by smartgrid to provide correct names for headers and links.