.. tutorial_index:

Tutorial
========

Introduction
------------

Gone are the days when web development was an easy quick buck. Today's web developers face a number of conflicting pressures. Modern websites are expected to deliver advanced functionality while being reliable, user-friendly and cross-browser. At the same time, developers need to increase productivity and reduce development times, to remain competitve. This situation has spawned a huge variety of web development platforms that aim to deliver solid high-level functionality, in a manner convenient for the developer. An interesting library for Python coders is ToscaWidgets, and will explore here how it can help you rapidly build useful web applications.

ToscaWidgets [1] is a framework for creating re-usable web components, called "widgets". A widget can contain an HTML template, JavaScript and CSS resources, Python code, validation information and more. There are several libraries of useful ToscaWidgets available, offering features such as popup calenders, rich text editors and ajax lookups. In this article, we'll be looking particularly at tw.dynforms. ToscaWidgets is purely a view component, not a complete MVC framework, so we'll use TurboGears to provide the other MVC components.

To demonstrate what tw.dynforms can do, we're going to walk through creating an internal business application for order tracking. This example would not be customer facing - think of call agents taking phone orders and keying them into an intranet application. ToscaWidgets and tw.dynforms certainly can be used for internet-facing applications; an intranet application is used just to keep the example simple.


Getting Started
---------------

The best way to install the required packages is using easy_install. Assuming you already have easy_install, you'll need to:

.. code-block:: python

    easy_install tw.dynforms

To start building our application, we'll use the TG quickstart facility, which creates a skeleton for the project. We'll use the -e switch to indicate the project will use Elixir as the database library. Issue:

.. code-block:: python

    tg-admin quickstart -e myproj
    
It will raise several prompts, and I suggest you just accept the defaults. This will create a directory structure under "myproj" (or whatever name you choose) with the skeleton files needed for a basic TurboGears application. We now need to register the directory with setup tools, so that python modules is any directory can use "import myproj". Issue:

.. code-block:: python

    cd myproj
    python setup.py develop
    
Inside the myproj directory are a number of script and configuration files, and also another "myproj" subdirectory. The top level directory contains files that you'll usually only make minor edits to, while the inner directory contains your source code. As we're using ToscaWidgets in the project, we need to enable this. Edit dev.cfg (in the top level directory) and add the following line to the [global] section:

.. code-block:: python

    toscawidgets.on = True

We'll now start the application to check that it works correctly. Issue:
    
.. code-block:: python

    python start-myproj.py

In a web browser, go to http://localhost:8080 If everything worked ok, this will present a TurboGears welcome screen. 


Model
-----

The first step is to define the database tables. We'll use Elixir as our object-relational mapper; this is an active record style ORM that builds on SQLAlchemy. Add the following to model.py (in the inner directory):

.. code-block:: python

    class People(Entity):
        name = Field(String(100))
        email = Field(String(100))
        def __str__(self):
            return self.name

    class Status(Entity):
        name = Field(String(100))
        def __str__(self):
            return self.name

    class Order(Entity):
        name = Field(String(100))
        status = ManyToOne(Status)
        customer = ManyToOne(People)
        assignee = ManyToOne(People)
        delivery = Field(Boolean)
        address = Field(String(200))
    
    setup_all()

The next step is to actually create the database tables. By default, TurboGears connects the application to an sqlite database, which will work fine for us. Issue:

.. code-block:: python

    tg-admin sql create

We also want to create some test orders, so we have some data to work with later on:

.. code-block:: python

    tg-admin shell
    
    jb = People(name='Joe Bloggs')
    jd = People(name='Jane Doe')
    sp = Status(name='Pending')
    sd = Status(name='Dispatched')
    Order(name='Garden furniture', status=sp, customer=jb, assignee=jd)
    Order(name='Barbeque', status=sd, customer=jd, assignee=jb)
    session.flush()


Front Page
----------

The front page of the application needs to be a list of orders. Usually we'd like it to show the list of orders that the user needs to work on, and it also needs functionality to search for orders. The FileringGrid widget from tw.dynforms is useful for this; we can set up a basic static grid by adding the following to the top of controllers.py:

.. code-block:: python

    import tw.forms as twf, tw.dynforms as twd
    from myproj import model

    class OrderGrid(twd.FilteringGrid):
        datasrc = lambda s: model.Order.query()
        columns = [('name', 'Name'), ('status', 'Status'), ('customer', 'Customer'), ('assignee', 'Assignee')]
    order_grid = OrderGrid('orders')

Inside the root controller, change the index method to:

.. code-block:: python

    @expose(template="myproj.templates.index")
    def index(self, **kw):
        return dict(order_grid=order_grid, kw=kw)

You'll also need to create a template. In the templates directory, copy welcome.kid to index.kid, then edit index.kid to remove all the TurboGears welcome text. Now, inside the <body> tag add:

.. code-block:: python

    ${order_grid(kw)}
    
With all this done, refresh the browser page and you will see the list of orders. We can add filtering and searching by adding the following to the OrderGrid class:

.. code-block:: python

    data_filter = ['status', 'customer', 'assignee']
    search_cols = ['name']   

FilteringGrid has some other features. It supports "code filters" as well as "data filters". Code filters work at the SQL level, providing greater flexibility and efficiency, at the cost of more coding effort. It also supports checkboxes that apply filters, useful for features like "Only show orders over 250".


Form Editing
------------

Users need to be able to click on an order to get further information. We'll build an inital version of the detail form using ToscaWidgets. Add the following to controllers.py:

.. code-block:: python

    import formencode as fe

    class FilteringSchema(fe.schema.Schema):
        filter_extra_fields = True
        allow_extra_fields = True

    class OrderForm(twf.TableForm):
        validator = FilteringSchema
        action = 'save_order'
        class children(twc.WidgetsList):
            id = twf.HiddenField()
            name = twf.TextField()
            status_id = twf.SingleSelectField(options=twd.load_options(model.Status), label_text='Status')
            customer_id = twf.SingleSelectField(options=twd.load_options(model.People), label_text='Customer')
            assignee_id = twf.SingleSelectField(options=twd.load_options(model.People), label_text='Assignee')
            delivery = twf.CheckBox()
            address = twf.TextArea()
    order_form = OrderForm('order')  

And, inside the Root controller class:

.. code-block:: python

    @expose(template="myproj.templates.order")
    def order(self, id):
        return dict(order_form=order_form, order=model.Order.query.get(id))

Copy index.kid to order.kid and put in:

.. code-block:: python

    ${order_form(order)}

We need to link to this page from the front page. Lets add another column to the table, 'ID', where the ID is a link to the detail form. The approach described here is convenient, although it does break the MVC model a little, by bringing some view functionality into the model. Edit model.py to add:

.. code-block:: python

    import genshi
    
and, inside the Order class:
    
.. code-block:: python

    @property
    def id_link(self):
        return genshi.XML('<a href="order?id=%d">%d</a>' % (self.id, self.id))

Edit the FilteringGrid definition in controllers.py:

.. code-block:: python

    columns = [
        ('id_link', 'ID'), 
        ('name', 'Name'), 
        ('status', 'Status'), 
        ('customer', 'Customer'), 
        ('assignee', 'Assignee')
    ]

The final piece is to make the form save when you click submit. Add the following to controllers.py:

.. code-block:: python

    @expose()
    @tg.validate(form=order_form)
    @tg.error_handler(order)
    def save_order(self, id, **kw):
        model.Order.query.get(id).from_dict(kw)
        tg.redirect('index')
      
You can now use your browser to edit orders in the system. This arrangement provides the basis for a highly functional system. In particular, validation can easily be added, with the error messages reported in a user-friendly way. It's also easy to adapt this to form a "create new order" function.


Hiding
------    

The address field only applies to orders that need delivery; there's no need to show it for other orders. Dynforms helps you build dynamic forms like this, using a set of Hiding controls. In this case, we'll use HidingCheckBox. Update controllers.py as follows:

Add to the top:

.. code-block:: python

    import tw.dynforms as twd

Change

.. code-block:: python

    class OrderForm(twf.TableForm):

to

.. code-block:: python

    class OrderForm(twd.HidingTableForm):

and change

.. code-block:: python

    delivery = twf.CheckBox()

to

.. code-block:: python

    delivery = twd.HidingCheckBox(mapping={1:['address']})

The mapping defines what controls should be visible when the Hiding control has a particular value - in this case, when it is checked, the address field will become visible. Other hiding controls are available, including HidingSingleSelectField and HidingCheckBoxList, and you can also create your own using HidingComponentMixin. Dynforms fully supports nested hiding and other complex arrangements.


Growing
-------

In this application, each Order can contain a number of Items. Most orders will just have a handful, but potentially some orders may have a large number of items. What we really want is a dynamic form that grows spaces to enter items, as needed. Dynforms supports a variety of Growing forms to allow this. To implement this, first we need to add a new class to model.py:

.. code-block:: python

    class Item(Entity):
        order = ManyToOne(Order)
        code = Field(String(50))
        description = Field(String(200))

Also, add the following to the Order class:

.. code-block:: python

    items = OneToMany('Item')
    
And to create the new table, at the shell:

.. code-block:: python

    tg-admin sql create

To create the corresponding widgets, add this to controllers.py:

.. code-block:: python

    class ItemForm(twd.GrowingTableFieldSet):
        class children(twc.WidgetsList):
            order_id = twf.HiddenField()
            code = twf.SingleSelectField(options=['', 'A', 'B', 'C'])
            description = twf.TextField()

and inside OrderForm's children:

.. code-block:: python

    items = ItemForm()
    
Also, change

.. code-block:: python

    class OrderForm(twd.HidingTableForm):

to    

.. code-block:: python

    class OrderForm(twd.CustomisedForm, twd.HidingTableForm):
    
Take a look at this in your browser - the growing form provides delete and undo functionality, and it's fun to play with!


Ajax Lookups
------------

Presenting the contacts as a dropdown list will become unmanagable as the number of contacts grows. The AjaxLookup widget helps to resolve this. It presents to the user as a text field, and when the control loses focus, it uses ajax to search on the server for names matching the text. If there is an exact match, the field changes visually to show this. If there are multiple matches, the user is presented with a list to choose from. When the form is submitted, the application just sees the ID associated with the contact. 

To setup the lookup, first we need to add search and __json__ methods to the People class in the model; add this:

.. code-block:: python

    def __json__(self):
        return {'id':self.id, 'value':self.name, 'extra':self.email}
    @classmethod
    def search(self, search):
        return People.query.filter(People.name.like('%'+search+'%')).all()

In controllers.py, add this:

.. code-block:: python

    import cherrypy as cp

    class ContactLookup(twd.AjaxLookupField):
        datasrc = model.People
        ajaxurl = 'findperson'

Change this:

.. code-block:: python

    customer_id = twf.SingleSelectField(options=twd.load_options(model.People), label_text='Customer')
    assignee_id = twf.SingleSelectField(options=twd.load_options(model.People), label_text='Assignee')

to:

.. code-block:: python

    customer_id = ContactLookup(label_text='Customer')
    assignee_id = ContactLookup(label_text='Assignee')

And add this to the Root controller class:

.. code-block:: python

    @expose(format='json')
    def findperson(self, search):
        return ContactLookup().ajax_helper(cp.request.method, search)

Try this in your browser - enter just "j" in the box and press tab to make it lose focus.


Select with Other
-----------------

Over time, users will want to use more status codes for orders, beyond "pending" and "dispatched", such as "awaiting supplier" and "returned". Dynforms provides OtherSingleSelectField, which adds an "other" choice to the list, and when this is selected, prompts the user for a free-text value. To use this, edit controllers.py:

Change

.. code-block:: python

    status_id = twf.SingleSelectField(options=twd.load_options(model.Status), label_text='Status')

to

.. code-block:: python

    status_id = twd.OtherSingleSelectField(dataobj=model.Status, field='name', label_text='Status')

When you try this in your browser, you'll see that once a user enters an "other" value, it is then available in the select field for all users.


Further Customisation
---------------------

To give the site your own look, you can edit the templates to provide your own layout. Customising the appearance of the forms can be done using CSS. If you need more flexibility, you can override widget templates with your own versions.

tw.dynforms has several other features. Cascading fields - when a value is selected in one field, it causes an ajax request that can set the value of others fields. LinkContainer - lets you attach a "view" link to a control, particularly useful with SingleSelectFields and AjaxLookupFields. There's also WriteOnlyTextField for secret data, such as passwords, that the server does not disclose to clients.


Future Directions
-----------------

Powerful as these tools are, there is plenty of scope for future development. One current challenge is the need to code both the database model and the widgets definition. There are two relatively new libraries, rum [2] and dbsprockets [3], to build the widgets automatically from the model, and both allow you to put hints on the model to tailor generation. Another challenge is that adding a column to a table currently requires the user to manually issue SQL. There are also two newer libraries to help with this, SQLAlchemy Migrate [4] and miruku [5]. Beyond this, future work will allow more concise specification

| [1] http://toscawidgets.org/
| [2] http://toscawidgets.org/documentation/rum/
| [3] http://code.google.com/p/dbsprockets/
| [4] http://code.google.com/p/sqlalchemy-migrate/
| [5] http://trac.ollix.org/miruku/