.. _aggregation: Aggregating resources with ToscaWidgets ========================================= Introduction ------------ One of the most powerful features of ToscaWidgets is the ability to create widget collections that depend on other widgets. These dependencies will be resolved automatically, injecting whatever resources are needed into the delivered page. As convenient and powerful this feature is at development time, it creates a situation where potentially a large number of files is loaded for a given site, reducing loading speed immensely. One way to deal with this is the aggregation of resources - mostly Javascript and CSS - into one large file, potentially even compressed uisng e.g. ``YUICompressor``. This can then be delivered with proper caching headers and thus reduce the loading time of a site significantly. Aggregating resources --------------------- The first step is to aggregate the resources into a single file. This is done through a :class:`setuptools.Command`, :class:`tw.core.command.aggregate_tw_resources`. Let's assume we have a TurboGears2 project called `MyProject` that lives in the package `myproject`. Inside this project, there is a directory-structure used to serve static resources, referenced via widgets. This structure looks like this:: myproject/public/javascript/ myproject/public/javascript/a.js myproject/public/javascript/b.js `MyProject` needs an entry-point in it's ``setup.py`` like this:: entry_points= { ... "toscawidgets.widgets": [ "widgets = myproject.widgets.load_all" ] ... } This entry-point will need to trigger the instantiation of all the widgets used by `MyProject`. So the ``load_all.py`` should look something like this:: import os from paste.script.util.logging_config import fileConfig import paste.fixture from paste.deploy import loadapp # locate a viable config, development.ini should be good APP_CONFIG = os.path.normpath(os.path.join(__file__, "..", "..", "development.ini")) # load the actual application, so that we then can instantiate # controllers. the_application = loadapp("config:" + APP_CONFIG) import myproject.controllers.root as root # instantiate the root-controller to pull in all widgets root.RootController() Now if we want to aggregate both Javascript-files to a single one, we'd use the following command from within the working-directory of `MyProject`:: $ python2.5 setup.py aggregate_tw_resources -o myproject/public/javascript/ -d MyProject --package=myproject This will - gather the sources found in `MyProject` as well as all referenced widgets of other packages. - filter these to contain only the ones that live within the package ``myproject``. - concatenate the sources to one file. - compute the md5-checksum of that file. - store the resulting aggreate file under the name ``-normal.js`` into the diretory ``myproject/public/javascript``. - place a mapping-file ``-normal.js.map`` alongside the aggregated source. - print out which files it created. This is the first step to serve aggregated resources. Most of the actions should be clear - with the exception of the mapping-files, so let's take a look at these. Mapping-files ~~~~~~~~~~~~~ Mapping-files are produced alongside aggregated resource files to specify **which** resources are actually contained inside an aggregation file. The format it simple:: | For our current example, the file will look like this:: myproject|public/javascript/a.js myproject|public/javascript/b.js When configuring the hostframework to serve aggregated resources, the mapping file is read and used to decide if for a given injected resource which aggregation file has to be served. Filtering out unwanted resources ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Some resources that are declared and loaded through the entry-point for a given distribution one does not want to include. A prominent example are the i18n-files of the ``tw.jquery.ui.ui_datepicker_js``. To accomodate for this need, a second entrypoint can be defined, like this:: entry_points= { ... "toscawidgets.widgets": [ "resource_aggregation_filter = myproject.widgets.filter_resources:filter_resources" ] ... } The entry-point is expected to be a predicate that gets passed the resource in question. When returning ``True``, the resource is included, otherwise filtered out. A possible implementation to filter out ``i18n``-files looks like this:: def filter_resources(resource): # we don't want the localization files of # datepicker and others filename = resource.active_filename() if "i18n" in filename: return False return True **IMPORTANT**: If there are several distributions given, and some of them define a filter, the predicates are and'ed together - so **all** predicates must agree to include a specific resource! Configuring the HostFramework ----------------------------- Once the aggregation has taken place, we are ready to configure our webapp to actually serve the aggregation file. This is done via the so-called `HostFramework`. The `HostFramework` is the link between ToscaWidgets and your application. One of it's responsibilities is to inject resources into various locations of the rendered page. Excactly there the aggregation takes place. To set it up, we pass a configuration to the `HostFramework` via the keyword-argument ``aggregation_configuration``. This is a dictionary with the following layout:: dict( js=[dict(modname=, filename=, map=), ...], # more mappings to come. css=... # as js ) For our actual project, this would look like this:: dict( js=[dict(modname="myproject", filename="public/javascript/-normal.js")] ) As you can see, the ``map``-key can be ommitted - it then defaults to the above mentioned naming-convention. Now whenever a page is served that has either ``a.js`` or ``b.js`` (or both of course) injected, the aggregation file will be used instead. The above configuration must of course be made within your actual application. Various frameworks set up the `HostFramework` differently, so you need to figure out how to set it up. For TurboGears2, it's shown here. Configuration of the HostFramework for TurboGears2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In an TG2-application, the `HostFramework` is set up for you throug the `AppConfig`. To override the default-behavior, you need to change ``myproject.config.app_cfg`` and put the following in there:: from tg.configuration import config as tg_config from tw.api import make_middleware as tw_middleware import tw.core # this is needed because otherwise the following import fails import tw.mods.base class MyAppConfig(AppConfig): def add_tosca_middleware(self, app): """Configure the ToscaWidgets middleware.""" aggregated_hash = tg_config.get("aggregated_hash") aggregation_config = dict() if aggregated_hash is not None: aggregation_config["js"] =[ dict(package="myproject", filename="public/javascript/%s-normal.js" % aggregated_hash, ) ] app = tw_middleware(app, { 'toscawidgets.framework' : tw.mods.base.AggregatedHostFramework, "toscawidgets.framework.aggregation_config" : aggregation_config, 'toscawidgets.framework.default_view': default_renderer, 'toscawidgets.middleware.inject_resources': True, }) return app As you can see, the decision if the aggregation is to be used or not depends on the existence of a specifig parameter ``aggregated_hash`` in the configuration of the webapp. Of course you are free to hardcode this, or name the parameter different. The :class:`setuptools.Command` ------------------------------- .. autoclass:: tw.core.command.aggregate_tw_resources The :class:`tw.mods.base.HostFramework` ------------------------------------------------- .. autoclass:: tw.mods.base.HostFramework