Preface¶

Prerequisites¶

• Python 2.4 or later
• Optional: Setuptools 0.6c3 or later

Setuptools is only required for the template engine plugin, which can be used to integrate Genshi with Python web application frameworks such as Pylons or TurboGears. Genshi also provides a Setuptools-based plugin that integrates its internationalization support with the Babel library, but that support can also be used without Setuptools being available (although in a slightly less convenient fashion).

Installing via easy_install¶

If you have a recent version of Setuptools installed, you can directly install Genshi using the easy_install command-line tool:

$ easy_install Genshi

This downloads and installs the latest version of the Genshi package.

If you have an older Genshi release installed and would like to upgrade, add the -U option to the above command.

Installing from a Binary Installer¶

Binary packages for Windows and Mac OS X are provided for Genshi. To install from such a package, simply download and open it.

Installing from a Source Tarball¶

Once you’ve downloaded and unpacked a Genshi source release, enter the directory where the archive was unpacked, and run:

$ python setup.py install

Note that you may need administrator/root privileges for this step, as this command will by default attempt to install Genshi to the Python site-packages directory on your system.

Genshi comes with an optional extension module written in C that is used to improve performance in some areas. This extension is automatically compiled when you run the setup.py script as shown above. In the case that the extension can not be compiled, possibly due to a missing or incompatible C compiler, the compilation is skipped. If you’d prefer Genshi to not use this native extension module, you can explicitly bypass the compilation using the --without-speedups option:

$ python setup.py --without-speedups install

For other build and installation options, please consult the easy_install and/or the Python distutils documentation.

Support¶

If you encounter any problems with Genshi, please don’t hesitate to ask questions on the Genshi mailing list or IRC channel.

Upgrading from Genshi 0.6.x to the development version¶

The Genshi development version now supports both Python 2 and Python 3.

The most noticable API change in the Genshi development version is that the default encoding in numerous places is now None (i.e. unicode) instead of UTF-8. This change was made in order to ease the transition to Python 3 where strings are unicode strings by default.

If your application relies on the default UTF-8 encoding a simple way to have it work both with Genshi 0.6.x and the development version is to specify the encoding explicitly in calls to the following classes, methods and functions:

• genshi.HTML
• genshi.Stream.render
• genshi.input.HTMLParser
• genshi.template.MarkupTemplate
• genshi.template.TemplateLoader
• genshi.template.TextTemplate (and genshi.template.NewTextTemplate)
• genshi.template.OldTextTemplate

Whether you explicitly specify UTF-8 or explicitly specify None (unicode) is a matter of personal taste, although working with unicode may make porting your own application to Python 3 easier.

Upgrading from Genshi 0.5.x to 0.6.x¶

Upgrading from Genshi 0.4.x to 0.5.x¶

Markup('<b>%s</b>', name)

Markup('<b>%s</b>') % name

Upgrading from Genshi 0.3.x to 0.4.x¶

The modules genshi.filters and genshi.template have been refactored into packages containing multiple modules. While code using the regular APIs should continue to work without problems, you should make sure to remove any leftover traces of the files filters.py and template.py in the genshi package on the installation path (including the corresponding .pyc files). This is not necessary when Genshi was installed as a Python egg.

Results of evaluating template expressions are no longer implicitly called if they are callable. If you have been using that feature, you will need to add the parenthesis to actually call the function.

Instances of genshi.core.Attrs are now immutable. Filters manipulating the attributes in a stream may need to be updated. Also, the Attrs class no longer automatically wraps all attribute names in QName objects, so users of the Attrs class need to do this themselves. See the documentation of the Attrs class for more information.

Upgrading from Markup¶

Prior to version 0.3, the name of the Genshi project was “Markup”. The name change means that you will have to adjust your import statements and the namespace URI of XML templates, among other things:

• The package name was changed from “markup” to “genshi”. Please adjust any import statements referring to the old package name.
• The namespace URI for directives in Genshi XML templates has changed from http://markup.edgewall.org/ to http://genshi.edgewall.org/. Please update the xmlns:py declaration in your template files accordingly.

Furthermore, due to the inclusion of a text-based template language, the class:

markup.template.Template

has been renamed to:

genshi.template.MarkupTemplate

If you’ve been using the Template class directly, you’ll need to update your code (a simple find/replace should do—the API itself did not change).

Basics¶

A stream can be attained in a number of ways. It can be:

• the result of parsing XML or HTML text, or
• the result of selecting a subset of another stream using XPath, or
• programmatically generated.

For example, the functions XML() and HTML() can be used to convert literal XML or HTML text to a markup stream:

>>> from genshi import XML
>>> stream = XML('<p class="intro">Some text and '
...              '<a href="http://example.org/">a link</a>.'
...              '<br/></p>')
>>> stream
<genshi.core.Stream object at ...>

The stream is the result of parsing the text into events. Each event is a tuple of the form (kind, data, pos), where:

• kind defines what kind of event it is (such as the start of an element, text, a comment, etc).
• data is the actual data associated with the event. How this looks depends on the event kind (see event kinds)
• pos is a (filename, lineno, column) tuple that describes where the event “comes from”.

>>> for kind, data, pos in stream:
...     print('%s %r %r' % (kind, data, pos))
...
START (QName('p'), Attrs([(QName('class'), u'intro')])) (None, 1, 0)
TEXT u'Some text and ' (None, 1, 17)
START (QName('a'), Attrs([(QName('href'), u'http://example.org/')])) (None, 1, 31)
TEXT u'a link' (None, 1, 61)
END QName('a') (None, 1, 67)
TEXT u'.' (None, 1, 71)
START (QName('br'), Attrs()) (None, 1, 72)
END QName('br') (None, 1, 77)
END QName('p') (None, 1, 77)

Filtering¶

One important feature of markup streams is that you can apply filters to the stream, either filters that come with Genshi, or your own custom filters.

A filter is simply a callable that accepts the stream as parameter, and returns the filtered stream:

def noop(stream):
    """A filter that doesn't actually do anything with the stream."""
    for kind, data, pos in stream:
        yield kind, data, pos

Filters can be applied in a number of ways. The simplest is to just call the filter directly:

stream = noop(stream)

The Stream class also provides a filter() method, which takes an arbitrary number of filter callables and applies them all:

stream = stream.filter(noop)

Finally, filters can also be applied using the bitwise or operator (|), which allows a syntax similar to pipes on Unix shells:

stream = stream | noop

One example of a filter included with Genshi is the HTMLSanitizer in genshi.filters. It processes a stream of HTML markup, and strips out any potentially dangerous constructs, such as Javascript event handlers. HTMLSanitizer is not a function, but rather a class that implements __call__, which means instances of the class are callable:

stream = stream | HTMLSanitizer()

Both the filter() method and the pipe operator allow easy chaining of filters:

from genshi.filters import HTMLSanitizer
stream = stream.filter(noop, HTMLSanitizer())

That is equivalent to:

stream = stream | noop | HTMLSanitizer()

For more information about the built-in filters, see Stream Filters.

Serialization¶

Serialization means producing some kind of textual output from a stream of events, which you’ll need when you want to transmit or store the results of generating or otherwise processing markup.

The Stream class provides two methods for serialization: serialize() and render(). The former is a generator that yields chunks of Markup objects (which are basically unicode strings that are considered safe for output on the web). The latter returns a single string, by default UTF-8 encoded.

Here’s the output from serialize():

>>> for output in stream.serialize():
...     print(repr(output))
...
<Markup u'<p class="intro">'>
<Markup u'Some text and '>
<Markup u'<a href="http://example.org/">'>
<Markup u'a link'>
<Markup u'</a>'>
<Markup u'.'>
<Markup u'<br/>'>
<Markup u'</p>'>

And here’s the output from render():

>>> print(stream.render())
<p class="intro">Some text and <a href="http://example.org/">a link</a>.<br/></p>

Both methods can be passed a method parameter that determines how exactly the events are serialized to text. This parameter can be either a string or a custom serializer class:

>>> print(stream.render('html'))
<p class="intro">Some text and <a href="http://example.org/">a link</a>.<br></p>

Note how the <br> element isn’t closed, which is the right thing to do for HTML. See serialization methods for more details.

In addition, the render() method takes an encoding parameter, which defaults to “UTF-8”. If set to None, the result will be a unicode string.

The different serializer classes in genshi.output can also be used directly:

>>> from genshi.filters import HTMLSanitizer
>>> from genshi.output import TextSerializer
>>> print(''.join(TextSerializer()(HTMLSanitizer()(stream))))
Some text and a link.

The pipe operator allows a nicer syntax:

>>> print(stream | HTMLSanitizer() | TextSerializer())
Some text and a link.

Using XPath¶

XPath can be used to extract a specific subset of the stream via the select() method:

>>> substream = stream.select('a')
>>> substream
<genshi.core.Stream object at ...>
>>> print(substream)
<a href="http://example.org/">a link</a>

Often, streams cannot be reused: in the above example, the sub-stream is based on a generator. Once it has been serialized, it will have been fully consumed, and cannot be rendered again. To work around this, you can wrap such a stream in a list:

>>> from genshi import Stream
>>> substream = Stream(list(stream.select('a')))
>>> substream
<genshi.core.Stream object at ...>
>>> print(substream)
<a href="http://example.org/">a link</a>
>>> print(substream.select('@href'))
http://example.org/
>>> print(substream.select('text()'))
a link

See Using XPath in Genshi for more information about the XPath support in Genshi.

Event Kinds¶

Every event in a stream is of one of several kinds, which also determines what the data item of the event tuple looks like. The different kinds of events are documented below.

START, (QName('p'), Attrs([(QName('class'), u'intro')])), pos

END, QName('p'), pos

TEXT, u'Hello, world!', pos

START_NS, (u'svg', u'http://www.w3.org/2000/svg'), pos

END_NS, u'svg', pos

DOCTYPE, (u'html', u'-//W3C//DTD XHTML 1.0 Transitional//EN', \
          u'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'), pos

COMMENT, u'Commented out', pos

PI, (u'php', u'echo "Yo" '), pos

START_CDATA, None, pos

END_CDATA, None, pos

Synopsis¶

A Genshi markup template is a well-formed XML document with embedded Python used for control flow and variable substitution. Markup templates should be used to generate any kind of HTML or XML output, as they provide a number of advantages over simple text-based templates (such as automatic escaping of variable data).

The following is a simple Genshi markup template:

<?python
  title = "A Genshi Template"
  fruits = ["apple", "orange", "kiwi"]
?>
<html xmlns:py="http://genshi.edgewall.org/">
  <head>
    <title py:content="title">This is replaced.</title>
  </head>

  <body>
    <p>These are some of my favorite fruits:</p>
    <ul>
      <li py:for="fruit in fruits">
        I like ${fruit}s
      </li>
    </ul>
  </body>
</html>

This example shows:

1. a Python code block in a processing instruction
2. the Genshi namespace declaration
3. usage of templates directives (py:content and py:for)
4. an inline Python expression (${fruit}).

The template would generate output similar to this:

<html>
  <head>
    <title>A Genshi Template</title>
  </head>

  <body>
    <p>These are some of my favorite fruits:</p>
    <ul>
      <li>I like apples</li>
      <li>I like oranges</li>
      <li>I like kiwis</li>
    </ul>
  </body>
</html>

A text template is a simple plain text document that can also contain embedded Python code. Text templates are intended to be used for simple non-markup text formats, such as the body of an plain text email. For example:

Dear $name,

These are some of my favorite fruits:
#for fruit in fruits
 * $fruit
#end

Python API¶

The Python code required for templating with Genshi is generally based on the following pattern:

• Attain a MarkupTemplate or TextTemplate object from a string or file-like object containing the template source. This can either be done directly, or through a TemplateLoader instance.
• Call the generate() method of the template, passing any data that should be made available to the template as keyword arguments.
• Serialize the resulting stream using its render() method.

For example:

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<h1>Hello, $name!</h1>')
>>> stream = tmpl.generate(name='world')
>>> print(stream.render('xhtml'))
<h1>Hello, world!</h1>

Using a text template is similar:

>>> from genshi.template import TextTemplate
>>> tmpl = TextTemplate('Hello, $name!')
>>> stream = tmpl.generate(name='world')
>>> print(stream)
Hello, world!

Using a template loader provides the advantage that “compiled” templates are automatically cached, and only parsed again when the template file changes. In addition, it enables the use of a template search path, allowing template directories to be spread across different file-system locations. Using a template loader would generally look as follows:

from genshi.template import TemplateLoader
loader = TemplateLoader([templates_dir1, templates_dir2])
tmpl = loader.load('test.html')
stream = tmpl.generate(title='Hello, world!')
print(stream.render())

See the API documentation for details on using Genshi via the Python API.

Template Expressions and Code Blocks¶

Python expressions can be used in text and directive arguments. An expression is substituted with the result of its evaluation against the template data. Expressions in text (which includes the values of non-directive attributes) need to prefixed with a dollar sign ($) and usually enclosed in curly braces ({…}).

If the expression starts with a letter and contains only letters, digits, dots, and underscores, the curly braces may be omitted. In all other cases, the braces are required so that the template processor knows where the expression ends:

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<em>${items[0].capitalize()} item</em>')
>>> print(tmpl.generate(items=['first', 'second']))
<em>First item</em>

Expressions support the full power of Python. In addition, it is possible to access items in a dictionary using “dotted notation” (i.e. as if they were attributes), and vice-versa (i.e. access attributes as if they were items in a dictionary):

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<em>${dict.foo}</em>')
>>> print(tmpl.generate(dict={'foo': 'bar'}))
<em>bar</em>

Because there are two ways to access either attributes or items, expressions do not raise the standard AttributeError or IndexError exceptions, but rather an exception of the type UndefinedError. The same kind of error is raised when you try to use a top-level variable that is not in the context data. See Error Handling below for details on how such errors are handled.

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<em>$foo</em>') # Wanted "$foo" as literal output
>>> print(tmpl.generate())
Traceback (most recent call last):
  ...
UndefinedError: "foo" not defined
>>> tmpl = MarkupTemplate('<em>$$foo</em>')
>>> print(tmpl.generate())
<em>$foo</em>

>>> tmpl = MarkupTemplate('<script>$(function() {})</script>')
>>> print(tmpl.generate())
<script>$(function() {})</script>

>>> tmpl = MarkupTemplate('<script>$$$("div")</script>')
>>> print(tmpl.generate())
<script>$$("div")</script>

<div>
  <?python
      from genshi.builder import tag
      def greeting(name):
          return tag.b('Hello, %s!' % name) ?>
  ${greeting('world')}
</div>

<div>
  <b>Hello, world!</b>
</div>

{% python
    from genshi.builder import tag
    def greeting(name):
        return 'Hello, %s!' % name
%}
${greeting('world')}

Hello, world!

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<p>${doh}</p>')
>>> tmpl.generate().render('xhtml')
Traceback (most recent call last):
  ...
UndefinedError: "doh" not defined

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<p>${defined("doh")}</p>')
>>> print(tmpl.generate().render('xhtml'))
<p>False</p>

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<p>${doh}</p>', lookup='lenient')
>>> print(tmpl.generate().render('xhtml'))
<p></p>

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<p>${doh.oops}</p>', lookup='lenient')
>>> print(tmpl.generate().render('xhtml'))
Traceback (most recent call last):
  ...
UndefinedError: "doh" not defined

>>> from genshi.template import MarkupTemplate
>>> tmpl = MarkupTemplate('<p>${type(doh) is not Undefined}</p>',
...                       lookup='lenient')
>>> print(tmpl.generate().render('xhtml'))
<p>False</p>

Template Directives¶

Directives provide control flow functionality for templates, such as conditions or iteration. As the syntax for directives depends on whether you’re using markup or text templates, refer to the XML Template Language or Text Template Language pages for information.

Template Directives¶

Directives are elements and/or attributes in the template that are identified by the namespace http://genshi.edgewall.org/. They can affect how the template is rendered in a number of ways: Genshi provides directives for conditionals and looping, among others.

To use directives in a template, the namespace must be declared, which is usually done on the root element:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      lang="en">
  ...
</html>

In this example, the default namespace is set to the XHTML namespace, and the namespace for Genshi directives is bound to the prefix “py”.

All directives can be applied as attributes, and some can also be used as elements. The if directives for conditionals, for example, can be used in both ways:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      lang="en">
  ...
  <div py:if="foo">
    <p>Bar</p>
  </div>
  ...
</html>

This is basically equivalent to the following:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      lang="en">
  ...
  <py:if test="foo">
    <div>
      <p>Bar</p>
    </div>
  </py:if>
  ...
</html>

The rationale behind the second form is that directives do not always map naturally to elements in the template. In such cases, the py:strip directive can be used to strip off the unwanted element, or the directive can simply be used as an element.

<div>
  <b py:if="foo">${bar}</b>
</div>

<div>
  <b>Hello</b>
</div>

<div>
</div>

<div>
  <py:if test="foo">
    <b>${bar}</b>
  </py:if>
</div>

<div py:choose="">
  <span py:when="0 == 1">0</span>
  <span py:when="1 == 1">1</span>
  <span py:otherwise="">2</span>
</div>

<div>
  <span>1</span>
</div>

<div py:choose="1">
  <span py:when="0">0</span>
  <span py:when="1">1</span>
  <span py:otherwise="">2</span>
</div>

<div>
  <span>1</span>
</div>

<py:choose test="1">
  <py:when test="0">0</py:when>
  <py:when test="1">1</py:when>
  <py:otherwise>2</py:otherwise>
</py:choose>

<ul>
  <li py:for="item in items">${item}</li>
</ul>

<ul>
  <li>1</li><li>2</li><li>3</li>
</ul>

<ul>
  <py:for each="item in items">
    <li>${item}</li>
  </py:for>
</ul>

<div>
  <p py:def="greeting(name)" class="greeting">
    Hello, ${name}!
  </p>
  ${greeting('world')}
  ${greeting('everyone else')}
</div>

<div>
  <p class="greeting">
    Hello, world!
  </p>
  <p class="greeting">
    Hello, everyone else!
  </p>
</div>

<div>
  <p py:def="greeting" class="greeting">
    Hello, world!
  </p>
  ${greeting()}
</div>

<div>
  <p class="greeting">
    Hello, world!
  </p>
</div>

<div>
  <py:def function="greeting(name)">
    <p class="greeting">Hello, ${name}!</p>
  </py:def>
</div>

<div>
  <span py:match="greeting">
    Hello ${select('@name')}
  </span>
  <greeting name="Dude" />
</div>

<div>
  <span>
    Hello Dude
  </span>
</div>

<div>
  <py:match path="greeting">
    <span>Hello ${select('@name')}</span>
  </py:match>
  <greeting name="Dude" />
</div>

<py:match path="body" once="true">
  <body py:attrs="select('@*')">
    <div id="header">...</div>
    ${select("*|text()")}
    <div id="footer">...</div>
  </body>
</py:match>

<div>
  <span py:with="y=7; z=x+10">$x $y $z</span>
</div>

<div>
  <span>42 7 52</span>
</div>

<div>
  <py:with vars="y=7; z=x+10">$x $y $z</py:with>
</div>

<ul>
  <li py:attrs="foo">Bar</li>
</ul>

<ul>
  <li class="collapse">Bar</li>
</ul>

<ul>
  <li>Bar</li>
</ul>

<ul>
  <li py:content="bar">Hello</li>
</ul>

<ul>
  <li>Bye</li>
</ul>

<div>
  <span py:replace="bar">Hello</span>
</div>

<div>
  Bye
</div>

<div>
  <py:replace value="title">Placeholder</py:replace>
</div>

<div>
  <div py:strip="True"><b>foo</b></div>
</div>

<div>
  <b>foo</b>
</div>

Includes¶

To reuse common snippets of template code, you can include other files using XInclude.

For this, you need to declare the XInclude namespace (commonly bound to the prefix “xi”) and use the <xi:include> element where you want the external file to be pulled in:

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:py="http://genshi.edgewall.org/"
      xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="base.html" />
  ...
</html>

Include paths are relative to the filename of the template currently being processed. So if the example above was in the file “myapp/index.html” (relative to the template search path), the XInclude processor would look for the included file at “myapp/base.html”. You can also use Unix-style relative paths, for example “../base.html” to look in the parent directory.

Any content included this way is inserted into the generated output instead of the <xi:include> element. The included template sees the same context data. Match templates and macros in the included template are also available to the including template after the point it was included.

By default, an error will be raised if an included file is not found. If that’s not what you want, you can specify fallback content that should be used if the include fails. For example, to to make the include above fail silently, you’d write:

<xi:include href="base.html"><xi:fallback /></xi:include>

See the XInclude specification for more about fallback content. Note though that Genshi currently only supports a small subset of XInclude.

<xi:include href="${name}.html" py:if="not in_popup"
            py:for="name in ('foo', 'bar', 'baz')" />

<xi:include href="myscript.js" parse="text" />

Comments¶

Normal XML/HTML comment syntax can be used in templates:

<!-- this is a comment -->

However, such comments get passed through the processing pipeline and are by default included in the final output. If that’s not desired, prefix the comment text with an exclamation mark:

<!-- !this is a comment too, but one that will be stripped from the output -->

Note that it does not matter whether there’s whitespace before or after the exclamation mark, so the above could also be written as follows:

<!--! this is a comment too, but one that will be stripped from the output -->

Template Directives¶

Directives are template commands enclosed by {% ... %} characters. They can affect how the template is rendered in a number of ways: Genshi provides directives for conditionals and looping, among others.

Each directive must be terminated using an {% end %} marker. You can add a string inside the {% end %} marker, for example to document which directive is being closed, or even the expression associated with that directive. Any text after end inside the delimiters is ignored, and effectively treated as a comment.

If you want to include a literal delimiter in the output, you need to escape it by prepending a backslash character (\).

{% if foo %}
  ${bar}
{% end %}

Hello

The answer is:
{% choose %}
  {% when 0 == 1 %}0{% end %}
  {% when 1 == 1 %}1{% end %}
  {% otherwise %}2{% end %}
{% end %}

The answer is:
  1

The answer is:
{% choose 1 %}\
  {% when 0 %}0{% end %}\
  {% when 1 %}1{% end %}\
  {% otherwise %}2{% end %}\
{% end %}

The answer is:
    1

Your items:
{% for item in items %}\
  * ${item}
{% end %}

Your items
  * 1
  * 2
  * 3

{% def greeting(name) %}
  Hello, ${name}!
{% end %}
${greeting('world')}
${greeting('everyone else')}

Hello, world!
Hello, everyone else!

{% def greeting %}
  Hello, world!
{% end %}
${greeting()}

Hello, world!

{% include base.txt %}

{% include ${'%s.txt' % filename} %}

Magic numbers!
{% with y=7; z=x+10 %}
  $x $y $z
{% end %}

Magic numbers!
  42 7 52

White-space and Line Breaks¶

Note that space or line breaks around directives is never automatically removed. Consider the following example:

{% for item in items %}
  {% if item.visible %}
    ${item}
  {% end %}
{% end %}

This will result in two empty lines above and beneath every item, plus the spaces used for indentation. If you want to supress a line break, simply end the line with a backslash:

{% for item in items %}\
  {% if item.visible %}\
    ${item}
  {% end %}\
{% end %}\

Now there would be no empty lines between the items in the output. But you still get the spaces used for indentation, and because the line breaks are removed, they actually continue and add up between lines. There are numerous ways to control white-space in the output while keeping the template readable, such as moving the indentation into the delimiters, or moving the end delimiter on the next line, and so on.

Comments¶

Parts in templates can be commented out using the delimiters {# ... #}. Any content in comments are removed from the output.

{# This won't end up in the output #}
This will.

Just like directive delimiters, these can be escaped by prefixing with a backslash.

\{# This *will* end up in the output, including delimiters #}
This too.

Legacy Text Template Syntax¶

The syntax for text templates was redesigned in version 0.5 of Genshi to make the language more flexible and powerful. The older syntax is based on line starting with dollar signs, similar to e.g. Cheetah or Velocity.

A simple template using the old syntax looked like this:

Dear $name,

We have the following items for you:
#for item in items
 * $item
#end

All the best,
Foobar

Beyond the requirement of putting directives on separate lines prefixed with dollar signs, the language itself is very similar to the new one. Except that comments are lines that start with two # characters, and a line-break at the end of a directive is removed automatically.

Usage¶

The basic usage pattern is simple: instantiate one TemplateLoader object and keep it around, then ask it to load a template whenever you need to load one:

from genshi.template import TemplateLoader

loader = TemplateLoader(['/path/to/dir1', '/path/to/dir2'],
                        auto_reload=True)
tmpl = loader.load('test.html')

When you try to load a template that can’t be found, you get a TemplateNotFound error.

The default template class used by the loader is MarkupTemplate, but that can be overridden both with a different default (as a keyword argument to the TemplateLoader constructor), as well as on invocation of the load() method:

from genshi.template.text import NewTextTemplate

tmpl = loader.load('mail.txt', cls=NewTextTemplate)

Caching¶

The TemplateLoader class provides a simple in-memory cache for parsed template objects. This improves performance, because templates do not need to be reparsed every time they are rendered.

The size of this cache can be adjusted using the max_cache_size option on the TemplateLoader constructor. The value of that option determines the maximum number of template objects kept in the cache. When this limit is reached, any templates that haven’t been used in a while get purged. Technically, this is a least-recently-used (LRU) cache, the default limit is set to 25 templates.

from genshi.template import TemplateLoader

loader = TemplateLoader('templates', auto_reload=True, max_cache_size=100)

from genshi.filters import Translator
from genshi.template import TemplateLoader

def template_loaded(template):
    Translator(translations.ugettext).setup(template)

loader = TemplateLoader('templates', callback=template_loaded)

Template Search Path¶

The template loader can be configured with a list of multiple directories to search for templates. The loader maps these directories to a single logical directory for locating templates by file name.

The order of the directories making up the search path is significant: the loader will first try to locate a requested template in the first directory on the path, then in the second, and so on. If there are two templates with the same file name in multiple directories on the search path, whatever file is found first gets used.

Based on this design, an application could, for example, configure a search path consisting of a directory containing the default templates, as well as a directory where site-specific templates can be stored that will override the default templates.

from genshi.template import TemplateLoader, loader
tl = TemplateLoader([loader.directory('/path/to/dir/')])

tl = TemplateLoader(['/path/to/dir/'])

from genshi.template import TemplateLoader, loader
tl = TemplateLoader([loader.package('myapp', 'templates')])

from genshi.template import TemplateLoader, loader
tl = TemplateLoader(loader.prefixed(
  core = '/tmp/dir1',
  plugin1 = loader.package('plugin1', 'templates'),
  plugin2 = loader.package('plugin2', 'templates'),
))
tmpl = tl.load('core/index.html')

Customized Loading¶

If you require a completely different implementation of template loading, you can extend or even replace the builtin TemplateLoader class.

HTML Form Filler¶

The filter genshi.filters.html.HTMLFormFiller can automatically populate an HTML form from values provided as a simple dictionary. When using this filter, you can basically omit any value, selected, or checked attributes from form controls in your templates, and let the filter do all that work for you.

HTMLFormFiller takes a dictionary of data to populate the form with, where the keys should match the names of form elements, and the values determine the values of those controls. For example:

>>> from genshi.filters import HTMLFormFiller
>>> from genshi.template import MarkupTemplate

>>> template = MarkupTemplate("""<form>
...   <p>
...     <label>User name:
...       <input type="text" name="username" />
...     </label><br />
...     <label>Password:
...       <input type="password" name="password" />
...     </label><br />
...     <label>
...       <input type="checkbox" name="remember" /> Remember me
...     </label>
...   </p>
... </form>""")
>>> filler = HTMLFormFiller(data=dict(username='john', remember=True))
>>> print(template.generate() | filler)
<form>
  <p>
    <label>User name:
      <input type="text" name="username" value="john"/>
    </label><br/>
    <label>Password:
      <input type="password" name="password"/>
    </label><br/>
    <label>
      <input type="checkbox" name="remember" checked="checked"/> Remember me
    </label>
  </p>
</form>

The filter will of course also handle radio buttons as well as <select> and <textarea> elements. For radio buttons to be marked as checked, the value in the data dictionary needs to match the value attribute of the <input> element, or evaluate to a truth value if the element has no such attribute. For options in a <select> box to be marked as selected, the value in the data dictionary needs to match the value attribute of the <option> element, or the text content of the option if it has no value attribute. Password and file input fields are not populated, as most browsers would ignore that anyway for security reasons.

You’ll want to make sure that the values in the data dictionary have already been converted to strings. While the filter may be able to deal with non-string data in some cases (such as check boxes), in most cases it will either not attempt any conversion or not produce the desired results.

You can restrict the form filler to operate only on a specific <form> by passing either the id or the name keyword argument to the initializer. If either of those is specified, the filter will only apply to form tags with an attribute matching the specified value.

HTML Sanitizer¶

The filter genshi.filters.html.HTMLSanitizer filter can be used to clean up user-submitted HTML markup, removing potentially dangerous constructs that could be used for various kinds of abuse, such as cross-site scripting (XSS) attacks:

>>> from genshi.filters import HTMLSanitizer
>>> from genshi.input import HTML

>>> html = HTML("""<div>
...   <p>Innocent looking text.</p>
...   <script>alert("Danger: " + document.cookie)</script>
... </div>""")
>>> sanitize = HTMLSanitizer()
>>> print(html | sanitize)
<div>
  <p>Innocent looking text.</p>
</div>

In this example, the <script> tag was removed from the output.

You can determine which tags and attributes should be allowed by initializing the filter with corresponding sets. See the API documentation for more information.

Inline style attributes are forbidden by default. If you allow them, the filter will still perform sanitization on the contents any encountered inline styles: the proprietary expression() function (supported only by Internet Explorer) is removed, and any property using an url() which a potentially dangerous URL scheme (such as javascript:) are also stripped out:

>>> from genshi.filters import HTMLSanitizer
>>> from genshi.input import HTML

>>> html = HTML("""<div>
...   <br style="background: url(javascript:alert(document.cookie); color: #000" />
... </div>""")
>>> sanitize = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style']))
>>> print(html | sanitize)
<div>
  <br style="color: #000"/>
</div>

Transformer¶

The filter genshi.filters.transform.Transformer provides a convenient way to transform or otherwise work with markup event streams. It allows you to specify which parts of the stream you’re interested in with XPath expressions, and then attach a variety of transformations to the parts that match:

>>> from genshi.builder import tag
>>> from genshi.core import TEXT
>>> from genshi.filters import Transformer
>>> from genshi.input import HTML

>>> html = HTML('''<html>
...   <head><title>Some Title</title></head>
...   <body>
...     Some <em>body</em> text.
...   </body>
... </html>''')

>>> print(html | Transformer('body/em').map(unicode.upper, TEXT)
...                                    .unwrap().wrap(tag.u).end()
...                                    .select('body/u')
...                                    .prepend('underlined '))
<html>
  <head><title>Some Title</title></head>
  <body>
    Some <u>underlined BODY</u> text.
  </body>
</html>

This example sets up a transformation that:

1. matches any <em> element anywhere in the body,
2. uppercases any text nodes in the element,
3. strips off the <em> start and close tags,
4. wraps the content in a <u> tag, and
5. inserts the text underlined inside the <u> tag.

A number of commonly useful transformations are available for this filter. Please consult the API documentation a complete list.

In addition, you can also perform custom transformations. For example, the following defines a transformation that changes the name of a tag:

>>> from genshi import QName
>>> from genshi.filters.transform import ENTER, EXIT

>>> class RenameTransformation(object):
...    def __init__(self, name):
...        self.name = QName(name)
...    def __call__(self, stream):
...        for mark, (kind, data, pos) in stream:
...            if mark is ENTER:
...                data = self.name, data[1]
...            elif mark is EXIT:
...                data = self.name
...            yield mark, (kind, data, pos)

A transformation can be any callable object that accepts an augmented event stream. In this case we define a class, so that we can initialize it with the tag name.

Custom transformations can be applied using the apply() method of a transformer instance:

>>> xform = Transformer('body//em').map(unicode.upper, TEXT) \
>>> xform = xform.apply(RenameTransformation('u'))
>>> print(html | xform)
<html>
  <head><title>Some Title</title></head>
  <body>
    Some <u>BODY</u> text.
  </body>
</html>

Translator¶

The genshi.filters.i18n.Translator filter implements basic support for internationalizing and localizing templates. When used as a filter, it translates a configurable set of text nodes and attribute values using a gettext-style translation function.

The Translator class also defines the extract class method, which can be used to extract localizable messages from a template.

Please refer to the API documentation for more information on this filter.

Limitations¶

Due to the streaming nature of the processing model, Genshi uses only a subset of the XPath 1.0 language.

In particular, only the following axes are supported:

• attribute
• child
• descendant
• descendant-or-self
• self

This means you can’t use the parent, ancestor, or sibling axes in Genshi (the namespace axis isn’t supported either, but what you’d ever need that for I don’t know). Basically, any path expression that would require buffering of the stream is not supported.

Predicates are of course supported, but path expressions inside predicates are restricted to attribute lookups (again due to the lack of buffering).

Most of the XPath functions and operators are supported, however they (currently) only work inside predicates. The following functions are not supported:

• count()
• id()
• lang()
• last()
• position()
• string()
• sum()

The mathematical operators (+, -, *, div, and mod) are not yet supported, whereas sub-expressions and the various comparison and logical operators should work as expected.

You can also use XPath variable references ($var) inside predicates.

Querying Streams¶

The Stream class provides a select(path) function that can be used to retrieve subsets of the stream:

>>> from genshi.input import XML

>>> doc = XML('''<doc>
...  <items count="4">
...       <item status="new">
...         <summary>Foo</summary>
...       </item>
...       <item status="closed">
...         <summary>Bar</summary>
...       </item>
...       <item status="closed" resolution="invalid">
...         <summary>Baz</summary>
...       </item>
...       <item status="closed" resolution="fixed">
...         <summary>Waz</summary>
...       </item>
...   </items>
... </doc>''')

>>> print(doc.select('items/item[@status="closed" and '
...     '(@resolution="invalid" or not(@resolution))]/summary/text()'))
BarBaz

Matching in Templates¶

See the directive py:match in the XML Template Language Specification.

Basics¶

The simplest way to internationalize and translate templates would be to wrap all localizable strings in a gettext() function call (which is often aliased to _() for brevity). In that case, no extra template filter is required.

<p>${_("Hello, world!")}</p>

However, this approach results in significant “character noise” in templates, making them harder to read and preview.

The genshi.filters.Translator filter allows you to get rid of the explicit gettext function calls, so you can (often) just continue to write:

<p>Hello, world!</p>

This text will still be extracted and translated as if you had wrapped it in a _() call.

You can control which tags should be ignored by this process; for example, it doesn’t really make sense to translate the content of the HTML <script></script> element. Both <script> and <style> are excluded by default.

Attribute values can also be automatically translated. The default is to consider the attributes abbr, alt, label, prompt, standby, summary, and title, which is a list that makes sense for HTML documents. Of course, you can tell the translator to use a different set of attribute names, or none at all.

<p xml:lang="en">Hello, world!</p>

<html xml:lang="$locale">
  ...
</html>

Template Directives¶

Sometimes localizable strings in templates may contain dynamic parameters, or they may depend on the numeric value of some variable to choose a proper plural form. Sometimes the strings contain embedded markup, such as tags for emphasis or hyperlinks, and you don’t want to rely on the people doing the translations to know the syntax and escaping rules of HTML and XML.

In those cases the simple text extraction and translation process described above is not sufficient. You could just use gettext API functions in embedded Python expressions for parameters and pluralization, but that does not help when messages contain embedded markup. Genshi provides special template directives for internationalization that attempt to provide a comprehensive solution for this problem space.

To enable these directives, you’ll need to register them with the templates they are used in. You can do this by adding them manually via the Template.add_directives(namespace, factory) (where namespace would be “http://genshi.edgewall.org/i18n” and factory would be an instance of the Translator class). Or you can just call the Translator.setup(template) class method, which both registers the directives and adds the translation filter.

After the directives have been registered with the template engine on the Python side of your application, you need to declare the corresponding directive namespace in all markup templates that use them. For example:

<html xmlns:py="http://genshi.edgewall.org/"
      xmlns:i18n="http://genshi.edgewall.org/i18n">
  …
</html>

These directives only make sense in the context of markup templates. For text templates, you can just use the corresponding gettext API calls as needed.

<p>
  Please visit <a href="${site.url}">${site.name}</a> for help.
</p>

<p i18n:msg="name">
  Please visit <a href="${site.url}">${site.name}</a> for help.
</p>

Please visit [1:%(name)s] for help.

Um Hilfe zu erhalten, besuchen Sie bitte [1:%(name)s]

<p>
  Um Hilfe zu erhalten, besuchen Sie bitte
  <a href="http://example.com/">Example</a>
</p>

<p i18n:msg="name">
  <i>Please</i> visit <b>the site <a href="${site.url}">${site.name}</a></b>
  for help.
</p>

[1:Please] visit [2:the site [3:%(name)s]] for help.

Um Hilfe zu erhalten besuchen Sie [1:bitte]
[3:%(name)s], [2:das ist eine Web-Site]

<p>
  Um Hilfe zu erhalten besuchen Sie <i>bitte</i>
  <a href="http://example.com/">Example</a>, <b>das ist eine Web-Site</b>
</p>

<p i18n:choose="len(messages); num">
  <i18n:singular>You have <b>${len(messages)}</b> new message.</i18n:singular>
  <i18n:plural>You have <b>${len(messages)}</b> new messages.</i18n:plural>
</p>

<i18n:choose numeral="len(messages)" params="num">
  <p i18n:singular="">You have <b>${len(messages)}</b> new message.</p>
  <p i18n:plural="">You have <b>${len(messages)}</b> new messages.</p>
</i18n:choose>

<p i18n:msg="name" i18n:comment="Link to the relevant support site">
  Please visit <a href="${site.url}">${site.name}</a> for help.
</p>

<div i18n:domain="examples">
  <p>Hello, world!</p>
</div>

Extraction¶

The Translator class provides a class method called extract, which is a generator yielding all localizable strings found in a template or markup stream. This includes both literal strings in text nodes and attribute values, as well as strings in gettext() calls in embedded Python code. See the API documentation for details on how to use this method directly.

# Python souce
[python:**.py]

# Genshi templates
[genshi:**/templates/**.html]
include_attrs = title

[genshi:**/templates/**.txt]
template_class = genshi.template.TextTemplate
encoding = latin-1

Translation¶

If you have prepared MO files for use with Genshi using the appropriate tools, you can access the message catalogs with the gettext Python module. You’ll probably want to create a gettext.GNUTranslations instance, and make the translation functions it provides available to your templates by putting them in the template context.

The Translator filter needs to be added to the filters of the template (applying it as a stream filter will likely not have the desired effect). Furthermore it needs to be the first filter in the list, including the internal filters that Genshi adds itself:

from genshi.filters import Translator
from genshi.template import MarkupTemplate

template = MarkupTemplate("...")
template.filters.insert(0, Translator(translations.ugettext))

The Translator class also provides the convenience method setup(), which will both add the filter and register the i18n directives:

from genshi.filters import Translator
from genshi.template import MarkupTemplate

template = MarkupTemplate("...")
translator = Translator(translations.ugettext)
translator.setup(template)

Related Considerations¶

If you intend to produce an application that is fully prepared for an international audience, there are a couple of other things to keep in mind:

Introduction¶

Some Python web frameworks support a variety of different templating engines through the Template Engine Plugin API, which was first developed by the Buffet and TurboGears projects.

Genshi supports this API out of the box, so you can use it in frameworks like TurboGears or Pylons without installing any additional packages. A small example TurboGears application is included in the examples directory of source distributions of Genshi.

Usage¶

The way you use Genshi through the plugin API depends very much on the framework you’re using. In general, the approach will look something like the following:

1. Configure Genshi as the default (or an additional) template engine
2. Optionally specify Genshi-specific configuration options
3. For any given view or controller (or whatever those are called in your framework of choice), specify the name of the template to use and which data should be made available to it.

For point 1, you’ll have to specify the name of the template engine plugin. For Genshi, this is “genshi”. However, because Genshi supports both markup and text templates, it also provides two separate plugins, namely “genshi-markup” and “genshi-text” (the “genshi” name is just an alias for “genshi-markup”).

Usually, you can choose a default template engine, but also use a different engine on a per-request basis. So to use markup templates in general, but a text template in a specific controller, you’d configure “genshi” as the default template engine, and specify “genshi-text” for the controllers that should use text templates. How exactly this works depends on the framework in use.

When rendering a specific template in a controller (point 3 above), you may also be able to pass additional options to the plugin. This includes the format keyword argument, which Genshi will use to override the configured default serialization method. In combination with specifying the “genshi-text” engine name as explained above, you would use this to specify the “text” serialization method when you want to use a text template. Or you’d specify “xml” for the format when you want to produce an Atom feed or other XML content.

Configuration Options¶

The plugin API allows plugins to be configured using a dictionary of strings. The following is a list of configuration options that Genshi supports. These may or may not be made available by your framework. TurboGears 1.0, for example, only passes a fixed set of options to all plugins.

Content-Type: text/html; charset=utf-8

genshi.filters.html¶

Implementation of a number of stream filters.

class genshi.filters.html.HTMLFormFiller(name=None, id=None, data=None, passwords=False)¶

A stream filter that can populate HTML forms from a dictionary of values.

>>> from genshi.input import HTML
>>> html = HTML('''<form>
...   <p><input type="text" name="foo" /></p>
... </form>''', encoding='utf-8')
>>> filler = HTMLFormFiller(data={'foo': 'bar'})
>>> print(html | filler)
<form>
  <p><input type="text" name="foo" value="bar"/></p>
</form>

class genshi.filters.html.HTMLSanitizer(safe_tags=frozenset(['em', 'pre', 'code', 'p', 'h2', 'h3', 'h1', 'h6', 'h4', 'h5', 'table', 'font', 'u', 'select', 'kbd', 'strong', 'span', 'sub', 'img', 'area', 'menu', 'tt', 'tr', 'tbody', 'label', 'hr', 'dfn', 'tfoot', 'th', 'sup', 'strike', 'input', 'td', 'samp', 'cite', 'thead', 'map', 'dl', 'blockquote', 'fieldset', 'option', 'form', 'acronym', 'big', 'dd', 'var', 'ol', 'abbr', 'br', 'address', 'optgroup', 'li', 'dt', 'ins', 'legend', 'a', 'b', 'center', 'textarea', 'colgroup', 'i', 'button', 'q', 'caption', 's', 'del', 'small', 'div', 'col', 'dir', 'ul']), safe_attrs=frozenset(['rev', 'prompt', 'color', 'colspan', 'accesskey', 'usemap', 'cols', 'accept', 'datetime', 'char', 'accept-charset', 'shape', 'href', 'hreflang', 'selected', 'frame', 'type', 'alt', 'nowrap', 'border', 'id', 'axis', 'compact', 'rows', 'checked', 'for', 'start', 'hspace', 'charset', 'ismap', 'label', 'target', 'bgcolor', 'readonly', 'rel', 'valign', 'scope', 'size', 'cellspacing', 'cite', 'media', 'multiple', 'src', 'rules', 'nohref', 'action', 'rowspan', 'abbr', 'span', 'method', 'height', 'class', 'enctype', 'lang', 'disabled', 'name', 'charoff', 'clear', 'summary', 'value', 'longdesc', 'headers', 'vspace', 'noshade', 'coords', 'width', 'maxlength', 'cellpadding', 'title', 'align', 'dir', 'tabindex']), safe_schemes=frozenset(['mailto', 'ftp', 'http', 'file', 'https', None]), uri_attrs=frozenset(['src', 'lowsrc', 'href', 'dynsrc', 'background', 'action']))¶

A filter that removes potentially dangerous HTML tags and attributes from the stream.

>>> from genshi import HTML
>>> html = HTML('<div><script>alert(document.cookie)</script></div>', encoding='utf-8')
>>> print(html | HTMLSanitizer())
<div/>

The default set of safe tags and attributes can be modified when the filter is instantiated. For example, to allow inline style attributes, the following instantation would work:

>>> html = HTML('<div style="background: #000"></div>', encoding='utf-8')
>>> sanitizer = HTMLSanitizer(safe_attrs=HTMLSanitizer.SAFE_ATTRS | set(['style']))
>>> print(html | sanitizer)
<div style="background: #000"/>

Note that even in this case, the filter does attempt to remove dangerous constructs from style attributes:

>>> html = HTML('<div style="background: url(javascript:void); color: #000"></div>', encoding='utf-8')
>>> print(html | sanitizer)
<div style="color: #000"/>

This handles HTML entities, unicode escapes in CSS and Javascript text, as well as a lot of other things. However, the style tag is still excluded by default because it is very hard for such sanitizing to be completely safe, especially considering how much error recovery current web browsers perform.

It also does some basic filtering of CSS properties that may be used for typical phishing attacks. For more sophisticated filtering, this class provides a couple of hooks that can be overridden in sub-classes.

Warn:	Note that this special processing of CSS is currently only applied to style attributes, not style elements.

is_safe_css(propname, value)¶

Determine whether the given css property declaration is to be considered safe for inclusion in the output.

Parameters:	propname – the CSS property name value – the value of the property
Returns:	whether the property value should be considered safe
Return type:	bool
Since:	version 0.6

is_safe_elem(tag, attrs)¶

Determine whether the given element should be considered safe for inclusion in the output.

Parameters:	tag (QName) – the tag name of the element attrs (Attrs) – the element attributes
Returns:	whether the element should be considered safe
Return type:	bool
Since:	version 0.6

is_safe_uri(uri)¶

Determine whether the given URI is to be considered safe for inclusion in the output.

The default implementation checks whether the scheme of the URI is in the set of allowed URIs (safe_schemes).

>>> sanitizer = HTMLSanitizer()
>>> sanitizer.is_safe_uri('http://example.org/')
True
>>> sanitizer.is_safe_uri('javascript:alert(document.cookie)')
False

Parameters:	uri – the URI to check
Returns:	True if the URI can be considered safe, False otherwise
Return type:	bool
Since:	version 0.4.3

sanitize_css(text)¶

Remove potentially dangerous property declarations from CSS code.

In particular, properties using the CSS url() function with a scheme that is not considered safe are removed:

>>> sanitizer = HTMLSanitizer()
>>> sanitizer.sanitize_css(u'''
...   background: url(javascript:alert("foo"));
...   color: #000;
... ''')
[u'color: #000']

Also, the proprietary Internet Explorer function expression() is always stripped:

>>> sanitizer.sanitize_css(u'''
...   background: #fff;
...   color: #000;
...   width: e/**/xpression(alert("foo"));
... ''')
[u'background: #fff', u'color: #000']

Parameters:	text – the CSS text; this is expected to be unicode and to not contain any character or numeric references
Returns:	a list of declarations that are considered safe
Return type:	list
Since:	version 0.4.3

genshi.filters.i18n¶

Directives and utilities for internationalization and localization of templates.

class genshi.filters.i18n.Translator(translate=<gettext.NullTranslations instance>, ignore_tags=frozenset([QName('http://www.w3.org/1999/xhtml}style'), QName('http://www.w3.org/1999/xhtml}script'), QName('style'), QName('script')]), include_attrs=frozenset(['prompt', 'title', 'standby', 'summary', 'abbr', 'alt', 'label']), extract_text=True)¶

Can extract and translate localizable strings from markup streams and templates.

For example, assume the following template:

>>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
...   <head>
...     <title>Example</title>
...   </head>
...   <body>
...     <h1>Example</h1>
...     <p>${_("Hello, %(name)s") % dict(name=username)}</p>
...   </body>
... </html>''', filename='example.html')

For demonstration, we define a dummy gettext-style function with a hard-coded translation table, and pass that to the Translator initializer:

>>> def pseudo_gettext(string):
...     return {
...         'Example': 'Beispiel',
...         'Hello, %(name)s': 'Hallo, %(name)s'
...     }[string]
>>> translator = Translator(pseudo_gettext)

Next, the translator needs to be prepended to any already defined filters on the template:

>>> tmpl.filters.insert(0, translator)

When generating the template output, our hard-coded translations should be applied as expected:

>>> print(tmpl.generate(username='Hans', _=pseudo_gettext))
<html>
  <head>
    <title>Beispiel</title>
  </head>
  <body>
    <h1>Beispiel</h1>
    <p>Hallo, Hans</p>
  </body>
</html>

Note that elements defining xml:lang attributes that do not contain variable expressions are ignored by this filter. That can be used to exclude specific parts of a template from being extracted and translated.

extract(stream, gettext_functions=('_', 'gettext', 'ngettext', 'dgettext', 'dngettext', 'ugettext', 'ungettext'), search_text=True, comment_stack=None)¶

Extract localizable strings from the given template stream.

For every string found, this function yields a (lineno, function, message, comments) tuple, where:

• lineno is the number of the line on which the string was found,
• function is the name of the gettext function used (if the string was extracted from embedded Python code), and
• message is the string itself (a unicode object, or a tuple of unicode objects for functions with multiple string arguments).
• comments is a list of comments related to the message, extracted from i18n:comment attributes found in the markup

>>> tmpl = MarkupTemplate('''<html xmlns:py="http://genshi.edgewall.org/">
...   <head>
...     <title>Example</title>
...   </head>
...   <body>
...     <h1>Example</h1>
...     <p>${_("Hello, %(name)s") % dict(name=username)}</p>
...     <p>${ngettext("You have %d item", "You have %d items", num)}</p>
...   </body>
... </html>''', filename='example.html')
>>> for line, func, msg, comments in Translator().extract(tmpl.stream):
...    print('%d, %r, %r' % (line, func, msg))
3, None, u'Example'
6, None, u'Example'
7, '_', u'Hello, %(name)s'
8, 'ngettext', (u'You have %d item', u'You have %d items', None)

Parameters:	stream – the event stream to extract strings from; can be a regular stream or a template stream gettext_functions – a sequence of function names that should be treated as gettext-style localization functions search_text – whether the content of text nodes should be extracted (used internally)
Note:	Changed in 0.4.1: For a function with multiple string arguments (such as ngettext), a single item with a tuple of strings is yielded, instead an item for each string argument.
Note:	Changed in 0.6: The returned tuples now include a fourth element, which is a list of comments for the translator.

setup(template)¶

Convenience function to register the Translator filter and the related directives with the given template.

Parameters:	template – a Template instance

genshi.filters.i18n.extract(fileobj, keywords, comment_tags, options)¶

Babel extraction method for Genshi templates.

Parameters:	fileobj – the file-like object the messages should be extracted from keywords – a list of keywords (i.e. function names) that should be recognized as translation functions comment_tags – a list of translator tags to search for and include in the results options – a dictionary of additional options (optional)
Returns:	an iterator over (lineno, funcname, message, comments) tuples
Return type:	iterator

genshi.filters.transform¶

A filter for functional-style transformations of markup streams.

The Transformer filter provides a variety of transformations that can be applied to parts of streams that match given XPath expressions. These transformations can be chained to achieve results that would be comparitively tedious to achieve by writing stream filters by hand. The approach of chaining node selection and transformation has been inspired by the jQuery Javascript library.

For example, the following transformation removes the <title> element from the <head> of the input document:

>>> from genshi.builder import tag
>>> html = HTML('''<html>
...  <head><title>Some Title</title></head>
...  <body>
...    Some <em>body</em> text.
...  </body>
... </html>''',
... encoding='utf-8')
>>> print(html | Transformer('body/em').map(unicode.upper, TEXT)
...                                    .unwrap().wrap(tag.u))
<html>
  <head><title>Some Title</title></head>
  <body>
    Some <u>BODY</u> text.
  </body>
</html>

The Transformer support a large number of useful transformations out of the box, but custom transformations can be added easily.

class genshi.filters.transform.Transformer(path='.')¶

Stream filter that can apply a variety of different transformations to a stream.

This is achieved by selecting the events to be transformed using XPath, then applying the transformations to the events matched by the path expression. Each marked event is in the form (mark, (kind, data, pos)), where mark can be any of ENTER, INSIDE, EXIT, OUTSIDE, or None.

The first three marks match START and END events, and any events contained INSIDE any selected XML/HTML element. A non-element match outside a START/END container (e.g. text()) will yield an OUTSIDE mark.

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')

Transformations act on selected stream events matching an XPath expression. Here’s an example of removing some markup (the title, in this case) selected by an expression:

>>> print(html | Transformer('head/title').remove())
<html><head/><body>Some <em>body</em> text.</body></html>

Inserted content can be passed in the form of a string, or a markup event stream, which includes streams generated programmatically via the builder module:

>>> from genshi.builder import tag
>>> print(html | Transformer('body').prepend(tag.h1('Document Title')))
<html><head><title>Some Title</title></head><body><h1>Document
Title</h1>Some <em>body</em> text.</body></html>

Each XPath expression determines the set of tags that will be acted upon by subsequent transformations. In this example we select the <title> text, copy it into a buffer, then select the <body> element and paste the copied text into the body as <h1> enclosed text:

>>> buffer = StreamBuffer()
>>> print(html | Transformer('head/title/text()').copy(buffer)
...     .end().select('body').prepend(tag.h1(buffer)))
<html><head><title>Some Title</title></head><body><h1>Some Title</h1>Some
<em>body</em> text.</body></html>

Transformations can also be assigned and reused, although care must be taken when using buffers, to ensure that buffers are cleared between transforms:

>>> emphasis = Transformer('body//em').attr('class', 'emphasis')
>>> print(html | emphasis)
<html><head><title>Some Title</title></head><body>Some <em
class="emphasis">body</em> text.</body></html>

after(content)¶

Insert content after selection.

Here, we insert some text after the </em> closing tag:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em').after(' rock'))
<html><head><title>Some Title</title></head><body>Some <em>body</em>
rock text.</body></html>

Parameters:	content – Either a callable, an iterable of events, or a string to insert.
Return type:	Transformer

append(content)¶

Insert content before the END event of the selection.

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//body').append(' Some new body text.'))
<html><head><title>Some Title</title></head><body>Some <em>body</em>
text. Some new body text.</body></html>

Parameters:	content – Either a callable, an iterable of events, or a string to insert.
Return type:	Transformer

apply(function)¶

Apply a transformation to the stream.

Transformations can be chained, similar to stream filters. Any callable accepting a marked stream can be used as a transform.

As an example, here is a simple TEXT event upper-casing transform:

>>> def upper(stream):
...     for mark, (kind, data, pos) in stream:
...         if mark and kind is TEXT:
...             yield mark, (kind, data.upper(), pos)
...         else:
...             yield mark, (kind, data, pos)
>>> short_stream = HTML('<body>Some <em>test</em> text</body>',
...                      encoding='utf-8')
>>> print(short_stream | Transformer('.//em/text()').apply(upper))
<body>Some <em>TEST</em> text</body>

attr(name, value)¶

Add, replace or delete an attribute on selected elements.

If value evaulates to None the attribute will be deleted from the element:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em class="before">body</em> <em>text</em>.</body>'
...             '</html>', encoding='utf-8')
>>> print(html | Transformer('body/em').attr('class', None))
<html><head><title>Some Title</title></head><body>Some <em>body</em>
<em>text</em>.</body></html>

Otherwise the attribute will be set to value:

>>> print(html | Transformer('body/em').attr('class', 'emphasis'))
<html><head><title>Some Title</title></head><body>Some <em
class="emphasis">body</em> <em class="emphasis">text</em>.</body></html>

If value is a callable it will be called with the attribute name and the START event for the matching element. Its return value will then be used to set the attribute:

>>> def print_attr(name, event):
...     attrs = event[1][1]
...     print(attrs)
...     return attrs.get(name)
>>> print(html | Transformer('body/em').attr('class', print_attr))
Attrs([(QName('class'), u'before')])
Attrs()
<html><head><title>Some Title</title></head><body>Some <em
class="before">body</em> <em>text</em>.</body></html>

Parameters:	name – the name of the attribute value – the value that should be set for the attribute.
Return type:	Transformer

before(content)¶

Insert content before selection.

In this example we insert the word ‘emphasised’ before the <em> opening tag:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em').before('emphasised '))
<html><head><title>Some Title</title></head><body>Some emphasised
<em>body</em> text.</body></html>

Parameters:	content – Either a callable, an iterable of events, or a string to insert.
Return type:	Transformer

buffer()¶

Buffer the entire stream (can consume a considerable amount of memory).

Useful in conjunction with copy(accumulate=True) and cut(accumulate=True) to ensure that all marked events in the entire stream are copied to the buffer before further transformations are applied.

For example, to move all <note> elements inside a <notes> tag at the top of the document:

>>> doc = HTML('<doc><notes></notes><body>Some <note>one</note> '
...            'text <note>two</note>.</body></doc>',
...             encoding='utf-8')
>>> buffer = StreamBuffer()
>>> print(doc | Transformer('body/note').cut(buffer, accumulate=True)
...     .end().buffer().select('notes').prepend(buffer))
<doc><notes><note>one</note><note>two</note></notes><body>Some  text
.</body></doc>

copy(buffer, accumulate=False)¶

Copy selection into buffer.

The buffer is replaced by each contiguous selection before being passed to the next transformation. If accumulate=True, further selections will be appended to the buffer rather than replacing it.

>>> from genshi.builder import tag
>>> buffer = StreamBuffer()
>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('head/title/text()').copy(buffer)
...     .end().select('body').prepend(tag.h1(buffer)))
<html><head><title>Some Title</title></head><body><h1>Some
Title</h1>Some <em>body</em> text.</body></html>

This example illustrates that only a single contiguous selection will be buffered:

>>> print(html | Transformer('head/title/text()').copy(buffer)
...     .end().select('body/em').copy(buffer).end().select('body')
...     .prepend(tag.h1(buffer)))
<html><head><title>Some Title</title></head><body><h1>Some
Title</h1>Some <em>body</em> text.</body></html>
>>> print(buffer)
<em>body</em>

Element attributes can also be copied for later use:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body><em>Some</em> <em class="before">body</em>'
...             '<em>text</em>.</body></html>',
...             encoding='utf-8')
>>> buffer = StreamBuffer()
>>> def apply_attr(name, entry):
...     return list(buffer)[0][1][1].get('class')
>>> print(html | Transformer('body/em[@class]/@class').copy(buffer)
...     .end().buffer().select('body/em[not(@class)]')
...     .attr('class', apply_attr))
<html><head><title>Some Title</title></head><body><em
class="before">Some</em> <em class="before">body</em><em
class="before">text</em>.</body></html>

Parameters:	buffer – the StreamBuffer in which the selection should be stored
Return type:	Transformer
Note:	Copy (and cut) copy each individual selected object into the buffer before passing to the next transform. For example, the XPath *|text() will select all elements and text, each instance of which will be copied to the buffer individually before passing to the next transform. This has implications for how StreamBuffer objects can be used, so some experimentation may be required.

cut(buffer, accumulate=False)¶

Copy selection into buffer and remove the selection from the stream.

>>> from genshi.builder import tag
>>> buffer = StreamBuffer()
>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em/text()').cut(buffer)
...     .end().select('.//em').after(tag.h1(buffer)))
<html><head><title>Some Title</title></head><body>Some
<em/><h1>body</h1> text.</body></html>

Specifying accumulate=True, appends all selected intervals onto the buffer. Combining this with the .buffer() operation allows us operate on all copied events rather than per-segment. See the documentation on buffer() for more information.

Parameters:	buffer – the StreamBuffer in which the selection should be stored
Return type:	Transformer
Note:	this transformation will buffer the entire input stream

empty()¶

Empty selected elements of all content.

Example:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em').empty())
<html><head><title>Some Title</title></head><body>Some <em/>
text.</body></html>

Return type:	Transformer

end()¶

End current selection, allowing all events to be selected.

Example:

>>> html = HTML('<body>Some <em>test</em> text</body>', encoding='utf-8')
>>> print(html | Transformer('//em').end().trace())
('OUTSIDE', ('START', (QName('body'), Attrs()), (None, 1, 0)))
('OUTSIDE', ('TEXT', u'Some ', (None, 1, 6)))
('OUTSIDE', ('START', (QName('em'), Attrs()), (None, 1, 11)))
('OUTSIDE', ('TEXT', u'test', (None, 1, 15)))
('OUTSIDE', ('END', QName('em'), (None, 1, 19)))
('OUTSIDE', ('TEXT', u' text', (None, 1, 24)))
('OUTSIDE', ('END', QName('body'), (None, 1, 29)))
<body>Some <em>test</em> text</body>

Returns:	the stream augmented by transformation marks
Return type:	Transformer

filter(filter)¶

Apply a normal stream filter to the selection. The filter is called once for each contiguous block of marked events.

>>> from genshi.filters.html import HTMLSanitizer
>>> html = HTML('<html><body>Some text<script>alert(document.cookie)'
...             '</script> and some more text</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('body/*').filter(HTMLSanitizer()))
<html><body>Some text and some more text</body></html>

Parameters:	filter – The stream filter to apply.
Return type:	Transformer

invert()¶

Invert selection so that marked events become unmarked, and vice versa.

Specificaly, all marks are converted to null marks, and all null marks are converted to OUTSIDE marks.

>>> html = HTML('<body>Some <em>test</em> text</body>', encoding='utf-8')
>>> print(html | Transformer('//em').invert().trace())
('OUTSIDE', ('START', (QName('body'), Attrs()), (None, 1, 0)))
('OUTSIDE', ('TEXT', u'Some ', (None, 1, 6)))
(None, ('START', (QName('em'), Attrs()), (None, 1, 11)))
(None, ('TEXT', u'test', (None, 1, 15)))
(None, ('END', QName('em'), (None, 1, 19)))
('OUTSIDE', ('TEXT', u' text', (None, 1, 24)))
('OUTSIDE', ('END', QName('body'), (None, 1, 29)))
<body>Some <em>test</em> text</body>

Return type:	Transformer

map(function, kind)¶

Applies a function to the data element of events of kind in the selection.

>>> html = HTML('<html><head><title>Some Title</title></head>'
...               '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('head/title').map(unicode.upper, TEXT))
<html><head><title>SOME TITLE</title></head><body>Some <em>body</em>
text.</body></html>

Parameters:	function – the function to apply kind – the kind of event the function should be applied to
Return type:	Transformer

prepend(content)¶

Insert content after the ENTER event of the selection.

Inserting some new text at the start of the <body>:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//body').prepend('Some new body text. '))
<html><head><title>Some Title</title></head><body>Some new body text.
Some <em>body</em> text.</body></html>

Parameters:	content – Either a callable, an iterable of events, or a string to insert.
Return type:	Transformer

remove()¶

Remove selection from the stream.

Example:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em').remove())
<html><head><title>Some Title</title></head><body>Some
text.</body></html>

Return type:	Transformer

rename(name)¶

Rename matching elements.

>>> html = HTML('<html><body>Some text, some more text and '
...             '<b>some bold text</b></body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('body/b').rename('strong'))
<html><body>Some text, some more text and <strong>some bold text</strong></body></html>

replace(content)¶

Replace selection with content.

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//title/text()').replace('New Title'))
<html><head><title>New Title</title></head><body>Some <em>body</em>
text.</body></html>

Parameters:	content – Either a callable, an iterable of events, or a string to insert.
Return type:	Transformer

select(path)¶

Mark events matching the given XPath expression, within the current selection.

>>> html = HTML('<body>Some <em>test</em> text</body>', encoding='utf-8')
>>> print(html | Transformer().select('.//em').trace())
(None, ('START', (QName('body'), Attrs()), (None, 1, 0)))
(None, ('TEXT', u'Some ', (None, 1, 6)))
('ENTER', ('START', (QName('em'), Attrs()), (None, 1, 11)))
('INSIDE', ('TEXT', u'test', (None, 1, 15)))
('EXIT', ('END', QName('em'), (None, 1, 19)))
(None, ('TEXT', u' text', (None, 1, 24)))
(None, ('END', QName('body'), (None, 1, 29)))
<body>Some <em>test</em> text</body>

Parameters:	path – an XPath expression (as string) or a Path instance
Returns:	the stream augmented by transformation marks
Return type:	Transformer

substitute(pattern, replace, count=1)¶

Replace text matching a regular expression.

Refer to the documentation for re.sub() for details.

>>> html = HTML('<html><body>Some text, some more text and '
...             '<b>some bold text</b>\n'
...             '<i>some italicised text</i></body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('body/b').substitute('(?i)some', 'SOME'))
<html><body>Some text, some more text and <b>SOME bold text</b>
<i>some italicised text</i></body></html>
>>> tags = tag.html(tag.body('Some text, some more text and\n',
...      Markup('<b>some bold text</b>')))
>>> print(tags.generate() | Transformer('body').substitute(
...     '(?i)some', 'SOME'))
<html><body>SOME text, some more text and
<b>SOME bold text</b></body></html>

Parameters:	pattern – A regular expression object or string. replace – Replacement pattern. count – Number of replacements to make in each text fragment.
Return type:	Transformer

trace(prefix='', fileobj=None)¶

Print events as they pass through the transform.

>>> html = HTML('<body>Some <em>test</em> text</body>', encoding='utf-8')
>>> print(html | Transformer('em').trace())
(None, ('START', (QName('body'), Attrs()), (None, 1, 0)))
(None, ('TEXT', u'Some ', (None, 1, 6)))
('ENTER', ('START', (QName('em'), Attrs()), (None, 1, 11)))
('INSIDE', ('TEXT', u'test', (None, 1, 15)))
('EXIT', ('END', QName('em'), (None, 1, 19)))
(None, ('TEXT', u' text', (None, 1, 24)))
(None, ('END', QName('body'), (None, 1, 29)))
<body>Some <em>test</em> text</body>

Parameters:	prefix – a string to prefix each event with in the output fileobj – the writable file-like object to write to; defaults to the standard output stream
Return type:	Transformer

unwrap()¶

Remove outermost enclosing elements from selection.

Example:

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em').unwrap())
<html><head><title>Some Title</title></head><body>Some body
text.</body></html>

Return type:	Transformer

wrap(element)¶

Wrap selection in an element.

>>> html = HTML('<html><head><title>Some Title</title></head>'
...             '<body>Some <em>body</em> text.</body></html>',
...             encoding='utf-8')
>>> print(html | Transformer('.//em').wrap('strong'))
<html><head><title>Some Title</title></head><body>Some
<strong><em>body</em></strong> text.</body></html>

Parameters:	element – either a tag name (as string) or an Element object
Return type:	Transformer