Orbeon Forms User Guide

Performance and Tuning

1. Tuning the Java Virtual Machine (JVM)

1.1. Set -Xms and -Xmx to the same value

The heap is a section of memory used by the JVM to store Java objects. You can set constraints on the size of the heap with two parameters passed to the JVM at startup: -Xms sets the initial size of the heap and -Xmx sets the maximum size of the heap. If you set those two parameters to different values, say 512MB and 1GB, you tell the JVM to start with a heap size of 512MB and increase the size up to 1GB "as necessary". In this case, the JVM has to balance two conflicting constraints: not request too much memory from the operating system (getting too fast to 1GB), and not request too little as it would increase the amount of time the to spends on garbage collection which would will reduce the performance of your application.

Asking the JVM to balance memory usage and performance by setting different values for -Xms and -Xmx is very reasonable for desktop applications where a number of applications are running and competing for resources, in particular memory. In a server environment, you often have one or two major applications running on the server, like the JVM for your application serer and maybe a database server. In this situation you have more control over how much memory can be used by each application, and we recommend you set both -Xms and -Xmx to the same value.

1.2. Allocate a large heap but don't cause swapping

The larger the heap, the faster your application will be get. This for two reasons: first, the JVM garbage collector works more efficiently with a larger heap, and second, this enables you to increase the size of the Orbeon Forms cache (more on this later) which will also improve the performance of your application. However, don't use a heap size so large that it would cause swapping, as this would then drastically reduce the performance of your application.

We recommend that you first set the heap size based on how much memory the server has and what other major applications are running. Say you have 2GB of physical memory, and no other major application: then you could set the heap to 1.5GB, which leaves 512MB to the operating system and minor applications. Say you have 4GB of physical memory and also a database running on the same server, then you can set the heap size to 2GB, assign 1.5GB to the database server, and leanve 512MB to the operating and minor applications.

Then, with a "reasonable" setting in place, monitor the server under normal load and look if the machine is swapping or if on the contrary the operating system is reporting a lot of available memory. In the first case, reduce the heap size. In the second, increase it.

2. Tuning the application server

2.1. WebLogic: Disable automatic redeployment

WebLogic can be configured to check on a regular basis if the files of your application have changed on disk, and redeploy the application if they did. Redeploying at application server level is useful when you change the JAR files or some of the underlying configuration files, like the web.xml. As checking if files have changed is incredibly time consuming with WebLogic, and as you are pretty unlikely to change any of those files on a regular basis, we recommend you disable the automatic web application redeployment feature, which is enabled by default.

To do this, after you have installed your Orbeon Forms application, stop WebLogic, and open the config.xml file in an editor. Look for the <WebAppComponent Name="orbeon"> element and add the attribute: ServletReloadCheckSecs="-1".

2.2. All application servers: Disable DNS lookup

You can configure your application server to perform a DNS lookup for every HTTP request. The server always know the IP address of the machine where the HTTP request originated. However, to get the name, the application server needs to send a DNS lookup query to the DNS server. In most cases, performing this query only has a negligible impact on performance. However, the request can take a significant amount of time in certain cases where the network from which the request originated is badly configured. In most case, the application server is doing DNS lookups for "aesthetic reasons": that is to able to in include in the logs the name of the client's machines, instead of their IP address (note that web analysis tools can usually do this reverse DNS lookup much more efficiently when analysing log files subsequently, typically on a daily basis). So we recommend you change the configuration of your application server to disable DNS lookup, which is in general enabled by default.

On Tomcat 5 (external documentation), look for the enableLookups attribute on the <Connector> element and set it to false. If the attribute is not present, add it and set it to false (the default value is true).

2.3. Enable gzip compression for generated text content

HTML and XML content usually compresses extremely well using gzip. You can obtain sizes for the content sent by the server to the web browser that are up to 10 times smaller. A very complex XForms page taking 100 KB, for example, may end up taking only about 10 KB. This means that the data will reach the web browser up to 10 times faster than without gzip compression.

Most web and application servers support gzip compression. For example, Tomcat 5 supports the attribute gzip on the <Connector> element. For more information, please visit the Tomcat HTTP connector documentation.

2.4. Reduce the number of concurrent processing threads

Servlet containers like Tomcat, or application servers like WebLogic, by default allow a very large number of concurrent threads to enter a servlet. For Tomcat, the default is 200. This means that memory usage cannot be effectively bound. It is enough to have a few requests slightly overlapping to cause extra memory consuption that can lead to Out of Memory errors. In addition, extra memory usage leads to poorer performance.

It is recommended to start by changing the container parameters that control threads. With Tomcat (see the documentation), try first:

  • maxThreads="3"
  • acceptCount="50"

This setting sets a small maximum of 3 concurrent requests, and 50 more requests being queued. This ensures that memory usage does not go out of control.

Then, experiment with a tool like Apache JMeter to test load, and adjust those parameters, as well as your virtual machine heap, until you get results that are satisfactory as far as memory consumption and number of concurrent requests are concerned.

Note that it is typical for Java virtual machines to have maximum heap sizes of 256 MB to 1.5 GB in production.

3. Tunning Orbeon Forms

3.1. Tune the Orbeon Forms cache size

One way to increase the performance of your application is to increase the size of the Orbeon Forms cache. You setup the size of the Orbeon Forms cache with the oxf.cache.size property. Due to limitations of the JVM, you cannot set the size of the Orbeon Forms cache in MB. Instead, the value you specify the maximum number of objects that Orbeon Forms can store in cache. As the size of each object stored in cache is different and the average size of those objects can change widely depending on your application, we can't give you an equivalence between number of objects and memory used. Instead, we recommend you follow the suggestions below to tune your Orbeon Forms cache size.

First keep the default value for the Orbeon Forms cache size and follow the suggestions above to set your heap size. Then, adding the -verbosegc parameter, monitor how the heap usage varies under typical load. In particular, pay a close attention to how much memory is freed by a full GC done when the memory used gets close the heap size. Note that a full GC will not get rid of objects in the Orbeon Forms cache. So assuming your heap size is 2GB, if after getting close to 2GB a full GC brings back the heap usage to 0.5GB, as the cached objects are included in those 0.5GB, it means you can and should to increase the size of your Orbeon Forms cache.

However, you should not increase the size to a point where a full GC can only reclaim a small amount of memory, as this will trigger more frequent full GC and so degrade the performance of your application. Typically, after the application has been used for a while, you want a full GC to be able to reclaim 20 to 30% of the heap. If your heap size is 2GB, this means reducing the heap usage to a value around 1.5GB range. The optimal value will change depending on your JVM, server, and application, so you might want to try multiple values and see how they impact performance.

3.2. Disable Saxon Source Location

Make sure the following property is set to false in properties.xml:

<property as="xs:boolean" processor-name="oxf:saxon8" name="generate-source-location" value="false"/>

While this property should be set to true during development in order to obtain better line number information, it has a performance impact. To optimize for speed, it should be set to false.

4. Tunning your application

4.1. Reduce the size of your XML documents

With XML, it is very easy to add data to an existing document and then extract just the data you need from that document. This creates a tendency for the size of the documents manipulated by your application to grow as you progress on the development of your application. Who has never said "let's just add this information to this existing document", or "let's keep this information in the document and pass it around; you never know, we might need it in the future". While this might be just fine in some cases, you need to make sure that the size of your documents does not increase to the point where performance is impacted. If you uncover a performance issue, checking and reducing the size of the documents you manipulate is one of the first thing you might want to do.

If you need to be further convinced, consider an application where pages are generated based on some information contained in an XML schema. This XML schema is stored in an XML database and takes about a 100KB or 4000 lines when serialized. Because data contained in the file is needed in multiple locations, the file is passed around in a number of pipelines while generating a page, and is used overall as input to 10 processors. Each processor will create its own representation of the data in memory, which can take 10 times the size of the serialized XML. That means that each processor has to allocate 1MB of objects and do some processing one those objects. At the end of the request, 10MB of memory have been allocated to process this data, and the garbage collector will eventually have to spend CPU cycles on freeing this memory. What if out of the 4000 lines, only 400 are actually used? Starting by extracting those 400 lines and then passing only those to the processors means that the processors now need to do only one tenth of the work they were doing before. Cleary this type of modification can drastically improve performance.

If you are required to work with large documents, also consider using an XML database such as the open source eXist database, and delegate complex queries to the database: this should be more efficient than continually retrieving large XML documents and processing them in Orbeon Forms.

4.2. Tune your XSLT code

Some operations in XSLT or XPath can be very expensive. This in particular the case for XPath expressions where the engine has to go through the whole input document to evaluate the expression. Consider this XPath expression: //person. To evaluate it, the engine iterates over every element in the document looking for a <person>. If you know that given the structure of the document, a person is inside a department, which is in a company, you can rewrite this as /company/department/person, which will typically run more efficiently.

4.3. Enable XPL profiling

XPL profiling has been introduced with Orbeon Forms 3.0 to give you detailed information on how much time is being spent in each processor involved to generate a web page. When enabled, for each HTTP request, the XPL profiler will output a tree with all the processor calls. Each node of the tree represents the execution of a processor. When a processor starts other processors, those are represented as child nodes in the tree. With each node, the profiler outputs 2 numbers: the first is for the time spent specifically by this processor; the second is for the time spent cumulatively by this processor and all its children; both are in milliseconds.

By default, XPL profiling is disabled. To enable XPL profiling, configure processor.trace, processor.trace.host, and processor.trace.port as described in the properties documentation.

4.4. Do not run your code in the examples portal

Note

This applies to versions of Orbeon Forms prior to 3.5.

The examples portal in Orbeon Forms 3.0 (i.e. the home page of Orbeon Forms, which lists all the Orbeon Forms examples), while convenient to run examples, is not implemented optimally and relies on portlets which cause overhead. The first thing to do is make sure that your code runs outside of the examples portal, i.e. that your page flow entries are present in the main page flow file (resources/page-flow.xml).

4.5. Customize the standard epilogue

The standard Orbeon Forms epilogue can be optimized for your own needs. For example:

  • If you do not use XForms Classic, you can remove the test for the XForms classic engine. (If you are wondering what XForms Classic is, you are probably not using it.)
  • If you do not use "widgets" (i.e. tabs, as this is the only standard "widget" at this point), you can remove the inclusion of xforms-widgets.xsl.
  • If you are not using portlets (which is likely), you can remove the test for portlets.
  • If you are using the theme-plain.xsl theme and have not modified it, you can bypass completely the theme stylesheet.

4.6. Make sure you remove all your XPL debug attributes

While using debug attributes is one of the best ways to debug XPL pipelines, those also have an impact on performance as they locally disable XPL caching and also require time to serialize XML documents to your logger. For performance testing and production, always remove all the debug attributes.

4.7. Don't serve your static files through Orbeon Forms

It is overkill to serve static files such as static images through Orbeon Forms, which is optimized to run XPL pipelines. Instead, use your servlet container's facilities for serving static files, or even better, use a simple web server such as Apache Server.

5. Tuning XForms

5.1. Enable minimal resources

Make sure that minimal JavaScript and CSS resources are enabled, as they will load faster in the user's web browser. See the XForms reference for details. Also enable gzip compression in addition.

5.2. Enable combined resources

Make sure that combined JavaScript and CSS resources are enabled, as they will load faster in the user's web browser when the resources are not yet in browser cache. See the XForms reference for details. Also enable combined resources caching and gzip compression in addition.

5.3. Consider using read-only and shared instances

Some XForms instances that never change or that are simply replaced during a submission can be set as read-only, and can also optionally be shared between pages and even applications. Such instances take less memory and are more efficient to build. However they cannot hold data that changes over time and they cannot use XForms Model Item Properties (MIPs). See the XForms reference for details.

5.4. Consider not refreshing sets of items on selection controls

The xxforms:refresh-items attribute on selection controls allows for performance improvements in certain cases. See Controlling Item Sets Refreshes with xxforms:refresh-items.

5.5. Consider server-side XForms state handling

The state of the XForms engine can be either stored on the server, or sent to the client and exchanged with the server with each request. By default, the XForms state is stored on the client. This option works in most cases and doesn't require any tuning. However, you might want to consider storing the XForms state on the server if your XForms instances are getting large or if your users have low bandwidth connection to the server. See the XForms reference for the benefits and drawbacks of each option and for details on how to configure state handling.

5.6. Use xforms:instance to load dynamic instances

There are several ways of initializating XForms instances. For instances whose content is generated dynamically, try to use the src attribute of the xforms:instance instead of xi:include or XSLT. Alternatively, you can use a submission upon receiving xforms-ready. This helps ensures that the source XForms page is cacheable.

5.7. Tune XForms caches

[TODO]

6. Other recommendations

6.1. Use a performance analysis tool

To obtain your numbers, use a tool such as Apache JMeter. Be sure to warm up your Java VM first and to let the tool run for a significant number of sample before recording your performance numbers.

If you feel comfortable with the source code of Orbeon Forms, you can also use a Java profiler such as YourKit to figure out if a particular part of the Orbeon Forms platform is a bottleneck.

6.2. Make sure of what you are measuring

If you are testing the performance of an application that talks to a database or backend services, be sure to be able to determine how much time your front-end or presentation layer, versus your backend and data layers, are respectively taking.