## The 2.4 Servlet Specfication

### The Good, the Bad and the Ugly Greg Wilkins

## 1. Introduction

After a longer and more difficult process than expected, the 2.4 version of the Servlet Specification has been completed by JSR-154 and is due for release before the end of 2003. The aims of this revision of the specification were some modest feature enhancements, improved deployment descriptors and general clarifications of existing functionality. These aims, with some notable exceptions have largely been met. This article reviews the most important additions, changes, omissions and remaining problems in the specification. The key new features are introduced and some explanations are given for those that are missing in action. Problem areas that remain in the specification and those that have newly been added are also discussed.

## 2. New Features

Developers fearing revolution, need not be worried by the 2.4 servlet specification as it introduces no new functionality. However, existing mechanisms have been extended and enhanced with sometime powerful results. The short list of new features includes:

- XML Schema definition of the deployment descriptor.

- Increased (improved?) integration with other J2EE specification and components.

- Filtering of RequestDispatchers and Error pages.

- Event listeners for requests and request attributes.

- Methods to query client connection details.

- Improved internationalization methods.

Of these changes, only those to the RequestDispatcher and internationalization methods are likely to be relevant to your average servlet developer.

### 2.1. XML Schema for web.xml

The 2.4 servlet deployment descriptor is now defined by an XML schema rather that a DTD (as used for all previous versions). Relatively few new elements are introduced in 2.4, but the schema's means that there are some subtle yet significant changes to all elements in the descriptor.

#### 2.1.1. Schema's Declaration

For many developers the change to schema's will mean little more than an change of the mantra used at the start of the web.xml file ( and an increase in the number of times they download xercesImpl.jar). To use the 2.4 schema's, the XML envelope of the deployment descriptor must change from the DTD form of:

>     <!DOCTYPE web-app PUBLIC
>      "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
>      "http://java.sun.com/dtd/web-app_2_3.dtd" >
>     <web-app>
>         <!-- deployment elements go here -->
>     </web-app>

to the even less readable:

>     <?xml version="1.0" encoding="ISO-8859-1"?>
>     <web-app
>      xmlns="http://java.sun.com/xml/ns/j2ee"
>      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>      xsi:schemaLocation=
>      "http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"
>      version="2.4">
>         <!-- deployment elements go here -->
>     </web-app>

Most developers need not concern themselves with the exact meaning of this envelope and can simply copy it verbatim or indeed have it generated for them from their IDE, by Xdoclet or similar. The older DTD based descriptors are still supported, but the new features of the 2.4 deployment schema will not be usable unless the 2.4 envelope is used.

#### 2.1.2. Validation

With previous DTD based versions of the deployment descriptor, validation was optional and many containers defaulted to no validation. But with the introduction of schema's, validation has been made a requirement for all J2EE containers:

> SRV.13.2 “The containers and tools that are part of J2EE technology-compliant implementation are required to validate deployment descriptor against the XML schema for structural correctness. The validation is recommended, but not required for the web containers and tools that are not part of J2EE technology-compliant implementation.”

Schema validation is significantly more powerful than DTD validation and more than simply document structure is checked:

- String values such as mime-types and URL patterns are checked for allowed characters.

- Numerical values are check for numeric only characters and for the allowed sign.

- Names for elements such as Servlet, Filter, EJB-Ref, etc. are checked for uniqueness.

- Enumerations (e.g. user-data-constraints: NONE, INTEGRAL & CONFIDENTIAL) are checked for allowed values.

The increase rigour of schema validation should assist development by detecting errors earlier in the process and reduce portability issues as less container specific validation implementation is required. Because of the accepting nature of unvalidated XML, it is likely that many validation errors will be detected when converting descriptors that were generated by hand and never validated. For developers that generate their descriptors or use XML aware editors, the transition should be mostly painless.

#### 2.1.3. Order Independence

One of the reasons that validation is often turned off for previous servlet versions, is that the DTDs impose a strict order of top level elements in the web.xml. For example, it is not legal to place a servlet mappings directly after the servlets declaration if you have more than one servlet. The following descriptor has a logical ordering that is illegal with the DTD based validation:

    <web-app>
      ...
      <servlet>
        <servlet-name>ServletA</servlet-name>
        <servlet-class>org.mortbay.servlet.ServletA</servlet-class>
      </servlet>

      <servlet-mapping>
        <servlet-name>ServletA</servlet-name>
        <url-pattern>/aaa/*</url-pattern>
      </servlet-mapping>

      <servlet> <!-- ILLEGAL in specs<=2.3 as all servlet elements must be together -->
        <servlet-name>ServletB</servlet-name>
        <servlet-class>org.mortbay.servlet.ServletB</servlet-class>
      </servlet>

      <servlet-mapping>
        <servlet-name>ServletB</servlet-name>
        <url-pattern>/bbb/*</url-pattern>
      </servlet-mapping>
      ...
    </web-app>

The 2.4 schema has been written so that all top level elements may now occur in any order and in any number. So the above descriptor will now be legal and more logical descriptor layouts and simpler generation will be possible. It still would have been better if the original descriptor specification had allowed url-patterns in the `<servlet>` element, but that's history now.

As a side effect of arbitrary element ordering, all top level elements may now occur 0 or many times. While this is logical for most elements, some additional interpretation is required for some elements:

- Multiple instances of the list elements like welcome-file-list and the new locale-encoding-mapping-list result in concatenated lists.

- Multiple instances of `<jsp-config>`, `<login-config>` and `<session-config>` are errors that must be detected and reported by the container rather than XML validation.

- Multiple instances of the distributable element are treated the same as a single occurrence.

It is hoped that this relaxation of ordering will avoid a major source of invalid descriptors now that validation is required for J2EE containers.

#### 2.1.4. New Descriptor Elements

There are only a few new elements in the 2.4 deployment descriptor, which are summarized below.

Table 1. New Deployment Descriptor Elements

New Element | Parent Element | Description
`<dispatcher>` | `<filter-mapping>` | Defines the request type that a filter mapping applies to (see 2.2 below).
`<locale-encoding-mapping-list>` | `<web-app>` | Defines list of locale to character encoding mappings.
`<locale-encoding-mapping>` | `<locale-encoding-mapping-list>` | Defines entry in locale encoding list.
`<locale>` | `<locale-encoding-mapping>` | The name of a locale in a locale encoding mapping.
`<encoding>` | `<locale-encoding-mapping>` | The character encoding of a locale encoding mapping.
`<message-destination>` | `<web-app>` | Declaration of a message destination resource. Defined in J2EE specification.
`<message-destination-ref>` | `<web-app>` | Declaration of a reference to a message destination resource. Defined in J2EE specification.
`<service-ref>` | `<web-app>` | Declares a reference to a Web service. Defined in J2EE specification.

The nested elements of the J2EE elements (e.g. `<ejb-ref>`) and JSP related elements (e.g. `<jsp-config>`) are no longer defined in the servlet specification itself but by other specifications from J2EE and the JCP.

#### 2.1.5. J2EE Schema's

The design of the deployment descriptor schema's for servlets represents more than just technical changes. The decision to make the schema's dependant on other J2EE schema's represents a significant and not completely welcome change in the specification process.

Prior to 2.4, the web application deployment descriptor was specified entirely by servlet specification and was the responsibility of the relevant JSR. With the 2.4 schema definitions, significant portions of the definitions are imported from the schema's produced by other parts of the J2EE specification and under the guidance of other JSRs.

The practical benefit of this change is that a single consistent definition of deployment descriptor elements may be used for JSPs, EJB, Web Services etc. Furthermore, these elements will be defined by the JSRs responsible for the technologies to which they apply to (e.g. the JSP descriptor schema is produced by JSR152 (JSPs) not JSR-154 (Servlets)). But some major concerns about this approach have been expressed regarding how this change effects the Servlet specification and the whole JCP process.

Importing J2EE schema's means that the servlet specification is no longer standalone and can only be considered as a subordinate specification of J2EE. Apart from the questionable object oriented design of a component having dependencies on it's container, this represents an administrative change that was forced on JSR-154 by the managers of the Java Community Process. The result being that the complete contents of the servlet deployment descriptor is now beyond the control of the servlet JSR and expert panel.

Servlets are much more than just a component of J2EE and many (if not most) deployments are on J2SE. There are even implementations for some J2ME profiles. By making J2EE the controlling specification for servlets, it has left J2SE implementation with a less definitive specification and with concerns that their interests may not be given equal consideration in future revisions of the specification.

This conflict of interests may have be the result of earlier mistakes in the servlet specification and is very much related to the discussion of extensible schema's later in this article. It could be argued that EJB elements should not have been added to the servlet specification, nor should JSPs have been given "favoured son" status over other content generation technologies. The `WEB-INF` directory could easily have been used as the home of separate deployment descriptors for servlet related technology without extending servlet deployment descriptor and specification itself.

The next revision of the specification may be a difficult one, as it has both significant technical and political issues that it must deal with. It is the authors hope that the result will be a servlet specification with clear J2ME, J2SE and J2EE profiles and a clear and reasonable dependency tree between them.

### 2.2. Filter Mapping Enhancements

When Filters were introduced in the 2.3 specification, they only applied to requests as they enter the container. Thus this powerful feature was not available to forwarded or included requests. Additionally it was undefined how they applied to error pages and welcome files. The 2.4 specification corrects this by allowing a filter to be mapped to specific types of requests and by clarifying the specification of the error page and welcome file mechanisms.

#### 2.2.1. The Problem

Consider the use of a filter to transform xml content as it is served from a web application. The transform could be to HTML, WML or whatever mark up is appropriate for the client. Using 2.3, the following descriptor elements could be defined:

      <filter>
        <filter-name>Transform</filter-name>
        <filter-class>com.acme.TransformFilter</filter-class>
      </filter>

      <filter-mapping>
          <filter-name>Transform</filter-name>
          <url-pattern>*.xml</url-pattern>
      </filter-mapping>

      <error-page>
        <error-code>404</error-code>
        <location>/error404.xml</location>
      </error-page>
      <error-page>
        <exception-type>java.lang.Exception</exception-type>
        <location>/except.xml</location>
      </error-page>

      <welcome-file-list>
        <welcome-file>index.xml</welcome-file>
      </welcome-file-list>

A request for an XML resource that exists (e.g. `/foo/bar.xml`) will match the Transform filter the content served will be handled by that filter as desired. Additionally, for many 2.3 containers this also worked for non-existent XML URIs (e.g. `/doesNotExist.xml`) which also matched the filter mapping and thus transformed the `error404.xml` error page that was served from the error-page mapping.

Unfortunately, this configuration fails for errors for non XML URIs. If a request for `/doesNotExist.JPG` is received, this does not match the *.xml filter mapping so the Transform filter should not be applied. This results in the error404.xml file being served without transformation.

A similar problem exists for welcome files, as a request to a URI like /foo/ may be handled by serving the `/foo/index.xml` resource, but it is unclear if the *.xml filter mapping should apply. For a container that uses a HTTP redirect to a welcome file, the filter would be applied as that results in a new request entering the container. For containers that serve welcome files directly, the filter should not be applied.

Finally, if a controller style servlet is introduced, where requests to URIs like `/controller/info` are dispatched as forwards or includes to the xml generating resource. The decision whether or not to apply the Transform filter cannot be made until the Controller servlet has selected a resource to which to dispatch.

Developers wanting to use 2.3 filters for such uses have often ended up applying the filter to all URIs and the filter itself decides if it applies to any particular request. This works well if your application mostly serves content that requires transformation, but is very inefficient in any mixed media environment. It also makes poor usage of the filter mapping mechanism.

#### 2.2.2. The Solution

To better support flexible filter usage, the 2.4 specification now allows one or more `<dispatcher>` sub elements to optionally be added to a `<filter-mapping>` element. Each `<dispatcher>` element specifies a value of `REQUEST`, `FORWARD`, `INCLUDE` or `ERROR` to indicate to which type of request the filter mapping applies.

For the above example, the filter mapping in the deployment descriptor can now be written as:

      <filter-mapping>
          <filter-name>Transform</filter-name>
          <url-pattern>*.xml</url-pattern>
          <dispatcher>REQUEST</dispatcher>
          <dispatcher>FORWARD</dispatcher>
          <dispatcher>INCLUDE</dispatcher>
          <dispatcher>ERROR</dispatcher>
      </filter-mapping>

This specifies that the Transform filter applies to any *.xml request that is received as a normal request, is the target of a RequestDispatcher.forward, is the target of a RequestDispatcher.include or is served as an error page.

#### 2.2.3. No Silver Bullet

The enhanced filter mapping mechanism will make it easier to use filters for clever applications, but it does not absolve the developer totally from the practise of careful design. The main problem now is to avoid situations where the same filter is applied more than once. For example, if an exception was thrown during the generation of a XML resource, then the Transform filter may get applied twice: once for the original request and again for the error page dispatch. Thus simply applying filters to all types of requests will not always be the solution.

### 2.3. RequestDispatcher Forward Attributes

The current specification of the RequestDispatcher.forward method does not well define the request methods that need to change when a request is forwarded:

> SRV.8.4 “The path elements of the request object exposed to the target servlet must reflect the path used to obtain the RequestDispatcher.”

This has resulted various implementation for the getRequestURI() method as it was unclear if getRequestURI was a path method or not. However, the issue is resolved elsewhere in the specification where it states:

> SRV.4.4 “It is important to note that, except for URL encoding differences between the request URI and the path parts, the following equation is always true: `requestURI = contextPath + servletPath + pathInfo`”

Thus the value returned by HttpServletRequest.getRequestURI() must change after a forward to reflect the resource used to obtain the RequestDispatcher. This resulted in the situation where the target of a forward was unable to easily determine the original URI of the request, as used by the client. This made it very difficult to generate URL links based on the request that the client had issued.

To resolve this, the 2.4 specification has added a set of forward attributs that are set the first time a request is forwarded:

> javax.servlet.forward.request_uri
> javax.servlet.forward.context_path
> javax.servlet.forward.servlet_path
> javax.servlet.forward.path_info
> javax.servlet.forward.query_string

These attributes capture the path values and query string as originally received by the container and are not hidden by any subsequent forwards or includes of the request. Thus the original request paths will always be available.

Unfortunately, the result is that if a servlet is to be portable and reusable, it now needs to double check getRequestURI to determine the URI that the client used and the URI (or context, servlet or info path) of the resource to be served:

    public void doGet(HttpServletRequest request, HttpServletResponse response)
    {
      String uriUsedByClient=(String)
        request.getAttribute("javax.servlet.forward.request_uri");
      if (uriUsedByClient==null)
        uriUsedByClient=request.getRequestURI();
      String uriToServe=(String)
        request.getAttribute("javax.servlet.include.request_uri");
      if (uriToServe==null)
        uroToServer=request.getRequestURI();
      // ...
    }

### 2.4. New Event Listeners

The servlet event model has been rounded out with new listeners defined for requests and request attributes. However, there is probably very little need for any of these new listeners other than completeness and do not provide anything that could not simply be implemented with a filter.

#### 2.4.1. Request Event & Listener

The `javax.servlet.ServletRequestListener` and `javax.servlet.ServletRequestEvent` classes have been introduced for handling events when a request enters and leaves the scope of a web application, which is defined in the javadoc as:

> “A request is defined as coming into scope when it is about to enter the first servlet or filter in each web application, as going out of scope when it exits the last servlet or the first filter in the chain.”

Unfortunately the events themselves were named without considering that a web application can dispatch a request to another, resulting in a request being in scope of more than one web application. Thus instead of names such as `enterContext`/`exitContext,` the events are defined as:

    public interface ServletRequestListener extends EventListener {
        /** The request is about to come into scope of the web application. */
        public void requestInitialized ( ServletRequestEvent sre );

        /** The request is about to go out of scope of the web application. */
        public void requestDestroyed ( ServletRequestEvent sre );
    }

Note that the functionality provided by this listener is also provided by the extended filter mechanism. A trivial filter installed at the default / path equally allows for handling of requests as they enter and exist a web application.

#### 2.4.2. Request Attribute Event & Listener

The `javax.servlet.ServletRequestAttributeListener` and `javax.servlet.ServletRequestAttributeEvent` classes have been introduced for handling events when a request attribute is modified while the request is within the scope of a web application:

    public interface ServletRequestAttributeListener extends EventListener {
        /** Notification that a new attribute was added to the
         ** servlet request. Called after the attribute is added.
         */
        public void attributeAdded(ServletRequestAttributeEvent srae);

        /** Notification that an existing attribute has been removed from the
         ** servlet request. Called after the attribute is removed.
         */
        public void attributeRemoved(ServletRequestAttributeEvent srae);

        /** Notification that an attribute was replaced on the
         ** servlet request. Called after the attribute is replaced.
         */
        public void attributeReplaced(ServletRequestAttributeEvent srae);
    }

As for the ServletRequestListener, a request can be in scope of multiple web applications and events are generated for all in-scope web applications. Thus if ServletA in context /webappA dispatches a request to ServletB in context /webappB, then ServletRequestListeners in both web applications will receive events if ServletB modifies an attribute.

As the value of the attribute is passed in the event object, use of this listener may lead to some interesting class loader issues as web applications are passed an object of a class instance that is only available to another web application. However, this is unlikely to ever be an issue, as there is very little need for this event and the author will buy a large beer to anybody that can suggest a real world use-case. Note that this listeners functionality could also have been provided with the extended filter mechanism and request wrapping.

### 2.5. New APIs

A few minor improvements have been made to the servlet APIs of existing interfaces.

#### 2.5.1. Client Connection Methods

The `javax.servlet.ServletRequest` interface in 2.3 had a partial set of methods that describe the logical and actual connection of a request. In 2.4 four additional methods have been added to make this part of the API more complete:

    public interface ServletRequest
    {
        // Methods about the logical name of the server, as seen by the client.
        public String getServerName();
        public int getServerPort();

        // Methods about the local end of the actual TCP/IP connection
        public String getLocalName();  // New in 2.4
        public String getLocalAddr();  // New in 2.4
        public int getLocalPort();     // New in 2.4

        // Methods about the remote end of the actual TCP/IP connection
        public String getRemoteAddr();
        public String getRemoteHost();
        public int getRemotePort();    // NEW in 2.4

        ...

The ServerName is the name used by the client to create the connection with the server. The LocalName is the DNS name of the interface that the connection was received on, which may have been proxied or otherwise mapped and thus can often be different from the ServerName. The same difference applies to the ServerPort and LocalPort.

These APIs will not be of much use for most web applications, but are vital for those that need to be aware of their network environment.

#### 2.5.2. Internationalization

Prior to 2.4, the content type and character encoding of a response could be set with a call to setContentType like:

    response.setContentType("text/html; charset=Shift_JIS");

The same result could be achieved with better separation of concerns using setLocale:

    response.setContentType("test/html");
    response.setLocale(Locale.JAPAN);

Unfortunately the mapping between Locale and character encoding is not well defined and it was up to each container to provide a reasonable mapping. There was no way to tell if a Locale was known to the container, nor to retrieve the resulting combined content-type to check if a character encoding had been set.

The 2.4 specification addresses this by firstly allowing the Locale to encoding mapping to be specified in the descriptor using the new `<locale-encoding-mapping-list>` element. Locales may be mapped with either just a language code (e.g. zh), or a language and country code (e.g.. zh_TW):

    <locale-encoding-mapping-list>
      <locale-encoding-mapping>
        <locale>ja</locale><encoding>Shift_JIS</encoding>
      </locale-encoding-mapping>
      <locale-encoding-mapping>
        <locale>zh</locale><encoding>GB2312</encoding>
      </locale-encoding-mapping>
      <locale-encoding-mapping>
        <locale>zh_TW</locale><encoding>Big5</encoding>
      </locale-encoding-mapping>
    </locale-encoding-mapping-list>

The encoding for a particular locale may still be set by a container default, but the webapp now has the option to override the container or ensure that the locale is supported. There is also a new `getContentType` method on `ServletResponse` that allows the resulting content-type string to be observed:

    response.setContentType("test/html");
    response.setLocale(Locale.JAPAN);
    assert response.getContentType().indexOf("charset=Shift_JIS")>0;

Finally for those that want direct control of the character encoding and do not wish to go via the Locale mechanism, there is a new response `setCharacterEncoding` method:

    response.setContentType("test/html");
    response.setCharacterEncoding("Shift_JIS");

## 3. Missing Features

Some of the features listed in the aims of JSR-154 and that have been much requested by users, did not make it into the 2.4 specification. Mostly this was because they proved to be beyond the scope of a point release

### 3.1. Programmatic Authentication

The JSR-154 aims and early drafts of the 2.4 specification contained an API for programmatic authentication of requests, which was dropped during the review process. While all parties agreed that this is a much needed feature, it was not possible to agree a one-size-fits-all solution without some significant changes to the existing security mechanism. Some of the key questions that need to be resolved include:

- What parts of authentication and authorization are the responsibility of the servlet specification? Should it be providing a complete solution or just implementing the HTTP specific protocols and linking them with an externally specified mechanism?

- Should the servlet security mechanisms be integrated with the java.security.* architecture? If so, how should role based authentication be mapped to the Permission model?

- What is the scope of Authentication? Is it tied to a URL space, a HTTP Cookie, a HttpSession or a distributed HttpSession? The existing protocols all have different scopes and it is only FORM authentication that tied authentication to a HttpSession.

- What requests are authenticated? Only those that match an authentication constraint, only those that have security methods called or all requests?

- Under what circumstances is authentication passed onto a J2EE call?

- What about JAAS, SAML, single sign on, etc.?

These are all tough questions and security is something that you do not wish to get wrong. The result was that programmatic login was thrown into the too hard basket for 2.4. For now programmatic login can be achieved on many containers with a dispatch a request to j_security_check.

### 3.2. Extensible Deployment Descriptors

The aims of JSR-154 and the early drafts of the 2.4 specification contained a proposal to allow third party extension of the deployment descriptor. The intent was to avoid the need for multiple descriptors when adding container specific deployment.

It is a attribute of J2EE that the standards based deployment descriptors are often insufficient to actually deploy real world applications and that vendor specific descriptors are commonly required. Thus as well as `web.xml` descriptor, you will commonly see in `WEB-INF` descriptors such as `jboss-web.xml`, `weblogic.xml`, `jetty-web.xml`, etc.. These extra descriptors provide vendor/container specific configuration for the web application.

The extensible deployment mechanism was intended to allow the vendor specific elements to be added to the standard descriptor without validation issues. Thus the configuration for elements such as EJBs would not be spread over multiple files.

While initially attractive, this proposal was eventually removed for a combination of technical, design and procedural difficulties:

- The aim of the java specification process is to encourage multiple implementations and to allow portability of compliant applications between those implementations. A single extensible deployment descriptor is counter to the aims of portability. While a single descriptor could include extensions from multiple vendors, it is doubtful if tools support for this could be provided simply or rapidly. Thus, apart from the visual confusion of multiple configurations within one descriptor, this feature invited the possibility of one vendor's tools removing or damaging the configurations of the others.

- It can be argued that the non-servlet deployment elements should never have been included in the standard deployment descriptor. It would have been possible for EJB, Resources and JSPs to be configured from their own descriptors within the `WEB-INF` directory. Such an approach avoids unnecessary technical coupling and would also lessen the jurisdiction disputes like those caused by the inclusion of J2EE schema's in the descriptor.

- For this feature to be acceptable, it had to be adopted for all J2EE component specifications. Indeed, other J2EE technologies such as CMP and EJBs suffer much greater issues of multiple descriptors and duplicate structure. However, there is no formal JCP process for getting such coordinated agreement from all JSRs involved and to impose a solution from management would diminish the value of community consultation.

### 3.3. NIO Support

A much requested feature for the servlets API is support for the `java.nio` library. This is desirable to allow usage of the more efficient direct buffering mechanism as well as potentially using non blocking IO to free threads from the many idle persistent connections that HTTP/1.1 servers maintain.

Unfortunately there is not simple way that NIO can be retrofitted to the servlet API. There is enough chaos and confusion caused by readers/writers vs streams API duality, so adding NIO Channels into that mix is not going to be easy without a major overhaul of the API. It is also not workable to slip non-blocking semantics under the existing API. Most servlets are written expecting blocking IO and will either turn into busy loops or fail if zero bytes because a legal return for a read or a write.

An argument can also be made that moving the Servlets API towards NIO is going in the wrong direction as it will encourage the servlet writer to become more involved with the buffering and scheduling performed by the container. It may be better to increase the level of abstraction provided by the servlet API, so that they can be used more as application components and less as protocol handlers. A higher level of abstraction could allow a container more scope to use efficient mechanisms such as NIO to deal with the low level protocols, which may not be HTTP. For example, the container is already responsible for reading and parsing `application/x-www-form-urlencoded` content, which it could currently do with NIO behind the scenes. It may be better to extend this approach to allow the container have more responsibility for content reading/writing and let the Servlets focus more on content handling/generation.

### 3.4. SingleThreadModel Deprecated

While not missing, the SingleThreadModel interface has been deprecated and will go missing in a future release. It was generally agreed that SingleThreadModel was a false friend that allowed a servlet developer to believe that they did not need to be concerned about synchronization and threading. Unfortunately this mechanism only makes this true for instance variables of the servlet itself and does not offer any protection for access to sessions, datasources or any other shared resources. As passing parameters between methods via instance variables is hardly best OO practise, there is no real benefit from this interface. However it does result in some significant additional implementation complications for containers.

## 4. Updates & Clarifications

### 4.1. Java & HTTP Versions

The versions of java required by the servlet specification have been updated to J2SE 1.3. Containers may still be developed to work with older versions of java, but a servlet can now be written to use 1.3 APIs and still be considered portable. Considering the length of time that 1.3 has been released and that 1.4 is now the norm, this is a more than reasonable change.

The specification also now requires that a servlet container supports HTTP/1.0 and HTTP/1.1 (as per RFC2616). This is actually a requirement of the Web Server associated with the servlet container and so could not strictly be policed from a compliance point of view. However the expectation is that filters and servlets may be written to read and generate the HTTP headers supported by these protocols and that whatever actual transport is used by the Servlet Container will respect the semantics of these. For example it is expected that HTTP/1.1 cache control headers will be transported to the client and any proxies even if the AJP13 protocol and apache mod_jk are the actual transport protocol used by the Servlet Container

### 4.2. Path Mappings & Path Parameters

It is a little known and little used feature of URIs that each path element may be given parameters such as:

    /foo;a=1;b=2/bar;c=3

The only common use made of such parameters is by the Servlet specification itself for tracking HttpSession with URL rewriting, e.g.:

    /foo/bar;jsessionid=a5d5f25s727a01

The servlet specification currently has no other support for path parameters and has always excluded them from the request parameter API:

> SRV.4.1 “Path parameters that are part of a GET request (as defined by HTTP 1.1) are not exposed by these APIs. They must be parsed from the String values returned by the getRequestURI method or the getPathInfo method.”

The 2.4 servlet goes further in clarifying that path parameter are not considered when mapping URIs to servlets:

> SRV.11.1 “The path used for mapping to a servlet is the request URL from the request object minus the context path and the path parameters.”

In practise, most containers will continue to only correctly handle trailing path parameters until later versions of the specification give a more detailed treatment of path parameters.

### 4.3. Error pages

The 2.4 specification now better defines the mechanism that a container may use to implement the error pages. Previously the dispatch to use was undefined, so that it was not clear how filters should be applied or what values the path methods should return.

> SRV.9.9.1 “The request path and attributes are set as if a RequestDispatcher.forward to the error resource had been performed.”

The enhancement to filters already discussed have also made it clear when filters are applied to error pages.

The triggering of the error page mechanism has been explicitly restricted to use of the sendError method and for exceptions thrown back to the container from an original request. Use of the setStatus method and exceptions thrown by RequestDisptachers or calls to doFilter will not invoke the error page mechanism. Finally, it has been made explicit exactly when setStatus and sendError will be used:

> SRV.9.9.2 “The default servlet and container will use the sendError method to send 4xx and 5xx status responses, so that the error mechanism may be invoked. The default servlet and container will use the setStatus method for 2xx and 3xx responses and will not invoke the error page mechanism.”

Thus it is now clear for example, that setting an error page mapping for the 200 status will not result in the error page being invoked for normal responses.

### 4.4. Welcome Files

The dispatch mechanism for welcome files has also been somewhat addressed:

> SRV.9.10 “The container may send the request to the welcome resource with a forward, a redirect, or a container specific mechanism that is indistinguishable from a direct request.”

In Australia this would be called a Claytons clarification, after a famous advertising campaign for a non alcoholic drink that used the tag line "the drink I have when I'm not having a drink". The 2.4 specification only clarifies that there is no clarity as to what dispatch mechanism should be used for welcome files. Thus it is unclear what filters will apply and what the request path methods should return and as a result many containers offer an option of several different welcome dispatch mechanisms .

The use of HTTP redirection for welcome files is common, as in 2.3 it was the only way to allow Filters to be applied to welcome files with the same semantics as normal requests. The down side if this is that another round trip to the client is required and that the full welcome file URL is exposed to the user.

The improved filter mechanism makes the HTTP forward mechanism a viable and well defined alternative for welcome file dispatch. Hopefully it will be this that future specifications recommend.

Other "clarifications" to the welcome file mechanism are discussed later in New and Unresolved Problems.

### 4.5. Life Cycles & Events

The life cycle of a web application and the events associated with it have been clarified with several changes throughout the specification and javadoc, plus the addition of section SRV.9.12 on web application deployment.

It has now been made explicit that the `ServletContextListener.contextInitialized` method is called before any filters or servlets are initialized. It thus joins `HttpSessionListener.sessionDestroyed` as an event whose name has the incorrect tense and thus can be misleading. Hopefully future versions of the specification will deprecate these methods and rename them `contentInitialize` and `destroySession.`

## 5. New and Unresolved Problems

Despite the numerous clarifications (and in some cases because of them), there are still some significant problems and ambiguities in the 2.4 specification.

### 5.1. Welcome Files

In addition to the non-clarification of welcome file dispatch mechanisms, the 2.4 servlet specification contains a new feature that allows servlets to be welcome "files" even if no real file exists:

> SRV.9.10 “The Web server must append each welcome file in the order specified in the deployment descriptor to the partial request and check whether a static resource or servlet in the WAR is mapped to that request URI.”

The intent of this change is to allow the welcome file mechanism to invoke template mechanisms such as TEA and Velocity that often store their templates outside of the normal web application resource hierarchy. Thus if a request is received for `/foo/`, the welcome mechanism can now dispatch this `to /foo/index.tea` even if the file `index.tea` does not exist in the `foo` directory.

Other than stretching the meaning of the word "file", this change initially looks appealing. Unfortunately it represents a significant change in the semantics of the welcome file mechanism and could break many web applications if implemented as specified.

Consider the following section of a deployment descriptor:

    <welcome-file-list>
        <welcome-file>index.jsp</welcome-file>
        <welcome-file>index.html</welcome-file>
    </welcome-file-list>

which is applied to a web application with the following file hierarchy:

    webapp
    |-- WEB-INF
    |   `-- web.xml
    |-- dirA
    |   `-- index.jsp
    `-- dirB
        `-- index.html

A request to `/dirA/` will be handled as expected, as a file `index.jsp` exists and the request will be dispatched or redirected to `/dirA/index.jsp`. On many containers, this will match a JSPServlet mapped to *.jsp and execute normally.

Unfortunately, a request to `/dirB/` will not serve `/dirB/index.html` as would be the case with the current 2.3 specification. Instead the first welcome file mapping of /dirB/index.jsp will be tried and will result in a match of the JSPServlet mapped to *.jsp. Thus a 2.4 compliant container will dispatch or redirect this request to the non-existent dirB/index.jsp resource. At best this will result in a 404 error, or possibly a 500 error complete with a nasty exception.

The situation is further muddied by the specification, as it's own examples have not been updated to reflect the inclusion of servlet mappings to the welcome file mechanism. It shows requests being passed to the default servlet, when a mapping to a JSPServlet is the more likely result.

The great pity of this flawed solution is that the problem really just reflects the deficiancy of the servlet URL paterns supported by the specification. If the URL patterns were able to support even simple wildcard pattern matching, it would not be difficult to map a servlet to every URL ending in /. Even without such a change to the specification, the desired functionality of a servlet welcome resource can easily be provided with the following simple Filter:

    public  class WelcomeFilter implements Filter
    {
      private String welcome;

      public void init(FilterConfig filterConfig)
      {
        welcome=filterConfig.getInitParameter("welcome");
      }

      public void doFilter(ServletRequest request,ServletResponse response,FilterChain chain)
        throws IOException, ServletException
      {
        String path=((HttpServletRequest)request).getServletPath();
        if (welcome!=null && path.endsWith("/"))
          request.getRequestDispatcher(path+welcome).forward(request,response);
        else
          chain.doFilter(request, response);
      }

      public void destroy() {}
    }

### 5.2. RequestDispatchers vs Wrappers

The request wrapping features introduced in 2.3 are well defined within the section of the specification that discusses filters, where it is made clear that specific types of wrapped requests and responses may be passed along a filter chain and over RequestDispatchers:

> SRV.6.2.2 “... When a filter invokes the doFilter method on the containers filter chain implementation, the container must ensure that the request and response object that it passes to the next entity in the filter chain, or to the target web resource if the filter was the last in the chain, is the same object that was passed into the doFilter method by the calling filter. ”

> “The same requirement of wrapper object identity applies to the calls from a servlet or a filter to RequestDispatcher.forward or RequestDispatcher.include, when the caller wraps the request or response objects. In this case, the request and response objects seen by the called servlet must be the same wrapper objects that were passed in by the calling servlet or filter..”

While this direct passing of wrapped objects is straight forward for filters, unfortunately this requirement proves ambiguous and undesirable when applied to RequestDispatchers. Fundamentally the container cannot guarantee that request and responses will be passed unwrapped through a RequestDispatcher as the enhanced Filter mechanism now allows filters and thus wrappers to applied to dispatching.

Furthermore, the specification of the RequestDispatch mechanism is very explicit about the treatment of path methods in a passed request:

> SRV.8.4 “The path elements of the request object exposed to the target servlet must reflect the path used to obtain the RequestDispatcher. The only exception to this is if the RequestDispatcher was obtained via the getNamedDispatcher method.”

Thus, as a user supplied wrapper is free to change the values return from the path methods, SRV.8.4 indicates that the container must override any such user supplied wrappers and force the values to be those specified by the RequestDispatcher.

Containers commonly implement the RequestDispatcher requirements by applying their own request wrapper during dispatch. Containers that respect SRV.6.2.2 “wrap under” user supplied wrappers, while those that respect SRV.8.4 “wrap over” user supplied wrappers. These two approaches can both be justified by refering to the specification, but result if very different behaviours.

#### 5.2.1. “Wrap Over” vs “Wrap Under”

Consider the following HttpServletRequestWrapper. It has been written to assist converting a webapplication from a case insensitive environment to a case sensitive one, so it converts all path methods to return lowercase values so that mixed case bookmarks and static content continue to work. The filter has been written to cache the results of toLowercase to avoid multiple conversions:

    public LCRequestWrapper extends HttpServletRequestWrapper
    {
        LCRequestWrapper(HttpServletRequest r) {super(r);}

        public String getServletPath()
        {
            if (servletPath==null)
                servletPath=super.getServletPath().toLowercase();
            return servletPath;
        }

        public String getPathInfo()
        {
            if (pathInfo==null)
            {
                pathInfo=super.pathInfo();
                if (pathInfo!=null)
                    pathInfo=pathInfo.toLowercase();
            }
            return servletPath;
        }

        private String servletPath;
        private String pathInfo;
    }

This wrapper looks straight forward, but can have very unexpected behaviour if used in a “wrap under” container. When a dispatch is done, the container will insert a request wrapper underneath this user supplied wrapper, so that the target resource will call the user wrapper directly. The user wrapper may call super methods which may return different values for the duration of the dispatch.

If the dispatching resource calls a path method before the request is forwarded to the target resource, then the target resource will see the cached value calculated before the dispatch, which will not be the correct value for the target.

If a path method is first called by the target resource, then the cached value will be set from dispatched values of the path. This value will be incorrectly returned to any call made by the dispatching servlet after the dispatch.

If “wrap over” semantics are applied to this example, then the behaviour is well defined as the wrapped path methos are not visible during dispatch as they are hidden by the SRV.8.4 compliant wrapper.

While this example is somewhat contrived and can simply be fixed by removing the value cache, it does indicate that “wrap under” semantics can be more complex than first appearances. To avoid such indeterminate behaviour, a wrapper must avoid having any fields that are derived from methods that may be changed during a request dispatch.

Unfortunately some values, such as streams and writers must be cached by a wrapper as they cannot be reproduced on each call. The following filter follows a similar pattern to most filters that wrap the responses stream or writer with a specialized version for compression or encryption etc.

    public MyResponseWrapper extends HttpServletResponseWrapper
    {
        MyResposneWrapper(HttpServletResponse r) {super(r);}

        public ServletOutputStream getOutputStream() throws IOException
        {
            if (_writer!=null)
                throw new IllegalStateException("getWriter called");
            if (_out==null)
                _out=new MyOutputStream(super.getOutputStream());
            return _out;
        }

        public PrintWriter getWriter() throws IOException
        {
            if (_out!=null)
                throw new IllegalStateException("getOutputStream called");
            if (_writer==null)
                _writer=new MyWriter(super.getWriter());
            return _writer;
        }

        ServletOutputStream _out;
        PrintWriter _writer;
    }

This wrapper contains the state of which type of output stream has been requested. Consider if this wrapper is passed to a servlet that calls getOutputStream before including of a resource that is generated using getWriter.

The call to getOutputStream will initialize the specialized MyOuputStream. If wrap under semantics are used during the include, then this wrapper will also be passed to the second resource. When getWriter is called an IllegalStateException will result. This implies that the user of RequestDispatch.include must be aware of the implementation details of the requested resource and can only use those with the same flavour of implementation.

With “wrap over” semantics, the container is given the opportunity to hide the state of the including resource. The included resource can be passed a response wrapper provided by the container that itself wraps the MyResponseWrapper instance. When getWriter is called, it initially calls the wrapped getWriter which will thow an IllegalStateException. The container supplied wrapper can catch this exception and then call the wrapped getOutputStream and wrap the specialized stream returned in an OutputStreamWriter.

#### 5.2.2. Resolution of RequestDispatchers vs Wrappers

The next version of the specification must address this conflict if portable filters and servlets are to be written using request and response wrapping. To resolve which part of the specification should prevail we need to examine the motivation for the requirement for direct passing of objects:

> SRV.6.2.2 “... In this model, the developer not only has the ability to override existing methods on the request and response objects, but to provide new API suited to a particular filtering task to a filter or target web resource down the chain.”

This close coupling of components is not best practise in an environment where the development of utility filters and wrappers is encouraged to intercept arbitrary requests. For example if a web application was written to assume the direct passing of a wrapper type from a dispatching servlet to a resource producing HTML, then the application of the common compression filter to all HTML resources would break this assumption and cause a class cast exception.

There exist several other mechanism to allow required coupling between web application components that are more flexible and not vulnerable to the extended filter mechanism. Static methods, request attributes, session attributes, ThreadLocals and unwinding of wrappers can all be used to achieve the aim of providing new API for a web application.

It is the author's hope that the next revision of the servlet specification will relax the object passing requirements of SRV.6.2.2 and discourage the associated closely couple programming model. The “wrap over” semantics should be specifically required for RequestDispatching.

### 5.3. Distributable Web Applications

The specification of distributed HttpSessions remains very poor in the 2.4 specification with no clarifications made to any of the significant omissions in the specification:

- Should the HttpSession.setAttribute method be considered pass-by-value or pass-by-reference? i.e. at what time should the session values be distributed. Non distributed sessions allow the value to be changed at any time by any thread or mechanism and the next call to getAttribute will return the updated value. However, unlike EJB's there is no uniform access path to session values nor transactional model, so such changes cannot simply be distributed. Many distribution implementation avoid this issue by changing the semantics for distributed sessions to be pass-by-value and only distributing session values when setAttribute is called.

- How are bind and unbind events handled in a distributed session? Is bind called in every JVM that value is distributed to? Or is it only called in the JVM where the value was originally added and then distributed to other JVMs without further events.

- How is non-gentle migration handled? If a node fails and no session passivation event was handled, should a session activation event be issued on the new node?

Until these issues are clarified, distribution wil remain a very container specific mechanism.

## 6. Conclusion

The 2.4 servlet specification is definitely a much better specification than it's forerunners, with the main achievement being the extension of Filters to be an integral part of all request dispatching within a container. There have also been numerous small but valuable clarifications that have not been covered by this article but are listed in SRV.S.16.

Unfortunately significant problems remain in the specification, some new ones have been added and some much requested features have been deferred. Other changes are at best of dubious added value and the benefits or otherwise of the structural and procedural changes represented by the J2EE based schema's have yet to play out.

Hopefully, most of these issues will be below the radar of your average servlet user as IDE's, tools and web frameworks are already doing much of the work to deal directly with the servlet API. Developers should be encorouged to start evaluations of containers supporting this revision.

## A. References

Servlet Home page:

  http://java.sun.com/products/servlet

JSR-154 Servlet 2.4 Specification working group:

  http://www.jcp.org/en/jsr/detail?id=154

Java Community Process:

  http://www.jcp.org

Jetty servlet container:

  http://jetty.mortbay.org

Jakarta Tomcat servlet container:

  http://jakarta.apache.org/tomcat