An Asynchronous Servlet API?

Now that the 2.5 servlet specification is final, we must start thinking about the next revision and what is needed. I believe that the most important change needed is that the Servlet API must be evolved to support an asynchronous model.

I see 5 main use-cases for asynchronous servlets:

1. Non-blocking input - The ability to receive data from a client without blocking if the data is slow arriving. This is actually not a significant driver for an asynchronous API, as most request arrive in a single packet, or handling can be delayed until the arrival of the first content packet. More over, I would like to see the servlet API evolve so that applications do not have to do any IO.
2. Non-blocking output - The ability to send data to a client without blocking if the client or network is slow. While the need for asynchronous output is much greater than asynchronous input, I also believe this is not a significant driver. Large buffers can allow the container to flush most responses asynchronously and for larger responses it would still be better to avoid the application code handling IO.
3. Delay request handling - The comet style of Ajax web application can require that a request handling is delayed until either a timeout or an event has occured. Delaying request handling is also useful if a remote/slow resource must be obtained before servicing the request or if access to a specific resource needs to be throttled to prevent too many simultaneous accesses. Currently the only compliant option to support this is to wait within the servlet, consuming a thread and other resources.
4. Delay response close - The comet style of Ajax web application can require that a response is held open to allow additional data to be sent when asynchronous events occur. Currently the only compliant option to support this is to wait within the servlet, consuming a thread and other resources.
5. 100 Continue Handling - A client may request a handshake from the server before sending a request body. If this is sent automatically by the container, it prevents this mechanism being meaningfuly used. If the application is able to decide if a 100-Continue is to be sent, then an asynchronous API would prevent a thread being consumed during the round trip to the client.

All these use cases can be summarized as "sometimes you just have to wait for something" with the perspective that waiting within the Servlet.service method is an expensive place to park a request while doing this waiting as:

• A Thread must be allocated.
• If IO has begun then buffers must be allocated.
• If Readers/Writers are obtained, then character converters are allocated
• The session cannot be passivated
• Anything else allocated by the filter chain is held

These are all resources that are frequently pooled or passivated when a request is idle. Because comet style Ajax applications require a waiting request for every user, this invalidates the use of pools for these resources and requires maximal resource usage. To avoid this resource crisis, the servlet spec requires some low cost short term parking for requests.

The Current Solutions

• Jetty has Continuation which are targetted at comet applications
• BEA has a future response mechanism also targetted Comet applications
• Glassfish has an extensible NIO layer for async IO below the servlet model
• The tomcat developers have just started developing Comet support in tomcat 6

class DefaultCoordinator implements ServletCoordinator
{
   void doRequest(ServletRequest request)
   {
       request.continue();
       request.service();
   }

   void doResponse(Response response)
   {
       response.complete();
   }
}