Friday Aug 15, 2008
Asynchronous Restful Webapplication
This blog annotates the Jetty 7 example web application that uses Jetty asynchronous HTTP client and the proposed suspendable servlets 3.0 API, to call an eBay restful web service. The technique combines the Jetty asynchronous HTTP client with the Jetty servers ability to suspend servlet processing, so that threads are not held while waiting for rest responses. Thus threads can handle many more requests and web applications using this technique should obtain at least ten fold increases in performance.
The screen shot above shows four iframes calling either a synchronous or the asynchronous demonstration servlet, with the following results:
The code for the example asynchronous servlet is available here and works as follows:
The screen shot above shows four iframes calling either a synchronous or the asynchronous demonstration servlet, with the following results:
- Synchronous Call, Single Keyword
- A request to lookup ebay auctions with the keyword "chair" is handled by the synchronous implementation. The call takes 660ms and the servlet thread is blocked for the entire time. A server with a 100 threads in a pool would be able to handle 151 requests per second.
- Asynchronous Call, Single Keyword
- A request to lookup ebay auctions with the keyword "chair" is handled by the asynchronous implementation. The call takes 669ms, but the servlet request is suspended so the request thread is held for only 2ms. A server with a 100 threads in a pool would be able to handle 5000 requests per second (if not constrained by other limitations)
- Synchronous Call, Three Keywords
- A request to lookup ebay auctions with keywords "mouse", "beer" and "gnomes" is handled by the synchronous implementation. Three calls are made to ebay in series, each taking approx 900ms, with a total time of 2706ms and the servlet thread is blocked for the entire time. A server with a 100 threads in a pool would be able to handle only 40 requests per second!
- Asynchronous Call, Three Keywords
- A request to lookup ebay auctions with keywords "mouse", "beer" and "gnomes" is handled by the asynchronous implementation. The three calls can be made to ebay in parallel, each taking approx 900ms, with a total time of 906ms and the servlet request is suspended, so the request thread is held for only 2ms. A server with a 100 threads in a pool would be able to handle 5000 requests per second (if not constrained by other limitations).
The code for the example asynchronous servlet is available here and works as follows:
- The servlet is passed the request, which is detected as the first dispatch, so the request is suspended and a list to accumulate results is added as a request attribute:
if (request.isInitial() || request.getAttribute(CLIENT_ATTR)==null)
The request is suspended before starting the searches in order to avoid races if the searches somehow complete before the request is suspended.
{
String[] keywords=request.getParameter(ITEMS_PARAM).split(",");
final List<Map<String, String>> results =
Collections.synchronizedList(new ArrayList<Map<String, String>>());
final AtomicInteger count=new AtomicInteger(keywords.length);
request.suspend();
request.setAttribute(CLIENT_ATTR, results);
- After suspending, the servlet creates and sends an asynchronous HTTP exchange for each keyword:
for (final String item:keywords)
The API for the Jetty Http client exchanges was inspired by the callback style of javascript XHR.
{
ContentExchange exchange = new ContentExchange()
{
protected void onResponseComplete() throws IOException
{
// see step 4 below
}
};
exchange.setMethod("GET");
exchange.setURL("http://open.api.ebay.com/shopping?MaxEntries=5&appid=" +
_appid +
"&version=573&siteid=0&callname=FindItems&responseencoding=JSON&QueryKeywords=" +
URLEncoder.encode(item,"UTF-8"));
_client.send(exchange);
}
- Once all asynchronous HTTP exchanges are sent, the servlet saves some timing information for the demo and then returns. Because the request is suspended, the response is not flushed to the browser, but the thread is returned to the thread pool so it can service other requests:
request.setAttribute(START_ATTR, start);
request.setAttribute(DURATION_ATTR, new Long(System.currentTimeMillis() - start));
return;
} - All the rest requests are handled in parallel by the eBay servers and when each of them completes, the call back on the exchange object is called. The code (omitted above, shown below)extracts auction information from the JSON response and adds it to the results list. The count of expected responses is then decremented and when it reaches 0, the suspended request is resumed:
protected void onResponseComplete() throws IOException
{
Map query = (Map) JSON.parse(this.getResponseContent());
Object[] auctions = (Object[]) query.get("Item");
if (auctions != null)
{
for (Object o : auctions)
results.add((Map) o);
}
if (count.decrementAndGet()<=0)
request.resume();
} - After being resumed, the request is re-dispatched to the servlet. This time the request is not initial and has results, so the results are retrieved from the request attribute and normal servlet style code is used to generate a response:
List<Map<String, String>> results = (List<Map<String, String>>) request.getAttribute(CLIENT_ATTR);
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.println("<html><head><style type='text/css'>img:hover {height:75px}</style></head><body><small>");
for (Map<String, String> m : results)
{
out.print("<a href=\""+m.get("ViewItemURLForNaturalSearch")+"\">");
...
Posted at 05:59AM Aug 15, 2008 by gregw in General | Comments[4]
Posted by Fredrik Tyboni on October 01, 2008 at 03:16 AM EST #
Posted by Valery Silaev on October 30, 2008 at 04:01 AM EST #
Posted by Troy Kelley on August 08, 2009 at 08:35 AM EST #
Posted by Greg Wilkins on August 08, 2009 at 02:46 PM EST #