Pitfalls to watch out for when creating Edge Side Includes

Tags:

When creating Edge Side Includes (ESI) there are - as pointed out earlier - pitfalls one can fall into. Falling into such a pitfall will definitely reduce the effects of applying ESI so it's important to try to avoid doing so. There are no strict rules for what defines a pitfall, but in this post I will try to give some example for what to look out for and how we can try to work around them.

To get maximum effects of ESI we want as much reuse of fragments as possible and it's important to remember that a ESI enabled proxy does cache the mark-up and content of a fragment. ESI does – or shall - not be applied to CSS and JavaScript files. We must also remember that each fragment going into the ESI cache live on an unique URL. In other words; each unique URL the ESI enabled proxy talks to shall hold an unique set of mark-up and content which is the fragment.

Pitfall one - Unequal query parameter order

ESI are based on including fragments by http and URLs can be a great source for a pitfall. In a large and dynamic system there will definitely be the need to let the application server serve different content to the ESI enabled proxy depending on different query parameter values in the request from the ESI enabled proxy. This is fine, but query parameters has the downside that their order in the URL are unimportant for the application server and for a ESI enabled proxy the order is important.

Let's say we have a fragment in our application which takes two input parameters; type and id. The ESI enabled proxy can then request the fragment at the application server by accessing:

http://www.yoursite.com/component?type=element&id=123

If the ESI enabled proxy call the same URL from different locations on our site we do great use of ESI since one cached fragment are reused on several spots on our site. We have optimal effect.

But, we can also call the same fragment from the application server with a different order of the query parameters and still get the same result returned by the application server:

http://www.yoursite.com/component?id=123&type=element

This is correct behavior by the application server but this is not optimal for the ESI enabled proxy server. ESI does keep track of the ESI fragments by their URLs and not the result returned from the application server. For the ESI enabled proxy this is two different URLs and it will hold two cached fragments in its cache even if the fragments are equal. This is not optimal use of ESI.

It's important to keep query parameters in a strict order to prevent such problems. The best way to avoid this problem is to use nice URLs for the fragments since nice URLs has a strict order:

http://www.yoursite.com/component/element/123/

Pitfall two – Not getting HTTP headers right

A ESI enabled proxy server works just as an ordinary proxy server except that it caches fragments instead of complete pages. The way the ESI enabled proxy talk to the back-end server are the same as old fashion proxy server which relies on http headers for when to get updated content. One pitfall is not to get the http headers on the application server right. One should consider that a ESI enabled proxy server will have more to keep track of than a traditional proxy because pages are broken into fragments. To deal with this it might be a good solution to implement HTTP Cache Channels if your ESI proxy server supports HTTP Cache Channels. Squid and Varnish (in writing, trough an extension provided by Redpill Linpro) does support HTTP Cache Channels.

Pitfall three – Turning reusable fragments into non-reusable fragments

The key factor for succeeding with ESI is to detect global fragments one can reuse across a site and make sure those are reused. Locating such global fragments are first done by analyzing the content of a potential fragment and if one find the same content to be presented on two or more locations on a site, one have a good start to make a reusable fragment. The next step in creating a reusable fragment is to make sure the mark-up and content are kept equal on all locations the fragment is supposed to be used. One possible pitfall is when a potential reusable fragment is modified to fit different visual presentation.

A ESI enabled proxy does, as mentioned, cache the content and mark-up of a fragment. CSS and JavaScript are not part of the fragment which goes into the ESI cache and because of this we have a great set of tools which can help us produce reusable fragments even if the presentation of the same fragment is supposed to be different on all locations the fragment is presented. I do not have one formula on how to be creative in this process but I will give an example which demonstrate how a bit of creativity in the presentation layer can give benefit in the ESI cache:

I work with new sites and on news sites it's pretty common to have a list of the latest published news articles on the site on almost all pages. This list of latest news articles are a great example of a fragment which fits perfectly into the ESI mind set since it's so much reused and tossed almost all over the site. Such a list can also be pretty resource intensive for the application server to generate. It's definitely something one will keep as one fragment in the ESI cache.

Recently I got the following question from our interaction designer:

Is it possible to make the list of latest news articles have different height and number of articles in the list depending on the height of an image placed to the left of the latest news articles list? The image to the left of the image will change and the height will be different from page to page.

Illustration displaying equal height of image and list of articles

In a ESI world this would be two different ESI fragments. The list of latest news articles would be one ESI fragment, which has a high reusability value, and the mark-up and content for the image would be another ESI fragment. But, solved poorly this could be a killer for the reuse of the fragment holding the list of latest news articles.

It could be solved poorly in two ways:

One - The two fragments got joined into one ESI fragment and to solve the height issue one grab the height of the image and use that to set the height of the list of latest news articles – by setting an in-line CSS style attribute with the images height on the list – and then try to calculate how many articles there should be in the list on the server side.

Two - Keep one ESI fragment for the image and one ESI fragment for the list of latest news articles and apply the same technique as in solution one to solve the equal height issue. 


<!-- Start ESI fragment 1 -->
<div id="image">
    <img src="resources/image_1.jpg" width="600" height="338" />
</div>
<!-- End ESI fragment 1 -->

<!-- Start ESI fragment 2 -->
<ul id="list" style="height:338px;">
    <li><a href="#">Lorem ipsum dolor 1</a></li>
    <li><a href="#">Lorem ipsum dolor 2</a></li>
        <!--  [SNIPP]  -->
    <li><a href="#">Lorem ipsum dolor 14</a></li>
    <li><a href="#">Lorem ipsum dolor 15</a></li>
</ul>
<!-- End ESI fragment 2 -->

Both solutions will kill the reuse of the fragment holding the list of the latest news articles. Solution one will end up with one ESI fragment less in the ESI cache since we join the image and the list but this will force the application server to render the list of latest news articles for each new image. Solution two will increase the number of ESI fragments by the same number as different heights we have on the different image fragments.

The solution which gives best effect in both the ESI cache and in the final design is actually to keep the list of latest news articles as one ESI fragment and manipulate it in the browser with a small JavaScript. In other words; we keep the list of latest news articles equal for all pages which give us one ESI fragment we can reuse in the ESI proxy. Then we use a small JavaScript - which will be cached in the browser between pages – to read the height of the image and then adjust the list of the latest news articles to the same height as the image.

I am not going into detail how this can be implemented but here are some working examples:

Example one - Page with 338px height image
Example two - Page with 452px height image

The solution has one downside; the list of latest news articles will probably contain more mark-up and content than we are going to display to the user. Which in other words mean more data transferred to the browser than the two solutions I defined as poor. In such situations one must be pragmatic and ask the question; what can we live with? Is the extra amount of data something which justifies what one gains on the server side and the effect in graphical presentation? If yes; go for it. If no; perhaps the answer for the original request is not possible.

This was just an example on how to think a little bit out of the box to gain performance effect on the server side.

21.10.2009 02:27 - Posted by Trygve - Comments: 0 - Technical

Comments:

Post a Comment:

HTML Syntax: NOT allowed