Edge Side Includes explained simple

Tags:

Edge Side Includes, or ESI, are a technology which has been around for many years. Lately I've been working a lot with component based web development where ESI have been used and seeing the benefits ESI have on the server side, have made me wonder why ESI aren't more used. I do also find a lot of developers, specially web developers, go kind of whoooot?!? when mentioning ESI so in this article I will explain what ESI are and what the benefits it provides.

ESI country; the proxy

ESI is a XML based "language" which operate in proxy servers. To understand the benefits of ESI we need to understand what a proxy is used for and how its used in server environments. On high traffic websites its pretty common to insert a proxy in front of the application servers to take some load of the application servers.

Illustration of client connecting to proxy and proxy connecting to application server

These proxy servers are also called web cache or web accelerators. Squid and Varnish are examples of such.

The traditional proxy and the problem

Traditionally a proxy will cache a whole page. The application server generate the page dynamically and hand it to the proxy. The proxy will then hold a static copy of the page and serve the static page to the user until something change in the page on the application server. When something does change in the page at the application server, the proxy throws away the cached version it's holding and cache an updated version of the whole page. No mater how large or small the change in the page is, the proxy asks the application server for the whole page again, forcing the application server to render all parts of the page.

Performance wise this is not optimal for the application server. Let's look at an example:

Lets say we have a news site holding about five thousand pages and we use a Content Management System (CMS) to maintain the site dynamically. If this site is quite popular we might have all those five thousand pages cached in our proxy.

On our pages we have areas which are similar between all our pages. A footer holding copyright and contact information is typically such an area which is common between all pages. To maintain a common footer we will in our CMS hold the footer as one object so when we need to update something in the footer we have one place to do so.

Illustration displaying the same footer used on many pages

But; updating such a global object causes some problems for the proxy. When we change a global object in our CMS we do actually update all our pages. For the proxy this means it must throw out all the pages it's holding and ask the application server for updated pages, which again gives the application server a lot to render. This create spikes on the application server and such spikes can be a killer for a poorly performing application server.

Cache fragments, not pages

A proxy server which supports ESI caches things a little bit different. Instead of caching whole pages it does cache pages structure and fragments on the page.

Illustration displaying a page divided into several components

The illustration shows how a simple page are separated into a structure and different fragments (components). Here we have a structure and a header-, left column-, center column- and a footer fragment.
How this is done technically will be discussed soon, but in a ESI environment the proxy will fetch the structure from the application server. In the structure there are "ESI URL's" referring to each fragment in the page. When the proxy sees these "ESI URL's" it will fetch each fragment these "ESI URL's" refer to from the application server and replace each "ESI URL" with its representative fragment.

Now; lets take another look at our example with the global footer. If we treat the footer, which are common between all 5000 pages, as a ESI fragment, the footer will be one cached fragment in the proxy which all 5000 proxied pages use. If we then change something in the footer, only that single cached footer fragment must be updated in the proxy. This saves the proxy from updating all pages in the cache just because one common fragment was changed.

How to start using ESI

The key to make use of ESI is to be able to break up and serve our pages as a structure and fragments. As mentioned we need to hand the ESI enabled proxy a structure which hold references to each fragment in the page. These references are URL's pointing to each fragment. In other words; each fragment we will include with ESI must be available on it's unique URL.

If we take our simple example again, the markup structure handed to the proxy can be something like this: 


<body> 
  <div id="head"> 
    <esi:include src="http://www.oursite.com/header.html" /> 
  </div> 
  <div id="left"> 
    <esi:include src="http://www.oursite.com/left.html" /> 
  </div> 
  <div id="content"> 
    <esi:include src="http://www.oursite.com/content.html" /> 
  </div> 
  <div id="foot"> 
    <esi:include src="http://www.oursite.com/footer.html" /> 
  </div> 
</body>

In this example there are four fragments included trough the esi:include tag and each fragment lives on there unique URL. You can in a way look at ESI as Server Side Includes (SSI) with URLs instead of local file system paths.

There are more to Edge Side Includes so this is just a very simple explanation of the ESI theory. Please refer to the Edge Side Includes documentation for more in depth documentation. Akamai does also hold a good documentation on ESI.

Please feel free to read my other blog posts about ESI:
What to think of when creating Edge Side Includes

05.08.2009 12:28 - Posted by Trygve - Comments: 2 - Technical

Comments:

Thanks Trygve, you provide 3 key pieces of information that have been confused, missing or buried by other sources: 1) ESI is object caching - ESI is not pure static page cache 2) ESI supports format caching -is ESI a CSS cache or other? 3) Location of ESI between HTTP server and browser- some I have seen put ESI between App server and MemCache. That seemed not right. Thanks for helping me understand

30.09.2009 07:13 - Posted by Ralph - Permalink

Thanks it's very clear and useful to know! Your website is in my bookmarks! Good job!

23.02.2010 02:44 - Posted by Neuh - Permalink

Post a Comment:

HTML Syntax: NOT allowed