Posts about performance - Ars Lounge Feedjack

*

Varnish - Trac [ma.gnolia] (Put together quickly (Haligan)) by MichaelBiven

Varnish is a state-of-the-art, high-performance HTTP accelerator. Varnish is targeted primarily at the FreeBSD 6/7 and Linux 2.6 platforms, and will take full advantage of the virtual memory system and advanced I/O features offered by these operating systems.

Saved By: Michael Biven | View Details | Give Thanks

Tags: proxy, web, performance, caching, http, accelerator, reverse proxy

Posted in: accelerator , caching , http , performance , proxy , reverse proxy , web

*

Hijacking the SubSonic relationship load process (MrKurt) by Kurt

In my quest to make SubSonic work for me, I've been stumbling through the source code and making improvements on my own super special branch. Of primary interest to me was the mechanism for loading objects related through a foreign key. A Review object might have a parent Author, accessible like so:

Review review = new Review(1);
Author author = review.Author;

The default SubSonic templates generate an accessor like this:

public Test.Author Author
{
    get { return Test.Author.FetchByID(this.AuthorID); }
    set { SetColumnValue("AuthorID", value.Id); }
}

This gets unwieldy in a hurry. Every single reference to the Author property on my Review class results in a database call. Code like this is scary:

for(int i = 0; i < 1000; i++)
{
    Author a = review.Author;
}

There is a LazyLoad option which changes the generated code slightly, persisting foreign key relationships in a private variable when they're referenced. It's a good start, but doesn't help with something like this:

foreach(Review review in someReviewCollection)
{
    Author a = review.Author;
}

In many applications, you might end up with thousands of reviews that share a small set of authors. Not only would that previous loop hit the database for every iteration, it might be reloading authors that have been seen before. This can quickly become a huge amount of load on a database server.

Yay delegates

The easiest fix for this problem would be to cache all the authors, and retrieve the cached instances when they're referenced through the Author property. Wouldn't it be nice to tell SubSonic "Hey, I want you to check the cache for all foreign key relationships in this object". Something like that would be possible with a foreign key property that looks like this:

public Test.Item Item
{
	get { return LoadSingleObject<Item, int>(
            this.ItemID,
            test.Item.FetchByID
            );
    }

	set { SetColumnValue("ItemID", value.Id); }
}

The above getter uses a new LoadSingleObject function to retrieve the related object. This function takes the related object's primary key value and a default function for retrieving that object.

For this to work, SubSonic needs some mechanism that allows us to hijack its relationship loading process. At a super high level, I attempted to make this possible by creating delegates for getting/storing individual objects, then keeping a list of those delegate pairs in each ActiveRecord object. When a foreign key property is referenced, it will run through that list looking for a hit with the "get" delegate (ie: a result that's not null). Assuming there are no hits, it will use the standard ActiveRecord.FetchById function to return the object. At that point, it iterates the list again and passed the found object back to the "store" delegate.

This has proven to be pretty flexible in my applications, and even provides a foundation for putting in eager loading at some point. When I cache my objects, I now use the RegisterSingleObjectLoader function on each of them to ensure that the cache is the first place they'll look when I reference one of their related objects.

You can have a look at my branch (/branches/kurt) to see exactly what I did. The lion's share of the changes are contained in these two files, which I've stuck on monoport so you can see them in all their syntax highlighted glory:

Posted in: c# , code , delegates , performance , subsonic

*

TruCluster: an interesting performance problem (Kilala.nl (Cailin Coilleach)) by Cailin Coilleach

The past two weeks we've been having a rather mysterious problem with one of our TruClusters.

During hardware maintenance of the B-node we moved all cluster resources to the A-node to remain up and running. Afterwards we let TruCluster balance all the resources so performance would benefit again. Sounds good so far and everything kept on working like it should.

However, during some nights the A-node would slow to a crawl, not responding to any commands and inputs. We were stumped, because we simply couldn't find the cause of the problem. The system wasn't overloaded, with a low load average. The CPU load was a bit remarkable, with 10% user, 50% system and the rest in idle. The network wasn't overloaded and there was no traffic corruption. None of the disks were overloaded, with just two disks seeing moderate to heavy use. It was a mystery and we asked HP to help us out.

After some analysis they found the cause of the problem :) Part of one of the applications that was failed over to the A-node were two file systems. After the balancing of resources these file systems stuck with the A-node, while the application moved back to the B-node. So now the A-node was serving I/O to the B-node through its cluster interconnect! This also explains the high System Land CPU load, since that was the kernel serving the I/O. :D

We'll be moving the file systems back to the B-node as well and we'll see whether that solves the issues. It probably will :)

Posted in: disk serving , performance , performance issue , trucluster

*