Archive for the ‘Lussumo’ Category

Benchmarking Magic (revisited)

Wednesday, October 15th, 2008

Since I started development on Garden (the next Lussumo development framework), I have spent a LOT of time crawling the web for information on everything from PHP optimization, to wacky new design patterns; anything I can get my hands on that will spark some new ideas in my head. In my travels, I’ve found a few very interesting posts that I’ve gone back to and read over and over again; getting new insights with each new read (and subsequent contemplation). One of these articles is called Benchmarking Magic by Larry Garfield (If you are interested in PHP development, I highly recommend you read his blog regularly).

I’ve been entranced by the possibilities of PHP’s magic methods for some time; but all I’ve ever heard about them was that they are slow. Larry took the time to benchmark PHP’s magic methods next to the standard alternatives to see exactly how slow they are. His results, though not too surprising, were definitely enlightening. Calling PHP’s magic __call method is slow, but the real issue is that you can’t do much with a __call unless you are then able to take the requested method and arguments and do something with them. This usually involves the use of PHP’s call_user_func or call_user_func_array functions. And Larry’s results showed that while the use of __call was slow, if you combined __call with the use of call_user_func*, it resulted in an abysmally slow return.

Now, I’ve been struggling to come up with a slick way of integrating plugins with Garden, and I decided to take the plunge and use PHP’s __call magic method to help me accomplish part of the plugin framework. But after reading and re-reading Larry’s article (and this other one by Paul M Jones) I decided that I just can’t justify using either of the call_user_func* functions – it would simply be too slow!

But, what are the alternatives?
The __call method takes two arguments: $Method and $Arguments. These represent the name of the method that was called, and the arguments that were passed into that method. The real problem, and the reason why you’d need to use call_user_func_array() 99% of the time is because when you redirect that method call somewhere else, you need your arguments to be split so that they go into the resultant method in individual pieces rather than as one single array called $Arguments.

In other words, if I am redirecting a method call to another method on the request object, this would work:

function __call($Method, $Arguments) {
     call_user_func_array(array($this, 'DestinationMethod'), $Arguments);
}

But this would NOT work:

function __call($Method, $Arguments) {
     $this->DestinationMethod($Arguments);
}

Because DestinationMethod is expecting the arguments to come in as:

$this->DestinationMethod($Arguments[0], $Arguments[1], $Arguments[n]);

But in a general sense within the constructs of Garden, I might not always know what the “DestinationMethod” will be (implying I’ll end up having to reference the destination method as a variable variable, like $this->$DestinationMethod()), let alone how many arguments that method is expecting!

So, last night I was thinking about how I could possibly work around the necessity of call_user_func_array(), and a wacky idea came to me: Why not pad the $Arguments array up to a length of 10 and pass in all 10 arguments to the variable call of $DestinationMethod? Sure, $DestinationMethod might only have 3, 4, or x number of arguments, but since I have full control over how many arguments go into all of the methods in Garden, I can certainly examine it to ensure that there are never more than 10; and PHP doesn’t care if you don’t define extra arguments explicitly.

For example:

function Foo() {
     return;
}

// This does not result in any errors being thrown:
Foo(1, 2, 3, 4, 5, 6, 7, 8, 9, 10);

But the question remains: How much faster is a variable method call than a call_user_func_array call?

Using similar methods to Larry Garfield, I created a benchmarking script to test everything from a literal function call straight down to a variable method call via PHP’s __call magic method. Just like Larry’s results, my exact numbers in the following tests weren’t particularly interesting; What’s interesting is their relative value. All tests are run in a single script (below) on the following system:

Macbook Pro on AC Power
Intel Core Duo 2 GHz
2 GB RAM
Apache 1.3.39 on Windows XP SP3
PHP 5.2.5

To quote Larry:

Because it’s a fairly beefy system, all tests are run 2,000,000 times so that we have worthwile numbers to compare. All times listed below are in seconds. Of course, any such tests will vary a bit between runs, and even between two tests in the same script. We’re looking for overall trends here, not exact numbers, but it’s important to keep in mind that micro-benchmarks are an inexact science. Also keep in mind that I don’t know the internals of the PHP engine well at all, so my analysis is based on logical extrapolation, not actual knowledge of the PHP engine itself.

The Results

Testing Method Time
Literal Function 1.941
Variable Function 2.792
call_user_func() 5.445
call_user_func_array() 6.376
Literal Method 2.912
__call 6.931
__call with call_user_func_array 17.903
__call with Variable Method 10.727

There you have it, a simple variable method call is most definitely faster than call_user_func_array(), and you can overcome the necessity of call_user_func_array by passing in x number of arguments and letting the extras “go to waste”.

So, did this solve all of my problems with the addon framework in Garden? Nope. But it lets me sleep a bit easier. I’m sure that resorting to __call on every method call would result in a slow application; but I doubt that plugin authors *need* that much access to the core. So it all comes down to deciding where it would be most useful to implement in the application.

For anyone interested, here is the script that gave me the results above:

<?php
error_reporting(E_ALL | E_STRICT);
define('ITERATIONS', 2000000);

function Foo($Bar) {
   return;
}

$Foo = 'Foo';

class TestCall {
  function Foo($Bar) { return; }
  function __call($Method, $Args) {
   if ($Method == 'VariableMethod') {
      $Call = 'Foo';
      return $this->$Call($Args[0]);
   } else if ($Method == 'Return') {
      return;
   } else {
      return call_user_func_array(array($this, 'Foo'), $Args);
   }
   return;
  }
}

$t = new TestCall();

function Row($Test, $Start, $Stop) {
   echo '<tr><td>'.$Test . '</td><td>' . ($Stop - $Start) . ' seconds</td></tr>' . PHP_EOL;
}

echo '<table border="1" cellpadding="0" cellspacing="3">';

$Start = microtime(true);
$TestName = 'Literal Function';
for ($i = 0; $i < ITERATIONS; ++$i) {
   Foo(1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = 'Variable Function';
for ($i = 0; $i < ITERATIONS; ++$i) {
   $Foo(1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = 'call_user_func()';
for ($i = 0; $i < ITERATIONS; ++$i) {
   call_user_func('Foo', 1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = 'call_user_func_array()';
for ($i = 0; $i < ITERATIONS; ++$i) {
   call_user_func_array('Foo', array(1));
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = 'Literal Method';
for ($i = 0; $i < ITERATIONS; ++$i) {
   $t->Foo(1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = '__call';
for ($i = 0; $i < ITERATIONS; ++$i) {
   $t->Return(1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = '__call with call_user_func_array';
for ($i = 0; $i < ITERATIONS; ++$i) {
   $t->CallUserFuncArray(1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

$Start = microtime(true);
$TestName = '__call with Variable Method';
for ($i = 0; $i < ITERATIONS; ++$i) {
   $t->VariableMethod(1);
}
$Stop = microtime(true);
Row($TestName, $Start, $Stop);

echo '</table>';

Addendum
Upon implementing the method described above, I discovered one caveat to the solution. If you always fill in the full argument list for the method being called, any default values assigned in the method declaration are lost.

For example, if the method I am calling is defined as:

function MyMethod($Arg1 = 'default') {}

And I call it in my __call method as described above:

$this->$MyMethodName($Arg[0], $Arg[1], $Arg[2], $Arg[3], $Arg[4], $Arg[5], $Arg[6], $Arg[7], $Arg[8], $Arg[9]);

The ‘default’ value will be lost and replaced with whatever the value I padded the array with. Obviously this is a big problem. The two solutions are to (a) explicitly enforce the default values within all of your methods [eeek], or (b) count the number of arguments and call the method selectively:

$Count = count($Args);
if ($Count == 0) {
   $this->$Call();
} else if ($Count == 1) {
   $this->$Call($Args[0]);
} else if ($Count == 2) {
   $this->$Call($Args[0], $Args[1]);
} else if ($Count == 3) {
   $this->$Call($Args[0], $Args[1], $Args[2]);
} else if ($Count == 4) {
   $this->$Call($Args[0], $Args[1], $Args[2], $Args[3]);
} else {
   $this->$Call($Args[0], $Args[1], $Args[2], $Args[3], $Args[4]);
}

I ran this kludgey if statement through the same benchmarking code as listed above and found that it took on average 2 – 3 more seconds to run. So, that brings the entire summary of benchmarks up to:

Testing Method Time
Literal Function 1.941
Variable Function 2.792
call_user_func() 5.445
call_user_func_array() 6.376
Literal Method 2.912
__call 6.931
__call with call_user_func_array 17.903
__call with Variable Method 10.727
__call with Variable Method & if statement 12.911

Not as good as I had hoped, but still better than call_user_func_array.

One final note on the subject:
Remember that all of my benchmarks were achieved by calling the methods 2,000,000 (that’s two million) times each. What if I ran another test where the iterations were bumped down to something more like I would expect for one page-load of Garden? Imagine if every single method call on every single object in Garden were run through __call instead of directly to themselves. How many would there be? I’m going to go on the high side and say that there are 200 method calls on a single page-load. And let’s say that the application is pretty busy with 30 people loading pages at the same time; What would the numbers stack up like, then?

200 methods * 30 requests = 6000 calls:

Testing Method Time
Literal Function 0.005
Variable Function 0.008
call_user_func() 0.015
call_user_func_array() 0.019
Literal Method 0.008
__call 0.021
__call with call_user_func_array 0.053
__call with Variable Method 0.033
__call with Variable Method & if statement 0.040

We’re really talking about fractions of a second of difference. Granted, on a heavy-load application that could mean the difference between a slow application and a fast one. But 99% of the people who download and use Garden will *not* be running extremely high-traffic websites where a slowdown would be noticeable, and as Larry Garfield pointed out in his article, you can always throw more CPU at the code to speed it up – and if you’ve got a high profile website, I’m sure that’s just what you’d do.

Personally, I think the benefits far outweigh the micro-seconds in slowdown. I’d be willing to bet that when the application is in alpha, and I start profiling to find bottlenecks, this will not be one of the sore spots.

Calling Regular Expressions Masters

Monday, October 13th, 2008

I’m working on a bit of code that prepends database table prefixes to a string of SQL. For example, I would take the following SQL:

select User.UserID, Permission.PermissionID from User
left join UserRole
     on User.UserID = UserRole.UserID 
left join RolePermission 
     on UserRole.RoleID = RolePermission.RoleID 
left join Permission 
     on RolePermission.PermissionID = Permission.PermissionID 
     and Permission.Name = 'can_sign_in' 
where User.Name = 'mark' 
     and User.Password = md5('password')

… and turn it into …

select LUM_User.UserID, LUM_Permission.PermissionID from LUM_User
left join LUM_UserRole
     on LUM_User.UserID = LUM_UserRole.UserID 
left join LUM_RolePermission 
     on LUM_UserRole.RoleID = LUM_RolePermission.RoleID 
left join LUM_Permission 
     on LUM_RolePermission.PermissionID = LUM_Permission.PermissionID 
     and LUM_Permission.Name = 'can_sign_in' 
where LUM_User.Name = 'mark' 
     and LUM_User.Password = md5('password')

I’ve got 99% of it working properly, but I am having difficulty with the “on” clause of the join. In a simple join, like the first join above, the following code works perfectly:

$DbPrefix = 'LUM_';
$On = 'User.UserID = UserRole.UserID';
$On = preg_replace('|([\w\.]+)([\W\s]+)(.+)|', $DbPrefix . "$1$2" . $DbPrefix . "$3", $On);
// Returns: LUM_User.UserID = LUM_UserRole.UserID

But the regex I have here is only good for one join. So, for example, this would be a problem:

$DbPrefix = 'LUM_';
$On = "RolePermission.PermissionID = Permission.PermissionID and Permission.Name = 'can_sign_in' ";
$On = preg_replace('|([\w\.]+)([\W\s]+)(.+)|', $DbPrefix . "$1$2" . $DbPrefix . "$3", $On);
// Returns: LUM_RolePermission.PermissionID = LUM_Permission.PermissionID and Permission.Name = 'can_sign_in' 

As you can see, it only prepends the prefix on the first two. I’m afraid I’m at a loss as to how I can get it to keep matching on subsequent joins for that table. In this example you might say that I should put the second part of my join into the where clause; but that’s not what I’m trying to accomplish; so just pretend that it’s a regular table-to-table join and needs to be here.

Anyone regex-ey enough to conquer this one?

UPDATE!
Thanks to Jeff, who solved it with:

$DbPrefix = 'LUM_';
$On = "RolePermission.PermissionID = Permission.PermissionID and Permission.Name = 'can_sign_in' ";
$On = preg_replace('/(\\w+\\.)/', $DbPrefix.'$1', $On);
// Returns: LUM_RolePermission.PermissionID = LUM_Permission.PermissionID and LUM_Permission.Name = 'can_sign_in' 

Vanilla 1.1.5 Released

Monday, September 22nd, 2008

I am pleased to announce the release of Vanilla 1.1.5. This release was entirely the result of the Lussumo Community’s participation. Particular kudos go out to Wallphone, (who contributed over 50% of the fixes and enhancements), and Dinoboff (who keeps me in the loop!). A list of all contributors can be found here.

Vanilla 1.1.5 can be downloaded from getvanilla.com now.

A full list of all fixes and features can be seen in the release notes.

Upgrading instructions can be found here.

An ongoing discussion of this release can be found on the community forum.

Personal Note
I have never before been so disconnected from the goings on in my software. Every other release previous to Vanilla 1.1.5 has had my fingerprint on almost every aspect of it. This one, however, doesn’t have any part of me in it. I may have offered advice to the contributors when asked, but I don’t think I personally wrote even a single line of code for it. It is both bitter and sweet that it has come together so well without me. I cannot say enough how grateful I am to all of the contributors.

Donkey Work

Wednesday, September 17th, 2008

Weeee!

After fruitless efforts into alternate configuration setups for Garden, I’ve been spending some time doing more trivial but necessary development. I started working on some ideas I’ve had for animated “progress” images, and I finally got to work on the javascript behind the menu UI component (yes, there will be some minor jQuery behind the menu – but it will also be non-js compatible).

I have to say, there is nothing that can renew your love for server-side programming like client-side programming. I’m sure some of you will know exactly what I mean.

Load Testing == Confirmed Fears

Friday, September 12th, 2008

Serious Statistics

After my previous post about being stuck, I decided to do some definitive testing to see what is a better solution for saving and loading of configuration settings in Garden. The three methods that were being discussed in the post (and the comments below it) were (1) sqlite, (2) mysql, and (3) a flat php file containing the configuration settings in an associative array (the way Vanilla does it’s config settings now).

In order to make the test have any meaning, I had to use far more configuration settings than Vanilla has or will ever need. So, for both sqlite and mysql I had a table with 400 bogus configuration settings, and I queried the database 400 times to retrieve each and every one. then I spat out the time it took to get the data. For the flat file, I simply created an array containing 400 settings and echo’d them all out to the screen, recording the time taken as well.

The results were not very thrilling. I found that both sqlite and mysql were all over the map, taking anywhere from 0.1 seconds to 1 second to load the 400 configuration settings. The flat php file with the associative array took approximately 0.3 seconds every time.

Unhappy with the results of the database comparisons, I decided to download and try a load testing tool so I could really see how 5 or more users loading that data could affect the outcome. I did some searching and eventually found and downloaded Verisium vPerformer. The tool was pretty straightforward to use (although my visual studio debugger came up a number of times asking if I wanted to fix problems in the app). I performed the same test on all three files: Request the pages as five different users every second for 30 seconds and take the average.

The result was that MySQL averaged at 4.3 seconds per page-load, SQLite averaged at 3.7 seconds per page load, and the flat file averaged at 1.2 seconds per page load.

These are not pleasing results. I was hoping that SQLite would be the definitive answer to all of my problems in this regard. The only thing that I have left to keep me happy is that NONE of these results represent a real-world request in Garden. In reality I’ll never have 400 configuration settings in Garden, it will be something more like 20 to 30. Beyond that, I’ll never call 400 configuration settings individually on a single page load, it will be something more like 2 or 3 per page load.

When you reduce the numbers and increase the load, it will probably be a pretty even playing field with SQLite only slightly better than MySQL, and the flat (insecure) php file only slightly better than that.

So, it’s decision time. Any thoughts?

Stuck

Thursday, September 11th, 2008

Over the last few weeks since I’ve been back from vacation, I’ve found myself stuck. Every time I look at something in Garden, I am left questioning if I’ve made good decisions. I don’t know if I should proceed as-is or if I should go back and redo things. I’ve never had this happen to me before, and the only thing that I can think of that is even slightly similar is writer’s block.

For example: in Vanilla 1 I used php-writable configuration files to handle saving/changing of configuration settings so that the application wouldn’t have to do a round-trip to the database in order to figure out basic, rarely-changing settings that will be used on every page-load. This method of saving application settings in files was simultaneously loved and loathed by the developer community. Some people loved how it sped the application up, others hated how it introduced a potential security hole into their server.

Granted, I’ve had NO reports of servers being compromised because of this “hole”. However, I have never liked the fact that it is there. In garden, I’ve gone way further with this type of “setting caching” method and I have a ton of files that are built and rebuilt whenever the application is run; and I’m not sure if I want Garden to do that anymore. I’ve read that php5 is going to (or maybe does already?) have built-in support for SQLite – which could replace all of my php-editable files (from the things I’ve read online, it looks like Drupal is going to be moving in this direction). However, who wants to be limited to using SQLite as a database engine? Or worse: using both SQLite and MySQL? That sounds like a more nightmarish setup than the editable files.

So I’m kind of stuck. I don’t know how to proceed on this, or any other avenue of programming in the new framework. Today I spent a number of hours tinkering with various parts of the application fixing small bugs. I guess that’s a start.

Gotta get my head in the game.

Fun with jQuery (part 2)

Thursday, June 12th, 2008

CPR

I recently posted about having some problems getting things to work properly with jQuery. Admittedly, I’m not that great with javascript. But I’m finding this to be a very steep learning curve. I understand the basics of jQuery. The problem I keep having is that I run into things that I just don’t think jQuery can do. Here is my latest one:

How do you use selectors to identify a parent element?

Much like you can identify a span within a div like this:

$('div > span').doSomethingNeato();

How can I identify the span’s parent through selectors? I mean something like this:

$('div < span').doSomethingNeato();

Of course I can go the distance with something like this:

$('span').doSomethingNeato(function() {
    var mydiv = $(this).parent();
});

But what if I need to get at the div directly through the selectors alone? Is there a way to do it that I don't know about? Doesn't this pose a problem?

In my case it is a big problem because I am trying to write a general purpose plugin that needs to perform an action on some element, get a response, and then place the response in the page somewhere. The place where the response is placed is going to be relative to the button that was clicked which caused the action. It could be within the place that was clicked, it could be the place that was clicked, or it could be within a parent of the place that was clicked. I need to be able to specify a selector to identify the place.

Edit
The only idea I've had is to pass in a function that takes "this" as it's only parameter and use that function to return the target whenever it is needed. I don't know if that will work yet - I'll have to try...

Edit #2
Thanks to everyone for their input - but especially thanks to dinoboff who pointed me to John Resig's blog where he shows that you can accomplish what I want with:

$("div:has(span)")

Oh jQuery, I'll never doubt you again.

jQuery, closures, objects, instances, suicide

Tuesday, June 10th, 2008

CPR

Do you have a fantastic grasp of jQuery and closures and would like to help me out?

I’m stumped on how to keep a settings array alive through the life-cycle of a jQuery plugin I’m working on, and I can’t find any answers on the Google or the Internet.

Want to help me? Please reply to this entry. I’ll continue slitting my wrists with the dull edge that is javascript until you do…

Update
After a couple of suggestions on plugin architecture, I’m still no further ahead. I’ve put together a summary of my problem. Check out the following code:

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-ca">
<head>
    <title>test</title>
    <script src="/garden/js/jquery.js" type="text/javascript"></script>
<script type="text/javascript">
<!--

(function($) {
    var opts;

    $.fn.identify = function(options) {
        opts = $.extend({}, $.fn.identify.settings, options);
           
        this.click(function() {
            $.fn.identify.speak();
            return false;
        });
           
        return this;
    }
       
    $.fn.identify.speak = function() {
        alert(opts.parameter);
    }

    $.fn.identify.settings = { parameter: 'default' };

})(jQuery);

jQuery(document).ready(function($) {
   $('a:first').identify({parameter:'first'});
   $('a:last').identify({parameter:'last'});
});


// -->
</script>
<body>
<a href="http://localhost/garden/test.php">http://localhost/garden/test.php</a>
<br /><br /><br />
<a href="http://localhost/garden/test.php">http://localhost/garden/test.php</a>
</body>
</html>

If you create this example and then click the first link, your popup will say “last” even though the parameter that was passed to the anchor was “first”. Simply put: I need it to say “first”.

If I change my javascript slightly, I can make it say first like so:

(function($) {
    $.fn.identify = function(options) {
        var opts = $.extend({}, $.fn.identify.settings, options);
      
        this.click(function() {
            alert(opts.parameter);
            return false;
        });
          
        return this;
    }

    $.fn.identify.settings = { parameter: 'default' };
})(jQuery);

jQuery(document).ready(function($) {
   $('a:first').identify({parameter:'first'});
   $('a:last').identify({parameter:'last'});
});

But the whole problem is that I need to be able to call a *different* function (like the “speak” one above) within the plugin and access the correct parameter. The only way that I have been able to do this successfully is by doing something like this:

(function($) {
    $.fn.identify = function(options) {
        var opts = $.extend({}, $.fn.identify.settings, options);
       
        this.click(function() {
            $.fn.identify.speak(opts);
            return false;
        });
           
        return this;
    }
       
    $.fn.identify.speak = function(opts) {
        alert(opts.parameter);
    }

    $.fn.identify.settings = { parameter: 'default' };
})(jQuery);

jQuery(document).ready(function($) {
   $('a:first').identify({parameter:'first'});
   $('a:last').identify({parameter:'last'});
});

This doesn’t really work for me because I can’t guarantee that opts will be able to get passed into my function if, say, I’m using a callback function that can’t take any parameters.

So, the bottom line: How can I save the parameters specific to the correct instance so that they are properly accessible from an outside function?

Any ideas?

Update #2

Dinoboff suggested that I use jQuery’s data() method to store and retrieve the settings. However, his code didn’t work for me. To get it to work, I had to do this:

(function($) {
    $.fn.identify = function(options) {
        this.data('identify_settings', $.extend({}, $.fn.identify.settings, options));
		
        this.click(function(e) {
            $.fn.identify.speak(e);
            return false;
        });
		
        return this;
    }

    $.fn.identify.speak = function(e) {
        var settings = $.extend({}, $.fn.identify.settings, jQuery.data(e.target, 'identify_settings'));
        alert(settings.parameter);
    }

    $.fn.identify.settings = { parameter: 'default' };

})(jQuery);

Which basically tells me that the source of the problem is that there is no way for the outside function to know what sent it unless you pass in a reference. The click event on the affected element knows what was sent via “this” and/or event arguments.

Unless someone comes up with something different, I’m left thinking that there is no way to do this.

Update #3

Thanks to everyone for their input. In the end I decided to go about it by passing in my settings wherever necessary. I also wanted to be able to programmatically call my identify method, so in the end I went with this type of thing:

(function($) {
    $.identify = function(options) {
        var settings = $.extend({}, $.identify.settings, options);
        $.identify.speak(settings);
    }

    $.fn.identify = function(options) {
        var settings = $.extend({}, $.identify.settings, options);
		
        this.click(function() {
            $.identify.speak(settings);
            return false;
        });
		
        return this;
    }
	
    $.identify.speak = function(settings) {
        alert(settings.parameter);
    }

    $.identify.settings = { parameter: 'default' };
})(jQuery);

jQuery(document).ready(function($) {
    jQuery.identify({parameter:'programmatically set'});
    $('a:first').identify({parameter:'first'});
    $('a:last').identify({parameter:'last'});
});

The page loads and alerts “programmatically set”. Clicking the first link alerts “first”, and clicking the last link alerts “last”.

Plugging In (Part 3)

Saturday, June 7th, 2008

green light

Just a note to say that after testing the new extension method in Garden, I haven’t noticed any slowdown in performance at all. All you Vanilla addon developers out there are going to love this.

More Scaffolding

Friday, June 6th, 2008

Yeah, I know. The scaffolding screenshots are getting boring. But see if you can figure out what’s interesting about this one…