Wednesday, July 23, 2008

GAE: Batch operations

Given the limit on individual requests with GAE, doing things like batch updates to your database (live) can be an interesting challenge. I recently had to update all of the votes for Challenge-You! to support a new rating system. The best approach I found was an AJAX-style solution. Have one request that returns a list of references to the entities you need to change (or if you're working with more than 1000 entities, you might want to partition the list into chunks by date or some other order, and work with one chunk at a time). Keeping those references in a stack in Javascript, pop them off one-by-one and send each to the controller that applies the change, repeating until the stack is empty. If the operations aren't too expensive, you could even send a handful of them per-request to speed things up. It's also trivial to implement some kind of progress bar or other graphical update to keep you informed.

And example of the Javascript code (using jQuery) that handles the client side:
var data = null;

function doupdate() {
if(data.length == 0) return;
$.ajax({
type:"POST",
url:"doupdate",
data:"id=" + data.pop(),
success:function(msg) {
setTimeout(function() {
doupdate();
}, 1000);
}
});
}

$(document).ready(function() {
$.ajax({
type:"POST",
url:"getlist",
data:"",
success:function(msg) {
data = msg.split(" ");
doupdate();
}
});
});

In this example, my site would have two urls for handling this, "getlist" which returns a space-delimited list of references/ids (however you're handling the entities) and "doupdate" which takes in one reference and applies the needed change.

Sunday, July 20, 2008

Speed kiddies

Speed kiddie:

Someone who learns or uses a low-level (often compiled) language for the constant coefficient performance increase. These people often have no regards for computational complexity and are usually confused by asymptotic notation. They frequently suffer from severe cases of premature optimization and believe that manageability comes second to runtime speed (in all applications). You can often find them trolling around web-forums bragging about their pointer or bit manipulation skills (the code is generally characterized by non-reusable design and a significant lack of readability).

Examples:
  • "What do you mean my algorithm runs at O(n!)? It's twice as fast as yours with this small test set"
  • "How do you write efficient code in Python... It doesn't even have pointers!"
  • "I don't need to write readable code, I'm not going to forget my own code..."
  • "High level languages are for the weak, real men enjoy pain... PAIN!!!"
  • "Lisp doesn't offer anything new to me, C has macros too!"
  • "RAWRRR POOINTERRRZZZ!!!!!!!!!!!!!!!!!!"