April 16, 2013 11:30
Empirical analysis of statistical algorithms often demands time-consuming experiments which are best performed on high performance computing clusters. For this purpose we developed two R packages which greatly simplify working in batch computing environments.
The package BatchJobs implements the basic objects and procedures to control a batch cluster within R. It is structured around cluster versions of the well-known higher order functions Map, Reduce and Filter from functional programming. An important feature is that the state of computation is persistently available in a database. The user can query the status of jobs and then continue working with a desired subset.
The second package, BatchExperiments, is tailored for the still very general scenario of analyzing arbitrary algorithms on problem instances. It extends BatchJobs by letting the user define an array of jobs of the kind "apply algorithm A to problem instance P and store results". It is possible to associate statistical designs with parameters of algorithms and problems and therefore to systematically study their influence on the results.
More details, the source code, installation instructions and much more can be found on the project's web site.
