Jakub T. Moscicki, CERN IT
How to survive complexity and dynamics in computing grids

Thousands of scientific users witness every day inherent instabilities and
bottlenecks of large-scale task processing systems: lost or incomplete jobs
and hard-to-predict completion times.
This talk presents scientific research from the problem statement, system
analysis, modeling and simulation, to validation through experimental results.
It captures and characterizes complexity and dynamics of global task
processing systems using as an example the largest scientific grid to date -
the EGEE/EGI Grid. A task processing model developed in this work allows to
rigorously explain why the late-binding method is superior to traditional task
scheduling based on early binding. A study of statistical properties of task
processing times is complemented by Monte Carlo simulation.
Presenting several successful application examples from diverse domains, it
explains how heterogeneity and dynamics of global task processing systems may
be addressed and mastered in a cost-effective way directly by the users.
It describes a User-level Overlay, based on two software packages, Ganga and
DIANE, as well as advanced resource selection strategies and scheduling
approaches.