The size, diversity and continued growth of the World Wide Web combine
to  make its understanding difficult even at the most basic levels.
The  focus of our work is in developing novel methods for measuring
and analyzing the Web which lead to a deeper understanding of its
performance.  We describe a methodology and a distributed
infrastructure for  taking measurements in both the network and
end-hosts.   The first unique characteristic of the infrastructure is
our ability to generate  requests at our Web server which closely
imitate actual users.  This ability is based on detailed analysis of
Web client behavior and the  creation of the Scalable URL Request
Generator (SURGE) tool.  SURGE provides  us with the flexibility to
test different aspects of Web performance.  We  demonstrate this
flexibility in an evaluation of the 1.0 and 1.1 versions of the Hyper
Text Transfer Protocol.

The second unique aspect of our approach is that we analyze the
details of Web transactions by applying critical path analysis (CPA).
CPA enables us to precisely decompose latency in Web transactions into
propagation delay, network variation, server delay, client delay and
packet  loss delays.

We present analysis of performance data collected in our
infrastructure.   Our results show that our methods can expose
surprising behavior in Web  servers, and can yield considerable
insight into the causes of delay  variability in Web transactions.