The esoteric discipline of website traffic analysis first came under public scrutiny in the early days of the dotcom revolution. In the absence of real financial figures, new wave internet companies were rated on their ability to attract "eyeballs". The more eyeballs or page views a website could claim, the higher its stock was valued.

The number of page views on a website is, of course, now recognised as an unreliable measure of the value of a business plan. Curiosity about the content of a site does not necessarily translate into sales and profits.

But the software technology developed to measure and analyse website traffic has continued to evolve and grow in importance. It is now central to the success of any website and, by implication, to any plans to move into electronic business.

Clear and accurate data about the performance of a site is essential if it is thrive and continue to attract those all-important eyeballs. Lengthy response times from a site that is overloaded with traffic will lose visitors in the time it takes to click a mouse. Similarly, bad web page design or a complicated site topography will deter browsers. Web traffic analysis software can help.

"The user community on the web has high expectations. It is so easy to use, they think they can do anything and they want performance that compares with their desktop applications. If something goes wrong they will leave the site," says Andy Crosby, European field market manager for performance and system testing specialist Mercury Interactive.

"Site managers need to be able to identify a problem quickly, diagnose the cause, and take remedial action. That is why performance monitoring is so important," he adds.

Mercury provides software that not only monitors current performance but also can anticipate surges in traffic. Mr Crosby says it detects potential spikes by analysing patterns of traffic: "You can set multiple thresholds with alerts to show a potential problems. We use a simple traffic light type system with green for all is well, red for problems and amber for potential degradation in the performance."

Monitoring the performance of computer systems and communications networks is a long-established discipline reaching back to the earliest mainframe computers and the terminal networks they supported. The banks' automated teller machine (ATM) networks and airline flight reservation systems, for example, have operated under the control of advanced monitoring software for decades.

But the web brings new dimensions to performance monitoring. Traffic loads are much less predictable than those that move through an ATM network. It is more difficult to anticipate peaks because access is global. Unexpected external events can trigger enormous surges in traffic. On the positive side, monitoring can also collect data about user's website behaviour which can be used to improve page designs.

"The big problem is that much of the internet is outside of your control. A user's experience of your site might be a nightmare and you might not know about it.

"People are putting more and more complicated applications on the web - and there are no manuals to help users. There is a huge opportunity to foul up," says Malcolm Duckett, vice president of marketing at web traffic specialist Speed-trap.com.

Along with a growing number of specialist software developers, Speed-trap.com aims to solve these problems by monitoring site activity at a number of different levels.

These facilities range from monitoring traditional network traffic, through tracking performance of web servers down to detailed data about where users click on individual web pages.

"It is all about performance-related usability - how well does the site work at the user level," says Mr Duckett. "You have to start at a general level to highlight problem areas in the network and then focus on the detail at page level."

Stephen Eick, chief technology officer at Visual Insights, one of the leaders in the web traffic analysis software market, also sees ease of use extending to the way performance data is presented: "Our approach is to make it easier to see how a site is performing by using visualisation techniques. You can, for example, see people moving about a website gathering pages as they go and gain a sense of what is happening."

This type of visual aid can help engineers to analyse and diagnose problems quickly and take corrective measures - such as adding more server capacity or more communications bandwidth. The tools also collect data about user experiences on a site. "We have built instrumentation on the web server that can capture the clicks and recreate the user experience," explains Mr Eick.

The trend towards so-called "caching" and "supercaching" - where often-used web pages are stored in high-speed servers to speed up access - can, however, skew the results. Cached pages will not necessarily refer to their original source site.

Mr Duckett of Speed-trap.com says that monitoring the web server alone is inadequate. "You can monitor traffic through the web server, but it does not give the full story. Web servers have no sense of a 'session' and treat each request as separate. This makes it hard to trace a full user experience only from the clickstream analysis."

He says that Speed-trap.com's approach is to send non-intrusive software agents to the browser device to collect data about the user's movements through a web page. Written in the portable programming language Java, the agents can work only within the web page that sent them and they disappear once the browser leaves the page.

Web traffic analysis techniques still have room for further development. Businesses need proven ways to interpret the detailed data about users and their responses to a website if they are to gain the full benefit from capturing all those clicks.