The Secrets to S&OP Success

Ask any supply chain professional this question: Is sales and operations planning (S&OP) important to your company? The chances are good that no matter what type of company they work for the reply will almost always be “Yes!” But if this attitude is so commonly held, why are so few companies willing to step forward and say they are actually doing something to institutionalize a strong S&OP process in their business?

It is no secret that most companies struggle with even the basics of balancing supply and demand in their supply chains. Retailers have excess inventories for some products while facing high product shortages for others. Consumer products companies are challenged with building ahead of the seasonal curve, which is based at best on questionable histories and more often on uninformed hunches. High tech and industrial companies work hard to put master plans in place only to have them unravel in the face of customer and supplier churn and uncertainty. And distributors must balance not wanting to have an oversupply in their distribution centers against the hefty discounts that usually result from carrying too much inventory of their product in the stores. Even worse, stories abound of companies that were not prepared for the demand created by highly discounted promotions, resulting in product shortages and a record number of unsatisfied customers.

Could all of these symptoms have a common root cause? Could significant progress be made in reducing these undesirable effects in so many companies? Certainly all supply chain professionals and executives want to reduce uncertainty and continuously improve how they manage risk. If there are so many opportunities for improvement and so much support for S&OP, what are the obstacles and what are the steps to overcome them? In this article, we address these key questions.

The Potential of S&OP
Regardless of size and industry, most companies in recent times have faced significant business challenges including shrinking profit margins, reduced customer loyalty, growing global competition, and increased supply chain velocity. At the same time, business growth is either below plan or below potential. These market factors have created new obstacles for supply chain executives and managers and have altered the global competitive environment into one of high uncertainty and risk. What has become clear is the importance of integration across operational silos as well as trading partners.

Sales and operations planning is one of the key strategies and approaches that successful companies are taking to respond to an increasingly complex business climate. The need for S&OP is being fueled by customer demand for faster response to market shifts and for more made-to-order products and services. For companies looking to create a sound strategy to mitigate and manage risks while increasing profitability—S&OP is the answer.

First a definition: S&OP is the set of business processes and technologies that enable an enterprise to respond effectively to demand and supply variability with insight into the optimal market deployment and most profitable supply chain mix. S&OP strategies help companies make “right-timed” planning decisions for the best combination of products, customers, and markets to serve. The typical planning period ranges from four weeks to as long as two years. When applied correctly, S&OP has the power to enable an enterprise to achieve an immediate and significant increase in return on investment. It can have a direct impact on profitability, performance, customer satisfaction, and the product portfolio.

Many company executives, however, are unsure what the key elements of an S&OP program really are. Common questions include: Isn’t S&OP only for large enterprises? And, why would my company need to implement S&OP now? To help answer questions like these, Oracle sponsored The Sales and Operations Planning Benchmark Report, a study of more than 200 companies conducted by the Aberdeen Group.

The goal of the S&OP study was to survey a wide variety of large and mid-size companies to identify successful strategies for sales and operations planning. One of the key findings was that most enterprises have some sort of S&OP process to align demand and supply. In fact, more than 70 percent of respondents to Aberdeen’s survey indicated that they are actively engaged in enhancing their existing S&OP capabilities. As shown in Exhibit 1, a significant number of participants revealed that they expect an enhanced S&OP program to significantly improve operational performance across the value chain, from sales to procurement. These respondents are motivated to improve their S&OP process by a combination of decreased business growth, diminishing profit margins, and competitive pressures.

The benchmark study also revealed that leading S&OP practices are evolving away from merely balancing supply and demand. They now involve a more powerful and holistic process that allows companies to improve revenues, reduce inventory, increase profits, create a more dynamic product portfolio, and maintain longer-term customers. The study results also demonstrated that as S&OP strategies have evolved, the market drivers have changed.

This article details the results of the study and highlights the best practices being used by leading companies to meet customer demand while maximizing financial gain and synchronizing all operational plans. This article will also help readers determine if they need to update their current S&OP and rethink outdated processes.

ADVERTISEMENT

<script language="Javascript1.1" src="http://ad.doubleclick.net/adj/scm.rbi/;abr=!webtv;desc=CA6327044;lay=article;type=boombox;sz=336x280;u=1085321165%7C615100179%7C724440032%7C0%7Carticle%7CCA6327044%7C%7C;ord=1149062768?" type="text/javascript"> </script><script type="text/javascript"> <!-- if ((!document.images && navigator.userAgent.indexOf('Mozilla/2.') >= 0) || (navigator.userAgent.indexOf("WebTV") >= 0)) { document.write('<a href="http://ad.doubleclick.net/jump/scm.rbi/;abr=!webtv;desc=CA6327044;lay=article;type=boombox;sz=336x280;u=1085321165|615100179|724440032|0|article|CA6327044||;ord=1149062768?">'); document.write('<img src="http://ad.doubleclick.net/ad/scm.rbi/;abr=!webtv;desc=CA6327044;lay=article;type=boombox;sz=336x280;u=1085321165|615100179|724440032|0|article|CA6327044||;ord=1149062768?" border="0" width="336" height="280"></a>'); } //--> </script><noscript></noscript>

Current Realities in Today’s Market
In the past few years, we’ve seen a sharp increase in the number of trends that are putting growing pressure on traditional S&OP practices. For example, brand loyalty has been declining, demand for customized and configured solutions has increased, and market uncertainty and global competition has eroded margins. Acquisitions, joint ventures, and outsourcing are changing organizational structures and are requiring rapid changes in plan objectives and targets. Companies are forced to concentrate efforts and resources on the best products, customers, markets, and channels while being constantly on guard in this unpredictable environment.

Companies have also come to realize that the risks and costs associated with poor decision making have increased—particularly in the area of aligning supply and demand and linking that to profitability. When companies target the wrong customers, products and services, or channels and geographies, they court disaster! Today’s market is less forgiving and much riskier. Excess inventory is frequently discounted. Unsatisfied customer demand now runs the real risk of lost sales and revenue.

Further, current market dynamics have rendered traditional S&OP processes and technologies obsolete. In the past, S&OP was characterized by demand-supply balancing in aggregate units such as tons, cases, cubic feet, and so forth. A key difference for today and for the future is that S&OP is no longer just about balancing supply and demand. It is about searching for and executing the most profitable strategy out of many possible scenarios. It is about relying on and enhancing those critical factors that give the business a sustainable competitive advantage.

Beyond the need for analyzing scenarios is the need to respond in real time. In the past, companies were faced with a big dilemma: Daily events, such as better-than-expected success with a big promotion, required a quick response, but the current planning-cycle capabilities were not up to the challenge. Planners could not replan quickly enough and in a consistent, measurable way to take advantage of opportunities as they arose. This failure has led senior management and staff to lack confidence in their companies’ current S&OP “balancing act.”

This is important because executive sponsorship is one of the necessary conditions to S&OP success. Lack of executive-level influence hinders a company’s ability to improve its planning capabilities. This is because, in real time, urgent priorities arise. When management lacks confidence in the S&OP process, reaction and expediting become the norm.

Other internal difficulties that present obstacles to the implementation of better S&OP programs and methods include budget constraints and the lack of credible benchmarks.

Key S&OP Business Findings
The results of the Aberdeen research demonstrate that improving S&OP practices drives gains in key performance areas across the value chain—in sales and marketing, distribution, manufacturing, and procurement. There is an explicit and clear relationship between the quality of an S&OP program and actual business performance (Exhibit 2). Enterprises that leverage S&OP best practices significantly outperform companies that are S&OP laggards. These findings hold true across companies of all sizes and industries.

Most enterprises today have at least some S&OP process in place to align demand and supply. However, there is a growing gap in performance between those that are focusing on a holistic S&OP strategy and those that operate S&OP tactically. Tactical S&OP, for instance, merely brings all stakeholders together once per month to agree on how the business plan will be implemented by operations. Holistic S&OP, on the other hand, introduces tools such as scenario-based modeling into the process so that rapid assessment of daily opportunities can be made. Holistic S&OP means the company’s local decisions can be measured against the success of business-unit and corporate goals.

S&OP can benefit all companies regardless of their size or S&OP maturity. For companies at all stages, S&OP programs have generated significant positive improvements in complete order fill rate, gross margins, and customer retention, as shown in Exhibit 3. The analysis further indicates that S&OP programs have a critical impact on profitability, help reduce inventory, and are crucial to enabling an effective and diversified product portfolio.

An effective S&OP program can substantially improve a company’s ability to plan and take orders for customized products and services—at the same time allowing the organization to focus on the most profitable and strategic customers and products.

The Effect of Globalization
One trend having a profound impact on the development of more advanced S&OP practices is globalization. In the 1980s through the ‘90s, much attention was given to the Japanese supply chain success stories. At that time, many of these companies maintained a local supplier base, which led to short lead times and enhanced flexibility to demand changes. Even some U.S. companies, such as Dell, became famous for their demand-driven supply chains where suppliers were local, often just across the street.

With the advent of global supply chains—where links of the chain are moved to where capital is best deployed and resources are best found—the situation has changed dramatically. Now, even the Japanese and Korean companies are engaged in offshoring.

The globalization of the supply chain brings rewards but also new challenges and risks. In turn, it further shapes the need for and approach to sales and operations planning. Some of the challenges of globalization that can affect S&OP include:

• Longer supply chain lead times.
• Increased reliance on security and risk management programs.
• Less control over data quality.

The extended supply chain means that in many cases logistics lead times are longer than production lead times. In the past, the focus of S&OP might have been on in-house or localized production capabilities and capacities. More and more, however, the focus has begun shifting to the capabilities and capacities of distant logistics and supply chain partners.

Increasingly, strategic planning depends on outsourced and third-party providers. As a result, management begins to see their influence decrease, which introduces a heightened need for risk management, scenario modeling, and execution strategies. All of these factors enable companies to cope with an environment of increased variance not only in demand forecasting but also in capability management.

As lead times become longer and more global partners must be considered in the S&OP process, the quality and predictability of data is also at risk. Therefore, the S&OP process must be flexible, focusing improvement efforts on gathering critical constraint information rather than detailed data from every node in the supply chain. As a result, the demand management component of S&OP shifts away from “one-number” forecasts for each and every SKU. Instead it focuses more on analysis of pessimistic and optimistic scenarios, with the executive team giving guidance on what products to push.

Best Candidates for S&OP
Companies that are prime candidates for a significant S&OP upgrade will share similar characteristics and face many of the same challenges. For example, one of the strongest indicators that your S&OP is outdated is failure to achieve key business metrics for targeted markets, channels, or product families. Also, if your organization lacks the agility to respond to new business opportunities or threats in real time (between planning cycles), you need to rethink your S&OP.

In general, the following are some early warning signs of an ineffective S&OP process that is in need of major change, replacement, or technology support. If you are trending negative on three or more of these for one or two periods, you should consider taking major action soon.

• Forecasts are missed for some critical product families.
• Earnings projections are missed.
• Inventories are increasing.
• Backlogs are increasing.
• Expedite charges are increasing.
• Agreed-upon alignment of supply and demand is not being sustained across the entire organization.
• Inventories are not aligned with planned deployments.
• The planning process takes too long.

Another sign that you are in need of S&OP emergency care: Operational plans are not tied to business plans. Many companies can say they have alignment only once per month, or worse. This is unacceptable and is definitely not best practice.

Implementing a Successful S&OP Strategy
The more successful enterprises focus their S&OP on profitability and continuous improvement. Their S&OP initiatives empower decision making at all levels and are supported closely by an integrated business information architecture. A successful sales and operations planning program will include the following five components: people, process, technology, strategy, and performance.

1. People
First and foremost, it is essential to obtain executive-level sponsorship. S&OP strategies fail without top-down support for the plan. Next, it’s important to create cross-functional teams that consist of sales, operations, finance, product development, and R&D. This step will eliminate the organizational silos and promote shared communication and collaboration across the enterprise. You’ll also gain better visibility into the pain points and success factors within every department.

Some additional S&OP conditions that will affect the people in the organization include:

• Only operational metrics approved by the S&OP team should be used, and all parties must be trained on the actions to take as business intelligence is delivered
• Management must establish guidelines for real-time response; for instance, what types of business events will trigger what possible response scenarios?
• All departments and business units must follow the formal S&OP system.

Finally, don’t forget to collaborate with your business network. It’s critical that the S&OP process leverage the capabilities and influence of customers and suppliers to expand the scope of potential improvements. This is particularly important when a company needs to exert its influence several “links” deep in their supply chain.

2. Process
Effective S&OP involves more than just holding formal monthly and quarterly meetings—although those meetings are extremely important. It is also about having real-time supply and demand visibility and making sure that business intelligence is continuously monitored at a strategic level, made relevant at the operational/actionable levels, and kept in alignment with the business goals. Take, for example, a new product introduction that is driving loads to exceed capacity for a critical, constrained resource, such as a plant or key raw material. Based on that information, the company must put into effect preplanned actions to cut back on other production (and possibly other sales) to optimize net profits and product-introduction goals.

To ensure this alignment, the formal meetings should begin with the CEO (or general manager) setting the tone for the meeting by bringing up any special, driving themes or situations that require priority attention. This way, all other aspects of the business can be subordinated to satisfying the stated needs.

All key functional areas of the business must be represented in the S&OP process, and a logical progression through the business must be made, including:

• Reviewing consolidated demand for all product families.
• Achieving consensus on the demand-side of the business.
• Testing the effect of plans on key constraints.
• Making adjustments to ensure optimal profit and achievement of strategic goals.
• Gauging the effects of new product introductions.
• Reviewing other special projects.
• Documenting all decisions and actions to date since the last meeting.
• Discussing possible process improvement.

It is very important to ensure alignment of performance metrics. Managing plants only on efficiency and keeping score at the corporate level based only on profitability is certain to doom S&OP improvement to failure!

Once you’ve got your metrics aligned, deploy contingency plans based on multiple “what if” scenarios to determine the risks and opportunities. This helps to provide clarity on how to respond when the actual demand and/or supply is different from the plan. Further, this approach empowers people to anticipate any potential action or outcome. It also helps to pre-empt the competition and focus on the areas where the greatest risk currently lies in achieving business objectives. When creating “what ifs,” be sure to establish benchmarks that enable you to study outcomes and modify future plans accordingly.

Finally, create a documented plan. The plan should drive execution and continuously challenge base assumptions, processes, and technologies including benchmarking against other best-in-class enterprises.

3. Technology
Technology upgrades and advancement are also necessary parts of success. Incredibly, many otherwise excellent companies are hamstrung when it comes to technology.

For example, we see many companies relying on spreadsheets for S&OP. The result: siloed, inaccurate data; nonrepeatable outputs and outcomes from period to period; and an inability to scale up or down as the business changes. Further, we see results that don’t provide a comprehensive view across all areas including procurement, manufacturing, sales, operations, marketing, and finance.

We’ve also noticed that a growing number of companies are finding that their current business processes and supporting systems for enterprise resource planning, supply chain management, planning, and budgeting don’t deliver what they need to keep ahead of the competition and customer demand. It’s important to utilize technology enablers by developing and implementing a strategy that leverages transaction, decision-support, and business-intelligence capabilities in a real-time environment.

4. Strategy
Through the course of the Aberdeen research, three strategic elements continually outshone the others as delivering the most business benefits and the best capabilities for mitigating risks. The first element is the formal alignment of supply and inventories to demand. This effort is supported by a planning process that spans the entire enterprise and crosses all functions.

Start by focusing on the greatest risk to achieving your business objectives. In nearly 90 percent of the cases, this involves demand management. Improving demand management has an incredible way of resolving supply problems such as low customer fill rates and excessive inventories. Noted author Richard C. Ling has pointed out that the areas for greatest potential change center on demand and new product introduction. Yet historically, companies have focused most of their planning systems initiatives on the supply and financial side.

The second strategic element is a focus on profit. We know that S&OP has moved beyond the practice of merely balancing supply and demand volume. Today, it’s important to measure all planning scenarios based on their profitability impact. Cherished assumptions regarding which products are most profitable must constantly be challenged and changed if necessary.

The third essential element involves paying attention to the value chain. Working collaboratively with customers and suppliers is one of the main ingredients of a successful S&OP program. Listen to your customers and include your business network partners in the design of the process and success metrics. By leveraging the capabilities and influence of customers and suppliers, you’ll be able to expand the scope of potential improvements.

It is impossible to try to compete in all customer, product, channel, and market segments. The key is to dominate the most profitable markets with the most profitable products. This requires the ability to define different segments, forecast revenue and profit potential, prioritize segments by desirability, and dominate the most attractive markets for you.

Finally, when it comes to strategy, you must emphasize continuous improvement. Keep the roadmap dynamic—you cannot effectively deploy plans too far ahead because you can’t predict the risks and market changes. It’s necessary to continually reassess the S&OP process and progress against actual results. It’s more important to develop guidelines to follow for making decisions during the period (such as what to do when product mix changes dramatically) rather than to rely strictly on the forecast for each product (which is probably wrong).

ADVERTISEMENT

<script language="Javascript1.1" src="http://ad.doubleclick.net/adj/scm.rbi/;abr=!webtv;desc=CA6327049;lay=article;type=boombox;sz=336x280;u=1085321165%7C615100179%7C724440032%7C0%7Carticle%7CCA6327049%7C%7C;ord=1149062866?" type="text/javascript"> </script><script type="text/javascript"> <!-- if ((!document.images && navigator.userAgent.indexOf('Mozilla/2.') >= 0) || (navigator.userAgent.indexOf("WebTV") >= 0)) { document.write('<a href="http://ad.doubleclick.net/jump/scm.rbi/;abr=!webtv;desc=CA6327049;lay=article;type=boombox;sz=336x280;u=1085321165|615100179|724440032|0|article|CA6327049||;ord=1149062866?">'); document.write('<img src="http://ad.doubleclick.net/ad/scm.rbi/;abr=!webtv;desc=CA6327049;lay=article;type=boombox;sz=336x280;u=1085321165|615100179|724440032|0|article|CA6327049||;ord=1149062866?" border="0" width="336" height="280"></a>'); } //--> </script><noscript></noscript>

5. Performance
You cannot improve what you cannot measure. Even companies with very informal S&OP practices must measure performance. Historically, metrics have been specific to a single function (such as sales forecasting accuracy) and involve volumetric types of measurement (such as actual vs. planned sales volumes). The drive for competitive advantage, however, has spurred a rethinking of what metrics should be used to determine the success of the S&OP program. The emerging best metrics, such as gross margin, encompass the two-way impact of demand and supply decisions, rather than having separate and unrelated metrics for each.

Business intelligence systems today can give decision makers an accurate, up-to-the-minute picture based on pre-defined key performance indicators (KPIs). These KPIs are related to value chain processes, product and customer profitability, order fill rates, customer satisfaction or retention, sales per employee, percent volume growth, and gross margins. The metrics can all be delivered through highly flexible role-based portals and executive S&OP dashboards.

For a S&OP program to succeed over the long term, companies must consider how performance measurement itself must change. This means putting new metrics into place as business conditions change as well as increasing the frequency of reporting and analysis.

Going forward, the S&OP system will feel growing pressure to manage consensus across key stakeholders. Exception-based workflows are not enough when external partners are key to the success of the program. Instead, the system must support proactive decision rules (such as permissible buffer levels) to enable the performance levels needed to achieve strategic goals.

S&OP Best Practices
The Aberdeen study discovered that companies utilizing S&OP best practices share a common set of approaches, namely: (1) reliance on a phased approach; (2) development of an “outside-in” sequence of S&OP initiatives; and (3) a focus on critical information, not just more data.

Rely on a Phased Approach
S&OP is much more an integrated set of business processes and technologies than a single, all-encompassing process or technology. If you just focus on the implementation of a new technology and think that S&OP will miraculously take shape, you’re wrong.

Begin the S&OP plan by creating specific phases with defined business objectives and identify the metrics to be improved. These phases should be designed to generate measurable business benefits in three to four months. With a longer time horizon, you may be late in detecting fatal flaws in the initiative, thereby losing critical time and value before the issues could be identified and, hopefully, resolved.

Develop an “Outside-In” Sequence of S&OP Initiatives
Typically, the events that will have the most profound and negative impact on your sales and operations planning are those outside of your control. For the most part, these are due to the decisions and actions of your customers, partners, and competitors, which have a direct impact on your revenue and your competitor’s strategy.

Again, your biggest return will likely come from improving how you manage and respond to the demand component. This is true because no amount of added flexibility and adaptiveness invested in the supply chain can compensate for an ineffective demand management process.

Focus on More Information, Less Data
Another key to successful S&OP is clean, current, and accurate data. The Aberdeen research indicates that often times, less than 10-15 percent of all available data has a critical impact on the business objective. Plans are often slowed down by the effort of gathering data that has minimal importance to the overall project. It is important to ensure that you know exactly what business problem you are trying to resolve and understand the minimum data necessary for the project. The 80-20 rule is very much alive when it comes to S&OP!

No Standing Still
Clearly, there is a strong link between sales and operations planning and dramatic improvements in a company’s growth, profitability, and customer satisfaction. The market is placing increased pressure on companies to align sales with operations. S&OP has never been more strategic, difficult, or crucial than it is today during a time of lean manufacturing, shortened production schedules, demand for customized offerings, global sourcing, and increased market competitiveness.

We cannot just stand still with supply and demand matching. We must push harder to use the S&OP process as a “capability improvement” tool.

The Aberdeen study demonstrated that all companies can benefit from enhancing their S&OP capabilities, regardless of size and S&OP maturity. All winning approaches are based on a tight integration of business processes—across the entire organization and value chain. With integration in place, companies can move down the path of proactive rather than reactive sales and operations planning.

An S&OP Success Story In the mid 1990s, one of the largest telecommunications and data services providers in the United States realized that its business would benefit significantly from improved collaboration between suppliers and internal operations. The company was challenged with balancing supply and demand because of the extremely short life cycle of its cell phone products. Additionally, order lead times for handset accessories were as long as 16-18 weeks, making it difficult to achieve forecast accuracy and supply chain responsiveness. The repercussions of this environment were significant. Limited product availability meant that the company lost customers to competitors with little hope of recapturing them in the future. Excess inventory produced lower margins as incentives were used to sell product or disposal costs were accrued for obsolete product. The company recognized that the challenging nature of phone supply, short product life cycles, and the competitive market pressures made it critical to collaborate better with suppliers and internal stakeholders to improve forecast accuracy while maintaining supplier fill rates. A Business Transformation What the company needed was a way to understand its demand drivers and supply risk so that it could make sourcing decisions that ensured product availability while minimizing inventory. Good decision making was especially critical at the end of the product’s life cycle. Inventory varied significantly because constant changes in pricing and promotions made demand unpredictable. However, rather than allow the market to dictate demand, the company decided it would adjust its promotion strategies to increase consumer interest in products at the end of life that were slow to sell or that were in short supply. The company implemented a multi-step process to initiate and drive S&OP across the organization. A formal, monthly S&OP meeting was conducted with the stakeholders to review demand patterns and supply risks and to establish the supply plan for the next quarter. Participants of these regular meetings included executive staff and key stakeholders from operations, marketing, and finance. The S&OP team meetings were organized around a “stoplight” system (green=okay, yellow=warning, red=problem), where all products were reviewed to identify the opportunities, risks, and areas that need the most attention. Once the supply chain plan had been developed during the S&OP process, the marketing organization committed to making sure that pricing and promotion programs consistently supported the supply plan. Between monthly S&OP meetings, a portion of the S&OP team began to hold weekly meetings to analyze the current sales pace. The goal of these meetings was to determine what was working and what was not and to project performance eight weeks into the future. A second meeting involved suppliers in order to complete weekly collaborative planning, forecasting, and replenishment. A critical success factor for this company’s S&OP process was uniting the key stakeholders. For this reason, suppliers were engaged in the formal process. This resulted in improvements in the supply base in the areas of cost, product availability, and capability. Technology Enablers As part of the S&OP process, the telecommunications leader created unconstrained demand plans using a demand planning software. This functionality enabled marketing to manage changes to the current product forecast, the introduction of new products, and the retirement of selected current products. Then a constrained plan was created and utilized, acknowledging supplier shortages and taking into account the need to drive unfulfilled demand to other suppliers. The company used price and promotion changes to manipulate demand. Finally an advanced supply chain planning solution was leveraged to create planned orders, drop requisitions, and plan the final packaging. Relying on the demand planning software, the telecommunications company was able to sustain S&OP best practices in a high-volume, data-intensive environment. The solution served as an S&OP “demand hub” data repository, which alleviated the burden of data collection. All relevant sources of critical information were consolidated, enabling data transfer and validation. As a result of implementing S&OP, the telecommunications company realized substantial benefits. First, the company demonstrated the ability to triple the number of products offered. Through the course of an eight-year period, forecast accuracy was improved by 36 percent and inventory-related working capital was reduced by 25 percent. Finally, store service levels increased to 98 percent or better. Authors’ note: This case study is based on material appearing in the Aberdeen Group’s Sales & Operations Planning Benchmark, 2004.