In System Center 2012 SP1 a new feature is introduced called Chargeback. In a series of blog posts we will explain how this can be implemented in the Datacenter using System Center 2012 SP1
Blog Post 1: Discuss the background behind Chargeback and how System Center makes use of this. (This Blog post)
Why chargeback is important, what are the different granularities of chargeback
Blog Post 2: Show how Virtual Machine Manager and Operations Manager should be configured to make Chargeback work.
Step by step guidance that explains how to configure the tools around Service Manager for Chargeback to work
Blog Post 3: How should Service Manager be configured to make Chargeback work and generate reports on the data.
Walk through the different options that can be used for chargeback and how reporting can be made available based on the defined price sheets that have been created in Service Manager
Background
Chargeback is something more and more companies are starting to look at, as they are getting more structured around how they deliver IT-services to the business. This enables IT to show and or bill the department that is consuming it-services from them. This can help to educate that IT is not an unlimited resource and having the ability to show and bill for delivered services will in most cases make the consumers think about how they spend these resources and use their resources effectively. If you have VM Server sprawl, or your cloud consistently expands, but never contracts or you never delete VMs, your chargeback costs are not providing enough incentive for consumers to measure their behavior.
The Theory
Yes I know it’s boring, but I just want to spend a little space explaining some of the background behind Chargeback and some of the things to keep in mind when thinking about implementing Chargeback / Show back into the Datacenter. In ITIL Chargeback is a discipline under “Financial Management for It Services” described as part of Service Strategy. The objectives are among some designed to establish an understanding between the relationship of expenses and income and ensuring that the two are balanced. In order to drive chargeback some equations needs to make up the charging model, this is normally referred to as a cost model. A cost model provides a structured way that identifies where money is being spent, once this is understood it becomes easier to find out what should be charged back to the consumer.
When defining cost models there are several things to take into account. Among these are:
- Identify Opex: What are my ongoing cost
- Identify Capex: What are my capital asset
Once these are identified it should be broken into the following:
- Direct cost
- Indirect cost
It’s not easy to identify costs and break them down into cost models, and this work should not be underestimated.
Two things to keep in mind when making these cost models is that it should be:
- Fair: One party or individual should not pay for others
- Transparent: It should be clear to why the cost is applied to a cost model. The bill should be easy to read and understandable by the person who receives the bill. Like in the old days we charged by MIPS, can you quantify the value per MIP? No that is way too difficult to understand what you are being charged for.
Chargeback can be as complicated as you want to make it, it’s a journey so taking a practical approach in the beginning could be a good idea.
Charging for the lamp in the server room might not be realistic, so it’s about finding a balance of what the Consumer will pay for, realistically.
Chargeback vs. Show back
Many companies take a two step approach as they are simply not ready to chargeback departments for usage. The alternative here is called Show back, where you are showing who is using what, but without applying a cost to the service. In some cases it might be appropriate to create a “Wall of shame” to help drive more cost effective use of resources. This can in some cases help to drive a fiscally responsible behavior from the consumers of the service. Once this is working well, the next step can be to start charging for the service.
System Center Integration is the key
One of the very strong features within System Center is the integration between the components of the suite. Having information of updates to important CI flowing automatically into the CMDB is a key driver for keeping the CMDB up to date with the latest changes, so these can be reflected in the charging model.
The flow of Chargeback information through the components is shown in the illustration below
Virtual Machine Manager:
Virtual Machine Manager is the place in the System Center Suite where we manage resources in the fabric and delegate these resources in a controlled manner to private clouds.
By defining private clouds as well as delegated access to these, VMM can control whom uses what VMs and at the same time use these clouds for Chargeback.
Private Clouds in Virtual Machine Manager looks like this:
Operations Manager:
Operations Manager has the capability of monitoring the other components in the System Center suite and Virtual Machine Manager is no exception.
Integration between Virtual Machine Manager and Operations Manager is made via connectors, which can be configured in Virtual Machine Manager. This will allow Virtual Machine Manager and Operations Manager to exchange information between them when changes happens in either component.
Operations Manager will monitor resources in VMM that has a capacity of some sort. eg. Virtual Machines, Hosts, Private Clouds. As Operations Manager uses an object oriented model to monitor resources, it will automatically create an object for each new resource created in VMM and begin to monitor it.
Below is an example of monitored Private Clouds and an example of one object of a Private Clouds and it’s attributes.
If a Private Cloud is running low on capacity, Operations Manger will notify the operator, so remediation can be done in time.
Service Manager:
Service Manager is the central component for holding information about Private Clouds. This is done in the CMDB which is a key component of Service Manager.
In order to feed and keep information in the CMDB up to date, Service Manager uses connectors to the other System Center components and Active Directory.
Operations Manager holds information about the private clouds and which VM that are part of which Private clouds, as shown above. By connecting Service Manager and Operations Manager via a connector we can now import this information into Service Manager CMDB.
Service Manager also has a connector to Virtual Machine Manager. This connector will however not import VMs, Private Clouds and hosts, but objects that are not monitored by Operations Manager. Eg. VM Templates, Host Profiles, Logical Networks and similar object types.
Once the needed information is in the CMDB, we can now start to define our price sheets in Service Manager.
A price sheet is an object class that allows you to associate a set of prices for various attributes for a VM and associate these to Private clouds.
All VM belonging to a private cloud will then have the price for the defined attributes applied to it and based on these a total price for this VM will then be calculated over time.
Example of a Price Sheet:
Service Manager Data warehouse:
Service Manager Data warehouse is the component in the System Center Suite that is used for reporting.
Service Manager alone cannot provide the information on the price that given tenant or user should pay over a month. This information is stored in the Service Manager Data Warehouse.
The Data Warehouse uses RTL (Report Transfer Load) between Service Manager and Data Warehouse to get reporting data into the Data warehouse. BI Cubes are then created to make it easier for the Reporting Analyst to generate reports based on the data in the cubes.
This allows the Reporting Analyst to generate Reports in three ways.
- Use Reporting Services
- Use Power Pivot in Excel
- Use SharePoint Performance Point
Excel Example:
Performance Point Example:
Summery Overview
Hopefully this gives a high level introduction to how Chargeback can be used in System Center. In this version the main focus is on VMs and defining a charging model for Private Clouds. Acknowledging that Chargeback can be more than VMs, SP1 has implemented the first step to start looking into Charging back for a service. We have partner solutions that can help taking a broader look at charging for other services than VMs.
For a quick introduction video on Chargeback by Bill Anderson please have a look here
In the next blog post we will take a look at how Virtual Machine Manager and Operations Manager should be configured to make Chargeback work in Service Manager.
In the previous blog post we talked about how Chargeback is working overall and some of the areas to consider when implementing Chargeback.
In this blog post we will walk through the configuration that needs to be done in Virtual Machine Manager and in Operations Manager and how you can verify that these are working as expected. This blog post is based on that the reader has no previous experience of Virtual Machine Manager or Operations Manager and therefore explains every step on the way to configure these components.
Knowing that Chargeback is a new feature in Service Manager 2012 SP1 to configure it correctly there are a list of things that must be done as pre-requisites in other System Center components for Chargeback to work. These things must be configured before starting on this guide.
- Virtual Machine Manager Server and Console is installed and managing at least one Hyper-V Server
- Operations Manager Console and Agent is installed on the Virtual Machine Manager Server
- Service Manager Server and Service Manager Data Warehouse servers are installed and connected
Configuring Virtual Machine Manager (VMM) for Chargeback
In Virtual Machine Manager the following configuration need to take place:
- Create one or more Clouds
- Provision or assign VMs to the Clouds
- Configure integration between Virtual Machine Manager and Operations Manager
- Verify that the right information is exchanged between Operations Manager and Virtual Machine Manager.
Create one or more Clouds in Virtual Machine Manager
The way Chargeback works in Service Manager is by grouping the VMs in Clouds. This means that all VMs in a Cloud will have the Price Sheet associated to them.
The way to create a cloud in VMM is the following
1. Open the VMM Console as a user that is Administrator in VMM.
2. Select the “VMs and Services” Wunderbar
3. In the Ribbon bar select Create Clouds
4. Give the Cloud a Name and click Next
5. In the Host groups, select the Host Group that should host the VMs for the Cloud.
TIP: To make Chargeback easier it’s recommended that Servers of the same or similar configuration inside the Host group is selected as this would make it easier to apply similar cost of the servers based on the CPU, Memory and other, so the servers have similar configuration.
5. Click Next
6. Select one or more Networks that the VMs should be able to connect to, click next
Note: In this example, we are not making use of Load Balancers, VIP Templates, Port Classifications, Storage and Library so no configuration was done here.
The Capacity of the Cloud is the one area where it can be defined how much resource a Cloud can consume for the Fabric.
Setting limits for a cloud can be a good way to avoid VM sprawling and control that resources are not over consumed. This can also ensure that chargeback does not go out of control.
The following limits can be set for a cloud
7. Set your limits and click next
8. Select Hyper-V and click next and finish creating the cloud.
We have now created a cloud that we can use for Chargeback.
More clouds can be made is needed, the view should now have been updated to show the new cloud. In this example, we created two clouds called Marketing Cloud and Finance Cloud
Next step is to make sure we have provisioned or assigned VMs to the new cloud so we can get some data out for Chargeback.
Provision or assign VMs to the Clouds
A VM can be added in two ways to a cloud.
- When it’s provisioned to a cloud
- When the VM is assigned to a cloud
In order to Provision a VM to a cloud, select create Virtual Machine, select a template and configure settings for the VM.
1. Once this is done, select Deploy the Virtual Machine to a private cloud
2. Select the new Private Cloud that was just created and provision the VM
The alternative is to assign an already running VM to the Cloud
1. Right click on a VM and select properties
2. Under the General Menu select Cloud from drop down and select the cloud that the VM should be assigned to.
3. Click OK
4. To view the VMs that are assigned to a cloud select the Cloud in the Cloud views and the list of VMs will show.
Now we have created a Cloud and assigned VMs to it. Next step is to configure VMM and Operations Manager to exchange this information.
Configure integration between Virtual Machine Manager and Operations Manager
For Operations Manager and Virtual Machine Manager to integrate following are the main items to configure:
- VMM and dependent Management Packs needs to be imported into Operations Manager
- OM and VMM system accounts needs to be added as Admins in VMM and OM.
- OM settings need to be configured in VMM
VMM and dependent Management Packs needs to be imported into Operations Manager
For Operations Manager to monitor and discover clouds, hosts and VMs, the VMM Management pack needs to be imported into Operations Manager. The VMM Management has a list of management packs that it depends on such as SQL, IIS and Windows Server. In order to make this easier a PowerShell script is available to import these Management Packs. These Management packs can be found on the Service Manager Server under the install dir \Chargeback (Eg.
\Program Files\Microsoft System Center 2012\Service Manager\Chargeback).
1. Copy the dependencies folder form Service Manager server to Operations Manager temp folder
2. Start an Operations Manager PowerShell prompt and go to the dependencies directory copied earlier
3. Run ./importToOm.ps1
4. Verify that script runs without errors
To add VMM and OM Service Account to VM and OM admins do the following
To make sure OM and VMM can exchange information between the two systems they need admin access between them . To verify that the service accounts has the right access do the following.
1. Open Services Console on VMM and OM servers
2. On the VMM Server verify the user for System Center Virtual Machine Manager is running under domain account (eg. CONTOSO\!vmm)
3. On the OM Server verify the user for System Center Data Access Service is running under domain account (eg. CONTOSO\!om_das)
4. If OM Administrators are managed through AD group then add the VMM Service Account to the OM Admin Security Group using Active Directory Users and Computers console Eg. Add CONTOSO\!VMM user to the OM Admins Security Group
Similarly, if VMM Administrators are managed through AD group then add the VMM Service Account to the OM Admin Security Group
OM Settings need to be configured in VMM
1. Open VMM Console
2. Select Settings > System Center Settings
3. Give the FQDN of the OM Server
4. Use default Service Account suggested, Click Next
5. Specify the Service Account for Operations Manager added to the VMM Security Group earlier, click Next
6. Click Finish to enable the integration.
To verify that the integration between Operations Manager and Virtual Machine Manager is working perform the following items
1. Open VMM Console as an administrator
2. Select Settings > System Center Settings
3. Double click on Operations Manager Server
4. Verify that connection Status is OK
5. Select Management Pack and verify that the list of MP is Version 3.1.6011.0 or higher.
6. Open Operations Manager console as Administrator
7. Select Monitoring > Monitoring > Virtual Machine Manager Views
8. Wait for the Diagram to show and extend Managed Resources > All Clouds
9. Verify that the list of Clouds that are showing in VMM is represented in the diagram like below:
To see information about each cloud in a state view the following steps can be done.
1. Select Administration > Management Packs
2. Select Create Management Pack in the Action Menu
3. Name it Private Clouds and Click Next and click Create
4. Select “Monitoring” Wunderbar
5. In the monitoring view right click on Monitoring (Top level) and select > New > Folder
6. Name the folder Private Clouds and Select Private Clouds in the Management Pack drop down and Click OK
7. Right Click on the Private Clouds and select New > State View
8. Use the following Data:
Name: Private Clouds
Class: Private Cloud
9. Click Ok
10. Verify that a list of Clouds shows up in the new view with Data about each cloud
This view now shows that Operations Manager is now monitoring the Private Clouds.
Now that the VMM clouds are monitored objects in the Operations Manager DB, we have now successfully integrated Operations Manager and Virtual Machine Manger, so they can exchange data between the two components.
As mentioned in the beginning of the blog, the goal of this blog post was to show how Virtual Machine Manager and Operations Manager should be configured in order to get Chargeback working probably, without having any expert knowledge about these two tools. In the next blog we will go through how Service Manager should be configured and how Chargeback price sheets and Reporting based on these can take place using Service Manager Data warehouse.
Happy Chargeback configuring
In this post, I will focus on service manager and take a deeper look at what needs to be configured to make chargeback work.
As noted in the previous posts, there are a few elements that need to be in place before you begin:
- Confirm that the steps outlined in the previous posts have been followed and cloud objects are appearing in Operations Manager State View.
- Confirm that Service Manager has been installed.
- Check that Service Manager Data Warehouse has been configured and registered with the Service Manager instance so that Data Warehouse and Reporting is showing in the Service Manager Console.
- Make sure that Excel 2010 or Excel 2013 is installed on the same computer as the Service Manager console.
Configuring Service Manager (SCSM) for Chargeback
Here are the primary steps I’ll cover in this post:
- Import the Chargeback Management Pack
- Configure the SCOM connector in Service Manager
- Create a view to see Clouds objects under Configuration Items
- Configure Price sheet for Chargeback
- Verify data has moved to data warehouse
- Use Microsoft tools to analyse data for chargeback
Import the Chargeback Management Pack
The Service Manager Chargeback Management Pack is not installed by default in Service Manager 2012 SP1, so this management pack (MP) needs to be imported for chargeback to work correctly.
To do this, follow these steps:
- Log on to the server that is running the Service Manager 2012 SP1 Server as a Service Manager Administrator.
- On Windows 2008 Server: Click Start > All Programs > Microsoft System Center 2012 > Service Manager > Service Manager Shell..
- On Windows 2012 Server: Click low left corner to go to “Start,” the type “Service Manager Shell,” then right click on “Service Manager Shell,” and select “Run as Administrator” in the bottom of the screen.
TIP: Remember to enable PowerShell to execute scripts: Set-ExecutionPolicy –force RemoteSigned
- The path in the PowerShell window should show
\Service Manager\, type cd chargeback - Type .\ImportToSM.ps1
- Verify that the script runs without errors (You might get a message that the Management Packs are already imported, and this is Ok).
- Type “Exit” to leave the PowerShell window
Configure the SCOM connector in Service Manager
In the previous blog post we configured the integration between Virtual Machine Manager and Operations Manager. To get the Cloud objects we were seeing in Operations Manager into the CMDB in Service Manager, the Operations Manager connector in Service Manager needs to be configured.
To do this, follow these steps:
- Open the Service Manager console as an Administrator.
- Select “Administration” in the Wunderbar and expand “Administration and Connectors.”
- In the action menu, select: Create Connector > Operations Manager CI Connector
TIP: If you have already configured the Operations Manager CI Connector, then select the connector in the central view, select “Properties,” select “Refresh” under Management Packs, and verify that all VMM MP are selected.
- Click “Next” to move to the “Before you begin” page.
- Give the connector a name, e.g. SCOM OM01 CI Connector, then click “Next.”
- Type the FQDN server name of the Operations Manager server
- Under credentials click “New.”
- In “Display Name” write a name for the account e.g. Operations Manager Run as Account.
- Under Accounts select “Windows Account.”
- Give a User Account and Password for an account in AD that has Operations Manager Admins Access.
- Select the domain for the user.
- Click “OK.”
- Click “Test Connection.”
- Give the Password for the user and click “OK.”
- A list of Management Packs will show, select all Management Packs, and verify that none are gray.
- Pay special attention to the Management Packs listed below and ensure they are selected and are not grey.
Tip: If any of the MP’s are gray and can’t be selected, make sure to check that the Management Packs in Operations Manager and Service Manager is the same version. In some cases the Operations Manager MP might be newer than the MP loaded into Service Manager. If this is the case, download the latest MP from the Operations Manager Catalog (located in OM Console > Under Administration > Management Packs > Management Packs) and select “Download to disk.” Then import MP into OM if it is newer than the one currently loaded. Next, copy this file to the Service Manager Server and import the Management pack (Administration > Management Pack > Import Management Pack) form the Service Manager Console.
- Click “Next.”
- Select a schedule for when the Connector should run and click “Next.”
- Verify the information in the Summery view and click “Create.”
- Select the Connector in the Connectors view and select “Synchronize Now” in the Action Menu.
- Wait 5 minutes and then select “Refresh” in the action menu and verify that the Status says “Finished Success” for the connector.
- Select “Data Warehouse” in the Wunderbar, and then select “Data Warehouse Jobs.”
- In the jobs view, select MPSyncJob and select “Resume” in the action view.
- Wait for the job to finish, and then verify that all MPs are associated in the MPSyncJob Detailed view. In particular, make sure the MP below is associated. This will ensure the right data can be moved to the Data Warehouse from Service Manager in order to enable reporting for chargeback.
Create view to see Clouds objects under Configuration Items
Just like we did in Operations Manager (where we successfully created a view to see all Private Clouds), we want to create a similar view under Configuration Management to see all Private Clouds in the CMDB.
To do this, follow these steps:
- Open the Service Manager Console as Administrator.
- In the Service Manager Console select “Configuration Items” in the wunderbar
- Right click on Configuration Items in the tree structure view, and select “Create folder.”
- In Folder Name type “Private Clouds.”
- Next to Management Pack click “New.”
- Type Chargeback and click “Ok.”
- Click “Ok” to create the folder
- Right click on the newly created folder and select “Create view.”
- Name the view “Private Cloud.”
- Under Advanced Search select the “Browse” button.
- In the drop down on the left select “All basic classes”
- In the filter type “Private cloud.”
- Select the class that has a cloud as an icon and click “Ok.”
- Under Display select the following values:
- Display Name
- Maximum Memory
- Maximum Storage
- Maximum vCPU Count
- Maximum VM Count
- Description
- Click “Ok” to create the view.
- Now select the new view under the Private Cloud folder.
- Verify that the same information shows as in Operations Manager State view.
Configure Price sheets for Chargeback
In the first chargeback blog post we talked about how cost models were a way to break down cost as a method for chargeback. In Service Manager, this can be defined as price sheets.
A price sheet can be applied to one or more clouds as a model for how a cost for a VM is calculated based on a defined cost model.
The Cost model for VM price sheet is divided into the following areas:
- VM Base Price per day
- Cloud Membership Price per day
- VM CPU Price per core/day
- VM Memory Price per GB/day
- Highly Available VM Price per day
- Static IP Price per day
- Expanding VHD Price per day
Based on these values, a cost for a VM can be calculated. With this established, let’s make a price sheet!
In this example, we will only create one price sheet for all the VMs in the Finance and Marketing Clouds. If it is required, a price sheet could be created for each cloud, and you can then apply different prices for VM’s in different clouds:
- Select “Administration” in the Wunderbar.
- Under “Administration” expand Chargeback > Infrastructure > Price Sheets > All Price Sheets.
- Select “Create Price Sheet” in the Action Menu.
- Give the Price sheet a name, e.g. Contoso Price Sheet.
- Select “Price” in the ribbon bar.
- Assign prices for the areas that has been defined on the cost model. An example model is shown below.
- Cloud VM per day: 5
- VM CPU per Day: 1
- VM Mem per Day: 1
- VM Storage per Day: 1
- Select “Publish” in the action menu.
- Select “Assigned Clouds” in the ribbon bar.
- Under “Assigned Clouds” click the Add button.
- Select one or more clouds (in this example we will select Finance and Marketing) and click “Add” to add them to selected objects. Then click “Ok.”
- Click “Ok” to create the price sheet.
Verify data has moved to the Data warehouse
For the Data Warehouse to enable reporting, it needs to get data from the Service Manager server at given intervals. The Service Manager Server has a list of jobs that will run automatically at a defined interval (Min, Hour or Daily). To speed up this process, the jobs can be run manually in a given order so that data will be transferred faster to the Data Warehouse.
To do this, follow these steps:
- In Service Manger console, select Data Warehouse > Select Data Warehouse jobs
- Resume the jobs in the following order (don’t start a job before the previous is ended)
- Extract:
- Extract_SM
- Extract_DW
- Transform:
- Transform.common
- Load:
- Load.Common
- Cube Processing:
- Process.SystemCenterServiceManagerChargeBackCube
- Once the data has been processed by the cube, reporting data is not ready to be analysed.
Use Microsoft tools to analyze Data for Chargeback
Service Manager Data Warehouse gives several way to analyze data for Chargeback. Among the options are:
- Excel Power pivot
- Sharepoint Perfromance Point
- SQL Reporting Services
These Microsoft features (Excel, SharePoint & SQL) provide different ways to present the Chargeback data to the users.
Now I’ll overview an example with Power Pivot, and identify how this can be exposed to users in the Service Manager console using built in features.
To do this, follow these steps:
- Open Service Manager Console.
- Select Data Warehouse > Cubes
- Select the Service Manager Chargeback Cube and click on “Analyze Cube in Excel.”
- Excel will open with a data connection to the Chargeback cube.
- Select the following fields to the left in the power pivot:
- Group: ServiceManagerInfraChargeback
- Fiscal Year Hierarchy
- Group: Service Manager Infra Daily
- Cloud Cost
- VM Cost
- VM Total Cost
- The Power Pivot should now look similar to this:
Note: The data shown may be limited because the chargeback configuration has only run for a limited time. Over time this report will have richer data to analyze.
- Save this file in a folder on the Service Manager Server. (e.g. c:\temp\Chargeback) and close Excel.
- Share this folder (c:\temp\Chargeback) that users of the Service Manager console has read access:
- Right click on the folder in File explorer “Share with > Specific People”
- Add a Domain Group that represents SM Console users
- Select Read Access and click “Ok”
- In the Service Manager Console, select Data Warehouse > Data Warehouse > Analysis Libraries
- Select “Add Library Folder”
- Name: Chargeback
- UNC Path: Type UNC Path to the share just created, e.g. \\SM01\Chargeback
- Click “Ok.”
- Go to Reporting > Analysis Library > Chargeback
- Select “Contoso Chargeback” file and click “Open in Excel” in the Action Menu
- The Excel file now opens in Excel
Hopefully this series of posts has provided a helpful overview for how chargeback for virtual machines can be done with the use of System Center components. Another goal of these posts has been to demonstrate how well the System Center products are integrated, and how nicely data flows between these components.
Please let us know if you have any feedback to this series of blog posts.
Stay tuned for more IT Service Management blog posts from the TED team!