ILMT / BigFix Inventory Best Practices & Lessons Learned

By Mark Feinman

The deployment of the IBM License Metric Tool (ILMT) or BigFix Inventory (BFI) is a mandatory requirement to take advantage of sub-capacity pricing. Most customers initially consider the implementation of ILMT as a burden that forces the I/T team to support a technology with yet another agent. Some companies think that the agent uses excessive CPU resources, and as a result, they do not run the agent at the recommended intervals. Some companies aren’t even aware that the agent can be scheduled to run during off hours. Another issue companies have is that the comparison of entitlements and actual usage isn’t even done in ILMT – Passport Advantage has to be checked for entitlements while ILMT reports on actual usage. Some companies think the Audit Snapshot is for IBM’s use only; they do not realize that anyone can use it to help with the process of bundling to establish a license position. Others feel that because ILMT did not link software components together properly, the tool must be “broken”; they don’t realize that the initial software classification that ILMT makes is a best guess based on available information.

The purpose of this paper is to hopefully ease some of these issues by identifying ILMT/BFI lessons learned and best practices obtained with experience deploying and working with these products in the past year. Whenever real-life cases are described, the identity of any clients worked with has been changed to fictitious names.

[In the text, BigFix Inventory is a superset of ILMT. Every function that ILMT contains is also available in BFI. The difference between the two products is ILMT is for IBM software only while BFI can work with other vendor’s software. Therefore, if ILMT is mentioned alone, BFI is also included.]

Measuring Savings

So why was ILMT/BFI deployed in the first place? Usually, there are three reasons:

  • IBM requires it to qualify for sub-capacity licensing.
  • Sub-capacity licensing results in substantial financial savings due to virtualization.
  • You’ll get an idea of what’s deployed in your environment, which helps determine your software license position.

The measure of savings is in “processor value units” … but that’s not a meaningful measurement. What does that mean exactly? That’s like saying that the Alpha Centauri galaxy is 4.367 light years from Earth (which it is) … you know it’s far but you don’t quite grasp how far. It’s best to translate the PVU measurement into currency, which is universally understood. Management might not understand PVUs but they do understand the value of the local currency. Your license agreement will tell you how much a PVU is for each product, so simply multiplying that value by the number of PVUs you have will come up with a value in your currency. If you do this for both your full capacity and subcapacity numbers, and determine the difference between them, you’ll see how much the use of ILMT/BFI is saving your company. It is this amount which should serve as your business case to obtain executive sponsorship for the tool’s deployment.

Executive Sponsorship

It’s just not enough to just have the tool installed … you need a software asset management process that works with that tool. That process needs buy-in at the executive level; if the executives haven’t bought into the process, then why should the rank and file care about it?

Executive sponsorship is truly needed for your SAM program to succeed. Management has to make a commitment to the success and longevity of the program, because there is real savings in understanding your software license position. Companies like IBM and Microsoft can audit you, and you’d better be prepared to justify your software usage. In addition, two dedicated teams should be set up: one to maintain the tool (the I/T team) and one to manage the data that the tool provides (the SAM team). The tool in this case doesn’t need to be ILMT or BFI; it could be any SAM tool. In the case of ILMT, the tool might be free, but using it is not. BFI is definitely not a free tool; you purchased it, so you should maximize its value!

Maximizing Tool Value: Skills Transfer

The best way I can think of to maximize the value of the SAM tool you chose is to learn how to use and maintain it. You will probably need to bring in an outside expert on that tool, but the expense will be worth it.

I mentioned that two teams need to be set up, the I/T (technical) team and the SAM team. What kind of knowledge would the technical team need to best support ILMT/BFI?

  • Troubleshooting TCP/IP Connectivity
  • Knowing how to use the ping, netstat, tracert and telnet commands to troubleshoot failed network connections
  • Relational Database (DB2 or SQL Server)
  • OS level skills: Windows and/or Linux
  • BigFix product skills as a master operator
  • ILMT product specific tasks, such as starting & stopping the VM Manager Tool
  • Forensics: Knowing where the logs are to find an issue. Software nowadays is complex and there are many moving parts, and as a result, there are usually several logs that need to be checked before an actual issue is found.
Give the technical team the BigFix training they need.

What kind of knowledge would the SAM team need?

  • CSAM (Certified Software Asset Manager) or similar experience with respect to:
  • Licensing skills. Have a good understanding of how IBM licenses software with an emphasis on PVU and RVU licensing. These are the most expensive and most pervasive products installed.
  • Passport Advantage navigation skills. Know how to run Entitlement reports so that they can be compared to deployments which highlights gaps.
  • The SAM team processes the Confirmation, Reassignment, and Exclusion of the automated bundling rule associations.
  • If the SAM team doesn’t understand IBM Passport Advantage Licensing, get help from a consultant.

Oftentimes, knowledge picked up in “the lab” doesn’t quite match reality. So how do you build “realistic” skills? I have found when assisting customers with BigFix/ILMT implementations that conducting WebEx sessions on technical topics and troubleshooting issues works very well. Instructions would be provided, and when appropriate, an explanation of why certain actions were being taken. This goes a long, long way in building the “realistic” knowledge base. Using one of my clients (whom I’ll call the Smart Cookie Company) as an example, those smart cookies took snapshots of the installation panels during their ILMT deployment to serve as a record of chosen options at a particular time of the installation process. At any time during their WebEx calls, the smart cookies were able to ask questions about how something was set up, what the significance of a particular action might have been, or why a particular action was taken. During subsequent webex troubleshooting sessions, at the close of an issue, a summary of what the issue was, and what was done to correct it, was provided. This was all compiled into a support & maintenance document to which the support team could refer back. These documents helped form an internal ILMT knowledgebase which could be consulted down the road if issues occurred.

As for the SAM team, the same techniques can be used. The senior SAM staff should learn the tool from an external tool consultant (in one or more “train the trainers” sessions), and at the end of that engagement, the consultant should create a document that acts as a reference guide to what was covered over the previous sessions. This is not meant to replace product documentation; rather, the reference document complements it by summarizing company specific decisions that were made. When needed, your in-house trainers can run WebEx sessions with the new members of the team so they can learn how to use the tool.

For ILMT/BFI, here is a suggested list of topics:

  • What is subcapacity and why is it is important.
  • IBM License Metrics – Definitions, examples and how they apply to ILMT/BFI.
  • Overview of ILMT/BFI features and functions to include:
  • Navigation of ILMT web pages and an explanation of functionality.
  • Creating required IBM Audit Snapshot Reports.
  • Explain audit period and required frequency of running reports.
  • Explain purpose and interpretation of each report.

For ILMT & BFI specifically, use the reports created by the “Audit Snapshot” button on the ILMT “All Metrics” page as a reference document in preparation for and during the bundling tasks. The “Audit Snapshot” is not just for IBM’s use – you can use it as well to see the same data that you’d give IBM if they requested it. You could also use the spreadsheets (CSV files) that are embedded in the Audit Snapshot to perform your own “what-if” analysis using Excel. Some teams find that easier that using the ILMT web pages.

IBM requires the Audit Snapshot to be run at least quarterly, saved for two years, and be provided to IBM upon request.

Software Scan Best Practices

ILMT has several ways that it can be deployed, and if you choose to deploy it using Linux & DB2, you have an “all-in-one” deployment option, where you can install both the BigFix platform and the ILMT tool on a single server in an “express” fashion. The “all-in-one” install method is meant to be a shortcut to deploy BigFix when you are using it specifically for ILMT. (BigFix Inventory doesn’t have an “all-in-one” installer.) While the “all-in-one” option hides some of the complexity from you, and sets some things up automatically for you, it doesn’t consider company specific requirements that can be accounted for if the components were installed separately. For example, the default software scan schedule using the all-in-one installer is to scan every computer every 7 days, and the scan can run at any time during the day. However, this default does not consider maintenance windows or peak/off-peak hours for business. It is a best practice to schedule the software scans yourself, spreading the load over a weekly period (or longer) and running them at off-hours, staggering the start times. The all-in-one installer could not possibly know what your “off hours” are, though you could stop the default scans and then create your own that way.

It is also a best practice to create automatic computer groups in BigFix and base the scan off these groups. The all-in-one installer does not do that for you either. An automatic computer group is a definition of a group of computers that share some common properties, such as “all hostnames which start with ‘abc’”. The definition is left up to you. When the group is saved, all computers that meet that definition get automatically included in the group. Any computers that get added in the future which meet the definition of the group will also get automatically placed in the group. You won’t have to change your scan schedules; the new computer added to the group would simply obtain the schedule set up for that group.

Deployment Best Practices

A key best practice for an ILMT or BFI deployment is the placement of Relays in your environment. Simply put, a Relay handles communication between BigFix clients and the BigFix server, relieving the BigFix server from the responsibility of handling network connections. It allows the BigFix server to focus on the inclusion of computer properties in its database (such as the results of an ILMT software scan) without worrying about gathering them over the network. Generally speaking, the ratio of relays to computers should be about 1,000 to one, and relays are used to cross firewall boundaries. They are also used when computers are widely disbursed; it’s better to minimize traffic over your Wide Area Network (WAN) by have a local relay connect to the clients, and have one WAN connection to another relay on the other side of the WAN.

The deployment of BigFix really requires careful planning (“plan, plan and plan some more”). The all-in-one deployment option for ILMT raises the temptation of rushing to install because it’s “easy”. However, your I/T team must still plan for the placement of relays in your environment, unless your deployment is very small.

Bundling Tips

Software Classification or “Bundling” is the process of managing software instances that aims at optimizing pricing calculations and generating accurate IBM sub-capacity reports. It involves several actions:

  • Confirming: Accepting what ILMT has found
  • Reassigning: IBM product instances that should be counted under the sub-capacity license of another product rather than its own; for example: DB2 that is a component of WebSphere Application Server
  • Excluding: IBM product instances that can potentially be under sub-capacity licensing but are currently not used under such a license and therefore should not be counted

When the ILMT Data Import runs, the process makes a best guess on how software components are bundled or linked together. Most people don’t realize that this is a best guess – it’s not the final decision. As a result, some companies think ILMT is “broken” because it didn’t properly link, or bundle, DB2 with WebSphere on a particular computer. ILMT is not broken – it guessed wrong based on the information it had. The one way you can improve ILMT’s guess is to load your PVU-based part numbers from Passport Advantage into ILMT and then run the Data Import. ILMT will lean towards your uploaded part numbers when it has to make bundling decisions. If anything is still “wrong”, then it’s up to you – the software asset manager – to correct it by reassigning components to another product or excluding the component from pricing calculations because the server is an evaluation or disaster recovery server.

Bundling is time consuming and there are no shortcuts. To help focus:

  • Import part numbers only for your sub-capacity products
  • Hide free software instances and filter out non-sub-capacity products from the Software Classification panel.

The resulting list is sub-capacity products on which to focus.

Here is a recipe to perform bundling:

  • Determine your entitlements (PPA)
  • Determine what is installed on every server (ILMT)
  • Determine the use of the server (production, test, DR, evaluation, etc.). Evaluation software can usually be excluded from pricing calculations.
  • Look for packaging patterns on every server. For example: WebSphere, DB2 and MQ
  • Verify that the version of the software title applies to your software entitlement
  • Check “Supporting Programs” in the License Information Documents repository
  • Example: DB2 Enterprise Server V10 is a supporting program licensed with WebSphere Application Server (WAS) The version of WAS must be and the version of DB2 must be 10; anything else is not considered a supporting program, unless your own license agreement states otherwise.
  • Consult your license agreement(s)
  • Make changes on ILMT (confirm, reassign, exclude)
A Case Study

The Acme Cracker Company (slogan: “We’re Crackers!”) recently deployed ILMT. Let’s see what happened.

ILMT was deployed at the Acme Cracker Company some time ago (let’s say two years). It was the all-in-one deployment on Linux, and the team (or external consultant, no one knows for sure) that deployed it is long gone, along with their knowledge of the decisions made and why. The deployment is a single server with everything on it, and all of the clients (or at least those that were working) were set to connect directly to the BigFix server. There were no BigFix relays deployed. No scan schedules were set up; everything ran with whatever defaults were set by the all-in-one installer. The initial deployment was about 1,000 servers; two years later, there were about 3,500 servers being managed.

When Acme Crackers decided to address their ILMT deployment, the version of ILMT was quite old; it was barely version 9. Mysterious issues would occur every now and then, and it seemed almost impossible to get rid of the dreaded “No VM Manager Data” issue. Given the fact that there was no planning involved with the deployment, management decided that ILMT should be deployed all over again with a better thought out architecture that would utilize relays and adhere to best practices whenever possible.

Unfortunately, the rank & file at Acme Crackers cracked up at the thought of a planned architecture with several relays. They did not want to go through the work of setting up firewall rules from clients to the relays. They had existing firewall rules that they wanted to use. They didn’t want to support the additional relays even though all of that is done through the BigFix console. “Too much work”, they said. They decided to go with the all-in-one architecture again, this time with up-to-date versions of the software, and hopefully someone will record the installation decisions that were made. All clients would connect to the BigFix server directly, as before.

Although the company name was changed, this story is real. Learn from Acme Crackers mistakes:

  • The rule of thumb is to have one relay for every 1,000 servers in a BigFix infrastructure. With about 3,500 servers, this would mean 4 relays. This doesn’t consider isolated networks, where it is a best practice to have one connection from relay to relay cross a firewall boundary, rather than have “n” number of client connections cross the firewall to a relay on the other side. Allowing all clients to cross the firewall boundary is a security exposure!
  • The BigFix server’s main job is to take the client computer properties and load the data to the BigFix database. If you have a large number of computers trying to send data to the BigFix server while BigFix is trying to do its main job, that’s going to bog down the server because it’s going to try to do both. Now add the work of ILMT licensing calculations (the data import) to this, and one or more analysts performing bundling activities, which might require licensing recalculations. All of these activities are data intensive, so ultimately there would be response time issues. Ideally, the workload should be spread out across multiple servers and not have a single server do everything (unless you have a really small deployment). The use of relays offloads the task of communication management from the BigFix server, and these relays package the data from multiple clients together to make that exchange more efficient. It is the use of relays that enables BigFix to scale to hundreds of thousands of clients.
  • The all-in-one installation creates a faux schedule that runs the software scan every 7 days, but the time that the scan is launched can be any time during the day. The best practice for running these scans is to stagger them and run them at off-peak times. You do that by setting up your own schedule.
  • Acme Crackers did not have executive sponsorship for the initial deployment. It is unclear if they have it even now. Without ownership of the process, there isn’t incentive to work with the tool, unless the threat of an IBM audit is driving this deployment.

Perhaps Acme Crackers will realize that this second deployment is not all it’s cracked up to be and take a step back to examine their situation. Don’t be Acme Crackers! Applying these best practices to your ILMT/BFI deployment should make it a more manageable experience.

About the Author

Mark Feinman is the Senior Solutions Architect at Siwel Consulting Inc.