Choosing a Capacity Management Tool

What is capacity planning, and why do I need it? Capacity planning is the process of determining the production capacity needed by an organization to meet changing demands for its products. In the IT world, though, people often refer to capacity planning when they really mean capacity management. Capacity planning is an integral part of capacity management, but there is a lot more to it.

ITIL breaks capacity management into three sections: capacity management, service capacity management and component capacity management. The combination of all three results in businesses attempting to meet the demands of each group in a cost effective manner to achieve their goals. To optimize IT performance and efficiency it is important to make sure that you have a solid capacity management program in place, and that it includes performance monitoring, analysis, tuning, and capacity planning.

What is the right tool for me? There are two main types of solutions for capacity tools in the IT environment. The first is the modeling method, which lets you create situations, put them into a simulator, and observe the potential outcome.

The second is the forecasting method, which lets you look into the future and predict what your environment is going to look like in the days and weeks to come. This method is based on the depth of data available and the math behind the scenes. For many companies this may be the best option, because it ensures that you have the correct information to make accurate assumptions based on your current environment.

Consider the following questions when choosing a tool:

  1. Can this tool work on everything across my environment?
  2. Can it accurately perform forecasting and trending?
  3. Can I analyze workload data with this tool to identify over-allocated capacity and eliminate waste?
  4. Can this tool audit changes to my infrastructure?

Why Sightline? Now that we know what a capacity management solution should do and why, the new question to answer is, “Why Sightline?” Sightline combines all of these capabilities into one tool and by doing so provides the best solution to the performance data management and capacity management needs of any company.
Read more

Third Party Data in Sightline

Speaking of data, what about third-party data? Is there a way to pull third party-data into Sightline? And if so, how?  Yes! Sightline offers multiple methods of pulling third-party data into Sightline for display and analysis by EDM or EA/V. We offer an SDK, the Log File Interface Agent, and the configurable SNMP Interface Agent. The SDK allows you to write your own Interface Agent to get data into Sightline. The Log File IA can be used to read instrumentation data from custom applications and also standard logs such as the syslog and Apache HTTP server. Finally, SNMP has a mapping file that lets you pull data from any SNMP sub-agent. If you would like to look and any of these options, contact us and we’ll help you get it implemented at your site.

Firewall Settings for Sightline Communication

A frequently asked question for us here at Sightline is, what ports do I need for communication between the EDM and the Power Agent?  

Sightline uses TCP/IP to communicate between EDM and the Power Agents and firewall rules are often required to allow this communication. All port settings are configurable, but we’ve created a document to provide specific settings that will get you up and running quickly. Click here for details.

Using Workloads in Sightline

Workloads are a powerful tool in the Sightline software suite. But what exactly are workloads, and what can you do with them?

Workloads allow you to combine processes into a logical group for reporting and tracking resource utilization. Workloads are defined in the Power Agent’s configuration file, and are normally mutually exclusive. Workloads are evaluated when the data is collected by the Power Agent. One purpose for developing these groups is to provide a small set of workloads so that manageable metrics can be used to evaluate performance.
You may want to be able to answer questions such as:

  • How much CPU is a particular group of users using?
  • Do the order entry (or other) people have enough memory?
  • How much I/O is my database application really using?

In many cases, you’ll find that the 80/20 rule applies; that is, 80% of the work is done by 20% of the users of applications. Your objective should be to define your workloads so that the Power Agent delivers meaningful data for the groups that have an impact on the system.

Click here to read more about workloads and see configuration examples.

Capacity Planning Made Easy With Sightline

Leveraging your IT infrastructure investment is critical to your company’s profitability. User demands can fluctuate dramatically at any moment, making it increasingly important to maintain service delivery at all levels at all times. Adding hardware or network capacity in order to avoid service level issues is no longer the most efficient way to meet your business goals.

Sightline’s advanced solution enables IT professionals to accurately plan for infrastructure capacity changes that assure consistent service levels and optimal ROI. If an IT organization builds out their infrastructure with too much capacity, the burdens of a larger, more complex infrastructure come with it – higher costs, more time, more support issues and decreased manageability.

If the infrastructure does not have enough capacity, the IT organization will be quick to hear about it – slow response times, accessibility issues, and down time lead to downturns in productivity, unhappy customers and lost revenues.

The key to offering new services to internal and external customers is ensuring you have enough capacity in your infrastructure to successfully roll those services out. Failure to ensure the capacity is there is a recipe for disaster – bottlenecks, dreadfully slow response times, and ultimately, dissatisfied end users will all result. Limited resources available within IT organizations can mean that proper planning never takes place, and that services are launched in a reactionary fashion only when the demand gets high enough.

Sightline’s software allows IT organizations to proactively manage capacity in a way that not only exceeds users’ expectations, but that is cost-effective. Sightline’s software collects hundreds of performance metrics on each server in your network in real time. All of that information can be used to perform historical trending to see how resources have been consumed over a certain period of time. The information can also be used to benchmark the infrastructure, providing you a basis from which to offer new services or bring on additional applications or users. An IT organization armed with this information can now easily predict if the existing infrastructure will allow for any planned growth, as well as showing if spikes in activity will be able to be handled when they do occur.

The benefits of using Sightline for capacity planning include:

  • Do more with less – utilize existing assets and delay any expensive add-ons or upgrades
  • Correctly predict capacity to ensure uninterrupted service, even when demand spikes
  • Eliminate finger pointing and firefighting by knowing where potential issues may arise, where and when bottlenecks are likely to occur, and who is consuming what resources
  • Justify your budget by proving when and where changes or additional investments are needed through the use of Sightline’s extensive capacity planning reports
  • Determine where opportunities exist for server or data center consolidation
  • Test and deploy new services with confidence.

Having the right information about performance and workloads is critical to be able to make sure the hardware you deploy is sized correctly for current use and future growth. Sightline’s software allows you to understand both the processes of your infrastructure at a very granular level and the workloads that applications, services or users are putting on it.

Setting Your Data Retention Policy

An important consideration in performance management and capacity planning is your data retention policy. Some data consumes a tremendous amount of space and loses its value quickly, such as process data. The granularity of data also loses its value over time. For example, for analyzing bottlenecks, detailed performance and process data is quite useful. For Service Level Reporting, a small subset of data is required and no process data is needed. For trending and capacity planning, only highly summarized workload and resource utilization data representing peak periods is necessary.

A sound data retention policy provides for storing different types of data for different periods. It may be advantageous to store detailed performance and process data for two weeks to one month, while data summarized hourly will be stored indefinitely. It may also be desirable to store daily capsules of peak demand periods that contain fine granularity and process information. For example, month-end processing demand may be worth storing permanently as a baseline of peak demand. Government regulations may also specify data retention rules.

The capacity to store collected data directly on the monitored system in a host trace file (HTF) provides a temporary store of data which facilitates flexible strategies for downloading data to the central Sightline server. Sightline Enterprise Data Manager (EDM) or Expert Advisor/Vision (EA/V) acts as the central management console within the Sightline deployment. It aggregates, manages, displays and analyzes heterogeneous information from networks, platforms, operating systems, databases and applications. Your data storage strategy is implemented at this level of the Sightline deployment. This offloads a significant portion of performance management overhead.

By default, individual HTFs are uploaded or transferred to the Sightline server, where data is maintained at the raw data collection interval. Summarization can then be applied, allowing for long-term storage of data. Using Sightline EDM, the connection template allows you to set your data retention policy once, and then apply it to all of your new and existing connections. What’s our recommendation? We start with these settings:

  • Live data: 14 days at the data collection interval, including process-level data
  • Archive data: 32 days of daily archives, including process-level data
  • 10-minute summary: 90 days, process data is not included. In EDM, this is often used for visualization.
  • 30-minute summary: 13 months, process data is not included. This long-term data is used for visualization, trending and forecasting.

Keep in mind that forecasting requires at least twice the amount of data as the intended forecast. For example, a one-month forecast requires at least two months of input data. As with many things, more is better! The more historical data you can put into the forecast, the more accurate the forecast will be.

Configuring Logging Levels in EDM

There is a lot going on in the EDM application, and a lot of it is logged in the Data Collector’s wrapper.log files. Let’s face it, EDM logs a lot of stuff, and the log files can grow quite large. In the recent versions of EDM, we have limited the amount of logging that EDM does by default. But if you have an existing installation, you may need to do this manually. Edit the /Data Collector Service/resources/logger/ logging-types.xml file and replace all of the INFO tags with WARN. No need to restart anything, the changes will be picked up dynamically.

Sightline in the vCenter Environment

Sightline in the vCenter Environment. The Sightline Performance Management Solution is a perfect complement to your organization’s vCenter implementation. When used together, Sightline can save you time and money, while providing the peace of mind that your systems are running smoothly.

Read about Sightline in the vCenter Environment

Monitoring VMware vCenter. Being able to correctly predict what your infrastructure is going to do in the future can be a truly monumental task. Using the capacity planning functionality within Sightline you can easily create supporting documentation to provide the necessary information to accurately determine what your infrastructure should look like in the weeks, months and years to come.

Read about Capacity Planning with EDM

Using the VM Count metric from vCenter. We’re often asked about the metrics we collect, whether we collect certain information, or how we would represent a specific situation. An interesting metric collected from vCenter is VM Count. Is there a metrics you’re interested in? Let us know, and we’ll investigate!

Read about VM Count

Your environment under a single umbrella — Sightline. As your organization evolves, so does the IT environment. It is critical that the IT team can intelligently monitor all physical, virtual and cloud components,with accurate information available to correct situations as they occur. Once collected, information can be used to assist with migrating business critical applications to different platforms and ensure that the environment is correctly sized to maximize efficiency and limit spending. Sightline provides this and more – all under the same umbrella.

Read about the Sightline Unified Monitoring Solution

Data visualization in Sightline EDM: who you are influences what you want to see. Visualization of data sounds like an easy proposition; collect data and display it. But there’s a lot more to it than that! What data should be displayed? How much data? And in what format? With the depth of data available from Sightline data collection agents, and the versatility of Sightline‘s display interface, we can show you whatever you need to see.

Read about Data Visualization in EDM

Maximize performance monitoring of your VMware environment using Sightline. Sightline offers in-depth monitoring of your entire VMware environment. We use two different strategies to provide you with the best overall view of your virtual environment. First, we look at the vCenter servers to provide basic monitoring of the virtual space; then, we utilize Power Agents and Interface Agents to provide in-depth data collection for your mission-critical systems and applications. You decide just how deep you want to go into each instance in the environment.

Read more about Sightline in your VMware environment

Ask John
Questions? Comments? Suggestions? Ask John! If we use your input in a future newsletter, we’ll send you a $10 Amazon gift card.

Doubling the Charting Speed of Sightline EDM

Sightline can collect data from over 7,500 different metrics across multiple OS platforms.  One of the important features of Sightline Enterprise Data Manager (EDM) is to visualize the data. Once the data is collected we use a charting tool inside EDM  to display the data in a web browser interface. It helps give IT teams a fast way to look at their server performance, spot trends or clearly see when problematic events began.

Yet, we were increasingly unhappy with the performance of our current charting tool because it would often take several minutes for our customers to view a report containing their performance data.

Recently we came across CanvasJS. It claimed to render 100,000 data points in 100 milliseconds -- just the type of performance we were seeking. After evaluating CanvaJS, we decided to include it in the next release of EDM.

Below are a few before and after comparisons to illustrate why we’re excited by EDM's upcoming new performance charting features.

Before

After

blog-chart3Only 1 chart type (zoomline) supports zoom capability.
blog-chart2EDM can now display more data without affecting performance. This increases the granularity at which we can plot the data.
blog-chart1We were forced to summarize data in order to limit the number of data points rendered in the chart. As a result the data was less granular and data spikes were not shown.
blog-chart4The majority of charts have zooming capability.
blog-chart5
To find the value for a particular point the user has to hover over the exact data point in the chart. Depending on the number of data points in the chart this was hard to do and often frustrating.
blog-chart6Charts are now more interactive. The user does not need to hover over an exact data point to view the data information.
Customers complain about charts taking too long to load.
Charts render in the UI at least twice as fast.

Tip: Clustering EDM Instances

This week’s Ask John question is from Oscar Jiminez at Megadata Information Solutions in Costa Rica. Thanks, Oscar!

Question: In an EDM high availability hypothetical scenario (2 EDMs working as active/active nodes), can both EDMs can share the same EDM DB? Under this scenario, I can put this database in a SQL cluster and provide it also high availability.

Answer:  Certainly. This implementation is supported; click here for an example of configuring JBoss with Apache HTTPD.