Newsletter Archive

20 Nov, 2008

SIFT Note 2008-02

Metrics For Measuring The Effectiveness of Information Security Controls
There is clearly a pressing need in today's business world for organisations to ensure that their IT infrastructure is operating in a secure fashion. This has served as the catalyst for the production of a number of standards that enumerate a variety of IT security controls businesses should consider implementing to cater to this need (a chief example being ISO's 27001 standard). However, a key difficulty that often arises in this context relates the question of how an organisation can properly gauge the effectiveness with which implemented security controls are operating. This difficulty has arisen due to a lack of available standardised metrics which can be applied to operative security controls. In response to this, the National Institute of Standards and Technology (NIST) has produced the Performance Measurement Guide for Information Security which seeks to assist organisations in the development, selection and implementation of IT security performance measures. The guide identifies three principal categories of metrics, which may be used independently or together with regard to a particular point of assessment:

  • Implementation: considering the extent of progress that has been made in implementing a particular security control;
  • Effectiveness and efficiency: considering whether specific security controls have been implemented correctly and are operating as intended; and
  • Impact: considering what impact a security control has on an organisation's 'mission' (for example, cost savings and improvement in or consolidation of public trust).

Appendix A to the guide provides examples of specific measures that fall into one of these three categories. These include:

  • Vulnerability Management: the number of 'high' (i.e. severe in nature) security vulnerabilities identified and mitigated within an organisationally determined time period, in comparison with the total number of high vulnerabilities identified within that time period;
  • Access Control: the number of remote access points through which unauthorised access to company systems was obtained, in comparison with the total number of remote access points;
  • Awareness and Training: the number of organisational personnel that have completed information security training within the past year in comparison with the total number of information security personnel;
  • Audit and Accountability: the frequency with which system audit logs are reviewed and analysed for inappropriate activity;
  • Contingency Planning: the number of systems that have undergone annual contingency plan testing in comparison with the number of systems in the organisation. This is essentially concerned with an organisation's disaster recovery planning, in that the measure determines whether the plans that are in place are sufficiently well designed to deal with emergencies, system outages and post disaster recovery;
  • Planning: the number of company employees who are given access to information systems only after they sign an acknowledgement that they have read and understood organisational rules of behaviour, as compared with the total number who have system access;
  • System and Information Integrity: the number of identified operating system vulnerabilities that have been patched or otherwise mitigated. This measure focuses upon available patches for operating systems installed on company hardware.

It is important to note that these metrics will only be useful if they are compared against a pre-determined benchmark so that the performance of a particular security control can be properly assessed. As the guide confirms, it is relatively straightforward to devise benchmarks for 'implementation' metrics since the ultimate aim for any organisation should be to strive for 100% implementation of all security controls. Conversely, producing benchmarks for 'impact' or 'effectiveness and efficiency' metrics is an inherently more complex task, since determining what constitutes an acceptable level of performance will often require a degree of subjective reasoning (for example, it is difficult to assess how much an 'impact' metric should improve the public's trust in an organisation). Moreover, an ideal benchmark that one organisation sets for a particular control may be completely different to that set by a different organisation. Indeed, it may even be the case that different systems within the one organisation will require different benchmarks for the same metric. As the guide suggests, one possible approach to dealing with this issue is to avoid using a benchmark the first few times a particular impact or effectiveness and efficiency metric is applied to a security control. During this time, a performance baseline can be established and corrective actions identified to improve the control's performance. Once these actions are performed, the baseline can be compared with the current performance of the control to determine how well it is functioning. The NIST guide provides an important means through which organisations can measure and report about the efficacy of their IT security controls in a standardised and consistent manner. Whilst obtaining the data necessary to use the metrics identified in the guide will often have associated time and monetary costs for organisations, these costs will usually be outweighed by the potential expense associated with having failing security controls in place that are not detected promptly or that remain undetected. Further information: Performance Measurement Guide for Information Security A Guide to Security Metrics

SIFT Presenting at Ruxcon, Sydney 2008
The Ruxcon information security conference is once again being held in Sydney on the 29th to the 30th of November. The not-for-profit conference is regarded throughout Australia and the world as one of the leading information security research events. With just a $60 conference entry fee, we would encourage everyone to attend the presentations of the cutting edge research to be showcased at Ruxcon. This year, three SIFT researchers have been accepted to present and the prestigious event. A short summary of each is provided below. SCADA Penetration Testing: Hacking Modbus Enabled Devices - Daniel Grzelak (SIFT) Modbus is a roughly 30 year old application messaging protocol for interacting with Supervisory Control and Data Acquisition (SCADA) devices. While people have been interested in SCADA security for a while now, specific information only started being published recently and there are now a few publicly-available testing toolkits. This presentation seeks to fill that gap by introducing a toolkit and methodology for Modbus security testing as well as providing some interesting insights on what is out there on the Internet. Attacking Rich Internet Applications - Alex Kouzemtchenko (SIFT), Stefano Di Paola (Minded Security) In recent years rich internet applications (RIAs) have become the mainstay of large internet applications and are becoming increasingly attractive to the industry due to their similarity to desktop applications. This presentation will examine the largely under-researched topic of RIA security in the hopes of illustrating how the complex interactions with executing environments and generally poor security practices can lead to exploitable applications. Browser Rider: Your way to Fun Browsing - Benjamin Mosse (SIFT) Browser exploitation is in fashion but there doesn't appear to be a robust framework to build and run attacks. Browser Rider will try to fill the gap by providing a framework to build, deploy and manage payloads that exploit the browser. The long term aim of this project is to provide penetration testers with a powerful, simple and flexible interface to client side attacks and targets. Other Upcoming Presentations

  • Same Origin Policy Weaknesses - Power of Community, Seoul & XCon, Beijing - Alex Kouzemtchenko (SIFT)

Other Recent Presentations

  • Mobile Device Security: spyPhones, HackBerrys & Smartphonies - Security 2008, Sydney - Victor Caringal (SIFT)
  • Blackhat Search Engine Optimisation - Security 2008 - Paul Theriault (SIFT)
  • Fatigue - Affecting the Culture of your Organisation - ISACA Oceania CACS 2008 - Nick Ellsmore (SIFT)
  • ITSEAG - Defence in Depth Presentation/User Access Management - SCADA Community of Interest, Sept 2008 - Nick Ellsmore (SIFT)



SIFT IN 2008 BRW FAST 100
In the second half of 2008, SIFT was recognised for our rapid and consistent growth through inclusion in the 2008 BRW Fast 100, which recorded SIFT as the 92nd fastest growing company in Australia, the 25th fastest growing IT company, and the only IT security company in the Fast 100. We were also finalists in the MyBusiness Awards 2008. As the largest pure-play information security consulting firm in New South Wales, and one of the largest in Australia, the firm has developed a deserved reputation for excellence in delivery. Since inception, SIFT has invested in research and development and has positioned itself as an Australian information security thought leader. This is demonstrated by SIFT research findings having been tabled at meetings of the Asia-Pacific Economic Co-operation (APEC) Telecommunication & Information Working Group, and referenced in reports issued by the European Parliament. SIFT consultants have delivered presentations both locally at events such as the AusCERT annual conference on the Gold Coast and the Annual IT Security Summit in Sydney, and internationally in countries including Japan, Hong Kong, the Philippines, Chile, China, Canada, Fiji, South Korea, New Zealand, Singapore, and the USA. Thank you to all our clients for your ongoing support; and most importantly thank you to the SIFT team for all the hard work to build the company to where it is today. 2009 promises to be another year of growth and shared success.

This article is tagged in these categories

SIFT, Metrics