Measurement based development approach to get your SOC to the next level

As you can see -and most probably already know-, the number of attacks on information security is rising sharply:

The cornerstone of an organization’s security is the ability to detect attacks as quickly as possible and to respond to them appropriately. This is ensured by a properly functioning Security Operations Center (shortly: SOC) established internally or applied as an external service, even in a hybrid model.

The most common motivations behind the development of a SOC are the need to centralize cybersecurity operations, to improve the visibility of the organization as a whole, the need to manage ever-increasing risks, to improve threat detection and to reduce exposure to threats. In addition, laws and regulations may also define centralized cyber defense monitoring and operations.

In our experience however, in the vast majority of cases organizations developing such capabilities do this without continuous or regular re-measurement. Yet, if a company wants to achieve its goals, it needs to be aware of where it stands at a given time, where it heads, and based on that it is possible to determine and prioritize the tasks to be fulfilled.

This article describes the underlying considerations and the design process of a framework to measure the capabilities of the Security Operations Center in any organization.

When a SOC is being built, the process has the ordinary life cycle steps of Plan, Assess, Design, Implement and Monitor:

The target operating model during Design phase already defines the time frame; when and where we would like to get, so during the Implement and Monitor phase we need maturity levels.

It is a crucial need to measure the maturity of any capability, because the result of such measurement provides information on:

• How good the capability is in its current state;
• Whether development is still needed to reach the desired state and;
• How a development plan should be defined or altered to reach our goals.

Before we started to develop our maturity assessment framework, we researched if there are existing free frameworks which we could reuse. We soon had to realize that although several methodologies exist, most of them are specialized in specific domains of cyber defense but there is no one comprehensive guide.

Furthermore, the definition of maturity changes over time. Totally different level was considered mature some 15 years ago, or even 5 years ago than it is today. We need a framework that always evaluates according to the current situation and possibilities. The definition of a mature SOC capability in our case means the degree of formality and optimization of the processes that build up or enable incident detection and response capability.

A good SOC is not just a series of alert processing operations, but also a producer and user of Threat Intelligence (TI) data, working closely with the Incident Management team (or it may be part of the SOC) and proactively looking for potential threats (hunting). However, the primary role of a Security Operations Center is to detect threats and respond to them, thus minimizing the business impact of a cyber security incident. As a consequence, the focus of our framework is to measure the maturity of central detection and response capabilities. For the same reason, measuring the daily maintenance of security equipment (firewalls, IDPs, etc.) and the maturity level of all other activities not related closely to the above categories was out of scope.

The approach was to create an assessment framework that is aligned to standard methodologies and which delivers a tangible development plan. Therefore, the concept has been adjusted to the NIST Cyber Security Framework (CSF) and RSA Advanced SOC (ASOC) characteristics, as both are well-adopted in the industry. NIST CSF refers to functions and capabilities, while RSA ASOC characteristics define tangible deliverables that are enablers for such functions and capabilities.

Based on these, the aim of the concept was set to:

• Assess the maturity of the capability referenced to NIST CSF;
• And based on the results, provide deliverables for development aligned with RSA ASOC.

The following reference maturity levels have been determined for the framework:

• Initial refers to a state when only ad hoc processes exist, they are performed inconsistently and results are hardly predictable.
• Managed means that disciplined processes exist which ensure repeatability, but results may depend on who performs it.
• Defined is the level where standard consistent processes exist and they integrate best practice methods.
• Measured means that the processes are predictable, outcomes can be predicted from intermediate states.
• Optimized refers to the state where the processes are continuously improved and improvement actions close gaps between the organization's current and required capabilities.

The used frameworks in a nutshell:

As previously mentioned, we used the NIST CSF framework as a starting point, as it perfectly summarizes the prerequisites for the detect and response capabilities required to operate a SOC. It defines the 5 core capabilities required to achieve cyber security:

The Identify capability enables us to know what we need to protect. The Protection capability defines how to build up our standard architectures to protect our valuable assets. Next is Detect, as even if we identified what we want to protect and built up some protection we still need to be able to detect if someone is attacking it. The Respond capability ensures that when an incident occurs, we know what response to give. And last but not least, even if we have good incident response capability, it doesn’t mean the organization won't suffer any harm in case of cyber attack, and that’s why we need Recover activity, to know how to rebuild our critical business processes.

For the development of the enhancement plan -which is the expected outcome of this measurement- we used RSA Advanced SOC (or shortly ASOC) characteristics as this is based on best practices describing the main characteristics of an effective cyber security incident "detection" and "response" capability.

These characteristics say that if we would like to have an effective cyber defense capability:

• We need to be Business and Risk Aligned (manage our business and risk management processes in a coordinated way);
• We need Visibility on the assets which we must protect;
• Need appropriate Content within our detection technologies (e.g., optimized, up-to-date alarm rules). The SIEM -which is the soul of the SOC- is never finished; the rules which generate alerts must constantly meet changing attack methods;
• Proper Security Operations (the internal processes of the cyber defense center) who are providing the response in case of a cyber security incident commensurate with the risk and potential impact;
• And in order to be aware of the latest threats, we need Applied Intelligence & Analytics.

According to this methodology if you satisfy these 5 main characteristics, you can state you have an effectively working SOC.

During our assessment, the subcategories of NIST CSF "Detect" and "Respond" functions are matched 1-to-1 to the RSA Advanced SOC domains, as the RSA ASOC defines tangible deliverables that help implement NIST CSF functions and capabilities. During the survey phase, we review all NIST CSF subcategories with a detailed and targeted questionnaire to check the level of maturity of the organization.

In order the ensure trustworthy results, a series of questions have been assigned to each CSF subcategory.

The link between the NIST CSF subcategories and the RSA ASOC domains allows the results to be visualized on maturity radars /spider webs/ and development recommendations to be assigned to areas for improvement.

The overall results are presented on maturity radars on both CSF subcategory levels and due to the one-to-one mapping, we out of the box get it for ASOC domains too. The results show which domains should be developed.

This framework has been implemented in RSA Archer Suite. The solution can track how maturity changes over time and thus how we can achieve the set milestones.

The questions and answers are saved by the system. Evidences can also be attached to ensure proper documentation, which makes this a useful tool during an audit. We must note that no confidential information (such as asset lists, etc.) needs to be handed over.

After finishing the survey, the solution generates a management summary explaining the maturity levels and scoring, furthermore it automatically generates findings in case of incorrect answers.  

To sum it up, we intended to create a framework that helps us to measure the detection and response capability of any SOC. As a deliverable of the assessment, a feasible development plan is produced that highlights if we lack any capabilities. The survey can be repeated as desired, changes in the maturity levels can be monitored, thus enabling the organization to address the gaps between the current and desired state to optimize its operation.

Although the evaluation of our survey and its report does not solve all the problems of a SOC, the results can be drilled deeper, thus it provides a good foundation for developing an effective SOC.

Get an idea from SOCWISE to build or develop your SOC!

Some CISOs have built their SOCs over time with a mix of internal and external resources. But, given the ongoing evolution of cybersecurity techniques and the need to constantly adopt new skills and tools, managing this mix is becoming increasingly complicated.

Benchmarking : The Key to Creating an Efficient Security Operations Center (SOC)

See how we built it, how it works, and what technologies we use!