IntelligentProcessLifecycle

The Intelligent Process Lifecycle of Active Cyber Defenders

Description

This github hosts the poster related files of false positive and error categories in security operation services. The goal is to define an open source reporting standard for Security Operation Center reports. The hereby published KPIs are focused on creating statistics relevant for continuous improvement of operational cyber defense tasks.

This information was first presented at FIRST 2020 together with Eireann Leverett(who injected his experience in regards to risk management), the video is available here: https://www.youtube.com/watch?v=pR02cZlPakU The peer reviewed paper created for this content can be found here: https://dl.acm.org/doi/10.1145/3499427

The recording of the talk i gave at SwissCyberStorm 2021 about the taxonomy for integrity as well as compliance configuration monitoring can be found here: https://www.youtube.com/watch?v=ra4LZouxIyk

The recording of the talk I gave at Area41 2022 about the problems in vulnerability management can be found here: https://www.youtube.com/watch?v=qdgY6aAfUAk

Quick Overview

Discipline:	Security Monitoring	Configuration Anomalies	Vulnerability Management
Validation via:	SIEM Use Cases, EDR /AV Logs, IDS/IPS, NDR Logs	Integrity Monitoring, Compliance Configuration Monitoring	Vulnerability Scans, Patch Verification
Paper published:	Peer Reviewed Version	Self published paper	Peer Reviewed Paper
Presentation link:	Hack.Lu 2019 Youtube	SwissCyberStorm 2021 Youtube	Area41 2022 Youtube
Slides:	Hack.Lu 2019 Slides	SwissCyberStorm 2021 Slides	Area41 Slides
JSON Taxonomy file:	MISP JSON File Security Monitoring	MISP JSON file Integrity Compliance Monitoring	MISP JSON file Vuln Mgmt & MISP JSON file Detection failures

Continuous Improvement metrics for integrity or technical security compliance monitoring

The following metric suggestions are best correlated with values for system responsible team or source systems type. The target values are in comparison to the total amount of events generated by this service per time unit (month, week, quarter, etc).

KPI	Explanation	Target Value	Owner	Risk Type	Business Impact	Motivating Example
Number of 'legitimate violations authorized by change'	This value reflects events which usually are classic false positives, where all official change processes were correctly followed but the SOC was not included in the process and therefore could not prevent the false alarm	< 10 %	Compliance	Endogenous	Governance Risk	Officially approved change to Apache changes configuration format, and detection tools alert on the change.
Number of 'configuration errors in baseline'	This value reflects what system configurations (or even configuration templates) needs improvement.	< 10 %	Compliance/ Operational	Endogenous	Change and Compliance Management Risk	The baselines for configuration templates were taken from development instead of production systems.
Number of 'Limitation in verification products' found	If too many of these events were created by configurations, the causing tool should be questioned.	< 5 %	Compliance/ Operational	Endogenous	SOC Operational risk	Snort rules can't be scoped narrowly to detect the change we're interested in, but if they're given wider scope, produce false positives.
Number of 'activities with no change required'	There seems to be a mismatch between the defined security scope and the verified security scope. Gaps should be verified	< 5 %	Policy	Endogenous	Policy-operational mismatch leading to overworked SOC	Sysadmin clears logfiles to save space, which needs no approval, but it triggers an alert in the SOC.
Number of 'unauthorized changes without legitimate cause'	Very high numbers → Security process and IT process integration needs rework; Very low numbers → The configurations aren‘t detecting or you are safe	it depends :)	Policy	Endogenous	Potential intrusion/Prioritise investigation	IIS server has added a user and administrator denies knowledge of event.
Number of changes without formal documentation	Number of legitimate violation with missed change documentation is highlighting where the SOC had no chance of automating false alerts, as well as where employees are not complying to formal processes.	<5%	Policy/Compliance	Endogenous	Shadow IT Administration Risk	Sysadmin changes configuration of Apache server without formal change management documentation (but it would have been approved).