Log Management: The Complete Guide for Compliance and Security Teams

What Is Log Management?

Log management is the systematic practice of collecting, centralising, storing, and analysing log data generated by systems, applications, and network devices across an organisation's IT environment. Logs provide the audit trail that enables security monitoring, incident investigation, compliance evidence, and operational visibility.

For compliance-driven organisations, log management is a foundational control required by ISO 27001, SOC 2, NIS2, and DORA to demonstrate that security events are recorded, monitored, and available for audit.

Log Sources

Source	Log Types	Examples
Operating systems	Authentication, system events, process execution	Windows Event Log, Linux syslog/journald
Applications	User actions, transactions, errors	Web server logs, application audit logs
Network devices	Traffic flows, firewall rules, DNS queries	Firewall logs, proxy logs, flow data
Cloud platforms	API calls, resource changes, IAM events	AWS CloudTrail, Azure Activity Log, GCP Audit
Security tools	Alerts, detections, scan results	EDR, vulnerability scanners, WAF
Databases	Queries, schema changes, admin operations	Audit logs, slow query logs
Identity systems	Authentication, MFA, SSO events	IdP logs, Active Directory events

Log Management Architecture

Component	Function	Examples
Collection agents	Gather logs from sources	Fluentd, Filebeat, rsyslog, cloud-native agents
Transport	Securely transmit logs to central platform	Kafka, AWS Kinesis, syslog-TLS
Processing	Parse, normalise, enrich log data	Logstash, Fluentd, cloud ETL pipelines
Storage	Store logs with appropriate retention	Elasticsearch, S3, Azure Blob, Splunk
Analysis	Search, correlate, and visualise logs	Kibana, Splunk, Grafana, cloud analytics
Alerting	Real-time notification on critical events	SIEM rules, CloudWatch Alarms, PagerDuty

Log Retention Requirements

Framework	Minimum Retention	Recommended Retention
ISO 27001	Risk-based (typically 12 months)	1-3 years
SOC 2	Audit period (12 months)	1-2 years
NIS2	Sufficient for investigation	12-18 months
DORA	5 years for ICT incidents	5 years
GDPR	Proportionate to purpose	6-12 months for access logs
PCI DSS	12 months (3 months immediately available)	12 months

Essential Log Events

Event Category	What to Log	Why
Authentication	All login attempts (success and failure)	Detect brute force, credential stuffing
Authorisation	Access grants, denials, privilege escalation	Detect unauthorised access
Data access	Access to sensitive data and systems	Audit trail for data protection
Configuration changes	System, network, and application changes	Detect unauthorised modifications
Administrative actions	All privileged operations	Accountability for admin activities
Security events	Alerts, policy violations, anomalies	Security monitoring and response
Error conditions	Application errors, system failures	Operational monitoring and incident detection

Compliance Requirements

Framework Mapping

Requirement	ISO 27001	SOC 2	NIS2	DORA
Event logging	A.8.15	CC7.2	Art. 21(2)(g)	Art. 10(2)
Log protection	A.8.15	CC7.2	Art. 21(2)(g)	Art. 10(2)
Clock synchronisation	A.8.17	CC7.2	Art. 21(2)(g)	Art. 10(2)
Log review	A.8.15	CC7.2	Art. 21(2)(b)	Art. 10(1)
Log retention	A.8.15	CC7.2	Art. 21(2)(g)	Art. 10(2)
Administrator logging	A.8.15	CC6.3	Art. 21(2)(i)	Art. 9(4)

Audit Evidence

Evidence Type	Description	Framework
Logging policy	Documented policy defining what is logged and how long	All frameworks
Log source inventory	List of all systems sending logs with coverage gaps	All frameworks
Retention configuration	Evidence of log retention settings matching policy	All frameworks
Log integrity controls	Documentation of tamper protection mechanisms	All frameworks
NTP configuration	Evidence of time synchronisation across all systems	ISO 27001, SOC 2
Log review procedures	Documented process for regular log review	All frameworks
Access control evidence	Proof that log access is restricted appropriately	All frameworks

Common Mistakes

Mistake	Risk	Fix
Logging everything without strategy	Excessive cost, alert fatigue, compliance gaps	Define logging requirements by use case and compliance need
No log integrity protection	Logs can be tampered with, undermining audit value	Use immutable storage, cryptographic hashing, centralised collection
Inconsistent timestamps	Cannot correlate events across systems	Implement NTP synchronisation, monitor for clock drift
No log review process	Logs collected but never analysed	Establish daily automated alerts and weekly manual review
Insufficient retention	Logs deleted before investigation or audit needs them	Align retention with longest applicable framework requirement
Logging sensitive data	PII or credentials in logs create additional compliance burden	Mask or redact sensitive fields before storage

How Orbiq Supports Log Management Compliance

Orbiq helps you demonstrate log management controls:

Evidence collection — Centralise logging policies, retention configurations, and review procedures
Continuous monitoring — Track logging coverage and retention compliance
Trust Center — Share your logging and monitoring posture via your Trust Center
Compliance mapping — Map logging controls to ISO 27001, SOC 2, NIS2, and DORA
Audit readiness — Pre-built evidence packages for auditor review