Logging and SIEM Basics: What to Collect and How to Use It
A SIEM, or security information and event management system, collects logs from across your environment, normalizes them, and lets you search them and generate alerts when patterns suggest an attack. Good security monitoring depends first on collecting the right logs and second on using tools to make sense of them.
Why logs matter
Without logs, an incident becomes guesswork. You cannot answer basic questions: how did the attacker get in, what did they access, and are they still there? Logs also support troubleshooting, compliance, and accountability. Many breaches are discovered only after weeks because nobody was looking at the evidence that existed.
What to log first
Prioritize sources that show access and change:
- Authentication logs: successful and failed logins, multi-factor events, password resets, and account lockouts, from directories, VPNs, and cloud identity services.
- Cloud audit logs: who created, changed, or deleted resources and permissions.
- Firewall and network logs: allowed and denied connections, particularly outbound traffic.
- DNS logs: the names devices attempt to resolve.
- Endpoint logs: process starts, security tool alerts, and administrative actions.
- Web server and application logs: requests, errors, and access to sensitive functions.
- Email security logs: delivered and blocked messages, forwarding rule changes.
Centralize and protect
Send logs to a central location rather than leaving them on individual systems, where an attacker can erase them. Use a consistent time source on all systems so events can be correlated in order. Restrict who can modify or delete logs, and keep a copy in a separate account or immutable storage. Encrypt log transport and storage, and avoid writing passwords, tokens, or full card numbers into logs in the first place.
What a SIEM adds
- Normalization: converting different log formats into a common structure.
- Correlation: linking events across sources, such as a login from a new country followed by a mailbox rule change.
- Search and dashboards: fast investigation across large volumes of data.
- Alerting: rules and analytics that notify you of suspicious patterns.
- Retention and reporting: evidence for audits.
Cloud-hosted and open-source options exist, and the choice depends on volume, staff skills, and budget. A smaller team may start with centralized log search and a handful of alerts before adopting a full platform.
Building useful alerts
- Start with high-value scenarios: repeated failed logins followed by a success, administrator accounts created, logging disabled, impossible travel, large data transfers, and new inbound firewall rules.
- Define who responds and what they do for each alert, in a short playbook.
- Tune relentlessly. An alert that fires constantly gets ignored. Adjust thresholds, add exceptions, and retire rules that never yield action.
- Prioritize by severity so urgent items reach a person quickly while low-priority ones are reviewed in batches.
- Test detections by simulating the behavior in a safe, authorized way.
Retention and privacy
Decide how long to keep logs based on regulatory requirements, investigation needs, and cost. Many organizations keep recent data readily searchable and older data in cheaper archive storage. Because logs can contain personal data, limit access, document the purpose, and follow applicable data-protection obligations.
Common pitfalls
- Collecting everything but analyzing nothing.
- Leaving critical systems out of scope, such as cloud accounts or identity systems.
- Letting log sources silently stop. Alert when a source goes quiet.
- Forgetting that people must be available to respond, including outside working hours for serious alerts.
A first-90-days plan
In the first month, identify critical systems and send authentication and cloud audit logs to a central store with synchronized clocks. In the second month, build a small set of alerts for administrator changes, failed-login bursts, and disabled logging, and write a short response note for each. In the third month, add firewall, DNS, and endpoint sources, review alert quality, and retire or tune the noisiest rules. At the end, test by simulating a few of the scenarios and record how long detection and response took, then repeat the cycle.
Example detection scenario
Consider a cloud account where someone signs in from an unfamiliar country, disables multi-factor authentication, creates a new access key, and then downloads a large volume of data. Each event alone may look minor, but a correlation rule that links them within a short window raises a high-priority alert. The responder disables the account, revokes the key, reviews the logs to see what was accessed, and notifies the owner. This shows why diverse log sources and good time synchronization matter: correlation only works when events can be compared across systems.
Frequently asked questions
Do small businesses need a SIEM?
Not necessarily a full platform, but they benefit from centralized logs for key systems and a few alerts. Managed monitoring services can fill the gap without a large team.
How long should I keep logs?
It depends on regulation and risk. A common approach is a few months of readily searchable data plus longer archival storage, but confirm your own obligations.
What is the difference between a log and an alert?
A log is a record of an event. An alert is a notification generated when events match a rule or pattern indicating possible trouble.
Key takeaways
- Collect authentication, cloud audit, network, DNS, endpoint, and application logs first.
- Centralize logs, synchronize time, and protect them from tampering.
- A SIEM correlates and alerts, but needs tuning and people to respond.
- Start small with high-value alerts and expand as you mature.
