Best practices for audit, log review for IT security investigations

At the heart of most devices that provide protection for IT networks is an ability to log events and take actions based on those events. This application and system monitoring provides details both on what has happened to the device and what is happening. It provides security against lapses in perimeter and application defences by alerting you to problems so defensive measures can be taken before any real damage is done. Without monitoring, you have little chance of discovering whether a live application is being attacked or has been compromised.

Critical applications, processes handling valuable or sensitive information, previously compromised or abused systems, and systems connected to third parties or the Internet all require active monitoring. Any seriously suspicious behaviour or critical events must generate an alert that is assessed and acted on. Although you will need to carry out a risk assessment for each application or system to determine what level of audit, log review and monitoring is necessary, you will need to log at least the following:

• User IDs
• Date and time of log on and log off, and other key events
• Terminal identity
• Successful and failed attempts to access systems, data or applications
• Files and networks accessed
• Changes to system configurations
• Use of system utilities
• Exceptions and other security-related events, such as alarms triggered
• Activation of protection systems, such as intrusion detection systems and antimalware

Collecting this data will assist in access control monitoring and can provide audit trails when investigating an incident. While most logs are covered by some form of regulation these days and should be kept as long as the requirements call for, any that are not should be kept for a minimum period of one year, in case they are needed for an investigation.  However, monitoring must be carried out in line with relevant legislation, which in the UK is the Regulation of Investigatory Powers and Human Rights Acts. Employees should be made aware of your monitoring activities in the network acceptable use policy.

No matter how extensive your logging, log files are worthless if you cannot trust their integrity.

Log files are a great source of information only if you review them. Simply purchasing and deploying a log management product won’t provide any additional security. You have to use the information collected and analyse it on a regular basis; for a high-risk application, this could mean automated reviews on an hourly basis. ISO/IEC 27001 control A.10.10.2 not only requires procedures for monitoring the use of information processing facilities, but demands the results are reviewed regularly to identify possible security threats and incidents.

However, even small networks can generate too much information to be analysed manually. This is where log analysers come in, as they automate the auditing and analysis of logs, telling you what has happened or is happening, and revealing unauthorised activity or abnormal behaviour. This feedback can be used to improve IDS signatures or firewall rule sets. Such improvements are an iterative process, as regularly tuning your devices to maximise their accuracy in recognising true threats will help reduce the number of false positives. Completely eliminating false positives, while still maintaining strict controls, is next to impossible, particularly as new threats and changes in the network structure will affect the effectiveness of existing rule sets. Log analysis can also provide a basis for focused security awareness training, reduced network misuse and stronger policy enforcement.

ISO/IEC 27001 controls A.10.10.4 and A.10.10.5 cover two specific areas of logging whose importance is often not fully appreciated: administrator activity and fault logging. Administrators have powerful rights, and their actions need to be carefully recorded and checked. As events, such as system restarts to correct serious errors, may not get recorded electronically, administrators should maintain a written log of their activities, recording event start and finish times, who was involved and what actions were taken. The name of the person making the log entry should also be recorded, along with the date and time. The internal audit team should keep these logs.

There are two types of faults to be logged: faults generated by the system and the applications running on it, and faults or errors reported by the system's users. Fault logging and analysis is often the only way of finding out what is wrong with a system or application. The analysis of fault logs can be used to identify trends that may indicate more deep-rooted problems, such as faulty equipment or a lack of competence or training in either users or system administrators.

All operating systems and many applications, such as database server software, provide basic logging and alerting faculties. This logging functionality should be configured to log all faults and send an alert if the error is above an acceptable threshold, such as a write failure or connection time-out. The logs should be reviewed on a regular basis, and any error-related entries should be investigated and resolved. While analysing all logs daily is likely an unrealistic goal, high-volume and high-risk applications, such as an e-commerce Web server, will need almost daily checking to prevent high-profile break-ins, while for most others a weekly check will suffice.

There should be a documented work instruction covering how faults are recorded or reported, who can investigate them, and an expected resolution time, similar to a service contract if you use an outside contractor to support your systems. Help desk software can log details of all user reports, and track actions taken to deal with them and close them out.

No matter how extensive your logging, log files are worthless if you cannot trust their integrity. The first thing most hackers will do is try to alter log files to hide their presence. To protect against this, you should record logs both locally and to a remote log server. This provides redundancy and an extra layer of security as you can compare the two sets of logs against one another -- any differences will indicate suspicious activity.

If you can’t stretch to a dedicated log server, logs should be written to a write-once medium, such as a CD-R or DVD-R, or to rewritable media such as magnetic tape data storage or hard disk drives that automatically make the newly written portion read-only to prevent an attacker from overwriting them. It's important also to prevent administrators from having physical and network access to logs of their own activities. Those tasked with reviewing logs should obviously be independent of the people, activities and logs being reviewed.

The protection of log information is critical. Compromised logs can hamper IT security investigations into suspicious events, invalidate disciplinary action and undermine court actions.

Another point to bear in mind is system clocks need to be synchronised so log entries have accurate timestamps. Check computer clocks and correct any significant time variations on a weekly basis, or more often, depending on the error margin for time accuracy.

Clocks can drift on mobile devices and should be updated whenever they attach to the network or desktop. Always record the time of an event in a consistent format, such as Universal Coordinated Time (UTC) across all files. For additional security, add a checksum to each log entry so you can detect if any entries have been tampered with. Controls also need to be in place to ensure there is ample log storage. If your logs can be trusted, they can help you reconstruct the events of security incidents and provide legally admissible evidence.

Logging and auditing work together to ensure users are only performing the activities they are authorised to perform, and they play a key role in preventing, as well as in spotting, tracking and stopping unwanted or inappropriate activities.

About the author:
Michael Cobb, CISSP-ISSAP, CLAS is a renowned security author with more than 15 years of experience in the IT industry. He is the founder and managing director of Cobweb Applications, a consultancy that provides data security services delivering ISO 27001 solutions. He co-authored the book IIS Security and has written numerous technical articles for leading IT publications.Cobb serves as SearchSecurity.com’s contributing expert for application and platform security topics, and has been a featured guest instructor for several of SearchSecurity.com’s Security School lessons.