First published January 2007
School of Computer and Information Science
Edith Cowan University
Modern techniques and methodologies for detecting attacks and malicious activities on computers and networks has evolved a lot over the last couple of years. The need for detecting intrusion attempts before the actual attack simplifies the job of securely administering computer networks. Often an attacker will probe different ports and services on a network to get intelligence about the structure of the network. Afterwards how and what services can be compromised is decided. This is a common strategy applied by most of the attackers and this is where Intrusion Detection Systems (IDS) comes in. They simplify the job of detecting attacks well before the actual attack by tracing the trails that the attacker leaves while gathering intelligence about a network. Government legislations however often act as a barrier in accessing/ monitoring private communications. This paper will particularly focus on the potential of using IDS logs as evidence in legal proceedings. It will also address the Commonwealth Telecommunication Interception Act to identify some conflicting issues that at some extent acts as a barrier for deployment of IDS tools.
Intrusion Detection System (IDS), IDS logs, telecommunication interception.
There is a growing need for use of Intrusion Detection Systems (IDS) in private and public corporations. These systems are very important to safeguard the huge distributed computing environment that a certain organization controls and manages. The log files that IDS generate can be massive depending on the volume of traffic and information they handle. It is important to understand that the use of IDS is a measure for securing the information system of companies and organization and they provide valuable support for diagnosing and reviewing security problems. Government legislators however, don’t consider this and they will often pass legislations that will stand on the way of public and private corporations in terms of using IDS as a security tool. The legislators need to understand that it is not only the police and intelligence agencies that need to intercept communications, private and public sector companies also need to intercept not for interception’s sake but for the sake of maintaining a secured information system. This paper will try to address these issues in general it will also discuss the recent amendment in the telecommunications interception laws in Australia. AIM
The aim of this paper is to determine the potential of using IDS log files as evidence. This paper will not make any conclusions regarding the matter. However, in certain cases personal arguments will be coming up. It is the intent of this paper to examine the telecommunication legislations particularly in Australia and examine some of the implications for the use of IDS’s in protecting computers and networks. Reference will be made to legislation from Australia, the United Kingdom and the United States in order to demonstrate points of potential arguments. It is not the intent of this paper to substitute the considered legal advice. The opinions made in this paper are strictly of the authors and do not reflect any government or political body.
INTRUSION DETECTION SYSTEM (IDS)
What it is
Intrusion Detection is the act of discovering or determining the existence, presence, or fact of the wrongfully entering upon, seizing, or taking possession of the property of another (F.C & Associates 1996). “Intrusion Detection System (IDS) is any system or set of systems that has the ability to detect a change in the status of a system or network” (Lane 2001). There are two major types of IDS’s. They are Signature-based IDS and Anomaly-based IDS. The deployment of IDS can be in two forms one is Network-based IDS and the other is Host-based IDS.
Why use IDS
IDS’s has become a part of every organization’s security system now days. They reduce risks of intrusions and prevent serious attempts to attack a system by alerting the administrators. IDS are capable of detecting preambles to attacks and with this they help to document and present the risks and threats. IDS serve as a quality control mechanism of the security system of an organization providing diagnosis, causes and details about different aspects of the security system. Mell (n.d) “IDS can detect when an attacker has penetrated a system by exploiting an uncorrected or uncorrectable flaw. Furthermore, it can serve an important function in system protection, by bringing the fact that the system has been attacked to the attention of the administrators who can contain and recover any damage that results. IDSs verify, itemize, and characterize the threat from both outside and inside your organization’s network, assisting you in making sound decisions regarding your allocation of computer security resources.”
IDS LOG FILES AS FORENSIC EVIDENCE
The first thing that needs to be considered is the legal dimension. While gathering and processing the IDS logs the legal dimensions of conducting forensic analysis needs to be considered thoroughly because it may cause problems later on. Turner (2002) says ” The principles of ‘chain of custody’ or continuity of evidence and ‘auditability’ are well known in forensic circles, there remains a general lack of awareness of these principles within the computer security community. As a consequence the dangers of ‘dirtying the data’ remain prevalent. An additional issue that emerges during analysis concerns ‘acontextual’ presentation of individual entries in log files. This can lead to a misrepresentation of the significance or insignificance of individual entries and of the log file as a whole”. Therefore, these things need to be carefully considered and practiced before using log files as forensic evidence.
Admissibility and validity
All forensic evidences have to overcome two tests. One of them is admissibility and the other is weight. According to turner (2002) “The USA – code title 28, section 1732 states that ‘ logs files are admissible as evidence if they are collected in the regular course of the business’. However, this principle of admissibility does not provide any guarantee that in any particular case log files will be deemed legally valid”. There are other issues, which are inevitable to avoid. “The ability to identify, track, trace and analyze log files is central to forensic investigations where digital evidence is main source of data. However, the forensic computing perspective moves beyond these technical skills to develop sensitivity towards questions over the admissibility of evidence and legal validity of particular data sets” (Turner 2002). Therefore, from forensic perspective the log files need to be valid and admissible.
IDS logs have definitely got evidentiary value provided that the IDS have not been compromised at the system level. IDS logs fall into the category of documentary evidence. But there are debates about this. “The issues aligned to evidence, acquisition and the suitability of Intrusion Detection Systems (IDS) for preparing legally admissible evidence, reveals strong disagreement amongst technical and legal experts over the suitability of IDS as a tool for collecting, collating and presenting forensic evidence” (Turner 2002). There are some reasons behind this. The difference in legal systems has a lot to do with this debate. “In Continental Europe the criminal procedure sees investigations being carried out by a specialist judge – juge d’instruction – in countries like England, the US, Australia and many former members of the old British Empire, investigations are carried out by the police or other law enforcement agency, the decision to prosecute is made by a separate body -District Attorney in the US, Crown Prosecution Service in England, and at trial the role of the judge is as chairman of the proceedings and enunciator of law. Separate opposing legal teams represent the arguments of prosecution (the Crown, the People) and the defence. The trier of fact is a jury. The procedure, known as adversarial, has lead to the development of complex rules of evidence, describing what can and cannot be put before the court for its consideration of fact” Sommer (1998,1999). This is a fact that places a lot of challenge in front the network security and forensic investigator community. It also makes it difficult for the police and other organizations to prosecute criminals involved in an attacks or intrusions. The need for understanding the technical details can be well carried out by a specialist judge as in Europe. Juries/ judges/lawyers however, have little knowledge and understanding on technical matters. This makes the cases involving technical matters really challenging. IDS logs are generally recognised means of investigation based on a network /system traffic and they are potential legal proofs. Turner (2002) says “Admissibility and weight are the legal validity of evidence for a submission in a particular jurisdiction and the ability of the court to be convinced by its presentation”. Therefore, there is a need for the legal system to set a baseline standard on the admissibility of evidence and the potential use of that evidence as legal proof. “The use of cyber-based evidence is becoming more important, and there is no reason to suppose that law enforcement agencies would not consider IDS logs as a potential source of cyber-based evidence” (Johnston).
Strengths and weaknesses
There are significant strengths and weaknesses of IDS tools that are available. The first worry is “The intrusion detection systems are themselves susceptible to a variety of attacks and some authors argue that the majority of these systems are fundamentally flawed. In other words, the data collected by these systems may itself have been tampered with before the attack was discovered and/or investigated.” This is a very important issue because the strength and the advantage of using IDS will be demolished if it gets invaded or compromised. The possibility of having a 100% secured IDS is arguable but the focus should go on getting to build these system in such a way that that can be of value to the organization or agency. There is another problem with the IDS’s today which is their ability to cope with high volume of traffic and having high-speed connection backbones.
Lot of the products is still not capable of coping with volume of traffic and processing of the packets. Therefore, it is still a challenge but research is going on to improve the performance of the IDS to cope with the future bandwidth requirements. The IDS log files are not always are able to prove the point of the start and end of a specific conduct. Therefore, they often lack sufficient details. Some IDS log files will have limited capability of recording all real time events because of huge traffic flows. Often the IDS log files will not distinguish the legitimate and unwanted traffic. Therefore, follow up and review can become tedious. There are other situations where IDS log files might fail to identify the intruders and have been tempered or altered. All these situations are very common with log files. However, still they are potential documentary evidence. The logs are bit and pieces of the big scenario. Given the challenge of detecting an incident, when the system log, firewall logs, IDS logs, application logs and all the other log information are combined together and haven’t been tampered, security and forensic experts will be able to identify the potential point of incident. Let’s not forge that IDS are serving our main purposes of real time analysis of network and host devices and they also capture and monitor the packets that arrive in the network. They provide fault tolerance and contribute to the overall security measures of an organization.
There remains a significant challenge in making the IDS devices to work as attack/ intrusion blocking systems that is why the intrusion prevention system has come up. They act as a detection system to attacks and intrusions but the alerts is generated after the attack. There are problems with IDS false positive and false negatives as well. The challenge still remains in handling encrypted data and OS specific application protocols. Signature based IDS need regular updating of their signature database this is still an issue but we have to deal with it. Privacy concern and legislations make it impossible to gather data for analysis and prosecution most of the time. There are significant things to argue about when we talk about privacy and legal aspects. Which will follow shortly.
Point of Privacy violation
The legislation does not clearly state what points are considered as privacy violations. Looking at packet headers, looking at packet data or something else. “According to Johnston (2002) “United States Code (criminal law) contains provisions for the use of pen registers and trap and trace devices. These devices are capable of monitoring and identifying the specific phone numbers dialed from a particular telephone line – they do not capture or record the content of any such communication. Certain legislation distinguishes between traffic data and content implies that traffic data is not considered to be private communications.” If that is the case then there should not be any problem when IDS are used to analyze traffic data for monitoring purposes. When it comes to the right to protect the information system and respect privacy laws, there should not be any distinction between government agencies and private/public sector agencies. In Australia however, the legislators have enabled ” Certain government agencies to intercept telecommunications services on the basis of a telecommunications devices. A further issue is how the law distinguishes between stored and real-time communications such as telephone conversations” Bartlett (2006). The point I’m trying to make is that “IDS are not sufficiently discriminating to distinguish between malicious activity (which should be monitored and logged) and benign activity (the privacy of which should be respected).” However, to determine whether a packet is malicious or not the header information (public information) of a packet is not enough for the IDS. Therefore, IDS do need to look into/intercept the packet.
Laws of Interception in Australia
Under the telecommunication interception law in Australia there are some ambiguous issues that should have been resolved but hasn’t so far. The recent amendment this year has enabled government agencies like ATO, ASIC, Customs to intercept private communications of innocent people who may not be involved in a crime. Even though the intercepted evidence may be relevant the government agencies have the right to use that information. Not only that. The recent amendment “creates the potential for Government agencies to misuse the power or apply it in an arbitrary fashion. This arbitrariness extends to another aspect of the Bill. It differentiates between stored communications, such as email and SMS, and real-time communications, such as telephone conversations. Under the Bill, it is much easier to access stored communications, apparently because SMS and email are thought to be less private than phone conversations” Williams G. (2006).
The question remains that why is no the government concerned about the other public and private sector organizations. The legislators are giving immense power to some government agencies where as the other agencies are not considered and neglected when it comes to interception issues. There is significant debate about handing over too much power to these government agencies but that is not the issue here. There has been some media releases regarding this topic but no one have ever mentioned about the interception right of other organizations and agencies. The private/public sector organizations and agencies have right to protect their IT infrastructure and therefore they need to intercept for security reasons. Where as on the other had some of the government agencies will be retriving and using intercepted information that is irrelevant.
“Besides Security and Intelligence (S&I) agencies and Federal Police, Public and private sector organizations also have a requirement to intercept and examine network traffic. While these organizations may need to do this for intelligence or evidentiary purposes, most such interception would likely be conducted by law enforcement or S&I agencies. Instead, interception performed by these organizations is more commonly done in order to identify an attempted or actual intrusion into a protected system or network, and to initiate an incident response process (of course, this also applies to law enforcement and S&I agencies). While existing legal regimes adequately address interception by S&I and law enforcement agencies, they generally do not adequately address interception of network traffic by other organizations” Johnston (2002). The government and legislation makers specially need to understand that “intelligence function is no longer the exclusive domain of ‘national security’ agencies – private sector organizations need to generate competitive business intelligence” Johnson (2002). Technological progress today in the IT sector is astonishing. Yet the progress of developing new IDS tools and techniques are still not that of news in the headlines. This is due to the negligent response of government legislation makers towards interception and IDS usage. Private agencies are kind of hopeless without a specific legislative basis for the conduct of intrusion detection.
Facts and findings
Johnston (2002) “The use of IDS may constitute an interception as defined in criminal law, and that the existing exemptions did not adequately address the interception of private communications for network protection purposes. Johnston (2002) also added ” Prior to the introduction of anti-terrorism legislation, the interception of private communications was conducted in accordance with strict rules. Specific conditions had to be met before an authorization could be granted, and strict conditions applied to the actual conduct of the interception. Interception was also generally limited to telephone communications”. This is a fact and it is true that for anti-terrorism purpose the government has desperately passed a bill that provided too much power to Government agencies. Williams G. (2002) says “This law goes too far. It contains more power to access our emails and text messages than is needed and contains too few safeguards. Rather than rushing the law through Parliament, the Government should have listened to the report of its members. It should have come up with a law that better protects the private communications of innocent people”. The recent ammedement did not satisfy parliament members nor does it satisfy the general community of people who are dependent on these legilations a lot. Private/public organizations, security professuionals, forensic exminers are looking forward towards th elegislator community to enable them to legally obtain information and safeguard their IT environment.
The potential of using log files, as evidence is there. It is not impossible as long as the admissibility and weight restrictions are meet. The legal aspect and value of using IDS logs as forensic evidence will change if the legislation allows the companies to legally intercept communications. The companies and the media do not bring this issue too loud in public because the legislation here in Australia is always in the way. Parliament members and responsible politicians might find this problem interesting. But the fact is the legislation limits the use of IDS logs as forensic evidence and the legislations need to change in order to use the logs as evidence.
1) Sommer P. Intrusion detection systems as evidence Retrieved 9/9/2006 from
2) Bace1 R. and Mell P. Intrusion detection systems Retrieved 11/9/2006 from
3) Johnston S. Development of a legal framework for intrusion detection Retrieved 19/9/2006 from
4) Broucek V, Turner P. Bridging the divide: rising awareness of forensic issues amongst systems administrators Retrieved 9/9/2006 from
5) Lebihan R. Anti-terrorist law: australia pushes e-mail interception Retrieved 29/9/2006 from
6) Bartlett A. The week ahead 4 – changes to telecommunications interception laws updated
Retrieved 9/10/2006 from http://www.andrewbartlett.com/blog/?P=183
7) Williams G. & Hume D. Someone else might be listening Retrieved 9/10/2006 from
8) Inquiry into the provisions of the telecommunications (interception) amendment bill 2006 Retrieved 1/11/2006 from
9) Williams G. and Hume D. More than ever, watch what you say Retrieved 13/10/2006 from
Fahmid Imtiaz 2006.