Regulation

ISO 17025 For Digital Forensics – Yay Or Nay

by Robert Merriott

“Much of the digital forensic community desires to have their evidence seen in court as forensically sound and bulletproof, yet do not want to go through the rigors that other traditional forensic sciences have done to prevent evidence spoliation and other mishandling and misinterpretations.”
~ Josh Moulin, Deputy Chief Information Officer, US Federal Government, National

As many forensic experts will already know, ISO 17025 is now the mandatory standard in the United Kingdom (UK) for all Digital Forensic laboratories as of October 2017.

Any digital forensic laboratory in the UK which isn’t ISO 17025 certified will be required to declare itself as “non-compliant” on each issued report.

2.1 Declaration of Non-Compliance
Failure to comply with the Regulator’s standards could significantly detract from the credibility of a forensic science professional, particularly when acting as an expert witness, and/or have a bearing on reliability…
~ Source: FSR – Annual Report

The question of the day is:

Will the digital forensics community in the United States, Canada, Australia and elsewhere adopt ISO 17025, another standard, or wait for one to be imposed on them?

To find the best solution for those working outside the UK, this article calls for an in-depth analysis of IS0 17025, and everything it represents. It also requires input from those working in ISO 17025 labs in the UK and other countries to help the rest of us better understand the reasons for and against this accreditation.

The Forensic Focus forum has many experts in the UK and elsewhere who are working in ISO-accredited labs, so this is the perfect place for such a discussion.

It’s possible that the ISO 17025 will continue to be adopted around the world and become the defacto standard for digital forensics. ISO 17025 has been met with mixed opinions within the digital forensic community; some digital forensic examiners (DFE) have expressed their support and approval, while others have declared this standard completely unsuitable to the field of digital forensics.

What is your opinion? I encourage you to read the following information and then continue the discussion in the forums.

The last thing we want to happen is for this accreditation to become the standard throughout the world if it’s not a good fit for digital forensics. But unless our community works towards the correct solution, we will likely be left with a standard that is forced upon us by outside regulators.

The following article is a summary and update of the original article “ISO 17025 – Right for Digital Forensics?”.

A Little About ISO 17025

Published in 1999, ISO 17025 endeavours to standardize laboratories worldwide in terms of testing, quality control, calibration; and to ensure that results generated by any accredited lab are reliably valid. Any laboratories hoping to become ISO 17025 accredited require assessment by official national accreditation bodies who evaluate the labs for conformity. The “methods of evaluation” for such assessments have been developed by the International Laboratory Accreditation Cooperation (ILAC).

The ISO 17025 standard involves adherence to standards in Scope, Normative Resources, Technical Requirements, Management Requirements, and Terms and Definitions. Areas which need to be addressed in order to comply with ISO 17025 include competence of staff, testing and calibration standards, standard of equipment, and quality management. Some North American forensic experts are of the opinion that it is best for digital forensic laboratories to become ISO 17025 certified.

The main reason for this support is because currently there are no other international standards in place for digital forensics and they believe that ISO 17025 best fits this profession, as discussed in a conference paper on ResearchGate.

Without national mandated standardization of all digital forensics labs, there remains a certain extent of inconsistency between the outputs and processes of the laboratories and the level of training and expertise possessed by the digital forensic examiners within those labs.

The Forensic Science Regulator in the UK, Dr. Gillian Tully, shared this belief and was responsible for mandating ISO 17025 within the UK.

Update ISO 17025:2017

The update to ISO 17025:2005 is referred to as ISO 17025:2017 or ISO 17025:20xx.  This standard was released at the end of 2017. It appears that only minor amendments to ISO 17025:2005 were made. The 2005 version was a revision of the original standard introduced in 1999.  Laboratories that are already accredited will have 3 years to transition to this new 2017 standard.

ISO 17025 for Digital Forensics – Yay or Nay?

ISO has been met with resistance and backlash by many digital forensics examiners. Although it endeavours to create a positively reinforced standardization and validation of lab results, many are opposed to the idea of ISO 17025 because of a number of concerns. This feedback was obtained via an early 2017 UK Survey by Pat Beardmore, Geoff Fellows, and Peter Sommer.

This survey was conducted because of the growing concerns among practitioners regarding the impact of planned regulation of forensic science services in general within the United Kingdom. Here are some of the prominent concerns which were revealed by this survey, and through discussions taking place within online forums.

Concerns about ISO 17025

The Costs

It has been a matter of considerable debate that ISO 17025 may be undesirable due to the excessive costs involved for the practitioners in upgrading and realigning their laboratory and practices to be ISO 17025 certified.

Digital forensic examiners have expressed concerns about whether their smaller laboratories and companies can afford to comply with the extra costs. However, the survey revealed that of those concerned, the majority of practitioners (70.8%) were unaware of the actual costs involved in becoming ISO 17025 accredited.

In addition to those numbers, 14% of practitioners believed that the costs involved in ISO 17025 standardization ranged between £20,000-£50,000, with 15% further thinking that the costs exceeded £50,000. These survey results indicate that a large proportion of practitioners who are in opposition of ISO accreditation are largely unaware of the actual costs involved in the process. Therefore, it is possible that their opposition is based in fear of the unknown, perhaps as a result of lack of research and/or past experiences with standardization.

Lack of Understanding

Another result from this survey which indicated that the opposition may be due to fear of the unknown was that majority of participants did not have a complete understanding of ISO 17025.

Even of the practitioners surveyed in the UK, a country where practitioners were actively preparing for mandatory standardization, not even a quarter (less than 25%) themselves believed that they had a high or clear understanding of the details involved in ISO 17025.

On the other hand, almost half (47.7%) of the participants believed they had a “reasonably good” understanding of ISO 17025, whereas 25% admitted to a poor understanding of it.

With approximately 73% of the DFE community not having a well-informed understanding of ISO 17025, is it prudent to shun it rather than to first allow said practitioners to develop a more in-depth understanding of this standardization?

Poor Implementation

Discussions on online forums dedicated to forensics professionals such as Forensic Focus, as well as general Twitter discussions, revealed that one of the foremost reservations of professionals considering ISO 17025 accreditation was the chaotic impact of poor implementation.

Some practitioners observed the poor and mismanaged way in which ISO 17025 had been implemented in the UK. They felt that ISO 17025 should have been implemented in the UK from the top (uniformed governmental level).

Others said that digital forensics examiners in the Unites States and elsewhere should learn from UK’s “mistake” of implementing mandatory standardization of ISO 17025, because a number of experienced self-employed DFEs have been forced out of business in the process of becoming accredited due to cost and time-related issues.

Impact of Inconsistency

One of the main concerns voiced by digital forensics practitioners is regarding the fact that while ISO 17025 necessitates straightforward compliance with a set of policies, the interpretation of these ISO standards for digital forensics are not so straightforward.

A gathering of DF industry leaders would be required to collaborate to devise a standard other than ISO 17025, or to direct a sole interpreted set of policies via ISO 17025 to be implemented uniformly across all labs in the DF industry.

This lack of uniformity and understanding results in further disagreements among experts is causing the rest of us to be more confused and push harder against accreditation.

In addition, a number of digital forensics practitioners also strongly felt that ISO 17025 was not the solution to the problems in the digital forensics industry, and its standardization processes were not suited to the processes and practices of DF.

Online Podcast Discussion

Phill Moore discusses requirements for regulation within the DFIR industry in his “This Month in 4N6 – November 2017” podcast. Phill sees the DFIR community either pushing forward and saying we need it or the legal entities saying that the DF industry needs to have some sort of regulation.

Phill refers to a recent blog post by Brett Shavers which discusses regulations within the Digital Forensics community suggesting that a lot of labs and examiners are probably going to push back from regulations. Phill suggests that one of the benefits of implementing standards is that you are not going to have “Joe IT guy rockin’ up to court with his expert report”.

Support for ISO 17025

Standards

The necessity for uniform standardization is, however, more apparent than ever. Without proper standardization, forensic labs’ and examiners’ credibility may be questionable within the courts in the years to come.

“Credibility” of digital forensic examiners and their accredited laboratory-issued results will continue to have greater scrutiny in high-profile cases.

This is especially true in the events where examiners do not have prior proper training or expertise to complete a complex digital forensic examination. Without accreditation, it is not uncommon for examiners in smaller offices with only basic training to be required to work on high-profile investigations.

The Deputy Chief Information Officer of the US Federal Government echoes these thoughts in his thesis paper Dishevelled Digital Forensics: The Impact of Inconsistent Standards, Certifications, and Accreditation.

“The lack of requirements for digital forensic practitioners to be certified in their discipline, be accountable to industry best practices and standards, or work out of accredited laboratories places the credibility of this forensic science in jeopardy.”
Josh Moulin, Deputy Chief Information Officer, US Federal Government, National Security

Consistency

Most practitioners in the industry believe that ISO 17025 standardization and accreditation will result in an overall improvement in the quality of results generated from the certified labs, as consistency of quality is the principle objective of ISO 17025.

Consistent and uniform standard of quality, even if it isn’t superior in quality than prior to implementing accreditation, at least sets the minimum quality standards which need to be adhered to.

Consequently, if all DF labs of a region were to become ISO 17025 accredited, they would, at the very least, start on a level playing field, and set a minimum quality bar below which standards would no longer fall.

This is exemplified by the following excerpt from a discussion thread on the Forensic Focus forum:

When all laboratories of a region produce results, which all follow a set standard, it results in an overall elevation of digital forensic standards and lab results as a whole. It is important to recognize that accreditation does not mean that certified laboratories would never make mistakes.

ISO 17025 would denote that, rather than laboratories utilizing the best possible practices for each case, they would simply adhere to an established set of standards of quality, and rely on acceptable, approved practices within the given requirements.

Further Reservations

ISO 17025 – Not the “Be All End All”

Laboratory accreditation according to practices such as ISO 17025 do serve to remove reservations regarding the lab’s result quality and personnel competence assurance.

However, as Josh Moulin states, accreditation is not the one-stop magical solution which will solve all the problems plaguing the professional field of digital forensics.

“Having a laboratory accredited according to best practices such as ISO 17025 removes many questions about the quality assurance of the laboratory and the personnel performing work. Accreditation is not the be-all and end-all or a magic solution to issues plaguing the digital forensic discipline.”
Josh Moulin, Deputy Chief Information Officer, US Federal Government, National Security

Accredited laboratories are not exempt from issues arising with their findings after being certified. The only difference is that accredited laboratories are easily able to identify and correct mistakes.

In Conclusion

I believe it is undeniable that some level of accreditation is required for the digital forensics community, however it remains a matter of debate whether that accreditation should be ISO 17025 as mandated within the UK.

Of the discussions held among members of the digital forensics community thus far, the recommendation I believe has the most potential is the one suggested by Brett Shavers. Brett suggests that front-facing DFIR organizations such as HTCIA, IACIS etc. should cooperate on a model for standards and implementation.

In this way, each state-specific DFIR organization would comply with their respective state regulatory agencies.

With the DFIR organizations making efforts to push the board to cooperate with their adjacent organizations’ members, communications would be streamlined, and North America would come one step closer to making a unanimous, customized and ideally suited decision regarding the matter of standardized accreditation. Other countries could follow a similar path to accreditation.

If we as a community are not willing to discuss this topic and come to a consensus, we will be forced to adhere to a standard that we may not agree with or be able to work within.

What are your thoughts on accreditation within the digital forensics field?

Do you support the above recommendation or do you have other ideas that would streamline this process?

About The Author

Robert Merriott is the founder of Forensic Notes. Forensic Notes was specifically designed for the digital forensics community to speed up and increase the quality of notes during an investigation, ensuring you have credible contemporaneous notes that will withstand court scrutiny.

Robert has over 12 years of experience as a Municipal Police Officer, most recently working as a Digital Forensics Examiner. Prior to his policing career, Robert obtained a Bachelor of Science in Computer Information Systems and worked in the private sector as a web application developer.

NOTE: The ideas and opinions presented in this article do not reflect the policies, procedures, and regulations of the author’s agency.

About scar

Scar de Courcier is an assistant editor at Forensic Focus.

Discussion

10 thoughts on “ISO 17025 For Digital Forensics – Yay Or Nay

  1. Would be worth asking why the 27000 series wasn’t considered instead of 17025 as this actually has some relevance to Digital Investigations

    Posted by minime2k9 | January 24, 2018, 11:06 am
    • It basically boils down to accreditation verses certification. You cannot accredit to the 27000 series, you can only get certified and if you are familiar with the differences between accreditation and certification you understand why.

      Posted by Clayton D Schilling | July 15, 2018, 3:53 pm
  2. ISO 17025 is not yet fully mandated here in the UK though the Forensic Science Regulator would like it to be. The current position can be seen from the FSR’s most recent Annual Report: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/674761/FSRAnnual_Report_2017_v1_01.pdf. Concerns here in the UK about the value of the standard to digital forensics are as your article states.

    Posted by Peter Sommer | January 24, 2018, 12:47 pm
  3. I’m going to overly simplify the issue but it gets the grey matter working:

    If you are accused of a crime, who would you rather use as your defence expert?

    Someone who has decades of experience, given evidence at the highest level, respected by their peers, lectures at Uni, published research etc etc (but not 17025)

    Or someone who has just left University with a degree in the topic (and has 17025),

    I know the two are not mutually exclusive but we need to focus on what we mean by quality and what we mean by administrative consistency. There is a real danger that we see 17025 as a route to quality.and I struggle to see this.

    Posted by Patb | January 29, 2018, 11:19 am
    • Two completely different topics (i.e., practitioner vs. organization). People do not get accredited, they get certified. In your example, the experienced practitioner could be highly respected in either scenario, but in my experience they courts have given higher credit to an expert who operates within an accredited system.

      Posted by Clayton D Schilling | July 15, 2018, 4:00 pm
  4. ISO 17025 is designed for laboratory activity; some aspects of digital forensics, particularly the essential early stage of acquisition/preservation, are lab activities. But thereafter much of DF work consists of interpretation – chronologies of events, observations of patterns of behaviour. Other aspects include the ability to research and report on artefacts within computer operating systems and programs. In the UK these are challenged by defence experts – and the Criminal Procedure Rules require the prosecution expert report to set out in detail all its arguments, activities and conclusions so that there can be proper peer review.
    These latter facets are surely better addressed by accrediting the individual expert, not the laboratory. The UK did have such a scheme; it was not perfect but instead of seeking to improve it, the UK government summarily abandoned it. Interestingly, many of the ideas of the UK scheme were later adopted in the Netherlands, though it must be admitted that criminal justice procedures there are different.

    Posted by Peter Sommer | February 1, 2018, 7:58 am
  5. Reblogged this on Precision Digital Forensics, Inc. and commented:
    Great read about the ongoing issues with ISO accreditation and a demonstration of the continuing confusion and misunderstandings that exist in the community.

    Posted by Clayton D Schilling | July 15, 2018, 3:35 pm
    • I have written a commentary on this topic for Digital Investigation; it’s behind a paywall for most of you I guess but a pre-print is available at: https://goo.gl/ynFUDd. I argue that there is no single simple method for re-assuring our customers of the quality of our work. We probably need a combination: accreditation of labs and processes, certification of individuals, reliance on court procedures for testting expert evidence, good practice guides.

      Posted by Professor Peter Sommer | July 17, 2018, 9:57 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 1,112 other followers

%d bloggers like this: