An Evaluation of Linux Cybercrime Forensics Courses for European Law Enforcement - Paul Stephens

Sixth International Symposium on Human Aspects of Information Security & Assurance (HAISA 2012)

     An Evaluation of Linux Cybercrime
    Forensics Courses for European Law
               Enforcement
                     Paul Stephens
                paul.stephens@bcs.org
            paul.stephens@canterbury.ac.uk

1

Harmonisation of Computer
    Forensics Investigation Training
                              Falcone
          (Law Enforcement, Trainers, Academics, Industry
        Professionals – Duplication of effort & Fragmentation)

                                Agis
                  (Courses Developed at PG Level)

                               ISEC
            (Courses Developed into an MSc Programme)

2

Funded by
    l European Commission

    l An Garda Siochana

    l National Policing Improvement
       Agency

    l Landesamt für Ausbildung,
       Fortbildung und
       Personalangelegenheiten der Polizei
       NRW
3

Project Partners

4

Agis Courses Developed
    2003-2004 (Basic)

    • Introductory IT Forensics and Network Investigations

    2005-2006 (Intermediate)

    • Applied NTFS Forensics
    • Intermediate Internet Investigations
    • Intermediate Network Investigations

    2006-2008 (Advanced)

    • Linux as an Investigative Tool
    • Mobile Phone Forensics
    • Wireless LANs and VOIP

5

ISEC Developments
                              (2008-2011)

    New Courses funded by EC/Partners

    • Forensic Scripting Using Bash
    • Malware Analysis & Investigations
    • Live Data Forensics

    Update all Agis courses

    • Seven courses in all
    • Funded by

    Full run of the MSc

    • Initially accredited by University College Dublin

6

Development of Linux as and Investigative Tool and
      Forensic Scripting Using Bash Modules

                    Law enforcement,
                 Academics, and Industry
                      Professionals

    Suggested course titles and discussions/presentations

7
                  Courses to be developed

Individual Course Development

    	
  	
                    	
  	
                     	
  	
                	
  	
                	
  	
  
    	
  	
                    	
  	
                     	
  	
                	
  	
                	
  	
  
    	
  	
                    	
  	
                     	
  	
                	
  	
                	
  	
  
    	
  	
                    	
  	
                     	
  	
                	
  	
                	
  	
  

                   	
  	
                	
  	
     	
  	
          	
  	
                	
  	
  
                   	
  	
                	
  	
     	
  	
          	
  	
                	
  	
  
                   	
  	
                	
  	
     	
  	
          	
  	
                	
  	
  
                   	
  	
                	
  	
     	
  	
          	
  	
                	
  	
  

8

Linux as an Investigative Tool
     (Proposed Timetable following pilot in 2007 – Week One)

9

Linux as an Investigative Tool
     (Proposed Timetable following pilot in 2007 – Week Two)

10

Week 1: Online Element
     l Cost issues
        l Conceived as a one week course
        l Break required between week one
           and week two
        l Pilot for all courses of MSc scheme
           lWiden availability of training
     l Difficult to find instructors

11

Issues Unique to this Course
            Development (1)
     l Course development team are truly
        international
        l Division of materials
        l Use of Moodle server
     l Students are also international
        l English skills
     l Hardware/Software difficulties
        l Online support
12

Issues Unique to this Course
            Development (2)
     l Content of the course
       l Linux is notoriously difficult to learn
          and to teach
       l Linux does not work quite the way a
          Windows user expects
       l Command Line Interface (CLI)

13

Formal Evaluations
l Kirkpatrick Model
   l Level 1: Reaction
      lHappy Sheets
   l Level 2: Learning
      lStudent Assessment
   l Level 3/4: Behaviour/Results
      lStudent Learning Journals/Manager Feedback

Level 1: Reaction
Aggregate Rating for Overall Session Grading for MSc run of Linux as an Investigative Tool

   15

Level 2: Learning
      Results for MSc run of MSc run of Linux as an Investigative Tool

     l Two students out of the 28 that sat the
        course failed the final assessment
     l Overall the student average was 80%
     l All passed on resit

16

Level 3/4: Behaviour/Results
     l Student Quote:
        l “The course itself was excellently presented, I found
           the subject matter fascinating, and I am utilising my
           knowledge in the workplace already. I have spent the
           last few days stripping out IP/time data from a 900MB
           text document containing compromised data using
           Linux, … it is most definitely not something I could
           have achieved prior to this course.”

17

Level 3/4: Behaviour/Results
     l Student Quote:
        l “I have learned to convert a DD image to another
           evidence file format to suit the tools I’m using such as
           EnCase. This is only one example of how what I have
           learned can be used to my advantage, other examples
           include extracting metadata from images and using
           the file system to undelete files.”
     l Managers’ feedback for the MSc as a whole was positive
        and encouraging

18

Forensic Scripting Using Bash
      (Timetable Indicating Course Content)

19

Level 1: Reaction
Aggregate Rating for Overall Session Grading for Pilot of Forensic Scripting Using Bash

      20

Level 2: Learning
           Results for Pilot of Forensic Scripting Using Bash

     l Approximately one-third of students
        failed the pre-course assessment and
        five (out of 20) students went on to fail
        the course assessment at the end
     l The pre-course assessment was
        therefore indicative of the number of
        students that would fail the course
     l Overall the student average was 58%

21

Level 1: Reaction
Aggregate Rating for the Structure and Method of Delivery for the MSc Run of Forensic Scripting Using Bash

     22

Level 1: Reaction
Aggregate Rating for the Level of Student Understanding for the MSc Run of Forensic Scripting Using Bash

     23

Student Concerns About
            Subject Matter
     l Some students were concerned about the
        difficulty of the subject matter. It is worth
        noting that trainers put a lot of work into the
        course including outside of the classroom as
        shown by the following comment:
        l “Working through the exercises in the
           evening is very beneficial as is the availability
           of the trainers for that time. Much
           appreciated.”

24

Level 2: Learning
          Results for MSc run of Forensic Scripting Using Bash

     l Five students out of 28 failed the end of
        course assessment worth 50%
       l Passed on resit
     l The overall student average for the test
        element was 68%
     l All students passed the other 50%
        element for which the average mark was
        78%
25

Level 3/4: Behaviour/Results
     l Manager Quote:
        l “has also developed different useful forensics tools
           and software packages that are used by all members
           of the unit.”
     l Student Quote:
        l “The web spider we have learnt is incredibly valuable
           for our work of monitoring…websites. We were highly
           surprised when we saw how easy is with a non very
           long script, to have a real time monitoring system to
           display all the changes in a website”

26

Level 3/4: Behaviour/Results
     l Student Quote:
        l “From a confidence point of view, the last few ‘Linux’
           months, and in particular the scripting course and
           post‐course assignment, having proven to be
           invaluable. … Over the last three years, I have been
           constantly mindful of the expertise that surrounds me,
           the knowledge that my colleagues have acquired over
           many years of hard work, and for which I feel I can
           only ever aspire to. Having completed my script, I was
           asked by two of the most experienced colleagues if I
           would provide them with a copy of my script, as they
           wished to look at it and learn from it. I am still in shock
           that I am seen as somewhat of a relative ‘expert’ on
           this subject!”
27

Conclusions
     l Unique Development and Delivery
     l Management and Development Models
     l The courses worked (with plenty of hard
        work from staff and students) and the
        data we have (I believe!) shows this
     l But, presenting this data in a
        presentation/paper is problematic! (for
        me!)
28

Questions and Suggestions?

            paul.stephens@bcs.org
        paul.stephens@canterbury.ac.uk
29

Thank You!
     Shameless Advertising:
     l The 6th International Conference on
        Cybercrime Forensics Education &
        Training (CFET 2012)
     l 6th & 7th of September 2012
     l Canterbury Christ Church University, UK

30