Forensic Image and Video Analysis: The complete workflow.





Step 1. Retrieve the data: get the bits!

This step may be most important and can be very complex and critical for evidence integrity.  Unfortunately, this is generally not done by us, or by those who know to take the care to document the steps.  Worst case (which seems to be the most common for police investigators), the original evidence is gathered by a patrol officer who is the first on the scene.  A low-res copy of an original is output to a disk or thumb-drive, and the original DVR is left to be copied over by the next day’s footage.  This happens far too frequently, and gets everyone off to a pretty bad start.

Another way we may get the data is if it comes from our colleagues on the digital forensics side of things. If they are meticulous and take the steps necessary to document their process, we can be in a good position to start.  It seems that often we start working on a video burned on DVD or on some images we receive by email, however we need to be conscious and document the previous steps, which may be, for example:

  • Analyze a disk image for complete and deleted files
  • Export a video from a DVR
  • Copy a file from an hard disk


For police investigators working in the real world case of evidence gathered by a non-scientific responder, with a little better communication, training, and diplomacy; the situation can be improved.  Let them know what you need them to gather, document, and what instructions to leave with the video source (like don’t copy over the DVR drive!!!) At the very least have them know to call you if they have any questions.

Step 2. Decoding the data: what are these bits?

Now that we have the bits, we must get something from them to analyze. If we have some plain video files, it is not a big problem, given we have the proper codecs. Since there are hundreds of proprietary video files out in the wild, that in itself may turn into a challenge.  To determine which approach we take, we need to know what we are looking at.  This step generally defines the challenges that we may be facing:

  • We have a disk image and we need to reconstruct images and videos, even the ones that may have been deleted (this is common in child pornography cases, for example).
  • We have a dump of a DVR drive and don’t know how the data is encoded.

With this step, we may need to do some research to really define what we need to complete the job.  Do we need to focus first on data recovery?  Do we need to find a better decoder?  Do we need to find a better player for the video or means to export it properly?



At this step we should be able to view the videos or the images, but we need to find the right ones! Two examples of what we may face may be:

  • Finding images of interest in a large database

This step can be helped with communication from other team member working on this case.  For most cases, the basic thing we need to understand is: what happened, and when did it happen? And of course, technology may help too, with technologies such as video content analysis and face recognition.


Depending on the situation we may need to understand how the original files have been generated. With some generalization this may be called image ballistics. Understanding the type of file and the source can help us understand several things in a case. Some analysis that may be done could be:

  • Identify the camera model used for taking the picture
  • Identify the specific device that has taken the picture

We need to document the source so that we can maintain the integrity of our evidence.  This will help us if we have to go to court later.


At this point we may be interested in understanding if we can trust the data we have gathered.  Is there a probability that someone altered it?  This can be done on various levels.

  • Verify if the file the file has been manipulated, for example altering the metadata
  • Verify if the image has been manipulated, for example converting the format, resizing or cropping it
  • Verify if the content has been manipulated, for example removing or adding a subject

Tampering is becoming more commonplace.  It can be done innocently (like converting formats from original to a low-res media file) or purposeful “photoshopping” to manipulate facts.  In this digital age, it is something that we need to address.  This is why anyone who has ever spoken to anybody here at Amped will attest to something that we always emphasize: Work with an original if possible.

Step 6. Estimate the quality of the data: do we have enough bits?

At this point we can see something in the image, but we must understand if the quality is enough for our purposes. For example, if we see a car, are able to read the license plate?  Or if we have a face, do we have enough pixels for a reliable identification?

  • Does the image effectively contains the information we need (e.g. the license plate has enough pixels)?

Technical knowledge and experience is very important to estimate quickly if we have enough quality or not. It is not always easy to estimate the minimum quality to get useful results.  A shortcut with things like faces and license plates is zooming in and counting pixels.  If you only have six or eight pixels to draw all the characters in a license plate, the probability of success is pretty low here.


Once we have identified the problems affecting the images or videos, given the right tools (Amped FIVE, obviously), we can enhance and restore the data. This step is actually pretty vast, and can involve processes like:

  • Image restoration techniques: understand the mathematical model of a known disturb and try to invert the model to recover the image without the defect (deblurring, Fourier filtering, frame integration…)

For us hardcore video enhancement people, this is the most fun part of the process.  It does involve a bit trial and error, but it is at least the fun part.  While no one can guarantee the great results found with Hollywood magic on the CSI shows, we can often see some amazing results.

One thing that is really important to remember in this step is documenting the enhancement process.  Amped Five Professional and On Demand do this automatically, so you have documentation to take to court.  General purpose software does not usually provide enough tools for this issue.


This is where you see what you have gained.  The enhancement step would be useless if there’s no improvement to the content of the image so we can understand and classify it. In this step you can do the following:

  • Compare a face in two different images
  • Compare a face with a known subject
  • Read the license of a vehicle
  • Identify the place where a picture is take
  • Measure the height of a subject
  • Find the corresponding fingerprints in a database

If you don’t get the results you need, you can go back and repeat the steps until you do; or you determine that you can’t get what you need with the data provided.  Again, this is a scientific process and should be left without emotion.  I know it is too easy to get bent out of shape over a ton of work without results; but that is sometimes the cards that are dealt.

Step 9. Validation: did I get the right bits?

Validation isn’t just focused on the quality of the result.  It is also about the quality of the process used to gain the result.  We must always maintain that the techniques used must be valid both from the scientific point of view and follow a procedural set of standards accepted by the courts that have jurisdiction over what we are doing.  This is extremely important for the verification of image integrity and for documenting the enhancement workflow.

A few things to consider are:

  • The results must be scientific and repeatable.
  • A detailed audit trail must be kept to explain how we go from the original image to the enhanced one.

This may be seen (and actually is) as manipulation of evidence, and thus we must be able to justify it properly from the scientific point of view.  In this case, documentation is key.

Step 10. Presentation: I’ll show you the right bits!

Getting the results it’s not enough. You must explain them to the court and the jury: you must be able to make them understand and accept the techniques you used. Scientists, engineers, attorneys and normal people speak different languages. It is really important to organize any facts you present with the idea that you must explain the terms you use in your context, be clear and open.


The key to overcoming this type of questioning is to document your workflow and stick to the science. Remember I mentioned organization?  Again, I mention that this is scientific and a scientific process; but it has to be explained to non-scientific people.   Can you present the facts of the case and explain the science in plain language?  If you are organized, clear, and concise this will help.  At the very minimum, you need to show:

  • The original.
  • Where it came from.
  • How you got it.
  • What steps were taken to get the result.
  • How that result relates to the case.
  • What scientific methods were used to validate the process.


The bottom line

You may not use all of these steps in all cases.  Often, you may work as a team and only concentrate on a couple of them.  Again, the point to drag away from this whole outline of workflow is organization and methodology of your process.

Depending on what we are working with, we may not formally define these steps so clearly.  Critical and systematic thinking are really the foundation for what we do.  When documenting the workflow, I am reminded daily of the old Gen. Dwight Eisenhower maxim: “those who fail to plan, plan to fail”.