Author Archives: SecurityOps

About SecurityOps

Information Security Professional

Timeline Analysis – Playing with a “gold” image

This blog is just to show some documentation related to timeline analysis and how much noise can happen on a Windows system just by performing simple activities in a short period of time.  I know this has probably been documented many times in other places, however I wanted to see it for myself on a system that I could control.

It’s not the scope of this entry to demonstrate how or the best method of doing this (I do show the command lines and the steps taken).  Timeline analysis is discussed in more details by other people much more prepared than I.  Some examples:

http://windowsir.blogspot.ca/2011/09/creating-mini-timelines.html

http://thedigitalstandard.blogspot.ca/2010/03/creating-timeline-of-live-windows.html

http://thedigitalstandard.blogspot.ca/2010_03_01_archive.html

http://computer-forensics.sans.org/blog/2011/12/07/digital-forensic-sifting-super-timeline-analysis-and-creation

Basically this is what I did:

1. Created a “gold” VM image of a Windows 7 64-bit install.  Nothing else was installed, aside from the MS patches.  This image was fully patched as of July 24, 2013.

2. Generate a HASH list of the image.  This will be used later to understand what files were changed/added after the user activities.  I also want to make it clear that no exploits or malware were used.  This is simply user/OS activities being shown here.

sha1deep64.exe -rc J:\ > Windows7-64-bit-AllPatches-July24-2013.txt

3. Followed a list of activities and recorded the time.  The activities happened between 20:36 and 21:07 EST5EDT, on July 26 2013.

4. After all activities were executed, the VM was suspended and a snapshot got created.  I used FTK Imager to create a RAW DD like image of the VM.

5. Used the hash in point #2 to see what changed.  I used sha1deep64 to make the comparison against the mounted image.  Results can be downloaded at the end of this post.

6. The timeline analysis activities:

A. File system – I used fls from “TSK – The Sleught Kit” http://www.sleuthkit.org/sleuthkit/.  I had to run it from a Linux distribution because the Windows version didn’t work for me.  Didn’t bother spending time troubleshooting why it didn’t work since I had the other Linux VM running anyway. First step is to create the body file, so here it is the command I used to do that:

fls -m C:/ -f ntfs -i raw -r /mnt/image/Win7-VM.001 > /mnt/j/Win7-VM.001.body

After creating the body file, I used bodyfile.pl which is one of the TLN tools provided by Harlan Carvey (https://code.google.com/p/winforensicaanalysis/downloads/list):

bodyfile.pl –f Win7-VM.001.body –s WIN-HM1OUI8PPLE > FS_events.txt

The above file will be used later on to create the master timeline.

B. Event Logs – Extracted the main Windows logs, Application, Security, and System from the image.  Here are the steps taken to extract the data into the TLN format.

i.      Use LogParser from Microsoft and export to CSV:

LogParser.exe -i:evt -o:csv “select * from System.evtx” > system.csv

LogParser.exe -i:evt -o:csv “select * from Security.evtx” > security.csv

LogParser.exe -i:evt -o:csv “select * from Application.evtx” > Application.csv

ii.      Use the “evtxparse.pl” Perl script to parse out the necessary event data into TLN format:

evtxparse.pl -f system.csv -t > sys_events.txt

evtxparse.pl -f security.csv -t > sec_events.txt

evtxparse.pl -f Application.csv -t > app_events.txt

C. Prefetch Files – The image was mounted using FTK imager, so I just pointed the pref.pl script to the mount point and the correct folder:

pref.pl -d J:\Windows\Prefetch -v -t -s WIN-HM1OUI8PPLE > prefetch_events.txt

D. Registry Data – Extracted all registry files from the image and created the timeline files using regtime Perl script.  I used 2 user ID’s during the test.  My own, “elias”, and the MS shipped “administrator”.

regtime.pl –m HKEY_USER –r NTUSER.DAT –s WIN-HM1OUI8PPLE –u elias > reg_elias_event.txt

regtime.pl –m HKEY_USER –r UsrClass.dat –s WIN-HM1OUI8PPLE –u elias >> reg_elias_event.txt

regtime.pl –m HKEY_USER –r NTUSER.DAT –s WIN-HM1OUI8PPLE –u ADMIN > reg_admin_event.txt

regtime.pl –m HKEY_USER –r UsrClass.dat –s WIN-HM1OUI8PPLE –u ADMIN >> reg_admin_event.txt

regtime.pl -m HKLM/SOFTWARE -r SOFTWARE -s WIN-HM1OUI8PPLE > reg_software_event.txt

regtime.pl -m HKLM/SYSTEM -r SYSTEM -s WIN-HM1OUI8PPLE > reg_system_event.txt

regtime.pl -m HKLM/SECURITY -r SECURITY -s WIN-HM1OUI8PPLE > reg_security_event.txt

7. After generating all the event files, I had to combine them all into one file (all-events.txt in my case).  After, simply use the parse.pl Perl script to create the timeline:

parse.pl -f all-events.txt -r 07/26/2013-07/27/2013 -c > all-events.csv

Since the time in the event file is UTC, I had to include also June 27 as part of the range.  So, basically July 26 20:36 is July 27 00:36 in the all-events.csv file.  I could have simply used the range 07/27/2013-07/27/2013 and I would get the result I wanted.

8. Now, on to the results, which is what this whole thing is about anyway.  In the document I tried to match the actual activities with the timelines from the artifacts processed above.  This should help with context.

2013-08-04_213958

I know I could have used log2timeline to do this, but I wanted to take a more manual approach.  I’m also aware that there are be other sources of information that can be added to a timeline like this one.

Download the full results here:

Timeline_activity (XLSX format) or TLN CSV Format

List of files changed: hash_FilesAddedOrChanged

Cheers.

Advertisements

HTTP_Exec – Automated VT analysis of downloaded executable files

Hi Everyone,

I wanted to start this blog by posting this program I’ve been working on.  The function is very simple: take input from a CSV file with URL’s of executable files, download, hash, check Virus Total for infections, and report it.

It was created for a group of colleagues that needed to perform this very same task manually on a regular basis. The source of the data (CSV file) is as NIDS signature created by IBM’s X-Force called “HTTP_Executable_Transfer“.  To make use of this program you’ll need:

1. Create a filter/view with the HTTP_Executable_Transfer signature in SiteProtector, and export the contents to CSV.  The following columns are required:

Time,Source IP,Target IP,Object Name,:URL,:server,:arg

If you don’t use Site Protector and still want to use this program, here’s the data mapping for the columns:

Time = Self-explanatory, date format: YYYY-MM-DD HH:MM:SS TIME_ZONE (2013-04-22 08:48:34 EDT)
Source IP = Self-explanatory
Target IP = Self-explanatory
Object Name = Destination port, i.e: 80
:URL = The path of the URL, needs to have the “/”. Example: /CodeStuff/PDFStreamDumper_Setup.exe
:server = The server where the file is, for example file.google.com or http://www.google.com. Don’t add HTTP/HTTPs or slashes
:arg = Some URLs use arguments. Do not add question mark, the program will do that for you. Example of argument: file=updates/early_up_20130331.exe

Example of a properly formatted CSV file:
Time,Source IP,Target IP,Object Name,:URL,:server,:arg
“2013-04-04 08:58:41 EDT”,”10.10.1.1″,”207.70.25.201″,”80″,”/fileDownloader.php”,”www.styleadvisor.com”,”file=updates/early_up_20130331.exe”

2. A VirusTotal API number.  The program will make connections to Virus Total via its public API, so you’ll need to get one.

3. Internet connection – Well, yes, you need one….as mentioned before, the program will download the file for you from the URL provided.

The program will NOT submit the file to VT automatically.  It will simply submit the HASH (MD5) and get the information back.  No files are submitted AT ALL. (we plan to add the option later).

Download:

Linux Version

MD5: 6d471f9b19993014ba07e9969af4714d

SHA1: f10fb9f05996031c50e5d75dd2cae7638bbaf3e4

Windows version

MD5: f4d3505785541ff20d75e6296c7ee3db

SHA1: 242603da6bf630d940edc4db218a751f440b31de

This is the 0.1 version, so there’s still a lot to do.  Read the information below for more details on usage or the read me file included in ZIP file.

By posting it here, we hope other people with a similar requirement can make use of it.  If you like the program and have some other use cases, please let us know.  We want to continue the development based on what other users need.

INSTALLATION

The program was created in Python 2.7.X, however it is a static file.  That means you don’t need to have Python installed.  It will run on Windows (tested on Windows 8/7) and Linux (tested on Ubuntu 64-bit).  It may work on other versions of Windows/Linux but I have not tested them.

All you need to do is extract the ZIP file and run the executable from the command-line.  Do not double-click from Windows, you need to go to command prompt and execute from there.

DATABASE

This program uses SQLite. All information obtained will be recorded in the following files:

db\http_exec.db – Contains the data related to files reported as infected by VT, and other
things. It will have all of the records that couldn’t be processed for whatever reason.
db\temp.db – Temporary DB used to record the data from the CSV file. You may delete this file after the program finished using it.

The report provided is basic. Feel free to explore the DB files and see what you can come up with.

WHITE LIST FILE
Inside the directory config there’s a file named white-list.txt
There you should have the list of domains you trust and don’t want to check.
It’s one domain per line. Example:
microsoft.com
download.windowsupdate.com
adobe.com

FILES & DOWNLOADS
The program will only download files of size 10MB or less. Anything over the 10MB limit,
the download will be interrupted and the temporary file deleted. This information is written
to the discarded table inside the db\http_exec.db file.

When the program can’t download a file, for whatever reason, that information is recorded
in the discarded table inside the db\http_exec.db file.

Http_exec automatically creates the following sub-directories relative to the location
of the program:

temp – Contains the files while it’s downloading. Files found to not have any detection in Virus Total will be deleted.
unknown – Contains files that were unknown by Virus Total. In other words, nobody previously submitted those files. You have to manually submit them.
malware – Contains files detected by AV engines from Virus Total.
reports – Where the report.csv file will be written to. (default)

SAMPLE

A sample CSV file (http_exec.csv) file is provided with the package.

USAGE
http_exec.exe -h
Basic usage: http_exec.exe -a <VT API>
Use with -v for a more detailed information regarding this program.
For help, use with the -h option

Options:
-h, –help show this help message and exit
-i SRCFILE Specify source CSV file. Default: http_exec.csv
-w WHITELISTFILE Specify white list file. One domain per line. Default:
config\white-list.txt
-o REPORTFILE Specify name of report CSV file. Default:
reports\report.csv
-a VT_API Virus Total API Number
-V, –verbose Debug mode for logs. Default is to log errors and info
only.
-q, –quiet Only log errors to log file.
-c, –continue Continue where it stopped. Won’t process any new records.
-r, –report Use this to simply generate a report on whatever is in the
DB already.
-v, –version Show program’s information and exit.

If using all the default settings, all you need to do is:
http_exec.exe -a <VT API>

The -c|–continue option can be used if the CSV file was already imported into the temp.db file and all you need is for the program to process whatever is in that DB and that has not been processed by the program yet. There is a column in that DB that flags whether the row was already processed or not. If you simply want to import everything again from a  CSV and process everything, delete the temp.db file and proceed as if this was the first time importing it. Caution: You will get duplicated entries in your main http_exec.db if you import the same CSV files over and over.

The -r|–report can be used if all you need is to generate the report. Nothing is processed, no CSV file is required, no connections or downloads are performed, only the report is generated by looking into the http_exec.db file.

LOGS
The log is located inside the \logs directory. If you use the -V option, it will
be more verbose. Default is INFO level. Please send this file along in case you need
support. The log can also provide information for failed download or VT calls.

CONTACT
Author: Elias Silva
Email: http_exec [ @ ] securityops [ . ] ca
URL: https://securityops.wordpress.com/
Copyright: Elias Silva. All rights reserved.

Continue reading