Category: Linux

UNIX Command History

One of the great things about Mac OSX is that it is based on Unix. One of the greatest thing in Unix is its terminal. However in my new mac I see that I need to re-type every command even I have just typed before… Usually you expect your command to be stored in .bash_history and when you hit upper arrow key you should be able to see the previous commands you typed. That was not the case with this Mac.

I checked/Users/ismail directory for .bash_history file. There was none. I created with touch command.

touch .bash_history
I tried with no help… Next I see there was no .bash_profile file neither. I created that with touch too:
touch .bash_profile

I then edited the .bash_profile :
HISTFILESIZE=5000 HISTSIZE=5000 HISTFILE=/Users/ismailg/.bash_history

The maximum number of lines contained in the history file. When this variable is assigned a value, the history file is trun-
cated, if necessary, to contain no more than that number of lines.

The number of commands to remember in the command history

Now you should be able to use arrow keys to get previous commands. In the next blog post we will discuss how to make Unix systems not records your commands even if you are a regular user in box and how to control attackers to delete .bash_history

Hiding Data: Steganography on Linux

My last blog post was about hiding info on slackspace by using a special tool called Bmap. Today I am going to discuss Steganography in more general. Steganography is the art and science of writing hidden messages in such a way that no one, apart from the sender and intended recipient, suspects the existence of the message. It is different than encryption since encryption may not care existence of cipher text from outside world. However steganography deals with hiding information even if it is encrypted.

See following example:

Alex wants to send a message to Bob. She wants only Bob reads the message. She can use encryption (symmetric or asymmetric). The risk here is possible attack for deciphering her message. She can also try hiding the message in a different format (say in a jpeg file) and send over the Bob. Since the message will be in a picture probably attacker Tom will not recover the message from the file.

Of course in case he desperately wants to read the message, he can use some forensics tools to read the message. For this reason combining encryption with a stenography will be best choice for Alex.

Today I am going to discuss a Linux tool steghide that does both encryption and stenography. On debian based system you can install steghide by following command:

apt-get install steghide

By default steghide compress the embedded data, and encrypted with rijndael-128 algorithm.

I have two files under my Private folder:

root@bt:~/Private# ls -l
total 24
-rw-r--r-- 1 root root    20 Jan 30 18:24 myMessageToBob.txt
-rw-r--r-- 1 root root 17875 Jan 30 18:23 soccer.jpg

My goal is embedding text file into jpeg file.

Let’s check if we have enough space on jpeg file to do that:

steghide info soccer.jpg
format: jpeg
capacity: 1.0 KB
Try to get information about embedded data ? (y/n)

So we can embed 1.0KB data and we only have 20B data (see ls -l output)

root@bt:~/Private# steghide embed -cf soccer.jpg -ef myMessageToBob.txt
Enter passphrase:
Re-Enter passphrase:
embedding "myMessageToBob.txt" in "soccer.jpg"... done

-cf stands for cover file whereas -ef stands for embedded file.

Let’s now check the size of the jpeg file.

ls -l
total 24
-rw-r--r-- 1 root root    20 Jan 30 18:24 myMessageToBob.txt
-rw-r--r-- 1 root root 18396 Jan 30 18:27 soccer.jpg

Himm, it got bigger and that was expected but the original data was just 20B and we know that steghide will compress data before embedding.

As you guess the reason of 521B (18396-17875) is encryption and crc check sum of the embedded data that will also added into the jpeg file.

Try to open the jpeg file. You will not see any difference from the original jpeg file.

Now, we want to extract the data out of the jpeg file.

root@bt:~/Private# steghide extract -sf soccer.jpg
Enter passphrase:
the file "myMessageToBob.txt" does already exist. overwrite ? (y/n) y
wrote extracted data to "myMessageToBob.txt".

root@bt:~/Private# ls -l
total 24
-rw-r--r-- 1 root root    20 Jan 30 18:58 myMessageToBob.txt
-rw-r--r-- 1 root root 18396 Jan 30 18:27 soccer.jpg

After we extracted the text file from jpeg file the file in the jpeg is still there (check the size after embedding and after extracting, they are same)

The only disadvantage I can think of is not being able to wipe the data from the cover file (the file you embed data into).

Hiding Data: Slack Space on Linux

I am going to cover slack space in various operating systems. the first post would be about the slack space in Linux.

What is Slack Space?

Before going to explain slack space, one should know what blocks (on Linux) and clusters mean (on Windows).  Blocks are specific sized containers used by file system to store data. Blocks can also be defined as the smallest pieces of data that a file system can use to store information. Files can consist of a single or multiple blocks/clusters in order to fulfill the size requirements of the file. When data is stored in these blocks two mutually exclusive conditions can occur; The block is completely full, or the block is partially full. If the block is completely full then the most optimal situation for the file system has occurred. If the block is only partially full then the area between the end of the file the end of the container is referred to as slack space. Linux write nulls on slack space. This means to find data in slack space on Linux systems are rare. However, it is not impossible.

Today, I am going to show how to hide data on slack spaces using a tool called bmap.


Bmap ,a data hiding tool, can utilize slack space in blocks to hide data.

It can  perform lots of functions interesting to the computer forensics community and the computer security community. However, in this article we will focus on its data hiding capability.

Installing Bmap

Click this link and save the tar.gz file on your Linux desktop.tar xvzf bmap-1.0.17.tar.gz

After untaring the file, we now can go inside of the directory and compile the program.
cd bmap-1.0.17

Optional: I placed bmap into /sbin so don’t need to go into the bmap directory each time I want to run the program.
ln -s yourBmapFilePath /sbin/bmap

Hiding Data on Slack Space

In the following example we will hide some text into slack space. Let’s see what options we have with bmap:
bmap --help
bmap:1.0.17 (12/25/10)
Usage: bmap [OPTION]... []
use block-list knowledge to perform special operations on files
--doc VALUE
where VALUE is one of:
version display version and exit
help display options and exit
man generate man page and exit
sgml generate SGML invocation info
--mode VALUE
where VALUE is one of:
map list sector numbers
carve extract a copy from the raw device
slack display data in slack space
putslack place data into slack
wipeslack wipe slack
checkslack test for slack (returns 0 if file has slack)
slackbytes print number of slack bytes available
wipe wipe the file from the raw device
frag display fragmentation information for the file
checkfrag test for fragmentation (returns 0 if file is fragmented)
--outfile write output to ...
--label useless bogus option
--name useless bogus option
--verbose be verbose
--log-thresh logging threshold ...
--target operate on ...

The option I am going to use –mode option with slack, putslack, wipe, map VALUEs. I have created a text file named Ismail.txt. Let’s see what sectors this file uses.
bmap --mode map ismail.txt

As you can see from the output of bmap, ismail.txt uses 8 sectors starting from 3113400. This corresponds a block in Linux. This text file is too small to use all of these sector in the block.

bmap --mode slack ismail.txt
getting from block 389175
file size was: 8
slack size: 4088
block size: 4096

The file only uses 8 bytes (1 sector is 512 bytes, so it is in the first sector). All of 7 sectors and 504 bytes of the first sector are empty (Linux write null on the slack space, so all they have 0s.)

We can use this slack space to hide data.

echo "I'm hiding this and you cannot easily see it" | bmap --mode putslack ismail.txt
stuffing block 389175
file size was: 8
slack size: 4088
block size: 4096

Now check what we have in slack space:

bmap --mode slack ismail.txt
getting from block 389175
file size was: 8
slack size: 4088
block size: 4096
I'm hiding this and you cannot easily see it

(This command also gives the block number of the file, 389175)

We can now wipe the data we put on slack space.
bmap --mode wipe ismail.txt

bmap --mode slack ismail.txt
getting from block 389175
file size was: 8
slack size: 4088
block size: 4096


Slack space can be used for hiding data. Even though advanced forensics tools reveal the hidden information in slack space, most system owners will not check all slack space since this is so time consuming activity. Bmap is one of the tool that helps attackers to hide information into slack space on Linux systems. As security engineers, we need to be careful when we analyze a compromised Linux systems and check slack space as well.

Introduction to Linux Forensics- Part I

It has been two weeks since I have not made a new blog post. There are some reasons behind this. I am busy with the work.

However, I don’t ignore my blog and actually was writing 2 new blog posts; one for the e-mail security with GPG and another one  for my third Nessus blog post. Those are still in progress. I just saved them and will complete as soon as I have more free time.

I am currently visiting Rackspace Cloud at San Antonio. I started to write this blog post in the plane and now I will complete it in my hotel room…


I am currently writing an article for the Slicehost customers to show them how to investigate their slices (Linux VPS) during a  possible compromise.

I am doing some research and implementing my knowledge on the Slicehost environment which takes quite time to complete the article.

I thought it would be good to have a blog post about a more general environment. This is the my first forensic related post. Yes, I have huge interest on Computer Forensics.


First of all, this article covers only the basic of Linux forensics. By saying that I won’t cover any highly sophisticated forensic techniques here ( at least in first two articles)

The aim of this blog post is simply showing you the way you can investigate your compromised Linux machine and learn from your mistakes. ( I will have articles about some advanced forensics tools such as autopsy, vinetto and MboxGrep later)

IMPORTANT WARNING: Before you do anything, you need to make an important decision—do you plan to involve law enforcement and prosecute the attacker? If the answer is yes, you should leave the compromised system alone and make no changes to it.

Any changes you make post-attack could complicate and taint the evidence, and because of that, many people have a policy of unplugging a system once they detect an attack and leaving it off until law enforcement arrives.

Investigators likely will want the complete system, or at least the drives, so they can store it safely; thus, your forensics analysis might end here until your system is returned. [1]

Nobody is perfect. Everybody can make mistakes. However, I avoid as much as possible to make same mistake twice. I believe only stupid people do that. At least, I feel stupid if I do same mistakes.

Ok, back to our lesson. We have a compromised Linux machine. First be calm. It is ok to get hacked. We are not only ones whose boxes got cracked. Of course, good system administrators will do everything to avoid this type of situation.

However, even if you believe you are so knowledgeable system admin, your machines can be hacked by an attacker who exploited a new discovered vulnerability…

Checking Network Connection

Check the network connections and open ports with netstat command.


netstat -an

By running this command you can see the any backdoors that are listening

tcp 0 0* LISTEN
tcp 0 0* LISTEN
tcp 0 0* LISTEN

In this case we see port 6697 is open. It is not a good sign because that port is used by IRC. We can sniff the connection by tcpdump. For more info on tcpdump, check this blog post.

tcpdump src port 6697

You can check here for more info on IRC bots.

Checking Last Logged in IPs

Brute force attack is a very popular type of attack. You may be able to find who was attacked you by checking last logged in IPs with the last command.

Using last you can determine the time a user logged in and out. It also provide you the hostname / IP address from where the user logged in from.

last -25

This will give us last 25 users’ IP who logged in the system.

/var/log/auth_log file can also have valuable information regarding to successful or failed login attempts.

Checking Last Commands

You may have heard “No crime is perfect” a lot if you have ever watched the Forensic Files TV show. It is true. Only a few good hackers cannot leave their finger prints on their digital crime.

For example, most of the time intruders leave their  their .bash_history files. .bash_history file contains the last commands used with the bash shell.

This can give  us a lot information about what they did, what they installed and where they got their files from. Typical entries may include,

wget http://malware.tar.gz
gunzip malware.tar.gz
tar xf malware.tar
cd hpd
cd ..
rm malware.tar
cd /dev/.hpd

This tells us the url they got the malware from, how they ran it, and where it was
installed. A good starting point for looking for their directory! [2]

Be aware of the way .bash_history store the information! It only show the all commands which has been run by a spesific user after he logged out.

In case attacker is logged in and you are trying to check his .bash_history, you may see an empty file.

Use who command to see active users on the machine.

user1 pts/0 Nov 18 23:33 (
user2 pts/1 Nov 16 10:22 (

We see two active users on the system. If user2 is compromised account, we should tcpdump and monitor his activity:

tcpdump host -w demo.dump

You can also use thehistory command to list the history of the last executed commands.

To get more useful information from history command and .bash_history file, let’s modify /etc/profile directory.

Add following line at the end of the file:

export HISTTIMEFORMAT=”%h/%d – %H:%M:%S “

You can now see the time when the commands run. (You will be able to see all commands with time stamps on the history ‘s output.

However, for .bash_history, you will be only able to see time stamps for new commands which is not useful for us.


We learn some basic information for investigating compromised Linux machines such as checking network connection, active users on the system, getting bash history, last logged in users IP etc… All of these are so critical information to track intruders and find security holes on the system.

The next post will discuss integrity checks and some helpful tools such as rootkit scanners.