Amahi Wiki - User contributions [en]

Virus Scan Shares

2014-07-20T00:29:14Z

Whetu:

ClamAV is an open source (GPL) antivirus engine designed for detecting Trojans, viruses, malware and other malicious threats on Linux. In this article, we will only be configuring ClamAV to run scheduled/on-demand scans; not resident scans.

== Install ==
* Install required ClamAV packages
yum install clamav clamav-update
* Edit '''/etc/freshclam.conf''' and make the following changes:
** Comment out “Example”
** Uncomment lines
*** “DNSDatabaseInfo current.cvd.clamav.net”
*** “DatabaseMirror db.XY.clamav.net” (replace XY with your [http://www.iana.org/cctld/cctld-whois.htm country code])
** Ensure line “DatabaseMirror database.clamav.net” is uncommented
* Update ClamAV’s signatures
/usr/bin/freshclam

'''NOTE:''' ClamAV will update automatically, as part of /etc/cron.daily/freshclam.

== Configure Daily Scan ==
In this example, we will configure a cronjob to scan the Docs share every day:
* Create '''/etc/cron.daily/manual_clamscan''' and add the text from "a" or "b":
:a. <u>Scan</u> - Change SCAN_DIR to the directory that you want to scan.
#!/bin/bash
SCAN_DIR="/var/hda/files/docs"
LOG_FILE="/var/log/clamav/manual_clamscan.log"
/usr/bin/clamscan -i -r $SCAN_DIR >> $LOG_FILE
:b. <u>Scan with email notifications</u> - Change SCAN_DIR to the directory that you want to scan, EMAIL and EMAIL_FROM to your email addresses.
<pre>#!/bin/bash
# Email alert cron job script for ClamAV
# Original, unmodified script by: Deven Hillard
#(http://www.digitalsanctuary.com/tech-blog/debian/automated-clamav-virus-scanning.html)
# Modified to show infected and/or removed files
# Directories to scan
SCAN_DIR="/var/hda/files/docs"
# Location of log file
LOG_FILE="/var/log/clamav/manual_clamscan.log"
# Uncomment to have scan remove files
#AGGRESSIVE=1
# Uncomment to have scan not remove files
AGGRESSIVE=0
# Email Subject
SUBJECT="Infections detected on `hostname`"
# Email To
EMAIL="your.email@your.domain.com"
# Email From
EMAIL_FROM="clamav@server.hostname.com"
check_scan () {
# If there were infected files detected, send email alert
if [ `tail -n 12 ${LOG_FILE} | grep Infected | grep -v 0 | wc -l` != 0 ]
then
# Count number of infections
SCAN_RESULTS=$(tail -n 10 $LOG_FILE | grep 'Infected files')
INFECTIONS=${SCAN_RESULTS##* }

EMAILMESSAGE=`mktemp /tmp/virus-alert.XXXXX`
echo "To: ${EMAIL}" >> ${EMAILMESSAGE}
echo "From: ${EMAIL_FROM}" >> ${EMAILMESSAGE}
echo "Subject: ${SUBJECT}" >> ${EMAILMESSAGE}
echo "Importance: High" >> ${EMAILMESSAGE}
echo "X-Priority: 1" >> ${EMAILMESSAGE}
if [ $AGGRESSIVE = 1 ]
then
echo -e "\n`tail -n $((10 + ($INFECTIONS*2))) $LOG_FILE`" >> ${EMAILMESSAGE}
else
echo -e "\n`tail -n $((10 + $INFECTIONS)) $LOG_FILE`" >> ${EMAILMESSAGE}
fi
sendmail -t < ${EMAILMESSAGE}
fi
}
if [ $AGGRESSIVE = 1 ]
then
/usr/bin/clamscan -ri --remove $SCAN_DIR >> $LOG_FILE
else
/usr/bin/clamscan -ri $SCAN_DIR >> $LOG_FILE
fi
check_scan</pre>
* Give our cron script executable permissions:
chmod +x /etc/cron.daily/manual_clamscan
* Create empty log file
mkdir -p /var/log/clamav
touch /var/log/clamav/manual_clamscan.log
* (OPTIONAL) Run the script
/etc/cron.daily/manual_clamscan

And you’re done! That should be the minimum required to install ClamAV and Perform a daily scan of a specific directory.

'''NOTE:''' You will need to enable email on your HDA to use option "b". See [[Main_Page#Community_Tutorials|Community Tutorials]] for guidance.

==== Incremental Daily and Full Weekly Scans ====
You can expand on the above by making a daily cronjob script that only scans files changed in the last 24 hours, and a weekly cronjob script that does a full filesystem scan. You may choose to do this for reasons of efficiency, as scanning only the files changed in the last 24 hours is an order of magnitude faster. Your daily cron file would look something more like this:
{{Code|Code =
#!/usr/bin/env bash

# Variables
DATE=$(date +%Y%m%d)
DIR="/var/log/clamav"
LIST="${DIR}/scan.$DATE"
RESULTS="${DIR}/scanresults.${DATE}"
# A list of partitions and/or directories to scan, in this example /, /boot and /home are on separate partitions
SCANDIRS="/ /boot /home"
# Number of days to keep files generated by this script. Default is 7.
LOGROTATION=7

# Remove files older than the number of days set with the $LOGROTATION variable.
# One might choose to comment this out and keep the files for trend-tracing, breakfix etc),
find ${DIR} -name "scan*" -mtime +${LOGROTATION} -exec rm -f {} \;

# Generate a list of files created in the last 24 hours, this list is fed into clamscan
# Because we're using -mount to prevent unwanted filesystem traversal,
# you'll need to specify per partition or directory with the SCANDIRS variable.
for S in ${SCANDIRS}; do
find "${S}" -mount -mtime 0 >> "${LIST}"
done

# Run clamscan against the list of files and pipe the results out to the results file
# -i prints only infected files, -f is the list of files to scan, and -l is the output log
clamscan -i -f "${LIST}" -l "${RESULTS}"
}}

== Using Greyhole ==
You will need to do some additional setup to scan files when using Greyhole.

* Set up [https://wiki.amahi.org/index.php/Mount_Shares_Locally mount shares locally]
* Ensure the SCAN_DIR= <code>/mnt/samba/share</code> and not <code>/var/hda/files/share</code>

Now when the daily scan runs, it will scan the Greyhole enabled share correctly.

==References==
* [https://www.centosblog.com/how-to-install-clamav-and-configure-daily-scanning-on-centos/ How to Install ClamAV and Configure Daily Scanning on CentOS]

* [https://hacking.im/automated-clam-antivirus-scanning-for-centos-servers Automated Clam AntiVirus scanning for CentOS/Ubuntu/Debian servers]

Partitions Over 2.1 TB

2011-08-24T01:18:02Z

Whetu:

=Introduction=
As hard drives have increased in capacity over time, some clever workarounds have been engineered to keep breaking barriers, such as the move from CHS to LBA. Unfortunately, with older style MBR partitions, there is a limit of 2.1TB that simply cannot be worked around using an MBR partition. We have to instead use newer GPT style partitions.

I'm not going to delve into the history too much, you can google/wikipedia it yourself if you like.

So, if you've just purchased a snazzy new 3TB drive like a Western Digital Caviar Green, and you're scratching your head as to why you can't get the full capacity, here's the fix.

=Assumptions=
The below documentation assumes you have an existing boot drive with Amahi, namely /dev/sda. Your >2.1TB HDD will be /dev/sdb

The instructions also assume a Linux terminal, such as that on your Amahi box (either using its console, the Terminal application within the GUI, or SSH'd in)

=Warning=
BE ABSOLUTELY SURE THAT YOU ARE RUNNING THESE COMMANDS AGAINST THE CORRECT DRIVE. I ACCEPT NO RESPONSIBILITY FOR ANY DATA LOSS ETC!

=OK, Let's do it=
First we label the drive with a GPT partition
{{Code|
parted /dev/sdb mklabel gpt
}}

Now we select sectors as a unit of measurement
{{Code|
parted /dev/sdb unit s
}}

Now we make the partition.
{{Code|
parted /dev/sdb mkpart primary ext2 2048 -1
}}
Parted may complain about the last sector and will automatically suggest a correct sector, just say Yes to the prompt.

We make it as a primary partition, ext2 filesystem, and we carefully select the sectors. The first number MUST be divisible by 8, so you can try 40, 48, 56, 64 etc... For me these numbers threw up errors, so I just went to 2048 and everything worked fine, and in the grand scheme of a 3TB drive, the extra sectors don't really matter :)

The reason that the starting sector is divisible by 8 is for aligning the sectors. This is because HDD's have traditionally been based around 512-byte sectors, yet newer, higher capacity HDD's are now arriving based around 4096-byte (a.k.a. 4k) sectors. The partitioning will still work with 512-bytes, but the drive's performance will be drastically impacted - in some testing it can be twice as slow, in others up to 25 times slower, depending on the sector alignment. So it's important to get this right!!!

=Alternative Parted usage=
Instead of the sequence of one liners up there, you can enter into parted itself like this. I give this as I don't know how your version of parted is going to behave. The interactive stuff from parted has been removed:
{{Code|
parted /dev/sdb
(parted) mklabel gpt
(parted) unit s
(parted) mkpart primary ext2 2048 -1
(parted) quit
}}

=Format the drive!=
Finally, let's get that drive formatted to a nice ext4 volume:
{{Code|
mkfs.ext4 -T largefile4 /dev/sdb1
}}

=Adding the drive to your HDA=
Normally to add a drive to Amahi, you would run the script '''hda-diskmount''', which searches for any unmounted drives and mounts them. Unfortunately, this script doesn't yet cater for GPT partitions. It runs an '''fdisk -l''' and searches out drives with a Linux system type, and with what it finds it mounts any that aren't mounted.

If we run '''fdisk -l''' by itself, we will see that our new drive doesn't return a system type of 'Linux', instead it returns 'GPT' as well as a warning:

{{Code|
Code= WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdb: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdb1 1 4294967295 2147483647+ ee GPT
Partition 1 does not start on physical sector boundary.
}}

So hda-diskmount correctly ignores the drive. You can either [http://forums.amahi.org/viewtopic.php?f=15&t=2326 fix hda-diskmount], or you can manually mount the drives with the instructions below.

First we make our mount point

{{Code|
mkdir /var/hda/files/drives/sdb1
}}

Next we exercise good practice and backup our fstab file, I like to use `date +%Y%m%d` to give the name a date stamp e.g. /etc/fstab.20110731 represents a backup done on the 31st of July, 2011. This isn't necessary, you could just use something like 'fstab.old', but date-stamping your file backups is a good habit to get into
{{Code|
Code= cp /etc/fstab /etc/fstab.`date +%Y%m%d`
}}

Now we adjust the fstab file

{{Code|
Code= echo -e /dev/sdb1$'\t\t'/var/hda/files/drives/sdb1$'\t'ext4$'\t'defaults$'\t'1 1 >> /etc/fstab
}}

What this does is adds a line to the /etc/fstab file, telling it that when Amahi boots, it has to mount the drive. The $'\t' entries simply mean 'put in a tab space'

If you now run '''cat /etc/fstab''' you should see a nice entry at the bottom, similar to this

{{Code|
Code= cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Thu Jul 28 23:56:48 2011
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg_hda-LogVol01 / ext4 defaults 1 1
UUID=cc1204dd-a4f8-42f8-8736-b5970251a865 /boot ext4 defaults 1 2
/dev/mapper/vg_hda-LogVol00 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/sdb1 /var/hda/files/drives/sdb1 ext4 defaults 1 1
}}

Now you can either reboot to allow the fstab file to mount your new drive, or you can simply run '''mount -a'''

Now if you run mount by itself, you should see your drive listed e.g.

{{Code|
Code= mount
/dev/mapper/vg_hda-LogVol01 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
/dev/sdd1 on /media/520abb16-ac7c-4f65-9cc3-9e536c93dca2 type ext4 (rw,nosuid,nodev,uhelper=udisks)
/dev/sdb1 on /var/hda/files/drives/sdb1 type ext4 (rw)
}}

=A note about Western Digital drives=
Western Digital make some nice hardware, but they are also incredibly stupid. Many of their hard drives will automatically park their heads every 8 seconds, which wears out the hard drives very quickly, in some cases the HDD's will reach their design limit (300k parks for consumer, 600k parks for enterprise) within 3-4 months!!! This isn't to say that the drives will necessarily die within 3-4 months, but it's still not good!

Fortunately we can fix this using a Western Digital tool named wdidle3. You can download it [http://support.wdc.com/product/download.asp?groupid=609&sid=113 here].

Create a bootable usb drive using [http://www.freedos.org/freedos/files/ FreeDOS] and [http://unetbootin.sourceforge.net/ unetbootin], then copy wdidle.exe onto the usb drive. Boot up your Amahi box to FreeDOS and enter the following commands:

{{Code|
c:\
wdidle3 /R
wdidle3 /S300
}}

This cd's you to the usb drive's root, then we run wdidle3 /R to display the current drive status, then we run wdidle3 /S300 to set the park time to 5 minutes. Alternatively you could run wdidle3 /D to completely disable the parking.

When you're done, simply restart the PC!

n.b. Unetbootin was a bit finnicky for me, I found that if I created the drive with a FreeDOS live CD that it wouldn't work, but if I then re-created using unetbootin's automatic distribution download feature, the USB stick worked a treat. YMMV.

Partitions Over 2.1 TB

2011-07-31T01:36:53Z

Whetu:

=Introduction=
As hard drives have increased in capacity over time, some clever workarounds have been engineered to keep breaking barriers, such as the move from CHS to LBA. Unfortunately, with older style MBR partitions, there is a limit of 2.1TB that simply cannot be worked around using an MBR partition. We have to instead use newer GPT style partitions.

I'm not going to delve into the history too much, you can google/wikipedia it yourself if you like.

So, if you've just purchased a snazzy new 3TB drive like a Western Digital Caviar Green, and you're scratching your head as to why you can't get the full capacity, here's the fix.

=Assumptions=
The below documentation assumes you have an existing boot drive with Amahi, namely /dev/sda. Your >2.1TB HDD will be /dev/sdb

The instructions also assume a Linux terminal, such as that on your Amahi box (either using its console, the Terminal application within the GUI, or SSH'd in)

=Warning=
BE ABSOLUTELY SURE THAT YOU ARE RUNNING THESE COMMANDS AGAINST THE CORRECT DRIVE. I ACCEPT NO RESPONSIBILITY FOR ANY DATA LOSS ETC!

=OK, Let's do it=
First we label the drive with a GPT partition
{{Code|
parted /dev/sdb mklabel gpt
}}

Now we select sectors as a unit of measurement
{{Code|
parted /dev/sdb unit s
}}

Now we make the partition.
{{Code|
parted /dev/sdb mkpart primary ext2 2048 -1
}}
Parted may complain about the last sector and will automatically suggest a correct sector, just say Yes to the prompt.

We make it as a primary partition, ext2 filesystem, and we carefully select the sectors. The first number MUST be divisible by 8, so you can try 40, 48, 56, 64 etc... For me these numbers threw up errors, so I just went to 2048 and everything worked fine, and in the grand scheme of a 3TB drive, the extra sectors don't really matter :)

The reason that the starting sector is divisible by 8 is for aligning the sectors. This is because HDD's have traditionally been based around 512-byte sectors, yet newer, higher capacity HDD's are now arriving based around 4096-byte (a.k.a. 4k) sectors. The partitioning will still work with 512-bytes, but the drive's performance will be drastically impacted - in some testing it can be twice as slow, in others up to 25 times slower, depending on the sector alignment. So it's important to get this right!!!

=Alternative Parted usage=
Instead of the sequence of one liners up there, you can enter into parted itself like this. I give this as I don't know how your version of parted is going to behave. The interactive stuff from parted has been removed:
{{Code|
parted /dev/sdb
(parted) mklabel gpt
(parted) unit s
(parted) mkpart primary ext2 2048 -1
(parted) quit
}}

=Format the drive!=
Finally, let's get that drive formatted to a nice ext4 volume:
{{Code|
mkfs.ext4 -T largefile4 /dev/sdb1
}}

=Adding the drive to your HDA=
Normally to add a drive to Amahi, you would run the script hda-diskmount, which searches for any unmounted drives and mounts them. Unfortunately, this script doesn't yet cater for GPT partitions. It runs an '''fdisk -l''' and searches out drives with a Linux system type, and with what it finds it mounts any that aren't mounted.

If we run '''fdisk -l''' by itself, we will see that our new drive doesn't return a system type of 'Linux', instead it returns 'GPT' as well as a warning:

{{Code|
Code= WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.

Disk /dev/sdb: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdb1 1 4294967295 2147483647+ ee GPT
Partition 1 does not start on physical sector boundary.
}}

So hda-diskmount correctly ignores the drive.

First we make our mount point

{{Code|
mkdir /var/hda/files/drives/sdb1
}}

Next we exercise good practice and backup our fstab file, I like to use `date +%Y%m%d` to give the name a date stamp e.g. /etc/fstab.20110731 represents a backup done on the 31st of July, 2011. This isn't necessary, you could just use something like 'fstab.old', but date-stamping your file backups is a good habit to get into
{{Code|
Code= cp /etc/fstab /etc/fstab.`date +%Y%m%d`
}}

Now we adjust the fstab file

{{Code|
Code= echo -e /dev/sdb1$'\t\t'/var/hda/files/drives/sdb1$'\t'ext4$'\t'defaults$'\t'1 1 >> /etc/fstab
}}

What this does is adds a line to the /etc/fstab file, telling it that when Amahi boots, it has to mount the drive. The $'\t' entries simply mean 'put in a tab space'

If you now run '''cat /etc/fstab''' you should see a nice entry at the bottom, similar to this

{{Code|
Code= cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Thu Jul 28 23:56:48 2011
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/vg_hda-LogVol01 / ext4 defaults 1 1
UUID=cc1204dd-a4f8-42f8-8736-b5970251a865 /boot ext4 defaults 1 2
/dev/mapper/vg_hda-LogVol00 swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
/dev/sdb1 /var/hda/files/drives/sdb1 ext4 defaults 1 1
}}

Now you can either reboot to allow the fstab file to mount your new drive, or you can simply run '''mount -a'''

Now if you run mount by itself, you should see your drive listed e.g.

{{Code|
Code= mount
/dev/mapper/vg_hda-LogVol01 on / type ext4 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext4 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
/dev/sdd1 on /media/520abb16-ac7c-4f65-9cc3-9e536c93dca2 type ext4 (rw,nosuid,nodev,uhelper=udisks)
/dev/sdb1 on /var/hda/files/drives/sdb1 type ext4 (rw)
}}

=A note about Western Digital drives=
Western Digital make some nice hardware, but they are also incredibly stupid. Many of their hard drives will automatically park their heads every 8 seconds, which wears out the hard drives very quickly, in some cases the HDD's will reach their design limit (300k parks for consumer, 600k parks for enterprise) within 3-4 months!!! This isn't to say that the drives will necessarily die within 3-4 months, but it's still not good!

Fortunately we can fix this using a Western Digital tool named wdidle3. You can download it [http://support.wdc.com/product/download.asp?groupid=609&sid=113 here].

Create a bootable usb drive using [http://www.freedos.org/freedos/files/ FreeDOS] and [http://unetbootin.sourceforge.net/ unetbootin], then copy wdidle.exe onto the usb drive. Boot up your Amahi box to FreeDOS and enter the following commands:

{{Code|
c:\
wdidle3 /R
wdidle3 /S300
}}

This cd's you to the usb drive's root, then we run wdidle3 /R to display the current drive status, then we run wdidle3 /S300 to set the park time to 5 minutes. Alternatively you could run wdidle3 /D to completely disable the parking.

When you're done, simply restart the PC!

n.b. Unetbootin was a bit finnicky for me, I found that if I created the drive with a FreeDOS live CD that it wouldn't work, but if I then re-created using unetbootin's automatic distribution download feature, the USB stick worked a treat. YMMV.

Partitions Over 2.1 TB

2011-07-30T03:51:23Z

Whetu: Created page with "=Introduction= As hard drives have increased in capacity over time, some clever workarounds have been engineered to keep breaking barriers, such as the move from CHS to LBA. Unf..."