App troubleshooting

App install should never fail. Period. App install failures are high priority.

Debugability

Debugging app install issues is very important, so any time something permanently destructive is attempted, it should be done with care to preserve the ability to analyze and debug what happened:

• Preserve as much as possible from the existing (buggy) environment

• Make sure httpd configuration is sane (result should be "Syntax OK"):

bash code
​httpd -t​

• The idea is to not delete anything permanently - copy or move things to, say, a folder in /tmp if you have to get something out of their place. For example:

bash code
​mkdir /tmp/app-debug/​

• Then add things there as necessary for later analysis, like:

bash code
​cp -a /var/log/amahi-app-installer.log /tmp/app-debug cp -a /etc/httpd/conf.d /tmp/app-debug​

The user can provide this information:

• Copy of the app install log:

bash code
​tail -300 /var/log/amahi-app-installer.log | fpaste​

• Copy of the app error and access logs (i.e. sabnzbd is the app):

bash code
​cat /var/hda/web-apps/sabnzbd/logs/error.log cat /var/hda/web-apps/sabnzbd/logs/access.log​

• To check if process is running (i.e. sab is the app):

bash code
​ps guax | grep sab | fpaste​

Miscellaneous

Filesystem Corrupt

Filesystems are corrupted, due to a power failure or system crash. Generally, after a system crash or power outage (what? No UPS?), the system will come up and repair itself. If you are using a journalling filesystem like ext3fs, jfs, xfs or resiserfs, it will usually perform a roll-forward recovery from its journal file and carry on. Even with the older ext2fs, the system usually runs an fsck (file system check) on the various file systems and repairs them automatically. However, just occasionally manual intervention is required - ; you might have to answer 'Y' to a string of questions (answering 'N' will get you nowhere unless you intend to perform really low-level repairs yourself in a last-ditch attempt to avoid data loss). In the worst case, you might have to reboot from rescue media and manuall run the e2fsck (or similar) command against each filesystem in turn. For example:

bash code
​e2fsck -p /dev/hda7​

• If the program complains that the superblock - the master block that links to everything else - is corrupted, it is useful to remember that the superblock is so critical that it is duplicated every 8192 blocks through the filesystem and you can tell e2fsck to use one of the backups:

bash code
​e2fsck -b 8193 /dev/hda7​

• One or more filesystems cannot be found and mounted: Check the contents of /etc/fstab - in making quick alterations here, typographical errors are common. You can use the e2label command to view the label of each filesystem: some distributions set these to the mount point so you can figure out what is what.

Filesystem Full

Is a filesystem full? This can show up in lots of different ways: being unable to save files, print jobs not spooling correctly (especially on Samba print/file servers), and so on. Use the df command to see available space:

[root@freya home]# df -H
Filesystem              Size  Used Avail Use% Mounted on
/dev/Volume00/LogVol00 520MB 254MB 240MB 52%  /
/dev/hda3              128MB 2 1MB 101MB 17%  /boot
/dev/Volume00/LogVol03 2.2GB 134MB 1.9GB  7%  /home
/dev/Volume00/LogVol05 520MB 8.5MB 485MB  2%  /opt
none                   264MB     0 264MB  0%  /dev/shm
/dev/Volume00/LogVol02 1.1GB  36MB 969MB  4%  /tmp
/dev/Volume00/LogVol01 4.3GB 3.0GB 1.1GB 75%  /usr
/dev/Volume00/LogVol06 1.1GB 101MB 903MB 11%  /usr/local
/dev/Volume00/LogVol04 3.2GB 2.3GB 756MB 75%  /var
/dev/hda1               16GB  13GB 2.8GB 83%  /mnt/winc

• Remember that a filesystem can fill up either because almost all of its data blocks are used up (some are reserved for the root user, just to get out of trouble) or because all its i-nodes (there is one of these per file) are used up.
• If you need to make space by deleting some large files, use the command 'ls -lS' to get a directory listing that is sorted by file size. To scan an entire filesystem (e.g. /home or /var) for the largest files, use the command:

sort -n code
​du ​

The largest files will be at the end of the listing.