The Festival Speech Synthesis System with MPlayer output

This post is about setting up the Festival Speech Synthesis System to output speech through MPlayer playback. This is necessary if the built-in output modules, like ALSA or linux16audio, do not work for you. This can also help to increase the quality and performance of the synthesized audio. (For Slackware Linux, for instance, MPlayer is the best already-installed option.) Once you have built Speech Tools and Festival following the instructions in the INSTALL files, you can specify MPlayer for output in your ~/.festivalrc file by adding the following lines:

(Parameter.set 'Audio_Command "mplayer -really-quiet -noconsolecontrols -nojoystick -nolirc -nomouseinput -demuxer rawaudio -rawaudio channels=1:rate=$SR $FILE")
(Parameter.set 'Audio_Method 'Audio_Command)

University of Toronto email on the Android email client

This post will provide you with quick and dirty instructions for setting up the Android email client (simply called “Email” in your App Launcher) with your University of Toronto email account. Ready?

1. Fire up the Android Email client app.
2. Press the Menu button on your Android device.
3. Choose “Add Account”.
4. Fill in your full email address (john.doe@utoronto.ca) and current password.
5. Choose the “Manual Setup” button (it will become available after you have filled in your email and password).
6. Pick an IMAP account.
7. For your username, do not use your email address. Use the username for portal login instead, without specifying a domain (so just “doejhon” and *not* “jhon.doe@utoronto.ca” !)
8. To fill in the “server” field, get the address of the mailserver hosting your account from your UTor ID Information Page ( https://www.utorid.utoronto.ca/cgi-bin/utorid/info.pl ). It will appear as “mailbox###.utcc.utoronto.ca” and different mailboxes reside on different servers. We need to know yours.
9. Choose SSL for encryption.
10. Choose port 993 (if it was not chosen automatically).
11. Click “Next”.
12. Use same username/password as for incoming mail in step #7.
13. Enter the SMTP server as “smtp.utoronto.ca”.
14. Choose TLS encryption.
15. Choose port 587.

You are done. Faculty spam (along with the odd useful message) will now begin flowing to and from your Android device!

Making an Ethernet Loopback Adapder

This post will be short (and sweet). Ethernet Loopback Adapters are little handy pieces of equipment that route the transmitting pins in an Ethernet jack back to the receiving pins in the same jack. This is good for testing link connectivity on an Ethernet card – if the adapter can establish a link with itself the hardware on the adapter is probably OK. In real life, this can save you hassle quickly testing ADSL modems, routers, switches, desktops and laptops without plugging the device into another jack to get the link light to come on.

This how-to will use an existing Ethernet cable which will be converted to a Loopback cable.  There are lots of guides and video online about creating such an adapter using an Ethernet jack and wires, but this requires having an uncrimped  Ethernet jack, some wires, and a crimper. In my case, a trip to the store to buy the components I already have at home on ready Ethernet cables seemed wasteful (most people will have a cable or two, or can buy a short cat 5 cable for under 2 dollars). I strongly recommend using a cable with a broken or missing jack – after all, we only need one Ethernet jack which is properly wired to a cat5 or higher cable.

1. Cut the cat5 cable a few ( 2 or 3 ) inches from the jack.

2. Strip about an inch from the shielding of the cat 5 cable, revealing 8 separately shielded color coded wires inside.

3. Strip about half an inch from the shielding on four wires: green-white, green, orange-white, and orange.

4. Twist the green-white and orange-white stripped ends together, connecting pin 1 to pin 3.

5. Twist the green and orange stripped ends together, connecting pin 2 to pin 6.

If you have some tape, you may want to cover the tips up. Otherwise, make sure the tips don’t touch each other. The end result will look like this:

twisted pairs on an Ethernet loopback adapter

You can test your new loopback adapter in any working Ethernet jack by plugging it in!

You can see the Port line is on for a wireless router with the loopback adapter plugged in:

Link light on with loopback adaper plugged in

Monitoring Amazon EC2 instances and other Cloud Resources with Hyperic HQ (and other monitoring platforms)

I’ve had to tackle this task recently and could not find a write-up. Nice folks from Hyperic, and others on Twitter, suggested OpenVPN or an SSH tunnel. I opted for the second option, and after setting up two tunnels and properly configuring the agent, I now have an Amazon EC2 Windows instance show up as a platform in my Dashboard. Note that those instructions will work for other software (Zabbix comes to mind). Here’s how you can have yours too:

1. Install an SSH server on the to-be-monitored cloud instance. For Linux, OpenSSH is easy to install and setup, and usually already comes with most distributions. All you have to do is create a user and a password, or keys. On Windows, CopSSH will do the trick – you just have to add a new user and configure it through the CopSSH control panel. Make sure the SSH server runs, and the login credentials work.

2. Install an SSH client on your Hyperic HQ server. For Linux, again, OpenSSH will do the trick and is most likely already there. For Windows, try CygWIN or PUTTY.

3. Designate a unique name for localhost in the hosts file of both the Hyperic server and the cloud instance. In Linux, it would be under /etc/hosts. In windows, it moves between versions but is usually under C:\Windows\system32\drivers\etc\hosts . Call it cloudagent1. The line should look like this:

127.0.0.1     localhost cloudagent1

4. From the Hyperic server, initiate an SSH tunnel which forwards two ports. First from the cloud instance to the Hyperic server (usually on port 7443). Second from the Hyperic server to the cloud instance, to the port on which the Hyperic agent runs. If you already have a Hyperic agent on your Hyperic server, you MUST use a different port. As the local agent usually runs on port 2144, you may want to pick something like port 22144. With OpenSSH on CygWin and Linux you can create the tunnels like this (assuming your username is “user” and your cloud instance is “cloud-instance.com”):

$ ssh user@cloud-instance.com -R 7443:cloudagent1:7443 -L 22144:cloudagent1:22144 -N -f

5. Configure the Hyperic agent on your cloud instance to use port 22144. The rest of the settings can be copied from your locally monitored agents. You can use “cloudagent1″ (or whichever name you have assigned to the localhost) in the configuration.

Hope this helped!

The Quest For The Fastest Linux Filesystem

What’s this thing about?

This post has a few main points:

1. Speeding up a filesystem’s performance by setting it up on a tuned RAID0/5 array.

2. Picking the fastest filesystem.

3. The fastest format options for Ext3/4 or XFS filesystems.

4. Tuning an Ext3/4 filesystem’s journal and directory index for speed.

5. Filesystem mount options that increase performance, such as noatime and barrier=0.

6. Setting up LILO to boot from a RAID1 /boot partition.

The title is a bit of an oversimplification The article is intended to keep being work in progress as “we” learn, and as new faster tools become available. This article is not intended to cover the fastest hardware (yet). The goal is the “fastest” filesystem possible on whatever device you have. Basically, “we” want to setup and tweak whatever is possible to get our IO writes and reads to happen quicker. Which IO reads? random or sequential? long or short? The primary goal is a quick Linux root filesystem, which is slightly different than, lets say, a database-only filesystem, or a /home partition for user files. Oh, and by the way, do not use this on your production machines, people. Seriously.

RAID

WTF is RAID?!

The first question is, how many devices would you like your filesystem to span? The simple and correct answer is – the more the faster. To use one filesystem across multiple devices, a single “virtual” device can be created from multiple partitions with RAID. (Recently developed filesystems, like BTRFS and ZFS, are capable of splitting themselves intelligently across partitions to optimize performance on their own, without RAID) Linux uses a software RAID tool which comes free with every major distribution – mdadm. Read about mdadm here, and read about using it here. There’s also a quick 10 step guide I wrote here which will give you an idea about the general procedure of setting up a RAID mdadm array.

Plan your array, and then think about it for a while before you execute – you can’t change the array’s geometry (which is the performance sensitive part) after it’s created, and it’s a real pain to migrate a filesystem between arrays. Not to mention a Linux root filesystems.

Deciding on a performance oriented type of RAID ( RAID0 vs. RAID5 )

The rule of thumb is to use 3 or more drives in a RAID5 array to gain redundancy at the cost of a slight performance loss over a RAID0 array (10% CPU load at peak times on my 2.8 GHz AthlonX2 with a 3 disk RAID5 array). If you only have 2 drives, you cannot use RAID5. Whatever your situation is, RAID0 will always be the fastest, but less responsible, choice.

RAID0 provides no redundancy and will fail irrecoverably when one of the drives in the array fails. Some would say you should avoid putting your root filesystem on an un-redundant array, but we’ll do it anyways! RAID0 is, well, the *fastest* (I threw that caution to the wind and I’m typing this from a RAID0 root partition, for what it’s worth). If you are going to be or have been using a RAID0 array, please comment about your experiences. Oh, and do backup often. At least weekly. To an *external* drive. If you only have one drive you can skip to the filesystem tuning part. If you do are going to use RAID0/5, remember to leave room for a RAID1 array, or a regular partition, for /boot. Today, LILO cannot yet boot a RAID0/5 array.

Deciding on a RAID stripe size ( 4 / 8 / 16 / 32 / 64 / 128 / 256 … )

You will need to decide, for both RAID0 and RAID5, about the size of the stripe you will use. See how such decisions affect performance here. I find the best results for my personal desktop to be 32kb chunks. 64 does not feel much different. I would not recommend going below 32 or above 128 for a general desktops root partition. I surf, play games, stream UPnP, run virtual machines, and use a small MySQL database. If I would be doing video editing, for example, a significantly bigger stripe size would be faster. Such specific usage filesystem should be setup for their own need and not on the root filesystem, if possible. Comments?

RAID 5 – deciding on a parity algorithm ( Symmetric vs. Asymmetric )

For RAID5, the parity algorithm can be set to 4 different types. Symmetric-Left, Symmetric-Right, Asymmetric-Left, and Asymmetric-Right. They are explained here, but they appear to only affect performance to a small degree for desktop usage, as one thread summarized.

Creating a RAID0 array

Using the suggestions above, the command to create a 2-disk RAID0 array for a root partition on /dev/md0 using the partitions /dev/sda1 and /dev/sdb1 should look like this:

# mdadm --create /dev/md0 --metadata=0.90 --level=0 --chunk=32 --raid-devices=2 /dev/sd[ab]1

Note the –metadata option, which with 0.90 specifies the older mdadm metadata format. If you will use anything other than 0.90, you will find Lilo failing to boot.

The Fastest Filesystem – Setup and Tuning

Deciding on a Filesystem ( Ext3 vs. Ext4 vs. XFS vs. BTRFS )

The Ext4 filesystem does seem to outperform Ext3, XFS and BTRFS, and it can be optimized for striping on RAID arrays. I recommend Ext4 until BTRFS catches up in performance, becomes compatible with LILO/GRUB, and gets an FSCK tool.

Deciding on a Filesystem Block Size ( 1 vs. 2 vs. 4 )

It is impossible to stress how important this part is. Luckily, if you don’t know what this is and just don’t touch it, most mkfs tools default to the fastest choice – 4kb. Why you would not want to use 1 or 2 is neatly shown in the benchmarking results of RAID performance on those block sizes. Even if you are not using RAID, you will find 4kb blocks to perform faster. Much like the RAID geometry, this is permanent and cannot be changed.

Creating an optimized for RAID Ext4 ( stride and stripe-width )

Use those guidelines to calculate these values:

stride = filesystem block-size / RAID chunk.
stripe-width = stride * number of drives in RAID array ( - for RAID0, and that minus one for RAID5 )

pass the stride and the stripe-width to mkfs.ext4, along with the block size in bytes, like this:

# mkfs.ext4 -b 4096 -E stride=8,stripe-width=16 /dev/md0

A handy tool to calculate those things for you can be found here.

Creating an optimized XFS filesystem ( sunit and swidth )

The XFS options for RAID optimization are sunit and swidth. A good explanation about those two options can be found in this post. A quick and dirty formula to calculate those parameters was taken from here:

sunit = RAID chunk in bytes / 512
swidth = sunit * number of drives in RAID array ( - for RAID0, and that minus one for RAID5 )

The sunit for a 32kb (or 32768 byte) array would be 32768 / 512 = 64

The command to create such a filesystem for a 32kb chunk size RAID0 array with 2 drives and a 4096 (4kb) block size will look something like this:

# mkfs.xfs -b size=4096 -d sunit=64,swidth=128 /dev/md0

Tuning the Ext3 / Ext4 Filesystem ( Journal )

There’s a good explanation about the 3 modes in which a filesystem’s journal can be used on the OpenSUSE Wiki. That same guide will rightly recommend avoiding writing actual data to the journal to improve performance. On a newly created but unmounted filesystem, disable the writing of actual data to the journal:

# tune2fs -O has_journal -o journal_data_writeback /dev/md0

Turning on Ext3 / Ext4 Directory Indexing:

Your filesystem will perform faster if the directories are indexed:

# tune2fs -O dir_index /dev/md0
# e2fsck -D /dev/md0

Filesystem Mounting Options ( noatime, nodiratime, barrier, data and errors options ):

Some options should be passed to the filesystem on mount to increase its performance:

noatime, nodiratime – Do not log access of files and directories.

barrier=0 – Disable barrier sync (Only safe if you can assure uninterrupted power to the drives, such as a UPS battery)

errors=remount-ro – When we have filesystem errors, we should remount our root filesystem readonly (and generally panic).

data=writeback – For Ext3 / Ext4. If your journal is in writeback mode (as we previously advised), set this option.

My fstab looks like this:

/dev/md0         /                ext4        noatime,nodiratime,data=writeback,stripe=16,barrier=0,errors=remount-ro      1   1

And my manual mount command will look like this:

# mount /dev/md0 /mnt -o noatime,nodiratime,data=writeback,stripe=16,barrier=0,errors=remount-ro

Did I mention to NEVER do this on a production machine?

Installing your Linux

Install as usual, but do not format the root partition you’ve setup! If you are using RAID0/5, you have to setup a separate, RAID1 or primary /boot partition. In my experience, the leaving the boot partition unoptimized does not affect regular performance, but if you are keen on shaving a few milliseconds off your boot-time you can go ahead and tune that filesystem yourself as well.

Making sure LILO boots

If you are using RAID0/5 for your root partition, you must setup a separate non-RAID or RAID1 partition as /boot. If you do setup your /boot partition to be on a RAID1 array, you have to make sure to point lilo to the right drive but editing /etc/lilo.conf :

boot = /dev/md1

and make sure LILO knows about the mirroring of the /boot partitions by adding the line:

raid-extra-boot = mbr-only

Then, LILO must be reinstalled to the Master Boot Record while the /boot partition is mounted on the root partition. From a system rescue CD, with a properly edited lilo.conf file this will look something like this:

# mount /dev/md0 /mnt
# mount /dev/md1 /mnt/boot
# /mnt/sbin/lilo -C /mnt/etc/lilo.conf

… and reboot.

Experience and thoughts:

I’ve been following my own advice for the last couple of weeks. The system is stable and best of all, *fast*. May those not be “famous last words”, but I’ll update this post as I go. The only thing we all really need is comments and input. If you use something else that works faster for you – let us know. If something downgraded your stability to the level of Win98, please let us know. More importantly – if you see any errors, you got it – let us know.

TO DO:

Test this interesting post about Aligning Partitions

Test BTRFS on 2 drives without RAID/LVM

Checking for Rootkits

What are rootkits? You can ask Wikipedia, or take this over-simplification:

The Windows flock has to worry about various security concerns – viruses, worms, trojans, bots. So may, in fact, they had been categorized into AdWare (unwanted commercial content), SpyWare (compromise of personal information) and Viruses (generally malicious code).

Linux folks, on the other hand, have to fear only one thing – a rootkit. Because Windows vulnerabilities allow exploiting a machine without administrative access for specific goals, a diversity of exploits was invented, while properly setup Linux can only be compromised with root (Administrator) access. However, once that had been achieved, the machine is fully compromised along with the network it’s on. Namely, there are very few things an attacker will not be able to do with a rooted Linux machine.

Periodic scans are a good measure to stay clean, similar to scheduled anti-virus scans in Windows PCs. chkrootkit is such an implementation, and appears to be the most popular today, for good reasons: it’s easy to build, easy to run, and easy to understand. A good idea might be to run it periodically and email the output to a system administrator.

Great, until you realize the catch22 of checking for root-kits: Once a machine had been infected with a rootkit (usually through an badly secured SSH account), there’s virtually no way to determine that is was compromised, unless a reference set of binaries can be used to compare the infected binaries to “clean” ones. This means that you may not use a machine to check itself for rootkits, because if it was already compromised at the time it was checked, the check would be useless.

There is, of course, a hack. Namely, Network Security Hack #99 from O’Reilly suggests we can run chkrootkit in Busybox! Right, Busybox has some 200 programs built into its binary, and we could use those instead of the host’s own suspected ones. Unfortunately, it does not show how.

Let’s think about this: We assume the machine might be compromised when we scan it. We would not want to use the machine’s own binaries (that could also be compromised). We would not want to use software that had been lying around on the machine while it was compromised, for the same reason, including Busybox or chkrootkit. Ideally, then, we would use a fresh copy of Busybox to download a fresh copy of chkrootkit, run one in the other and dump both (that way, one cannot compromise something that is not there).

I ended up with the (almost entirely self-sufficient) solution below. It will run on any machine that has internet access, GNU wget (to download Busybox and chkrootkit), and GNU make (to build chkrootkit). Other than wget and make, the script makes use of Busybox applets instead of the system installed binaries, for extra safety

#!/bin/sh

# Temporary folder path. (It's a good idea to use a random name):
TEMPORARY_FOLDER=/tmp/security-$RANDOM

# A URL from which Busybox binaries can be downloaded
BUSYBOX_URL=http://www.busybox.net/downloads/binaries/1.17.2

# A URL from which the chkrootkit source can be downloaded:
CHKROOTKIT_URL=ftp://ftp.pangeia.com.br/pub/seg/pac

# Download and prepare the Busybox binary in the temporary folder:
/usr/bin/mkdir -p $TEMPORARY_FOLDER
cd $TEMPORARY_FOLDER
/usr/bin/wget $BUSYBOX_URL/busybox-"`uname -m`"
/usr/bin/chmod +x $TEMPORARY_FOLDER/busybox-"`uname -m`"
BUSYBOX_BIN=$TEMPORARY_FOLDER/busybox-"`uname -m`"

# Download, confirm the MD5 sum, and extract the chkrootkit source using Busybox:
$BUSYBOX_BIN wget $CHKROOTKIT_URL/chkrootkit.tar.gz
$BUSYBOX_BIN wget $CHKROOTKIT_URL/chkrootkit.md5
if [ "`$BUSYBOX_BIN cat chkrootkit.md5`" != "`$BUSYBOX_BIN md5sum chkrootkit.tar.gz`" ]; then
  $BUSYBOX_BIN echo " !!! MD5 SUM check FAILED !!! "
  exit
else
  $BUSYBOX_BIN echo "MD5 checksum for chkrootkit source passed..."
fi
$BUSYBOX_BIN tar vxf $TEMPORARY_FOLDER/chkrootkit.tar.gz

# Build chkrootkit. (It would have been better to run make in Busybox, but that fails):
/usr/bin/make -C $TEMPORARY_FOLDER/chkrootkit* sense

# Run the newly built chkrootkit in the just downloaded Busybox:
$BUSYBOX_BIN sh $TEMPORARY_FOLDER/chkrootkit*/chkrootkit

# Clean up:
cd /
$BUSYBOX_BIN echo "Done."
$BUSYBOX_BIN rm -rf $TEMPORARY_FOLDER

Eavesdropping with Linux – A BigBrother Network Bridge (or: kiss your privacy goodbye)

All wires can be tapped. Luckily Ethernet wires are no exception. This How-to will cover the fastest way to get everything you need for eavesdropping on your network neighbors: setting up 2 bridged Ethernet adapters and configuring them for an uninterrupted network flow, which then can be monitored or shaped.

The most reliable way to achieve full transparency and control over a specific network segment is to have a full featured operating system on a network bridge (a HUB or a Switch) – Having network tools running locally on a relay point is hard to beat. There are lots of commercial grade relay points you can purchase, but any Cisco router which will give you network management tools is likely to still be lacking, and expensive. In other words – good-luck running graphical or 3rd party propriety applications on your 300$ switch.

This How-to will leave you with a box which will always come up with 2 bridged adapters. There will be uninterrupted network flow between the box and the network segment it fragments. There will be remote access and tools to play with. It shall be a real privacy violator. Let’s get to it!

You will need:

• 1 Desktop/Workstation/Server
• 2 Ethernet cards
• Linux distribution (we like Slackware Linux) with this software installed:
  • net-tools
  • bridge-utils
  • iptables
  • X11 (recommended for VNC)
  • xfs (recommended for VNC)

1.  Prepare Ethernet adapters for bridging:

The about to be bridged adapters should have no assigned IPs and placed in promiscuous mode:

# ifconfig eth0 0.0.0.0 promisc up
# ifconfig eth1 0.0.0.0 promisc up

2. Bridge the Ethernet adapters using bridge-utils:

# brctl addbr br0
# brctl addif br0 eth0
# brctl addif br0 eth1

3. Configure and initialize the new network bridge:

# ifconfig br0 192.168.0.233 netmask 255.255.255.0 broadcast 192.168.0.255 up

4. Ensure uninterrupted flow between adapters and bridge:

# iptables -A INPUT -i eth0 -j ACCEPT
# iptables -A INPUT -i eth1 -j ACCEPT
# iptables -A INPUT -i br0 -j ACCEPT

5. Setup networking for the eavesdropping machine itself:

# route add default gateway 192.168.0.1 br0
# echo "nameserver 204.11.104.3" > /etc/resolv.conf

6. Install a TightVNC server for remote access:

Having access to the graphic environment on the eavesdropping machine is very handy, when you consider the awsome GUI of Wireshark and Zenmap (or the lame fact Wireshark MUST have a GUI to run). TightVNC is the best VNC server out there, and it happens to be free! Get the source, compile and install, then start the X Font Server and TightVNC:

# xfs -daemon
# vncserver :1

7. Install Wireshark (and other tools)

This is where it all comes together – the power of having color-coded packet entries fly by your eyes at the speed of lite, or, well, electricity, is hard to beat. Wireshark is the best network packet analyzer out there. Luckily, it’s also the easiest to run. Get the source, compile and install. ZenMap also goes a long way on such a setup, but as it ships with Slackware, we’ll leave those steps out.

8. Script the whole thing:

This bit is here only to ensure you’ll keep eavesdropping after a reboot The easiest way to achieve this is to dump all the above commands into a file, make it executable, and place an entry for it in /etc/rc.local.

You can use this tested rc.bigbrother script if you are feeling lazy. It will do all of the above and more. Just place it anywhere and add to /etc/rc.local.

Screen-shot:

Screen-shot of BigBrother in action

A Linux Tablet PC – Slackware 13 on the HP Pavilion tx2500

This post is a brief summary of how to get your Linux running on a tablet PC. We’ll be using the HP Pavilion tx2500 and Slackware13. These steps can be adapted for other tablets. They should be easier on distribution such as OpenSUSE or Ubuntu, where a lot of the system configuration is done automatically behind the scenes. Overall, you may find they all worked for you with minimal tweaking, or, that you could use one or 2 bits to help you where you got stuck.

This document will cover the ‘few’ bits that do not work out of the box with a pure GNU/Linux, mostly due to them being propriety hardware: Graphics (FGLRX), WiFi (Broadcom), Touch/Stylus (Wacom), Touchpad (Synaptics) and Sound (Intel HD). This how-to should leave you with a fully working tablet, minus a fingerprint reader.

1. ATi’s FGLRX:

ATi’s Radeon 3200 is a propriety chip-set, but the ATi drivers outweigh their open-source brother when it comes to 3D performance. Download the latest FGLRX driver from ATi, make the binary executable, and run it with the ‘–buildpkg Slackware/All’ flag. This will create a .txz package you can install with pkgtool. Run ‘aticonfig –initial’ and ‘ldconfig’ as root, and reboot. If FGLRX is not loaded, ‘modprobe fglrx’.

2. Broadcom’s WL:

Well, what can we say, Broadcom’s 4328 R3 wireless chip-set is also propriety hardware. To get your WiFi working, download the latest Linux STA 802.11 driver from Broadcom, unpack and compile with ‘make’. You must unload the ‘ssb’ module (if loaded), and you must load ‘lib80211_crypt_tkip’, before you may load the newly built wl.ko module with ‘insmod /path/to/wl.ko’.

3. Stylus/Touch and Touchpad

This part is not much different in practice, but very different in essence – we’ll be using opensource drivers from the Linux Wacom project. Download the latest LinuxWacom version from the Linux Wacom Project, and build with X.org support. Then place the HAL and UDEV rules along with the xorg driver at their designated locations:

$ ./configure --enable-wacom
$ ./make
# ./make install
# cp src/xdrv/wacom_drv.so  /usr/lib64/Xorg/modules/drivers/wacom_drv.so
# cp src/util/10-linuxwacom.fdi /etc/hal/fdi/policy/10-linuxwacom.fdi
# cp src/util/60-wacom.rules /etc/udev/rules.d/60-wacom.rules
# cp src/2.6.27/wacom.ko /opt/

Load the new module you made:

# insmod /opt/wacom.ko

Now, we have to modify /etc/X11/xorg.conf to let it know about those new inputs by adding new ‘InputDevice’ sections and adding them to ‘ServerLayout’. This involves identifying the correct /dev/input/event# for the touch, stylus and eraser. You can do it by watching the dump from each event file while (read: at the same time as) you  send input events by dragging the pen across the screen:

# cat /dev/input/event8

If you are running into difficulties with your xorg.conf, you can use this file for refrence: HP_Pavilion_tx2500.xorg.conf

Calibrate your tablet with ‘xwacomcpl’. Download and compile CellWriter for handwriting recognition.

Screen Rotation script:

Since this is a tablet, screen rotation is pretty handy. Because we’re using Wacom modules, we will also need to change the oriantation of the wacom devices. Basicaly, xrandr can be used in conjunction with xsetwacom to rotate the tablet and the screen together. Here’s how it works – the following lines will rotate everything “right”:

# xrandr -o right
# xsetwacom set WacomTouch rotate 1
# xsetwacom set WacomStylus rotate 1
# xsetwacom set WacomEraser rotate 1

You have to match the device name to the device identifier in xorg.conf, and use the right path for xsetwacom. Instead of “right” for xrandr you may use “left” “inverted” or “normal”, and instead of “rotate 1″ for xsetwacom you may use 0 for normal, 2 for left and 3 for inverted. Most people script this together into something, like the example I made here: wacom_rotate.sh

4. Sound

This was the easiest bit to get working. All it took, is running alsaconf, and selecting the ‘snd-hda-intel’ option.

5. Automate

The easiest way is usually to put all the stuff that is not scripted on its own into ‘/etc/rc.d/rc.local’, or for some distribution ‘/etc/rc.local’ (That would probably include loading ‘wl.ko’ and ‘wacom.ko’).

I hope you found this helpful. Leave comments, or ask for help if you crashed your Linux following these instructions

RAID for Everyone – Faster Linux with mdadm and XFS

I crashed my Linux. It took a lot of skill and root access, but I’ve accidentally hosed my desktop and backtracking will be more time consuming than running through a quick Slackware install. If you find yourself in this situation, and have more than one drive on your machine, it makes sense to RAID the drives. RAID will either greatly increase performance for the drives, which are the bottleneck of any desktop, or mirror the drives for disk failure protection. To read more about RAID, which becomes more and more popular, try The Linux Software RAID How To.

This quick how-to will try to cover the basics, but all the basics, needed to install any Linux Desktop distribution on any machine with 2 or more drives. It begins with installing a Linux system on a RAID1 partition, and continues with adding a RAID0 home partition after the install. For the home partition, XFS will be used as a file-system, and tweaked to illustrate some of its strengths with RAID. Finally, It’ll cover replacing a failed drive in an array. Any bits of it will try to be relevant to other scenarios. Mostly, it will attempt to demonstrate how simple it is to administer RAID arrays with mdadm.

Why software RAID (mdadm)? Chances are, your motherboard already comes with an on-board RAID controller. Those are present on motherboards as cheap as 60$. I won’t be using mine, however, and this tutorial will not cover that part. I had the most miserable experience with my ATI on-board RAID, which is a propriety chipset, working out of the box only with SuSE, and failing drives left, right and center. Even if your lucky to have a decent Linux supported controller, you will still have a hard time finding a decent interface for the firmware, the options will be lacking at best, and you will not find RAID 5 and 6 options on motherboards or low end cards. You will also have no cheap way to recover data from failed controllers, spare buying the same hardware again. Propriety software cards are not even worth mentioning. Since the CPU penalty for software RAID is fairly low on modern chips, and all Linux distributions supports mdadm out of the box, that’s what I recommend.

Why Slackware? I’ll be using Slackware 13 because that’s what I like, and because Slackware install CD gives the most partitioning freedom (read: a console with all the console-tools) before the install. But this will work on anything Linux. Here it goes:

1.  What you need:

Get a Slackware CD/DVD ready, or any Linux installation CD from which you can access a console before the installation starts. Live CDs are great! Backup your data. No, really, do it now. And we assume you have at least 2 drives installed. The size does not have to be identical, but the partition layout will be limited to that possible on the smallest of the drives.

2. BIOS:

Reboot with the Linux installation CD in the drive. In your BIOS, make sure the RAID controller is off (and that you can boot from CD/DVD).

3. Partition Drives:

Boot the CD/DVD and get to a console. Slackware just takes you to one on its own, from a Debian DVD one can access an alternative console with Alt+F2, etc, and in a LiveCD there’s probably a terminal program. Pull one up. Log in as root, or do everything with $ sudo. Here goes the destructive part. Identify the drives you’ll use with

# fdisk -l

This examples uses /dev/sda and /dev/sdb. We also assume you have no active RAID arrays on the drives you manipulate. If you do, stop them:

# mdadm --manage --stop /dev/md0

Create parition tables on drives:

# cfdisk /dev/sda

delete all partitions. Create a Swap type partition (one of two) and a Linux Raid Autodetect type partition. Write and exit. repeat on other drive(s) with the same sizes for both partitions. Note that you do not have to use the entire space right away. In this case, we will setup a root file system on a mirrored partition, for redundancy and ease with bootloaders as most recommend. Only later we’ll attach a striped (for size and speed) home partition.

4. RAID the drives:

mdadm is our weapon of choice. It’s mighty but simple. Here’s a RAID1 (mirrored) device /dev/md0, using /dev/sda2 and /dev/sdb2 (assuming /dev/sda1 and sdb1 were used for swap):

# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2

Now, RAID1 drives will take some time to rebuild (sync the mirror), depending on your partition size. I’ve seen 20GB partitions rebuilt in 15 minutes and 500GB partitions go for almost 2 hours over 2 7,200 RPM SATA drives. You can tell what the status is by glancing at /proc/mdstat:

# cat /proc/mdstat

Wait for the array to rebuild before proceeding. Many corners of the internet, including this one in the past, suggested you had to wait for an mdadm array to rebuild before using it. However logical this sounds, in reality, “the reconstruction process is transparent, so you can actually use the device even though the mirror is currently under reconstruction”, and we can move right along regardless of the type of RAID we chose:

5. Start Linux install and choose /dev/md0 as your root partition. Install the OS.

6. Setting up a RAID home partition, or any other partition, is not much more complicated. We’ll use RAID 0 for home, because of the volume it provides, as well as the speed performance. We’ll be using /dev/sda3 and /dev/sdb3 for a mirrored RAID. So head over to the terminal:

# cfdisk /dev/sda3

partition as you wish, type Linux RAID Autodetect, and repeat with /dev/sdb3. Make sure the partition sizes match.

7. Set up RAID:

This is a simple striped setup across 2 partitions

# mdadm --create /dev/md1 --level=0 --chunk=256 --raid-devices=2 /dev/sda3 /dev/sdb3

here, we had to specify the RAID 0 chunk, which is the stripe size, in addition to the options we used with RAID 1. The optimal chunk size depends entirely on your primary usage.

8. Setup a file-system:

You can use anything and just skip this section, but XFS has special tweaking for RAID, and its worth taking advantage of them for performance. XFS allows specifying the partition RAID dimensions to the file-system, and takes them into consideration with file reads/writes, to match the operations. 2 parametrs are used with XFS upon creation and mounting: sunit, which is the size of each chunk in 512byte blocks, and swidth, which is sunit * amount-of-drives (…for RAID 0 and 1; that minus 1 for RAID 5; or that minus 2 for RAID 6). More about RAID and XFS can be found here. To create a matching XFS file-system:

# mkfs.xfs -d sunit=512, swidth=1024 /dev/md1

9. Move home to its new home.

To quickly move the contents of the old /home directory to the new RAID partition, simply rename the old home, create a new home, mount, and copy stuff over. We’ll put an entry in fstab to mount the file-system properly, and with no access time logging, to get the performance boost. All of this must to be done as root with all other users logged out. (If home was on a separate partition already, you must unmount it and remount it as something else rather than moving it):

# mv /home /home.old
# mkdir /home
# echo "/dev/md0     /home     xfs     noatime,sunit=512,swidth=1024    0    2" >> /etc/fstab
# mount /home
# cp -aR --preserve=all /home.old/* /home
# rm -rf /home.old

10. Thats it for the setup. Now, lets give our new RAID array a real test drive.

We can check the status of all our arrays with:

# cat /proc/mdstat

We can monitor RAID1 arrays (but not RAID0) with:

# mdadm --monitor --oneshot /dev/md0

But the most rewarding bit will be performing some speed tests with hdparm. Lets check on a speed performance of a single drive:

# hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads:  366 MB in  3.01 seconds = 121.45 MB/sec

Compare this to the speed of our RAID 0 array:

# hdparm -t /dev/md1
/dev/md1:
 Timing buffered disk reads:  622 MB in  3.01 seconds = 206.71 MB/sec

Yup, that’s right folks — the speed on a 2 drive RAID 0 array is twice as high. That being expected, it is by no means less satisfying

Bonus: Recovering a failed drive from a RAID1 array. This will be handy for your root partition. Needless to say, you will not be able to recover anything from your RAID0, because it has 0 redundancy. With RAID1, however, the machine just keeps humming along after a drive gave up. How will you know you have a failed drive then? If the drive failed partially (repeatedly failing on some seeks but not on all), you will notice your performance degrade. You can test for performance degradation even before it becomes severe, with hdparm as explained above. If the drive failed totally, you might not notice.  It’s good to occasionally peak at /proc/mdstat to see that the array is up. In this case, the fix is easy – just pop in a new drive when the system is off. However, if you have a partially failed drive in a RAID1 array and you do not wish to wait for a reboot (very reasonable on a server that keeps working, if you could just avoid the horrible seek delays for the failing drive), you could drop it yourself in 2 commands.

My /dev/md0 is a RAID1 array which is made of /dev/sda2 and /dev/sdb2.  In my case, it was easy to see the drive  access light throwing fits, and the desktop freezing occasionally, indicating a problem with the drives. A quick run of hdparm revealed that /dev/sdb was the failing drive, as it displayed much slower reads. It caused the file-system on /dev/sdb2 to be barley accessible, which slowed my RAID1 array during writes (reads were still fast because they could be completed from the good drive alone, but writes needed to happen on both drives). So as soon as I got my desktop back from an occasional freeze, I fired up a terminal, marked the drive in the array as failed, and removed it from the array:

# mdadm --manage /dev/md0 --fail /dev/sdb2
# mdadm --manage /dev/md0 --remove /dev/sdb2

Past that point, its just a matter of powering off and replacing the drive at your earliest convenience. Once you got a new drive, pop it in, boot the system up, clone the partition table, and add the new partition to the array:

# sfdisk -d /dev/sda | sfdisk /dev/sdb
# mdadm --manage /dev/md0 --add /dev/sdb2

… watch the array rebuild itself by looking at /proc/mdstat , and you’re done. Phew.

I hope the minimal amount of code and steps will demonstrate how easy it is for any person with 2 hard drives to enjoy the benefits of RAID, which will make your Linux Desktop even faster/safer, without investing any significant amount of time or money.

Are YOU paranoid? 10 steps for encrypting your Ubuntu HOME.

Are you paranoid? I am.

If you keep most of your work on a net-book/laptop, you should consider the possibility of having it lost or stolen. All the backups in the world will not prevent someone else from having full access to all your personal and embarrassing information. One way to get around this is to encrypt your hard drive. The safest way is to encrypt the whole drive before the operating system is installed. Encrypted installation is offered by Debian, and soon by SUSE.

To be really safe, you should keep check-sums of your unencrypted boot sectors, etc, on your encrypted partition, to prevent tempering with the few unencrypted bits. But, if you didn’t piss the KGB off very recently, you might feel safe with encrypting only the folder that contains your user documents. Under Windows Vista +, that would be \Users\YOUR USER NAME, but I suspect that would inevitably leave some loose ends. One way or another, if your are even slightly paranoid, you are probably not using Windows. On Linux, encrypting your home directory is probably reasonable enough.

Here are 10 steps to accomplish this on Debian/Ubuntu systems (Adapt to your Linux/BSD/OpenSolaris):

1. Install the packages: initramfs-tools, hashalot and lvm2:

$ sudo apt-get install initramfs-tools hashalot lvm2

2. load (or make sure have been built into the kernel) the modules: aes-x86_64 or aes-i586, dm-crypt and dm-mod:

$ sudo modprobe aes-x86_64 dm-crypt dm-mod

3. Create (or designate) a partition that would be encrypted. Allow enough room because I’m not sure growing it later is an option. Any data on it will be destroyed:

$ sudo fdisk /dev/YOUR DRIVE ON WHICH THE TO-BE-ENCRYPTED PARTITION IS

4. Check for bad blocks:

$ sudo /sbin/badblocks -c 10240 -s -w -t random -v /dev/YOUR TO-BE-ENCRYPTED PARTITION

5. Fill you to-be-encrypted partition with random data. Note: (This takes AGES, but makes things safe. By ages I mean ~ 2 hours for every 10 GB):

$ sudo dd if=/dev/urandom of=/dev/YOUR TO-BE-ENCRYPTED PARTITION

6. Setup an encrypted luks volume

$ sudo cryptsetup -y –cipher aes-cbc-essiv:sha256 –key-size 256 luksFormat /dev/YOUR TO-BE-ENCRYPTED PARTITION

7. Unlock it:

$ sudo cryptsetup luksOpen /dev/YOUR TO-BE-ENCRYPTED PARTITION pvcrypt

8. Create a volume group and a volume:

$ sudo pvcreate /dev/mapper/pvcrypt
$ sudo vgcreate vg /dev/mapper/pvcrypt
$ sudo lvcreate -n VOLUME-NAME -L VOLUME-SIZE vg

9. Create a file-system on /dev/mapper/vg-VOLUME-NAME:

$ sudo mkfs.ext3 /dev/mapper/vg-VOLUME-NAME

10. Edit /etc/fstab. Add the line:

mount -t ext3 /dev/mapper/vg-VOLUME-NAME /home

9. Edit /etc/crypttab. Add the line:

pvcrypt /dev/YOUR-ENCRYPTED-PARTITION none luks,retry=1,lvm=vg

10. This is the fun part: log out all users, switch to a console and login as root. Move the /home directory to be /home-SOMETHING. Create a new empty /home directory, and mount the encrypted volume in it. Then copy the entire contents of your /home-SOMETHING into /home preserving all attributes, times and ownerships. Here’s a few simple steps to do it:

$ sudo mv /home /home-plain

$ sudo mkdir /home

$ sudo mount /dev/mapper/vg-VOLUME-NAME /home

$ sudo cp -aR –preserve=all /home-plain/* /home/

Ready to see if it worked? Reboot! you should do this from the console directly:

$ sudo /sbin/reboot

Upon boot up, when your system tries to mount your /home partition, which is now encrypted, you will be asked for a password before booting continues. After the correct password is supplied, the system boots on.

If you have more locations you would like to encrypt, you can create more volumes on the encrypted volume group. To understand how to, or for a detailed guide for installing Ubuntu Linux on an encrypted volume group  to begin with, see the page from which I’ve adapted the steps above.