Testing AoE over SCSI

In a previous post, I demonstrated that it was possible to serve an LVM logical volume over ATA-over-Ethernet. That particular LV was backed by a parallel ATA physical device. Now I want to see if ATA-over-Ethernet is a misnomer of sorts, and if it is in fact possible to serve an LV backed by a SCSI physical device. I suspect this to be the case.

The test system:

Intel(R) Pentium(R) 4 CPU 2.53GHz
1.0 GB RAM
Adaptec AIC-7892A U160/m (rev 02) SCSI controller
80G PATA hard disk (root file system)
5 Seagate ST373207LW 73GB U320 disks (Yes, I know I only have a U160 controller installed. I don't really care about the speed of the thing. It's a waltzing bear.)
SOYO mainboard

The SCSI disks will be configured as a RAID5 array using Linux software RAID.

mdadm --create /dev/md/0 -f --level=raid5 --raid-devices=5 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

As I am impatient:

echo 100000 > /proc/sys/dev/raid/speed_limit_max
echo 100000 > /proc/sys/dev/raid/speed_limit_min

Which brings my construction speed up to ~10,600 k/s. It will still take a good two hours to build, though.

Once the RAID has been constructed, I will then create a physical volume for LVM on it:

pvcreate /dev/md0

And create a volume group named "scsi":

vgcreate scsi /dev/md0

And create a logical volume named "raid":

lvcreate --name raid -l 70001 scsi

Note that the "scsi" VG had 70001 free extents.

Next, I'll create an ext3 filesystem on the LV:

mkfs.ext3 /dev/scsi/raid

And now, the fun part - sharing the LV via AoE:

vbladed 0 1 eth0 /dev/scsi/raid

Syslog on the test server shows that vbladed is running. Let's see what I get when I run aoe-discover and aoe-stat on my desktop (which is connected to the same network the AoE test server is on):

aoe-discover
aoe-stat

e0.1 293.605GB eth0 up

That looks pretty good, except that lvdisplay on the server says the LV is only 273.44 GB in size. However, 273.44 * 1024³ (273.44 gibibytes) == 293.604 gigabytes. So it's likely the disparity is just two different definitions of "GB". Still worth testing, though.

Anyway, now it's time to mount the AoE drive:

mount /dev/etherd/e0.1 /mnt/

No errors reported on either server. But here's the real test: 1 GiB of random data.


dd if=/dev/urandom of=/mnt/random bs=1024 count=1048576
1048576+0 records in
1048576+0 records out
1073741824 bytes (1.1 GB) copied, 169.715 s, 6.3 MB/s


No errors. Next I'll take an MD5 sum of the file, unmount the AoE share, then mount the LV on the test server. In point of fact, I'm not required to unmount the AoE share first, but I want to ensure that there is no possible way both systems would be attempting to write to the test file. Sure enough, the MD5 sums are identical. This shows that you can, in fact, export any LVM logical volume via ATA over Ethernet, no matter what the physical medium backing it. It remains to be seen, however, if it is possible to export a single partition on a SCSI drive via AoE.