Ubuntu 10.04 might bring a new look, but it breaks a few things too.

UPDATE 16/09/2010 | Despite 3ware updating its software to 10.2, the name resolution bug still exists. Webmin’s update to 1.520 fixes the Samba issue though. You’ll either need a clean install, or to follow these comments for the workaround.

Samba, for one, is now treated entirely differently, and this of course breaks a number of things including Webmin. Apparently this will be addressed in the next release of Webmin, whenever that is. In the meantime, a distro upgrade from 9.10 to 10.04 will likely bork your Samba like it did mine, requiring a clean install to get things working again.

Another thing also broken in 10.04 is a library that affects name resolution, which plays havoc with 3DM2′s email notification feature, crashing 3DM2 in the process. There’s a simple workaround for this: use your mail server’s IP instead of its name. Not optimal, but it will work until the Ubuntu devs catch up and stop palming the problem off to other devs.

New 3DM2 + CLI 10.1

One thing I did notice during the reinstall was that a new version of 3ware’s 3DM2 + CLI package was out, 10.1. Of course 3ware has never explicitly claimed Ubuntu support, so moving off the trusted 9.5.3 was a bit of a concern. I did it anyway, it worked fine (name resolution bug aside) and here’s how you can too.

The installation package has now moved to 100% text, away from the previous Installshield efforts. It’s fairly trivial to set up too. Not Windows trivial, but then, nothing ever is on Linux.

First, uninstall any previous version of 3DM2 — how you do this will depend on the distribution you’ve used. If you’ve used a .deb file, you should find it in Synaptic Package Manager, but if you’ve used the old Installshield package, you’ll need to open up a terminal and issue the following commands:

cd /opt/AMCC/_uninst
sudo ./uninstaller.bin

And follow the prompts. This assumes you installed to the default path of /opt/AMCC, if you’ve installed elsewhere, you’ll need to find uninstaller.bin yourself. Next, you’ll want to download the 3DM2 & CLI Linux 10.1 code set from 3ware.

Open a terminal and navigate to the directory that 3DM2_CLI-Linux-10.1.zip is in.

Type:

unzip 3DM2_CLI-Linux-10.1.zip -d 3dm2
cd 3dm2
chmod +x install.sh
sudo  ./install.sh --install

Then follow the prompts. If you’re still having grief, our old friend has made some .deb files for us.

After a successful install, simply open a browser, point it at https://127.0.0.1:888 and log in with the default password 3ware. Just remember to use your mail server’s IP instead of its host name.

Firmware aside, 3ware splits its application into two parts — its command line tool tw_cli and its web management tool 3dm2. While tw_cli is perfectly fine for managing the array directly, you’ll need 3dm2 to setup mail notifications and scheduled maintenance.

3ware's 3dm2 is where most of your card management will be done, as the CLI is sadly limited.

Until version 9.5.3 was released at the end of November, 3ware’s tools simply didn’t install in Ubuntu 64-bit thanks to a broken installer. To get things working, you needed a third party release.

3ware’s install is simply weird; rather than simply provide a .deb file, once you’ve extracted the .tar.gz “Linux” bundle you’re presented with a .bin file. After making it executable, and running it:

chmod +x setupLinux_x64.bin
sudo ./setupLinux_x64.bin

A Java runtime environment extracts, and, shock — a graphical version of Installshield loads. Who said it was just for Windows?

Installshield on Linux... who'd have thought?

After going through the process and a restart, the webserver was running on https://127.0.0.1:888 with the default password 3ware and everything was as easy as pie. Complemented with GapcMon, apcupsd, Webmin, Samba, Proftpd and Gadmin-proftpd, I’m now ready to start filling up my file server.

Except I no longer care.

Too impatient to wait for the RMA on the dead Asus P5Q-E (of which the replacement is now a spare swap-in board), thanks to an incredibly generous friend I picked up a Gigabyte GA-EP45-Extreme… which OpenSolaris b127 hated, and refused to boot with. After a few days of hair pulling and switching off almost everything I could in the BIOS to try and rectify the issue, I finally admitted OpenSolaris was not to be.

The Gigabyte GA-EP45 Extreme, great board, hated by OpenSolaris

Not willing to risk Nexenta, I dropped to FreeBSD 8, the last bastion of ZFS hope (no folks, FUSE does not count).

FreeBSD worked wonderfully from a compatibility front, but I soon discovered that when it came to virtualisation, it had the same options as a prisoner faced with the Spanish inquisition: basically none. There is, ironically, a version of Sun’s VirtualBox floating around, but it’s a hack job that hates 64-bit, and like most things FreeBSD if you’re not running from the command line you’re asking for pain.

And so, hoping that one day Larry Ellison would open up ZFS licensing a little more so the GPL crowd would stop whining and just integrate it already, I sighed, flicked the 3ware 9650SE into hardware RAID 6 and reached for the Ubuntu 9.10 64-bit disc.

It worked.

Post mortem: List of controller cards that will work with OpenSolaris

While I note with grim satisfaction that Areca has still failed to produce a Solaris driver for it’s ARC-1300ix series, here’s a list of PCI-Express cards known to work with OpenSolaris without requiring any RAID 0/JBOD workarounds, and being able to control at least eight drives.

1. LSI SAS3081E-R
2. Intel SASUC8I flashed with the SAS8031E-R’s IT (initiator target) firmware
3. 3ware 9650SE series

Tiny, yes? The last, which I ended up with due to non-availability of the first two in Australia, is significantly more expensive as it has hardware RAID capability as well.

Post mortem: Final system

Rack: HP 10622
OS: Ubuntu 9.10
PSU: Corsair TX-850
CPU: Intel Q9550
Memory: 8GB Corsair Dominator PC-2 8500
Motherboard: Gigabyte GA-EP45 Extreme
GPU: Geforce 7600GS silent (to be swapped out with a PCI card when a second 3ware controller card is bought)
Controller card: 3ware 9650SE-8LPML
Network card: HP NC364T
Case: Chenbro RM41416B
UPS: APC Smart-UPS 750
Switch: Netgear GS724T
System drives: Samsung HD501LJ SATA
Array drives (RAID 6 w/XFS): WD RE3 1TB x3, Samsung HD103UJ 1TB x2, Seagate 7200.11 x2, Seagate 7200.12

The only problem left is the Seagate 7200.12, which seems to keep dropping from the array. I’ll have to see if a firmware update to the 3ware card fixes it, otherwise I may need to swap in a new drive (Update: turns out the ridiculously expensive Mini-SAS to SATA cables I bought were dodgy. Upon replacing, I’ve had no dropouts).