Posts Tagged ‘File Server’

Getting the 3ware 9650SE working in Ubuntu 9.10 64-bit

December 13th, 2009

While the Linux kernel has included 3ware drivers that have worked perfectly for a long time, to manage your array you need access to applications.

Firmware aside, 3ware splits its application into two parts — its command line tool tw_cli and its web management tool 3dm2. While tw_cli is perfectly fine for managing the array directly, you’ll need 3dm2 to setup mail notifications and scheduled maintenance.

3ware's 3dm2 is where all your card management should be done.

3ware's 3dm2 is where most of your card management will be done, as the CLI is sadly limited.

Until version 9.5.3 was released at the end of November, 3ware’s tools simply didn’t install in Ubuntu 64-bit thanks to a broken installer. To get things working, you needed a third party release.

3ware’s install is simply weird; rather than simply provide a .deb file, once you’ve extracted the .tar.gz “Linux” bundle you’re presented with a .bin file. After making it executable, and running it:

chmod +x setupLinux_x64.bin
sudo ./setupLinux_x64.bin

A Java runtime environment extracts, and, shock — a graphical version of Installshield loads. Who said it was just for Windows?

Installshield on Linux... who'd have thought?

Installshield on Linux... who'd have thought?

After going through the process and a restart, the webserver was running on https://127.0.0.1:888 with the default password 3ware and everything was as easy as pie. Complemented with GapcMon, apcupsd, Webmin, Samba, Proftpd and Gadmin-proftpd, I’m now ready to start filling up my file server.

Popularity: 47% [?]

Sometimes giving in is easier

December 13th, 2009

OpenSolaris’ ZFS implementation recently picked up one of the tastiest things it possibly could: block level dedupe.

Except I no longer care.

Too impatient to wait for the RMA on the dead Asus P5Q-E (of which the replacement is now a spare swap-in board), thanks to an incredibly generous friend I picked up a Gigabyte GA-EP45-Extreme… which OpenSolaris b127 hated, and refused to boot with. After a few days of hair pulling and switching off almost everything I could in the BIOS to try and rectify the issue, I finally admitted OpenSolaris was not to be.

The Gigabyte GA-EP45 Extreme, great board, hated by OpenSolaris

The Gigabyte GA-EP45 Extreme, great board, hated by OpenSolaris

Not willing to risk Nexenta, I dropped to FreeBSD 8, the last bastion of ZFS hope (no folks, FUSE does not count).

FreeBSD worked wonderfully from a compatibility front, but I soon discovered that when it came to virtualisation, it had the same options as a prisoner faced with the Spanish inquisition: basically none. There is, ironically, a version of Sun’s VirtualBox floating around, but it’s a hack job that hates 64-bit, and like most things FreeBSD if you’re not running from the command line you’re asking for pain.

And so, hoping that one day Larry Ellison would open up ZFS licensing a little more so the GPL crowd would stop whining and just integrate it already, I sighed, flicked the 3ware 9650SE into hardware RAID 6 and reached for the Ubuntu 9.10 64-bit disc.

It worked.

Post mortem: List of controller cards that will work with OpenSolaris

While I note with grim satisfaction that Areca has still failed to produce a Solaris driver for it’s ARC-1300ix series, here’s a list of PCI-Express cards known to work with OpenSolaris without requiring any RAID 0/JBOD workarounds, and being able to control at least eight drives.

  1. LSI SAS3081E-R
  2. Intel SASUC8I flashed with the SAS8031E-R’s IT (initiator target) firmware
  3. 3ware 9650SE series

Tiny, yes? The last, which I ended up with due to non-availability of the first two in Australia, is significantly more expensive as it has hardware RAID capability as well.

Post mortem: Final system

Rack: HP 10622
OS: Ubuntu 9.10
PSU: Corsair TX-850
CPU: Intel Q9550
Memory: 8GB Corsair Dominator PC-2 8500
Motherboard: Gigabyte GA-EP45 Extreme
GPU: Geforce 7600GS silent (to be swapped out with a PCI card when a second 3ware controller card is bought)
Controller card: 3ware 9650SE-8LPML
Network card: HP NC364T
Case: Chenbro RM41416B
UPS: APC Smart-UPS 750
Switch: Netgear GS724T
System drives: Samsung HD501LJ SATA
Array drives (RAID 6 w/XFS): WD RE3 1TB x3, Samsung HD103UJ 1TB x2, Seagate 7200.11 x2, Seagate 7200.12

The only problem left is the Seagate 7200.12, which seems to keep dropping from the array. I’ll have to see if a firmware update to the 3ware card fixes it, otherwise I may need to swap in a new drive (Update: turns out the ridiculously expensive Mini-SAS to SATA cables I bought were dodgy. Upon replacing, I’ve had no dropouts).

Popularity: 49% [?]

Curse of the server

September 29th, 2009
Gasp! Intrigue! Another fucking motherboard!

Gasp! Intrigue! Another fucking motherboard!

I suppose at this point I really shouldn’t be surprised. The number of things that have gone wrong to date rival most government run projects.

Not covering old ground:

  1. The Areca ARC-1300ix 16 was returned on the basis of no Solaris driver and it being a glorified port multiplier. I waited for about a month for the Adaptec 31605 on back order. After showing no signs of turning up any time soon, I cancelled the order and decided to pony up for the highly featured and crazily expensive 3ware 9650SE-8LPML instead.  The day after, I find out Digicor has started distributing the SAS3081E-R again, a significantly cheaper option. Take in mind either choice locks me into a motherboard with at least three PCI-E x4 slots (due to needing 16 channels, and to equip the Intel quad gigabit Ethernet card).
  2. The moment the 3ware arrives, I plug it in, and lo and behold, the machine no longer posts. Having seen a similar behaviour on the previous server (would or would not boot based on random hardware plugged in and how many times half the male population has scratched its crotch in the last hour while the wind is blowing west), I proceeded to disconnect everything until only RAM, CPU and GPU remained — and it still didn’t boot. Being that the only remaining part from the old server was the 850W CoolerMaster PSU, I ordered a Corsair TX-850 at AU$240 to remove all doubt, plugged it in AND;
    Corsair TX-850

    The Corsair TX-850, a PSU with a beefy 12V rail and five year warranty - AU$240.

  3. Discovered that the Asus P5Q-E motherboard, which replaced the exploded MSI was the dead part, despite no sparking, despite working a week ago, despite nothing being physically wrong with the board — it just stopped posting,  meaning I’ve once again spent more cash than I have to.

    The excellent Asus P5Q-E. Alas, it was not to be.

    The excellent Asus P5Q-E. Alas, it was not to be.

Well, fuck. That’s two motherboards gone in one build, which is making me wonder if the case is shorting something somehow. At this point I’ve had the chassis for over ten months, with no working system. It’s enough to make you want to buy a prebuilt NAS.

Meanwhile the brand new Netgear GS724T, APC SmartUPS 750 and HP 22RU rack just sit there, waiting for some action…

The mighty 3ware 9650SE-8LPML.

The mighty 3ware 9650SE-8LPML, coming to a server near you - AU$915.

Netgear GS724T

Netgear GS724T - picked up for AU$280.

APC SmartUPS 750

APC SmartUPS 750 - AU$231 on eBay.

HP 10622 rack - the one I bought on eBay for AU$180 likely has a bit more wear and tear than this image.

HP 10622 rack - the one I bought on eBay for AU$180 likely has a bit more wear and tear than the one in this image.

Popularity: 29% [?]

Silencing the chassis, perpetual controller issues

May 26th, 2009

The Chenbro RM41416B is a 4U file server case. Thus it has fans built for pushing around huge volumes of air, with complete disregard for the auditory senses of normal human beings. Obviously for a file server in a house, this is an issue.

The case by default comes kitted out with five Y.S. Tech FD128032HB 80mm fans, with room for another two at the back to draw hot air out. Upon spinning up, these were loud enough to shatter a pensioner’s dentures at 50 paces. The decision was made to swap them out with Noctua NF-R8s —certainly not a one to one replacement in terms of air flow, but a heck of a lot quieter.

The Noctua NF-R8 is pretty darn quiet.

The Noctua NF-R8 is pretty darn quiet.

Y.S. Tech FD128032HB Noctua NF-R8
Size (mm) 80 80
Depth (mm) 32 25
Bearing Ball Self lubricating oil pressure
RPM 4,000 1,800
CFM 46.9 31.19
Feels like A gale force wind A gentle breeze
dBA 40 17
Sounds like A 747 flying over seven simultaneous heavy metal concerts during the apocalypse World peace

While slightly less deep than the fans they’re replacing and featuring a cable long enough to require four zip-ties to keep under control, the Noctuas are dramatically quieter than the Y.S. Techs. Putting a human value on how much quieter is an issue though it seems; while there’s no doubt a 3dBA increase is a doubling in sound intensity, there’s arguments over how this relates to perceived volume.

They also put out significantly less cubic feet per minute of air – however I’m confident the server will survive with the extra heat, and some of the load will be lifted by two additional 80mm fans at the back drawing hot air out.

Power Supplies are Noisy too

As a consequence of all this silencing, a new problem became apparent: the power supply. The immensely noisy Emacs Zippy MX3-5750P 750W triple redundant power supply to be precise, worth around AUD$1,000. Looks like picking up the case for AUD$503 on eBay was a magic deal, around an AUD$1,800 discount thanks to the also included SATA backplane.

The power supply features three 375W hot-swappable units (MX1-5375P), of which two are active and one is spare. The top power supply sadly has an electrical squeal, which I’m not sure how to address. I’m quite sensitive to high pitched noises (like say, CRTs being left on) and so it’s destined to become annoying.

Squeal aside, the main issue is the horrific noise generated by the three PSU fans. These 40x40x28mm banshees are AVC F4028B12HBs, like the Y.S. Techs are ball bearing based, and howl like there’s no tomorrow. After some creative screwdriver work the PSU came apart with little resistance, revealing that the fine chaps at Emacs had glued the fan’s three pin power plug down in an attempt to stop fan replacement. Fortunately it’s nothing some short work with a scalpel shouldn’t be able to fix.

There’s no specs available on the AVCs, but I’m pretty damn sure the 40x40x10mm Scythe Mini Kaze SY124010L, rated at 14dBA and pushing 4.11CFM is going to be a lot quieter. The rated amperage is lower (0.06A vs 0.3A) so powering shouldn’t be an issue, however I am slightly concerned that the incredibly low CFM could impact the performance of the PSU due to excess heat build up, potentially already a threat due to the lowered exhaust power of the swapped in Noctuas.

The third issue with the power supply comes down to replacements — specifically, it seems only stores in Belgium, Russia and the Netherlands list the individual replacement modules for sale (often under the Chieftec brand) yet none of them have them in stock, while in Australia we’re limited to buying the whole expensive power supply again. At least, this is the case through regular channels, perhaps a disty may be able to help out.

Areca Abandonment

After distributors Digicor failed completely to reply to my query about bringing the ARC-1300ix-16 into the country, I decided to hunt them down at CeBIT Australia. The rep schmoozed that I shouldn’t bother with Areca and that they’re “focusing on 3Ware” — distributor speak for “we lost the contract”. Sure enough, FortuneTec picked up the deal shortly after and were blazingly fast in responding. Sadly there’s no intent to bring the 1300 series into Australia since as non-RAID cards, they’re perceived as niche market.

Looks like I’ll have to import, unless the Promise SuperTrak EX16350 shows, er, promise in Solaris.

The Promise Super Trak EX16350 is based off the Intel IOP333 and is cheap due to being EOL - but there are no Solaris guarantees

The Promise Super Trak EX16350 is based off the Intel IOP333 and is extremely cheap due to being an end of life product - but there are no Solaris guarantees.

Popularity: 30% [?]

Hardware musings

April 14th, 2009

It seems Apple managed to update its Mac Mini, with 9400M hardware and all — without a Blu-ray drive. The stabbing may commence.

It’s also HDMI deficient, meaning you’ll need a DVI > HDMI dongle (or Mini Display-Port > HDMI if you’re willing to get unofficial), and to push your sound through a receiver via optical audio. Not necessarily a deal breaker for the high end, rather annoying for the low to mid-end.

The Mac Mini. Now with more graphical power doing absolutely nothing.

The Mac Mini. Now with more graphical power doing absolutely nothing.

I’ve had the good fortune to play with LG’s BE06LU10 external Blu-ray player, but sadly it arrived too late to test in conjunction with the new Mac Mini to see if the CPU had enough grunt to run a heavy VC-1 decode. I suspect it does, but wouldn’t mind proof before laying down AUD$1049 for the Mac Mini, and then another AUD$400 for the LG. Given the aesthetic is totally different for both devices though, upon a successful test I’d be more likely to gut the Mac Mini DVD drive and insert a slot loading Blu-ray.

LG's BE06LU10 external Blu-ray drive while fine by itself, would likely ruin the aesthetic of the Mac Mini.

LG's BE06LU10 external Blu-ray drive while fine by itself, would likely ruin the aesthetic of the Mac Mini.

Given the failure of the Mac Mini to capitalise on the streaming/Blu-ray market, and Ion’s inability to process VC-1 without frame skipping, it seemed like I was going to have to fall back on a homebrew solution, until I spotted this — MSI has a contender for the best all-in-one solution with its Wind Box DE200, also based on the seemingly ever-expanding 9400M chipset. While the 2GHz Celeron M 550 CPU is a worry, it’s a Socket P-based solution, meaning it can at least be upgraded to a 2.266GHz Celeron M 570, and if the chipset supports, a Core 2 Duo mobile processor. Hopefully MSI will take the guesswork out for us and just release a higher model version.

MSI's Wind Box DE200 - with a little more CPU, this could be the all in one we're looking for.

MSI's Wind Box DE200 - with a little more CPU, this could be the all in one we're looking for. Image credit: Engadget

On the server front, Areca have been darlings and released what we’ve been waiting for in controllers — an internal 16 port SAS HBA with no RAID engine, to keep the costs down, in the form of the ARC-1300ix-16. Best of all, it has Solaris support out of the box. Looks like we have our target, now we just need to wait for a local release.

The Areca ARC-1300ix-16 goes for CAD$499, has no RAID engine and comes with Solaris support. It's love at first sight.

The Areca ARC-1300ix-16 goes for CAD$499, has no RAID engine and comes with Solaris support. It's love at first sight, so long as it comes out in Australia. Image credit: Tweakers.net

In bad news, Anand gives us all a reason why we have to wait a little longer for SSDs to really live up to the promise, as they get slower over time. Unless you want to drastically shorten their lifespan by continuously hard wiping them, that is. This is a shame, as the improvement in Windows UI snappiness (loading an expanded Control Panel from the Start Menu is near instant) is almost entirely worth it on its own.

In other news, while the new Xeon 5500 setups have me crying at the ridiculous performance that can be made available with wads of cash, I’m sure my setup will do just fine within the confines of home use. The bought hardware seems to test well, short of the MSI P7n Diamond continuously resetting the boot order to what it thinks is smart rather than using the options set by the user. If there’s a USB drive left in or a new hard drive hooked up, it tries to boot off that. If you finish installing an OS off the optical drive, it sets the primary boot device to the hard drive where the OS is installed. Extremely frustrating and vastly stupid on behalf of the MSI engineers.

Popularity: 36% [?]