Gasp! Intrigue! Another fucking motherboard!

I suppose at this point I really shouldn’t be surprised. The number of things that have gone wrong to date rival most government run projects.

Not covering old ground:

1. The Areca ARC-1300ix 16 was returned on the basis of no Solaris driver and it being a glorified port multiplier. I waited for about a month for the Adaptec 31605 on back order. After showing no signs of turning up any time soon, I cancelled the order and decided to pony up for the highly featured and crazily expensive 3ware 9650SE-8LPML instead.  The day after, I find out Digicor has started distributing the SAS3081E-R again, a significantly cheaper option. Take in mind either choice locks me into a motherboard with at least three PCI-E x4 slots (due to needing 16 channels, and to equip the Intel quad gigabit Ethernet card).
2. The moment the 3ware arrives, I plug it in, and lo and behold, the machine no longer posts. Having seen a similar behaviour on the previous server (would or would not boot based on random hardware plugged in and how many times half the male population has scratched its crotch in the last hour while the wind is blowing west), I proceeded to disconnect everything until only RAM, CPU and GPU remained — and it still didn’t boot. Being that the only remaining part from the old server was the 850W CoolerMaster PSU, I ordered a Corsair TX-850 at AU$240 to remove all doubt, plugged it in AND;
   

   The Corsair TX-850, a PSU with a beefy 12V rail and five year warranty - AU$240.

3. Discovered that the Asus P5Q-E motherboard, which replaced the exploded MSI was the dead part, despite no sparking, despite working a week ago, despite nothing being physically wrong with the board — it just stopped posting,  meaning I’ve once again spent more cash than I have to.
   

   The excellent Asus P5Q-E. Alas, it was not to be.

Well, fuck. That’s two motherboards gone in one build, which is making me wonder if the case is shorting something somehow. At this point I’ve had the chassis for over ten months, with no working system. It’s enough to make you want to buy a prebuilt NAS.

Meanwhile the brand new Netgear GS724T, APC SmartUPS 750 and HP 22RU rack just sit there, waiting for some action…

The mighty 3ware 9650SE-8LPML, coming to a server near you - AU$915.

Netgear GS724T - picked up for AU$280.

APC SmartUPS 750 - AU$231 on eBay.

HP 10622 rack - the one I bought on eBay for AU$180 likely has a bit more wear and tear than the one in this image.

This adds an extra layer of complexity to rebuilding disks, as when a disk fails, the card assumes a RAID 0 array has died, regardless of what you’re doing with ZFS. Apart from removing the ability to yank a disk on a live array then pop it back in and continue as normal, this adds extra overhead as the card is managing RAID 0 data for every drive attached to it on top of the RAID-Z already being done on the software side. Bad, bad, bad.

The LSI MegaRAID SAS 84016E

We have a new contender though, the LSI MegaRAID SAS 84016E (also known as the Intel SRCSASPH16I), which definitely has OpenSolaris driver support, but as usual is not available in Australia (the Intel is, but is over AU$1,000). It’s more expensive than the vapourous ARC-1300ix-16, thanks to it being PCI-E 8x rather than 4x. It’s also a true RAID card with 256MB of memory, and can handle up to RAID 60 thanks to a 500MHz Intel IOP333 processor.

PC Pitstop sells them at US$689, and the site even has a section saying it ships to Australia. Now if only a certain eBay seller wasn’t selling it for almost US$100 cheaper with free shipping…

Then there’s the Intel RAID Controller SRCSATAWB. This is a modified LSI MegaRAID SAS 8708ELP, doesn’t work in PCI-E 2.0, seems to have virtually the same featureset as the 84016E, but with only two mini-SAS ports. EYO Drop Shipping is currently selling it for AU$576.18.

The Intel RAID Controller SRCSATAWB

For both, the manuals mention nothing about JBOD, which may resign them to the same scrap heap as the P400. They do mention virtual drives, but these seem to only be accessible when creating an array. There’s no mention of running single drives in order to access software RAID.

Edit: Neither card offers JBOD functionality. At this rate I’ll end up buying the crazily expensive Adaptec 31605 just to get working gear.

• Intel Q9550 ~ AUD$450
• 8GB Corsair DDR2 8500 ~ AUD$380
• MSI P7N Diamond ~ AUD$360

MSI’s P7N Diamond was chosen for one point alone — four PCI-e x16 slots. While a lot of boards have a number of physical x16 slots, they fail to back this up electrically beyond two slots. The MSI board has three x16 electrical slots, with the fourth yellow one being an x8 — perfect for expansion.

The P7N Diamond has just the right amount of PCI-E lanes to satisfy our expansion needs.

OpenSolaris 2008.11 was installed on this setup, on a 500GB drive hooked up to one of the NV sata ports, a DVD drive hooked up to the JMB363 controlled IDE port, a previously acquired GeForce 7600GS inserted, alongside a HighPoint RocketRaid 2340. For kicks, an Intel X25-E was hooked up to check out some awesome transfer speeds.

It wasn’t to be.

Things I’ve learned:

• OpenSolaris loves the MSI board, pretty much enabling everything. While it recognises the X-Fi sound, sound does not actually work. This isn’t a deal breaker. To my never ending surprise, JMB363 seems to work just fine.
• Turning off AHCI only results in the rear eSATA ports turning off.
• Most curiously, OpenSolaris will not recognise the X25-E drive at all. Whether this is related to the NV sata ports or otherwise, I do not know.
• The HighPoint RocketRaid 2340 is not supported. The dual Marvell 88SX6081 chips on it technically are with voodoo beyond the install process, but are the cause of some problems. These have been patched it seems, but all up it seems less trouble to grab something based off LSI chipsets. While FreeBSD certainly supports the 2340, once again the sturdiness of its implementation of ZFS gives me pause.
• There’s something called Solaris eXpress Community Edition, which abbreviates to the unfortunate SXCE, or “sexy”. It’s basically a beta containing future code, and sadly also didn’t recognise the X25-E, 2340 or X-Fi.

The remaining options are few to be able to set up a 16 drive array in Solaris. Either acquire the Adaptec 31605 for around AUD$1200, or two HP P400s for around AUD$700. Obviously the HP option is significantly cheaper – so long as it works.

While Solaris may seem ideal, it certainly isn’t cheap to get working thanks to limited hardware support. It could seriously be a wait for Snow Leopard and some Hackintoshing, although this is much better suited to an Intel board than this 780i.