Posts Tagged ‘OpenSolaris’

Sometimes giving in is easier

December 13th, 2009

OpenSolaris’ ZFS implementation recently picked up one of the tastiest things it possibly could: block level dedupe.

Except I no longer care.

Too impatient to wait for the RMA on the dead Asus P5Q-E (of which the replacement is now a spare swap-in board), thanks to an incredibly generous friend I picked up a Gigabyte GA-EP45-Extreme… which OpenSolaris b127 hated, and refused to boot with. After a few days of hair pulling and switching off almost everything I could in the BIOS to try and rectify the issue, I finally admitted OpenSolaris was not to be.

The Gigabyte GA-EP45 Extreme, great board, hated by OpenSolaris

The Gigabyte GA-EP45 Extreme, great board, hated by OpenSolaris

Not willing to risk Nexenta, I dropped to FreeBSD 8, the last bastion of ZFS hope (no folks, FUSE does not count).

FreeBSD worked wonderfully from a compatibility front, but I soon discovered that when it came to virtualisation, it had the same options as a prisoner faced with the Spanish inquisition: basically none. There is, ironically, a version of Sun’s VirtualBox floating around, but it’s a hack job that hates 64-bit, and like most things FreeBSD if you’re not running from the command line you’re asking for pain.

And so, hoping that one day Larry Ellison would open up ZFS licensing a little more so the GPL crowd would stop whining and just integrate it already, I sighed, flicked the 3ware 9650SE into hardware RAID 6 and reached for the Ubuntu 9.10 64-bit disc.

It worked.

Post mortem: List of controller cards that will work with OpenSolaris

While I note with grim satisfaction that Areca has still failed to produce a Solaris driver for it’s ARC-1300ix series, here’s a list of PCI-Express cards known to work with OpenSolaris without requiring any RAID 0/JBOD workarounds, and being able to control at least eight drives.

  1. LSI SAS3081E-R
  2. Intel SASUC8I flashed with the SAS8031E-R’s IT (initiator target) firmware
  3. 3ware 9650SE series

Tiny, yes? The last, which I ended up with due to non-availability of the first two in Australia, is significantly more expensive as it has hardware RAID capability as well.

Post mortem: Final system

Rack: HP 10622
OS: Ubuntu 9.10
PSU: Corsair TX-850
CPU: Intel Q9550
Memory: 8GB Corsair Dominator PC-2 8500
Motherboard: Gigabyte GA-EP45 Extreme
GPU: Geforce 7600GS silent (to be swapped out with a PCI card when a second 3ware controller card is bought)
Controller card: 3ware 9650SE-8LPML
Network card: HP NC364T
Case: Chenbro RM41416B
UPS: APC Smart-UPS 750
Switch: Netgear GS724T
System drives: Samsung HD501LJ SATA
Array drives (RAID 6 w/XFS): WD RE3 1TB x3, Samsung HD103UJ 1TB x2, Seagate 7200.11 x2, Seagate 7200.12

The only problem left is the Seagate 7200.12, which seems to keep dropping from the array. I’ll have to see if a firmware update to the 3ware card fixes it, otherwise I may need to swap in a new drive (Update: turns out the ridiculously expensive Mini-SAS to SATA cables I bought were dodgy. Upon replacing, I’ve had no dropouts).

Areca are liars: The ARC-1300ix 16 does not support Solaris

August 3rd, 2009
Oh damn... why did no one tell me?

Oh damn... why did no one tell me?

Well, that about settles it. This file server is cursed.

After replacing the CPU and motherboard, I booted up my once again completed fileserver to find out, unlike as advertised, the ARC-1300ix 16 does not support Solaris in any fashion. At all.

Areca has altered the product page too, conveniently after I purchased, to read “Solaris 10/11 (will be available with 6Gb/s Host Adapter)”.

Excuse me? It will be out when something is available that doesn’t even come on motherboards yet, let alone standalone adapters? Not to mention the drivers might possibly not support the card I bought which explicitly claimed support?

Bullshit.

The manual, of course, still claims Solaris support — both the one that came with the card, and online in the form of revision 1.1 (2009/5/8, hosted here in case Areca takes it down). Incidentally, it only offers RHEL for Linux support, and this comes as a floppy image!

For anyone that’s curious, it uses a Marvell 88se6440 plugged into an LSI SASX28. The latter is a port multiplier, supported by Solaris. The former is the controller, supports four drives and is most definitely not supported. Effectively, Areca is jamming in four drives per 3Gb/s channel, squeezed over a PCI-E 4x connection. I am more than slightly vexed.

I have filed a complaint through Areca’s web support form, but given that it didn’t even confirm that anything had been sent, I don’t hold out any hope for a response. If there isn’t one coming quickly, I’ll be requesting a refund from the distributor. My only concern is that there is now really no option but to pony up for the Adaptec 31605, since I no longer have the luxury of four PCI-E x16 slots to play with.

Update:
Areca has responded, and in a timely manner. Q3 huh? Soooo maybe October, factoring in delays. That’s a long time to wait to set up a file server. Have some nice people on the Sun side helping, but I believe it may still be dwelling in refund territory.

—-

Dear Sir,

in our plan, we will release the driver for solaris as soon as the 6G SAS HBA available, the planned schedule is the end of Q3. the driver can be used for 6G and 3G SAS HBA both.  sorry for the inconvenience.

Best Regards,

[Removed for privacy]

Areca Technology Tech-support Division
Tel : [Removed for privacy]
Fax : 886-2-87975970

Http://www.areca.com.tw

Ftp://ftp.areca.com.tw

Mirror Ftp :
ftp://areca.starline.de

The continuous controller conundrum

June 8th, 2009

Strike another one off the list — the HP Smart Array P400 doesn’t present drives through JBOD to the OS, only through RAID 0.

This adds an extra layer of complexity to rebuilding disks, as when a disk fails, the card assumes a RAID 0 array has died, regardless of what you’re doing with ZFS. Apart from removing the ability to yank a disk on a live array then pop it back in and continue as normal, this adds extra overhead as the card is managing RAID 0 data for every drive attached to it on top of the RAID-Z already being done on the software side. Bad, bad, bad.

The LSI MegaRAID SAS 84016E

The LSI MegaRAID SAS 84016E

We have a new contender though, the LSI MegaRAID SAS 84016E (also known as the Intel SRCSASPH16I), which definitely has OpenSolaris driver support, but as usual is not available in Australia (the Intel is, but is over AU$1,000). It’s more expensive than the vapourous ARC-1300ix-16, thanks to it being PCI-E 8x rather than 4x. It’s also a true RAID card with 256MB of memory, and can handle up to RAID 60 thanks to a 500MHz Intel IOP333 processor.

PC Pitstop sells them at US$689, and the site even has a section saying it ships to Australia. Now if only a certain eBay seller wasn’t selling it for almost US$100 cheaper with free shipping…

Then there’s the Intel RAID Controller SRCSATAWB. This is a modified LSI MegaRAID SAS 8708ELP, doesn’t work in PCI-E 2.0, seems to have virtually the same featureset as the 84016E, but with only two mini-SAS ports. EYO Drop Shipping is currently selling it for AU$576.18.

The Intel RAID Controller SRCSATAWB

The Intel RAID Controller SRCSATAWB

For both, the manuals mention nothing about JBOD, which may resign them to the same scrap heap as the P400. They do mention virtual drives, but these seem to only be accessible when creating an array. There’s no mention of running single drives in order to access software RAID.

Edit: Neither card offers JBOD functionality. At this rate I’ll end up buying the crazily expensive Adaptec 31605 just to get working gear.

Silencing the chassis, perpetual controller issues

May 26th, 2009

The Chenbro RM41416B is a 4U file server case. Thus it has fans built for pushing around huge volumes of air, with complete disregard for the auditory senses of normal human beings. Obviously for a file server in a house, this is an issue.

The case by default comes kitted out with five Y.S. Tech FD128032HB 80mm fans, with room for another two at the back to draw hot air out. Upon spinning up, these were loud enough to shatter a pensioner’s dentures at 50 paces. The decision was made to swap them out with Noctua NF-R8s —certainly not a one to one replacement in terms of air flow, but a heck of a lot quieter.

The Noctua NF-R8 is pretty darn quiet.

The Noctua NF-R8 is pretty darn quiet.

Y.S. Tech FD128032HB Noctua NF-R8
Size (mm) 80 80
Depth (mm) 32 25
Bearing Ball Self lubricating oil pressure
RPM 4,000 1,800
CFM 46.9 31.19
Feels like A gale force wind A gentle breeze
dBA 40 17
Sounds like A 747 flying over seven simultaneous heavy metal concerts during the apocalypse World peace

While slightly less deep than the fans they’re replacing and featuring a cable long enough to require four zip-ties to keep under control, the Noctuas are dramatically quieter than the Y.S. Techs. Putting a human value on how much quieter is an issue though it seems; while there’s no doubt a 3dBA increase is a doubling in sound intensity, there’s arguments over how this relates to perceived volume.

They also put out significantly less cubic feet per minute of air – however I’m confident the server will survive with the extra heat, and some of the load will be lifted by two additional 80mm fans at the back drawing hot air out.

Power Supplies are Noisy too

As a consequence of all this silencing, a new problem became apparent: the power supply. The immensely noisy Emacs Zippy MX3-5750P 750W triple redundant power supply to be precise, worth around AUD$1,000. Looks like picking up the case for AUD$503 on eBay was a magic deal, around an AUD$1,800 discount thanks to the also included SATA backplane.

The power supply features three 375W hot-swappable units (MX1-5375P), of which two are active and one is spare. The top power supply sadly has an electrical squeal, which I’m not sure how to address. I’m quite sensitive to high pitched noises (like say, CRTs being left on) and so it’s destined to become annoying.

Squeal aside, the main issue is the horrific noise generated by the three PSU fans. These 40x40x28mm banshees are AVC F4028B12HBs, like the Y.S. Techs are ball bearing based, and howl like there’s no tomorrow. After some creative screwdriver work the PSU came apart with little resistance, revealing that the fine chaps at Emacs had glued the fan’s three pin power plug down in an attempt to stop fan replacement. Fortunately it’s nothing some short work with a scalpel shouldn’t be able to fix.

There’s no specs available on the AVCs, but I’m pretty damn sure the 40x40x10mm Scythe Mini Kaze SY124010L, rated at 14dBA and pushing 4.11CFM is going to be a lot quieter. The rated amperage is lower (0.06A vs 0.3A) so powering shouldn’t be an issue, however I am slightly concerned that the incredibly low CFM could impact the performance of the PSU due to excess heat build up, potentially already a threat due to the lowered exhaust power of the swapped in Noctuas.

The third issue with the power supply comes down to replacements — specifically, it seems only stores in Belgium, Russia and the Netherlands list the individual replacement modules for sale (often under the Chieftec brand) yet none of them have them in stock, while in Australia we’re limited to buying the whole expensive power supply again. At least, this is the case through regular channels, perhaps a disty may be able to help out.

Areca Abandonment

After distributors Digicor failed completely to reply to my query about bringing the ARC-1300ix-16 into the country, I decided to hunt them down at CeBIT Australia. The rep schmoozed that I shouldn’t bother with Areca and that they’re “focusing on 3Ware” — distributor speak for “we lost the contract”. Sure enough, FortuneTec picked up the deal shortly after and were blazingly fast in responding. Sadly there’s no intent to bring the 1300 series into Australia since as non-RAID cards, they’re perceived as niche market.

Looks like I’ll have to import, unless the Promise SuperTrak EX16350 shows, er, promise in Solaris.

The Promise Super Trak EX16350 is based off the Intel IOP333 and is cheap due to being EOL - but there are no Solaris guarantees

The Promise Super Trak EX16350 is based off the Intel IOP333 and is extremely cheap due to being an end of life product - but there are no Solaris guarantees.

One step forwards, two steps back

February 16th, 2009

Some purchasing has recently happened to start the file server project:

  • Intel Q9550 ~ AUD$450
  • 8GB Corsair DDR2 8500 ~ AUD$380
  • MSI P7N Diamond ~ AUD$360

MSI’s P7N Diamond was chosen for one point alone — four PCI-e x16 slots. While a lot of boards have a number of physical x16 slots, they fail to back this up electrically beyond two slots. The MSI board has three x16 electrical slots, with the fourth yellow one being an x8 — perfect for expansion.

The P7N Diamond has just the right amount of PCI-E lanes to satisfy our expansion needs.

The P7N Diamond has just the right amount of PCI-E lanes to satisfy our expansion needs.

OpenSolaris 2008.11 was installed on this setup, on a 500GB drive hooked up to one of the NV sata ports, a DVD drive hooked up to the JMB363 controlled IDE port, a previously acquired GeForce 7600GS inserted, alongside a HighPoint RocketRaid 2340. For kicks, an Intel X25-E was hooked up to check out some awesome transfer speeds.

It wasn’t to be.

Things I’ve learned:

  • OpenSolaris loves the MSI board, pretty much enabling everything. While it recognises the X-Fi sound, sound does not actually work. This isn’t a deal breaker. To my never ending surprise, JMB363 seems to work just fine.
  • Turning off AHCI only results in the rear eSATA ports turning off.
  • Most curiously, OpenSolaris will not recognise the X25-E drive at all. Whether this is related to the NV sata ports or otherwise, I do not know.
  • The HighPoint RocketRaid 2340 is not supported. The dual Marvell 88SX6081 chips on it technically are with voodoo beyond the install process, but are the cause of some problems. These have been patched it seems, but all up it seems less trouble to grab something based off LSI chipsets. While FreeBSD certainly supports the 2340, once again the sturdiness of its implementation of ZFS gives me pause.
  • There’s something called Solaris eXpress Community Edition, which abbreviates to the unfortunate SXCE, or “sexy”. It’s basically a beta containing future code, and sadly also didn’t recognise the X25-E, 2340 or X-Fi.

The remaining options are few to be able to set up a 16 drive array in Solaris. Either acquire the Adaptec 31605 for around AUD$1200, or two HP P400s for around AUD$700. Obviously the HP option is significantly cheaper – so long as it works.

While Solaris may seem ideal, it certainly isn’t cheap to get working thanks to limited hardware support. It could seriously be a wait for Snow Leopard and some Hackintoshing, although this is much better suited to an Intel board than this 780i.