How To Build a Custom SAN (Storage Area Network)
Hardware, Part 1
Why I Wanted to Roll My Own SAN
In my constant search to find the perfect SAN (Storage Area Network) I came to the realization a long time ago that it really doesn’t exist. Now of course the perfect SAN would be described differently depending on who you talk to and what the SAN is going to be used for. I wanted a SAN for the purpose of doing just about anything. That would mean I needed the SAN to be highly flexible in all possible aspects while keeping it practical and affordable since I’m working from a hobby budget, at least in my case. Given the current economic conditions that exist for most small business these days wasteful spending is not really an option if you are considering a SAN build for your company or employer.
After searching high and low for a black-box SAN solution I came up disappointed every time. Whether it was USB capable only or relied too heavily on RAID based software I really couldn’t find anything that perfectly fit my wishful-thinking requirements. My goal was to find a SAN with the following features.
Home / Small Business SAN Requirements:
- Nobody likes spending their own cash
- SAN solutions are already too expensive
- This market is ripe for innovation
- Improves uptime by avoiding system shutdown to replace hardware
- Takes advantage of SATA hot-swap specification
- You can have your cake and eat it too. Large storage in a small space is always fashionable.
- In case you sleep next to it
Low Power Consumption
- Because hibernating a SAN gets people fired
- If the SAN is going to be used infrequently, little is gained by burning through the watts
- Personal preference on this one as the great debate between hardware/software RAID rages on
- Prefer the audible alarm signaling drive failure and error LED on drive bay (Software method less obvious)
Dependable IO Controller
- 10+ SATA II Ports to match the 10 hot-swap drive bays
- Quality controllers will provide vast performance improvements over cheap boards
- Features like Staggered Spin-Up, Out-of-Band managements, and Audio/Visual Alarms
- Your requirements may dictate RAID 10,50 etc..
- Are you looking for speed, safety, or both?
Dependable 1Gb NIC Port
- Intel and Broadcom are two industry standards compliant and reliable manufacturers
- Avoid bottleneck by using a fast 1Gbps+ connection
- 1Gb connectivity minimum if using iSCSI or NFS
- 110MB Transfer Rate (WD10EARS) equates to 880Mbps. Add RAID and you are over 1Gbps
Bootable USB Port
- Handy when you don’t have room for a CDROM and need to load the SAN OS or additional software
On-board Video Card
- Reduce costs
- Improve airflow
- Improve simplicity
- Hard drives are going to generate heat. A cool system is a happy system.
- Capable PSU supporting 12+ HDs, MB, and CPU
SAN Operating System
- Slick interface that makes quick effective work of slicing up your storage as you desire
- Protocols: iSCSI, NFS, CIFS/SMB, FTP (everything else would be icing on the cake)
Upgradable Software and Hardware
- Something new and fresh is always on the horizon
- Don’t drop a ton of money on an entirely new all-in-one SAN every time some new feature comes out. Just upgrade as you need. Vendors love you a lot more if you buy a new SAN to get the latest do-dad.
From my list of requirements I was already thinking this task was going to be tricky trying to track down a “Ready-to-Run” black box SAN. I started looking at solutions like using a Drobo, but it fell short on several key areas. So I turned to direct attached storage (DAS) in the form of eSATA, but they were not network accessible. It just seemed like the market for high-end SAN feature with a low end budget was non-existent. That was unacceptable to me in the end and I was determined to come up with something feasible. I knew it could be done, and for some reason the industry has always seemed to lag behind in the storage market for the prosumer techie like me. Time to roll up my sleeves and make it happened the good old fashioned way. It was time to Do It Yourself!
So I rolled up my sleeves and started researching components for my home made SAN. After all I was only going to use it as my backup server, VMware Host, Oracle Database Block Storage, media streaming server, home file server, and raw video data store. I just wanted to be prepared in case anything else came up. My search started with finding a good RAID enclosure, so I could see visual indicators of drive activity, hot swap failed drives, and squeeze 5 3.5″ drives into 3 case bays with the enclosure. I considered the RAID enclosure and the RAID Controller the two core important pieces of hardware I really wanted to get right the first time.
I jumped right in to researching the enclosure. I looked at so many different enclosures and the majority of these enclouses had one commonality between them. They were all cheaply constructed and of poor quality. None of them rated all that well and bothered me quite a bit as this was the keystone in my build. Right off the bat I was mildly discouraged in my quest for greatness. After stepping back and thinking about it a while I realized the majority of entry level business SANs were using Supermicro components for their SAN solutions. So I stopped beating myself over the head with a power dongle and went straight to the source. I found the Supermicro CSE-M35T-1B. In the end I was quite pleased with this purchase, especially the quality all things considered.
Next on my list was the all important RAID controller. My searching lead me to an Areca ARC-1260 (I picked up one used, and this model is no longer produced) for their good reviews, very capable controllers and useful out-of-band web user interface for monitoring and managing the hardware aspect of the RAID configuration. This would avoid rebooting your storage area network to check a configuration, tweak a setting, or swap a failing drive. That would kind of defeat the purpose of using a SAN.
Next I wanted to find a solid motherboard. Since I was quite happy with the purchase of my two Supermicro RAID enclosures I thought a Supermicro motherboard deserved consideration. In my requirements I stated on-board video and non-generic network port so this was a logical choice. I was able to hunt one down, although I admit at the time it was hard to find a single socket, embedded video, and built-in RAID controller. My strategy here was to locate my operating system hard drives away from the primary RAID controller. I didn’t want to create a single point of failure or controller bottleneck. This would allow for expanded flexibility too. So I configured two 250GB drives using RAID 1 attached to the Supermicro on-board RAID controller to protect against loosing my SAN operating system and RAID configuration. I also wanted to avoid co-locating the SAN OS drives with the hot swap production drives. I didn’t even want to take the risk of accidentally pulling an OS drive from the RAID cage.
Next up I had to find a home for all this beautiful hardware to reside. I was looking for something convenient to work with internally, had plenty of ventilation, fairly quiet, plenty of bays for stacking two RAID enclosures that would consume six bays, and a sleek black box feel. I’ve owned Lian Li cases in the past and found what I was looking for. I particually liked this case because there was no unnecessary wasted real-estate on the front of the case, just nine 3.5″ bays oriented from top to bottom with a very clean look. This prevented from from purchasing a server town in turn keeping prices down and size to a minimum. Check, check and check!
Lian Li PC-A17B