Just a quick note to let everyone know that I have decided to post articles related to VMware, virtualization and cloud computing to a new blog: http://vmclouds.e-wilkin.com
I will leave current posts as-is but new content will be posted to the new blog. I will continue to post my other ramblings here moving forward.
Friday, September 8, 2017
Sunday, March 12, 2017
New NAS Build Update 1
After working with Rockstor and weighing my options with HDDs, I've decided to go a slightly different direction...
NAS OS
I'm a big CentOS fan. I also have been wanting to learn more about Docker and BTRFS. This makes Rockstor with its "Rock-ons" and BTRFS filesystem the perfect choice. Or does it?
BTRFS
BTRFS seems to be Linux's answer to ZFS. However, it is not nearly as mature, feature complete or ready for primetime in my opinion. I wanted to work with 4+ drives which implies a RAID5/6 type of setup. I quickly found that BTRFS has some serious problems with its RAID5/6 implementation to the point where it corrupts data and therefore is not recommended for production. I could buy a true hardware-based RAID card for around $100, but I'm trying to avoid spending additional coin on this build (see Hardware section).
XPEnology
I'm "upgrading" from a Synology NAS. I love the simplicity and the 'it just works' factor it provides. Finding a NAS OS that provides a similar experience has proven to be difficult. Rockstor will not let you use any non-BTRFS filesystems on data drives. Per the RAID issue above and the fact that I did not get very good support from their somewhat inactive forums rules out Rockstor for this build. About the same time I started running into obstacles with Rockstor, I stumbled upon XPEnology.
While not officially supported by Synology, it is based on their OSS code and the forum appears fairly active. I've been testing it in a VM while waiting for some additional hardware parts to arrive (see hardware section). This is basically DSM on your hardware of choice. The real test will be the installation on the TS140-based system. This is in no way my final decision on the NAS OS as further testing is required.
Hardware
The more I worked with the TS140's case, the less I liked it. Both internal drive trays require an L-shaped SATA connector since they're so close to the side of the case. Lenovo only provides one. That was the final straw. Luckily, I had a storage server case, an ARK 4U-500-CA that I was about to list on Craigslist but decided to use for this build. It fits nicely with my 'low cost, greater flexibility' theme. The rackmount hardware is removable. Just add some rubber feet on the bottom and presto-chango - it's a server tower case! It can hold at least 10 drives, has great air-flow and best of all - its free (to me).
In the next update, I'll detail the fun of moving the TS140 guts into a new case and some additional decisions required based on the number of drives needed.
NAS OS
I'm a big CentOS fan. I also have been wanting to learn more about Docker and BTRFS. This makes Rockstor with its "Rock-ons" and BTRFS filesystem the perfect choice. Or does it?
BTRFS
BTRFS seems to be Linux's answer to ZFS. However, it is not nearly as mature, feature complete or ready for primetime in my opinion. I wanted to work with 4+ drives which implies a RAID5/6 type of setup. I quickly found that BTRFS has some serious problems with its RAID5/6 implementation to the point where it corrupts data and therefore is not recommended for production. I could buy a true hardware-based RAID card for around $100, but I'm trying to avoid spending additional coin on this build (see Hardware section).
XPEnology
I'm "upgrading" from a Synology NAS. I love the simplicity and the 'it just works' factor it provides. Finding a NAS OS that provides a similar experience has proven to be difficult. Rockstor will not let you use any non-BTRFS filesystems on data drives. Per the RAID issue above and the fact that I did not get very good support from their somewhat inactive forums rules out Rockstor for this build. About the same time I started running into obstacles with Rockstor, I stumbled upon XPEnology.
While not officially supported by Synology, it is based on their OSS code and the forum appears fairly active. I've been testing it in a VM while waiting for some additional hardware parts to arrive (see hardware section). This is basically DSM on your hardware of choice. The real test will be the installation on the TS140-based system. This is in no way my final decision on the NAS OS as further testing is required.
Hardware
The more I worked with the TS140's case, the less I liked it. Both internal drive trays require an L-shaped SATA connector since they're so close to the side of the case. Lenovo only provides one. That was the final straw. Luckily, I had a storage server case, an ARK 4U-500-CA that I was about to list on Craigslist but decided to use for this build. It fits nicely with my 'low cost, greater flexibility' theme. The rackmount hardware is removable. Just add some rubber feet on the bottom and presto-chango - it's a server tower case! It can hold at least 10 drives, has great air-flow and best of all - its free (to me).
In the next update, I'll detail the fun of moving the TS140 guts into a new case and some additional decisions required based on the number of drives needed.
Wednesday, March 1, 2017
Time for a (NAS) Change
I have been running the DVR feature of Plex for several months now. During the same time the message "this server is not powerful enough to convert video" kept popping up intermittently preventing some videos from playing. Oddly enough, it doesn't happen all of the time on all shows, but enough to be more than annoying. Another thing I couldn't understand is why it was trying to transcode video at all when I have this option turned off for local streaming.
I was finally able to determine that the tuners record TV shows using the older MPEG-2 video standard which is still used to broadcast HD television today. On the other hand, Plex wants to stream video using the newer H.264 standard. Therefore, Plex will convert recorded TV shows on-the-fly (transcode) when streaming to Plex clients.
My existing NAS is a Synology DS713+ running Plex Media Server along with several other applications. Upon researching this problem, I found a list of compatible NAS devices along with performance notes here: Plex NAS Compatibility List.
What this list tells me is that I have to spend around $1500 to get a Synology box that does not have any problems streaming 1080P HD video (mostly due to the requirement a modern Intel Core I3 CPU or higher). I love my Synology but not that much - its hard to justify that kind of coin when I can build a system twice as powerful for half the price (or better).
So now I've decided to build my own. I used to build and repair systems over twenty years ago and its interesting how little has really changes with system components. I have spare disks, a RAID controller (if needed) and so my requirements are relatively light:
Form-factor: Tower
CPU: Xeon proc (no more problems with transcoding)
Memory: At least 4GB ECC RAM but expandable
Disk: capable of at least 4 disks, hot-swappable nice-to-have]
Quiet (will live in my office)
Cheap (I'm poor... and cheap)
I narrowed the system choice down to the Dell PowerEdge T20 or the Lenovo ThinkServer TS140. These units are nearly identical and I wanted to go with the Dell but the ThinkServer prides itself in being quiet (26 decibels) and has an extra PCI slot that pushed it over the 'edge (hidden pun). I ordered the ThinkServer from eBay/Newegg here: Lenovo ThinkServer TS140 70A4003AUX Tower Server
A note on memory: I'm going to try the 4GB of memory to start. It comes as a single DIMM (as opposed to two or more DIMMs) which leaves three open slots. So if more memory is needed I can add another 4GB DIMM later. I had tried running a custom built storage server three or four years ago using a desktop board with non-ECC memory running FreeNAS. I had read on the forums multiple warnings that only ECC memory should be used due to the risk of bit rot and eventual data corruption. Well, it only took a couple weeks before VMs stored on that systems starting becoming corrupted. Having been bit by this bug, I won't run a storage server without ECC memory and highly, highly, highly recommend you do the same.
A note on storage: I'm going to reuse the two 4TB drives in my Synology for this system in addition to two more 4TB drives I'll purchase for this build. The TS140 has an integrated hardware RAID controller. So for the data volume, I'll install the two new drives and pull one of the Synology drives for a total of three. Then transfer the data off the Synology to the new system. Once I'm satisfied everything is in good working order, I'll pull the fourth drive and install into the new system, expand the volume, etc. Then I'll have a very nice DS713+ for sale!
A note on the OS: I will miss the easy UI experience Synology DSM provides. However, I have found a decent alternative: Rockstor. I looked at several options: OpenFiler (dead) NexentaStor, FreeNAS, etc. But Rockstor offers features that I really want: web-based interface, based on CentOS 7, Docker-based plugin system called Rock-ons, optimized to run from an USB drive and Apple Time Machine Support. I also use the Synology for security cameras but found Rockstor also supports ZoneMinder video surveillance system.
It's not all rainbows and unicorns though. I'm not very confident about BTRFS (pronounced 'butter FS'). Rockstor recommends against using RAID5 or 6 for production systems. No worries for me since I'm using hardware-based RAID. I've also read that newer versions are much more stable than early releases. So while they tout BTRFS as a big feature, it may actually be their biggest drawback. Time and testing will tell. There must be a reason that many if not most NAS companies are now offering this filesystem as an option on their devices.
Also note that the TS140 has an internal USB port - perfect for a USB flash drive to host the OS.
Here's the parts list:
System: Lenovo ThinkServer TS140 70A4003AUX Tower Server = $320
4TB Enterprise SATA x 2 = $177
16GB USB Flash Drive: Transcend 16GB JetFlash 820 = $10
Operating System: Rockstor = Free
RAID Adapter (if needed): PERC H310 = Free
NIC Adapter = Intel Quad port = Free
Application Mapping:
Plex Media Server (Synology package) = PMS Rock-on (Rock-on)
Surveillance Station (Synology package) = ZoneMinder (Rock-on)
CrashPlan (Community Package) = CrashPlan (Rock-on or native)
Photo Station (Synology Package) = TBD
Download Station (Synology Station) = CouchPotato? TBD
So for $100 less I'll have a systems that's much more powerful and expandable than a Synology 916+ with twice the capacity. I'll follow-up with some additional thoughts once I have run this build through its paces.
I was finally able to determine that the tuners record TV shows using the older MPEG-2 video standard which is still used to broadcast HD television today. On the other hand, Plex wants to stream video using the newer H.264 standard. Therefore, Plex will convert recorded TV shows on-the-fly (transcode) when streaming to Plex clients.
My existing NAS is a Synology DS713+ running Plex Media Server along with several other applications. Upon researching this problem, I found a list of compatible NAS devices along with performance notes here: Plex NAS Compatibility List.
What this list tells me is that I have to spend around $1500 to get a Synology box that does not have any problems streaming 1080P HD video (mostly due to the requirement a modern Intel Core I3 CPU or higher). I love my Synology but not that much - its hard to justify that kind of coin when I can build a system twice as powerful for half the price (or better).
So now I've decided to build my own. I used to build and repair systems over twenty years ago and its interesting how little has really changes with system components. I have spare disks, a RAID controller (if needed) and so my requirements are relatively light:
Form-factor: Tower
CPU: Xeon proc (no more problems with transcoding)
Memory: At least 4GB ECC RAM but expandable
Disk: capable of at least 4 disks, hot-swappable nice-to-have]
Quiet (will live in my office)
Cheap (I'm poor... and cheap)
I narrowed the system choice down to the Dell PowerEdge T20 or the Lenovo ThinkServer TS140. These units are nearly identical and I wanted to go with the Dell but the ThinkServer prides itself in being quiet (26 decibels) and has an extra PCI slot that pushed it over the 'edge (hidden pun). I ordered the ThinkServer from eBay/Newegg here: Lenovo ThinkServer TS140 70A4003AUX Tower Server
A note on memory: I'm going to try the 4GB of memory to start. It comes as a single DIMM (as opposed to two or more DIMMs) which leaves three open slots. So if more memory is needed I can add another 4GB DIMM later. I had tried running a custom built storage server three or four years ago using a desktop board with non-ECC memory running FreeNAS. I had read on the forums multiple warnings that only ECC memory should be used due to the risk of bit rot and eventual data corruption. Well, it only took a couple weeks before VMs stored on that systems starting becoming corrupted. Having been bit by this bug, I won't run a storage server without ECC memory and highly, highly, highly recommend you do the same.
A note on storage: I'm going to reuse the two 4TB drives in my Synology for this system in addition to two more 4TB drives I'll purchase for this build. The TS140 has an integrated hardware RAID controller. So for the data volume, I'll install the two new drives and pull one of the Synology drives for a total of three. Then transfer the data off the Synology to the new system. Once I'm satisfied everything is in good working order, I'll pull the fourth drive and install into the new system, expand the volume, etc. Then I'll have a very nice DS713+ for sale!
A note on the OS: I will miss the easy UI experience Synology DSM provides. However, I have found a decent alternative: Rockstor. I looked at several options: OpenFiler (dead) NexentaStor, FreeNAS, etc. But Rockstor offers features that I really want: web-based interface, based on CentOS 7, Docker-based plugin system called Rock-ons, optimized to run from an USB drive and Apple Time Machine Support. I also use the Synology for security cameras but found Rockstor also supports ZoneMinder video surveillance system.
It's not all rainbows and unicorns though. I'm not very confident about BTRFS (pronounced 'butter FS'). Rockstor recommends against using RAID5 or 6 for production systems. No worries for me since I'm using hardware-based RAID. I've also read that newer versions are much more stable than early releases. So while they tout BTRFS as a big feature, it may actually be their biggest drawback. Time and testing will tell. There must be a reason that many if not most NAS companies are now offering this filesystem as an option on their devices.
Also note that the TS140 has an internal USB port - perfect for a USB flash drive to host the OS.
Here's the parts list:
System: Lenovo ThinkServer TS140 70A4003AUX Tower Server = $320
4TB Enterprise SATA x 2 = $177
16GB USB Flash Drive: Transcend 16GB JetFlash 820 = $10
Operating System: Rockstor = Free
RAID Adapter (if needed): PERC H310 = Free
NIC Adapter = Intel Quad port = Free
Application Mapping:
Plex Media Server (Synology package) = PMS Rock-on (Rock-on)
Surveillance Station (Synology package) = ZoneMinder (Rock-on)
CrashPlan (Community Package) = CrashPlan (Rock-on or native)
Photo Station (Synology Package) = TBD
Download Station (Synology Station) = CouchPotato? TBD
So for $100 less I'll have a systems that's much more powerful and expandable than a Synology 916+ with twice the capacity. I'll follow-up with some additional thoughts once I have run this build through its paces.
Subscribe to:
Posts (Atom)