If you want to get the most out of today’s storage hardware, the ZFS File System is a technology you are probably already using or thinking about using in the near future. Today’s vast number of storage products in the way of High-RPM spinning hard-drives, Solid State drives, PCI-E Cache drives, etc., give IT Storage administrators a wealth of options to choose from when engineering an efficient, tiered storage solution. The latest, SAS 3.0 technology also plays into faster, more efficient storage models by offering new features on a number of different RAID and HBA cards. Just throwing new hardware into a storage solution, however, is hardly the basis for creating efficient, fast and robust storage solutions. The ZFS File System is unique in that it quickly adopts some of the very latest storage products on the market while rejecting some of the others. To understand this better, it helps to gain some perspective on what the ZFS File System actually is or does.
What is ZFS?
The ZFS File System is a highly innovative and powerful file system that completely changes some of the rules and methods of managing storage. There are gads of powerful storage administration features packed inside the ZFS file system which are not found in other software operating systems. For instance, the ZFS file system completely makes even the very latest and best RAID cards totally obsolete. RAID is handled through the operating system itself, but more on that later. ZFS allows storage administrators the flexibility and power to manage an extremely robust and scalable storage architecture. Rather than use a single, physical device for managing storage volumes, ZFS utilizes storage pools for managing and administrating multiple storage devices. Through a volume manager, the ZFS file system allows these pools of storage built upon multiple devices to be represented as a single device. Consequently, there is no need for any modifications to take place when additional devices are added to the storage array. ZFS is a smart, robust and tidy file system which is completely scalable. For example, if new drives are added, multiple file systems within the pool can immediate utilize the additional capacity without incurring any performance hits. You can think of it as a something similar to when additional memory is added to a storage server. The storage benefits are immediately beneficial to the entire storage system. With the ZFS File System, storage can be added on the fly and be utilized accordingly without making any sacrifices of other hardware resources. In a ZFS file system, the physical description of every single hardware device is described within the storage pool. These characteristics include the layout of your device hardware, redundancy levels, etc., etc.. All of the disk space in a storage pool can be shared. This eliminates the headaches of planning and predicting the size of your file system. Rather, the file system is automatically accommodated by the amount of disk space required by the storage pool.
Traditional file systems overwrite data in its same place which means that a power failure can result in data loss during power losses or other hardware failures. Additionally, the ZFS file system uses a method of data and metadata verification with use of a user-selected checksum formula. The chances of incorrect data being written is all but eliminated since ZFS checksums are stored in away that makes these failures readily detectable. This, of course, is all done at the file system layer. The reality of the ZFS file system is that this where all of the data verification is done, thus greatly reducing overhead when recovering data from applications. ZFS storage offers a much more streamlined, transparent process for correcting and recovering data. So far, we’ve only touched a little bit on the benefits of ZFS storage.
Benefits of ZFS
In addition to offering the typical NAS and SAN storage benefits such as Snapshots and Deduplication, the ZFS file system makes things a whole lot easier for administrators to manage. With its efficient use of a hierarchical file system, it is extremely simple and easy to create and manage entire file systems with no requirements or knowledge of cryptic commands or editing of configuration text files. But, in today’s world of quickly advancing storage hardware, there is even more than meets the eye. The ZFS file system is a storage solution which makes more efficient use of today’s hardware.
How to Use ZFS
There is certainly no one-size-fits-all rule for using ZFS for any particular storage or server environment. However, there are some best practices when it comes to getting the most out of the ZFS file system. Knowledge of today’s storage hardware and its capabilities is definitely one of the keys in engineering a successful ZFS storage solution. Today’s fastest SSD drives are one of the keys to a successful ZFS implementation. But, besides just speed, it is important to choose SSD drives with extremely high endurance levels. SSD drives serve two purposes: Cache (as in the L2ARC cache) and ZIL (Zero Intend Log). ZIL is where data synchronous data writes are turned into asynchronous writes. In turn, this assists the database operations of the file system. Since the data is written into something similar to a journal log this means that it is constantly being written to which in-turn requires very high-endurance, high-speed caching drives. The very latest SSD drives on the market are best suited for this purpose. Look into STEC and Intel’s latest SSD DC3700 series SSD drives. Addtionally, affordable SSD drives are also important for extra L2ARC Cache. One important thing to note is that memory is the primary caching level of a ZFS file system. If you can afford lots of internal, system memory, this will reduce the need for an expensive SSD drive investment for caching. Perhaps, the most important consideration is choosing quality, spinning, enterprise-level data drives for storing long term data. Isn’t that pretty much true of any type of enterprise storage environment? With that said, what type of RAID redundancy is offered by the ZFS file system and what are its hardware requirements?
ZFS RAID Levels
When it comes to data redundancy, ZFS requires less, not more. There is no need to use a RAID card in a ZFS file system. In fact, using a RAID card will almost certainly add to the chances of complexity and problems with data corruption. The ZFS file system offers it’s own RAID, similar to Hardware levels of RAID 0, 1, 5, 6, etc., etc. For single, double or even triple drive redundancy, all that is required is a Host Bus Adapter (HBA) RAIDZ is similar to a mirror RAID; RAID Z1 the equivalant of a RAID 5; RAIDz2 a RAID-6 and RAIDz3 is like a RAID-6 on steroids with an additional drive’s worth of redundancy. In general, RAIDZ1 like RAID-5 should never be used with disks larger than 1TB. The chances of a big drive failing during a rebuild is too great. Another good practice is that RAIDz1 should be built with 3 to 7 disks in each vdev. RAIDz2 should beult with 6-10 disks per vdev. RAIDz3 can use betwen 7 and 15 disks. One thing to consider: Rather than waste an extra drive as a hot-swap, just add an extra measure measure of ZFS redundancy. The other consideration is that mirrors are better than any type of RAID for speed. As possible and budgets allow, consider 3-way mirrors. With RAIDZ, disk sizes in data pools cannot be mixed. It is also imperative that you plan ahead with your capacity requirements ad determined by the size and number of hard drive bays in your server, chassis and JBODs. For instance, if your initial drive capacity in a 24-bay storage chassis consists of 12 x 1TB drives, you will need to expand with 12 x 1TB drives.
How to use ZFS
Remember that ZFS is not a one size fits all approach. The first step in implementing an efficient storage solution is to plan ahead. How much storage do you need today and how much will you need tomorrow? Choose a hardware platform that not only fits the budget of your IT department today, but will accommodate the needs of tomorrow. Supermicro is an excellent manufacturer with storage solutions ranging from very small, to very large and expandable. Remember, what is fast and popular today in drives and performance will soon be outdated. If you keep your goals in line with your budget, you will quickly find an expandable ZFS storage solution that will accommodate your data capacity and performance requirements for years to come.
Important ZFS Resources and Links
Below are some of my own favorite links for ZFS tips, ZFS RAID Explanations, explanations on the ZFS File System intricacies and more.