ZFS pools are comprised of virtual devices. ZFS abstracts every physical device into a virtual device. A vdev can be a disk, a slice of a disk, a file, or even a logical volume presented by another volume manager such as Solaris Volume Manager (SVM) or a LUN from a hardware RAID device.
These are the virtual devices types:
• Dynamic stripe: A dynamic stripe is a nonredundant configuration of a simple disk or concatenation of disks.
• Redundant group (mirror, RAID-Z1, or RAID-Z2): A mirror can be a two-way or three-way mirror. RAID-Z groups are recommended to have up to nine disks in the group. If there are more, then multiple vdevs are recommended. Two disks minimum are needed for RAID-Z, and three disks at a minimum are needed for RAID-Z2. (Note that RAID-Z and RAID-Z1 are interchangeable terms. With the introduction of the RAID-Z2 feature, the term RAID-Z evolved into RAID-Z1 to differentiate it from RAID-Z2.)
• Spare: A spare vdev is for a hot standby disk replacement.
• Log: A log vdev is for ZFS Intent Log (ZIL). The ZIL increases the write performance on ZFS. Only dynamic stripe and mirrored vdev configurations are supported for this vdev type.
• Cache: A cache vdev is used to speed up random reads from a RAID-Z-configured pool. Its intended use is for read-heavy workloads. There is no redundancy support at this point for this vdev type. If there is a read error, then ZFS will read from the original storage pool.
Log and cache vdevs are used with solid-state disks (SSDs) in the Sun Storage 7000 series in hybrid storage pools.
Best-practice guidelines for ZFS pools include the following:
• Mirrored configuration beyond a three-way mirror should not be used. Think about using a RAID-Z configuration instead.
• Use RAID-Z or RAID-Z2 virtual device groups with fewer than ten disks in each vdev.
• Using whole disks is best. ZFS works best with multiple disks.
• Use slices for vdev groups only for boot disks.
• Use disks of a terabyte or less for boot devices.
• Use matched-capacity disks (mixed geometry is OK) for the best maximum storage results.
• Use matching sizes of vdevs in a ZFS pool. Match the number of disks and redundancy groups in each vdev in a pool for best performance.
Creating/adding new vdevs to a ZFS pool is the most unforgiving part about ZFS. Once committed, some operations cannot be undone. The
zpool command will warn you, however, if the operation is not what’s expected. There is a
force option in
zpool to bypass any of the warnings, but it is not recommended that you use the
force option unless you are sure you will not need to reverse the operation.
These are the rules for ZFS pools:
• Once a normal (dynamic stripe) vdev is added to a ZFS pool, it cannot be removed.
• Only the special-use vdevs can be removed: spares, log, and cache.
• Disks the same size or larger can be replaced within a vdev.
• Disks can be added to a single disk or mirrored vdev to form a mirror or a three-way mirror.
• New disks cannot be added to an existing RAID-Z or RAID-Z2 vdev configuration.