A change is coming in the hard drive industry. As storage densities have increased dramatically over the years, one of the most elemental aspects of hard drive design, the logical block format size know as a sector, has remained constant.
Beginning in late 2009, accelerating in 2010 and hitting mainstream in 2011, hard drive companies are migrating away from the legacy sector size of 512 bytes to a larger, more efficient sector size of 4096 bytes, generally referred to as 4K sectors, and now referred to as the Advanced Format by IDEMA (The International Disk Drive Equipment and Materials Association).
This paper provides the context for this migration as well as pointing out the long-term benefits to customers and potential pitfalls to be avoided in the move from 512 bytes to 4K sectors.
For over 30 years, data stored on hard drives has been formatted into small logical blocks called sectors. The legacy sector size is 512 bytes. In fact, many aspects of modern computer systems still have design assumptions with regard to this fundamental format standard.
The legacy sector format contains a Gap section, a Sync section, an Address Mark section, a Data section and Error Correction Code (ECC) section (Figure 1).
Figure 1. Legacy Sector Layout on Hard Drive Media
The structure of this sector layout was designed as follows:
- Gap section: The gap separates sectors.
- Sync section: The sync mark indicates the beginning of the sector and provides timing alignment.
- Address Mark section: The address mark contains data to identify the sector’s number and location. It also provides status information about the sector itself.
- Data section: The data section contains all the user’s data.
- ECC section: The ECC section contains error correction codes that are used to repair and recover data that might be damaged during the reading or writing process.
This low-level format has served the industry well for many years. However, as hard drive capacities have increased, sector size has increasingly become a limiting design element in improving hard drive capacities and error correction efficiency. For example, comparing the sector size to the total capacity of earlier hard drives versus more recent ones, you can see that the sector resolution has become extremely small. The resolution of the sector (the ratio of sectors as a percentage of total storage) has become very fine and increasingly inefficient (Table 1).
Very fine resolution is good when managing small, discrete amounts of data. However, applications common in modern computing systems manage data in large blocks, much larger in fact than the legacy 512-byte sector size.
More important, the small 512-byte sector has consumed a smaller and smaller amount of space on the hard drive surface as areal densities have increased. This is a problem in the context of error correction and the risk of media defects. In Figure 2, for example, the data in a hard drive sector is consuming smaller areas, making error correction more challenging because media defects of the same size can damage a higher percentage of the total data payload and, therefore, require more error correction strength.
Figure 2. Media Defects and Areal Density
A 512-byte sector can typically correct a defect of up to 50 bytes in length. Today’s hard drives are beginning to push the limits on error correction with leading areal densities. Consequently, the migration to larger sectors within the hard drive industry has become a fundamental need relative to gaining improvements in error correction and achieving format efficiencies.
The Transition to 4K Sectors (Advanced Format)
The storage industry has been collectively working on a transition to larger sector hard drive formats for years; significant work at Seagate and with our peers in the hard drive industry dates back at least five years (Figure 3). In December 2009, through a co-ordinated effort within IDEMA, Advanced Format was nominated and approved as the name for the standard of 4K-byte sectors. In addition, all hard drive manufacturers have committed to shipping new hard drive platforms for desktop and notebook products with the Advanced Format sector formatting beginning in January 2011. Prior to this date, Advanced Format drives will begin to enter the market. Western Digital began shipping an Advanced Format drive in December of 2009, and Seagate has been shipping large sector drives for some time to OEM customers and in branded retail products, most notably USB-attached external hard drives such as the Seagate® FreeAgent® series.
Figure 3. Key Milestones in the Development of the Advanced Format Standard
The Long-Term Benefit to 4K Sectors
As all hard drive manufacturers have agreed to transition to the Advanced Format sector design by January 2011, the industry must adapt to and embrace this change to minimise potential negative side effects. Short-term benefits to end users will not be dramatic in terms of immediate capacity increases. However, the migration to 4K-sized sectors will most definitely provide quicker paths to higher areal densities and hard drive capacities as well as more robust error correction.
Improved format efficiency by reducing the amount of space used for error correction code
In Figure 4, the legacy 512-byte sector layout is shown, where each 512-byte sector has a non-data-related overhead of 50 bytes for ECC and another 15 bytes for the Gap, Sync and Address Mark sections. This yields a sectorised1 format efficiency of about 88 per cent (512/(512 + 65)).
Figure 4. Legacy 512-Byte Sector Layout
The new Advanced Format standard makes the move to a 4K-byte sector, which essentially combines eight legacy 512-byte sectors into a single 4K-byte sector (Figure 5).
Figure 1. Advanced Format: 4K-Byte Sector Layout
The Advanced Format standard uses the same number of bytes for Gap, Sync and Address Mark, but increases the ECC field to 100 bytes. This yields a sectorised1 format efficiency of 97 per cent (4096/(4096 + 115)), almost a 10 per cent improvement.
Over time, these format efficiencies will pay off, helping to yield higher capacity points while also improving data integrity.
Reliability and Error Correction
While the physical size of the sectors on hard drives has shrunk, taking up smaller and smaller amounts of space, media defects have not. Consider Figure 6, which displays images of what we would consider very small objects. In relation to the fly height of a hard drive read/write head, these objects are relatively large. Microscopic particles much smaller than those in this diagram can create media defects on a hard drive.
Figure 6. Small-Scale Representation of Hard Drive Fly Height
The larger 4K sector in the Advanced Format standard approximately doubles2 the size of the ECC block from 50 bytes to 100 bytes, providing a much needed improvement in error correction efficiency and robustness against particles and media defects.
Together, the benefits of improved format efficiency and more robust error correction make the transition to 4K sectors well worth the effort. Properly managing this transition to capture the long-term benefits with minimal side effects is a key focus for the hard drive industry.
Understanding the Impacts to the 4K Transition
As noted earlier, there are many aspects of modern computing systems that continue to assume that sectors are always 512 bytes. To transition the entire industry over to the new 4K standard and expect all these legacy assumptions to change suddenly is simply not realistic. Over time, the implementation of native 4K sectors, where both host and hard drive exchange data in 4K blocks, will take place. Until then, hard drive manufacturers will implement the 4K sector transition in conjunction with a technique called 512-byte sector emulation.
512-Byte Sector Emulation
The introduction of 4K-sized sectors will depend heavily on 512-byte sector emulation. This term refers to the process of translating from the 4K physical sectors used in Advanced Format to the legacy 512-byte sectors expected by host computing systems.
The 512-byte emulation is acceptable in that it does not force complex changes in legacy computing systems. However, it carries the potential for negative performance consequences, particularly when writing data that does not neatly correspond to eight translated legacy sectors. This becomes clear when considering the reading and writing process required by 512-byte emulation.
Emulated Read and Write Processes
To read data from a 4K sector formatted drive in 512 emulation mode, the process is very straightforward, as shown in Figure 7.
Figure 7. Potential Read Sequence for 512-Byte Emulation
The process of reading the 4K block of data and reformatting the specific 512-byte virtual sector requested by the host computer is performed in the drive’s DRAM memory and does not measurably impact performance.
A write process can be more complicated, particularly when data the host computer attempts to write is a subset of a physical 4K sector. In these cases, the hard drive must first read the entire 4K sector containing the targeted location of the host write request, merge the existing data with the new data and then rewrite the entire 4K sector (Figure 8).
Figure 8. Potential Write Sequence for 512-Byte Emulation
In this instance, the hard drive must perform extra mechanical steps in the form of reading a 4K sector, modifying the contents and then writing the data. This process is called a read-modify-write cycle, which is undesirable because it has a negative impact on hard drive performance. Minimising the probability and frequency of read-modify-write instances is the most important aspect of making the transition to 4K sectors smooth and painless.
- Write requests that are misaligned because of logical to physical partition misalignment
- Write requests smaller than 4K in size
Aligned Versus Unaligned Hard Drive Partitions
Up to now we have not discussed how host systems and hard drives communicate the location of sectors on the media. It is time to introduce the Logical Block Address (LBA).
Each 512-byte sector is assigned a unique LBA, from zero (0) to the number required based on the size of the disc. The host requests a specific block of data using the assigned LBA. When the host requests to write data, an LBA address is returned at the end of the write telling the host where the data is located. This becomes important in the transition to 4K sectors, since there are eight different possibilities for where the host LBA starts.
When LBA 0 is aligned to the first virtual 512-byte block in the 4K physical sector, the logical to physical alignment condition for 512-byte emulation is termed Alignment 0. Another possible alignment is when LBA 0 is aligned to the second virtual 512-byte block in the 4K physical sector. This situation is termed Alignment 1 and is shown in comparison to the Alignment 0 condition in Figure 9. There are six additional possibilities for unaligned partitions that can result in read-modify-write events similar to the Alignment 1 condition.
Figure 9. Alignment Conditions
Alignment 0 conditions work very well with the new 4K sectors in the Advanced Format standard. This is because a hard drive can easily map eight contiguous 512-byte sectors into a single 4K sector. This is accomplished by storing 512-byte write requests in the hard drive’s cache until enough contiguous 512-byte blocks are received that form a 4K sector. Since modern computing applications deal with chunks of data which are typically larger than 4K, runts are extremely rare. However, the Alignment 1 situation is another matter.
When hard drive partitions are created that result in an unaligned condition as shown in Figure 9, this results in read-modify-write cycles that can slow hard drive performance. This is the key condition that must be avoided in the implementation of Advanced Format hard drives, and is discussed later.
In modern computing applications, data such as documents, pictures and video streams are much larger than 512 bytes. Therefore, hard drives can store these write requests in cache until there are enough sequential 512-byte blocks to build a 4K sector. As long as hard drive partitions are aligned, the hard drive can easily map 512-byte sectors into 4K sectors without any performance penalties. There are, however, certain low-level processes that can force a hard drive to deal with runt situations that are not associated with unaligned partitions. These occur in rare instances where the host makes discrete write requests that are actually smaller than 4K. These are typically operating system level activities dealing with the file systems, journaling or similar low-level activities. Generally, these occur in small enough numbers that overall performance is not significantly impacted. Still, it is recommended that system designers consider proper modifications to any of these processes to maximise performance as the 4K transition becomes reality.
Preparing for and Managing the 4K Transition
Now that we understand the benefits of migrating to 4K sectors as well as the potential impacts to performance, it is time to examine how best the industry can manage this transition. This topic is best discussed in the context of the two most popular operating systems deployed in modern computing: Windows and Linux.
Managing 4K Sectors in the Windows Environment
The single most important aspect of managing the transition to 4K sectors is related to the alignment issues described above. Advanced Format drives work well in an Alignment 0 condition, where the physical to logical starting positions are equal. Alignment conditions are created when the hard drive partition(s) is created. Partitions are created by software that falls into two general categories:
- Windows OS releases
- Hard drive partitioning utilities
In the case of partitions created with the Windows operating system, there are three Windows releases that warrant discussion: Windows XP, Windows Vista and Windows 7. Microsoft was involved with the community planning the transition to larger sectors. Consequently, they released 4K-sector-compatible software beginning with Windows Vista Service Pack 1. Software that creates partitions of Alignment 0 (those that work well with the Advanced Format standard) is referred to as “4K aware”. Table 2 describes the situation relative to current generations of the Microsoft Windows operating system.
|Operating System Release
||Creates primary partition with Alignment 1 condition (unaligned)
|Windows Vista - Pre Service Pack 1
||Large sector aware but creates partitions incorrectly (unaligned)
|Windows Vista - Service Pack 1 or later
||Creates partitions with Alignment 0 condition (aligned)
||Creates partitions with Alignment 0 condition (aligned)
Clearly, new computer systems shipped with the latest versions of Windows will be in a very good situation relative to the use of Advanced Format hard drives. However, for systems still using Windows XP or Windows Vista, pre-Service Pack 1, there is significant risk of reduced performance relative to the partitions created by the OS.
In addition to the potential for unaligned partitions created with older versions of the Windows OS, there are also a number of software utilities that are widely used by system builders, OEMS, value-added resellers and IT managers that can also result in unaligned partitions. In fact, it is more common for partitions to be created with these types of utilities rather than with the Windows OS itself. Thus the risk of creating unaligned partitions, and therefore an environment that can result in degraded hard drive performance for drives using 4K sectors, is significant. To complicate the matter, hard drives shipped in systems today typically consist of multiple hard drive partitions. This means that each partition on the hard drive must be created with 4K-aware partitioning software to make sure proper alignment and performance is ensured. Figure 10 shows the potential results of creating multiple hard drive partitions with software that is not 4K aware.
Figure 10. Multiple Partitions and Alignment Conditions
Dealing with Unaligned Conditions
There are three potential methods to avoid and/or manage an unaligned condition that has the potential to impact hard drive performance.
- Use a new version of the Windows OS or work with your partitioning utility provider to obtain a 4K-aware version of the software.
- Use a hard drive utility to realign disc partitions.
- Depend on your hard drive supplier to manage performance regardless of the alignment condition.
Using a 4K-aware version of Windows to create hard drive partitions is a simple and straightforward method to avoid unaligned conditions. Vendors of software utilities that create hard drive partitions should be able to tell you if 4K-aware versions are available. If so, migrate to these versions to avoid further concern.
Some hard drive manufacturers are dealing with this issue by offering utilities that examine existing hard drive partitions and realign as needed. This alternative takes additional time and adds steps to the system building or upgrading process.
Ultimately, hard drive manufacturers will develop more sophisticated methods to manage unaligned conditions while avoiding negative performance impacts.
As the transition to Advanced Format drives gains momentum, all these methods will play a role in maximising the industry benefits while also avoiding any potential performance impacts.
Managing 4K Sectors in the Linux Environment
The key strategies in managing the transition to 4K sectors in a Windows environment also apply to Linux. Most Linux system users have access to the source code, giving them the ability to customise the OS to fit their specific needs. This provides an opportunity to proactively update their Linux system to manage Advanced Format hard drives properly.
Creating disc partitions properly that are well-aligned to Advanced Format drives and minimising small system level writes that can generate runts independently of alignment issues, can largely be avoided by making modifications to your Linux system.
Changes have been made to both the Linux kernel and utilities to support Advanced Format drives. These changes ensure that all partitions on Advanced Format drives are aligned properly on 4K sector boundaries. Kernel support for Advanced Format drives is available in kernel versions 2.6.31 and above. Support for portioning and formatting Advanced Format drives is available in the following Linux utilities:
Fdisk: GNU Fdisk is a command line utility that partitions hard drives. Versions 1.2.3 and above support Advanced Format drives.
Parted: GNU Parted is a graphical utility for partitioning hard drives. Versions 2.1 and above support Advanced Format drives.
The industry transition away from the legacy 512-byte sector is a certainty. Hard drive manufacturers have all agreed to adopt the Advanced Format standard no later than January 2011 for new models shipped to the notebook and desktop market segments.
This transition will provide hard drive engineers with another tool to continue to drive improved areal densities and more robust error correction. Consumers will benefit as hard drives continue to offer the higher capacities, lower costs per gigabyte and continued reliability levels expected by hard drive technology.
The key to a smooth transition is a well-educated storage community, so that potential performance pitfalls can be avoided. The most critical aspect of a smooth and successful transition to 4K sectors used in Advanced Format is to promote the use of 4K-aware hard drive partitioning tools. As a system builder, OEM, integrator, IT professional or even an end user who is building or configuring a computer, be sure to:
- Use Windows Vista (Service Pack 1 or later) or Windows 7 to create hard drive partitions.
- When using third-party software or utilities to create hard drive partitions, check with your vendor to make sure that they are updated and confirmed to be 4K aware.
- If you have customers who commonly re-image systems, encourage them to make sure that their imaging utilities are 4K aware.
- If you are using Linux, check with your Linux vendor or your engineering organisation to make sure that your system has adopted the changes to become 4K aware.
- Check with your hard drive vendor for any other advice or guidance on using Advanced Format drives in your systems.
Together with our industry colleagues and customers, we can make the transition to Advanced Format 4K sectors smooth and efficient, leveraging the long-term potential benefits for the entire storage industry.
1Sectorised format refers strictly to the data sectors and does not take into account the overhead associated with servo data and other sector layout inefficiencies.
2Not every implementation of a 4K sector exactly doubles the ECC bytes when moving from 512-byte to 4K sectors.