This Page

Windows Defragmentation

Windows Storage

Windows File Systems
Windows NTFS
Windows DFS
Windows CIFS
Virtual Disk Services
Volume Shadow Copy Services
Removable Storage System
Windows Volume Mgmt.
Defragmentation
Compression
Windows System state

Novell Netware

File Systems
Disks and Volumes
NDS and eDirectory
Clustered Servers
iFolder
Netware Volume Statistics

Who are we?

Welcome to Lascon Storage. The site was founded in 2000 and provides hints and tips on how to manage your data, strategic advice, news items and a discussion forum. Here is a list of recent major updates
TSM hints and tips from IBM, updated May 2013
VCS Cluster Backups added April 2013
Enterprise disk subsystems updated March 2013
HDS Remote Replicator added February 2013
Some 6.x database updates added February 2013
FDRABR pages updated, February 2013
Oracle Databases added January 2013
SRDF mirroring updated December 2012

Lasconet Forum

Visit the Lasconet Forum

Windows Storage - Defragmentation

Fragmentation explained
The Windows Disk Defragmenter
Windows versions
Third party Products

What is Fragmentation?

When a Windows operating system is first installed on a disk and then applications are added, all the files are contiguous and arranged in the optimal place on the disk for performance. This means that the read-write heads do not need to move much to read a single file. A file is fragmented when it is not contiguous, but is split up into pieces in various parts of the disk. Now the read-write heads have to move back and forward over the disk to read the whole file. This takes time, and causes 'seek delay' performance problems.

When a file is updated, it usually needs space to grow. The NTFS file system leaves space before and after a file, to allow it a bit of room to grow without causing fragmentation. Microsoft initially thought that this meant that the NTFS was fragmentation proof, and were so confident that they did not supply a defragmentation tool. However, this is not quite correct. The 'guard space' which NTFS leaves on either side of files is about 30% of the space used by the file, so once an NTFS partition hits the 70% full mark, the data has actually hit the end of the disk. When more new files are added, they will be allocated into the 'guard bands', and may well be fragmented.
Over time, as files are deleted and new files added, the free space becomes fragmented, so it is impossible to allocate a new file into a single space. Fragmentation and performance will get worse with time.

In the real world, performance does not die when partitions are loaded beyond 70%. This is probably due to a combination of the other NTFS performance features, and also that the big cache in RAID disks mask sequential access and seek problems (because the files are no longer allocated in nice contiguous chunks, but are split up and striped over several disks).

A defragmentation tool basically works by moving files around on the disk, so each file becomes consolidated into one contiguous extent, and to consolidate the freespace into a single contiguous extent for new allocations. This ideal cannot be achieved in practice, as some files cannot be moved.

Unmovable files

Files which cannot be moved with Windows active are
Bootsect.dos
Safeboot.fs
Safeboot.csv
Safeboot.rsv
Memory.dmp
FAT directories
Change journal
and any other files open for exclusive use.

Master File Table: The MFT is an index file that maps everything stored on a disk, and is usually held right in the centre of the disk. The MFT is explained in the NTFS page. As MTF entries are being constantly added or deleted as files are created or deleted, the MTF gradually becomes fragmented, resulting in longer disk access times and diminished performance.
Paging File: Windows uses the Paging File to swap data in and out of Random Access Memory (RAM). When the RAM fills up, Windows needs to create more room to load programs or data, so it swaps the oldest data from memory out to the paging file on disk. If the Paging File becomes fragmented, performance will suffer, and you can usually hear the hard drive thrashing while you wait for a response.
Hibernate File: The Hibernate File is where memory data is stored when the computer is turned off. Because the location of the Hibernate File is determined very early in the boot process, it cannot be moved.

It is impossible to defragment any of the three files above using normal Windows compression tools, you need a third party tool which works at boot time

Directories can also become fragmented; adding to the time it takes to access files. Windows 2000 directories on an NTFS disk can be defragmented while the operating system is running. However, FAT directories can only be defragmented at boot time.

Good defragmentation policies

Delete any files that you don't need before starting a defrag
Purge the deleted files using a utility like spybot
Run a chkdsk to make sure your disk is not dirty.
Make sure you have at least 15% free space to be used as a sorting area
Kill off as many active processes as possible to minimise files in use
Do regular defrags.

The Windows Disk Defragmenter

Microsoft provided a system defragmenter tool from Windows 2000 onwards. It is based on the Diskeeper product described below. It has two functions, an analyze function to determine if a disk needs defragging, and the defragmentation function itself. To run a defrag -

Click Start - Programs - Accessories - System Tools - Disk Defragmenter; Select the volume that you want to check for fragmentation; Click Analyze and the message box will tell you if a defragment is necessary. If so, click Defragment.

A few pointers -

It is best not to do other tasks while Disk Defragmenter is defragmenting a volume, because response times will be poor, and because if you hold exclusive locks on files, they will not be consolidated.
Defragment will not run if the disk has any errors. You may want to use Scandisk to check this first.
While the defragmenter is running, you can see a detailed colour map of what's happening. Check out the Legend box to see what the the colours mean. Each coloured pixel can represent thousands of clusters, each of which can be one of several states. The pixel value represents the state of the biggest number of clusters in that group.
You need a reasonable amount of free space to run a defrag successfully. Microsoft recommend 30% free space, excluding any free space reserved for the MFT.
A defrag can take several hours. Defragmentation speed is related to the number of files on a disk, so it will take much longer for large, full disks.

Defragmentation Improvements in various Windows Versions

Windows 2008

Windows 2008 has a command line defrag facility. To analyse your C: drive to see if a defrag is required, open up a command window and enter defrag c: /A. The command has several switches, some of which are:

/C - all volumes, for example 'defrag /C' will defrag all volumes
/X - consolidate freespace
/V - verbose
/U - provide progress report
/M - run defrages in parallel, for example 'defrag C: D: /M' will defrag both the C and D drives at the same time.

Windows 2003

The main enhancements are -

A fragmentation process can be scheduled or scripted
Performance is faster than in Windows 2000
You can defragment the MFT online. The MFT is an index to all the files on the system including itself, so it needs to be optimised. Disk Defragmenter cannot move the first 16k of the file, as this describes the file system. However it will consolidate the rest of the file.
You can now defragment any NTFS cluster size
Defrag works fine on Windows cluster volumes, as long as defrag scheduling is done by virtual node name or cluster name
Disk Defragmenter will try to place all the files needed at boot time into a single contiguous extent to optimise boot performance.
You can run reports on the defragmentation status of remote servers with the DefragAnalysis report. This only works when running on a Windows XP workstation, reporting on Windows 2003 servers

Third party tools

Diskeeper

Diskeeper is supplied by Executive Software, and in fact, the Microsoft defrag tool used Diskeeper code. So why would you buy a third party tool when Microsoft provide one with the operating system?
Diskeeper defrags your directories at boot time, which might mean the boot takes longer, but it can improve performance.
Diskeeper can defragment the paging file and the MFT at boot time. Its also possible to schedule this for remote servers using remote boot t
The Server Enterprise edition will defrag very large volumes, up to 4 terabytes, and can do 16 simultaneously. It can also can be managed with Diskeeper Administrator. This edition is required for Windows Server 2003 Enterprise Edition and Windows 2000 Advanced Server

PageDefrag

PageDefrag is a freeware utility that is specifically designed to let you defragmented your paging files and Registry hives, event log files and Windows hibernation files. This could make it an ideal complement to the Windows Defragmentation tool. It supports Windows systems from NT 4.0 upwards.

When you run PageDefrag, it gives you a listing of how many clusters make up your paging files, event log files, and Registry hives as well as how many fragments those files are in. You then select a radio button to decide if you want to defragmented the files at the next reboot. Immediately after CHKDSK examines your hard drives PageDefrag uses the standard file defragmentation APIs to defragment the files. As it processes each file PageDefrag will print on the boot-time startup screen the file name and its success at defragmenting it. If it is successful at reducing the fragmentation it will tell you the number of clusters the file started with and the number it consists of after the defragmentation.

O&O Defrag

O&O Defrag has the usual defrag facilities, it can defrag systems files at boot time, and it will run in 'screen saver' mode to automatically defragment disks when a computer is not is use. It integrates with Windows explorer to present a simple user interface. It will defragment an unlimited number of drives or partitions, and can defragment almost any type of disk including USB sticks, memory cards and RAID devices, with only about 5% free space needed for operation (the Windows defragmenter needs 15% free space).