Defraging

[TRN]

This message is posted on behalf of Muskrat ... until he finds his way into our forums. He sent a pm asking howto defrag his GNU/Linux system. Well, one of the great things of GNU/Linux is that you don't need to defrag ! My only suggestion is to watch the number of processes chugging along your desktop because too many can slow the little guy down. Keep it light. Keep it simple. Keep it happy :-)

[muskrat]

I understand due to the nature of Linux it don't need defraged but about your personal files in the home directory.

If I right a file to home the write more files. After some time I delete some of said files writng more and deleting more. My question is wouldn't those files become fragmented?

All the righting and deleting going on, after some time I would think that it would need to defrag, or is that something linux does on it's own at boot. :?

[Davepet]

I would like some more info on this as well. Although I have heard that Linux doesn't need to be defragged, I have never heard a good explaination why.

When you delete a file, does it leave a "hole" & wait until you save a file that will fit? Or do you end u with a disk full of holes.

Dave

[TRN]

OK. Let's see how far we can get into this discussion without getting me lost :-)

First, there is a defrag utility available for ext2 partitions. My assumption is that most of us visiting this site are using ext3, ReiserFS, JFS, or XFS. Well, we know about assumptions. So, let's step back for a second.

There are numerous filesystems available on a GNU/Linux system, including ext, ext2, ext3, ReiserFS, XFS, JFS, GFS, a bunch more that I can't think of off the top of my head (someone else might want to post them). The original was the Minix filesystem - since Linux was developed with Minix in mind. Just realize that each filesystem handles the data differently. So, check to see what filesystem you are using. You also might want to know that the VFS layer in the kernel allows these newer filesystems to be implemented. THis was a really cool idea. Bravo to the people behind it.

OK, back to my story. ext is an extended filesystem and was used in the earliest versions of Linux to extend support for features not supported in the Minix filesystem. ext2 was the second extended filesystem. ext3 was the third. ext2 was chosen years ago after the Linux kernel was able to support newer features (the VFS layer was put into the kernel). It was chosen because there was extra code that could handle the data structure in a faster manner (and many other reasons). Basically, there is meta-data that handles the placement of files. ext was limited in the size of the hard drive. I think it was 2 GB partitions (?). ext2 was largely based on BSD implementations.

Update

Here is a good link:
http://web.mit.edu/tytso/www/linux/ext2intro.html

The link explains the VFS far better than I could ever explain it. It also states that the ext2 allowed the longer file names. :oops: forgot that part!

ext3 and ReiserFS are really new for Linux users. They help speed up the boot time as well as added journaling (keeping track of things so that you can roll back to it on a reboot if there is a problem). Linux systems were great servers because they didn't have to be rebooted. However, upon reboot there was a checker run. Depending upon the size of the disk, this could take a long time. It was important to run it - but it took time. The newer filesystems allow us to skip through this check. However, if you turn off your system daily then you should run it every so often. Actually, I'd suggest leaving your machines on. It's better on the hardware and the OS can definitely handle it. There is no need to turn off a GNU/Linux system.

Another thing that might interest you - if I haven't bored you with non-essential stories - is the idea that we can mount and unmout filesystems to our GNU/Linux system. This was a cool feature that may confuse many of us. This is why we have to mount the floppy or CD when we add these removable media from our system. In order to mount them - these must be in the text file called fstab. You should read the man pages about fstab -- in order to get a handle on maintaining it. Yes, automount and all of these newer features might do all of this automatically for you -- but it's a file worth learning about.

Now, back to my story about the filesystem. Each file is a text file and is represented by an inode, a number.

Update
I had to look this up for a good defn.

Each inode contains the description of the file: file type, access rights, owners, timestamps, size, pointers to data blocks.

Each directory is a file with a list of inodes. Symbolic links are also important in the filesystem and they are a feature that was added after the VFS was implemented in the kernel.

Therefore, to answer your question about defraging. This short history hopefully gives you enough information so that you understand the filesystem is handling the text files so that there aren't holes on the disk. Remember, deleting a file is simply changing the indexing of it. You don't delete it unless you use a utiltity to delete it. Typically, the space if overwritten when you add new files.

I'd suggest leaving your machines on. It's better on the hardware and the OS can definitely handle it. There is no need to turn off a GNU/Linux system.

Except perhaps to save energy. I know my electric bill went way down when I decided to only have the computer on when I'm using it. Of course, I have an Athlon, so what do I know about energy efficiency. But I'm switching to one of VIA's energy-efficient EPIA designs Real Soon Now(tm).

http://www.mini-itx.com/

[TRN]

Energy is definitely best conserved.

I keep four machines on at all times. These four machines plus any additional ones, the refrig, tv, dvd player, stereo, water heater and lights in the apt cost approximately $26 per month. There is a light swtich that turns on some vanity lights in the bathroom which cost an arm and a leg. The dial on the meter literally turns like a fan ! Therefore, I have a small wattage lamp that sits on the counter and we use it rather than the light switch.

The heat from the machines plus a small floor heater keeps the place warm enough. During the summer it is actually over 100 degrees F in the apt and that causes some problems. Otherwise, all is pretty well.

Good point, though. Not everyone needs to keep their machines on when using Linux as a desktop OS ...

[Davepet]

Thanks for all the info. I skimmed tha articles as well.

I'm stilll in the dark, though about how the filesystem avoids fragmentation. The article said something about write operations reserving (something like) 8 blocks when it begins a write operation, but what happens (for example) if you have a file that requies 30 blocks & you also have just deleted some files leaving 2 8 block "holes" in the middle of the disk (sorry, "hole" just is the way I think unused space surrounded by used space).

Does it know the file is too big to fit in the 8 block "holes" & allocate contiguous space somewhere else?

I know W98 would fill in the holes first & fragment the file.

Also, say your file is a work in progress & keeps getting bigger. How does the FS keep the file from getting fragmented?

I'm just throwing out questions here, you don't need to do the research for me, but if you happen to know the answers... :wink:

Maybe I'm just too used to my weekly defrag in windoze.... what will I do with that extra time? :D

Dave

[muskrat]

Maybe it's because of our MS windows upbringing. I realize how the system files can never need defraging. But due to the nature of disks construction. I can't help but think one would have to run some sort of utility on your home directory at least some time. Unless the OS has different ways of retriving this information and the fragmentation isn't a factor any more.

[TRN]

I've done a fairly poor job in explaining it so .. how about if I direct you to the best place I've found so far ...

http://www.reiserfs.org

You'll see a link http://www.namesys.com/disk_usage.html on space efficiency. Maybe that will help.

[Davepet]

Thanks for the link... lots to digest over there.

Going to take some reading time.

Dave