Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Changes in HDD Sector Usage After 30 Years

CowboyNeal posted more than 8 years ago | from the new-and-improved dept.

360

freitasm writes "A story on Geekzone tells us that IDEMA (Disk Drive, Equipment, and Materials Association) is planning to implement a new standard for HDD sector usage, replacing the old 512-byte sector with a new 4096-byte sector. The association says it will be more efficient. According to the article Windows Vista will ship with this support already."

cancel ×

360 comments

Sorry! There are no comments related to the filter you selected.

4MB (1, Funny)

Anonymous Coward | more than 8 years ago | (#14986235)

Why not a 4MB sector?

Re:4MB (2, Funny)

irimi_00 (962766) | more than 8 years ago | (#14986252)

Why not a 32768 bit sector?

Re:4MB (3, Insightful)

LardBrattish (703549) | more than 8 years ago | (#14986313)

Simple answer - every file would then have a minimum size of 4MB

Re:4MB (2, Insightful)

Anonymous Coward | more than 8 years ago | (#14986337)

It only means that a 4MB block would be the smallest atomic unit you could write on a disk. Writing to parts of it would require to first read it, then modify it, then write it. A lot of FS would implement this by always caching full blocks. But you could still pack many files in a single block. Most FS already work with pretty large (logical) block sizes (16KB ain't uncommon) and will "fragment" them for very small files. Databases often compact records end to end in a block.

But of course, 4MB is fscking large. One problem would be to make them truly atomic. Current drives are supposed to have enough power to be able to complete a 512 bytes writes even if power is lost and stuff likes that.

Because of R-M-W (3, Insightful)

alanmeyer (174278) | more than 8 years ago | (#14986342)

Want to write a single byte? Then read 4MB, modify 1 byte, and write 4MB back to the disk.

Re:Because of R-M-W (2, Informative)

Mortlath (780961) | more than 8 years ago | (#14986431)

Want to write a single byte? Then read 4MB, modify 1 byte, and write 4MB back to the disk.

That's why there is a system (Level 1, 2, and main memory) cache. Write-backs to the physical disk only occur when needed. That doesn't mean that 4MB would be a good sector size; it just means that write-backs are not the issue to consider here.

Re:4MB (1)

Nefarious Wheel (628136) | more than 8 years ago | (#14986370)

Why not a 4MB sector?

Depends on the size of your device and system cache, really. There's a functional difference between the size of a sector and the size of the space allocated for new files or file extensions. My advice? Let the hardware vendors decide what they want for the sector size, doesn't matter all that much. Then make sure any file extension or file initial size allocation is a healthy multiple of that. If you don't use it all, truncate on close. It's small allocations, not small sector sizes, that determine the rate of file fragmentation.

Re:4MB (3, Informative)

Alioth (221270) | more than 8 years ago | (#14986497)

4Kbyte is the size of a page of memory on all modern architectures. Given all modern operating systems use demand page loading of executables, and implement paging (swap space), a sector size that matches the size of a memory page will probably result in better performance.

Re:4MB (1)

Warg! The Orcs!! (957405) | more than 8 years ago | (#14986501)

I'm thinking that the Anonymous Coward was thinking that 4096 bytes = 4Mb and so why not just call it a 4Mb sector instead of a 4096 byte sector.

Of course 4096 bytes is 4Kb not 4Mb.

Ah, error correction. (5, Insightful)

wesley96 (934306) | more than 8 years ago | (#14986239)

Well, CD-ROMs use 2352 bytes per sector, ending up with 2048 actual bytes after error correction. Looking at the size of the HDDs these days a 4096-byte sector seems pretty reasonable.

Re:Ah, error correction. (5, Informative)

Ark42 (522144) | more than 8 years ago | (#14986308)

Hard drives do the same thing - for each 512 bytes of real data, they actually store near 600 bytes onto the disk with information such as ECC and sector remapping for bad sectors. There is also tiny "lead-in" and "lead-out" areas outside each sector which usually contain a simple pattern of bits to let the drive seek to the sector properly.
Unlike CD-ROMs, I don't believe you can actually read the sector meta-data without some sort of drive-manufacturer-specific tricks.

Re:Ah, error correction. (2, Interesting)

bjpirt (251795) | more than 8 years ago | (#14986499)

I wonder if the 4096 bytes are before or after error correction. If it's after, it might make sense because (and I'm sure someone will correct me) isn't 4K a relatively common miimum size in today's filesystems. I know that the default for HFS+ on a mac is.

What's the case for Linux? (1)

szobatudos (642289) | more than 8 years ago | (#14986241)

Is Ingo Molnar working on this?

Re:What's the case for Linux? (1)

A beautiful mind (821714) | more than 8 years ago | (#14986368)

Why would he be working on this? He works with RTS, not with filesystems.

Ask Hans Reiser maybe.

Re:What's the case for Linux? (2, Informative)

Anonymous Coward | more than 8 years ago | (#14986392)

All major Linux file systems (except XFS) already support arbitrary sector sizes up to 4096 bytes, e.g. for s/390 Mainframes that traditionally use 4096 byte sectors on Linux.
The poeple who would need to write support for this are Jeff Garzik (libata) and James Bottomley (scsi). It's not that this would require a terribly complicated patch though.

Re:What's the case for Linux? (5, Funny)

Aggrav8d (683620) | more than 8 years ago | (#14986455)

I know I'm tired because I misread the first name as Inigo and the next thing through my head was

"Hello. My name is Inigo Molnar. You changed the sectors. Prepare to die."

That's nice (0, Troll)

Mancat (831487) | more than 8 years ago | (#14986243)

So... If I write down a little 16-byte message to myself in Notepad containing a name and a phone number, it will take up 4096 bytes. That's good. Thanks. I guess since disks are getting so large, they have to find a way to help me waste the space even faster.

Re:That's nice (1)

Trevahaha (874501) | more than 8 years ago | (#14986256)

Will something like NTFS disk compression actually make it less than one sector, or are you right that every file will take up at least 4096 bytes? Does anyone know if this will do anything to improve retrieval speed?

Re:That's nice (1, Informative)

AuMatar (183847) | more than 8 years ago | (#14986284)

OSes allocate disk space by the sector. If the size of a file is less than 1 sector, the rest is wasted. This is called internal fragmentation. Every file wastes sector_size-file_size%sector_size bytes. On average, thats sector_size/2 bytes per file.

Re:That's nice (2, Insightful)

Eunuchswear (210685) | more than 8 years ago | (#14986472)

Informative, but wrong.

Some file systems can pack multiple tail fragments into one block.

Re:That's nice (4, Informative)

jcr (53032) | more than 8 years ago | (#14986257)

So... If I write down a little 16-byte message to myself in Notepad containing a name and a phone number, it will take up 4096 bytes.

On most systems in use today, it already does.

Blame the file system, not the sector size on the media.

-jcr

Re:That's nice (1, Insightful)

arrrrg (902404) | more than 8 years ago | (#14986458)

Anyway, why do you give a flying fuck? You can get a 250 GB hard drive for less than $100 these days ... at that rate, that 4096 bytes costs you about $100/64000000~= $.0000016. Was that really worth your time bitching? I didn't think so.

No, that's not 'sector' (4, Informative)

wesley96 (934306) | more than 8 years ago | (#14986259)

You're thinking of 'cluster'. This is tied to the file system that is actually used on the disk. Even with the current 512-byte sector, a normal NTFS partition of, say, 200GB, uses 4KB cluster and a single file takes up a minimum of 4KB already.

Re:No, that's not 'sector' (1)

A beautiful mind (821714) | more than 8 years ago | (#14986376)

So, all they doing is pushing this abstraction layer to the hardware, thus getting rid of an unnecessary layer, if I understand it correctly?

Re:No, that's not 'sector' (1, Informative)

Anonymous Coward | more than 8 years ago | (#14986481)

Even with the current 512-byte sector, a normal NTFS partition of, say, 200GB, uses 4KB cluster and a single file takes up a minimum of 4KB already

No. If the file can fit into the MFT record (resident file) then it takes 0 bytes outside of the file's metadata, which is usually 1 KB altogether. The maximum size of such files are usually 700-800 bytes. Though not many other filesystems have this capability.

Re:That's nice (2, Informative)

Beryllium Sphere(tm) (193358) | more than 8 years ago | (#14986268)

NTFS will write something that small into the MFT.

Re:That's nice (1)

entrex (580367) | more than 8 years ago | (#14986280)

You think thats bad, you should see my porn collection.

Re:That's nice (1)

Valdoran (887940) | more than 8 years ago | (#14986475)

You've got porn vids of less than 4KB? Cool!

Re:That's nice (1)

GoingDown (741380) | more than 8 years ago | (#14986283)

It is perhaps already taking it or even more.

Today's filesystems are usually using larger chunks than 512 bytes to save data. But of course it depends of the filesystem you are using.

And judging of your talk about Notepad, you are using Windows. Windows NTFS uses 4k blocks by default on large (> 2GB) disks.

Re:That's nice (3, Informative)

ars (79600) | more than 8 years ago | (#14986290)

Um, it already does take up 4K or more. Unless you have a hard disk smaller then 256MB.

See: http://www.microsoft.com/technet/prodtechnol/winxp pro/reskit/c13621675.mspx [microsoft.com] and scroll down to Table 13-4

If you notice, in most of the useful cases the custer size is 4K. Making the hard disk match this seems like a good idea to me.

And EXT2 also uses a 4K block size.

Also remember it's for large disks, no FS that I know of supports a cluster (or block) size smaller then 4K for large disks.

Re:That's nice (1)

tacocat (527354) | more than 8 years ago | (#14986375)

like /swap?

Re:That's nice (1)

ars (79600) | more than 8 years ago | (#14986434)

What about /swap? Do you mean the swapfile partition in linux? That too uses a 4K page size.

Re:That's nice (0)

Anonymous Coward | more than 8 years ago | (#14986294)

Too bad, it sucks for those of you that keep gazillion 2-line files. On the other hand, you free up a few bits that were previously used to keep track of all those extra sectors.

Re:That's nice (3, Informative)

Foolhardy (664051) | more than 8 years ago | (#14986304)

Actually, if you're using NTFS, the data will be stored directly in the file entry in the MFT, taking zero dedicated clusters or sectors. The maximum size for this to happen is like 800 bytes.

Here [ntfs.com] 's a short description of how NTFS allcates space. On volumes larger than 2GB, the cluster size (the granularity the FS uses to allocate space) was 4k already unless you specified something else when formatting the drive. Also, Windows NT has supported disk sector sizes larger than 512 bytes for a long time; it's just that anything else has been rare.

Re:That's nice (1)

carterhawk001 (681941) | more than 8 years ago | (#14986363)

Who the hell uses notepad to make notes?

Re:That's nice (0)

Anonymous Coward | more than 8 years ago | (#14986365)

Hello?!?!

NOTEpad? What do you think it's for?

Re:That's nice (1)

Kraeloc (869412) | more than 8 years ago | (#14986424)

What would you use, Word? For simple text-only notes, less is better. If I need to jot down a phone number quickly before I forget, I'm just gonna hit notepad (or pico on unix) and be typing it in seconds; I don't want to wait several minutes for Word to get it's bloated ass in gear.

How do you know how your data is actually stored ? (4, Insightful)

Horus1664 (692411) | more than 8 years ago | (#14986394)

Modern DASD architecture is almost completely hidden from the user. In the (good?) old days system software needed to interface closely with the DASD and needed to understand the hardware architecture to gain maximum performance from the devices (I know because I work on such systems within an IBM mainframe environment on airline systems which require extremely high speed data access).

Nowadays the disk 'address' of where the data actually resides is still couched in terms that appear to refer to the hardware itself but in 'serious' DASD subsystems (e.g. the IBM DS8000 enterprise storage systems [ibm.com] )the actual way in which the hardware handles the data is masked from the operating system. Data for the same file is spread across many physical devices and some version of RAID is used for integrity.

The 4096 value for data 'chunks' has to do with the most efficient amount of data that can be transmitted down a DASD channel (between a host and storage in large systems or the bus in self-contained systems)

The idea of a 'file address' would cease to exist and it would be replaced by a generic 'data address' if it weren't for the in-built assumptions about data retrieval within all current Operating Systems.

Quick Explain How! (0)

realcoolguy425 (587426) | more than 8 years ago | (#14986248)

Someone explain to me how this works! why is this better? oh yeah you can only use sci-fi/startrecky terminology or else it doesn't count.

Re:Quick Explain How! (5, Interesting)

AngelofDeath-02 (550129) | more than 8 years ago | (#14986270)

Best analogy is a gym locker room
You have say, 10 lockers up and 20 lockers accross
You can only put one thing in a locker, so you cant put your gym shorts in the same one as your shoes. But if you have lots of socks, you can pile them in, and take up two or three if neccessary.

Space is wasted if you have a really big locker, but it's only holding a sock.

Now, you've got to record where all of this stuff is, or you will take forever to find that sock. So you set asside a locker to hold the clipboard with designations.

Now to bring this back into real life. There are a _lot_ of sectors on a disk. So keeping track of all of them starts requiring a substantial amount of resources. I imagine they are finding it easier to justify wasting space for small files in order to make it easier to keep track of them. Average file sizes are also going up, so it's not as big of a problem as it used to be either. It's all relative...

Re:Quick Explain How! (0)

Anonymous Coward | more than 8 years ago | (#14986296)

Who the hell has more than one gym locker and can't put their shorts with their shoes? LOL :-) Other than that, good explanation.

Re:Quick Explain How! (5, Funny)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986299)

I'm willing to sell this account for the right price.

Re:Quick Explain How! (1)

Bob54321 (911744) | more than 8 years ago | (#14986340)

Funniest comment I have read in ages...

Re:Quick Explain How! (5, Funny)

realcoolguy425 (587426) | more than 8 years ago | (#14986307)

I'm sorry, Your response has to be in some form of star-trek (or sci-fi) I would have accepted this however...

  Best analogy is Spock's gym locker room

Spock has say, 10 space lockers up and 20 space lockers accross

Spock can only put one thing in a locker, so Spock cant put his gym shorts in the same one as your shoes. But since Spock has lots of socks, He can pile them in, and take up two or three if neccessary.

Space is wasted if Spock uses a really big locker, but it's only holding a sock.

Now, you've got to record where all of this stuff is, or you will take forever to find that sock. (I guess the tricorders are broken) So Spock sets aside a locker to hold the clipboard with designations.

Now to bring this back into real life. There are a _lot_ of sectors on a disk. So keeping track of all of them starts requiring a substantial amount of resources. I imagine they are finding it easier to justify wasting space for small files in order to make it easier to keep track of them. Average file sizes are also going up, so it's not as big of a problem as it used to be either. It's all relative...

Re:Quick Explain How! (2, Funny)

Nefarious Wheel (628136) | more than 8 years ago | (#14986383)

I think it all started with the first Vax 780, or possibly the first IBM 370 channel controller. Those old machines booted with a 7" floppy that had a capacity of 0.5k. Yep, 512 bytes. Early bootstraps could store the entire contents on to a hard disk with very few instructions if the sector size matched.

Man that takes me back. Where's my toupee....

Wrong attribution on your sig (1)

IvyKing (732111) | more than 8 years ago | (#14986446)

It was Bill Stout, not Ed Heineman who coined the phrase "Simplicate and add lightness".

Nevertheless, Heineman was still one heck of an airplane designer.

As far as the 8" floppy, ISTR that they were intended to replace punched cards, 77 tracks with 26 sectors (hard coded) came out to be pretty close to a box of 2000 hollerith cards (80 columns with 12 bits per column). 8" drives were available before the end of 1975, and the VAX came out in 1977(?). One of the uses for the flopies was loading the microprogram store on the VAX and IBM machines of the same era.

Re:Quick Explain How! (1)

MichaelSmith (789609) | more than 8 years ago | (#14986460)

I think it all started with the first Vax 780, or possibly the first IBM 370 channel controller. Those old machines booted with a 7" floppy that had a capacity of 0.5k. Yep, 512 bytes. Early bootstraps could store the entire contents on to a hard disk with very few instructions if the sector size matched.

The 11/750 loaded its microcode from a little magnetic tape. It used to take (seemingly) ages to get going. I used to boot PDP 11/84's and 83's from TK50 tape. This was in traffic signal cabins out in the middle of nowhere, usually at 0200 or so. I could go for a walk and listen for the console printer to start chattering as the system came up.

I think the pdp's are the reason I now use NetBSD. Not sure why. Just a similar feel.

Cluster size? (2, Interesting)

dokebi (624663) | more than 8 years ago | (#14986251)

I thought cluster sizes were already 4KB for efficiency, and LBA for larger drive sizes. So how does changing the sector size change things? (Especially when we don't access drives by sector/cylinder anymore?)

Re:Cluster size? (5, Informative)

scdeimos (632778) | more than 8 years ago | (#14986366)

I thought cluster sizes were already 4KB for efficiency, and LBA for larger drive sizes.
Cluster sizes are variable on most file systems. On our NTFS web servers we tend to have 1k clusters because it's more efficient to do it that way with lots of small files, but the default NTFS cluster size is 4k. LBA is just a different addressing scheme at the media level to make a volume appear to be a flat array of sectors (as opposed to the old CHS or Cylinder Head Sector scheme).

problems for older hardware??? (1)

3seas (184403) | more than 8 years ago | (#14986254)

so long as this new format is transparent, built internally in the drives and doesn't effect older hardware or software, there shouldn't be a problem. It also should not contain any DRM junk.

All to often an advantage in speed improvements and such are more than countered by adding overhead junk.

now maybe I should RTFA...

It can't be. (1)

Myria (562655) | more than 8 years ago | (#14986339)

It can't be, at least not efficiently. Like flash devices, it's impossible to write less than a sector at a time.

If this were transparently implemented by the hardware, the OS would frequently try to write a single 512 byte sector. In order for this to work, the hard drive controller would have to read the existing sector then write it back with the 512 bytes changed. This is a big waste, as a read then a write costs at least a full platter rotation (1/7200 second). Do this hundreds or thousands of times per second, and you have a nice slow hard drive.

Melissa

Hrm, that kind of makes sense... (2, Insightful)

I kan Spl (614759) | more than 8 years ago | (#14986265)

Most "normal use" filesystems nowadays (FAT32, Ext3, HFS, Reiser) all use 4K blocks by default. That means that the smallest amount of data that you can change at a time is 4k, so every time you change a block, the HDD has to do 8 writes or reads. That would leave the drive preforming 8x the number of commands that it would need to.

As filesystems are slowly moving towards larger block sizes, now that the "wasted" space on drives due to unused space at the ends of blocks are not as noticable, moving up the size on the underlying hardware also makes sense. I don't think that this can make things too much faster, but it would allow SATA drives (and SCSI also) to quesu more commands in their internal buffers, as they will onyl be recieving one command per read/write that the filesystem does, instead of 8.

Re:Hrm, that kind of makes sense... (3, Informative)

Anonymous Coward | more than 8 years ago | (#14986327)

You're talking bullshit. In SCSI/SATA you can read/write big chunks of data (even 1MB) in just one command. Just read the standards.

Vista (1)

kvant (939634) | more than 8 years ago | (#14986273)

"According to the article Windows Vista will ship with this support already."

HAHAHAHAHAHA

Re:Vista (2, Funny)

Anonymous Coward | more than 8 years ago | (#14986288)

Also, Solitaire will be replaced by Duke Nukem Forever on every shipped copy of Vista. And if you're one of the first 100 in line at any Best Buy when you pick up Vista, you will also get a free Phantom game console.

Good for small devices (4, Interesting)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986276)

Small devices like cellphones typically save files of several kilobytes, whether they be the phonebook database or something like camera images. Whether the data is saved in a couple large sectors or 8 times that many small sectors isn't really an issue. Either way will work fine, as far as the data is concerned. The biggest problem is the amount of battery power used to transfer those files. If you have to re-issue a read or write command (well, the filesystem would do this) for each 512-byte block, that means that you will spend 8 times more energy (give or take a bit) to read or write the same 4k block of data.

Also, squaring away each sector after processing is a round trip back to the filesystem which can be eliminated by reading a larger sector size in the first place.

Some semi-ATA disks already force a minimum 4096-byte sector size. It's not necessarily the best way to get the most usage out of your disks, but it is one way of speeding up the disk just a little bit more to reduce power consumption.

Re:Good for small devices (1)

scdeimos (632778) | more than 8 years ago | (#14986382)

I guess that's why they call you bad analogy guy. :) R/W filesystems tend to abstract media into clusters, which are groups of sectors, so that they can take advantage of multi-sector read/write commands (which have been around since MFM hard disks with CHS addressing schemes, by the way) to get more than one sector's worth of data on/off the hard disk in a single command.

Re:Good for small devices (1)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986390)

The single command to the taskfile, sure. But the actual sector access must still be handled on a sector-by-sector basis. So to read 4k of data, you go from:

1 taskfile write + 8 sector reads

to

1 taskfile write + 1 sector read.

Re:Good for small devices (0)

Anonymous Coward | more than 8 years ago | (#14986427)

interleaving will allow all the cluster's sectors to be read/written in 1 shot.

Re:Good for small devices (1)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986440)

At what level of abstraction? Is this truly a continuous 8-sector access?

Things don't work that way. (1)

woolio (927141) | more than 8 years ago | (#14986389)

If you have to re-issue a read or write command (well, the filesystem would do this) for each 512-byte block, that means that you will spend 8 times more energy (give or take a bit) to read or write the same 4k block of data.

Well sorry, but that's the way it is.

Hard drives generally have the ability to read/write multiple sectors with a single command. (Go read the ATA standards). And DMA is usually used [ program I/O just plain sucks].

I don't see how changing the sector size is going to save power... Either way they have to increase the size of the buffers for the read/write multiple operations. So these could just be increased while keeping 512-byte sectors and the same benefit would result.

Re:Things don't work that way. (1)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986402)

Yes, as I mentioned in another reply, the command itself is handled as a single operation, then the data transfer is handled as another operation.

At the DMA level, the HDD hardware will still have to either access the sectors as 8 separate sectors or as 1 large sector, whether this is invisible to the software and CPU or not. The microseconds eliminated from those sector seeks add up.

There really isn't much data... (1)

rice_burners_suck (243660) | more than 8 years ago | (#14986286)

...to back up the claim that this is more efficient, though it intuitively "feels" like it should be faster, but not necessarily more efficient in terms of space. Suppose a file with a size of 1 byte takes up 512 bytes of space on the disk. With this larger sector size, that file would take 4k. I don't see why this isn't an option that can be set through drive initialization parameters, and why you can't choose any size for the sectors, depending on whatever tweaking you can do to figure out what's best for your application.

Re:There really isn't much data... (0)

Anonymous Coward | more than 8 years ago | (#14986316)

As others have noted, common file systems all use block size of 4K or bigger. Matching the disc sector size will simply expedite the IO.

In Vista already? (5, Funny)

sinnerman (829959) | more than 8 years ago | (#14986289)

Well of course Vista will ship with this supported already. Just like WinFS...er..

Finally!! (1)

5pp000 (873881) | more than 8 years ago | (#14986295)

I've wondered about this for years. In 1981 I had a CP/M system -- CP/M!!! -- that could happily handle 128-, 256-, 512-, or 1024-byte sectors on its 8-inch floppy drives. (I wrote the reblocking code in the BIOS myself.) There was no question that larger sizes were better for both capacity and performance, if perhaps slightly more prone to unrecoverable errors (not that I really noticed, though). But somehow the idea that drivers could support multiple sector sizes was forgotten.

To answer someone's question, if all disk transfers are multiples of 4K anyway, you're better off using that as the hardware sector size because there's less overhead -- less spatial overhead on the disk because you have fewer sector headers and intersector gaps, and less temporal overhead in the I/O protocol because you're only sending one transfer command instead of eight.

Re:Finally!! (1)

IvyKing (732111) | more than 8 years ago | (#14986421)

At the same time, 86-DOS (the original name for what is now MS-DOS) could also handle varying sized sectors - generally used 128 byte sectors for SSSD 8" disks and 1024 byte sectors for DSDD 8" disks. Tim Paterson recommened using a cluster size that was the square root of the disk capacity in bytes, so the SSSD disks used four sectors per cluster (512 bytes) and the DSDD used 1 sector per cluster (1024 bytes).

What was done with CP/M and 86-DOS, but not PC/MS-DOS, was distributing source code for the BIOS (called io.sys under 86-DOS). With that source code, it was relatively easy to support whatever sector size you wanted.

Re:Finally!! (1)

MichaelSmith (789609) | more than 8 years ago | (#14986436)

I had a CP/M system as well and IIRC it stored contiguous files on every third or sixth sector because the CPU could not always keep up with the disk.

I remember writing some slow basic code around the same time on an apple ][ which caused the floppy drive to stop and wait for the CPU.

Also there was something about batch files in CP/M. I think the files were structured so that the shell could go back to the disk for the next step in the script. Those were the days when memory was really scarce.

Wasted space? (1)

solarbob (959948) | more than 8 years ago | (#14986302)

Taking the cost per GB currently and that most "small files" now are 10K+ does the overhead this cause by "wasted space" really need to matter. It still takes a hell of a lot of documents to fill up even a 250GB disk and as you can now get these disk for next to nothing I'm happy to get the extra performance

Apple in the forground again (1, Interesting)

danmcn (960457) | more than 8 years ago | (#14986310)

Isn't this what apple tried to do 5+ years ago with HFS+

Re: Apple in the forground again (4, Informative)

n.wegner (613340) | more than 8 years ago | (#14986378)

You could have added MS with FAT32 and NTFS. The problem is we're not talking about filesystem cluster sizes, which are software-configurable, but the disks' actual sector size, which is hardware that HFS+ has no effect on.

Re: Apple in the forground again (0)

Anonymous Coward | more than 8 years ago | (#14986401)

No, it isn't, you don't know what you're talking about.

Hmm, apple fanboy who doesn't know what he's talking about ... I'm sure that's a coincidence.

Re: Apple in the forground again (1, Informative)

Anonymous Coward | more than 8 years ago | (#14986417)

On the filesystem layer everbody moved the block size up more than 15 years ago. Even Microsoft's POS FAT32 has a 4096 byte cluster. This time Microsoft was only 10 years behind Unix. We're talking hardware layer here. If you can address 2^48 sectors (or whatever is the limit these days) and your sector size is 8x you can have much bigger disks. Think in term of SAN arrays and you'll see that such a need might be here right now on big iron already.

Re: Apple in the forground again (1)

ocelotbob (173602) | more than 8 years ago | (#14986443)

Apple is hardly at the forefront in FS research here. Pretty much every FS on the planet has had variable cluster sizes for many, many years; I remember back in the DOS days being able to format disks with cluster sizes from 512 bytes to 32 or even 64K. This proposal is talking about the physical sector sizes and getting them equalized with what the default cluster size for most operating systems.

Why just one standard? (2, Interesting)

dltaylor (7510) | more than 8 years ago | (#14986312)

Competent file system handlers can use disk blocks larger or smaller than the file system block size, but there are some benefits to using the same number for both. Although it may provide more data-per-drive to use larger blocks and you can index larger drives with 32-bit numbers, the drive has to use better (larger and more complex) CRCs to ensure sector data integrity integrity, the granularity of replacement blocks may end up wasting more space simply to provide an adequate count of replacements, and there are still some disk space management tools that insist on working in terms of "cylinders", regardless of the fact that the disk drives have had variable density zones for ages. The range from 4K (common disk block size) to 16K works as a decent compromise.

"Back in the day" running System V on SMD drives, where you could use almost any block size from 128 Bytes to 32K (the CRCs were weak after that) and control the cylinder-to-cylinder offset of block 0 from the index, I spent a few days trying different tuning parameters and found that, due to the 4K size of the CPU pages, and of the file blocks and swap it really did give a significant improvement in performance. I tried 8K and 16K, because the file system handler could be convinced to break them up, but didn't get any better performance, so used 4k for the spares granularity.

Perhaps I should take one of my late-model SCSI drives, which support low-level reformatting, and try the tests again. 16KByte file system blocks on 16KByte sectors might really be a win now. Have to do some research to see what I can do with CPU page sizes, too.

It's all about Format Efficiency (5, Informative)

alanmeyer (174278) | more than 8 years ago | (#14986320)

HDD manufacturers are looking to increase the amount of data stored on each platter. With larger sector sizes, the HDD vendor can use more efficient codes. This means better format efficieny and more bytes to the end user. The primary argument being that many OSes already use 4K clusters.

During the transition from 512-byte to 1K, and ultimately 4K sectors, HDDs will be able to emulate 512-byte modes to the host (i.e. making a 1K or 4K native drive 'look' like a standard 512-byte drive). If the OS is using 4K clusters, this will come with no performance decrease. For any application performing random single-block writes, the HDD will suffer 1 rev per write (for a read-modify-write operation), but that's really only a condition that would be found during a test.

Seems good to me. (3, Informative)

mathew7 (863867) | more than 8 years ago | (#14986331)

Almost all filesystems I know of use at least 4Kb clusters. NTFS does come with 512 byte on smaller partitions.
LBA accesses on sector boundaries, so for larger HDD's, you need more bits (currently 28-bit LBA, which some older bioses support, means a maximum of 128GB- 2^28*512=2^28*2^9=2^37) Since 512-bytes were used for 30 years, I think it is easy to assume it will not last for 10 more years (getting to LBA32 limit). So why not shave off 3 bits and also make it an even number of bits (12 against 9).
Also there is something called "multible block access" where you make only one request for up to 16 (on most HDD's) sectors. For 512-byte sectors you have 8K, but for 4K sectors that means 64K. Great for large files (IO overdead and stuff).
On the application side this sould not affect anyone using 64-bit sizes (since only the OS would know of sector sizes), as for 32-bit sizes it already is a problem (4G limit).
So this sould not be a problem because on a large partition you will not have too much wasted space (i have around 40MB wasted space on my OS drive for 5520MB of files, and I would even accept 200MB)

Re:Seems good to me. (1)

farnz (625056) | more than 8 years ago | (#14986381)

But we've solved the LBA28 limit, by switching to LBA48 (48-bit sector addresses, maximum of 128PiB (131,072TiB) with 512 byte sectors, going up to 512PiB (524,288TiB) with 4k sectors). Multiple block access allows a drive in PIO mode to return up to 16 contiguous sectors; in DMA modes, the drive is allowed to set a limit of up to 128MiB of contiguous sectors. Thus, a larger sector size helps with MSDOS (PIO mode only), but not Windows 2000 or above, or Linux 2.0 or above (or probably Linux 1.2, but I've not investigated that).

The gain is that for large disks, the filesystems already use 4K blocks; having the sector size == 4K just means that the sector size and block size match, so that sector and block numbers match, and the drive can use more efficient ECC to get more usable space.

Finally! (1, Redundant)

Zo0ok (209803) | more than 8 years ago | (#14986336)

Finally! This is what I really wanted for years. Cant believe this innovation has not been materialized earlier. This is great for perfomance, TCO, iPods, everything. I cant wait to get my hands on one of those new goodies!

Block size issue (1)

danmcn (960457) | more than 8 years ago | (#14986355)

Ok lets all get real for a minute if block size is not consistent how the hell will disk defragment and optimizer soft wear work. The Whole trick to those soft wear is moving the blocks around. Now if you want to loose the ability to fix and speed up your hard drive go for variable block size. But those of us who run multiple large drives want a small fixed block (to save space on things like fonts and e mail) otherwise our drives will quickly become unmanageable. Dan

Re:Block size issue (1)

ocelotbob (173602) | more than 8 years ago | (#14986482)

Most drives nowadays are formatted at 4k already because it's faster than a 512 byte format. With the size of drives, the lost space caused by large clusters is not much of an issue these days. Finally, people concerned about disk space could use a file with a tail gathering scheme, like reiserfs, in order to gain more space.

Re:Block size issue (1)

ocelotbob (173602) | more than 8 years ago | (#14986487)

Oh, and with inconsistent blocksizes, they're already inconsistent. Some people use 512 byte blocks, some use 32k blocks. The defragger will continue working just as it always has; the OS will take care of things and not create a partition smaller than the sector size.

Boot sector virii (5, Funny)

TrickiDicki (559754) | more than 8 years ago | (#14986358)

That's a bonus for all those boot-sector virus writers - 8 times more space to do their dirty deeds...

You're all complaining about tiny files... (2, Insightful)

NalosLayor (958307) | more than 8 years ago | (#14986364)

But really...think about this: if each sector has overhead, then any file over 512 bytes will have less overhead, and you'll effectively get more space in most cases. What percentage YOUR files are less than 4k?

Re:You're all complaining about tiny files... (1)

NalosLayor (958307) | more than 8 years ago | (#14986372)

Sorry to reply to my own post, but I'd like to add: TFA makes it sound like the engineers on the comittee made this analysis and concluded that 4k sectors WERE in most cases more efficient.

Re:You're all complaining about tiny files... (1)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986374)

On average, you'll have about 2048 bytes of wasted space per sector vs. 256 bytes per sector for the 512-byte sectors. If the filesystem does such things, you may already have 2048 bytes of wasted space per sector anyway.

The problem isn't small files, but files that spillover into another sector and wasting space.

Re:You're all complaining about tiny files... (1)

NalosLayor (958307) | more than 8 years ago | (#14986380)

Don't you think that the large # of sectors in the middle of a file outweighs the one at the end?

Re:You're all complaining about tiny files... (1)

BadAnalogyGuy (945258) | more than 8 years ago | (#14986411)

It depends on usage, of course. As an extreme example (and extremely simplified), if I have 1,000,000 4097 byte files, on a 512-byte sector disk there will be 511*1,000,000 bytes of wasted space With a 4096-byte sector size, there will be 4095*1,000,000 bytes wasted.

On average, the wasted space will be 256 and 2048 bytes respectively. And I'm only talking about space which is in use but not usable.

file size (0)

danmcn (960457) | more than 8 years ago | (#14986384)

A 5K font file now uses 8k put 500 fonts on your machine as a designer you see the problem. Email !.2k average file 4k How much mail do you keep. It keeps adding up and you loose massave amounts of space

Re:file size (2, Insightful)

Anonymous Coward | more than 8 years ago | (#14986407)

5 K becomes 8 K.. times 500 ... is a whopping 1.5 MEGAbytes wasted. I mean, that is more than fits on a floppy. What a waste.

Actually a 200 GB drive can still store 25 million files. How many fonts do you have?

FWIW the advantage is in the error correction. For a 1 bit secotro size, you'd need 3 bits to store it with error correction. As the block becomes larger, the error correction becomes more powerful. That is where the advantage is.

Of course data can still be stored byte-wise on the disk - it is only that a small update will require a read-modify-write transaction.

Re:file size (1)

Rob Simpson (533360) | more than 8 years ago | (#14986507)

Uh, don't most email programs store mail as a single large database file, rather than zillions of individual files?

Re:file size (1)

Alioth (221270) | more than 8 years ago | (#14986512)

All that is still trivial on even the smallest disks supplied today. In any case, your OS is already using 4K blocks so you're already doing that anyway regardless of the underlying sector size.

It's so that ECC can handle bigger bad spots (3, Interesting)

Animats (122034) | more than 8 years ago | (#14986391)

The real reason for this is that as densities go up, the number of bits affected by a bad spot goes up. So it's desirable to error correct over longer bit strings. The issue is not the size of the file allocation unit; that's up to the file system software. It's the size of the block for error correction purposes. See Reed-Solomon error correction. [wikipedia.org]

Re:It's so that ECC can handle bigger bad spots (1)

danmcn (960457) | more than 8 years ago | (#14986397)

error corection will step in to the next available block reread the story

Redmond thinks they're so smart... (3, Funny)

filterchild (834960) | more than 8 years ago | (#14986409)

Windows Vista will ship with this support already.

Oh YEAH? Well Linux has had support for it for eleventeen years, and the Linux approach is more streamlined anyway!

Of course Vista will support it... (0)

Anonymous Coward | more than 8 years ago | (#14986414)

It will also come bundled with a universal translator and cold fusion power source (both to be developed circa 2200 AD)

30 years and now it's bumped up only 8x? (0, Redundant)

WoTG (610710) | more than 8 years ago | (#14986433)

Bumping this up makes sense; however, I wonder if 4KB (KiB?) is a bit low. I know that hard drive sizes have gone up way more than 8x. Let's see, I started with a 20MB drive in an XT. My desktop downstairs has a 200GB drive. So, that's about 10,000 times bigger. Intuitively, a 8x bump in sector size seems a bit small. Wouldn't something like 16KB be useful for a few extra years?

30 years doing what? (1)

glas_gow (961896) | more than 8 years ago | (#14986467)

So after thirty years these guys have come up with the idea of consolidating disk density through an 8x decrease in sector resolution. By now I'd rather hoped the magnetic HD had been replaced. I seem to be repairing and replacing these things a lot lately. I doubt this breakthrough will alter that.

I have my eyes peeled for a bio-drive, something noxious smelling that you feed with potato rinds which stores your data directly in its DNA. What d'you reckon? Another thirty years.

is any body reading the mail (1)

danmcn (960457) | more than 8 years ago | (#14986483)

You all miss the point. To answer coward over 1100 plus 1300 active emails mostly text. Under 6k Unlike you I use HFS+. When I copied my font and email over to HFS it grew by over 2gigs, I still don't get the error thing error correction should not be affected by block allocation size. as it just links to the next block. Then we still have the defrag and optimize problem.
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?