Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

HDD Manufacturers Moving To 4096-Byte Sectors

CmdrTaco posted more than 4 years ago | from the zomg-the-world-will-collapse dept.

Data Storage 442

Luminous Coward writes "As previously discussed on Slashdot, according to AnandTech and The Tech Report, hard disk drive manufacturers are now ready to bump the size of the disk sector from 512 to 4096 bytes, in order to minimize storage lost to ECC and sync. This may not be a smooth transition, because some OSes do not align partitions on 4K boundaries."

Sorry! There are no comments related to the filter you selected.

Factors of 10 (5, Funny)

suso (153703) | more than 4 years ago | (#30571456)

Why not just move it to 1000 byte sectors, then we could minimize the space lost to advertising.

(Note to accuracy nazis, this is meant to be funny)

Re:Factors of 10 (-1, Troll)

Anonymous Coward | more than 4 years ago | (#30571462)

because 2^10 = 1024 and 10^3 = 1000. there's a difference

Re:Factors of 10 (1, Offtopic)

Iyonesco (1482555) | more than 4 years ago | (#30571518)

If they want to use base10 the first thing they should do is respecify a byte to be 10bits.

I somehow think their enthusiasm for base10 would diminish when it would case a a 25% drop in stated capacity.

Re:Factors of 10 (4, Funny)

rolfwind (528248) | more than 4 years ago | (#30572046)

If they want to use base10 the first thing they should do is respecify a byte to be 10bits.

How about leaving the word byte alone and using another, distinct group of letter to do the job? Respecifying only confuses the issue, even those who know, because you're still be working with two different definitions in the same field for a long time.

Re:Factors of 10 (1, Funny)

Anonymous Coward | more than 4 years ago | (#30572316)

What, like bite?

Re:Factors of 10 (3, Informative)

TheRaven64 (641858) | more than 4 years ago | (#30572086)

A byte can be 10 bits; it's an architecture-specific quantity. An octet is always 8 bits.

Re:Factors of 10 (0)

houghi (78078) | more than 4 years ago | (#30572394)

Stupid bits. Oh and there are only 11 types of people. Those who can read binary and those who can't.

(Yes, I did read and re-read the above and it is correct from my point of view.)

Re:Factors of 10 (1)

Aqualung812 (959532) | more than 4 years ago | (#30572596)

I'll bite. The third type is...? People that can read binary but don't understand why you say there are three types and list only two?

Re:Factors of 10 (1)

ivan_w (1115485) | more than 4 years ago | (#30572694)

Ok.. Care to explain how you came to the conclusion it was 11 and not 10 types of people ?

--Ivan

Re:Factors of 10 (0)

Anonymous Coward | more than 4 years ago | (#30572542)

Historically, yes, but nowadays "byte" is always used to refer to an octet in a system with an 8-bit, 16-bit, 32-bit, or 64-bit word (and so likely all the other 2^(3+n)-bit word systems to come down the pipe in the future).

Re:Factors of 10 (4, Funny)

drainbramage (588291) | more than 4 years ago | (#30572158)

Mine goes to 11.

So only XP is out of luck? (5, Informative)

7o9 (608315) | more than 4 years ago | (#30571486)

According to the Anandtech article, only the pretty much end-of-life Windows XP is out of luck. Linux, OS X and modern Windows versions all work ... Non news?

Re:So only XP is out of luck? (5, Insightful)

gbjbaanb (229885) | more than 4 years ago | (#30571538)

whoooooo. WinXP is end-of-life? You'd best tell that to all the millions of users (including big businesses) out there.

What that's you say? Upgrade to Windows 7 and use its perfectly infallible XP mode?

Ah, I understand now. Hi Bill, how's Steve getting on, still a bit sweaty and concerned he's not selling enough?

Re:So only XP is out of luck? (2, Interesting)

iamhassi (659463) | more than 4 years ago | (#30571760)

"WinXP is end-of-life? You'd best tell that to all the millions of users (including big businesses) out there."

Couldn't agree more. Hopefully I don't have to rehash how horrible Vista was [pcworld.com] , and Windows 7 came out a few months ago so it's a bit early to proclaim XP is dead when it's hopeful replacement just showed up.

I think 4096-byte sectors are Very Bad News. I have no experience with these drives but XP doesn't like them [anandtech.com] which is reason enough for me to avoid them. I hope hard drive manufactures come out with a standard naming scheme for these new drives so they're easy to identify online, like IDE, SATA, PATA, etc. Maybe AFD for Advanced Format Drive?

Re:So only XP is out of luck? (3, Informative)

iamhassi (659463) | more than 4 years ago | (#30571840)

ah this was what I was looking for: Drobo, XP Users: Beware of 4K “Advanced Format” Drives! [fosketts.net]

Article states that not only will XP have problems but so will many other devices like media centers, USB drives, game consoles, and anything else that uses a hard drive. USB drives will be the worse though since 4k drives formatted for XP won't work with Windows 7 and vise versa. Honestly I think this is too soon, put it off another 10 years, by then we'll have OS's that would have supported 4k for 10+ yrs already and all devices should be compatible by then.

Re:So only XP is out of luck? (4, Insightful)

lorenlal (164133) | more than 4 years ago | (#30571928)

Eventually, you have to put a line in the sand. If you push off the deadline, manufacturers will still take their time, and they'll be in the same place 9 years and 11 months from now.

Example: IPv6.

Re:So only XP is out of luck? (0)

Anonymous Coward | more than 4 years ago | (#30571996)

Except that pretty much every OS in use now has IPv6 support. The problem isn't the installed base it's the fact that so long as NAT holds out there's no pressure to switch to it.

Re:So only XP is out of luck? (4, Insightful)

drinkypoo (153816) | more than 4 years ago | (#30572150)

Except that pretty much every OS in use now has IPv6 support.

Except that name resolution is broken for IPv6 on Windows XP, which is the operating system not supporting 4k sectors that people are complaining about... so IPv6 was a super shitty example for you to try to defend.

Re:So only XP is out of luck? (1)

Surt (22457) | more than 4 years ago | (#30572416)

Consumer OS, and not XP, which is in widespread use.
(why is consumer OS important to clarify? Because all that really matters for ipv6 adoption is the router OSs.)

Re:So only XP is out of luck? (1)

BenoitRen (998927) | more than 4 years ago | (#30572614)

Comparing this issue to IPv6 is not fair, because we've known about its existence for longer, and there is more support for it out there.

Not to mention that there is no real need to switch to 4096-byte sectors. Sure, it's nice, but not necessary to keep going.

Then again, there are people who doubt the need for IPv6 as well, saying it's a solution in search of a problem.

Re:So only XP is out of luck? (4, Informative)

kill-1 (36256) | more than 4 years ago | (#30571858)

The new hard drives will have a compatibility mode. It will be slower though because it has to read-modify-write behind the scene.

Re:So only XP is out of luck? (1)

pslytely psycho (1699190) | more than 4 years ago | (#30571884)

How about FUBAR?

Re:So only XP is out of luck? (3, Insightful)

alen (225700) | more than 4 years ago | (#30571960)

MS has a clear support policy. Maybe you like Apple's 3 year support policy better than Microsoft's 10 year 7/3 policy?

Re:So only XP is out of luck? (1, Insightful)

LOLLinux (1682094) | more than 4 years ago | (#30572164)

Or the Linux policy of barely a year.

Re:So only XP is out of luck? (2, Insightful)

Sir_Lewk (967686) | more than 4 years ago | (#30572648)

That's ok, the linux policy of free upgrades more than makes up for that for me.

But hey, you're a known troll, logic doesn't have much to do with this does it?

Re:So only XP is out of luck? (1)

alen (225700) | more than 4 years ago | (#30571974)

and it's not like corporations will buy all the new 2TB drives to use as a OS drive in their ancient XP workstations? they will just buy a new PC with Windows 7 installed or use a 7 corporate image

Re:So only XP is out of luck? (1)

Paradigm_Complex (968558) | more than 4 years ago | (#30571712)

This is news, but not because of the potential problems that could arise. It's interesting from a technological standpoint. Why/how would changing the sector size effect performance? What are the downsides - why wasn't it done before? Why is it now chosen at 4k, why not something even larger? Those questions are what make it news (for nerds). It doesn't have to break something to be newsworthy.

Re:So only XP is out of luck? (4, Informative)

AlecC (512609) | more than 4 years ago | (#30571848)

Why wasn't it done before? Sheer inertia. 512 bytes has been the HDD sector size since time immemorial. Some HDDs in the past could be re-sectored to different sizes, and sometimes were. I did it on one generation of disks to optimise storage for a particular reasons, but it didn't work reliably on the next generation of disks, so I dropped it. Some disks had a sector of 1080 bits, I think to handle the 33rd bit on IBM System/38.

What is the advantage? Every sector has a preamble, a sync mark, a header, the payload data, ECC, and postamble. These can amount to tens of bytes, especially as you have stronger ECC for weaker signals. By having fewer sector, you recover this space from most of the sectors. This could easily add 10% to the capacity of a drive. And, as posted elsewhere, most OSes do 4K transfers most of the time.

Re:So only XP is out of luck? (1)

Jeffrey_Walsh VA (1335967) | more than 4 years ago | (#30572670)

Isn't the purpose of some of these overhead items error correction? Will this lead to more unrecoverable read errors?

Re:So only XP is out of luck? (0)

Anonymous Coward | more than 4 years ago | (#30571914)

Why not something even larger? I guess it goes to average file size. Hard drives can only read and write whole sectors. If the file is smaller than a sector, this means extra data is read; and when writing, the entire sector must be read, the portion the file occupies is updated from memory (the hdd's cache), then the entire sector is written out.

Re:So only XP is out of luck? (1)

rickb928 (945187) | more than 4 years ago | (#30571830)

Really? Go back to your fvwm desktop and stop the rumor-mongering. Ok?

Re:So only XP is out of luck? (1)

petermgreen (876956) | more than 4 years ago | (#30571834)

According to the Anandtech article, only the pretty much end-of-life Windows XP
I wouldn't call XP pretty much end-of-life just yet, you can still purchase it with new systems and it's still supported until april 8 2014 (that's after desktop support expires for the NEXT release of ubuntu LTS) and I haven't seen much use of either vista or win7 in buisness/academia yet.

This isn't that bad though, the logical sectors will still be 512 byte so it's just a matter of getting the partitions aligned right and wd will apparently be supplying a tool for doing this :).

Windows XP end-of-life? (1)

Jim Efaw (3484) | more than 4 years ago | (#30572030)

I don't know what "pretty much end-of-life Windows XP" you speak of. I'm replying to this from Windows XP Media Center Edition. 10-20% of the computers on display at Best Buy last week were netbooks and nettops with Windows XP. Most HP workstations [hp.com] have "Windows XP Professional 32-bit (available through downgrade rights from Genuine Windows® 7 Professional 32-bit)" and "Windows XP Professional 64-bit (available through downgrade rights from Genuine Windows® 7 Professional 64-bit)" as options as of today; until this week (last week of December 2009), if I remember, they didn't have any operating system options except "Vista® Business 32-bit with downgrade to Windows® XP Professional 32-bit custom installed" and "Genuine Windows Vista® Business 64-bit with downgrade to Windows® XP Professional 64-bit custom installed". Why? Because people who buy computers for a business environment will not buy Vista, at any price, for real production work — fair or not. I have clients who will not buy a computer unless it has Windows XP. Despite Microsoft again attempting to remove the previous OS from the supply chain by force despite overwhelming demand, just like they have before, XP is still being sold new on a very large portion of computers.

Re:So only XP is out of luck? (1)

conureman (748753) | more than 4 years ago | (#30572182)

Who's gonna install old XP on one of these new HDDs? HIBT?

WD is already shipping them (5, Informative)

daha (1699052) | more than 4 years ago | (#30571494)

There are certain models of the Western Digital Caviar Green drives that are already shipping with a 4K sector size, such as this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16822136490 [newegg.com]

Re:WD is already shipping them (1)

Pedrito (94783) | more than 4 years ago | (#30571814)

There are certain models of the Western Digital Caviar Green drives that are already shipping with a 4K sector size, such as this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16822136490 [newegg.com] Where do you get the 4K sector size from? From here: [wdc.com] User Sectors Per Drive 1,953,525,169 1.9 billion * 4K sectors = 7.6 GB 1.9 billion * 512 byte sectors = 972 MB Or am I missing something?

Looks like 512 (1)

midicase (902333) | more than 4 years ago | (#30571918)

From the WD website:
http://www.wdc.com/en/products/products.asp?DriveID=763 [wdc.com]

Capacity 1 TB
User Sectors Per Drive 1,953,525,169

That would be 1 TB / 1,953,525,169 = 512. I tried to verify with the spec sheet but the model's pdf is password protected.

Re:Looks like 512 (4, Informative)

butlerm (3112) | more than 4 years ago | (#30572124)

Those are "logical" sectors, which can be different from the physical sector size. According to the Anandtech article [anandtech.com] the Western Digital hard drive model numbers that end with "EARS" use the larger, 4KB physical sector size, while presenting a 512 byte logical sector size to the operating system for compatibility reasons.

Please note, of course, that the logical sector size is a drive interface level concept distinct from the filesystem cluster or block size. Filesystem block sizes have generally been larger than the logical or physical sector size for quite some time.

Re:Looks like 512 (1)

chill (34294) | more than 4 years ago | (#30572268)

So, if you're using a compatible OS, will you be able to take advantage of the drive by having it present you with 4,096 byte logical sector sizes? Or is that all in the disk format?

Re:WD is already shipping them (0)

Anonymous Coward | more than 4 years ago | (#30571986)

Where is that indicated in any of the specs? How would someone determine if a drive has a 4K sector size?

http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-701229.pdf [wdc.com]

Re:WD is already shipping them (1)

noidentity (188756) | more than 4 years ago | (#30572136)

There are certain models of the Western Digital Caviar Green drives that are already shipping with a 4K sector size

I think you mean 4.096K sector size. It's a hard drive [wikipedia.org] , after all.

Re:WD is already shipping them (1)

jc42 (318812) | more than 4 years ago | (#30572744)

... a 4K sector size

I think you mean 4.096K sector size.

Heh. You're fighting a losing battle. 99% of the human population is unable to handle numbers with more than one significant digit. The tech crowd is a little better, i.e., most of us can handle two digits most of the time.

Yes, you might observe that 4.096 rounds to 4.1, but that doesn't matter, because computer types always truncate. So 4.096 truncates to 4.0 and the final ".0" is dropped because it's the default.

If you want people to distinguish numbers that differ in the 3rd digit, you have to talk to engineers. They're less than 10^-5 of the population, and you can't expect such fine distinctions in anything outside of technical journals.

(Actually, I just made up that 10^-5 number. I wonder what the real order-of-magnitude estimate is. Yeah, google knows, but it knows several estimates that differ by several OoMs. ;-)

Care to provide examples? (1)

bogaboga (793279) | more than 4 years ago | (#30571532)

"...This may not be a smooth transition, because some OSes do not align partitions on 4K boundaries..."

In cases like these, it always helps to provide examples. Care to do so? Thanks.

Re:Care to provide examples? (3, Informative)

DeHackEd (159723) | more than 4 years ago | (#30571628)

I just checked my system. /dev/sda1 is /dev/sda + 32256 bytes, which is 63 512-byte sectors. /dev/sda2 is also on an odd-numbered sector alignment.

Fedora 11 fresh install, which is less than a year old.

Re:Care to provide examples? (5, Funny)

andrewd18 (989408) | more than 4 years ago | (#30571662)

I realize this is Slashdot, but both of the articles linked talk about the affected operating system. Hint: It shares an ending with a colloquial name for urine.

Re:Care to provide examples? (5, Funny)

tepples (727027) | more than 4 years ago | (#30571690)

I realize this is Slashdot, but both of the articles linked talk about the affected operating system. Hint: It shares an ending with a colloquial name for urine.

Wii? PSP?

Re:Care to provide examples? (1)

BobMcD (601576) | more than 4 years ago | (#30572218)

That's a terrible hint. Now I have to RTFA. THANKS!

Re:Care to provide examples? (3, Interesting)

Gulthek (12570) | more than 4 years ago | (#30572244)

It took me longer than it should've to answer this riddle. Shortcut for the similarly caffeine deprived: andrewd18 means "P" as in Windows XP.

Seriously, I was like "Win...dows?" "U...nix?" "Micro...soft?" "OS...X"? "BS...D"?

Re:Care to provide examples? (5, Funny)

Anonymous Coward | more than 4 years ago | (#30572322)

Solar... isssss

D:

intelligent interfaces (1)

FuckingNickName (1362625) | more than 4 years ago | (#30571580)

Why does the sector size presented by the interface have to reflect anything about the hardware? isn't this like the CHS/LBA conversion done under the hood? What about the ability to request a particular sector size, with the default being 512 bytes and the recommended amount being the hardware amount for optimisation purposes? Memories of 512 versus 2048 in the CD booting of older versions of VMS...

Re:intelligent interfaces (-1, Redundant)

Anonymous Coward | more than 4 years ago | (#30571704)

Adapters = Overhead. Even if they're in hardware and on the drive.

Re:intelligent interfaces (1)

SharpFang (651121) | more than 4 years ago | (#30571772)

But almost the same conversion is done already!
Do you really believe your hard drive has 256 heads?

Re:intelligent interfaces (5, Funny)

Thanshin (1188877) | more than 4 years ago | (#30572140)

Do you really believe your hard drive has 256 heads?

It had only six, at first, but we didn't know the thing about burning the stumps.

Re:intelligent interfaces (1)

Anti_Climax (447121) | more than 4 years ago | (#30572606)

Well played, sir... Well played

Re:intelligent interfaces (3, Informative)

tepples (727027) | more than 4 years ago | (#30571708)

Why does the sector size presented by the interface have to reflect anything about the hardware?

If the OS clusters aren't aligned to physical sectors, the hard drive's controller has to read-modify-write all the time.

Re:intelligent interfaces (1)

petermgreen (876956) | more than 4 years ago | (#30571966)

It doesn't, and indeed these WD drives will still have 512 byte logical sectors so there will be 8 logical sectors to one physical sectors.

The problem is if the partition is misaligned the OS is likely to make a load of unaligned writes. Those unaligned writes will force the drive to do a read-modify-write (which afaict will mean waiting for a complete rotation in the middle of the operation)

Add this to the fact that some systems (most notablly XP) have a habbit of aligning partitions on the boundries of cylinders* and that a typical cylinder is 63 sectors and you have a pretty much guaranteed misalignment.

It's easilly fixed if you know about it and have the right tools (WD supply one) but if you aren't aware of it you will likely get poor performance for no obvious reason.

*cylinders don't really exist on modern drives but bioses emulate them since the traditional real mode hard drive access mechanisms use them.

Isn't this just a firmware change? (3, Informative)

SharpFang (651121) | more than 4 years ago | (#30571666)

It doesn't sound like the 512 bytes per sector is tightly bound to hardware. More like a low-level reformat plus change of some #defines in the firmware to transform from one to another type. Which would mean there could be i.e. a jumper setting for sector size, allowing for backward compatibility.

Also, the fact an OS doesn't enforce partition alignment doesn't mean it won't respect a disk formatted to aligned partitions. Just provide a 3rd party partitioning tool that aligns the partitions right, and install the OS on pre-made partitions. If your business depends on WinXP so much, your IT dept should be capable of doing it.

Re:Isn't this just a firmware change? (0)

Anonymous Coward | more than 4 years ago | (#30571946)

These new drives will most likely work with older OSs, it's just that when you go buy a 4k native drive, plug it into WinXP and format it at 512Bps, you'll end up with less capacity than you expected. Your new 1TB (4k native) may only give you 900GB at 512BpS.

By shipping drives in this configuration, and (finally, after 10 years!) getting OS suppliers to agree to it, hard drive manufacturers are going to be able to reduce their areal density design requirements, making it easier to ship drives.

You can probably format your existing hard drive at 4k if you want to, and if your OS supports it. You'll get anywhere from an 8 to 14% capacity increase in doing so. So, your 1TB drive (512 B native) would end up with perhaps 1.1TB at 4k sector size.

Re:Isn't this just a firmware change? (0)

Anonymous Coward | more than 4 years ago | (#30572226)

It doesn't sound like the 512 bytes per sector is tightly bound to hardware.

The hardware is fundamentally designed to work optimally for whatever data unit, (sector, PDU, packet, whatever) is specified. There are buffers (physical memory, very fast, not general purpose) sized precisely for the packets. The time required to modulate the contents of the buffers across the bus(ses) is a function of the packet size and bus frequency; i.e. it does not vary and is therefore assumed throughout the system. This is real low level stuff we're talking about. The chips on the bottom of your disks are not general purpose devices that change their nature because you recompile something.

If your business depends on WinXP so much

Heh. I think it's the other way around. NTFS has been 4K aligned for a long time now; there are actually tools in the world to align legacy FAT file systems to 4K for conversion purposes. The phrase "some OSs" instead of "micro$oft" is a subtle clue that it's actually the non-microsoft systems that will have 4K unaligned partitions. You were expected to detect that.

disable ECC? (4, Interesting)

Anonymous Coward | more than 4 years ago | (#30571740)

I heard some talks from the ZFS folks at Sun about how they were floating the idea to HD mfgr's of just disabling ECC on the drives. ZFS checksums every block, and in a RAID configuration, it would be able to transparently correct any checksum errors. I think this may have also been the motivation behind bringing triple-redundant RAID to ZFS.

The motivating idea was that this would reduce the overhead involved on ECC and gain extra space.

Thoughts?

Re:disable ECC? (4, Interesting)

MobyDisk (75490) | more than 4 years ago | (#30571796)

That wouldn't work with existing file systems that assume the drive does this. That's like deciding to remove the checksums from TCP and IP because a few protocols provide their own checksums. Might work in specialized cases. Probably just adds risk though for no benefit.

Re:disable ECC? (1)

Abcd1234 (188840) | more than 4 years ago | (#30572248)

Well, presumably the idea would be to add an ATA command which allows one to disable ECC on a drive on-the-fly. Or, at minimum, a hardware switch of some kind.

Re:disable ECC? (3, Informative)

Junta (36770) | more than 4 years ago | (#30572302)

That's like deciding to remove the checksums from TCP and IP because a few protocols provide their own checksums.

Funny you should mention IP checksums, that's one feature removed from the IP layer in IPv6 precisely because the 'important' protocols do it themselves anyway (i.e. TCP).

Re:disable ECC? (4, Insightful)

Waffle Iron (339739) | more than 4 years ago | (#30572162)

It doesn't seem like a great idea to me. There are a lot of different ECC algorithms and implementations. It seems to me that it would be better to let the hard drive manufacturer select one that closely matches the expected signal and noise characteristics of a particular disk drive rather than some generic algorithm in the filesystem.

Re:disable ECC? (1)

beelsebob (529313) | more than 4 years ago | (#30572538)

The point though is that zfs does this *anyway* even if the disk does it too, so why have the disk do it too and lose more space.

Re:disable ECC? (3, Interesting)

Jeff DeMaagd (2015) | more than 4 years ago | (#30572168)

I can see this working for drives made specifically for RAIDs. Lose ECC on single drive configurations and you're asking for trouble. At least for RAIDS, a controller would need to be aware of this and do the remapping themselves, in the end, I don't know if it's worth doing this at all. If some enterprising RAID controller company could prove it works better to do it this way, then I can see it happening.

Re:disable ECC? (3, Interesting)

TheLink (130905) | more than 4 years ago | (#30572412)

If they really did that, I'd say they were clueless. Such a feature would increase the odds of error.

ZFS might checksum every block. But what happens when ZFS is not everywhere? Does the BIOS or whatever equivalent support ZFS checksumming for reading the boot sectors? So those sectors better be 100% or you better be turning it off for boot drives. You have to use ZFS everywhere and for everything. For example, if you ever try to image a 1TB disk without ECC, the odds of bit errors will be high. Even if ZFS can repair it - you'd only find out much later (too late?) and likely after another error prone write.

Such a feature would just be creating more opportunities for people to get things wrong.

And for what benefit?

> The motivating idea was that this would reduce the overhead involved on ECC and gain extra space.

I think the people who'd want ZFS or RAID would rather have better reliability than the 10% or so extra space.

Even if they don't know it at first ;).

Re:disable ECC? (5, Informative)

butlerm (3112) | more than 4 years ago | (#30572430)

That's insane. ECC at the hardware / firmware level corrects the vast majority of bit errors transparently in a manner that is invisible to the operating system. If you took out sector level ECC, the drives would be useless in anything other than a ZFS RAID configuration, and even then performance would drop in the presence of trivially ECC correctable errors, due to the re-reads and stripe reconstructions at the filesystem level.

Drive performance would probably drop because the heads would have to stay in closer alignment without the ability of ECC to correct data read errors caused by small vibrations and electrical noise. In addition, sector relocations would probably increase because tiny flaws that do not impair the ability of a drive to write an ECC correctable sector would force the drive to remap that sector to another part of the disk.

It is a similar issue with various wire level data transmission schemes. If DSL connections did not use error correcting codes, they would suffer much higher packet loss rates than they do now, especially at distance. Most those packets would generally get retransmitted due to transport level checksum errors, but why resort to performance impairing fall back measures when the problem can be largely eliminated at a lower level?

Re:disable ECC? (2, Insightful)

Izmunuti (461052) | more than 4 years ago | (#30572468)

Ugh. Sounds like a bad idea. Hard drive channels are noisy. How will ZFS fare if lots and lots of sectors read from every drive have at least a couple of bits in error? With no ECC in the drive, errors would be common.

Re:disable ECC? (2, Insightful)

Anonymous Coward | more than 4 years ago | (#30572588)

One of the reasons there is so much BS and hype surrounding ZFS...

Drives already do and have for years done error correction and most revector badblocks dynamically and the OS never even knows. It's part of how drives are so reliable as it is. Pushing that logic up in to the filesystem just introduces more complexity to the filesystem and reduces overall reliability.

Talk to any drive engineer, what they want the most is for filesystem engineers to stop trying to outsmart the drive guys. THere is not a guaranteed relationship between a sector and it's location on the disk, the drive and the firmware it has will try to figure that out the best way it can.

What they really should spend their time on is filesystems and their relationship with cache will have to change in the next decade, solid state medias don't need caching the same way disks do, I fully expect disks to start coming as hybrid devices with solid state storage and disk based storage as a singular device. There are some relatively complex problems to solve to provide media awareness to all the storage algorithms.

Re:disable ECC? (1, Insightful)

Anonymous Coward | more than 4 years ago | (#30572674)

An ECC should be implemented as close to where the problem occurs as possible. For a hard drive, this means on sectors, not (abstract) blocks. Otherwise you'll see the OS rewriting clean sectors that are parts of unclean blocks (if the block size is greater than the sector size), and gain nothing if the block size is smaller than the sector size.

Re:disable ECC? (2, Insightful)

drinkypoo (153816) | more than 4 years ago | (#30572700)

If you were going to eliminate ECC in one place or another, it wouldn't be on the drive. The drives have to operate in the real world of analog states, while the filesystem works in the virtual world of "whatever the disk actually feeds me". Disks have to have correctable ECC just to reliably give you accurate data from magnetic media at these densities. It would make more sense to upgrade the on-disk ECC and give the filesystem better access to the disk's ECC.

Why are there sectors? (2, Interesting)

AP31R0N (723649) | more than 4 years ago | (#30571776)

i'm asking because i don't know, not to troll.

What is their purpose? Does the purpose still matter?

Re:Why are there sectors? (4, Informative)

JordanL (886154) | more than 4 years ago | (#30571916)

A sector on a HDD is the minimum writeable space. Think of it as a lot in a subdevelopment. If each lot is 50,000 sq. ft. on a 20 acre plot, and you move to 60,000 sq. ft. lots instead, the plot is still 20 acres, but the development now has less lots on it.

In computers, larger sectors are often better for large files, while smaller sectors are better for smaller partitions and smaller files. If a sector is 4096 bytes, and you create a 1024 byte file, it still occupies 4096 bytes on the disk, as the HDD won't write anything else but that file to the sector. If you have files that are hundreds of megabytes though, you can access the file, with minimum wastage, by using fewer sectors, which reduces thrashing and similar issues.

The discrepancy between file sizes and sector sizes is what the difference is in Windows when you view a hard drive and it displays "size" and "size on disk". "Size" is the actual file size, while "size on disk" is the amount of space the file occupies on the hard drive.

Re:Why are there sectors? (1)

AP31R0N (723649) | more than 4 years ago | (#30572024)

+1 Informative.

So why have the sectors at all? Bigger sectors seem to incur more waste, though with HDDs growing at the rate they do it might not matter. But if there was no sectoring, then a file could take as much space, and only as much space as it actually needs. The 1024 byte file could then take 1024 bytes.

i'm not saying sectors aren't needed, i just don't understand why they exist in the first place.

It is it some kind of addressing thing?

Re:Why are there sectors? (3, Interesting)

JordanL (886154) | more than 4 years ago | (#30572076)

It is indeed. Unless HDD makers were going to create firmware, and programmers made partition formats, which address each bit individually (which itself would require an enormous amount of space... much larger than the HDD in fact), you will always be unable to live without sectors. The subdivision idea is again relevant. Imagine if every part of the 20 acre plot had to be "addressable" down to the square inch.

Re:Why are there sectors? (2, Interesting)

AP31R0N (723649) | more than 4 years ago | (#30572220)

Ah, so it's saying, "Turn left on Evergreen... it's on that block". And the monstrous estate is from Elm to Fern at State. As opposed to GEOCOORD 32'57"(bunchOfDigits) by 32'57"(more digits).

Got it.

With the 1024 byte example, could the address just be "from bit X to bit X+1023"? i guess that too would be too much. All those tiny .dlls and .inis would take more space to define than they actually take.

Thanks!

Tail packing (4, Funny)

tepples (727027) | more than 4 years ago | (#30572246)

Unless HDD makers were going to create firmware, and programmers made partition formats, which address each bit individually (which itself would require an enormous amount of space... much larger than the HDD in fact), you will always be unable to live without sectors. The subdivision idea is again relevant. Imagine if every part of the 20 acre plot had to be "addressable" down to the square inch.

It's called block suballocation [wikipedia.org] : store a small file in its entirety in another file's slack space. And yes, it's a "killer" feature.

Re:Why are there sectors? (1)

MBCook (132727) | more than 4 years ago | (#30572170)

Let's take a 1TB disk, which is becoming common. To address it all in 512B sectors, you need 31 address bits (since there are 2 billion sectors).

If you change to 4KB sectors, you now have 1/8 as many, so you only need to address ~270 million sectors, which takes 28 bits of address space.

The thing is, disks are given addresses of a certain size. If all address are 16 bits, and the sectors only have 512B in them, your disk can't be bigger than 32MB. By using 32 bits, you can go up to ~2GB. If you are using 4KB clusters, you could have 16GB.

In short, it saves address lines. That prevents having to change protocols, saves memory, saves hardware costs, etc.

The FAT filesystem [wikipedia.org] article on Wikipedia discusses how this same basic problem happened to FAT.

Re:Why are there sectors? (2, Informative)

Thanshin (1188877) | more than 4 years ago | (#30572194)

So why have the sectors at all? [...]
The 1024 byte file could then take 1024 bytes.

That's not "not having sectors", that's having sectors 1 byte long.

Thus, apply the reasoning of "bigger sectors, faster treatment of bigger files, and vice-versa".

Re:Why are there sectors? (0)

Anonymous Coward | more than 4 years ago | (#30572250)

The short answer is, essentially, addressing as you guessed. Sectors are essentially the addresses on the disk. Whereas computer memory (RAM) will have an address for each piece of information (e.g. a 'word' where 'word size' might be 8 bits or 32 bits or 64 bits), a disk will have an address for a sector or block. For RAM you could have 32 bit addressing which would give 2 ^32 bytes of memory (4 GiB) or 64 bit addressing could handled 2 ^ 64 bytes of memory. For disk sectors, you don't have an address for each byte, but for each sector. In theory, you could have access similar to memory with a 1 byte sector, but performance would suffer due to the overhead from addressing, access etc.

(This is kind of rough and can vary, but hopefully it gets the idea across. In a computer architecture class you would go into a ton of detail.)

You can read some here:
http://en.wikipedia.org/wiki/Logical_block_addressing [wikipedia.org]
http://en.wikipedia.org/wiki/Hard_disk_drive_partitioning [wikipedia.org]

Re:Why are there sectors? (1)

bdsesq (515351) | more than 4 years ago | (#30572278)

Sectors used to be needed because the drives would lose sync. The sector header would help keep it in sync.

One thing that didn't get covered very well is "breakage". Breakage is the amount of space lost due to sector size. For each file you lose, on average, one half of a sector. This is because the last sector used by a file has somewhere between one and 512 bytes of data. For the new drives that is between one and 4096 bytes.

So if you have 1,000,000 files on a drive with 512 byte sectors you lose half of 512 bytes 1,000,000 times of 256,000,000 bytes. For a disk with a 4096 sector size you would lose half of 4096 bytes 1,000,000 times or 2,048,000,000 bytes.

This is not very important to the drive manufacturers because they are really interested in bragging rights for the biggest disk. And they know Joe Consumer does not understand any of this.

Re:Why are there sectors? (1)

AP31R0N (723649) | more than 4 years ago | (#30572420)

What does sync mean in this context?

So they are making the drives faster in the sense that there are fewer sectors, so it's easier to get to a city than to a particular block of a city. They are also keeping the address space small. And they are wasting space because most of the blocks of the city have huge yards and a tiny house.

Actually that first sentence should be a question. Does having bigger sectors make the drive appear to be faster?

Fascinating.

Re:Why are there sectors? (2, Informative)

TheRaven64 (641858) | more than 4 years ago | (#30572308)

Yes, it's an addressing thing. The grandparent is confusing sectors with allocation units. A filesystem is perfectly at liberty to allocate sub-sectors to different files (some do). A 32-bit disk interface can address 2^32 sectors. If you have one-byte sectors then that means you're limited to 4GB disks. If you have 512 bytes sectors then you're limited to 2TB. If you want a disk bigger than 2TB then you can either make the interface wider or can make the sector size bigger. Making the address wider requires defining a new interface[1], although ATA currently supports 48-bit addresses, so this isn't really a problem for a while. It is convenient for filesystems, because they can continue to use 32-bit sector indexes for partitions larger than 2TB.

The real advantage of bigger sectors is that they reduce the command overhead. To write 4KB to the disk you just need to send one write command and the data, rather than eight. All modern operating systems cache data from disk in RAM and so will write it out or read it in as a group of pages. The smallest page size of any modern architecture is 4KB, so having 4KB sectors is a lot more convenient.

Re:Why are there sectors? (3, Interesting)

DarkOx (621550) | more than 4 years ago | (#30572106)

You are confusing physical sector size with cluster size. May file systems are already addressing data in larger blocks. 4096 is very commonly used. They are generally multiples of 512 which is the physical sector size; so that its is easy to calculate the physical sector that needs to be changed when you know the logical.

Its quite possible to have a cluster size smaller than the sector size; the file system would need to be smart enough to determine what other clusters fall on that sector and write them all though.

Re:Why are there sectors? (1)

JordanL (886154) | more than 4 years ago | (#30572114)

Sector does have several definitions, but from reading the article I'm fairly certain they are talking about sector size, and not clusters.

Re:Why are there sectors? (1)

blueg3 (192743) | more than 4 years ago | (#30572336)

The first part is true, but the second part isn't. The logical cluster size and its implications are all in software and dependent on the filesystem. It's entiirely possible to have filesystems that put multiple small files into a single logical cluster and, for that matter, into a single physical sector. This does mean things are a bit more complicated, though.

The real answer is simply that a sector is the size of a unit of data that can be read from or written to a hard drive. That constrains how the operating system accesses the disk, which has performance implications for different disk-use strategies.

Re:Why are there sectors? (1)

Izmunuti (461052) | more than 4 years ago | (#30572306)

Hard drives are random-access devices and sectors are the smallest atomic unit that a drive can normally physically read and write. It doesn't read or write half a sector. When emulating a write to a 512 byte logical block with 4096 byte physical blocks on the media, it has to read the whole 4K sector, modify it with the changed 512 bytes, and rewrite the entire 4K sector.

The concept of sectors could be hidden from the interface, theoretically. You could put the whole file system into the drive (OSD), for example, or allow the host to address bytes, hiding all the read/modify/writes. But, all the common hard drive interfaces (ATA/SCSI) use blocks/sectors.

Re:Why are there sectors? (0)

Anonymous Coward | more than 4 years ago | (#30572326)

Sectors allow the address space of the drive to be smaller by grouping bytes. This reduces the amount of space on the drive needed for keeping track of where the files are located. The downside, of course, is that in general, no more than one file can occupy a given sector (though a file usually occupies multiple sectors). Although it could be done to eliminate sectors by using a run-length addressing system within the filesystem, this would quickly break down if aggressive defragmentation was not performed on a highly active drive (though a filesystem could abstract sectoring on it's own, and virtual filesystems, such as ISO files do)

Re:Why are there sectors? (1, Informative)

donscarletti (569232) | more than 4 years ago | (#30572496)

A sector used to be quite literally a sector of a disc in the mathematical sense, like a wedge shape that spins around. Now with LBA (labeling hard drive's blocks in series from zero rather than by their physical position) it is just like a block on your filesystem, but on the hardware instead, it is a blob of data that must be read or written as a whole. The rationale is that you are not likely to ever want to read or write one byte at a time, so there is no reason to make the hard disk handle requests for one byte. The difference between a "sector" and a block is that a block on a file system should not be smaller than a sector on the hard drive since an OS can pretend two, four, etc. sectors is a single block, it cannot cut a sector in half.

The upshot of this, is unlike memory which is addressable to the byte, hard discs can be much bigger compared to the address range since it only needs volume/blocksize addresses to locate the data, so even with a block size of 512, a 2 Terabyte (base2) volume may be sufficiently covered with a 32 bit address space, this makes everything a lot easier and more efficient.

Anyway, in answer to your question, sectors are still as useful as they ever were, just they might not actually be sectors anymore because of LBA. Maybe they are, I'm not sure, I've only written hard disc drivers, I've never built one of the things.

use of current cultural context.... (2, Insightful)

Himring (646324) | more than 4 years ago | (#30571818)

This may not be a smooth transition, because some OSes do not align partitions on 4K boundaries.

"One life ends; another begins"

Give me some context (1)

KnownIssues (1612961) | more than 4 years ago | (#30572014)

How many Libraries of Congress can you store in that?

Re:Give me some context (1)

compro01 (777531) | more than 4 years ago | (#30572342)

Approximately 186 picoLOCs.

Paying for More Slack Space (1)

Doc Ruby (173196) | more than 4 years ago | (#30572258)

Larger sectors means more empty space at the end of the last sector of a file. Lots of files means lots of wasted space. Modern OS'es, especially Windows, have many more, smaller files than in past versions, and the trend continues upwards.

So larger sectors means more space bought on a drive that isn't used. Which means more drives bought.

I can see how drive manufacturers would like that.

Re:Paying for More Slack Space (1)

ettlz (639203) | more than 4 years ago | (#30572386)

I believe a filesystem with tail-packing support would overcome this.

Re:Paying for More Slack Space (1)

644bd346996 (1012333) | more than 4 years ago | (#30572478)

Except that larger sectors also means more efficient ECC, so the same drive will present a higher OS-accessible capacity. Unless you plan to deal mostly in files that are less than 4K, you'll come out ahead.

Re:Paying for More Slack Space (1)

Volante3192 (953645) | more than 4 years ago | (#30572536)

Well, the trend today, especially with large drives, is to go for a cluster size of 4k anyway. Sure, there'll be a lot of system files under 4k, but there's going to be much more music and pictures over 4k that will likely take more space.

Re:Paying for More Slack Space. (2, Informative)

butlerm (3112) | more than 4 years ago | (#30572634)

Any change in sector size that doesn't affect the filesystem block size will not affect the number of KB required to store a file at all. Since virtually every filesystem already uses 4 KB block sizes by default a change to 4KB logical or physical sector sizes will not have an effect on storage requirements.

Okay. So, what if we have to image an old box? (0, Offtopic)

E-Sabbath (42104) | more than 4 years ago | (#30572444)

Let's say, two situations.
A: Moving an XP box from an old 512 sector drive to a new, 4k sector drive. Image using, say, Acronis. Just have to run the CLI tool, after imaging?

B: Moving an XP box, using Acronis, from a 4k to a 4k. Would I have to run the tool?

Load More Comments
Slashdot Login

Need an Account?

Forgot your password?