Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Linux Not Quite Ready For New 4K-Sector Drives

CmdrTaco posted more than 4 years ago | from the when-more-is-less dept.

Data Storage 258

Theovon writes "We've seen a few stories recently about the new Western Digital Green drives. According to WD, their new 4096-byte sector drives are problematic for Windows XP users but not Linux or most other OSes. Linux users should not be complacent about this, because not all the Linux tools like fdisk have caught up. The result is a reduction in write throughput by a factor of 3.3 across the board (a 230% overhead) when 4096-byte clusters are misaligned to 4096-byte physical sectors by one or more 512-byte logical sectors. The author does some benchmarks to demonstrate this. Also, from the comments on the article, it appears that even parted is not ready, since by default it aligns to 'cylinder' boundaries, which are not physical cylinder boundaries and are multiples of 63."

cancel ×

258 comments

first misaligned post (2, Funny)

Anonymous Coward | more than 4 years ago | (#31135388)

damnit, obviously since this is not technically the 'first post', my web browser must be misaligned by a post

oh great.. i try to make a joke and ... (2, Interesting)

decora (1710862) | more than 4 years ago | (#31135414)

the first time i have ever actually gotten 'first post'... it is when i try to make a joke about not having gotten first post. ya see my first post was supposed to come up like second or third.. it would have been HILARIOUS . .. but oh no in soviet russia, the fates mock you!!!!

Re:oh great.. i try to make a joke and ... (0)

Anonymous Coward | more than 4 years ago | (#31135854)

And now you miss the "Post anonymously" box explaining it all!

Parted / GPT (1, Interesting)

Anonymous Coward | more than 4 years ago | (#31135392)

I heard using parted and GPT labels instead of MSDOS will optimize it on 4096 byte sectors automatically. Any truth to it?

Re:Parted / GPT (1)

houstonbofh (602064) | more than 4 years ago | (#31135486)

Even if that works for new partition, what about existing ones? These drives have been out a while, and people have been installing software to them... The best solution will be a program that moves and/or resizes existing partitions to align them properly. The question is will it be an updated gparted, or something new? What ever it is, I have no doubt that it will be out for Linux far before it is out for XP. :)

Set 32 sectors per track (4, Insightful)

tchuladdiass (174342) | more than 4 years ago | (#31135416)

The simple solution is to set you Sectors per Track to 32. This would make sure that everything is properly aligned (except the first partition, usually /boot, which is mis-aligned by one cylinder).

Re:Set 32 sectors per track (3, Insightful)

Brian Gordon (987471) | more than 4 years ago | (#31135620)

Sectors, blocks, clusters, cylinders... I hope that as we move to solid state drives, devs have the sense to exorcise these anachronisms from the kernel. We haven't been able to get rid of terminals in the 20 years since they've even existed.. this document [linux.org] is heart wrenching. Try reading it; it'll make you cry to see how deeply the now-irrelevant concept of a terminal runs in Linux.

Re:Set 32 sectors per track (4, Insightful)

walshy007 (906710) | more than 4 years ago | (#31135686)

the now-irrelevant concept of a terminal

Speak for yourself sir, I for one like my rs-232 terminals to be handy for when ethernet is down and you can't ssh (and can't be assed hooking up keyboard and monitor). Seriously, anyone adept at the command line uses it far more than the gui to get things done, terminals will never disappear.

Re:Set 32 sectors per track (0, Offtopic)

tagno25 (1518033) | more than 4 years ago | (#31135734)

the now-irrelevant concept of a terminal

Speak for yourself sir, I for one like my rs-232 terminals to be handy for when ethernet is down and you can't ssh (and can't be assed hooking up keyboard and monitor). Seriously, anyone adept at the command line uses it far more than the gui to get things done, terminals will never disappear.

or like on the SheevaPlug, Cobalt RaQ, or Cobalt Qube when you cannot hook up a keyboard and monitor

Re:Set 32 sectors per track (0, Offtopic)

0100010001010011 (652467) | more than 4 years ago | (#31135808)

So why aren't more machines built like the SheevaPlug?

Yet is has console, but it's a USB connection. It has the USBSerial built in. I don't need anything for my Mac other than the cable that came with it.

Re:Set 32 sectors per track (1, Offtopic)

jedidiah (1196) | more than 4 years ago | (#31135698)

Terminals are only irrelevant if you have a strictly Windows and DOS notion of computing.

The document you cited touches on this a little bit.

Re:Set 32 sectors per track (0, Offtopic)

rubycodez (864176) | more than 4 years ago | (#31135740)

but in windows server 2008 and Exchange 2007 the command line and scripting (powershell) is back with a vengeance. Even Microsoft realized dah Powah of dah Toiminal!

Re:Set 32 sectors per track (4, Insightful)

rubycodez (864176) | more than 4 years ago | (#31135726)

terminals are a very necessary and relevant part of Linux. That's how most server administration is done. That's how sending commands to many network appliances is done. That's how setting up high end computers is done (e.g. set up a midrange Integrity or Superdome and you'll start with terminal on the serial port, whether cu in linux or hyperterminal in windowws or a real terminal). Also how certain tasks are performed in GUI environments. It doesn't matter that the terminal is now mostly virtual, the cursor control and font attribute features make convenient applications possible. Even on the weekend here I am chatting via IRC to some tech friends with irssi in terminal under screen, and reading server status emails with mutt. the terminal, it's 21st century tool.

Re:Set 32 sectors per track (5, Insightful)

Anonymous Coward | more than 4 years ago | (#31135860)

terminals have nothing to do with the command line!
i think the op is complaining about the fact that things like
baud, stopbits and whatnot are deeply embedded in the
linux kernel. these concepts are not necessary to
have a command line. c.f. plan 9.

Re:Set 32 sectors per track (1)

Brian Gordon (987471) | more than 4 years ago | (#31136142)

Yes, thank you. That's exactly what I mean.

Re:Set 32 sectors per track (3, Informative)

Z00L00K (682162) | more than 4 years ago | (#31135984)

Essentially we are back to the old problems of the ST412 interface where we had to figure out the best interleave for the drives as well when we were formatting them. Most drives then did have a fairly conservative interleave, but a reformat of them could improve the throughput considerably. A reformat could be done so that the whole track could be read in 2 rotations instead of 3, and what that does to performance is fairly easy to understand. C800:5 was a commonly used BIOS address where the low level format routine did reside.

But from what I understand this problem is an offset problem when the head steps from track to track, and that's also an issue to be considered. And today it's not common knowledge/practice to low level format hard drives.

And why stick at 4k sectors? Depending on the system you may want to use a different sector size. If you run Oracle on some systems the block size is 8k, and in that case you may want to have 8k disk blocks too since it would be good for performance.

Anyway - sooner or later we will have flash drives instead, and then this isn't a problem.

Re:Set 32 sectors per track (4, Insightful)

amorsen (7485) | more than 4 years ago | (#31136186)

Anyway - sooner or later we will have flash drives instead, and then this isn't a problem.

Actually this problem is potentially much worse on SSD's. Erase blocks are huge, and read-modify-write really sucks on flash.

Good thread on this. (4, Informative)

Anonymous Coward | more than 4 years ago | (#31135426)

Re:Good thread on this. (1)

darkpixel2k (623900) | more than 4 years ago | (#31135488)

One of the comments in that thread suggests switching to GPT if you aren't using Windows.
I haven't used Windows at home since ~2001.

Can you just wipe/reinstall using GPT? I thought the BIOS was involved with the type of partition table and that I had to be using the msdos partition type because of the BIOS. Can a geek with deeper knowledge of partitions and and all things boot drop some knowledge?

Re:Good thread on this. (2, Informative)

Anonymous Coward | more than 4 years ago | (#31135570)

The BIOS has no understanding of partition tables. It merely reads the first sector of the harddrive to 0x7C00 and then jumps to that location. The DOS partition table is used by convention for interoperability between operating systems. If you wanted to use a different partitioning scheme, there is no technical reason your operating system couldn't.

Re:Good thread on this. (3, Interesting)

Anonymous Coward | more than 4 years ago | (#31135610)

GPT wraps itself in a MBR partition map. At the very least the GPT is supposed to include an MBR map that claims the whole disk as used by GPT to avoid issues with old disk tools and the like. And if you've got a partition scheme that's compatible with the MBR scheme they can both contain the same information, assuming your disk tool supports this, so that MBR-only environments can still find your partitions.

It's also possible to format with GPT and then use an MBR-only tool (fdisk) to go back and manipulate the (fake) MBR to contain a partition that points to the same start/end points as the GPT boot partition -- GPT-aware systems will just ignore the MBR record, and non-GPT systems will at least be able to find the boot partition.

As to whether your motherboard/firmware supports GPT, it can be hard to say. Anything with EFI is required to support GPT. Some systems with a legacy BIOS pre-boot environment also have support for GPT, because it's the only way to support large disks. But I can't name particular firmware versions that do/don't support GPT.

Re:Good thread on this. (3, Informative)

marcansoft (727665) | more than 4 years ago | (#31135644)

Unless your BIOS is trying to be too smart and peeking into your partitions instead of launching the MBR (sadly, some do), it won't matter. It's the MBR's job to boot your system after the BIOS hands off control to it, and on most Linux systems the bootloader is installed straight into the MBR.

Re:Good thread on this. (2, Informative)

Z34107 (925136) | more than 4 years ago | (#31135834)

Even if you are using Windows, Vista and up support GPT. It's handy for servers where you expect to have partitions larger than 2 TB.

But I guess if one were using a modern version of Windows, you wouldn't have the 4K alignment problems to begin with.

Interesting (2, Interesting)

Murdoch5 (1563847) | more than 4 years ago | (#31135432)

I actually have 2 of the these drives in my desktop right now. There is a slight decrease in performance compared to Windows 7 but nothing that it unacceptable or even a need for concern. If you need to worry about the performance lost with the 4k sectors then just go solid state.

Re:Interesting (3, Insightful)

ScytheBlade1 (772156) | more than 4 years ago | (#31135476)

TFA [osnews.com] disagrees with you.

$ time cp winxp.img /mnt/sdc # ALIGNED
real 5m9.360s
user 0m0.090s
sys 0m20.420s

$ time cp winxp.img /mnt/sdd # UNALIGNED
real 13m26.943s
user 0m0.110s
sys 0m19.350s

$ time cp -r Computer Architecture/ /mnt/sdc # ALIGNED
real 42m9.602s
user 0m0.680s
sys 1m59.070s

$ time cp -r Computer Architecture/ /mnt/sdd # UNALIGNED
real 138m54.610s
user 0m0.660s
sys 2m15.630s

The first two being a single file, the latter two being multiple files in a larger directory structure.

I would heartily disagree with you on the matter.

Re:Interesting (1)

markus_baertschi (259069) | more than 4 years ago | (#31135564)

What do you disagree with ? There is a performance problem and the author thinks the kernel people should look into the matter. I don't think they can do much, after all the kernel just writes 4k block to the drive. The problem is with the drive microcode which should attenuate the problem in the case of sequential writes, like the first case. Obviously it does not do that at all. Markus

Re:Interesting (2, Informative)

ScytheBlade1 (772156) | more than 4 years ago | (#31135632)

I disagree with the "but nothing that it unacceptable or even a need for concern" part. If copying the same image to the same disk where the only difference is where the partition begins -- by one sector difference -- will amount to a 2.6x decrease in speed, and where copying a large directory with many subdirectories amounts to a 3.2x decrease in speed... that should qualify as an unacceptable decrease in speed that warrants concern.

While a kernel tweak may help alleviate the issue, it is primarily an issue with our current (userspace) disk partitioning and formatting utilities. I'd also disagree with you on the point where the problem is the drive microcode; drives should do what they are told, and not guess on behalf of the instructions they are given what to do. Admittedly, the microcode tweak would be minor and largely trivial, but I'd rather not fix (primarily) userspace software problems in the kernel, nor the device firmware.

Re:Interesting (5, Interesting)

markus_baertschi (259069) | more than 4 years ago | (#31135818)

About the microcode part. The drive pretends to be a 512byte drive, but internally is using 4k sectors and and claims to 'translate transparently'. I can understand that in a random-access scenario it it has to read-modify-write 2 sectors each time and performance suffers (2 additional reads and one additional write). But in a sequential access scenario, the penalty should be once per sequence/file, not once per sector. Here the microcode fails completely to make the best out of the suboptimal situation.

Re:Interesting (2)

Murdoch5 (1563847) | more than 4 years ago | (#31135670)

This problem isn't really kernel involved, the problem comes from the fact the write is not being issued in an effective manor. This more so leads to the fact that the system has not been fitted to use the 4k sectors. If you just put a little work into the install you can over come this problem.

By default yes you will likely see a big performance hit, but like any good system once it's tuned and fitted to it's job it will run great. People can post all the results they want from default installs and it means very little.

It's the same as if you bought a muscle car and never took the time to tune the engine, you can't try to make the claim that the car wasn't ready. you need to put the time in and get the car up and going.

The same thing is true of the install. You need to put the work in, tune it up and reap the benefits of a high performance install.

Re:Interesting (5, Insightful)

hedwards (940851) | more than 4 years ago | (#31135902)

That's true, but it's also true that having hardware lie to the OS isn't a great situation to be in. At the very least there should be some way of forcing it to be honest for the benefit of OSes that can handle the reality. A lot of the gunk and instability in computing comes from hardware that does things that are more appropriately done by software and vice versa.

Forcing users to optimize isn't inherently wrong, it's just that they shouldn't need to do it for things which are somewhat standard as a work around for weird hardware designs. And yes, I realize that the 4096byte sectors aren't being implemented arbitrarily.

Re:Interesting (1)

Murdoch5 (1563847) | more than 4 years ago | (#31136106)

I could not say it better my self, I'm going to show that post to my prof if it's okay with you. That might be the best way of explaining hardware and software interfacing I've ever heard.

Re:Interesting (2)

Murdoch5 (1563847) | more than 4 years ago | (#31135572)

That is more a distro problem then, I run a full stable Gentoo system and I see less then a 5% over head difference. Not saying your wrong because you posted output here but it can have a lot to do on how you have optimized your disk access and setup the kernel to interface the disks. Even the file system in use.

But then again that's just what I see on my system. Possibly running a different configuration would yield different results.

Re:Interesting (1)

ScytheBlade1 (772156) | more than 4 years ago | (#31135646)

Any chance you can post drive model number, output of fdisk -l, and a few similar benchmarks?

Re:Interesting (1)

Murdoch5 (1563847) | more than 4 years ago | (#31135768)

I will tomorrow. Right now I'm not on my desktop. I can easily get 70+ Mb/s write to them I think somewhere around 77.6 Mb/s. As for my partition scheme I used parted and had it properly align the disk.

Sorry that's the best I can do from where I'm, I'll make sure to update the numbers tomorrow. if it will help I'll even post the gparted version I used to partition with.

Until then have a good one.

Re:Interesting (1)

maeka (518272) | more than 4 years ago | (#31136062)

I hope you mean 70+ MB/s.

Re:Interesting (1)

Murdoch5 (1563847) | more than 4 years ago | (#31136070)

lol my bad ya MB/s

Open Source to the rescue (3, Insightful)

bogaboga (793279) | more than 4 years ago | (#31135460)

I am no kernel hacker but I can almost guarantee that some kernel hacker will provide a solution to this "short coming" fairly soon.

That's the beauty of Open Source.

I am aware though that "fairly soon" means many things to many people; which means that there could be a substantial delay before we get a working solution to this issue.

I am optimistic nevertheless.

Request to Western Digital: Provide all the information needed to develop a solution.

Re:Open Source to the rescue (0)

bhtooefr (649901) | more than 4 years ago | (#31135478)

Here's the information: Sectors are 4096, not 512, bytes.

That's all that a developer needs to know, AFAIK.

Re:Open Source to the rescue (1)

MoHaG (1002926) | more than 4 years ago | (#31135558)

And they pretend to be 512 bytes for "compatibility"... (Unlike CD-ROM / SD card sectors...)

Re:Open Source to the rescue (4, Informative)

marcansoft (727665) | more than 4 years ago | (#31135584)

Exactly. Drives are pretending to have 512-byte sectors because Windows can't deal with 4k sectors, and then silently reducing performance when you believe them and use 512-byte sector sizes. Had the drives reported 4k sector sizes, they'd work great under Linux and not at all under Windows.

This isn't a Linux problem, it's a drive problem caused by Windows. The solution is to implement yet another workaround for stupid devices, and start aligning partitions to 4k by default.

Nitpick: SDHC card sectors are always 512 bytes, and most SD card sectors are 512 bytes too. Flash memory would benefit from larger sector sizes too, but they've probably stuck to 512 bytes for Windows compatibility.

Re:Open Source to the rescue (5, Insightful)

rsmith-mac (639075) | more than 4 years ago | (#31136082)

On the contrary, this has (almost) nothing to do with Windows - it has everything to do with old OSes. The IDEMA didn't approve the 4K sector standard until 2006; it was only in the late 90's that the first meaningful research was begun by IBM on whether 512B sectors would be an issue.

As it turns out, yes, 512B sectors would be an issue, and drive manufacturers would be best served by moving to larger sectors (with some arguing over whether to go to 1K or 4K). So the IDEMA hashed this out over the first half of the decade, and finally in 2006 approved the 4K specification.

The point of all of this is that software written at the turn of the century was all done well before changing drive sector sizes was a serious discussion. WinXP was released in 2001, Mac OS X 10.0 was in 2001, and of course Linux 2.4 was also in 2001. None of those OSes know what to do with anything other than a 512B sector - the only reason Windows factors in to this equation is that WinXP just happens to be with us (no doubt trying to eat our brains) while the other two are dead. Anything circa 2005 or later such as WinVista, Linux 2.6, and Mac OS X 10.5 know full well what to do with a 4K drive.

But even that is beside the point. You don't just make major jumps like this, you have to do it in a transition so that you don't break old hardware and old software alike. Even if XP/Lin2.4/MacOSX knew what to do with 4K sectors, at some point you'd run in to hardware, 3rd party devices, etc that would not. A transition is necessary to let old hardware and software get flushed out of the ecosystem, and as such we're still years out from consumer drives offering native 4K access.

In short: drives are pretending to have 512-byte sectors because there's a lot of old stuff, including Windows XP that can't deal with 4K sectors.

Re:Open Source to the rescue (1)

TheKidWho (705796) | more than 4 years ago | (#31136088)

My Hard drive with 4k sector sizes works perfectly under Windows 7?

Re:Open Source to the rescue (1)

marcansoft (727665) | more than 4 years ago | (#31136164)

There are no hard drives with logical 4k sector sizes out there, as they all emulate 512-byte sectors for compatibility, so your point is moot. However, if they did exist, they would theoretically work with Windows 7. I was talking about Windows XP.

Re:Open Source to the rescue (1)

Miseph (979059) | more than 4 years ago | (#31136176)

s/windows/windowsxp

Whatever you may think of Vista/7/Server2008, this isn't a problem for them. I agree that it's stupid for WD to make their drives lie for compatibility with what is effectively a legacy OS (by modernity, not use), but there's no call to tar newer releases for it.

Re:Open Source to the rescue (1)

nomadic (141991) | more than 4 years ago | (#31135510)

That's the beauty of Open Source.

And that couldn't possibly happen with closed source?

Re:Open Source to the rescue (0)

Anonymous Coward | more than 4 years ago | (#31135530)

That's the beauty of Open Source. And that couldn't possibly happen with closed source?

Not with Microsoft.

Re:Open Source to the rescue (1)

bogaboga (793279) | more than 4 years ago | (#31135562)

I guess you only read but did not understand! Key words in my piece are: "Fairly soon."

Re:Open Source to the rescue (1)

nomadic (141991) | more than 4 years ago | (#31135832)

"Fairly soon."

Again, that's still possible with closed source.

Re:Open Source to the rescue (1)

Blakey Rat (99501) | more than 4 years ago | (#31136138)

Well, since Windows Vista and Windows 7 already support this, I'd say "fairly soon" is demonstrably false. Unless you're happy with "fairly soon" being "after everybody else has been doing it for several years."

Re:Open Source to the rescue (1)

hedwards (940851) | more than 4 years ago | (#31135922)

Trust me, it's right after adding GPT support to Windows XP home on their agenda. Shouldn't take more than, I don't know forever to add.

Re:Open Source to the rescue (0)

Anonymous Coward | more than 4 years ago | (#31135568)

The beauty of Open Source is that you can code it by yourself, so why don't you start right now? Another history is distribution and maintability.

Re:Open Source to the rescue (0, Troll)

buddyglass (925859) | more than 4 years ago | (#31135658)

That's the beauty of Open Source.

I guess the beauty of Closed Source, then, is that the OS supports it out of the box, without some user having to notice the problem, benchmark the performance hit, figure out (more or less) why its happening, make a big blog post, then wait for a qualified dev to fix the problem and for the major distros to pick up the fix?

Re:Open Source to the rescue (1)

jedidiah (1196) | more than 4 years ago | (#31135838)

> I guess the beauty of Closed Source, then, is that the OS supports it out of the box

      Or not. The sheep-like closed source user is never likely to notice the problem.

      Although the real problem here seems to be that you can't consider a bit of hardware
as it presents itself. Now I wonder why a hard drive company feels the need to have it's
hardware LIE to the OS?

Re:Open Source to the rescue (2, Interesting)

Pentium100 (1240090) | more than 4 years ago | (#31136076)

Now I wonder why a hard drive company feels the need to have it's hardware LIE to the OS?

So the hardware is compatible with more software. For example, hard drives still report some number of cylinders, heads and sectors to the BIOS and the OS, but hard drives have been using ZBR [wikipedia.org] for 20 years now (IIRC) so the sector number is meaningless.

But, as it is now, if my old system needs a new hard drive, I do not need to find an old drive to be compatible with my system (as long as it is IDE or SCSI, I don't know of any adapter from the newer interfaces to ESDI or ST-506, but they probably exist).

They could have made it a jumper setting set to 512B by default though. I assume the hard drive is faster using 4KB sectors instead of true 512B sectors, they could have made an option to reformat the drive to 512B (or maybe it's not possible with modern drives, I have an old 4GB SCSI drive that can be reformatted to a different sector size (I never tried it though)).

Re:Open Source to the rescue (2, Insightful)

DAldredge (2353) | more than 4 years ago | (#31136104)

Drives have been lying to the OS for more than 20 years in regards to quite a few different things so why pick now to yell and get upset?

Re:Open Source to the rescue (0, Troll)

marcansoft (727665) | more than 4 years ago | (#31135944)

The beauty isn't Closed Source, it's Market Monopoly. You know, the one guaranteeing that device manufacturers make up for your failures by deviating from standards in order to make sure that their devices work out of the box with your broken OS.

Re:Open Source to the rescue (1)

Blakey Rat (99501) | more than 4 years ago | (#31136128)

Don't forget that this "beauty of open source" doesn't even get around to thinking about fixing the issue until a competing OS has had support for it for 3 entire years.

Re:Open Source to the rescue (0)

Anonymous Coward | more than 4 years ago | (#31136060)

ah yes, the "beauty" of open source: "someone else will do it."

So don't do that... (1, Informative)

russotto (537200) | more than 4 years ago | (#31135492)

Author claims a massive performance drop if things aren't aligned right. Ubuntu already does it with parted and fdisk can do it manually. So, no big problem; fdisk ought to be fixed to have sane defaults with a 4096 byte block size, sure. That can't be all that difficult.

The author also seems to think that only a 30% increase in times for misaligned writes should be expected. I'm not sure why. In a naive implementation I'd expect a 100% increase in time (each block now needs to be written twice). Linux, obviously, doesn't use a naive implementation. It's expected that if the hardware violates the assumptions behind the techniques Linux uses to achieve high performance, that those techniques end up making things very slow instead.

Re:So don't do that... (2, Insightful)

marcansoft (727665) | more than 4 years ago | (#31135604)

fdisk doesn't need to be fixed, it needs to be deprecated. DOS partition tables are a ridiculously bad artifact of the past. We won't be using them for much longer anyway; they're limited to 2TB for 512-byte-sector drives (or 4K drives with 512-byte emulation).

Re:So don't do that... (0)

Anonymous Coward | more than 4 years ago | (#31135704)

Fdisk is so retardedly out-of-date that 4k-block alignment is pretty low on the list of worries. It's fairly easy, so it might get done, but there are much more pressing matters for fdisk -- GPT support, not assuming MBR/DOS as the fall-through label, etc. -- that 4k-blocks are pretty low on the list.

It's still Windows' fault (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31135496)

Of course fdisk/parted only follow the convention to align to "cylinder" boundaries because Windows and some of its tools will do weird things if a partition is not aligned like that. Linux doesn't have any problems with arbitrary partition offsets and lengths.

Check with your distribution (4, Interesting)

macemoneta (154740) | more than 4 years ago | (#31135536)

I know that Fedora seems to have addressed this with parted 2.1.1 [fedoraproject.org] and util-linux-ng 2.1 [fedoraproject.org] . Both are scheduled for Fedora 13, but can be pulled into Fedora 12 by those getting the hardware early.

Partitions are obsolete (1, Interesting)

Anonymous Coward | more than 4 years ago | (#31135556)

Easiest fix: stop dividing your disks into partitions.

Re:Partitions are obsolete (0)

Anonymous Coward | more than 4 years ago | (#31135674)

yep just use the whole drive (sda not sda1)

Re:Partitions are obsolete (1)

jedidiah (1196) | more than 4 years ago | (#31135868)

Nope. Read the original article. The first partition will be mis-aligned if you use the first available block rather than padding to the next 4K boundary.

You won't even know it unless you look at the partition table in expert mode.

Re:Partitions are obsolete (1)

marcansoft (727665) | more than 4 years ago | (#31136074)

What the GP is advocating is not using a partition table, instead of partitioning drives into one partition. This doesn't cause the issue mentioned in the article. This isn't a novel idea - there are some USB drives that do this already, and even Windows seems to cope fine.

partition table (2, Funny)

gd2shoe (747932) | more than 4 years ago | (#31136160)

There won't be a partition table with his suggestion. The boot sector set aside by the filesystem will be the very first sector of the disk.

Re:Partitions are obsolete (3, Insightful)

hedwards (940851) | more than 4 years ago | (#31135952)

Which is nice if you're wanting to ensure that you've got the lowest possible reliability and safety for your data. While you're at it, make sure you're using a striped non-redundant array of disks as well, best use at least 4 in the array, otherwise you might get some of your data back.

You've got it exactly backwards, people shouldn't be partitioning disks into one huge partition. They should be able to split things up a bit to keep rapidly changing directories from mostly static ones and to manage the risk of filesystem corruption destroying important files.

Oh slashdot.. (5, Insightful)

JeffSh (71237) | more than 4 years ago | (#31135576)

Dear Slashdot,

I've been around for a while. Enough to understand, nay, love the fact that you are linux supporters and all that. But I remain an ardent supporter of truth and speaking in ways which are concise and leads the reader in the direction of truth. Nothing in this news story is inaccurate, but to make it a point to say that Windows XP is incompatible with no mention of Vista and 7 being perfectly compatible should be an embarrassment of journalistic integrity.

Windows XP may not work with the new WD Green drives, but Vista and on have been perfectly comfortable with 4096 byte sectors. A lay reader may read this story and not "Read between the lines" as I have learned to do here. Their take away may be that Microsoft operating systems are broken in some way (which they are in a lot of ways), but not this one!

slashdot is not journalism (4, Insightful)

SuperBanana (662181) | more than 4 years ago | (#31135668)

should be an embarrassment of journalistic integrity.

Slashvertisements, basic English grammar and spelling problems, completely wrong summaries and titles...

...and you a)think that Slashdot is "journalism" and b)it's had integrity to lose in the first place?

I like Slashdot, but gimme a break...it's a user-driven blog which directs readers to existing stories (now often lagging behind the major news wires) with good categorization and semi-sophisticated commenting system, utilized by a larger commenter population. Not much more, and definitely not journalism.

Re:slashdot is not journalism (1)

JeffSh (71237) | more than 4 years ago | (#31135708)

Yeah, I don't disagree, maybe I'm even well aware of that, but if I had admitted that in my original post I'd have no room to indict the continued problems. I think they should aspire to be better even if Slashdot is everything you say it is.

Re:slashdot is not journalism (3, Interesting)

Blakey Rat (99501) | more than 4 years ago | (#31136092)

I'm with you, but on the other hand that doesn't mean they should just not give a shit about the quality of their end-product. We know from experience that they can edit and correct stories as corrections arise in the comments, but how often does that happen in practice? (Hardly ever.) Somewhere between a third and half of the stories posted here are either outright lies, or extremely misleading-- I may be exaggerating, but not by much-- and almost never are they corrected.

Look, any site that posts this article: http://tech.slashdot.org/article.pl?sid=09/02/16/2259257 [slashdot.org] without a single correct simply Does. Not. Give. A. Shit.

I don't think anybody's expecting the New York Times when they visit here, but some minimum level of competence would be nice. I don't fault anybody for complaining.

Re:Oh slashdot.. (0)

Anonymous Coward | more than 4 years ago | (#31135764)

I think it goes without saying that a major supplier of PC disk drives would make sure that they worked with the current edition of Windows, otherwise they stay in the labs and warehouses until the drivers are ready.

Re:Oh slashdot.. (1)

rubycodez (864176) | more than 4 years ago | (#31135788)

nonsense, I guessed (correctly it turns out) from the description that Xp and older versions of windows would have problem. But those of us who keep older computer around for windows, with no interest in going beyond Xp, aren't fretting.

You have an imagined stereotype that isn't true. Here's my stereotyping, Less than 0.1% of readers of slashdot would be "lay readers" in matters of computers so you needn't worry.

Re:Oh slashdot.. (0)

Anonymous Coward | more than 4 years ago | (#31135796)

I think you're nutso. I explicitly understood XP to mean that Vista and 7 were okay.

Re:Oh slashdot.. (0)

Anonymous Coward | more than 4 years ago | (#31135828)

Some people don't understand the Windows Community is fragmented.

Re:Oh slashdot.. (0)

Anonymous Coward | more than 4 years ago | (#31135806)

Well, there are standards for web and yet the major browser out there ignores them forcing the web to be coerced into a de facto standard. So if a new hardware arrives and breaks the de facto OS... that is newsworthy, in that most people, who are still using it, will be screwed if they want to upgrade their machine with this new hard disk.

And guess who is going to have to "support" their cries...

Re:Oh slashdot.. (1, Flamebait)

hduff (570443) | more than 4 years ago | (#31135878)

Dear Slashdot,

Nothing in this news story is inaccurate, but to make it a point to say that Windows XP is incompatible with no mention of Vista and 7 being perfectly compatible should be an embarrassment of journalistic integrity.

You are correct. It is a non-issue because nobody uses WindowsXP anymore and there are no legacy systems running it. Nor will there ever be a need to use a new drive in a WindowsXP stsem since there are no WindowsXP systems in use. You are correct.

Re:Oh slashdot.. (1)

edxwelch (600979) | more than 4 years ago | (#31135932)

Maybe Vista and 7 support it, but 70% of all PCs are still using Windows XP.

Re:Oh slashdot.. (2, Insightful)

Vellmont (569020) | more than 4 years ago | (#31136030)


  but to make it a point to say that Windows XP is incompatible with no mention of Vista and 7 being perfectly compatible should be an embarrassment of journalistic integrity.
.
.
Their take away may be that Microsoft operating systems are broken in some way (which they are in a lot of ways), but not this one!

It only takes about 3 brain cells to realize that Windows XP != All Microsoft Operating Systems. Even the average person has more than 3 brain cells.

For those people with less than 3 brain cells, Slashdot still has you covered since the article clearly says:

According to WD, their new 4096-byte sector drives are problematic for Windows XP users but not Linux or most other OSes.

(emphasis mine)
It only takes 2 brain cells to understand that "most other OSes" likely includes Vista and Windows 7.

For those unlucky few with only 1 brain cell, you're correct. Slashdot has certainly failed the 1 brain celled individual.

Re:Oh slashdot.. (1)

maxume (22995) | more than 4 years ago | (#31136044)

It isn't really even a problem under XP (as long as the drive is formatted with a tool that is aware of the new sector size, so that the logical sectors of the operating system line up with the physical sectors of the drive; Western Digital, the crazy fools, have a tool that will make sure this is the case...).

I just bought one of these (1)

xhorder (232326) | more than 4 years ago | (#31135580)

Can some kind soul tell me specfically what version of what utility I need to use for me to be OK? Or what settings?

My head hurts from trying to understand cylinders and sectors and drive geometry...

thanks!

Re:I just bought one of these (5, Informative)

King Kwame Kilpatric (1315071) | more than 4 years ago | (#31135680)

The problem is that WD doesn't tell the system about the sector size.
dev/sdd:

Model=WDC WD15EARS-00Z5B1, FwRev=80.00A80, SerialNo=
Config={ HardSect NotMFM HdSw>15uSec SpinMotCtl Fixed DTR>5Mbs FmtGapReq }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=50
BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=16

It looks to me that this should *really* be fixed by WD with a firmware update

.

Solution: Instead of fdisk, call it as fdisk -H 224 -S 56 as per Theodore Tso's blog [thunk.org] .

Re:I just bought one of these (1)

King Kwame Kilpatric (1315071) | more than 4 years ago | (#31135738)

Also see this post [wdc.com] .

Checking for alignment issues? (0)

Anonymous Coward | more than 4 years ago | (#31135586)

How can one quickly check for alignment issues?

Re:Checking for alignment issues? (0)

Anonymous Coward | more than 4 years ago | (#31136026)

Well, if you're lawful good and you come across an old lady and you slice her throat for her coin purse I would say you have an alignment issue.

if vista/win7 really do support this correctly... (-1, Troll)

buddyglass (925859) | more than 4 years ago | (#31135630)

...then I find this to be somewhat of an indictment against the open source model when applied to OS development. The default seems to be to fix fires as they arise. If there are no drives with 4k sectors then we don't need to support drives with 4k sectors. Once drives with 4k sectors arrive its up the individual maintainers of each affected tool (fdisk, et. al.) to update their code. Contrast this with a dictatorial model used by Microsoft, where they said, basically, "We know these are going to arrive sometime in the next couple years and we want to be ready when they do. So all you subsystem maintainers whose code is affected by it better build in support now, ahead of time."

Of course, if the Win7 support is crappy or only partially works, then its no indictment at all. Not having used one of these new drives in conjunction with a recent version of Windows, I can't really say one way or the other.

Re:if vista/win7 really do support this correctly. (4, Insightful)

walshy007 (906710) | more than 4 years ago | (#31135754)

The real problem is that it is lying about it's sector size, it's reporting 512 bytes when it's using 4k, if it told linux it was using 4k everything would be fine and dandy.

Why does it lie about it's sector size when it doesn't need to? because if it didn't the drives would not work on windows XP at all. Which would not bode well for sales.

Once drives with 4k sectors arrive its up the individual maintainers of each affected tool (fdisk, et. al.) to update their code.

Kernel handles sector sizes, and could handle 4k sectors ages ago, but when the hardware reports something it tends to trust it, which is now apparent it shouldn't. (512 byte sectors being implemented as an emulation layer of sorts on these drives.. and enabled by default)

Re:if vista/win7 really do support this correctly. (3, Insightful)

ArghBlarg (79067) | more than 4 years ago | (#31135996)

I see it rather as an indictment against closed-source OSes, if XP turns out to be incompatible with these new drives and MS never releases a patch to add support. People will need to upgrade for no good reason to one of MS's new operating systems. People should not have to deal with a complete upheaval of their tested and true systems due to a small hardware change such as this.

I can imagine MS is quietly chuckling with glee to itself, if this issue becomes a deal-breaker for machines still running XP.

Drive lies and future fixes (4, Interesting)

Sits (117492) | more than 4 years ago | (#31135762)

There is an excellent thread talking about how recent (2.6.31+) linux kernels try to report the underlying hard drive architecture [gmane.org] (found via the OSNews comments [osnews.com] ). Alas, it looks like some of these drives are not reporting this data correctly and thus automatic adjustment (at partitioning time) is not taking place. It looks like in the future rather than trying to do detection by reported capability fdisk (and hopefully gparted) will default to sectors of 1MiB if the topology can't be found by default [gmane.org] (unless your media is small).

Additionally, I gather that recent Fedoras will try to adjust things like LVM to match larger sectors too [storagemojo.com] . Hopefully whatever is laying out LVM will also be fixed too.

Coincidentally, it looks like Oracle have a very committed dev trying to make this stuff work by default...

Lies! (-1, Flamebait)

Anonymous Coward | more than 4 years ago | (#31135798)

These are lie because teh linux is the best!

This only effects the newer 1TB+ WD Green drives (1)

HouseOfMisterE (659953) | more than 4 years ago | (#31135864)

It appears that this does not effect the older 1TB+ Western Digital Green drives such as the WDC10EADS. Those use 333GB platters and are native 512-byte sectors. The newer (newest) Western Green drives, like the WDC10EARS, use 500MB platters and have 4K sectors. One way to tell the drives apart with a quick glance is the old Green drives had 32MB of cache and the new ones have 64MB of cache.

A minor correction... (1)

HouseOfMisterE (659953) | more than 4 years ago | (#31136016)

I posted that the newer WD Green drives use 500MB platters and I meant 500GB. 500MB platters would make for a very physically-large 1TB+ hard drive!

Linux isn't good enough to wipe my ass. (-1, Troll)

Anonymous Coward | more than 4 years ago | (#31135916)

Linux is one of the most over rated lumps of shit out there today.

I was worried about this... and am still unclear (4, Informative)

bmajik (96670) | more than 4 years ago | (#31136018)

I just got one of the 1TB 64mb WD drives that is known to be 4kb sector based.

Here is how it shows up in dmesg:
[ 3.420488] sd 1:0:0:0: [sdb] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)

and here's what hdparm -I says:
ATA device, with non-removable media
                Model Number: WDC WD10EARS-00Y5B1
                Serial Number: WD-WCAV55227529
                Firmware Revision: 80.00A80
                Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6
Standards:
                Supported: 8 7 6 5
                Likely used: 8
Configuration:
                Logical max current
                cylinders 16383 16383
                heads 16 16
                sectors/track 63 63
                --
                CHS current addressable sectors: 16514064
                LBA user addressable sectors: 268435455
                LBA48 user addressable sectors: 1953525168
                Logical/Physical Sector size: 512 bytes
                device size with M = 1024*1024: 953869 MBytes
                device size with M = 1000*1000: 1000204 MBytes (1000 GB)
                cache/buffer size = unknown
Capabilities:
                LBA, IORDY(can be disabled)
                Queue depth: 32
                Standby timer values: spec'd by Standard, with device specific minimum
                R/W multiple sector transfer: Max = 16 Current = 1
                Recommended acoustic management value: 128, current value: 254
                DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
                          Cycle time: min=120ns recommended=120ns
                PIO: pio0 pio1 pio2 pio3 pio4
                          Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
                Enabled Supported:
                      * SMART feature set
                                Security Mode feature set
                      * Power Management feature set
                      * Write cache
                      * Look-ahead
                      * Host Protected Area feature set
                      * WRITE_BUFFER command
                      * READ_BUFFER command
                      * NOP cmd
                      * DOWNLOAD_MICROCODE
                                Power-Up In Standby feature set
                      * SET_FEATURES required to spinup after power up
                                SET_MAX security extension
                                Automatic Acoustic Management feature set
                      * 48-bit Address feature set
                      * Device Configuration Overlay feature set
                      * Mandatory FLUSH_CACHE
                      * FLUSH_CACHE_EXT
                      * SMART error logging
                      * SMART self-test
                      * General Purpose Logging feature set
                      * 64-bit World wide name
                      * {READ,WRITE}_DMA_EXT_GPL commands
                      * Segmented DOWNLOAD_MICROCODE
                      * Gen1 signaling speed (1.5Gb/s)
                      * Gen2 signaling speed (3.0Gb/s)
                      * Native Command Queueing (NCQ)
                      * Host-initiated interface power management
                      * Phy event counters
                      * NCQ priority information
                                DMA Setup Auto-Activate optimization
                      * Software settings preservation
                      * SMART Command Transport (SCT) feature set
                      * SCT Features Control (AC4)
                      * SCT Data Tables (AC5)
                                unknown 206[12] (vendor specific)
                                unknown 206[13] (vendor specific)
Security:
                Master password revision code = 65534
                                supported
                not enabled
                not locked
                not frozen
                not expired: security count
                                supported: enhanced erase
                204min for SECURITY ERASE UNIT. 204min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 50014ee203bda3eb
                NAA : 5
                IEEE OUI : 0014ee
                Unique ID : 203bda3eb
Checksum: correct

Why is it lying to the linux kernel and insisting that it is a 512byte device? That's irritating.

In any case, this disk has no fdisk-style partitions on it at all... i made the whole device into LVM space using a basic/naive pvcreate command. So I have no idea [and no idea how to check] if I am misaligned or not.

The machine is ubuntu 9.10.

Anyone have any notion of how to check? fdisk and parted of course say there is no partition table at all, and pvdisplay says nothing about underlying alignment.

I have such a problem (1)

orkysoft (93727) | more than 4 years ago | (#31136042)

I have a tiny 1.8" usb harddisk with 4096-byte sectors, and the Ubuntu installer crashes when it tries to read the partitioning information. Very annoying.

DragonFly's solution (4, Interesting)

m.dillon (147925) | more than 4 years ago | (#31136120)

We're adjusting our disklabel64 utility and kernel support to set the partition base offset such that it is physically aligned instead of slice-aligned, and we are using 32K alignment. That should fix the problem without having to mess around with fdisk.

The DragonFly 64-bit disklabel structure uses 64-bit byte offsets instead of sector addressing to specify everything. It ensures things are at least sector aligned but we wanted to make disk images more portable across devices with potentially different sector sizes. The HAMMER fs uses byte-granular addressing for the same reason, 16K aligned.

-Matt

Poorly researched article. (4, Insightful)

Vellmont (569020) | more than 4 years ago | (#31136190)

The article represents one data point, for one particular way to install a drive, on one (un-named) version of Gentoo, on one particular model of a WD drive that had a bugzilla entry entered by the author all of 2 days ago. So this is supposed to be an indictment of all of Linux?

The author even mentions that Ubuntu has an option on parted that accomplishes the task properly. I'd be much more interested in an article that talks about how the default installer handles this task rather than concentrating on one particular expert tool that does so. It's still good to know that fdisk on his un-named Gentoo distribution does the wrong thing.. but this hardly means we should fire up the klaxon and declare "Linux not fully prepared for 4096 sector hard drives!". It's certainly interesting, but I'll withhold judgment until we actually know more about the implications of this across the entire spectrum of Linux distributions and the various 4096 sector HDs.

Load More Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...