Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Comments

top

The State of ZFS On Linux

Vesvvi Re:Unfamiliar (366 comments)

So "p" is the probability of a drive being down at any given time. A hard drive takes a day to replace, and has a 5% chance of going dead in a year. A given hard drive has a "p" of ~1.4e-4.

For RAID6 with 8 drives, you can drop 2 independent drives: failure = 1.4e-10. It's out in the 6+ nines.

It would take 6x sets of mirrors to get the same space. Each mirror has a failure probability of (p^2), 1.9e-8. Striped over the mirrors, all sets have to stay active: success = (1-p^2)^6, failure = 1.1e-7. Way easier to calculate without the binomial coefficient, by the way.

Technically, the mirrors are 3 orders of magnitude more likely to fail, but the odds are still ridiculously good. Fill a 4U with 22 drives (leave some bays for hot-swap) as mirrors and it's failure = 2e-7. Statistically, neither of these is going to happen: you just won't see two drives happen to go down together by random chance.

People already know this. There are much more advanced models that account for the what-happens-next situation after you've already lost a single drive, and of course it non-linearly worse. But just to keep it simple, going back to the naive model, for the RAID6 with 7 remaining drives, the failure probability is now up to 4e-7 during the re-silver time. The mirror model stays at a "huge" failure = 1.4e-4 during a re-silver, but it's brief, predictable, and with low system impact. It's my stance that that kind of probability keeps it in the category of less-important compared to many other factors for a risk analysis.

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:above, below, and at the same level. ZFS is eve (366 comments)

Sorry, I'm not that familiar with OpenSolaris.
Don't the first and second commands create a zpool backed by a file? That's not what's at question here, I want to know if you can back a zpool with a zvol created on that same zpool.

A quick test showed that it does work on FreeBSD to create a zpool upon a zvol from a different zpool. The circular version has made it hang for a not-insignificant amount of time...

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:Unfamiliar (366 comments)

That's a nice writeup.
I'm sure you've chosen that configuration for a reason, but I think it's a good example for why stripes over mirrors can be a better choice for some applications

You are running raidz2(7x4TB)+raidz2(8x2TB). Let's say that instead it was 3x(mirror(2x4TB))+4x(mirror(2x2TB)). Your capacity is 32TB as-is, or 20TB as mirrors: obviously that's a huge loss, and factoring in heat/electricity/performance/reliability it's likely that the raidz is a good choice for a home setup. Bandwidth would also be more that sufficient for home use.

But as you mention, the upgrades either take forever (one drive at a time) or require ridiculous free ports (add 7x at once?!). Even if you were to do them all at once, it would still be a fairly slow process with a massive performance hit.
On the other hand, with mirrors you can increase capacity 2 drives at a time, and at that level it's reasonable to leave both drives active as part of the "mirror" (now, 4-way) for some time. This is my preferred approach: new drives get added to a mirror set and run along with the system for a month or two. This stress-tests them, and if any point there are warning signs the drives can be dropped out immediately. If all is good after the test period, the old 2x of the mirror are removed and the space is immediately available (autoexpand=on). The process can then be repeated. Overall it takes as much or more time than your approach, but the system is completely usable during that time with no real performance hits, and of course the overall system performance is substantially improved with the equivalent of 7 devices running in parallel instead of 2.

There are definitely situations in which raidz2/3 makes more sense than mirrors, but if you're regularly expanding or looking for performance, I think the balance favors mirrors.

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:above, below, and at the same level. ZFS is eve (366 comments)

Have you confirmed using a zvol underneath a zpool, and if so was it a different zpool?
I've wanted to do that in the past, but it was specifically blocked. It's a pretty ugly thing to do, but it does give you a "new" block device that could be imported as a mirror on-demand. With enough drives in the zpool, that new device is nearly independent from its mirror, from a failure perspective.

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:production ready? (366 comments)

Is it a problem to add them by ID? I intentionally use partition IDs because they're stable, and it works well in both Linux and FreeBSD, but the FreeBSD people seem to like labels or raw device names.
Regardless, every import should bring the same pool online in the same way, regardless of the device names.

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:I agree... (366 comments)

Maybe your ZIL comments are specific to Linux? It used to be the case in FreeBSD that you had to have the ZIL present to import, and a dead ZIL was a very big problem, but that was many versions ago (~3-4 years?). I personally went through this when I had a ZIL die and the pool was present but unusable. I was able to successfully perform a zpool version upgrade on the "dead" pool, after which I was able to export it and re-import it as functional without the ZIL.

Note that this was NOT a recommended sequence of operations, and I wouldn't suggest it unless you have no choice.

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re: Working well for me (366 comments)

Not all Adaptec controllers are supported by FreeBSD. It would be a "safer" choice to use LSI, since they work great in Linux and FreeBSD: that gives you the option to migrate your host OS should you desire.

Admittedly, if you're changing over that much then buying new controllers isn't a big deal, but I like to have the option of having the "reference" implementation of ZFS just a few minutes away.

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:Unfamiliar (366 comments)

If you want to add 4 more TB, then you attach a new set of mirror, and you're left with RAID6(12x)+RAID1(2x). There is zero rebalancing (for better or worse): it's available immediately and transparently. The only catch is that you can't remove it again, but you can replace it with any combination of storage that provides equal or greater capacity to your RAID1(2x).

You could also grow your RAID6, and it's more efficient that it would be on most normal hardware RAID. But please don't do that: RAID5/6 really should be phased out, and it's not a good idea to create huge RAIDZ groups, even as RAIDZ2+. If you really want to stick with RAID5/6, it's better to just make a new group: leave your RAID6(12x) and add another RAID6(n

about two weeks ago
top

The State of ZFS On Linux

Vesvvi Re:above, below, and at the same level. ZFS is eve (366 comments)

I think you're giving the wrong idea here. I have yet to find a format of storage capacity that zfs won't support, with one exception: you can't create a zvol on a zpool, then attach that zvol as back-end storage for the same zpool. That is specifically disallowed, and I'm guessing that you can't use a zvol from one zpool to back-end another zpool either. This is a very bizarre (also, probably dumb) thing to do, but even this can be overridden if you're really desperate. For more practical applications, everything else just works: at least in FreeBSD, you can "hide" the block devices behind all different kinds of abstractions to provide 4k writes, encryption, whatever, and zfs will consume those virtual block devices just fine.

about two weeks ago
top

Climate Change Skeptic Group Must Pay Damages To UVA, Michael Mann

Vesvvi You're both absolutely, painfully correct. (497 comments)

This is the saddest part about the AGW debate. From my viewpoint, it looks like the pro-AGW people pushed back against criticism overwhelming their "opponents" with data and consensus, and tried to extinguish them via marginalizing them.

That was absolutely the worst thing that could have been done, for the reasons you note above. On the other hand, if they had embraced and extended, the whole debate could have been extinguished.

We could be working with alternate energy sources for reasons of dominance in international trade, national security / energy independence, etc. Instead, we've actively been pushed backward by the the pro-AGW agenda, and they deserve some of the blame for that.

about 2 months ago
top

How Did Those STAP Stem Cell Papers Get Accepted In the First Place?

Vesvvi Re:The same way many global warming papers got pub (109 comments)

I believe you are misreading the previous post. What they are saying is that political and financial pressure has an impact on scientists. It was not a comment on politics/finance vs science.

about 3 months ago
top

Oklahoma Moves To Discourage Solar and Wind Power

Vesvvi Re:But They Sell at Wholesale (504 comments)

They only make a profit if they can sell it or store it at the time it is being generated. When the demand dips low enough (a substantially non-zero number, given technical and business overheads), they start losing money.

about 4 months ago
top

Mathematicians Devise Typefaces Based On Problems of Computational Geometry

Vesvvi Re:zeroes (60 comments)

I really wish I had mod points for this.

about 5 months ago
top

Stanford Team Tries For Better Wi-Fi In Crowded Buildings

Vesvvi Re:and how do they track users across muilt units? (43 comments)

I assume this is something like Ubiquiti's "zero-handoff" system, where the wireless APs coordinate to form an virtual network that spans across all the APs. I'm not a networking pro, but I guess you could say it's like the opposite of a VLAN? Although based on the article, it sounds like there are effectively VLANs running on their mesh to partition it into per-user virtualized segments. In that situation, there is nothing that prevents you from having a static or even public IP.

about 6 months ago
top

Ask Slashdot: What's the Most Often-Run Piece of Code -- Ever?

Vesvvi Re:Solved. Next? (533 comments)

You could break it down further to look at the lower-level "operations" being performed.

In order to see if the next amino acid should be a glutamine instead of an asparagine, each of the two are going to be bounced into the active site to see if they "fit": if they fit, they're incorporated and if not, they bounce out and the next one is tried.

So you could think of this as an if, else repeated really ridiculously quickly: at the speed of molecular diffusion, minus just a tiny bit. That makes it both extremely fast and extremely parallel, as you observe.

As a side note, nearly ever molecular interaction operates in this way, unless a substrate is literally "handed-off" between two enzymes. That would definitely make a molecular recognition operation the most highly "executed" routine (at least down to that scale) by untold orders of magnitude.

about 8 months ago
top

OpenSSH Has a New Cipher — Chacha20-poly1305 — from D.J. Bernstein

Vesvvi What about HPN? (140 comments)

This is all well and good, but what's the status of seeing HPN-SSH or similar incorporated? FreeBSD has incorporated it, but it's still messy on Linux systems.

about 9 months ago
top

Storing Your Encrypted Passwords Offline On a Dedicated Device

Vesvvi Re:if you can access it on a website (107 comments)

It doesn't specifically solve any of those problems (except forbidden punctuation mark), although it simplifies them a bit.

Required characters (uppercase, punctuation, numbers) can be added post-hash as an insecure suffix to meet site requirements. These don't add any security, so you can carry them around with you, put them on a public website, or leave them on a sticky note on your monitor: "work suffix: #U1_. Github suffix: (#$JHi/."

The same thing can be said for length issues, although I've found that most systems these days are happy with 16. Admittedly, with the character set restriction it would be better to keep it long, but I would argue that by avoiding sending plaintext to the servers, we're avoiding the vast majority of vulnerabilities.

Expiration is made more simple by making it easier to remember passwords: changing one isn't a big deal. This continues onto your next point as well: you'll never have an error message that your new password is too similar to your old one.

I think your last shows another benefit of terminating private passwords as soon as possible. On insecure hardware, your public (hashed) password is exposed, and of course it could be captured for future use. But that will limit exposure to a single service, and it won't reveal any hints about your password trends.

You actually overlooked the most important point: if we're hashing passwords on a secured and user-controlled device, it's very easy (space-, energy-, speed-efficient) to get the public/hashed passwords off (LCD), but it's still a bit annoying to get the private passwords onto the device. UI concerns are a problem: I can do it extremely fast an efficiently if I'm working on a desktop, but it's a bit slower even on a tablet. The further we go towards hardware which can be fully locked down (keyfob with a single chip), the harder it is to get the data onto the device.

about 9 months ago
top

Storing Your Encrypted Passwords Offline On a Dedicated Device

Vesvvi Re:if you can access it on a website (107 comments)

I don't understand why there is so much effort placed on storing passwords. We already know what to do with passwords from the perspective of the server: discard them as soon as possible!

The password should be salted and hashed immediately, and it should never be stored in plaintext. So let's not store them at all: let the user remember the risky password, and encrypt it as soon as possible. It's a validated methodology, and it removes many/most of the trust issues of the user/server relationship: I don't care if the server fails to salt my password if it's already encrypted.

Now take this to the next step. The user-side "passwords" can be pretty weak, since they need to be memorable but not high-entropy. We don't want to re-use the same "password" everywhere (different sites/services), since that's a risk, but we can come up with a weak per-site salt that's easy to remember. Combine that with a relatively weak password and we have a winner

Use-everywhere password: invsqrt
Site: slashdot.org. "Salt": modmadness. Full password: invsqrtmodmadness
hashlib.sha256(getpass.getpass()).hexdigest()[::2][:16]
Password sent to server: "dee4ea048518f588"

Use-everywhere password: invsqrt
Site: stackexchange.com. "Salt": xyproblem. Full password: invsqrtxyproblem
hashlib.sha256(getpass.getpass()).hexdigest()[::2][:16]
Password sent to server: "be6065c67f055583"

Yes, I know it's just a hash, but this is a simple example. There's some loss of strength from key vs hash lengths, re-using "passwords" etc, and I've thrown in some complication, but I think the general idea is sound. The most important fact is that insecure, memorable, secret information never leaves my brain. Ok, in practice it does: I enter it onto an offline encryption device, but it never goes anywhere else.

  • There is no private key to lose.
  • I don't have to store private information.
  • The public-side "passwords" are high-entropy and pseudo-random.
  • The user-side "passwords" are highly memorable.
  • An offline encryption device adds security, but it isn't necessary: in an emergency I can generate hashes nearly anywhere, since I carry my secure passphrases around with my in my brain.

You can stack additional levels of complication to make it more robust, but even the crudest implementation put you in the top 0.01% of hardest-to-crack passwords. For example, your encryption fob can contain a private key: smash the fob and you have securely destroyed the ability to re-create passwords. It also would make the outgoing passwords much more secure.

about 9 months ago
top

A Protocol For Home Automation

Vesvvi Please don't re-invent the wheel. (116 comments)

Please don't re-invent the wheel unless you need to. By that, I mean to say that automation and interconnection of "gadgets" is a well-established field in industry and tech. For example, vehicle ECU and sensor systems, factory automation, and data acquisition systems are all now decades old, and we should have a really solid idea of how to do these things properly.

Of course these existing systems aren't the same as what we're talking about here, with modules that span different physical link layers, protocols, etc. I just hope that we can take the best lessons from existing "gadget integration" attempts to make forward progress more successful and not just something doomed to rapid obsolescence.

For some fun and background, have a look at the old HPIB/GPIB physical/protocol standard (http://en.wikipedia.org/wiki/IEEE-488), which was used in many different pieces of scientific equipment. When that somewhat died out it was replaced by CAN (http://www.team-cag.com/support/theory/chroma/hplc_bas_at/system/cableConnections.html). Agilent uses that for their HPLCs (maybe test equipment, too?), and Waters uses the same physical link, but with a different protocol? Other vendors still work with contact-closure, and USB is becoming more popular, but that pushes so much onto the host computer and really enforces lock-in.

I will personally be watching this closely from the perspective of someone who operates a lot of data-acquisition equipment. Could this be the foundation for better interop between different vendors at the more commercial/research level, in addition to the consumer? I hope so.

about a year ago
top

DNA Sequence Withheld From New Botulism Paper

Vesvvi Re: Right move (182 comments)

It looks you are correct: the costs have dropped low enough that it's feasible to just make it synthetically. I guess that really shouldn't be surprising, given that this is basically a "parallelizable" technology.

about a year ago

Submissions

Vesvvi hasn't submitted any stories.

Journals

Vesvvi has no journal entries.

Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>