Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

DragonFlyBSD 3.6 Brings AMD/Intel Graphics Drivers & Better SMP Scaling

timothy posted about 8 months ago | from the not-just-hovering-there dept.

Operating Systems 48

An anonymous reader writes "DragonFlyBSD 3.6 was released [Monday] with the big new features being dports, Intel and AMD Radeon KMS kernel graphics drivers, major SMP improvements, and improved language support. Dports is the new package management system based upon the FreeBSD Ports collection and replaces pkgsrc as the default; over 20k packages are available via dports. Major SMP scaling improvements come via reducing lock contention within the kernel and other multi-core enhancements. The Intel and Radeon graphics drivers on DragonFlyBSD were ported from the FreeBSD kernel, which in turn were ported from the upstream Linux kernel."

cancel ×

48 comments

All I can say to that is... (-1)

Anonymous Coward | about 8 months ago | (#45526371)

who?

Re:All I can say to that is... (1, Funny)

Anonymous Coward | about 8 months ago | (#45526553)

who?

It's a DOS clone that thinks it's still relevant. These losers should just move to Windows and stop clinging to text-mode DOS clones. The worst part is it isn't even binary compatible with old DOS games making it doubly irrelevant.

Re:All I can say to that is... (0)

Anonymous Coward | about 8 months ago | (#45526791)

If it can't run Excel, it's pretty useless to us. I have a ton of VBA code that needs to just work - I don't have time to dick around with obscure OS's.

Re:All I can say to that is... (0)

Anonymous Coward | about 8 months ago | (#45573097)

How your opinion correlates with Windows application and server OS?

Still a useless hobby project... (-1)

Anonymous Coward | about 8 months ago | (#45526393)

... that nobody but its developers should run. If you want BSD, FreeBSD (or even OpenBSD) are just better for tinkering and furthering BSD development. DragonFly is just a hobby me-too effort.

Re:Still a useless hobby project... (1)

Tarlus (1000874) | about 8 months ago | (#45530841)

So are the vast majority of Linux distributions, but it sure is nice to have options, isn't it?

Comparison (0)

Anonymous Coward | about 8 months ago | (#45526407)

Does anyone know if a three way performance comparison between Freebsd, Dragonfly and Linux has been done lately?

Re:Comparison (1)

dreamchaser (49529) | about 8 months ago | (#45526489)

You might find this interesting [freebsd.org] . It compares several versions of BSD with Linux.

Re:Comparison (1)

epyT-R (613989) | about 8 months ago | (#45526823)

I am sure I will find an unbiased report about toyotas on honda.com too. Seriously, those graphs suggest a hard limiter has been hit, like a settable sysctl/kernel knob, rather than a serious architectural deficiency.

Re:Comparison (1)

Desler (1608317) | about 8 months ago | (#45527095)

Have anything that isn't 5 years old?

Re:Comparison (1)

ZorkZero (6507) | about 8 months ago | (#45530171)

Ummm, that's from March 2008. That's over five and a half years ago.

Re:Comparison (1)

nurb432 (527695) | about 8 months ago | (#45532549)

Ya, in 2008, which makes it totally worthless.

Timothy's favorite distro (2, Interesting)

Sarten-X (1102295) | about 8 months ago | (#45526443)

Has anybody else noticed that over the last decade, almost all of the DragonFlyBSD release stories have been posted by timothy, and the majority of those were submitted anonymously?

It's not a particularly popular distro, coming in at #77 [distrowatch.com] today, in an unscientific poll. I get that it's news for nerds, but I'm starting to suspect a wee bit of bias.

Re:Timothy's favorite distro (0)

Anonymous Coward | about 8 months ago | (#45526579)

> It's not a particularly popular distro, coming in at #77 [distrowatch.com] today

Versus SuSE (SLE) at #78, OpenBSD at #72, and NetBSD at #85.

Re:Timothy's favorite distro (1)

Sarten-X (1102295) | about 8 months ago | (#45526657)

And DSL at #60, Tiny Core at #50, and PC-BSD at #40.

I did say it's unscientific, but my point stands. We don't get release notices for most of the more-popular distros, so the promotion of DragonFly seems suspicious.

Re: Timothy's favorite distro (0)

Anonymous Coward | about 8 months ago | (#45527219)

It's an operating system NOT a distribution of some other BSD OS.

Re:Timothy's favorite distro (3, Interesting)

geek (5680) | about 8 months ago | (#45526857)

Of the BSD's DragonFly is probably one of the more important and active ones. The filesystem they use is one of the biggest advancements in BSD in a long time. I'm not a DragonFly user but I do track it in hopes of one day using it at a point that it fits my needs. FreeBSD 10 however is addressing all of my concerns so DragonFly may slip away from my sites soon.

Re:Timothy's favorite distro (0)

Anonymous Coward | about 8 months ago | (#45530165)

DragonFly may slip away from my sites soon.

Web sites? Work sites? Or did you mean "sights"? You're not being clear there...

Re:Timothy's favorite distro (1)

Anonymous Coward | about 8 months ago | (#45527409)

You might note that about half of bsd.slashdot.org's [slashdot.org] stories are from anonymous readers. It's just that this site has grown hostile towards the BSDs, as evidenced by many BSD vs GPL threads which degenerate into strawmen and insults. They probably have driven out many BSD-using folks here, or at least made them not bother with registration.

(Try to read the comments posted on those bsd.slashdot.org stories. The signal-to-noise ratio is dismal.)

Oh, and I was one of those anonymous submitters for a past DFly release announcement. And I'm not Timothy in disguise, in case you're wondering.

Anyway, these announcements are more interesting than the stories of the Linux kernel getting a RC release, or the biyearly Ubuntu stories that get silly threads with everyone deciding to discuss Unity and nothing but Unity.

Re:Timothy's favorite distro (0)

Anonymous Coward | about 8 months ago | (#45531145)

They probably have driven out many BSD-using folks here, or at least made them not bother with registration.

Yes, the people in the BSD camps never starts an argument, and have always been polite and nice towards Linux users. It couldn't possibly be a case of you reap what you sow.

Re:Timothy's favorite distro (1, Informative)

bluefoxlucid (723572) | about 8 months ago | (#45528019)

DragonflyBSD is the only interesting BSD. Minix is the other interesting Unix-like. Dragonfly has a whole hell of a lot of novel concepts or novel implementations of concepts: HAMMERFS (runs 30 second snapshots for 24 hours, then 1 day snapshots, then 1 week, 1 month--a versioning file system, semi-useful), checkpointing with freeze/thaw (you can actually freeze an application and reboot, to the point that you can even move the application to another machine running Dragonfly with the same files at the same paths and thaw the application, continue running it as if you just ran sigstop/sigcont), and extremely good scheduling on SMP that scales to thousands of processors and threads way better than FreeBSD (unsure on how it compares to Linux).

Minix of course is obvious: It's a fully fault-tolerant self-healing microkernel.

I would enjoy seeing the Linux extensions (events, iptables, filesystems, etc.) rolled into Minix so that udev, dbus, systemd, and basically a straight Linux distro stack work without modification; and then seeing the DragonFlyBSD scheduling concepts and freeze/thaw checkpointing implemented in Minix, with the tools to call checkpointing ported from DragonFlyBSD. Then we could drop the whole thing straight into a Linux distribution like Ubuntu or Fedora and play with it in a VM, and it would work as expected. Obviously you'd need to port tons of drivers to get a usable desktop system; but to get "you can run Linux on this now", you just need to port some Linux-specific subsystems onto Minix and run it in VMware or KVM or VirtualBox. Straight direct comparisons can then be made.

Single System Image clustering (1)

Jizzbug (101250) | about 8 months ago | (#45528427)

The cool checkpointing and process migration features are part of DragonFly's goal of providing seamless Single System Image clustering some day... A feature and goal that I highly respect and find cool-as-hell, having been an early user of MOSIX and openMosix back in the day.

Re:Timothy's favorite distro (1)

unixisc (2429386) | about 8 months ago | (#45529629)

I think an interesting project might be to have pFsense or m0n0wall redone using Minix, rather than BSD as the kernel. With full IPv6 support

BSD Fragmentation (1)

n1ywb (555767) | about 8 months ago | (#45526527)

You know, I hear a lot of folks complain about Linux fragmentation, tyrany of choice, etc. But at least we can say that, for the most part, there is one true canonical Linux kernel (Linus' tree) and all the other kernels are for the most part shallow forks tweaking a few things.

Now in BSD land we have NetBSD, FreeBSD, OpenBSD, and DragonflyBSD, each with their own true kernel.

Why?

If the project goals have diverged so widely as to take the kernel off in a completely different direction from all the other BSD's why even call it BSD anymore?

What do the four big BSD distros have in commmon besides the name and a kernel they used to use years (decades?) ago?

I am admitedly ignorant and perhaps I am underestimating the degree of cooperation between these projects.

Re:BSD Fragmentation (0)

Anonymous Coward | about 8 months ago | (#45526633)

Well, let's think about it.

1. The license and copyright. 2. The legacy. 3. The userland. 4. The overall quality. 5. Not so many prima donna devels (Theo excluded).

Although the userlands have diverged, there's still a great deal of commonality.

Re:BSD Fragmentation (-1)

n1ywb (555767) | about 8 months ago | (#45526787)

Well, let's think about it.

1. The license and copyright. 2. The legacy. 3. The userland. 4. The overall quality. 5. Not so many prima donna devels (Theo excluded).

Although the userlands have diverged, there's still a great deal of commonality.

1. I'll give you the license but they obviously no longer share much copyright as they have all been largly rewriten since the 386bsd days. If that were not true then they would all be the same.

2. That's like saying nothing differentiates us from monkeys because we share a common ancestor. I call BS. A common origin implies very little of practical value especially considering how much of the code has been rewritten.

3. So first you say they share a user land then you say the userlands have diverged. Which is it?

4. Overall LACK of quality due to not having a critical mass of developers, you mean?

5. I think they're ALL prima donna devs otherwise why are there so many forks? It's because everybody in the BSD camp wants to take their ball and go home instead of working together on a common goal.

I don't hate BSD or wish anybody ill. These are just my perceptions based on what I've seen.

Re:BSD Fragmentation (1)

Brian Feldman (350) | about 8 months ago | (#45527251)

I don't hate BSD or wish anybody ill. These are just my perceptions based on what I've seen.

No, they're not. They're your perceptions based upon what other Linux trolls have posted on Slashdot. Grow up.

Re:BSD Fragmentation (1)

unixisc (2429386) | about 8 months ago | (#45527453)

What exactly is different b/w the different BSD licenses? Can someone describe in plain terms?

Re:BSD Fragmentation (2, Interesting)

Anonymous Coward | about 8 months ago | (#45527801)

The only real difference is the level of bitterness felt about the overwhelming success of the supposedly "anti business" and "anti freedom" linux project and the GPL license it's published under.

I think it's safe to say at this point the GPL was the right choice. Linux flourishes today with enormous growth and a huge community. BSD is still around, and it can claim a lot of success.. But, exactly as everyone feared, the big users of BSD don't contribute their improvements back to the main project. Without this, the BSD community stagnates in the relative obscurity it's been facing for the past decade or so. So there's BSD in the playstation3 and 4. So? What good does that do other than to fuel the egos (and possibly the paychecks) of a few developers. The community sees nothing.

RMS is one of those guys that's annoyingly, uncomfortably, maddeningly correct. He's rude, creepy looking, and hard to get along with. He doesn't sugar coat anything or are about anyone's ego or feelings. Much community may not like what he does or what he has to say, but he's been there since the start and he's always right in the end. He's like the Socrates of computing.

Re:BSD Fragmentation (1)

fuzzyfuzzyfungus (1223518) | about 8 months ago | (#45526683)

How widely do the various BSD kernels actually diverge?

It is true that Linux has resisted fragmentation pretty well, with the exception of the various fossilized versions usually present at roughly 80% of the completeness that GPL compliance demands in the unending waves of shitty BSPs for assorted hardware, may they sink into hell. That is the 9th circle of hell when it comes to 'Linux fragmentation'. x86 is downright civil, by contrast, with most of the fragmentation, real and alleged, occurring above the kernel level (the various x86 distro kernels aren't pure kernel.org tree; but they tend to track it pretty closely, mostly either trailing a bit to tame its more unpredictable tendencies, or incorporating specific pet features that haven't been mainlined yet)

Do the BSDs actually differ much more substantially from one another than the various Linux distros do? To the degree that they may differ more, do they represent major duplications of effort, or are most of the areas of variation areas that are central to their missions (eg. I'd expect that OpenBSD would have an abnormally well-developed chunk of code around PF, and certain security-critical features, with comparatively vestigial and/or stock FreeBSD components elsewhere, that would seem logical in line with their objectives; but something like one of the BSDs marching off into the weeds with its own driver model would seem deeply unsustainable.

Re:BSD Fragmentation (2)

bluefoxlucid (723572) | about 8 months ago | (#45528045)

Dragonfly BSD is substantially different from FreeBSD in that it's not using the same fundamental memory management or task scheduling strategies.

Re:BSD Fragmentation (1)

fuzzyfuzzyfungus (1223518) | about 8 months ago | (#45528845)

Forgive my ignorance; but how modular are memory management and task scheduling relative to what would seem (to my admittedly untrained eye) to be the two big areas where fragmentation would be a killer: kernel drivers and applications/DEs/etc. (including those of significant complexity, not just Hello World).

If Dragonfly can still use FreeBSD drivers, and run common Linux and BSD applications, they may or may not have a particularly good memory manager or task scheduler; but if somebody wants to go off and pursue his vision of memory and task management, good for him.

If it cuts compatibility with either of those, it might as well join Minix running in a few VMs of antiquarians; because you are going to have a hell of a time without a critical mass to keep your drivers and your userspace more or less current. If it doesn't, there's plenty of room for quibbling about a desktop vs. server priorities; but modularity helps you out.

Re:BSD Fragmentation (1)

bluefoxlucid (723572) | about 8 months ago | (#45540483)

It's significantly critical that drivers are either incompatible or sub-optimal if migrated from one base to another. In theory this can be separated; in practice, monoliths don't separate in that way.

Re:BSD Fragmentation (1)

unixisc (2429386) | about 8 months ago | (#45527425)

Generally, FreeBSD seems be the popular face of BSD, just like Red Hat is of commercial Linux. OpenBSD & NetBSD seem to have a relatively low base in comparison, while the others, such as DragonFly are even smaller. Also, most of the BSD clones - GhostBSD, MidnightBSD, et al are FreeBSD based, as opposed to the other 2.

OTOH, Minix is its own (micro)kernel and uses NetBSD userland, as opposed to FBSD. Interesting.

Huh (1)

Vince6791 (2639183) | about 8 months ago | (#45527397)

It's a fork of FreeBSD 4.8 from 2003. I ran the most recent freebsd and pc-bsd and both are rock solid. It was easier to install flash player on bsd than opensuse 13.1(it did not work even installing from the terminal). I ran linux open source applications just fine with no issue on pc-bsd 9.2.

Re:Huh (1)

unixisc (2429386) | about 8 months ago | (#45528955)

If FBSD/PC-BSD simply added Hammer/2 support, wouldn't that make DragonFly instantly irrelevant? B'cos then it would just be a fork of FBSD 4.8 & nothing more

Re:Huh (1)

jones_supa (887896) | about 8 months ago | (#45529143)

The desktop experience under Linux is still better than of PC-BSD. I did some experimenting with PC-BSD and everything had a bit crusty and unpolished feeling. The robustness of BSDs is of course very good thing, but I need a good desktop too. Of course the PC-BSD guys have made installing the desktop much more easier, so they still get a plus from me.

site hacked? someone check the features page (1)

Gothmolly (148874) | about 8 months ago | (#45527745)

Check http://www.dragonflybsd.org/features/ [dragonflybsd.org] , there's a list of random links under the DEVFS bullet point.

Re:site hacked? someone check the features page (1)

Gothmolly (148874) | about 8 months ago | (#45529731)

And now they're gone, I guess someone over there read my post.

SMP contention basically gone from critical paths (5, Informative)

m.dillon (147925) | about 8 months ago | (#45528079)

This release removes almost all the remaining SMP contention from both critical paths and most common paths. The work continued past the release in the master branch (some additional things which were too complex too test in time for the release). For all intents and purposes the master branch no longer has any SMP contention for anything other than modifying filesystem operations (such as concurrent writing to the same file). And even those sorts of operations are mostly contention free due to the buffer cache and namecache layers.

Generally speaking what this means is that for smaller 8-core systems what contention there was mostly disappeared one or two releases ago, but larger (e.g. 48-core) systems still had significant contention when many cores were heavily resource-shared. This release and the work now in the master branch basically removes the remaining contention on the larger multi-core systems, greatly improving their scaling and efficiency.

A full bulk build on our 48-core opteron box took several days a year ago. Today it takes only 14.5 hours to build the almost 21000 packages in the FreeBSD ports tree. These weren't minor improvements.

Where it matters the most are with heavily shared resources, for example when one is doing a bulk build on a large multi-core system which is constantly fork/exec'ing, running dozens of the same process concurrently. /bin/sh, make, cc1[plus], and so on (a common scenario for any bulk building system), and when accessing heavily shared cached filesystem data (a very common scenario for web servers). Under these conditions there can be hundreds of thousands of path lookups per second and over a million VM faults per second. Even a single exclusive lock in these paths can destroy performance on systems with more than 8 cores. Both the simpler case where a program such as /bin/sh or cc1 is concurrently fork/exec'd thousands to tens of thousands of times per second and the more complex case where sub-shells are used for parallelism (fork without exec)... these cases no longer contend at all.

Other paths also had to be cleaned up. Process forking requires significant pid-handling interactions to allocate PIDs concurrently, and exec's pretty much require that locks be fine-grained all the way down to the page level (and then shared at the page level) to handle the concurrent VM faults. The process table, buffer cache, and several other major subsystems were rewritten to convert global tables into effectively per-cpu tables. One lock would get 'fixed' and reveal three others that still needed work. Eventually everything was fixed.

Similarly, network paths have been optimized to the point where a server configuration can process hundreds of thousands of tcp connections per second and we can get full utilization of 10GigE nics.

And filesystem paths have been optimized greatly as well, though we'll have to wait for HAMMER2 to finish that work for modifying filesystem calls to reap the real rewards from that.

There are still a few network paths, primarily related to filtering (PF) that are serialized and need to be rewritten, but that and the next gen filesystem are the only big ticket items left in the entire system insofar as SMP goes.

Well, the last problem, at least until we tackle the next big issue. There's still cache coherency bus traffic which occurs even when e.g. a shared lock is non-contended. The code-base is now at the point where we could probably drop-in the new Intel transactional instructions and prefixes and get real gains (again, only applicable to multi-chip/multi-core solutions, not simple 8-thread systems). It should be possible to bump fork/exec and VM fault performance on shared resources from their current very high levels right on through the stratosphere and into deep space. Maybe I'll make a GSOC out of it next year.

The filesystem work on HAMMER2 (the filesystem successor to HAMMER1) continues to progress but it wasn't ready for even an early alpha release this release. The core media formats are stable but the freemap and the higher level abstraction layers still have a lot of work ahead of them.

In terms of performance... well, someone will have to re-run bechmarks instead of just re-posting old stuff from 5 years ago. Considering the SMP work I'd expect DFly to top-out on most tests (but there's still always the issue of benchmark testers just blindly running things and not actually understanding the results they post about). Database performance with postgresql still requires some work for large system configurations due to the pmap replications (due to postgres fork()ing and using mmap() now instead of sysv-shm, e.g. if you used a large 100GB+ shared memory cache configuration for the test). We have a sysctl to enable page table sharing across discrete fork()s but it isn't stable yet... with it, though, we get postgres performance on-par with the best linux results in large system configurations. So there are still a few degenerate cases here and there that aren't so much related to SMP as they are to resource memory use. But not much is left even there.

Honestly, Slashdot isn't the right place to post BSD stuff anymore. It's too full of immature posts and uninformed nonsense.

-Matt

Re:SMP contention basically gone from critical pat (0)

rivaldufus (634820) | about 8 months ago | (#45528665)

Honestly, Slashdot isn't the right place to post BSD stuff anymore. It's too full of immature posts and uninformed nonsense.

-Matt

Agreed - The BSD section used to be somewhat lively. There's an awful lot of hostility towards the BSDs as they're not Linux. People feel really threatened when there is an alternative to their favorite OS. I also have to laugh at posts about BSD fragmentation. How many Linux distros are there now? Oh, but they're all related! Of course, the BSDs share ancestors, so they're related, too.

Re:SMP contention basically gone from critical pat (1)

KonoWatakushi (910213) | about 8 months ago | (#45529591)

The filesystem work on HAMMER2 (the filesystem successor to HAMMER1) continues to progress but it wasn't ready for even an early alpha release this release. The core media formats are stable but the freemap and the higher level abstraction layers still have a lot of work ahead of them.

Have you considered space maps [oracle.com] for tracking free space? I thought that was one of the more interesting ideas in ZFS.

Anyway, great work on the SMP scalability. It is refreshing to see a concerted effort in reworking the system to be more SMP friendly, rather than the profuse and convoluted locking that most others have adopted.

Re:SMP contention basically gone from critical pat (1)

m.dillon (147925) | about 8 months ago | (#45530487)

I've read the space map work but there are several issues that make them impractical for H2. The main one is that H2 snapshots are read-write (ZFS snapshots are read-only). Secondarily, my experience from H1 is that any complicated ref-counting or space accounting can lead to hidden corruption. Even if we assumed that the media data validation mechanism (a CRC or cryptographic hash) detects all media issues, there is still the problem of software complexity introducing bugs at a higher level.

So H2 is going to use a relatively simple freemap mechanism. In fact, it is using the same dynamic radix tree for the freemap as it does for the file and directory topology. It will still roll-up tiny allocations into a somewhat chunkier granularity to reduce freemap overhead (similar to what H1 did but at a finer 1KB grain vs H1's 2MB granularity), and it will roll-up space availability hints and propagate them up the tree. And it is type-aware so it can do things like collect inode allocations together for burstability. It isn't trivialized. But it does not attempt to implement ref-counting or any complex sub-topology to keep track of who and how many own which blocks.

The actual block freeing is going to be done in the background rather than in the 'rm' path (for a COW filesystem with snapshots which are actually used regularly, there's no point trying to free blocks in real time). This will allow H2 to implement several major features such as de-duplication, writable snapshots, and so forth with far less software complexity. So the complexity of the block allocation and any block duplication becomes trivialized while the complexity of the block freeing code increases. But it's an incrementally solvable problem. I can start with a brute-force memory-bounded scan and slowly add optimizations to recognize duplication in the topology, add topology hints to improve the efficiency of the block freeing scan, and so forth.

-Matt

Re:SMP contention basically gone from critical pat (1)

Burz (138833) | about 8 months ago | (#45530567)

...hundreds of thousands of tcp connections per second...

Hold on there, I'm still using DNet!

Re:SMP contention basically gone from critical pat (0)

Anonymous Coward | about 8 months ago | (#45533783)

Slashdot really isn't the place to post anything technical anymore.

Re:SMP contention basically gone from critical pat (1)

oddtodd (125924) | about 8 months ago | (#45535875)

Thanks for that post Matt, there's still a few old skool folks about that like this stuph. /. has a lot of noise, but there's still some signal in there now and again.

Re:SMP contention basically gone from critical pat (1)

trip23 (727132) | about 8 months ago | (#45543113)

Seconded.

Re:SMP contention basically gone from critical pat (0)

Anonymous Coward | about 8 months ago | (#45606457)

I understand why you don't post this stuff yourself but as a hardcore Linux user I like to follow the developments of Dragonfly here. It's always pleasing to see the kind of serious OS design work that you do. Linux doesn't have everything good and nor should it.

Check for New Comments
Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...