IRC Forum with Matthew Dillon of DragonFly BSD 223
weebl writes "Thursday October 9th at 6:00PM PDT (9PM EDT/1AM GMT) SlashNET's #forum channel will be hosting a Q&A session with Matthew Dillon of the DragonFly BSD Project. This is your opportunity to ask about DragonFly BSD, BSD in general, or any other questions you might have for him.
DragonFly BSD was first announced this past July." If you can't make it to the forum, SlashNET will have a bot running earlier in the day for question submissions, and logs available afterward.
Shame.... (Score:5, Interesting)
I'm sure I represent a large portion of the community who greatly appreciated his work on the VM subsystem (he even pointed the Linux folks in the right direction on more than one occassion), and am disappointed to see him leaving the project.
I understand that not everyone gets along, that goals differ between members; it's just a shame to see it happen.
Re:Shame.... (Score:1, Insightful)
Everyone knows that the BSD codebase is infinitely more clean than the Linux codebase.
Remember, the last few times there have been license issues with FreeBSD, it was people (*ahem* linux *ahem*) stealing FROM them.
Re:Shame.... (Score:1, Insightful)
Re:Shame.... (Score:1)
Nope, it ain't [lawyers.ca].
I think you'll find law in most other countries similarly clear on this issue.
Re:Shame.... (Score:5, Insightful)
Matt already wrote a SHITLOAD of code for DragonFly. He already overhauled the threading in the kernel, and put in a totally new slab allocator. Right now he's overhauling the namecache system.
Also, the DragonFly gang are doing chores nobody in the official FreeBSD camp cares enough about, like removal of __P(), removal of the 'register' keyword and ANSIfication of old K&R code. Of course we'll be seeing those efforts back in the other BSDs.
These changes that Matt and his merry gang of hackers are making are changes that would never be accepted by the FreeBSD deities with a commit bit,because they're intrusive. Hence the reason Matt forked off DragonFly.
He never left FreeBSD, dammit. Get over it.
Re:Shame.... (Score:1)
He was stripped of his commit bit. Effectively, that means his contributions are slow and filtered. He's not gone, but his role is diminished.
Re:Shame.... (Score:2)
*removal of __P()?
Could you explain what this is? I suppose that this is FreeBSD specific..
*removal of the 'register' keyword
If memory serves, the register keyword is only a hint to the compiler, so a global search and replace should be enough to remove the register keyword, no?
*ANSIfication of old K&R code
K&R, ouch my memory is fading, but I think that some preprocessor can do it automatically, no?
Not th
Re:Shame.... (Score:2)
>The problem probably isn't doing the work, but getting the changes accepted by the powers-that-don't-like-change.
Well now that he has his own codebase, this shouldn't be too much a problem
Re:Shame.... (Score:1)
Re:Shame.... (Score:5, Informative)
I still contribute to FreeBSD. If I see a bug or a security issue in DragonFly that also exists in FreeBSD they get a head's up, and some of the patch sets we've comitted to DragonFly have been organized in as legible a way as possible to promote a possible port from DragonFly to FreeBSD for those FreeBSD folks interested in the work. But my main focus these days is DragonFly.
The biggest loss to FreeBSD from my departure is that they have one less person tracking down bugs in the kernel. To be sure, I had done less and less of that over the last year as FreeBSD-5 continued to diverge from what I believed could be supported, but before anyone chimes in with a silly 'Matt hasn't done anything' comment I will note that I spent several days reviewing and commenting on Jeff's scheduler and slab allocator work, which benefited greatly from the comments, plus brainstorming sessions with Julian on KSEs, with Alan and Tor on VM issues, and so forth. I had attempted to put FreeBSD-5 on a better track several times, but the ideas were rejected in piecemeal. I don't think there are any big-picture/multi-disciplinary folk left in the project, which is a problem. Basically the FreeBSD-5 team seems to suffer from a sort of myopia. It is possible to defend a piece of work taken in isolation, such as priority stealing for mutexes or kernel preemption, but when you look at the big picture there are simply too many such pieces, each complex in its own way, in the FreeBSD-5 kernel. The result is a huge mess that only a few programmers can actually navigate through without introducing new bugs, and that is a real problem insofar as progress in FreeBSD-5 goes. Too many developers are working only in their own little corner of their world and too few developers are doing general debugging and architectural work. And, most especially, too few developers are looking out for and supporting the end-users, something that has been a significant part of my work ethic ever since I wrote DICE. There are plenty of FreeBSD developers, their numbers certainly are not decreasing, but the class of developers is far less balanced now then it was in 1998. That is my opinion, anyway.
Not having to deal with FreeBSD politics anymore as significant reduced my stress levels, and being able to work on innovative new designs rather then having to fix other people's bugs has improved my disposition dramatically :-).
-Matt
IRC Is Dead (Score:4, Funny)
A dragonfly bit me the other day (Score:2, Funny)
I don't want to use any operating system that reminds me of that traumatic experience.
I liked Dillon in Footloose, though.
Re:A dragonfly bit me the other day (Score:2, Funny)
focus (Score:3, Interesting)
We all know the 3 main (free) BSDs have their focuses, namely:
FreeBSD: ease-of-use and i386 platform (plus a few others)
NetBSD: portability
OpenBSD: security, plus some focus on ease-of-use
and then there are a few other minors, and Mac OS X
But the question is, what is DragonFly BSD's focus? What will it offer for us? How will it be useful?
Perhaps we'll learn at this chat session.
Re:focus (Score:5, Informative)
There's a LOT of work going into fine grained locking to allow faster SMP (which is a classic slowdown in FreeBSD 4) - light weight kernel threads, advanced caching, messaging APIs, and even plans for a package system that (if it works?) would completely change the way people think about installing third party software.
There are a lot of big things planned. It may be slow to start, hopefully it'll take off... and we'll see some cross development with the other BSDs.
Re:focus (Score:5, Insightful)
Unless I'm totally wrong, this is exacly the opposite from what DragonFlyBSD is about. Matt has never liked the way FreeBSD and Linux are designed, a kernel sprinkeled with locks.
Instead, he's trying to do a kernel where kernel-side subsystems communicate via message-passing, not too far from how exec.library worked in AmigaOS.
So Matt and his cowboys/cowgirls are actually removing locks.
Regards, Tommy
Re:focus (Score:4, Informative)
Now, of course, some data structures are more important then others. It doesn't make sense to try to avoid all locking... there are plenty of situations where you have to eat the duplication in the cache and plenty of situations where you have to eat cache mastership changes.
But there are a ton of situations where the waste is not necessary. In DragonFly these situations translate to: The scheduler, the slab allocator, device driver interrupt management, and the networking stack, just to begin with.
Lets take the networking stack as an example. In a fine-grained mutex model if a cpu is available it might be called upon to run pending work for the network stack... say processing packets for various TCP connections. In order to do the work the kernel will pull the next packet off the queue, figure out the protocol and port, obtain a fine-grained mutex on the PCB, and then do the work.
In the DragonFly model the network interrupt will queue the packet and the queueing code will peek at the packet and direct it to a particular protocol thread. Any given connection will be directed to a single thread so, for example, if you have a 4-cpu system and you have told the kernel to create 4 TCP protocol stack threads (one on each cpu), then each new TCP connection will be assigned to a particular thread. Only that thread will process packets associated with that particular TCP connection, which means that no locks are required to process packets for that TCP connection and no L1/L2 cache duplication will occur between cpus because only one thread will ever touch it. If you have 1000 TCP connections, each of your four TCP protocol stack threads will be assigned approximately 250 of them.
In DragonFly's case the trick then becomes to balance the assignments, but even here we get a leg-up because simply distributing the connections by ^^ mod the number of protocol threads will get you most of the way to that goal. When you add in the cache-locality you have just achieved (you have 4 times the available data cache by not sharing the PCBs across multiple cpus) the result is far higher performance then a fine-grained-but-perfectly-balanced model could ever give you.
-Matt
Re:focus (Score:4, Informative)
But why are you reading
Re:focus (Score:1, Insightful)
Perhaps you can learn at this webpage [dragonflybsd.org], dipshit.
Re:focus (Score:2)
Why Dragonfly? (Score:5, Interesting)
Re:Why Dragonfly? (Score:2)
Re:Why Dragonfly? (Score:4, Informative)
Re:Why Dragonfly? (Score:5, Interesting)
The biggest topic on our lists right now is the discussion on the infrastructure required to support a good binary and source-based package management system. FreeBSD's package management system currently suffers greatly from bitrot and upgrade-breakage (where upgrading one package breaks half a dozen others), amoung other problems. We are discussing the use of variant symlinks and VFS-based 'environments' for deploying and validating port dependancies in a manner that allows multiple versions of anything to coexist on the same system.
Basically you can think of DragonFly as being all the things I've always wanted to do but couldn't due to politics in the FreeBSD camp. FreeBSD-5 has gone down a road that I believe will lead to a slow, cumbersome, and unmaintainable result. There are plenty of examples of this but the most aggregious is thread preemption in the kernel, preemptive thread movement between cpus (also while in the kernel), and a ridiculously complex API for various types of mutex operations. There are good things in FreeBSD-5, and we will snarf them as appropriate.
The timing is appropriate. FreeBSD-4.x is nearing its end-of-life, and it is obvious that less care is being taken in preserving its vaunted stability then in the past. DragonFly will be the logical successor to FreeBSD-4.x statring in around a year, and I also fully expect to beat the shit out of FreeBSD-5 SMP-performance-wise by the time too.
DragonFly's kernel model is basically to operate on cpus in isolation, with movement beween cpus strictly managed. For example, there is a separate light weight kernel scheduler on each cpu which handles all operations related to that cpu and requires the use of no mutexes. The slab allocator is also per-cpu and mutexless. If an operation needs to work on a data structure owned by another cpu an asynchronous IPI (message) is sent to that cpu rather then performing a locking/mutex operation. For example, if cpu #1 needs to free data owned by cpu #2's slab allocator, cpu #1 queues the work to cpu #2 (and in the case of a memory free, the work does not have to be done immediately). The term 'asynchronous' is important in this context, because it means that the IPI latency between cpus can be entirely absorbed and that sequences of messages (in more heavily loaded systems) can be processed in a tight loop. When we get into more complex entities such as the networking stack, the intention is to run the protocol stacks as threads (possibly multiple threads, one on each cpu or as needed by the situation) and assign protocol control blocks to particular threads on particular cpus. This eliminates nearly all locking and mutex operations related to any given protocol control block and guarentees cache locality of reference. If done properly the individual cpu caches on SMP systems will wind up with less duplication and far higher efficiencies then one gets using a fine-grained locking mutex model. In fact, DragonFly's model works equally well in both UP and SMP environments and we expect the SMP support will cause virtually no loss in performance on UP boxes.
-Matt
Re:Why Dragonfly? (Score:1)
Re:Why Dragonfly? (Score:2)
The biggest benefit/drawback that I see is all the Slashdot trolling mirroring/mocking/echoing the "Imagine a Beowulf cluster of these" with "Imagine a Dragonfly cluster of these..." The change will be felt mostly when the pissing contest and flamewar between the trolls' biters resurrect the old Linux/BSD holy war.
I predict this will happen when people figure out how to get the Dragonfly threading/messaging code into NetBSD.
Future Architecture Idea (Score:2)
It seems to me that your SMP (Symmetric) kernel could just as easily (with a kernel module or something) become AMP (Asymmetric): creating one class of thread on a particular class of processor. What I mean is the loader would create an executable main memory thread/object tagged for a DSP or a x86 or an AMD64 or PPC processor, and the kernel would be able to execute these tagged objects on specific processors on expansion cards or a remote host or a FC WWPN target or such craziness..
The real shame (Score:3, Insightful)
Re:The real shame (Score:3)
Is this the guy that wrote dcron? (Score:1, Interesting)
Re:Is this the guy that wrote dcron? (Score:4, Interesting)
It is gratifying to see that DCron is still being used regularly, but I am a bit miffed that nobody has gone and made any truely significant improvements to it. That is part of the nature of open-source, I guess.
-Matt
Re:Is this the guy that wrote dcron? (Score:2)
As a happy user of DCron, I can't honestly think there's anything significant to improve upon. It works, and works well. Thanks for a great piece of software!
Too many Matt Dillons! (Score:5, Funny)
Then, I realized I must have been thinking about the wrong Matt Dillon, but I still thought it was weird that the guy from There's Something About Mary [imdb.com] became involved in a BSD project.
Finally I remembered the other Matt Dillon [backplane.com] who developed the DICE C compiler for the Amiga back in the good old days.
Re:Too many Matt Dillons! (Score:3, Informative)
Re:Too many Matt Dillons! (Score:5, Informative)
I also completely rewrote the swap management code from the old list-of-holes format to a fixed radix tree with dynamic size hinting, fixed issues in VN and MFS, and generally put out a lot of fires all over the tree. Most of what I did in BSDland was fix bugs, because there were a lot of bugs that needed fixing and, again, there isn't much of a point having an advanced operating system if it crashes every so often.
I probably spent far more time fixing bugs then anything else. My involvement with FreeBSD began during my BEST Internet days, in the 94 or 95 timeframe, with a commit bit coming in '98 I think, but my involvement with BSD began in 1985 at UC Berkeley, at about the same time I got involved with the Amiga.
The logs will be at slashnet.org (Score:5, Informative)
For those who can't make the chat, the log will eventually be at http://www.slashnet.org/forums/ [slashnet.org]
Editors: After the chat is over, any chance of having the log URL linked to the story text as an update?
Lets make a bet... (Score:2, Interesting)
Or better yet fooling the irc admins into thinking its a DDOS attack.
Wouldn't it be easier to use the handy slashdot moderation? I think CmdTaco would be willing to hand him 30 of the most highly rated questions.
Re:Owned by SCO (Score:3, Interesting)
I've seen c
Re:It is now official (Score:4, Funny)
It is now official - Netcraft has confirmed: *BSD is dying
I really shouldn't reply to this kind of comment, but doesn't that comment seem kinda weird if you consider the fact netcraft runs FreeBSD?
Re:DragonFly BSD Problems (Score:1)
Re:*BSD for Windows XP? (Score:1)
You could install it on another partition and actually dual boot BSD and Windows (dual booting is where you have two operating systems on the same machine, side by side). You can only run one operating system at a time, though.
You can download the latest FreeBSD ISO (bootable CD image) from here [linux.com]
Re:*BSD for Windows XP? Set up first! (Score:2, Funny)
dd if=dev/urandom of=dev/ad0
this command will fix things up for you first.
Re:Ahem (Score:2)
Re:Ahem (Score:1)
From a technological standpoint, why should the average person choose one or the other? ("Average" in this case, meaning someone debating with themself between linux/bsd/or even MS for next server.)
I mean to ask this strictly from a strictly technology standpoint. I have heard the arguements re: bsd vs gpl, and it is an important issue as far as present and future development is concerned. But, I'm loo
Re:And in other similar comparisment (Score:1)
Sorry, just taking the pith.