Interview with Matthew Dillon of DragonFly BSD 233
JigSaw writes "Well-known FreeBSD/DragonFly/Linux/Amiga system hacker Matthew Dillon discusses a number of interesting points regarding where the BSDs are going, the status and goals of his latest project DragonFly BSD, the status of his innovative Backplane distributed database, his exciting plans to develop DragonFly into a transparently cluster-capable system implementing native SSI (Single System Image) which is something that no other operating system can do today, and more."
I guess that'll show em. (Score:3, Funny)
Not by a long shot. (Score:5, Interesting)
I don't really know what he's talking about, but:
If he's right, everybody wins.
Even if he's wrong and we find out why, everybody wins.
It sounds like Linux isn't hurting BSD any, and methinks for a number of reasons, Linux wouldn't be what it is today without the BSD's.
Amazing. (Score:5, Funny)
Re:Amazing. (Score:2)
Re:I guess that'll show em. (Score:1)
Re:I guess that'll show em. (Score:5, Funny)
Re:I guess that'll show em. (Score:2)
Or soon will be
Re:I guess that'll show em. (Score:2)
Opposite of "advanced" (Score:2)
In technical jargon (e.g. wave theory), the opposite of "advanced" is "retarded".
Re:I guess that'll show em. (Score:1)
I'm not critisizing either one. I'm just curious, any thoughts?
Re:I guess that'll show em. (Score:1, Interesting)
Redhat9 binaries won't work with redhat7 and debian does things their own way. While mandrake and gentoo do it this way. Then suse jumps in and blah blah blah.
Sure they all use the same kernel but usually it is never the same kernel.
Re:I guess that'll show em. (Score:4, Insightful)
That said, some tools (esp those using kmem) should be kept in sync with the kernel, and when at it, why not just build a new userland, its easier then figuring out what you have to update.
The concurrently developing BSD variatiens allow trying out a variety of low level solutions to problems while sharing a lot of their experiences.
Such diversity doesn't really exist in Linux despite its zillion distributions (which provide a lot of variation in user experience tho)
Re:I guess that'll show em. (Score:2)
What's the problem?
If you want to run a.out binaries you need to run:
kldload aout
If problem in 2.x binaries - install compatibility libraries.
Re:I guess that'll show em. (Score:1)
I am quite aware that that is possible, I have maintained a Linux distribution for abotu 1 1/2 years for a small company, and we had no problem with binary compatability.
That said, it is a rather common experience for a Linux end-user to run into a package meant for an older version of their distribution, and having to actually know w
Re:I guess that'll show em. (Score:1)
Re:I guess that'll show em. (Score:5, Interesting)
Re:I guess that'll show em. (Score:5, Interesting)
Re:I guess that'll show em. (Score:5, Insightful)
Which is unfortunate in many ways. For example, Matt has introduced variant symlinks into DragonFly, and has major plans involving vfs namespaces etc which will really solve a lot of problems in package management, like allowing two different conflicting versions of a package to exist at the same time. He can do all this because he's looking at the whole picture, and so are the others: the entire source tree for the base system is there on my machine, in one nicely-arranged subdirectory. I don't foresee major changes happening in the linux kernel driven by distributors. To this day, breakages with binary-incompatible glibc etc are constant annoyances with linux unless you choose a stable distributed version from a branded linux distro and stick to it. the linux kernel is what "linux is supposed to look like" to linus.
What is "the linux kernel"? There's a Red Hat kernel, a Mandrake kernel, a SuSE kernel, and you can't really drop a generic Linus kernel into any of the commercial distros and expect it to work properly. (Debian and Gentoo are better.)
I'm not dissing linux, it's better than the mainstream alternatives and has far better hardware support and graphical system administration tools than the BSDs. In fact after 2 years with FreeBSD I myself had switched to Linux on my new machine because of hardware issues (I've now mostly switched to DragonFly and the hardware issues are mostly gone). And I use Linux at work and have no desire to change that. But there are reasons why a lot of technically aware people find the BSDs nicer systems to play with.
Re:I guess that'll show em. (Score:2)
Huh???? Generic kernels work just fine. You just need to configure them with the correct options. All RH and others do is take a generic kernel and add some additional patches such as IPSec, updated drivers, etc., but none of those are needed for day to day usage for the most part. These are NOT forks of the linux kernel.
In fact, it's a very good idea to maintain your own kernel with just the opt
Re:I guess that'll show em. (Score:5, Insightful)
Merely my brief experience with Gentoo, when they first upgraded glibc (from 2.2 to 2.3 iirc) and broke half the packages, then downgraded it again and broke everything else. This is really a pet peeve: aren't minor versions supposed to be compatible? And a zillion similar but smaller-scale annoyances, well expressed by Bill Paul many years ago [freebsd.org] and the years haven't eased the pain all that much.
And BSDs are more likely to introduce binary incompatibilities
Clearly you haven't used the BSDs. You may have library incompatibilities between major versions, but just install the earlier "compat libraries" and you're set. I upgraded from FreeBSD 4 to FreeBSD 5 -- a huge upgrade, over 2 years in the making -- and all my software just worked, even complex stuff like KDE and Mozilla that had been compiled under 4.x.
Re:I guess that'll show em. (Score:4, Informative)
If you are unhappy with your executables be broken, simply keep a copy of the older libraries. (With Gentoo, simply delete the old package file in /var/db/pkg before updating.)
Re:I guess that'll show em. (Score:2)
This is a gentoo problem, not a Linux one; I've been through two glibc changes on Debian without breaking anything.
Re:I guess that'll show em. (Score:2, Interesting)
If I'm not mistaken Linus has said he won't endorse or release a distro for that reason alone. Friendly competition between the distros is a good thing. It sparks invention and true innovation. Even the various *BSDs help each other out.
If you sit down at a Linux system you have no idea what you're going to find.
I disagree there. Most *nixes are fairly similar. You have a
Re:I guess that'll show em. (Score:5, Informative)
Granted, if you ran an all RedHat shop or an all Mandrake shop things would be easier than simply an all Linux shop, but the same would be true for an all OpenBSD shop vs an all FreeBSD or NetBSD shop. But if each department is free to buy what they want I'd rather find who-knows-which-BSD on the box than who-knows-which-Linux.
Re:I guess that'll show em. (Score:1, Interesting)
You guys are lucky! We have 5 admins to handle over 300 machines that vary from sparcstation 5's, to V880's, to HP ProLiants, to high-end PA-RISC hardware.
You better believe we pray that we don't get called when one of the "insert product here" experts goes on vacation. And when I go on vacation, I pray nobody manages to get ahold of me !
Being seriously understaffed sucks,
Re:I guess that'll show em. (Score:2)
Isn't the point of a single system to have exactly identical machines? You either mount / over NFS or have an image you blast onto the machines regularly. You absolutely mount
I think HP was right on the money; allowing your infrastructure to heterogenize is a bad thing.
Note that you can still have dept-local configurations, as long as they are centrally managed images. The idea is that if any
Re:I guess that'll show em. (Score:2)
If you're in a company with any significant number of computers, you have literally tons of spare parts lying around. All the machines will drift as they're maintained, and get customized. Someone might need more memory, s
Re:I guess that'll show em. (Score:2)
Actually, the setup, I'm trying to develop, keeps everyone's home directory on their own desktop -- the machine, they are most likely to use most of the time. It is automounted elsewhere, when needed with automounter's map available through NIS. This gives the speed of local disk, the features of file system, that may not work through NFS, and reduces network traffic.
On the downsid
Re:I guess that'll show em. (Score:4, Insightful)
Is this just MACH? (Score:1)
Maybe he should fork Darwin instead and then I could run it on my iBook.
Different threading model (Score:3, Interesting)
Re:Different threading model (Score:5, Funny)
Re:Different threading model (Score:5, Funny)
~Darl
Re:Different threading model (Score:2)
Re:Different threading model (Score:5, Informative)
Re:Different threading model (Score:5, Informative)
Re:Different threading model (Score:3, Informative)
The s
Re:Different threading model (Score:5, Informative)
Kernel threads almost universally stay on the cpu they were originally assigned to. High performance threaded subsystems, such as the network stack, are replicated. That is, the network stack creates multiple threads (one per cpu) and those threads do not migrate because, obviously, they do not need to.
Generally speaking, the purpose of making thread migration explicit instead of automatic is to partition a larger data set across available cpu caches rather then cause the same data to be shared amoungst all cpu caches. The processors operate a lot more efficiently and SMP scales a lot better. Most people do not realize the horrendous cost of moving threads between cpus because the cache mastership change is invisibly handled by hardware, but the cost is still there and still very real.
-Matt
Re:Different threading model (Score:1, Interesting)
A big problem is sharing cachelines between CPUs, where one or more of those CPUs writes to that cacheline. This problem is second only to lock serialisation when it comes to SMP scalability.
The Linux kernel for example may move tasks between CPUs (when the task isn't running, of course). This doesn't in any way prevent the
Re:Different threading model (Score:5, Informative)
In regards to cache issues, lets say you have a quad opteron system. Each cpu has a 1MB L2 cache. If you migrate threads willy nilly you basically wind up in a situation where each of the four cpu's L2 caches contain the same data. In effect, you wind up with a system that globally has only a tad more then 1MB of L2 cache. If you partition data (such as TCP protocol data) across distinct threads, and place those threads on different cpus, then you are in effect partioning your system's memory across all four cpu caches and you wind up with a system that globally has 3-4MB of L2 cache instead of 1-2MB.
There are two costs being saved here. (1) the cost of having to go to main memory when a piece of data is not in the L1/L2 cache, which can run into the hundreds of cpu cycles, and (2) the cost of cache mastership changes for all the data associated with the thread that was migrated (repeated each time the thread migrates).
-Matt
Re:Different threading model (Score:2, Insightful)
Do you do anything special in the mutex code to take advantage of the four async top level threads and prevent mutex waits?
Re:Different threading model (Score:5, Informative)
The LWKT scheduler on any given cpu is only allowed to operate on threads owned by that cpu. If you attempt to wakeup a thread owned by a different cpu, an asynchronous IPI message is sent to the target cpu's LWKT subsystem requesting that the specified thread be woken up. It's really that simple. Same goes for cross-cpu scheduling.
IPI messages themselves are lockless and require no mutexes to operate because the cpucpu messaging uses a software crossbar (array of FIFOs) approach.
Re:Different threading model (Score:2, Insightful)
Re:Different threading model (Score:5, Informative)
For a project that gets no press (Score:5, Interesting)
The further away they get from their 4.x FreeBSD roots, though, the more I wish they'd release an ISO. Particularly since the last ISOs for the 4 series of FreeBSD are probably going to be totally gone in a few months.
Re:For a project that gets no press (Score:4, Informative)
Re:What?! (Score:3, Funny)
Re:What?! (Score:2)
I hope I get one of these in MetaMod.
Here we go again (Score:1, Funny)
Why can't we all just get along??
Re:Here we go again (Score:3, Funny)
So, you see, it's obvious... BSD made a deal with the devil! And it's users weigh the same as ducks, therefor they are made of wood, and since they're made of wood, they are witches!
I mean, didn't you ever wonder why we call them "holy wars"?
Re:Here we go again (Score:4, Funny)
Divide and conquer (Score:4, Funny)
Re:Divide and conquer (Score:5, Interesting)
1: All the BSDs are entirely different operating systems, which are lumped into one category becuase of their roots.
2: Since no extra bullshit is thrown in like linux, there is less need for reworking the base.
3: BSD is not obscure in the least, it is rather alive and florishing.
BTW you forgot to mention Solaris, which has it's roots in BSD too.
Re:Divide and conquer (Score:1)
SunOS 1-4 were primarily BSD. SunOS 5 is primarily System V, with some 'leftovers' from earlier releases. [My guess would be that the BSD bits were left in primarily for backwards compatibility or because SysV didn't have equivalents.]
In any case, modern Solaris is much more SysV than BSD.
SunOS ISA BSD, Solaris ISA SVR4 (Score:1)
Re:Divide and conquer (Score:1, Interesting)
From that interview, it sounds like DragonFly is going to have a different package management system in the future. Which means either the base is going to change, you will stop calling it bsd or you will say ports isn't a basic part of bsd
Re:Divide and conquer (Score:5, Informative)
The BSD base isn't packaged. BSD types like having a source tree for their entire base system and being able to do "make buildworld" and "make installworld" to upgrade it. The package management system is entirely for third party applications. This is not Debian or Gentoo who have no code maintained by themselves other than installation and package management stuff. The BSDs maintain the kernel, the libc, other key libraries, and all the base utilities like ls, cp, mount, etc. And there's also a lot of "contrib" software in the base system -- some of it necessary to build the system (gcc and binutils), some of it just there out of tradition or regarded as "too useful to be moved to ports" (bind, sendmail).
Re:Divide and conquer (Score:1)
Useful, eh?
Re:Divide and conquer (Score:4, Insightful)
hold on cowboy...
linux drove usb support? check your history...
linux has better support for smp? right... 'cos the linux smp support isn't a rip of free-bsd's first smp incarnation, and free's 'new' smp code is some hack up by a big school kid is it?
linux has better support for enterprise hardware? shall we start with... i dunno... scsi support... get your history book out and do some experimenting with old linux v's old *bsd installs - try backing up a raid and restoring, then come back and tell me how good scsi isn't fundamental enterprise computing...
next you'll tell me that open's code auditing and goal of bug-free secure code is inferior to linux's free for all crap-code fest
excuse my rant, i'd don't mean to bag linux - every OS has its place - even windows.
but man... linux zealots and their damn superiority complex, re-writing history... i even heard someone try to explain the sco crap the other day... he actually said that 'unix is a brand of linux'
Re:Divide and conquer (Score:1, Insightful)
linux drove usb support? check your history...
USB support in both of these evolved around the same time (kernel 2.2 / FreeBSD 3.0). There are more drivers available for Linux though.
linux has better support for smp? right... 'cos the linux smp support isn't a rip of free-bsd's first smp incarnation
Woah, way to rewrite your history. Linux was first to even have SMP support in 1996. FreeBSD followed in 1998. Linux developers steadily worked on improving SMP, causing Matt [google.com] to note that Li
Re:Divide and conquer (Score:1, Insightful)
I was Solaris SysAdmin for four years. And SysV-style inits only ever got in the way. Having learnt all about SysV run levels, the fact that I can hardly remember a thing about them surely tells us that this is not really a significant BSD/SysV differance.
The new style rc scripts BSD are not perfect. But in my opinion are just a tiny bit less irritat
Re:Divide and conquer (Score:3, Insightful)
Re:Divide and conquer (Score:3, Funny)
If my post was "Flamebait" then answer my question: Who actually uses GNU/HURD? The GNU guys don't even use it! [netcraft.com] That's a statement of fact, not a flame. I certainly don't expect them to use anything that's not released under the GPL, so their use of Linux is no real surprise, but given Stallman's history with the whole "Linux" vs "GNU/Linux" thing I'm assuming he'd use GNU/HURD if it was any good.
SSI? (Score:1)
umm...unicos/mk?
Re:SSI? (Score:5, Informative)
Re:SSI? (Score:4, Funny)
I am always amazed at the rockin' shit OpenVMS can do... just about everything that DragonFly is suggesting... plus the fact that a hacker that got in would likely just say WTF? and log out.
Re:SSI? (Score:2)
Re:SSI? (Score:5, Insightful)
The commercial world is full of SSI systems although its never been clear if transparent SSI is the right answer to any problem except "I need it to work now", because coding good apps for SSI setups is hard.
Dragonfly looks a good project, and looks like it has old BSD folks who actually knew what they were doing working on it.
Michael (Score:3, Interesting)
Re:Michael (Score:2, Informative)
"The three chief virtues of a programmer are: Laziness, Impatience and Hubris." -- Larry Wall
Something no other OS can do? (Score:5, Informative)
It's simply not true that "a transparently cluster-capable system implementing native SSI" is "something that no other operating system can do today." We were doing it at Locus in 1994 with SVR4 then with Tandem in 1996 with NonStop Clusters for Unixware. Now some of the same folks at HP have introduced OpenSSI [openssi.org], which is essentially the same code, less all the Unixware-related bits, ported to Linux and placed under the GPL. They are coming up hard on their 1.0 release, which is not bad for five people and such a large task.
OpenSSI is the real thing, it has processes that migrate from node to node, distributed file systems, the works. And it's running now on clusters literally all over the world. (Not many clusters, true, but maybe that will change if the Slashdot crowd finds out about it.)
I'm happy to say that there's a lot of my code in that system, as well.
I know a little about what Matt wants to do with his SSI in Dragonfly, but he should certainly take a look at OpenSSI; we had to solve a lot of the problems you run into when you build such a beast.
(And a beast it is. As complex as a kernel can be, when you have what is essentially a distributed kernel across several nodes, the complexity goes up by orders of magnitude. Makes tracking down those weird hangs pretty exciting, in a painful, time-consuming kind of way.)
Re:Something no other OS can do? (Score:2)
Re:So I looked at OpenSSI's website. .. (Score:3, Interesting)
And it almost made no sense to me. Those buzzwords work great one at a time, but the brain starts to make a noise kind of like the one the TV makes after the TV channel goes off the air when you string too many together at once. Especially when nothing but commas separates them.
Did anyone at HP's marketing department take an courses in English at college? Or were they just as non-clueful about what OpenSSI is when they wrote that blurb as I was
Re:Something no other OS can do? (Score:1, Funny)
This looks like much stronger grounds for a lawsuit than anything IBM did. Kiss your UNIX licence good-bye HP!!
Re:Something no other OS can do? (Score:3, Interesting)
Not so much, no. The bits that were ported were never tainted and the bits that were tainted weren't ported. Because of the way we did our development, what belonged to us was never mixed with what was merely licensed. So when I said "strip out all the bits related to Unixware" I meant precisely that. Not "strip out all the Unixware bits" but strip out all the stuff in the locally-developed code that was Unixware-specific.
Of course, I was only there for the very beginning of the port; by the time the
Re:Something no other OS can do? (Score:2)
Same with IBM and AIX but somehow it didn't stop SCO claiming that IBM's code couldn't be licensed under the GPL.
As for OpenSSI, is it an integral part of the Linux kernel?
If not then it isn't really native given that it is a third party add on to Linux (or we might have a different idea of what native means in this context).
Same with your Unixware example, if it uses a third party product then it isn't really native.
Now, is there an OS out there that natively* implements SSI? (not a rethorical question
Re:Something no other OS can do? (Score:3, Interesting)
He didn't say that, here's the paragraph from the interview (emphasis mine)..
Re:Something no other OS can do? (Score:1)
(Sigh, kids these days, whatta ya gonna do?)
The Ballad of Matthew Dillon (Score:2, Funny)
Whose Dragonfly project was illin'.
He found, to his dread,
His *BSD dead
And Linux was doin' the killin'.
Re:The Ballad of Matthew Dillon (Score:2)
I guess I'm the only one that likes limericks around here.
Re:The Ballad of Matthew Dillon (Score:5, Funny)
Who thought it most wonderful news
That the AC's post
was more funny than most
But not all mods agreed with his views.
KFG
Re:The Ballad of Matthew Dillon (Score:2)
Re:The Ballad of Matthew Dillon (Score:2)
Saw ScrewMaster's prose
And rhymed thru his nose
But the joke was a crime
And I'm out of time. Phooey.
Re:The Ballad of Matthew Dillon (Score:5, Funny)
Whom everyone thought was a villain [aboutmary.com],
He cried, "That's not me!"
"I use BSD!"
"Because I find it fulfillin'."
W
Re:The Ballad of Matthew Dillon (Score:1)
Linux has no SSI? (Score:3, Interesting)
Anybody have thoughts comparing the DragonFly SSI [shiningsilence.com](warning, PDF) and the Linux [sourceforge.net] one?
(Open)Mosix has had craploads of work done on it, and by the time DragonFly's is done, it will be even further ahead. I somehow doubt DragonFly's will end up being better.
PK
Re: SSI? (Score:1, Insightful)
The ultimate goal of which Matt speaks will go far beyond even that, and it'll take time. For Linux to do the same thing, they'd basically have to reinvent their kernel as Matt has done with FreeBSD's, and that's not about to happen
Re: SSI? (Score:4, Insightful)
You need to take a break IMHO. It seem it is you who has some sort of imbecile rage against BSD, as if it killed one of your relatives or something. Saying things like 'this will be better than what linux has or will probably have' shouldn't throw you in such a fit. Look. Matt might be up to something. He might succeed. He might not. Either way,(even linux) developers might/will learn something from this 'experiment'. Can't hurt, right?
Now go, fetch that valium before you have stroke.
ps: (modded as insightful? - come on ...)
Re: SSI? (Score:1)
Ok, I myself might have been a bit harsh in my post. But look around. If you compare the level of atrocity BSD users are exposed to around here (and even osnews), that post wasn't even close to being a troll. I don't think cr
Re: SSI? (Score:2)
I don't know why you say that. Care to point out where can you see serious linux bashing sessions in the BSD community? (I'm not interested in years old derogatory statements/jokes). BSDforums has its own linux section, for instance. Occasionally linux newbies post there - and you won't see 'hey, use bsd instead' remarks there. People simply try to help. Or just a few minutes ago I saw this post [freebsdforums.org] - talking very positively (g
Re: Linux has no SSI? (Score:2, Interesting)
Backplane non-free, non-relational (Score:4, Interesting)
Re:Backplane non-free, non-relational (Score:2)
It's an interesting system in that it is really quite dif
Re:Backplane non-free, non-relational (Score:2)
Well, I don't consider copyleft a downside. But the Backplane license does not seem even to be free.
Actually Dataphor isn't implmented on top of anything. It just uses an SQL DBMS. AFAIK they don't intend yet to have their own, nor integrate into any, DBMS engine;
License contradiction? (Score:5, Interesting)
followed by:
If you power an application using the Backplane database that you market or sell, or use that application to conduct any form of online commerce (selling/buying products or services over a website) you need to purchase the Backplane Commercial License.
The example given is if you run an email service from which you sell access to other companies, you must buy the commerical license.
My question is, what if the program that provides the email service is GPL. Do I have to buy a commercial license or not? One of the great things about GPL software is that if it's an internal piece of software, you can mix proprietary and GPL code as much as you want, as long as you never redistribute the program to anyone.
Also, how does dual licensing work with this? Can I license it under the GPL to myself, and then sell copies under another license to other people? Obviously THEY would have to buy a commercial license, but do I?
Just trying to point out some holes in the licensing..
Oops, just noticed the part at the end saying:
NOTE: In any of these examples, if the entire application or service is 100% GPL compatible, you may use the Backplane Free License.
But that still leaves open the question about dual licensing..
Patents (Score:1)
It seems that you are working in some
inovative features.
I hope that in the way, you fill some patents
about your work (even if you don't agree with
software patents), because we are going to
need it in the upcoming patent fight against
Microsoft.
Re:Patents (Score:2)
Re:Patents (Score:1)
Device drivers (Score:2)
Re:Very interesting! (Score:1, Funny)
>security fiasco last year.
so what do u suggest windows? LOL
sorry