FreeBSD 7.0 Bests Linux In SMP Performance 288
cecom writes "After major improvements in SMP support in FreeBSD 7.0, benchmarks show it performing 15% better than the latest Linux kernels (PDF, see slides 17 to 19) on 8 CPUs under PostgreSQL and MySQL. While a couple of benchmarks are not conclusive evidence, it can be assumed that FreeBSD will once again be a serious performance contender. Some posters on LWN have noted that the level of Linux performance could be related to the Completely Fair Scheduler, which was merged into the 2.6.23 Linux kernel."
Update: 03/06 21:32 GMT by KD : An anonymous reader sent in word that Linux kernel developer Nick Piggin reran the benchmark today and came to a different conclusion: In his benchmark Linux was faster than FreeBSD.
Well (Score:5, Interesting)
Not that FreeBSD hasn't made major performance improvements.
Also, I think that a database test isn't a complete picture. For example, some OSes like IRIX or Mac OS X perform very well on streaming of local video and audio, but I wouldn't benchmark Oracle or PostgreSQL on either.
Re:Well (Score:4, Interesting)
About the only really good news here is that MySQL performance is actually adequate. As MySQL has always been a dog (usable, but a dog) on FreeBSD, the general rule of thumb was that if you needed MySQL you should stick to Linux; all other factors being equal. So now at least we can get down to other factors that are important instead of one database that performs poorly on one system.
Re:Well (Score:5, Funny)
Re: (Score:2)
Re:Well (Score:5, Informative)
Re: (Score:3, Funny)
Re: (Score:3, Informative)
Re: (Score:3, Insightful)
From TFA... (Score:2)
Short version:
Linux pre-CFS is faster than post-CFS, but FreeBSD still comes out ahead, by maybe 5%.
Re:From TFA... (Score:4, Informative)
The article also describes a FreeBSD 7.0 pre-release from October last year. This still had debugging code turned on in the builds, as mentioned on the NetBSD lists when Andrew Doran was comparing NetBSD -current SMP performance.
Seriously: who cares? (Score:4, Interesting)
Nobody living outside their parents' basement is going switch from Linux to BSD for a 15% performance increase. Somebody already using BSD might upgrade if the latest BSD kernels and environment are significantly better than past environments, but 15% is so slight as to be basically undetectable in a real-world environment!
My rule of thumb for upgrading equipment has been to not bother until we hit a full order of magnitude improvement. In other words, if 1) we can 10X the performance of a system AND 2) there have been complaints about performance, then the upgrade is probably worth it. Even then, the value is dubious. For example, in Postgres, (or any other database application) it's very typical to see 100x improvement simply by creating an index!
Maybe this is good for frail BSD egos, who have been long bruised by the mindshare success of Linux over the more historic and "more free" BSD. So be it. But it's not performance that's kept me from using BSD, it's familiarity and the pain of switching. And that's also what kept me running it yesterday, will today, and tomorrow too.
Don't get me wrong - I would hate to see BSD "die" in any meaningful way. The different cultures between Linux and BSD create a very rich, diverse environment where ideas can be tested, and the cross-feed of proven concepts and technologies (EG: Open SSH) benefits all involved!
But the benefit of a 15% performance increase is almost never going to be sufficient reason to pick one computing technology over another!
Re:Seriously: who cares? (Score:5, Interesting)
So if you are google, and your software will all 100% run with Linux or BSD, you don't see the idea that 15% better performance means the same work with 15% less machines means something? In certain cases, 15% can mean thousands or millions of dollars, all for changing to an operating system that will basically run on the exact same hardware and run the exact same software, after a recompile.
No for most it isn't a big deal and may not make people CHANGE operating systems on existing hardware. We use both Linux and BSD, so it *might* make me consider BSD instead of Linux on the next new box. I'm likely not alone in this.
Re:Seriously: who cares? (Score:4, Insightful)
If it takes more than a few weeks to make the switch, you've already lost your benefit, as well as the potential of destabilizing your administration of those systems. Backups have be revisited, since the file tree will have changed. Network monitors will have to be updated, and tested for compatibility changes. Little one-off scripts to solve problem X or Y in a hurry will break. Admins will have to be trained, and will make more mistakes for a while until they find out what not to do. Unforeseen wrinkles will inevitably appear, Etc... Etc... Etc...
Worth it for Google? Not a chance!
Re: (Score:3, Insightful)
Obviously, it depends on cost:benefit, which is exactly why there is no rule of thumb. Blanket generalisations and "rules of thumb" are a bad way of making a decision for everyone that
finally (Score:2)
That toast isn't going to serve itself!
Re:finally (Score:4, Funny)
Re:finally (Score:5, Funny)
Re: (Score:2)
How to recover your mod points (Score:2, Informative)
This assumes that youare still within the time frame of modability. I don't know what happens if you have used all your modpoints; does that automatically end your modability time slot?
Re: (Score:2)
I did this once (Score:3, Informative)
Re: (Score:3, Funny)
Depending on which SMP chip you're talking about, FreeBSD will now run 15% faster than Linux on your toaster.
It's that year again! (Score:5, Funny)
Wait, what?
BSD Desktops (Score:3, Interesting)
Re: (Score:3, Interesting)
Download the FBSD isos, install the gnome packages. Not new enough, build from source using ports. While it probably doesn't include the absolutely latest Gnome, the FreeBSD people tend to appreciate stability over cutting edge features, so its probably going to be a little behind the bleeding edge, for something popular like gnome though, it should match up with the latest stable release within a very short period
Silly, ignorant children. (Score:2)
Quite nice, if not my cup of tea. IMHO, BSD is for servers, but if I were tempted to use BSD for desktops, it'd be Desktop BSD.
That being said, it seems VERY clean and useful, but it does use KDE (no load time on my laptop, when accessing menus, which means something has been fixed on BSD that hasn't been fixed on Linux KDE yet.)
I think one can force it to use something other than KDE and still keep the "Desktop Tools", which make Desktop BSD quite useful. (Only OS that detected my SD M
Re:BSD Desktops (Score:4, Informative)
There are FreeBSD-based PC-BSD [pcbsd.org] and DesktopBSD [desktopbsd.net] Both of them are using KDE, though.
Re: (Score:2, Informative)
FreeBSD is not just a kernel like Linux. FreeBSD is the entire package. You don't have other distros, you have forks of the original.
Re:BSD Desktops (Score:5, Informative)
http://desktopbsd.net/wiki/doku.php?id=doc:faqs [desktopbsd.net]
Of course, I appreciate the fact that FreeBSD's base is an integrated system which is maintained as a unit as opposed to Linux distributions which are sourced from multiple projects.
Re: (Score:3, Informative)
In fact, a lot of the D
Re:BSD Desktops (Score:5, Insightful)
I'm a FreeBSD fan, but what kind of logic is that? You pick one example out of a fragmented set, and compare it to an entire other set of operating systems.
You act as if NetBSD, OpenBSD, DragonflyBSD, Darwin, etc, do not exist. Of course and item cannot be fragmented if if you define it's containing set as "itself". Makes about as much sense as:
Re: (Score:3, Insightful)
Compare/Contrast to Linux distributions. The kernel, assuming version matchs, is rather interchangable between the distros. Its the file system layout, the utilities included, and default configuration
Re: (Score:3, Insightful)
Eivind (ex/inactive FreeBSD kernel developer).
Re: (Score:2)
Why do you "need" distros? (Score:2)
What's the advantage of having multiple distros?
that everything needs to be compiled from source (ala gentoo) or that everything is precompiled for somebody else's needs (ala debian)
If you want to compile from source, you can, that's how Ports works. If you want to use precompiled versions you can install packages. You don't need to have multiple distros to be able to configure the system the way you need, you start with a core OS (which you can rebuild from sou
Re: (Score:3, Informative)
FreeBSD doesn't have distros because FreeBSD itself is very much like a distro. It's not a requirement from the FreeBSD team--rather, FreeBSD is a complete operating environment akin to a Linux distribution.
There's nothing technically or legally preventing me from creating a new installer that uses the FreeBSD kernel and FreeBSD userland, with some modifications to the default packages installed. In fact, people [desktopbsd.net] have [pcbsd.org] done [pfsense.com] just that. PFSense even calls it a distributi
Re: (Score:2)
Re:BSD Desktops (Score:4, Informative)
Possibly (Score:3, Funny)
Re:Possibly (Score:5, Informative)
You can tell by his smile. It doesn't look freakish or anything
Pre-emptable kernel? (Score:2, Interesting)
Re: (Score:2, Interesting)
Re: (Score:2)
I look forward (Score:5, Funny)
Re:I look forward (Score:5, Funny)
Wait, you forgot to mention
I have Vista (Score:5, Funny)
Can you help? Sorry, I am not good with computers.
Re: (Score:3, Funny)
HTH, HAND, etc.
Re: (Score:2)
-Smartphone xyz has a touchscreen, similar to the iphone, and the animations are even nicer.
-web based product abc is better than google efg. You can select more fonts.
-[Notebook from notable series] is nearly as thin as the apple air
-[free office suite x] can read [some standard] better than open office
etc.
General scheme:
[alternative product well known to everybody interested] has [arbitrary unsuprising advantage withing the "version flucuation" in some feature] since [last
Bad news for Linux? (Score:2, Insightful)
Re:Bad news for Linux? (Score:4, Insightful)
Its a safe bet Linux will do some other set of things faster than FreeBSD does them, possibly even another specific set of PostgreSQL queries for that matter. Linux is definately more concerned with desktop app performance. I can say this safely because Linux actually cares about it, FreeBSD does not. Its there to serve, not run X. It will run X, and if they see a way to make performance better for the desktop apps AND the server apps, then it may go in the source tree. If its going to hurt the server side, don't bet on it.
While I use FreeBSD for my servers because its got a clean filesystem layout and is designed to be a server OS, I'd be willing to bet that someone with deep knowledge of PostgreSQL on Linux could give it a run for its money by tweaking the kernel for server performance.
Re:Bad news for Linux? (Score:5, Insightful)
In my opinion, the article summary is a pretty big red herring because the SMP performance may not have a huge impact on the result.
Re:Bad news for Linux? (Score:5, Insightful)
Re: (Score:2)
Re: (Score:3, Informative)
CFS = Completely Fair Scheduling = CPU scheduler = what process gets how much of the CPU
CFQ = Completely Fair Queuing = I/O scheduler = what process gets how much of the hard disc
FWIW, on our database loads at least, I find that whilst deadline tends to give the lowest single transaction rate, CFQ gives better overall performance (i.e. more transactions served) over a given time period. Anyone tried the CFQ, deadline and no-op schedulers on a solid state disc yet?
Re: (Score:2)
In short, wow (Score:5, Interesting)
Still, I hope Linux can at least match this sort of superb scalability. CFS is fairly new, and I know there's optimisation work been done to it in
Even so, it's refreshing to see precious little of the "BSD fudged their benchmarks!" trollspeak in the LKML thread, and plenty of talk about how to make Linux better. Open source is hippy capitalism - it also needs healthy competition to keep it in check
Offtopic: bug linked to in the LKML pointed me at this http://www.latencytop.org/ [latencytop.org] Sounds quite nifty
Re: (Score:2)
Oh they are legit. The guy who did them has a thing for proper benchmark procedure and statistical analysis and is a physicist in real life.
He isn't an expert in MySQL and PostgreSQL but if he did underoptimize them, you can be assured he uderoptimized them in exactly the same way on both OS-es :)
Re: (Score:3, Informative)
If you need transactions, stored procedures, subqueries, complex queries with joins, multi-value insert, etc., then you'll get much better performance results with Postgresql >= 8.2 than with Mysql. 8.3 adds a nuce performance gain on 8.2 aswell, while the database is much more consistent than mysql is.
I think this boils down to the philosophy:
Re: (Score:3, Informative)
Bogus PostgreSQL tests (Score:3, Interesting)
The claimed "15% faster than linux" is pretty clear in the MySQL tests; the PostgreSQL ones have a weird dip in them but are in general much closer. I'd be comfortable if the result of this study was "FreeBSD 7 has been optimized to be 15% faster running MySQL than Linux", because that matches what they did (note the specific libpthread patch for example). But the fact that they used such an awful PostgreSQL benchmarking methodology leaves me hesitant to draw a broader conclusion than that based on their tests.
Where does this leave dragonfly? (Score:3, Interesting)
Agreed, on both points. What I want to know though, is where this performance improvement, and 7.0 in general, leaves Dragonfly BSD... do they still feel that Dragonfly's choice to split off at 4x and start making radical changes is paying off? Is dragonfly making progress towards better performance, in general, or on particular workloads?
I saw what Matt Dillon did back in Amiga days. I
Re: (Score:3, Informative)
Re: (Score:2)
As we know, unwritten code has no bugs and runs at twice the speed of thought, so maybe so
Linux ups the bar (Score:4, Informative)
good news, everybody! (Score:2)
FreeBSD SMP threads + boehm-gc = totally broken (Score:4, Informative)
boehm-gc is totally broken when using threads on FreeBSD SMP. And it's still totally broken on FreeBSD 7.
The Neko virtual machine is in ports, but it's unuseable due to this, I don't even understand why it's in the ports tree. Was it ever tested before being imported?
Just creating a thread:
$loader.loadprim("std@thread_create", 2)(function(z) { $print(z) }, "OK");
makes is crash with a corrupted stack. It works on every other operating system. It seems to work on an UP FreeBSD system, but on a FreeBSD 7 SMP system, it crashes, crashes, crashes.
Re: (Score:3, Insightful)
Some clarifications on my benchmarks (Score:5, Informative)
* The point of this benchmark is not to unilaterally declare victory over Linux, but to point out that FreeBSD is once again competitive with it on modern high-end hardware and certain workloads. Of course, we are working on other workloads too, and currently perform better than Linux on other benchmarks, and still worse on others. There will no doubt be further friendly competition between the two OSes that will work to the benefit of both. Our message to the Linux developers is that they should not expect to get away with resting on their laurels
* I benchmarked both mysql and postgresql, and FreeBSD 7.0 performs better than all Linux kernels (at least up to 2.6.23) with both databases. Incidentally postgresql is much faster than mysql, contradicting common wisdom. Other fun facts are that mysql 5.0.51 has poorer scaling than 5.0.47, and 5.1.x has *much* worse performance and scaling than 5.0.47 on my tests.
* I benchmarked several versions of Linux including 2.6.20.x, 2.6.22 and 2.6.23. 2.6.20.x has terrible performance http://people.freebsd.org/~kris/scaling/scaling.png [freebsd.org]. This graph is from Feb 2007 and the FreeBSD performance also improved after this point.
* 2.6.22 (which is pre-CFS) mostly fixed this but still performs worse than FreeBSD http://people.freebsd.org/~kris/scaling/os-mysql.png [freebsd.org]. 2.6.23 included the new scheduler and was a major performance regression. I did not yet retest with 2.6.24, so maybe they have fixed CFS by now.
* Contrary to some commenter's assertions that this is not a CPU benchmark, this benchmark is *extremely* sensitive to CPU performance and especially scheduling (in fact, as noted in the PDF, I/O performance is not a factor here). The scheduler really matters here, which is why Linux took a big hit when they switched to CFS (similarly, on FreeBSD the 4BSD scheduler performs much worse). Tuning the scheduler is critical to performance on this kind of workload. The other critical aspect is having a highly optimized kernel without concurrency bottlenecks. 2.6.20 fell over on kernel concurrency, and 2.6.23 fell over with the scheduler.
Hope this helps to clarify things.
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Interesting)
Hope you enjoyed a your 5 minutes in the spotlight (Score:4, Informative)
http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/ [kernel.org]
Re:Hope you enjoyed a your 5 minutes in the spotli (Score:2)
Also noted in the FreeBSD 7.0 release announcement (Score:3, Interesting)
Dramatic improvements in performance and SMP scalability shown by various database and other benchmarks, in some cases showing peak performance improvements as high as 350% over FreeBSD 6.X under normal loads and 1500% at high loads. When compared with the best performing Linux kernel (2.6.22 or 2.6.24) performance is 15% better.
http://people.freebsd.org/~kris/scaling/bind-pt.png [freebsd.org]
Summary:
* FreeBSD 7.0-R with 4BSD scheduler has close to ideal scaling on this test.
* The drop above 6 threads is due to limitations within BIND.
* Linux 2.6.24 has about 35% lower performance than FreeBSD, which is significantly at variance with the ISC results. It also doesn't scale above 3 CPUs.
* 7.0 with ULE has a bug on this workload (actually to do with workloads involving high interrupt rates). It is fixed in 8.0.
* Changes in progress to improve UDP performance do not help much with this particular workload (only about 5%), but with more scalable applications we see 30-40% improvement. e.g. NSD (ports/dns/nsd) is a much faster and more scalable DNS server than BIND (because it is better optimized for the smaller set of features it supports).
eight? (Score:3, Interesting)
Get it to scale on some serious SGI kit, for example, then we'll talk.
Re: (Score:3, Informative)
But I'm probably rising to a troll. Or an idiot. Or both.
Re:You don't have to be Kreskin (Score:5, Funny)
Re:You don't have to be Kreskin (Score:5, Funny)
Because netcraft confirms it.
Re:You don't have to be Kreskin (Score:5, Insightful)
Re:You don't have to be Kreskin (Score:4, Interesting)
Re:You don't have to be Kreskin (Score:4, Insightful)
Re:You don't have to be Kreskin (Score:4, Interesting)
You're right that mere numbers of packages is a weird metric, but what else can we offer? FreeBSD has great performance, and has everything necessary to be either a good server *or* a good desktop. It's much like Gentoo that way -- it doesn't focus on being either one or the other, it focuses on being a solid basis. What you put on top of that basis is your choice. It honestly seems to me that the distinction between server OS and desktop OS is its own entire discussion; if we can come to a good notion of what either means, we can reach a nice conclusion. If we take the current crop of Linux desktop OSes, though, I don't see any more integration between, say, Fedora and Gnome and FreeBSD and Gnome, or Ubuntu and Gnome and FreeBSD and Gnome.
If I think about it, it does seem that Ubuntu starting with a GUI interface and letting you find the command line by yourself is more friendly to the average user; I haven't installed FreeBSD using anything other than minimal-install for so long that I don't know whether you can have a GUI start up by default. And FreeBSD's installer, whilst excellent for its audience, is less friendly to a first timer. If we take those metrics, the idea of "can I sit down and first time use it without documentation?" then a lot of the linux crop are friendlier, yeah. But the documentation *is* very hand-holdy, and very very thorough for FreeBSD. And nicely available online.
Re:You don't have to be Kreskin (Score:4, Insightful)
Re: (Score:3, Interesting)
Linux will run virtually everything bsd will (after a recompile)... And most linux apps will recompile for bsd, but bsd's linux emulation isn't perfect when it comes to precompiled linux apps...
There's also hardware support, does bsd have drivers for modern ati videocards yet? I know the linux drivers suck, but its slightly better than nothing.
Re:You don't have to be Kreskin (Score:5, Insightful)
It does (I use it too) BUT only in specific environments. FreeBSD hardware support is not bad, but it is nowhere near as complete as that found in the various Linux distro's. My wireless keyboard + mouse is supported under any recent Linux distro, on FreeBSD, only the keyboard works (fixable with a unofficial ums.ko though). No support under FreeBSD for my DVB-C PCI card either.
Re: (Score:3, Interesting)
I use Linux as my desktop OS, but I am no prepared to ignore it's shortcomings. From where I'm s
Re:You don't have to be Kreskin (Score:4, Informative)
With a debian "package", I know exactly how to install it (the same way as all the others), and I know that there is a set version of that package that corresponds to, say, "Debian Sarge". I know that if I install it, it will pull along any libraries it needs, and that it won't break anything already on my system. I know it doesn't always work like that, but that's the idea. I think of a "package" as part of the distribution. Somebody has decided that it forms part of the distribution, and has hopefully tested it as such.
A "program" is what Windows has so many of. But all bets are off when it comes to versioning, library dependencies, etc. Even how to install it. If you think of Windows as a "distribution", then it doesn't come with all that many packages at all. A Desktop environment, a browser, some photo and media tools. Mac OS X doesn't really fare all that much better. I love OS X to bits, but the first thing I did was install a third party program (firefox).
Re: (Score:3, Funny)
Whatever happened to Windows?
Vista. That's a non-operating system.
Re: (Score:3, Informative)
A program is quite hard to define. A handwritten script could be considered a program by some, other
Re: (Score:3, Interesting)
Funny you should mention that. If you rule out junk software like sparkly mouse cursors, Windows seems to have less software than any other major OS (given that most Unix software is already ported to OS X, or at least can be). I feel constricted every time I have to use a Windows box because none of the programs I want to use are installed, or even readily available. No, I'm not joking.
Kreskin sez (Score:5, Funny)
Re: (Score:3, Interesting)
Does this apply to single processor machines with dual cores or just multiple processors?
Interesting point, but from the OS kernel's point-of-view a single-processor, dual-core machine looks exactly like a machine with two processors. So it runs the same code to support SMP whether we're talking single-processor dual-core or multiple processors.
IOW, if there is a performance difference, I would expect it to show up exactly the same in both FreeBSD and Linux (as well as any other OS that supports SMP).
Re: (Score:2)
Re: (Score:2)
Re:Dual Core diff; more so quad core (Score:3, Interesting)
Re: (Score:3, Interesting)
Re: (Score:3, Informative)
Don't dual core CPUs share components (cache maybe?) that an aware OS can exploit for performance improvements?
The same way an HT CPU shows up as 2 CPUs (with disasterous effects) unless the OS is away and can properly exploit it?
Some dual cores share L2/3 cache, but not all. Another important factors are the shared connections to external world, such as memory. So I presume inter-CPU communication is faster, but external communication can be slower.
That aside, HT is a hack which should not be compared to dual core systems at all. In fact, "dual core processor" is a rather silly marketing term, because it means "two processors on one piece of silicon". In other words, you could interpret the phrase "dual-core CPU" as "a CPU th
Re:Dual Core (Score:4, Interesting)
Now if you complained about Intel shoving two dies into a multi-chip package and calling that quad-core, I'd agree with you. All the reduced bandwidth of a shared connection to the FSB with none of shared cache! Sign me up!
Re: (Score:3, Interesting)
Both Intel and AMD used a shared cache design with a single connection to the system bus (FSB and HT, respectively).
In AMD's case, the shared cache sits on the other side of the fully-connected crossbar, which allows intra-core communication to happen without using HyperTransport at all. So yes, it's shared, but each core has its own "port" to it and can access it independently. Same deal for the Intel shared L2. The phrase "single connection to the system bus" is misleading because it implies a bottleneck where there (most times) isn't one.
In the case of AMD, it also means a shared memory controller.
The memory controller on Intel systems is shared as well. It's just sitt
Re: (Score:3, Informative)
Shared cache is hardly a necessity. The original Pentium D didn't have any. And (I misspoke, er typed) neither does the Athlon X2. It's just an option that makes sense when you're sharing silicon.
Another differentiator is that multi-core designs can communicate at native clockspeed, rather than resorting to an interconnect. Hypertransport is fast, but shared s