Forgot your password?
typodupeerror
Operating Systems Software BSD Linux

FreeBSD 7.0 Bests Linux In SMP Performance 288

Posted by kdawson
from the coulda-been-a-contender dept.
cecom writes "After major improvements in SMP support in FreeBSD 7.0, benchmarks show it performing 15% better than the latest Linux kernels (PDF, see slides 17 to 19) on 8 CPUs under PostgreSQL and MySQL. While a couple of benchmarks are not conclusive evidence, it can be assumed that FreeBSD will once again be a serious performance contender. Some posters on LWN have noted that the level of Linux performance could be related to the Completely Fair Scheduler, which was merged into the 2.6.23 Linux kernel." Update: 03/06 21:32 GMT by KD : An anonymous reader sent in word that Linux kernel developer Nick Piggin reran the benchmark today and came to a different conclusion: In his benchmark Linux was faster than FreeBSD.
This discussion has been archived. No new comments can be posted.

FreeBSD 7.0 Bests Linux In SMP Performance

Comments Filter:
  • Well (Score:5, Interesting)

    by morgan_greywolf (835522) on Thursday March 06, 2008 @09:44AM (#22662176) Homepage Journal
    I'd be interested to see results from pre-CFS kernels.

    Not that FreeBSD hasn't made major performance improvements.

    Also, I think that a database test isn't a complete picture. For example, some OSes like IRIX or Mac OS X perform very well on streaming of local video and audio, but I wouldn't benchmark Oracle or PostgreSQL on either.
    • Re:Well (Score:4, Interesting)

      by archen (447353) on Thursday March 06, 2008 @09:57AM (#22662298)
      And I like the article summary stating that FreeBSD may now be considered "a serious performance contender". Like FreeBSD was 1000% slower than Linux? Most servers spend their time spinning their wheels anyway, generally I'd rather look at security, how it handles under load and other metrics than whatever "performance" is considered in this instance. Linux is good for some things, BSD for others.

      About the only really good news here is that MySQL performance is actually adequate. As MySQL has always been a dog (usable, but a dog) on FreeBSD, the general rule of thumb was that if you needed MySQL you should stick to Linux; all other factors being equal. So now at least we can get down to other factors that are important instead of one database that performs poorly on one system.
      • Re:Well (Score:5, Funny)

        by calebt3 (1098475) on Thursday March 06, 2008 @11:15AM (#22663094)

        Most servers spend their time spinning their wheels anyway
        And then along comes a /. article...
      • by makomk (752139)
        Acoording TFA, FreeBSD didn't even have proper SMP support until relatively recently. Until 5.0, released in 2003, only one CPU could run kernel code at once. (To be fair, I think Linux still hasn't fully got rid of its own Big Kernel Lock, but...)
    • Re:Well (Score:5, Informative)

      by Zpin (921535) on Thursday March 06, 2008 @10:28AM (#22662598)
      The linked PDF contains pre-CFS kernel benchmarks. Conclusion:

      The new CFS scheduler in 2.6.23 is "Completely Fair" ...to FreeBSD
      • Re: (Score:3, Funny)

        Yep. Thanks, Linus, for dropping the pluggable schedulers in favor of The One Scheduler...
    • Re: (Score:3, Insightful)

      by saleenS281 (859657)
      You also wouldn't run Oracle or PostgreSQL on them unless you wanted to lose your job. The test is a great test for linux vs. freebds, because it's the type of environment where they'd compete.
    • As Zpin wrote a few posts above, the linked PDF contains pre-CFS kernel benchmarks.

      Short version:
      Linux pre-CFS is faster than post-CFS, but FreeBSD still comes out ahead, by maybe 5%.
    • by mcrbids (148650) on Thursday March 06, 2008 @12:38PM (#22664120) Journal
      Yes, I meant that: who cares?

      Nobody living outside their parents' basement is going switch from Linux to BSD for a 15% performance increase. Somebody already using BSD might upgrade if the latest BSD kernels and environment are significantly better than past environments, but 15% is so slight as to be basically undetectable in a real-world environment!

      My rule of thumb for upgrading equipment has been to not bother until we hit a full order of magnitude improvement. In other words, if 1) we can 10X the performance of a system AND 2) there have been complaints about performance, then the upgrade is probably worth it. Even then, the value is dubious. For example, in Postgres, (or any other database application) it's very typical to see 100x improvement simply by creating an index!

      Maybe this is good for frail BSD egos, who have been long bruised by the mindshare success of Linux over the more historic and "more free" BSD. So be it. But it's not performance that's kept me from using BSD, it's familiarity and the pain of switching. And that's also what kept me running it yesterday, will today, and tomorrow too.

      Don't get me wrong - I would hate to see BSD "die" in any meaningful way. The different cultures between Linux and BSD create a very rich, diverse environment where ideas can be tested, and the cross-feed of proven concepts and technologies (EG: Open SSH) benefits all involved!

      But the benefit of a 15% performance increase is almost never going to be sufficient reason to pick one computing technology over another!
      • by Pharmboy (216950) on Thursday March 06, 2008 @01:18PM (#22664684) Journal
        But the benefit of a 15% performance increase is almost never going to be sufficient reason to pick one computing technology over another!

        So if you are google, and your software will all 100% run with Linux or BSD, you don't see the idea that 15% better performance means the same work with 15% less machines means something? In certain cases, 15% can mean thousands or millions of dollars, all for changing to an operating system that will basically run on the exact same hardware and run the exact same software, after a recompile.

        No for most it isn't a big deal and may not make people CHANGE operating systems on existing hardware. We use both Linux and BSD, so it *might* make me consider BSD instead of Linux on the next new box. I'm likely not alone in this.
        • by mcrbids (148650) on Thursday March 06, 2008 @02:31PM (#22665784) Journal
          Compare 15% against Moore's law, and you find that it equates to a few weeks delay in the price-performance curve.

          If it takes more than a few weeks to make the switch, you've already lost your benefit, as well as the potential of destabilizing your administration of those systems. Backups have be revisited, since the file tree will have changed. Network monitors will have to be updated, and tested for compatibility changes. Little one-off scripts to solve problem X or Y in a hurry will break. Admins will have to be trained, and will make more mistakes for a while until they find out what not to do. Unforeseen wrinkles will inevitably appear, Etc... Etc... Etc...

          Worth it for Google? Not a chance!
      • Re: (Score:3, Insightful)

        by Cal Paterson (881180) *

        Nobody living outside their parents' basement is going switch from Linux to BSD for a 15% performance increase. Somebody already using BSD might upgrade if the latest BSD kernels and environment are significantly better than past environments, but 15% is so slight as to be basically undetectable in a real-world environment!

        Obviously, it depends on cost:benefit, which is exactly why there is no rule of thumb. Blanket generalisations and "rules of thumb" are a bad way of making a decision for everyone that

  • by Zashi (992673)
    I can finally make full use of my quad-core toaster!

    That toast isn't going to serve itself!
    • Re:finally (Score:4, Funny)

      by BitZtream (692029) on Thursday March 06, 2008 @09:55AM (#22662280)
      Your comment would have been funny if we were talking about NetBSD. NetBSD is the one that runs on everything, including a toaster.
      • Re:finally (Score:5, Funny)

        by UbuntuLinux (1242150) on Thursday March 06, 2008 @10:04AM (#22662376)
        Hello. I don't believe you know me, but you might be able to help. I use Ubuntu Linux, and I am helping the daughter of a friend to install Linux onto her PC this evening. I have never really spoken to a girl before, and was thinking that maybe if I made a joke, it might break the ice. You are obviously excellent at humour, and I was wondering if you could give me some tips? For instance, it is inevitable that something will go wrong during the process, and I was thinking that maybe if I said something like 'this is almost as unreliable as my beard trimmers!' then it would demonstrate that I am a funny guy. Can you offer any kind of critique of this line, or offer any other advice?
        • Holy carp, that was funny. One of the few comments that's gotten me to laugh aloud for more than a half-second. Oh, to recant my hastily-spent mod points...
          • Post a comment. When faced with the two incompatibilities, slashcode keeps the comment and tosses the mods, presumably because comments might have attracted subcomments and thus be impossible to remove.

            This assumes that youare still within the time frame of modability. I don't know what happens if you have used all your modpoints; does that automatically end your modability time slot?
            • What is with the offtopic police and their poor usage? Ah, well - anyway. The points were actually spent earlier, in a couple of different articles. Though I think posting in the conversation after modding actually results in not only losing the mod, but also not regaining the lost mod points...
      • Re: (Score:3, Funny)

        Depending on which SMP chip you're talking about, FreeBSD will now run 15% faster than Linux on your toaster.

  • by n3tcat (664243) on Thursday March 06, 2008 @09:48AM (#22662208) Homepage
    Maybe now we can finally declare year of the linux desktop!

    Wait, what?
    • BSD Desktops (Score:3, Interesting)

      by parvenu74 (310712)
      Speaking of which: are there any "distros" out there ship a combination of FreeBSD and the latest Gnome desktop? I think that would be a better combination than Ubuntu's Debian+Gnome combo, personally.
      • Re: (Score:3, Interesting)

        by BitZtream (692029)
        Yes, the only FreeBSD 'distro'. FreeBSD is not fragmented like the 100 and 1 Linux distros

        Download the FBSD isos, install the gnome packages. Not new enough, build from source using ports. While it probably doesn't include the absolutely latest Gnome, the FreeBSD people tend to appreciate stability over cutting edge features, so its probably going to be a little behind the bleeding edge, for something popular like gnome though, it should match up with the latest stable release within a very short period
        • Dig up Desktop BSD.

          Quite nice, if not my cup of tea. IMHO, BSD is for servers, but if I were tempted to use BSD for desktops, it'd be Desktop BSD.

          That being said, it seems VERY clean and useful, but it does use KDE (no load time on my laptop, when accessing menus, which means something has been fixed on BSD that hasn't been fixed on Linux KDE yet.)

          I think one can force it to use something other than KDE and still keep the "Desktop Tools", which make Desktop BSD quite useful. (Only OS that detected my SD M
        • Re:BSD Desktops (Score:4, Informative)

          by ninjaz (1202) on Thursday March 06, 2008 @10:28AM (#22662596)

          Yes, the only FreeBSD 'distro'. FreeBSD is not fragmented like the 100 and 1 Linux distros

          There are FreeBSD-based PC-BSD [pcbsd.org] and DesktopBSD [desktopbsd.net] Both of them are using KDE, though.

          • Re: (Score:2, Informative)

            by BitZtream (692029)
            No, those are forks.

            FreeBSD is not just a kernel like Linux. FreeBSD is the entire package. You don't have other distros, you have forks of the original.
            • Re:BSD Desktops (Score:5, Informative)

              by ninjaz (1202) on Thursday March 06, 2008 @12:19PM (#22663844)

              No, those are forks. FreeBSD is not just a kernel like Linux. FreeBSD is the entire package. You don't have other distros, you have forks of the original.
              Well, both PC-BSD and DesktopBSD claim to not be forks. DesktopBSD explicitly identifies itself as a distribution of FreeBSD and PC-BSD says "PIF" (PC-BSD is FreeBSD). Here are the references: [pcbsd.org]http://faqs.pcbsd.org/index.php?action=artikel&cat=14&id=304&artlang=en [pcbsd.org]

              http://desktopbsd.net/wiki/doku.php?id=doc:faqs [desktopbsd.net]

              Of course, I appreciate the fact that FreeBSD's base is an integrated system which is maintained as a unit as opposed to Linux distributions which are sourced from multiple projects.

            • Re: (Score:3, Informative)

              by sremick (91371)
              Incorrect. Both track the main FreeBSD tree. Consider them "value add" packages of software and tweaks layered on top of the official FreeBSD. A fork would be traced to a single point in time with ever-growing divergence (think DragonflyBSD). Both DesktopBSD and PC-BSD include updated bases of the main FreeBSD with their own updates. For example, PC-BSD 1.0 was based on FreeBSD 6.0. The current version of PC-BSD, 1.4.1, is based on FreeBSD 6.3. PC-BSD 2.0 will be based on FreeBSD 7.0.

              In fact, a lot of the D
        • Re:BSD Desktops (Score:5, Insightful)

          by Anonymous Coward on Thursday March 06, 2008 @10:28AM (#22662604)

          FreeBSD is not fragmented like the 100 and 1 Linux distros

          I'm a FreeBSD fan, but what kind of logic is that? You pick one example out of a fragmented set, and compare it to an entire other set of operating systems.

          You act as if NetBSD, OpenBSD, DragonflyBSD, Darwin, etc, do not exist. Of course and item cannot be fragmented if if you define it's containing set as "itself". Makes about as much sense as:

          Ubuntu is not fragmented like the 100 and 1 BSD distros
          • Re: (Score:3, Insightful)

            by BitZtream (692029)
            Of those you listed, Only DragonflyBSD and Darwin use a FreeBSD kernel, Darwin considerably modified, to the point that it might as well be counted on its own. You can not drop the FreeBSD kernel in any of them and have things work, except for Dragonfly, but even then it still requires (minor) modification.

            Compare/Contrast to Linux distributions. The kernel, assuming version matchs, is rather interchangable between the distros. Its the file system layout, the utilities included, and default configuration
            • Re: (Score:3, Insightful)

              by Eivind Eklund (5161)
              I would count both Darwin and DragonflyBSD as their own kernels; they are substantially modified.

              Eivind (ex/inactive FreeBSD kernel developer).

        • by Vexorian (959249)
          I guess that's the main reason for its lack of success.
      • Gnome in FreeBSD is at something like 2.20 -- pretty damn recent. Just install FreeBSD, and either use pkg_add to grab the most recent binaries or build your own gnome by grabbing the latest ports tree ("portsnap fetch extract"'ll do that ) then "cd /usr/ports/x11/gnome2 && make install clean". Done.
      • Re:BSD Desktops (Score:4, Informative)

        by Ed Avis (5917) <ed@membled.com> on Thursday March 06, 2008 @10:39AM (#22662746) Homepage
        You could try Debian GNU/kFreeBSD, though this is not really a distribution of the full FreeBSD system but just its kernel with a GNU userland.
  • Possibly (Score:3, Funny)

    by Malevolent Tester (1201209) * on Thursday March 06, 2008 @09:49AM (#22662214) Journal

    While a couple of benchmarks are not conclusive evidence, it can be assumed that FreeBSD will once again be a serious performance contender
    Right up until someone displays a crucifix, that is.
  • Pre-emptable kernel? (Score:2, Interesting)

    by Gothmolly (148874)
    Does FreeBSD have a pre-emptable kernel? One of the things Linux has really focused on lately is desktop interactive performance, so there may be performance tradeoffs vs. a kernel which can't pre-empt itself.
    • Re: (Score:2, Interesting)

      by archen (447353)
      in the kernel config there is a preemption option which I believe is enabled in the default kernel. Generally you can turn this on and off, and use a different scheduler if you want to, but there isn't the myriad of options to tinker with in Linux. Since many don't bother switching from the defaults, and many others tend to choose wrong anyway, this might be a good thing.
    • Does FreeBSD have a pre-emptable kernel? One of the things Linux has really focused on lately is desktop interactive performance, so there may be performance tradeoffs vs. a kernel which can't pre-empt itself.
      Yes. The FreeBSD kernel has preemption options to enable preemption. The distributed generic kernels have it enabled.
  • by Clover_Kicker (20761) <clover_kicker@yahoo.com> on Thursday March 06, 2008 @09:55AM (#22662276)
    to the enlightening and respectful conversation this article will provoke.
  • by Chrisq (894406)
    Since they switched to the Completely Fair Scheduler [wikipedia.org] to improve performance then it means one or two things. Either they have failed, as it seems to run slower than earlier Linux versions in the BSD test, or the tests that BSD chose are "untypical", maybe selected to show a particular advantage to BSD. I don't have the expertise to tell which, but I would be happier seeing some benchmarks from an independent source rather than BSD.
    • by BitZtream (692029) on Thursday March 06, 2008 @10:04AM (#22662370)
      Benchmarks are almost but not completely useless. In this particular setup, FBSD 7.0 runs postgres doing some specific set of queries faster than Linux.

      Its a safe bet Linux will do some other set of things faster than FreeBSD does them, possibly even another specific set of PostgreSQL queries for that matter. Linux is definately more concerned with desktop app performance. I can say this safely because Linux actually cares about it, FreeBSD does not. Its there to serve, not run X. It will run X, and if they see a way to make performance better for the desktop apps AND the server apps, then it may go in the source tree. If its going to hurt the server side, don't bet on it.

      While I use FreeBSD for my servers because its got a clean filesystem layout and is designed to be a server OS, I'd be willing to bet that someone with deep knowledge of PostgreSQL on Linux could give it a run for its money by tweaking the kernel for server performance.
    • by Azarael (896715) on Thursday March 06, 2008 @10:52AM (#22662880) Homepage
      The article is probably misleading (surprise, surprise), as the tuning documentation for PostgreSQL *states* that good IO performance has more of an impact than good CPU performance. Additionally, some other information I've read (search for postgres tuning/optimi(z|s)ation online) recommends FreeBSD because of its strong IO performance. I'll go out on a limb and assume that MySQL's performance attributes are similar.

      In my opinion, the article summary is a pretty big red herring because the SMP performance may not have a huge impact on the result.
    • by atomic-penguin (100835) <wolfe21&marshall,edu> on Thursday March 06, 2008 @10:54AM (#22662896) Homepage Journal
      For one, CFQ is not supposed to be an optimized I/O scheduler for database loads. That's where the Deadline scheduler comes in. You wouldn't want a "Fair" scheduler on your database server, as you would end up putting the database in I/O wait to handle lower priority processes.
      • by Fweeky (41046)
        CFS is a CPU scheduler, and rather similar to FreeBSD's ULE; CFQ and Deadline are IO schedulers, and completely different beasts.
      • Re: (Score:3, Informative)

        by MrNemesis (587188)
        As t'other poster pointed out;
        CFS = Completely Fair Scheduling = CPU scheduler = what process gets how much of the CPU
        CFQ = Completely Fair Queuing = I/O scheduler = what process gets how much of the hard disc

        FWIW, on our database loads at least, I find that whilst deadline tends to give the lowest single transaction rate, CFQ gives better overall performance (i.e. more transactions served) over a given time period. Anyone tried the CFQ, deadline and no-op schedulers on a solid state disc yet?
    • by Vexorian (959249)
      There are lies, damn lies, statistics and benchmarks.
  • In short, wow (Score:5, Interesting)

    by MrNemesis (587188) on Thursday March 06, 2008 @10:07AM (#22662398) Homepage Journal
    Very, very nice scaling performance under PGSQL is evident in the PDF, and I've no reason to assume the benches aren't legit. I think part fo the reason that PG was traditionally slower than MySQL was that it did lots of complicated locking to provide better scalability across processors, whereas we see MySQL performance dropping off after we go to more than eight cores. I think this was the same philosophy Sun took with "Slowaris", which was also far more scalabe than Linux at the time the moniker was in widespread use.

    Still, I hope Linux can at least match this sort of superb scalability. CFS is fairly new, and I know there's optimisation work been done to it in .24 and .25, although it was a little sad to see the first iteration of CFS performing more poorly than its predecessor (and, if this is the case, I can see why Linus stonewalled CK's patches for so long, since they were mainly tested on desktop workloads). Are there any apples-to-apples comparisons out there that test various flavours and versions of Linux and BSD with a wide range of benchmarks? At the best review sites do a few benches with MySQL, and six months later everything has changed so it's incredibly difficult to do good performance comparisons.

    Even so, it's refreshing to see precious little of the "BSD fudged their benchmarks!" trollspeak in the LKML thread, and plenty of talk about how to make Linux better. Open source is hippy capitalism - it also needs healthy competition to keep it in check :)

    Offtopic: bug linked to in the LKML pointed me at this http://www.latencytop.org/ [latencytop.org] Sounds quite nifty
    • by ivoras (455934)

      Very, very nice scaling performance under PGSQL is evident in the PDF, and I've no reason to assume the benches aren't legit.

      Oh they are legit. The guy who did them has a thing for proper benchmark procedure and statistical analysis and is a physicist in real life.

      He isn't an expert in MySQL and PostgreSQL but if he did underoptimize them, you can be assured he uderoptimized them in exactly the same way on both OS-es :)

    • Re: (Score:3, Informative)

      Well, as someone who uses Pg with 100k unique user visits a day, Pg beats the crap out of Mysql if we're talking about anything else than very simple selects.

      If you need transactions, stored procedures, subqueries, complex queries with joins, multi-value insert, etc., then you'll get much better performance results with Postgresql >= 8.2 than with Mysql. 8.3 adds a nuce performance gain on 8.2 aswell, while the database is much more consistent than mysql is.

      I think this boils down to the philosophy:
      • Re: (Score:3, Informative)

        by MrNemesis (587188)
        Agree 100%. I've got to the stage where if I see something list MySQL support without Postgres support, I'm generally of the opinion they're not taking things seriously enough :) Postgres has been by DB of choice for about two years now, and I've yet to have it balls up on me, and speed has never been an issue - like you say, as soon as you start doing non-trivial stuff, Postgres utterly thrashes MySQL.
  • by greg1104 (461138) <gsmith@gregsmith.com> on Thursday March 06, 2008 @10:24AM (#22662554) Homepage
    While I'm glad for FreeBSD they're showing good numbers again, their testing of PostgreSQL in this study is rather odd. The results are using the read-only tests from sysbench. You can see from its sourceforge page [sourceforge.net] that sysbench is a MySQL benchmarking tool that has some rudimentary PostgreSQL support bolted on top. That particular code is so bad that the last time I checked, turning on the write OLTP tests deadlocked the PostgreSQL server, as it wasn't putting statements into transactions correctly (which of course the ancient MySQL versions this code targeted doesn't care about). As the sysbench tool hasn't been actively maintained in ages I doubt that has improved.

    The claimed "15% faster than linux" is pretty clear in the MySQL tests; the PostgreSQL ones have a weird dip in them but are in general much closer. I'd be comfortable if the result of this study was "FreeBSD 7 has been optimized to be 15% faster running MySQL than Linux", because that matches what they did (note the specific libpthread patch for example). But the fact that they used such an awful PostgreSQL benchmarking methodology leaves me hesitant to draw a broader conclusion than that based on their tests.
    • While I'm glad for FreeBSD they're showing good numbers again, their testing of PostgreSQL in this study is rather odd.

      Agreed, on both points. What I want to know though, is where this performance improvement, and 7.0 in general, leaves Dragonfly BSD... do they still feel that Dragonfly's choice to split off at 4x and start making radical changes is paying off? Is dragonfly making progress towards better performance, in general, or on particular workloads?

      I saw what Matt Dillon did back in Amiga days. I

      • Re: (Score:3, Informative)

        by renoX (11677)
        The dev of Dragonfly BSD has switched its aim from being 'a better way to do SMP' to 'SSI clustering' so I doubt that Dragonfly BSD is going to compete with FreeBSD in SMP scalability anytime soon if ever.
      • by kkenn (83190)
        Dragonfly has still made no progress towards SMP performance with even 2 CPUs (they are still "competitive" with FreeBSD 4.x performance, and sometimes much slower), but this somehow doesn't deter their advocates from believing that Matt Dillon's -- still largely unimplemented -- ideas are still vastly superior.

        As we know, unwritten code has no bugs and runs at twice the speed of thought, so maybe so :)
  • Linux ups the bar (Score:4, Informative)

    by argggh (1251840) on Thursday March 06, 2008 @10:37AM (#22662722)
  • Sounds good. Presuming that there is merit in the methodology here, we may see some more competition in the market. I'd like it more if there were more code sharing going on, but maybe if we don't get tempers too hot we'll see a little more of that too.
  • by chrysalis (50680) on Thursday March 06, 2008 @11:02AM (#22662958) Homepage
    Cool, however it would be better if software working on Linux were also working on FreeBSD.

    boehm-gc is totally broken when using threads on FreeBSD SMP. And it's still totally broken on FreeBSD 7.

    The Neko virtual machine is in ports, but it's unuseable due to this, I don't even understand why it's in the ports tree. Was it ever tested before being imported?

    Just creating a thread:

    $loader.loadprim("std@thread_create", 2)(function(z) { $print(z) }, "OK");

    makes is crash with a corrupted stack. It works on every other operating system. It seems to work on an UP FreeBSD system, but on a FreeBSD 7 SMP system, it crashes, crashes, crashes.
    • Re: (Score:3, Insightful)

      by Sancho (17056)
      Did you file a bug report and contact that port's maintainer?
  • by kkenn (83190) on Thursday March 06, 2008 @11:11AM (#22663042)
    Hi, I am the one who performed these benchmarks and I'd like to clarify a couple of things:

    * The point of this benchmark is not to unilaterally declare victory over Linux, but to point out that FreeBSD is once again competitive with it on modern high-end hardware and certain workloads. Of course, we are working on other workloads too, and currently perform better than Linux on other benchmarks, and still worse on others. There will no doubt be further friendly competition between the two OSes that will work to the benefit of both. Our message to the Linux developers is that they should not expect to get away with resting on their laurels :-)

    * I benchmarked both mysql and postgresql, and FreeBSD 7.0 performs better than all Linux kernels (at least up to 2.6.23) with both databases. Incidentally postgresql is much faster than mysql, contradicting common wisdom. Other fun facts are that mysql 5.0.51 has poorer scaling than 5.0.47, and 5.1.x has *much* worse performance and scaling than 5.0.47 on my tests.

    * I benchmarked several versions of Linux including 2.6.20.x, 2.6.22 and 2.6.23. 2.6.20.x has terrible performance http://people.freebsd.org/~kris/scaling/scaling.png [freebsd.org]. This graph is from Feb 2007 and the FreeBSD performance also improved after this point.

    * 2.6.22 (which is pre-CFS) mostly fixed this but still performs worse than FreeBSD http://people.freebsd.org/~kris/scaling/os-mysql.png [freebsd.org]. 2.6.23 included the new scheduler and was a major performance regression. I did not yet retest with 2.6.24, so maybe they have fixed CFS by now.

    * Contrary to some commenter's assertions that this is not a CPU benchmark, this benchmark is *extremely* sensitive to CPU performance and especially scheduling (in fact, as noted in the PDF, I/O performance is not a factor here). The scheduler really matters here, which is why Linux took a big hit when they switched to CFS (similarly, on FreeBSD the 4BSD scheduler performs much worse). Tuning the scheduler is critical to performance on this kind of workload. The other critical aspect is having a highly optimized kernel without concurrency bottlenecks. 2.6.20 fell over on kernel concurrency, and 2.6.23 fell over with the scheduler.

    Hope this helps to clarify things.
    • FYI. Your observations of mysql performance are right in line with what others including google have noticed. The famous google patches significantly improve the scalability regressions introduced. They should eventually make there way into the official Mysql in a couple months or so.
      • by kkenn (83190)
        Thanks, it's useful to know. As a "consumer" of mysql it's kind of disappointing they don't do this kind of performance analysis in-house.
  • by Lost+Found (844289) on Thursday March 06, 2008 @11:15AM (#22663092)
    Looks like it didn't last for long:

    http://www.kernel.org/pub/linux/kernel/people/npiggin/sysbench/ [kernel.org]
  • by bconway (63464) on Thursday March 06, 2008 @11:35AM (#22663328) Homepage
    Here [freebsd.org], and it applies to a significant number of other network servers.

    Dramatic improvements in performance and SMP scalability shown by various database and other benchmarks, in some cases showing peak performance improvements as high as 350% over FreeBSD 6.X under normal loads and 1500% at high loads. When compared with the best performing Linux kernel (2.6.22 or 2.6.24) performance is 15% better.

    http://people.freebsd.org/~kris/scaling/bind-pt.png [freebsd.org]

    Summary:

    * FreeBSD 7.0-R with 4BSD scheduler has close to ideal scaling on this test.

    * The drop above 6 threads is due to limitations within BIND.

    * Linux 2.6.24 has about 35% lower performance than FreeBSD, which is significantly at variance with the ISC results. It also doesn't scale above 3 CPUs.

    * 7.0 with ULE has a bug on this workload (actually to do with workloads involving high interrupt rates). It is fixed in 8.0.

    * Changes in progress to improve UDP performance do not help much with this particular workload (only about 5%), but with more scalable applications we see 30-40% improvement. e.g. NSD (ports/dns/nsd) is a much faster and more scalable DNS server than BIND (because it is better optimized for the smaller set of features it supports).
  • eight? (Score:3, Interesting)

    by dwater (72834) on Thursday March 06, 2008 @12:07PM (#22663716)
    Is 8-way still considered SMP? I mean, 8-way is kind of consumer level now, isn't it? Even Apple produce 8-way machines SSI machines.

    Get it to scale on some serious SGI kit, for example, then we'll talk.
    • Re: (Score:3, Informative)

      by PitaBred (632671)
      ...what? I think you don't know what "SMP" stands for. It has nothing to do with big iron vs. consumer kit, and everything to do with the processor configuration. And SSI [thefreedictionary.com] means nothing unambiguous as far as I can tell, at least in relation to CPU's. Perhaps you can help? The serious SGI kit you reference really only has a difference in the CPU architecture... it should scale just as well on Itanic as well as it does on x86/x86-64.

      But I'm probably rising to a troll. Or an idiot. Or both.

Karl's version of Parkinson's Law: Work expands to exceed the time alloted it.

Working...