Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Links Operating Systems BSD

Benchmarks of *BSD, Linux, and Solaris at LinuxTag 224

Posted by Nik from the unbiased-hmm dept.
AnonymousCow writes "At LinuxTag, an unbiased comparison of performance of FreeBSD, NetBSD, OpenBSD, Linux, and Solaris." I'll let Tim's comment on this story stand: "Unbiased is hard to claim - all tests can be seen as biased in their formulation - but this is thorough, with 45 slides and well-explained methodology -- BSD does very well ..."
This discussion has been archived. No new comments can be posted.

Benchmarks of *BSD, Linux, and Solaris at LinuxTag More

Benchmarks of *BSD, Linux, and Solaris at LinuxTag

Comments Filter:

  • Re:Benchmarks (Score:1)

    by Anonymous Coward

    Solaris is a long way removed from BSD. Saying Solaris is a form of BSD is just plain wrong at this point. And as for the odds being "stacked against" Linux, oh, come on, quit whining. Consider the case of I/O performance for instance. Here, the kernel's interface to drivers, and the drivers themselves play a large role. Linux has a single I/O lock, system wide, which all I/O operations must obtain before I/O is permitted. So _all_ I/O is _single threaded_. That's bad. It's kind of an architectural change to fix this, but it's being worked, or at least the infrastructure for a fix looks like it's being put in place...(the drivers will have to change to take advantage of it of course). In the meantime, expect to be routinely beaten by other OSes on I/O performance. Linux is a good OS, but, there are definitely areas where improvements could be made, and this is one of them.

  • Just dont touch the GPL! (Score:1)

    by Anonymous Coward
    Oh please, a service is no more a tangible thing than software is.

    You open sourcers are all alike. You apply one piece of logic to one aspect of a problem and then turn aound and be hypocrites about something else. Just look at Napster. You can do anything you want with other people's licensed music. Oh but don't touch my GPL'd code or I'll sick ESR and RMS and Bruce Perens and all the rest of the Goddamn Whiners on you.

  • Re:More reports like ths needed (Score:1)

    by Anonymous Coward

    I agree, good stuff.

    I use Linux myself at home, just because I wanted a "free" OS, it was there (a friend offered a CD copy of RedHat 5.0 to me), the challenges of learning something new always are fun for me, and due to the price of NT (starving college student) I figured, why not? It suits me (now at RedHat 6.2) *shrug* I am also learning *BSD's / *NIX's as well for a number of college courses (2 OS courses of differing complexity, one networking/data communications class, and programming theory), and later work experience. I like "mucking about under the hood" when it comes to an OS. I eventually will revisit M$, Novell, Mac OS's (fingers crossed that X meets the expectations of Apple fans/users) and BeOS, dependent upon the needs of my future employers.

    However, I think Bob DeNiro put it best in "Ronin". I'm paraphrasing here, but I think it was to the point of "It's a tool, that's all; you have some tools in a box, you use the right tool for the job."

    If you have a favorite "tool", by all means, use it (heh, I'm talking technology you freaks), but be open to the fact that sometimes you have to use other methods to be successful and to make your employer, boss, customer, or clients happy. Be proficent in the use of other methods. Just my $0.02

    hermit "I could have been The Walrus, but I'd still have to bum rides off people." --Ferris Bueller

    BTW, nice sig Argyle BTW, kid

  • BSD ruining open source and stealing code! (Score:1)

    by Anonymous Coward
    BSD does nothing for the open source community but drag it down. it is an inferior product, not as time tested as linux and full of coders who steal linux innovation and try to pass it off as their own.

    its time open source be made linux only so the real work and innovation can be done without hinderance.

    Linux, the choice of a GNU generation

  • Re:BSD TCP/IP Stack? (Score:1)

    by Anonymous Coward
    The code is better.
    There is real fragment support.
    The kernel -> user space movement is superior.
    It'll be better on a uniprocessor machine, and once BSD/OS's smp has been integrated into FreeBSD, it'll be better there.
  • User tests (mine) show the FreeBSD mouse pad to be far superior to the Red Hat product. The FreeBSD pad has a far smoother feel to it with the mouse gliding effortlessly over the surface. Red Hat's offering has a less appealing surface texture which grabs a little at the mouse. Micro-benchmarks indicate the Red Hat pad to offer faster overall mouse velocity however in end user tests the FreeBSD pad performs far better. The struture and color of the FreeBSD pad is more vibrant than the Red Hat pad which uses only three colors (red, black, white). The structure of the FreeBSD pad is also more organized with a firm support base and multilayer overlay. The Red hat base is thicker. The test platform employed was a standard office desk. A "Microsoft Intellimouse with Intelleye" to eliminate any chance of ball friction (funny how a software company makes better hardware than they do software).
  • The thing that shocked me was that it looks like Linux NFS did really well, when I thought the NFS stack was always considered to be pretty hacked up.

    :wq!

  • Whup, looks like I spoke too soon. :)

    :wq!

  • Solaris doesn't really do well until you give it more than 1 processor.

    :wq!

  • Really, then please tell me what the three servers I've been playing with today are. Well, okay, they're not quite that big, after all they're basically FTP servers to carry away the data generated by the big machine. (little ones: 1 PIII/800, 1G of RAM, 2 RAID-1 arrays. big one: 14 Ultrasparcs with 8MB cache for each, 14 gigs of ram, and a whompingly large RAID array on fibre channel).

    Big machines do exist, and are quite common among the set of people who keep their load averages over 1, and don't run seti or dnetc. After all, am I supposed to tell my company 'yeah, I know we've got a couple hundred million bucks in the bank, but I really think we should run the mission critical apps on cheap PC hardware'.
    ----------------------------
  • Twit.

    He said they have no bearing on reality. That's pretty clear what he meant. As for those of you who wouldn't pay $10k on a server, would you say the same if downtime costs were currently $50k/hour, and this is during the 'test' phase of the project?

    If you wouldn't, then I'm damned glad you don't work with me. $10k is a bargain if it saves me from one outage.
    ----------------------------

  • To All of Those who Say DMA (Score:3)

    by mosch ( 204 ) on Monday July 24, 2000 @04:16PM (#908595) Homepage
    Lots of people are saying Linux didn't have DMA turned on, and that's the reason for the low scores. You can see on Slide 41 [dynamine.net] that hdparm is mentioned with the -d flag. I think that means that he turned it on, ladies and gentleman.
    ----------------------------
  • The numbers Linux puts forth are absolutely anemic and, considering EXT2FS has been proven in other tests to be slightly faster than UFS/FFS, it's obvious that DMA wasn't turned on for these tests.

    I imagine, with DMA turned on (as it is by default, of course, with FreeBSD), Linux would've looked much better in these tests (and probably most of the others too since they all involve disk transfers to some degree.)

    - A.P.
    --


    "One World, one Web, one Program" - Microsoft promotional ad

  • They didn't turn DMA on in Linux -- this would've improved hard drive performance by a factor of at least 3.

    This obviously would've affected Linux's showing in nearly every test.

    Maybe they should re-do this thing. And get rid of the horrid jpeg slides.

    - A.P.
    --


    "One World, one Web, one Program" - Microsoft promotional ad

  • Um, these results look flawed. (Score:3)

    by Wakko Warner ( 324 ) on Monday July 24, 2000 @03:31PM (#908598) Homepage Journal
    Not that I'm just saying that in some "rabid linux zealot" fashion which is pretty typical of these kind of damning "benchmarks", but it looks like they forgot to turn DMA on under Linux.

    This would kill performance in nearly every test.

    The hard disk appears to max out at around 9MB/second in BSD but only around 4 in Linux, which is odd because it's been proven in other places that EXT2fs is at least a little bit faster than the BSD filesystem (even with async turned on).

    Maybe they should redo these tests?

    - A.P.
    --


    "One World, one Web, one Program" - Microsoft promotional ad

  • Update on the site (Score:3)

    by Wakko Warner ( 324 ) on Monday July 24, 2000 @04:45PM (#908599) Homepage Journal
    I don't remember seeing this the first time around:


    small update: some updates about the bad io results of linux will follow after more tests with different ide drivers (which
    might be the reason for the extreme difference)


    - A.P.
    --


    "One World, one Web, one Program" - Microsoft promotional ad

  • It shows what linux groupies always try to deny... The *BSD's perform better under heavy I/O.

    Unless, of course, your heavy IO involves NFS, HTTP, or parallel compiles. Or did you just look at the first two slides and rush back to post?

  • To me it menas that he didn't turn it on and considers this one of the things that can still be tuned. Look at the other tuning items on the list on slide 42, and you will see that it is so. IMHO, at least.

    --

  • I know this was a test of limited scope, but other than a home web server, who runs a "real" server on one processor and an IDE drive? I would be much more interested in a test using a plain adaptec 2940UW with a fast disk (10k rpm) and at least two processors.
    Solaris is not aimed at uniprocessor/IDE machines. That hardware setup should be reserved for a test of workstation benchmarks. A test of "real" servers should include more than one client and much higher loads than the machines were subjected to. After all, can you consider it a "real server" if there's only one client doing read/writes?

    Warning: I am a Sun employee. I try to keep my biases in check, but I am only human.


    _damnit_
  • % uname
    Linux

    % ls -l /lib/libc-2.1.3.so
    -rwxr-xr-x 1 root root 888596 May 1 20:26 /lib/libc-2.1.3.so

    % file /lib/libc-2.1.3.so
    /lib/libc-2.1.3.so: ELF 32-bit LSB shared object, Intel 80386, version 1, stripped

    /mill
  • The vast majority of NFS is done over UDP, not
    TCP. TCP stack idiosyncrasies/incompatibilities
    would not affect the results.
  • Sure, for servers that's good, but I want bleeding edge. It's just more fun and interesting.

    Use an unstable branch then?

    Plus, need I say much more than "Quake"?

    Yes, apparently. The Quakes run just as well on FreeBSD as they do on Linux.

  • No need to be snotty... (Score:5)

    by slothbait ( 2922 ) on Monday July 24, 2000 @03:04PM (#908606)
    > FreeBSD... The choice of those, who know how to choose...

    You sound like a few wine snobs I know.

    FreeBSD may have it's technical merits, but I know plenty of people who run it just for the "fringe appeal". These are the people that ran Slackware way back because it seperated them from the crowd.

    After the Red Hat IPO, Linux officially arrived and was no longer properly counter-culture. Once big money was involved, Linux became establishment, and that's just not chic. Then the people who ran Linux just to be different had to look for a new system -- luckily for them, the BSD's provide an easy migration. (yes, I know that BSD dates back to before Linux)

    I feel some of this effect myself. I confess that part of why I use Debian is to seperate myself from the kids who purchased a RH boxed set at CompUSA, clicked their way through an install and now proudly play Solitaire in Gnome.

    In the old days it *did* mean something to hack together a functional system from the disk sets that Slack provided, or that you downloaded by hand. There was a base level of proficiency that was necessary even to become a Linux user. This exclusion is rapidly breaking down in the Linux world, but lives on in the BSD world.

    I'm not sure that any of the BSD's want to lower the entry barrier. Arguments of elitism aside, lowering the barrier definately lowers the mean competence of your user base, and I think that's something FreeBSD and the others would rather live without.

    It seems to me that a lot of the *BSD users on Slashdot just wait around for good news about their system so that they can triumphantly reveal how superior their tastes are. It gets a little old.

    FreeBSD... Because Linux just isn't leet enough anymore.

    --Lenny
  • DMA is part of the protected mode disk drivers, along with 32-bit operations.
  • I guess Roblimo was right [slashdot.org].
    Some people in the design business say the best way to learn what the WWW will look like in six months is to keep up with Jeff [Zeldman]'s famous www.zeldman.com site.

    Or are all these BSD guys color blind?
  • BSD, Linux...
    Which serves web pages fastest?
    Aaah! Orange! I'm blind!
  • Someone correct me if I am wrong, but compiling has alot more to do with the compiler, libs, headers then the actual operating system. I am guessing that linux has alot larger/more libs and headers to take into consideration when compiling an application then either solaris or *bsd. Second I think NFS was a bad choice in testing network performence, NFS is unique to the OS, and as such it is testing the OS specific implmetnation of NFS more so then the OS (Not to mention Linux is WELL KNOWN to have crappy NFS). Samba would of been much more useful in benching the operating systems ability. The IO difference between freebsd and the other operating systems looks rather like what I see with UDMA disabled :) I get better performence marks on a pentium 200 with UDMA enabled (under linux). I would agree that this benchmark isn't biased, it's just poorly done.
  • And your point?
  • Uhmm. I didn't say as much. The header and libs are not the same on Freebsd as they are on linux.
  • Have you not heard of package management? You know, dependency resolution, etc.?
    --
  • Damnit! That wasn't a troll! That was damn funny. Especially the,
    Their current version, 5.0, is not their release version; their released versions have frequently in the past not been stable; and their stable versions are not current.
    hahaha
  • Absolutely! If you are gonna spend that kind of dough on hardware, get a Sun box, or a fridge sized AIX beast. Get real heavy iron. Who buys the giant intel boxes except ZDNet for an NT test?
  • There is no such thing as a $500 Sparc, unless you are using an old Sparc 5/10, circa 1995.

    That is one of the best things about this comparison: it's in crappy hardware. I am sick to death of these useless 4CPU, 4G of RAM, 12 disk RAID array tests that just have no bearing on reality. Give me $500 of intel crap, load it with pr0n, and see which one the 'net kills first. THAT would be a test!
  • I don't think RedHat's 'fancy-looking-ness' should be mistaken for ease of use. The user friendliness of FreeBSD is the reason why I use it at home. The config tool sysinstall actually works, the BSD package manager actually works, all unlike the bits RedHat added to linux.
    It also annoyed me no end, what a mess RedHat makes of the directory tree, with separate library directories, binary directories etc for every package, which requires you to keep long lists of environment variables set (QTDIR=.., KDEDIR=.. etc).
    BSD just keeps the libraries in the libraries dir, binaries in the binaries dir, packages not part of the OS go under /usr/local.. etc. All package information goes under /var/db/pkg, so files don't get lost, and the uninstall tool actually works. My guess is it works well, because FreeBSD is being developed and maintained as an entire OS. Linux is a kernel, and the usability really depends on what a certain distro company wanted to add.

    The one thing were Linux is easier on the user, is make xconfig, whereas BSD requires you to edit a textfile, but I don't think Redhat had a lot to do with that tool, or did they? I know I should really try a different Linux distribution, to give it a fair chance, but since I haven't needed it yet, I haven't gotten around to it yet. One thing is for sure: it won't be RedHat.

    Reiserfs looks promising, I'll probably use linux for that fileserver I'm setting up.
  • I agree, and in addition to that, the Solaris x86 CDs perform exceptionally well as beer-mats.
  • Actually, the sysinstall is a nice, menu driven program. It's just not graphical, which means it will also work without X and Gnome. Packages can be installed via the ports (this will mean compiling, except in the case where the ports are only available in binary format, like netscape), or as binaries with pkg_add.

    It would be a very good idea if you read some docs on XFree86 configuration... you can physically damage your monitor and your videocard if it's done wrong by the configuration program, as appears to have been the case with you. The autogenerated XF86Config file looks intimidating, but most of it you don't need at all. In the case of XF86-4.0 it's been made a bit easier too.

  • What fun is that?

    Sure, for servers that's good, but I want bleeding edge. It's just more fun and interesting.

    Plus, need I say much more than "Quake"?

  • oh. fine, then.

    I actually like BSD. I was just being a snot. (bored at work today. Don't often get the chance to be bored, so, I thought I'd do some trolling)

  • Remember when SLS was king? Ahhh, the good old days.

    And taking over a PC lab at 1 in the morning, downloading all 50 disks, dedicating a row just to format floppies (when unformated disks were cheaper (and AVAILABLE!)) and another to ftping to tsx-11.mit.edu to get the most recent SLS, getting it home and finding out disk N was bad (where N=the most critical disk possible.) Ah, yes, the days where men were men and university computing departments wouldn't give out SLIP accounts because "you can't have just anybody writing raw IP packets to the Internet! *gasp!*"

    I miss those days sometimes, then I realise I can just grab a CD of anything for next to nothing and have a pretty decent OS to fuck around with.

    Yeah, I was a teenage Unix hacker, sue me.
  • Am I going blind or are all those graphs damned near impossible to read? They were done in JPEG format which is sub-optimal for graphs and text. I found it almost impossible to correlate the legend with the data points. It sure would be nice to see them in PNG format.
  • Well said. I think all of the *nix'es have their specialties. I use Linux on my laptop, FreeBSD on my desktop and web servers, and OpenBSD on my firewall.

    I will say this, though. New Linux users need to be very careful in their choice of distributions. You'll find that the security, performance, and reliability does vary from Linux distribution to distribution.

  • Re:Well.. (Score:3)

    by Jeffrey Baker ( 6191 ) on Monday July 24, 2000 @02:10PM (#908625)
    Yeah, 2.4 scales a lot better than 2.2 on paper. In reality, the network stack in 2.4 is superior to that in 2.2. However, disk I/O is sucking in all ways: IDE, SCSI, software RAID, everywhere. Many people have reported I/O performance down 60% or more from Linux 2.2 on linux-kernel and linux-raid mailing list.

    I think you should be more cautious in your cheerleading. 2.4 has potential, but the realization of that potential is diffcult and not gauranteed.

  • IIRC, bonnie uses the stock getc() and putc(), which under Linux are threadsafe. These functions acquire a lock in userspace, and so have more overhead than is necessary for a single-threaded application. As Doug Ledford explains on this page [redhat.com], bonnie performs much more realistically when modified to use the non-threadsafe versions (which is a valid thing to do in a single-threaded application).

    --Joe
    --
  • I know what you mean. However, there is hope. After hearing all about the better security of OpenBSD for quite some time now, I decided to take the plunge this afternoon: I've ordered my official OpenBSD CDs and they should be here soon! I can't wait to play with it...
  • It's called Nightmare File System for a reason you know. =)

  • weak. (Score:1)

    by juuri ( 7678 )
    Catch many with that der' pole?

    Whats up with trolls these days being so blatently obvious. You trolls need to goto some creative writing classes or something... hell spend some time on USENET. It can only help.
    ---
    Solaris/FreeBSD/Openstep/NeXTSTEP/Linux/ultrix/OSF /...
  • This highly depends on your chipset. With my Aopen Board which runs the old MVP3+ (Apollo?) chipset. I can get 27 megs a second with my UDMA66 drives.

    Ony my DFI motherboard I can not get above 12.
  • Quote:

    Nah, the mention couldn't actually be there because he did NOT use it.

    *scratch* Nah.

    End quote:

    The slide on which hdparm was mentioned was in a series of slides at the end which had a rather speculative tone ie "these are things we might think about". It doesn't say it was used, it IMHO implies in fact that it wasn't.

    I'm pleased to see you know what sarcasm is, but it's not a very useful first step to convincing people that you're right.

  • DMA problems? (Score:3)

    by Colitis ( 8283 ) <jj.walker@NOSpaM.outlook.co.nz> on Monday July 24, 2000 @02:52PM (#908632)
    The patterns in the disk read performance figures look suspiciously like the sort of gaps you get with and without DMA enabled on a disk. I note that the motherboard is a VIA board; Linux is known for having problems with DMA on via boards. Also, depending on distro, some may or may not enable DMA by default on bootup.

    My opinion is that the BSDs having in the range of (say) 10% improvement over competitors would be easily explained by possibly better file system and VM architecture. But when we see a difference of five to one surely there's got to be something seriously wrong there. I've gotten better bonnie figures than that on an old Pentium.
  • Quote:

    Lots of people are saying Linux didn't have DMA turned on, and that's the reason for the low scores. You can see on Slide 41 that
    hdparm is mentioned with the -d flag. I think that means that he turned it on, ladies and gentleman.

    End quote:

    Unfortunately, it's rather typical behaviour of DMA driver problems for DMA to be turned on, then turn itself off again soon afterwards when it gets a timeout error. And yes, I did notice that slide, but it doesn't state that option was actually used during the tests, does it?
  • Can anyone explain why NFS performs better on Linux with a Linux client and better on FreeBSD with a FreeBSD client?

  • Re:Well duh! (Score:3)

    by orabidoo ( 9806 ) on Tuesday July 25, 2000 @12:30AM (#908635) Homepage
    seite 34 is one of the few where the Y unit is "time taken", not "amount processed per unit of time", which means that the *lower* line is the best one there. it's a bit surprising, but on seite 34, it's NetBSD that outperformed everyone. same thing on seite 32 (the SQL tests): Solaris was actually the slowest, and Linux the fastest.

    in any case, we should take these tests with a large grain of salt. there are MANY factors unaccounted for: driver quality for the hardware used on each OS, IDE settings (hdparm on Linux), choice of filesystem, sync/async mounts, softupdates or not on BSD, journaling or not, and the large can of worms that is kernel tuning (did they do any?).

    at least there are a few things we can tell kind of reliably from these tests: 1) the BSDs and Linux are all great; Solaris/x86 is generally slower (but may have better SMP, which wasn't tested here), 2) Linux 2.4 is a real improvement over 2.2, and 3) nfs and network stuff is faster when the client and the server use the same OS flavor.

  • Don't believe about the code quality? well, the BSD C lib is half the size of the GNU lib C, with the same functionality..


    This is the second time I've seen this comment of 2x. It's a crock of shit:

    % uname
    Linux
    % ls -l /lib/libc-2.1.2.so
    -rwxr-xr-x 1 root root 4118299 Sep 20 1999 /lib/libc-2.1.2.so

    % uname
    FreeBSD
    % ls -l /usr/lib/libc.so.4
    -r--r--r-- 1 root wheel 537268 Jun 12 00:26 /usr/lib/libc.so.4

    Looks closer to 10x to me...
  • Hey, what's wrong with Solitaire in GNOME???
  • hdparm doesn't enable/disable DMA itself, that requires kernel support for the chipset. It will set the mode for types of DMA transfers, and can disable DMA. Using hdparm on a kernel which doesn't have DMA support will not do anything to the DMA characteristics of the drive.
  • I agree with the above comment. The filesystem performance differences should be at WORST a magnitude of 100-200 % worse/better in either case. BSD by default enables DMA, (for detected/supported chipsets, in my experience), while linux tends to require specific support for the udma transfers (recompiled UDMA patched kernel, my experience). Also, the hdparm -unmask option for servicing other interrupts greatly improves the overall responsiveness and feel of the system (in X, once again, my experience).

    It doesn't appear these tuning parameters were applied, so I would expect any linux i/o bound results do be dog slow relative to any of the other OS's. The solaris results are so similar to the linux results for prolly the same reason... SCSI would have been a better test for this one benchmark.

    --Adrian
  • How does one turn on DMA in Linux?
  • One nitpick (and I have FreeBSD 4.0-Current on one machine and Debian potato on another so don't say I'm biased) but the BSD libc does *not* have all the same functionality as glibc (at least, not that I could find). Some examples are argz vectors and obstacks (neither are really necessary, but they exist in glibc and not BSD libc, i know, i was planning on using both and needed to rewrite the functionality for FreeBSD cause it wasn't in the libc).

    The BSD's are a great system, but the ports tree isn't all it's cracked up to be (it NEEDS an upgrade procedure, I'm sorry but apt-get is the best package method out there). Fix the ports tree and SMP (I don't care if Linux's is sloppy, it works today) and get support for bleeding edge graphics cards (I like being able to play my video games too =) and I'll switch completely.

    (No this isn't a flame, just honest criticism from someone who uses both on a daily basis, and both as workstations, not servers).
  • You're right, the ports collection is nice, but it NEEDS a good upgrade system. Uninstalling something like the gnome just takes too long and I've yet to get everything uninstalled properly to do an upgrade. apt-get is my current favorite for installs/upgrades. As easy as ports, but it can upgrade everything too.
  • I've been benching our machines, and the numbers he gave are typical for DMA turned off. When I turn DMA on my home machine on, it goes 3 times faster. I get ~10 Meg/sec on my IDE drive.
  • Which is why you're obviously an amateur and not a professional working in the glass room.

    At your level, saving money is important and whatever job you're doing can be done on trash hardware, and the people who write the checks don't care what you do so long as it doesn't cost any money. Don't be insulted, I've had those jobs before and there's a certain independence to them. But it's not real data processing.

    In the major leagues the concern is with meeting the business goals, and that means getting the transactions through the system on whatever hardware is the fastest and most reliable. That usually means big(ger) iron than these tests use, and yes, it often means Sun, IBM or HP enterprise hardware.

  • and here's our crappy Powerpoint display... Ick! I'm not sure if I should run out and install the Brownish redline with triangle or the Blueish grey line with diamond! Someone tell me which is best? Man whats with the techie Yuropieons and their lack of color matching skills? Would someone please write an open source version of the RBG color spectrum so the rest of the nonCOLORBLIND WORLD could read stuff like this!
  • Oddly enough, with 2.4.0-test4 I'm seeing the opposite, a 60% disk performance increase, much better than 2.4..

    Geoff
  • What's the deal with these damn slides! They suck ! What's wrong with just posting the results as a simple text file or plain old HTML????

  • amazing (Score:3)

    by cpeterso ( 19082 ) on Monday July 24, 2000 @02:49PM (#908648) Homepage
    the I/O performance varies from development kernel to development kernel and they're still tracking issues down (ie, someone reports 200% improvement on one kernel, but says it's worse in another.. really amazing actually).

    What this means is that the Linux developers are playing with black magic and don't really know the effects of their performance tuning. Each dev kernel is a "hmm, does this fix the problem? or what about this fix?".

  • I think you're right here.
    It's the same thing with Linux vs. Windows; people tend to say Windows is easier to install. I tend to say it's because it's different and difference doesn't mean difficult.

    Actually I'm a Linux freak, but an *BSD doesn't scare me. The BSD's are cousins to Linux, therefore they are family, thus, important.


    Best regards,
    Steen Suder
  • Solaris really performs on sparc.
    And it's not designed for 'serving pages'. THat is a computationally light task.

    Try a real server, serving many users on interactive X sessions at once, all with wierd simulation packages running. Solaris reigns.

  • Re:Well.. (Score:1)

    by nd ( 20186 )
    Cheerleading? Perhaps you should check the latest Spec99 results. Quite impressive, disk I/O suckage notwithstanding.

    Actually, as I understand it from following the LKML, the I/O performance varies from development kernel to development kernel and they're still tracking issues down (ie, someone reports 200% improvement on one kernel, but says it's worse in another.. really amazing actually).

  • Re:Well.. (Score:2)

    by nd ( 20186 )
    > The *BSD's perform better under heavy I/O.

    I'm not so sure this is the case had they compared against 2.4 rather than 2.2. But, the goal was to compare against production kernels, so it was a fair test I guess.

    2.4 does scale WAY WAY better than 2.2, though.
  • Well, If you will read at the bottom of my post, DMA is turned on for the numbers I posted. The numbers would have been much closer, but I was playing MP3's at the time. They're is still a 1.5 second difference today with no MP3's playing.

    Could be something wierdwith this PC, but that's the straight scoop from here.

  • Re:To All of Those who Say DMA (Score:3)

    by HamNRye ( 20218 ) on Monday July 24, 2000 @06:03PM (#908654) Homepage
    I'm with this guy on the DMA issue, bu I also noticed that they never set the IDE drive to use 32 bit access. Here's some sample output:

    pharlap:~ # hdparm -c /dev/hda
    /dev/hda:
    I/O support = 0 (default 16-bit)

    pharlap:~ # hdparm -t /dev/hda
    /dev/hda:
    Timing buffered disk reads: 32 MB in 7.48 seconds = 4.28 MB/sec

    pharlap:~ # hdparm -c 1 /dev/hda
    /dev/hda:
    setting 32-bit I/O support flag to 1
    I/O support = 1 (32-bit)

    pharlap:~ # hdparm -t /dev/hda
    /dev/hda:
    Timing buffered disk reads: 32 MB in 4.42 seconds = 7.24 MB/sec

    These benchmarks were taken while playing MP3's, so the numbers will be low. DMA was/is enabled.

  • Re:There is no "best" system after all (Score:3)

    by Azog ( 20907 ) on Monday July 24, 2000 @02:56PM (#908655) Homepage
    Obviously, people didn't read the tests very carefully.

    Solaris had big numbers for the SQL tests. Unfortunately, those were in seconds -- so Solaris actually did very POORLY on the SQL tests.

    Please, everyone, read carefully. There's some good info on those slides but it is not clearly presented.

    Torrey Hoffman (Azog)

  • Solaris x86 (Score:4)

    by arodrig6 ( 22052 ) on Monday July 24, 2000 @01:55PM (#908656) Homepage
    It looks like they used Solaris x86, which is not as finely tuned as Solaris/SPARC. It would be intersting to see what Solaris on SPARC could do against linux, assuming they both had similar hardward cost constraints.

    Also, I thought an IDE hard drive an odd choice for a high performance server test - wouldn't SCSI or RAID be more appropriate?
  • How can solaris suck at everything if it poststed the best postgresql scores?
  • Don't forget that the main reason Sun maintains x86 port is to get students, geeks, developers, etc interested in their Solaris platform. As far as Solaris production systems are concerned, most of them are Solaris on ultrasparc.

    Also the later versions of the OS really need more RAM to operate efficiently. 64MB? This is a joke for Solaris. I bet there would be a -significant- performance increase if they used ~128MB.

    IDE drives, another joke. Solaris IDE drivers never were good (look even at the ultrasparc based Ultra5/10 with IDE disks). On intel they gotta be worse by definition.

  • Why is this topic was posted in BSD cathegory?

    *BSD weren't the only systems featured in the test and in addition while they performed well, they didn't beat the other OSes in all tests, just in some of them.
  • Well, i'll tackle most of the responses i saw

    Why?
    well, am bored, and i stopped working for a while.

    * The *BSD's have just become the latest thing you show you are knowleadgeable, just like linux was a few years back...

    WHile this could be true, i personally don't think so. Most New FreeBSD users can be categorized into two main sectors. One, ex linux users, that want to try it out, and see what is all about. Mostly because of problems they've had with linux, other times because they just want to try something new.Two, current Solaris,commecial unix users, that are looking for a reliable platform, on relative inexpensive hardware.

    If you're somehow wondering my background, my first unix experience was on solaris, back in 1993. FreeBSD in early '96. Linux, debian, in mid '97. My preference... isn't it odvious?

    * 2.4 will close any gaps in heavy I/O FreeBSD may have an advantage in

    EHHHHH, nope. At least not in my testing or anyone's testing... 2.4 is slower on many things than 2.2 is.

    * SMP is better on Linux

    One comment... the difference between doing it right, and doing it slopy is a great one my friend.

    * Linux dev is Free'er than the BSD's

    In linux kernel dev, you are subjected to the views of two main controlling forces... Linus and Alan... FreeBSD... about 15 core team members, with greatly equal vote.. plus the rest of the commiters.. plsu everything is available right away on CVS, unlike the linux kernel... AGAIN BRANCHES MY FRIEND...


    FreBSD just follows a more mature approached to developement... I advise you to read up on it.. Linux is still the hobbyist OS that it started as.. and hopefully that will change.

    Don't believe about the code quality? well, the BSD C lib is half the size of the GNU lib C, with the same functionality..

    Well, i don't wish to change anyones preference.. You use the best or the job... and on many tasks... But that doesn't change the gfact, that FreeBSD is still a better engineered system.... again... face the facts.. i mean... damn kernel httpd? ok, give me a break, maybe on a micro kernel arch, but on a monolithic? maybe am wrong... heh

  • scrap that last paragraph,, i think i omitted a line or two... :P
  • It shows what linux groupies always try to deny...

    The *BSD's perform better under heavy I/O.

    Unlike the general linux mentality of, get it out the door if it barely works, the BSD mentality has alwasy been... it only goes out, if it's ready, and mature enough to go out... Hence the different release branches...

    FreeBSD... The choice of those, who know how to choose...


    Flame filters... ON... you can flame all you want, but you cannot change the facts.

  • It took me awhile, but I finally managed to get a mirror of the site up. Hopefully this should ease the Slashdot effect. It will be up unless the author asks me to remove it.

    The URL is http://www.dynamine .net/mirrors/innominate.org/projects/tuning/ [dynamine.net]

    The server is on a 100Mb Exodus link, running FreeBSD 4.0. Have fun!

  • "Their latest version, 5.0, is roughly comparable to Linux 2.3. Since the current version of Linux is 6.2, this is clearly unacceptable"

    eh? the current version of linux is 6.2? freebsd 5.0 is comparible to linux 2.3? sounds like someone else who has fallen into the "redhat=linux" myth, yet is also confused in other ways too. Geeze...how long has it been since redhat 2.3? was there ever one? (lol) I know I used to use slack 2.3...(shrug) current version of REDHAT is at 6.2. Current "version" of Linux (stable) is 2.2.16

  • Re:OS bias in NFS? (Score:4)

    by pingbak ( 33924 ) on Monday July 24, 2000 @02:38PM (#908682)
    I think one could attribute this to network stack idiosyncracies. For example, if the Linux stack only sets its recv buffer to some specific size that's smaller than the *BSD size, would account for the difference. IIRC, Van Jacobson pointed out a number of problems with the TCP implementation doing weird things in slow start and congestion avoidance that might also account for impedance mismatching.

    A similar problem cropped up when Samba was benchmarked against NT's SMB implementation.

    -scooter
  • Looks like I'll be installing some flavor of BSD on my spare machine. Last time I tried Linux, it was just so unorganized and non-standard (yes, yes, I know LSB, but still...). I'm hoping the BSDs are a bit more tame as far as organization and standardization. I would even take whatever performance hits there were with NetBSD if it is actually written and designed elegantly enough that someone as rusty as me in C and C++ might actually be able to tinker with it (I am assuming most Linux users don't even dare putzing directly with their kernel). I wish there were some force for convergence instead of divergence as far as Linux distributions are concerned. Maybe that LDPS will help. It just seems that nothing really has enough teeth to tame the wildly fragmenting Linux distributions (how about a single package standard, a single update standard, even compliance with LSB?). I don't know, maybe I'm just talking out my ass.

  • Nice benchmark (Score:3)

    by Chalst ( 57653 ) on Monday July 24, 2000 @05:50PM (#908695) Homepage Journal
    There is a very thorough benchmark [usc.edu] comparing Linux (kernel 2.2.12) to
    FreeBSD (4.0). The benchmark takes time to analyse file system
    performance, kernel timings such as contexts switches and use of
    memeory managers and thread/process creation, all tied up with an
    excellent summary.
  • The problem is that most systems ship with DMA off. It causes problems in some configurations, and is usually a liability for the vendor.
  • This compared to the Linux crowd whose benchmarks are, "oh, I just installed this and have only used Windows before and Linux is the fastest thing I've ever seen," or "on my 386 33 with 4 MB of RAM, Linux absolutely BLOWS AWAY Windows 2000" or my favorite "I've been using Linux ever since kernel .9 and its so fast! I mean, I played around with the Win2K alpha release, and Linux is SO much faster. What WM am I using? I don't run X, what a silly question! What do you MEAN that my comparison is invalid?"
  • You know it just hit me.
    How does one turn on DMA in Linux?
    "Compile the right chipset into the kernel, then play with hdparm."
    How does one turn on DMA in Windows?
    "Click on the DMA enabled checkbox in device manager."
  • Of course, VIA chipset's also required an upgrade to the kernel as I remember, Not MS's fault that VIA chipsets tend to be flaky. Even still, download drivers, or recompile kernel. I can probably choose which one I want to do on a rainy day ;)
  • Actually, over the years, MaximumPC's editors have suggested all manners of ways to punish vendors who ship with DMA off. I think one involved a ferret and a wet noodle...
  • Linux goes through the same levels of development as any other operating system, the difference is that each step is available to everyone, and not locked up behind closed doors.

    Oh, please. That troll is very old now. Exactly what do you base that on? The development of Free, Net, and OpenBSD is just as open, no, more open, than the development of Linux. With Linux, you get only the snapshots that Linus or Alan feels that you should get- with FreeBSD, you can get from CVS the version of any given day, hour, minute, and second.

  • Re:No need to be snotty... (Score:3)

    by vyesue ( 76216 ) on Monday July 24, 2000 @03:49PM (#908713)
    I'm not really sure that the barriers to installing BSD are as high as you seem to be implying. if you can insert a cdrom and read english, you can install openbsd in a few minutes. the only entry barrier that I see is that everyone and their cousin is obsessed with linux and not a lot of people have even heard of the bsds, comparitively speaking.

  • Well duh! (Score:4)

    by randombit ( 87792 ) on Monday July 24, 2000 @01:48PM (#908725) Homepage
    OK, not trying to start a flamewar here (though I think that for this story, that may be an impossiblity). But was anyone really expecting *BSD to just roll over and die in the face of Linux? BSD has one badass TCP/IP stack, and it can't be much of a suprise that it did well. Keep in mind, readers and moderators, that I mostly use Linux. I'm not trying to be a BSD evangelist. I feel that FreeBSD 4s user-friendlyness is fairly low comared to, say, RH 6.2. But OTOH the *BSDs, especially FreeBSD, seem to really try and make their networking fast as hell (and the succeed).

    Simliarly, the fact that Solaris cleaned up in the SQL test shouldn't be suprising.

    BTW, some of those graphs were really hard to read, but in some didn't 2.2 wildly outperform 2.4? Specifically Seite 34, parellel compiling, real time. I'm confused!

  • abuse (Score:2)

    by emir ( 111909 )
    thats why there is meta-moderation, if you are modded down unfairly than you will get moderated up when meta-moderation steps in , and the guy that moderated you down unfairly will get - points in karma.

  • Re:There is no "best" system after all (Score:3)

    by softsign ( 120322 ) on Monday July 24, 2000 @04:55PM (#908752)
    In all fairness to Solaris, it IS Solaris-x86. Although they're based on most of the same code, x86 and SPARC Solaris are worlds apart in terms of performance.

    I'm always wary when I see a test such as this that says "let's keep the hardware the same and compare just the software". There's a very dangerous illusion of equality which is just a fantasy.

    Linux may have better disk access -- for IDE drives. But if FreeBSD handles SCSI better, is it fair to conclude that Linux is the better OS simply because you only used IDE drives in your comparison?

    The reality is that you can't compare OSes in any meaningful sort of way and generalize the results to say "X is better than Y". To be scientific about it, you have to say "X is better at Y, only if run on <this_platform>, and then only at doing <this_task>, etc..."

    And to be fair, the guy who did this study, pretty much said as much. His tests are basically valid for anybody with a stock K6-450 PC wanting to run a free Unix(like) system and possibly a web server.

    Somehow, this gets transmogrified into "thorough" and "unbiased". =) Not that I'm disputing the results, I run FreeBSD myself. I just think it's important to take a step back and look at the whole picture.

    --

  • Testing? What's that? If it compiles, it is good, if it boots up, it is perfect.
    -- Linus Torvalds

    This has to be my favorite argument with linux tcp/ip

    http://www.spatula.net/proc/linux/localhost.src

    With the linux development process the users are the alpha/beta testers.

  • VIA Apollo driver (Score:3)

    by Furry Ice ( 136126 ) on Monday July 24, 2000 @02:51PM (#908765)
    It seems that the test system was a K6-400 with a VIA ide chipset. It was probably an Apollo Pro, which isn't very well supported by Linux at the moment. Examining the filesystem performance graphs shows around 3 MB/s throughput, which is exactly what I was getting out of my Apollo before I got UDMA working with it. You have to apply the IDE patches to 2.2 to even have it available, though it's in the 2.4-test series. You need to enable support for VIA82CXXX, ATA Works in Progress, Use PCI DMA by default when available, and VIA82CXXX Tuning support (a work in progress). After doing these things, I get 22 MB/s (!) out of my IBM, and 19 MB/s out of my Western Digital. If the Linux kernels had been compiled with these options, the results of the tests would have been quite different, I'd say.
  • Why didn't they include Windows NT or 2000? Aint they sposed to replace unix in the future?
    Adam's Preliminary Page of BANG~!

  • There is no "best" system after all (Score:4)

    by achurch ( 201270 ) on Monday July 24, 2000 @02:25PM (#908797) Homepage

    Questions of bias aside, I think this benchmark makes it pretty clear that there's no such thing as a absolute "best" system--even the author says as much in his conclusions. While the *BSDs performed very well for filesystem I/O, Linux did nicely in the HTTP tests and Solaris topped everyone in SQL performance.

    Also, something the holy-war people seem to forget is that even the comparatively badly-performing systems are more than sufficient for the majority of users. How many people, for example, serve more than 10 dynamic pages per second?

    I guess what it boils down to is use the one you like best. Linux, *BSD, Solaris, etc. all have things to recommend them, which all appeal to different people. I personally have the most experience with Linux--I was introduced to it first--so it's what I use daily, but I've had my eye on FreeBSD for a while as well. (No spare computer to put it on yet, though...)

    And think of the irony of trying to tear down a Windows monopoly only to replace it with a {Linux,FreeBSD,...} one. Competition is good, and variety is the spice of life. (^:

  • 1) the IO numbers look very suspect, Linux was always capable of saturating IO bandwidth up to 60MB/sec even on modest hardware. This is so basic that they should have stopped the test when they saw how far the block IO numbers are apart. It certainly does not show alot of experience in tuning Linux.

    2) the 'per character' numbers of bonnie are utterly meaningless. Bonnie is not a smart benchmark tool, but it gets quoted often. The 'per char' numbers simply measure the performance of libc's "getc()" function [and both Linux and FreeBSD use glibc.]. So the effect of the 'per char' measurement only slows Bonnie down and skews the numbers with additional CPU time. In fact i use a hacked Bonnie that just does the 'block IO' numbers - looking at the 'char IO' numbers is a waste of time.

    3) While i understand the deadline issues, the 2.4-ac series were seriously buggy. It's only 2.4.0-test5 that started behaving properly, VM and IO-wise.

    4) Linux's name is 'Linux', not 'linux' - they got it right with 'FreeBSD', so it's not hard :-)

    5) the apparent IO misconfiguration then reflects in all the 'high load' numbers, so all the slides (except maybe the network numbers) should be redone IMO.

  • Anyone else notice NFS performance for *BSD to to *BSD to be significantly faster than Linux to *BSD? Same goes for Linux to Linux. Hmm...

    */

Slashdot Top Deals

FORTRAN is not a flower but a weed -- it is hardy, occasionally blooms, and grows in every computer. -- A.J. Perlis

Close