NetBSD Focuses On Scalability

An anonymous reader writes "Felix von Leitner recently performed some benchmarks (previous story) for a talk about scalable network programming he held at Linux Kongress 2003. The winners in this scalability lineup were Linux and FreeBSD 5, followed by NetBSD and finally OpenBSD. What's interesting is that in only two weeks time the NetBSD team made dramatic improvements. Felix performed his benchmarks again and the results are nothing short of astonishing. NetBSD now has better scalability than FreeBSD." Read on for a list of improvements.

the submitter lists these changes:

• socket: previously O(n), now O(1).
  
• bind: greatly improved, but still O(n). Much less steep, though.
  
• fork: a modest O(n) for dynamically linked programs, O(1) for statically linked.
  
• mmap: a bad O(n) before, now O(1) with a small O(n) shadow.
  
• touch after mmap: a bad strange graph in 1.6.1, a modest O(n) a week ago, now O(1).
  
• http request latency: previously O(n), now O(1)

This is a very good job from the NetBSD team! I hope to see more benchmarks and more improvement for a great OS like NetBSD."

Target

What's interesting is that in only two weeks time the NetBSD team made dramatic improvements.

Colour me cynical, but just maybe the improvements are targeted to produce a better benchmark rather than broader scalability.

Tell me I'm wrong.

Re:Target

um... the benchmarks in question all have to do with improving web server performance, (specifically, the author's pet project of a web server) so does it matter whether or not the goals *are* targeted to produce a better benchmark, if the *results* end up being broader scalability?

Benchmarks are great tools to use for improving performance, and as long as you don't have to cheat to do better, (like some major video card companies who shall remain nameless) improving your scores on a good benchmark largely equates to improving performance across a whole host of applications.

If you'll remember, the same thing happened with the Netcraft debacle; performance deficiencies in Linux wrt. NT were highlighted, and fixed, and Linux is the better for it, with even faster webserving, and a better TCP/IP stack. I don't care about the alleged reasons, I care about the positive results. :)

Re:Target

Yup, you're cynical.

NetBSD's team is scary good. There are some advantages to keeping a tight core development team, and if the team is good, one of them is quality.

In my experience with NetBSD, when they do something, they do it right.

Let me put it to you this way.

Say there was this huge alien spaceship coming from outer space to blow up the white house and use us as food. We'd need to send a rag-tag group of crazy operating system geniuses up into space in a rocket to intercept them and upload a virus into their system (this would no doubt piss off the stiff-necks in the military, but the orders would no doubt come from up top). That rag-tag group of crazy OS geniuses would be the NetBSD team.

NetBSD is very cool

It's some of the best source code I've ever looked at. As far as having consistantly good source code, it whoops Linux. I really tried to get my operating systems class teacher to use it instead of Linux, because of it's clean design. He decided to use QNX instead. WTF~?

Anyway if you've never tried NetBSD, I think you should. At least get it installed and compile a kernel. It's a good learning experience. Plus it's been ported to every fsking hardware platform ever (just about.)

Not Bad for a Dying OS :-)

The Debian Project is supporting a GNU-on-NetBSD-kernel configuration, Debian GNU/NetBSD. Benefits to users and to the Debian Project include:

• demonstrating Debian GNU kernel independence, enforcing package portability.
• supporting processor architectures that Linux is not ported to, yet, and many it will probably never be ported to.
• improving diversity: driver bugs on various peripherals are likely not to match, so one kernel might work with devices where the other fails.
• cool!

I run GNU on my machines. I'm not picky about kernels.

What?

mmap: a bad O(n) before, now O(1) with a small O(n) shadow.

What the hell is this supposed to mean? Either you are O(1) or you are O(n) - what is "small O(n) shadow" mean?

Re:What?

RTFA - specifically the graph [bulk.fefe.de] of the mmap benchmark in question. Note that for the most part the red line goes straight across. But there are a few data points that follow a O(n) graph above that (what the author called a shadow). So the interpretation is that the typical case is O(1), but occasionally it has a worst-case performance of O(n). Plus, the factor of the O(n) case is much lower than the previous version.

big doh notation

The O(N) shadow statement is a sufficient statement of O(N) behaviour for the big O pedants. I looked at the graph, and I vote we keep the wording as it was.

O notation is overrated. Sorting is always described as O(N*log N), but for any practical architecture using a radix sort with L1/L2 cache locality, replace log N with the constant factor of 3 or 4. A million cache local buckets can radix sort 10^30 elements in 3 log N time.

Using all of main memory as your bucket store, I'd guess you could sort every proton in the known universe in 8 passes. So what exactly is that log N term trying to tell us?

Trying this out

I'd like to play around more with NetBSD if it can produce results like this. Did the author just install an "off the shelf" version of 1.6.1 or did he have to apply 100 pre-alpha patches from some guy named Joe in order to get this performance?

Felix von Leitner "papers" ... cum grane salis

I can really only warn of using OpenBSD for scalable network servers.

Don't use OpenBSD for network servers.

...again, I would advise against using OpenBSD for scalable network servers.

If you are using OpenBSD, you should move away now.

http://news.netcraft.com/archives/2003/11/02/secur e_dog_hosting_most_reliable_hosting_company_site_d uring_october.html [netcraft.com]

little question

Since the author actually reads the slashdot comments, I've a little question.

I'm no real kernel hacker, so I could be totally off, but:

Are all OSes located on the same part of the (same) HD? This because linear performance probably scales linear with the cylinder the OS partitions is on.

Great... running it on my VAX

...wake me up when I can take NetBSD v1.7 and run it on my VAXstation 3100 with 8MB of memory like I can with v1.5.1
NetBSD is starting (not yet, but close) to become dangerously close to the precipice of being Bloatware(*)
First on my list to replace is GCC which has ballooned in size way way too much for the "features" that have been recently included.

TDz.
(*Bloatware is a TM of Microsoft corp)

Open Source, +1 RedHat

Or the NetBSD guys looked at all of Ingo Molnar's kernel hacks and incorporated the same logic. 2 weeks to implement all these things is an awfully short period of time, IF YOU ARE DOING THE HEAVY LIFTING OF THE DESIGN. Implementing someone else's algorithm is much easier.

Re:Nice

Actually, it's OpenBSD that doesn't support SMP. FreeBSD and NetBSD both support it rather well.

Re:Failure

*yawn*

Sometimes, its just too easy:
http://news.netcraft.com/archives/2003/11/0 2/secur e_dog_hosting_most_reliable_hosting_company_site_d uring_october.html

Choke on it and die.