While I'm glad for FreeBSD they're showing good numbers again, their testing of PostgreSQL in this study is rather odd. The results are using the read-only tests from sysbench. You can see from its sourceforge page [sourceforge.net] that sysbench is a MySQL benchmarking tool that has some rudimentary PostgreSQL support bolted on top. That particular code is so bad that the last time I checked, turning on the write OLTP tests deadlocked the PostgreSQL server, as it wasn't putting statements into transactions correctly (which of course the ancient MySQL versions this code targeted doesn't care about). As the sysbench tool hasn't been actively maintained in ages I doubt that has improved.
The claimed "15% faster than linux" is pretty clear in the MySQL tests; the PostgreSQL ones have a weird dip in them but are in general much closer. I'd be comfortable if the result of this study was "FreeBSD 7 has been optimized to be 15% faster running MySQL than Linux", because that matches what they did (note the specific libpthread patch for example). But the fact that they used such an awful PostgreSQL benchmarking methodology leaves me hesitant to draw a broader conclusion than that based on their tests.
While I'm glad for FreeBSD they're showing good numbers again, their testing of PostgreSQL in this study is rather odd.
Agreed, on both points. What I want to know though, is where this performance improvement, and 7.0 in general, leaves Dragonfly BSD... do they still feel that Dragonfly's choice to split off at 4x and start making radical changes is paying off? Is dragonfly making progress towards better performance, in general, or on particular workloads?
The dev of Dragonfly BSD has switched its aim from being 'a better way to do SMP' to 'SSI clustering' so I doubt that Dragonfly BSD is going to compete with FreeBSD in SMP scalability anytime soon if ever.
Dragonfly has still made no progress towards SMP performance with even 2 CPUs (they are still "competitive" with FreeBSD 4.x performance, and sometimes much slower), but this somehow doesn't deter their advocates from believing that Matt Dillon's -- still largely unimplemented -- ideas are still vastly superior.
As we know, unwritten code has no bugs and runs at twice the speed of thought, so maybe so:)
You are correct that the pgsql driver in sysbench is broken for read-write loads. It needs a small patch to get it to detect deadlocks properly. However your objection is irrelevant to this benchmark, which is for read-only loads. Furthermore, the goal is not to evaluate mysql vs postgresql, it is to evaluate FreeBSD vs Linux.
If sysbench has some performance problem running PostgreSQL, and the tests I've done never gave me any confidence in it there, those issues could end up bottlenecking the benchmark results. Now: what if that's the case, but that bottleneck is less of a problem with FreeBSD than Linux? All you've proven is that FreeBSD is faster at running sysbench, not PostgreSQL. Unless you're using a good benchmark for the database you're testing, you can't draw any strong conclusion about how that database performs, an
The database is putting load on the kernel, and the performance of the kernel is what is being tested here. If you like, sysbench+pgsql is just a fancy concurrent syscall load generator and the underlying workload is irrelevant as long as the pattern of syscalls models the kind of thing that a reasonable application could be doing. The sysbench output numbers are just a function of the number and type of syscalls performed, i.e. they measure kernel throughput. Analysis of the syscalls being performed indee
I agree that "15% faster than Linux" isn't defendable, but I think the point of the benchmarks was to get a taste of the SMP scalability. pgsql and mysql were just the vehicles to demonstrate it.
Now once the scheduler becomes topology-aware... look out!
The closest to perfection a person ever comes is when he fills out a job
application form.
-- Stanley J. Randall
Bogus PostgreSQL tests (Score:3, Interesting)
The claimed "15% faster than linux" is pretty clear in the MySQL tests; the PostgreSQL ones have a weird dip in them but are in general much closer. I'd be comfortable if the result of this study was "FreeBSD 7 has been optimized to be 15% faster running MySQL than Linux", because that matches what they did (note the specific libpthread patch for example). But the fact that they used such an awful PostgreSQL benchmarking methodology leaves me hesitant to draw a broader conclusion than that based on their tests.
Where does this leave dragonfly? (Score:3, Interesting)
Agreed, on both points. What I want to know though, is where this performance improvement, and 7.0 in general, leaves Dragonfly BSD... do they still feel that Dragonfly's choice to split off at 4x and start making radical changes is paying off? Is dragonfly making progress towards better performance, in general, or on particular workloads?
I saw what Matt Dillon did back in Amiga days. I
Re: (Score:3, Informative)
Re: (Score:2)
As we know, unwritten code has no bugs and runs at twice the speed of thought, so maybe so
Re: (Score:1)
Basically, dragonfly is both slower with a single thread and fails to scale at all
Re: (Score:2)
Re: (Score:2)
Unless you're using a good benchmark for the database you're testing, you can't draw any strong conclusion about how that database performs, an
Re: (Score:2)
Analysis of the syscalls being performed indee
Re: (Score:1)
I agree that "15% faster than Linux" isn't defendable, but I think the point of the benchmarks was to get a taste of the SMP scalability. pgsql and mysql were just the vehicles to demonstrate it.
Now once the scheduler becomes topology-aware... look out!