Solaris DTrace To Be Ported to FreeBSD 151
daria42 writes "It looks like Sun's famous Dynamic Tracing tool - one of the best features in Solaris 10 - is getting ported to FreeBSD. Sun open-sourced the code back in January and it has been picked up by FreeBSD developer Devon O'Dell. The tool provides insanely great advanced performance analysis and debugging features for server software. Good to see some result come out of the Sun open-sourcing process." From the article: "O'Dell told ZDNet Australia the aim of the project -- which commenced a month ago -- was that all scripts and applications that utilised DTrace under its native Solaris environment should be able to run in FreeBSD with no changes. While FreeBSD's existing ktrace function was similar to DTrace, it was limited in scope, according to O'Dell. 'FreeBSD implements a somewhat similar facility for dynamically instrumenting syscalls for any given application,' he said."
License? (Score:3, Interesting)
Re:License? (Score:5, Insightful)
To create a BSD-licensed version, someone would have to *clone* it, which is different from porting.
Re:License? (Score:2)
Re:License? (Score:5, Insightful)
Also, I don't think FreeBSD is committed to removing all non-BSD code.
Re:License? (Score:5, Insightful)
Modifications I need to make to already BSD licensed code will remain BSD licensed. New pieces of code I write to get it working that are not taken from Sun will be BSD licensed. Everything else will be porting work and will be CDDL.
Re:License? (Score:1)
FreeBSD is mostly about making things work fast on x86.
Good for Ruby! (Score:5, Insightful)
I'm not sure how this benefits Sun, but something as awesome as this, I'm willing to assume it's altruism, and I appreciate it.
Re:Good for Ruby! (Score:5, Informative)
S
Re:Good for Ruby! (Score:5, Insightful)
Thats easy. You used to be a Mac only person (making some guesses here...) but now you are a Solaris user.
How many other people are trying solaris for the first time because of this feature?
Suck in the developers and they may turn into server sales or even just positive PR.
Sounds like more than altruisim to me.
Re:Good for Ruby! (Score:3, Interesting)
No, I think he's talking about Sun making dtrace open source, which might turn him into a FreeBSD user, or perhaps allow him to use OS X exclusively (not likely with the kernel changes needed, but maybe Apple will see the light.)
So, sacrificing your value-added product to the public domain seems to be entirely altruistic AFAICT. With something like NFS, they stood to gain directly by allowing others to use it, but
Re:Good for Ruby! (Score:2)
I'd hope so too, but doesn't it depend on the kernel? OS X doesn't have a FreeBSD kernel, it's a MACH-based affair.
It clearly can be ported between kernels because this is precisely what the article is describing. However, that doesn't translate to the work actually taking place to run it against MACH.
Cheers,
Ian
Re:Good for Ruby! (Score:2, Interesting)
I should point out that the PowerPC port is not tier 1 yet so its not perfect. I know there have been a few problems with X11 and keyboards on laptops that use ADB protocol are broken (all ibooks for example) I think some powerbook models use USB so y
When will it be available in Linux ? (Score:2, Interesting)
Re:When will it be available in Linux ? (Score:5, Interesting)
Re:When will it be available in Linux ? (Score:2)
Re:When will it be available in Linux ? (Score:2)
My first impression is that FBSD is like another distribution of Linux. I don't mean pigeonhole FBSD. And I realize it may come as an insult, but after using so many different f
Re:When will it be available in Linux ? (Score:5, Informative)
Is there any reason why I shouldn't look at FBSD as if it were a flavor of Linux? Yeah, it has a different kernel. I guess FBSD might be a little faster? That is what the benchmarks say, but the difference isn't staggering. I certainly don't notice. Is it more stable? I haven't had many problems with Linux that couldn't be blamed on cheap PC hardware.
Yes, a very important reason - FreeBSD is not Linux, just as surely as SCO UnixWare is not Solaris. Their codebase is certainly not the same, and in fact FreeBSD's code lineage dates back many years before Linux.
FreeBSD and Linux, being F/OSS systems, share a very large base of F/OSS software, so looking at kde on X on FreeBSD really won't appear that different from looking at kde on X on linux. I could just as well ask why anyone would want to use Linux when it just looks like a derivative of FreeBSD, which predates it. but that would not be a fair assessment because Linux is a seperate work built by another party. Yes, it is a unix-like system. Yes, it strives to adhere to POSIX standards. Yes, it runs all the same software. But no, it is a different system.
I have been using FreeBSD and NetBSD for many years, and where I work all of our stuff is on SuSE. In my opinion, SuSE is impossible to upgrade, its package system is inadequate, and shorewall is a lousy attempt at ip filtering. If I had my way I'd probably replace everything with FreeBSD. But did you notice somehting about the attitude of my opinions? Wasn't your first thought "Well gee, you use FreeBSD all the time and you've probably barely given SuSE Linux a shot?" If it was, you would be right. Because I learned to accomplish tasks in FreeBSD, I favor it - the same way I favor speaking in english over german because english is my native language. I'm sure if you sit down and think about it, when you picked up FreeBSD you tried to do things in the Debian idiom, expecting Debian results. But you didn't get them. So you're underwhelmed. It's natural, but please don't try to attribute it to FreeBSD being an inadequate copy of your favorite system, because that simply is a lie.
On the packages/ports system, I think you've really overdramaticized your plight with the BSD way-of-doing-things. First, you can cvsup the ports tree and compile from source. But you can also use pkg_add to add binary packages. If you don't want to fetch the package tarball yourself, you can use pkg_add's remote fetching feature. Simply pkg_add -r and you're on your way. It will take care of dependencies and the package database will record the package's information. You can also install portupgrade and use it to magically update a port and its dependencies when it is time to upgrade. It's not a difficult or time consuming system to use. I'm unfamiliar with Debian's package system, so I can't make any comments on it, but FreeBSD's package system has always been very useful fo me, and it gets more powerful all the time.
Overall, though, Linux and BSD really do feed from eachother's growth. What's good for one is good for the other. I may use FreeBSD, but that doesn't mean Linux is useless; and the opposite is true as well. All this bickering is really pointless because both projects will continue on in their own directions; some people will favor the one while some people will favor the other. It's simply a matter of preference
your .sig (Score:2)
Re:your .sig (Score:1)
I took this to another level and not only 'unpacked' th
Re:When will it be available in Linux ? (Score:5, Insightful)
Linux has its uses and is great for many tasks, but only Gentoo comes close to the ports system and how well it manages software installation.
Either way, I'm hoping that yes, it will be ported to Linux as well, if it hasn't been already.
Re:When will it be available in Linux ? (Score:1)
I've used many different package systems - solaris's, linux (debian's apt, suse's yast, redhat's), ipkg on zaurus... and maybe I'm missing something, but I didn't find FreeBSD's ports better than debian's system, or even much better than yast's... and it wasn't entirely unbreakable either.
I'm sorry, FreeBSD guys, but it's still too much of a minority interest, with too many real-world solutions missing.
Re:When will it be available in Linux ? (Score:3, Informative)
Re:When will it be available in Linux ? (Score:2)
I've heard such paeans to the FreeBSD ports system for years, but when I actually spent time working with it, I found it a big pain.
First, ports can only really handle installation of software packages under one's /usr/local tree. It doesn't do anything for upgrading other pieces of the file system.
Second, the whole thing isn't very user friendly. I spent a week or so developing a new port, that involved dependencies on a dozen or so other ports. Every time I tried doing a 'make package', it went thr
Re:When will it be available in Linux ? (Score:1)
But yes, there's MUCH room for improvement in the "ports" system, but the concept is sound, and the idea of a system that builds the dependencies for you is great, IMHO.
Anyways, back to the topic!
Re:When will it be available in Linux ? (Score:4, Informative)
make package-recursive
Re:When will it be available in Linux ? (Score:2)
Thank you! I read the entire Porter's guide extensively and never came across that.
Re:When will it be available in Linux ? (Score:2)
You might also want to try man ports then :).
Re:When will it be available in Linux ? (Score:3, Informative)
Yes, there is: SystemTap by Red Hat, IBM and Intel.
Perhaps you should read
http://groups.yahoo.com/group/solarisx86/message/2 7818 [yahoo.com]
and
http://milek.blogspot.com/2005/08/linux-and-solari s.html [blogspot.com]
Two discussions on some differences between SystemTap and Dtrace. (And yes, both links are in favor of Dtrace, and for good reason it appears.)
Re:When will it be available in Linux ? (Score:2, Informative)
Re:When will it be available in Linux ? (Score:3, Insightful)
Re:When will it be available in Linux ? (Score:3, Interesting)
That said, I have downloaded the FreesBIE LiveCD; I just haven't burned it yet.
Re:When will it be available in Linux ? (Score:2)
Install the sysutils/coreutils [freshports.org] port. You'll get all the GNU utilities with a 'g' prefix, i.e. gls, gcp, etc. You can alias the ones you want to use.
Re:When will it be available in Linux ? (Score:2)
Re:When will it be available in Linux ? (Score:1, Flamebait)
From all the posts on
Re:When will it be available in Linux ? (Score:4, Interesting)
You can see a fairly detailed analisis in the 2005 Proceedings, Volume 2, page 57 [linuxsymposium.org] of the linux symposium
Also some doc from IBM: http://www-128.ibm.com/developerworks/linux/libra
also there's a "linux trace toolkit". A post about LTT vs dtrace [theaimsgroup.com]...whatever, too much flamewar for my taste.
Re:When will it be available in Linux ? (Score:3, Informative)
Oh well, it's already there and it seems to have been there for ages: http://kernel.org/git/?p=linux/kernel/git/torvald s
Re:When will it be available in Linux ? (Score:5, Informative)
dtrace vs. krpobes [blogspot.com]
systemtap is in its infancy and being designed without safety as a priority, dtrace was created to be 100% safe to run anytime, even in production. systemtap is being made for the kernel hacker to debug the kernel. With possibly some userland probes and safety as an after thought. Sure they talk about safety as a goal. But as documented dtrace_usenix.pdf [sun.com]
dtrace was created from the start to be safe and secure. They even sacrafice some functionality to keep production servers safe. Systemtap is like building a bank they build the building, bring in the money, and desks, and machines, and promise that top of the line doors, windows and safe will top of the line and installed any day now.
Re:When will it be available in Linux ? (Score:1, Informative)
RUNTIME issues:
how do you sudgest you solve the halting problem?
do you have a solution for errant pointers?
and many more.
compile time issues:
You should read this latest tidbit from the systemtap mailing list, basicly it says that systemtap will not understand include files, so if you want to track data in a userland or kernel based struct.
Re:When will it be available in Linux ? (Score:2)
Re:When will it be available in Linux ? (Score:1, Informative)
Re:When will it be available in Linux ? (Score:1)
Tons of links in the article (Score:5, Interesting)
Do you need to instrument the calls you expect to profile? If so, how can you avoid taking that performance hit when deciding whether to perform the profiling or not, even when the profiler is off? It's still got to check the profiler level each time, doesn't it?
Re:Tons of links in the article (Score:2)
None. DTrace patches code when you use it, and then un-patches itself when you're done.
Re:Tons of links in the article (Score:1)
Does it swap out normal binaries for instrumented binaries on the fly?
How is it able to manage a zero penalty hit?
Re:Tons of links in the article (Score:2, Informative)
The overhead when in use is low enough that you can turn on a blanket Dtrace of all functions in the kernel without killing the OS. If you target your trace points sensibly the overhead is low enough that its not an issue. Its designed to be safe to use, so the Dtrace scripts that do in-kernel filtering can't d
Re:Tons of links in the article (Score:1)
Linux had this for ages (Score:2, Funny)
The official reason is that it wasn't release was because Linus didn't want the BSD folks using it, but the real reason is the Department of Homeland security didn't want the BSD folk to find the last bug in their release.
Thats what I just head right now. (Thanks, voices)
Re:Linux had this for ages (Score:5, Informative)
Can you monitor how much network bandwidth each process uses? -- Sure you can see listening ports and IP traffic, and ntop is fantastic at showing what network bandwidth is used for (i.e. spotting p2p and IM traffic, eg). However if you have a trojen and you see suspecious network activity, there is a certain amount of guess work to try to find the process. Solaris will show exactly what process is making what connection where and the bandwidth it is using.
Can you monitor how much IO utilization each process has? -- No, only IO wait and CPU consumption which is normally enough, but say you have a script thats just reading all content on the disk and redirects it to
Sure you're usually right making an educated guess but system profiling is far ahead on Solaris.
Re:Linux had this for ages (Score:2)
Although perhaps the voices were right...who's to say...
Wikipedia:DTrace (Score:5, Informative)
For we that don't have a clue what DTrace is, here's what the [wikipedia.org] has to say: DTrace allows to do performance tuning with applications and troubleshoot production systems--all with little or no performance impact. DTrace provides improved visibility into kernel and application activity, giving the user operational insights with which they can make performance gains..
The no performance penalty sounds really cool to me.
--
Superb hosting [dreamhost.com] 4800MB Storage, 120GB bandwidth, $7,95.
Picaday! [picaday.host.sk] Soon to be open "Picture of the day web".
Re:Wikipedia:DTrace (Score:2)
It's not on solaris 9, just checked (checked solaris 8 for fun too), so can't make any real comparisons.. anything that makes solaris debugging less than a total 'mare sounds like a good idea though.
(shouldn't be too hard on solaris though... I have to do an HPUX port too - that's an OS I wouldn't wish on my worst enemy...)
Re:Wikipedia:DTrace (Score:4, Informative)
--dave
Re:Wikipedia:DTrace (Score:1)
Is this mostly a developer tool, or is it useful to SAs, too?
Are you seeing most 3d party software vendors supporting Solaris 10? Zones?
Re:Wikipedia:DTrace (Score:2)
Re:Wikipedia:DTrace (Score:1)
Features like the new SMF (aka launchd on Mac OS X) are a little turn-off IMHO, because it introduces XML and a full-blown database instance to something simple as the start-
Re:Wikipedia:DTrace (Score:1)
The TCP/IP stack is nice, as is SMF.
Although you can go to Sun's site to find out. I am no cheerleader for them.
Re:Wikipedia:DTrace (Score:4, Informative)
dtrace is much different, you have areas of your kernel that have probes, places that accumulate data. dtrace is a language where you can read these probe areas (including the syscall interface) and print them out to user level and figure out whats going on (wrong) in your kernel.
For the people who say Sun isn't real about open source, they should realize this is a differentiating technology, years ahead of what anything in Linux/bsd or commercial linuxes have. If it's going into the BSD kernel, it's probably also BSD licensed, meaning all UNIXes can take this.
Re:Wikipedia:DTrace (Score:2)
And that's why Sun and Solaris have been such smashing successes recently?
Face it, most people would not know how to use DTrace if their life depended on it. That leaves the few who do. Many of those don't have a choice in platforms, so it's academic. And many of the performance problems gurus encounter and can fix are blatantly obvious anyway. And even if DTrace may be a little better, it's
Re:Wikipedia:DTrace (Score:1)
The company that I work for (huge/global etc). Whilst migrating some things to RHEL is leaving a lot of stuff on Solaris (currently 10000+ systems).
Developers are attending special courses on Dtrace and they love it I think the benefits will be remarkable.
If you see what this can do in a real-world scenario you will be suprised.
Re:Wikipedia:DTrace (Score:2)
And where are those benefits supposed to come from? Well-tuned applications already run close to what the hardware is capable of. Doing that isn't rocket science.
Whilst migrating some things to RHEL is leaving a lot of stuff on Solaris (currently 10000+ systems).
I pity you.
Re:Wikipedia:DTrace (Score:2)
FreeBSD really needs this (Score:5, Interesting)
A tool like this could really aid in finding all the bottlenecks. Benchmarks have become an embarrassment for FreeBSD as of late, and it is really sad to see that FreeBSD has fallen so far behind. Hopefully this could start to turn things around.
Re:FreeBSD really needs this (Score:5, Interesting)
Re:FreeBSD really needs this (Score:4, Insightful)
I do agree with the parent poster as well since the threading and the code quality has made many old FBSD timers leave and work on Dragonfly. I no longer run FBSD as a result.
But I wold mod the parent down for the that reason. However I would mod him up if it was a general FBSD post about i/o or BSD vs Linux story.
Re:FreeBSD really needs this (Score:2)
But!!! (Score:1)
start sarcasm
But it is not BSD! It can't be better than anything BSD has created.
We all know that Solaris is just a crappy system that has no use in the enterprise.
end sarcasm
bsd-style proc accounting? (Score:1)
this is great (Score:2, Interesting)
1) Considering the fact that we are currently going through the Beta's for FreeBSD 6, I am curious how, if at all, a fully implemented DTrace would help the devs with tracking down and solving the current beta problems. From my current understanding, it seem
Re:this is great (Score:5, Informative)
Quite a lot, actually. I've talked with Eric Schrock about his thesis work, which was implementing some lock analysis tools using DTrace. This allowed him to detect (very precisely) things like LORs, deadlocks, and the like. His thesis is available at http://www.cs.brown.edu/publications/theses/ugrad
On 2):
When I've seen demonstrations on this stuff, it has been Bryan Cantrill doing fun stuff with libumem, mdb, and DTrace. I suspect that, at the minimum, we'd need libumem to find and fix this stuff with the accuracy that it can be done in Solaris.
Hope this is useful information.
--Devon
Re:this is great (Score:1)
Thanks for the link.
"best" feature of Solaris 10 (Score:1, Flamebait)
Re:"best" feature of Solaris 10 (Score:2, Flamebait)
Then there was a slight HR issue with many of the engineers...
Re:"best" feature of Solaris 10 (Score:1)
SMF, new IP stack, zones (whilst not true virtualisation is a good start) blah blah blah
Whilst I am no cheerleader for Sun. I think some hardware lines are poor - I do SA callout work on Solaris/Linux and it has kept me up a few times. Solaris is a quality rock-solid OS.
Re:"best" feature of Solaris 10 (Score:2)
I seem to remember being quite supportive of Solaris around here, and the Sun engineers.
Now, the PHBs. That's a different story.
Just for the record: Solaris : good, Java : good, RedHat : bad.
Opteron : good. UltraSPARC : mediocre and falling behind.
Pentium : bad.
Linux : OK. Solaris : better.
Windows : Bad.
Unrestrained capitalism : bad.
I hope that clears a few things up. If you want to know my shoe size and inside leg measurement, just ask.
Re:"best" feature of Solaris 10 (Score:2)
Re:"best" feature of Solaris 10 (Score:1)
Re:"best" feature of Solaris 10 (Score:2)
Clear up a few things (Score:5, Informative)
FAQ #1 seems to be about the license. Obviously, the CDDL is `viral' in the sense that changes in the code need to be provided under the same terms of the CDDL. In my understanding, this applies only to files in which modifications take place. Extension of something CDDL by adding extra files seems to not require those files to be released under the CDDL. That said, this is a porting effort, and most of the changes I will make will be inside CDDL-licensed files. Thus, anything imported will be under the CDDL. This does not require anything external files to be under the CDDL and thus it can be shipped with FreeBSD without `infecting' other files.
FAQ #2 seems to be whether Sun is happy about this or not. If you have read the article, you would have seen that I've been encouraged to work on this by Sun kernel engineers. Whether Sun as a whole is happy about this is not known to me, but everybody involved in getting it this far has been, so I'm not terribly worried about the rest.
FAQ #3 is about performance incurrences. Certain aspects of DTrace incur performance penalties, but only when DTrace is running. DTrace by itself is a library and a userland tool. All instrumentation is done dynamically and when DTrace is not instrumenting something, no performance hits happen whatsoever. When it is running, we have several advantages to other tools because (unlike e.g. truss) we are instrumenting single processes. Processes which are not being instrumented will not take any performance hits other than the fact that they have a bit less CPU usage since DTrace is instrumenting something.
How do you not take a performance penalty when the profiler is off? You must be root to run DTrace. When you instrument functions inside an application, this is done on-the-fly by rewriting the code that is being executed. When it is not being executed, nothing is being rewritten and it's not even looking to rewrite something. It's just some code resident in memory (in fact, modules are even loaded as needed). It sounds like it might be prone to security flaws, but keep in mind that this has been working in production for a while now.
When will this be in Linux? I don't know. I won't be working on it. I challenge _you_ to do this
Is this vaporware? No. I'm continuing development from about a week off (since I lost my development machine) this evening.
Feel free to ask more questions, I'll try to address them as I see them. But please refrain from bad-mouthing Sun or myself out of spite, jealousy, or whatever. I know it's fun to troll (if you're a troll), but the rest of us really don't appreciate it.
--Devon
Re:Clear up a few things (Score:1, Informative)
By the way, you don't need to be root to run DTrace. The Solaris privelege model allows assignment of dtrace priveleges to users. So you can selectively allow users to trace their own processes are more.
Are you planning to also support kernel level tracing? Dtrace is also really useful to Solaris kernel developers (my job) and allows tracing of kernel functions, system calls, etc.
Re:Clear up a few things (Score:3, Informative)
Yes, I am planning on implementing every provider I can.
Re:Clear up a few things (Score:2)
Re:Clear up a few things (Score:1, Interesting)
Good challange! But isnt the big problem here the license issue? Someone can do something like dtrace but a port is hard...
Re:Clear up a few things (Score:5, Informative)
No, the real difficulty here is that Linux is by itself only a kernel. Getting this integrated into a full operating system is hard because you have to work with varying userland utilities and make sure that it's integrated properly. I'd expect that somebody would probably do it in Debian, Gentoo, Redhat, or another distribution. In FreeBSD, this is easier, because you are working with an entire system that you know exists and is going to be available for use. You know exactly what will and will not be there.
Integrating it into Linux might thus be a bigger challenge than doing so in FreeBSD (or any other BSD, for that matter). But if somebody were to choose a distribution and JUST GET IT DONE (this is the key), I'm sure others would pick it up.
Re:Clear up a few things (Score:3, Insightful)
Certainly, licensing should be a primary issue. Before one does anything with a program, there needs to be an answer to the questions, "Who does it belong to? What am I allowed to do with it?" At the very least, those questions ought to be answered "Mine," and "Whatever I want." Ideally, it should be answered, "Ours, and we want." If it doesn't belong to you and you can't do certain things with it, you can only get by ignoring this for so long until it becomes a maj
Re:Clear up a few things (Score:1, Insightful)
no no no... Licensing *is* secondary. Considering it primary is a sign of the dementia that is affecting the world today. A dementia that the GPL was created exactly to *fight*!
Follow the spirit of GPL, not the license itself.
Re:Clear up a few things (Score:2)
Re:Clear up a few things (Score:2)
Well, that's certainly everything I wanted to hear. I'm seeing your point a bit more clearly now, and while it's always debateable about how restrictive a license is or isn't as long as the licensing is such that you can legally use the software. It's a "right tool, right job" relationship.
Re:Clear up a few things (Score:2)
The ported parts are licensed under the cddl. I imagine some parts which reach deep into the kernel will require reimplementation as opposed to porting and it would make sense for those parts to be under a bsd license. The cddl is designated by the Free Software Foundation [fsf.org] as a Free Software GPL-incompatible license [fsf.org]. It is a derivative of the mozilla public license. Due to the requirements of the GPL it is easy to be GPL-incompatible. The LGPL is more sens
Waiting... (Score:2)
Re:Insanely great (Score:2, Funny)
Google shows 229,000 hits for "insanely great" (as a phrase).
Welcome to, umm, Geek English!
Re:Insanely great (Score:5, Informative)
"insanely great" is well known. In fact, it's in the Jargon File [catb.org]
Re:Insanely great (Score:1)
Re:MOD PARENT UP! (Score:1)
And then proceeded to cancel that moderation by posting in the story.
Let me guess, you must be new here?
Re:MOD PARENT UP! (Score:1)
which is why I posted...
sigh...
weird tho, the score is the same as before I posted.
so someone else modded it up +1 to cancel the cancel on my +1 ? erg, you're making my brain hurt!
bad slashdot user, bad!
Re:MOD PARENT UP! (Score:1, Interesting)
And what does it mean that a former FreeBSD core member admits that FreeBSD is dying? Well, in my opinion FreeBSD's leadership has been a little out of touch lately. That doesn't mean that OpenBSD, NetBSD, and now DragonFly (for the disaffected FreeBSD people) can't continue kicking ass.
Re:Looking for a DragonFlyBSD Port (Score:2)
Check facts, this is true.