Debugging the FreeBSD Kernel Transparently

An anonymous reader writes "To debug the FreeBSD kernel core dump efficiently, it is best to set up a remote debugging session between a development machine and the target machine, and remotely debug the kernel using serial communication. This article shows you how you can debug as many kernel images as you want; it becomes transparent to users once debugging starts, and your system's performance is not affected at all."

Surely...

[BluhDeBluh]

If this news day gets any slower it's just going to just stop.

Re:

[MattPat]

Please, enlighten me as to how something can be "1, Redundant" if it's the first post?

Re:

[pembo13]

I've wondered that myself.

Re:

[mrbluze]

I think people will see through this transparent debugging process for what it really is... A thinly veiled attempt to strip the code naked, make the kernel look sexy, and get something posted on slashdot!

Re:

[Brandybuck]

I've got no idea what time it is. I'm lucky to be posting at all, what with all the traffic signals not working and ATM machines crashed and people rioting in the streets...

Re:

[lord_sarpedon]

You might say that we've reached the break(ing)point...

I was just about to post this story....

[concernedadmin]

Hehe, good to see others saw this as an influential story of the day.
However, I have a question that perhaps others can answer.
What does the diagram actually work out?
Listing 1. 25-Pin NULL modem cabling

2 3 Transmit Data
3 2 Receive Data
4 5 Request to Send
5 4 Clear to Send
6 20 Data Set Ready and Carrier Detect
7 7 Signal Ground
20 6 Data Terminal Ready

What does that mean? I thought serial -> serial is merely a connection. Do I have to solder something?

Regards

Re:I was just about to post this story....

[ampmouse]

What does the diagram actually work out?

This is a Null modem adapter [wikipedia.org]. It allows you to connect to terminals, such as computers, together without a modem. It is the serial equivalent of a ethernet crossover cable.

Re:

[Spaceball_3000]

Normally from PC to modem the modem doesn't have wires crossed just a straight through cable, as the device automatically will swap certain pins like pins 2 and 3 in the device itself. But if your connecting from one computer to another using an straight through cable, the same pin like receive will connect to the same pin on the other PC (which is the receive), thus this will not work; as for normal operation you need to connect the Transmit to Receive from each PCs. Hope this helps.

Re:

[mattjb0010]

What does that mean?

You were going to submit the article without RTFA? From the article, comes this link [nullmodem.com], which explains things rather well (IMHO).

Do I have to solder something?

No, you should be able to buy a null modem cable.

Re:

[Alwin Henseler]

Do I have to solder something?

You must be in the wrong place here. A true Slashdot user would ask: "Please, can I solder something to make this work?"

Re: I was just about to post this story....

[Dolda2000]

I thought serial -> serial is merely a connection. Do I have to solder something?

You seem to think in strange ways. Indeed, serial -> serial is a connection, but surely, you realize that you have to make that connection with something, or did you think it was wireless?

Either way, to actually answer the question, you don't actually have to solder anything -- you should be able to buy a null modem cable anywhere. However, considering this is Slashdot, don't you think it would be easier to solder it y

Computers these days

[DeadCatX2]

We just bought a few brand new PCs recently, and they didn't come with RS-232 ports at all.

Laptops almost never have serial or parallel connectors.

But, you are correct, 25-pin serial is rather rare...

Re:

[King tweak]

Humm. Sun boxens. And if you lucky you don't get an OK India friends OK

Re:

[King tweak]

No you already have to many whirs. twee will do. 1,2,7 xon baby...

Re:

[King tweak]

oopy 2,3 and the mighty 7 oh my

Re:

[DeadCatX2]

tx
rx
gnd
ftw

Re:

[dioxide]

it's the pinout of the null modem cable. you can't use a straight through serial cable, the tx/rx lines need to be reversed so that computer a's transmit ends up at computer b's receive.

you can either cut a straight through cable and resolder, buy the db9 or db25 ends and make your own cable (can be soldered, or solderless), or just go buy a null modem cable.

gdb remote is useful

[EmbeddedJanitor]

This is a very cool feature in gdb that is very commonly used to debug various embedded systems etc. Sometimes gdb remote uses a serial port or tcp from the host to a stub running in the target (as is the case with Redboot).

Also, gdb remote is often used to talk to a server that mimics a gdb remote stub and then turns the commands into some other connection into the target (eg. a JTAG debugger).

All up, this makes debugging embedded systems a lot simpler than it would otherwise be.

Stop pissing about

[DrSkwid]

Get yourselves a system, with a proper debugger [bell-labs.com]

debugging kernels

[digitalderbs]

This article shows you how you can debug as many kernel images as you want

including zero.

Re:

[King tweak]

Go away to another land and find your fortune cookie u kinky kernel.

Unfortunately

[edderly]

Jayesh V. Rane works as a Systems Software Engineer for IBM and has five years of experience in product development, mostly working on network programming and storage management software. He has worked extensively on FreeBSD and AIX operating systems. Jayesh is currently working with the IBM India Systems and Technology Lab in Pune, India. You can contact him at [e-mail]. Unfortunately Jayesh doesn't understand that compiling your kernel with -g will in fact affect the systems performance and is currently looking for a job where he can pretend to be a kernel engineer. Contact Jayesh at [e-mail]

Which system?

[WereRaven]

The "system performance" he is referring to is on your development machine, i.e. where you are running kgdb. Of course compiling your kernel with -g will affect performance on the target machine! By not running a development kernel on your development machine you are not affecting your system's performance. I don't know about you but it really affects my system's performance when the kernel panics.

Wyse terminals work well for this.

[Anonymous Coward]

About 5 years ago I used an old Wyse terminal [wyse.com] I got at a junkyard for exactly this job using a homemade cable, it worked beautifully. Once complete I then used the terminal to control and update the server from the living room couch.

I also used the same terminal and SoftIce to *ahem* "debug" several windows applications.

Wyse terminals (at least the older ones) are excellent gear, very sturdy, nice keyboards, though the monochrome monitor was a bit burnt in on the one I used. An industrial strength terminal for a rock solid OS. A match made in heaven. FreeBSD is an excellent OS.

Re:

[archen]

Is it possible to use wyse terminal keyboards with a regular PC? I've got a lot of them laying around and really like them but nothing to hook them up to really (only two serial terminals).

Transparent to users?

[istartedi]

Something tells me that if the kernel has dumped core and dropped down into a debugger, the users are going to notice something. Just a hunch.

Re:

[flithm]

A core dump does not necessarily signify a crash, or dropping "down" into a debugger or anything. In fact you can actually force pretty much any app to generate a core dump. Usually all you need to do is send a SIG_QUIT signal to the app. However, this depends on your security settings and so forth, since a core dump of a running program might reveal sensitive information.

At any rate, FreeBSD provides a mechanism for allowing core dumps of the kernel to happen whenever you (ie someone who's authorized) w

It was a dark day for Linux

[Chatz]

It was a dark day for Linux when Linus said he didn't believe in kernel debuggers.

Last time I had a good look at what was on offer while developing a filesystem for multiple operating systems, I compared the tools for Linux to Windows, MacOSX, IRIX and Solaris.

Windows was clearly the best for remote kernel debugging with windbg and I don't see that changing anytime soon. A fully fledged debugger, automatic download of kernel symbols from Microsoft and your own repository, reliable capture of dumps etc.

Re:

[ThePhilips]

But when it comes to kernel debugging on Linux, the picture is still bleak.

I recently was customizing for myself frame buffer driver. I have overhauled about 20k of kernel code - and after several compilation errors it worked perfectly fine. As in "just works."

On Linux side I can tell only one thing: if you need a debugger for kernel, then you are better off doing your stuff in user space. 2.6 is heck a lot cleaner and polished than any linux kernel I have worked before - 2.0/2.1/2.2/2.4. Most stuff

Re:

[nuzak]

> On Linux side I can tell only one thing: if you need a debugger for kernel, then you are better off doing your stuff in user space.

Yeah, it's too bad Linus doesn't believe in microkernels either. This is the same guy who thought the OOM killer was a great idea.

Re:

[Eivind Eklund]

DDB is in FreeBSD, too, since always. In fact, DDB was ported from Mach to 386BSD, which predates the FreeBSD/NetBSD split / diverge, which again predates the NetBSD/OpenBSD split.

Eivind.

quantum

[null-sRc]

maybe the mere act of viewing the event will change it... is debugging ultimately futile?

Re:

[nuzak]

This is sometimes known as a Heisenbug (ah yes the Jargon File did have some nice entries back in the old days). Multithreaded or just plain time-sensitive code (like that found in device drivers) is notorious for this sort of thing.