Sometimes, the answers are fairly easy to find. Usually, when Caudium runs into problems, it will dump backtraces into the debug log (usually logs/debug/default.1). This file is always a good first place to look for answers.

Thread Dump

To trigger a thread status dump (sort of like a backtrace for each running thread), you can perform the following:

kill -5 <caudiumpid>

Sending this signal to Caudium should cause the current thread state to be written to the default debug log (usually logs/debug/default.1). This is often very helpful for seeing what each thread is up to, and may provide clues to the problem.

Attach GDB to the process

Problems deeper down in the C level code may require the use of a debugger. GDB allows users to debug a running process.

http://www.delorie.com/gnu/docs/gdb/gdb_23.html

Of particular interest is the current call stack, which when observed over time, can provide details about many types of C level problems.

Running a system call trace

Most UNIX-like operating systems provide a facility for inspecting system calls made by a process in real time. This can often be useful in tracking down I/O related problems.

- Solaris users can use "truss" or "dtrace" (Solaris 10+)

• Darwin and OSX users can use "ktrace" or "dtrace" (OSX 10.5+)
• BSD users can use "ktrace"
• Linux users can use "strace"

  Open file descriptors

  Often, problems occur when a process runs out of available file descriptors (fds). While this usually is the result of heavy load on an untuned system, sometimes file descriptor shortages can be traced to coding errors on the Pike or C level.

  Tools such as lsof or the proc utilities on systems such as Solaris can be useful for locating fd leaks.

Powered by PikeWiki2