iTunes is one of a few Apple-provided applications that prevent you from attaching a debugger. This can be problematic if you are, say, developing an iTunes visualizer. iTunes accomplishes this by calling ptrace(PT_DENY_ATTACH, 0, 0, 0), which prevents future attempts at tracing the process. Debuggers like gdb use ptrace to control the process they debug. Luckily, there's a way to intercept the ptrace-call before it completes, simply by starting iTunes in the debugger. Let's see what happens when we run iTunes from gdb in Terminal, without doing anything:

[#] cd /Applications/iTunes.app/Contents/MacOS/
[#] gdb ./iTunes
gdb shows some license info
(gdb) run
Starting program: /Applications/iTunes.app/Contents/MacOS/iTunes 
Reading symbols for shared libraries ... done
Program exited with code 055.

Not quite what we want. Let's see what happens when we put a breakpoint on ptrace before starting iTunes:

(gdb) br ptrace
Breakpoint 1 at 0x7fff90592d14
(gdb) run
Starting program: /Applications/iTunes.app/Contents/MacOS/iTunes 
Reading symbols for shared libraries . done

Breakpoint 1, 0x00007fff90592d14 in ptrace ()
(gdb)

We're stopped at the entry to the ptrace() function. Now, at this point we have two options: We can force the function to return, setting its return value to 0 to indicate no error, or we can modify the arguments to the ptrace call. Either way works, but this post will demonstrate the latter approach.

We know that iTunes calls ptrace with PT_DENY_ATTACH as its argument. Looking at the ptrace header file, we discover that PT_DENY_ATTACH corresponds to the value 31:

From /usr/include/sys/ptrace.h:
#define PT_DENY_ATTACH 31

Now, let's look at the contents of the registers:

(gdb) info registers
rax            0x24a0   9376
rbx            0x7fff5fc35120 140734800023840
rcx            0x0      0
rdx            0x0      0
rsi            0x24a0   9376
rdi          0x1f   31
Rest of register contents snipped

The register rdi looks promising. Let's try changing it to the value matching PT_DETACH:

(gdb) set $rdi=11
(gdb) info registers
rax            0x24a0   9376
rbx            0x7fff5fc35120 140734800023840
rcx            0x0      0
rdx            0x0      0
rsi            0x24a0   9376
rdi          0xb    11
(gdb) continue
Continuing.
Reading symbols for shared libraries . done
Symbols keep loading, and iTunes launches. And we're still attached!

So that seemed to work brilliantly. At this point, you can use the debugger as normal, pausing iTunes when you need to, getting backtraces, and catch errors in your visualizer. The only caveat is that detaching iTunes using gdb will probably hang, since we replaced PT_DENY_ATTACH with PT_DETACH earlier. This can be worked around by using the "return immediately" approach I alluded to earlier; doing this will be left as an exercise for the reader. To make gdb exit, you will probably have to kill it. As a final note, I should point out that this approach has only been tested on Lion (10.7), although I suspect it will work just as well in earlier releases of Mac OS X. Have fun!