Jamdog's blog

Using GDB to Debug - A Quick Guide

A common question asked by MUD owners is "My MUD keeps crashing, how do I use GDB to debug my MUD and find the problem". I hope to be able to answer that question with my quick guide below.

This is not a comprehensive tutorial that will teach you everything there is to know about GDB, but it will give you enough basic knowledge to hopefully root out the cause of any MUD crash.

What is GDB
GDB is a programmers debug tool. It allows you to run the program within a controlled environment which will 'catch' a crash, and let you examine the status of the program at the time of the crash. It does much more than just this (like letting you pause your running program, and step through it one line at a time, or even examine your crashed program days after a crash, even when it wasn't running within GDB).

Running your MUD in Debug Mode
GDB should be run on your MUD from it's root folder (not in the bin subfolder as you might expect).

Next, type gdb bin/circle to start GDB, and tell it to use your MUD executable. You will now be given the gdb prompt instead of your normal unix-shell one.

Next, type run <port> to run your mud (obviously entering a port number to use, not typing "<port>"). You should see all your normal syslog info appear on your screen, and your MUD will eventually give a message saying "No connections, going to sleep.".

You should then be able to connect to your MUD on the specified port in the normal way. Your next step is to crash the MUD. Obviously, if you have a crash bug, you might have some idea of what's causing it. When you manage this, the MUD will appear to pause, and you will get a message in GDB, and your gdb prompt will return. You can now start debugging.

Normally, the first command you type will be bt (for 'backtrace') which will show you the function tree, with the lowest level at the top. This shows the files each function call is in, and the line number it's called from. Below is an example of how this could look:


Program received signal SIGSEGV, Segmentation fault.
load_char (name=0x22c540 "Jamdog", ch=0x10ebcb48) at players.c:248
248 GET_SKILL(ch, i) = 0;
(gdb) bt
#0 load_char (name=0x22c540 "Jamdog", ch=0x10ebcb48) at players.c:248
#1 0x0045f967 in nanny (d=0x10eb62f0, arg=0x22ca50 "Jamdog") at interpreter.c:1417
#2 0x00440f80 in game_loop (mother_desc=3) at comm.c:903
#3 0x004410e0 in init_game (port=4000) at comm.c:531
#4 0x004414f6 in main (argc=1, argv=0x10011c80) at comm.c:378

Now, this may look scary and confusing, but it's relatively simple. The top line (beginning #0) is the point of the crash - it's the line that was being executed when the crash happened. Here is the breakdown of that line:

#0 - indicates the function depth. As you can see above, #0 is called from within the function at #1, and #1 is called from within the function at #2. The 'main' function will always be the last in the list.

load_char - This is the name of the function where the crash happened. Sometimes it is a function in the C core code, and not the MUD code, so look for the first function you recognise in the list.

(name=0x22c540 "Jamdog", ch=0x10ebcb48) - This shows the variables that were passed to the function. In this case, a string at memory address 0x22c540 and a char_data struct at memory address 0x10ebcb48. GDB handily shows the string for you, which was 'Jamdog'. You can see the contents of ch with the print command (see below)

at players.c:248 - This shows the file and line number where the crash happened.

You can now use the following commands to debug most problems:

  • info local - display the status of 'local' variables, variables created in the current function.
  • list - list the source code. Shows 3 or 4 lines above and below the current location.
  • print <variable> - display the staus of the specified variable (does not need to be local, and can view within structs this way - e.g. print ch->name or print *ch).
  • up - move 'up' the function tree (shown by bt) to the previous function.
  • kill - stop the current MUD program.
  • quit - exit from gdb (prompts to kill a running program)

Here is an example of these in use:

(gdb) info local
id = 0
i = 2
fl = (FILE *) 0x10011f74
fname = "plrfiles/F-J/jamdog.plr"
buf = "\204b\026aPÄ\"\000\000\000\000\000ÿ\001", '\0' , "\001"
buf2 = "X\006\000\000ðÃ\"\000\001\000\000\000$Ä\"\000(Ä\"", '\0' , "PE\02"
line = "ðé$\000í"...
tag = "þÿ\000\000\2268"
ch = (struct char_data *) 0x10ebcb48
(gdb) print *ch
$1 = {pfilepos = 50, nr = 65535, in_room = 791, was_in_room = 65535, wait = 1, player = {passwd = "Ja3pvo1SmcUB6", name = 0x8b117a0 "Jamdog", short_descr = 0x0, long_descr = 0x0, description = 0x0, bio = 0x0, title = 0x8b16308 "the Omnipresence", sex = 1 '\001', chclass = 4 '\004', level = 200, adm_level = 4, time = { birth = 1243951538, logon = 1288030404, played = 2148763}, weight = 146 '\222', height = 174 } }
(gdb) print ch->in_room
$2 = 791
(gdb) list
243 /* character initializations */
244 /* initializations necessary to keep some things straight */
245 ch->affected = NULL;
246 mudlog(NRM, LVL_GOD, TRUE, "Setting skills to zero");
247 for (i = 1; i <= MAX_SKILLS; i++)
248 GET_SKILL(ch, i) = 0;
249 mudlog(NRM, LVL_GOD, TRUE, "Setting sex");
250 GET_SEX(ch) = PFDEF_SEX;
251 GET_CLASS(ch) = PFDEF_CLASS;
252 GET_LEVEL(ch) = PFDEF_LEVEL;
(gdb) up
#1 0x0045f967 in nanny (d=0x10eb62f0, arg=0x22ca50 "Jamdog")
at interpreter.c:1417
1417 if ((player_i = load_char(tmp_name, d->character)) > -1) {
(gdb)

Running GDB on your MUD After a Crash
Obviously, you can't keep your MUD in debug mode all the time. If it crashed while you were asleep, it would 'pause' the MUD until you found it in the morning.

For that reason, modern tbaMUD releases do a 'core dump' in the event of a crash. This is basically a snapshot, or photograph of the MUD at crash time, and this can still be used by GDB to trace the problem.

The core dump is saved in your lib folder, but because it's so big (commonly 10-15Mb), only the mot recent core dump is kept.

To load the core dump into GDB, you need to be in the root folder of your MUD, and type:
gdb bin/circle lib/core
GDB will load in the normal way, but you don't need to run the MUD. You can immediately type bt to see the backtrace from crash-time, and you can use all the above commands to debug what happened. Because the MUD isn't running however, you won't be able to step-through the program (which usually isn't necessary anyway, and why I didn't add it to this guide).

New Admin and Mortal Levels

A major fundamental change has been made to the way TBA works, and although we are still ironing out the creases, I hope everyone agrees that it's a change for the better.


With the old system, MUD 'Imms' were always Imms, and were forced to log on a second, mortal character for proper testing of their zones. A few imms may have had the ability to change their level to become mortal, but this required a code change to allow them to return to normal.


The old admin levels were tacked onto the end of the mortal levels, like an afterthought of the original coders.


The new system that is now being used on TBA, and will be in the next release of the tbaMUD codebase, changes everything.


Mortal and Admin levels have been separated, and at any time, any imm can disable their godhood, becoming truly mortal for testing their shiny new zone, then can just turn their 'immness' back on. This is done by use of the new mortal command to become mortal, and the return command to return to normality.


As part of the new admin levels, admin privileges can be assigned (or removed) to any player, even mortals. You no longer have to be an Imm to build your own zone. A level 1 player could be given a zone and build privileges for that zone, and while in that zone, the player will be able to build using OLC, but outside that zone, be a normal mortal again.


Obviously, this has been a HUGE update, affecting virtually every file in the MUD source, so there may be teething problems following it's introduction. Please feel free to log any bugs you find (using bug report <header> in TBA), and I'll look into them as soon as I can.


prefedit, autodoor and autokey

There have been a few new features added to TBA today, and the SVN version of the tbaMUD codebase.

They are all things I have added to a few MUDs, and have always been very popular with players.

autokey Autokey, when enabled, will automatically use a door key (if it's in your inventory) when you try to open a locked door. You do not need to use the separate unlock command.
Autokey is a player toggle. It can be turned on and off using the autokey command, the toggle autokey command, or prefedit (see below).
autodoor Autodoor, when enabled, will automatically find the next available door when using the open, close, lock, unlock or pick commands. For example, in a room with several closed doors, one door will be opened each time the player types open door until all the doors are open. The player can still specify a direction to open a particular door.
prefedit An addition to OasisOLC for players! The menu-driven interface allows players to change all of their preferences and toggles.

Here's an example with autokey AND autodoor turned on

> look
A large room
You are in a large room, surrounded by doors. The doors to the north and south are lock, and the doors east and west are closed.
Exits: (n) (e) (s) (w)
> open door
The door is locked, but you have the key.
*click*
You open the door
> open door
You open the door
> open door
The door is locked, but you have the key.
*click*
You open the door
> look
A large room
You are in a large room, surrounded by doors. The doors to the north and south are lock, and the doors east and west are closed.
Exits: n e s (w)
>
Syndicate content