MySQL, is it a viable alternative?

Login to reply  Page: « < 1 of 2 > »
16 Jun 2009 - 10:201621
MySQL, is it a viable alternative?
This is an idea that has been at the back of my mind for a while now, and the more I think about it, the more it seems like a a whole lotta hard work, but will it be worth it?

The idea is using MySQL for MUD data. At first, maybe just pfiles, then maybe text files, then maybe the whole world...

Advantages
* Most servers restrict disk usage, but not the size of MySQL databases.
* Players would be able to change their login passwords on the MUDs website.
* The MUD website could easily have player profile pages.
* If the world was in a database, builders could use an OLC, or just build in their browser!
Disadvantages
* Not all MUD servers have PHP and MySQL
* MUD Admin would require knowledge of C, *nix Shell, MySQL and PHP, making it harder for new MUD owners
* Running a MUD from your home PC requires lots of extra stuff to be installed (Apache (or IIS)/PHP/MySQL)

Because of the disadvantages, I've always planned to have the ability to turn MySQL on and off in cedit, and providing a new tool in src/utils to convert pfiles->MySQL and MySQL->pfiles.

I'm basically posting this here to see what everyone else thinks of this idea? Would it be used by anyone, or is it simply an insane idea that should be locked away, never to see the light of day.

If I did create the patch to convert pfiles to MySQL, I would also provide PHP website scripts for changing user passwords, and showing player profiles...


__________________
16 Jun 2009 - 17:391623
I personally think that it would be a worthwhile thing to look into...the primary complaint that I have heard about transactional data storage for the mud system has been disk I/O concerns. You would have a hard time convincing me that MMORGs are not running in at least a partially setup transactional database. Some portions may be more static for speed, but I can't imagine handling that much data by flat file.


16 Jun 2009 - 21:201624
There are more, simple, reasons.

For:
- flatfiles are so last century.
- flatfiles require special parsers which are prone to break on small syntax changes.
- making changes to a "live" mud using flatfiles requires changes to in-memory structures, or a restart.
- Simply switching to use sql (of any kind) instead of flatfiles is relatively simple. Can be done in an afternoon or two, depending on how normalized you want the database schema.

Against:
- making use of this structure for anything but storing static things is difficult. Circlemud, and thus tbaMUD, is based on a conception of having all objects (and players, descriptors, mobs, rooms etc.) in memory when running. This means that a change in the lib dir has no effect on a running mud, just as a change in a running database would have no impact - unless more or less the entire mud is rewritten.
This also means that if you want the switch to databases to make sense beyond using less disk space, a lot of stuff needs to be changed.

If you have the time for it (some months), go for it. But generally, this last point makes me want to suggest you leave it alone. This is something that is easier added if writing something from scratch.


__________________
You know who I am.
17 Jun 2009 - 09:011628
Quote welcor:
Against:
- making use of this structure for anything but storing static things is difficult. Circlemud, and thus tbaMUD, is based on a conception of having all objects (and players, descriptors, mobs, rooms etc.) in memory when running. This means that a change in the lib dir has no effect on a running mud, just as a change in a running database would have no impact - unless more or less the entire mud is rewritten.
This also means that if you want the switch to databases to make sense beyond using less disk space, a lot of stuff needs to be changed.

If you have the time for it (some months), go for it. But generally, this last point makes me want to suggest you leave it alone. This is something that is easier added if writing something from scratch.

This is something I hadn't considered, but is a pretty big negative...

You could argue though, that a copyover would reload everything anyway. Thinking along these lines, Aderon has a 'reload' command, here's the help file:
RELOAD

Usage: reload < '*' | all | file >

Reload is used to reload text files such as the MOTD from disk into memory.
reload * reloads all the text files below.

Valid files are:
  background (/mud_dir/lib/text/background)
  credits    (/mud_dir/lib/text/credits)
  greetings  (/mud_dir/lib/text/greetings)
  handbook   (/mud_dir/lib/text/handbook)
  help       (/mud_dir/lib/text/help)
  immlist    (/mud_dir/lib/text/immlist)
  imotd      (/mud_dir/lib/text/imotd)
  news       (/mud_dir/lib/text/news)
  motd       (/mud_dir/lib/text/motd)
  info       (/mud_dir/lib/text/info)
  policy     (/mud_dir/lib/text/policies)
  wizlist    (/mud_dir/lib/text/wizlist)
  xhelp      (/mud_dir/lib/text/help/*)
I'm thinking that this maybe could be extended to include:
  world      (/mud_dir/lib/world)
  rooms      (/mud_dir/lib/world/wld)
  mobs       (/mud_dir/lib/world/mob)
  zones      (/mud_dir/lib/world/zon)
  objects    (/mud_dir/lib/world/obj)
  pindex     (/mud_dir/lib/plrfiles/index)
Then, all of the above converted to load from the database, rather than files.
If not, then I guess a copyover would have to suffice...


__________________
17 Jun 2009 - 10:051629
The reload command is nice, since it - on-the-fly - loads changed text into the mud.

In theory, there's nothing stopping a similar functionality (think "reload world"), when it comes to rooms, objects, etc. It's not written in an afternoon, but it is certainly doable, and might even be viable.

Whys and hows to consider if implementing something like this:
- are there advantages to reloading the world that can't be accomplished with a copyover ?
- how much state (if any) should be saved about the world as it is right now - if I'm mid-battle with an aggressive, tough mob, is the current fight state saved before the world is reloaded, or will I have to start over with half hitpoints after the reload (if my "half-fatigued" state is stored but the mob's just reloaded) ?
- which parts of the internal structures will we need to alter to get the functionality we want? Can we load the new data, compare it to existing data, and only change internal memory structures if the data has changed (thus eliminating the problem above, if neither mob or player are altered)?

Welcor


__________________
You know who I am.
18 Jun 2009 - 20:341637
Quote Jamdog:
Aderon has a 'reload' command, here's the help file


Not looking at a copy of tbaMUD at the moment, but fairly sure it has a command named xreload which does the same thing.

EDIT: Just checked it's actually named the same as yours, perhaps it used to be named xreload and was changed, or perhaps my memory is just altogether wrong.


__________________

Last edited by Fizban (18 Jun 2009 - 20:36)
19 Jun 2009 - 06:171640
There's a Diku-based MUD out there called Shadows of Isildur that uses MySQL. You can download the source code for their engine at their website and see what they've done. I hate that MUD, but I have to admit that the website/game integration is pretty slick.

I've done a little work to convert tbaMUD's flatfile DB to PostgreSQL and coded in a DB options menu accessible from the cedit menu. Here's what it looks like:

Configuration Editor
G) Game Play Options
C) Crashsave/Rent Options
R) Room Numbers
O) Operation Options
A) Autowiz Options
D) Database Options
Q) Quit
Enter your choice : 
d

PostgreSQL Configuration Options (Connection active)
E)      pgSQL databasing: On
H)      server hostname: localhost
A)      server address: undefined
P)      server port: 5432
D)      database name: tinamud2data
O)      client options: undefined
L)      server SSL mode: Off
N)      server servicename: undefined
U)      server user: tinamud2admin
S)      Set pgSQL user password (Set).
R)      Reinitialize connection (save changes first).
Q)      Exit To The Main Menu
Enter your choice : 

If you'd like, I could try to make a patch or something. It saves the settings to the same configuration file as all the other config data. Though it's for postgresql, you could probably change it to work with MySQL in a pinch.

To me, designing a good database that will fulfill all my MUD's needs seems as tricky as writing the code to interface the MUD with the SQL server/ converting all the flatfile stuff over. There is no doubt conversion would be an extremely tedious task, though.


21 Jun 2009 - 05:211647
Another mud that uses MySQL is Worlds of Carnage. Blobbie incorporated it into that DikuMUD codebase as well.. But he also wrote an entirely new scripting language called BlobScript, which is neat. I believe you can as of right now, build areas,objects,npcs, commands,skills,spells and various other things like resistant tables, racial/class based stuff, all on the browser on their website. I'm sure if need be I can talk to him and have him post on here his pro's/con's issues of adding it. But I also believe he is using backend xml support? Actually here is the link describing that:

http://www.wocmud.org/info/system.php?PHPSESSID=1cba9509a390ed5a62c7a9208316897e

"but then most MUD servers don't run on an XML backend templating system for object, room and mob creation." Quote from that link.



Last edited by Elervan (21 Jun 2009 - 05:22)
22 Jun 2009 - 17:001653
Using a SQL Database Back End
What you need to remember is that a database management system is more than just a "different way to store structured data". The main advantage of an RDBMS is to manage a transactional data environment - where elements of that data is changing regularly, and you want to manage those changes - for example, to wrap a number of individual changes into a single "transaction" that gets committed if they are all successful, or rolled back if not all of them can be made.

You also have to manage your transaction log files - over time these grow and grow and grow.... There is a big overhead to pay if you never make use of those saved transactions.

A poorly designed database (generally an over-normalized data model) will kill your disk performance.

Just some thoughts....


30 Jun 2009 - 00:471687
Nice
I really want to see a MUD running with MySQL. One of my ideas is to make a site card-game integrated with the mud, so, a mud MySQL database will be exactly what I need.

Good luck making it o/

Hope this will solve the Loss of player equipment bug that all circlemud's mud have.


30 Jun 2009 - 20:151690
Quote vdarkn:
Hope this will solve the Loss of player equipment bug that all circlemud's mud have.

???

Has there been a bugreport on this ? Steps to reproduce?

I can't say I've heard tbaMUD should suffer from this.

Also - this is not likely to be solved by switching to mySql - or any database. It is most likely caused by a code error, not a backend malfunction.


__________________
You know who I am.
04 Jul 2009 - 00:081703
Well, i really don't know any step to reproduce this or if it was reported.

But I've seen this bug on many circlemud's based worlds. It only happens when a lot of players is registered and many plays the mud for a long time. Its extremely random, so, i cant make something-like a step-to-step to reproduce it.

I always thought it was some problem on the circlemud rent code... but I never catch an error there.

Anyway... i'm waiting for the MySQL database conversion for tba and hoping for many new stuff for the website of our project.

Att.


06 Jul 2009 - 19:421716
An example
This is an simple example how to use mysql in a c program..


The first point i guess is make a new function: save_char_mysql(ch).. and put in the end of the existent save_char ... so the char is saved to ascii and to mysql....

The second is make a function to load_char_mysql(ch) .. so the players file will be in mysql.. =).. and can make a cedit preference that saves/loads in ascii, in mysql or both...


But i don't know how to save the char objects to a file.. is more complicade i guess..


[]'s


Stoneheart


 
/* Simple C program that connects to MySQL Database server*/
#include <mysql.h>
#include <stdio.h>

main() {
   MYSQL *conn;
   MYSQL_RES *res;
   MYSQL_ROW row;

   char *server = "localhost";
   char *user = "root";
   char *password = "PASSWORD"; /* set me first */
   char *database = "mysql";

   conn = mysql_init(NULL);

   /* Connect to database */
   if (!mysql_real_connect(conn, server,
         user, password, database, 0, NULL, 0)) {
      fprintf(stderr, "%s\n", mysql_error(conn));
      exit(1);
   }

   /* send SQL query */
   if (mysql_query(conn, "show tables")) {
      fprintf(stderr, "%s\n", mysql_error(conn));
      exit(1);
   }

   res = mysql_use_result(conn);

   /* output table name */
   printf("MySQL Tables in mysql database:\n");
   while ((row = mysql_fetch_row(res)) != NULL)
      printf("%s \n", row[0]);

   /* close connection */
   mysql_free_result(res);
   mysql_close(conn);
}


And this site has more useful examples too:

http://www.geocities.com/jahan.geo/mysql_c_by_example.html


__________________
Stoneheart from Brazil! =)
Projeto BrMUD!
http://brmud.com.br

Last edited by stoneheart (06 Jul 2009 - 19:51) Reason: Add some information
07 Jul 2009 - 09:321718
@Stoneheart:
That's the easy part - it's the designing of the db + the conversion utilities that'll take a little time to write.

@topic:
As we've seen in the previous posts, the "how" is not difficult, merely tedious.

The original question of "why", can be summed up in just a few words - external integration. Be it from website or other external utilities, it's easier to interface against an SQL database than a flat file, which needs to be parsed to be altered. And another thing - getting rid of the parsing of text files removes the ~ issue completely!

Pitfalls include, as was previously mentioned, transaction logs handling and making a sensible reload mechanism. Since the database in tbamud's case would primarily be a data storage - ie. no direct alterations of the world database from within the mud except via OLC - the transaction log problem should not be too massive (only player records are changed periodically).


__________________
You know who I am.
23 Jul 2009 - 18:131754
I think it'd be well worth the effort to convert to mysql from flat files. The web integration could be a huge benefit... not only for players, but for admins. Think of the possibilities in configuration management. You could go from changing constants in header files and recompiling/rebooting... to changing values in the db from a web interface. You could have a pulse event in the code check a value in the db to see if a reload is necessary. Maybe I'm dreaming here, but I'm envisioning being able to do things like tweaking rolls on damage spells on the fly... adding/removing levels... setting permissions on levels... giving access to or changing the level of a spell in a class... all on the fly. Things like this would obviously require a tremendous amount of work, but look at how far the code has come already. If people are going to continue work on the code for years to come, this might open the door to a lot a really cool stuff in the future.

John


29 Jul 2009 - 16:551768
Most definitely for MySQL integration here.
tbaMUD is an excellent start for a framework upon which builders can build their MUDs, and separating the code from the data would make it ideal. The design could be phased in approach until completion (player files and object catalog could be a start). There is even a MUD that has its source available in which they did this (Awakened Worlds) that was based upon Circle and could possibly be utilized as a start.
I'd love to be able to populate some spreadsheets, import it into my MySQL tables, and have a completely redesigned world at my fingertips...


07 Aug 2009 - 07:031800
Quote vdarkn:
I really want to see a MUD running with MySQL. One of my ideas is to make a site card-game integrated with the mud, so, a mud MySQL database will be exactly what I need.

Good luck making it o/

Hope this will solve the Loss of player equipment bug that all circlemud's mud have.


I think a banking system (and gambling) with player character coin from site to site would be interesting. Especially if it reflects the actions of player characters in different mediums from multiple sites. Even if the results of the interaction, were simple and cosmetic, it would be pretty cool.


11 Aug 2009 - 05:361806
Most definitely for MySQL integration here and mabe PHP to. lol
Also hope to good things to come.
Also MySQL could be added in tbaMUD 3.59 to make it tbaMUD 4.0.


12 Aug 2009 - 02:321808
Quote De:
Most definitely for MySQL integration here and mabe PHP to. lol
Also hope to good things to come.
Also MySQL could be added in tbaMUD 3.59 to make it tbaMUD 4.0.


Funny you mention that...
I had originally started my mud coding journey a while back by building out a mud server all coded in php and a browser-based builder with a MySQL backend for rooms, items, toons and such. My boss at work loved the idea and had started to help out but then I found this code-base and realised that it was all already done and I was just duplicating everything already done in another language, lol.


27 Jun 2010 - 19:372779
I've finally decided to give this a go, hopefully I'll be able to get this working enough for integration into stock tbaMUD (with an on/off toggle in cedit).

It's all going surprisingly well so far, the biggest hiccup I had was figuring out the changes to Makefile:
- LIBS =  -lcrypt -lm 
+ LIBS =  -lcrypt -lm -lmysqlclient 
That's all I needed for a clean compile, so posting it here for anyone else who got confused by why their mysql code wouldn't compile.

Keep watching this thread for updates!


__________________
27 Jun 2010 - 21:572780
I am very interested on seeing how much progress you get with it Jamdog. What kind of integration are you going for anyway? Are you going to consider perhaps data basing most things? (pfiles, wld files, other?)


27 Jun 2010 - 23:362781
I too am quite interested, what is the progress on it?


__________________
Zahn - www.insymniamud.com 3300
27 Jun 2010 - 23:572782
The final result I would like to achieve would be removal of pfiles and world files. This would probably need to be a separate branch of tbaMUD, as I would be writing specifically for gcc (which is used to compile most MUDs anyway) and MySQL. Not all servers would have these capabilities, and therefore there would be a need for the non-SQL based code too.

Rather than bloat the code by having both versions in one codebase, I'm aiming to (at least in the first instance) try to completely convert a fresh tbaMUD so that it doesn't use the lib folder at all, but loads everything from the MySQL database on startup. I may later try to create a version that allows MUD owners to toggle between the two options in cedit.

I'm starting with pfiles (database connection is already working), and hope to have something workable within a week. I've worked a lot with MySQL in PHP, and the method is similar in C, so it shouldn't be too hard. I'll keep y'all posted ;)


__________________
28 Jun 2010 - 13:282784
I just thought I'd let you in on the progress, so if anyone can see any obvious errors at this stage, bugs can be avoided later, and it gives other the chance to see the method.
Obviously, you first need to make the Makefile change I showed above, so that MySQL code will work.
I have created 2 files, these are
mysql.h:
#ifndef _MYSQL_H
#include <mysql/mysql.h>
#define _MYSQL_H 1
#endif

#define SQL_ERR mysql_error(SQL_conn)

#define MAX_SQL_FIELDNAME 30

/* mysql_field_count doesn't exist before v3.22.24 */
#if !defined(MYSQL_VERSION_ID) || (MYSQL_VERSION_ID<32224)
#define mysql_field_count mysql_num_fields
#endif

#ifdef _MYSQL_C
MYSQL *SQL_conn      = NULL;         /* MySQL Connection handler is a global  */
char  *SQL_server    = "localhost";  /* MySQL Server URL (usually localhost)  */
char  *SQL_user      = "my_user";    /* MySQL Server Username                 */
char  *SQL_password  = "my_password";/* MySQL Server Password                 */
char  *SQL_database  = "my_db";      /* MySQL MUD Database Name               */
char  *SQLTBL_users  = "my_users";   /* MySQL Table for player index          */
char  *SQLTBL_pfiles = "my_pfiles";  /* MySQL Table for player data           */
#else
extern MYSQL *SQL_conn;  /* Extern MySQL Connection for outside mysql.c  */
#endif
mysql.c:
/* Simple C functions that connect to MySQL Database server */
/* Created for TrigunMUD by Jamdog - 27th June 2010 */

#define _MYSQL_C 1

#include "conf.h"
#include "sysdep.h"
#include "structs.h"
#include "utils.h"
#include "db.h"

#include "mysql.h"

/*
 * connect_to_mysql function
 * Sets up the actual database connection, called at MUD startup
 */
MYSQL *connect_to_mysql(void) {
   SQL_conn = mysql_init(NULL);

   /* Connect to database */
   if (!mysql_real_connect(SQL_conn, SQL_server,
         SQL_user, SQL_password, SQL_database, 0, NULL, 0)) {
      log("SYSERR: MySQL: %s\n", SQL_ERR);
      return NULL;
   }
   return(SQL_conn);
}

/*
 * run_database_query function
 * Query the database, and return the results (or NULL)
 */
MYSQL_RES *run_database_query(const char *q)
{
   MYSQL_RES *res;

   if (!SQL_conn)   return NULL;
   if (!q || !(*q)) return NULL;

   /* send SQL query */
   if (mysql_query(SQL_conn, q)) {
      log("SYSERR: MySQL: %s\n", SQL_ERR);
      return(NULL);
   }

   res = mysql_use_result(SQL_conn);
   return res;
}

/*
 * MySQL Version of the load_char function
 */
void mysql_load_char(int pid, struct char_data *ch)
{
  MYSQL_RES   *res;
  MYSQL_ROW   row;
  MYSQL_FIELD *field;
  char q[MAX_STRING_LENGTH];
  char f1[128], f2[128], f3[128], f4[128];
  int i;

  sprintf(q, "SELECT * FROM %s WHERE id='%d'", SQLTBL_pfiles, pid);
  if (!(res = run_database_query(q))) {
    log("SYSERR: MYSQL: Invalid or missing pfile data (ID: %d), %s", pid, SQL_ERR);
  }

  /* Should only be one row, so grab it */
  row = mysql_fetch_row(res);

  mysql_field_seek (res, 0);                      /* Go to the first column       */
  for (i = 0; i < mysql_num_fields(res); i++)     /* Cycle through result columns */
  {
    field = mysql_fetch_field (res);
    switch (*(field->name)) {
      case 'a':
             if (!strcmp((field->name), "ac")) GET_AC(ch) = atoi(row[i]);
        else if (!strcmp((field->name), "act_flags")) {
          if (!sscanf(row[i], "%s %s %s %s", f1, f2, f3, f4) == 4) {
            for (i=0; i<PR_ARRAY_MAX; i++)
              PLR_FLAGS(ch)[i] = 0;
            log("SYSERR: MySQL: Invalid act_flags field in %s (ID: %d)", SQLTBL_pfiles, pid);
          } else {
            PLR_FLAGS(ch)[0] = asciiflag_conv(f1);
            PLR_FLAGS(ch)[1] = asciiflag_conv(f2);
            PLR_FLAGS(ch)[2] = asciiflag_conv(f3);
            PLR_FLAGS(ch)[3] = asciiflag_conv(f4);
          }
        }
        break;

      default:
        log("SYSERR: MySQL: Unhandled field %s in table %s", (field->name), SQLTBL_pfiles);
        break;
    }
  }

  /* free MySQL results table */
  mysql_free_result(res);
}

I decided to make SQL_conn a global, permanently 'open' connection to the database rather than keep opening a db connection every time you want to run a query, then closing it. This should save time.

Obviously, mysql_load_char is unfinished, but has a couple of example fields done, to show method.

The database connection data (host, username, password, etc...) would be editable by Imps in cedit eventually. Obviously, you would need to set these for your server...

If no-one sees and glaring errors in my code, I'll continue... ;)


__________________
28 Jun 2010 - 19:152785
Quote Jamdog:
I decided to make SQL_conn a global, permanently 'open' connection to the database rather than keep opening a db connection every time you want to run a query, then closing it. This should save time.

Yes, but you may run into some trouble with this method. Sql connections time out. So, if you don't have transactions happening over the connection for a while (default 28800 seconds or 8 hours), the connection will be closed at the database end. To avoid this I see a couple of options:
1. Alter the timeout to some preposterously high amount (31536000 seconds (= a year!) is the maximum value). Simple fix.
2. Connect when you need it. Close afterwards. This is the most labour intensive (ie. a little more code every time you access mysql), but also the cleanest if you ask me. Instead of having a global, you use your function to create the connection and then call mysql_close() when done with it.
3. You close the connection when going to sleep, open it when waking up, and then, every hour, you do something on the connection to keep it alive.

Quote Jamdog:
Obviously, mysql_load_char is unfinished, but has a couple of example fields done, to show method.
....
If no-one sees and glaring errors in my code, I'll continue... ;)

I can see from your example that you've settled on a schema that's very "wide". One column for each value. This is arguably the easiest way to translate the values from db to memory.

There's already a large parser for player files. It works generally on the principle "tag x means value is type Y, and must be set using algorithm Z". This can be leveraged by your code either by having "tagnames" as column names, or by using a different schema altogether.


Your suggestion looks like this in schemaform:
table my_pfiles
ac varchar(40),
act_flags varchar(160),
player_flags varchar(160),
title varchar(250),
..
and so on...
..
..
and so forth...
..
..
and it goes on forever!

This is where schema normalization comes into play. We basically have a handful of different types in some 120 different columns. I'll normalize a bit, and I'll post a suggestion for database schema here later tonight.


__________________
You know who I am.
Login to reply  Page: « < 1 of 2 > »