tbaMUD Continued development of the CircleMUD core

Following this thread, I want to take a crack at curing the stock telnet protocol negotiaton (or lack of it). I know that thread is now almost 2 years old, but this has been on my todo list since then.

Telnet negotiation was fixed in SMAUG and ROM codebases, but never in Circle, and tbaMUD unfortunately inherited this problem.

I've contacted elanthis on Facebook, and he's more than happy for me to use his libtelnet patch as a basis.

This comes with support for the following:
MCCP2 (MUD Client Compression Protocol, v2)
This effectively compresses the data passed between the server and the client, like if you zip a file, email it to a friend, and they unzip it. The player should see no visible difference except a speed improvement, as less physical data is being passed. This is used in CWG releases I believe, but in the days of modern broadband has become pretty obsolete.

MSSP (MUD Server Status Protocol)
This is useful is you register your MUD with a MUD list that supports it. It enables the MUD list to connect to your MUD and grab all kinds of info (MUD name, server URL/port, website URL, genre, number of areas/rooms/mobs/objects, etc...), which it then uses to keep the MUD list updated. Full MSSP support would include two update methods, telnet negotiation method and a 'plaintext' method. More info about MSSP here.

ZMP (Zenith MUD Protocol)
Used for sending messages outside of the normal data stream. This may be used for many things, such as keeping a small area of the client screen to display inventory or stats. Unlike other protocols, ZMP commands are passed entirely through telnet negotiation. More info on ZMP here.

I just wanted some feedback on these, should full support for these protocols be added to stock tbaMUD?

Personally, I think that MCCP is a waste of time. It was handy to compress the data stream at 9,600 baud when any speed-up was very noticable. Broadband makes MUD traffic virtually instant anyway, and compressing/decompressing the data stream may actually even slow it down...

MSSP however I think is very useful, and I would like to add a new cedit menu option to configure MSSP, with the option to send MUD info via telnet, plaintext, both or none. I've added a plaintext-only version to Trigun, and it updates this MSSP MUD List about 4-5 times per day...

ZMP I'm not sure about, but willing to make it a separate patch for anyone who wants it...

Adding proper telnet negotiation also paves the way for other protocols:
MXP (MUD Extension Protocol)
Allows the MUD to send HTML-like info to the client, to display in-line bitmap images, hypertext links, font changes and more. More MXP info here.

MSP2 (MUD Sound Protocol, v2)
Add sounds and background music to your MUD (different music for each zone? Light music for walking in fields, oppressive atmospheric music for dungeons, etc...). You can have the sound of swords clashing during battle, a gurgling death cry when a mob bites the dust. The original MSP only used telnet negotiation for turning MSP on, then used plaintext messages for the sound. MSP2 has moved the sound control purely into telnet negotiation.

Both of the above I plan to make patches for, as I feel they probably shouldn't be in the stock codebase.

Please let me know your thoughts, ideas and suggestions...

I believe that we should have the fixes in, and generally agree with you. Compression is probably unnecessary now, but if the telnet library includes it, then why not have it? I seem to recall some weirdness with MCCP and zMud, if you got disconnected, then strange characters were send. I could well be confused though.

Other extensions are pretty good - MSSP have as a standard feature that can be configured and disabled through cedit. I like the other protocol too (the name escapes me because I gut the bulk of your post from the quote). I've been playing another mud (non-circle codebase) where the admin has developed his own Telnet client, and lots of stuff is displayed out of band - map, stats, groups status... it's very good.

Nope, you are correct, CWG releases have been dogged by this problem, but I believe that it's due to incorrect or incomplete telnet negotiation handling. The telnet handler I plan to add to tbaMUD is far more advanced than the CWG one.


There will be cedit options to disable anything I add.
This sounds like ZMP, or something very similar...

Also, I forgot to mention that this patch comes with TTYPE recognition, so when a player logs in, it can work out what MUD client they are using. This could be very handy in several situations. For example, the SimpleMU client won't allow / commands in the text editor, and you have to use // instead. If the editor recognises the player is using SimpleMU, it could change these to # commands instead (#s to save, #f to format, etc...), or at least display the editor help with // instead of /. I'm sure other people could come up with plenty of other examples where TTYPE recognition would be useful...

Hi, I checked out your forums after reading a post on MudGamers about tbaMUD working on MSSP support, and I stumbled across this thread. Hope you don't mind if I make a few observations about the protocols you're discussing.

The main benefit of MCCP is not for the players, but for the mud, particularly for larger muds. The owner of Aardwolf, for example, posted stats showing that MCCP reduced his bandwidth usage to less than a fifth of its previous amount, and went on to say that "if we ever have to move to a host that charges for bandwidth it could very well mean the difference between being able to afford to stay online or not". Of course this probably won't be an issue for smaller muds, but it's still nice to have the option available.

There are various out-of-band protocols - ZMP, MSDP, ATCP, GMCP, NEW-ENVIRON, 102, 94, etc - even the latest version of MXP supports OOB communication. They each have their own pros and cons, but can all be used to do much the same sort of thing. With ZMP you'd have to design your own packages for sending and receiving data, though, as the specification is very open-ended. Other muds would have their own approaches, so plugins/scripts using ZMP probably won't be very portable unless you can encourage other people to use your extension of the ZMP specification.

MSSP is still fairly new and not used much, but it's gradually picking up popularity. It also uses the same data format as MSDP (both protocols were designed by the same person), which makes it a little easier to add one if you've already implemented the other.

MXP is one of the harder protocols to implement, but it does offer some nice features - however most implementations only seem to support a subset of those features. You can generally rely on having links and 24-bit colour, but things like fonts and embedded graphics don't seem to be supported much. You could also just use 256 colours instead, as that's really easy to add. But I have to admit the links are a nice feature.

MSP2 is only a proposal, I don't know of any servers or clients that actually support it. If you want OOB sound triggers, you could do it through ZMP/MSDP/etc (much like MXP supports MSP-style sounds as a package). The negotiation in the original MSP is pointless, as you can't even use it to switch on sound because some clients support MSP sound triggers but not the negotiation, and even those that support the negotiation can't assume that the user wants to play sounds (or even has the soundpack) just because their client supports MSP.

Thanks KaVir, some really interesting points there.

I've been pretty ill with flu for the last couple of weeks, so not done any coding. The telnet patch is in on my home PC (in my local SVN copy, ready for committing to the SVN server), but still largely untested, and I'm not happy it's 100% complete yet. When I feel a little better, I'll be testing then committing it.

I didn't realise that MSP2 was only a proposal. Does that mean it's unsupported by any clients?

I also didn't realise that MCCP made such a big difference to bandwidth (I though it maybe only halved data traffic), but if it cuts it down by 80%, then it's definitely worth having.

As you said, MSSP isn't globally supported yet, but MUDBytes use it for their MUD lists and a couple of other sites, and TMC have plans to add it to theirs in future. Of course, any client can be used to test the plaintext version (by entering "MSSP-REQUEST" at the Username prompt), but only a few clients (like Tintin++, which I'm using for testing) can be used to debug or view telnet negotiation. It is more designed for MUD Lists rather than players, and I think if more codebases had MSSP built in, more listings sites would support it. For this reason, I've emailed the owners of almost every MUD listings website to ask if they support MSSP, and to let them know that the tbaMUD codebase is soon to have full MSSP support built-in.

To the best of my knowledge MSP2 isn't natively supported by any clients, and there were also a few concerns raised about the specification when it was discussed on MudStandards last year, but of course some clients (including MUSHclient, TinTin++ and CMUD) allow you to add your own support through plugins/scripts. However MSP1 is natively supported by several clients, and (because it's in-band) can also be used by a wider range of clients through simple triggers. So even if you do decide to add support for MSP2, I would personally suggest supporting MSP1 as well.

The same doesn't apply to MCCP. Every client that supports MCCP1 also supports MCCP2, so the former is now redundant and can be ignored. It's worth noting that although MCCP can significantly reduce bandwidth, you pay for those savings with a higher memory and CPU overhead, so it may not be desirable for all muds. Games using a dedicated host that charges based on bandwidth will definitely want MCCP, but those using a mud hosting service that charges based on memory and CPU usage will probably want to deactivate MCCP entirely.

MudBytes uses a few of the MSSP fields to produce a data cloud, and MudStats uses the "players" field to generate its charts, but other than Scandum's crawler I don't know of anyone that makes extensive use of it. However the protocol is still young, and I remain optimistic that it will continue to grow in popularity, eventually being incorporated into at least some of the mud listing sites.

I've already added a simple state machine to my code base to fix the telnet protocol handling and add window size autonegotiation. It's a pretty trivial fix. If you have a better one though there's no need for mine.

As tbaMUD is a codebase very friendly to novice administrators Im very interested in what these changes will mean to the playing experience. Will I be able to use these protocols to capture information without it being sent to the main output window? What kind of information can I retrieve and which one of the protocols should I be using to do this? Is it the kind of thing a novice programmer can pick up?

Ive been working on and off with using triggers on MUSHclient to capture information sent to output (such as inventory, equipment, score, etc) but if I could do this using one of these telnet negotiation protocols it seems I could do alot more with it. I think KaVir you used MXP in creating your GodWars client plugin, but I didnt see too much mentioned about MXP in this thread and I was wondering if one of the other ones listed is doing the same thing and trying to figure out which one I should learn how to use.

If the data is sent out-of-band, yes, as it will be separate from the main data stream (i.e., all the text that's normally displayed in the window). MSSP, MSDP, ZMP, ATCP, GMCP, etc, all send their data out-of-band. This means you can have energy bars, maps, icons, etc, which update even if there is no text scrolling up the window.

On the other hand if you use an in-band protocol like MCP, SFXP or MXP, you'll end up with blank lines even if you gag the information from being displayed. That looks pretty nasty when you've got energy bars updating every second.


You can retrieve any information you wish, although of course you'll need to update the mud to provide it. For example if the client wants to retrieve health and mana, the mud will need to recognise the request and send back the appropriate data. If the protocol is already part of the mud, then I imagine adding new variables would probably involve updating a table.

As for which out-of-band protocol you wish to use, that's a difficult one, but personally I would suggest either MSDP or GMCP. These are the options I'm aware of (although of course you could also make up your own):

ZMP: A lot of the documentation seems to have vanished (dead links), and although the specification is much less vague than most of the others, it's also much less complete - you'd need to design your own packages for it to be any use, it's not an "out of the box" protocol like the others. I don't know of any operational muds that support it, and it uses NUL bytes which quite a few people dislike.

102: Created and used exclusively by Aardwolf, until they dropped it in favour of GMCP, so it doesn't have much of a future now. It uses a Lua-style syntax and is natively supported by Mudlet and MUSHclient.

ATCP: The IRE protocol, natively supported by Mudlet and CMUD, it's now redundant having been replaced with GMCP (which some people insist on calling ATCP2, due to some major disagreements with the way the protocol was designed).

GMCP: I've not been following this much lately, but when I last looked the specification was rather vague. There are apparently certain options that can be selected, but data is sent in packages (such as a 'vitals' variable which has a group of associated values), which I dislike for a couple of reasons (firstly because it's not very customisable for plugins, and secondly because it can result in unchanged data being sent to the client). It uses a (mostly) JSON format and is natively supported by Mudlet and CMUD.

MSDP: Specification is clear, design is simple but open-ended, and my script-writers dropped ZMP and starting using MSDP as soon as I'd added it to the mud. Allows the client to specify exactly what data should be sent. I don't know of any other operational muds that support it, but the design and data format is very similar to MSSP, which has been adopted by quite a few muds, so it's not entire untested. Native support is being added to at least two new clients that are under development.

94: A custom protocol created for (and used by) ConQUEST for their custom client. As far as I know it doesn't have any specification.

MXP: The latest version of MXP allows you to transmit out-of-band data, but as far as I know only CMUD supports it, and MXP itself isn't really that widely supported anyway.

NEW-ENVIRON: The Telnet Environment Option, you could in theory use this for mud-specific data.


The concept itself is very much like ANSI colour - if you want to display some text in red, you send the escape character followed by "0;31m". If the client supports 256 colours you might decide to display some text in orange, by sending the escape character followed by "38;5;202m". Well this is much the same sort of thing - you send some special characters to indicate special behaviour.

Having said that, adding protocol support to the mud can involve some effort, and might prove overwhelming for a novice. But if it's already been added to the codebase (or is available as a well documented snippet) it should be fairly easy to use and extend.


Well you'll probably find you can do the same amount, but much more easily and cleanly.


Nope - I do support MXP, but I only use it for clickable links. The plugin uses MSDP.

I actually implemented ZMP first, then later switched to MSDP, because I found it better suited to my needs. I've also added support for psuedo-ATCP
- basically MSDP data treated as a custom ATCP package, so that Mudlet users can create their own GUIs as well if they want to.

MUSHclient, TinTin++, zMUD and CMUD allow you to add your own out-of-band protocols via scripts/plugins, so you could use them for any of the options I listed earlier, or even make up your own. However Mudlet only supports 102, ATCP and GMCP.

Awesome, thanks that gives me alot to chew on. So now my obvious question is Jamdog do you think we could add MSDP support to tbaMUD?

I'm curious what the status of this is? I've been finding that mudstats/etc has the wrong stats for my mud (# of players, etc) more often than not, and I wanted to implement mssp - but if you're already working on it...

Plus I have some crappy MXP stuff in place (that I more or less gave up on except for some cool links) and some of this other stuff looks great, too, if it's all going in at once...

Will it be a patch or just part of the core?

I downloaded tbaMUD last night and tried adding MSDP. Got it partially working, but the low-level stuff works differently to anything I've encountered before - particularly the output buffer - so it's not quite the 10-minute job I'd been expecting.

Is it worth continuing, or is someone else already in the process of adding MSDP?

KaVir -

I dont know if anyone else is working on this or not or if they are, how far along those projects might be, but if this would allow MUSHclient to make use of some of the graphical GUI scripts with tbaMUD I would be forever grateful.

You can now grab the snippet from here, and a simple GUI plugin from here. I'm afraid I've not yet finished the installation instructions for TBA (they're only partially done) - perhaps someone who knows TBA better than me could finish them? If not, I'll have another look when I get the chance.

KaVir,

I have not heard anything from the community here and don't want to let this go without a comment. I do not think there is anyone here that is working on adding that piece of code in. I also feel that it is one of the most important pieces of code that could be added to circle/diku. I have tried to patch it in, but ran into issues and could not get it working right. I am sure we would all appreciate any assistance in developing a full tutorial / patch for tbaMUD.

Hey folks,

I've gone ahead and tried pluggin Kavir's snippet into the source and making the necessary extra additions to get it to work. Attached is a link to the patch file I created against my changes in the 3.62 source I have it hosted on megaupload. I set up an mxp link for autoexits so whenever you look with autoexits on it SHOULD display a link to click for each direction according to the usage guide Kavir wrote. At first just using write_to_output It wouldn't interpret the \t tags it would just display tabs on the mudclient whenever I looked at something with an mxp tag. I pushed the processoutput function to vwrite_to_output and now it strips the tags but doesn't display a link. I tried using the advanced color options and got slightly more success but all it did was turn text standard red not orange, when I used the manual tag to force dark green foreground it worked, but using dark brown([B210]) created a dark red background not dark brown. I think it might be a problem when vsnprintf is run to populate arguments on the data it may be tearing out some of the tag data but I have my doubts but I've not really used the va set of functions that much so its really guesswork on my part. I've turned on verbose logging of mxp in mushclient and The only notice I get is that it's turned on when I have Use MXP set to query. This is my first time back writing in C in many many years I didn't want to try rewriting any more of KaVir's actual code then I had to. Im hoping maybe another set of eyes will see what I missed but I'll be trying in the meantime. I'd really really like to get this working in tba since I have plans for a project I think this would be really great for. I've included screenshots of the data(note im on a mac but mushclient is running on a win7 vm in parallels) Also please note that mxp enabled links on the room exits are added in all these screenshots as well but no dice.

MXP enabled room exits:

Dark brown background on exits:

Dark green foreground on exits:


Download for the patch to add kavir's snippet:
http://www.megaupload.com/?d=WGIV2Z1I

Edit - Oh also I tried this with cmud my normal client and mxp doesn't work as well. Also my duhness forgot to mention that along with mxp not working none of the variables seem to be sent either to update the bars or anything.

The snippet will automatically strip out links if the client doesn't support MXP, and downgrade colours to ANSI if the client doesn't support XTerm 256 color. Without MXP there's no way to detect if MUSHclient supports the extended colour, so "210" would just be treated as red (as that's the closest match).

In other words, it looks like you've got the output working correctly. The problem appears to be with the negotiation - it's failing to switch on MXP. If the code is working correctly, you shouldn't need to configure anything in MUSHclient, it should do it all automatically.

What you could try is downloading WinTin++, entering "#config {debug} on", and then connecting to your mud. Does the mud send a load of negotiation requests? If so, then the first part is working. Does the mud then send you a list of MSSP variables? If so, then the second part is working. WinTin++ also uses a different method of switching on the extended colour (cyclic TTYPE), so it'd be interesting to see if that fixes the colour issue.

Okay I had a look at the code, and it seems I made a mistake in my (admittedly unfinished) TBA instructions. This:

ProtocolInput( t, read_buf, strlen(read_buf), t->inbuf );

Should actually be:

ProtocolInput( t, read_buf, bytes_read, t->inbuf );

In addition, just before you return from the enter_player_game() function, you need to do this:

MXPSendTag( d, "<VERSION>" );

That just leaves the problem of blank lines being sent. For that, you need to make sure you never send a new prompt (or blank line) if t->pProtocol->WriteOOB is > 0.

Holy cows batman that worked. that one change on the protocolinput fixed it all. Works great, mxp, the variables, updaing the generic gui and everything, cheers! I've put the new patch together and learned how to use mudbytes paste bin rather than megaupload. I'll submit the patch to the snippets over their as well.

--tba-prot.patch--
http://www.mudbytes.net/index.php?a=pastebin&s=view&pid=45830

I got all of this patched in and working fine. The only problem now is that I have the problem you mentioned above. On clients without MXP enabled everything works fine. With MXP enabled everything works fine except for extra prompts. I have code that splits regen over time so that a player is constantly regening points if they are below their max on any point. With the MXP enabled, if a player is below max on a point and they are updated an extra prompt is returned.

example...

first is no points to update over a span of 5 minutes..
Now look after I set my hit to below maxhit... the below is over the time span of 10 seconds...


I edited comm.c and added the below, but it doesn't seem to help.

I am beating my head against a wall and this one single issue is the only thing standing in the way of me having a production MXP system. Thanks in advance for any replies/assistance I get.

EDIT:

This seems to be a bug in the GUI Plugin -- I enabled tracing and each update got:

TRACE: Executing Plugin Generic_GUI script "OnPluginTelnetSubnegotiation"

It seems this is sending an extra return to the MUD which is generating a new prompt. Can you think of an easy way to suppress this?

Kewlb: It's not MXP causing the blank lines, it's MSDP - to fix it, you need to add three WriteOOB checks to your process_output() function in comm.c:

(Sorry about the formating, but it's actually WORSE with the code tags)

/* add the extra CRLF if the person isn't in compact mode */
if (STATE(t) == CON_PLAYING && t->character && !IS_NPC(t->character) && !PRF_FLAGGED(t->character, PRF_COMPACT))
if ( !t->pProtocol->WriteOOB ) /* <--- Check 1 */
strcat(osb, "\r\n"); /* strcpy: OK (osb:MAX_SOCK_BUF-2 reserves space) */

/* add a prompt */
if ( !t->pProtocol->WriteOOB ) /* <--- Check 2 */
strcat(i, make_prompt(t)); /* strcpy: OK (i:MAX_SOCK_BUF reserves space) */

/* now, send the output. If this is an 'interruption', use the prepended
* CRLF, otherwise send the straight output sans CRLF. */
if (t->has_prompt && !t->pProtocol->WriteOOB) { /* <--- Check 3 */
t->has_prompt = FALSE;
result = write_to_descriptor(t->descriptor, i);
if (result >= 2)
result -= 2;
} else
result = write_to_descriptor(t->descriptor, osb);

I added all the above code, but it is still doing the same thing. Each time the player obtains a point update a new prompt is being generated and or an extra CrLf is being sent which generates a new prompt. I would be willing to monetarily compensate for work done to try and resolve this problem and or some tutoring to help me better understand the system and how to put it to use.

Okay, here are the updated instructions for TBA:

http://www.godwars2.org/download/INSTALL_TBA.TXT

I just added them on a fresh copy of TBA to make sure they work.

minor changes to avoid compiler warnings / errors:


By the way -- this did fix the extra returns/prompts issue. Now if I can only figure out why MXP is not working correctly when I tag text for output via \t(text\t)

Very strange, I've no idea why you'd need to do that.


Use \t when it's a compiled message (like a send_to_char() or something). If you're adding it to a help file, you need to literally press the tab key.

I assume you get the message about MXP being enabled when you log on? And it's using orange and brown colours?