You are an unregistered user, you can register here
Navigation

Information

Site

Donations
If you wish to make a donation you can by clicking the image below.


BeyondUnreal News

 
Go Back   The Unreal Admins Page > Forums > Unreal Admins > Unreal Tournament > UT Server - General Chat

Reply
Thread Tools Display Modes
  #1  
Unread 1st September, 2013, 01:33 PM
Sp0ngeb0b's Avatar
Sp0ngeb0b Sp0ngeb0b is offline
Godlike
 
Join Date: Sep 2008
Location: Germany
Posts: 488
Default [Nexgen 1.12 // TCP Transfer // empty Strings SOLUTION]

[Nexgen 1.12 // TCP Transfer // serious bug with empty Strings on arrays (formatCmdArg) // SOLUTION]

This info is only related to programmers!

Introduction
Although I don't know how many of you people are still developing for UT and especially for Nexgen, but I don't want to keep this information for myself anyways as it might help others alot. While developing my newest Nexgen Plugin, I made extensive use of the Server <-> Client TCP communication system introduced in Nexgen 1.12. A feature which has not really established itself in the modding community, which is pretty incomprehensible from my point of view. It offers a way way faster data transfer between Server and Client, compared to this the original UT netcode sucks even more. Anyways, while working with it I ran into some strange stuff, which brought me to digging deep into Defrost's work in Nexgen 1.12 ...


What was the problem?
The data transfer worked pretty well generally, but there was one issue I discovered which made me wonder: I wasn't able to save empty strings clientside. They somehow turned into strange numbers while beeing transfered to the server side. After intensive bug hunting on my plugin I was pretty sure the issue was not on my side... And then a light dawned on me: The strange numbers turned out to be the index number of the specifc array entry! It was pretty clear that somehow during the TCP sending process, the empty string must have been completely ignored and therefore the following Index data must have been erroneously recognized as the new value.

A look into the formatCmdArg() function of NexgenUtil.uc confirms this suspicion:

Code:
/***************************************************************************************************
 *
 *  $DESCRIPTION  Formats the specified string so that it can safely be added to a command string as
 *                an argument of that command.
 *  $PARAM        arg  The argument string that is to be formatted.
 *  $RETURN       The properly formatted argument string.
 *
 **************************************************************************************************/
static function string formatCmdArg(coerce string arg) {
	local string result;
	
	result = arg;
	
	// Escape argument if necessary.
	if (arg == "") {
		arg = "\"\"";
	} else {
		result = arg;
		result = class'NexgenUtil'.static.replace(result, "\\", "\\\\");
		result = class'NexgenUtil'.static.replace(result, "\"", "\\\"");
		result = class'NexgenUtil'.static.replace(result, chr(0x09), "\\t");
		result = class'NexgenUtil'.static.replace(result, chr(0x0A), "\\n");
		result = class'NexgenUtil'.static.replace(result, chr(0x0D), "\\r");
		
		if (instr(arg, " ") > 0) {
			result = "\"" $ result $ "\"";
		}
	}
	
	// Return result.
	return result;
}
As you can see in line 8 of the function code, the wrong variable got assigned (arg instead of result). Therefore, the real expression for an empty string ( \"\" ) isn't returned, but instead just an empty string which isn't recognized on the other machine.


Solution

I don't want to overrun you with to many details, since you would probably need some time to understand how Nexgen's TCP stuff works (I needed a lot of time aswell).
There are basically 2 ways to fix this:
  • 1) Ofcourse, you can fix the invalid formatCmdArg() function directly. But this would mean you have to recompile the whole core package of Nexgen + all its plugins, which I wanted to avoid. This method is only recommended if you want to create your own custom Nexgen Core version.
    But if there's ever gonna be an updated, official version of Nexgen (1.13), we need to make sure to include this fix!
  • 2) Manual fix for a plugin:
Into your PluginController class, insert this:
Code:
/***************************************************************************************************
 *
 *  $DESCRIPTION  Fixed serverside set() function of NexgenSharedDataSyncManager. Uses correct
 *                formatting.
 *
 **************************************************************************************************/
function setFixed(string dataContainerID, string varName, coerce string value, optional int index, optional Object author) {
	local NexgenSharedDataContainer dataContainer;
	local NexgenClient client;
	local NexgenExtendedClientController xClient;
	local string oldValue;
	local string newValue;

  // Get the data container.
	dataContainer = dataSyncMgr.getDataContainer(dataContainerID);
	if (dataContainer == none) return;

	oldValue = dataContainer.getString(varName, index);
	dataContainer.set(varName, value, index);
	newValue = dataContainer.getString(varName, index);

	// Notify clients if variable has changed.
	if (newValue != oldValue) {
		for (client = control.clientList; client != none; client = client.nextClient) {
			xClient = getXClient(client);
			if (xClient != none && xClient.bInitialSyncComplete && dataContainer.mayRead(xClient, varName)) {
				if (dataContainer.isArray(varName)) {
					xClient.sendStr(xClient.CMD_SYNC_PREFIX @ xClient.CMD_UPDATE_VAR
						              @ static.formatCmdArgFixed(dataContainerID)
						              @ static.formatCmdArgFixed(varName)
						              @ index
						              @ static.formatCmdArgFixed(newValue));
				} else {
					xClient.sendStr(xClient.CMD_SYNC_PREFIX @ xClient.CMD_UPDATE_VAR
						              @ static.formatCmdArgFixed(dataContainerID)
						              @ static.formatCmdArgFixed(varName)
						              @ static.formatCmdArgFixed(newValue));
				}
			}
		}
	}

	// Also notify the server side controller of this event.
	if (newValue != oldValue) {
		varChanged(dataContainer, varName, index, author);
	}
}



/***************************************************************************************************
 *
 *  $DESCRIPTION  Corrected version of the static formatCmdArg function in NexgenUtil. Empty strings
 *                are formated correctly now (original source of all trouble).
 *
 **************************************************************************************************/
static function string formatCmdArgFixed(coerce string arg) {
	local string result;

	result = arg;

	// Escape argument if necessary.
	if (result == "") {
		result = "\"\"";                      // Fix (originally, arg was assigned instead of result -_-)
	} else {
		result = class'NexgenUtil'.static.replace(result, "\\", "\\\\");
		result = class'NexgenUtil'.static.replace(result, "\"", "\\\"");
		result = class'NexgenUtil'.static.replace(result, chr(0x09), "\\t");
		result = class'NexgenUtil'.static.replace(result, chr(0x0A), "\\n");
		result = class'NexgenUtil'.static.replace(result, chr(0x0D), "\\r");

		if (instr(arg, " ") > 0) {
			result = "\"" $ result $ "\"";
		}
	}

	// Return result.
	return result;
}

Into your Client class, add the following:
Code:
/***************************************************************************************************
 *
 *  $DESCRIPTION  Fixed version of the setVar function in NexgenExtendedClientController.
 *                Empty strings are now formated correctly before beeing sent to the server.
 *
 **************************************************************************************************/
simulated function setVar(string dataContainerID, string varName, coerce string value, optional int index) {
	local NexgenSharedDataContainer dataContainer;
	local string oldValue;
	local string newValue;

	// Get data container.
	dataContainer = dataSyncMgr.getDataContainer(dataContainerID);

	// Check if variable can be updated.
	if (dataContainer == none || !dataContainer.mayWrite(self, varName)) return;

	// Update variable value.
	oldValue = dataContainer.getString(varName, index);
	dataContainer.set(varName, value, index);
	newValue = dataContainer.getString(varName, index);

	// Send new value to server.
	if (newValue != oldValue) {
		if (dataContainer.isArray(varName)) {
			sendStr(CMD_SYNC_PREFIX @ CMD_UPDATE_VAR
			        @ class'PluginController'.static.formatCmdArgFixed(dataContainerID)
			        @ class'PluginController'.static.formatCmdArgFixed(varName)
			        @ index
			        @ class'PluginController'.static.formatCmdArgFixed(newValue));
		} else {
			sendStr(CMD_SYNC_PREFIX @ CMD_UPDATE_VAR
			        @ class'PluginController'.static.formatCmdArgFixed(dataContainerID)
			        @ class'PluginController'.static.formatCmdArgFixed(varName)
			        @ class'PluginController'.static.formatCmdArgFixed(newValue));
		}
	}
}


/***************************************************************************************************
 *
 *  $DESCRIPTION  Corrected version of the exec_UPDATE_VAR function in NexgenExtendedClientController.
 *                Due to the invalid format function, empty strings weren't sent correctly and were
 *                therefore not identifiable for the other machine (server). This caused the var index
 *                beeing erroneously recognized as the new var value on the server.
 *                Since the serverside set() function in NexgenSharedDataSyncManager also uses the
 *                invalid format functions, we implemented a fixed function in PluginController. The
 *                client side set() function can still be called safely without problems.
 *
 **************************************************************************************************/
simulated function exec_UPDATE_VAR(string args[10], int argCount) {
	local int varIndex;
	local string varName;
	local string varValue;
	local NexgenSharedDataContainer container;
	local int index;

	// Get arguments.
	if (argCount == 3) {
		varName = args[1];
		varValue = args[2];
	} else if (argCount == 4) {
		varName = args[1];
		varIndex = int(args[2]);
		varValue = args[3];
	} else {
		return;
	}

	if (role == ROLE_Authority) {
  	// Server side, call fixed set() function
  	PluginController(xControl).setFixed(args[0], varName, varValue, varIndex, self);
  } else {
  
    // Client Side
    dataSyncMgr.set(args[0], varName, varValue, varIndex, self);

    container = dataSyncMgr.getDataContainer(args[0]);

		// Signal event to client controllers.
		for (index = 0; index < client.clientCtrlCount; index++) {
			if (NexgenExtendedClientController(client.clientCtrl[index]) != none) {
				NexgenExtendedClientController(client.clientCtrl[index]).varChanged(container, varName, varIndex);
			}
		}

		// Signal event to GUI.
		client.mainWindow.mainPanel.varChanged(container, varName, varIndex);
  }
}
This will force the correct formating of empty strings. Use the known way of updating variables clientside (setVar()). If you want to update a variable serverside, make sure to use the new setFixed() function in the PluginController.


I hope this might help somebody at some point.
Reply With Quote
  #2  
Unread 4th September, 2013, 01:20 AM
The_Cowboy's Avatar
The_Cowboy The_Cowboy is offline
Unstoppable
 
Join Date: Jan 2009
Location: India
Posts: 189
Default

Can you explain this TCP transfer algorithm in brief? Thanks.
__________________
My GitHub repositories

Quote:
Originally Posted by Wormbo
You learn UnrealScript mainly by reading other people's code. Removing code without an important reason (download size reduction and lack of helpfulness are not important in that sense) is extremely antisocial IMHO. Sure, without the source code the package is smaller, but let's be honest - text compresses well, so the difference isn't really noticeable for redirected downloads.
Quote:
Originally Posted by Feralidragon
Trial and error is sometimes better than any tutorial, because we learn how it works for ourselfs, which kills any doubts about anything
Reply With Quote
  #3  
Unread 4th September, 2013, 02:10 AM
[rev]rato.skt's Avatar
[rev]rato.skt [rev]rato.skt is offline
Godlike
 
Join Date: Aug 2010
Location: Brazil
Posts: 370
Default

Sp0ngeb0b:

Cold you please, put some support to ascii code and write in bold mode in the next upgrade?
__________________
Brazilian Server:
ip server 1.utbr.org:2017
ip server 1.utbr.org:2222
ip server 1.utbr.org:8888
ip server 1.utbr.org:5555
Reply With Quote
  #4  
Unread 4th September, 2013, 04:12 PM
Sp0ngeb0b's Avatar
Sp0ngeb0b Sp0ngeb0b is offline
Godlike
 
Join Date: Sep 2008
Location: Germany
Posts: 488
Default

Quote:
Originally Posted by The_Cowboy View Post
Can you explain this TCP transfer algorithm in brief? Thanks.
Basically, the system expands the existing UT client <-> server communication (which is nothing but replication) with a framework which works similiar to the known TCP model. An output buffer (simple String datatype) is created on both ends, server and client side. Inside this string is the actual data which is to be sent to the other machine. The data is shaped in commands so the opposite side is able to understand. So, if you want to send a variable, you will first send a beginning Command, then the command for updating the variable (containing the var name and the new value) and then a closing command. A basic example could look like this:
START VAR VARNAME VARVALUE END
This command will be put inside the output buffer and waits until it is ready to be sent. It won't be sent in one piece though, but will be splitted into several marked packets. Defrost implemented a system which will recognize packet timeouts (= the packet was lost during the transfer) and therefore enables reliable data transfer as the packet will be send until it is received. If you want to check it out for yourself, all the magic happens inside the 'NexgenNetClientController' class, completed by the specific recvStr() function inside the subclasses ('NexgenExtendedClientController').


Quote:
Originally Posted by [rev]rato.skt View Post
Cold you please, put some support to ascii code and write in bold mode in the next upgrade?
Not exactly sure what you mean... Can you be more precise?
And for your information, I'm not updating the Nexgen core itself, since Defrost said he don't want anyone to do an offical update plus I don't really see a reason what needs to be fixed/added what can't be done inside a plugin.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT +1. The time now is 05:46 PM.


 

All pages are copyright The Unreal Admins Page.
You may not copy any pages without our express permission.