The Unreal Admins Page - Forums

The Unreal Admins Page - Forums (https://unrealadmin.org/forums/index.php)
-   General Chat (https://unrealadmin.org/forums/forumdisplay.php?f=177)
-   -   ACE: Status, Plans, and Test Build (https://unrealadmin.org/forums/showthread.php?t=31841)

AnthraX 13th November, 2016 10:55 PM

ACE: Status, Plans, and Test Build
 
*** STABILITY ***

This build should be relatively stable.

*** STATUS UPDATE/BORING TECHNICAL DETAILS ***

As you may have noticed, I've had some time to work on ACE recently (after a 5 year hiatus). Lots of things have changed so I want to start with a status update first. If you're not interested in the status update, then just scroll down to download the v1.0e package but install it AT YOUR OWN RISK!

You may remember that pre-0.9 versions of ACE were plagued with connection issues. People playing over WiFi or connections with packet loss were particularly impacted by this. This was not easy to fix because ACE's communication requirements don't fit nicely into Unreal Engine. To fix the problem, I designed a new networking layer that was separate from Unreal Engine. This new layer was stable in v0.9e, but that version had some new issues, which is why it was never publicly released.

The biggest problem of all was (and is) getting ACE to work well with the Linux server binaries. The Linux server binaries were built using some very old tools (gcc 2.95) and the only way to get ACE to work on Linux servers is to build it using those same tools. GCC 2.95 has a lot of known bugs and it lacks a lot of useful features (no sanitizers, no stable C++ support, ...), which makes it very hard to create a stable version of ACE for this platform. As a (hopefully temporary) solution, I've split ACE's server side into two components: the "Player Manager" and the "Connector". The Player Manager contains the bulk of the server-side logic. It runs as a separate program, which is why I can build it using modern tools. The Connector is a small library that takes care of the communication between the game server and the player manager. This component runs inside the game server process, which is why I have to build it using the old tools. This solution is ridiculously complex, but it has the advantage that it should be relatively easy to stabilize ACE's server side. The down side is that some game server providers might explicitly prevent you from running additional processes alongside the game server.

The current build, v1.0e, seems relatively stable. There are some known issues, though (see below).

*** PLANS ***

I want to merge the server components back together, but this will not be possible unless we get updated server binaries. I've asked Epic for this. Last time we spoke, they said it might be possible. More news about this soon (hopefully).

I also want to gradually open source all of the non-security critical components. Some components have already been open sourced (GitHub link available in the README). I've also given Higor (the creator of XC_Core/XC_Engine) access to the NPLoader source code.

*** CHANGES ***

A lot of things have changed since the last official release. The most notable changes are:

- New networking layer. This one works better for players with bad connections.
- ACE now stores its client-side settings in ACE.ini and remembers its settings even when the client gets updated.
- The file whitelist/blacklist is now a plain text file. This means that server admins can now allow new renderers themselves, without having to wait for me to update the official list.
- Support for Windows 10
- Support for TO 3.40 (untested! might require a filelist update)
- ACE can now check up to 255 UPackages
- Fixed a lot of false positive detections
- Added several new protection/detection routines
- ACE now supports checking of skin textures (you really have no excuse to still run AnthChecker now!)
- ... and of course, a whole slew of bug fixes and performance improvements

A full changelog will be made available at http://utgl.unrealadmin.org/ACE/changes.txt

*** INSTALLATION ***

Basic installation instructions can be found in the INSTALL.txt document which is included in the ACE package.

*** CONFIGURATION ***

An overview of configurable settings is available in the SETTINGS.txt document which is included in the ACE package.

*** KNOWN ISSUES ***

Installation Issues: Some other testers have reported issues installing the server files. Most of these are caused by outdated system libraries. Please note that you will need glibc 2.15 or later on Linux. Linux distributions released before 2012 might have older versions of glibc.

A second problem seems to occur for certain admins running x86_64 Linux servers. These admins might see the following error message:
Code:

Path to the PlayerManager: /data/ut-server/System/PlayerManager/ACEv10e_M
ACE: ERROR - Communication with PlayerManager failed - poll errno: 4 Interrupted system call
ACE: ERROR - PlayerManager Connection Failed
[ACEv10e]: ERROR: ACE could not spawn the PlayerManager.
[ACEv10e]: ERROR: ACE is now disabled.
ACE: ERROR - PlayerManager Connection Shutting Down!

This is most likely caused by a corrupted PlayerManager file. Please make sure that you upload the files in binary mode rather than text mode!


Timeouts during check spawn: Some people have reported timeouts that are caused by ACE using the wrong network interface. To debug these issues, please add the following lines to your server.ini:

Code:

[ACEv10e_S.ACENICHelper]
bDebug=1

Kick logs that say "Not kicking because bStrictSystemLibraryChecks is set to false": This is not technically an issue. ACE v1.0 has some very aggressive detection routines that detect a lot of legitimate external programs (e.g. Fraps, Display Drivers, ...). If you see this kick status, just ignore the log. The player will not be kicked.

*** DOWNLOAD LINK ***

http://utgl.unrealadmin.org/ACE/ACEv10e.zip

ShaiHulud 14th November, 2016 12:17 AM

That's a praise-worthy post, thanks for the update AnthraX, and your continued work on the project.

Chamberly 14th November, 2016 02:24 AM

Note: Please test with various IP_Drv customs to pinpoint the connection problem.

From Dev 1 (fragnet):
& XC_Engine doesn't seem to have a problem with it I think if we're using regular IP_Drv. (By using rollback/older versions that doesn't require it's own IP_Drv.)
Still need to test that older XC versions and see if the map switch is breaking from another source.

Server 2:
Just have crazy bad connection timeout problem with NFOServers. Idk why. No XC_Engine set up here.

Server 3 (DigitalOcean VDS):
Linux, running ok? I need to check logs. No XC Setup here as well.

McNamara 14th November, 2016 04:45 PM

Got this running on a normal weapons server.
will also trial a bunnytrack server when i get time ;)

Will post any problems i find!

SoNY_scarface 15th November, 2016 03:07 PM

I've been running the intial v1.0 on our pug server and it's been pretty stable.

I'll get round to updating with 1b and feedback.

isense 15th November, 2016 07:14 PM

Thank you very much for this update!
And thank you very very much for working on ACE/UT in general! _O_
Nice to have a central topic now!

Curious the memory problem (playermgr) from ACE1 got solved.
Will get back here asap!

Issue #1
I get kicked for my engine.dll
So did 3 of the 4 testmates. (kicked after 3-4minutes)


Fixed it by using the dll from a fresh install. (didnt get kicked for 20mins and 4 mapswitches)
Compared the files, and they had diff. sizes.
(is this file being written to by ACE or UT, and becomes bigger?)

Here's a link to the zip with the 2 different engine.dll's (onedrive)

Issue #2
Virtual memory loads up at clients :(
Result ->Playermgr crashes. ACE shuts down.
UT can use up till 1.5GB of RAM and gives big-screen errors.
Screenshots are coming.

possible issue 3
Same story as issue 1.
Only then with Core.dll (2 of 4 testmates).
Didnt swapped files yet.

This all happened in 3-4 games.
Iam gonna look up which server of linux gameservers.com use.
I hope to bring more hard proof. :)

Currently 3/4 players on the test server. (iDM US)
unreal://66.55.154.81:7777/

AnthraX 15th November, 2016 09:19 PM

Quote:

Originally Posted by isense (Post 171966)
Issue #1

Do you have log files for this one?

Quote:

Originally Posted by isense (Post 171966)
Issue #2

Is it the client or the server that shuts down? And is it UT.exe or ACEv10b_M.exe using the 1.5Gb of RAM?

Quote:

Originally Posted by isense (Post 171966)
possible issue 3

Again, log files would be tremendously useful here!

isense 15th November, 2016 10:06 PM

Quote:

Originally Posted by AnthraX (Post 171967)
log files
useful!

Sended you pm :)

Quote:

Originally Posted by SoNY_scarface (Post 171963)
I've been running the intial v1.0 on our pug server and it's been pretty stable.

My experience:
Stable. But after a game or 5-6 (sometimes 10-11) UT.exe took up all the RAM memory and makes the client crash.

Logfile ACE1:
Code:

[ACEv10]: ERROR: ACE could not spawn the PlayerManager.
[ACEv10]: ERROR: ACE is now disabled.
ACE: ERROR - PlayerManager Connection Shutting Down!said - PlayerManager Shutting Down

What is wise to mention (but so obviously for me today):
I run my servers with NewNet v0.9, maybe there are some conflicts ofc.

I cannot seem to reproduce the above errors with the current ace1b in the serverlog, allthough almost the same situations occur.
Need to do some testing without NewNet! :)

SoNY_scarface 15th November, 2016 11:10 PM

Quote:

Originally Posted by isense (Post 171968)



My experience:
Stable. But after a game or 5-6 (sometimes 10-11) UT.exe took up all the RAM memory and makes the client crash.

Typically, pug games are 1-3 games at most so maybe the environment is not the best to discover the bugs you mention. I can make the release on our pub servers, however the main issue is admin tools that have a dependency on ACEv08h and my lack of time to update those.

isense 15th November, 2016 11:41 PM



Quote:

however the main issue is admin tools that have a dependency on ACEv08h and my lack of time to update those.
asc/nexgen ?

i will get back with testresults without newnet :)

medor 16th November, 2016 12:13 AM

Remember this CommonACEkickfiles http://unrealtournament.99.free.fr/u...Ekickfiles.zip

AnthraX 16th November, 2016 12:36 AM

I've identified the bug that caused both the memory leak and the PlayerManager crash. Will post an updated package soon.

AnthraX 16th November, 2016 02:59 AM

Quote:

Originally Posted by isense (Post 171968)
Sended you pm :)

Reviewed your log files. It seems that your server is having trouble loading the BeaEngineFork.dll/.so file that came with ACE. Do you have any info about your server setup? Is it a Windows or Linux server? Are you using any special scripts to boot the server? From which folder do you start your server/script?

@stability issues: I think I've found the cause. I'm testing the fix right now and will hopefully be able to upload the v1.0c package tomorrow.

isense 16th November, 2016 07:22 PM

Quote:

Originally Posted by AnthraX (Post 171974)
Do you have any info about your server setup? Is it a Windows or Linux server? Are you using any special scripts to boot the server? From which folder do you start your server/script?

@stability issues: I think I've found the cause. I'm testing the fix right now and will hopefully be able to upload the v1.0c package tomorrow.

Thank you for checking!
BeaEnginefork.dll is the file huh.

Serverinfo:
It's hosted @gameservers.com.
It's a Linux server (dont know which distribution, or version)
Iam using ActorCLP to load the mutators onto the server.
Mutators are in the logfiles yea.

If you need to know which version and distro of linux, i need to look it up.

Chamberly 16th November, 2016 11:01 PM

Anth, I tried using the update and since I had to contact my host (Fragnet) to review and upload it, it was denied because of this they found...

"Unfortunately the following is reported while performing a virus scan: trojanspy.win32.skeeyah.a!rfn

Please review and send a new virus free file."

:( Btw, it's the .exe file.

AnthraX 17th November, 2016 01:13 AM

Looks like a badly tweaked heuristic virusscanner. These are the virustotal results:
https://virustotal.com/en/file/4c4de...is/1479341392/

Surprisingly, the v1.0b file does not trigger any alarms. I wonder what's up with that:
https://virustotal.com/en/file/a0c8a...is/1479341556/

I'll see if there's anything I can do about this ;)

-=CoN=-Strych9 17th November, 2016 05:10 PM

Thank you for your work on this Anthrax!

The only thing that I have found so far is the Join Logs are not working for me. I am not getting any logs on player joins.

Here are my settings:
bExternalLogJoins=True
JoinLogPath=../Logs/ACE/Players/
JoinLogPrefix=[ACEPLAYERS]

Thank you again!

McNamara 17th November, 2016 08:45 PM

Hi, having random ACE timeouts also here are some kick logs 2 of the screenshots are purple like an sshot cleaner?

[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Player Kick |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: PlayerName.....:
[ACEv10b]: PlayerIP.......:
[ACEv10b]: OS.............: Microsoft Windows 10 x64 (Version 10.0.14393)
[ACEv10b]: CPU............: Intel(R) Core(TM) i5-4460 CPU @ 3.20GHz
[ACEv10b]: CPUSpeed.......: 3192.599365 Mhz
[ACEv10b]: NICDesc........: Realtek PCIe GBE Family Controller
[ACEv10b]: MACHash1.......: 74490E74CE97062DC15CEB1E580236BC
[ACEv10b]: MACHash2.......: 74490E74CE97062DC15CEB1E580236BC
[ACEv10b]: HWID...........: A566B2BE49611D02177BF2D5B92075BC
[ACEv10b]: GameVersion....: 436
[ACEv10b]: Renderer.......: D3DDrv.D3DRenderDevice
[ACEv10b]: SoundDevice....: Galaxy.GalaxyAudioSubsystem
[ACEv10b]: CommandLine....:
[ACEv10b]: TimeStamp......: 16-10-2016 / 21:15:30
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Kick Reasons |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: BaseAddress....: 0x10300000
[ACEv10b]: LibraryName....: Engine.dll
[ACEv10b]: LibraryPath....: C:\Users\popie\Desktop\jmdu13\Unreal Tournament\UnrealTournament\System\Engine.dll
[ACEv10b]: LibrarySize....: 1789952 bytes
[ACEv10b]: LibraryHash....: 30E34C2A9E0EAB908C5DA6F322F4E2D7
[ACEv10b]: LibraryVer.....: UT v436 Std
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: KickReason.....: Hooked Function
[ACEv10b]: HookType.......: CODE
[ACEv10b]: HookAddress....: 0x10301BB3
[ACEv10b]: HookOffset.....: 0x0000
[ACEv10b]: HookFunction...: [email protected]@@[email protected]@P [email protected]@@Z
[ACEv10b]: HookInfo.......: ?1030103c-e91f83cd45/e93f510e00
[ACEv10b]: AdditionalInfo.: Mismatch @ 1030103c
[ACEv10b]: Expected:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: Found:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: MetaData.......: 55fd9360 -> C:\Users\popie\Desktop\jmdu13\Unreal Tournament\UnrealTournament\System\ACEv10b_C.DLL!? [email protected]@[email protected]+244d0
[ACEv10b]: 55fd8ee0 -> C:\Users\popie\Desktop\jmdu13\Unreal Tournament\UnrealTournament\System\ACEv10b_C.DLL!? [email protected]@[email protected]+24050
[ACEv10b]: +----------------------------------------------------------------------------+
[ACEv10b]: KickReason.....: Hooked Function
[ACEv10b]: HookType.......: CODE
[ACEv10b]: HookAddress....: 0x10302DC4
[ACEv10b]: HookOffset.....: 0x0000
[ACEv10b]: HookFunction...: [email protected]@@[email protected]@@Z
[ACEv10b]: HookInfo.......: ?10301bb3-e92873cd45/e9f8b60700
[ACEv10b]: AdditionalInfo.: Mismatch @ 10301bb3
[ACEv10b]: Expected:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: Found:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: MetaData.......: 55fd98c0 -> C:\Users\popie\Desktop\jmdu13\Unreal Tournament\UnrealTournament\System\ACEv10b_C.DLL!? [email protected]@[email protected]+24a30
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Screenshot Status |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: Filename.......: ../Shots/[ACE] - [PURE] [GAMBINO] Normal Weapons 1v1 Duel Server EU_2016.11.14.21.07.38_DM-Deck16][_3_Thatch.jpg
[ACEv10b]: Status.........: Success
[ACEv10b]: +------------------------------------------------------------------------------+

[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Player Kick |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: PlayerName.....:
[ACEv10b]: PlayerIP.......:
[ACEv10b]: OS.............: Microsoft Windows XP x86 (Version: 5.1.2600)
[ACEv10b]: CPU............: Intel(R) Atom(TM) CPU 230 @ 1.60GHz
[ACEv10b]: CPUSpeed.......: 1596.055420 Mhz
[ACEv10b]: NICDesc........: 802.11n Wireless PCI Express Card LAN Adapter - Miniport d'ordonnancement de paquets
[ACEv10b]: MACHash1.......: 37CAEFB1A7D4DE7FD4C47CC39D35793B
[ACEv10b]: MACHash2.......: 8EEE883DA327876D6711F5FC1A2FAAFA
[ACEv10b]: HWID...........: 39F2E354702422672D5D8DEDC363DF6F
[ACEv10b]: GameVersion....: 436
[ACEv10b]: Renderer.......: D3DDrv.D3DRenderDevice
[ACEv10b]: SoundDevice....: Galaxy.GalaxyAudioSubsystem
[ACEv10b]: CommandLine....:
[ACEv10b]: TimeStamp......: 16-10-2016 / 21:16:15
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Kick Reasons |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: BaseAddress....: 0x10100000
[ACEv10b]: LibraryName....: Core.dll
[ACEv10b]: LibraryPath....: C:\Program Files\GOG.com\Unreal Tournament GOTY\system\Core.dll
[ACEv10b]: LibrarySize....: 786432 bytes
[ACEv10b]: LibraryHash....: CCF104341C7452B06295D421167DBA95
[ACEv10b]: LibraryVer.....: UT v436 Std
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: KickReason.....: Hooked Function
[ACEv10b]: HookType.......: CODE
[ACEv10b]: HookAddress....: 0x101030AD
[ACEv10b]: HookOffset.....: 0x0000
[ACEv10b]: HookFunction...: [email protected]@@[email protected]@[email protected]
[ACEv10b]: HookInfo.......: ?101030ad-e9ee69b6f9/e92eb70300
[ACEv10b]: AdditionalInfo.: Mismatch @ 101030ad
[ACEv10b]: Expected:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: Found:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: BaseAddress....: 0x10300000
[ACEv10b]: LibraryName....: Engine.dll
[ACEv10b]: LibraryPath....: C:\Program Files\GOG.com\Unreal Tournament GOTY\system\Engine.dll
[ACEv10b]: LibrarySize....: 1789952 bytes
[ACEv10b]: LibraryHash....: 30E34C2A9E0EAB908C5DA6F322F4E2D7
[ACEv10b]: LibraryVer.....: UT v436 Std
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: KickReason.....: Hooked Function
[ACEv10b]: HookType.......: CODE
[ACEv10b]: HookAddress....: 0x10302DC4
[ACEv10b]: HookOffset.....: 0x0000
[ACEv10b]: HookFunction...: [email protected]@@[email protected]@[email protected]
[ACEv10b]: HookInfo.......: ?10302dc4-e9676f96f9/e9c7c10600
[ACEv10b]: AdditionalInfo.: Mismatch @ 10302dc4
[ACEv10b]: Expected:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: Found:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Screenshot Status |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: Filename.......: ../Shots/[ACE] - [PURE] [GAMBINO] Normal Weapons 1v1 Duel Server EU_2016.11.14.21.07.38_DM-Deck16][_6_FearMom4Sum.jpg
[ACEv10b]: Status.........: Success
[ACEv10b]: +------------------------------------------------------------------------------+

[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Player Kick |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: PlayerName.....:
[ACEv10b]: PlayerIP.......:
[ACEv10b]: OS.............: Microsoft Windows 7/2008 R2 x86 (Version: 6.1.7601)
[ACEv10b]: CPU............: AMD Athlon(tm) Dual Core Processor 5400B
[ACEv10b]: CPUSpeed.......: 2721.555664 Mhz
[ACEv10b]: NICDesc........: NVIDIA nForce-Netzwerkcontroller
[ACEv10b]: MACHash1.......: C2C6AFD8A7554634038011E553E48DB8
[ACEv10b]: MACHash2.......: 2C94C86F4EAE03E1E319671E864BB33A
[ACEv10b]: HWID...........: 60664158D7CD739205595B0405F37D03
[ACEv10b]: GameVersion....: 436
[ACEv10b]: Renderer.......: D3DDrv.D3DRenderDevice
[ACEv10b]: SoundDevice....: Galaxy.GalaxyAudioSubsystem
[ACEv10b]: CommandLine....:
[ACEv10b]: TimeStamp......: 17-10-2016 / 17:38:04
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: | Kick Reasons |
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: BaseAddress....: 0x10100000
[ACEv10b]: LibraryName....: Core.dll
[ACEv10b]: LibraryPath....: C:\UnrealTournament\System\Core.dll
[ACEv10b]: LibrarySize....: 786432 bytes
[ACEv10b]: LibraryHash....: CCF104341C7452B06295D421167DBA95
[ACEv10b]: LibraryVer.....: UT v436 Std
[ACEv10b]: +------------------------------------------------------------------------------+
[ACEv10b]: KickReason.....: Hooked Function
[ACEv10b]: HookType.......: CODE
[ACEv10b]: HookAddress....: 0x10101B04
[ACEv10b]: HookOffset.....: 0x0000
[ACEv10b]: HookFunction...: [email protected]@@[email protected]@QAXPAVUFunct [email protected]@@Z
[ACEv10b]: HookInfo.......: ?101013f2-e94989a860/e919d10300
[ACEv10b]: AdditionalInfo.: Mismatch @ 101013f2
[ACEv10b]: Expected:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: Found:
[ACEv10b]: Unknown Opcode -- FIXME anth
[ACEv10b]: MetaData.......: 70b89d40 -> C:\UnrealTournament\System\ACEv10b_C.DLL!??0AACENa [email protected]@[email protected]+24d40
[ACEv10b]: +------------------------------------------------------------------------------+

AnthraX 17th November, 2016 11:49 PM

Could you try moving BeaEngineFork.dll and BeaEngineFork.so into the System/PlayerManager folder?

AnthraX 17th November, 2016 11:59 PM

By the way, when reporting the "Unknown opcode -- FIXME anth" problem, please include details about your server setup. OS, 32 or 64bit architecture, and the script you use to start your server would be very nice to see.


All times are GMT +1. The time now is 06:24 PM.

 


All pages are copyright The Unreal Admins Page.
You may not copy any pages without our express permission.