You are an unregistered user, you can register here
Navigation

Information

Site

Donations
If you wish to make a donation you can by clicking the image below.


BeyondUnreal News

 
Go Back   The Unreal Admins Page > Forums > Unreal Admins > Unreal Tournament 3 > UT3 Server - General Chat

Reply
Thread Tools Display Modes
  #1  
Unread 25th October, 2007, 01:31 PM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default Global Stats (API?)

The biggest news today were the Global stats hosted by Enemy Down, are they hosting officially the Stats or do they just use an interface to another Server from Epic or more probable at GameSpy?

Probably they use the Interface also used by the Leader-Board in UT3 right? anyone has already some information or do I need to start up wireshark?

the ED site:
http://www.enemydown.co.uk/game.php?gameselect=UT3

Too overloaded this site...
Reply With Quote
  #2  
Unread 26th October, 2007, 12:21 AM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Post

I started a bit with wireshark, if you don't want to start from scratch you can read my analysis I did here: (its probably more than you need in the end) http://download.apophis.ch/UT3/globalstats/debug.txt

I cannot make my own calls yet, however I see what happens, the Statistics get queried from ut3pcd.sake.gamespy.com but it looks like you have to login first to get two things, a "Ticket" and a secret Key. You get the ticket when you login in the game, I have no clue where the secret key comes from. The XML description only mentions the ticket, maybe this is enough! I'll try later.

If you have that information, you can do Soap Calls to the StorageServer here: http://ut3pcd.sake.gamespy.com/SakeS...ageServer.asmx

The answers are quite boring, you get big xml answers with plenty of "RecordValues" with different type (string, int, binary) and not much description. Its plaintext and you can compare with your stats so it will be easy to find out when a Query gets an answer.

ffs is there no documentation from gamespy about this?
Reply With Quote
  #3  
Unread 26th November, 2007, 02:40 AM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default Query Playerstats API

After some time a little update here, I did some research and thanks to Cruz I got the necessary information I needed to program an example client which queries directly Gamespy to get the information of the players.

The project is written in C# and works under .NET and Mono, the fast prototyping, and automation of the Writing of the proxy classes for the webservices made it very nice to develop.

This is quite sure the same method enemydown.co.uk uses, I tried to contact them, but I was faster with getting the info myself, and I want to make it easier for other interested people.

How it works:

we log in at gpcm.gamespy.com and get a valid user ticket (thats why we need a valid login)
then with this ticket we query a webservice at ut3pc.sake.gamespy.com, as described in the previous post. In principle this is all what we need.

The parameters needed can be obtained via WireShark, a little list is added to the client as well. For experimenting this should be enough.

I'm working on a web interface (here) and if anybody is interested I will release a console client.

Thanks to Cruz and the author of gslist

PS: some admin can move this into the main area, since its not demo related anymore. TIA

Here you can download the Source:
To make it work: add a valid Account in the beginning of the Main Procedure. The Code is GPL.
gameSpyWebService.zip

PS: consult the Source Code I tried to document it understandable.

Last edited by apophis.ch : 26th November, 2007 at 02:42 AM.
Reply With Quote
  #4  
Unread 30th November, 2007, 07:14 PM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Talking Job finished

Ok I derived a console Program, works on .NET and Mono, tested on Windows Vista, XP SP 2 and Linux (x86, debian)

here you can download it:
its gpl, and I hope you have fun, here is the help for the Program (you get it with -h) ... I use it to get the stats on http://prx.apophis.ch/player.php which is a 60 minutes project so very rudimentary, but you see what you can do.

Code:
Usage:
ut3player.exe [-g <username> <pass>][Options]

Options:
  -c  --count            Only get the count of records
  -d  <player>           Part or Complete Name of player
  -e  <filter>           Filter, containing fields (-f)
  -f  --fields <param>   List field/coloumnames
               all         param All fields
               main        param Main Fields (3)
               general     param General fields
               weapon      param Weapon fields
               vehicle     param Vehicle fields
               turret      param Turret fields
  -g  <username> <pass>  All queries to the GS Database need a valid login
  -h  --help             This help
  -l  --limit <num>      Get at most <num> records (10)
  -n, --surround <num>   Get surrounding entries (only use with -u)
  -o, --offset <num>     Order by column
  -p,                    easy parsing, not for humans, no markup
  -q, --query  f1 f2 f3  Query Coloumns f1,f2,f3, for coloum names use -f
      --q-all              Query All fields
      --q-main             Query only 3 values per Records (*)
      --q-general          Query General Stats
      --q-weapon           Query Weapon Stats
      --q-vehicle          Vehicle fields
      --q-turret           Turret fields
  -s, --sort <col> [a/d] Sort by column [ascending/descending] (ELO desc)
  -u, --user <uid> ...   Get data of certain user
  -v, --verbose          Debug information
      --version          Version information

Examples:
ut3player.exe -g  <username> <pass> -d player
Lists all players with Nick containing 'player', and list their complete name, ownerid and row values

ut3player.exe -g  <username> <pass> -f-all -u 1337
Lists all attributes from the user with id 1337

ut3player.exe -g  <username> <pass> -q Nick -l 3 -s Pure_VehiclesDM_VEHICLEKILL_UTVEHICLE_GOLIATH_CONTENT d
This will give the three nicks which used the tank the most.
Console Program:
http://download.apophis.ch/UT3/ut3playerStats.exe

Source Program:
http://download.apophis.ch/UT3/gameSpyWebService.zip

So for me its finished
Reply With Quote
  #5  
Unread 5th January, 2008, 07:08 PM
dirrty[starbuck]'s Avatar
dirrty[starbuck] dirrty[starbuck] is offline
Killing Spree
 
Join Date: Jan 2006
Posts: 18
Default

Hey apophis.ch... nice work on this, you know your stuff.

Hoping you can help me get this working on my end. I downloaded the source, compiled without issue. Ran it supplying my username and password (that I use to login to the game) - the return was:
Quote:
The uniquenick provided is incorrect.
Webservice returned 'LoginTicketInvalid' instead of success.
Reading your posts and digging around a bit I realized I needed to supply my own secretKey and loginTicket to go with my username. I loaded up Wireshark and obtained both, here is part of what I pulled from Wireshark:
Code:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:ns1="http://gamespy.net/sake"><SOAP-ENV:Body><ns1:SearchForRecords><ns1:gameid>1727</ns1:gameid><ns1:secretKey>nT2Mtz</ns1:secretKey><ns1:loginTicket>nxzZEA6Sb7s[k[nA04kOMi__</ns1:loginTicket><ns1:tableid>PlayerStats_v2</ns1:tableid>
I updated secretKey(although this is the same as yours) and loginTicket in all locations in QueryCounts.xml and QueryRecords.xml in the source code files. Recompiled and ran - same result
Using Wireshark again I analyzed the soap packets when I ran your program. Here's what it's sending:
Code:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><GetRecordCount xmlns="http://gamespy.net/sake"><gameid>1727</gameid><loginTicket /><tableid>PlayerStats_v2</tableid><filter>NUM_Pure_PlayerDM &gt; 0</filter><cacheFlag>false</cacheFlag></GetRecordCount></soap:Body></soap:Envelope>
Here's what I'm getting back:
Code:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><soap:Body><GetRecordCountResponse xmlns="http://gamespy.net/sake"><GetRecordCountResult>LoginTicketInvalid</GetRecordCountResult><count>0</count></GetRecordCountResponse></soap:Body></soap:Envelope>
Clearly it's not actually sending the secretKey and loginTicket...

Do you have any idea what I'm missing? Am I going about it the wrong way? I appreciate any help - thanks.
__________________
http://forum.dirrtyclan.com
http://blogs.dirrtyclan.com/starbuck
UT3 Server: ut3.dirrtyclan.com:7777
Reply With Quote
  #6  
Unread 5th January, 2008, 10:01 PM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default

Hi,

The provided XML files in the source are not used in the program. They have been crucial for my development, and I included them as a reference for anyone who wants to develop anything with it.

If you want to check if its working you can try the precompiled version and try if it works. For myself it doesn't work with my ingame-id but it works with my regular GameSpy ID which was registered with GameSpy and not within ut3. But I am uncertain why its like that, it should work with my other ID as well.

You don't need to change anything, if it compiles it should work, and its more likely that your user ID and password are wrong.

the secret-key and loggin ticket will be calculated and/or received from the GameSpy Server by the program on runtime with your User ID and password. (the login ticket is only valid for some hours, or until you logout)

PS: yes its the error message you get if username or password is incorrect, and it looks like I have an uncached Null Pointer Exception there sorry! missed that one.

Last edited by apophis.ch : 5th January, 2008 at 10:05 PM.
Reply With Quote
  #7  
Unread 5th January, 2008, 10:46 PM
dirrty[starbuck]'s Avatar
dirrty[starbuck] dirrty[starbuck] is offline
Killing Spree
 
Join Date: Jan 2006
Posts: 18
Default

Thanks for the quick reply! After I posted I walked through the code (which I probably should have done before posting)... and realized the xml files are not used, and that your code does everything.

It was exactly as you said, my ID and password were wrong... that is, it wouldn't accept my UT3 login info. I signed up a new account at login.gamespy.com and within about 5 minutes it was active and worked with your app.

Any chance you've written a C# version of the UT3 Server Query? All my attempts have failed... I got as far as obtaining the challenge string from the first query I send... but after that my code fails (gamespy never replies to the second query I send).

Thanks again for your help.
__________________
http://forum.dirrtyclan.com
http://blogs.dirrtyclan.com/starbuck
UT3 Server: ut3.dirrtyclan.com:7777
Reply With Quote
  #8  
Unread 6th January, 2008, 12:15 AM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default

No I have written it directly in php and didn't needed it for C#, but it should be even simpler in C# and maybe I can help you with whatever you want to do.

The only thing you probably do wrong is the calculation of the magic number, you can check here if you do it correct, just enter a challenge number (you get from the server) and it calculates you the magic number which is always 4bytes.

and take notice that the server sends you a number encoded in ASCII (upto 10 characters including the minus sign -) but the magicnumber is binary and therefore 4 bytes you see it easily in Hex.

you can check if your implementation is right here: enter a challenge (in the URL) and see the magic number in hex.
http://download.apophis.ch/UT3/check...?chlng=1265345
PS: and you consulted the Wiki? -> http://wiki.unrealadmin.org/UT3_query_protocol

Last edited by apophis.ch : 6th January, 2008 at 12:18 AM.
Reply With Quote
  #9  
Unread 6th January, 2008, 07:40 AM
dirrty[starbuck]'s Avatar
dirrty[starbuck] dirrty[starbuck] is offline
Killing Spree
 
Join Date: Jan 2006
Posts: 18
Default

I was so close... that was just the bit of help I needed... having a point of reference for what that magic number should be.

Thanks!
__________________
http://forum.dirrtyclan.com
http://blogs.dirrtyclan.com/starbuck
UT3 Server: ut3.dirrtyclan.com:7777
Reply With Quote
  #10  
Unread 8th March, 2008, 02:56 PM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default Update

I found a question in the Epic Forum (Link) for Querying non-pure Server statistics, obviously this is only possible ingame at the moment, not even Enemy Down is supporting it.

It was a very small hack into my Querying Program so I updated the software to support non-pure Query Flags aswell.

Now you can query with additionally with --custom and can use non "Pure" Query columns aswell. I added two simple examples into the help. If you have any questions dont hesitate to ask.

I also updated the example webpage which is and will stay as a rudimentary example, I dont have the urge to make it to something more useful than it is right now:

Code:
Usage:
ut3playerStats.exe [-g <username> <pass>][Options]

Options:
  -c  --count            Only get the count of records
      --custom           Non-pure Server stats
  -d  <player>           Part or Complete Name of player
  -e  <filter>           Filter, containing fields (-f)
  -f  --fields <param>   List field/coloumnames
               all         param All fields
               main        param Main Fields (3)
               general     param General fields
               weapon      param Weapon fields
               vehicle     param Vehicle fields
               turret      param Turret fields
  -g  <username> <pass>  All queries to the GS Database need a valid login
  -h  --help             This help
  -l  --limit <num>      Get at most <num> records (10)
  -n, --surround <num>   Get surrounding entries (only use with -u)
  -o, --offset <num>     Order by column
  -p,                    easy parsing, not for humans, no markup
      --pure             Pure Server stats (*)
  -q, --query  f1 f2 f3  Query Coloumns f1,f2,f3, for coloum names use -f
      --q-all              Query All fields
      --q-main             Query only 3 values per Records (*)
      --q-general          Query General Stats
      --q-weapon           Query Weapon Stats
      --q-vehicle          Vehicle fields
      --q-turret           Turret fields
  -s, --sort <col> [a/d] Sort by column [ascending/descending] (ELO desc)
  -u, --user <uid> ...   Get data of certain user
  -v, --verbose          Debug information
      --version          Version information

Examples:
ut3playerStats.exe -g  <username> <pass> -d player
Lists all players with Nick containing 'player', and list their complete name, ownerid and row values

ut3playerStats.exe -g  <username> <pass> -f-all -u 1337
Lists all attributes from the user with id 1337

ut3playerStats.exe -g  <username> <pass> -q Nick -l 3 -s Pure_VehiclesDM_VEHICLE KILL_UTVEHICLE_GOLIATH_CONTENT d
This will give the three nicks which used the tank the most.

ut3playerStats.exe -g  <username> <pass> --custom
Lists The Top 10 Players on non Pure servers

ut3playerStats.exe -g  <username> <pass> --custom -q Nick PlayerDM_ELO Pure_PlayerDM_ELO
Lists The Top 10 Players on non Pure servers and compares there Custom Elo Ranking to the Pure Elo Ranking
Console Program:
http://download.apophis.ch/UT3/ut3playerStats.exe

Source Program:
http://download.apophis.ch/UT3/gameSpyWebService.zip

Example Webpage:
http://prx.apophis.ch/player.php

Last edited by apophis.ch : 9th April, 2008 at 04:05 PM. Reason: Make help readable
Reply With Quote
  #11  
Unread 8th April, 2008, 09:13 PM
mandelbug mandelbug is offline
Forum Newcomer
 
Join Date: Mar 2008
Posts: 8
Default

This looks fantastic and I am hoping that I can add this to my site. I am using vb.net at the mo so it will be fun to get it working :-)

When its all up and running, due credit will be given
__________________

Reply With Quote
  #12  
Unread 9th April, 2008, 09:49 PM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default

You're Welcome, if you want to use it from VB or from aspx best would be probably to compile it as a dll and use it via an interface. If you need help to do that in C# I can help you there.

Right now its quite some Work to make a usable Api from it.

Hope I can help

ut3.de uses it for the top 10... and BTW you can write quite complex queries with it, once I listed all players with a 10:1 or more Ratio in kills ;D so Cheater spotting could be a hobby with it
Reply With Quote
  #13  
Unread 10th April, 2008, 06:17 AM
mandelbug mandelbug is offline
Forum Newcomer
 
Join Date: Mar 2008
Posts: 8
Default

At the moment I am thinking of using it as a windows service to get the info I want and then dump that to a SQL database every half hour to an hour. My reasoning for this is using ut3.de and your demo front end it was not exactly quick (putting this down to gamespy) so I will essentially cache the info however it will refresh faster than ED
__________________

Reply With Quote
  #14  
Unread 13th December, 2008, 04:37 PM
apophis.ch apophis.ch is offline
Killing Spree
 
Join Date: Oct 2007
Posts: 41
Default

Gamespy Changed their Login Policie therefore the Query was failing for some time, but nobody cared so it took me a while to notice it

I updated the ut3 Global Player Stats Projects Page and hope I fixed all the remaining issues.

Please Notice: You need to use the UT3 Account now, not an Account from login.gamespy.com. I know it was confusing before and now it should be more logical!

The Links remain the same!

Console Program:
http://download.apophis.ch/UT3/ut3playerStats.exe

Source Program:

http://download.apophis.ch/UT3/gameSpyWebService.zip

greets
Apophis
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT +1. The time now is 03:27 PM.


 

All pages are copyright The Unreal Admins Page.
You may not copy any pages without our express permission.