Author Topic: Unleashing a HTTP-based API for the Soldat Lobby  (Read 3481 times)

0 Members and 1 Guest are viewing this topic.

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Unleashing a HTTP-based API for the Soldat Lobby
« on: April 10, 2017, 05:37:08 am »
Update docs here: https://wiki.soldat.pl/index.php/Lobby_HTTP_API

For almost a decade, the only way of querying the Soldat Lobby to see what servers are online has been to use an aging, proprietary, protocol over sockets which is inconvenient if you want to build an app to make use of lobby data.

To make building 3rd party apps or other tools for Soldat easier, I just implemented support for getting and filtering the currently online Soldat servers over JSON+HTTP, slightly similar to the API for KAG.

List of all servers: http://api.soldat.pl/v0/servers

Example of some queries with the currently-supported filters:

http://api.soldat.pl/v0/servers?empty=no&bots=no&full=no&version=1.7.1
http://api.soldat.pl/v0/servers?empty=yes

Info on one specific server: http://api.soldat.pl/v0/server/88.198.8.206/23020
Players for that one server: http://api.soldat.pl/v0/server/88.198.8.206/23020/players

Note: the json returned is compacted for speed reasons. To view it easier, install a browser extension for pretty printing json (there are many) or from the command line, run a command like this:

curl -s 'http://api.soldat.pl/v0/servers' | python -m json.tool

I look forward to seeing people build cool things using this new API. Please provide any suggestions or feature requests in this thread.  :D
« Last Edit: September 01, 2017, 03:11:19 am by jrgp »
Playing Soldat and active on these forums from 2004 - present

Offline Shoozza

  • Soldat Developer
  • Veteran
  • ******
  • Posts: 1599
  • Soldat's Babysitter
    • Website
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #1 on: April 10, 2017, 11:09:45 am »
Sounds like the soldat.pl lobby website could possibly use this with clientside javascript.

Great job  8)
Rules
Tools: ARSSE - SARS - SRB - chatMod

Offline SyavX

  • Soldat Beta Team
  • Camper
  • ******
  • Posts: 335
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #2 on: April 10, 2017, 11:56:35 am »
GJ

I suggest to rename some fields:
"OsType"    "OSType"
"Ip"        "IP"
"Wm"        "WM"
"Be"        "AC"
(as it's called Anti-Cheat in Soldat client)

Offline Furai

  • Administrator
  • Veteran
  • *****
  • Posts: 1887
    • TransHuman Design
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #3 on: April 10, 2017, 12:24:27 pm »
Nice. thanks for working on this! :)
"My senses are so powerful that I can hear the blood pumping through your veins."

Offline machina

  • Soldat Beta Team
  • Veteran
  • ******
  • Posts: 1014
  • The world isn't nice. Why should I be...?
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #4 on: April 10, 2017, 02:28:00 pm »
Cool! :D

How about serving in-game players' nicknames? It was possible over TCPbut it worked pretty randomly (what actually discouraged me to implement anything with this thing).

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #5 on: April 10, 2017, 07:37:17 pm »
Sounds like the soldat.pl lobby website could possibly use this with clientside javascript.

Definitely could.

GJ

I suggest to rename some fields:
"OsType"    "OSType"
"Ip"        "IP"
"Wm"        "WM"
"Be"        "AC"
(as it's called Anti-Cheat in Soldat client)

That makes sense.

How about serving in-game players' nicknames? It was possible over TCPbut it worked pretty randomly (what actually discouraged me to implement anything with this thing).

The soldat servers themselves don't provide the lobby with this information. You can reliabily get this information using REFRESH(X) over rcon, but that of course requires logging into each server.

Alternatively, I think it might be possible to use the tcp asset download port to grab logs/gamestat.txt without needing to authenticate. If this works then it would be possible to make this API interface provide players; however, this would only be on the individual server endpoints rather than the full list.
Playing Soldat and active on these forums from 2004 - present

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #6 on: April 10, 2017, 07:51:01 pm »
Alternatively, I think it might be possible to use the tcp asset download port to grab logs/gamestat.txt without needing to authenticate. If this works then it would be possible to make this API interface provide players; however, this would only be on the individual server endpoints rather than the full list.

Yep, this works for grabbing gamestat.txt: https://wiki.soldat.pl/index.php/Download_thread

I'll see if I can implement support for that into this API on the server endpoints.
Playing Soldat and active on these forums from 2004 - present

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #7 on: April 11, 2017, 05:13:48 am »
I'll see if I can implement support for that into this API on the server endpoints.

Just did it. If you tack on /players to the end of an individual server URL, it will attempt to download gamestat.txt from that server and show the parsed out players.



(From http://api.soldat.pl/v0/server/108.61.204.90/23075/players )
Playing Soldat and active on these forums from 2004 - present

Offline machina

  • Soldat Beta Team
  • Veteran
  • ******
  • Posts: 1014
  • The world isn't nice. Why should I be...?
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #8 on: April 11, 2017, 03:51:34 pm »
Alternatively, I think it might be possible to use the tcp asset download port to grab logs/gamestat.txt without needing to authenticate. If this works then it would be possible to make this API interface provide players; however, this would only be on the individual server endpoints rather than the full list.

Yep, this works for grabbing gamestat.txt: https://wiki.soldat.pl/index.php/Download_thread

I'll see if I can implement support for that into this API on the server endpoints.
Yea, I meant precisely this thing.

Just did it. If you tack on /players to the end of an individual server URL, it will attempt to download gamestat.txt from that server and show the parsed out players.
Great work, man! Now I have to think what to do with this API :D Most desirably a mobile app.

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #9 on: April 11, 2017, 05:19:08 pm »
Great work, man! Now I have to think what to do with this API :D Most desirably a mobile app.

Thanks! I really would love to see you build something with this API, even if it's something small.
Playing Soldat and active on these forums from 2004 - present

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #10 on: April 18, 2017, 07:56:32 pm »
Updates:

- I've renamed these keys:
OsType -> OS
Ip -> IP
Wm -> WM
Be -> AC


- OS is now linux, mac, or windows
- GameStyle is now the uppercase representation of the game mode rather than the number

- Added more filters. Each of these are optionally specified via URL params. Pipe (|) denotes allowed options. 3 dots denotes options that you'd expect.

?os=linux|mac|windows
?gamestyle=DM|PM|TM|CTF|RM|INF|HTF
?country=US|DE|...
?version=1.7.1|1.6.1|...
?empty=yes|no
?full=yes|no
?bots=yes|no
?private=yes|no
?realistic=yes|no
?survival=yes|no
?advanced=yes|no


For example:
https://api.soldat.pl/v0/servers?os=linux&country=US&full=no&empty=no&version=1.7.1&gamestyle=DM
« Last Edit: April 18, 2017, 07:58:31 pm by jrgp »
Playing Soldat and active on these forums from 2004 - present

Offline machina

  • Soldat Beta Team
  • Veteran
  • ******
  • Posts: 1014
  • The world isn't nice. Why should I be...?
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #11 on: April 19, 2017, 05:11:19 am »
Nice to see an ongoing development. Good job :D

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #12 on: April 19, 2017, 05:23:55 am »
Nice to see an ongoing development. Good job :D

:)

I just rewrote https://soldat.pl/en/lobby to hit this API directly via ajax, rather than the old implementation which was a php shim against the old protocol.

Looking at the html source for that page (it's not obfuscated), and the accompanying javascript file, should make it obvious how to write a similar lobby browser via ajax.
Playing Soldat and active on these forums from 2004 - present

Offline machina

  • Soldat Beta Team
  • Veteran
  • ******
  • Posts: 1014
  • The world isn't nice. Why should I be...?
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #13 on: April 19, 2017, 06:21:28 am »
I just rewrote https://soldat.pl/en/lobby to hit this API directly via ajax, rather than the old implementation which was a php shim against the old protocol.
Once you got there you could set default ordering by amount of players (descending obviously). It would make more sense in my opinon to get servers where people actually play on the top of the list.

This should also be done in the Soldat GUI (game) I think. Ordering by players amount - not ping. It's not within your cup of tea though.


A bit off topic here right now:

I see that you've (just?) removed top bar with language switching as I proposed to do so long time ago because of unsupported translations etc. Note that main domain i.e.: https://soldat.pl/ points to https://soldat.pl/pl/ (where there's still that messy frontpage etc.) instead of https://soldat.pl/en/. Also, the background picture should be elevated.

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #14 on: April 19, 2017, 11:10:41 am »
I just rewrote https://soldat.pl/en/lobby to hit this API directly via ajax, rather than the old implementation which was a php shim against the old protocol.
Once you got there you could set default ordering by amount of players (descending obviously). It would make more sense in my opinon to get servers where people actually play on the top of the list.

This should also be done in the Soldat GUI (game) I think. Ordering by players amount - not ping. It's not within your cup of tea though.


A bit off topic here right now:

I see that you've (just?) removed top bar with language switching as I proposed to do so long time ago because of unsupported translations etc. Note that main domain i.e.: https://soldat.pl/ points to https://soldat.pl/pl/ (where there's still that messy frontpage etc.) instead of https://soldat.pl/en/. Also, the background picture should be elevated.

Hmm soldat.pl shouldn't be redirecting directly to soldat.pl/pl/:

$ curl -I https://soldat.pl
HTTP/1.1 302 Found
Date: Wed, 19 Apr 2017 16:07:32 GMT
Location: https://soldat.pl/en/


I did just remove the language switching. Elevating the entire site makes sense and I'll try to get to that too.

Sorting by players descending also makes sense for the site. Not sure about ingame because that way you might have really high ping servers at the top rather than ones that you'll be able to play on comfortably.
Playing Soldat and active on these forums from 2004 - present

Offline machina

  • Soldat Beta Team
  • Veteran
  • ******
  • Posts: 1014
  • The world isn't nice. Why should I be...?
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #15 on: April 19, 2017, 02:35:36 pm »
Hmm soldat.pl shouldn't be redirecting directly to soldat.pl/pl/:

$ curl -I https://soldat.pl
HTTP/1.1 302 Found
Date: Wed, 19 Apr 2017 16:07:32 GMT
Location: https://soldat.pl/en/

I tested that using a browser (FF precisely) and it doesn't behave this way i.e. redirecting to /en. That's weird though as I had tested that also on Postman this time and it returned an English version. Maybe these are some browser cached language preferences but really dunno why ???

Sorting by players descending also makes sense for the site. Not sure about ingame because that way you might have really high ping servers at the top rather than ones that you'll be able to play on comfortably.
Well, if sorted by ping than you most likely get plenty of empty servers on the top.

Recently, players living in "weirder" locations of the globe have become a problem. They usually don't have servers located close enough to their place where players would attend frequently enough. This makes them join popular European servers but for the price of lags and ruining fun of other players. On the other hand I don't necessirly want to banish them for that as it is not entirely their fault.

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #16 on: September 01, 2017, 02:54:42 am »
Just added a wiki doc: https://wiki.soldat.pl/index.php/Lobby_HTTP_API

Also made the frontpage have a link to that instead of just showing 404: http://api.soldat.pl/

Also made the /players endpoints support Windows servers instead of just Linux ones. (line endings)
« Last Edit: September 01, 2017, 03:09:14 am by jrgp »
Playing Soldat and active on these forums from 2004 - present

Offline pewpew

  • Major(1)
  • Posts: 2
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #17 on: November 04, 2017, 08:45:58 pm »
I'm getting a 403 forbidden code when I try to get a player list (with /players in url). :'(

Is this just for me or is it a problem with the API? Or maybe it's being worked on?  ???

Offline machina

  • Soldat Beta Team
  • Veteran
  • ******
  • Posts: 1014
  • The world isn't nice. Why should I be...?
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #18 on: November 05, 2017, 03:06:06 pm »
I'm getting a 403 forbidden code when I try to get a player list (with /players in url). :'(

Is this just for me or is it a problem with the API? Or maybe it's being worked on?  ???
I've just checked and I also encounter this problem. E.g.: http://api.soldat.pl/v0/server/45.76.84.88/23075/players -> 403

Offline jrgp

  • Administrator
  • Flamebow Warrior
  • *****
  • Posts: 4959
Re: Unleashing a HTTP-based API for the Soldat Lobby
« Reply #19 on: November 05, 2017, 03:57:02 pm »
I'm getting a 403 forbidden code when I try to get a player list (with /players in url). :'(

Is this just for me or is it a problem with the API? Or maybe it's being worked on?  ???

I'm getting a 403 forbidden code when I try to get a player list (with /players in url). :'(

Is this just for me or is it a problem with the API? Or maybe it's being worked on?  ???
I've just checked and I also encounter this problem. E.g.: http://api.soldat.pl/v0/server/45.76.84.88/23075/players -> 403

Hey,

I blocked that endpoint due to it being a potential attack vector during the Soldat server attacks over the past few weeks. Technically it should not have been—as it will only hit the same soldat server once every 30 seconds due to caching—but I disabled it anyway.

I just removed the block so now the players endpoint will work. I will add the block back if it gets abused again.
Playing Soldat and active on these forums from 2004 - present