Frenchie's MAD! Again!

[Frenchie]

Frenchie's MAD!

Hey Everyone!
I'm here to announce MAD 2.0 a revamp of the original

For those who don't know the original, find it here
http://forums.soldat.pl/index.php?topic=11121.0
(Includes a bigger/different explanation) (Still Works except no new maps added as old lobbies have been taken down)


MAD...  Map Automation Daemon


What?
It's a script which fetches the entire server list, then goes through it and downloads all news maps currently in play on the servers.

Why?
Simple.... I'm lazy.

• I'd rather spend time playing Soldat then wasting my time following dead links searching the internet for maps.
• And I like a 1 STOP SHOP for all my map needs

And I'm sure there are alot of you out there that hate having just downloaded a map and it's missing a bloody texture or scenery!! Now your going to have to get in contact with the author or website to tell them about it, in the hopes they get the missing file.
Or scrounge around downloading other maps hoping that they have that 1 missing scenery.

For those of you who fall in the above category (Most of you who have been around Soldat for a while) this is exactly what MAD is here to relieve you from

Without further ado

MAD 2.0

It still has the same look.. I will change it when I get around to it.........

For those who want to know more.. Or whats different with the first one.. READ ON

How?
It is a huge upgrade in terms of code. Basically everything is new or re-written.

Fetch Servers?
OLD - HTML PARSER ----> NEW - Lobby Protocol (PHP Parser by PerroAZL)
Read Maps?
OLD - Doc's Unreleased PMS Map Reader ----> NEW - Doc's PMS Map Reader V4
D/L Maps?
OLD - Buggy Script by Me ----> NEW - Sooper Dooper Script by Me (it's alright)

Anything Else New?
Yeah! It includes a whole New Database backend which records every single scenery that belongs to every single map (Along with the normal map details, as before) - This feature will be used in future

Yeah! It also doesn't limit it to only 2 ip's/ports per map d/l as before -- Now it will go through all servers with same map till it gets it.

Yeah! Map Previews are not available at the moment. (Look at old MAD to see what I mean)

Yeah! Just a few new functions which have improved efficiency of Map/Scenery/Texture downloading

Thanks?
Delphille - For providing a code snippet of his about New Lobby Protocol (Protocol was very new to me)
PerroAZL - For his great Server List Grabber (use with Lobby Protocol)
Doc - For his PMS map reader (what can I say? works well)
Devin Doucette - Still using his TAR/ZIP archive class

Now it's still sorta a beta.. I have done quite alot of testing... But I just updated it today and.. you never know


And if anyone has any comments or suggestions please post!
Thanks Everyone.

[jrgp]

That all sounds great, but that link turns up 404.

[Frenchie]

That all sounds great, but that link turns up 404.

Ummm mate it's working here..
You talking about this link?
http://tumler.bendaweb.com/MAD2/

Ooppss sorry just found what you meant.. (the download links) - Will Fix

Fixed -- I had copy and pasted another index on it just before creating topic and forgot to check (turned out it was old one)

[BombSki]

sweet! but:
The requested URL /MAD2/imagemap.php was not found on this server.

[Frenchie]

sweet! but:
The requested URL /MAD2/imagemap.php was not found on this server.

Yes I've removed the image preview for the time being

-Fixed the links pointing to it (FTP client/server connection hasn't been working right for last day)

Date Posted: June 13, 2008, 06:37:41 amHad to restart database and MAD
-As the files I had updated yesterday weren't ftp'ed to server (Well they were...)
-And the cron config had changed from what it was yesterday (Must of been something done to server)
-Did abit of fiddling with map downloads. Now when you open the zip it tells you | Title:/Waypoints/And some others..

[jrgp]

Maybe the list could also include the map's polycount and filesize for both the zip and the gzipped-tarball?

[Frenchie]

Yes polycount is easy to add.

But as the archives are done on-the-fly I can't really get filesize. (Might later make them save to server first but it's not really a big priority)

So any more comments? What daayaa reckon?

[jrgp]

But as the archives are done on-the-fly I can't really get filesize. (Might later make them save to server first but it's not really a big priority)

Just cache the filesize in the database. Maybe have it so when a map is downloaded in either of those two formats, the script checks for a filesize database entry, and it it's not there it enters it in. That would save you time. You don't even need to have them saved, just use output buffering. I'm not really sure what libraries you use for the zip / tgz generating, but the following should give you an idea on how to capture the filesize.

Code: [Select]

<?php
//.......

//start output buffering so we can see what would be sent out
ob_start();

//whatever function you use to sent out the .newly generated .zip / .tgz.
archive_send();

//the size of what's in the buffer (your filesize)
$file_size = ob_get_length;

//send out the archive and kill the buffer.
ob_ end_ flush();

//update mysql database.. (I'm assuming all your map's have their own unique ID)
mysql_query("update `maps` set `filesize` = '$file_size' where `id` = '$map_id' limit 1");
?>


[Frenchie]

Wow you learn something new everyday...

Cool thanks alot jrgp, I will look into that (Hopefully adding it soon)

Date Posted: June 13, 2008, 11:01:36 pmAdded FileSizes (Of Zip Archive-- Tgz are usually smaller)

I edited the archive script to get it to spit out filesizes and they are added once a download is initiated

[DorkeyDear]

I love MAD for getting random maps I know the name of that I need where I don't know where to get. The more features there are, the merrier.

Yeah! Map Previews are not available at the moment. (Look at old MAD to see what I mean)

It seemed not to smooth before, but hopefully you'll reimplement it being smoother. It helps figuring out of the map you found is the map your looking for w/out downloading it. (Did the old version generate the map one time, and then saved the image; and next time it is requested, it would instead load the image? If not, hope newer version does this (deletes image if map is modified in any way, unless its something that doesn't effect the image))

[Frenchie]

Thanks for the coment dorkeydear

It seemed not to smooth before, but hopefully you'll reimplement it being smoother. It helps figuring out of the map you found is the map your looking for w/out downloading it. (Did the old version generate the map one time, and then saved the image; and next time it is requested, it would instead load the image? If not, hope newer version does this (deletes image if map is modified in any way, unless its something that doesn't effect the image))

I will be reimplementing the Map Previews. The old MAD loaded each pic dynamically but the new will be saving to disk.
The old mad screwed up alot with previews as it was pretty buggy.
The new  MAD 2.0  has actually fixed all the bugs the old one had. (And it fetching maps faster than ever)

Date Posted: June 16, 2008, 07:21:20 amForgot to post - For those who don't know already
MAD will not download default scenery-gfx or textures (much asked about feature from old MAD) - Making smaller and faster downloads

[BombSki]

what if there are 2 versions of 1 map?
edit: or 2 maps with the same name

[Frenchie]

Soldat Maps themselves do not follow a strict naming convention (ie. Mappers can & do name for eg. ctf maps as 'AwEsUmmOO' instead of 'ctf_Awesummoo' which confuses many people but there isn't much we can do about it.) -- MAD does have a naming convention which changes map from eg 'CTF_GiBBerish' to 'ctf_Gibberish' so they are more readable

If there are all multiple versions of maps such as 'ctf_Balony' & 'ctf_Balonyv2' etc..
MAD will download them each as long as they have different names.
I see no point removing different versions, as some servers could be hosting old maps which can no longer be found on internet (as old links are updated for new map) Except for MAD


BUT if there are 2 maps with the same name, that is BAD practise by the mappers regardless if they are edits to there same named original map. (This will cause map wrong version bugs to all those players who have different versions of the same named maps trying to join servers)

Bottomline. MAD downloads different named maps, skipping ones it already has. 

[DorkeyDear]

Any way that we could "fix" the "bug" where if somebody update's their map, the new one will be on MAD? Maybe if the filesize is different, it'll save it as a subcategory under that map name?
BTW, I see it moved to http://www.soldathq.com/mad/

Date Posted: June 16, 2008, 02:40:19 amYou know what would be a nice feature? A list of servers the map has been seen on. (mayb a sublink off of a map)
If you know of a not-so-popular gamemode, and know some maps; you may be able to find servers hosting it

[Frenchie]

Any way that we could "fix" the "bug" where if somebody update's their map, the new one will be on MAD? Maybe if the filesize is different, it'll save it as a subcategory under that map name?

I don't know if you can get the map filesize without actually downloading it off the server. (maybe Enesce could enlighten us)
Sooo.. the only way possible at the moment would be to download every map off every single server (500+) ever time MAD Updates (often) just to check their filesizes. This is quite a waste of resources.  So looking at it this way I won't be implementing it.

BTW, I see it moved to http://www.soldathq.com/mad/

Date Posted: June 16, 2008, 02:40:19 am

Yes I have been with my free host for years now and would recommend them to anyone. (namepad.net 500 mb 10000mb BW, mysql, CPanel all FREE!)
But I've wanted a domain for a while and there was a good host that doesn't oversell with a good deal (costing less than $1.5 per month and unlimited features besides space & bw) and upgraded to that

I'll be keeping my free host as well for hosting stuff as well.
But as you noticed  MAD has been transferred to soldathq.com

You know what would be a nice feature? A list of servers the map has been seen on. (mayb a sublink off of a map)
If you know of a not-so-popular gamemode, and know some maps; you may be able to find servers hosting it

So you basically want a server list which displays servers by their gaemode?
Well my ASE Soldat Lobby had that feature built in (a long with many others. )
Yes with my ASE Soldat Lobby you could view that stats of the servers (including no. of gamemodes being played throughout servers easily viewable in a list, and all the maps currently in rotation) but for some reason it's not accessible atm.

edit--- Actually yeah you can see the stats, there at http://www.tumler.bendaweb.com/lobby/stats/

Plus you could make it show specific gamemodes only such as http://www.tumler.bendaweb.com/lobby/mode/CTF/
But it doesn't have many servers 
It's because it was made for ver 1.3 and when the old lobbies were taken down it couldn't update it's list anymore (but it still lists servers which havent changed ip/port since then)
But I will be fixing that

Date Posted: June 16, 2008, 06:32:43 pmUpdate: MAD index now cached on server (Faster/More Efficient)

I re-read what you said Dorkey and yeah wouldn't be a bad idea.
Plus I'm thinking of a new feature where it would allow people to create mappacks. So instead of searching through your computer and finding maps/sceneries, then compressing the maps and then uploading.
 
They could log on to MAD and tick the maps they want, add a description and then get a link to their mappack.

What do you guys reckon?

[DorkeyDear]

Mappack idea would be wicked awsome! Not sure how it would work though... maybe you can type in "(cs|de|)_.*" (regular expression of some sort, doesnt have to be POSIX) and it'll get all cs maps and put them into a mappack; then it'll add it to a list of previously generated map packs, and you can name it and stuff
Maybe a list of "other" maps not found

Maybe a search thing, and then you check off maps you want to put into a mappack?

Or you can upload your own mappacks

Many different ways to do it

[Frenchie]

Or you can upload your own mappacks

I definitely don't want users to upload mappacks, as they will be able to create them online using the maps in MAD

Maybe a search thing, and then you check off maps you want to put into a mappack?

Yeah I wanna add a search function and I'm hoping to add link that can be clicked on any page so it automatically puts check boxes next to all the maps on the screen and allows users to select maps

maybe you can type in "(cs|de|)_.*" (regular expression of some sort, doesnt have to be POSIX) and it'll get all cs maps and put them into a mappack;

I actually wasn't thinking along this track but it's interesting.

Basically I was thinking of giving users the option to make mappacks by ticking maps they want from a list they would want.
Plus I would have some 'auto-generating' mappacks that would be hardcoded into MAD (Like a cs mappack [fetch maps with cs_,de_] or zombie maps [has zombie in name, zm_]

BUT
---------------------------------
Your idea got me thinking....
I like the idea of having regular expression searches for users making mappacks

So I think the best way to do it will be like this:
 [BASIC]
While your browsing or searching  MAD and you like the maps you found. Click on create a mappack and it will add checkboxes to all the maps on screen, select maps, then at the bottom of page add a description. DONE! (you will be given link)
+
[ADVANCED - makes auto-generating mappacks]
regexp: click create mappack, click on advanced, then add regexp for mapname AND/OR add regexp for title DONE! (you will be given link)

Possiblites with this: The regexp will provide quite alot of power and make auto-generating mappacks (mappacks that grow in size without user input) as they can be set to search for specific mappers in titlenames and game types in mapname. (eg. download DM maps by chakapo [new maps will automatically be added to mappack when added to MAD])

What do you reckon?

-----------------
on another note. Map Previews have been re-added

[DorkeyDear]

Maybe the search can be local, using javascript... that way the page doesn't have to reload; i have hardly any experience in that so i have no idea if its possible.
and then every map you check off will be added to a list at the bottom
at the bottom, under the list, you can put in the name of the mappack; and there will be a button saying "Generate ZIP" or "Generate RAR" or whatever grouping/compressing file types there are
and also ability to see all generated mappacks
if you check something off, then change the filture; it'll stay inside the list of checked off maps

Yippy! Map previews! *checks it out*

[chrisgbk]

As a headsup, the file download code in 1.5.0 has changed. I don't remember offhand if it has compatability with the old way, but I don't believe it does, so be prepared to change it once 1.5.0 is released.

On the brightside, it allows multiple download requests combined into one, so you can request every scenery file at once in one connection.

[DorkeyDear]

Should request the protocol in advance, they should give it away sense it will be public information, or at least for sure will be once 1.5 comes out.