323s Item Name Grabber

[323]

Grabs item names from the marketplace, you can input a range of item IDs. Outputs the ID + Item Name to a text file.

Enjoy! Going to have different versions that also output values and stuff later on, but for now this is it.

import java.util.*;
import java.io.*;
import java.io.UnsupportedEncodingException; 
import org.apache.commons.lang.StringUtils; 

class ItemNameGrabber
{
    public static void main(String[] args)
    {
        //Item Name Grabber
        //For Gaia Online
        //Made by 323
        //Grabs item names from the market and outputs them to a text file
        
        Scanner scan = new Scanner(System.in);
        
        int startvalue;
        int endvalue;
        int current;
        
        HTTPRequest request = new HTTPRequest(); 
        request.setURL("http://www.gaiaonline.com/");
        
        System.out.println("ING by 323");
        System.out.println("Enter Start Value:");
        startvalue = scan.nextInt();
        System.out.println("Enter End Value: (Don't make it too big!)");
        endvalue = scan.nextInt();
        System.out.println("Scanning Marketplace...");
        String source = request.get();
        
        if (source != null) { 
            //"Grab Sources" test successful
            System.out.println("GS Test Successful.");
            System.out.println("Still Scanning Market...");
        } 
        else { 
            System.out.println("Error: Couldn't grab page source! ABORTING."); 
        }
        
        current = startvalue;
        
        try {
            PrintWriter out = new PrintWriter(new FileWriter("MarketNames.txt"));
            while(current < endvalue) {
                request.setURL("http://www.gaiaonline.com/marketplace/itemdetail/" + current);
                source = request.get();
                String name = StringUtils.substringBetween(source, "<h2 id=\"vend_item_title\">", "</h2>");
                if (name != null )
                {
                    out.println(current + ": " + name);
                }
                current++;
            }
            out.close();
        }
        catch ( IOException error ) {
            System.err.println( "Error writing to output file: " + error );
        }
        
        System.out.println("Finished Scanning Marketplace!");
        System.out.println("Check MarketNames.txt for your output!");
    }
}

I'll upload an exe of it in a little while!

[Use]

whats the point of it?

[323]

whats the point of it?

I had been using it to find hidden/unreleased item names, which surprisingly enough I found a ton of. I'm actually going to release a hidden item finder version of it later today or tomorrow, and it will be able to output the names of hidden items.

[Use]

I had been using it to find hidden/unreleased item names, which surprisingly enough I found a ton of. I'm actually going to release a hidden item finder version of it later today or tomorrow, and it will be able to output the names of hidden items.

oh okay that makes more sense
cool

[Tree]

Why are you using the marketplace instead of the GSI.

import requests

item_id = 1404
uri = 'http://www.gaiaonline.com/chat/gsi/'
post_data = {'v':'json', 'm':'[[712,[{0}]]]'.format(item_id)}
item_info = requests.post(uri, params=post_data).json()
print list(item_info)[0][2][0]['name']

I didn't test, did it from memory but it should work. Though this is for one item, you can easily pass multiple requests (explained in my tutorial) at once, allowing you to be able to view upwards of 75 to 100 item details through one request.

Basically, combine all the things I've shown in my release and you can make an item scanner that can scan huge ranges within just a few seconds as opposed to minutes.

I'll throw together a quick item scanner in a second.

[Tree]

Threw this together really quickly to demonstrate how you can get huge ranges from the GSI a lot faster than the market. It's not threaded, but if you did thread it it could scan thousands of item ranges within seconds. Could have made it more compact, or done some other misc things to it but I didn't feel like it and it was a five minute job.

Edit: Update part of the code to be a bit more dynamic.

import requests
import json

def get_item_info(start, end):
    uri = 'http://www.gaiaonline.com/chat/gsi'
    scan_range = [i for i in xrange(start, end+1)]
    for i in xrange(0, len(scan_range), 50):
        m = [json.dumps([(712, [id]) for id in scan_range[i:i+50]])]
        httpget = requests.get(uri, params={'v': 'json', 'm': m})
        [(yield ret) for ret in httpget.json()]

scan_range = None

while True:
    try:
        scan_from = int(raw_input('Scan from: '))
        scan_to = int(raw_input('Scan to: '))
        break
    except:
        print 'Invalid number.'

item_info = get_item_info(scan_from, scan_to)

with open('items.txt', 'a') as file:
    for item in item_info:
        try:
            item_id = item[2][0]['item_id']
            item_name = item[2][0]['name']
            file.write('{} : {}\n'.format(item_id, item_name))
        except:
            pass

[323]

Huh, that's actually pretty cool. I wasn't even aware you could get marketplace data through GSI, haha. Cool though, I'm going to have to implement that.

[Tree]

Huh, that's actually pretty cool. I wasn't even aware you could get marketplace data through GSI, haha. Cool though, I'm going to have to implement that.

Yup, I've made item scanners that can scan pretty much every conventional item ID they would have within under a minute with threading.

[323]

Yup, I've made item scanners that can scan pretty much every conventional item ID they would have within under a minute with threading.

Wow, that's pretty cool. I've never really looked into multithreading, would you say it's an interesting topic to learn? It seems like it could be a cool way to speed up my programs.

[Tree]

Threading isn't necessarily just for speed increases. It can be fairly simple to learn so you might as well try.

[MattSmith]

Yup, I've made item scanners that can scan pretty much every conventional item ID they would have within under a minute with threading.

Of course, internet speeds providing the ability to do so

[323]

Of course, internet speeds providing the ability to do so

Haha that's true. With my 250kbps download speed, it might take five or ten minutes, but that's still a lot faster than the hours it would take to do it without multi-threading.

[MattSmith]

Haha that's true. With my 250kbps download speed, it might take five or ten minutes, but that's still a lot faster than the hours it would take to do it without multi-threading.

I guarantee that threading will speed things up! Make sure you don't have any memory leaks though Very unlikely you will however.

[323]

I guarantee that threading will speed things up! Make sure you don't have any memory leaks though Very unlikely you will however.

Oh god, memory leaks suck e_e one of the drivers on my old dell laptop has a memory leak, and the drivers are so old that they don't receive updates anymore. So, I can use the computer for two to three hours, but then it completely crashes because it doesn't even have enough memory to run explorer.exe, I hate it. I use the laptop a lot too, despite how old it is. Dell Latitude D610, 1GB RAM, Intel Centrino CPU, found in the trash on my friends College Campus.

[itsmike0]

great!