323s Item Name Grabber

Grabs item names from the marketplace, you can input a range of item IDs. Outputs the ID + Item Name to a text file.

Enjoy! Going to have different versions that also output values and stuff later on, but for now this is it.

import java.util.*;

import java.io.*;

import java.io.UnsupportedEncodingException; 

import org.apache.commons.lang.StringUtils; 



class ItemNameGrabber

{

    public static void main(String[] args)

    {

        //Item Name Grabber

        //For Gaia Online

        //Made by 323

        //Grabs item names from the market and outputs them to a text file

        

        Scanner scan = new Scanner(System.in);

        

        int startvalue;

        int endvalue;

        int current;

        

        HTTPRequest request = new HTTPRequest(); 

        request.setURL("http://www.gaiaonline.com/");

        

        System.out.println("ING by 323");

        System.out.println("Enter Start Value:");

        startvalue = scan.nextInt();

        System.out.println("Enter End Value: (Don't make it too big!)");

        endvalue = scan.nextInt();

        System.out.println("Scanning Marketplace...");

        String source = request.get();

        

        if (source != null) { 

            //"Grab Sources" test successful

            System.out.println("GS Test Successful.");

            System.out.println("Still Scanning Market...");

        } 

        else { 

            System.out.println("Error: Couldn't grab page source! ABORTING."); 

        }

        

        current = startvalue;

        

        try {

            PrintWriter out = new PrintWriter(new FileWriter("MarketNames.txt"));

            while(current < endvalue) {

                request.setURL("http://www.gaiaonline.com/marketplace/itemdetail/" + current);

                source = request.get();

                String name = StringUtils.substringBetween(source, "<h2 id=\"vend_item_title\">", "</h2>");

                if (name != null )

                {

                    out.println(current + ": " + name);

                }

                current++;

            }

            out.close();

        }

        catch ( IOException error ) {

            System.err.println( "Error writing to output file: " + error );

        }

        

        System.out.println("Finished Scanning Marketplace!");

        System.out.println("Check MarketNames.txt for your output!");

    }

}

I'll upload an exe of it in a little while!

whats the point of it?

Use wrote:

whats the point of it?

I had been using it to find hidden/unreleased item names, which surprisingly enough I found a ton of. I'm actually going to release a hidden item finder version of it later today or tomorrow, and it will be able to output the names of hidden items.

Flareboy323 wrote:

I had been using it to find hidden/unreleased item names, which surprisingly enough I found a ton of. I'm actually going to release a hidden item finder version of it later today or tomorrow, and it will be able to output the names of hidden items.

oh okay that makes more sense
cool

Why are you using the marketplace instead of the GSI.

import requests



item_id = 1404

uri = 'http://www.gaiaonline.com/chat/gsi/'

post_data = {'v':'json', 'm':'[[712,[{0}]]]'.format(item_id)}

item_info = requests.post(uri, params=post_data).json()

print list(item_info)[0][2][0]['name']

[/0][/2][/0]

I didn't test, did it from memory but it should work. Though this is for one item, you can easily pass multiple requests (explained in my tutorial) at once, allowing you to be able to view upwards of 75 to 100 item details through one request.

Basically, combine all the things I've shown in my release and you can make an item scanner that can scan huge ranges within just a few seconds as opposed to minutes.

I'll throw together a quick item scanner in a second.

Threw this together really quickly to demonstrate how you can get huge ranges from the GSI a lot faster than the market. It's not threaded, but if you did thread it it could scan thousands of item ranges within seconds. Could have made it more compact, or done some other misc things to it but I didn't feel like it and it was a five minute job.

Edit: Update part of the code to be a bit more dynamic.

import requests

import json



def get_item_info(start, end):

    uri = 'http://www.gaiaonline.com/chat/gsi'

    scan_range = [i for i in xrange(start, end+1)]

    for i in xrange(0, len(scan_range), 50):

        m = [json.dumps([(712, [id]) for id in scan_range[i:i+50]])]

        httpget = requests.get(uri, params={'v': 'json', 'm': m})

        [(yield ret) for ret in httpget.json()]



scan_range = None



while True:

    try:

        scan_from = int(raw_input('Scan from: '))

        scan_to = int(raw_input('Scan to: '))

        break

    except:

        print 'Invalid number.'



item_info = get_item_info(scan_from, scan_to)



with open('items.txt', 'a') as file:

    for item in item_info:

        try:

            item_id = item[2][0]['item_id']

            item_name = item[2][0]['name']

            file.write('{} : {}\n'.format(item_id, item_name))

        except:

            pass

[/0][/2][/0][/2][/id]

Huh, that's actually pretty cool. I wasn't even aware you could get marketplace data through GSI, haha. Cool though, I'm going to have to implement that.

Flareboy323 wrote:

Huh, that's actually pretty cool. I wasn't even aware you could get marketplace data through GSI, haha. Cool though, I'm going to have to implement that.

Yup, I've made item scanners that can scan pretty much every conventional item ID they would have within under a minute with threading.

Tree wrote:

Yup, I've made item scanners that can scan pretty much every conventional item ID they would have within under a minute with threading.

Wow, that's pretty cool. I've never really looked into multithreading, would you say it's an interesting topic to learn? It seems like it could be a cool way to speed up my programs.

Threading isn't necessarily just for speed increases. It can be fairly simple to learn so you might as well try.

Tree wrote:

Yup, I've made item scanners that can scan pretty much every conventional item ID they would have within under a minute with threading.

Of course, internet speeds providing the ability to do so

MattSmith wrote:

Of course, internet speeds providing the ability to do so

Haha that's true. With my 250kbps download speed, it might take five or ten minutes, but that's still a lot faster than the hours it would take to do it without multi-threading.

Flareboy323 wrote:

Haha that's true. With my 250kbps download speed, it might take five or ten minutes, but that's still a lot faster than the hours it would take to do it without multi-threading.

I guarantee that threading will speed things up! Make sure you don't have any memory leaks though Very unlikely you will however.

MattSmith wrote:

I guarantee that threading will speed things up! Make sure you don't have any memory leaks though Very unlikely you will however.

Oh god, memory leaks suck e_e one of the drivers on my old dell laptop has a memory leak, and the drivers are so old that they don't receive updates anymore. So, I can use the computer for two to three hours, but then it completely crashes because it doesn't even have enough memory to run explorer.exe, I hate it. I use the laptop a lot too, despite how old it is. Dell Latitude D610, 1GB RAM, Intel Centrino CPU, found in the trash on my friends College Campus.

great!