Shit, can anyone help?

[323]

To get any further in my Gaia bot development, I need to be able to login. I have the entire login down, the only problem I'm having is managing cookies. The HTTP Wrappers I'm using right now were made by Isonyx and don't support cookies. I was originally going to use JSoup, but I need the HTTP Wrappers to be able to export the page source as a string, which JSoup can't do. I was then going to try Apache HTTP Utilities, but it seems way too complicated for what I need. I then downloaded cURL and it looked really promising and everything, and then I noticed the Makefile. So, the cURL for Java is made for linux systems, and thus I can't use it.

So, does anyone know of HTTP Wrappers for Java that support cookies? And if not, anyone know a simple tutorial for Apache HTTP Utilities or on how to install cURL for Java on a windows system? I'm using NetBeans as an IDE if that helps.

Thanks!

[The Unintelligible]

I'll ask Isonyx and see if he can do anything about cookie support. Shouldn't be too much of a problem.

[323]

I'll ask Isonyx and see if he can do anything about cookie support. Shouldn't be too much of a problem.

Oh okay, that would be really awesome actually. Much easier for me than learning an entirely new library that I can't manage to get set up, haha. If you don't know what HTTP Wrapper I'm talking about I could PM it to you if you want.

[The Unintelligible]

I think I know which one you're referring to. Or at least Isonyx would.

[323]

I think I know which one you're referring to. Or at least Isonyx would.

You had shared it with me from Isonyx's Github a while ago, it was really useful until I ran into this cookie problem haha.

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;


public class HTTPRequest {
    private URL myURL;
    private HttpURLConnection URLConnection;
    public HTTPRequest(String myURL) {
        this.URLConnection = null;
        try
        {
            this.myURL = new URL(myURL);
        }
        catch (Exception e) 
        { e.printStackTrace();
        }
    }
    public HTTPRequest() 
    {
    }
    public URL getURL()
    {
        return myURL;
    }
    public void setURL(String URL)
    {
        try {
            myURL = new URL(URL);
        }
        catch (MalformedURLException e) 
        {
            e.printStackTrace();
        }
    }
    public String get()
    {
        StringBuffer returnString = new StringBuffer();
        String temporary = "";
        try
        {
            URLConnection = (HttpURLConnection)myURL.openConnection();
            URLConnection.setRequestMethod("GET");
            URLConnection.setInstanceFollowRedirects(true);
            URLConnection.setDoOutput(true);    
            URLConnection.setDoInput(true);
            URLConnection.connect();
            BufferedReader reader = new BufferedReader(new InputStreamReader((InputStream)URLConnection.getContent()));
            while((temporary = reader.readLine()) != null)
            { 
                returnString.append(temporary + "\r");
            }
            reader.close();
            return returnString.toString();
        }
        catch (Exception e) {
            e.printStackTrace();
            return null;
        }
        finally
        {
            if (URLConnection != null)
            {
                URLConnection.disconnect();
            }
        }
    }
    public String post(String theLink, String parameters)
    {
        StringBuffer returnString = new StringBuffer();
        String temporary = "";
        try
        {
            URLConnection = (HttpURLConnection)new URL(myURL + theLink).openConnection();
            URLConnection.setRequestMethod("POST");
            URLConnection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
            URLConnection.setRequestProperty("Content-Length", "" + Integer.toString(parameters.getBytes().length));
            URLConnection.setRequestProperty("Content-Language", "en-US");
            URLConnection.setUseCaches (false);
            URLConnection.setDoInput(true);
            URLConnection.setDoOutput(true);
            DataOutputStream write = new DataOutputStream(URLConnection.getOutputStream());
            write.writeBytes(parameters);
            write.flush();
            write.close();
            BufferedReader reader = new BufferedReader(new InputStreamReader((InputStream)URLConnection.getInputStream()));
            while((temporary = reader.readLine()) != null)
            { 
                returnString.append(temporary + "\r");
            }
            reader.close();
            return returnString.toString();
        }
        catch (Exception e)
        {
            e.printStackTrace();
            return null;
        }
        finally
        { 
            if (URLConnection != null)
            {
                URLConnection.disconnect();
            }
        }
    }
    public String toString()
    {
        return myURL.getHost();
    }
}

EDIT: Found the link you had given me https://github.com/Isonyx/HTTPRequest

[ya bish]

I understood none of what you said, but it sounds awesome.

[323]

I understood none of what you said, but it sounds awesome.

I need a programming library that will let me interact with websites. Gaia stores a little number, your "Session ID", in a cookie. The library I had that interacted with the web couldn't store this little number because it couldn't accept cookies. Because of this, Gaia would always log me out, because I didn't have a session, because it didn't see a "Session ID" on my computer.

[Isonyx]

Here's an update to the HTTP Wrapper. Added cookie support as well as a README file.

https://github.com/Isonyx/HTTPRequest

Let me know if I didn't implement something correctly, I rushed this update during a history class.

Edit: That part should be fixed. Test and let me know how it works for you.

[323]

Here's an update to the HTTP Wrapper. Added cookie support as well as a README file.

https://github.com/Isonyx/HTTPRequest

Let me know if I didn't implement something correctly, I rushed this update during a history class.

Edit: That part should be fixed. Test and let me know how it works for you.

Thanks a ton, but it appears to be having problems posting for some reason. I have all of my post data working and everything, it's saying it's having a problem at this line of HTTPRequest.java:

BufferedReader reader = new BufferedReader(new InputStreamReader((InputStream)URLConnection.getInputStream()));

Not sure why.

I was using this to check for a login success:

        request.setURL("http://www.gaiaonline.com/auth/login/");
        String login = request.post("", postdata);
        if (login.contains("login_success")) {
            System.out.println("Logged in successfully!");
        }
        else {
            System.out.println("Login Failed!");
        }

But I'm getting an error because the string login is null, it doesn't contain any data because there's something wrong with the actual POST haha. Thanks for updating this though, it's going to be super helpful to me once we can get all of these bugs ironed out.

[Stapled]

Thanks a ton, but it appears to be having problems posting for some reason. I have all of my post data working and everything, it's saying it's having a problem at this line of HTTPRequest.java:

BufferedReader reader = new BufferedReader(new InputStreamReader((InputStream)URLConnection.getInputStream()));

Not sure why.

I was using this to check for a login success:

        request.setURL("http://www.gaiaonline.com/auth/login/");
        String login = request.post("", postdata);
        if (login.contains("login_success")) {
            System.out.println("Logged in successfully!");
        }
        else {
            System.out.println("Login Failed!");
        }

But I'm getting an error because the string login is null, it doesn't contain any data because there's something wrong with the actual POST haha. Thanks for updating this though, it's going to be super helpful to me once we can get all of these bugs ironed out.

Does the post give an error or just return nothing? If it returns nothing, like the one I'm using in C# sometimes does, you can just do what I did, post login, get Welcome to Gaia | Gaia Online and check for "Welcom back".

[Isonyx]

Thanks a ton, but it appears to be having problems posting for some reason. I have all of my post data working and everything, it's saying it's having a problem at this line of HTTPRequest.java:

BufferedReader reader = new BufferedReader(new InputStreamReader((InputStream)URLConnection.getInputStream()));

Not sure why.

I was using this to check for a login success:

        request.setURL("http://www.gaiaonline.com/auth/login/");
        String login = request.post("", postdata);
        if (login.contains("login_success")) {
            System.out.println("Logged in successfully!");
        }
        else {
            System.out.println("Login Failed!");
        }

But I'm getting an error because the string login is null, it doesn't contain any data because there's something wrong with the actual POST haha. Thanks for updating this though, it's going to be super helpful to me once we can get all of these bugs ironed out.

What's the actual error?

Also do you mind providing me with some post data so I can test with Gaiaonline?

Keep in mind, without specifying certain headers the code will error with java.netSocketException.

For example

HTTPRequest request = new HTTPRequest();
		
request.setURL("http://www.gaiaonline.com/auth/");
		
		
//request.setUserAgent("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061015 Firefox/3.0");
//request.setAccept("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", 
//		"en-us,en;q=0.5", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
	
//request.setKeepAlive("300");
//request.setConnection("keep-alive");
//request.setReferer("http://www.gaiaonline.com/auth/login");
//request.setContent("application/x-www-form-urlencoded", "", "en-US");
		
String postData, thePost;

postData = "username=admin&password=helloworld&sid=1234";
	
thePost = request.post("", postData);

System.out.println("Post: " + thePost);

Isn't going trk. Post will return "null" and a java.netSocketException will be thrown.
However,

HTTPRequest request = new HTTPRequest();
		
request.setURL("http://www.gaiaonline.com/auth/");
		
	
//request.setUserAgent("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061015 Firefox/3.0");
//request.setAccept("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", 
//		"en-us,en;q=0.5", "ISO-8859-1,utf-8;q=0.7,*;q=0.7");
	
//request.setKeepAlive("300");
//request.setConnection("keep-alive");
//request.setReferer("http://www.gaiaonline.com/auth/login");
request.setContent("application/x-www-form-urlencoded", "", "en-US");
		
String postData, thePost;
	
postData = "username=admin&password=helloworld&sid=1234";
		
thePost = request.post("", postData);

System.out.println("Post: " + thePost);

Will work.

You may not have set the content type.

request.setContent("application/x-www-form-urlencoded", "", "en-US");

[323]

-stuff-

My full source: (The file is called GaiaLogin2, btw)

import java.io.UnsupportedEncodingException; 
import org.apache.commons.lang.StringUtils; 
class GaiaLogin2 { 
    public static void main(String[] args) throws UnsupportedEncodingException { 
        String lookfor = "The username or password"; 
        HTTPRequest request = new HTTPRequest(); 
        request.setURL("http://www.gaiaonline.com/");
        System.out.println("URL: " + request.getURL()); 
        String source = request.get(); 
        if (source != null) { 
            System.out.println("Source successfully grabbed: " + source); 
        } 
        else { 
            System.out.println("Error: Couldn't grab page source!"); 
        } 
        String title = StringUtils.substringBetween(source, "<title>", "</title>"); 
        System.out.println("Title: " + title); //Print out page title, for testing purposes 
        String posts = StringUtils.substringBetween(source, "</fieldset>", "</form>").replace("data-value", "");//limits the string search to the area with the data 
        //posts = posts.replace("data-value", ""); 
        String[] names = StringUtils.substringsBetween(posts, "name=\"", "\""); //finds the names of the strings 
        String[] values = StringUtils.substringsBetween(posts, "value=\"", "\"");//finds the values of the strings 
        System.out.println(names[0] + " => " + values[0]); //prints the first name and value 
        System.out.println(names[1] + " => " + values[1]); //prints the second name and value 
        System.out.println(names[2] + " => " + values[2]); //prints the third name and value 
        System.out.println(names[3] + " => " + values[3]); //prints the fourth name and value 
        
        //login starts here 
        String user = "usernamehere"; 
        String pass = "passwordhere";
        String postdata = names[0] + "=" + values[0] + 
            "&" + names[1] + "=" + values[1] + 
            "&" + names[2] + "=" + values[2] + 
            "&" + names[3] + "=" + values[3] + 
            "&" + "&username=" + user + "&password=" + pass; 
        request.setURL("http://www.gaiaonline.com/auth/login/");
        String login = request.post("", postdata);
        if (login.contains("login_success")) {
            System.out.println("Logged in successfully!");
        }
        else {
            System.out.println("Login Failed!");
        }
    } 
}

The specific error is:

java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:718)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:715)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
	at HTTPRequest.post(HTTPRequest.java:204)
	at GaiaLogin2.main(GaiaLogin2.java:36)
	at __SHELL10.run(__SHELL10.java:6)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at bluej.runtime.ExecServer$3.run(ExecServer.java:725)
java.lang.NullPointerException
	at GaiaLogin2.main(GaiaLogin2.java:37)
java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:718)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:715)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
	at HTTPRequest.post(HTTPRequest.java:204)
	at GaiaLogin2.main(GaiaLogin2.java:36)
	at __SHELL12.run(__SHELL12.java:6)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at bluej.runtime.ExecServer$3.run(ExecServer.java:725)
java.lang.NullPointerException
	at GaiaLogin2.main(GaiaLogin2.java:37)

This has only happened after updating the wrapper.

If I remove the check for login success, the only difference is no NullPointerException, but still the other error.

[Isonyx]

My full source: (The file is called GaiaLogin2, btw)

import java.io.UnsupportedEncodingException; 
import org.apache.commons.lang.StringUtils; 
class GaiaLogin2 { 
    public static void main(String[] args) throws UnsupportedEncodingException { 
        String lookfor = "The username or password"; 
        HTTPRequest request = new HTTPRequest(); 
        request.setURL("http://www.gaiaonline.com/");
        System.out.println("URL: " + request.getURL()); 
        String source = request.get(); 
        if (source != null) { 
            System.out.println("Source successfully grabbed: " + source); 
        } 
        else { 
            System.out.println("Error: Couldn't grab page source!"); 
        } 
        String title = StringUtils.substringBetween(source, "<title>", "</title>"); 
        System.out.println("Title: " + title); //Print out page title, for testing purposes 
        String posts = StringUtils.substringBetween(source, "</fieldset>", "</form>").replace("data-value", "");//limits the string search to the area with the data 
        //posts = posts.replace("data-value", ""); 
        String[] names = StringUtils.substringsBetween(posts, "name=\"", "\""); //finds the names of the strings 
        String[] values = StringUtils.substringsBetween(posts, "value=\"", "\"");//finds the values of the strings 
        System.out.println(names[0] + " => " + values[0]); //prints the first name and value 
        System.out.println(names[1] + " => " + values[1]); //prints the second name and value 
        System.out.println(names[2] + " => " + values[2]); //prints the third name and value 
        System.out.println(names[3] + " => " + values[3]); //prints the fourth name and value 
        
        //login starts here 
        String user = "usernamehere"; 
        String pass = "passwordhere";
        String postdata = names[0] + "=" + values[0] + 
            "&" + names[1] + "=" + values[1] + 
            "&" + names[2] + "=" + values[2] + 
            "&" + names[3] + "=" + values[3] + 
            "&" + "&username=" + user + "&password=" + pass; 
        request.setURL("http://www.gaiaonline.com/auth/login/");
        String login = request.post("", postdata);
        if (login.contains("login_success")) {
            System.out.println("Logged in successfully!");
        }
        else {
            System.out.println("Login Failed!");
        }
    } 
}

The specific error is:

java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:718)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:715)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
	at HTTPRequest.post(HTTPRequest.java:204)
	at GaiaLogin2.main(GaiaLogin2.java:36)
	at __SHELL10.run(__SHELL10.java:6)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at bluej.runtime.ExecServer$3.run(ExecServer.java:725)
java.lang.NullPointerException
	at GaiaLogin2.main(GaiaLogin2.java:37)
java.net.SocketException: Unexpected end of file from server
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:718)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:715)
	at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:579)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1322)
	at HTTPRequest.post(HTTPRequest.java:204)
	at GaiaLogin2.main(GaiaLogin2.java:36)
	at __SHELL12.run(__SHELL12.java:6)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at bluej.runtime.ExecServer$3.run(ExecServer.java:725)
java.lang.NullPointerException
	at GaiaLogin2.main(GaiaLogin2.java:37)

This has only happened after updating the wrapper.

If I remove the check for login success, the only difference is no NullPointerException, but still the other error.

You should probably read the post I made again.
As I said setting the content type eliminated all errors for me.

I took your code and simply added

request.setContent("application/x-www-form-urlencoded", "", "en-US");

before the POST.

Although I haven't actually tested the login it runs error free on my machine.

[323]

You should probably read the post I made again.
As I said setting the content type eliminated all errors for me.

I took your code and simply added

request.setContent("application/x-www-form-urlencoded", "", "en-US");

before the POST.

Although I haven't actually tested the login it runs error free on my machine.

Awesome, works great thanks! I'll have to remember that. Now that I've got the basic code running, I'm going to try to get cookies up and working. From there, I can start my bot development! All thanks to you haha, thanks a ton for the wrappers and help and stuff!

Oh and Stapled, you too! You've helped me with a ton of my code!

[Isonyx]

Awesome, works great thanks! I'll have to remember that. Now that I've got the basic code running, I'm going to try to get cookies up and working. From there, I can start my bot development! All thanks to you haha, thanks a ton for the wrappers and help and stuff!

Oh and Stapled, you too! You've helped me with a ton of my code!

No problem. Drop me a message if you have any more issues.

[Stapled]

Awesome, works great thanks! I'll have to remember that. Now that I've got the basic code running, I'm going to try to get cookies up and working. From there, I can start my bot development! All thanks to you haha, thanks a ton for the wrappers and help and stuff!

Oh and Stapled, you too! You've helped me with a ton of my code!

Lol thanks. I've been lurking this thread because I was having the same problem with these Java wrappers as you. By the way, did this get the login working?

[323]

Lol thanks. I've been lurking this thread because I was having the same problem with these Java wrappers as you. By the way, did this get the login working?

Yeah, got it working perfectly! I'm going to open source it on Github when I get home, and then when I finish my refresher I'll throw it up on Github too. They'll still have threads on LG with links to 'em, though.

EDIT: Shit, still having cookie problems, damn. I tried to grab the title of "My Gaia", because it displays as "My Gaia" if you're logged in, or "Log In" if you aren't, so it's a good testing page. Lo and behold, the page said Log In instead of My Gaia. Shit.

[ya bish]

Yeah, got it working perfectly! I'm going to open source it on Github when I get home, and then when I finish my refresher I'll throw it up on Github too. They'll still have threads on LG with links to 'em, though.

EDIT: Shit, still having cookie problems, damn. I tried to grab the title of "My Gaia", because it displays as "My Gaia" if you're logged in, or "Log In" if you aren't, so it's a good testing page. Lo and behold, the page said Log In instead of My Gaia. Shit.

Dont they have solutions of this on google ? (noob)

[323]

Dont they have solutions of this on google ? (noob)

Nah, these are custom HTTP Wrappers made by Isonyx, so you won't find anything on google haha.

[ya bish]

Nah, these are custom HTTP Wrappers made by Isonyx, so you won't find anything on google haha.

Oh all right.

[Isonyx]

For the record the cookie problems have been fixed.
The revised code has been pushed to Github.

I don't know if or when Flareboy is planning to release his Gaia login code and refresher though.

On the topic of use, there's information in the README file in the repository.