I'm having a problem with my code/bot, the problem is outlined in more detail below:
I am using the requests library to try and post on a phpBB forum.
My code looks something like the following:
Then I check whether the topic was posted successfully by searching for a string in the response.Code:self.r = requests.session() d = self.r.get('http://examplephpbbforum.com/community/posting.php?mode=reply&f=' + forumid + '&t=' + topicid) sid = stringBetween('style.php?sid=', '&id=2', d.content) lastclick = stringBetween('lastclick" value="', '" />', d.content) creation_time = stringBetween('creation_time" value="', '" />', d.content) form_token = stringBetween('form_token" value="', '" />', d.content) topic_cur_post_id = stringBetween('topic_cur_post_id" value="', '" />', d.content) payload = {'addbbcode20':'100', 'message':message, 'topic_cur_post_id':topic_cur_post_id, 'lastclick':lastclick, 'post':'Submit', 'attach_sig':'on', 'creation_time':creation_time, 'form_token':form_token} headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1', 'Referer':'http://examplephpbbforum.com/community/posting.php?mode=reply&f=' + forumid + '&t=' + topicid, 'Content-Type':'application/x-www-form-urlencoded', 'Connection':'Keep-alive', 'Host':'examplephpbbforum.com', 'Accept-Language':'en-us,en;q=0.5', 'Accept-Encoding':'gzip, deflate', 'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'} d = self.r.post('http://examplephpbbforum.com/community/posting.php?mode=reply&f=' + forumid + '&sid=' + sid + '&t=' + topicid, data=payload, headers=headers, allow_redirects=True)
The flow of the traffic is that after the first post, there is a 302 redirect which the request then follows to the page it successfully posted to.
Sometimes it works (maybe 1 in 50 times) but generally it doesn't work. The response is just the page from the first get request. (The post a reply page)
It should redirect to the posted message pretty much every time, but it doesn't for some reason.
If I sniff the traffic with wireshark it seems the response is generally sent with a URL to redirect to, but it seems something goes wrong there.
Anyone know what might be causing this to fail in the first place and how to fix it?
Results 1 to 3 of 3
- 17 Mar. 2012 06:40pm #1
Python requests library not following 302 properly?
Visit my blog - http://rosewillcode.tumblr.com/
- 18 Mar. 2012 07:37pm #2
Try putting a referrer on it. Sometimes that's the problem. Other then that I would normally go through and make sure you aren't hitting captcha or anything of the sorts.
- 22 Mar. 2012 08:38pm #3
I have a referrer Tried throwing a referrer on there, and I know its not hitting a CAPTCHA, theres no CAPTCHA on the page.
Visit my blog - http://rosewillcode.tumblr.com/