Thursday, July 26, 2007

MultipartPostHandler doesn't work for unicode files

I am in the middle of upgrading the PeerIt client to use the latest and greatest bittorrent code. While I am at it, I have decided to make the sell process easier by having the client post the torrent to the web server and start seeding automatically. I feel this will ease the listing process. In order to do this, I need to post a multipart form with the torrent file. Unfortunately urllib2 does not have support for this. http://pipe.scs.fsu.edu/PostHandler/MultipartPostHandler.py was a nice piece of code I found, in order to use urllib2 to do the posting. I need to still use urllib2 in order to maintain cookies which are used by PeerIt.com login. Unfortunately this piece of code did not work as is for uploading torrent files. This is because torrent files have unicode characters in them. The following is the MultipartPostHandler.py that does work with unicode. The fix was to create the header using StringIO instead of plain old string class...

8 comments:

Anonymous said...

Thanks, it works like a charm! :)

Anonymous said...

OH MY GOD!! YES!!

Thank-you! I've been trying to modify the very same code from the link in your article to accept files with unicode characters, for the last two hours!

I was getting so frustrated, must have searched around 50 mailing lists. Then finally found this site halfway down google for "python urllib2 support for unicode".

It works perfectly. Thanks! This blog is now the first site I've ever bookmarked. Haha.

Anonymous said...

Thak you! Works for me.

Anonymous said...

thanks for the lib, but I have a question for my case. I cannot find the solution, can you help me out? thanx.

Even for logout action, I have to supply the 'data' var as following
[code works]
import MultipartPostHandler, cookielib
cookies = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookies),
MultipartPostHandler.MultipartPostHandler)
urllib2.install_opener(opener)
#in fact, they are read from the logged-in page
utmpnum, utmpkey, utmpuserid='12345678', '2345678', 'myid'
data = urllib.urlencode({
'utmpnum':utmpnum,
'utmpkey':utmpkey,
'utmpuserid':utmpuserid})
myrequest = urllib2.Request('http://host/cgi-bin/bbslogout', data)
[/code works]

If I do not supply 'data', I was told "you are not logged in"

for my case, I finished this code, but the response still says "you are not logged in"!
[code does not work]
import MultipartPostHandler, cookielib
cookies = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookies),
MultipartPostHandler.MultipartPostHandler)
urllib2.install_opener(opener)
bbs_att_url='http://host/cgi-bin/bbsdoupload'

data = {
'utmpnum':utmpnum,
'utmpkey':utmpkey,
'utmpuserid':utmpuserid,
'up' : open("myfile.ico", "rb") ,
}
data=urllib.urlencode(data)
request = urllib2.Request(bbs_att_url, data)
fd=urllib2.urlopen(request)
data=fd.read()
print data
[/code does not work]

Anonymous said...

hey guys, I'm running into a problem myself but I am unsure if this is a MultipartPostHandler error or not.

I am passing a form, it seems to encode just fine with MPhandler but I still seem to get an error stating I am missing form data. all form fields are filled out, my login/cookies seem to be fine, and from what I can see the headers are all set ok except for "Connection" it is set to "Connection: close" yet I am passing this value manually with request.add_header('Connection', 'Keep-Alive").

I have no idea where the error is, I am using fiddler2 to examine the form/header/cookie data and when comparing there is barely any difference from what I am using in my code and what FireFox3 sends.

My other q is why am I finding my local ip address mixed into the Content-Disposition line?? eg:

**START**
--192.168.1.101.1.13524.1217999622.504.1
Content-Disposition: form-data; name="url"

blah
**END**

192.168.1.101 is/was my local address assigned to my pc from my router. is this information needed? from what I see from FireFox3's sent request it is not and if I don't have to include this detail in my form I would prefer not too. :P

Any help with these two problems would be greatly appreciated :D

TIA!!

kcris said...

how do I handle the upload on the server?

thanks,

Anonymous said...

Very helpful, thank you!

Anonymous said...

thx a lot.

arkayne