HowTo retrieve URL content using Python

With the following code you can fetch any content from web using http or ftp:

def fetchContent(self, url):
try:
fetchResult = urlfetch.fetch(url)
except:
return 'Exception: Not an URL'
if fetchResult.status_code == 200:
encTypeRaw = fetchResult.headers['content-type']
encStartIdx = encTypeRaw.find('=')
encType = encTypeRaw[encStartIdx + 1:]
fetchResultContent = fetchResult.content
return fetchResultContent.decode(encType)
return 'Warning: URL could not be retrieved successfully'

Comments

Post a Comment

Popular posts from this blog

Eclipse sucks, so use NetBeans!

Eclipse is the most ugliest tool I have ever used: It has more bugs than Windows 95 pre-alpha Edition ever had. The shortcuts used in Eclipse are used nowhere else . So is the search method implemented inconsequently because there are a bunch of search UIs and almost all of them you will never use but you see them all the time.   If you google for " eclipse sucks " you will find about 1.2 million hits. It seems that many people have suffered from this IBM product. In fact it cost me approx. ten years of my life time working six months with Eclipse and many swear words. There are also many former Eclipse users that have switched over to NetBeans and have not regret this decision. And there are many articles in the blogosphere which were written by frustrated Eclipse users to work off the aggression they gathered using this IDE. And is there any good Eclipse alternative? Yes: NetBeans - for everybody who is fed up with Eclipse!   If you want to tell the w...
Read more

Tuning ext4 for performance with emphasis on SSD usage

The obligatory disclaimer: The more performance you gain, the more data integrity you loose! But if you want to tune something like a file system, I strongly assume that you know what you are doing :) These tips apply to all major Linux distributions like Fedora/Red Hat, OpenSUSE, and Ubuntu that are already using ext4 or are going to make ext4 their default file system. As write operations on Solid State Disks (SSD) are expensive the tips below are focused on SSD usage but also apply to HDD usage as well . My Ubuntu 9.04 Jaunty /etc/fstab after a fresh installation: [...] UUID=d818ddf9-ff01-e21a-a67d-3ceab43a9e2b / ext4 relatime,errors=remount-ro 0 1 UUID=0d339122-74e0-e0ea-805a-7879b1fa3172 /home ext4 relatime 0 2 [...] My tuned Ubuntu 9.04 Jaunty /etc/fstab : [...] UUID=d818ddf9-ff01-e21a-a67d-3ceab43a9e2b / ext4 noatime,barrier=0,nobh,commit=100,nouser_xattr 0 1 UUID=0d339122-74e0-e0ea-805a-7879b1fa3172 /home ext4 noatime,barrier=0,data=writeback,nobh,commit=100,nouser_xattr 0 2...
Read more

Tuning ext3 for performance without losing its data integrity potential

Ext3 as the by journaling capability enhanced successor filesystem of ext2 has quite conservative default parameters which ensure an adequate data integrity after crashes. But if you make backups and using a notebook you will maybe pay a greater attention to the read/write speed of your HDD than an exaggerated and performance wasting data integrity. In the following you can see the settings which I am using for my LUKS encrypted partitions accommodated by a LVM . You have to alter the both files below as followed: /boot/grub/menu.lst #[...] title Ubuntu 7.10, kernel 2.6.22-14-generic root (hd0,1) kernel /vmlinuz-2.6.22-14-generic root=/dev/mapper/vg-starlight ro rootflags=data=writeback,nobh,commit=100 splash usbcore.autosuspend=1 initrd /initrd.img-2.6.22-14-generic quiet /etc/fstab #[...] /dev/mapper/swap none swap sw 0 0 /dev/sda2 /boot ext3 noatime,nobh,d...
Read more