Cumartesi, Mayıs 26, 2012

Php 5.1.6 json_encode

I was using Spiros Kabasakalis great ajax crud gii template for Yii on a server where PHP is of version 5.1.6. It was giving an error when one creates a new record. Some digging showed that it is because the 5.1.6 version does not include json library which is used in the template code. I installed the json library usein pecl, included ir in php.ini and voila.
Meanwhile I have heard o another library upgradephp which gives PHP 5.3/5.4 special properties to earlier versions of PHP. Now its time to enjoy Kabasakalis' library.

Salı, Mayıs 22, 2012

Python mechanize, beautiful soup and html scraping

I used to have bash,sed,awk,curl,beautifiul  soup  when I was doing html parsing,scraping and automating tasks. I was aware of mechanize but was not using it. Today I decided to give it a try for an automation task and I didnt regret. It was a real fun(!) learning(!) it. Actually I learned a bit of it. It helped me a lot on automating browser requests. Although I didnt yet use it for form handling, I know it has some magical power there too. I also realised that one must use Beautiful Soup with Mechanize. These make an awesome combo worth trying and using. Thanks to all the folks behind them.
here is a snippet
br = mechanize.Browser()

br.open("http://www.site.com")
all_links=[l for l in br.links(url_regex="pattern")]
for i in all_links[5:]:


br.follow_link(i)
temp=br.response().read()
soup=BeautifulSoup(temp)                                                                                                  
link=soup.find('a', href=re.compile("mp3"))   #title\/tt[0-9]*\/"))
if hasattr(link, "href"):
    lin=link['href']
    file=lin.split("/")[-1]
    print file+"----"+lin
    br.retrieve(lin,file)

Here are some helper links
http://stockrt.github.com/p/emulating-a-browser-in-python-with-mechanize/
http://stockrt.github.com/p/handling-html-forms-with-python-mechanize-and-BeautifulSoup/