Ana içeriğe atla

Beautiful Soup get contents of a tag

I was embodying multiple body tags into a huge body tag to make a web page printable more efficiently. I used bash and python BeautifulSoup to get the web pages and parse them. Actually all of the job could have been done with python BeautifulSoup as the name suggests, but because I used some sed stuff previously, I did not attempt to change them into python equivalents (this requires usage of some libraries like urllib2 which I did not use since much time).
By using such a code
str(soup.find('body'))
BeautifulSoup returns all the contents of body including and tags. Because I aggregate these tags into a new one, multiple tags appear in the new html document. This might cause some CSS problems in the presentation of the document, I propose. Thats why I needed to eliminate those superfluous tags. 
This page from stackoverlow has helped me.
 The required code was like that
 
body=body+''.join(map(str,soup.find('body').contents))

I really like  BeautifulSoup as I like sed.

Yorumlar

Bu blogdaki popüler yayınlar

An Awk feature

I was trying to use sub (or gsub) function in awk to delete an \r from the end of line. For that purpose I used dum=sub(/\r/,"",$0) and thought that the variable "dum" will be filled with the changed value of $0. But I realised that sub function only make the change and returns 0,1 accordingly. So I wrote like this sub(/r/,"") ; x=$0;

Centos 6 Postgresql 9.1 Install

Today I have changed my mind and decided to begin learning and using Postgresql. On a Centos 6 host there are some ways of installing Postgresql. I have chosen the easy one: use repos. Here http://yum.pgrpms.org/repopackages.php is the official Postgresql repos for Red Hat related distros. I have chosen the x86_64 rpm for Centos 6 and installed it. Later using yum install postgresql91-server I have installed three packages postgresql91,postgresql91-server and postgresql91-libs which is enough for now to start Postgresql 9.1. Using the instructions at /usr/share/doc/postgresql91-9.1.1/README.rpm-dist , I first did service postgresql-9.1 initdb and then service postgresql-9.1 start.Voila! Now let the journey begin! Edit:  Here is another place to look instructions http://www.postgresonline.com/journal/archives/203-postgresql90-yum.html

OpenBSD Nfsen install

I was  trying to find a solution for network monitoring as a friend's request. OpenBSD is the firewall. I saw the Nfsen packages and installed to a 5.2 server. After some effort trying to understand what is going on behind, I realised that Netflow is a protocol by Cisco designed to export router traffic to outside. And OpenBSD has a support of it with pflow interface. First I made a new pflow0 interface like ifconfig pflow0 flowdst 127.0.0.1:9998 flowsrc 127.0.0.1 up Then made some configuration changesto /etc/nfsen.conf and /var/www/htdocs/nfsen/conf.php. Because Apache in OpenBSD runs in chrooted by default, if you dont make the necessary changes in these pages you got an error stating  ERROR: nfsend connect() error: Permission denied!  ERROR: nfsend - connection failed!!  ERROR: Can not initialize globals! which is because apache cant reach the communication socket which is initially  located at $PIDDIR/nfsen.comm namely /var/db/nfsen/run direc...