Beautiful Soup get contents of a tag

I was embodying multiple body tags into a huge body tag to make a web page printable more efficiently. I used bash and python BeautifulSoup to get the web pages and parse them. Actually all of the job could have been done with python BeautifulSoup as the name suggests, but because I used some sed stuff previously, I did not attempt to change them into python equivalents (this requires usage of some libraries like urllib2 which I did not use since much time).
By using such a code
str(soup.find('body'))
BeautifulSoup returns all the contents of body including and tags. Because I aggregate these tags into a new one, multiple tags appear in the new html document. This might cause some CSS problems in the presentation of the document, I propose. Thats why I needed to eliminate those superfluous tags. 
This page from stackoverlow has helped me.
 The required code was like that

body=body+''.join(map(str,soup.find('body').contents))

I really like  BeautifulSoup as I like sed.

Html to Pdf conversion with wkhtmltopdf

Still need to convert html to pdf. I was looking for a batch converter. wkhtmltopdf seems to be a nice converter. It has many options including batch converting (ie. converting multiple html pages into one pdf file) and it does its job nicely except some pitfalls.
When installed from standard Ubuntu repo, a cropped version is installed which does not have the ability of batch conversion. After you install full version from its website, it gives you broader options.
By the way, since the web pages I needed to print to pdf contain many mathematical formulas, wkhtmltopdf conversion ability was limited and I preferred not to use it. Instead I used some bash hacking, and Firefox's print to pdf file option with Stylish addon as I have described in a previous post. 

Some tmux commands

I spent a lot of time in shells. Tmux is a late-discovery must-have for me. It is terminal multiplexer. I used to use screen command. But tmux is next generation of it.
Tmux has 3 basic constructs: session,window and pane. Think session as a global container of insider elements: windows and panes. As described here Ctrl-b is the prefix command. After prefix
s shows sessions and enables navigation
d detaches from that session (not quit)
w lists windows
c create new window
% splits a window horizontally
" splits a window vertically
n passes to next window
p passes to previous window
x kills the current pane
& kills the current window
! breaks the pane out of window (making it a seperate window)
arrow keys enables navigation through panes, windows

$ tmux a 
attaches a previous session.

Have fun.

Centos 7 FirewallD Add/Delete/Log Rich Rule

Centos 7 has firewalld which has many options and a new language "rich language". I was looking for deleting a rich rule syntax which I found at last at fedora project wiki .
Here is a delete rich rule
 firewall-cmd --zone=public --remove-rich-rule='rule family="ipv4" source address="ip adress" reject'
 
which is added previously to the active zone=public by the following

firewall-cmd --add-rich-rule="rule family=ipv4 source address=ip_adress reject"

Here is to list all rich rules at a zone

firewall-cmd --zone=public --list-rich-rules

 If you wish to log packets you might need a rule like this
 firewall-cmd --zone=public --add-rich-rule='rule family="ipv4" source address="ip adress" log prefix="badboy" level="info" limit value="1/m" reject'

For further information please consult to wiki and official Redhat documentation.

Fail2ban Error on Centos 7 KVM instance

On Centos 7 I was struggling with fail2ban on different vm templates eg. OpenVZ and KVM. Previous post is about OpenVZ problems (not only about fail2ban) to which I could not find a solution yet.
Now I would say some words about KVM instance. fail2ban 0.9.2 was running on Centos 7.1.1503 with systemd backend. I was struggling to ban failed ssh attempts with the following config options in jail.local file

[DEFAULT]
backend = systemd
banaction = firewallcmd-ipset
bantime = 3600

[sshd]
enabled = true

Nothing seemed to be wrong with these options and log files. But I was getting an error like this

ERROR   ipset create fail2ban-sshd hash:ip timeout 3600
firewall-cmd --direct --add-rule  filter INPUT 0 -p tcp -m multiport -m set --match-set fail2ban-sshd src -j REJECT --reject-with icmp-port-unreachable
-- stdout: '\x1b[91mwrong priority\nusage: --direct --add-rule { ipv4 | ipv6 | eb } \x1b[00m\n'

As is easily seen, this is an error about ipset command which is triggered by fail2ban's actionstart command found in file /etc/fail2ban/action.d/firewallcmd-ipset.conf

actionstart = ipset create fail2ban- hash:ip timeout
             firewall-cmd --direct --add-rule ipv4 filter 0 -p -m multiport --dports -m set --match-set fail2ban- src -j

ipv4 option in the above command was missing and therefore I kept getting the error. By looking at the error output, it is understood that there must be ipv4|ipv6|eb keywords besides --add-rule. Hence adding the keyword ipv4 temporarily solves the problem. Temporarily because, I dont know what would happen if some ipv6 address try logging in your server.

Update: firewallcmd-ipset only works if your kernel has ipset module support. If your kernel is like

2.6.32-042stab108.8

dont expect firewallcmd-ipset work. So one must use firewallcmd-new action in this case, because it only uses firewalld not kernel ipset module.
The problem I faced was because a wrong template file firewallcmd-ipset.conf.

Centos 7 Minimal OpenVZ Template Problems

I am having trouble with this minimal installs. Although I have heard that Centos 7 has some issues with OpenVZ minimal template, I planned to use it with some struggle.
First problem I encounter is about misconfiguration of network cards in this OpenVZ template. Here is the offical bug report and here is a solution for the template. One needs to make an ethernet device up everytime after reboot if he did not have installed this patch to OpenVZ template using a command like this
ifup ifcfg-venet0   
or a temporary solution is adding 
ARPCHECK="no"
line to /etc/sysconfig/network-scripts/ifcfg-venet0:0 for my case which survives until next reboot.
Second issue is about fail2ban and various system logs. Banning failed ssh attempts is a common task of a system administrator. Fail2ban seems to work in case all ssh logs are streamed to file /var/log/secure. Unfortunately in Centos 7 minimal installs ssh logs are not streamed to this file. Actually there are very few files under /var/log directory. It seems that an important portion of logs are left to be arranged by journalctl command. I installed fail2ban but iIt does not work. After some search I realized that because there are no logs on /var/log/secure file, fail2ban does not recognize failed attempts. After reading this I made sure that the problem is as described. And now it is time to route all ssh logs to /var/log/secure and try to make fail2ban work.

Update: You do not need to stream logs to /var/log/secure or any other file. An option

backend = systemd

to [DEFAULT] section of config file (like jail.local) solves the problem easily. One can use systemd logs directly.

Chrome print web page to pdf font size

Lately I am in a search for converting some web pages to pdf. There are a lot of options, ways to do this. I tried a simple and working one for my case, because there are lots of mathematical symbols on the web page. As you might guess, not all of web page converters are good at math.
Thanks to this post, I found an easy and powerful way. I used Chrome, but I believe this can be done with no hassle with Firefox. Both of the browsers allow saving a web page as a pdf file seamlessly. But in my case I wished to decrease web page's font size a little, to make it fancy looking. I tried zooming out options installed in browser with no help. Because as you might know, those options are futile when printing the page.
I must have found a way to interfere printing progress to make font size smaller. This extension Stylish, which I wasnt aware till I read the post in stackoverflow's site, make this easy. As suggested by the poster I have added some css instructions by using Stylish. It was so simple, just a line made the job.
@media print { body  {font-size: x-small ; } ;}
I have added this line to stylish and when I was printing web page to pdf, this line is activated and I was happy.
Story didnt stop here. I realised later that Chrome adds web address and page numbers as headers and footer to pdf file. You can easily disable them but can not customize them. Namely I wished only page numbers appear on pdf file not the web adress it is originated. After some search I understood that this is not possible with the version of Chrome I am using (actually not possible with any Chrome) so I must switch to my old friend Firefox. Because it has the feature enable customizing headers and footers. Later of course I needed to install Stylish extension to Firefox. But as you guessed it, it was just a click away. So the story seems to end here.
BTW let me mention also a Google service which seems to be helpful when converting web pages to pdf files, Google Cloud print. It has some other features which I dont want to learn. But here is a good place to mention.