tag:blogger.com,1999:blog-80459436102417255882024-03-06T07:58:58.709+00:00MGD BlogMarcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-8045943610241725588.post-28079393258642628502014-08-26T12:01:00.000+01:002014-08-26T12:08:17.123+01:00RHEV and RHEL Clustering - Fencing without RHEVM - The script<div class="separator" style="clear: both; text-align: center;">
<a href="http://upload.wikimedia.org/wikipedia/fr/archive/c/cb/20130513144223!Red_hat_logo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://upload.wikimedia.org/wikipedia/fr/archive/c/cb/20130513144223!Red_hat_logo.png" height="104" width="320" /></a></div>
<br />
<br />
Following up on the blog post from a few weeks ago (<a href="http://marcosgildavid.blogspot.com/2014/08/rhev-and-rhel-clustering-fencing.html" target="_blank">here</a>) I finally got around to creating a fence script that allows for fencing a VM without an available RHEV manager.<br />
<br />
<br />
I've placed the script in my GitHub <a href="https://github.com/marcosgildavid/RedHat_Clustering/blob/master/fence_rhev_nomgt" target="_blank">HERE</a>, you need to copy it to <span style="font-family: Courier New, Courier, monospace;">/usr/sbin</span> and give it execution permission.<br />
<br />
<br />
How does it work?<br />
<br />
I replicated the fence_virtsh script and changed the code to add the necessary commands.<br />
The list of hosts where the script checks for the VM is passed in the "<span style="font-family: Courier New, Courier, monospace;">ipaddr</span>" field.<br />
<br />
Since this is a custom fencing script you cannot configure it directly in Luci, you need to edit the <span style="font-family: Courier New, Courier, monospace;">cluster.conf</span> file manually.<br />
<br />
The fence device can be added to a node like this:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><clusternode name="server1" nodeid="1"></span><br />
<span style="font-family: Courier New, Courier, monospace;"> <fence></span><br />
<span style="font-family: Courier New, Courier, monospace;"> <method name="RHEV-NOMGT"></span><br />
<span style="font-family: Courier New, Courier, monospace;"> <device name="rhev-nomgt" port="Linux-Serv1"/></span><br />
<span style="font-family: Courier New, Courier, monospace;"> </method></span><br />
<span style="font-family: Courier New, Courier, monospace;"> </fence></span><br />
<span style="font-family: Courier New, Courier, monospace;"></clusternode></span><br />
<br />
The fence device itself is define like this:<br />
<br />
<<span style="font-family: Courier New, Courier, monospace;">fencedevice agent="fence_rhev_nomgt" ipaddr="192.168.1.1,192.168.1.2" login="root" name="rhev-nomgt" passwd="password"/></span><br />
<br />
In the <span style="font-family: Courier New, Courier, monospace;">clusternode </span>fence method definition, the "<span style="font-family: Courier New, Courier, monospace;">port</span>" is the name of VM in the RHEV system.<br />
<br />
The "<span style="font-family: Courier New, Courier, monospace;">ipaddr</span>" parameter, in the fence device, is a list of the hostnames (or ip addresses) of the hypervisors where the VM can run separated by a comma. The login and password refer to the login and password of the root user on the hypervisors. I know, this is not very safe but the hypervisors don't allow the creation of other users and for my scenario this won't be an issue.<br />
<br />
Example <span style="font-family: Courier New, Courier, monospace;">cluster.conf</span> for a two-node cluster:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><?xml version="1.0"?></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><cluster config_version="1" name="RHEVCLUS"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <clusternodes></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <clusternode name="server1" nodeid="1"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <fence></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <method name="RHEV"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <device name="rhev-nomgt" port="Linux-Serv1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </method></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </fence></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </clusternode></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <clusternode name="server2" nodeid="2"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <fence></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <method name="RHEV"></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <device name="rhev-nomgt" port="Linux-Serv2"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </method></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </fence></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </clusternode></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </clusternodes></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <cman expected_votes="1" two_node="1"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <fencedevices></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> <fencedevice agent="fence_rhev_nomgt" ipaddr="172.18.56.251,172.18.56.252" login="root" name="rhev-nomgt" passwd="password"/></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> </fencedevices></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"></cluster></span><br />
<br />
<br />
<b>PLEASE TAKE NOTE:</b><br />
This fencing method is intended as a fail-safe option when there is no other possible fencing option available and to ensure the cluster doesn't halt in specific situations where there RHEV Manager isn't available. <b><span style="color: red;">This should no be used as a primary fencing method!</span></b> And please note that it can cause issues in databases such as loss of data and file corruption. To use this method you should be fully aware of the risks.<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-23307390107785644422014-08-11T20:38:00.001+01:002014-08-27T17:48:47.241+01:00RHEV and RHEL Clustering - Fencing without RHEVM<div class="separator" style="clear: both; text-align: center;">
<a href="http://upload.wikimedia.org/wikipedia/fr/archive/c/cb/20130513144223!Red_hat_logo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://upload.wikimedia.org/wikipedia/fr/archive/c/cb/20130513144223!Red_hat_logo.png" height="104" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
UPDATE: I've added the script see the post: <a href="http://marcosgildavid.blogspot.pt/2014/08/rhev-and-rhel-clustering-fencing_26.html">here</a>.<br />
<br />
<br />
For an upcoming project I'll be using a Red Hat Cluster inside a RHEV environment.<br />
At first glance I didn't see any problems since RHEL's High Availability add-on already includes a fencing script for the RHEV-M.<br />
But what happens when the RHEV-M is down or unresponsive and the cluster need to fence one of the nodes?<br />
<br />
This could mean trouble since the cluster would stop every service it manages resulting in a potential downtime for our applications.<br />
<br />
After some research I've come up with a possible solution that allows for the fencing of a VM without a RHEV-M.<br />
<br />
The process is quite simple but needs a few steps:<br />
<br />
1 - Get a list of all the hypervisors inside your RHEV system where the VM can run<br />
<br />
2 - For each of these hosts do the following operations until we find the VM:<br />
<br />
<ul>
<li>Connect to the host as root</li>
<li>Check if there is a QEMU process for our VM in our current host. If there is proceed with the following commands, if not then try the next hypervisor.</li>
<li>Create a new set of credentials to interact with libvirt: </li>
<span style="font-family: Courier New, Courier, monospace;">saslpasswd2 -p -a libvirt fenceagent</span><br />
<br />
<span style="font-family: inherit;">(fenceagent is a username and this command will ask for a password)</span>
</ul>
<ul><br />
<li>Restart the VM with the following command:</li>
<span style="font-family: Courier New, Courier, monospace;">virsh qemu-monitor-command --hmp VM_NAME system_reset</span><br />
<br />
<span style="font-family: inherit;">(Change VM_NAME for the name of the VM as it appears on RHEV)</span>
</ul>
<ul>
<li>Remove the user you created.</li>
<li>Log off from the hypervisor</li>
</ul>
<div>
<br /></div>
<div>
In a few days I'll transform this into a Python script so I can add it to the Cluster. </div>
<div>
I've already validated this process manually so I think there will be no major issues with it.</div>
<div>
<br /></div>
<div>
But there is a potential issue, since this requires an iteration over all the hypervisors (or at least until you find the VM) it can that a lot of time if there are lots of hypervisors, but at least your cluster won't go berserk :D</div>
<div>
<br /></div>
<div>
This will also need some extra configuration like a list of Hypervisors where the VM can be run and the VM name also needs to be passed as an argument to the fence script</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
For future reference, I based this "algorithm" in the following information:</div>
<div>
<br /></div>
<div>
<a href="http://comments.gmane.org/gmane.comp.emulators.ovirt.user/6551">http://comments.gmane.org/gmane.comp.emulators.ovirt.user/6551</a></div>
<div>
<br /></div>
<div>
<a href="http://blog.vmsplice.net/2011/03/how-to-access-qemu-monitor-through.html">http://blog.vmsplice.net/2011/03/how-to-access-qemu-monitor-through.html</a></div>
<div>
<br /></div>
<div>
More updates on this to follow.</div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-84096768260851323532014-07-28T09:57:00.002+01:002014-08-10T18:50:41.876+01:00Remembering Red Hat Linux 5.2 (from 1998) - Part I (Installation)<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
<a href="http://en.wikipedia.org/wiki/Red_Hat_Linux" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;" target="_blank"><img alt="" border="0" src="http://upload.wikimedia.org/wikipedia/en/4/49/Redhat_5.2_box.jpg" height="320" title="Red Hat Linux 5.2 Box" width="267" /></a>My first contact with Linux was when I was finishing High-School in 1999. A friend from class gave a copy of<a href="http://www.redhat.com/" target="_blank"> Red Hat</a> Linux 5.2 and a small 20 pages manual.<br />
I remember getting home firing up my "powerful" AMD K6 machine and getting immediately frustrated: I didn't have enough disk space for a dual boot...<br />
<br />
No problem, since I had recently installed my Windows 98 I decided to backup any important files to floppy disks and format the hard drive.<br />
<br />
Before starting the installation process I opened the PC case and took note of most of my Hardware chips since my friend warned me this would be required in the setup process.<br />
<br />
And so I happily proceeded with the installation. Everything was going great until I had to choose my video card... a Riva TNT2 64 which, at the time, wasn't supported. This ruined the experience for me since I was stuck with a 640x480 8 bit display. Since it was my first time running Linux I didn't know enough command-line to get started so I had to accept defeat and re-installed Windows 98 :(<br />
<br />
Although it was a bad first experience, at the time I was very intrigued and excited about the concept of a free operating system and that "regular" people we're joining up to create something completely open-source and that anyone could join in and contribute.<br />
I had no idea that this "community" project would be such a big part of my life :-)<br />
<br />
Anyway, for the sake of nostalgia and to experience the O.S. I never had the change to try, here is a guide to running Red Hat Linux 5.2 on QEMU.<br />
<br />
<br />
1 - Download a Red Hat Linux 5.2 iso image from the Internet Archive:<br />
<a href="https://archive.org/details/redhat-5.2_release">https://archive.org/details/redhat-5.2_release</a><br />
<br />
2 - Download QEMU for your linux distro (use your package management tool) or Windows from here: <a href="http://qemu.weilnetz.de/">http://qemu.weilnetz.de</a><br />
I used version (2.1.0-rc2)<br />
<br />
3 - Extract the contents of the installation package of QEMU or run the installer.<br />
<br />
4 - Create a disk image to use with your Red Hat 5.2 Virtual Machine, I'll be creating a disk image with 1GB since this was more than enough at the time (my first disk had about 850 MB)<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">qemu-img create -f qcow2 redhat52.img 1G</span><br />
<br />
5 - Run qemu with a special vga card (cirrus) that is compatible with these old linux distributions, also use 256M of RAM and attach the cdrom image to the VM:<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">qemu-system-i386 -hda "<path to disk image>\redhat52.img" -cdrom "<path to iso>\redhat-5.2-i386.iso" -m 256 -vga cirrus</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: inherit;">The </span><span style="font-family: Courier New, Courier, monospace;">-vga cirrus</span><span style="font-family: inherit;"> parameter is essential to get a desktop later on.</span><br />
<br />
6 - Begin the installation! I made a small video (~6min) with the installation process. I tried not to complicate things too much and chose a Workstation default configuration. Video below:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='640' height='480' src='https://www.youtube.com/embed/exLceN2YsXw?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
<br />
<br />
On the next post I'll be going over the desktop and software that came with this version.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-17382225727742188702014-03-27T17:09:00.004+00:002014-03-27T17:15:13.202+00:00Moodbar and playlist creation - Part II like music! I listen to music everyday since it helps relax, get inspired and even lifts me up when I have the blues.<br />
<br />
Unfortunately I can't access online music services during the day (damn proxies...) so I have to rely on my own music collection for all my needs.<br />
<br />
You can imagine that listening to the same tracks day in and day out can get a bit tiresome...<br />
<br />
I always tried to use players that had "smart" playlist generation in order to create some nice coherent playlists. However these "smart" playlist creating apps only work well to find similar artist/bands to the one you're listening to or to suggest popular tracks from other similar artists/bands.<br />
<br />
That's great when you have a huge music collection with lots of artists/bands and albums or for web services that have artists and bands you never heard of...<br />
<br />
Besides, different tracks from the same artist/bands don't always have the same "mood".<br />
I'll illustrate this with an example from my favourite band: Pearl Jam<br />
<br />
Listen to the following songs:<br />
<br />
Do the Evolution: <a href="http://www.youtube.com/watch?v=aDaOgu2CQtI">http://www.youtube.com/watch?v=aDaOgu2CQtI</a><br />
and<br />
Sirens: <a href="http://www.youtube.com/watch?v=qQXP6TDtW0w">http://www.youtube.com/watch?v=qQXP6TDtW0w</a><br />
<br />
They are in completely different contexts and can easily have been from two distinct bands.<br />
This is something that usually happens within a band, songs that translate into completely different states of mind and that can really mess up a playlist and your mood :-)<br />
<br />
So how can you make this less of a problem?<br />
<br />
A few years ago I was introduced to a great music player for the KDE desktop: Amarok (<a href="http://amarok.kde.org/">amarok.kde.org</a>). Amarok had a nice feature that almost went unnoticed but had great potential, the moodbar.<br />
<br />
What is a moodbar you might ask?<br />
Read this: <a href="http://blog.randomprocesses.net/2007/02/moodbar.html">http://blog.randomprocesses.net/2007/02/moodbar.html</a><br />
and this: <a href="http://cratoo.de/amarok/ismir-crc.pdf">http://cratoo.de/amarok/ismir-crc.pdf</a><br />
and maybe this as well: <a href="http://userbase.kde.org/Amarok/Manual/Various/Moodbar">http://userbase.kde.org/Amarok/Manual/Various/Moodbar</a><br />
<br />
Making a quick summary, a moodbar is a graphical representation of the different frequencies in a song as they occur over time.<br />
<br />
It looks something like this for the songs I mentioned earlier:<br />
<br />
<div style="text-align: center;">
<b>Moodbar for "Sirens"</b></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWFHhZpqle7dSR_HImwwU1FmgNsb-XzknfuP1c1IZBJ6EwC_21a7dIhD5XTfcMcwa6HljBHn3X-M60J-SlDQBYHMjTRMihi98hD1Sspl0QJsuVvjoYOMejjcl8fow-J25jB0Om1Hd5lF2u/s1600/moodbar_sirens_song.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWFHhZpqle7dSR_HImwwU1FmgNsb-XzknfuP1c1IZBJ6EwC_21a7dIhD5XTfcMcwa6HljBHn3X-M60J-SlDQBYHMjTRMihi98hD1Sspl0QJsuVvjoYOMejjcl8fow-J25jB0Om1Hd5lF2u/s1600/moodbar_sirens_song.png" height="40" width="400" /></a></div>
<div class="" style="clear: both; text-align: center;">
<br /></div>
<div class="" style="clear: both; text-align: center;">
<b>Moodbar for "Do the Evolution"</b></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVRMHATJhxOZBsJT-ax1ceLiNt84BI4B1hDHXwDh1rW5kud8zEeR8FBoxTxg9V_xp3S2A_vTn322qPpd_N56HN4LCMRmb9-mv4yyX_v-RRWRNnYniTTGaPNvdTu4SM4Q8dkV1wt39DvGV3/s1600/moodbar_evolution_song.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVRMHATJhxOZBsJT-ax1ceLiNt84BI4B1hDHXwDh1rW5kud8zEeR8FBoxTxg9V_xp3S2A_vTn322qPpd_N56HN4LCMRmb9-mv4yyX_v-RRWRNnYniTTGaPNvdTu4SM4Q8dkV1wt39DvGV3/s1600/moodbar_evolution_song.png" height="40" width="400" /></a></div>
<div class="" style="clear: both; text-align: center;">
<br /></div>
<br />
You can easily identified when something happens in the song since the vertical bars are ordered with the song's duration. You can also see that the two moodbars are very different in overall aspect/hue, just like the songs.<br />
<br />
So this got me wondering if we could use this to select similar songs....<br />
<br />
<br />
Turns out you actually can... :D<br />
<br />
I used a simple Python script to read moodbar files generated by my current music player, Clementime (<a href="http://www.clementine-player.org/">http://www.clementine-player.org/</a>) which is based on Amarok, and select songs that have a similar "mood" to a given song.<br />
<br />
I started by creating a playlist of Bands/Artists I wanted to listen to and then run the script to process the files in that list and created a new list of songs that had a similar mood to my selected song.<br />
<br />
<br />
This is what I came up with:<br />
<br />
<br />
<div style="text-align: center;">
<b>Playlist based on "Sirens" (first 20 songs):</b></div>
<br />
<div style="text-align: center;">
Pearl Jam - Sirens</div>
<div style="text-align: center;">
Rage Against The Machine - Year Of Tha Boomerang</div>
<div style="text-align: center;">
The Black Keys - I Got Mine</div>
<div style="text-align: center;">
Pearl Jam - Supersonic</div>
<div style="text-align: center;">
Nirvana - Spank Thru</div>
<div style="text-align: center;">
Pearl Jam - Swallowed Whole</div>
<div style="text-align: center;">
The Black Keys - The Breaks</div>
<div style="text-align: center;">
The White Stripes - Screwdriver</div>
<div style="text-align: center;">
Rage Against The Machine - Down Rodeo</div>
<div style="text-align: center;">
Ramones - Rock 'N' Roll High School</div>
<div style="text-align: center;">
The White Stripes - Black Math</div>
<div style="text-align: center;">
The White Stripes - Instinct Blues</div>
<div style="text-align: center;">
Nirvana - Come As You Are</div>
<div style="text-align: center;">
Pearl Jam - Mind Your Manners</div>
<div style="text-align: center;">
Pearl Jam - Even Flow</div>
<div style="text-align: center;">
Ramones - Sheena Is A Punk Rocker</div>
<div style="text-align: center;">
Pearl Jam - Why Go</div>
<div style="text-align: center;">
Nirvana - Beeswax</div>
<div style="text-align: center;">
Rage Against The Machine - Down Rodeo</div>
<div style="text-align: center;">
The Black Keys - Nova Baby</div>
<br />
<div style="text-align: center;">
And the combined moodbar look like this:</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiljl3WFWHum3Eg5IM4J-QPhBpL7AcfQwtcNvqwEfj8u3c3tM_dUNAWRgPP_XueE5YqyVa84Z9zrBZweeUSVXJ6jFrqKOtf_kAsWJIuDDQlylRAGTwhqz16NjULUkYaSh7ZU83i5XCweByF/s1600/moodbar_sirens_pl.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiljl3WFWHum3Eg5IM4J-QPhBpL7AcfQwtcNvqwEfj8u3c3tM_dUNAWRgPP_XueE5YqyVa84Z9zrBZweeUSVXJ6jFrqKOtf_kAsWJIuDDQlylRAGTwhqz16NjULUkYaSh7ZU83i5XCweByF/s1600/moodbar_sirens_pl.png" height="80" width="400" /></a></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
<b>Playlist based on "Do the Evolution" (first 20 songs):</b></div>
<br />
<div style="text-align: center;">
Pearl Jam - Do The Evolution</div>
<div style="text-align: center;">
The White Stripes - Now Mary</div>
<div style="text-align: center;">
The White Stripes - Jack The Ripper</div>
<div style="text-align: center;">
Nirvana - Rape Me</div>
<div style="text-align: center;">
The White Stripes - Hotel Yorba</div>
<div style="text-align: center;">
The Black Keys - Stack Shot Billy</div>
<div style="text-align: center;">
The Black Keys - Keep Me</div>
<div style="text-align: center;">
Pearl Jam - State of Love and Trust</div>
<div style="text-align: center;">
Led Zeppelin - Dazed And Confused</div>
<div style="text-align: center;">
Pearl Jam - Corduroy</div>
<div style="text-align: center;">
Temple Of The Dog - Pushin Forward Back</div>
<div style="text-align: center;">
Nirvana - Moist Vagina</div>
<div style="text-align: center;">
Pearl Jam - Arc</div>
<div style="text-align: center;">
Pearl Jam - Help Help</div>
<div style="text-align: center;">
Rage Against The Machine - Microphone Fiend</div>
<div style="text-align: center;">
Rage Against The Machine - Sleep Now In The Fire</div>
<div style="text-align: center;">
The White Stripes - I Think I Smell A Rat</div>
<div style="text-align: center;">
Rage Against The Machine - Microphone Fiend</div>
<div style="text-align: center;">
Nirvana - Radio Friendly Unit Shifter</div>
<div style="text-align: center;">
The Black Keys - Just Got To Be</div>
<br />
<div style="text-align: center;">
<b>Moodbar for the playlist:</b></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-wWArK7aujApOKN37C7l1nlB74op9Q6Dr2X4jxq6O8s_mwse7kcnDCHCPhYC6au1BbcYlq01g_H01h1QYt_wKHXCvzISI0ocJ6AVhkcxAbxfFKzTi1qVXq7DHs4nrcRf1X1ka9kbbHrvL/s1600/moodbar_evolution_pl.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-wWArK7aujApOKN37C7l1nlB74op9Q6Dr2X4jxq6O8s_mwse7kcnDCHCPhYC6au1BbcYlq01g_H01h1QYt_wKHXCvzISI0ocJ6AVhkcxAbxfFKzTi1qVXq7DHs4nrcRf1X1ka9kbbHrvL/s1600/moodbar_evolution_pl.png" height="80" width="400" /></a></div>
<div style="text-align: center;">
<br /></div>
<br />
Not bad, the process can be refined a bit but you can immediately see that the overall hue of the playlists is different and so are the songs but they are similar in overall sound.<br />
<br />
<br />
<br />
I'll post the Python screen in another post since I still have to clean it up and optimize some stuff.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com1tag:blogger.com,1999:blog-8045943610241725588.post-68765819071588964192014-02-26T15:54:00.000+00:002014-02-27T09:24:22.951+00:00First adventures in PythonOne of my goals for the next couple of months is to learn Python. I guess this is long overdue and I should probably have picked it up a few years ago... Oh well, never too late to start...<br />
<br />
Since I have an issue with following basic tutorials and writing code that has no practical use, I decided to create a small app to monitor databases.<br />
<br />
Nothing to fancy, just a listing a couple of running SQL queries, table locks and user sessions.<br />
The nice part is that it can do this for three different database systems: Oracle, MySQL and PostgreSQL (this one is not yet implement).<br />
<br />
I placed all of the code on github (<a href="https://github.com/marcosgildavid/dbmonitor">HERE</a>) so you can check it out by yourself.<br />
<br />
<br />
How does it work?<br />
<br />
Well I created a simple interface using ncurses the is divided into 4 areas: Header, Running Queries, Current DB Locks, Current Sessions.<br />
<br />
The interface is the same for all database systems and the information to display is the same (within each DB's possibilities).<br />
<br />
Here are some screenshots:<br />
<br />
MySQL Database:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4oJPBvoN34Q3fuPrhKaoiLq_yK6ptAXFqNx2MMkAEHmShUVaKUOdLboy8JT-5D759p0QdTHQYSjYKO_WqyBe_wmz7IZ5HimghMOXJ-XgnJKxtME7onWG5rtqEm4hWipn_iHj9kjYvGF9M/s1600/main_screen_MySQL.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4oJPBvoN34Q3fuPrhKaoiLq_yK6ptAXFqNx2MMkAEHmShUVaKUOdLboy8JT-5D759p0QdTHQYSjYKO_WqyBe_wmz7IZ5HimghMOXJ-XgnJKxtME7onWG5rtqEm4hWipn_iHj9kjYvGF9M/s1600/main_screen_MySQL.png" height="296" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Oracle database:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1aLlqUQuZdiTJFs031hjW5aeJs4ZvFxIdQBCtOBepITlhVb70m4aJFh3u-gB7fCd2FZtE6-4ceGuiNkKMXF81jW4tU5IGlUhpwxnrCWnMcM_09NmPwlY_6kXdeejJQzLIt9alB7uPiJ6a/s1600/main_screen_Oracle.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1aLlqUQuZdiTJFs031hjW5aeJs4ZvFxIdQBCtOBepITlhVb70m4aJFh3u-gB7fCd2FZtE6-4ceGuiNkKMXF81jW4tU5IGlUhpwxnrCWnMcM_09NmPwlY_6kXdeejJQzLIt9alB7uPiJ6a/s1600/main_screen_Oracle.png" height="298" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
It currently is in a very alpha stage, and there are some things I need to polish. For instance the terminal it runs on needs to have at least 51 lines (which is easily changed in the code), there are few exception handling, etc...<br />
But there also some nice things: long running queries (over 5 minutes) should appear in RED, the query text is divided into two lines so you can more easily identify bad queries.<br />
<br />
<br />
The information is constantly refreshing (currently once per second) and it should have a low impact on your database since the queries it runs are on internal database tables/views.<br />
<br />
I plan on adding some more features like system/db load, traffic, query cache hit, key efficiency, etc....<br />
<br />
So stay tuned for more updates...<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-32362357239586121052014-02-21T16:16:00.001+00:002014-08-11T23:15:16.463+01:00Linux script to monitor an Apache Tomcat WebApp<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://tomcat.apache.org/images/tomcat.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://tomcat.apache.org/images/tomcat.png" /></a></div>
<br />
Sometimes the world is against us and stuff goes wrong.<br />
This can happen a lot with Tomcat and bad behavioured WebApps.<br />
<br />
To mitigate crashes and unresponsive WebApps sometimes it is useful to check if they are working or that they haven't crashed Tomcat... (not my fault! :-))<br />
<br />
Since I was running a Tomcat server on a Cluster system I needed a script that could check the status of a particular webapp and if tomcat was also alive.<br />
<br />
I came up with <a href="https://drive.google.com/file/d/0B5v28FoE-4LUeGtLVWplNHBqOEk/edit?usp=sharing">THIS</a>, a simple script that starts/stops a WebApp, checks if it is running and even checks if Tomcat hasn't died...<br />
<br />
<br />
So how does it work?<br />
<br />
First we need to setup some variables:<br />
<br />
The first one tells the script where the webapp is running, this can be change to include a specific IP Address (Virtual address used in clusters for instance)<br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_SERVER=`hostname`</span><br />
<br />
Next is the script that controls tomcat (if you aren't using one you can change the code to use the built in startup and shutdown commands)<br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_SCRIPT=/etc/init.d/tomcat</span><br />
<br />
This is the port where Tomcat is running, usually its 8080<br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_PORT=8080</span><br />
<br />
This port is used to check if Tomcat is running.<br />
Why use the Shutdown port instead of the normal port? Because I had an issue where the normal port was open but there was no shutdown port and Tomcat was not running properly...<br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_SHUTDOWN_PORT=8005</span><br />
<br />
We also need the credentials for a user with permission to start/stop webapps.<br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_USER=user</span><br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_PASS=pass</span><br />
<br />
Next we need to define the WebApp name to be controlled. There are two ways to do this, either by the script, assuming we use something like: tomcat_MyWebApp or tomcat-MyWebApp.<br />
Or just set it manually.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">#The Line below is used to extract the Webapp name from the script</span><br />
<span style="font-family: Courier New, Courier, monospace;">#the webapps name is assumed to the last sequence of alphanumeric</span><br />
<span style="font-family: Courier New, Courier, monospace;">#chars in the scriptname (for instance: tomcat_MyWebApp -> WA name=MyWebApp)</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#TOMCAT_WA_NAME=`echo $0|egrep -o "[a-zA-Z0-9]*$"`</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#Or you can manually set the WS name here </span><br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_WA_NAME=MyWebApp</span><br />
<br />
<br />
Finally we need to define the installation dir for Tomcat, this should be the same as the CATALINA_DIR in Tomcat's configuration. This is used to check the log for Memory errors.<br />
<span style="font-family: Courier New, Courier, monospace;">#Tomcat dir aka CATALINA_DIR</span><br />
<span style="font-family: Courier New, Courier, monospace;">TOMCAT_DIR=/usr/local/jakarta-tomcat</span><br />
<br />
The script then uses curl to invoke Tomcat start/stop WebApps commands.<br />
<br />
<br />
The initial version of the status routine just opened the WebApp default page to see if it got some text.<br />
Later, I added another verification to see if Tomcat hadn't died with a Permgen error (memory issues, bad app..).<br />
<br />
You can extend the status routine to check for internal variables in your WebApp and determine it's status or any other condition that signals a fault and requires a restart.<br />
<br />
<br />
To make this restart webapps and Tomcat in scenarios where you don't have a cluster you can use the watchdog template script on my blog (<a href="http://marcosgildavid.blogspot.pt/2013/12/linux-rhel-simple-application-watchdog.html">This one!</a>) to make a periodic checks and mimic cluster behaviour.<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-20672154665685070692014-02-19T11:02:00.001+00:002014-08-26T17:57:26.731+01:00Getting an ILI9341 SPI screen working on the RaspberryPII recently bought a small SPI screen from ebay which uses the ILI9341 controller.<br />
The screen is identical to the one made by ElecFreaks (<a href="http://www.elecfreaks.com/store/22%E2%80%99-tft-lcd-tft0122sp-p-672.html">This one!</a>).<br />
Although I got mine really cheap from eBay....<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzUcAfCOyBbAPxBfsztxulGfd5l5Wg8vEizxSFVRHnnMjq8cId6oauL6nvE4TX3fQV3Q5Oap6UP6s-fGp3MDawlCzE3D1q4JYg2mDVqUS52EALIXIvy7hgMyu7aoODHXLElnD5iYZynsOm/s1600/2.2SP_A.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzUcAfCOyBbAPxBfsztxulGfd5l5Wg8vEizxSFVRHnnMjq8cId6oauL6nvE4TX3fQV3Q5Oap6UP6s-fGp3MDawlCzE3D1q4JYg2mDVqUS52EALIXIvy7hgMyu7aoODHXLElnD5iYZynsOm/s1600/2.2SP_A.JPG" height="231" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I decided to use notro drivers since they are flexible and already provide a driver for the ILI9341. You can find the source code and install instructions <a href="https://github.com/notro/fbtft">here</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
I actually compiled a custom kernel where I added his drivers.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
After compilation and module installation I just had to figure out the parameters I needed to use.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
But first we still need to hook it up to the RaspPI :-)</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I used the following connections, also from notro's Wiki</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I used the wiring schema for an Adafruit LCD, check the original table <a href="https://github.com/notro/fbtft/wiki/LCD-Modules#wiki-adafruit-22">here</a>.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<table align="center" border="1" cellpadding="3" cellspacing="0">
<thead>
<tr><th>Display</th><th>Driver gpio name</th><th>Raspberry Pi</th><th>Remark</th></tr>
</thead>
<tbody>
<tr><td>BL</td><td>led</td><td>GPIO18</td><td></td></tr>
<tr> <td>SCK</td><td></td><td>SCLK</td><td></td></tr>
<tr> <td>MISO</td><td></td><td></td><td>Used by SD-card</td></tr>
<tr> <td>MOSI</td><td></td><td>MOSI</td><td></td></tr>
<tr> <td>CS</td><td></td><td>CE0</td><td>Chip Select</td></tr>
<tr> <td>SDCS</td><td></td><td></td><td>SD-card Chip Select</td></tr>
<tr> <td>RST</td><td>reset</td><td>GPIO25</td><td></td></tr>
<tr> <td>D/C</td><td>dc</td><td>GPIO24</td><td></td></tr>
<tr> <td>VIN</td><td></td><td>3.3V</td><td></td></tr>
<tr> <td>GND</td><td></td><td>GND</td></tr>
</tbody>
</table>
<br />
Aside from a normal SPI connection, please notice the BL(led), RST(reset) and D/C(dc) pins since these will be used as parameters for the module.<br />
<br />
<br />
Once you have everything hooked up, compiled and installed you can test out the screen by loading the module:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">sudo modprobe fbtft_device custom name=fb_ili9341 gpios=reset:25,dc:24,led:18 speed=16000000 rotate=90 bgr=1</span><br />
<br />
If you change one of the pins above, please change the gpios parameter accordingly.<br />
Also, you have the rotate option to put the screen in landscape mode, by default it will be in portrait mode.<br />
The bgr=1 option is to exchange the blue and red colors, by default the colors don't appear correctly and this option is necessary.<br />
Once you load the module nothing will happen on your screen (this made think it wasn't working...) you can test it out by either sending the console to the framebuffer with:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">con2fbmap 1 1</span><br />
<br />
(it means, send console 1 to framebuffer 1)<br />
<br />
or using X<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">FRAMEBUFFER=/dev/fb1 startx</span><br />
<br />
<br />
If you want to make the module load automatically, add the following line to /etc/modules:<br />
<span style="font-family: Courier New, Courier, monospace;">fbtft_device custom name=fb_ili9341 gpios=reset:25,dc:24,led:18 speed=16000000 rotate=90 bgr=1</span><br />
<br />
EDIT: For the built-in version, you can add this to the kernel command line:<br />
<span style="font-family: Courier New, Courier, monospace;">fbtft_device.name=fb_ili9341 fbtft_device.gpios=reset:25,dc:24,led:18 fbtft_device.speed=16000000 fbtft_device.rotate=90 fbtft_device.bgr=1 fbtft_device.custom=1</span><br />
<br />
<br />
Hope you find this useful.<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com54tag:blogger.com,1999:blog-8045943610241725588.post-6366999082916307862014-02-03T13:03:00.000+00:002014-02-03T13:16:02.158+00:00Chrome - Create a button to "proxify" a siteSome time ago I created a custom proxy with the help of this tutorial:<br />
<a href="http://www.labnol.org/internet/setup-proxy-server/12890/">http://www.labnol.org/internet/setup-proxy-server/12890/</a><br />
<br />
Is pretty cool since it helps me circumvent some problems I have from incorrect proxy blacklists, to bypass geographically blocked content, etc.<br />
<br />
Recently I found that you can add a bookmark to Chrome that calls some javascript code.<br />
<br />
Putting the two together I came up with a simple Button on my Chrome Bookmark bar that instantly redirects a page to my proxy.<br />
<br />
All you need to do is to create a bookmark, add it to the bookmark bar and use the following text in the URL (all in a single line!)<br />
:<br />
<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">javascript:window.location="http://PROXY_URL/"+window.location.href.substring(window.location.href.search("://")+3);</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<br />
Just replace PROXY_URL with your own custom proxy or you can search the for one.<br />
<br />
The final bookmark url should look like:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">javascript:window.location="http://labnol-proxy-server.appspot.com/"+window.location.href.substring(window.location.href.search("://")+3);</span><br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4zLucy0qANvx31V1EJ7JwfapyRSNN-E4p-XhwUpju3oRaZkIWsDDWb7U3aT3-xZPMSTYKoSoZEizVvPB7GdhyphenhyphenxhxiFU7t3FXiavpKCWSzZ8MAK6cbySIiT4OpmVRi9x6JU0Q4NpUL7ZMn/s1600/proxify0.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh4zLucy0qANvx31V1EJ7JwfapyRSNN-E4p-XhwUpju3oRaZkIWsDDWb7U3aT3-xZPMSTYKoSoZEizVvPB7GdhyphenhyphenxhxiFU7t3FXiavpKCWSzZ8MAK6cbySIiT4OpmVRi9x6JU0Q4NpUL7ZMn/s1600/proxify0.png" height="400" width="350" /></a></div>
<br />
The proxy I mention there is the one that is used as an example in the tutorial link.<br />
<br />
So how do you use it?<br />
<br />
After adding the bookmark you should be able to see it in the bookmark bar.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdRWOQgstUbpTfvGK7rUj3m_R1yZqfwbv-7CL47gIm-AZFUzO-KY8wBQoG36LBjihCWvf3gIiu7sa3GGMvah96v9kPdvyBWRC8MpTUQYQlAuzoxFJUzgSq0ok_DkRSRC9FG3mx82hs7yCa/s1600/proxify1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdRWOQgstUbpTfvGK7rUj3m_R1yZqfwbv-7CL47gIm-AZFUzO-KY8wBQoG36LBjihCWvf3gIiu7sa3GGMvah96v9kPdvyBWRC8MpTUQYQlAuzoxFJUzgSq0ok_DkRSRC9FG3mx82hs7yCa/s1600/proxify1.png" height="252" width="400" /></a></div>
<br />
<br />
When you reach a site that you want to filter to the proxy, just it the button, and It will automatically redirect you:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqZIf4yOGje018Tk9ZnL7U76YFBybxYiTiHgpBs2jIQvwJQ76tYWi3JgrcmVatHBQ9XZDcjroB1QyyYBbjJOHcDo3qhPrUZMaBfWzmKm7nyLEsf7n_agez0GgGEmH-85kOJer_ztvOHqAR/s1600/proxify2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqZIf4yOGje018Tk9ZnL7U76YFBybxYiTiHgpBs2jIQvwJQ76tYWi3JgrcmVatHBQ9XZDcjroB1QyyYBbjJOHcDo3qhPrUZMaBfWzmKm7nyLEsf7n_agez0GgGEmH-85kOJer_ztvOHqAR/s1600/proxify2.png" height="200" width="400" /></a></div>
<br />
<br />
<br />
Of course this means you can do other types of redirections.<br />
<br />
I hope you find this useful!<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com3tag:blogger.com,1999:blog-8045943610241725588.post-87479334927420500692014-01-30T17:47:00.003+00:002014-01-30T17:53:28.291+00:00Importing a VirtualBox VM into RHEVAfter several trials and errors I finally found a "working" method of importing VirtualBox (and probably VMWare and other Virtualizations solutions) VM into a RHEV system.<br />
<br />
What you need:<br />
<br />
1 - Lots of disk space! At least twice the size of your VM's disks.<br />
2 - A VirtualBox installation and the qemu-img program (available in your distro or from <a href="http://www.qemu.org/">www.qemu.org</a>).<br />
3 - A libvirtxml template file (you can get the one I use from <a href="https://drive.google.com/file/d/0B5v28FoE-4LUVVVLbHNHM2dpUEE/edit?usp=sharing" target="_blank">here</a>).<br />
4 - The virt-v2v program installed in one the RHEV machines (or the RHEV manager machine).<br />
<br />
So how do you do it?<br />
<br />
Convert VirtualBox disks to QCOW format (if someone knows how to easily convert them directly to raw please let know!).<br />
You could be tempted to convert the disks directly from VMDK format to RAW with qemu-img, unfortunately I found this is not possible since qemu gives an "unsupported format error".<br />
<br />
Anyway, just follow these steps:<br />
<br />
<br />
Go to the VirtualBox File Menu and select: "Virtual Media Manager".<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipvKeNdU9igau8MIo87aHsXH-of7oacaCoqzTrLr3hDJ4qQr_VrR_DrX8gfp5VPwwFfKuE7kaVB2Jibp3Tz7O38fSXCCmQjkZsnG30WJU9AJHZjpNuKz3bFkgW8OQoOg51W8XaUPdrq2pW/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEipvKeNdU9igau8MIo87aHsXH-of7oacaCoqzTrLr3hDJ4qQr_VrR_DrX8gfp5VPwwFfKuE7kaVB2Jibp3Tz7O38fSXCCmQjkZsnG30WJU9AJHZjpNuKz3bFkgW8OQoOg51W8XaUPdrq2pW/s1600/1.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Then select the first disk and right-click it and select "Copy...", or press the Copy Button:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_EzR8Xe8wBBT5nYhqRhLwh21OS8V5e3RlprsAXfn7GFXckcPFqFrRgsxINOI9A4Md4o9tAV-5NEew5HSXPfuwXpVplwJejkhidN2cp3oJmMgUsYM99_BC5bSlVZ17QgZ7iKcXVU7bmpKb/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_EzR8Xe8wBBT5nYhqRhLwh21OS8V5e3RlprsAXfn7GFXckcPFqFrRgsxINOI9A4Md4o9tAV-5NEew5HSXPfuwXpVplwJejkhidN2cp3oJmMgUsYM99_BC5bSlVZ17QgZ7iKcXVU7bmpKb/s1600/2.png" height="312" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Check if the right file was selected and press Next:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih3YOuAyaYUrgTYl__DAaAJRPxYODDsaJCkPl1Euvg9AQNblbPfkvFAiJq0vF1hyEmuwhFCzn1ZJHBL_MLlGLsXMCY8cNZD0NBVt3N7zzk9Ks3l4JtCPnVDKyIREcMDLzxUDHbvkrCPZC3/s1600/3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih3YOuAyaYUrgTYl__DAaAJRPxYODDsaJCkPl1Euvg9AQNblbPfkvFAiJq0vF1hyEmuwhFCzn1ZJHBL_MLlGLsXMCY8cNZD0NBVt3N7zzk9Ks3l4JtCPnVDKyIREcMDLzxUDHbvkrCPZC3/s1600/3.png" height="337" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Select the QCOW (QEMU Copy-On-Write) format and press Next:</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggDTnGK8Ql7XYWJcKKy7WP-6uAxWPAOE3Q3OqVd3gOcXDNpeQoPxnXho53uKefYfSD8y2kOL1ieDLDO6l3BlrkPSuDw_id8IW7XK2DhKxrzLlC5iMXi31FoV2ngsmFVt-GI5Eu6hiX6Mb6/s1600/4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggDTnGK8Ql7XYWJcKKy7WP-6uAxWPAOE3Q3OqVd3gOcXDNpeQoPxnXho53uKefYfSD8y2kOL1ieDLDO6l3BlrkPSuDw_id8IW7XK2DhKxrzLlC5iMXi31FoV2ngsmFVt-GI5Eu6hiX6Mb6/s1600/4.png" height="337" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Select the output filename and path, finally press Copy:</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjSOWvJo3nJVuWo-Yf_q2dPug6l5UE2-GXPUCNf01f-vuJ0YkZjKQgN142HgCz9RB-KU384k3jAag74kYASgLu4GlizlOu-DEbz_PUhjOUGsh6RPC5W-GRE8QdT11kCLToLCJm-W8UXgEU/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjSOWvJo3nJVuWo-Yf_q2dPug6l5UE2-GXPUCNf01f-vuJ0YkZjKQgN142HgCz9RB-KU384k3jAag74kYASgLu4GlizlOu-DEbz_PUhjOUGsh6RPC5W-GRE8QdT11kCLToLCJm-W8UXgEU/s1600/5.png" height="270" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Wait for it to finish... </div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Once it is done you still need to convert them to the RAW format. This is where qemu-img comes in. Issue the following command to convert a disk to the RAW format:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Courier New, Courier, monospace;">qemu-img convert disk1.qcow -O raw disk1.img</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Repeat for every disk you need converted.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Now, for the real fun part! </div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
Open the libvirtxml file you got from above and edit the following fields:</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><name>VMNAME</name></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><uuid>b8d7388a-bbf2-db3a-e962-12346789abcd</uuid></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><memory>2097152</memory></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"> <type arch='x86' machine='pc'>hvm</type></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
The <b><span style="font-family: Courier New, Courier, monospace;">vmname</span> </b>should be changed to whatever you want, this will be the name that appears on the imported VM.</div>
<div class="separator" style="clear: both; text-align: justify;">
The <span style="font-family: Courier New, Courier, monospace;"><b>uuid </b></span>is not important at this point since it will be changed once the import is completed.</div>
<div class="separator" style="clear: both; text-align: justify;">
You can also change the <span style="font-family: Courier New, Courier, monospace;"><b>memory </b></span>size (in KB).</div>
<div class="separator" style="clear: both; text-align: justify;">
The <b><span style="font-family: Courier New, Courier, monospace;">arch </span></b>property should be set to <span style="font-family: Courier New, Courier, monospace;">x86 </span>or <span style="font-family: Courier New, Courier, monospace;">x86_64 </span>if it has a 32bit or a 64bit guest OS.</div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
In the disk section you need to change a few things.</div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><disk type='file' device='disk'></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"> <driver name='qemu' type='raw' cache='none' io='threads'/></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"> <source file='disk1.img'/></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"> <source dev='/dev/sda'/></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"> <target dev='sda' bus='scsi'/></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><shareable/></span><span style="font-family: 'Courier New', Courier, monospace;"> </span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"></disk></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">Change the </span><b><span style="font-family: Courier New, Courier, monospace;">source file</span></b><span style="font-family: inherit;"> and </span><span style="font-family: Courier New, Courier, monospace;"><b>target dev</b></span><span style="font-family: inherit;"> properties to what you need. In this example the disk1.img is the first disk (sda). </span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">The example XML file I provide has two disks, you can remove the section for the second one or y</span><span style="font-family: inherit;">ou can add more disk sections if you need to.</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">Once you have the file properly </span>set up<span style="font-family: inherit;"> it's time to import it into RHEV.</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">Use the following command:</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<span style="font-family: Courier New, Courier, monospace;">virt-v2v -i libvirtxml -o rhev -osd IP:/DIR --network rhevm VM_libvirt.xml</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">The </span><span style="font-family: Courier New, Courier, monospace;"><b>IP </b></span><span style="font-family: inherit;">and </span><span style="font-family: Courier New, Courier, monospace;"><b>DIR </b></span><span style="font-family: inherit;">should be the IP, or hostname, of the export domain and DIR the NFS dir path (I'm using an NFS export domain, you can change this to reflect you export domain).</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">Check the documentation if you have any doubts.</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">After a couple of minutes or hours, depending on the size of the disks and your network speed, you should have the machine in your export domain. From there you can import it to RHEV.</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: justify;">
<span style="font-family: inherit;">I have used this method on both windows and Linux machines and it seems to work fine!</span></div>
<div class="separator" style="clear: both; text-align: justify;">
<br /></div>
Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com3tag:blogger.com,1999:blog-8045943610241725588.post-37756957886036701392014-01-15T15:12:00.002+00:002014-02-05T12:32:06.167+00:00Windows - Batch file to clean old Log FilesAfter dealing with several hundred GB of log files from Tomcat, I created a simple cleanup script for old log files. This script is meant to be run by a Schedule Task in Windows so its configuration is very simple. It only needs to know three things: Path to search, Type of Files and the number of days to keep the files.<br />
<br />
You can download the batch file <a href="https://drive.google.com/file/d/0B5v28FoE-4LUNjVTazhOenU2V0k/edit?usp=sharing">here</a>, or copy paste the following text:<br />
<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">echo off</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM ********************************</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM * USER VARIABLES *</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">REM CLEAN FILES OLDER THAN NUM_DAYS</span><br />
<span style="font-family: Courier New, Courier, monospace;">set NUM_DAYS=7</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">REM PATH WHERE TO RECURSIVELY SEARCH FOR FILES</span><br />
<span style="font-family: Courier New, Courier, monospace;">set SRCDIR=C:\LOGS</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">REM FILE EXTENSION TO LOOK FOR (*.* FOR ALL FILES)</span><br />
<span style="font-family: Courier New, Courier, monospace;">set EXT="*.log"</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM ********************************</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">echo "Cleaning Files older than %NUM_DAYS% on %SRCDIR%"</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">forfiles /p "%SRCDIR%" /s /m %EXT% /D -%NUM_DAYS% /C "cmd /c del @path"</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">echo "DONE!"</span><br />
<span style="font-family: Courier New, Courier, monospace;">echo on</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: inherit;">Super simple! It requires the forfiles command to be installed (it already comes Windows Server 2003 and onwards, Windows Vista and onwards. Windows XP users have to install it manually)</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">You can used this in conjunction with a schedule task to make the script run automatically at any given day/time. </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">You can also modified it to move the files, zip them or event rename them.</span><br />
<span style="font-family: inherit;">I suppose this can also be done with Windows Powershell, but since I'm more into Linux I don't have a clue on how to do it. If you have a similar Powershell script please share it, or let me know where I can download it from! </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><br /></span>Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-6638405378208250912013-12-20T10:47:00.000+00:002013-12-20T10:47:38.270+00:00Windows - Automatic VPN dialer Batch scriptSometimes I need to work on several customers or environments at the same time. This means that sometimes I need to have several (Windows) VPN connections open at the same time or I need to connect and disconnect from them several times a day.<br />
<br />
In order to simplify the process and to enable me to keep my network settings mostly undisturbed when I connect to VPNs I have created a simple batch file to allow me to quickly connect to a VPN and set the appropriate routes.<br />
<br />
The batch file is available here: <a href="https://drive.google.com/file/d/0B5v28FoE-4LUOVVpUWFnb0RKazA/edit?usp=sharing">vpn_dialer_template.bat</a><br />
<br />
Usually when you connect to a VPN you lose your default gateway and become unable to access your local resources. This can be avoided by configuring the connection not to use the VPN gateway, but this comes with a cost: you have to manually set the required routes yourself.<br />
<br />
In order to make this process simpler and quicker I came up with a batch file that makes the VPN connection and set the routes all by itself. However, It has some "requirements": I only tested it on Windows 7, although it should work on Vista and Windows 8; it has to be run as Administrator since setting the routes requires privilege escalation; to make it effective you have to disable the "use default gateway" option on the VPN connection this implies you have to know the remote gateway address and other routes that you also need.<br />
<br />
I'll guide you through the process of deactivating the remote gateway option:<br />
<br />
First goto "Network and Sharing Center". Next, choose the "Change Adapter Settings" option on the left. Then right-click on the VPN connection and choose "Properties"<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiLrU3QY7NHbNwAvx5aEvysIMa1R3RXrYpGrBKvVL2T7lQhKDE7y2fLuj95FTzY_NvEYpC7Cl0muR6ofo8UThy10sLlOpubVqyAas62GojWPEFAS5PPQcu8sWKOz2GFnexyv4BU8Q1Pqjs/s1600/VPNList.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="92" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiLrU3QY7NHbNwAvx5aEvysIMa1R3RXrYpGrBKvVL2T7lQhKDE7y2fLuj95FTzY_NvEYpC7Cl0muR6ofo8UThy10sLlOpubVqyAas62GojWPEFAS5PPQcu8sWKOz2GFnexyv4BU8Q1Pqjs/s400/VPNList.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5eh8ZaH5jMwrl4uK7ANWRsAOj8WZWXG8ysx2CMFgl8KZUppONK3T-BN9qKZPnQPwKZqMnaXuuRDtJ_850u-g9fpFwxJxhtv1tY3frzkvHT-Nd0xQGadcpD49ScsE3p6Q1VYy0qbB4e0lq/s1600/VPNProperties.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5eh8ZaH5jMwrl4uK7ANWRsAOj8WZWXG8ysx2CMFgl8KZUppONK3T-BN9qKZPnQPwKZqMnaXuuRDtJ_850u-g9fpFwxJxhtv1tY3frzkvHT-Nd0xQGadcpD49ScsE3p6Q1VYy0qbB4e0lq/s400/VPNProperties.png" width="312" /></a></div>
<br />
Then select the Networking tab.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTmAjyC7CgTqmxEDHDaDfSqGymu0S81OMHpLNBLgtPZ7jqe9c5NdbMX3IzcZh2IelVzFycgsij-ZL-vyGH0RcWKRoBvDA7LYwkCZodJLWIQJVLVZCs2aOAVed74VFTewwuaGAIM1gKNtYG/s1600/VPNNetworking.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiTmAjyC7CgTqmxEDHDaDfSqGymu0S81OMHpLNBLgtPZ7jqe9c5NdbMX3IzcZh2IelVzFycgsij-ZL-vyGH0RcWKRoBvDA7LYwkCZodJLWIQJVLVZCs2aOAVed74VFTewwuaGAIM1gKNtYG/s400/VPNNetworking.png" width="312" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
In the Networking tab, select "Internet Protocol Version 4" and then click on Properties:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7uhsKVUjfzookbhjyI3gwrmjEitEGJidu9VYuynIxZIoBVqNFFVZoOUI6gJgd9DArMlsDcGWGuAkvkaZtPak8BJdc5bk_NYvlmAuEp7zphzACBlMb3S0QbQKFIF9HlxY54ANFcVVh7CqE/s1600/VPNNetworkingAdvanced.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi7uhsKVUjfzookbhjyI3gwrmjEitEGJidu9VYuynIxZIoBVqNFFVZoOUI6gJgd9DArMlsDcGWGuAkvkaZtPak8BJdc5bk_NYvlmAuEp7zphzACBlMb3S0QbQKFIF9HlxY54ANFcVVh7CqE/s400/VPNNetworkingAdvanced.png" width="358" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
Click Advanced:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGldAX7j51DRhBh1wtLuiJt8toItMEANr4J6M5dj5Q5c8swoLpj4Xs_OVHEN0cOqXff4adEi8vsFXupxK5aOk-Aew4_0yub2yAeaM70acp1SzCjyx-Z168pT6Jx3XixIaKCyyynNqv6UjL/s1600/VPNDisableGateway.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGldAX7j51DRhBh1wtLuiJt8toItMEANr4J6M5dj5Q5c8swoLpj4Xs_OVHEN0cOqXff4adEi8vsFXupxK5aOk-Aew4_0yub2yAeaM70acp1SzCjyx-Z168pT6Jx3XixIaKCyyynNqv6UjL/s400/VPNDisableGateway.png" width="335" /></a></div>
<br />
Uncheck the option: "Use default gateway on remote network" and click OK->OK->OK.<br />
<br />
<br />
Now the VPN Connection is ready to use with the batch file.<br />
Of course, you can also use this trick on your own.<br />
<br />
<br />
Next I'll explain how the batch file is used.<br />
<br />
<br />
You can configure your setting in the USER VARIABLES section in the beginning of the file:<br />
<span style="font-family: Courier New, Courier, monospace;">REM **************************************</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM USER VARIABLES</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM **************************************</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">SET VPNNAME=VPNNAME</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET VPNUSER=vpnuser</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET VPNPASS=vpnpass</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM If you do not use a Domain then set VPNDOMAIN to empty: SET VPNDOMAIN=</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET VPNDOMAIN=vpndomain</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">REM ROUTES TO ADD AFTER CONNECT</span><br />
<span style="font-family: Courier New, Courier, monospace;">REM EXAMPLE: ROUTE1=route add 10.110.0.0 mask 255.255.0.0 10.255.255.2</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET ROUTE1=</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET ROUTE2=</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET ROUTE3=</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET ROUTE4=</span><br />
<span style="font-family: Courier New, Courier, monospace;">SET ROUTE5=</span><br />
<br />
REM **************************************<br />
<div>
<br /></div>
<div>
You need to change the VPNNAME to the name you gave your VPN.</div>
<div>
The VPNUSER, VPNPASS are the username and password used to connect to the VPN. If you have an issue with having the plain-text password in a file then you should placed it in an encrypted folder (that works for me).</div>
<br />
The VPNDOMAIN is optional and you only need to set it if the VPN connection requires it. Otherwise simply leave it empty.<br />
<br />
By default you can configure the script to add up to 5 routes (more than enough for my needs).<br />
But you can add more with a simple tuning of the file.<br />
<br />
The ROUTE# variables need to contain the actual route command used to add the route.<br />
For example:<br />
<span style="font-family: Courier New, Courier, monospace;">SET ROUTE1=route add 10.110.0.0 mask 255.255.0.0 10.255.255.2</span><br />
<br />
(See the syntax of the route command for further explanations).<br />
<br />
The batch file works as follows:<br />
1 - Connect the VPN with the credentials supplied in the USER VARIABLES section.<br />
2 - If the connection is successful then loop trough all the connections to check for the interface id. This only work properly on more recent versions of Windows as I mentioned earlier. Windows XP for instance uses an hexadecimal notion for the interface ID which totally messes things up.<br />
3 - Go through the ROUTE# variables, add the interface id to the route command and set the route.<br />
4 - Pause until user presses a key.<br />
<br />
<br />
The batch file doesn't disconnect the VPN automatically, but if you take a look at it you can see this option is commented out since I was accidently pressing a key and sending the VPN connection down too often. :D<br />
<br />
In the (unlikely) case you need to add more routes you can add another ROUTE# variable and then duplicate on the "Set route" sections on the last part of the file:<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">echo . </span><br />
<span style="font-family: Courier New, Courier, monospace;">if defined ROUTE# (</span><br />
<span style="font-family: Courier New, Courier, monospace;"> set CMD=%ROUTE#% if %IFNUMBER%</span><br />
<span style="font-family: Courier New, Courier, monospace;"> %CMD%</span><br />
<span style="font-family: Courier New, Courier, monospace;"> IF %ERRORLEVEL% GEQ 1 (</span><br />
<span style="font-family: Courier New, Courier, monospace;"> echo Error Setting %ROUTE#%!</span><br />
<span style="font-family: Courier New, Courier, monospace;"> pause</span><br />
<span style="font-family: Courier New, Courier, monospace;"> goto :disconnect</span><br />
<span style="font-family: Courier New, Courier, monospace;"> )</span><br />
<span style="font-family: Courier New, Courier, monospace;">)</span><br />
<br />
<br />
Just replace the 3 references to the ROUTE# variable with your one variable.<br />
<br />
<br />
Hope you find this useful!<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com2tag:blogger.com,1999:blog-8045943610241725588.post-82977801419503771322013-12-17T14:55:00.001+00:002013-12-17T15:50:48.352+00:00Linux (RHEL) - Simple application watchdog script (aka how to make an application automatically restart)In order to make sure some of applications are running non-stop and required no human intervention to be restarted in case of failures we usually really on a cluster infrastructure (in our case Red Hat Cluster Suite). This works great when you have the hardware to run it but sometimes we also need to make sure we have this behaviour on a standalone machine.<br />
<br />
This is accomplished by using a watchdog script that mimics the behaviour of the cluster (in regards to application monitoring).<br />
<br />
In the new Red Hat EL versions (>6.0) a similar effect can be achieved with upstart respawn option. But the watchdog script has some advantages since it allows for a bit more fine-tuning. Besides, I'm still stuck on RHEL 5 which has no upstart, only sysvinit.<br />
<br />
To make implementation and deployment fast and easy I have a template script which enables me to add monitoring to a script/service in a couple of minutes.<br />
<br />
The template script can be downloaded here: <a href="https://drive.google.com/file/d/0B5v28FoE-4LUX1NhbjYyRWR0TTA/edit?usp=sharing">app_wathdog_template.sh</a><br />
<br />
I'll try to explain how it works:<br />
<br />
First, you must replaces all instances of the text: <app> with the name of the script/service you are trying to monitor.<br />
<br />
This can quickly be done in vi with:<br />
<br />
Press ESC (to enter command mode)<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">:%s/<app>/myapp/g</span><br />
<br />
(myapp is your script/service name :-) )<br />
Press Enter<br />
<br />
Next you need to alter the settings in the script:<br />
<span style="font-family: Courier New, Courier, monospace;">#***************************************************</span><br />
<span style="font-family: Courier New, Courier, monospace;">#APP specific Configurations</span><br />
<span style="font-family: Courier New, Courier, monospace;">INTERVAL=30 #Interval between status checks</span><br />
<span style="font-family: Courier New, Courier, monospace;">LOCKFILE=/var/run/<app>.pid<span class="Apple-tab-span" style="white-space: pre;"> </span>#monitor app if this files exists.</span><br />
<span style="font-family: Courier New, Courier, monospace;">SERVICE_SCRIPT=/etc/init.d/<app></span><br />
<span style="font-family: Courier New, Courier, monospace;">SYSLOGNAME=<app>watchdog</span><br />
<span style="font-family: Courier New, Courier, monospace;">#***************************************************</span><br />
<br />
INTERVAL is the number of seconds between each status check<br />
<br />
LOCKFILE is a file which is used to start the monitor cycle, usually it can be the application's pidfile ou lockfile or anything you want, just make sure it only exists when your application should be running.<br />
<br />
SERVICE_SCRIPT is the complete path to the script that is used to monitor the application.<br />
<br />
SYSLOGNAME is the tag name that is send to syslog. This makes it easy to see what the watchdog's actions in the system's logs.<br />
<br />
The main action happens in the monitor function, which I'll explain below:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">monitor()</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> #infinite loop until stop is called</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> while [ 1 -eq 1 ]</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> do</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> #no restarting if there is no pidfile, this ensures there are no unsolicited restarts</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> #it also means that the application must delete its pidfile when stop is called.<span class="Apple-tab-span" style="white-space: pre;"> </span></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> if [ -f $PIDFILE ]</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> then</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> $SERVICE_SCRIPT status 2>&1 >/dev/null</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> STATUS=$?</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">if [ $STATUS -gt 0 ]</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">then</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">logger -t $</span><span style="font-family: 'Courier New', Courier, monospace;"><span style="font-size: x-small;">SYSLOGNAME</span></span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> "Status returned ERROR!"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">doRestart</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">fi</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> fi</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">sleep $INTERVAL</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> </span><span style="font-family: Courier New, Courier, monospace; font-size: x-small;">done</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">}</span><br />
<br />
First it enters an infinite loop which checks the app at $INTERVAL seconds.<br />
<br />
The monitoring is only done if it finds the LOCKFILE, this ensures it only restarts the application if it is supposed to be running. This also implies the application's script needs to remove the LOCKFILE when it stops. If the LOCKFILE is not removed once the application is stopped the watchdog will continue to check the script status which could lead to a restart loop.<br />
<br />
The watchdog relies on the service/script status exit code to verify if a restart is required.<br />
<br />
By default, if the status returns a value other than 0 then the script/service is restarted. This also sends messages through syslog so you can debug later.<br />
<br />
You can configure this loop to do whatever you want based on the status return value. :-)<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-3266274311775753112013-12-13T17:00:00.000+00:002014-02-05T11:51:24.684+00:00Squeezeplay 7.8 compiled for Raspberry PiFor my upcoming Raspberry PI project, an Internet Radio with built-in speakers, I had to compile the Logitech Squeezeplay (aka jive) player for the Raspberry Pi.<br />
<br />
I followed several instruction and eventually got it to compile. It was very tricky....<br />
I started with the information from slimdevices <a href="http://wiki.slimdevices.com/index.php/SqueezePlay_Build_Instructions" target="_blank">wiki</a> and started the compilation process.<br />
<br />
The compile process takes a long time on the pi (over 10 hours) and along the way there were several errors that needed to be addressed. I googled most of the errors so anyone should be able to overcome the difficulties in the compilation process.<br />
<br />
Since this was cumbersome and painful to complete, I'm making the compiled version available for anyone to use.<br />
<br />
You can download it <a href="https://drive.google.com/file/d/0B5v28FoE-4LUZl9yaUlBRUtiVEU/edit?usp=sharing">here</a>. It's compiled for the Raspbian Wheezy.<br />
I used the version from SVN so it should be mostly up to date.<br />
<br />
<br />
If you also need squeezecenter you can use the version provided by Logitech, the ARM version works fine in the Pi.<br />
You can run Squeezecenter and squeezeplay on the same Pi, but it can be a bit slow to start, at least on my version A Pi (only 256MB RAM).<br />
<br />
I've also purchased a LCD module and I'm planning on using it to display the Jive app which will basically turn the Pi into a DIY Logitech Touch.<br />
<br />
There are other options for this, namely SoftSqueeze (get it from <a href="http://softsqueeze.sourceforge.net/" target="_blank">here</a>) which is Java based and squeezeslave (from <a href="https://code.google.com/p/squeezeslave/" target="_blank">here</a>) which is a command line version (which can also be used with a char LCD).<br />
Now all I need is a DAC, some speakers and I'll be ready to go...<br />
<br />
Hope you find it useful!Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-89095952840853079362013-12-11T16:14:00.002+00:002013-12-17T15:50:48.356+00:00Red Hat Cluster Suite fence agent for HP C7000 enclosureSome of the projects I've worked on used the C7000 blade enclosure from HP.<br />
<br />
It's a neat solution to implement clustering since it has almost all of the required hardware in one package.<br />
<br />
Our software also uses Red Hat Enterprise Linux 5 (RHEL5) and Red Hat Cluster Suite.<br />
The system seemed to work properly but there were some issues with the fencing.<br />
<br />
We were using the iLO of the Blades servers for fencing nodes. Everything works fine until you pull out one node... Since the iLO port is no longer active and neither is the server the cluster stops and goes into a loop trying to fence the node. From this point on nothing works until you do a <span style="font-family: Courier New, Courier, monospace;">fence_ack_manual</span>...<br />
A similar situation as also occurred when one of the nodes suddenly died, by this I mean it had a complete hardware failure and it stopped responding. This hardware fault also affected the iLO port and so the cluster went into a fencing failed loop...<br />
<br />
<br />
In order to solve this problem a new fencing level must be added that uses the C7000.<br />
When fencing through the iLO port fails the cluster must try to do a fence through the enclosure management (OnBoard Administrator).<br />
To make this work a new fence agent was added.<br />
<br />
I modified the fence script for a IBM Bladecenter (<span style="font-family: Courier New, Courier, monospace;">/sbin/fence_bladecenter</span>) so it works with the C7000 enclosure, the configuration in the <span style="font-family: Courier New, Courier, monospace;">cluster.conf</span> is the same as with the IBM Bladecenter except that for the agent property:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><fencedevice <b>agent="fence_c7000"</b> ipaddr="10.1.0.4" login="Administrator" name="HP_C7000_OA_P" passwd="passwd"/></span><br />
<br />
<br />
the fence level configuration can be set as:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;"><clusternode name="node1" nodeid="1" votes="1"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><fence></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><method name="1"></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><device name="node1_ilo" action="off"/></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><device name="node1_ilo" action="on"/></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span></method></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><b><method name="2"></b></span><br />
<b><span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span><device blade="1" name="HP_C7000_OA_P"/></span></b><br />
<span style="font-family: Courier New, Courier, monospace;"><b><span class="Apple-tab-span" style="white-space: pre;"> </span></method></b><span class="Apple-tab-span" style="white-space: pre;"> </span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span></fence></span><br />
<span style="font-family: Courier New, Courier, monospace;"></clusternode></span><br />
<br />
This enables the cluster to fence through the iLo port first and then try the Onboard Administrator, in this case it would power cycle the blade in slot 1 (blade="1").<br />
<br />
You can get the fence script here: <a href="https://drive.google.com/file/d/0B5v28FoE-4LUYXJXMk1lOVA0VUk/edit?usp=sharing">fence_c7000</a><br />
<br />
To get this to work you need to place the script in the <span style="font-family: Courier New, Courier, monospace;">/sbin </span>directory and give it execute permissions:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">chmod 0755 /sbin/fence_c7000</span><br />
<br />
Then it's just a matter of configuring the <span style="font-family: Courier New, Courier, monospace;">cluster.conf</span> with the examples above.<br />
<br />
Hope this helps!<br />
<br />Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com0tag:blogger.com,1999:blog-8045943610241725588.post-51192751401386815282013-07-25T15:18:00.002+01:002022-01-25T09:32:49.438+00:00Adding custom apps to Iomega IX2-200 (CE, non-CE and possibly ix2 and ix2-dl)After getting some apps from <a href="http://lifelineapps.com/">http://lifelineapps.com</a> and checking out how they work I decided to create my own.<br />
This will be useful to launch some custom scripts and some custom apps I installed with ipkg (in my case Transmission).<br />
<br />
To install ipkg I followed the instruction in this link: <a href="http://techmonks.net/installing-transmission-and-dnsmasq-on-a-nas/">http://techmonks.net/installing-transmission-and-dnsmasq-on-a-nas/</a><br />
<br />
The method to install transmission on that page is somewhat tricky since it implies altering the firmware. My method works without the need to alter the /<span style="font-family: Courier New, Courier, monospace;">boot/images/apps</span> file.<br />
<br />
You simply install the application which consists of a <span style="font-family: Courier New, Courier, monospace;">start_init_scripts.sh</span> file (based on the information from this link: <a href="http://chrispont.co.uk.boardwalkit.com/?p=619">http://chrispont.co.uk.boardwalkit.com/?p=619</a>) that allows you to start any script whose name starts with an S in the <span style="font-family: Courier New, Courier, monospace;">/opt/etc/init.d </span>directory.<br />
<br />
I also include the app source code so you can modify it to your liking.<br />
To generate an app package just run the mkpackage.sh script on any debian (ubuntu) system.<br />
<br />
Link to app (startscripts_1.0_armel.tgz) and source code (ix2_custom_app.tgz):<br />
<br />
<span style="color: #0000ee;"><u><a href="https://drive.google.com/drive/folders/0B5v28FoE-4LUbkZGcmx0dWprV3c?resourcekey=0-b2REX0DqY7c2pfUz3nAMqg&usp=sharing">Google Drive Folder</a></u></span><br />
<br />
The startscripts file is the application you install on the ix2, the ix2_custom_app is a "template" to help you create other custom apps.<br />
<br />
The start scripts simplifies the process of starting apps that are not part of the original firmware since a reboot to the NAS implied I had to connect through SSH and startup the Transmission daemon manually. This was not very easy when I wasn't at home...<br />
Also this enables you to make full use of the optware packages since you can write your on custom app to enable some command to run in the background (like transcoding files).<br />
<br />
After installing the app, whenever you want to run the scripts just press the start app button in the ix2 web interface.Marcos Davidhttp://www.blogger.com/profile/00050487653756352601noreply@blogger.com2