Wednesday 26 February 2014

First adventures in Python

One of my goals for the next couple of months is to learn Python. I guess this is long overdue and I should probably have picked it up a few years ago... Oh well, never too late to start...

Since I have an issue with following basic tutorials and writing code that has no practical use, I decided to create a small app to monitor databases.

Nothing to fancy, just a listing  a couple of running SQL queries, table locks and user sessions.
The nice part is that it can  do this for three different database systems: Oracle, MySQL and PostgreSQL (this one is not yet implement).

I placed all of the code on github (HERE) so you can check it out by yourself.


How does it work?

Well I created a simple interface using ncurses the is divided into 4 areas: Header, Running Queries, Current DB Locks, Current Sessions.

The interface is the same for all database systems and the information to display is the same (within each DB's possibilities).

Here are some screenshots:

MySQL Database:

 Oracle database:


It currently is in a very alpha stage, and there are some things I need to polish. For instance the terminal it runs on needs to have at least 51 lines (which is easily changed in the code), there are few exception handling, etc...
But there also some nice things: long running queries (over 5 minutes) should appear in RED,  the query text is divided into two lines so you can more easily identify bad queries.


The information is constantly refreshing (currently once per second) and it should have a low impact on your database since the queries it runs are on internal database tables/views.

I plan on adding some more features like system/db load, traffic, query cache hit, key efficiency, etc....

So stay tuned for more updates...


Friday 21 February 2014

Linux script to monitor an Apache Tomcat WebApp




Sometimes the world is against us and stuff goes wrong.
This can happen a lot with Tomcat and bad behavioured WebApps.

To mitigate crashes and unresponsive WebApps sometimes it is useful to check if they are working or that they haven't crashed Tomcat... (not my fault! :-))

Since I was running a Tomcat server on a Cluster system I needed a script that could check the status of a particular webapp and if tomcat was also alive.

I came up with THIS, a simple script that starts/stops a WebApp, checks if it is running and even checks if Tomcat hasn't died...


So how does it work?

First we need to setup some variables:

The first one tells the script where the webapp is running, this can be change to include a specific IP Address (Virtual address used in clusters for instance)
TOMCAT_SERVER=`hostname`

Next is the script that controls tomcat (if you aren't using one you can change the code to use the built in startup and shutdown commands)
TOMCAT_SCRIPT=/etc/init.d/tomcat

This is the port where Tomcat is running, usually its 8080
TOMCAT_PORT=8080

This port is used to check if Tomcat is running.
Why use the Shutdown port instead of the normal port? Because I had an issue where the normal port was open but there was no shutdown port and Tomcat was not running properly...
TOMCAT_SHUTDOWN_PORT=8005

We also need the credentials for a user with permission to start/stop webapps.
TOMCAT_USER=user
TOMCAT_PASS=pass

Next we need to define the WebApp name to be controlled. There are two ways to do this, either by the script, assuming we use something like: tomcat_MyWebApp or tomcat-MyWebApp.
Or just set it manually.

#The Line below is used to extract the Webapp name from the script
#the webapps name is assumed to the last sequence of alphanumeric
#chars in the scriptname (for instance: tomcat_MyWebApp -> WA name=MyWebApp)

#TOMCAT_WA_NAME=`echo $0|egrep -o "[a-zA-Z0-9]*$"`

#Or you can manually set the WS name here 
TOMCAT_WA_NAME=MyWebApp


Finally we need to define the installation dir for Tomcat, this should be the same as the CATALINA_DIR in Tomcat's configuration. This is used to check the log for Memory errors.
#Tomcat dir aka CATALINA_DIR
TOMCAT_DIR=/usr/local/jakarta-tomcat

The script then uses curl to invoke Tomcat start/stop WebApps commands.


The initial version of the status routine just opened the WebApp default page to see if it got some text.
Later, I added another verification to see if Tomcat hadn't died with a Permgen error (memory issues, bad app..).

You can extend the status routine to check for internal variables in your WebApp and determine  it's status or any other condition that signals a fault and requires a restart.


To make this restart webapps and Tomcat in scenarios where you don't have a cluster you can use the watchdog template script on my blog (This one!) to make a periodic checks and mimic cluster behaviour.


Wednesday 19 February 2014

Getting an ILI9341 SPI screen working on the RaspberryPI

I recently bought a small SPI screen from ebay which uses the ILI9341 controller.
The screen is identical to the one made by ElecFreaks (This one!).
Although I got mine really cheap from eBay....


I decided to use notro drivers since they are flexible and already provide a driver for the ILI9341. You can find the source code and install instructions here.
I actually compiled a custom kernel where I added his drivers.

After compilation and module installation I just had to figure out the parameters I needed to use.

But first we still need to hook it up to the RaspPI :-)

I used the following connections, also from notro's Wiki

I used the wiring schema for an Adafruit LCD, check the original table here.

DisplayDriver gpio nameRaspberry PiRemark
BLledGPIO18
SCKSCLK
MISOUsed by SD-card
MOSIMOSI
CSCE0Chip Select
SDCSSD-card Chip Select
RSTresetGPIO25
D/CdcGPIO24
VIN3.3V
GNDGND

Aside from a normal SPI connection, please notice the BL(led), RST(reset) and D/C(dc) pins since these will be used as parameters for the module.


Once you have everything hooked up, compiled and installed you can test out the screen by loading the module:


sudo modprobe fbtft_device custom name=fb_ili9341  gpios=reset:25,dc:24,led:18 speed=16000000 rotate=90 bgr=1

If you change one of the pins above, please change the gpios parameter accordingly.
Also, you have the rotate option to put the screen in landscape mode, by default it will be in portrait mode.
The bgr=1 option is to exchange the blue and red colors, by default the colors don't appear correctly and this option is necessary.
Once you load the module nothing will happen on your screen (this made think it wasn't working...) you can test it out by either sending the console to the framebuffer with:

con2fbmap 1 1

(it means, send console 1 to framebuffer 1)

or using X

FRAMEBUFFER=/dev/fb1 startx


If you want to make the module load automatically, add the following line to /etc/modules:
fbtft_device custom name=fb_ili9341  gpios=reset:25,dc:24,led:18 speed=16000000 rotate=90 bgr=1

EDIT: For the built-in version, you can add this to the kernel command line:
fbtft_device.name=fb_ili9341 fbtft_device.gpios=reset:25,dc:24,led:18 fbtft_device.speed=16000000 fbtft_device.rotate=90 fbtft_device.bgr=1 fbtft_device.custom=1


Hope you find this useful.

Monday 3 February 2014

Chrome - Create a button to "proxify" a site

Some time ago I created a custom proxy with the help of this tutorial:
http://www.labnol.org/internet/setup-proxy-server/12890/

Is pretty cool since it helps me circumvent some problems I have from incorrect proxy blacklists, to bypass geographically blocked content, etc.

Recently I found that you can add a bookmark to Chrome that calls some javascript code.

Putting the two together I came up with a simple Button on my Chrome Bookmark bar that instantly redirects a page to my proxy.

All you need to do is to create a bookmark, add it to the bookmark bar and use the following text in the URL (all in a single line!)
:



javascript:window.location="http://PROXY_URL/"+window.location.href.substring(window.location.href.search("://")+3);


Just replace PROXY_URL with your own custom proxy or you can search the for one.

The final bookmark url should look like:


javascript:window.location="http://labnol-proxy-server.appspot.com/"+window.location.href.substring(window.location.href.search("://")+3);


The proxy I mention there is the one that is used as an example in the tutorial link.

So how do you use it?

After adding the bookmark you should be able to see it in the bookmark bar.


When you reach a site that you want to filter to the proxy, just it the button, and It will automatically redirect you:




Of course this means you can do other types of redirections.

I hope you find this useful!