14 May 2012

Deploying Jenkins in CentOS

1. Download the software to "/usr/src" and Install Java JDK - jdk-6u30-linux-i586.rpm

    $ rpm -ivh jdk-6u30-linux-i586.rpm

2. Download the software to "/usr/src" and Install Apache Tomcat

    $ cd /usr/src
    $ tar -xzvf apache-tomcat-6.0.35.tar.gz
    $ mv apache-tomcat-6.0.35 tomcat

3. Download the software to "/usr/src" and Install grails-1.3.7.zip

    $ cd /usr/src
    $ unzip grails-1.3.7.zip

4. Edit "/etc/profile" and add the path below

    $ nano /etc/profile

    export JAVA_HOME
    export PATH

    export TOMCAT_HOME
    export PATH

    export CATALINA_HOME
    export PATH

    export GRAILS_HOME
    export PATH

To maximized java memory utiliztion, add the lines below in the "catalina.sh" file

Note. on 32bit Java (3gig is the max), on 64bit

    $ nano /usr/src/tomcat/bin/catalina.sh

    JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms2560m -Xmx2560m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"

5. Create a Tomcat User

    $ cd /usr/src/tomcat/conf
    $ nano tomcat-users.xml

    Add the lines below under

<role rolename="tomcat"/>
<role rolename="role1"/>
<role rolename="manager"/>
<user username="tomcat" password="tomcat" roles="tomcat"/>
<user username="role1" password="tomcat" roles="role1"/>
<user username="both" password="tomcat" roles="tomcat,role1"/>
<user username="manager" password="manager" roles="manager"/>

6. Download Jenkins .war file.


7. Login to the server's webui, eg http://localhost:8080 and click Tomcat Manager

8. Under file to deploy, browse the downloaded jenkins.war, and click deploy.

Thant's it. . .

26 August 2010

How to enable clustering in Openfire Enterprise?

What is clustering?

A cluster is when you have several servers hosting the same domain. Before Openfire 3.4.0 it was possible to only have one machine hosting a domain. Even though a single machine can scale to very big numbers (e.g. more than 100K concurrent users) there is still a limit in the number of users you can scale. Moreover, if that machine suffers a problem and the server stops then all users will be affected.

When using clustering you avoid those two problems. You will be naturally distributing the load among several machines thus even if one of them goes down the entire service will still continue to respond. The users that were connected to the machine that went down will just reconnect to any of the remaining machines.

How do I use clustering in Openfire?

Clustering is a commercial feature that is available in the Openfire Enterprise 3.4.0 or later. That means that you need to get Openfire Enterprise to be able to use clustering. If you have an existing Openfire Enterprise license you will need to update it since the max number of members that could be part of the cluster is kept in the license. If you don't have an enterprise license and would like to try the clustering feature you can get an evaluation license for 30 days.

Follow these steps to install Openfire Enterprise with clustering:

1. Identify the machines where you are going to install the server. Any machine with at least 512MB of RAM is fine. Of course you need to have more than one machine to have a cluster running.

2. Download Openfire from here


3. Run the installer to install Openfire in each of the machines. Follow the installation guide for more help.


4. Start the server and complete the setup wizard in each machine. Make sure to use the same XMPP domain, an external database and that all servers are using the same database.

5. Download the Openfire Enterprise plugin from here


6. Log into the admin console and go to the Plugins page. Upload the enterprise.jar file from that page. Run this step for each machine.

7. Go to the Enterprise tab to enter a valid license for Openfire Enterprise.

At this point you have several machines running Openfire Enterprise. The last step is to enable clustering and see how the machines meet each other. Follow these steps to enable clustering:

1.Go to the clustering page in the admin console.

2. Click on Enabled clustering and save the settings. It will take a few seconds for the first machine to start the cluster. Enabling cluster in the other machines will be faster. As each node joins the cluster you will see in the clustering page the nodes that conform the cluster.

What's next?

You may want to use a load balancer to distribute load between active servers. The load balancer does not need to support sticky sessions since XMPP connections are longed lived.

29 January 2010

Scraping an Entire Website using LINUX

Web scraping (also called Web harvesting or Web data extraction) is a computer software technique of extracting information from websites. Usually, such software programs simulate human exploration of the Web by either implementing low-level Hypertext Transfer Protocol (HTTP), or embedding certain full-fledged Web browsers, such as the Internet Explorer (IE) and the Mozilla Web browser. Web scraping is closely related to Web indexing, which indexes Web content using a bot and is a universal technique adopted by most search engines. In contrast, Web scraping focuses more on the transformation of unstructured Web content, typically in HTML format, into structured data that can be stored and analyzed in a central local database or spreadsheet. Web scraping is also related to Web automation, which simulates human Web browsing using computer software. Exemplary uses of Web scraping include online price comparison, weather data monitoring, website change detection, Web research, Web content mashup and Web data integration. [wikipedia]

Scarping a website using wget command in Linux.

$ wget -m --tries=7 "http://www.linuxnuggetz.blogspot.com"

Spoofing the browser that you are using.

$ wget --user-agent="Mozilla 2.0" -m http://www.linuxnuggetz.blogspot.com

28 November 2009

Changing Date and Time in Linux

1. date -s "DD Mon YYYY hh:mm:ss"

2. date set="DD Mon YYYY hh:mm:ss"

3. date MMDDhhmmYYYY.ss

1. date -s "28 NOV 2008 13:45:00"

2. date set="28 NOV 2008 13:45:00"

3. date 112813452009

Setting Time in Linux

date +%T -s "hh:mm:ss"

date +%T -s "10:13:13"

04 November 2009

Deleting Files/Folders based on OWNERSHIP

1. List the files and directories.

$ ls -lah

total 52K
drwxrwxrwt 11 root root 4.0K Nov 4 03:52 .
drwxr-xr-x 25 root root 4.0K Oct 6 03:54 ..
srwxr-xr-x 1 george george 0 Oct 8 00:47 .dguardianipc
srwxr-xr-x 1 george george 0 Oct 8 00:47 .dguardianurlipc
-rw-r--r-- 1 root root 0 Nov 4 03:52 file1
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file2
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file3
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file4
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file5
-rw-r--r-- 1 root root 0 Nov 4 03:52 file6
-rw-r--r-- 1 root root 0 Nov 4 03:52 file7
drwxrwxrwt 2 root root 4.0K Oct 1 07:55 .font-unix
drwxrwxrwt 2 root root 4.0K Oct 1 07:55 .ICE-unix
drwxr-xr-x 2 root root 4.0K Nov 4 03:43 test1
drwxr-xr-x 2 root root 4.0K Nov 4 03:51 test2
drwxr-xr-x 2 root root 4.0K Nov 4 03:43 test[2-7]
drwxr-xr-x 2 root root 4.0K Nov 4 03:43 test2-7
drwxr-xr-x 2 george george 4.0K Nov 4 03:51 test3
drwxr-xr-x 2 george george 4.0K Nov 4 03:51 test4
drwxr-xr-x 2 george george 4.0K Nov 4 03:51 test5

2. Let's try deleting the files with an owner 'gopher'

List the files with gopher owner.

$ ls -lah | grep gopher

-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file2
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file3
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file4
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file5

The output are from file2 to file5, next task is to list the exact filenames using awk.

$ ls -lah | gopher | awk '{print $9}


Now that the output list only filenames, we can now delete the files.

$ rm -fv `ls -lah | grep gopher | awk '{print $9}'`

removed `file2'
removed `file3'
removed `file4'
removed `file5'

If its a directory you can issue this pattern:

$ rm -fvR `ls -lah | grep george | awk '{print $9}'`

removed directory: `test3'
removed directory: `test4'
removed directory: `test5'

Note: be sure to replace the 'grep (owner)' with the specified owner/user.

28 October 2009

Easy way to extract Logs on a Date/Line Range in LINUX

Example: range of date to be extracted in the /var/log/message. Oct 26-27
our x will be Oct 26, y will be Oct 27

1. Get the first line number which will be our x reference (x - Oct 26).

$ grep -n 'Oct 26' /var/log/messages | head

The initial reference of x = 29933

Sample Output:

29933:Oct 26 00:00:00 postfix nagios: LOG ROTATION: DAILY
29934:Oct 26 00:00:00 postfix nagios: LOG VERSION: 2.0
29935:Oct 26 00:00:00 postfix nagios: CURRENT HOST STATE: localhost;UP;HARD;1;PING OK - Packet loss = 0%
29936:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Current Load;OK;HARD;1;OK
29937:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Current Users;OK;HARD;1
29938:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;HTTP;WARNING
29939:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;PING;OK;HARD;1;PING OK - Packet loss = 0%
29940:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Root Partition;OK;HARD;1;DISK OK - free space
29941:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;SSH;OK;HARD;1;SSH OK
29942:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Swap Usage;OK;HARD;1;

2. Get the last line number which will be our y reference (x - Oct 27).

$ grep -n 'Oct 27' /var/log/messages | tail

The last reference of y = 89712

Sample Output:

89703:Oct 27 23:59:14 postfix snmpd[18448]: -- IF-MIB::ifPhysAddress.1
89704:Oct 27 23:59:16 postfix snmpd[18448]: Connection from UDP: [192.168.0.x]:1055
89705:Oct 27 23:59:26 postfix last message repeated 15 times
89706:Oct 27 23:59:36 postfix snmpd[18448]: Connection from UDP: []:4118
89707:Oct 27 23:59:36 postfix snmpd[18448]: Received SNMP packet(s) from UDP: [192.168.0.x]:4118
89708:Oct 27 23:59:36 postfix snmpd[18448]: send response: Failure in sendto
89709:Oct 27 23:59:36 postfix snmpd[18448]: -- SNMPv2-MIB::sysObjectID.0
89710:Oct 27 23:59:36 postfix snmpd[18448]: -- IF-MIB::ifPhysAddress.1
89711:Oct 27 23:59:36 postfix snmpd[18448]: Connection from UDP: [192.168.0.x]:1055
89712:Oct 27 23:59:56 postfix last message repeated 23 times

3. Use the x,y reference to extract the file : x = 29933, y =89712

$ sed -n '29933,89712p' /var/log/messages
> newfilename

06 September 2009

Google Sitemap Generator Installation in a CentOS Platform

1. Download Google Site Map Generator:


2. Unpack the downloaded file

$ tar –zxvf sitemap_linux-i386-beta1-20090225.tar.gz

3. Run the installation script

$ cd sitemap-install/
$ ./install.sh -t /usr/sbin/apachectl -g apache

Other options:
* -h displays help about these command line options.
* -d directory specifies a non-default installation directory.
* -a Apache-binary-filename specifies the path to the Apache binary file (httpd).
Alternatively, you can specify this value during installation.
* -c Apache-config-filename specifies the Apache root configuration file.
* -g Apache-group specifies the user group name under which Apache is running.
* -t Apache-control-filename specifies the Apache control script (apache ctl).
If you specify this value, the installer lets you choose to automatically
restart Apache at the end of the installation.
If you omit this option, you'll need to restart Apache manually.

Just simply follow the installation, everything is interactive.

4. By default you can access the sitemap generator via port 8181

e.g. http://yourserver:8181/

5. Initialy you will be denied accessing it remotely. Run this in the terminal to enable remote access.

$ /usr/local/google-sitemap-generator/bin/sitemap-daemon remote_admin enable

6. Install SSL requirements.

$ yum install mod_ssl openssl

7. Generate a self-signed certificate

Generate private key
$ openssl genrsa -out ca.key 1024

Generate CSR
$ openssl req -new -key ca.key -out ca.csr

Generate Self Signed Key
$ openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt

Move the files to the correct locations
$ mv ca.crt /etc/pki/tls/certs
$ mv ca.key /etc/pki/tls/private/ca.key
$ mv ca.csr /etc/pki/tls/private/ca.csr

8. Then update the Apache SSL configuration file

$ vi +/SSLCertificateFile /etc/httpd/conf.d/ssl.conf

Comment the existing and add the following:

SSLCertificateFile /etc/pki/tls/certs/ca.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca.key

Restart the apache server

$ /etc/init.d/httpd restart

9. Edit the sitemap generator httpd configuration.

$ vi /usr/local/google-sitemap-generator/conf/httpd.conf

Add the SSL lines, or you can simply paste the code below

Listen 8181
NameVirtualHost *:8181
<VirtualHost *:8181>
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/ca.crt
SSLCertificateKeyFile /etc/pki/tls/private/ca.key
DocumentRoot "/usr/local/google-sitemap-generator/admin-console"
ScriptAlias /cgi-bin/ "/usr/local/google-sitemap-generator/admin-console/cgi-bin/"

<Directory "/usr/local/google-sitemap-generator/admin-console" >
# Allow from all
AllowOverride All
Options ExecCGI
DirectoryIndex index.html

LoadModule google_sitemap_generator_module "/usr/local/google-sitemap-generator/lib/mod_sitemap.so"

10. Restart Apache and that's It.

$ /etc/init.d/httpd restart

Starting and Stopping Google Sitemap Deamon

$ /usr/local/google-sitemap-generator/bin/sitemap-daemon service start
$ /usr/local/google-sitemap-generator/bin/sitemap-daemon service stop
$ /usr/local/google-sitemap-generator/bin/sitemap-daemon service restart