15 August 2015

Linux OS Backup and Recovery

Source :http://relax-and-recover.org/documentation/getting-started
This quick start guide will show you how to run Relax-and-Recover from the git checkout and create a bootable USB backup.
Start by cloning the Relax-and-Recover sources from Github:
git clone https://github.com/rear/rear.git
Move into the rear/ directory:
cd rear/
Prepare your USB media. Change /dev/sdb to the correct device in your situation. Relax-and-Recover will ‘own’ the device in this example.
This will destroy all data on that device.
sudo usr/sbin/rear format /dev/sdb
Relax-and-recover asks you to confirm that you want to format the device:
The device has been labeled REAR-000 by the ‘format’ workflow.
Now edit the ‘etc/rear/local.conf’ configuration file:
cat > etc/rear/local.conf <
Now you are ready to create a rescue image. We want verbose output.
sudo usr/sbin/rear -v mkrescue
The output I get is:
Relax-and-Recover 1.13.0 / $Date$
Using log file: /home/jeroen/tmp/quickstart/rear/var/log/rear/rear-fireflash.log
Creating disk layout
Creating root filesystem layout
WARNING: To login as root via ssh you need to setup an authorized_keys file in /root/.ssh
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Creating initramfs
Writing MBR to /dev/sdb
Copying resulting files to usb location
You might want to check the log file for possible errors or see what Relax-and-Recover is doing.
Now reboot your system and try to boot from the USB device.
If that worked, you can dive into the advanced Relax-and-Recover options and start creating full backups. If you USB devices has enough space, initiate a backup using:
sudo usr/sbin/rear -v mkbackup
That is it. Your hard disk can now safely fail.

14 May 2012

Deploying Jenkins in CentOS

1. Download the software to "/usr/src" and Install Java JDK - jdk-6u30-linux-i586.rpm

    $ rpm -ivh jdk-6u30-linux-i586.rpm

2. Download the software to "/usr/src" and Install Apache Tomcat

    $ cd /usr/src
    $ tar -xzvf apache-tomcat-6.0.35.tar.gz
    $ mv apache-tomcat-6.0.35 tomcat

3. Download the software to "/usr/src" and Install grails-1.3.7.zip

    $ cd /usr/src
    $ unzip grails-1.3.7.zip

4. Edit "/etc/profile" and add the path below

    $ nano /etc/profile

    export JAVA_HOME
    export PATH

    export TOMCAT_HOME
    export PATH

    export CATALINA_HOME
    export PATH

    export GRAILS_HOME
    export PATH

To maximized java memory utiliztion, add the lines below in the "catalina.sh" file

Note. on 32bit Java (3gig is the max), on 64bit

    $ nano /usr/src/tomcat/bin/catalina.sh

    JAVA_OPTS="-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms2560m -Xmx2560m -XX:NewSize=1024m -XX:MaxNewSize=1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC"

5. Create a Tomcat User

    $ cd /usr/src/tomcat/conf
    $ nano tomcat-users.xml

    Add the lines below under

<role rolename="tomcat"/>
<role rolename="role1"/>
<role rolename="manager"/>
<user username="tomcat" password="tomcat" roles="tomcat"/>
<user username="role1" password="tomcat" roles="role1"/>
<user username="both" password="tomcat" roles="tomcat,role1"/>
<user username="manager" password="manager" roles="manager"/>

6. Download Jenkins .war file.


7. Login to the server's webui, eg http://localhost:8080 and click Tomcat Manager

8. Under file to deploy, browse the downloaded jenkins.war, and click deploy.

Thant's it. . .

26 August 2010

How to enable clustering in Openfire Enterprise?

What is clustering?

A cluster is when you have several servers hosting the same domain. Before Openfire 3.4.0 it was possible to only have one machine hosting a domain. Even though a single machine can scale to very big numbers (e.g. more than 100K concurrent users) there is still a limit in the number of users you can scale. Moreover, if that machine suffers a problem and the server stops then all users will be affected.

When using clustering you avoid those two problems. You will be naturally distributing the load among several machines thus even if one of them goes down the entire service will still continue to respond. The users that were connected to the machine that went down will just reconnect to any of the remaining machines.

How do I use clustering in Openfire?

Clustering is a commercial feature that is available in the Openfire Enterprise 3.4.0 or later. That means that you need to get Openfire Enterprise to be able to use clustering. If you have an existing Openfire Enterprise license you will need to update it since the max number of members that could be part of the cluster is kept in the license. If you don't have an enterprise license and would like to try the clustering feature you can get an evaluation license for 30 days.

Follow these steps to install Openfire Enterprise with clustering:

1. Identify the machines where you are going to install the server. Any machine with at least 512MB of RAM is fine. Of course you need to have more than one machine to have a cluster running.

2. Download Openfire from here


3. Run the installer to install Openfire in each of the machines. Follow the installation guide for more help.


4. Start the server and complete the setup wizard in each machine. Make sure to use the same XMPP domain, an external database and that all servers are using the same database.

5. Download the Openfire Enterprise plugin from here


6. Log into the admin console and go to the Plugins page. Upload the enterprise.jar file from that page. Run this step for each machine.

7. Go to the Enterprise tab to enter a valid license for Openfire Enterprise.

At this point you have several machines running Openfire Enterprise. The last step is to enable clustering and see how the machines meet each other. Follow these steps to enable clustering:

1.Go to the clustering page in the admin console.

2. Click on Enabled clustering and save the settings. It will take a few seconds for the first machine to start the cluster. Enabling cluster in the other machines will be faster. As each node joins the cluster you will see in the clustering page the nodes that conform the cluster.

What's next?

You may want to use a load balancer to distribute load between active servers. The load balancer does not need to support sticky sessions since XMPP connections are longed lived.

29 January 2010

Scraping an Entire Website using LINUX

Web scraping (also called Web harvesting or Web data extraction) is a computer software technique of extracting information from websites. Usually, such software programs simulate human exploration of the Web by either implementing low-level Hypertext Transfer Protocol (HTTP), or embedding certain full-fledged Web browsers, such as the Internet Explorer (IE) and the Mozilla Web browser. Web scraping is closely related to Web indexing, which indexes Web content using a bot and is a universal technique adopted by most search engines. In contrast, Web scraping focuses more on the transformation of unstructured Web content, typically in HTML format, into structured data that can be stored and analyzed in a central local database or spreadsheet. Web scraping is also related to Web automation, which simulates human Web browsing using computer software. Exemplary uses of Web scraping include online price comparison, weather data monitoring, website change detection, Web research, Web content mashup and Web data integration. [wikipedia]

Scarping a website using wget command in Linux.

$ wget -m --tries=7 "http://www.linuxnuggetz.blogspot.com"

Spoofing the browser that you are using.

$ wget --user-agent="Mozilla 2.0" -m http://www.linuxnuggetz.blogspot.com

28 November 2009

Changing Date and Time in Linux

1. date -s "DD Mon YYYY hh:mm:ss"

2. date set="DD Mon YYYY hh:mm:ss"

3. date MMDDhhmmYYYY.ss

1. date -s "28 NOV 2008 13:45:00"

2. date set="28 NOV 2008 13:45:00"

3. date 112813452009

Setting Time in Linux

date +%T -s "hh:mm:ss"

date +%T -s "10:13:13"

04 November 2009

Deleting Files/Folders based on OWNERSHIP

1. List the files and directories.

$ ls -lah

total 52K
drwxrwxrwt 11 root root 4.0K Nov 4 03:52 .
drwxr-xr-x 25 root root 4.0K Oct 6 03:54 ..
srwxr-xr-x 1 george george 0 Oct 8 00:47 .dguardianipc
srwxr-xr-x 1 george george 0 Oct 8 00:47 .dguardianurlipc
-rw-r--r-- 1 root root 0 Nov 4 03:52 file1
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file2
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file3
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file4
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file5
-rw-r--r-- 1 root root 0 Nov 4 03:52 file6
-rw-r--r-- 1 root root 0 Nov 4 03:52 file7
drwxrwxrwt 2 root root 4.0K Oct 1 07:55 .font-unix
drwxrwxrwt 2 root root 4.0K Oct 1 07:55 .ICE-unix
drwxr-xr-x 2 root root 4.0K Nov 4 03:43 test1
drwxr-xr-x 2 root root 4.0K Nov 4 03:51 test2
drwxr-xr-x 2 root root 4.0K Nov 4 03:43 test[2-7]
drwxr-xr-x 2 root root 4.0K Nov 4 03:43 test2-7
drwxr-xr-x 2 george george 4.0K Nov 4 03:51 test3
drwxr-xr-x 2 george george 4.0K Nov 4 03:51 test4
drwxr-xr-x 2 george george 4.0K Nov 4 03:51 test5

2. Let's try deleting the files with an owner 'gopher'

List the files with gopher owner.

$ ls -lah | grep gopher

-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file2
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file3
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file4
-rw-r--r-- 1 gopher gopher 0 Nov 4 03:52 file5

The output are from file2 to file5, next task is to list the exact filenames using awk.

$ ls -lah | gopher | awk '{print $9}


Now that the output list only filenames, we can now delete the files.

$ rm -fv `ls -lah | grep gopher | awk '{print $9}'`

removed `file2'
removed `file3'
removed `file4'
removed `file5'

If its a directory you can issue this pattern:

$ rm -fvR `ls -lah | grep george | awk '{print $9}'`

removed directory: `test3'
removed directory: `test4'
removed directory: `test5'

Note: be sure to replace the 'grep (owner)' with the specified owner/user.

28 October 2009

Easy way to extract Logs on a Date/Line Range in LINUX

Example: range of date to be extracted in the /var/log/message. Oct 26-27
our x will be Oct 26, y will be Oct 27

1. Get the first line number which will be our x reference (x - Oct 26).

$ grep -n 'Oct 26' /var/log/messages | head

The initial reference of x = 29933

Sample Output:

29933:Oct 26 00:00:00 postfix nagios: LOG ROTATION: DAILY
29934:Oct 26 00:00:00 postfix nagios: LOG VERSION: 2.0
29935:Oct 26 00:00:00 postfix nagios: CURRENT HOST STATE: localhost;UP;HARD;1;PING OK - Packet loss = 0%
29936:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Current Load;OK;HARD;1;OK
29937:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Current Users;OK;HARD;1
29938:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;HTTP;WARNING
29939:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;PING;OK;HARD;1;PING OK - Packet loss = 0%
29940:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Root Partition;OK;HARD;1;DISK OK - free space
29941:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;SSH;OK;HARD;1;SSH OK
29942:Oct 26 00:00:00 postfix nagios: CURRENT SERVICE STATE: localhost;Swap Usage;OK;HARD;1;

2. Get the last line number which will be our y reference (x - Oct 27).

$ grep -n 'Oct 27' /var/log/messages | tail

The last reference of y = 89712

Sample Output:

89703:Oct 27 23:59:14 postfix snmpd[18448]: -- IF-MIB::ifPhysAddress.1
89704:Oct 27 23:59:16 postfix snmpd[18448]: Connection from UDP: [192.168.0.x]:1055
89705:Oct 27 23:59:26 postfix last message repeated 15 times
89706:Oct 27 23:59:36 postfix snmpd[18448]: Connection from UDP: []:4118
89707:Oct 27 23:59:36 postfix snmpd[18448]: Received SNMP packet(s) from UDP: [192.168.0.x]:4118
89708:Oct 27 23:59:36 postfix snmpd[18448]: send response: Failure in sendto
89709:Oct 27 23:59:36 postfix snmpd[18448]: -- SNMPv2-MIB::sysObjectID.0
89710:Oct 27 23:59:36 postfix snmpd[18448]: -- IF-MIB::ifPhysAddress.1
89711:Oct 27 23:59:36 postfix snmpd[18448]: Connection from UDP: [192.168.0.x]:1055
89712:Oct 27 23:59:56 postfix last message repeated 23 times

3. Use the x,y reference to extract the file : x = 29933, y =89712

$ sed -n '29933,89712p' /var/log/messages
> newfilename