NDMP communication failure error

Guys,

Issue :

Netbackup server sends alert NDMP communication failure once everyday. But there is no issue to run scheduled backup jobs.

Environment : Netbackup server is RHEL 6.2 which also works as Media server, Netbackup 7.1 is running on it. NDMP host is  Netapp filer 2240. Backup is being taken on HP MSL 4049 tape.

Troubleshooting steps followed :

Everyday I receive a NDMP communication error alert messages once. I don't know why. I tried to troubleshoot but no luck. Even Symantec doesn't have any answer apart from what they have in their KB. They said, pw authentication might have wrong but for our setup it is correct. This is already verified multiple times. Enabled all logging even. Network communication is also fine. Following error I could see in bptm log file :

avrd[15164]: ndmp_public_session_create() failed: Unexpected NDMP message - not a connected message
tldd[24890]: TLD(0) ndmp_public_session_create_wCred failed with error code -1005

I don't see any suspicious messages on filer.

Solution : I don't still find out. Please share if you anyone knows.

FYI: I observed that tape drives periodically go off line. Hence, I had to up them. I have deployed a work around script which all check if drive is down or not. It'll up if it is down.

Check on netbackup server : tpconfig -l    or vmoprcmd -d
up it :  vmoprcmd    

Read More

CISCO UCS makes easy to manage IT setup and provides optimized usage of resources

CISCO has launched UCS (unified computing system) few years back and this makes really easy to manage IT environment. CISCO UCS 5k chassis (containing 8 blades), fabric interconnect(6k) and fabric extender (2k) ie FEX make UCS environment where FCOE protocol is used. In a tipical setup,  two FEX will be connected to one chassis. Each FEX has 8 Ethernet  and 2 FC ports. Two downlink connection will be for single blade, each from FEX. You don't need to manually do cabling. You need to insert FEX module. That's it. This is also called IO module.

Now, there will have one FCOE uplink to Fabric(A and B, for redundancy) to each CISCO fabric interconnect switch. Both A and B Fab will be connected in order to maintain failover. This is UCS setup. Number cable dramatically reduced. Hence, power consumption too.

In order to send out traffic to uplink, Nexus 5k switch can be used and this supports FCoE and other  features too. In DC, generally we prefer Nexus 5k. All zonning, vlan and trunking are done in this switch. There is also redundancy here ie NX A and NX B. Your storage systems will be connected to this NexUS switch. Even backup will be connected to it. In order to sendout traffic from NEXUS,  there are other lot of switches. In our environment, we use cat 6K. You can use NexUS 7k too. I heard somewhere MDS switch is also used. This is basically core switch which interfaces to ISP routers.

Now, in order to manage UCS, there is UCS manager which is accessible through GUI. Even you can manage everything through backend. You need to login into primay fabric interconnect switch and execute commands. From server installation to patching, upgrading(FW and software) can be done. UCSM has nice features.

If vmware ESX is setup on blades and if they form ESX cluster with FT enabled, then your IT environment is 100% up. I'll explain and provide more unusual details from time to time.

So, stay tune....take care :)

Read More

multipath details on RHEL 6

Guys,

I'll mention one default configuration file(/etc/multipath.conf)

Environment : RHEL 6 :
Default : /etc/multipath.conf [comment should be removed]
-------------
#multipath.conf
#NetApp recommended settings


defaults
{
        user_friendly_names yes
        max_fds max
        queue_without_daemon no
        bindings_file "/var/lib/multipath/bindings"
        uid=500
        gid=500
}
blacklist
{
        wwid DevId
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z]"
        devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}
devices
{
        device
        {
                vendor "NETAPP"
                product "LUN"
                getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
                prio_callout "/sbin/mpath_prio_ontap /dev/%n"
                features "1 queue_if_no_path"
                hardware_handler "0"
                path_grouping_policy group_by_prio
                failback immediate
                rr_weight uniform
                rr_min_io 128
                path_checker directio
                flush_on_last_del yes
        }
}

--------------


Example of paths :
$multipath -ll
mini_p (360a98000572d45394b34715579354446) dm-23 NETAPP,LUN
[size=1.0T][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=8][active]
 \_ 1:0:0:0  sda        8:0     [active][ready]
 \_ 2:0:1:0  sdca       68:224  [active][ready]
\_ round-robin 0 [prio=2][enabled]
 \_ 2:0:2:0  sdct       70:16   [active][ready]
 \_ 1:0:1:0  sdq        65:0    [active][ready]

    Explanations:

mini_p (360a98000572d45394b34715579354446) dm-23 NETAPP,LUN
------  ---------------------------------  ---- --- ---------------
   |               |                         |    |          |-------> Product
   |               |                         |    |------------------> Vendor
   |               |                         |-----------------------> sysfs name
   |               |-------------------------------------------------> WWID of the device
   |-----------------------------------------------------------------> User defined Alias

[size=1.0T][features=1 queue_if_no_path][hwhandler=0][rw]
 ---------  ---------------------------  ----------------
     |                 |                        |--------------------> Hardware Handler
     |                 |---------------------------------------------> Features supported
     |---------------------------------------------------------------> Size of the DM


Path Group 1:
\_ round-robin 0 [prio=8][active]
-- -------------  ------  ------
 |    |              |      |----------------------------------------> Path group state
 |    |              |-----------------------------------------------> Path group priority
 |    |--------------------------------------------------------------> Path selector
 |-------------------------------------------------------------------> Path group level

First path on Path Group 1:
  \_ 1:0:0:0  sda        8:0     [active][ready]
    -------- --- ----   ------  -----
      |      |     |        |      |---------------------------------> Physical Path state
      |      |     |        |----------------------------------------> DM Path state
      |      |     |-------------------------------------------------> Major, minor numbers
      |      |-------------------------------------------------------> Linux device name
      |--------------------------------------------------------------> host,chan,scsiid,lun

Second path on Path Group 1:
  \_ 2:0:1:0  sdca       68:224  [active][ready]

Path Group 2:

 \_ 2:0:2:0  sdct       70:16   [active][ready]
 \_ 1:0:1:0  sdq        65:0    [active][ready]

-----------------

polling_interval :  Specifies the interval between two path checks in seconds.
udev_dir     : The directory where udev device nodes are created. The default value is /dev.
multipath_dir : /var/lib/multipath/bindings, The directory where the dynamic shared objects are stored.

path_selector     : Specifies the default algorithm to use in determining what path to use for the next I/O operation.

Possible values include:

    round-robin 0: Loop through every path in the path group, sending the same amount of I/O to each.
    queue-length 0: Send the next bunch of I/O down the path with the least number of outstanding I/O requests.
    service-time 0: Send the next bunch of I/O down the path with the shortest estimated service time, which is determined

by dividing the total size of the outstanding I/O to each path by its relative throughput.
The default value is round-robin 0.

path_grouping_policy     : Specifies the default path grouping policy to apply to unspecified multipaths. Possible values include:

    failover: 1 path per priority group.
    multibus: all valid paths in 1 priority group.
    group_by_serial: 1 priority group per detected serial number.
    group_by_prio: 1 priority group per path priority value. Priorities are determined by callout programs specified as global, per-controller, or per-multipath options.
  group_by_node_name: 1 priority group per target node name. Target node names are fetched in/sys/class/fc_transport/target*/node_name.

The default value is failover. 

getuid_callout     :

Specifies the default program and arguments to call out to obtain a unique path identifier. An absolute path is required.
The default value is /lib/udev/scsi_id --whitelisted --device=/dev/%n.

prio     : Specifies the default function to call to obtain a path priority value. For example, the ALUA bits in SPC-3

provide an exploitable prio value. Possible values include:
    const: Set a priority of 1 to all paths.
    emc: Generate the path priority for EMC arrays.
    alua: Generate the path priority based on the SCSI-3 ALUA settings.
    tpg_pref: Generate the path priority based on the SCSI-3 ALUA settings, using the preferred port bit.
    ontap: Generate the path priority for NetApp arrays.
    rdac: Generate the path priority for LSI/Engenio RDAC controller.
    hp_sw: Generate the path priority for Compaq/HP controller in active/standby mode.
    hds: Generate the path priority for Hitachi HDS Modular storage arrays.
        The default value is const. 


path_checker     :

Specifies the default method used to determine the state of the paths. Possible values include:
    readsector0: Read the first sector of the device.
    tur: Issue a TEST UNIT READY to the device.
    emc_clariion: Query the EMC Clariion specific EVPD page 0xC0 to determine the path.
    hp_sw: Check the path state for HP storage arrays with Active/Standby firmware.
    rdac: Check the path stat for LSI/Engenio RDAC storage controller.
    directio: Read the first sector with direct I/O.
The default value is directio.

failback     :
Manages path group failback.
    immediate :  A value of immediate specifies immediate failback to the highest priority path group that contains

active paths.
    manual : A value of manual specifies that there should not be immediate failback but that failback can happen only

with operator intervention.
    followover : A value of followover specifies that automatic failback should be performed when the first path of a

path group becomes active. This keeps a node from automatically failing back when another node requested the failover.
A numeric value greater than zero specifies deferred failback, expressed in seconds.
The default value is manual.


 rr_min_io :    Specifies the number of I/O requests to route to a path before switching to the next path in the current

path group. This setting is only for systems running kernels older than 2.6.31. Newer systems should use rr_min_io_rq. The

default value is 1000.

rr_min_io_rq :    Specifies the number of I/O requests to route to a path before switching to the next path in the current

path group, using request-based device-mapper-multipath. This setting should be used on systems running current kernels. On

systems running kernels older than 2.6.31, use rr_min_io. The default value is 1.


rr_weight  :    If set to priorities, then instead of sending rr_min_io requests to a path before calling path_selector to

choose the next path, the number of requests to send is determined by rr_min_io times the path's priority, as determined by

the prio function. If set to uniform, all path weights are equal. The default value is uniform.

no_path_retry     : A numeric value for this attribute specifies the number of times the system should attempt to use a

failed path before disabling queueing.

    fail : A value of fail indicates immediate failure, without queueing.
    queue: A value of queue indicates that queueing should not stop until the path is fixed.
The default value is 0.

user_friendly_names     :If set to yes, specifies that the system should use the /etc/multipath/bindings file to assign a

persistent and unique alias to the multipath, in the form of mpathn. If set to no, specifies that the system should use the

WWID as the alias for the multipath. In either case, what is specified here will be overridden by any device-specific

aliases you specify in the multipaths section of the configuration file. The default value is no.

queue_without_daemon :    If set to no, the multipathd daemon will disable queueing for all devices when it is shut down. The

default value is no.

flush_on_last_del     : If set to yes, the multipathd daemon will disable queueing when the last path to a device has

been deleted. The default value is no.

max_fds  :    Sets the maximum number of open file descriptors that can be opened by multipath and the multipathd daemon.

This is equivalent to the ulimit -n command. As of the Red Hat Enterprise Linux 6.3 release, the default value is max,

which sets this to the system limit from /proc/sys/fs/nr_open. For earlier releases, if this is not set the maximum number

of open file descriptors is taken from the calling process; it is usually 1024. To be safe, this should be set to the

maximum number of paths plus 32, if that number is greater than 1024.

checker_timeout     : The timeout to use for path checkers that issue SCSI commands with an explicit timeout, in

seconds. The default value is taken from sys/block/sdx/device/timeout.

fast_io_fail_tmo  :    The number of seconds the SCSI layer will wait after a problem has been detected on an FC remote

port before failing I/O to devices on that remote port. This value should be smaller than the value of dev_loss_tmo.

Setting this to off will disable the timeout. The default value is determined by the OS.

dev_loss_tmo :     The number of seconds the SCSI layer will wait after a problem has been detected on an FC remote port

before removing it from the system. Setting this to infinity will set this to 2147483647 seconds, or 68 years. The default

value is determined by the OS.

Read More

How to verify UDP packet communication between two linux system?

Guys,

Today, I had to check UDP packet communication between linux and a windows system. Main purpose of the windows system was to capturing or receiving syslog data from various linux system on arcsight. Hence, remote log forwarding was enabled on client. Following steps I followed :

Sending UDP packets from client :

$nc  -uv   IP_of_system_where_UDP_is_sending   port_number_to_which_UDP_is_receiving
Hello
This is test UDP packet
Are you capturing it
Please let me know


Example :

A$ nc   192.1.2.10   514
Hello
This is test UDP packet
Are you capturing it
Please let me know


If you capture the packets using wireshark / tshark or tcpdump, you'll see above packets on windows system/linux system

If your system wants to listen or receive UDP packets on any linux box, you can execute this :

$ nc -luv port

Example :

B$ nc -luv   514

If you don't have nc command, install it like :

$ yum install nc -y

Try with your own risk :)

Read More

New posts are coming soon..

Hi Guys,

It's been a long time I didn't post any article or issue here. There were few transitions in my career and I was bit busy. Hence, I didn't get time to update or post. There are more technologies on which I'll discuss. Following are in pipeline :

Linux Kernel
A bit about Red Hat Company and Product
Linux troubleshooting step and basic concept
CISCO unified computing system( UCS )
CISCO Fabric Interconnect Switch, VLAN, port channeling etc
FCOE, FC protocol
Red Hat Cluster
Veritas Cluster
Veritas Volume Manager
DMP or Veritas Dynamic Multipathing
SAN/NAS
Vmware vSpehere Virtualization, ESX 5.0.0
vCenter and vCenter HeartBeat
vMotion, Update manager
Netapp Storage Filers, Volume, LUN,masking, mapping, exporting, system log analysis etc
SnapVault
Symantec Netbackup Tecnology
Scripting : Python and Perl

So, stay tune...Good Luck :)

Read More

Steps to develop patch and apply it to original source file

1. Create test.c

 Above file contains :

--------
[kamalma@test-1 C_Programming]$ cat test.c
#include
#include
int main()
 {
 printf("\n I'm kamal \n");
return 0;
}
--------

2. Compile it and check its output.

OP :

-------
[kamalma@test-1 C_Programming]$ ./test

 I'm kamal
[kamalma@test-1 C_Programming]$
-------

3. Modify test.c and add new lines or codes in it. Lets say it test_modifed.c. This file contains following codes :

------
[kamalma@test-1 C_Programming]$ cat test_modified.c
#include
#include
int main()
 {
 printf("\n I'm kamal \n");
 printf("\nI have added one more line. This comes from modifed code\n" );
return 0;
}
[kamalma@test-1 C_Programming]$
[kamalma@test-1 C_Programming]$ gcc -o test_modified test_modified.c
[kamalma@test-1 C_Programming]$ ./test_modified

 I'm kamal

I have added one more line. This comes from modifed code
[kamalma@test-1 C_Programming]$
------

3. Then execute following command to create a patch in the same directory.  :

 diff -u test.c test_modified.c > test.patch.1

Here test.path.1 will contain following :

-----
[kamalma@test-1 C_Programming]$ cat test.patch.1
--- test.c      2012-07-17 07:52:39.000000000 +0530
+++ test_modified.c     2012-07-17 07:54:41.000000000 +0530
@@ -3,5 +3,6 @@
 int main()
  {
  printf("\n I'm kamal \n");
+ printf("\nI have added one more line. This comes from modifed code\n" );
 return 0;
 }
[kamalma@test-1 C_Programming]$
-----

5. In order to apply patch to test.c file, you need to execute following command :

patch test.c < test.patch.1

Example :

---------
[kamalma@test-1 C_Programming]$ patch -u test.c < test.patch.1
patching file test.c
[kamalma@test-1 C_Programming]$ cat test.c
#include
#include
int main()
 {
 printf("\n I'm kamal \n");
 printf("\nI have added one more line. This comes from modifed code\n" );
return 0;
}
[kamalma@test-1 C_Programming]$
-----------

Reverting back :

----------
[kamalma@test-1 C_Programming]$ patch -R test.c < test.patch.1
patching file test.c
[kamalma@test-1 C_Programming]$ cat test.c
#include
#include
int main()
 {
 printf("\n I'm kamal \n");
return 0;
}
[kamalma@test-1 C_Programming]$
----------

You can do dry-run (test prior to be originally chaning codes in test.c)

patch -p0 --dry-run test.c < test.patch.1

6. Now recompiling the test.c program :

---------
[kamalma@test-1 C_Programming]$ gcc -o testnew test.c
[kamalma@test-1 C_Programming]$ ./testnew

 I'm kamal

I have added one more line. This comes from modifed code
[kamalma@test-1 C_Programming]$
---------

Read More

How to redirect tomcat log to syslog server?

I assume that you are going to use log4j tool to log the tomcat messages. In order to redirect these messages in syslog file, you should modify4j$CATALINA_BASE/lib/log4j.properties and make sure following entries are present :

 -------------------

log4j.rootLogger=INFO, CATALINA

# Define all the appenders
log4j.appender.CATALINA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.CATALINA.File=${catalina.base}/logs/catalina.
log4j.appender.CATALINA.Append=true
log4j.appender.CATALINA.Encoding=UTF-8
# Roll-over the log once per day
log4j.appender.CATALINA.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.CATALINA.layout = org.apache.log4j.PatternLayout
log4j.appender.CATALINA.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.LOCALHOST=org.apache.log4j.DailyRollingFileAppender
log4j.appender.LOCALHOST.File=${catalina.base}/logs/localhost.
log4j.appender.LOCALHOST.Append=true
log4j.appender.LOCALHOST.Encoding=UTF-8
log4j.appender.LOCALHOST.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.LOCALHOST.layout = org.apache.log4j.PatternLayout
log4j.appender.LOCALHOST.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.MANAGER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.MANAGER.File=${catalina.base}/logs/manager.
log4j.appender.MANAGER.Append=true
log4j.appender.MANAGER.Encoding=UTF-8
log4j.appender.MANAGER.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.MANAGER.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.HOST-MANAGER=org.apache.log4j.DailyRollingFileAppender
log4j.appender.HOST-MANAGER.File=${catalina.base}/logs/host-manager.
log4j.appender.HOST-MANAGER.Append=true
log4j.appender.HOST-MANAGER.Encoding=UTF-8
log4j.appender.HOST-MANAGER.DatePattern='.'yyyy-MM-dd'.log'
log4j.appender.HOST-MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.HOST-MANAGER.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Encoding=UTF-8
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %d [%t] %-5p %c- %m%n

# Configure which loggers log to which appenders
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost]=INFO, LOCALHOST
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager]=\
  INFO, MANAGER
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager]=\
  INFO, HOST-MANAGER

 -------------------

 Note : I have added SYSLOG logger and appender along with other properties in above file. For more information about log4j, you can log on to : http://tomcat.apache.org/tomcat-7.0-doc/logging.html#Using_Log4j


Once you added above file, you should restart tomcat application or instance. Then you'll able to see tomcat messages in system log (/var/log/messages). In order to redirect syslog client messages to syslog server, you should use following format in syslog.conf in syslog client machine :

 *.* @IP_OF_syslog server

Note : Please replace IP address with appropriate IP. Make sure syslog ports (514) are open in sysog server.

Read More

How to configure apache-tomcat on linux box?

Environment : RHEL 5.8
Package version : JDk 1.7.0_03
Apache tomcat : 6.0.35

Reference : http://www.puschitz.com/InstallingTomcat.html

JDK setup :

pwd
/jdk1.7.0_03

[root@vm13 jdk1.7.0_03]# export JAVA_HOME=/jdk1.7.0_03
[root@vm13 jdk1.7.0_03]# export PATH=$JAVA_HOME/bin:$PATH
[root@vm13 jdk1.7.0_03]# which java
/jdk1.7.0_03/bin/java
[root@vm13 jdk1.7.0_03]# java -version
java version "1.7.0_03"
Java(TM) SE Runtime Environment (build 1.7.0_03-b04)
Java HotSpot(TM) 64-Bit Server VM (build 22.1-b02, mixed mode)
[root@vm13 jdk1.7.0_03]#


Apache Tomcat setup :

[root@vm13 src]# rm -rf /var/spool/mail/root
[root@vm13 src]# groupadd tomcat
[root@vm13 src]# useradd -g tomcat -s /usr/sbin/nologin -m -d /home/tomcat tomcat
[root@vm13 src]# cd /var/lib
[root@vm13 lib]# tar zxvf /usr/local/src/apache-tomcat-6.0.35.tar.gz
[root@vm13 lib]# chown -R tomcat.tomcat /var/lib/apache-tomcat-6.0.35


/var/lib/apache-tomcat-6.0.35/bin/version.sh
Using CATALINA_BASE: /var/lib/apache-tomcat-6.0.35
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /var/lib/apache-tomcat-6.0.35/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
Server version: Apache Tomcat/6.0.35
Server built: Nov 28 2011 11:20:06
Server number: 6.0.35.0
OS Name: Linux
OS Version: 2.6.18-308.el5
Architecture: amd64
JVM Version: 1.7.0_03-b04
JVM Vendor: Oracle Corporation


Starting/Stopping Tomcat:

export CATALINA_HOME=/var/lib/apache-tomcat-6.0.35
You have new mail in /var/spool/mail/root
[root@vm13 lib]# export CATALINA_BASE=/var/lib/apache-tomcat-6.0.35
[root@vm13 lib]# su -p -s /bin/sh tomcat $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE: /var/lib/apache-tomcat-6.0.35
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /var/lib/apache-tomcat-6.0.35/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
[root@vm13 lib]#

$ps aux |grep java
tomcat 10711 9.4 10.1 463404 51696 pts/1 Sl 18:58 0:02 /jdk1.7.0_03/bin/java -Djava.util.logging.config.file=/var/lib/apache-tomcat-6.0.35/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/var/lib/apache-tomcat-6.0.35/endorsed -classpath /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar -Dcatalina.base=/var/lib/apache-tomcat-6.0.35 -Dcatalina.home=/var/lib/apache-tomcat-6.0.35 -Djava.io.tmpdir=/var/lib/apache-tomcat-6.0.35/temp org.apache.catalina.startup.Bootstrap start
root 10730 0.0 0.1 61232 724 pts/1 R+ 18:58 0:00 grep java
[root@vm13 lib]#


Stopping :

[root@vm13 lib]# su -p -s /bin/sh tomcat $CATALINA_HOME/bin/shutdown.sh
Using CATALINA_BASE: /var/lib/apache-tomcat-6.0.35
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /var/lib/apache-tomcat-6.0.35/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
You have new mail in /var/spool/mail/root
[root@vm13 lib]# ps aux |grep java
root 10763 0.0 0.1 61232 724 pts/1 R+ 18:59 0:00 grep java
[root@vm13 lib]#


Switching to Tomcat User Account :

[root@vm13 lib]# su - -s /bin/sh tomcat
-sh-3.2$ id
uid=502(tomcat) gid=503(tomcat) groups=503(tomcat) context=root:system_r:unconfined_t:SystemLow-SystemHigh
-sh-3.2$

Control+D to exit from tomcat shell


Setting Up First Tomcat JVM Instance :

[root@vm13 opt]# mkdir -p /opt/tomcat-instance/sales.example.com
[root@vm13 opt]# cd /opt/tomcat-instance/sales.example.com
[root@vm13 sales.example.com]# cp -a /var/lib/apache-tomcat-6.0.35/conf .
[root@vm13 sales.example.com]# ll
total 8
drwxr-xr-x 3 tomcat tomcat 4096 Apr 17 18:58 conf
[root@vm13 sales.example.com]# less conf/
Catalina/ catalina.properties logging.properties tomcat-users.xml
catalina.policy context.xml server.xml web.xml
[root@vm13 sales.example.com]# less conf/server.xml
[root@vm13 sales.example.com]# mkdir common logs temp server shared webapps work
You have new mail in /var/spool/mail/root
[root@vm13 sales.example.com]# chown -R tomcat.tomcat /opt/tomcat-instance
[root@vm13 sales.example.com]#

Most of the remaining steps are executed as the tomcat user. So make sure you switch from root to tomcat:

# su - -s /bin/sh tomcat
$ id
uid=1001(tomcat) gid=1001(tomcat) groups=1001(tomcat)
$

[root@vm13 sales.example.com]# su - -s /bin/sh tomcat
-sh-3.2$ pwd
/home/tomcat
-sh-3.2$ whoami
tomcat
-sh-3.2$ cat > /opt/tomcat-instance/sales.env << EOF
> export JAVA_HOME=/jdk1.7.0_03
> export PATH=\$JAVA_HOME/bin:\$PATH
> export CATALINA_HOME=/var/lib/apache-tomcat-6.0.35
> export CATALINA_BASE=/opt/tomcat-instance/sales.example.com
> EOF
-sh-3.2$ cat /opt/tomcat-instance/sales.env
export JAVA_HOME=/jdk1.7.0_03
export PATH=$JAVA_HOME/bin:$PATH
export CATALINA_HOME=/var/lib/apache-tomcat-6.0.35
export CATALINA_BASE=/opt/tomcat-instance/sales.example.com
-sh-3.2$

Configuring Tomcat Network Ports :

Since this is the first Tomcat instance that's being created here, the default port numbers can be left unchanged in $CATALINA_BASE/conf/server.xml (/opt/tomcat-instance/sales.example.com/conf/server.xml):



connectionTimeout="20000"
redirectPort="8443" />



Starting First Tomcat Instance :

sh-3.2$ source /opt/tomcat-instance/sales.env
-sh-3.2$ $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE: /opt/tomcat-instance/sales.example.com
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /opt/tomcat-instance/sales.example.com/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
-sh-3.2$

I have seen empty page : http://10.65.211.13:8080/

Relaying HTTP Port 80 Connections to Tomcat Port 8080 : that means if someone access http://10.65.211.13/ or http://10.65.211.13:80/, he'll get op of http://10.65.211.13:8080/

[root@vm13 sales.example.com]# iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080
[root@vm13 sales.example.com]# iptables -t nat -I OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 8080
[root@vm13 sales.example.com]# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere tcp dpt:http redir ports 8080

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere tcp dpt:http redir ports 8080
[root@vm13 sales.example.com]


Remove iptables rules : iptables -t nat -F

Setting Up a Web Application for First Tomcat JVM Instance :

-sh-3.2$ vi $CATALINA_BASE/conf/server.xml :

unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">



And the docBase attribute is set to mysales which stands for the application name within the URL, i.e. "http://10.65.211.13/mysales" or "http://10.65.211.13:8080/mysales".


Home Page for Web Application :

cat > $CATALINA_BASE/webapps/sales/index.html < > > "http://www.w3.org/TR/html4/loose.dtd">
>
>
>
>

Apache Tomcat Sales Home Page

>
>
> EOF
-sh-3.2$
Restarting First Tomcat Instance :

-sh-3.2$ source /opt/tomcat-instance/sales.env
-sh-3.2$ $CATALINA_HOME/bin/shutdown.sh
Using CATALINA_BASE: /opt/tomcat-instance/sales.example.com
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /opt/tomcat-instance/sales.example.com/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
-sh-3.2$ $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE: /opt/tomcat-instance/sales.example.com
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /opt/tomcat-instance/sales.example.com/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
-sh-3.2$

now access http://10.65.211.13/mysales/ , you'll see "Apache Tomcat Sales Home Page".

Deploying Java Servlet for Web Application in First Tomcat JVM Instance :

1. Setting up Java Servlet Layout :

-sh-3.2$ mkdir -p $CATALINA_BASE/webapps/sales/WEB-INF/classes
JAR Files :
-sh-3.2$ mkdir $CATALINA_BASE/webapps/sales/WEB-INF/lib

Creating a Java Servlet :

vi $CATALINA_BASE/webapps/sales/WEB-INF/classes/Sales.java

-sh-3.2$ cat $CATALINA_BASE/webapps/sales/WEB-INF/classes/Sales.java
import java.io.*;
import javax.servlet.*;
import javax.servlet.http.*;

public class Sales extends HttpServlet {

public void doGet(HttpServletRequest request, HttpServletResponse response)
throws IOException, ServletException
{
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.println("");
out.println("");
out.println("");
out.println("");
out.println("");
out.println("

Executing Sales ...

");
out.println("");
out.println("");
}
}

-sh-3.2$
The following command should now compile the Java servlet without errors:
-sh-3.2$ cd $CATALINA_BASE/webapps/sales/WEB-INF/classes
-sh-3.2$ javac -classpath "$CATALINA_HOME/lib/*" Sales.java
-sh-3.2$ ls
Sales.class Sales.java
-sh-3.2$

Configuring the Java Servlet :

-sh-3.2$ cat $CATALINA_BASE/webapps/sales/WEB-INF/web.xml


xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">


servlet_sales
Sales



servlet_sales
/execute





-sh-3.2$

Modify index.html :

-sh-3.2$ cat $CATALINA_BASE/webapps/sales/index.html
"http://www.w3.org/TR/html4/loose.dtd">

Apache Tomcat Sales Home Page

Execute Sales


-sh-3.2$

Testing and Executing the Java Servlet :

-sh-3.2$ source /opt/tomcat-instance/sales.env
-sh-3.2$ $CATALINA_HOME/bin/shutdown.sh
Using CATALINA_BASE: /opt/tomcat-instance/sales.example.com
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /opt/tomcat-instance/sales.example.com/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
-sh-3.2$ $CATALINA_HOME/bin/startup.sh
Using CATALINA_BASE: /opt/tomcat-instance/sales.example.com
Using CATALINA_HOME: /var/lib/apache-tomcat-6.0.35
Using CATALINA_TMPDIR: /opt/tomcat-instance/sales.example.com/temp
Using JRE_HOME: /jdk1.7.0_03
Using CLASSPATH: /var/lib/apache-tomcat-6.0.35/bin/bootstrap.jar
-sh-3.2$

OP : If you access : http://10.65.211.13/mysales/ , you'll see following :

Apache Tomcat Sales Home Page
Execute Sales

Now click on "Execute Sales", you'll see following :

Executing Sales ...

Disclaimer : I have referred above link and I have successfully setup it. In case you face any problem, please take a look at the link mentioned. I am not responsible for changing anything in above link. Use the commands at your own risk.

Read More

How to enable php in apache on RHEL ?

1. In order to enable php engine in apache, you should make sure php is installed in the system first. Then refer following steps :

locate libphp5.so

In most of cases, libphp5.so will be present inside /etc/httpd/modules

2. Add following in httpd.conf :

------------------
# Use for PHP 5.x:
LoadModule php5_module modules/libphp5.so
AddHandler php5-script .php

# Add index.php to your DirectoryIndex line:
DirectoryIndex index.html index.php

AddType text/html .php

# PHP Syntax Coloring
# (optional but useful for reading PHP source for debugging):

AddType application/x-httpd-php-source phps
------------------

OR

-----------------

LoadModule php5_module modules/libphp5.so


# Use of the "ZTS" build with worker is experimental, and no shared
# modules are supported.
LoadModule php5_module modules/libphp5-zts.so


#
# Cause the PHP interpreter to handle files with a .php extension.
#
AddHandler php5-script .php
AddType text/html .php

#
# Add index.php to the list of files that will be served as directory
# indexes.
#
DirectoryIndex index.php

#
# Uncomment the following line to allow PHP to pretty-print .phps
# files as PHP source code:
#
#AddType application/x-httpd-php-source .phps
-----------------

3. Restart apache.

Testing :

$ httpd -t

Create a php file inside web directory. Put the following content in that file like :

vi info.php

--------


--------

Then access the file through browser like http://localhost/info.php. If it shows all the enables php directives or modules then php works fine.

Read More

How to store sftp log messages in custom file on RHEL 6?

How to setup chrooted sftp account on RHEL 6?

$groupadd sftponly
$$useradd user123
$usermod -d /myhome -g sftponly -s /bin/false user123
$mkdir -p /chroots/user123 ; chmod -R 755 /chroots/user123
$mkdir /chroots/user123/myhome ; chown user123:sftponly /chroots/user123/myhome
$passwd user123

Also make sure below is successfully configured :


How to store sftp log messages in custom file on RHEL 6?

--------------------------------
1. Make sure "AUTHPRIV" facility is used in sshd_config file. Sample file looks like :


$ cat /etc/ssh/sshd_config :


-------------
Subsystem sftp internal-sftp -f AUTHPRIV -l VERBOSE

# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server

Match Group sftponly
ChrootDirectory /chroots/%u
AllowTcpForwarding no
ForceCommand internal-sftp -f AUTHPRIV -l VERBOSE
X11Forwarding no
-------------

2. Use proper additional socket file in /etc/sysconfig/rsyslog like :

vi /etc/sysconfig/rsyslog

SYSLOGD_OPTIONS="-m 0 -a /chroots/dev/log"

3. Following lines should be present in /etc/rsyslog.conf :

Add following lins :

-----------
$InputUnixListenSocketHostname internal-sftp
$AddUnixListenSocket /chroots/dev/log

# Log internal-sftp activity in a separate file

:programname, isequal, "internal-sftp" -/var/log/sftp.log
:programname, isequal, "internal-sftp" ~
:programname, isequal, "sshd" -/var/log/sftp.log
:programname, isequal, "sshd" ~

------------

4. Make sure you have done a hard link to socket file like :

$mkdir -p /chroots//dev
$ln /chroots/dev/log /chroots//dev/log

Note : Replace username in above command.

5. Restart rsyslog and sshd service :

$service rsyslog restart
$service sshd restart

Testing :

Open a terminal and execute following command on it like :

$tailf /var/log/sftp.log

Try to access SFTP account like :

$sftp username@IP
pw: << Enter password You'll see that all sftp logs messages are available in /var/log/sftp.log. Note : Note that all sshd and sftp related messages will be logged in above file. --------------------------- You may get following error : fatal: safely_chroot: stat("/chroots/"): Permission denied Solution : This is appearing due to SELinux issue. Please execute following commands : $chcon -t user_home_t /chroots/user123 $chcon -t user_home_t /chroots/user123/myhome $getsebool -a|grep sftp sftpd_anon_write --> off
sftpd_enable_homedirs --> off
sftpd_full_access --> off
sftpd_write_ssh_home --> off

Enable boolean like :

$setsebool -P sftpd_enable_homedirs 1

That's it.

Read More

How to setup multiple gateways for multiple Ethernets?

If you would like to use multiple gateways, you should use iproute package and define the gateway in two tables in conjunction with each interface.

Here is an example how to configure it :

Let's assume eth0 has gateway 10.1XX.69.1 and eth3's gateway is 10.1XX.66.1.

So, you should execute following steps :

echo 1 ISP1 >> /etc/iproute2/rt_tables
echo 2 ISP2 >> /etc/iproute2/rt_tables

The next step is to have some routing rules and routes:

For the ISP1 table:

ip route add default via 10.1XX.69.1 dev eth0 table ISP1
ip rule add from 10.1XX.69.0/24 table ISP1


For the ISP2 table:

ip route add default via 10.1XX.66.1 dev eth3 table ISP2
ip rule add from 10.1XX.66.0/24 table ISP2

------------------

In order to make above rules persistent, you should put them in rc.local file like :

cat /etc/rc.local

ip route add default via 10.1XX.69.1 dev eth0 table ISP1
ip rule add from 10.1XX.69.0/24 table ISP1
ip route add default via 10.1XX.66.1 dev eth3 table ISP2
ip rule add from 10.1XX.66.0/24 table ISP2

Note : You should replace the IP addresses in above commands.

Read More

How to create bridge on top of bonding in linux?

I have created following scripts :

------------
/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
NM_CONTROLLED=no
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no

---------------

/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth1
NM_CONTROLLED=no
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
TYPE=Ethernet
IPV6INIT=no
USERCTL=no

--------------

/etc/sysconfig/network-scripts/ifcfg-bond0

DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
IPV6INIT=no
BONDING_OPTS="mode=1 miimon=100 updelay=200 downdelay=200 primary=eth0"
BRIDGE=br0

-------------

/etc/sysconfig/network-scripts/ifcfg-br0

DEVICE=br0
TYPE=Bridge
BOOTPROTO=none
IPADDR=10.XXX.XXX.X7
NETMASK=255.255.255.XXX
GATEWAY=10.XXX.XXX.XXX
ONBOOT=yes
DELAY=0
IPV6INIT=no
-------------

Also add alias of bonding in modprobe.conf like :

# vi /etc/modprobe.conf

alias bond0 bonding

Restart network service :

# service network restart

That's it.

Read More

How to start apache uisng worker MPM?

1. Check which MPM is running currently :

/usr/sbin/apachectl -l

If you see worker.c in the list of loaded modules, your Apache is running Worker MPM. If prefork.c, it is running prefork.

2. To determine if apache has Worker MPM compiled in:

/usr/sbin/httpd.worker -l

If you see worker.c in the list of compiled-in modules, Apache can run Worker MPM.

3. Un-Comment HTTPD=/usr/sbin/httpd.worker in /etc/sysconfig/httpd
4. service httpd restart

Note : Please recompile php with thread-safe option(php still doesn't support worker MPM. You need mod_fcgid ), else it will throw error.

In order to temporary disable php, locate php

$locate php.conf

Then comment the php module and options. Then start apache.

Check again $/usr/sbin/apachectl -l

In order to compile mod_fcgid with worker MPM to be running php page, please refer following link :

http://www.lifelinux.com/installing-apache-with-worker-mpm-and-php-fastcgi/

Disclaimer : I have mentioned above link to help you on it. Note that I am not owner of this site or don't have privileges to edit this article. So, use it at your own risk.

Read More

How to create reverse DNS record in bind?

How to create reverse DNS record in bind :

Used Environment : RHEL 6

Steps followed :

1. Add following lines in /etc/named.conf :

---------
zone "0.168.192.in-addr.arpa" IN {
type master;
file "0.168.192.in-addr.arpa";
allow-update { none; };
};
---------

2. Create zone for this reverse record :

-------------------
$cat /var/named/0.168.192.in-addr.arpa

$TTL 600
@ IN SOA ns1.example.com. host.example.com. (
2012013001 ;Serial Number
86400 ;refresh
7200 ;retry
3600000 ;expire
86400 ;minimum

)

0.168.192.in-addr.arpa. IN NS ns1.example.com.
0.168.192.in-addr.arpa. IN NS ns2.example.com.
201 IN PTR example.com.
------------------

Testing :

-----------------------
$dig -x 192.168.0.201 @localhost

; <<>> DiG 9.7.3-P3-RedHat-9.7.3-8.P3.el6_2.2 <<>> -x 192.168.0.201 @localhost
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17992
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;201.0.168.192.in-addr.arpa. IN PTR

;; ANSWER SECTION:
201.0.168.192.in-addr.arpa. 600 IN PTR example.com.

;; AUTHORITY SECTION:
0.168.192.in-addr.arpa. 600 IN NS ns2.example.com.
0.168.192.in-addr.arpa. 600 IN NS ns1.example.com.

;; ADDITIONAL SECTION:
ns1.example.com. 86400 IN A 192.168.0.201
ns2.example.com. 86400 IN A 192.168.40.34

;; Query time: 0 msec
;; SERVER: ::1#53(::1)
;; WHEN: Wed Feb 22 15:00:32 2012
;; MSG SIZE rcvd: 137
---------------------------

DB of the zone example.com looks like :

--------------
$cat example.com
$TTL 24h

;
; zone data file
; comments can appear on any line after a semi-colon
;
example.com. IN SOA ns1.example.com. root.mail.example.com. (
2012013001 ; last updated January 30th,2012, once
24h
2h
4w
4d )

example.com. IN NS ns1.example.com.
example.com. IN NS ns2.example.com.
example.com. IN A 192.168.0.201
mail.example.com. IN A 192.168.0.201
ns1.example.com. IN A 192.168.0.201
ns2.example.com. IN A 192.168.40.34
www.example.com. IN CNAME example.com.
wwww.example.com. IN CNAME example.com.
ww.example.com. IN CNAME example.com.
example.com. IN MX 0 mail.example.com.

--------------

Read More

Which command will provide IO details of all processes in the system?

You can execute following command to get the details :

$ for i in {1..65353}; do if [ -f /proc/$i/io ] ; then echo "---------------------------------------------------"; echo "Process name :" ; cat /proc/$i/cmdline; echo "PID : $i" ; echo "IO Details:" ; cat /proc/$i/io ; fi; done

Read More

How to locally generate CA certificate and server - client certificate using CA on linux ?

Creating certification authority :

$cd /etc/newcerts
$openssl genrsa 2048 > ca-key.pem
$openssl req -new -x509 -nodes -days 1000 -key ca-key.pem > ca-cert.pem

NOTE: Last command will ask for details of certificate provider. So, provide short names

Creating certificate for server using above CA certificate :

$openssl req -newkey rsa:2048 -days 1000 -nodes -keyout server-key.pem > server-req.pem
$openssl x509 -req -in server-req.pem -days 1000 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 > server-cert.pem

NOTE: First command may ask for a password. Don't provide it. Just press enter key for two times.

Creating certificate for client using above CA certificate(similar like server) :

$openssl req -newkey rsa:2048 -days 1000 -nodes -keyout client-key.pem > client-req.pem .
$openssl x509 -req -in client-req.pem -days 1000 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 > client-cert.pem

NOTE : Provide details of client owner who will contact server. Short names may be same

Now client will contact to server using client-cert.pem and server will consult it its server-cert.pem and approve encryption.

Note that I have used same CA to generate cert for server as well as for client.

Read More

How to install mysql and configure SSL with it?

* How to install mysql and configure SSL with it?
* Received following error while I locally try to connect to mysql server over SSL.

[root@]# mysql --ssl-cert=/var/lib/mysql/openssl-md5/ca-cert.pem --ssl-key=/var/lib/mysql/openssl-md5/client-key.pem --ssl-cert=/var/lib/mysql/openssl-md5/client-cert.pem -u root -p -v -v -v

Enter password:

ERROR 2026 (HY000): SSL connection error

My used environment

* Red Hat Enterprise Linux 5.7

Required packages :

* perl-DBD-MySQL-3.0007-2.el5
perl-DBI-1.52-2.el5
mysql-server-5.0.77-4.el5_6.6
mysql-5.0.77-4.el5_6.6
mysql-5.0.77-4.el5_6.6

Resolution

1. Download all mysql related packages or use yum command to install the packages. You can use "rpm -ivh " to install them manually.
2. Start mysql.
4. Change mysql root password

Example :

[root@ /]# rpm -ivh perl-DBI-1.52-2.el5.x86_64.rpm

warning: perl-DBI-1.52-2.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186

Preparing... ########################################### [100%]

1:perl-DBI ########################################### [100%]

[root@/]# rpm -ivh mysql-5.0.77-4.el5_6.6.i386.rpm << This is needed on 64 bit os

warning: mysql-5.0.77-4.el5_6.6.i386.rpm: Header V3 DSA signature: NOKEY, key ID 37017186

Preparing... ########################################### [100%]

1:mysql ########################################### [100%]

[root@ /]# rpm -ivh mysql-5.0.77-4.el5_6.6.x86_64.rpm <<

warning: mysql-5.0.77-4.el5_6.6.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186

Preparing... ########################################### [100%]

1:mysql ########################################### [100%]

[root@ /]#

[root@ /]# rpm -ivh perl-DBD-MySQL-3.0007-2.el5.x86_64.rpm

warning: perl-DBD-MySQL-3.0007-2.el5.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186

Preparing... ########################################### [100%]

1:perl-DBD-MySQL ########################################### [100%]

[root@/]#



[root@ /]# rpm -ivh mysql-server-5.0.77-4.el5_6.6.x86_64.rpm
warning: mysql-server-5.0.77-4.el5_6.6.x86_64.rpm: Header V3 DSA signature: NOKEY, key ID 37017186
Preparing... ########################################### [100%]
1:mysql-server ########################################### [100%]
[root@ /



OR



$yum install mysql-server mysql

* Verify istalled packages :

$rpm -qa |egrep -i 'mysql|perl-DBD-MySQL|perl-DBI|mysql-server'

perl-DBD-MySQL-3.0007-2.el5

perl-DBI-1.52-2.el5

mysql-server-5.0.77-4.el5_6.6

mysql-5.0.77-4.el5_6.6

mysql-5.0.77-4.el5_6.6

* Start mysqld service :

$/etc/init.d/mysqld restart
Stopping MySQL: [FAILED]
Initializing MySQL database: Installing MySQL system tables...
OK
Filling help tables...
OK



To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system



PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h dhcp209-14.gsslab.pnq.redhat.com password 'new-password'



Alternatively you can run:
/usr/bin/mysql_secure_installation



which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.



See the manual for more instructions.



You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &



You can test the MySQL daemon with mysql-test-run.pl
cd mysql-test ; perl mysql-test-run.pl



Please report any problems with the /usr/bin/mysqlbug script!



The latest information about MySQL is available on the web at
http://www.mysql.com
Support MySQL by buying support/licenses at http://shop.mysql.com
[ OK ]
Starting MySQL: [ OK ]
[root@]# /etc/init.d/mysqld status
mysqld (pid 15065) is running...



[root@]# mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.0.77 Source distribution



Type 'help;' or '\h' for help. Type '\c' to clear the buffer.



mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| test |
+--------------------+
3 rows in set (0.00 sec)



mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A



Database changed
mysql> show tables;
+---------------------------+
| Tables_in_mysql |
+---------------------------+
| columns_priv |
| db |
| func |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| host |
| proc |
| procs_priv |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
+---------------------------+
17 rows in set (0.00 sec)



mysql> quit
Bye
[root@dhcp209-14 /]# netstat -plan |grep :3306
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 15065/mysqld
[root@dhcp209-14 /]#
----------

mysql> show variables like '%%ssl%%';
+---------------+----------+
| Variable_name | Value |
+---------------+----------+
| have_openssl | DISABLED |
| have_ssl | DISABLED |
| ssl_ca | |
| ssl_capath | |
| ssl_cert | |
| ssl_cipher | |
| ssl_key | |
+---------------+----------+
7 rows in set (0.00 sec)

mysql>

* Reset mysql root password :

$/usr/bin/mysqladmin -u root password 'mysql'

* Configurring SSL for mysql server and client(who will access server) :

$mkdir -p /etc/mysql/newcerts

$chown -R mysql:mysql /etc/mysql/newcerts

* Creating certificate autority :

$cd /etc/mysql/newcerts

$openssl genrsa 2048 > ca-key.pem
$openssl req -new -x509 -nodes -days 1000 -key ca-key.pem > ca-cert.pem

* Creating certificate for server using above CA certificate :

$openssl req -newkey rsa:2048 -days 1000 -nodes -keyout server-key.pem > server-req.pem
$openssl x509 -req -in server-req.pem -days 1000 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 > server-cert.pem

* Creating cerificate for client using above CA certificate(similar like server) :

$openssl req -newkey rsa:2048 -days 1000 -nodes -keyout client-key.pem > client-req.pem
$openssl x509 -req -in client-req.pem -days 1000 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 > client-cert.pem

* Make sure following entries are present in /etc/my.cnf file :

[mysqld]

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1



# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0
ssl # < ssl-ca=/etc/mysql/newcerts/ca-cert.pem # << Important.
ssl-cert=/etc/mysql/newcerts/server-cert.pem # << Important
ssl-key=/etc/mysql/newcerts/server-key.pem # << Important



[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

* Execute following commands (Used "mysql" password for user "mysql"):

$/etc/init.d/mysqld restart

$mysql -u root -p

$mysql> GRANT ALL ON *.* TO 'mysql'@'%' IDENTIFIED BY 'mysql' REQUIRE SSL;

* Testing :

$cd /etc/mysql/newcerts

$mysql --ssl-cert=ca-cert.pem --ssl-key=client-key.pem --ssl-cert=client-cert.pem -u root -p -v -v -v

Enter password: << pw = mysql

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.0.77 Source distribution



Reading history-file /root/.mysql_history
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.



mysql> SHOW STATUS LIKE 'Ssl_cipher';
--------------
SHOW STATUS LIKE 'Ssl_cipher'
--------------



+---------------+--------------------+
| Variable_name | Value |
+---------------+--------------------+
| Ssl_cipher | DHE-RSA-AES256-SHA | < +---------------+--------------------+
1 row in set (0.00 sec)



mysql> show variables like '%%ssl%%';
--------------
show variables like '%%ssl%%'
--------------



+---------------+-------------------------------------+
| Variable_name | Value |
+---------------+-------------------------------------+
| have_openssl | YES | < | have_ssl | YES | < | ssl_ca | /etc/mysql/newcerts/ca-cert.pem |
| ssl_capath | |
| ssl_cert | /etc/mysql/newcerts/server-cert.pem |
| ssl_cipher | |
| ssl_key | /etc/mysql/newcerts/server-key.pem |
+---------------+-------------------------------------+
7 rows in set (0.01 sec)



mysql> quit
Writing history-file /root/.mysql_history
Bye





[root@]# mysql --ssl-cert=/etc/mysql/newcerts/ca-cert.pem --ssl-key=/etc/mysql/newcerts/client-key.pem --ssl-cert=/etc/mysql/newcerts/client-cert.pem -u root -p -v -v -v
Enter password: << pw = mysql



Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.0.77 Source distribution



Reading history-file /root/.mysql_history
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.



mysql> show variables like '%%ssl%%';
--------------
show variables like '%%ssl%%'
--------------



+---------------+-------------------------------------+
| Variable_name | Value |
+---------------+-------------------------------------+
| have_openssl | YES |
| have_ssl | YES |
| ssl_ca | /etc/mysql/newcerts/ca-cert.pem |
| ssl_capath | |
| ssl_cert | /etc/mysql/newcerts/server-cert.pem |
| ssl_cipher | |
| ssl_key | /etc/mysql/newcerts/server-key.pem |
+---------------+-------------------------------------+
7 rows in set (0.01 sec)



mysql> SHOW STATUS LIKE 'Ssl_cipher';
--------------
SHOW STATUS LIKE 'Ssl_cipher'
--------------



+---------------+--------------------+
| Variable_name | Value |
+---------------+--------------------+
| Ssl_cipher | DHE-RSA-AES256-SHA | << Confirmed
+---------------+--------------------+
1 row in set (0.00 sec)


mysql>

mysql> quit

Read More

How to generate UDP packets and capture them?

Guys,

I had a chance to generate UDP packets and send them to another host. This was needed for testing a specific problem that I had faced. The problem is that I had to see huge Recv-Q at "netstat -apln|grep -i 'syslog' prior to RHEL 5.4. Now question is why does it show such huge value? But why do we see this at first place? No, we didn't see it at first place. Actually one monitoring software was complaining that there were lot of failure d UDP packets in "netstat -s" output and it also complained that number of such packets were also huge in Recv-Q. So, I had to investigate in detail.

As you know Recv-Q = Number of bytes received from network as well as from local machine and that will be processed by syslog, so syslogd was not processing received UDP pkts.

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
udp 653212 0 0.0.0.0:514 0.0.0.0:* LISTEN syslog

As per my knowledge 0 0.0.0.0 = any.any.any.any address. And syslog collects messages from /dev/log. Also klogd sends kernel error/warning messages to syslogd. Syslogd filters the messages as per rules defined in /etc/syslog.conf and redirects them to corresponding log file. Big question is that if rule doesn't match then do the messages will be really going to log files(say /var/log/messages)? I think they will go if we accepts network messages. Pass "-r" to syslogd ie /etc/sysconfig/syslog.

---------> | /dev/log, klogd ----->| ------->syslogd ------>filter------->system log, @remote_server
---------> | ----->|

My concept about receiving the packets :

NIC -->NIC_DRIVER---->hrd_IRQ---->soft_IRQ---->PRE_ROUTING---->packet_forwarding---->libpcap(tcpdump)---->hook(INPUT)--->SKB--->CPU

I tried to send UDP packets locally and captured UDP by tcpdump. I have used server.c and rawudp.c two programs. So, server.c should be running to listen udp packets whereas rawudp.c will send udp packets. I took help from a guy to get these programs. I was trying to do using perl but there were lot of dependencies. So, I avoided it to use it. My intenstion was to verify whether Recv-Q was increasing by hitting UDP packets to server. But this does't increase. Note that I sent raw UDP packets. tcpdump data gave clear picture about source, destination, source and destination ports.

Here are the exact source codes and testing procedures :

------server.c---------
#include
#include
#include
#include
#include
#include

#define BUFLEN 512
#define NPACK 10
#define PORT 9930

void diep(char *s)
{
perror(s);
exit(1);
}

int main(void)
{
struct sockaddr_in si_me, si_other;
int s, i, slen=sizeof(si_other);
char buf[BUFLEN];

if ((s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP))==-1)
diep("socket");

memset((char *) &si_me, 0, sizeof(si_me));
si_me.sin_family = AF_INET;
si_me.sin_port = htons(PORT);
si_me.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(s, &si_me, sizeof(si_me))==-1)
diep("bind");

for (i=0; i if (recvfrom(s, buf, BUFLEN, 0, &si_other, &slen)==-1)
diep("recvfrom()");
printf("Received packet from %s:%d\nData: %s\n\n",
inet_ntoa(si_other.sin_addr), ntohs(si_other.sin_port), buf);
}

close(s);
return 0;
}
------server.c--------


------rawudp.c-------
/*
Copyright 2010 Gabriel Serme

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
#include
#include
#include
#include
#include
#include
#include
#include
#include

#define LEN 512

typedef unsigned short u16;
typedef unsigned long u32;

//2nd
unsigned short csum (unsigned short *buf, int nwords);
uint16_t udp_checksum(const struct iphdr *ip,
const struct udphdr *udp,
const uint16_t *buf);



int main(int argc, char * argv[])
{

# if __BYTE_ORDER == __LITTLE_ENDIAN //reverse per 8 bits
printf("little endian\n");
#else
printf("big endian\n");
#endif

if(argc != 5)
{
printf("- Usage %s \n", argv[0]);
exit(1);
}else{
printf ("Args : \n"
"\tip source : %s:%s\n"
"\tip dest : %s:%s\n",
argv[1], argv[2], argv[3], argv[4]);
}
int PSOURCE = atoi (argv[2]);
int PDEST = atoi(argv[4]);
char * SOURCE = argv[1];
char * DEST = argv[3];
char * ANSWER = argv[5];
int s;
struct sockaddr_in daddr, saddr, answerip;
char packet[LEN];
/* point the iphdr to the beginning of the packet */
struct iphdr *ip = (struct iphdr *)packet;
struct udphdr *udp = (struct udphdr *)((void *) ip + sizeof(struct iphdr));
struct dnshdr *dns = (struct dnshdr *)((void *) udp + sizeof(struct udphdr));

if ((s = socket(AF_INET, SOCK_RAW, IPPROTO_RAW)) < 0) {
perror("error:");
exit(EXIT_FAILURE);
}

daddr.sin_family = AF_INET;
saddr.sin_family = AF_INET;
daddr.sin_port = htons(PDEST);
saddr.sin_port = htons(PSOURCE);
inet_pton(AF_INET, DEST, (struct in_addr *)&daddr.sin_addr.s_addr);
inet_pton(AF_INET, SOURCE, (struct in_addr *)&saddr.sin_addr.s_addr);

memset(daddr.sin_zero, 0, sizeof(daddr.sin_zero));
memset(saddr.sin_zero, 0, sizeof(saddr.sin_zero));
memset(udp, 0, sizeof(struct udphdr));
memset(ip, 0, sizeof(struct iphdr));

ip->ihl = 5; //header length
ip->version = 4;
ip->tos = 0x0;
ip->id = 0;
ip->frag_off = htons(0x4000); /* DF */
ip->ttl = 64; /* default value */
ip->protocol = 17; //IPPROTO_RAW; /* protocol at L4 */
ip->check = 0; /* not needed in iphdr */
ip->saddr = saddr.sin_addr.s_addr;
ip->daddr = daddr.sin_addr.s_addr;

udp->source = htons(PSOURCE);
udp->dest = htons (PDEST);

int sizedata = 100;
memset(((void *) udp) + sizeof(struct udphdr), 'A', sizedata);

int sizeudpdata = sizeof(struct udphdr) + sizedata;
ip->tot_len = htons(sizeudpdata + sizeof(struct iphdr)); /* 16 byte value */
udp->len = htons(sizeudpdata);

udp->check = udp_checksum(
ip,
udp,
udp);
printf ("Checksum : 0x%x\n", udp->check);
printf ("Sizes : \n\t[+] iphdr %d"
"\n\t[+] udphdr %d\n",
sizeof(struct iphdr), sizeof(struct udphdr));
printf ("Total size : %d\n", sizeudpdata);

int optval = 1;

if (setsockopt(s, IPPROTO_IP, IP_HDRINCL, &optval, sizeof(int))
< 0)
perror ("IP HDRINCL");

while(1) {
int sizepacket = sizeof(struct iphdr) + sizeudpdata;
if (sendto(s, (char *)packet, sizepacket, 0,
(struct sockaddr *)&daddr, (socklen_t)sizeof(daddr)) < 0)
perror("packet send error:");
else
printf("Sent packet\n");
sleep(2);
}
exit(EXIT_SUCCESS);
}

//http://www.linuxquestions.org/questions/linux-networking-3/udp-checksum-algorithm-845618/
//modified by Gabriel Serme
struct pseudo_hdr {
u_int32_t source;
u_int32_t dest;
u_int8_t zero; //reserved, check http://www.rhyshaden.com/udp.htm
u_int8_t protocol;
u_int16_t udp_length;
};

uint16_t udp_checksum(const struct iphdr *ip,
const struct udphdr *udp,
const uint16_t *buf)
{
//take in account padding if necessary
int calculated_length = ntohs(udp->len)%2 == 0 ? ntohs(udp->len) : ntohs(udp->len) + 1;

struct pseudo_hdr ps_hdr = {0};
bzero (&ps_hdr, sizeof(struct pseudo_hdr));
uint8_t data[sizeof(struct pseudo_hdr) + calculated_length];
bzero (data, sizeof(struct pseudo_hdr) + calculated_length );

ps_hdr.source = ip->saddr;
ps_hdr.dest = ip->daddr;
ps_hdr.protocol = IPPROTO_UDP; //17
ps_hdr.udp_length = udp->len;

memcpy(data, &ps_hdr, sizeof(struct pseudo_hdr));
memcpy(data + sizeof(struct pseudo_hdr), buf, ntohs(udp->len) ); //the remaining bytes are set to 0

return csum((uint16_t *)data, sizeof(data)/2);
}

/* Not my code */
unsigned short csum (unsigned short *buf, int nwords)
{
unsigned long sum;

for (sum = 0; nwords > 0; nwords--)
sum += *buf++;

sum = (sum >> 16) + (sum & 0xffff);
sum += (sum >> 16);
return ~sum;
}
--------rawudp.c--------

Compile programs :

$gcc -o server server.c
$gcc -o rawudp rawudp.c

You'll see server and rawudp files.

--------------------------------

Testing :

./server //server is ready to listen UDP which will be sent from rawudp prgram.
$tcpdump -i lo -X -vv 'port 9930' -w captured.pcap //I am sniffing it on another terminal.(Note that server port is 9930)

./rawudp 127.0.0.1 1111 127.0.0.1 9930 //sending raw packets to localhost on another terminal(destination is 127.0.0.1 and its port is 9930. source port is 1111 )
src src_port dst_h dst_p


$wireshark captured.pcap //Viewing captured UDP packets

------------------------------

Conclusion :

a) Firewall was enabled : $iptables -A INPUT -s 127.0.0.1 -p udp -j DROP ,
So, ./sever did't listen any udp packets. //Test was done

Ou put :

./server
Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA //16bytes

Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Received packet from 127.0.0.1:1112
Data: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

These messages will be stopped once I again execute "iptables -A INPUT -s 127.0.0.1 -p udp -j DROP". But I saw that number of packets capturing in tcpdump was increasing. And rawudp was continuosly sending udp packets.

Result :

[root@vm ~]# tcpdump -i lo -X -vv 'port 9930' -w gaby.dump3_blokupd
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
Got 37


[root@vm122 myUDP]# ./rawudp 127.0.0.1 1112 127.0.0.1 9930
little endian
Args :
ip source : 127.0.0.1:1112
ip dest : 127.0.0.1:9930
Checksum : 0x3317
Sizes :
[+] iphdr 20
[+] udphdr 8
Total size : 108
Sent packet


Sent packet
Sent packet
Sent packet
. . . .
---------------------

Why Recv-Q is so high :

1. DNS server was sending lot of UDP(LLMNR) packets.
2. Local machine was sending lot of UDP(LLC) packets.
3. Some machines which are in same subnet were sending UDP to other machines(Don't know why libpcap captured them. Did they route after passing through tcpdump? I still have doubt)
4. Increase size of rmem kernel memory.

Read More

How to configure rndc key with chrooted bind on linux?

Environment : RHEL 6.1 .

Here are the steps that I followed :

1. vi /etc/rndc.conf and following line

----
options {
default-server 127.0.0.1;
default-key "rndckey";
};

server 127.0.0.1 {
key "rndckey";
};

key "rndckey" {
algorithm "hmac-md5";
secret "secret key will be placed here";
};
---

2. cd /var/named/chroot/etc/

3. $dnssec-keygen -r /dev/urandom -a HMAC-MD5 -b 256 -n HOST rndc
4. cat *.private
5. copy the key from private file and put it in /etc/rndc.conf at "secret" line.
6. Do link :

$ln -s /var/named/chroot/etc/rndc.conf /etc/rndc.conf

7. vi /etc/named.conf put following :

----
controls {
inet 127.0.0.1 allow { 127.0.0.1; } keys { rndckey; };
};

key "rndckey" {
algorithm "hmac-md5";
secret "replace_keyhere";
};

----

8. Restart named and check status :

$rndc status

Output will look like :

$rndc status
version: 9.7.3-P3-RedHat-9.7.3-2.el6_1.P3.2
CPUs found: 1
worker threads: 1
number of zones: 20
debug level: 0
xfers running: 0
xfers deferred: 0
soa queries in progress: 0
query logging is OFF
recursive clients: 0/0/1000
tcp clients: 0/100
server is up and running

Read More

How to rotate sudo log?

Scenario : Sudo log files will keep the activities of sudo users and their login status in /var/log/sudolog.

Tested platform : RHEL 5

Solution :

1. Create a sudo log file and put it in /etc/sudoers. Then rotate it for 90 days. As an example :

$touch /var/log/sudolog

2. vi /etc/sudoers and add following lines :

-----
Defaults !syslog
Defaults logfile = /var/log/sudolog
-----

Then save it.

3. Rotating this log file :

vi /etc/logrotate.d/sudolog and put following :

------
/var/log/sudolog {
rotate 90
size 5M
postrotate
/usr/bin/killall -HUP syslogd
endscript
}
------

4. Now restart syslogd :

$service syslogd restart

This will keep maximum 5MB file of 90 different copies. Old copy will be removed first.

Read More