Use LFTP command line to transfer files from ftp server

Use LFTP command line to transfer files from ftp server

Example source server

Host: host.example.com
username: exampleuser
password: examplepass

Source folder: /public_html

To install lftp in ubuntu

#apt-get install lftp

Usage

To connect to the ftp server use

#lftp -u exampleuser,examplepass host.example.com -e ‘set ftp:ssl-allow no;’

Once your are connected you will see the below prompt

lftp exampleuser@host.example.com:~>

Use ls command to list files and directories

To copy the entire public_html directory use the following command

lftp exampleuser@host.example.com:~>mirror public_html /destination-path

Note: Lftp will not automatically create the destination-path. The destination path should exist

Kerberos-Based Apache SSO with Active Directory

Apache SSO with Active Directory

In this tutorial we will be setting up Active directory - Kerberos based sso with Apache.

Requirements

1. Active directory Server - Windows server 2008 R2
2. Apache server - Cent OS 7
3. mod_auth_kerb module in apache

Active Directory details

Domain Name : example.com
Domain Controller name: dc.example.com
IP address  : 192.168.0.100

Cent OS server details

Hostname : server.example.com
IP address: 192.168.0.101
DNS address: 192.168.0.100

Lets begin by installing apache with mod_auth_kerb module

#yum install httpd mod_auth_kerb krb5-workstation

Now lets configure and verify kerberos

#vim /etc/krb5.conf

------------------------------------------------------------------------
[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = EXAMPLE.COM
 default_ccache_name = KEYRING:persistent:%{uid}
 default_tkt_enctypes = rc4-hmac
 default_tgs_enctypes = rc4-hmac
 permitted_enctypes = rc4-hmac

[realms]
 EXAMPLE.COM = {
  kdc = dc.example.com
  admin_server = dc.example.com
 }

[domain_realm]
.example.com = EXAMPLE.COM
 example.com = EXAMPLE.COM

--------------------------------------------------------------------------


Now lets verify with the following command

For every kerberized host you have to create a service principal on the KDC. Lets create a service principle in the Command prompt on the windows server.

Before we create a service principal, we need to create a user account with any name which will be used for creating the service ticket. Le the name of the user be "krb5@example.com"

Use the below command to create the service principal

C:\>ktpass -princ HTTP/server.example.com@EXAMPLE.COM -mapuser krb5@EXAMPLE.COM -crypto RC4-HMAC-NT -ptype KRB5_NT_PRINCIPAL -pass long$longp2ass3word -out c:\temp\krb5.keytab

The ktpass command creates the pricipal HTTP/server.example.com@EXAMPLE.COM, maps it to AD account "krb5" and exports it's key to the keytabfile c:\temp\krb5.keytab. Copy that file to Centos server(apache server).


Check if the KDC sends correct tickets (kvno) by getting a serviceticket and using klist:

#kvno HTTP/server.example.com@EXAMPLE.COM
#klist -e

check this values against keytab krb5.keytab

#klist -e -k -t krb5.keytab

#kinit -k -t krb5.keytab HTTP/server.example.com

If this doesn't work then there is a problem with the Kerberos config.

Now lets create a test site with Apache and configure kerberos authentication

Copy the keytab file to /etc/httpd and change the ownership of the file to apache:apache

-------------------------------------------------
<VirtualHost *:80>

ServerName server.example.com

DocumentRoot /var/www/html/server.example.com

<Directory "/var/www/html/server.example.com">
AuthType  Kerberos
KrbAuthRealms EXAMPLE.COM
KrbServiceName HTTP
Krb5Keytab /etc/httpd/krb5.keytab
KrbMethodNegotiate on
KrbMethodK5Passwd off
require valid-user
</Directory>


</VirtualHost>
------------------------------------------------


Now lets enable Windows integrate authentication in Internet Explorer using the below link

https://docs.secureauth.com/display/KBA/Enable+Integrated+Windows+Authentication+(IWA)+in+Internet+Explorer


Now lets configure Firefox for sso

1. In the address bar of Firefox, type about:config to display the list of current configuration options.
2. In the Filter field, type negotiate to restrict the list of options.
3. Double-click the network.negotiate-auth.trusted-uris entry to display the Enter string value dialog box.
4. Enter the name of the domain against which you want to authenticate, for example, .example.com.
5. Repeat the above procedure for the network.negotiate-auth.delegation-uris entry, using the same domain.

Use the following link to enable Windows Integrated Authentication with chrome

https://www.specopssoft.com/configuring-chrome-and-firefox-for-windows-integrated-authentication/

Installing HA-Proxy on Cent OS 7

This example assumes that firewall and selinux is disabled. We are going to use the following three servers.

Apache1: 192.168.124.2 apache1.example.com
Apache2: 192.168.124.3 apache2.example.com
HAProxy: 192.168.124.10 haproxy.example.com



All three servers should be able to reach each other. Enter the following in the /etc/hosts file in all three nodes

192.168.124.2 apache1.example.com apache1
192.168.124.3 apache2.example.com apache2
192.168.124.10 haproxy.example.com haproxy

Now install apache service on the two apache nodes using the following command

#yum install httpd -y

Create a sample HTML file in the apache nodes in differentiate them and start the httpd service

#systemctl start httpd.service


Install and enable Epel repository on the haproxy node

#yum install epel-release -y

Install HA proxy

#yum install haproxy -y

The default configuration file haproxy.cfg is located in /etc/haproxy/

Lets make a backup of the original file

#cd /etc/haproxy

#cp haproxy.cfg haproxy.cfg.bkp


Now lets edit the configuration file. We need to clear the existing frontend/backend config and create our own config

Delete the following lines in haproxy.cfg

------------------------------------------------------------------------
frontend  main *:5000
    acl url_static       path_beg       -i /static /images /javascript /stylesheets
    acl url_static       path_end       -i .jpg .gif .png .css .js

    use_backend static          if url_static
    default_backend             app

backend static
    balance     roundrobin
    server      static 127.0.0.1:4331 check

backend app
    balance     roundrobin
    server  app1 127.0.0.1:5001 check
    server  app2 127.0.0.1:5002 check
    server  app3 127.0.0.1:5003 check
    server  app4 127.0.0.1:5004 check
------------------------------------------------------------------------

Now add the following lines
-------------------------------------------------------------------------
#webapp1 is the name we give for our frontend
frontend webapp1
#Makes the haproxy service listen on port number 80
bind *:80
#defining the backen name
default_backend webapp1_servers
#send X-Forwarded-For header
option  forwardfor


#Backend configuration
backend webapp1_servers
#Load balencing protocol
balance     roundrobin
server  apache1 192.168.124.89:80 check
server  apache2 192.168.124.51:80 check
-------------------------------------------------------------------------

Note: The X-Forwarded-For request header helps you identify the IP address of a client when you use an HTTP or HTTPS load balancer. Because load balancers intercept traffic between clients and servers, your server access logs contain only the IP address of the load balancer.





Save the config file and restart haproxy service

#systemctl restart haproxy.service


Now hit the haprosy node's IP address in the browser and it should direct us to one of the web servers.

Install and Configure Nginx with php-fpm on Cent OS 7.

Install Epel repository

#yum install epel-release -y

Install nginx and php

#yum install php php-fpm php-gd nginx -y

#systemctl start nginx.service

#systemctl start php-fpm.service

At this point we should be able to get test page for Nginx server when accessed in the browser. http://localhost

The default document root for nginx is /usr/share/nginx/html/

Now lets create a php info file in /usr/share/nginx/html

#vim phpinfo.php
------------------
<?php phpinfo(); ?>
------------------

Now try to open the file in the browser. http://localhost/phpinfo.php

The file will get downloaded instead of getting executed. To fix this we need to include the following configuration in /etc/nginx/nginx.conf. This will pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 for execution.

-----------------------------------------------
location ~ \.php$ {
        root           /usr/share/nginx/html;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME   $document_root$fastcgi_script_name;
        include        fastcgi_params;
}
-----------------------------------------------


Restart nginx and php-fpm service

#systemctl start nginx.service

#systemctl start php-fpm.service

Now we should be able to open http://localhost/phpinfo.php


Now Lets create another server block with a document root of /var/www/html

Create example.conf  in /etc/nginx/conf.d/example.conf

Include the following config in the example.conf

----------------------------------------------------------------
server {
        listen       80;
        server_name  example.com;
        root         /var/www/html;
        index index.php;

location ~ \.php$ {
        root           /var/www/html;
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME   $document_root$fastcgi_script_name;
        include        fastcgi_params;
}

}
-----------------------------------------------------------------

#mkdir -p /var/www/html

#vim /var/www/html/index.php
-----------------------
<? phpinfo(); ?>
------------------------

Save the file and restart nginx service and you should be have the example.com site up and running.

Split large sql file into multiple tables

The below bash script will split a large Mysql dump file into multiple files depending upon the number of tables in it

Usage
#./<script> <dump.sql>


#######################################################################
#!/bin/bash

####
# Split MySQL dump SQL file into one file per table
# based on http://blog.tty.nl/2011/12/28/splitting-a-database-dump
####

if [ $# -ne 1 ] ; then
  echo "USAGE $0 DUMP_FILE"
fi

csplit -s -ftable $1 "/-- Table structure for table/" {*}
mv table00 head

for FILE in `ls -1 table*`; do
      NAME=`head -n1 $FILE | cut -d$'\x60' -f2`
      cat head $FILE > "$NAME.sql"
done

rm head table*

########################################################################

Note: I did not create this, found it online

Cron with Pacemaker

Clustering cron daemon with pacemaker on Cent OS 7

This is a two node cluster with virtual IP and cron daemon as resources. The crontab has to be edited manually in both nodes to have the same task scheduled in it

node1.example.com
node2.example.com

Add the following to the /etc/hosts file in both nodes so that they are able to reach each other

X.X.X.X       node1.example.com        node1
X.X.X.X       node2.example.com        node2

Before starting with the cluster stop and disable the cron service in both nodes

#systemstl stop crond,service

#systemctl disable crond.service

Now install the pre-requisites on both nodes

#yum install -y pcs pacemaker corosync cman wget

Now set password for the hacluster user on both nodes

#passwd hacluster

Now start pcsd service and enable it on both nodes.

#systemctl start pcsd.service
#systemctl enable pcsd.service

Now authorize the cluster nodes. Will ask for username and password. Use "hacluster"

#pcs cluster auth node1 node2

Now create the cluster

#pcs cluster setup --name MYCLUSTER node1 node2
#pcs cluster start --all

Now disable STONITH and quorum as it is not required for a two node setup

#pcs property set stonithpcs resource create CRON systemd:crond-enabled=false

#pcs property set no-quorum-policy=ignore

Now add the resources. We need a virtual IP and tomcat resource

#pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=x.x.x.x cidr_netmask=24  op monitor interval=30s

 #pcs resource create CRON systemd:crond

Now to make both resources run in the same node we must have a constraint as follows

# pcs constraint colocation set VirtualIP CRON

Set the order in which the resources has to load

#pcs constraint order set VirtualIP CRON

All done. Now stop the cluster and start it.

#pcs cluster stop --all

#pcs cluster start --all

#pcs status

Installing KVM(libvirt) on Ubuntu 14.04

OS: Ubuntu 14.04 - 64 bit

Install the prerequisites and bridge utils for creating the bridge interface

apt-get install vim aptitude bridge-utils

Now lets configure the bridge interface
Change the eth0 interface to manual, create the bridge interface and link the bridge interface with eth0. After configuring the bridge interface the host machine will be connected to eth0 which inturn will be connected to the bridge. Now any traffic originating from the host machine will go through the bridge interface


vim /etc/network/interfaces
----------------------------------------------------------------
auto eth0
iface eth0 inet manual


auto br0
iface br0 inet static
address        x.x.x.x
netmask        255.x.x.x
gateway x.x.x.x
dns-nameservers        8.8.8.8 4.2.2.2
#bridge configuration
bridge_ports eth0
#turning off spaning tree
bridge_stp off
bridge_maxweight 0
bridge_fd 0
----------------------------------------------------------------

Now lets intall KVM

$aptitude install qemu-kvm qemu-system

Now install the virt-manager

$aptitude install virt-manager

After installation we can start virt manager by issuing the following command

$sudo virt-manager

This will launch the Virtual Machine Manager