SCM and evolution of DevOps

Software Configuration Management (SCM) is the application of Configuration Management (CM) principles in the context of Software Engineering (SE) in Computer Science (CS).

Software Configuration Management (SCM)  identifies and tracks the configuration of artifacts at various points in time and performs systematic control of changes to the configuration of artifacts for the purpose of maintaining integrity and traceability throughout the whole software life cycle.

It is essential for every project and applicable for every methodology of project Governance. To balance the demand of rapid software delivery and leveraging ROI, organizations started implementing automation everywhere. Domain and disciplines defined in SCM are categorized. Tools for every domain or for multiple disciplines were developed. All are interconnected as a Tool-chain that is expected to expedite software delivery.

In the era of rapid development and delivery, first we introduced Continuous Integration, though, before that, parallel development, source code merging, standardizing code commit by different hooks were in place. Many projects used customized script to integrate different software modules and packaged them together. A typical three team, three tire enterprise structure. Along with rapidness, automation also helps continuous improvement which is obvious. Over maturity of continuous integration (CI) we thought about Continuous Delivery (CD). Every project started talking about CI/CD but a few of them really succeed to Deliver Continuously. Integration of different software module is purely technical, however delivery involves many nontechnical, functional and client centric project related activities that demands agility, therefore, seems difficult to be continuous, so far monolithic application is concerned.

Merging Development, Operation and System Administration with the help of Automation introduced new concept called DevOps. The name reveals itself, Development+Operation, other than that there is no state forwards definition, rules, policies that guide us implementing the concept uniformly. Off course it has a principal, perhaps the goal to Developed as per requirement, Integrate, Deploy, Test automatically and getting Feedback for farther Improvement, then Release and finally Monitor for continuous operation. In background every change gets registered and ensure possibilities of rollback at any time. It seems like ITIL specified best practices of Service Lifecycle, however it is not a sequential framework, rather than it is Agile – an interactive approach where MVPs are passed through the life-cycle and delivered in a very short period. Altogether it’s an operation pipeline that accomplish the goal throughout a set of toolchains. 

Managing Automation and agility at the same time is not about simply letting loose a stream. To overcome the hurdle, monolithic application splits up into microservices. On the other hand, infrastructure becomes concise as a form of container. In combination of Microservices and Containerization organizations experiencing proven benefit of DevOps. Orchestration of huge containers is not a big hurdle today. Security and segregation mechanism imposed inside orchestration framework are simplifying coding complexity.

Principals of SCM are still being maintained silently inside DevOps engineering. Version Identification, Version Control, Artifact Versioning and Issue Tracking are essential disciplines of every software projects. Evolution of DevOps is continuing. Elimination or integration of tools name it differently, however core concept is automating SCM and IM to deliver rigorously Tested product in agile way. 

Monitor SVN commit using Perl script.

Usually it should be accomplished by configuring svn hooks, however in some cases when we do not have svn server access we are unable to notice if developers are committing as per SCM guideline.  Rather monitoring manually it’s better to automate the process which can check all commits in a day and notify the user/administrator if the commit in improper.

That is the purpose of the script. We use the same user ID and corresponding credential that is being used by Jenkins. To notify user or administrator the script will use SMTP. At the end the script will be scheduled as a user’s cron job at 22:30 PM every day.

Let’s have a look into the script…………

#!/usr/bin/perl

use strict;
use warnings;
use Net::SMTP;
use XML::Simple;
my $release = shift(@ARGV);
my $dt = `date +%Y-%m-%d`;
my $base = 'https://your.host.com/base/url';
chomp($dt);
my %hash;
my %nuhash;
my %users;
my $details;
my $xml = new XML::Simple;
my $contain = $xml->XMLin("/path/to/jenkins-home/hudson.scm.SubversionSCM.xml");
my $user = $contain->{credentials}->{'entry'}->[0]->{'hudson.scm.SubversionSCM_\

                                                      -DescriptorImpl_-PasswordCredential'}->{'userName'} . "\n";

my $auth = $contain->{credentials}->{'entry'}->[0]->{'hudson.scm.SubversionSCM_\

                                                      -DescriptorImpl_-PasswordCredential'}->{'password'};

my $encode = `echo $auth | python -m base64 -d`;
chomp($user);
chomp($encode);
if ($release =~ /REL_VER_/){
foreach my $repo ('app1', 'app2', 'app3') {
my $repo = "$repo" . "/branches/" . "$release";
my $URL = "$base" . "$repo\n";
my @to = ('your_mail_id@domain.com');
my $LOG = `svn --username=$user  --password=$encode log -r{$dt}:HEAD $URL`;

open LOG,'-|',"svn log -r{$dt}:HEAD $URL" or die $@;
my $i = 0;
while (&ltlog&gt) {
        next if /^----/;
        next if /^$/;
if (/^r/) {
  my($rev, $user) = split /\|/, $_;
                $hash{$rev} = '';
                $users{$rev} = $user;
                } else {
                my @keys = (keys %hash);
                my $key = $keys[$i];
                delete $hash{$key};
                $nuhash{$key} .= $_;
                }
        }

close(LOG);

foreach my $key (keys %hash) {
        if ($hash{$key} =~ /^$/){
        $nuhash{$key} .= '';
          }
        }

foreach my $tab (keys %nuhash) {
                if ($nuhash{$tab} =~ /^$/ || $nuhash{$tab} =~ /^\s+$/ ) {
                $details = `svn log -r$tab $URL`;
                &_send_mail('your_mail_id@domain.com',"$tab :" . "$users{$tab} \

                  => " . " NULL" , @to);
                        }
                }
$i++;
        }
}
#
# Check hash
#
#foreach my $k (keys %nuhash) {
#print "$k" . " => " . "$nuhash{$k}\n";
#}
#
# Send mail to the user
#
sub _send_mail {
my ($from, $sub, @to) = @_;

  my $smtp = Net::SMTP->new('YOUR.SMTP.SERVER');

  $smtp->mail($from);
  $smtp->to(@to);

  $smtp->data();
  $smtp->datasend("To: @to\n");
  $smtp->datasend("Subject: $sub\n");
  $smtp->datasend("\n");
  $smtp->datasend("$details\n");
  $smtp->dataend();

  $smtp->quit;
}

Secure Tomcat manager for production use

Tomcat manager is very useful for production environment when multiple applications are deployed in a single server. It helps to manage applications without restarting the server.  However, accessing HTML interface of manager application remotely is not a wise decision.

Therefore preventing remote access of tomcat manager using web browser and allowing access of tool-friendly plain text interface instead would be the best choice. This article illustrates a simple solution that has been designed to secure tomcat server for production use.

Tomcat provides a number of Filters to secure the server itself or an individual application. Please check here for more details. Our goal is to prevent web browser to access the Manager application from outside of local host. At the same time we must allow commands as a part of the request URI to get responses in the form of simple text that can be easily parsed and processed. Therefore filter should have logic to allow access based on HTTP request header. A very simple logic could be filtering Remote Address and embedded request properties available in HTTP request header as below.

private String checkHeader = "MyComp";
.
.
.
if (headerValue != null) {
   /*
    * Either connect from 127.0.0.1 or use "tomcatmanager" command
    */
   if (headerValue.equals(checkHeader) || remoteIp.equals("127.0.0.1")) {
    denyStatus = true;
   }
  }

Second part of this solution is a java utility which performs two basic functions. First it encrypts plain text password available in properties file and then decrypt the same again to connect tool-friendly text URI.  Properties file contain plain text user and password as per tomcat-user.xml. Whenever tomcat credential gets change, properties file should get modified accordingly. Another function is to setRequestProperty to prepare URLConnection.

urlConnection.setRequestProperty("referer", "MyComp");

Users with the manager-gui role should not be granted the manager-script or manager-jmx roles. Therefore, to use this client utility, configure tomcat-users.xml accordingly.

Demonstration:

Consider two systems A and B. System A is your Tomcat server where manager application is deployed and system B is your Desktop client. If you try to access HTML interface of tomcat manager from your desktop, it will redirect you to an error page, however if you run the utility it will show you all details in readable text format as below.

  

How it works?

As a client, when you hit web browser to access GUI interface of tomcat manager, filter checks Remote Address and redirect your request. However the filter will allow access of GUI interface from system A as, in such a case, request goes from localhost.

When we run the utility from system B, filter checks and found hardcoded request properties, therefore filter refrain Remote Address checking and allow access of plain text URI.

X and O puzzle for kids

A very simple game written in python is attached herewith for kids. You can  Download the game and unzip to run on your system.This is perfect for windows platform as a few Windows system commands like “color”, “TMP” file path etc have been used inside the script. However, modifying a few lines the script can be used on other OS like Linux. Please comments if you need source code. The script is converted into exe and zipped to attaché in blog.

Once you double click on the file it will show you the board. Please read the instruction carefully and place your position. Your choice will be placed as "X" and system will place "O" against your choice. 

If none of you win the game the result appears as below.

Different color at end of the game indicates the result. By any chance if you win the game Green Board will congratulate you.
let's enjoy the game and put your comments how is that!!!!

Search and test all links available in web page

Another small utility to check if all HYPERlinks inside a web page are working operationally. User can modify this utility and use according to their requirement.The nice part of this code is it stores search output in a file and display storage path on GUI console at the end of the process. To test the script simply store the code in a *.py file and double click on it. A popup will appear on desktop. Provide URL/web page and click on "show" button.
Searching will start and result will populate on windows command line console provided, python-3.3.2 is installed and placed in path properly.

#!python
'''
Created on Aug 19, 2013
@requires: Tested in Windows XP
@author: Jaydeb Chakraborty
@version: Python Version-3.3.2

'''
from tkinter import Tk, Label, Button, Entry
import urllib.request
import re
import os
import logging

""" 
Store all URLs in a file and get HTTP return code 
"""
def show():
    strn = entry.get()  
    if re.match('(?:ftp|https)://', strn):
        mesg="Currently HTTPS|FTP is not supported"
        t = Label(w, text=mesg)
        t.pack()
    else:
        l=strn.replace('http://', '')
        mesg='Please check output in ' + (os.environ.get('TEMP', '')) + '\INFO.log'
        t = Label(w, text=mesg)
        t.pack()                
         
    if strn:  
        stdinfo=((os.environ.get('TEMP', ''))+ '\INFO.log')
        logging.basicConfig(filename='%s' % stdinfo, format='%(asctime)s %(message)s', level=logging.INFO)
        logging.warning('*******  Accessing : %s' % strn + ' ********')
        logging.warning('**********************************************')
        local_filename, headers = urllib.request.urlretrieve('http://' + l)
        f = open(local_filename) 
        for lines in f:
            myString_list = [item for item in lines.split(" ")]
            for item in myString_list:      
                try:
                    o = re.search("(?Phttp?://[^\s]+)", item.expandtabs()).group("url")
                    url = re.sub(r'\?.*|".*', "", o)
                    conn = urllib.request.urlopen(url)
                    access = conn.getcode()
                    logging.warning('URL : %s' % url + ' -- Returncode is : %s' % access)
                    print(url, access)
                except :
                    pass 
                
       
w = Tk()
quitBotton = Button(w, text='Quit', command=quit).pack()
showBotton = Button(w, text='Show', command=show).pack()
Label(w, text="        Please provide URL...        ").pack()
entry = Entry(w)
entry.pack()
res = Label(w)
res.pack()
w.title('Test Links in web page')
w.maxsize(1000, 40000)
w.mainloop()

Windows schedule task: Restart Tomcat automatically

Managing tomcat with a bunch of application sometime becomes problematic. Without investing proper time for root cause analysis it is not possible to isolate the issue perhaps the application from the container which is troubling. Be it a production environment project cannot move forward w farther without resolving the issue. Well, I am talking about development environment where usually we do not think about the impact instead just bounce the server.

For UNIX/Linux it is easy at least can be managed using a few line of script and scheduling the job via cron. Maybe windows expert will do the same very easily. I thought to do the same thing via python. There are many reason and corresponding indication of hanging a server. Having a close look in server log one can found the common foot print that a server left every time it goes to hang state. High Memory utilization is one of them and very common.  Following steps demonstrate how to setup a job written in python in windows.

How to schedule jobs in windows?
>>schtasks /create /tn "PythonCron Job" /tr "path/to/your/executable Arg1 Arg2" /sc hourly
Above command will create job in "Control Panel--> Schedule Task".
In windows, schedule python script does not accept argument.
Therefore converted .py script to .exe file

How to convert .py to .exe?
To Convert .py to .exe for Python3.3 I used CX-Freeze.
For more details please have a look in http://cx-freeze.sourceforge.net/index.html
>>cxfreeze hello.py --target-dir dist
.........and schedule task as
>>C:\Path\To\Your\File.exe Arg1 Arg2

How to adjust schedule job in windows?
Open the task and adjust time in Advance Tab. Every task requires a user name and password
Administrator can use "NT AUTHORITY\SYSTEM"
How to check log?
To check if schedule job is running perfectly, go to "Control Panel--> Schedule Task"
Logs are usually available in "C:\WINDOWS\SchedLgU"
The script also populates logs ‘stdout/stderror’ in user "HOMEPATH". You can check logs if there is any error.
How to check login user in windows
>>echo %USERDOMAIN%\%USERNAME%

How to Test script in DEV?
To test this script please follows the following steps:
    1) Defile CATALINA_HOME in User Variable, if not defined.
    2) Start tomcat server
    3) Execute script in command line "python /path/to/the/script.py MAXMEM TOMCAT_PORT"
    4) For testing provide MAXMEM as low as for example '5'
        Example: >>>python /path/to/the/script.py 5 8080

#!python
'''
Python Version-3.3.2
Created on Aug 19, 2013
@author: Jaydeb Chakraborty
??? Restart Tomcat automatically
??? Monitor server behavior and modify script for different threshold 

!!! Assume CATALINA_HOME is defined and Tomcat starts via 'startup.bat'
'''
import logging
import sys
import re
import os
import socket
import subprocess
import urllib.request
from datetime import datetime
from time import sleep

class ServerProcess:
    def __init__(self, portnum):
        self.portnum = portnum
        self.port = (str(portnum).replace("'", "").replace("]", ""))
    def getPID(self):
        try:
            port = self.port
            p1 = subprocess.Popen('netstat -ano', stdout=subprocess.PIPE)
            p2 = subprocess.Popen('findstr :%s' % port, stdin=p1.stdout, stdout=subprocess.PIPE)
            result=p2.communicate()[0].split()
            pid=str(result[4]).lstrip("b,").replace(",", "").replace("'", "")
            return pid 
        except LookupError as l:
            logging.basicConfig(filename='%s' % stderror, format='%(asctime)s %(message)s', level=logging.DEBUG)
            logging.warning('Watch out! : %s' % l)
            return 0       
    def getMemory(self, pid):
            p3 = subprocess.Popen('tasklist /fi "MEMUSAGE ge 10" /fi "PID eq %s' % pid, stdout=subprocess.PIPE)
            mem = str(p3.communicate()[0].split()[17]).lstrip("b,").replace(",", "").replace("'", "")
            return mem
    def getConnect(self):
        try:
            port = int(self.port)
            sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            remoteServerIP  = socket.gethostbyname(socket.gethostname())
            conn = sock.connect((remoteServerIP, port))
            sock.close()
            return 'Listening'
        except socket.gaierror as name:
            logging.basicConfig(filename='%s' % stderror, format='%(asctime)s %(message)s', level=logging.DEBUG)
            logging.warning('Watch out! : %s' % name)
            sys.exit()
        except socket.error as s:
            logging.basicConfig(filename='%s' % stderror, format='%(asctime)s %(message)s', level=logging.DEBUG)
            logging.warning('Watch out! : %s' % s)
            return 'NotListening'
            sys.exit()
    def getHttpCode (self):
        port = self.port
        req=('http://localhost:'+ (str(Args[2]).replace(",", "").replace("'", "").replace("]", "")))
        status = urllib.request.urlopen(req)
        data=status.getcode()
        return data        
    def setLogentries(loglevel):
        if (loglevel == 'DEBUG'):
            logging.basicConfig(filename='%s' % stderror, format='%(asctime)s %(message)s', level=logging.DEBUG)
        else:
            logging.basicConfig(filename='%s' % stdout, format='%(asctime)s %(message)s', level=logging.INFO)

if __name__ == "__main__":
    global stderror, stdout 
    stdout=((os.environ.get('HOMEPATH', ''))+ '\INFO.log')
    stderror=((os.environ.get('HOMEPATH', ''))+ '\ERROR.log')
    Args = str(sys.argv).split()
    # Check Arguments
    if ((len(Args)) != 3):
        ServerProcess.setLogentries('DEBUG')
        logging.warning('Watch out! : Provide Hostname and Port name in arguments')
        print("Uses : python " + str(Args[0]).replace("['", "") + " MAXMEM PORT")
        sys.exit()

maxmem=str(Args[1]).replace(",", "").replace("'", "")        
process = ServerProcess(Args[2])
pid = process.getPID()
memory = process.getMemory(pid)      

for i in range(3):
    sleep(1)
    sys.stdout.flush()
    conn = process.getConnect()
    data = process.getHttpCode()
    """ If socket is listening and used memory is less than MAXMEM """
    if(( conn == 'Listening' ) and eval(maxmem)>eval(memory)  and ( data == 200 )):
        ServerProcess.setLogentries('INFO')
        logging.warning('Watch out! : Socket is %s ' % conn + 'when used memory is %s ' % memory + 'and return code is %s ' % data)
    elif(eval(maxmem) > eval(memory) and data == 200):
        ServerProcess.setLogentries('DEBUG')
        logging.warning('Watch out! : Can access url')              
    else:
        p = subprocess.Popen('tasklist /fi "MEMUSAGE ge 10" /fi "PID eq %s' % pid, stdout=subprocess.PIPE)
        proc = str(p.communicate()[0].split()[13]).lstrip("b,").replace(",", "").replace("'", "")
        os.system('taskkill /f /im %s' % proc)
        try:
            startup=(os.environ.get('CATALINA_HOME', '')+'\/bin\/startup.bat')
            os.system('%s' % startup)
            sys.exit()
        except OSError as sys:
            ServerProcess.setLogentries('DEBUG')
            logging.warning('Watch out! : Socket is %s' % sys) 

Software Configuration Management

SCM consist three domains and eight disciplines. Each discipline has a lot of tools that support the purpose itself.  Version Identification with syntactically and semantically strong scheme is primary requirement and initial part of  SCM. Selection of tools depends upon SDLC that a project follows. Not mandatory, however best practice is to  use advanced tools with advanced process. Trend shows that project use DVCS instead of VCS in Agile SDLC. However legacy Water Fall SDLC still prefers SVN. Consider Version Controlling discipline where Git continues to take-over developer's primary source code management system and at the expense of Subversion. Interestingly, Subversion continues to be the #1 SCM but just barely. If you combine Git and GitHub usage, Subversion leads by only 1.5% points. eclipse-community-survey-results-for-2013 predicts that next year Git and GitHub will be #1. The survey also sows decline of Ant usage, strongly indicates that Maven dominating Build and Deployment Management discipline. Another two disciplines - Artifact Version and Issue Tracking, that are tightly coupled with configuration management. Managing all input artifacts in build process is required a robust tool like Artifactory or Nexus.
Dependencies management is an important part of building source code. History of version control and history of Issue tracking both contain important information. Many popular issue tracking tools comes with VCS plug-in that helps to have a clear picture of committed changes in respect of a particular issue ID. To be facilitated by the feature of issue tracking tool changes should be committed to VCS with issue ID. Finally Change Control and Release Management put the final touch of product delivery process. Project and product gets change in every release. Customer change management in enterprise policy impacts on project delivery. To add a high value to the customer project change-control and customer change-management should go together. For example, in Agile software development process customer interaction is an important and prime part for having more clarity of the requirement. Release management can be divided in three sub category namely Release Engineering, Release management and Product management. A stable software product release follows 5+1 distinct Release Engineering  phases. Simply consider the five steps like Inception, Integration, Quality-Control and last-minute issue fixing. last (+1) phase of release engineering is to rebuild and delivery final release candidate version as the final release version. Maintaining all eight disciplines with proper documentation will provide real value in software development.