Jenkins – Continuous Integration (Development, Build, Test & Release)

Continuous Integration (CI)
Continuous Integration is a software development practice where team members integrate their work frequently, usually each individual integrates at least daily that leads to multiple integration everyday. Each integration is verified by an automated build that includes unit testing to find integration errors as fast as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
This article is about quick practical of Continuous Integration summarizing the technique and its current usage.
Continuous Integration is backed by several important principles and practices.
  • Maintain a single source repository
  • Automate the build
  • Make your build self-testing
  • Every commit should build on an integration machine
  • Keep the build fast
  • Test in a clone of the production environment
  • Make it easy for anyone to get the latest executable
  • Everyone can see what’s happening
  • Automate deployment
Continuous Deployment (CD)
Continuous deployment is very closely related to continuous integration and refers to the release into staging of software that passes the automated tests.
By adopting both Continuous Integration and Continuous Deployment, one not only reduce risks and catch bugs quickly, but also move rapidly to working software.With low-risk releases, you can quickly adapt to business requirements and user needs. This allows for greater collaboration between ops and delivery, fueling real change in your organisation, and turning your release process into a business advantage.
To create practical scenario of continuous integration we are going four instance or virtual machines as follows, diagram quick understanding
  1. Dev machine from where code changes are pushed
  2. Git (repository) machine for code management
  3. Jenkin machine, continuous integration (CI) server
  4. QA/UAT machine

CI

 

In nutshell, only four important commands require to complete entire process

  1. Git
    • Management tool where code resides
  2. MSbuild (Microsoft Build Engine)
    • build and create release package (.zip))
  3. Vstest.console
    • run unit testing of release assembly
  4. MSdeploy (Microsoft Deploy Engine)
    • deploy package and sql on QA machine

 

Jenkins configuration is so vital for successful completion of CI & CD automation, nothin to worry, it is not so complax if one knows process flow and associated commands, so let us go through it step by step

  1.  Repository setup to pull code
    • We are going to use git as code repository to pull code changes
    • ref  Git Setting
  2.  Set Build Trigger
    • We could choose periodically execute automatically
    • ref Poll SCM
  3.  Microsoft Build
    • MSBuild to clean and build solution here compilation errors are catched
    • ref  Build Setting
  4.  Unit Test
  5.  Set solution to publish and release package in zip format
  6.  Set MSBuild to create deployable package
  7.  Set Artifacts
  8.  How to do manual deployment
  9.  How to do auto deployment

Quick recapd videos for jenkins Configuration, Build, Test, Manual and Auto Deploy

ASP.NET MVC Partial View Validation From Client Side

I have come across one problem regarding client side validation for form submit where form resides in partial view
I have gone through many forums and articles but very few have explained it with complete example

Guess what I have done? Made one simple and small demo that I am going to explain
How unobtrusive validation works and problem associated with partial view
  • Unobtrusive validation is a unique feature. One can easily bind server side validation to client side validation provided by JQuery plugin.
  • Unobtrusive validation binds special attributes to field on load event, which are used to bind your own rules defined inside the model with rules implementation provided by JQuery library.
  • However, when you create a partial view fields in new form are without those special attributes and so validation is not working.
  • One of the solution is to load unobtrusive .js file again. It works but you need to load something twice wasting resources.
  • There is a method “parse” in unobtrusive plugin , which is responsible for creating validation
  • There is a method “parse” in unobtrusive plugin , which is responsible for creating validation attributes , adapters and also what’s really important errorPlacement adapter. Everything you need. Knowing all of this we can just use this “parse” on the form and everything should work fine.
Higher level we could divide this exercise in three sections moving forward step by step we are going to try to accomplish each section

Section 1 : Setup & Configuration

  • STEP 1 : Bundle following files in BundleConfig class BundleConfig
    • JQuery (for DOM management)
    • JQuer UI (for Dialogue box)
    • jquery.unobtrusive & jquery.validate (for validation)
    • “~/Content/css” for site.css (for site design)
    • “~/Content/themes/base/css” JQuery UI CSS (for dialogue box design)
  • STEP 2 : _layout page  _layout
    • Add STEP 1 bundles in _layout page
    • Note replace render section of script to bottom after jquery, jqueryui & jqueryval bunder
  • STEP 3 : Add appsettings in web.config  Web.config
    • set ClientValidationEnabled true
    • set UnobtrusiveJavaScriptEnabled true

#Section 1 is completed let us move forward to section 2

Section 2 : Back-end side

  • STEP 1: AddUserViewModel Model that includes following  Model
    • Properties
    • validate properties using DataAnnotations using System.ComponentModel.DataAnnotations
  • STEP 2: Create partial view incorporates followings  Partial View
    • Associated Model
    • Form elements without submit as it will be taken care of by dialogue box button
    • Validation message
  • STEP 3: HomeController has two responsibilities  HomeController
    • Render partial view (i.e. AddUserPartialView)
    • Handle submit data on form POST (i.e. AddUserInfo)

#Section 2 is completed let us move forward to section 3

Section 3 : Front end side (Index.chtml)

  • STEP 1: Add dialogue box container and adduser button Ref Code
  • STEP 2: Add button click event to open dialogue box (i.e. $(“#AddUserForm”).dialog({…}) Ref Code
  • STEP 3: Load rendered partial view in dialogue box
    • (i.e. $(‘#AddUserForm’).load(‘@Url.Action(“AddUserPartialView”, “Home”)’, function () {…}) ) Ref Code
  • STEP 4: Call “parse” in unobtrusive plugin, which is responsible for creating validation attributes Ref Code
  • STEP 5: Register form submit event on load of dialogue box (i.e. $form.submit(function () {..}) ) Ref Code
  • STEP 6: post serialized form data to /Home/AddUserInfo action by AJAX Ref Code
  • STEP 7: AJAX event has two handlers success and error Ref Code
  • STEP 8: Last but not the least add two buttons “Add User” and “Cancel” in dialogue box Ref Code
    • “Add User” call form submit
    • “Cancel” close dialog box

Source code:-
https://github.com/ganesha8shiva/MVCPartialViewValidation
Video Help :-

Note: If anybody faces error while building project

  • Right click on solution
  • Enable NuGet Package Restore (will take few minutes to finish)
  • Tools -> Library Package Manager -> Package Manager Console
  • Some NuGet Packages are missing from this solution. Click to restore.
  • Press Restore (will take several minutes)
  • Build Successful and works like charm

Video Help :-

Collaborative Software Development Model By Git Repository

Distribution feature of git repository gives leverage to scrum team to work in collaborative manner. Additionally, git with agile can give continuous development, testing and delivery.

Further more, we could separate out work by public-private repository based on developer role/responsibilities which aids to mitigate dispute among developers

Last but not the least, team could have their own unique work flow for development, testing and delivery

Here we go with an example, we have scrum team having below members for an example

  • UI/UX developer – (DEV1)
    • Responsibility to develop only front-end and integrate API
    • Can only pull API developer’s public repository in his/her private repository
  • API developer – (DEV2)
    • Responsibility to develop REST API based on UI/UX requirement
    • Can only pull UI/UX developer’s public repository in his/her private repository
  • Lead
    • Lead can only read both API and UI/UX developer public repository in his/her private repository
    • Push his/her separate work (e.g. configuration, static pages, etc..)
    • Does unit testing, engage API and UI/UX developers for fixes or change before giving it to QA
    • Lead can only push things on central server repository
  • QA / Tester
    • Only read Lead’s public repository in his/her private repository and do 360 degree testing

High level picture for this work flow 

 

git_dist_workflow

Auto Build & Prepare Release

If team use Agile (Scrum) sprint for development and use git as repository then team can easily make strategy of auto build and release

# Strategy
1 keep master and sprint related branches on separate line
2 keep sql changes separate in sprint specific folder under DBChanges

# Solution
Pretty simple logic,  first get changed list of files by executing diff command of git between master and any sprint specific branch. After getting this files we are going to create release folder and after that copy-paste files changed in sprint. Moreover, we explicitly copy-paste content of bin folder and DBChanges folder mentioned in setting file

# Core command is git command which gives list of changed files in development branch

git checkout sprint1
git diff --name-only master

# First, we are going to create setting file that has file ignore pattern, folder to get content of bin and last but not lease DBChange folder to get database related changes
>>> See Settings.xml

<Settings>
    <!-- Ignore file types while making release -->
    <ignore>.vb|.config|.vbproj|aspnet_client</ignore>    
    <!-- NOTE: Build should have project folders having bin sub-folder and need to be given in release -->
    <build>
        <folders>
            <folder>XYZRootWebApp</folder>
            <folder>XYZApp</folder>
        </folders>
    </build>    
    <!-- Location Of SQL Folder & Keep SQL Folder At The Same Level Of Solution -->
    <sql>        
        <folder>DBChanges</folder>        
    </sql>
</Settings>

# Second we start with powershell file which covers below code
1. Checkout and pull master branches
2. Load setting file
3. Checkout and pull development branch
4. Build solution
5. git diff command to get change file list
6. copy-paste change file list
7. copy-paste content of bin folder mentioned in setting file
8. copy-paste sql files

>>> See repareRelease.ps1

#---
# 1. Checkout and pull masters
#---
$cmdOutput=""
$cmdOutput = git checkout master -f 2>&1
if($cmdOutput -match 'error|conflict') {
    $cmdOutput
    exit
}

$cmdOutput=""
$cmdOutput = git pull 2>&1
if($cmdOutput -match 'error|conflict') {
    $cmdOutput
    exit
}

#---
# Terminate Script If There is no arg
#---
if(!$args) {
    write-host "Please enter branch name" -foreground "red" 
    exit
}

#---
# SET Branch
#---
$branch = $args[0]

#---
# Terminate Script If branch is not existsls
#---
$cmdOutput=""
$cmdOutput = git checkout $args[0]  -f 2>&1
if($cmdOutput -match 'error|conflict') {
    $cmdOutput
    exit
}

#---
# 2. load setting file and ignore folder
#---
#$build_folders = ('XYZRootWebApp','XYZWebApp')
#$ignore_pattern = '.vb|.config|.vbproj|aspnet_client'

[xml]$ConfigFile = Get-Content ".\Settings.xml"

#NOTE: Build should have project folders having bin sub-folder and need to be given in release
$build_folders = @()  # NOTE: list of folder 
$build_folders = $build_folders + $ConfigFile.Settings.build.folders.folder | Foreach-Object { $_}
if([int]$build_folders.length -eq 0) 
{
    write-host "Enter Valid Build Folders"  -foreground "red" 
    exit
}

# Ignore file types while making release
$ignore_pattern = $ConfigFile.Settings.ignore
if($ignore_pattern -eq '') 
{
    write-host "Enter Valid Ignore File Pattern"  -foreground "red" 
    exit
}

$sql_folder = $ConfigFile.Settings.sql.folder
if($sql_folder -eq '') 
{
    write-host "Enter Valid SQL Folder Path"  -foreground "red" 
    exit
}

#---
# sprint name 
#---
$sprint = $branch

#---
# 3 checkout and pull development branch
#---
git checkout -f $sprint
#git pull
$cmdOutput=""
$cmdOutput = git pull 2>&1

if($cmdOutput -match 'error|conflict|fatal') {
    $cmdOutput
    exit
}

#---
# collection projects that affected
# Purpose: move bin after build
#---
$projects = @()
$flag = 0
git diff --name-only master |  % {
    if($_.split('/')) {
       $_.split('/')[0]
    }
} | Sort-Object -Unique | %{     
    $file_path = $_        
    $flag=0
    $build_folders | Foreach-Object {
        if($file_path -match  $_) {
            $flag=1
        }
    } 
    if($flag -eq 1 ) {
        $projects = $projects + $_ 
    }    
}

#---
# 4. Build Projects and Solution
#---
msbuild ..\XYZSln.sln /t:Clean

$cmdOutput=""
$cmdOutput = msbuild ..\XYZSln.sln /t:Rebuild /p:WarningLevel=0 /p:Configuration=Release /clp:ErrorsOnly  2>&1
if($cmdOutput -match 'error') {    
    write-host $cmdOutput -foreground "red"     
    exit
}

#---
# build path
#---
$build_path = ".\" + $sprint

#---
# remove build folder if exists
#---
if(Test-Path -Path $build_path ) {
    Remove-Item $build_path -Force -Recurse
}

#---
# 5. git diff command to get change file list
# 6. copy paste change file list
#---
$flag = 0
git diff --name-only master | Foreach-Object {
    if($_ -notmatch $ignore_pattern) {       
        $file_path = $_
        $flag=0
        $build_folders | Foreach-Object {
            if($file_path -match  $_) {                
                $flag=1
            }
        }                
        if($flag -eq 1 ) {            
           $newfile=$build_path + "\" +  $_.replace("/","\") # OLD
           New-Item -ItemType File -Path  $newfile  -Force  # OLD           
        }        
    }
}

$flag = 0
git diff --name-only master | Foreach-Object { 
    if($_ -notmatch $ignore_pattern) {        
        $file_path = $_        
        $flag=0
        $build_folders | Foreach-Object {
            if($file_path -match  $_) {
                $flag=1
            }
        }          
        if($flag -eq 1 ) {        
            $source =  "..\" + $_.replace("/","\") # OLD
            $destination =   $build_path + "\" +  $_.replace("/","\")   # OLD  
            Copy-Item $source $destination -Force  # OLD            
        }        
    }
}

#---
# 7. copy paste content of bin folder mentioned in setting file
#---
$projects | % {
    $destination =   $build_path + "\" +  $_.replace("/","\")   + "\bin"    
    if(Test-Path -Path $destination ) {
        Remove-Item $destination -Force -Recurse
    }
}

$projects | % {        
    $source =  "..\" + $_.replace("/","\") + "\bin\*"
    $destination =   $build_path + "\" +  $_.replace("/","\")   + "\bin"    
    if(Test-Path -Path $destination ) {
        Remove-Item $destination -Force -Recurse
    }    
    if (!(Test-Path -Path $destination)) {
        New-Item $destination -Type Directory
    }
    Copy-Item $source -Destination $destination                
}

#---
# 8.copy paste sql files
#---
# Source Location /SQL/sprint{N}/*.sql
$source =  "..\" + $sql_folder + "\$sprint\*"
$destination =   $build_path
if(Test-Path -Path $source) {
    Copy-Item $source -Destination $destination
}

write-host "`n`n--------------------------------------" -foreground "magenta" 
write-host ">> " $args[0] "is ready !!!" -foreground "magenta" 
write-host "--------------------------------------" -foreground "magenta" 

#---
# TODO ZIP of build
#---

# Third, we make batch file for user facing
Ask which sprint to build and release
>>> See AutoRelease.bat

:: Enter Branch Name
set /p branch="Enter Branch Name: "

:: Execute ps1 file and pass parameters
Powershell.exe -executionpolicy remotesigned -File prepareRelease.ps1 %branch%

:: To keep console open
PAUSE

🙂 Here is entire code and can be scalped further as need of different team !!!
Git Hub AutoBuildRelease

Polymer Overrides Parent Method !!!

A short definition of Polymer Components
Decoupled bundle/component of script (Java Script), template (HTML) and design (CSS).

  • Developers can break big structure in small pieces of components
  • Developer can work/focus on single piece at time
  • letter on connect all components and achieve bigger picture

🙂 kind of Lego Game !!!

I want to discuss one aspect of Polymer, how to expose method to child element so child element can hook customized logic. This concept can be knows as one of the following single line statements
– method overriding
– late binding
– run-time polymorphism
– Overrides Parent Method

Keeping things short and plunge into code. 1, 2 & 3 let’s overrides urlFor method

1. Parent Element => bgs-parent

<polymer-element name="bgs-parent" attributes="name urlFor">
<template>
<button on-click="{{urlFor}}">{{name}}</button>
</template>
<script>
Polymer('bgs-parent', {
urlFor: function() {
alert("alert of " + this.name);
},
urlCaller: function() {
this.urlFor();
}
});
</script>
</polymer-element>

2. Child Element => bgs-child 

<link rel="import" href="../../bgs_components/bgs-parent/bgs-parent.html">
<polymer-element name="bgs-child">
<template>
<bgs-parent name="Parent1" urlFor="{{computeUrlFor}}"></bgs-parent>
<bgs-parent name="Parent2" ></bgs-parent>
</template>
<script>
Polymer('bgs-child', {
computeUrlFor: function() {
alert("alert of bgs-child")
}
});
</script>
</polymer-element>

3. Html Page that uses bgs-child element and click Parent1 and Parent2 button and see function overrides functioning  (i.e. index_function_overrides.html)

<!doctype html>
<html>
<head>
<title>App Router</title>
<link rel="import" href="/bower_components/polymer/polymer.html">
<link rel="import" href="/bgs_components/bgs-child/bgs-child.html">
</head>
<body>
<bgs-child></bgs-child>
</body>
</html>

Download Code: https://github.com/ganesha8shiva/polymer-examples/tree/master/polymer-method-overrides

Note: Please make sure download polymer bower_components and put at the level of bgs_components to make example work for you

Ref: https://www.polymer-project.org/0.5/docs/start/getting-the-code.html

Download Polymer: https://github.com/polymer/polymer

# Keep Your Folder Structure Like Below:

  • index_function_overrides.html
  • /bgs_components/bgs-parent/
  • /bgs_components/bgs-child/
  • /bowser_components

What Is Document Ready Event In JQuery ???

What is the role of below code in JQuery? For me it is always be part of understanding JQuery fundamentals

$(document).ready(function(){
   // jQuery methods go here...
});

Lets run below sample code and you wonder!!!
Why we get first alert “Alert 2” and second alert “Alert 1”
Interesting !!! Correct…

<!DOCTYPE html>
<html>
<head>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script>
$(document).ready(function(){
  alert("Alert 1");
});
alert("Alert 2");
</script>
</head>
<body></body>
</html>

Reason…guess????

$(document).ready(function(){
   // jQuery methods go here...
});

This code prevent any jQuery code from running before the document is finished loading (is ready).Good practice is to wait for the document to be fully loaded and ready before start working with it. This also allows developer to have JavaScript code before the body of document, in the head section.

Here are few examples of actions that can fail if methods run before the document is fully loaded:

  • Trying to hide an element that is not created yet
  • Trying to get the size of an image that is not loaded yet

Note: JQuery team has also made an even shorter method for the document ready event:

$(function(){
   // jQuery methods go here...
});

Both Left Join & Group By In Single LINQ Query

I had encountered with the complicated requirement, requirement was to display category and total products against it, I need both left outer join and group by on CategoryID of Categories table

I had provide below solution which I had tested on LINQPad Below is tested solution

I applied simple rule while I was coding and that is “Break one complex problem in small but reasonably simple problems” (Tip: Divide & Rule)

Step 1: Left Join Categories Products

var LO =  from a in Categories 
	       join b in Products			
	       on a.CategoryID equals b.CategoryID into leftjoin
	       from c in leftjoin.DefaultIfEmpty()			
	       select new {
		 ProductID = c.ProductID == null ? 0 : 1,
		 CategoryID = a.CategoryID,
		 CategoryName = a.CategoryName,
	       };

Step 2: Group by { CategoryID, CategoryName, ProductID }

var Group = from d in LO
		 group d by new {d.CategoryID, d.CategoryName, d.ProductID} into groupby
		 select new {
			CategoryID = groupby.Key.CategoryID,
			CategoryName = groupby.Key.CategoryName,
			Total = groupby.Sum(x => groupby.Key.ProductID)
		 };

Step 3: Iteration

foreach(var i in Group) {
	Console.WriteLine(i);
}

Output

Category ID: 1; Category Name: Beverages;      Total Products: 12
Category ID: 2; Category Name: Condiments;     Total Products: 12
Category ID: 3; Category Name: Confections;    Total Products: 13
Category ID: 4; Category Name: Dairy Products; Total Products: 10
Category ID: 5; Category Name: Grains/Cereals; Total Products: 7
Category ID: 6; Category Name: Meat/Poultry;   Total Products: 6
Category ID: 7; Category Name: Produce;        Total Products: 5
Category ID: 8; Category Name: Seafood;        Total Products: 12
Category ID: 9; Category Name: New;            Total Products: 0

Cheers !!!

JQuery Virtual Keyboard

A quick post on virtual keyboard which can be used to enter user name / password / Pine number, etc… Internet banking highly recommend to use while logging into Internet Banking Site.

Below are the advantages of virtual keyboard

  • The virtual keyboard can be an additional precautionary measure, protects you against malicious key logger programs which might have infected your  computer.
  • A key logger program can capture the keystrokes you enter in the physical keyboard to login which may be accessing your PC without your knowledge or permission.
  • A virtual keyboard prevents your username or password from being stolen, especially while using public computers, such as those available in Internet cafes. Also, the sequence in which the keys appear will change everytime the page is refreshed.

Before I start with an example go through Keith Wood’s solution for it build on top of JQuery

It is very clean, well documented, higly compatible and scalable solution, if I have to rate, I rate 5 out of 5 stars and highly recommended solution

Virtual Keyboard:  http://keith-wood.name/keypad.html

Quick Reference: http://keith-wood.name/keypad.html

jQuery Keypad Reference:     http://keith-wood.name/keypadRef.html

Last but not the least I am going through simple yet quick example before signing off

Step 0:- Create Folder structure as below

  • main
  • main/css
  • main/javascript

Step 1:-  Add jquery.keypad.css in main/css folder

Step 2:-  Add jquery-1.7.1.min.js and jquery.keypad.js in main/javascript folder

Step 3:-  Add index.html in main folder and copy-paste below code into index.html  “please read code carefully before copying it blindly :D”

<html>
<title>Virtual Keyboard</title>
<head>
 <link type="text/css" href="css/jquery.keypad.css" rel="stylesheet" />  
 <script type="text/javascript" src="javascript/jquery-1.7.1.min.js"></script>
 <script type="text/javascript" src="javascript/jquery.keypad.js"></script> 
    <script type="text/javascript">
        $(document).ready(function () {                    
            $("input[id$='chkKeyboard']").click(function () {            
                if (!$(this).is(':checked')) {
                    $('#txtUsername').keypad('destroy');
                    $('#txtPassword').keypad('destroy');
                } else {
                    $('#txtUsername').keypad({ keypadOnly: true, layout: $.keypad.qwertyLayout, randomiseAlphabetic: true, randomiseNumeric: true, randomiseOther: true });
                    $('#txtPassword').keypad({ keypadOnly: true, layout: $.keypad.qwertyLayout, randomiseAlphabetic: true, randomiseNumeric: true, randomiseOther: true });                    
                }
            });
        });
    </script>       

</head>

<body>
   <div style="text-align: left; margin-top: 15px;">                           
      Username : <input type="text" id="txtUsername" /><br/>
      Password : <input type="Password" id="txtPassword"></asp:TextBox><br/>
      <input type="checkbox" id="chkKeyboard" /> Use virtual keyboard (Recommended)<br/>
      <input type="button" id="btnSubmit" value="Submit"  />
   </div>
</body>

</html>

Step 4:- Open index.html in any browser (ie, chrome, firefox, etc…)
virtual_keyboard

Quote: “Great solutions are described in few words”