Jenkins and Git at Plaid Speed

Why mouse around with a funny Git Parameter Plugin when you can have something that shows a current list of branches and tags?

Why clone another copy of your Git repository to your Jenkins server when all you need is the code tree to build and deploy?

Following the lead in a Jenkins plugin comment by Pawel Ratajczak, I instead used the Extensible Choice Parameter plugin to launch an inline Groovy script to fetch current tag and branch data directly from a remote Git repo and present the list of choices to the user.  I then execute a git-archive command to fetch only the tip code and none of the Git repo metadata and historic data.

Did that sound too hard?  Still, maybe someone will steal this and make some better Jenkins plugins out of it.

Here’s how my Jenkins/Git testing went:

Set the Repo URL String

First, I needed a place to store the repo URL string.  Unfortunately, I need it in two distinctly different places (a Groovy script before build launch and a Bash script after) so I had to store it in two places, for now, for testing.  The best workaround I see to this at the moment is to write it to a file in the Groovy script for later pickup in the Bash script… but, well, let’s ignore that for now and look at my test case.  [I also tried embedding the URL in the Groovy parm list but the result was too messy so I dumped it.]

In a feeble attempt to keep this redundant data close together for easier maintenance, I put the first string in an ordinary “Choice Parameter” as seen in the top part of the screenshot below.  (I used this parameter type so that the user could see it but not fiddle with it.)

Jenkins job parameters

Extensible Choice Parameter plugin

Next, (assuming you’ve already added the Extensible Choice plugin to Jenkins) add “Extensible Choice” as a second parameter (as seen in the bottom part of the previous screenshot).  I’m not a Groovy programmer but, well, who cares?  Here’s my well-commented Groovy script to get you started.

This script will generate a list of all branches plus the 20 latest tags.  Adapt it to your needs.  It also adds a “choose one” option to force a selection.  [Written for readability, not brevity.]

def repo = "git@bitbucket.org:batboy/bat.git"

// Ask the Git repo to give up some metadata by pinging it with ls-remote commands.
// Then take the output and process it into list variables for return to the
// Extensible Choice plugin.

//def command = """git ls-remote --heads --tags ${repo}"""
def gitHeads = """git ls-remote --heads ${repo}"""
def gitTags = """git ls-remote --tags ${repo}"""
def commitID = ~/^\w+\s+/                    // regexp to match commit ID and whitespace after it

def proc = gitHeads.execute()
proc.waitFor()                               // Wait for the command to finish
def listHeads = proc.in.text.tokenize("\n")  // *out* from the external program is *in* for groovy
listHeads = listHeads.collect{it - commitID} // Strip the commit ID from each string

proc = gitTags.execute()
proc.waitFor()                               // Wait for the command to finish
def listTags = proc.in.text.tokenize("\n")   // *out* from the external program is *in* for groovy
listTags = listTags.reverse(mutate=false)    // List newest tags first and take the top 20
if (listTags.size() > 20) {
    listTags = listTags[0..19]
}
listTags = listTags.collect{it - commitID}   // Strip the commit ID from each string

def result = ["--choose one--"] + listHeads + listTags
return result

git-archive Shell Script

In the build section add an “Execute shell” build step.  Here’s my Bash script to get you started (which assumes you are using the AnsiColor plugin).  (Oh, yeah, I’m not much of a Unix guy either but, well, who cares?  I also fake DevOps expertise in case you haven’t noticed yet.  Jenkins and Git are brand new to me.)

This script executes a git-archive command to fetch the code into a clean directory I call “bundlespace”.  It then adds a VERSION file (for the heck of it) and then zips up the directory.  Alternatively, you can also use git-archive to request a zip outright (as seen in the commented-out line).  [Again, written for readability, not brevity.]

#!/bin/bash -x

if test "$GIT_TAG" == "--choose one--"
then
  echo -e "\e[31mERROR: User failed to choose a head or tag to build.\e[0m"
  exit 1
fi

# Tar an archive of the tree from Git, untar it locally.
rm -rf bundlespace
git archive --format=tar --prefix=bundlespace/ --remote=$GIT_REPO $GIT_TAG | tar -xf -
# Or, just grab a zip if you want.
#git archive -o bat.zip --prefix=bundlespace/ --remote=$GIT_REPO $GIT_TAG

# Inject the version number into a file.
TAG_VER="${GIT_TAG#refs/heads/}"
TAG_VER="${TAG_VER#refs/tags/}"
TAG_VER="${TAG_VER//\//_}.b$BUILD_NUMBER"
echo "$TAG_VER" > bundlespace/VERSION

rm -f bat.zip
zip -rq bat bundlespace

Run It

When you’re done setting up your project and select “Build with Parameters,” it looks like this:

Jenkins Build with Parameters

Whoa!  Like magic, eh?  Go ahead, give one or both of these ideas for a spin and see if you like them.

So… now what?

From here I was planning on taking this testing through a full CI and CD solution with a git-flow pipeline but I got distracted by testing Bamboo next and rethinking my whole universe.  So, good luck with this.  Maybe I’ll come back to it when something forces me back to Jenkins.

Regarding Bamboo, it presented some challenges as well with trying to set up a git-flow plan with tagging and versioning and whatnot but, in general, Bamboo is ENORMOUSLY more pleasant to work with.  Bamboo makes it easy to let it build every commit from various branches so then all I have to do is choose which builds to deploy without the need for a custom tag selector like I was working on here.  It was a complete turnaround in logic (that I was considering doing with Jenkins as well).  On the down side, git-archive does not accept SHA1 commit hashes with remote repositories (or maybe any repository?) so I can’t use this lean option when using this reverse logic where some builds may not be tagged or at the head (such as rebuilds).  I’m currently working up another post on my test solution in Bamboo so maybe I’ll explain more of this there.

Advertisements

7 thoughts on “Jenkins and Git at Plaid Speed

    1. It’s been a year since I had my head in Jenkins but as I recall I was testing this on a Windows master with a Linux slave. So, yes, it should all be slave friendly. I would be curious to hear if you find otherwise. Ultimately, we went with Bamboo instead of Jenkins. It just handled everything in a much smarter way.

      Like

  1. Hi
    I am trying to implement this but my list is not getting populated. the execute command in groovy is not retrieving any list of git heads or tags? i have copied the same script as above and just changed the repo to my bitbucket url for the repo.
    Pls help !!!

    Like

    1. Try the same commands from the command line — you’ll probably get a better sense of what is going on from there. For example:

      git ls-remote --heads git@bitbucket.org:accountname/reponame.git

      Most likely it is an authentication problem (as in, your SSH keys aren’t set up properly). Possibly a git version issue.

      Like

  2. I would like to do the same by bieng able to choose multiple git repositories from the first drop down (named GIT_REPO) based on which the second drop down should get populated. Kindly provide your insights on this requirement.

    Like

    1. I haven’t touched Jenkins since I wrote this, so I can only guess. I’m guessing it may not be possible because I’m not sure if Jenkins can have more than one parameter screen triggered. Alternatively, I’m guessing you can’t trigger dependency events on one screen to reload one parm based on another. Thus, consider doing it different and clone this setup for each repo. It may not be the best idea to make a single pipeline so broad. On that note, what I did here may be too broad.

      It may be possible to work out what you want in various ways but I’ll have to leave that to you to figure out.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s