How To Set Up Continuous Integration with Jenkins and Node.js

As you’ve probably guessed by my previous two articles I love writing (about) Node.js. What isn’t great, however, is creating deployment artifacts every time you check something in to GitHub and want to deploy. To make that process a little easier there are many tools available, and of of those is Jenkins, the friendly butler. In this “How-to” I’ll walk you through using Jenkins with the tibcli utility to deploy Node.js apps to TIBCO Cloud Integration every time updates are pushed to GitHub.

Some assumptions

There are a few assumptions that I’ve made while writing this tutorial, which should cover the majority of the people reading it. If you have any questions, feel free to post them at the TIBCO Community!

• You’re using GitHub to store your projects and you’ve got a repo for your Node.js app
• You’re kinda familiar with Jenkins
• You’ve modeled an API Spec on TIBCO Cloud Integration
• You’ve downloaded the tibcli utility from TIBCO Cloud Integration and the user that will run the JENKINS server has to login with the tibcli at least once

Python

Wait, Python?! I though we were doing Node.js! We are, don’t worry! There will be a few tasks that need to be executed during the build and deployment and one of them is obviously using the tibcli utility to deploy the app. Because the tibcli utility works in an interactive mode, we’ll use a python script to carry out the different build tasks and start the tibcli and push the app to TIBCO Cloud Integration. You’ll need to install the python module pexpect first which you can do with the command

sudo pip install pexpect

You can copy and modify the python script below. Be sure to save it in a location you’ll remember with a name you’ll remember too. In my case I’ve named the file server.py.

### Imports
import sys
import os
import shutil
import zipfile
import pexpect

### Constants
DEPLOYMENT_PATH = './deployment'
APP_NAME = sys.argv[1]
TIBCLI_PATH = sys.argv[2]
APP_PUSH_CMD = 'tibcli app push'
PASSWORD = sys.argv[3]

def replace_unicode(cmd_output):
    cmd_output = cmd_output.replace('\b', '')
    cmd_output = cmd_output.replace('\x1b', '')
    cmd_output = cmd_output.replace('[32m', '')
    cmd_output = cmd_output.replace('[31m', '')
    cmd_output = cmd_output.replace('[0m', '')
    return cmd_output

def with_interactive_login(child):
    cmd_output = str(child.before)
    child.sendline(PASSWORD)
    cmd_output += str(child.after)
    return cmd_output

def zipdir(path, ziph):
    for root, dirs, files in os.walk(path):
        for file in files:
            ziph.write(os.path.join(root, file))

if not os.path.exists(DEPLOYMENT_PATH):
    os.makedirs(DEPLOYMENT_PATH)

if os.path.exists('./' + APP_NAME + '/node_modules'):
    shutil.rmtree('./' + APP_NAME + '/node_modules')

shutil.copy2('manifest.json',DEPLOYMENT_PATH + '/manifest.json')

zipf = zipfile.ZipFile(DEPLOYMENT_PATH + '/app.zip', 'w', zipfile.ZIP_DEFLATED)
zipdir('./' + APP_NAME, zipf)
zipf.close()

cmd_output = ''

child = pexpect.spawn(TIBCLI_PATH + '/' + APP_PUSH_CMD,cwd=DEPLOYMENT_PATH)

if child.expect(["Password", pexpect.EOF, pexpect.TIMEOUT], timeout=300) == 0:
    cmd_output = with_interactive_login(child)
else:
    print("command time out occur")
    cmd_output += str(child.before)
cmd_output = replace_unicode(cmd_output)
print(cmd_output)

Getting your butler

Jenkins is a self-contained, open source automation server which can be used to automate all sorts of tasks such as building, testing, and deploying software. As much as I like new shiny things, for this tutorial I’ve chosen the Long-term Support Release (LTS). You can download and install Jenkins for just about any type of Operating System and there even is a Docker container available.

Note: It is definitely a good idea to secure your installation of Jenkins. There are a lot of good tutorials out there on securing Jenkins so I’ll skip that for now.

Plugins

Jenkins has many, many plugins available that allow you to do just about anything. If you’ve installed the latest version of Jenkins there is only one additional plugin that we’ll need to complete this tutorial:

• NodeJS Plugin

You can install new plugins by going to Manage Jenkins -> Manage Plugins and search for the plugins on the Available tab.

Connect to GitHub

I’ll use GitHub to store the source code of my project. We need to configure Jenkins to be able to access the repositories on GitHub and clone them. While Jenkins is able to pull from GitHub periodically it is much cooler to have GitHub tell Jenkins when updates are made to the project. To allow Jenkins to connect to GitHub you’ll need a Personal Access Token. You can find that in the Settings menu of GitHub. You’ll need to give it access to:

• repo
• notifications
• user

After clicking Generate token, be sure to save it somewhere! You’ll need it later on :)

Now let’s go back to Jenkins and make sure that ‘he’ understands to connect to GitHub as well. Go to Credentials -> System and add two new Global credentials:

• The first is for the GitHub Plug-in in Jenkins
  • Kind: Secret text
  • Scope: Global
  • Secret: The Personal Access Token from GitHub
  • ID: Something to remember this credential by
  • Description: A good description is helpful to remember this credential by
• The second will be for the Jenkins project accessing your GitHub repo
  • Kind: Username with password
  • Scope: Global
  • Username: Your GitHub username
  • Password: The Personal Access Token from GitHub
  • ID: Something to remember this credential by
  • Description: A good description is helpful to remember this credential by

Now go back to Manage Jenkins -> Configure System and scroll down to the GitHub section. Select Add GitHub server and select the first credential you created in the dropdown. You can select Test Connection to make sure everything works and you can leave the rest as default. Hit Save and you’re done with this section!

Adding Node.js to Jenkins

Now that GitHub is completely configured we also need to set up the Node.js installation for Jenkins. To do this go to Manage Jenkins -> Global Tool Configuration and scroll a bit down to the NodeJS section. The NodeJS plugin that we installed earlier allows you to install and manage different versions of Node.js and use those different versions when you’re building apps. You can add a new Node.js installation and if you keep the box install from nodejs.org selected you can choose the version that you want to work with. As soon as you hit Save it will install your new version.

Setting up CI and CD

Great! Now that we’ve got the set up out of the way let’s focus on setting up Continuous Integration for the Node.js project you have. To do that we’ll need to add a New Item. Enter any item name that you want (this will be the name of your project) and be sure to select Freestyle project. Now let’s configure the rest!

Source Code Management

In the SCM section select Git and paste in the repository URL for your GitHub repository. As credentials you now need to select the second set of credentials you created a while ago. The rest you can leave as default

Build Triggers

In the Build Triggers section you need to choose GitHub hook trigger for GITScm polling to allow Jenkins to inject a webhook trigger into your repository (which is why we did all the complicated set up earlier). That webhook will make sure that whenever a new commit is done a new build will be triggered

Build Environments

Be sure to check Provide Node & npm bin/ folder to PATH and select the version of Node.js you had configured to make sure that Jenkins is able to understand your Node.js app.

Build

This is where we work our magic! This section will allow us to build the zipfile that needs to be sent to TIBCO Cloud Integration. To do so add a build step Execute shell with the below code:

# Copy the deployment script to this folder
cp /path/to/server.py .
python server.py <YOUR APPNAME> <LOCATION OF TIBCLI> <YOUR PASSWORD>

Post-build Actions

To save a bit of diskspace we’ll also configure two Post-build actions. Add a step for Archive the artifacts and set the files to archive to deployment/**. This will keep the artifacts safe and sound. The other step you’ll want to add is Delete workspace when build is done to remove all unneeded stuff from your drive.