FlexNet Code Insight Scripting - Getting Started

[#1]

Summary

Palamida executes Groovy scripts through various modules, using the Palamida Public API to operate on a Palamida Core or Scan server.

Synopsis

Flexnet Code Insight executes Groovy scripts through various modules, using the Flexnet Code Insight Public API to operate on a Flexnet Code Insight Core or Scan server.

WARNING: As with any scripting, do not run scripts on a production server without thorough testing, to avoid data loss. If you will be developing scripts, Palamida recommends using a test server.

FlexNet Code Insight scripting modules:

1. ScriptRunner
2. Scan Server Autorun scripts
3. Detector Scripts
4. Workspace Reports

Discussion

ScriptRunner

ScriptRunner is a standalone module that can be used to perform maintenance tasks and bulk operations, deploy custom integrations with your build process, and to retrieve data from the Palamida application.

Installation

ScriptRunner is located in the Palamida installation directory under:

$palamida/scriptRunner

ScriptRunner is ready to run directly on a Palamida server with no further setup.

To run ScriptRunner from another machine, copy the scriptRunner directory to the desired machine. The Java JDK is required to run ScriptRunner.

Usage

The ScriptRunner executable is located in the scriptRunner/bin directory.

On Linux:

scriptRunner.sh [OPTIONS] <GROOVY_SCRIPT> [SCRIPT_ARGS...]

On Windows:

scriptRunner.bat [OPTIONS] <GROOVY_SCRIPT> [SCRIPT_ARGS...]

Options

-c, --coreEngine <coreServer>

Name or IP address of the Palamida Core server. Default is localhost.

-u, --username <userName>

User name (login) for authentication. See below.

-p, --password <password>

User password for authentication. See below. (Note: The -p flag has been REMOVED in 6.6 and later versions)

-g

Launch the GUI for the Groovy console, useful for debugging on a test server.

-w, --workspace <workspaceUri>

URI of workspace to open and set in the binding, e.g. http://scan.corp.com/workspace-name. See below about the Groovy binding.

User Authentication

Authentication is required to connect to a Palamida server. Users must have Scripting Administrator system role in order to execute scripts.

1. Launch the web browser as a user who has Scripting Administrator privileges.

2. After logging into the application, click on My Settings on the top right corner of the screen. Scroll to the bottom where you can add a Token.

3. Proceed to add a token with a name to identify it, the desired expiration date, and copy the token as you will need it to run ScriptRunner.

4. Navigate to the scriptRunner/bin directory to launch scriptRunner. You will need to The -c flag connects the core server, which needs to use the http(s) port used by the application, the same core server url specified in core.properties. Use the -u flag with the authorized user after you have copied the JWT Token from the Web UI.

   ./scriptRunner.sh -u username_of_scripting_administrator -c http(s)://core.server.url:8888/palamida/

5. This will prompt you to enter the JWT Token, paste it in for the one time authorization. If the authorization is successful, the groovy commandline console will be launched. Exit the console with the command exit.

For Palamida 6.8 or earlier:

User credentials are passed on the command line using the -u and -p options. Since passwords often contain special characters, wrap them in quotes, for example:

scriptRunner.sh -c core.server.corp.com -u username -p "mypa$$word" /path/to/myScript.groovy

The credentials are cached under the user's home directory in the following location:

$HOME/.palamida/config/scriptRunner/scriptRunner.properties

This allows you to supply the user credentials the first time ScriptRunner is run, and subsequent runs will use the cached credentials.

Deploying ScriptRunner

To deploy ScriptRunner for a build or CI machine, the recommended practice is to create a new user with the Scripting Administrator privilege for each machine or service that will need to execute scripts. After that:

1. Copy the scriptRunner directory to the new machine.

2. Execute ScriptRunner from the new machine, passing the machine user's Palamida credentials.

3. On Windows, exit the command prompt. On Linux, run the following commands:

   rm $HOME/.bash_history
   chmod 0600 $HOME/.palamida/config/scriptRunner/scriptRunner.properties

If you need to revoke access for a machine, remove the privilege from the machine user, or just delete the user.

Groovy Binding

When ScriptRunner executes a Groovy script, it pre-loads script variables using the Groovy binding mechanism. The following variables are available in the global scope in ScriptRunner scripts:

locator

WorkspaceLocatorCover object for manually opening workspaces on a scan server, and for accessing scanner scheduling services.

workspace

If the -w option was passed, ScriptRunner will handle the opening and closing of the specified workspace, and set this variable to the [WorkspaceCover][3] object.

options

Options passed to ScriptRunner (a list of CLI Option objects). These can be used to access the ScriptRunner options in scripts.

Adding external libraries

NOTE: It is good practice to document and backup any changes you make to the Palamida installation directory, including configuration files, to protect against accidentally losing custom changes during an upgrade.

ScriptRunner loads libraries through the config file:

scriptRunner/conf/scriptRunner.conf

If you need to use additional jar files in your scripts, place them in

scriptRunner/lib

and add the appropriate line to scriptRunner.conf:

load !{groovy.home}/lib/my-lib.jar

Autorun Scripts

Autorun scripts execute on a Palamida Scan server, and can trigger actions before and after scans.

These scripts are executed by the scan engine when a workspace is created, or when it is first opened after the scanner Tomcat service is launched. A WorkspaceCover instance is bound to the global variable workspace in scripts.

Scan event triggers are implemented by extending the ScanPhaseAdapter class, and calling an internal method to attach an instance of the adapter to the workspace.

Installation

NOTE: It is good practice to keep official copies of custom scripts, and record which autorun scripts are installed. This makes upgrades and new deployments easier.

To install an autorun script, copy the script to each scan server in the location:

$palamida/config/scanEngine/autorun

A restart of the scanner Tomcat service is required after adding or changing autorun scripts.

Autorun Template

The following is a basic autorun script template:

import com.palamida.workspace.event.ScanPhaseAdapter
import org.apache.commons.logging.LogFactory

class MyScanAdapter extends ScanPhaseAdapter {

    // reference to the WorkspaceCover instance
    def workspace

    // logging object
    def log = LogFactory.getLog('com.palamida.workspace.MyScanAdapter')

    MyScanAdapter(workspace) {
        this.workspace = workspace
    }

    // add event methods...
    void afterAnalysis() {
        // wrap all callbacks in try/catch!
        try {
            log.info("After Analysis triggered")
        } catch(Throwable t) {
            log.error('An error occurred', t)
        }
    }
}

// attach the listener to the workspace.
workspace.getInternalWorkspace().addScanPhaseListener(new MyScanAdapter(workspace))

Scan Events

Methods from the ScanPhaseAdapter class can be overriden to implement any event triggers. For example:

class MyScanAdapter extends ScanPhaseAdapter {
    // ... template code

    void beforeScan() {
        try {
            log.info("Beginning scan")
        } catch(Throwable t) {
            log.error('An error occurred', t)
        }
    }
}

Any of the following methods will be triggered on the corresponding scan event:

beforeScan

Called when the scan is just about to start.

scanInterrupted

Called when the scan was canceled by a user.

scanException

Called when the scan fails.

afterScan

Called when the scan is finished, before the scan results are committed.

afterCommit

Called after the scan results have been committed.

afterAnalysis

Called after automated analysis techniques (e.g. MID-rule group creation) have completed.

Detector Scripts

The Detector client interface has the ability to run scripts from the Script menu. This menu is only available to users who have the Scripting Administrator privilege. There is also a built-in Groovy console.

WARNING: Do not develop in the Groovy console on active projects. Thoroughly test scripts in a sandbox environment before running them on a production server.

The Detector context binds the current instance of the WorkspaceCover class to the global variable workspace in scripts.

Installation

To install a custom Detector script on a user's machine, place it in:

$HOME/.palamida/config/detector/scripts

The script will appear in the Script > Favorite Scripts menu in Detector.

Custom Filters

Custom filters have the same effect as running a filter from the GUI, and are used to filter the file tree based on custom logic.

NOTE: Custom filters execute on the client machine, and their performance is affected by database intensive scripting.

Custom filter scripts are passed a list of visible file paths in the global variable visible_nodes and should return a list of paths to be displayed in the Detector file panel.

For example, to show files with one of the extensions .zip, .rar, or .gz:

def extensions = [".zip", ".rar", ".gz"]
return visible_nodes.findAll { path ->
    extensions.any { path.endsWith(it) }
}

To install a custom filter on a user's machine, place it in:

$HOME/.palamida/config/detector/custom_filters

The filter will appear in the Script > Favorite Filters menu in Detector.

Workspace Reports

Workspace reports analyze scan results and other workspace-related data and output html and other documents. They are usually scheduled to run right after a scan. See also triggering reports.

Installation

NOTE: Custom reports and their XML definitions should be maintained in a separate repository so that they can be easily re-installed after an upgrade.

Workspace reports plug into the Scan Engine web application $palamida/tomcat/webapps/palamidaScanEngine/WEB-INF.

To add a report, add the following entry to classes/config/report.xml:

<report name="My Report" class="com.palamida.workspace.report.GroovyReport">
    <reportAttribute name="reportId" value="my_report" />
</report>

Create the directory classes/reports/my_report and add the file my_report.groovy. This script must write to the file index.html in the report output directory, which is located in $workspaceName/reports/My Report in the workspace base directory on the scan server. The report will be viewable in the web UI.

Workspace Report Template

The following is a minimal template for the report script my_report.groovy:

// write to index.html
new File(outputDirectory, "index.html").write("Hello World")

Palamida makes use of the Velocity template engine in reports. For details on using Velocity, see the documentation in classes/reports/readme.txt.

Workspace Report Binding

The following variables are available in the Groovy binding for workspace reports.

outputDirectory

The output directory that the report should write to.

reportDir

The install directory of the report.

reportId

The slug of the report, e.g. my_report.

reportName

The display name of the report.

workspace

Internal workspace object. Use new WorkspaceCover(workspace) to get the documented cover class. See below.

Getting the WorkspaceCover object in Workspace Reports

To get the WorkspaceCover object in a workspace report, instantiate the cover from the internal workspace object that is available in the binding:

import com.palamida.script.WorkspaceCover
def ws = new WorkspaceCover(workspace)

[tjvanwae]

Under Workspace Reports section the link for triggering reports no longer works.