Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Excerpt

Implement the Atlassian Way for a consulting project, using the ant-salesforce-git build file.

Status
colourBlue
title** ALPHA**

This setup guide extends the workflow found in the Atlasssian Cookbook for Collaborative Development. The original guide is geared toward customers maintaining their own Salesforce environment, using a mix of server-based and cloud-based tools. This version uses cloud-based products, and it works well for an ISV supporting multiple Salesforce orgs. 


For additional background, see also 



Workflow 

Setup 

To Do Steps

Before beginning work in the project, create a JIRA workflow with these seven steps, 

  1. To Do
  2. In Progress
  3. In Approval
  4. Approved
  5. In Testing
  6. Tested
  7. Done

and be sure the the Upgrade Steps field shows on your JIRA screens.

You should also configure your Agile board to include an Upgrade Steps swimlane.


Tip

For each sprint, you may also wish to provision task sandboxes in advance, since they can take several hours for Salesforce to process. 

To Do Steps

When you begin the project, do these seven things:

(1) From BitBucket, create a client repository, following the standard instructions provided by BitBucket.

(2) For the initial commit, at the root, add a .gitignore file

ant-salesforce-git
*SocialPost*
ExternalEventMapping.workflow
Idea.workflow
Question.workflow
Reply.workflow
SocialPersona.workflow
SocialPost.workflow

and two properties files. 

build_org.properties

sf_credentials=${env.bamboo_client_sf_username_password_token}

build_repo.properties

repo_user = ${env.bamboo_bitbucket_user}
repo_password = ${env.bamboo_bitbucket_password}
repo_host = ${env.bamboo_bitbucket_host}
repo_message = ${env.bamboo_bitbucket_message}
repo_path = ${env.bamboo_client_bitbucket_path}
repo_config_user = ${env.bamboo_ManualBuildTriggerReason_userName}
repo_config_email = ${repo_config_user}@nimbleuser.com

For "client", please substitute the project acronym. 

(These properties map the global variables and minimize the other setup we need for Bamboo.)

Be sure all three files (.gitignore, build_org.properties, build_repo.properties) are committed to the master branch. 

(3) From Bamboo, create and set two global variables 

  1. client_bitbucket_path 
  2. client_sf_username_password_token

The Bitbucket path is the rightmost part of the repository URL, usually "nimbleams/client", where "client" is the acronym. 

The Username Password Token value is the concatenation of the username, password, and security token, in the format "user@example.com:PasswordSecurityToken". (Note the colon.)

Because the variable name includes "password", Bamboo will substitute the value with asterisks, even in the logs.  

(4) From Bamboo, create and run the plan ...

_ Initialize Production (IP)

  • Prepares a fresh master branch for use with the production org.
  • plan branch: master (new project, new repository)
    • For ALL Bamboo plans
  • trigger: manual
  • requirements: Bash (custom)
  • ant InitProduction
    • setup, retrieve, fixup, checkOnly, commit

(4) From the command line, first install Git and Apache Ant, and then 

  • From BitBucket, clone ant-salesforce-git and the client repo, so that they are both subdirectories of the same parent directory. 
  • From the ant-salesforce-git folder, run 
    • ant -Dhome=client-repo 
      -Dsf_credentials=${client_sf_username_password_token} 
      -Drepo_path=${bitbucket_path} 
      -Drepo_config_user=yourUsername
      -Drepo_user=${bitbucket_user} 
      -Drepo_password=${bitbucket_password} 
      -Drepo_message=${bitbucket_message} 
      -Drepo_host=${bitbucket_host} 

(5) Also create the plan ...

^ Refresh Backup branches daily at 8a (RB)

  • Retrieves the current production org into the backup branch.
  • plan branch: backup
  • trigger: 8 AM
  • requirements: Bash (custom)
  • ant RefreshBackup
    •  retrieve, fixup, checkOnly, commit  
  • ant RefreshBackupStaging

(6) After creating these two plans, and running Initialize, from BitBucket,

  • create backup branch from master
  • create staging branch from master
  • create backup-staging from staging
  • create develop branch from staging

(7) Finally,

  • set develop as the Main Branch.

The other plans can be created now, or as they are needed.

In Progress Steps

When you are ready to work on a new task, 

  1. Move a JIRA issue (task) to In Progress.
  2. Ensure that your task sandbox is provisioned and ready. 
    1. If your JIRA task ID is ABC-123, name the sandbox ABC123 (important!).
  3. From Bamboo, select "1 Start New Task" to copy the develop branch to your task sandbox.  
    1. From the top right corner of 1 Start New Task, click Run and select Run customized...
    2. Click "Override a variable" and task appears by default.
    3. In the value field, enter the JIRA ID for your task (such as ABC-123).
  4. Make changes to the sandbox pursuant to that single JIRA task. Do not mix tasks!
    • Changes can be made via the Salesforce UI or an IDE, or both. 
      • Either or both
      • When using an IDE
        • Develop directly against the task sandbox. 
          • You do not need to checkout the repository locally.
        • Save to Server when work is complete. 
        • (TBD) Create and update the develop package. 
      • When using the UI
        • Make changes as you would to any sandbox or org. 
        • (TBD) Create and update the develop package. 
  5. Add any Upgrade Steps to the JIRA issue.
    • Some changes cannot be copied between orgs, and need manual upgrade steps when deployed to another org. 
      • Log such changes to the Upgrade Steps field. 
        • "Making a new Record Type (CI), making that Record Type available to user profiles (clicks and UI)"
    • Some components must be named explicitly in the package manifest.
      • Log any needed package.xml updates (if not using a package).
        • Example: new Email Templates aren't picked up as there isn't a wildcard option in the package.xml.
  6. From Bamboo, select the  "2 Ready to Review" plan, Run customized, and enter your JIRA ID again. 
  7. Move the issue to In Approval.
  8. Work the board right to left, and approve other work if you can.
  9. If no other work is pending, goto step 1 (snag another To Do).

In Progress Steps 

1 Start New Task (SNT)

  • Prerequisite:
    • Create task sandbox (ABC123)
  • Plan Configuration
    • Plan Details 
      • 1 Start New Task
      • Brings a task sandbox up to date with the latest development.
    • Repositories (2)
      • client-org develop (default)
        • It is essential that the names of the staging and develop sandboxes exactly match the names of the staging and develop sandboxes, and that the task sandbox name (AWP1) correspond to the task branch name (AWP-1). 
      • ant-salesforce-git
    • Trigger: Run Customized (ABC-123)
    • Variables: task
  • Tasks
    • Source Code Checkout Configuration
      • Repository: client-repo develop 
        • Checkout Dir: sf-org
      • Second Repository: ant-salesforce-git
        • Checkout Dir: ant-salresforce-git
    • Ant - StartNewTask
  • Requirements: Ant / JDK

2 Ready to Review (R2R)

  • Prerequisite: 
    • Task sandbox with changes
  • Plan Configuration
    • Plan Details
      • 2 Ready to Review
      • Retrieves the task org to a task branch and creates a pull request.
    • Repositories (1)
      • ant-salesforce-git (default)
    • Trigger: Run Customized (ABC-123)
    • Variables: task
  • Tasks
    • Source Code Checkout
      • ant-salesforce-git
        • Checkout Dir: ant-salesforce-git
      • x Force Clean Build
    • Script - /bin/bash \ rm -rfd client-repo
    • Ant
      • ReadyToReview 
        -Dhome=awp-org 
        -Dsf_credentials=${bamboo_client_sf_username_password_token} 
        -Drepo_user=${bamboo_bitbucket_user} 
        -Drepo_password=${bamboo_bitbucket_password} 
        -Drepo_host=${bamboo_bitbucket_host} 
        -Drepo_message=${bamboo_bitbucket_message} 
        -Drepo_path=${bamboo_awp_bitbucket_path} 
        -Drepo_config_user=${bamboo_ManualBuildTriggerReason_userName} 
        -Drepo_config_email=${bamboo_repo_config_user}@
        nimbleuser.com
    • Requirements: Bash, Ant, JDK

In Approval Steps

When you are able to approve someone's work, 

  1. Determine whether 
    1. the changes in the task org work as expected,
    2. the changes follow our general standards,
    3. the Upgrade Steps seem complete, and
    4. the scope of the pull request matches the scope of the JIRA issue. 
  2. If needed, update the package manifest (package.xml) with new component references. 
    1. Some components must be named explicitly in the package manifest.
    2. (TBD) – Or use the unmanaged package approach, which can create its own manifest.
  3. If there is an question, move the issue back to In Progress and assign it back.
  4. When two approvers approve,
    1. Merge the approved pull request into develop.
      • System runs "Deploy to Develop Sandbox" plan. 
    2. Move the issue to Approved.

In Approval Steps

^ Update develop sandbox after merge to develop branch (DTDS)

  • Plan Configuration 
    • Plan Details
      • ^ Update Develop sandbox after merge to Develop branch
      • Deploys to develop sandbox on any change to the develop branch.
    • Repositories (2)
      • client-repo develop
      • ant-salesforce-git
    • Trigger:  Repository polling, changes to develop. 
  • Tasks
    • Source Code Checkout
      • client-repo develop
        • Checkout Dir: sf-org
      • ant-salesforce-git
        • Checkout Dir: ant-salesforce-git
    • Ant
      • Deploy to Staging Sandbox
      • DeployToDevelop
        • By default, all deploys run local tests and fail if any test fails.
    • Ant
      • Lookahead to see if develop branch will deploy to staging sandbox.
      • CheckOnlyDevelopToStaging
    • Requirements: Ant, JDK.

Approved Steps 

When staging is ready for additional user acceptance testing (UAT), 

  1. From Bamboo, run the Retrieve from Staging plan.
    1. Address any direct changes to staging not on the branch. 
  2. From BitBucket, create and merge a pull request from develop to staging
  3. System runs "Deploy to Staging Sandbox" plan.
  4. When deploy completes, make any needed Upgrade Steps to staging.
  5. Move newly deployed Approved issues to In Testing.

Approved Steps

^ Update Staging sandbox after merge to Staging branch (DTSS)

  • Same as develop but with staging sandbox and staging branch.
  • Ant - CheckOnlyStagingToProuction

_ Retrieve from Staging (RFS)

  • Commits metadata from staging sandbox to staging branch (Sync).
  • plan branch: staging
  • trigger: manual
  • requirements: Bash (custom)
  • ant RetrieveFromStaging

_ Retrieve from Production (RFP)

  • Commits metadata from production org to the master branch (Sync).
  • plan branch: master
  • trigger: manual
  • requirements: Bash (custom)
  • ant RetrieveFromProduction

In Testing Steps

When staging is updated for additional user acceptance testing (UAT).

  1. Schedule customer walkthrough and other work for In Testing column. 
  2. Determine whether the acceptance criteria for the JIRA issue is met. 
  3. Move the issue to Tested, or back to In Progress, and assign to initial person. 

(None)

Tested Steps 

When the In Testing column is empty, and production is ready to be  updated with the Tested issues... 

  1. From Bamboo, run the Refresh from Production plan. 
    1. Address any direct changes to production not on the branch. 
  2. From BitBucket, create and merge a pull request from staging to master
  3. When ready, and with product owner "Go" approval, checkout master and run "DeployToProduction" from Ant command line. 
  4. Move Tested issues to Done.

Tested Steps

_ Update Production org after merge to Master branch (DPO)

  • prerequisite 
    • Run Refresh From Production plan
      • Check for direct changes 
      • From BitBucket, Sync if appropriate
  • plan branch: master
  • trigger: manual
  • requirements: Any
  • ant DeployToProduction
  • manual:
    • upgrade steps
    • smoke tests



Accommodating a Direct Change to Production 

  • After the change is made (or as the change is made), recreate it in a sandbox and roll it forward through the process, before the next staging-to-production deployment. 

Destructive Change

(TBD)



Changes from Original

  • Single repository with branches (rather than multiple forked repositories).
  • Shared repository for build tools (ant-salesforce-git).
  • All scripts in Git (build repository).
  • Org credentials stored once (not three times).
  • Minimal parameters needed for a build (either one or zero).
  • Contextual naming for plans (Start New Task, Ready to Review).
  • Task sandbox support (rather than individual developer sandboxes).
  • Both admins and Apex devs work in sandboxes – No direct changes to shared environments. 
  • Sample data sandbox scripts.
  • Holding on Destructive Changes support


Atlassian Way

  • build.xml (3 targets)
    • getCode
    • deployCode
    • deployEmptyCheckOnly
  • build.properties (3 properties)
  • Bamboo Global Variables (14, 4 per fork plus two)
  • Linked Repos (3)
  • Build Plans (7)
    • Dev All Tests
    • UAT All Tests
    • Dev Deployment
    • UAT Deployment
    • Production Deployment
    • Dev Save Changes
    • Production Save Changes
    • Full Backup Plan
      • Dev Backup
      • UAT Backup
      • Prod Backup

DreamOps Experience

  • build.xml (14 core targets)
    • InitProduction
    • StartNewTask
    • ReadyToReview
    • DeployToDevelop
      • CheckOnlyDevelopToStaging
    • DeployToStaging
      • CheckOnlyStagingToProduction
    • DeployToProduction
    • RetrieveFromProduction
      • RetrieveFromStaging
      • RetrieveFromDevelop
    • RefreshBackup
      • RefreshBackupStaging
  • build.properties (16 properties)
  • Bamboo Global Variables (2)
  • Linked Repos (3)
  • Build Plans (10)
    1. Everyday 
      1. 1 Start New Task
      2. 2 Ready to Review
    2. Automatic 
      1. ^ Backup Production and Staging
      2. ^ Deploy to Develop Sandbox
      3. ^ Deploy to Staging Sandbox
    3. Admin
      1. _ Initialize Production
      2. _ Deploy to Production Org
      3. _ Retrieve from Production
      4. _ Retrieve from Staging
      5. _ Retrieve from Develop


Questions

  • Task Sandboxes for Admins. 
    • Could administrators save some bother and just make changes directly to the develop sandboxes? Can't we pull changes from the develop sandbox and merge those into Git?
      • Yes, that approach is feasible, but there are two issues with a direct-change approach. 
        • There is a chance that the changes would be overwritten by a merge from a task sandbox. 
        • It's easy for direct changes to bypass JIRA and become undocumented, unapproved changes. 
      • If all changes, whether made by clicks or code, go through task sandboxes, then all changes go through Git first, and we have the chance to review and approval all changes. 
        • It's just as easy to make an incorrect click change as it is to make an incorrect code change. 
        • If quality matters to an organization, all changes should be peer reviewed before release to production. 
  • Automatic Syncing. 
    • Wouldn't it be better to for staging and develop to be forks, so that we could enable automatic syncing?
      • The automatic syncing feature is nice, but there are a couple of issues. 
        • First, automatic fork syncing is not available to Bitbucket Cloud.
          • Branch syncing is available, and we are investigating usage.
        • Second, it tends to encourage direct changes, since people know the change will flow down. 
          • The preferred approach in this guide is for all changes to be documented through JIRA issues and originate in Task Sandboxes. 
            • An emergency change can be made in production and in a task sandbox at the same time, or immediately after. 
    Develop Sandbox.
    • Do we need a develop sandbox? Couldn't we just merge from the task sandboxes to staging?
      • The develop sandbox is a failsafe that enables a couple of worthwhile protections. 
        • UAT Buffer - When staging is approved, everything in staging moves to production. With develop as a buffer, we can choose when to bring in the next batch of changes for acceptance testing. 
        • Integration Buffer - By merging to develop first, the build system can ensure that all of the latest Apex tests are passing before we merge to staging. 
  • Backup. 
    • If we have everything under version control, why do we need a parallel backup branch?
      • The backup branch is a failsafe against direct changes. Depending on timing a direct change to production may or may not create a conflict with a change made downstream in a task org. The backup branch ensures that we capture all changes to production. 
      • (TBD) We also refresh the production branch (master) from the org every day and before a deployment, to capture any changes.
  • Refresh. 
    • How often do we need to refresh staging and develop?
      • When metadata changes are deployed from version control, the driver for a refresh is (1) data updates and (2) to conform out-of-band changes made according to Upgrade Steps.
        • For out-of-band changes, if staging and develop are being properly maintained with the Upgrade Steps, then the need to refresh is minimal. 
        • For data changes, if the user acceptance tests are more effective with current data, then there is a need to refresh. 
        • After a refresh, ... (TBD)
  • Platform