Measure twice, cut once – As the scope of an enterprise-grade managed application grows, additional resources are needed to support rich implementations. The DreamOps Guide includes over thirty core capabilities that you can use to streamline and enrich package development.

Reality Check

Isn't this a lot of things? Yes, it is. And each of the things is a need enterprise developers have on every software development platform. The DreamOps Guide is specific to Salesforce, but, the missing bits of continuous delivery – static analysis, help topics, feature toggles, monitoring, and so forth – also need to be plugged into most delivery platforms.

DreamOps Walk Through

Let's take a brisk stroll through a DreamOps software delivery lifecycle. 

  • Making Changes - First, we put our right foot forward by specifying the changes to be made in a well-designed development environment. 
  • Validating Changes - The changes are then checked by another team member, to be sure that the change is done right, and that it's doing the right thing.
  • Distributing Changes - Once verified and validated, changes can be distributed to customer environments. 

Making Changes

New changes are specified with multiple subscribers in mind, including a mock customer, and provide sample data when appropriate. Admins and Coders can provision a development trial instance by asking Hubot in a HipChat window, or by completing a form on a secure Visualforce page.

Development environments are pre-processed (via proxy signup) and arrive ready-to-use. Developers just sign-in-and-go using a preset password. IDEs can be attached directly to the org without cloning the Git repository locally. Any changes made to the development environment can be added to an unmanaged package in the org, and automatically committed back to version control without special knowledge of Git or Bitbucket, including changes to the mock sample data.

The package's developer API is documented with ApexDocs for Global classes and Octopus for other shared components. Both Help for this Page and the Confluence Help site are updated each sprint to cover tasks, topics, and references meaningful to end-users. New regression tests for Global methods are created as part of the development task, and UI changes include modifications to the Selenium Nightwatch scripts. Issues found by tests are posted to cloud-based logging systems, just like production errors and messages.

Once a pull request is ordered, via ChatOps or a Visualforce page, changes are first verified through static analysis, and both Apex test coverage and Apex test throughout are confirmed. Selenium Nightwatch tests confirm that the user interface works as expected. Peer developers review and test all changes before code and sample data is merged back to version control.

The full suite of tests automatically run against the merged changes, to be sure everything still works with other recent changes. Failed builds are raised up for handling by the developer responsible for the last change.

A build uploads a new version at the end of each sprint, verifies the version, and then submits the version for end-to-end testing.

Validating Changes

Managed packages are kept in a state of continual release readiness. The latest work increment, including help topic updates, is examined at the close of each sprint, to determine whether the version is ready-for-primetime.

The examination includes both regression and acceptance tests, in automated and exploratory form. Apex tests can make configuration changes during a test run to confirm features work as expected when enabled or disabled. Support can enable or disable the same optional features on the customer's behalf through a secure Apex web service. Exploratory testing includes reviewing updated help topics, and testing like a customer. 

All error events, whether in test or production, are posted to cloud-based logging systems. The package also posts any feature configuration changes to our audit log, whether by an administrator or our web access point.

Distributing Changes

Using a secure Salesforce app in your business org, any certified version can be pushed to customer staging or production orgs, individually or in groups, typically with the latest features disabled, and other needed updates automatically applied and logged. User access to new components is managed through packaged permission sets and a stub profile. 

Usage metrics and logs are continually scanned to detect statistically significant variations in error volume, creating a "production immune system". Subscriber Apex tests are run periodically to detect any new failures. 

Each customer can enable new features when it is ready, or not at all. The package feature adoption dashboard is generated from analysis of the usage metrics and configuration audit log. Based on adoption feedback and other inputs, additional changes can be specified, closing the loop.

To assist with implementing the package for new customers, a Jumpstart trial is maintained with the latest GA versions. Support is also provided for maintaining implementation source under version control, using a separate developer sandbox for each task, populated with a subset of production data.

DreamOps Capabilities

The extended version of the pipeline draws on 37 distinct capabilities, ranging from checklists to jumpstart trial orgs. (Dimmed are TBD - 25/37 - 68%.)

The Pipeline and Bootstrapping sections walk through the DreamOps Experience and how to standup your own version of the way to manage your own packages.

Welcome to Force.com with all the trimmings. Yum!

  • Create Change (9/12)
    1. Collaborative development tools using the Atlassian platform (JIRA, Bitbucket, Bamboo, Confluence, HipChat) to lower communication overhead.
      1. AWS instance to host Bamboo server (no cloud version).
    2. Checklists to manage repetitive tasks in Confluence.
      1. Develop Task, Upload Version, Validate Help Content, Validate Version, Release Version, Seasonal Rollout, Maintenance Version, Launch Implementation
    3. Mock customers to enable deep solution crafting.
    4. Managed mock sample data to simplify development and testing.
    5. ChatOps to launch automated tasks.
    6. Separate environment for each development task with Trialforce.
      1. Trialforce Source Org (TSO) populated with development code.
      2. Signup Request and Proxy Signup enabled in business org.
      3. Visualforce form to request development trial instance. 
    7. Performant unit tests to reduce cycle time.
    8. Global class API documentation with ApexDocs.
    9. Shared component documentation with Octopus.
    10. Change log to document all updates to package.
      1. (TBD) - Do we need a Known Issues capability?
    11. Help content space with staged versions for milestone and preview releases, using Confluence with Scroll Versions and Google Analytics.
      1. AWS instance to host Confluence Server (10 user edition). 
    12. Help for this Page for context-sensitive content on custom tabs.
      1. Proxy server (NGINX) to route help requests to appropriate Confluence page.
  • Verify Change (4/5)
    1. Static analysis to ensure code quality with SonarQube and CodeScan.
    2. Dynamic test analysis to ensure code coverage with ApexUnit.
    3. Automated acceptance tests with Selenium and Nightwatch.
    4. Peer review and approval to validate new development. 
    5. Exploratory testing to verify new development.
  • Create Version (3/3)
    1. Release Dashboard to track package rollout (JIRA project with agile board).
    2. Regression sandbox for initial installation and testing.
    3. Regression unit test suite to exercise external global extension points.
  • Verify Version (4/4)
    1. Milestone and Preview versions to minimize change and reduce risk.
    2. Org Detail object to track subscriber environments.
    3. Exploratory testing in mock customer orgs using Trialforce to provision. 
    4. Acceptance testing with sample data developer sandboxes for select subscribers.
  •  Deploy Versions (2/8) - Milestone, Seasonal, and Maintenance Versions
    1. Automated push major upgrades to scale releases.
    2. Metadata automation to eliminate manual upgrade steps.
    3. Dark launching to separate deployment and enablement.
    4. Audit log to track package configuration changes.
    5. Cloud-based logging to surface errors, monitor performance, and track usage.
    6. Inbound messaging to parse email alerts.
    7. Usage monitoring to track adoption, performance, and quality trends.
    8. Production Immune System to detect failed changes (monitor business metrics for statistically significant variations).
      1. (TBD) - Do we need a Security/Performance/Adoption Health Check?
      2. (TBD) - Do we need a devops metrics capability?
  • Launch Implementation (3/5)
    1. Jumpstart orgs for implementations using Trialforce.
    2. Configuration Wizard to update implementation settings.
    3. Task Sandboxes to simplify version control.
    4. Continuous subscriber Apex test monitoring with Gearset test runner.
    5. Learning Management System to accelerate training.

DreamOps Tools