Improve your own software delivery practice by selecting new capabilities to implement as part of your continuous delivery pipeline.

• Build — Create new features, correct defects, and standup supporting infrastructure.
• Deploy — Install changes in staging and production environments.
• Operate — Track the effect of changes, capture feedback, and help customers succeed.
• Plan — Combine metrics with customer feedback and professional intuition to prioritize and describe the next round of changes.

Detailed Table of Practices

• Build — Create new features, correct defects, and standup supporting infrastructure.
  • Bootstrapping Infrastructure - Build
    • Password Manager — All resource credentials are kept in a secure, shared repository, such as LastPass.
      • Setup Password Manager
    • Email Alerts —  All systems are enabled for email alerts, and all email alerts go to a shared chat room, forum, email account, or distribution list. For a distribution list, to provide transparency, the alerts should also be available in a chat room or forum. (TBD)
    • Version Control System — All source materials used to create and distribute products or customizations are kept under some form of version control with a history of changes.
    • Post-Launch Subscriber Configuration — (sf) To provide longterm support, subscriber metadata is kept under version control, so that post-launch changes can be reviewed. (TBD)
    • Issue Tracker — All new work and modifications are made according to a request logged to a common system that assigns a unique identity to each item, such as an issue tracker or agile project management tool. 
    • Traceability — All changes can be traced from the Issue Tracker to the Version Control System, or from version control (Bitbucket) to the change request (Jira issue) to the original requirement (in Confluence). (TBD)
      • Issue IDs in Commit Logs — Each and every commit references an issue ID, so that we can trace from the commit to the issue tracker (and vice versa).  Ideally, this practice is enforced by a pre-commit hook so that the Version Control System will not accept a commit that does not reference an issue tracker ID. (TBD)
    • Collaborative Platform — Requirements and changes are specified using a collaborative environment, such as Confluence, which is integrated with development tools, such as the JIRA issue tracker and Git version control system. (TBD)
    • Agile Methodology — Development work is managed using a well-defined Agile process, where progress on open items is visible on a shared Agile board, and the team continuously improves through blameless retrospectives. (TBD)
    • Designated Workspace — Using a template, a separate Confluence workspace is created for each customer implementation or major release series (such as Winter, Spring, Summer, for Salesforce) in order to aggregate the resources generated for each release and to avoid creating a future maintenance burden. (TBD)
    • BuildOps Maintenance — The infrastructure used by the build pipeline is regularly maintained according to a set schedule and checklist, including pruning abandoned task branches. (TBD)
    • Resource Checklists — To ensure consistency and quality, checklists walk through the initial setup of recurring resources, such as managed packages, Trialforce orgs, workspaces, requirements, and internal scripting tools, as well as onboarding and offboarding resource users. (TBD)
    • On-board/off-board — The access new employees require is defined by role, and work instructions include removing access from business resources and development resources when an employee leaves. (TBD)
    • Shared Two-Factor Verification — For shared and standing development orgs, the verification text number is tied to a Google Voice number that posts to a chat room (such as HipChat or Slack), for shared access. 
    • Topic Branching — Within the Version Control System, topic branches are created from the default branch to represent individual change requests (Jira issues). The branches are short-lived and merged back into the default branch when the task is done. An alternative to Topic Branching ("Git Flow") is Trunk Based Development. (TBD)
    • Resource Management — Key resources shared across packages or implementations (such as the ant-salesforce.jar or SFDX CLI) are maintained in a common managed location, and not in the individual source repositories. (TBD)
    • Managed v Unmanaged — (sf) When to use an unmanaged or managed package is clearly defined by a decision tree or matrix. When to submit a package for security review is also supported by a decision tree. (TBD)
    • Resource Naming Conventions — Common resources share a strong naming scheme, so that new resources with a similar scope are easy to locate, regardless of which team member creates the resource. This practice applies to version control repositories, named packages, shared usernames, development environments, and so forth. (TBD)
    • Trunk Based Development — Within the Version Control System, new development is committed directly against the trunk, and checked in at least daily. An alternative to Trunk Based Development is Topic Branching. (TBD)
    • Branching Model — A high performing development team selects a branching model that works well in their own environment. The two most popular models are Topic Branching and Trunk Based Development. Branching is a core practice. Each team member must understand the branching model being used, and should understand why the selected model is the best choice for their team.  (TBD)
    • Configuration Management System — All server operating systems and application software are setup and updated using automated configuration tools, such as Puppet, Chef, or AWS Cloud Formation.  (TBD)
    • Resource Runbook — Outline the lifecycle of each infrastructure resource, and any operational parameters, in a runbook or set of runbooks. (TBD)
    • Equivalent Staging and Production Environments — To properly test ongoing changes to the work increment, we need a staging environment that matches the production environment as closely as possible. 
    • Continuous Integration Build Server — A build server integrates with other resources, such as the Version Control System, Configuration Management System, and other tools, in order to execute routine builds, launch automatic tests, and undertake other housekeeping tasks.  (TBD)
    • Deploy to Staging on Every Merge — On any change to the head of development, deploy the updated work increment to staging environment.  (TBD)
  • Provisioning Environments
    • Disposable Dev and Demo Environments — Single-use development or demo instances are provisioned from source control, used briefly for a single task or feature, and then discarded. For Salesforce, environments may be provisioned from an Enterprise Edition Org with Sandboxes, a Scratch Org Dev Hub (beta), or a Trialforce Source Org (partners). (TBD)
    • Build Request Form — Supplemental builds can be launched from the business org or secure web site, without visiting external sites or servers. (TBD)
    • Mock Demos — Sales demos are based on a thoughtfully designed mock customer, complete with realistic mock data, designed to be used with trials. (TBD)
    • MP Task Orgs — (sf) For managed packages, Trialforce is used to create new environments from on a secure Visualforce page, or DX is used at the command line by Platform Developers. (TBD)
    • CS Task Sandboxes — (sf) For consultants, developer sandboxes are provisioned and populated with a scrubbed subset of the customer's production data, and a full sandbox is used to integrate changes for customer acceptance testing. (TBD)
    • Org Request Form — Environment requests can be launched from the business org, or secure web site, without visiting external sites or servers. (TBD)
    • ChatOps — Environments can also be provisioned by querying a robot service in a chat room (Doggles in HipChat).  (TBD)
    • Clean Orgs — (sf) DX Scratch Orgs are used, or spurious metadata is scrubbed from other development environments, so that the environment only contains components found in the target package. (TBD)
    • Mock Orgs — For MPs, the development environment is based on the primary Sales Demo and includes realistic mock data bundled with the trial. For CSs, the development sandboxes include a subset of customer production data loaded with a build agent. (TBD)
    • Mock Package — The core Sales Demo is handled as a package so that it can be redistributed to more than one environment type. (TBD)
    • Role Orgs — MP staff and partners can provision different types of trials according to each person's role, with either source code or packages installed (development, testing, training, and/or sales use). (TBD)
    • Cohort Orgs — MP environments are provisioned to represent different segments of its subscriber base, pre-configured with settings and data appropriate to each segment. (TBD)
    • Development Environments on Demand — A working development environment can be automatically provisioned by developers, as needed, either through an online request form or ChatOps. 
  • Working with Environments
    • Demo Orgs Ready to Use — Sales demos and other trials also arrive pre-configured and ready to go, without routine manual steps. (SF orgs are configured by using Proxy Signup to modify the trials in-flight.) (TBD)
    • Dev Orgs Ready to Use — MP development environments arrive ready-to-go. Developers sign-in-and-go without so much as setting a password.  (TBD)
    • Start New Task — Automated Task Branching - Task branches and pull requests can be managed by running a build when development work begins, without special knowledge of Git. (TBD)
    • Ready to Review — Automated Pull Requests - Any changes made to the development environment can be automatically committed back to version control, without special knowledge of GitThis task automatically updates the package manifest (by retrieving the package by name). (TBD)
    • Data to Go — Ready to Review includes changes to the mock sample data. (TBD)
    • Ready to Use IDE Support — (sf) IDEs can be attached to development environments without cloning the Git repository to a local workstation. The native developer UIs can also be used to make changes to be placed under version control. (TBD)
  • Managing Change - Build
    • Permissions — (sf) User access to new components is handled through permission sets and an optional stub profile. For MPs, updating customer permsets or profiles is handled through Direct Deployment. (TBD)
    • Report Examples — New reporting examples are included with new features, as appropriate, and distributed through Direct Deployment. (TBD)
    • External ID Fields — (sf) External IDs are included with standard and custom objects to support capabilities like data conversion, sample data import, and staging refresh. (This item does not include implementing the capabilities, only the External ID fields.) (TBD)
    • Proxy Base — For extension packages being developed alongside a base package, to support base API changes for the extension, the base is represented by a proxy package using a different namespace prefix so that the base can be updated incrementally during the development sprint.  (TBD)
    • Proxy Upload — (sf) For managed packages using Proxy Base, a new version is automatically uploaded and installed into an integration or development environment, after every change to the mainline or on a nightly basis. (TBD)
    • Flows for the Win! — (sf) Flows are available for use in managed packages and local extensions. (TBD)
    • Cache Partitions — (sf) Platform cache partitions are used in managed packages and local extensions where needed.  (TBD)
    • Customer Collaboration — Customers have access to an issue tracker and wiki (such as JIRA and Confluence) that can be used to collaborate between themselves and with the implementation team. (TBD)
    • Unit Tests — As part of coding an item, include light-weight automated tests that exercise the internal interface for individual classes or components. A unit test provides a strict contract that the piece of code must satisfy.  (TBD)
    • Team Agile Board — A team agile board is a work and workflow visualization tool that optimizes the flow of work during an iteration. The board provides columns to represents phases of work, such as To Do, In Progress, and Done. Each work item is represented on the board as it moves across the columns. An agile board can be used with Scrum, Kanban, and other methodologies, and be physical or virtual. (TBD)
    • Feature Agile Board — A feature agile board tracks the progress of a feature from inception to adoption. The board provides columns to represents phases of work, such as To Do, Plan, Build, Operate, Evaluate. Other agile board may track the progress of each feature with the top-level phases.  (TBD)
    • Continual Builds of Complete Test Versions — At least once a day, automatically build a complete release candidate, including dependencies, documentation, and automatic tests. Deploy build to a staging or test environment. (TBD)
    • Short Running Tests — During code review both Apex unit test coverage and Apex unit test duration on new components is confirmed to be within expectations (less than five seconds per method). (See also ApexUnit https://github.com/forcedotcom/ApexUnit.) (TBD)
    • Go/NoGo Milestone Checkpoints​ — At the close of each iteration, stakeholders jointly determine whether to proceed with the next milestone version.  (TBD)
    • Peer Review and Testing — Peer developers review and test all changes before code and sample data is merged back to version control, following a test plan that is created for each issue in Confluence. (TBD)
    • Static Analysis — Once a pull request is created, changes are verified through static analysis, including quality scans and security scans. (TBD)
      • Security Scan — Common security issues are checked as part of peer review or static analysis (TBD)
        • Quality Scan — Once a pull request is created, changes are verified through static analysis and by running the Apex unit tests. (TBD)
    • Coding Style Guide — Document the set of conventions about how to write code for your project. It is much easier to understand a large codebase when all of its code is presented in a consistent style. (TBD)
  • Documenting Change
    • Help Site — Help topics are provided for all subscriber-facing capabilities. (TBD)
    • Context Help — Help for this Page is extended for new tabs, updated as needed, and linked to Help Site. (TBD)
    • Value Stream Map — A value stream is the series of activities used by an organization to deliver a product or service to an external or internal customer. A value stream map breaks down the process time for each activity, the lead time between activities, and the success rate. The map is depicted as a flow chart annotated with key metrics. Key metrics are lead time, process time, activity ratio, rolled % complete & accurate. Often a current state map is used to derive a future state map with process improvements.
    • API Documentation — Describe what services your application programming interface offers and how other people can use those services. (TBD)
    • Evergreen Help Site — The Preview and GA help sites are updated within each sprint to cover new tasks, topics, and references meaningful to end-users.  (TBD)
    • Developer Design Documents — Document key aspects of the application design for the benefit of other developers, and your future self. (TBD)
    • OctoDocs — Other shared components are documented with Octopus https://appexchange.salesforce.com/listingDetail?listingId=a0N3000000B5cEKEAZ. (TBD)
    • Technical Change Log — Itemize each work item merged into a given work increment, and provide a standard description of the change. (TBD)
      • New Components — Once the new increment is deployed, update the change log to list any new components (classes, fields, and so forth) being added to the version, by using a Javascript macro to scrape the package details page.
      • Jira Change Log in your Confluence Wiki — Update the change log to list the issues for the new version. (May be done in GNG Baseline.)
        • Setup Change Log
        • Change Log Fields — In the JIRA issue, fields are available for developers to summarize the actual change being made, and to indicate the need for any Upgrade Steps or Feature Toggles. (These fields are included in the change log generated for each version.) (TBD)
    • Troubleshooting Articles — Capture how unusual and everyday problems were solved, in case the issue reoccurs. (TBD)
    • ApexDocs — Global classes are documented with ApexDocs and available through the help site. (TBD)
  • Integrating Change - Build
    • CheckOnly Before Merge — (sf) The full set of Apex tests on the task branch is automatically validated (checkOnly) against the packaging org or production org when the pull request is created. (TBD)
    • CheckOnly After Merge — (sf) On merge to develop, the updated develop branch is validated (checkOnly) with the packaging org or production branch. (TBD)
    • You Break It — Failed builds are raised up in email alerts for handling by the developer responsible for the last change. (TBD)
• Deploy — Install changes in staging and production environments.
  • Bootstrapping Infrastructure - Deploy
    • Preview Channel — Development and validation of new practices is staged so that designated early adopters can use changes and enhancements first before the capability is available to all subscribers. (TBD)
    • Pilot, Beta, and GA Rollouts — New capabilities are often available as an optional Pilot, followed by a public beta, before being designated Generally Available (GA). (TBD)
    • Master and Patch Branches — (sf) Individual package branches are stood up under a master folder in the Version Control System, where each branch under master represents the main packaging org or a patch org. (TBD)
  • Integrating Change - Deploy
    • Batch Size of One — Successful builds are automatically uploaded to a proxy test package, push deployed to a firewall sandbox instance, and subjected to a full set of regression tests. (TBD)
    • Dev TSO Template Refresh — (sf) At the end of the sprint, a new development TSO templates is provisioned, in case it is needed for patch development. (TBD)
    • GNG Baseline — Just prior to the sprint retrospective, a master pull request and an updated change log are generated, for review at the retrospective and Go/NoGo meeting.  (TBD)
    • Go/NoGo — Following the Sprint Review and Retrospective, stakeholders meet to determine if the new work increment is potentially releasable, or whether there are known impediments. (TBD)
    • TSO Template Bot — A Selenium script automates provisioning the templates, and the confirmation email is processed as an inbound message and parsed. New template ID is captured in an object and made available to ChatOps and Visualforce Request form. (TBD)
    • UI Regressions — Selenium Nightwatch tests are included where appropriate to assure that the user interface will continue to work as expected. (TBD)
  • Preparing New Versions
    • Milestone TSO Source Template — (sf) When creating a transient branch, a new trial template is provisioned to represent the new version. (TBD)
    • New Version Agile Board — The version creation and validation process is tracked in a shared checklist and/or Agile board, and included in the issue tracker. (TBD)
    • Transient Branch — To freeze the work increment being released, a temporary branch is created from develop, and a pull request created from the transient branch to packaging or production. (TBD)
    • Tag Issues — Identify the changes for the new version by tagging the corresponding JIRA issues. (May be done in GNG Baseline.) (TBD)
    • Automatic Change Log — The change log is determined by observing the Git pull request, with little or no human intervention. (TBD)
    • Merge and Deploy — The new increment is merged, and deployed to the packaging org or production (using the Ant Migration Tool or SFDX tool). All metadata in the repository is deployed using a blank manifest (TBD)
    • Tag Repository — The packaging or production branch in Git is tagged using the JIRA version ID. (TBD)
    • Firewall Install Test — Install the new version into the firewall sandbox environment.  (TBD)
    • Upgrade Install Test — Upgrade a GA trial environment to the latest version.  (TBD)
    • Upgrade Preview TSO — Install the new version into the Preview TSO, which generates production-like environments. (TBD)
    • One Button Milestone Install — The Firewall and Upgrade Sandbox Install and Test tasks are automated, and, on success, trigger the GA Trial success, and ultimately the TSO install. (TBD)
    • No Pull Request Left Behind — Prior to merging a new work increment, open pull requests from prior sprints are reviewed, to ensure completed work is not left outstanding. (TBD)
    • Version Agile Board — A version agile board tracks the progress of a set of changes from build to operate. The board provides columns to represents phases of work, such as To Do, Building, Certifying, Deploying. The board may include preview, general, and maintenance releases.  (TBD)
  • Validating New Versions
    • Scope / Tests — Validation includes both regression tests and feature acceptance tests, in automated and exploratory form. #TestLikeACustomerr (TBD)
    • Version Certification — Each new version is validated following a detailed checklist and exploratory test plan. If all checklist items pass, the version is certified for production use. Uncertified versions are deprecated. (TBD)
    • Validation — The latest work increment, is examined at the close of each sprint, to determine whether the version is ready-for-primetime. For MPs, a milestone version is validated. For CSs, a full staging sandbox is validated. For MPs, patch versions are run through the same gauntlet as major versions. (TBD)
    • Version Issue Resolved — The issue for the version is resolved in the New Version issue tracker and indicates its resolution status (failed certification, certified but not deployed, certified and deployed). (TBD)
    • Toggle Tests — Validations are performed with all features toggled off, all features toggled on, and with only new features toggled on. (TBD)
    • Release Ready — The packaging and pre-launch production orgs kept in a state of continual release readiness.  (TBD)
    • Scope / Docs — Validation includes the relevant help topics and other documentation (ApexDocs, Developer Guides). (TBD)
    • Graceful degradation — Apex unit tests can make configuration changes during a test run to confirm features work as expected when enabled or disabled. (TBD)
    • Health Check — The packaging and business orgs continue to pass the Salesforce Security Health Check following each major release, with a score of 80% or above. New subscriber orgs are reviewed with the Salesforce Optimizer before launch. (TBD)
  • Deploying Versions
    • Configuration Wizard — New implementation setup is streamlined through use of a configuration wizard that collect and updates settings. (TBD)
    • Connected Deployment — Direct deployment leverages a connected application (and user credentials are not required). (TBD)
    • Deploy From Hub — (sf) Using a secure Salesforce app in our business org, any certified version can be pushed to subscriber staging or production orgs, individually or in groups, typically with the latest features disabled, and other needed updates automatically applied. (TBD)
    • Direct Deployment — (sf) New permissions and reports are deployed post-upgrade by Connected Deployment or by leveraging user credentials. (TBD)
    • Feature Configuration Audit Trail — Any change to feature configuration (including validation rules) is recorded in a supplemental audit trail log and made available as part of Push Metrics. (TBD)
    • Feature Toggles — (sf) Custom metadata feature toggles may be hidden from the subscriber and activated through the LMA. (TBD)
    • Jumpstart Org — (sf) To assist with implementing a package for new subscribers, a Jumpstart trial is maintained with the latest GA versions for use by new implementations. (TBD)
    • Org Details Object — (sf) To track additional detail about subscriber orgs, a custom object in the business org is linked to the LMA and the subscribers Account object. The org details aggregates the API applications installed for each subscriber, the instance, the my domain name, the production org ID, as well as the staging/preview sandbox IDs, and refresh history. Ideally, an automation posts an alert whenever a new org is added, to queue followup actions. (TBD)
    • Patch Org Validation — (sf) Prior to re-launching the preview channel, the GA patch orgs are provisioned early and tested with a deploy to be sure the org provisioned correctly. (TBD)
    • Push Major Upgrades — Queue automated deployments of new versions from build server o to individual subscribers or subscriber groups. On Force.com, Deploy Fom Hub queues deployments form the business org. (TBD)
    • Rock Steady API — (sf) Global components are stable with a low regression rate, including behaviors of Global Apex class methods and variables, Apex triggers, custom metadata, custom objects, fields, validations, and workflows. (TBD)
    • Rollout Tracking — The progress being made on a major version rollout is tracked in a shared checklist and/or Agile board. (TBD)
    • Scalable Deployment — When Direct Deployment is used, scale agents on a platform like AWS, so that many subscribers can be upgraded at once. (TBD)
    • Sharpen the AX — (sf) The AppExchange entry is updated to the latest version for each new GA major version, without concern of prompting a security review.  (TBD)
    • Toggle Off — Optional features promoted to GA or Beta in a release can be deployed in an activated or deactivated state and toggled on or off by the subscriber. (TBD)
    • Upgrade Slot Select — Subscribers can select their own upgrade slot within a window, or accept a default upgrade date. (TBD)
    • Zero Step Tolerance — (sf) Any upgrade steps are automated or optional and can be invoked by subscribers. (TBD)
  • Managing Change - Deploy
    • External Regression Test Suite — New regression tests for Global API methods are created in an external package as part of the development task, and UI changes include modifications to the Selenium Nightwatch scripts. (TBD)
    • Staging Data Refresh — External IDs are included with standard and custom objects so that updated records can be pulled from production at any time. (TBD)
• Operate — Track the effect of changes, capture feedback, and help customers succeed.
  • Monitoring and Measuring
    • Fail Events — (sf) Issues found by failing Apex Unit tests or Selenium tests are posted to cloud-based logging systems, just like production errors and messages. (TBD)
    • Remote Event Logging —  (TBD)
      • Log Rollover — Recent usage and error logs are combined maintained both in the business org (primary storage) and a data warehouse (secondary storage). Older logs are routinely pruned from the business org and maintained in the data warehouse for long-term analysis. (TBD)
    • Defect KPI — Defect reports and repairs are closely monitored and used as a key performance indicator. (TBD)
    • Trust Dashboard — When reliability issues are detected, the incident is posted to a system health dashboard that all subscribers can access. (TBD)
    • Root Cause Analysis — To ensure the quality and improvement of our change management process, and to prevent previously discovered problems from recurring, we perform a root cause analysis (RCA), by asking five whys. (TBD)
    • Information Radiator — Display key statistics, performance indicators, and inspirational messages in a highly visible location, for the benefit of team members and passers-by. (TBD)
    • Hammer Tests — (sf) Subscriber Apex tests are run periodically to detect any new failures.  (TBD)
    • Defect Timebox — Rather than assign story points, initial work on a defect proceeds within a predefined standard timebox. If the defect cannot be resolved within the timebox, then whether to proceed is reviewed at standup. For example, the timebox may be based on how long it takes to resolve 80% of the defects (TBD)
    • Heath Predictors — Defect Magnets; Hidden Features; Unilateral Changes (TBD)
    • Push Metrics — Feature usage and adoption is automatically monitored and subscriber orgs push configuration settings, logs, and statistics back to the MP business org. For Salesforce, the metrics include data storage used, installed packages, and Lightning adoption. For Communities, metrics include license counts and Community logins. (TBD)
    • Known Issues — Defects are reported to a public site for awareness and feedback that all subscribers can access. (TBD)
    • Usage Metics — For each subscriber, tally record creation volume, page access volume, and page render times, and summarize on a dashboard. (TBD)
    • Feature Auditing — (sf) FeatureSetupAuditTrail for subscribers are aggregated as part of Push Metrics. (Needs to be considered in the context of the Feature Management pilot.) (TBD)
    • Error Email Logging — (sf) Application error emails are forwarded to an inbound email service in the business org. (TBD)
    • Production Immune System — Usage metrics and logs are continually scanned to detect statistically significant variations in error volume, creating a "production immune system". (TBD)
    • Defect Resolution — Crucial defects are quickly repaired and (MP) made available as patch versions to affected subscribers. (TBD)
    • Score Card — A set of leading and lagging metrics are used to predict and then review the success of a CS project or MP major version and compared with actual results prior to the next event. (TBD)
    • Defect Analytics — Defect dashboards and reports are available in the business org. (TBD)
    • Auditing — SetupAuditTrails for subscribers are aggregated via the API and scanned for unhealthy changes. (TBD)
  • Managing Change - Operate
    • Alert Capture — (sf) Error alerts raised by email are captured in a business org object for analysis, reporting, and followup action. (TBD)
• Plan — Combine metrics with customer feedback and professional intuition to prioritize and describe the next round of changes.
  • Learning and Planning
    • Ideas Solicited — Subscribers are able to post feature enhancement requests ("Ideas") to a forum for peer and product management review. (TBD)
    • Results Driven Roadmap — Based on adoption feedback, subscriber ideas, prospect needs, and other inputs, additional changes are specified, closing the loop. (TBD)
    • Adoption Analytics — Feature adoption is visible from the business org in the form of dashboards and reports, drawing on usage metrics and the configuration audit logs.  (TBD)
    • Ready to Develop Checklist — (Help text, error messages, sample data ...) (TBD)
    • Extensibility First — New changes are specified with extensibility for multiple subscribers in mind, including the mock customer. (TBD)
    • Feature Adoption — Every new feature is a hypothesis, and we track feature adoption to determine whether to pivot or persevere.  (TBD)
    • Acceptance Criteria — As part of the definition of done, acceptance criteria lists the main points or business rules to consider when coding and testing a user story. Acceptance criteria is often used as the basis for Acceptance Test Suite.  (TBD)
    • Collaborative Requirements — User stories, use cases, wireframes, and other supporting materials, are maintained in a collaborative authoring platform, such as Confluence, or specified directly in a JIRA story or task. (TBD)
    • Acceptance Test Suite — Include a set of tests that describe how a feature or story will be used in practice. Acceptance tests often walk through the item's acceptance criteria with hands-on exercises and Sample Feature Data. (TBD)
    • Definition of Done — The definition of done is the standing set of acceptance criteria for all work items, in addition to the specific Acceptance Criteria for each item. It is a comprehensive checklist of necessary, value-added activities that assert the quality of any feature and not the functionality of that feature. A DoD often includes unit test coverage, coding to standards, peer review, and passing its own Acceptance Test Suite.  (TBD)
    • Sample Feature Data — New features provide new sample data to use with new components when appropriate. (TBD)
      • Sample Data Updated for New Features — During initial development, the Sample Feature Data set is extended to include new features, and the updated data set follows the feature as it goes into the GA and Preview channels. (TBD)
    • Analytics Considered — Actual and projected feature adoption rates are compared and used as input to roadmap planning. (TBD)
    • Actionable KPIs — Key performance indicators measure activity to determine whether a system is achieving it's defined objectives. Not all metrics are KPIs. KPIs should pass the IPA test – Important, Potential for improvement, and Authority (does the system have the means to improve this KPI?).   (TBD)
    • Ideas Considered — Subscribers Ideas play a strong role in steering the future roadmap, along with a strong internal vision of the product's future. (TBD)
    • Production Data Subset — Identify a representative subset of production data to extract into development environments. (TBD)