Track the effect of changes, capture feedback, and help customers succeed.

Functional Headings

• Monitoring and Measuring
• Managing Change - Operate

Operate Capabilities

• Monitoring and Measuring
  • Defect Analytics — Defect dashboards and reports are available in the business org. (TBD)
  • Score Card — A set of leading and lagging metrics are used to predict and then review the success of a CS project or MP major version and compared with actual results prior to the next event. (TBD)
  • Defect Resolution — Crucial defects are quickly repaired and (MP) made available as patch versions to affected subscribers. (TBD)
  • Production Immune System — Usage metrics and logs are continually scanned to detect statistically significant variations in error volume, creating a "production immune system". (TBD)
  • Error Email Logging — (sf) Application error emails are forwarded to an inbound email service in the business org. (TBD)
  • Feature Auditing — (sf) FeatureSetupAuditTrail for subscribers are aggregated as part of Push Metrics. (Needs to be considered in the context of the Feature Management pilot.) (TBD)
  • Usage Metics — For each subscriber, tally record creation volume, page access volume, and page render times, and summarize on a dashboard. (TBD)
  • Known Issues — Defects are reported to a public site for awareness and feedback that all subscribers can access. (TBD)
  • Push Metrics — Feature usage and adoption is automatically monitored and subscriber orgs push configuration settings, logs, and statistics back to the MP business org. For Salesforce, the metrics include data storage used, installed packages, and Lightning adoption. For Communities, metrics include license counts and Community logins. (TBD)
  • Heath Predictors — Defect Magnets; Hidden Features; Unilateral Changes (TBD)
  • Defect Timebox — Rather than assign story points, initial work on a defect proceeds within a predefined standard timebox. If the defect cannot be resolved within the timebox, then whether to proceed is reviewed at standup. For example, the timebox may be based on how long it takes to resolve 80% of the defects (TBD)
  • Hammer Tests — (sf) Subscriber Apex tests are run periodically to detect any new failures.  (TBD)
  • Auditing — SetupAuditTrails for subscribers are aggregated via the API and scanned for unhealthy changes. (TBD)
  • Information Radiator — Display key statistics, performance indicators, and inspirational messages in a highly visible location, for the benefit of team members and passers-by. (TBD)
  • Root Cause Analysis — To ensure the quality and improvement of our change management process, and to prevent previously discovered problems from recurring, we perform a root cause analysis (RCA), by asking five whys. (TBD)
  • Trust Dashboard — When reliability issues are detected, the incident is posted to a system health dashboard that all subscribers can access. (TBD)
  • Defect KPI — Defect reports and repairs are closely monitored and used as a key performance indicator. (TBD)
  • Remote Event Logging —  (TBD)
    • Log Rollover — Recent usage and error logs are combined maintained both in the business org (primary storage) and a data warehouse (secondary storage). Older logs are routinely pruned from the business org and maintained in the data warehouse for long-term analysis. (TBD)
  • Fail Events — (sf) Issues found by failing Apex Unit tests or Selenium tests are posted to cloud-based logging systems, just like production errors and messages. (TBD)
• Managing Change - Operate
  • Alert Capture — (sf) Error alerts raised by email are captured in a business org object for analysis, reporting, and followup action. (TBD)