Track the effect of changes, capture feedback, and help customers succeed.

Functional Headings

Operate Capabilities

  • Monitoring and Measuring
    • Defect AnalyticsDefect dashboards and reports are available in the business org. (TBD)
    • Score CardA set of leading and lagging metrics are used to predict and then review the success of a CS project or MP major version and compared with actual results prior to the next event. (TBD)
    • Defect ResolutionCrucial defects are quickly repaired and (MP) made available as patch versions to affected subscribers. (TBD)
    • Production Immune SystemUsage metrics and logs are continually scanned to detect statistically significant variations in error volume, creating a "production immune system". (TBD)
    • Error Email Logging(sf) Application error emails are forwarded to an inbound email service in the business org. (TBD)
    • Feature Auditing(sf) FeatureSetupAuditTrail for subscribers are aggregated as part of Push Metrics. (Needs to be considered in the context of the Feature Management pilot.) (TBD)
    • Usage MeticsFor each subscriber, tally record creation volume, page access volume, and page render times, and summarize on a dashboard. (TBD)
    • Known IssuesDefects are reported to a public site for awareness and feedback that all subscribers can access. (TBD)
    • Push MetricsFeature usage and adoption is automatically monitored and subscriber orgs push configuration settings, logs, and statistics back to the MP business org. For Salesforce, the metrics include data storage used, installed packages, and Lightning adoption. For Communities, metrics include license counts and Community logins. (TBD)
    • Heath PredictorsDefect Magnets; Hidden Features; Unilateral Changes (TBD)
    • Defect TimeboxRather than assign story points, initial work on a defect proceeds within a predefined standard timebox. If the defect cannot be resolved within the timebox, then whether to proceed is reviewed at standup. For example, the timebox may be based on how long it takes to resolve 80% of the defects (TBD)
    • Hammer Tests(sf) Subscriber Apex tests are run periodically to detect any new failures.  (TBD)
    • AuditingSetupAuditTrails for subscribers are aggregated via the API and scanned for unhealthy changes. (TBD)
    • Information RadiatorDisplay key statistics, performance indicators, and inspirational messages in a highly visible location, for the benefit of team members and passers-by. (TBD)
    • Root Cause AnalysisTo ensure the quality and improvement of our change management process, and to prevent previously discovered problems from recurring, we perform a root cause analysis (RCA), by asking five whys. (TBD)
    • Trust DashboardWhen reliability issues are detected, the incident is posted to a system health dashboard that all subscribers can access. (TBD)
    • Defect KPIDefect reports and repairs are closely monitored and used as a key performance indicator. (TBD)
    • Remote Event Logging (TBD)
      • Log RolloverRecent usage and error logs are combined maintained both in the business org (primary storage) and a data warehouse (secondary storage). Older logs are routinely pruned from the business org and maintained in the data warehouse for long-term analysis. (TBD)
    • Fail Events(sf) Issues found by failing Apex Unit tests or Selenium tests are posted to cloud-based logging systems, just like production errors and messages. (TBD)
  • Managing Change - Operate
    • Alert Capture(sf) Error alerts raised by email are captured in a business org object for analysis, reporting, and followup action. (TBD)

Please submit feedback to the DreamOps Success Group http://dreamops.org/group.