Monitoring and Measuring

  • AuditingSetupAuditTrails for subscribers are aggregated via the API and scanned for unhealthy changes. (TBD)
  • Defect AnalyticsDefect dashboards and reports are available in the business org. (TBD)
  • Defect KPIDefect reports and repairs are closely monitored and used as a key performance indicator. (TBD)
  • Defect ResolutionCrucial defects are quickly repaired and (MP) made available as patch versions to affected subscribers. (TBD)
  • Defect TimeboxRather than assign story points, initial work on a defect proceeds within a predefined standard timebox. If the defect cannot be resolved within the timebox, then whether to proceed is reviewed at standup. For example, the timebox may be based on how long it takes to resolve 80% of the defects (TBD)
  • Error Email Logging(sf) Application error emails are forwarded to an inbound email service in the business org. (TBD)
  • Fail Events(sf) Issues found by failing Apex Unit tests or Selenium tests are posted to cloud-based logging systems, just like production errors and messages. (TBD)
  • Feature Auditing(sf) FeatureSetupAuditTrail for subscribers are aggregated as part of Push Metrics. (Needs to be considered in the context of the Feature Management pilot.) (TBD)
  • Hammer Tests(sf) Subscriber Apex tests are run periodically to detect any new failures.  (TBD)
  • Heath PredictorsDefect Magnets; Hidden Features; Unilateral Changes (TBD)
  • Information RadiatorDisplay key statistics, performance indicators, and inspirational messages in a highly visible location, for the benefit of team members and passers-by. (TBD)
  • Known IssuesDefects are reported to a public site for awareness and feedback that all subscribers can access. (TBD)
  • Production Immune SystemUsage metrics and logs are continually scanned to detect statistically significant variations in error volume, creating a "production immune system". (TBD)
  • Push MetricsFeature usage and adoption is automatically monitored and subscriber orgs push configuration settings, logs, and statistics back to the MP business org. For Salesforce, the metrics include data storage used, installed packages, and Lightning adoption. For Communities, metrics include license counts and Community logins. (TBD)
  • Remote Event Logging (TBD)
  • Root Cause AnalysisTo ensure the quality and improvement of our change management process, and to prevent previously discovered problems from recurring, we perform a root cause analysis (RCA), by asking five whys. (TBD)
  • Score CardA set of leading and lagging metrics are used to predict and then review the success of a CS project or MP major version and compared with actual results prior to the next event. (TBD)
  • Trust DashboardWhen reliability issues are detected, the incident is posted to a system health dashboard that all subscribers can access. (TBD)
  • Usage MeticsFor each subscriber, tally record creation volume, page access volume, and page render times, and summarize on a dashboard. (TBD)