UTSQ Presetation (slides)

Unifying Theory of Software Quality (UTSQ) Because all Software is a series of Economic Events and Responses

Software is an instantiation of a concept believed to deliver economic value Requirements are a representation Code is a representation Test suites are a representation Documentation is a representation Concept = Hypothesis

Motivation Why A New Theory

Reality Hits in Quality Processes Many direct hits between what is taught and what is done ISO certification is required in Telco equipment Software Engineering Institute (SEI) is a driving force in software development processes for DoD. Ignition and power train software in automotive industry exceeds Six Sigma Motivation

Reality Gaps in Quality Processes Many disconnects between what is taught and what is done Game companies ($9 billion dollar industry) do not market ISO certification ATMs do not meet Six Sigma access levels (5 min per year down) Open Source development a different paradigm Apache serve 70% of the web pages daily Firefox 25 million users in first 90 says CEOs managing products in new categories consistently fund new features over big fixing Each release moves further form Six Sigma Ironically, this is commonly the winning strategy Motivation

Measurements Matter People are ruthless optimizers They will maximize results as per Motivation measurements If your measurements do not directly map to the desired outcome, then the results will be sub-optimal. Do not ask staff to be martyrs

Quality Quality

Economic Quality is Usually a Brand vs Product measurement A multi-period measurement Expected future return Quality R A time discounted measurement 1-d Vendor risk discounted measurement Less the probability cost of catastrophic failure R = R 1-d -

Quality (Economic Quality) Quality is delivering the user's expected economic return. Quality Quality= what we got so far expectations now return we initially expected Q= =1 R 1-d =1 n =n1 n R 1-d R 1-d

Normal User Cost/Value Timeline Value Value Value Value Value Quality Time Cost Cost Cost Cost Cost Cost Cost

Brand Value H euristic: Do everything possible to make your word your bond. Being a reliable vendor has more impact to a user's expected return than promise of new capabilities. Quality Expected Return= what we think about the future = R 1-d - R

Keystone of UTSQ Direct link between software quality and firm's revenue. Maximum a Rational user will pay is capped by expected economic return. Quality Principle: Developer's revenue opportunity is capped by User's economic return - Quality. max Rev= R u users

Risk of Catastrophic Failure Task Value Phi Cost 3 Sigma Risk (1 in a thousand) Legal Documents $50 / day $1,000 $1.00 $0.001 $.999 Air Plane $500,000 / day $500 million $500,000 $500 $499,500 Nuclear Plant Man. $1 million / day $100 Billion $100 Million $100 Thousand $99.9 Million Quality 6 Sigma Risk (1 in a million ) Gain H euristic: H uge catastrophic risks dictate "heavy" development processes. Small dictate "response" processes.

Heavy Processes H euristic: As the ratio of catastrophic failure to period return increases, Six Sigma and similar processes closely map to Economic Return process . Quality Return= period returns- probability of the big one R = R 1-d -

Alternative Dev Processes "First Article" Quality level X must be archived on initial launch. Large test suite. Focus on "corner cases." "User Driven" Quality Quality level X by release Y. Test suite for mainline. "Corner cases" user reported. Defect Density User Driven First Article Time to Market

"User in the Loop" Component of Theory

Brand Value to User Value

Task Fixed Value The work product for a Task has a value V that is independent of technology used to produce work product. Technology A User Technology B Work Product T

TaskWare User in the Loop: Users are active components of the system not just passive consumers of functionality. Tas k Wa r e Peo p l e Wa r e Ta s k Sof tWare Va l u e Costs R=v T -c T

PeopleWare Ta s k Wa r e Peo p le Wa r e Sof tWare Pe o ple War e R e s o u r c e Wa r e Kno wled geWare KnowledgeWare: Ability to execute required actions unaided, i.e. The users have the know-how in their head. ResourceWare: Access external authority.

Task Stimuli c= f A t s f K t i f R t k ts A ti K tk R H euristic: To increase User return, systematically replace ResourceWare with KnowledgeWare and KnowledgeWare with software.

Under Performing KnowledgeWare . f1 f2 f3 f4 f5 ta . 20 . . . t b 200 . 90 10 . t c 75 . . . 5 td . . . 50 40 H euristic: Poor use of application is a Value killer. Work on understanding how skilled your users are.

Parameters from Client Interface User Support Software Evaluate and Respond

Summary Summary

Two Key Points Probability-weighted cost of Catastrophic Summary Failure ( as measured by risk in UTSQ) is the best predictor of the development quality processes Software Quality is measured by User's Economic Return

What does UTSQ Unify? Unified Metric for: Direct link Quality and Revenue Features vs fix bugs resource allocation Help desk, documentation, training, UI research, ... resource allocation CEO decision domain and Engineering decision domain End user Economic Return is metric H euristic: Invest in activities that create the max End User Return per dollar committed Summary

What Next

Presentation and References www.sqi-inc.com