Lean Startup or Reinforcement Learning is all about evolving as quickly as possible to discover the best way forward using Statistical Hypothesis Testing.

Every new features being built into the Website or Application costs extra time and employee payroll that may or may not bring extra rewards in the form of longer usage time as well as better user experience and higher one time or subscription revenue.

Key Statistics or Metrics for a Software as a Service SAAS are:

Activation Metrics:

• Activation Rate: Percent of Signup Who Complete a Key Action like Upload an Image, Upload a Video, an Excel File or Draw First Design.
• Time To Value: How Long It Takes for User to reach first Meaningful Valuable Outcome like Run 1st Image Analysis, Download first AI-Edited Video or first Interface Design PNG File.

Retention and Engagement

• Customer Retention Rate: % of Customers still active after X days/months
• Engagement Ratio: DAU Daily Active User/ MAU Monthly Active User)

Revenue and Growth

• Monthly/Annual Recurring Revenue
• Customer Life Time Value: Total Exptected Revenue per customer
• Customer Acquisition Cost

Viral and Scaling

• Viral Coefficient: >1 == exponential growth. How many new users each user brings

For different stages of a high tech lean startup product we can focus on single most imporant metrics:

• Problem/Solution Fit ==> ACTIVATION RATE is CRITICAL ==> >40% users create first project
• Product/Market Fit ==> RETENTION RATE is CRITICAL ==> 90-day Retention > 30%
• Scale Up ==> LTV Life Time Value : CAC Customer Acquisition Cost ==> LTV/CAC > 3:1
• Mature Growth ==> NRR Net Revenue / Starting Revenue ==> NRR>= 120%

For the Case Study of a Medical Image/Video Analysis Startup, to determine of the Product achieve Problem/Solution fit our product team needs to focus on the ACTIVATION RATE of Finished Video Uploaded per Sign UP

We can define our HYPOTHESIS is whether Our Product Add More Values To User Life OR Not by testing if our AR Activation Rate is larger than 40% or Not

• Basline p0 === 40%
• H0 Null Hypothesis is p = p0 == 40%
• H1 Alternative Hypothesis is p > p0

We can then use our Facebook Friend List, Phone Contact List or Mailing List to collection data for our Lean Startup Hypothesis Testing.

According to the Central Limit Theorem, we need a Minimum Sample of 30 randomly selected users.

$$ Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}} \sim N(0, 1) $$

This minimum sample size makes our Sample Statistics a moderately Reliable Predictor of our Product Adoption Rate before we run large marketing campaigns.