[Amplitude Foundations 4️⃣] Getting Started with Amplitude Feature Experimentation

 

[Amplitude Foundations 3️⃣] Use Dashboards and Starter Templates to Monitor Important Metrics

---

Amplitude Experiment Product 구성
1) Feature Flags
2) Web Experimentation
3) Feature Experimentation

• Understand how Amplitude Feature Experiment fits into Amplitude’s suite of products
• Navigate key parts of the Amplitude Experiment user interface
• Understand the high-level Amplitude Feature Experiment workflow
• Explain the difference between experiments and flags in Amplitude Experiment
• Understand how to create and analyze an experiment

---

 

What is Amplitude Feature Experimentation?

Feature Experimentation vs Web Experimentation

 

Let's learn experiment terminology

 

 

Experiments vs. Feature Flags

Feature Flags와 Experiments는 동일한 데이터 모델을 기반으로 작동하며, 서로 간에 변환할 수도 있습니다. 즉, 이미 생성된 플래그를 실험으로, 또는 실험을 플래그로 전환 가능함.

​

다만, 사용 목적에 차이가 있다.

Experiments는 사용자 경험을 개선하고 데이터 기반 결정을 내리는데 적합한 도구.(ex. 어떤 플로우로 제안했을 때 가장 많은 전환이 이루어지는지)

​

Feature Flags는 세밀한 분석 없이 기능을 관리하고 즉각적인 변경을 실행하는데 유용하다 (ex. 미국 유저에게만 배너를 노출하기)

 

Plan for an Experiment

1. Problem statement and supporting data: A good test plan starts with a problem statement. A strong problem statement will clearly define the specific problem you’re trying to solve.
2. Goals, metrics, hypothesis: Think about your experiment goals, what metric you would use to track the effectiveness of your changes, and your hypothesis on what you expect to happen as a result of implementing those changes. Your goals should detail what you want to achieve as a result of the experiment. Think about your business and product objectives, product KPIs, and Product features that relate to your problem statement to help specify your goals.
3. Variants
4. Targeting and timeline

 

Complete an Experiment

​

When you analyze the results of an experiment, one of three scenarios will present themselves. You either have:

• Conclusive results where your variant won out over the control
• Conclusive results where your variant did not win out over the control
• Inconclusive results where it is not clear which variant won

​

When reviewing the results of an experiment, it’s important to have a perspective on what your next steps should be, depending on the outcome. Click the tabs below to review the available options within the three possible scenarios in more detail.