marketing powerpoint

profilemometag
project_2-ab_test_plan.docx

Project Deliverable 2: A/B Test Plan

What platform are you using for the test?

We are using Google Analytics as our platform for the experiment. We will be using Squarespace as our User Interface with designing the two different front pages. Squarespace offers simple control for web design, and has the capability of hiding webpages if needed.

What treatments will you test? (You may still be working with your client to figure that out, but give me one idea.)

Since the main goal of the website is to guide or convince the consumers to request a quote from us by hitting the button, “request a quote”, we and the company sought best to display two versions of the front page. The original front page will have a button on the top right that says, “Request a quote”. Once the button is pressed, it will guide the users to the page where they can request the specific chiller that their company requires. The second front page will have the same purpose; however, the visual display will be much more direct. The button will have an arrow pointing at the three words: “Request a quote”, and instead of having a visual slideshow, there will be a bland front cover with the company’s logo.

How will you assign treatments to customers?

The Google Analytics Interface offers an experiment mode that enables us to test almost any change or variation to the website to measure its impact. Within the configuration of the experiment, we will have the original page as Variation 1, and the direct-approach page as Variation 2. We then choose to split the experiment 50/50 for all users who approach our website. Google Analytics will automatically help us divide the population randomly for testing.

What response measures will you use to evaluate your treatments?

We will evaluate the experiment based on the number of page views on average over the time of 3 weeks. However, this is the tricky part. Since the goal of the website is to convince the user to click on the “request a quote” button to negotiate for a chiller, we cannot measure the page views of the two front pages. Instead we will design two “request a quote” pages. Afterwards we can measure how many page views the two pages will receive; therefore, finding out which front page is more effective.

How will this test help inform future decisions that the client will make?

The client’s main goal is to see how digital marketing can help drive their sales up for the company. Drake Refrigeration has always been communicating with wholesalers for clients, and thus decided to go for a more direct approach. This test will help them know which front page can drive more potential clients. The more potential clients they have, the more actual clients they will have, it’s a simple correlation. Our experiment will not only introduce them to the power of digital marketing, but will also show them the most effective way to execute.

Will you have enough data to make a conclusion? Make sure you make fake data and analyze it to determine if your confidence interval will detect a difference between the two groups.

According to our previous analytics over the past few months, it is clear that our website takes in an average page view of 5000 visits a month. This amount of visits, is a good indication that we will have enough data to make a conclusion.

We performed a mock test with fake data by using past 19 days from our website and generating another set of data using random number generator between the minimum integer and maximum integer of the real data. Here is the data:

Day

Visit to Targeted Page-A

Visitor(A)

P(A)

Visit to Target Page-B

Visitor(B)

P(B)

1

73

212

0.34434

29

90

0.322222

2

7

20

0.35

46

99

0.464646

3

13

44

0.295455

28

82

0.341463

4

49

170

0.288235

36

171

0.210526

5

62

233

0.266094

44

99

0.444444

6

64

235

0.27234

69

182

0.379121

7

50

220

0.227273

33

124

0.266129

8

50

196

0.255102

61

76

0.802632

9

11

22

0.5

68

155

0.43871

10

18

44

0.409091

38

122

0.311475

11

56

163

0.343558

11

66

0.166667

12

68

185

0.367568

49

50

0.98

13

74

227

0.325991

13

24

0.541667

14

41

163

0.251534

67

55

1.218182

15

63

175

0.36

23

102

0.22549

16

16

78

0.205128

17

162

0.104938

17

12

36

0.333333

29

95

0.305263

18

50

186

0.268817

43

21

2.047619

19

42

110

0.381818

12

85

0.141176

Targeted Page A= Request a Quote Page A Targeted Page B= Request a Quote Page B

P(A)=

P(B)= Based on these two data sets, the average of the average of P(A) is 0.318194 and P(B) has an average of 0.511177. Under these conditions, we can compute a confidence interval for the difference between the percentage of successes in group A and group B giving us the confidence interval of {0.10646014, -0.492426}. This means there is no significant difference between the two groups.