How we increased downloads of apps by 14% using the new icon design
This is another case about how we conducted an A/B test app icons in Google Play. We proved that only one new icon design can increase the organic load of the application by 14%. But before that, we happened to pass 16 stages of testing, change 6 concepts to deal with depressing accuracy and to bring his successful formula for conducting A/B tests.
For a start, try to guess the icon-winner:
the
“Just Facts: Did You Know?” is an application which with the help of slides tells about interesting facts from various fields of science and the world.
The main target audience was young men from India, USA, Philippines, Pakistan and South Africa.
the majority of users found the app for the words “did you know” or “facts”
So that was the icon of the application before contacting us.
2 key request to her
We have studied the competitors to understand which images to avoid and how to stand out among other icons.
the Most commonly used images, such as books, light bulbs and different kinds of abstraction
The customer also said that he has been trying to change the application icon.
He conducted several experiments:
the First icon brought less downloads, and the second icon was tested on the same app on quotes, but it didn't work
According to the customer, the icons didn't work because it was more abstract and did not give the idea of what the app is, unlike the existing icons, which included in its design keywords. These guesses, we still had to check so we don't take them as fact. The more the customer himself, asked to do something radically new.
Aims and wishes of the customer:
the
When a technical specification was drawn up, we had to create a few options of icons immediately to skip to A/B testing.
the
As requested by the customer, we decided to start to work completely different than the current options.
Here are some ideas we came:
Simple designs that attract attention:
Sketches # 1 and 2 — more major titles. Bet on the main key queries.
— Sketch No. 3 — icon in the style of American scientific and educational channel ТЕD. So we wanted to reach the audience of the United States.
— Sketch 4 — the guy on the toilet looking at the screen mobile. It's a little shocking and attracts attention.
Designs on the subject of entertainment:
Sketch No. 5, 6, 7 — card with the themes of the facts or images of different fields of knowledge, like those used in American quizzes or TV show.
— Sketch №8 — the letter "F" from the word "facts", in which the infinite universe.
designs — a psychological tricks:
— Sketch No. 9 — button with the inscription in English "do Not press", which will instinctively want to click.
— Sketch number 10 — the popular picture of a girl who looks like a floor lamp.
— Sketch No. 12 — very small app name. The reception, which should draw attention to themselves, because people will not be able to read from what is written on the icon, and he will have to look better.
Design expressing emotion:
— Sketch No. 13 — stylized character that conveys a strong emotion of surprise.
— Sketch No. 14 — the surprised alien.
— Sketch No. 15 — a man fallen to the floor jaw, from the English idiom “someone's jaw drops”, which means a very great surprise.
— Sketch No. 16 surprised Einstein or other famous in USA personality.
— Sketch No. 17 — a more classic character, looking in the direction of the name of the app in the store with surprise on her face. Thus we will force the user to pay attention to the application name.
In discussions with the client, we selected 5 ideas, each of which has been developed in several designs and colors.
Now that we have created in the end, before conducting A/B tests:
the New concept selected for A/B testing
Oh, and then it turned out not so easy, but the test results completely unexpected.
the
During the first testing all the icons worked worse than the current one. In addition, the settings of A/B testing customer left the current icon 25% of the traffic, and 75% were allocated for experiments.
Download sank
We started a second A/B-eats. This time, the current icon we have identified 70%, and three test — 10%. But the result was again unexpected.
When we first saw the old app icon, my head is spinning: “Yes, we can easily draw better!”. Here is a second test shows that high-quality visual graphics works in 2 times worse than the previous icon(
Most troubling was the similarity of the results of the new options. They all showed a decrease in downloads in the range 60-50%. All the time pursued the idea that we missed something.
the
After the first A/B test, when sank the boot, the customer no longer wanted to take risks and singled out for experiments only 30% of the traffic, i.e. each of the icons is 10%. The logic is clear: why allocate more traffic if Google scales the result.
So-called scaled installs (scalable installation) on which we orentiruetsya, is the number of plants during the experiment divided by audience share.
Ie, like how they distribute traffic less, and the result should be the same. But what of the error?
We decided to spend the so-called AAB-test to calculate the error. The essence of this test is that with the current icon to test it as a new variant and another alternative for the icon.
We spent a couple of AAB-tests and found that when 7-day test, and traffic distribution by 25% for the alternative icons, the error is from -0.2 to+10,1% when downloading at 1.5-3K.
Google says that the experiment is completed and to draw conclusions: the current icon won the current icon and download increased by 4.95% (on average) :)
That's what useful AAB test. We revealed a depressing uncertainty.
However, even taking into account the mean error 4.95%, alternative icons while working worse. We knew that the error would be smaller if the test longer. But we didn't see the point in waiting, because it was absolutely clear that the chosen strategy is not working.
the
Finally, we drew attention to the important caveat that did not betray the proper value initially. Whatever the customer wanted a radically new versions of the icons, we came to the conclusion that this cannot be done, and here's why.
The guesswork of the customer, which he expressed in the beginning of the work, that the text on the icon works much better than any abstraction, now looked quite justified.
All these observations led us to the conclusion that you need to finalize the design of the previous icon and not to create radically new.
We looked at we tested icons. Icon with a black and white surprised though Lincoln did not work, but the result was the best among alternatives. In addition, this image fits perfectly into the old design.
The task was to link the old design icons with the image of surprised Lincoln. And here is what happened:
Tests began to give positive results
So there is a new concept — to create a new icon based on the old design.
The designer has created several new options to test this idea.
Various variants of the integration of old and new design
Once again, we began testing different options. Now it was absolutely clear that we were not mistaken. The image of Lincoln attracted attention, and elements of the old design helped to convey the essence of the application.
The results of the latest tests are encouraging.
finally, we decided to stay when the average conversion rate became +14,15%
Here it is — the icon-won:
Here looks like the app icon in Google Play to date.
The number of downloads for the A/B test could be increased by 6.2% — 22.1%. And even with the mean error 4.95% this result, because your job my consider it done if the average conversion increased by 10%. The result is achieved. But, by and large, the experiment could still continue. This is probably how the path to perfection is an endless process.
the
With each new A/B testing we gain new experience and useful information to share with you. We've wrote to you about the case when the words “Free” unpredictable impact on the number of downloads of the application. This time we recorded the fact that is at odds with the conventional wisdom of the designers that the text on the app's icon is a taboo. Here's a good example of how the text on the icon in the new design worked great and increased the load by 14.15%. So, do not blindly follow the rules and standards in the design of the icons. Who knows, maybe your case will be no exception. Better follow the rules, which are confirmed by practical experience:
the
And the main conclusion that can be drawn from this case: an icon is a powerful tool that can significantly affect the conversion of your application.
Article based on information from habrahabr.ru
For a start, try to guess the icon-winner:
the
With what came to us customer
“Just Facts: Did You Know?” is an application which with the help of slides tells about interesting facts from various fields of science and the world.
The main target audience was young men from India, USA, Philippines, Pakistan and South Africa.
the majority of users found the app for the words “did you know” or “facts”
So that was the icon of the application before contacting us.
2 key request to her
We have studied the competitors to understand which images to avoid and how to stand out among other icons.
the Most commonly used images, such as books, light bulbs and different kinds of abstraction
The customer also said that he has been trying to change the application icon.
He conducted several experiments:
the First icon brought less downloads, and the second icon was tested on the same app on quotes, but it didn't work
According to the customer, the icons didn't work because it was more abstract and did not give the idea of what the app is, unlike the existing icons, which included in its design keywords. These guesses, we still had to check so we don't take them as fact. The more the customer himself, asked to do something radically new.
Aims and wishes of the customer:
the
-
the
- to increase the number of downloads of the application; the
- to test a radically different concept; the
- icon design to make colored, three-dimensional, positive; the
- to make a generic icon that, with a little variation, can be used for another application from the same industry.
When a technical specification was drawn up, we had to create a few options of icons immediately to skip to A/B testing.
the
As we have developed a new design concept icons
As requested by the customer, we decided to start to work completely different than the current options.
Here are some ideas we came:
Simple designs that attract attention:
Sketches # 1 and 2 — more major titles. Bet on the main key queries.
— Sketch No. 3 — icon in the style of American scientific and educational channel ТЕD. So we wanted to reach the audience of the United States.
— Sketch 4 — the guy on the toilet looking at the screen mobile. It's a little shocking and attracts attention.
Designs on the subject of entertainment:
Sketch No. 5, 6, 7 — card with the themes of the facts or images of different fields of knowledge, like those used in American quizzes or TV show.
— Sketch №8 — the letter "F" from the word "facts", in which the infinite universe.
designs — a psychological tricks:
— Sketch No. 9 — button with the inscription in English "do Not press", which will instinctively want to click.
— Sketch number 10 — the popular picture of a girl who looks like a floor lamp.
— Sketch No. 12 — very small app name. The reception, which should draw attention to themselves, because people will not be able to read from what is written on the icon, and he will have to look better.
Design expressing emotion:
— Sketch No. 13 — stylized character that conveys a strong emotion of surprise.
— Sketch No. 14 — the surprised alien.
— Sketch No. 15 — a man fallen to the floor jaw, from the English idiom “someone's jaw drops”, which means a very great surprise.
— Sketch No. 16 surprised Einstein or other famous in USA personality.
— Sketch No. 17 — a more classic character, looking in the direction of the name of the app in the store with surprise on her face. Thus we will force the user to pay attention to the application name.
In discussions with the client, we selected 5 ideas, each of which has been developed in several designs and colors.
Now that we have created in the end, before conducting A/B tests:
the New concept selected for A/B testing
Oh, and then it turned out not so easy, but the test results completely unexpected.
the
How we did the first a/B test icons
During the first testing all the icons worked worse than the current one. In addition, the settings of A/B testing customer left the current icon 25% of the traffic, and 75% were allocated for experiments.
Download sank
We started a second A/B-eats. This time, the current icon we have identified 70%, and three test — 10%. But the result was again unexpected.
When we first saw the old app icon, my head is spinning: “Yes, we can easily draw better!”. Here is a second test shows that high-quality visual graphics works in 2 times worse than the previous icon(
Most troubling was the similarity of the results of the new options. They all showed a decrease in downloads in the range 60-50%. All the time pursued the idea that we missed something.
the
As we are faced with the uncertainty of A/B testing
After the first A/B test, when sank the boot, the customer no longer wanted to take risks and singled out for experiments only 30% of the traffic, i.e. each of the icons is 10%. The logic is clear: why allocate more traffic if Google scales the result.
So-called scaled installs (scalable installation) on which we orentiruetsya, is the number of plants during the experiment divided by audience share.
Ie, like how they distribute traffic less, and the result should be the same. But what of the error?
We decided to spend the so-called AAB-test to calculate the error. The essence of this test is that with the current icon to test it as a new variant and another alternative for the icon.
We spent a couple of AAB-tests and found that when 7-day test, and traffic distribution by 25% for the alternative icons, the error is from -0.2 to+10,1% when downloading at 1.5-3K.
Google says that the experiment is completed and to draw conclusions: the current icon won the current icon and download increased by 4.95% (on average) :)
That's what useful AAB test. We revealed a depressing uncertainty.
However, even taking into account the mean error 4.95%, alternative icons while working worse. We knew that the error would be smaller if the test longer. But we didn't see the point in waiting, because it was absolutely clear that the chosen strategy is not working.
the
As we've found what you missed
Finally, we drew attention to the important caveat that did not betray the proper value initially. Whatever the customer wanted a radically new versions of the icons, we came to the conclusion that this cannot be done, and here's why.
-
the
- Icon with text is much better conveyed the essence of the application. It was hard to admit, because the presence of words in the icon design is considered to be unacceptable, except for the exceptions. Well, maybe we just got into this exception.
the - Text on the old icon contained key words which users normally and found the app. So why to remove them, if this is exactly what they are looking for?
the - in addition, our designer noticed that the search by name in Google shows a preview of the old icon of your app in the store — new. So, the user may get confused and close the tab.
The guesswork of the customer, which he expressed in the beginning of the work, that the text on the icon works much better than any abstraction, now looked quite justified.
All these observations led us to the conclusion that you need to finalize the design of the previous icon and not to create radically new.
We looked at we tested icons. Icon with a black and white surprised though Lincoln did not work, but the result was the best among alternatives. In addition, this image fits perfectly into the old design.
The task was to link the old design icons with the image of surprised Lincoln. And here is what happened:
Tests began to give positive results
So there is a new concept — to create a new icon based on the old design.
The designer has created several new options to test this idea.
Various variants of the integration of old and new design
Once again, we began testing different options. Now it was absolutely clear that we were not mistaken. The image of Lincoln attracted attention, and elements of the old design helped to convey the essence of the application.
The results of the latest tests are encouraging.
finally, we decided to stay when the average conversion rate became +14,15%
Here it is — the icon-won:
Here looks like the app icon in Google Play to date.
The number of downloads for the A/B test could be increased by 6.2% — 22.1%. And even with the mean error 4.95% this result, because your job my consider it done if the average conversion increased by 10%. The result is achieved. But, by and large, the experiment could still continue. This is probably how the path to perfection is an endless process.
the
Insights on testing of application icons
With each new A/B testing we gain new experience and useful information to share with you. We've wrote to you about the case when the words “Free” unpredictable impact on the number of downloads of the application. This time we recorded the fact that is at odds with the conventional wisdom of the designers that the text on the app's icon is a taboo. Here's a good example of how the text on the icon in the new design worked great and increased the load by 14.15%. So, do not blindly follow the rules and standards in the design of the icons. Who knows, maybe your case will be no exception. Better follow the rules, which are confirmed by practical experience:
the
- When selecting icon design better rely on A/B-tests than on their own opinions, trends or conventional rules. the
- do Not rely blindly on the results of A/B testing. Consider the error of using the AAB test and reduce it by altering the factors that affect it. the
- Remember Google recommendation that the longer be conducted A/B test, the more accurate data you get.
the clearer app icon conveys its essence, the better it works. the
And the main conclusion that can be drawn from this case: an icon is a powerful tool that can significantly affect the conversion of your application.
Комментарии
Отправить комментарий