Campaign Center

Overview

Vitech broadly serves three industrial verticals - Insurance, Investments, and Retirement. For the clients in our insurance segment, their entire claims processing happens on Vitech's platform. They currently need to export their user data from our platform and import it into other third-party campaign builders. With their data living in the Vitech platform, the product team identified that we could empower our customers to maximize their revenue and engagement by building a campaign feature right into the platform.

Introduction to the Platform

Vitech's next-generation React platform is a robust framework that serves as a base for all our new self-service apps. We use React Material under the hood that is modified to serve Vitech's Design System components.

Measures of Success

We will consider the product to be a success when:

1. We have proven usage by our customers to be equal to or higher than their current usage of third-party campaign builders.
2. We have proven that 80% count of customers have run campaigns.
3. Metrics to be further refined for early adopters and for when the product is generally available for all customers.
4. We've gotten positive feedback (qual data) from user interviews confirming that we've solved the problem, for example: When we have helped MetLife with their compliance needs or helped MTRS reduce their number of unenrolled members.

Our customers' business processes are manual labor intensive (like claim adjudication, claim adjustments, etc.) and require multiple activities. Each of these activities can require its result conditions which result in subprocesses. Our greatest opportunity and success would be to automate and manage these processes which in turn (the hypothesis) can help our clients support their larger goals.

Introduction

Before we drop in, I want to remind you will be walking through a clickable prototype rather than actual software. This means we have simulated some functionalities and not others. If you click somewhere and something doesn't happen or if you clicked somewhere and multiple things happen, it doesn't mean you clicked in the wrong spot, it just means that we didn't simulate that portion. If you get stuck, we want to hear your thought process as to why you are stuck, I may step in to nudge you in a direction if we need to move on to other sections of the design.

I am going to give you some background on your user role and the area of the Campaign Center tool we are in then give you a series of tasks to perform. Please remember to think out loud as much as possible so we can get insight into how you perceive the tool and the design concept. If you are ready, let's go ahead and get started.

From our last user testing session, we went through and set up the backbone of a campaign. Now let's take it a step further, you have gone through and created the campaign and several outbound activities (emails, messages, letters) associated with the campaign. You are going to be a user for one of our pension funds, your goal is to navigate around and perform several actions on the inflight campaign.

Questions to ask

Here are the tasks you need to execute to accomplish this goal:

1. What are your first impressions of this page? What information are you seeing here?
2. We will start on our Active Campaign Dashboard > Summary tab. How would you change your view on the summary statistics from being counts to percentages?
   • Is the data displayed here relevant to you as a user?
3. Please find where you would set up your outbound emails/letters/messages.
4. How would you filter this Activities list to only display the Future activities to send out?
   • Is the data for outbound touchpoints here sufficient to understand what is being sent out, to whom, and when?
   • Please click All to back out of the View drop-down.
5. Please find where you would view the full list of people targeted by this campaign.
6. How would you view a subset of the Audience that is 'Converted' and age 65 or over?
7. If the audience segment that you want is not already available, where would you go to create a new audience segment?
   • [Let them know that nothing here is currently automated] But looking at this list of filters how would you create an audience segment of people aged 30-40, who are in a "Potential" status, and who are missing an email address?
   • Where would you go to name that audience segment?
8. Let's say you need to perform a quick Audit on your interactions with people in the campaign. For example, the ones who converted on August 18th. How would you get the data to do that audit?
9. [Have them navigate back to the summary tab] Let's say you finished all the activities you wanted to accomplish on the current campaign, how would you navigate to another existing campaign?
10. What if you do not see the campaign you want to work on immediately displayed, what are the different ways you can find the campaign you need to work on?
11. What if you only wanted to see customer campaigns, how would you do that?
12. Now what if you change your mind, how would go back to viewing our current campaign in the full screen?

Follow up

Great, we're finished with the bulk of the test. You mentioned [something they said out loud perhaps?] earlier and I didn't want to jump in at that time. Can you say more about that?

On a scale of 1-7, how would you rate the experience of using the design?

What are 3 adjectives that come to mind when thinking about your experience with the design?

Research Methods

1. Remote (in-situation) usability testing was conducted on a convenience sample of 5 test participants (2 internal and 3 external) from 08/01/2021 - 08/15/2021.
2. Participants were shown the Create the Campaign clickable prototype and instructed to perform an aforementioned task and provide feedback on their experience interacting with the UX design concept/clickable prototype and potential features via the think-aloud method.
3. Sessions lasted approximately 30+ minutes and were recorded via MS Teams with both video and audio.
4. Following data collection, a content analysis was performed by watching recorded test sessions to extract user feedback in aggregate and identify overarching themes.

Overall Experience (Active Campaign Concept)

Overall Experience (Create Campaign Concept)

Test Participants

Create Campaign Concept

1. All 5 test participants successfully completed the task flow of creating the campaign as the first time user of the app's UX concept, from the initial screen with the text copy and the CTA button being contextually clear, to setting up the campaign name, start/end date of the campaign, specify the goal of the campaign, selecting appropriate target type, then choosing relevant target group, and performing additional interval settings, then clearly understand the information that's being displayed on the review & confirm screen.
2. 2/5 test participants didn't realize that they should scroll vertically downward to be able to select the target type perhaps due to the vertical whitespace between the sections on Step 1 screen.
3. All 5 participants successfully completed the task of choosing the target group(s) on Step 2 screen.
4. 2/5 participants were not aware they could change the frequency value on Step 2 screen. This may have been more to do with prepopulating fields on the prototype and less with the prototype itself.
5. 2/5 participants were not aware that the CTA button 'Edit' on the 'Review and Confirm' screen would change the page they were on.
6. 2/5 participants did not use the 'Return to Confirm' CTA button on page 1 to skip to the Confirm screen.
7. All participants successfully clicked the pencil icon to change the name.

Active Campaign Concept

1. All 5 test participants successfully completed tasks on the 'Summary' tab. Bulk of testers wanted the default to be the 'Percentage' view of summary stats and see more information related to data trends.
2. 3 of 5 participants successfully completed tasks on the 'Activities' tab. Number of participants found the name of the tab to not resonate with a term they would use for managing outbound correspondence.
3. 2 of 5 participants successfully searched for existing audience segments on the 'Audience' tab and 3 of 5 were not able to navigate to the segment creator on the first attempt.
4. All 5 test participants successfully navigated to and interacted with the Campaigns ribbon for navigating between campaigns.