• Provided a simple bar chart and scoring solution
  • Now that the site was going to show comparisons between a specific business and its peers, we needed a compelling way to present this information. To get started, we explored a variety of different design options for how best to show comparison data. Next, we usability tested our ideas and moved forward with the design that users liked best. Our final solution combined a simple bar chart with a visual quartile score, in a format that was easy for scanning.

• Created a clean, fresh, and consistent visual design
  • We all know that it is important for information on a site to not only be useful, but to also be presented in a clear, visually appealing way. To help the Trends site appeal to its audience and convey the desired tone, we created a modern, clean look and feel for the site that also followed Intuit’s brand guidelines. We also introduced a consistent color scheme to use for “Me” and “Peers” across the site to help users differentiate between the two types of data.

You can check out our redesign for yourself at: https://workplace.intuit.com/db/bejqb2kpn.
We’d love to hear what you think!

2. Create a goal – get commitment – provide feedback
Consider structuring the DR program so that participants get a specific difficult goal for participation, commit to the goal, and then get feedback on their goal. This type of structure has proven repeatedly to be one of the strongest approaches for motivating energy behavior change. In one study, researchers gave households a difficult goal (20% energy reduction), easy goal (2% reduction), or no goal for energy use. All groups (including the no goal control) were then given information on which appliances used the most energy. The goal was also combined with feedback or not. Households who received a difficult goal + feedback conserved the most (15.1%) and were the only group to significantly differ from the control. Participants with the easy goal did not differ in behavior from the control at all. To make an even stronger program, consider an extra reward if the goal is reached.

3. Provide frequent feedback
The more continuous the feedback, the more effective the intervention. In a seminal study conducted over 30 years ago in 1979, households were given continuous feedback over a period of 11 months about monetary costs of electricity use by means of a monitor displaying electricity use cents per hour. On average, households that had a monitor installed reduced electricity by 12%. Although hourly, daily, weekly, and monthly feedback all create savings effects, the more frequent the feedback, the more effective it is. Consider creative ways to deliver that feedback via web portals, in home devices, smart phones or on SMS (intermittently).

4. Emphasize choice and control
One study considered people’s resistance to installing automatic day/night thermostats. Once the thermostat was redesigned to allow residents to override the system temporarily, the thermostat was much more attractive to residents – even though in actual use most people never overrode them. Similarly, a DR program should emphasize choice and control – people can opt into the program and still have full control over their consumption.

5. Tap into the power of the group
One energy conservation program that had a lot of success enrolled people in groups where they discussed and compared conservation behavior with their social group over a long term basis. Similarly, virtual networks of known groups can be set up to motivate participation in DR programs – for example by tapping into existing social networks of friends on Facebook to encourage participation.
In addition, consider a structure that offers additional savings if everyone in a group or neighborhood participates and reaches a set goal (see point 1 above). One study indicated that an incentive that offered on an individual and group level – in this case for all residents of one apartment building — was more effective than solely an individual incentive.

6. Frame program benefits as avoiding loss rather than emphasizing gain
The amount of joy that someone experiences when winning $100 is not equal to the consternation suffered when losing the same amount. Most people are more willing to take risks to avoid or minimize a loss than to increase their fortune. So, focus on showing residents how much money they are losing every month by not enrolling in demand response. Once the loss is obvious, people will take action.

7. Integrate complex information
When calculating energy savings, people usually can’t take into account all the elements such as rising fuel costs, the real long terms benefit, etc… So, do the math for them! Give price information that shows the full savings, presented as avoidance of a negative consequence of non-action (see point above). Use the actual data you already know about the consumer’s energy usage to make the information actionable and real.

8. Present information using vivid personal stories and videos
Statistical data summaries and impersonal information are less effective than case studies and colorful stories for motivating participation. For example, imagine that you are considering a new car and are choosing between a Volvo and a Saab. Consumer Reports informs you that the consensus of its studies is that the Volvo has a better repair record. That evening, you go to a party and run into an acquaintance who tells you a horrific story about a Volvo. Although the Consumer Reports article is based on hundred of repair records and your friend’s story is just one additional data point, most people will be swayed by their friend not to buy that car.
When communicating the benefits of a direct response program, demonstrate benefits with concrete stories about real people who save more energy than average but are “just like you. To be even more effective, present the content in videos. Numerous studies have also shown that videos of people modeling the desired actions are more effective in getting people to change their behavior than written information or lectures.

9. Use a foot in the door strategy
Individuals who agree to a small initial task are much more likely to agree to a larger request. So, instead of asking people to enroll in the full DR program immediately, first ask people to participate in a small act, such as filling out a survey, and then later ask them to consider signing up for demand response as a follow up to the first request. For example, one representative study showed that the percentage of people agreeing to an unattractive sign being put on their front lawn encouraging people to drive carefully increased dramatically (from 17% to 55%) if they had first been given the opportunity to sign a petition favoring safe driving.

10. Communicate trust
One key differentiator for successful energy programs is successful marketing to get people to even consider trying it out. We’ve found in our research that people inherently don’t trust their utility so partner with a local organization people do trust to market your program. In a marketing experiment conducted in Minnesota, a county government contracted with a private company to install energy saving equipment in homes in exchange for payment of a percentage of the value of the energy saved. To market the program, households received one of three types of letters: one letter was sent on company letterhead with no mention of cooperation with the county, one letter went out on company letterhead and mentioned the county’s role, and the third went out on county letterhead and was signed by the County Board of Commissioners. The source of information had a profound effect on consumer response – request for energy audits came from 6%, 11% and 26% respectively of households receiving the three types of letters.

Next Steps

We’re continuing to do more research in this area and will publish more insight expanding into some areas mentioned above. In the meantime, here’s a partial list of references…happy reading!

• Abrahamsen, W., Steg, L., Vlek, C., & Rothengatter, T. (2005). A review of intervention studies aimed at household energy conservation. Journal of Environmental Psychology, 25, 273-291.
• Krantz, D. H., & Kunreuther, H. C. (2007). Goals and plans in decision making. Judgment and Decision Making, 2(3), 137-168.
• Geller, E. S., (1992) It takes more than information to save energy. American Psychologist, 814-815.
• Geller, E. S., Winett, R. A., & Everett, P. B. (1982). Preserving the environment: New strategies for behavior change. Elmsford, NY: Pergamon Press.
• Lutzenhiser, S. et al (2009) Beyond the Price Effect in Time-of-Use Programs: Results from a Municipal Utility Pilot, 2007-2008. Presented at the International Energy Program Evaluation Conference, Portland, OR, August 12-14, 2009. http://drrc.lbl.gov/pubs/lbnl-2750e.pdf
• McKenzie-Mohr, D. and Smith, W. (1999) Fostering Sustainable Behavior. Gabriola Island, B.C., Canada: New Society Publishers.
• Swim, Janet et al. (2009) Psychology and global climate change: addressing a multi-faceted  phenomenon and set of challenges, A report by the American Psychological Association Task Force on the Interface between psychology and global climate change. http://www.apa.org/science/about/publications/climate-change.aspx
• Winett, R. A., Hatcher, J. W., Fort, T. R., Leckliter, I. N., Love, S. Q., Riley, A. W., et al. (1982). The effects of videotape modeling and daily feedback on residential electricity conservation, home temperature and humidity, perceived comfort, and clothing worn: Winter and summer. Journal of Applied Behavior Analysis, 15(3), 381-402.
• Winett, R.A. and Geller, E.S., (1981) Comment on “Psychological research and energy policy”. American Psychologist, (425-426).
• Yates, S. and Aronson, E., (1983) A social psychological perspective on energy conservation in residential buildings. American Psychologist, (435-444).
• Stern, P. C., Aronson, E., Darley, J. M., Hill, D. H., Hirst, E., & Kempton, W. et al. (1986). The effectiveness of incentives for residential energy conservation. Evaluation Review, 10(2),
  147-176.

For related posts about designing for the smart grid, check out:

• Tips For Improving Greenbox’s Energy Portal
• Watts All the Buzz  About Smart Grid Energy?

1) Simplify, simplify, simplify!

My foremost suggestion is simplicity. At first glance, the Greenbox interface looks very busy. There are many controls and buttons, and it tries to accomplish too much on a single page. Do users really need to be able to adjust their thermostat controls right on the main landing page, and also view their overview and data details at the same time? Instead, the home page could show only the most important information at first glance, and use color to further highlight and differentiate content on the page. The user can always drill-in to inside pages to access the less urgent information or access info on mouseover.

2) Engage users immediately with compelling content and proactive recommendations

My first response to the Greenbox dashboard is, “So what?”  I’m missing context to understand the significance of my data. For example, is my overall usage good or bad? Am I using more or less energy than I did last month? Am I on track so far this month? Also, the portal doesn’t offer any suggestions for changing my behavior so it’s unclear what my next step should be.

To engage its users, Greenbox should focus on delivering insightful content that immediately provides context and motivation. Delight users by anticipating their questions and displaying answers at-a-glance. Provide simple charts and visuals on the dashboard to compare energy usage to different time periods and to others in the community so that people understand why this information matters.

3) Adhere to basic design principles

In addition to overall experience enhancements, here are some basic user interface guidelines to help GreenBox or any other site be more usable:

• Use clear titles Is the bar chart below showing data for all energy resource usage, just the electricity usage, or only heating and cooling (because that’s what is selected in green)? It’s hard to know for sure without a clear title for the chart.

• Use visual cues to help users appropriately group related itemsThe content relationships on GreenBox are a little hard to parse.  For example, in the screenshot below, the “Heating/Cooling” button selected on the left side of the dashboard, actually updates the content shown on the right side, which is not immediately obvious.

Instead, GreenBox could make these two items more visually related to connect them. For example, put the same  green border around the box on the right and give both elements the same background color (see update below). Then, I would be much more likely to quickly understand their relationship.

Updated design with green box:

• Visually identify hyperlinks & interactive featuresTo help users understand what’s clickable, it’s important to differentiate hyperlinks and interactive features from other static elements on the page. For example, on the GreenBox site buttons and links currently use the same dark text color as other static elements, which makes them difficult to identify.

A Minimal Redesign Idea

If we take the recommendations above and do a very minimal redesign of Greenbox limiting ourselves to only minor changes to content and layout, we already begin to see large improvements in terms of usability.

For example, in this minimal redesign, the relationships between content is clearer due to stronger visual cues and use of titles, the interactive elements more readily “pop” from the page, and inclusion of arrow icons begin to tell an interesting story about the user’s energy consumption.

But we’re not done yet…

Of course, it would take GreenBox more than just a simple redesign with no change in functionality to address all of our recommendations.  As a next step, I think Greenbox would benefit to take a more critical look at its overall structure and content to ensure they lend themselves to a good user experience – in particular thinking about the best items to put on the home page and trying to answer the question “So what?”

Will we see a new and improved Greenbox site in the near future?  We hope so.  In the meantime, let’s learn both from the things that Greenbox does well and from the areas that Greenbox falls short so that we can continue to focus our energy on creating intuitive, meaningful, and persuasive user experiences.

Related post: Watts all the buzz about smart grid energy?

But, do you mean retention or forced retention? For example, I can artificially increase my retention numbers by locking new users into a long contract, and that’s going to keep them with me for awhile. Once that contract’s up, they are free to move wherever they like, and then I need to acquire a customer to replace them. And, if my contract is longer than my competitors’, it can scare off new users.

Also, the retention metric is easy to affect with switching barriers, which may increase the number of months I have a customer while making them less happy. Of course, if those switching barriers are removed for any reason – for example, cell phone number portability – I can lose my hold over long time customers.

While retention can be an indicator of happy customers, increasing retention by any means necessary doesn’t necessarily make your customers happier.

Revenue

Revenue’s another metric that seems like it would point to happy customers. Increased revenue means people are spending more, which means they like your service!

There are all sorts of ways I can increase my revenue without making my customers happier. For example, I can rope them into paying for things they didn’t ask for or use deceptive strategies to get them to sign up for expensive subscriptions. This can work in the short term, but it’s likely to make some customers very unhappy, and maybe make them ex-customers in the long run.

Revenue is also tricky to judge for free or ad-supported products. Again, you can boost ad revenue on a site simply by piling more ads onto a page, but that doesn’t necessarily enhance your users’ experience or happiness.

While increased revenue may indicate that people are spending more because they find your product more appealing, it can also be caused by sacrificing long term revenue for short term gains.

NPS – Net Promoter Score

The net promoter score is a measure of how many of your users would recommend your product to a friend. It’s actually a pretty good measure of customer happiness, but the problem is that it can be tricky to gauge accurately. It generally needs to be obtained through surveys and customer contact rather than simple analytics, so it suffers from relying on self-reported data and small sample sizes. Also, it tends to be skewed in favor of the type of people who answer surveys and polls, which may or may not be representative of your customer base.

While NPS may be the best indicator of customer happiness, it can be difficult to collect accurately. Unless your sample size is quite large, the variability from week to week can make it tough to see smaller changes that may warn of a coming trend.

Conversion to Paying

For products using the freemium or browsing model, this can be a useful metric, since it lets you know that people like your free offering enough to pay for it. However, it can take awhile to collect the data after you make a change to your product because you have to wait for enough new users to convert to payers.

Also, it doesn’t work well on ad-supported products or products that require payment upfront.

Most importantly, it doesn’t let you know how happy your paying customers are, since they’ve already converted.

Conversion to Paying can be useful, but it is limited to freemium or browsing models, and it tends to skew toward measuring the free part of the product rather than the paid product.

Engagement

Engagement is an interesting metric to study, since it tells me how soon and often users are electing to come back to interact with my product and how long they’re spending. This can definitely be one of the indicators of customer happiness for ecommerce, social networking, or gaming products that want to maximize the amount of time spent by each user. However, increasing engagement for a utility product like processing payroll or managing personal information might actually be an indicator that users are being forced to do more work than they’d like.

Also, engagement is one of the easiest metrics to manipulate in the short run. One time efforts, like marketing campaigns, special offers, or prize giveaways can temporarily increase engagement, but unless they’re sustainable and cost effective, they’re not going to contribute to the long term happiness of your customers.

For example, one company I worked with tried inflating their engagement numbers by offering prizes for coming back repeatedly for the first few days. While this did get people to return after their first visit, it didn’t actually have any effect on long term user happiness or adoption rates.

Engagement can be one factor in determining customer happiness, but this may not apply if you don’t have an entertainment or shopping product. Also, make sure your engagement numbers are being driven by actual customer enjoyment of your product and not by artificial tricks.

Registration

While registration can be the fastest metric to see changes in, it’s basically worthless for figuring out how happy your users are, since they’re not interacting with the product until after they’ve registered. The obvious exception is products with delayed (i.e. lazy) registration, in which case it can act like a lower barrier-to-entry version of Conversion to Paying. When you allow users to use your product for awhile before committing, an increase in registration can mean that users are finding your product compelling enough to take the next step and register.

Registration is only an indicator of happy customers when it’s lazy, and even then it’s only a piece of the puzzle, albeit an important one.

Customer Service Contacts

You’d think that decreasing the number of calls and emails to your customer service team would give you a pretty good idea of how happy your customers are. Unfortunately, this one can be manipulated aggressively by nasty tactics like making it harder to get to a representative or find a phone number. A sudden decrease in the number of support calls might mean that people are having far fewer problems. Or, it might mean that people have given up trying to contact you and gone somewhere else.

Decreased Customer Service Contacts may be caused by happier customers, but that’s not always the case.

So which is it?

While all of these metrics can be extremely important to your business, no single one can tell you if you are making your customers happy. However, looking at trends in all of them can certainly help you determine whether a recent change to your product has made your customers happier.

For example, imagine that you introduce a new element to your social networking site that reminds users of their friends’ birthdays and then helps them choose and buy the perfect gifts. Before you release the feature, you decide that it is likely to positively affect:

• Engagement – every time you send a reminder of a birthday, it gives the user a reason to come back to the product and reengage.
• Revenue – assuming you are taking a cut of the gift revenue, you should see an increase when people find and buy presents.
• Conversion to Paying – you’re giving your users a new reason to spend money.
• (Lazy) Registration – if you only allow registered users to take advantage of the new feature, this can give people a reason to register.
• Retention – you’re giving users a reason to stay with you and keep coming back year after year, since people keep having birthdays.

Once the feature is released, you look at those numbers and see a statistically significant positive movement in all or most of those metrics. As long as the numbers aren’t being inflated by tricks or unsustainable methods (for example, you’re selling the gifts at a huge loss, or you’re giving people extra birthdays), you can assume that your customers are being made happy by your new feature and that the feature will have a positive impact on your business.

Of course, while you’re looking at all of your numbers and metrics and analysis, some good old fashioned customer outreach, where you actually get out and talk directly with users, can also do wonders for your understanding of WHY they’re feeling the way they’re feeling. But that’s another post.

Interested? You should follow me on Twitter.

For more information on the user experience, check out:

• 6 Reasons Users Hate Your New Feature
• Is Continuous Deployment Good For Users?
• A Faster Horse – When Not to Listen to Users
• Improving the ROI for Your Customer Feedback
• 5 Things People Get Wrong When Talking to Users