How did we squeeze a feature-rich desktop program into a lightweight mobile app?
In our case study we talk about redesigning desktop features for a mobile OS:
- The process we used to choose the essential tasks to support in our app
- How those tasks influenced which UI elements of the existing desktop program the app needed
- How we found ways to reduce the drawbacks of mobile devices
We also discuss the additional challenge we had: our app does computations in the cloud. We cover the UI feedback we provided to mimic the snappy performance of local apps. We include how we made dropped connections invisible to our users.
You could use our process for any desktop program that you’re moving to a mobile app.
Driving Behavioral Change for Information Management through Data-Driven Gree...
Mobilize: Make Good Things Come in Small Packages
1. Mobilize:
Make Good Things Come in Small Packages
Lissa Story & Claudia Wey
Copyright 2012 MathWorks, Inc.
2. The Move to Mobile
2011 Sales
120,000,000
100,000,000
80,000,000
60,000,000
40,000,000
20,000,000
0
Smart phones PCs
Source: International Data Corporation and Canalys
2
23. Move Data Cursor
Can tap anywhere to move
crosshair so you don’t
cover selection
Lesson Learned:
When a user needs to select a
specific location, find a way to
keep fingers from blocking it.
23
24. Activity:
• Form small groups
• List what features
you would include
for MS Project on
a smartphone
• Why?
24
26. Some Ideas:
• View tasks & timeline separately
• Check task dates & status
• See how changing a task date affects project
end date
• See what’s due now
• See what’s overdue
• Add a task from a template
26
27. Who: Our iPad User
Users’ Needs iPad’s Real Estate
27
33. Local App vs. Cloud Connection
User expectations:
• Local app
• Snappy performance
Lesson Learned:
Show feedback so users know
what’s happening while the
app completes a command.
33
36. Local App vs. Cloud Connection
Handling loss of
connectivity :
• History stored on the
device
• Status preserved on
Lesson Learned:
the server
Make dropped network
connections invisible to users by
saving session info and data on
the device and in the cloud.
36
37. Lessons Learned
Mobile
• Balance using real estate to show content & to make
interaction easier
• Check that OS standards are right for your users
• Provide a glimpse of essential information in context
whenever possible.
• Use the extra space of a tablet to address the pains of
the small phone screen
38. Lessons Learned
Mobile
• Know that fingers are fatter than mice
• Users tap - lots
Cloud
• Provide feedback and approximations
• Plan for dropped connections
Claudia is at CHI, so I’ll be the sole presenter todayThe Process of Translatinga Complex Desktop Program into a companion AppWho’s worked on a mobile app?Of those, who had a desktop program before they made an app version?Anyone start with a mobile app first?
[Consider skipping this slide]2011 was the 1st year smart phone sales exceeded PC sales for the year.IDC is predicting: By 2015, more U.S. Internet users will access the Internet through mobile devices than through PCs or other wireline devices. This is why we’re talking about mobile…International Data Corporation (IDC) Q4 was the 1st quarter the number of smart phones sold exceeded the number of PCs sold (100 million smartphone to the 92 million PCs).
BrieflyOur Product & User: what, who, & why – so you have some contextPhone Design: what & whyActivityTablet Design: what & whyCloud Connection ChallengesLessons LearnedQuestionsMATLAB is a programming environment that scientists and engineers use for computation and data visualizationAdd that this is a very technical app – we’ve drawn lessons relevant to all apps, like most users they don’t want to spend time learning software they just want to get their work done, but may be especially helpful to people with apps that have a lot of info or features….
This was our main product in ‘82 -- and all the commands you could do in it, including characters that have special functions associated with them (will talk more about that later)Also computation intensive
This is what a chart of data might look like – For those of you who were alive in ‘92, it might remind you a little of Asteroids
Like many desktop applications, it’s become very feature rich
We looked at our users’ needs: - We interviewed users, example: - Deal with work problems that come up when they’re away from the office: one real life user we talked to was at the airport leaving for vacation and got a call from their office about a problem that they had to fix before they leftand what iPhones are good for: - more consuming than editing – limited real estate and difficult to typeThe intersection: - type and run short commands, but not creating and editing whole programs - viewing content or data
We looked at our users’ needs:Deal with work problems that come up when they’re away from the office: one real life user we talked to was at the airport leaving for vacation and got a call from their office about a problem that they had to fix before they leftand what iPhones are good for: - more consuming contentthan editing – limited real estate and difficult to typeThe intersection: - running short commands, but not creating and editing whole programs - viewing dataMention that this is a companion app – still use desktop for more complicated functions. Reduces need to have all features on mobile, concentrate on what device is best for
For those tasks we needed 3 parts of the existing desktop application: - the command window where users enter and run commands And also to address the consuming vs. editing issue we needed 1 other part: - the command history of previously used commands that could be rerun without retypingthe figure window where you can view your resultsThis core functionality is similar to the core functionality in the original DOS version, so I guess our Mobile app is really just our DOS version with an iOS skin on it ;)
Command Window, History and Figures are the main functionalities we identify as required for the user to perform essential tasks.We organized this functionalities in separate tabs on the device.
The command window in MATLAB is the very core of our application.This is the place where are user enter commands, look at numeric result of his/her computation or can even query for help.Entering commands in MATLAB is very typing intensive. User complained that entering commands would almost always require them to access multiple secondary keyboards.
To alleviate this pain we decided to add an extra row of keys above the standard keyboard and to use popups to extend even more the number of keys we could display. We look at what our users typically type in MATLAB to identity and prioritize what keys we would show here.and if you’re wondering why we used up real estate for the banner, it scrolls away once commands fill up (provides initial branding_)
When we usability tested this, people didn’t discover the tap and hold pop-ups.We resolved this by opening the pop-up on just the initial tap. If the user didn't select one of the pop-up characters, the app inserted the character for the key they tapped.You may also want to increase the discoverability of tap and hold features
Our users workflow: they type a command which generates an image, look at the image, then type another command and so on.In a desktop you can seecommands and figures at the same time in separate windowsPhone didn’t have enough roomto show both side by side.So we show a thumbnail of the figure next to the command.The user can see that the figure was created and it’s roughly what they expected.They cantap on the thumbnail to view the full figure.
User often re-run previous commands, so like the desktop we save previous commands and they can select commands to re-run without retyping themUnlike the desktopwe also show the related figure and they can tap it to get the full figureThis reduced the need for re-running commands and also makes it easier to scan and find thecommands by seeingthe figures
The figure tab shows all the figuresTapping on any of them shows it full screen.
A navigation bar on the bottom displays all figures, so user can jump to any of the available figures.
The desktop has a data cursor so user can get values of a specific pointTapping on the show data cursor buttonwill show the values for a specific point
This is another example of an OS standard not working well for our users
Initially when users zoomed we enlarged the entire image, like a photo. When we usability tested this our users said: that’s uselessThey need to see their data. When the axes go off the screen, you can’t see the values without panning back and forth
We changed our zooming to keep the axes on the screen and just zoom the dataThis required to re-compute the plot, which introduced other challenges that we will see more in detail later.
Fingers are fatter than mouse pointers. It makes it hard to select a specific point – and can get in the way of seeing what you’re selecting.We provided crosshairs to narrow the selection point – and the user can tap and drag anywhere on the screen to move the crosshairs to the point they want(depending on time, talk about trying to avoid modes and failing)
Even if you haven’t used MS Project, I’m guessing you’ve all worked on projects and are familiar with what’s needed to plan a project~Five minutes Then I’ll ask each group to share their features and ideas with everyone
Even if you’re not familiar with MS Project, if you’ve ever worked on a project the concepts should be familiar --List of:Project tasksSubtasksTask dependencies based on orderDurationStart & End DatesTimeline view of same dataAgain, ~Five minutes Then I’ll ask each group to share their features and ideas with everyone
Some ideas for tasks that you might want to do when you’re not at your desktopSearchCalendar view
By the time we released our iPhone app the iPad was already out.Users asked for an app optimized for the iPadUsers and tasks were similarWe tried to address the pains of the phone’s small real estate with the tablet’s larger real estateThe main two pains were:Typing- while we had added an extended keyboard user still had to switch to a secondary keyboard to enter numbers.Tab Switching – we had added a thumbnail of the resulting figure, but user had still to switch to the figure tab to get to a more usabile view of their graph. They also needed to switch to the command history to get access to a set of commands they want to re-run.
Added all numbers to extra row of keys – can now program without ever switching to the other keyboardsThe figure generated are now display on a panel on a side in a much larger size, which makes them usable without the need of enlarging them for most cases.
User can keep typing commands and view the resulting graph on the side panel.
When hiding the keyboard user can view the entire list of figures.
And switching to the history tab user can get access to previous commands.
Tapping on a figure user will now have a beautiful large view of their graph.
This is another area we had to deal with:ML is computation intensive, so computation happens in the cloud - users expect apps to live locally on the device - and along with this expectation is that apps will be very responsive
One example when a figure is zoomed the app needs to get the updated information from the cloud.When user pinches, we zoom the bitmap or picture so they can see where they’re zooming and how muchThen we show a progress message while the app gets the dataThen we update the display – it happens faster in the app than I just did it
We also provide a progress message when the crosshairs are moved – also faster than what I just did
The other challenge was lost of connectivity.Mobile app which by definition means our users are mobile. They lose their signal, but we don’t want it to be too disruptive to the user.As I mentioned earlier, we save both figures and previous commands on the device, so the user could still look at his graphs and look at his previous commands.We would also save all this information on the cloudWhen the device reconnects we match up the data on the device with the data on the cloud – so they’re not starting from scratch
Just to recap, based on our experience, here’s what we found/recommendMobile:Bring a glimpse of essential information in context whenever possible, and let the user expand to the full view when needed (e.g. figure thumbnails).Don’t assume that just because you follow the iOS standards that the design will work for your app or user.Understand the specific needs of your app and the right solution for it.Usability test your app to make sure you’re meeting your users’ needs (e.g. in-axis zooming).When designing for a small screen like a smartphone, choose very wisely how you use your screen real estate. Balance between leaving enough workable real estate to display results and providing a way to make interaction easier (e.g. using some of it for extended keys).Understand the pains of your phone app that the limited real estate causes and take advantage of the extra real estate in a tablet to address them (e.g typing and context switching for us).
You can’t interact with a touch screen the way you can with a mouse. Because a fingertip is bigger than a mouse pointer, it’s harder to tap on a single point. Also fingers can cover your selection. When tapping a specific location, think about a way to keep the item in view. (e.g. data cursor).Another consequence of a touch interface is that users tend to expect a quick response to their interaction. If the app doesn’t respond immediately users tend to keep tapping, so that the app never have a chance to complete a command in response of a inputCloudYou can compensate for a lack of immediate response by providing feedback. In some cases you can also provide an approximation while waiting for results to come back (e.g. bitmap zoom).Make dropped cloud connections invisible to users by saving session info and data on the deviceMention that what we’ve done in mobile app is helping us rethink the design of the desktop UI…