How to open up all software

Tuesday, April 13, 2010 @ 06:04 PM gHale


Open software is often in the eye of the beholder, but there may soon be a way to really have an open environment and add custom features to the software of your choice.

“Microsoft and Apple aren’t going to open up all their stuff. But they all create programs that put pixels on the screen. And if we can modify those pixels, then we can change the program’s apparent behavior,” said James Fogarty, a University of Washington assistant professor of computer science and engineering who helped write a paper on the subject.

Fogarty’s approach hijacks the display to customize the user’s interaction with the program. “We really see this as a first step toward a scenario where anybody can modify any application,” he said. “In a sense, this has happened online. You’ve got this mash-up culture on the Web because everybody can see the HTML. But that hasn’t been possible on the desktop.”

These days a Web page might include a map from Google, an embedded video from YouTube and a list of recent headlines. This is not yet possible on the personal computer.

“Let’s say I’m writing a paper in Microsoft Word but I want to listen to music at the same time,” said co-author Morgan Dixon, a UW doctoral student in computer science and engineering.

Right now he would have to click back and forth between Word and iTunes, but the system he helped create can simply add a few iTunes buttons to the Word toolbar.

“I’m using some program that I love,” Dixon said, “and I’m going to stick in some features from some other program that I love, so I have a more unified interface.”

More importantly, having more control over widely used programs would allow people to benefit from accessibility tools.

An example is target-aware pointing, which can make interfaces easier for people with motor-control disabilities. One such tool, the bubble cursor, highlights the button closest to it, making it easier for people with disabilities to click a button without having to hit it dead on.

“The human-computer interaction community has done 30 years of research on how to make computers more accessible to people with disabilities. But no one change is perfect for everybody,” Fogarty said. “That’s why you don’t see these tools out there.”

This research will make it possible to personalize programs based on specific needs. The UW tool, named Prefab, takes advantage of the fact that almost all displays come from prefabricated blocks of code such as buttons, sliders, check boxes and drop-down menus. Prefab looks for those blocks as many as 20 times per second and alters their behavior.

Prefab unlocks previously inaccessible interfaces, allowing people to add the same usability tool to all the applications they run on their desktop. The system could translate a program’s interface into a different language, or reorder menus to bump up favorite commands.

Prefab can also produce more advanced effects. In one case, a user could create multiple previews of a single image in Photoshop. Behind the scenes, Prefab moves the sliders to different points, captures the output and then displays all of them on a single screen. This could save time by showing a range of effects the user frequently adjusts. The system could also allow programs to move from computer screens to mobile devices, which do not have a standard operating system.

“It dramatically lowers the threshold to getting new innovation into existing, complex programs,” Fogarty said.



Leave a Reply

You must be logged in to post a comment.