Senior computer scientist Ashutosh Sharma breaks down the magic behind the software that gets rid of photo bombers on monument Lincoln.
We’ve all been there. You arrive at the Lincoln Memorial monument only to encounter hordes of other tourists eager to snap a photo. Chances are you’re never going to get a shot that doesn’t have a bystander in the background. Adobe’s senior computer scientist Ashutosh Sharma realizes this, and he’s working on a new software tool that will help remedy the issue. Back at Adobe’s Max design conference in October, Sharma demoed Monument Mode: a software feature that makes those annoying photobombers disappear.
Monument Mode uses a software algorithm that distinguishes between moving (people, cars, etc.) and stationary objects to help you get a clean shot. When a person or vehicle is moving, it won’t appear in the image that your phone captures. By analyzing a camera’s live view, the software gets rid of anyone who may wander into the frame before a still image is taken. The time it takes to get a good photo depends on how long those photobombers remain stationary, of course.
“I travel a lot and I like to use photo apps, but I found that whenever I tried to take a good photo at a tourist spot, there would be a number of people coming into the shot,” Sharma explained. He says that he had the idea to take a collection of images of a monument or landmark and merge them together in Photoshop or another program. That’s all well and good, but using a handful of photos to compose a single good image takes a few minutes to complete, even with something like Photoshop Elements’ built-in features designed specifically for getting rid of unwanted people.
“It struck me that the best way to do this would be right in the camera,” he continued. “That’s when the idea came for people to be able to do this live. If I wanted to get a clean shot of a monument, all I would need to do is stand still for a few seconds.”
While it may sound simple, Sharma explained that there’s a good bit of heavy lifting going on underneath the hood. In order for the software to work, it has to analyze each frame every few milliseconds, demanding a great deal of computing power. However, Sharma and his team optimized the app to efficiently use GPU power. This means the act of capturing a good image won’t be too taxing on a device. “Speed is key for the user to get instant feedback on the screen and achieve the result they really want,” he said.
“Speed is key for the user to get instant feedback on the screen and achieve the result they really want.”
The Monument Mode demo at Adobe Max was driven by software on a mobile device. However, the team isn’t sure what the final implementation will look like or when it’ll be available to lend a hand with your vacation photos. It makes sense for the final product to arrive on phones, though, as that’s how so many people capture images nowadays. Sharma said standalone cameras that allow for third-party applications and better image quality, like Sony’s RX100, are also a possibility.
“On the desktop, it’s entirely possible that, after putting together 10 shots, the final photograph is still not what I wanted,” Sharma explained. “When I’m doing it live, I know how long I need to wait. If I see that there are people moving too slowly, I can adjust to make sure that I get the shot I want.” So what happens if someone walks into the frame and stops in the middle of your capture? By increasing the capture time to add a few more frames, Monument Mode can capture one that the person won’t be in, even if they walk into the frame and stand still.
Adobe has made a big push into mobile photography over the last few years. It released Lightroom mobile for both Android and iOS and regularly added new features that delivered more desktop-like functionality to those who like to edit on a phone or tablet. With that focus on its mobile apps, it’s easy to see how something like Monument Mode would further boost those efforts.
The mobile device that was used to show off Monument Mode last year was affixed to a tripod, but how long would you need to stay still in order for the software to do its thing? “We are still not at a point where this works handheld — there are still a few things we need to work out,” he said. However, you will definitely be able to do so in the final version. “As long as you’re holding the camera as steady as if you were taking a regular photograph, it should work fine,” he added.
Sharma isn’t planning to rest easy once he’s banished interlopers from your vacation pics. He’s already looking ahead to other uses for the tech, too. Monument Mode works by analyzing the dominant pixel values of an image or frame, so it could potentially be used for things like reducing noise in images taken in low-light environments. Image quality tanks in dimly lit conditions, and a feature like that could offer a welcome improvement.
The team is also looking into the potential for using video footage rather than the live camera feed to nab that perfect image. But again, you’ll have to wait a while to be able to put Monument Mode through its paces. “We’re still trying to work out how and where all of these features come together before we release this feature,” Sharma admits. “It might not be really good for a standalone application, so we’re still trying to work all of that out.”