Anybody with a smartphone knows the problem of snapping too many photos, with an itchy shutter finger quickly leading to hundreds of images that need to be sorted and culled.
But new software from researchers at the Massachusetts Institute of Technology (MIT) could help photographers curate and keep only the best images, thanks to an algorithm that can detect how memorable or forgettable they are almost as accurately as a human can.
The "MemNet" algorithm uses deep learning techniques, relying on a neural network that has learned to identify image patterns based on how memorable they are. You can try it out for yourself here by uploading your own photos or linking to images online.
"Understanding memorability can help us make systems to capture the most important information, or, conversely, to store information that humans will most likely forget," said Aditya Khosla, a researcher at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). "It's like having an instant focus group that tells you how likely it is that someone will remember a visual message."
The researchers schooled the algorithm in memorability recognition by feeding it a stream consisting of tens of thousands of images that had been ranked in terms of how memorable they were, based on testing with observers, which awarded each image an objective memorability score.
The collection, which at 60,000 images now stands as the world's largest image memorability dataset, was processed by the software. The algorithm independently taught itself how to find patterns in (and correlations among) the photos, by analysing the images and building its own understanding of image elements associated with memorability (based on the rankings) and distinguishing them from ones ranked as forgettable.
In addition to identifying and scoring images based on their memorability, the algorithm also produces a heat map on images to indicate which parts of them are distinctly memorable.
To test how successful their artificial intelligence was at distinguishing memorable images from forgettable ones, the researchers pitted the algorithm against a group of human subjects. The challenge was to see who was better at predicting how well another group of people would be able to recall a collection of never-before-seen images.
The results, published online in the team's paper, show that people still have a slight edge over machines in predicting human memorability, but the gap is certainly closing. MemNet scored 64 percent in correlating memorable images – representing a 30 percent improvement on existing algorithms – and was only narrowly beaten by the human testers, who scored 68 percent.
"While deep-learning has propelled much progress in object recognition and scene understanding, predicting human memory has often been viewed as a higher-level cognitive process that computer scientists will never be able to tackle," said another of the team, Aude Oliva. "Well, we can, and we did!"
The scientists have a number of future plans for their algorithm, including releasing an app that can subtly tweak photos to make them more memorable. But this kind of research doesn't just help our understanding of what makes images distinctive – it could also help shed light on the nature of human memory, such as how and why we're able to better recall particular visuals.
"You might expect that people will acclimate and forget as many things as they did before, but our research suggests otherwise," said Khosla. "This means that we could potentially improve people's memory if we present them with memorable images."
No comments:
Post a Comment