The Project
Come Swim is a 2017 short film that explores impressionism. It was Kristen Stewart’s directorial debut, premiering at Sundance 2017 and screening at Cannes 2017. For my part in the project, we took the novel approach of using Neural Style Transfer – a recently developed artificial-intelligence driven technique – to literally redraw key scenes in Come Swim in the impressionistic style of the painting that Kristen painted to inspire the film.
The creative and research work we undertook is detailed in our paper, Bringing Impressionism to Life with Neural Style Transfer in “Come Swim”,
co-authored by Bhautik Joshi, Kristen Stewart and David Shapiro. The paper was originally submitted to arxiv and was presented at the peer-reviewed ACM conference DIGIPRO2017 in August 2017.
Left: a still from ‘Come Swim’, Middle: a crop from Kristen’s impressionistic painting, Right: the still redrawn in the impressionistic style of the painting using Neural Style Transfer
In our paper we discuss how we took a cutting-edge research and brought it under creative control, helping the science to be service of story. We hope that by publishing the work we can help others to use turn this nascent, rich new AI-driven visual language and into a practical tool to help tell new kinds of stories.
Using the technique we developed in the paper, we were able to use a simple parameter – ‘u’ – to control the degree of style transferred and the strength of how impressionistic the result looked.
Here we show how the camera used to capture the painting affects the result. The painting is texturally complex; a cellphone photo of it, left-column, leads to a poorer quality result than a DSLR photo of it, right-column, which captures many of the fine details which brings richness to the Neural Style Transfer.
How it came together
In late-2016 I’d performed some early experimentation with using Neural Style Transfer for redrawing videos using a painting. This got the attention of Kristen and her team; Kristen’s directorial debut was built around a painting which served as the inspiration for the script of the film.
Kristen’s painting
The team reached out:
Excerpt from the Come Swim Press Kit
…and it was my job from there on in to bring a brand-new technique that lived in the domain of research and add enough controls and steering so it
could be used in a high qualit film production. The application was novel enough that we thought it warranted publishing a formal academic paper on
our investigation into the use of the technique in a production context (see Publications, below).
We even used Neural Style Transfer for generating media to go with the movie, including the official poster.
Media
Publications
Video
Kristen discussing our work in an interview with Deadline:
A trailer for Come Swim:
A feature in the WSJ:
More media:
The film itself (watch online!):
Press
2017-01-19: Techcrunch: Kristen Stewart co-authored a paper on style transfer and the AI community lost its mind
2017-01-10: The Cut: Kristen Stewart Co-Authored a Research Paper on Artificial Intelligence
2017-01-19: Inverse: Kristen Stewart, the Actress, Co-Wrote a Research Paper on A.I.
2017-01-20: Fortune: Kristen Stewart Just Helped Write a Paper on Artificial Intelligence
2017-01-20: The Guardian: Kristen Stewart co-authors research paper on ‘pioneering’ film technique
2017-01-20: Marie Claire: Kristen Stewart Used Artificial Intelligence In Her Directorial Debut
2017-01-20: The Verge: Kristen Stewart has co-authored a paper on artificial intelligence
2017-01-20: Metro: Kristen Stewart reveals how robots helped make her directorial film debut
2017-01-20: Business Insider: Kristen Stewart co-wrote an academic paper about artificial intelligence
2017-01-20: Newsweek: KRISTEN STEWART PUBLISHES PAPER ON ARTIFICIAL INTELLIGENCE
2017-01-20: Screencrush: KRISTEN STEWART CO-AUTHORED A SCIENTIFIC PAPER ABOUT ARTIFICIAL INTELLIGENCE, OF COURSE
2017-01-20: Hello Giggles: Kristen Stewart co-authored a research paper about artificial intelligence
2017-01-20: Tech Times: Kristen Stewart Co-Authored A Paper On Artificial Intelligence
2017-01-20: Daily Dish: Kristen Stewart Co-Authors Research Paper on Artificial Intelligence
2017-01-20: Engadget: Kristen Stewart co-wrote a paper on machine learning
2017-01-20: Mic: Kristen Stewart co-authored a research paper on AI because she contains multitudes
2017-01-20: Mashable: Kristen Stewart wrote an academic paper on artificial intelligence because she can
2017-01-20: Uproxx: Kristen Stewart Co-Authored A Research Paper On Artificial Intelligence
2017-01-20: Dazed Digital: Kristen Stewart releases paper on artificial intelligence
2017-01-20: Yahoo News: Kristen Stewart (yes, that Kristen Stewart) has co-authored a paper on AI
2017-01-21: io9: Kristen Stewart Co-Authored Research Paper About Artificial Intelligence
2017-01-23: cnet: Kristen Stewart gets all AI for trippy ‘Come Swim’
2017-01-24: Lawyer Herald: Twlight Star Kristen Stewart Publishes AI Research Paper For ‘Come Swim’
2017-01-25: Vogue: Kristen Stewart debutta alla regia con “Come Swim”
2017-01-26: Hindustan times: Kristen Stewart teamed up with Indian-origin scientist and this is what they made
2017-01-31: Science Node: Come Swim heralds a tech evolution in filmmaking
2017-01-31: BTRToday: Humans Need Not Apply
2017-01-31: Acadguild: Kristen Stewart Writes a Paper on AI-Based Neural Style Transfer
2017-02-21: Psych Central: Kristen Stewart on Directing Her Movie About Emotional Pain, and Being Highly Sensitive
