I used to be one of the congregants at the now defunct Macworld Expo, the annual gathering of the Apple faithful. Steve Jobs was a draw as were the diverse technology companies imagining out loud what the future might be. Having just graduated from an institute of technology, the promise was intoxicating. I had the first version of the iPhone, the first iPad, the first Macbook Air.
As those young technologies matured in unpredictable ways, so did my relationship with technology. I stopped going to Macworld. I stopped listening to Steve Jobs’ keynote address. Instead, each year when his talk was scheduled, I listened to a different speech: Neil Postman’s 1997 lecture titled, “The Surrender of Culture to Technology.”
The speech is a tonic for our dizzying, distracting world. It presents a moment of calm to think clearly about technology. This was the general ethos expressed by Postman at the beginning of his lecture:
“We need technology to live as we need food to live. But, of course, if we eat too much food or eat food that has no nutritional value or eat food that is infected with disease, we turn a means of survival into its opposite. The same might be said of the ways in which we use technology. It can be used as life enhancing. And it can be used as life diminishing. Which means it is stupid to be categorically anti-technology, but certainly not stupid to be deeply suspicious of technology. For it is clear that technologies and the media they become can have the most serious effects on our ways of living, on our social institutions, on our psychic habits, and on our ways of experiencing the world. Therefore, it seems to me that only a fool would blithely welcome any technology without having given serious thought not only to what technology will do but also to what it will undo.”
He then proceeds to pose a series of questions that one should apply when confronted with a new technology. He describes their purpose in this way:
“What I intend to do is to offer seven questions about technology, the answers to which can provide insights into the ways technology intrudes itself into a culture and therefore affects our social institutions. The answers to the questions are important although they will vary according to the answerer. But the questions are more important. Answers change over time and in different circumstances even for the same person. The questions endure which is why I think of them as a kind of permanent armament with which citizens can protect themselves against being overwhelmed by technology.”
As familiar as I’ve become with these questions over the years, I have never applied them myself to a new technology. I will do so now. And I do so not because I have found an ideal new technology to evaluate, nor because I expect to come to any particularly grand conclusion. Rather I apply this battery of questions for the exercise of doing so. I wish to become more adept with this armament. For today, technological changes come not single spies, but in battalions. A practiced defense would be worthwhile.
Martin Scorsese x Black Forest Labs
My focus for this first exercise will be the announcement made by Black Forest Labs that Martin Scorsese would partner with the company as an advisor. They would work together to develop “Visual Intelligence for Cinema.”
While such a heading is broad, the announcement focused on Scorsese discussing using Black Forest Labs’ FLUX model for storyboarding. The heart of the announcement comprised this statement by Scorsese:
And this demonstration video:
I find the limited scope of the new technology being discussed here helpful for this first exercise in applying Postman’s battery of questions. AI discussions can run broad, and a contained conversation can be a relief. At the same time, the prominence of the participants gives the exercise weight. Whatever one’s opinion of the merits of Scorsese’s films (which I don’t plan to explore), there is no denying his significant stature within the film industry. And while Black Forest Labs may not be as familiar a name to those outside the AI industry, as a company last valued at $3-4 billion and whose technology powers image generation at Microsoft, Meta, and xAI, it also enjoys significant stature within its industry.
Before asking any of Postman’s questions, it is worth discussing a distinction one immediately observes in the demonstration video: there is a significant difference between what the technology claims it is capable of doing versus what it is demonstrated to be capable of doing. This is a common feature of AI promotion, and the gap is elided by Scorsese’s proclamations, “Look at that. That’s it.” And, “You know, this conveys a cinematic intelligence…So, that’s the great thing about this tool.”
In the video, Scorsese is using FLUX.2—notably with a human intermediary—to create storyboards, visual representations of shots to be filmed. Storyboards help anchor the film team in a common vision for the film. Scorsese narrates a description of a road in a town. He begins by describing it as:
“I have in mind a town, not a village, not a city, a kind of gnarled old, almost medieval town in the Caucasus, let’s say. Even the streets are narrower, cobblestoned.” Then this is the image the video shows:
Scorsese continues: “The main road through the town is twisting and turning. It may be over small hills.” This is then the image the video shows:
Between the two images, the most noticeable difference—less snow on the road—was nowhere a part of the conversation. The image did not respond to Scorsese’s hill direction. The layout of the town does not seem to have changed, but the path the cars travel is slightly more curved. And there are now, inexplicably, two lanes of traffic. Note the very specific time period of the car when we did not witness Scorsese specify a time period.
Scorsese then gives feedback to what he sees by directing, “Even narrower.” The person running the model confirms, “Even narrower, okay.” We then get a shot of the computer screen that shows:
The street does appear to be slightly narrower. Of the other characteristics Scorsese has proposed—gnarled, medieval, twisting, hills—it would be surprising if all those elements appeared just as he had imagined without the need for qualification, particularly since some elements don’t seem to appear at all.
Next Scorsese describes the driver of a vehicle. This image appears as he begins his description:
“The driver would be again very Eastern European, almost like someone you’d find in the street there, older, well, he’d probably be in his 40s, but he’d look like he’s in his 60s. Longish hair.”
This is the only image of the driver that appears. It is unclear if it took multiple iterations to get there or how the image would respond to Scorsese’s feedback.
So this is what we have: a model that has demonstrated two instances of image generation (without clarity as to how many iterations were necessary to produce the initial images we saw), one instance of image adjustment unrelated to instruction, and one instance of image adjustment responding to instruction (without clarity if there were intervening iterations). The demonstration is spare enough that a third of the three minute video is just Scorsese talking about a shot in Goodfellas that has nothing to do with the technology discussion at hand.
A shocking aspect of this demonstration is the disproportionate control the technology has over the vast majority of features and details. Scorsese has made so few decisions (variously heeded by the machine) relative to the numerous decisions made by the technology. In contrast to his reaction in the video, I cannot imagine Scorsese being satisfied with such a result and using it in his creative process. He would certainly have more to say about the architecture, the light, the field of view, the camera angle, and countless other details. If the problem he is trying to address is how to communicate the fullness of picture within his mind to his collaborators, here very little has left his mind and found its way into the storyboards.
So it is a leap to go from this demonstration to saying we have a new storyboard technology. But to be able to continue the exercise of applying Postman’s questions, I will anticipate a point in the future when a new storyboard technology might function effectively for generating and iteratively developing a wide array of storyboards via the type of interactions depicted in the video. I will now apply Postman’s questions to this future, functioning storyboard technology.
Question #1: What is the problem to which a technology claims to be the solution?
Within Scorsese’s statement is his response to this question:
“For 70 years, l’ve been creating my own storyboards. There’s always been this problem of how do you communicate what you see in your head to your cast and crew…Now, with this tool, I can share what l’m visualizing more clearly and efficiently to my creative team…I recently tested this out on a scene and the ability to visualize and immediately share the storyboard was creatively freeing. During the pre-production process, time costs money, and this allowed us to move faster without sacrificing quality or craft.”
A way to rephrase Scorsese’s response: for 70 years, to create storyboards has not been a problem. And 70 years is not an exaggeration. Here are storyboards by an 11 year old Scorsese for his imagined Roman epic, The Eternal City.
If Scorsese no longer wished to create storyboards himself, there is no shortage of storyboard artists in the industry. For decades there has also been non-AI software to assist in the process of storyboarding. So with the array of options available, there is no problem in the ability to create storyboards.
Scorsese explains that this tool allows him to “more clearly and efficiently” share with his team. That to be able to do this immediately is creatively freeing. This is then followed by the adage that time is money. One of the distracting qualities for new technologies is the touting of features, whereas the power of this first question of Postman’s is to focus us on the task of identifying a problem that this technology fixes.
Was the clarity of storyboards a problem? I have never heard of such a thing. I believe they can be as detailed as one wishes. So now we are left with the issue of time. Let us assume that this new storyboard technology reaches a point where its ability to generate and iterate images far outpaces human abilities or human abilities assisted by non-AI software. Let us further assume that the economics of time saved exceeds the cost of the new technology. We have saved money, but have we identified a problem? Time is money, but I have yet to hear tales of a debilitating storyboard budget overrun. I have yet to hear tales of pre-production delays due to storyboarding.
This new technology cannot be seen as a solution to a significant obstacle faced by film production. It can only be seen as a solution to the problem of modest expenditure, replacing it with a different modest expenditure. This is as close as we can get to an answer to Postman’s question. It is unsatisfying as it is a problem that carries no urgency. If a modest expenditure were a true problem, it could be solved by no expenditure. This is the case on my movies as I do not use storyboards—I have other ways of communicating visuals with my collaborators. But I doubt there will be a wave of productions suddenly converting to my system, for this is no urgent problem. It is merely the closest we can get to identifying a problem that this new technology solves.
Question #2: Whose problem is it?
As the one paying the bills and managing the budget, this is the producer’s problem.
Question #3: What new problems might be created because we have solved the problem?
One of the most striking features of the demonstration in the video is that throughout the process, no one asks a question. A significant share of a director’s existence is spent answering questions. A great joy of being a director is to be able to ask questions. Here not a single question travels in either direction (except for the throwaway, “Does that kind of look like it?”).
Scorsese dictates, and the technology returns outputs, not questions. Scorsese dictates more. I believe the absence of questions in the process creates two types of problems. Both problems emerge from trading away a better sense of where you are going merely to get somewhere faster.
The first type of problem impacts practical communications. When a director communicates with a human storyboard artist, that storyboard artist will have questions. These questions may call attention to details the director has not yet considered. Informed by film experience, these questions may seek clarity for issues that impact collaborators later interpreting the storyboards. These questions strengthen the visual communication process and can help prevent misunderstandings and the need to redo storyboards later down the line. This is lost in the new technology. It is then plausible that the new technology actually results in more subsequent economic costs than were initially saved as new iterations are required. This is not unheard of in film production. There is a saying in the industry: “No one ever has the money to do it the right way. But they always find the money to do it again.”
The second type of problem impacts the artistry. The fundamental tool of art, more powerful than any technology present or future, is the question. One’s collaborators in art are as much partners in questioning as in creating. There is a loss here of a collaborator with a different perspective than you, with a different set of knowledge and experiences, but who can come to understand you and your project and put all those differences in your service. This new technology cannot come to understand you for to do that it must ask questions. Thus the vigor of art that is gained from collaboration—including its resilience to inadvertent error—is lost.
Question #4: Which people and what institutions might be most seriously harmed by a technological solution?
Storyboard artists lose their livelihoods with this technology. Note the demonstration did not show what a storyboard artist could do with the technology. It aimed to demonstrate that a storyboard artist was not necessary.
Previously we discussed how this new technology could complicate the work of designers down the line and even result in more costs for producers.
Now I would like to discuss an unintuitive candidate for potential loser with this technology: the director. First of all, it adds the burden of navigating the new technology and the responsibility of storyboard creation to the director’s workload. I assume Black Forest Labs does not intend to provide a live, admiring intermediary for each user of this technology. Second, the ease with which a director can generate storyboards with the new technology does not belong to the director alone. Now a producer can easily generate storyboards. Different departments can come to meetings with their own storyboards. Ease can lead to proliferation.
But let us consider just the case of the producer generating storyboards. If the new technology lives up to its promises of ease and economy, such acts will certainly come to pass. Once a producer generates storyboards (or shapes prompts alongside the director), this diffuses the center of vision for the film. Before the new technology, the director presumably earned the spot at the center of the vision for the film through investment of time and effort. Erode the need for time and effort, and the director’s role—perhaps not without reason—seems more arbitrary and open to questioning.
Question #5: What changes in language are being promoted by new technologies?
This announcement introduced me to the term ‘cinematic intelligence,’ the culmination of this sentence attributed to Scorsese:
“Now, with this tool, I can share what l’m visualizing more clearly and efficiently to my creative team—the production designer, art designer, and cinematographer—for them to build on to enrich cinematic intelligence.”
I consider it bad manners to introduce such a fanciful term and not leave a single context clue as to its meaning. I would have thought that cinematic intelligence would have been some quality assigned to the new technology, but this sentence places it as an end goal; it is what results after the creative team has done its work. More than that, their human work is meant to enrich it.
The language of the new technology is reorienting the whole filmmaking project around an undefined, hitherto unknown goal. It makes this goal in its own image. I expect that ‘cinematic intelligence’ will mean many things to many people. As a marketing term, I think it will come to mean that a new AI technology is capable of passably dressing an idea in the cinematic conventions of the time. Thus an average and obvious output is crowned with the title of intelligence.
Question #6: What sort of people and institutions acquire special economic and political power because of technological change?
There are three types of power that accrue to the makers of the new technology.
First, it has discretionary power over the training data and alignment procedures. Either data selection or alignment procedure could skew the range of what is possible with the new technology. This could happen inadvertently or as a more conscious—potentially political—act. This can apply to content as well as aesthetics.
Second, artistic output so clearly derivative of previous art—that was necessarily pervasive enough to produce a learnable distribution—is inherently conservative. That is, it conserves and continues the old ways of doing things. Brecht believed that presenting the world in the same manner as we already experienced it inherently perpetuated and conserved its problems. If there is any truth to this, then this new technology contains a bias toward conservatism and away from progress.
Third, it is worth noting the economic dynamics that often accompany new technologies like this. At its introduction (and subsidized by venture capital), it could be priced to undercut the readily available human alternatives. As the new technology use becomes more pervasive, human alternatives are priced out of the market, and so those skills become rarer. With this market change, the firm producing the new technology acquires market power. Once alternatives are scarce enough, the firm is able to exercise market power and increase price.
Question #7: What alternative uses might be made of a technology?
When I try to imagine an alternative use for this new technology, its use changes sufficiently that I am not certain it qualifies as the same technology. This may be a natural consequence of applying this question to a very specific, narrow technology. One can imagine many different ways to use the medium of television, fewer ways to use storyboard technology.
In general, the alternative uses I imagine are to assist—rather than replace—the work of a storyboard artist: perhaps to edit elements the artist has themselves created or to composite in elements other departments have created. However, I think the driving question for alternative use should be Question #1: identifying a problem for which this technology is the solution. Every film department is constantly developing technological solutions to problems they face. But those tools assist their ability to create rather than abdicate it.
So there it is: my first training session with Mr. Postman’s armament. A bit clumsy for being a beginner, but stumbling through feels like it has been worthwhile. Some answers I thought I had, and they turned out not to be answers at all. Questions I thought I could not answer, a response did emerge in the end. I feel a student again, instructed by the ghost in the machine how to beware machinery.
Hadn't seen that video. It's frightening to see him so readily accept what the program spits out, especially because (like you say) if he were actually using it to make a film, he'd probably have to spend a lot more time with it in order to have it accurately depict what he had in mind (time one could just spend storyboarding...). I also love your point about the questioning part of collaboration and how this is often what leads to creative discovery. I think this connects to problems of friction and de-skilling. I'd be interested to see an argument for what a genuinely worthy use case of this tech (in this context) could be. Upon first impression, it seems like it would be most useful as a novelty or a crutch. Maybe there's some interesting experimental stuff you could do though. I also think these problems of art and AI rely on the notion of art as product rather than process/creative meditation. But anyway, a great piece!