There’s a reason why my iPhone is often the only camera I’ll carry with me. Its imaging quality allows me to take good photos without having to carry any additional gear.

My DSLR still beats my smartphone considerably in terms of quality, but I don’t mind the quality loss in lots of non-professional scenarios. There’s a lot more happening besides good optical quality in my smartphone, however, and its camera wouldn’t be great if it didn’t have help from computers.

While DSLR and mirrorless technology is evolving, recent photography has seen some of its most revolutionary changes in places like the smartphone and compact-camera worlds. Since there are size and cost limits to the imaging hardware that can be used in these cameras, innovation has come in the form of computational imaging processes. With recent rises in the accessibility of artificial intelligence, computational photography is evolving and will continue to grow. This article will provide an overview of computational photography and why it matters in many different contexts.

What is Computational Photography?

If you’ve never heard the term before, you’re not alone. Computational photography uses computing techniques such as artificial intelligence, machine learning, algorithms, or even simple scripts to capture images. This is usually in conjunction with or after optical image capture. Although it might sound obscure, nearly every smartphone camera uses computational photography in some way. Without it, smartphone imaging capabilities would be much lower.

Examples of Computational Photography

Chances are good that you’ve used computational photography if you own a smartphone. If not, you’ve probably seen people use it in lots of different ways.

Portrait mode on a smartphone

In most professional cameras, the portrait-looking effect comes from using a wide lens aperture. This creates a shallow depth of field and, without getting too complicated, blurs the background of the image.

Most smartphone cameras have fixed apertures or apertures that don’t open wide enough to naturally create the portrait-esque blur. This would be really hard and expensive to implement using hardware, so smartphone manufacturers use computational photography to create a simulated background blur. Portrait mode recognizes subjects and essentially overlays a blurry filter on the background. This uses computational processes to recognize, isolate, and focus on the subject in the image in real-time.

Panorama modes on smartphones and cameras

Many smartphones and newer cameras have built-in panorama modes. With the push of a button, the imaging device will direct the photographer to move and keep their camera along a straight line so that the device can take multiple images and stitch them in near real-time. This creates a panorama in-camera, rather than having to manually stitch the images.

Computational photography is at work guiding the photographer, stitching the images, and creating a single panorama file that can be viewed instantly. Since most smartphones and smaller cameras have electronic shutters with minimal moving parts, the shutters can be activated very rapidly, allowing for quick panorama capture.

High dynamic range (HDR) modes on smartphones and cameras

With the press of one button, this feature uses algorithms and even machine learning to recognize the brightest and darkest parts of a scene. The camera will take photos at different exposures and combine them seamlessly, leading to a final product with detail in the brightest and darkest parts.

This all happens quickly in most smartphones because a smartphone camera is always imaging when the camera app is in use. Otherwise, the live “preview” couldn’t be shown. Only the images that you press the shutter button for are saved to your device’s photo album, but there is always a buffer of photos that your smartphone hangs onto for a little bit and then discards. This is how Live Photo works in iPhones, and this is why you will probably see movement from before you actually pressed the shutter button in any Live Photo. In smartphones, the shutter button is very much like a “pause and save” button in a continuous imaging stream.

Daytime “long exposures” on your smartphone

Similar to Apple’s Live Photo feature, many smartphones offer a feature to actually stack multiple buffered images to create a long exposure effect. This is a similar concept to time-stacking, which many landscape photographers use to get long exposures of water when it’s too bright to take a long exposure. The technique involves taking many shorter exposures and combining all of the imaging data using post-processing. Smartphones will use computational photography to align and combine shots from the buffer and create a long exposure effect.

How to do this on an iPhone: First, take a Live Photo in your Camera app. Then, find the image in your Photos app and click on the icon that says “Live” in the upper left corner. The Long Exposure button will combine all of the buffered images that comprise the Live Photo into one stacked image.

Night mode in many recent camera systems

Images at night are hard to make because there’s not enough light to create a well-exposed image with good contrast. In professional cameras, slow shutter speeds, wide apertures, and high ISOs help, but those usually require a tripod to minimize camera shake. Most people don’t want to use their smartphone camera with a tripod.

Many recent smartphone manufacturers have implemented a similar process to the buffering and stacking outlined above, which takes shorter exposures (minimizing motion blur) and combines them to create an overall higher-quality and brighter image because there is an increase in the total amount of data captured. If you combine that technique with images that are brighter and darker to capture more dynamic range (which some smartphone cameras do), you get images that look like they might have been taken with a DSLR when they were really taken with a tiny lens and sensor.

Computational Photography Uses Many Different Processes

This article is simply providing an introduction to computational photography, so the technical details of algorithms and other techniques are beyond its scope. However, there are some really interesting techniques being used in the context of photography.

Neural networks and machine learning

In many recent cases, manufacturers have implemented neural networks (basically, simulated brains that use “neurons” to “think”) in computational photography. By showing artificial intelligence images that are too bright, too dark, or discolored, the system will be able to recognize when an image has those characteristics and attempt to fix it. Although it won’t always be perfect, the billions of images that are made each day make for a huge database to use to teach systems how to judge and correct images.

Pixel shifting

In computational photography, image-making is often an additive process. In other words, one image is rarely the product of a single image. Rather, an image is a combination of many different images with different parameters. Pixel shifting is another example of this. This is often available in smartphones and very high-resolution mirrorless cameras. This process physically (using a mechanism or making use of naturally shaky hands) shifts the sensor one pixel at a time. By combining multiple images just a pixel apart, there is more overall information captured.

The exact means for how this happens is beyond the scope of this article, but the important thing is that this is all made possible by computer processes to shift, combine, and output one higher-resolution, sharp image.

Pixel binning

Very basically, pixels help capture light, and larger pixels capture more light. However, smartphone cameras need to be small, and therefore their pixels need to be small. Pixel binning allows the data from four pixels to be combined into one, which increases the overall quality of the image without sacrificing low-light capabilities. Without computational photography, this technology wouldn’t be as accessible.

Focus stacking

This is fairly simple and used to be a very laborious process that only extreme macro photographers would do often since they work with extremely shallow depths of field. This involves taking different images at different focal points and combining them to create one image with greater depth of field and detail. Smartphones and other cameras will do this automatically with the click of a single button and use computer algorithms to align and stack the images in seconds.

Why Does Computational Photography Matter?

Manufacturers are finding ways to make 12-megapixel sensors produce similar results to 50-megapixel sensors (although there are still limitations). This is all because of the power of algorithms, machine learning, scripts, and other computer-enabled processes that maximize the imaging capabilities of traditionally lower-grade hardware.

It’s not a perfect analogy, but a professional racecar driver could probably beat a novice driver in a race even if they swapped cars. The racecar driver, just like computational photography, uses skills to make the most out of the lower-grade hardware, while the novice driver has the hardware and lacks the skills. Computer processes are quick and efficiently taught, and they are being developed to maximize hardware that would typically not give great results. This lowers materials cost for expensive hardware and makes more possible with what we already have.

In some ways, this is a frightening aspect of modern photography. Traditionally, photographers have been championed because they have the knowledge, equipment, and experience to create stunning images in many different environments. If computational photography makes it possible to do more with less hardware, photographers need to compete with everyone who has a smartphone. There will always be the artistic side of photography, however, which will belong to photographers for at least a little bit longer. The rise of AI has brought important questions to the art world, and photography is no exception.

Computational photography is also making its way into the professional photography world. DSLRs and mirrorless cameras are seeing changes in autofocus systems that involve AI subject-identification, HDR, panorama, and other processes described above. It’s not just a smartphone thing, anymore, and technology is changing – for better or worse.

Conclusion

We’ve come a long way from loading film into a camera and advancing it to get to the next shot. Significant advances in computer technology have recently made their way into smartphones and other image-making devices, something that many would’ve never thought possible.

Computational photography has traditionally been used to make high-quality photographs using hardware that would otherwise not produce such quality, and it’s making its way to professional cameras. Regardless of anyone’s opinion on the innovation happening today, it’s important to stay informed on technology and how imaging processes work so that photographers can use the tools available to them.

Image credits: Header photo by Ted Kritsonis for PetaPixel

When it comes to sports portraits and action photography, employing a bit of artificial light can make a huge difference by giving you the ability to shape the look of the frame and add a lot of drama to the image. If you are ready to get started working with artificial light in your sports action portraiture, check out this excellent video tutorial that will give you a range of helpful tips and advice sure to put you on the right track. 

Coming to you from Matt Hernandez with Westcott Lighting, this awesome video tutorial discusses the ins and outs of working with artificial light for sports action photography. The beauty of adding a bit of artificial light to this sort of portraiture is that you can overwhelm the sun, making it easy to control the drama in the frame. While you might normally think that it is easier to work in a studio in this sort of scenario (and from a technical perspective, it is), working outside allows you to photograph your subjects on the tennis court, the baseball diamond, the football field, etc., lending your images a much higher level of realism. Check out the video above for the full rundown from Hernandez.

On April 5, Web3 photography platform Fellowship dropped Post Photographic Perspectives, its inaugural group show headlined and curated by designer and AI-collaborative artist Roope Rainisto. Featuring 500 new pieces from Reworld, Rainisto’s latest collection, and a further 1,000 works from 10 AI artists hand-picked by Rainisto and Fellowship, Post Photographic Perspectives is one of the most highly-anticipated NFT drops of the year. 

The show sits at the center of several exciting technological and cultural developments that have sparked intense debate both within and outside of Web3’s walls in recent months. We sat down with Rainisto and Fellowship’s Co-founder, Alejandro Cartagena, ahead of the drop to get their views on the discussions surrounding AI art and the importance of pushing technological and artistic boundaries forward. 

Rainisto’s vision meets Fellowship’s mission

Rainisto has seen a monumental rise to fame in both the NFT and AI art spheres in the aftermath of Life In West America, his successful BrainDrops release from February 2023 that presented a fever-dream vision of the idea of the American West. That collection helped cement BrainDrops’ status as one of the premiere platforms for AI art in Web3 and has done nearly 1,800 ETH in trading volume on the secondary market since its release.

Rainisto’s style is both immediately recognizable and compellingly bizarre. Rather than try to hide the fact that his works are the product of AI art tools, his AI photography leans hard into the “AI-ness” of his image outputs, creating something at once familiar and alien. Rainisto’s work, more so than perhaps any other artist’s in the space, plays with the brain’s inherent pattern-seeking capabilities, presenting viewers with scenes that they have both seen a million times and yet have never come across. 

“In my many years of studying photography and art, there’s something [in Rainisto’s work] that I’ve never seen before,” Cartagena said of Rainisto’s unique aesthetic while speaking with nft now. “He’s the only person doing the things he’s doing with AI. His vision is very particular, concise, and technically sound. What he has done is embrace the tool and its glitches, its defects, and found expression in precisely that.”

Fellowship’s goal is to capitalize on the opportunity that Cartagena sees in building a photography collection like no other on the planet. Himself a photographer with decades of experience in creating, curating, and collecting photographs, Cartagena believes Fellowship can be the organization to consolidate 180 years’ worth of photographic history into one cohesive and unique collection, unbound from the limits of the museums that store 1/1 physical photographic work in vaults and halls that few people will ever see. 

In November 2022, Fellowship collaborated with none other than Dmitri Cherniak for the release of the artist’s 100-piece generative photography collection, Light Years. Experimental releases like these align perfectly with Cartegena and Fellowship Co-founder Studio137’s vision, making the platform’s partnership with Rainisto a perfect fit.

“We see Fellowship as this core collection that is the base to build into the future what photography can be,” Cartagena elaborated. “We love the fringes of photography and the things that have made photography, photography. Because the NFT space moves so quickly, we can experiment in a very sophisticated and fast way. We’re a gallery, a collection, we commission work, and we want to be a charity to support marginalized artists. We want to be a lot of things because the NFT space permits that flexibility.”

Cartagena and Studio137 began collecting Rainisto’s work in early 2022. Connecting with the artist shortly after, they agreed to work together on Rainisto’s follow-up to Life In West America — Reworld. 

REWORLD and Post-Photographic Perspectives

The 500 pieces in Reworld thematically recall those from Life In West America but are distinct in their particular focus. Whereas the images in the latter centered on individuals and small-scale personal relationships, Reworld deals with grander concepts, forming a commentary on societal issues at a much higher level. 

Starting April 5, the pieces in the collection will be sold in a Dutch Auction format, starting at 10 ETH and dropping 0.4 ETH every minute until the price reaches a low-end cap of 4 ETH. After this, the price will drop 0.1 ETH each minute until it hits a floor of 2 ETH, after which the sale becomes public if any inventory remains. The first 35 minutes of the auction is reserved for Patron Pass holders, Fellowship’s annual membership token released in March. 

On the show’s second day, Rainisto switches from featured artist to curator, presenting 10 AI artists who were chosen for their ability to show the range of what AI tools can do in the post-photographic arena. While some artists in the show’s second stage are known for their own approaches to AI photography, others take the very concept “to the edge,” as Cartagena puts it. Selected artists for this part of the show include AI. S.A.M, Andres Hernandez, Antti Karppinen, Charlie Engman, Jess Mac, Julie Wieland, Katie Morris, Simon Raion, Ben Millar Cole, and Pierre Zandrowicz. 

The group show will also include at least one IRL activation, including an exhibition in May at an as-of-yet undisclosed gallery in Los Angeles and the Paris photo fair. 

An artist like no other

At 43 years old, Rainisto is only recently beginning to explore artistic expression on his own terms. Having worked as a designer since the late 1990s, Rainisto’s artistic journey reached a tipping point two years ago when he started seriously considering how VR and AI technologies would begin to drastically change how the world works.

“I thought I can be either one of the first people on this bandwagon or I can be one of the last,” Rainisto explained of his interest in AI tools while speaking to nft now. “I’m always excited about new technologies, trying to figure out how you can then use [them] to create something meaningful. That was my inspiration.”

The 500 pieces in REWORLD, a collection he has been working on for the better part of the past year, are the final cuts from over 50,000 outputs Rainisto created while building the series. They feature Rainisto’s singular style: a surreal photo-realistic familiarity steeped in Americana inspired in part by Robert Frank’s photographic book, The Americans. Rainisto also attributes the project’s origins to the idea of a road trip as seen through an algorithm’s eyes, technology that, he says, represents a shared understanding of the world.

“They see in five billion pictures. In some ways, [the collection] is a road trip into a shared kind of humanity,” Rainisto offered as an insight into his project and vision.

Rainisto is fond of his approach to AI photography, something rapidly becoming known as post-photography, precisely because of the way it embraces the imperfections and outright otherworldliness of the way algorithms “envision” the prompts given to them. 

“We’ve all seen that AI can do photos where you can’t really tell if it’s a real photo or not,” Rainisto continued. “I think that’s fine, but my kind of reference point is the camera. When Stable Diffusion came out, my idea was to make the outputs resemble a photo. I had successful renders and things that had ‘failed.’ But I found the failures to be much more interesting. Rather than try to hide the ‘AI-ness,’ what would happen if I tried to explore it?”

Reworld will be Rainisto’s last large-scale collection in 2023. Apart from the time and energy it took him to create and release two series in the span of a few months, he is hesitant to saturate the market with his work. Interested collectors can look forward to him continuing to experiment with AI art and working on 1/1s, animations, and potentially even small-scale limited editions. For now, Rainisto advises fans of his work to pick up a Patron Pass on the secondary market so they don’t miss out on the rest of Fellowship’s 2023 shows.