Samsung has a new trailer up on their channel, and it showcases some of the abilities of their new 200MP sensor. This is presumably the HP2X sensor that’s coming on the Galaxy S24 Ultra. No smartphone exists on the market with a 200MP sensor and the Snapdragon 8 Gen 3 chipset. So, we can assume this is a very early preview of the capabilities of the 8 Gen 3 ISP and the upcoming Galaxy S24 Ultra.
The 200MP sensor can crop in and use multiple parts of it for multiple zoom ranges. You can get 50MP 2X zoom photos and only use the middle of the sensor. You can also use 1/4th of the sensor and get 25MP 4X zoom photos from the HP2X. In-sensor zoom is much more practical.
The 8 Gen 3 and Exynos 2400 both have tremendous improvements in AI performance. The 8 Gen 3 also has a massively improved ISP. The ISP on the 8 Gen 2 could already perform semantic segmentation, which means it can identify objects like skies or trees and enhance them with AI. The 8 Gen 3 takes this a step further and includes real-time semantic segmentation for both photos and videos, and it can do it in up to 12 layers.
There’s a feature called Zoom Anyplace. It can automatically track subjects and zoom into them while also capturing the entire zoomed-out footage. This means that one of the streams the sensor is capturing is a zoomed-in scene, and the other is a full-frame view. It’s doing this in real time. Two streams in up to 4K resolution, one of the area that you’re cropping and one of the entire view.
You can now easily change the framing of your scene even after your shooting is done. This fixes a very common problem with photo or video angles. You can use these features at both 2X zoom and 4X zoom, and we finally have a good use case for a 200MP sensor.
This feature also tracks subjects, and the internal mechanisms always attempt to keep them in focus. If the main subject moves a lot in any direction, it can still track it and lock focus. Even if it gets closer to the lens, the sensor won’t have problems with focusing, even at a high 4K resolution.
The new image processing system does many things at once. Important aspects of a photo, like color, tone, noise reduction, image sharpening, application of HDR, color reconstruction, white balance, and lens shade correction, all happen in real time.
The older method did it one color and one layer at a time. This increases the processing time and also partly contributes to shutter lag. The new technology uses AI to assist it, and they call it AI Remosaic. It’s end-to-end encrypted for better security. This can give up to two times faster capture time and also less processing time. Noise reduction might improve as well.
An explanation with pictorial representations is on the official Samsung community blog. Samsung says that these are all for illustrative purposes. We’ll find out whether the S24 Ultra will have this hybrid AI zoom feature at Galaxy Unpacked in January. Many companies besides Samsung might use this technology on their telephoto lenses for even better-zooming capabilities.
