How Beamr scales subjective measurement using crowd sourcing.
Beamr Vice President of Technology and Algorithms, Tamar Shoham, discusses a tool that Beamr built, which enables ITU BT.500 style subjective quality testing to be performed at scale using crowdsourcing. Building on top of Beamr View, and leveraging Amazon Mechanical Turk, VISTA enables subjective evaluations to be performed cost-effectively and reliably. Tamar shares ground-breaking academic research from the Technion-Israel Institute of Technology that demonstrates mathematically why objective metrics like SSIM, PSNR, and even VMAF, are not sufficient for video encoder comparison and evaluation.