Review: Moscow State University Video Quality Measurement Tool 14.1

995美元的第14版 Moscow State University Video Quality Measurement Tool (VQMT) 有两个重要的新特性和多个增强，提高了实用性和可用性. If you’re an encoding professional who has somehow resisted this product before now, 这个新版本会把你逼到崩溃的边缘.

About VQMT

一些背景知识:VQMT是一个工具，它允许您将基于磁盘的编码文件与源文件进行比较，并计算超过20个指标, including VMAF, PSNR, SSIM, MS SSIM, and an alphabet soup’s worth of other metrics shown in Figure 1. You can compute most metrics for different color values like R, G, B, Y, U, V, or RGB and YUV, so the actual number of individual metrics that you can compute is easily in the hundreds, if not more.

VQMT metrics

Figure 1. VQMT 14中可用的度量.1.

每次在GUI中比较文件时(如果您喜欢的话，VQMT还有一个非常健壮的命令行版本), 程序创建如图所示的结果图 Figure 2. This plots the frame score for each encoded file over the duration of the file. 图形表示可以让您发现短暂的质量问题，如文件前面用红色和蓝色显示的问题，并测量质量可变性. As an example, 请注意，虽然紫色编码器在文件开始时不会遇到与蓝色和红色相同的低点, overall, 其质量表现出显著的可变性, 哪些会降低QoE.

VQMT结果图文件比较

Figure 2. Comparing five files in VQMT’s ultra-useful Result plot

您可以使用按钮上的滑块或单击图中的任何位置来导航结果图. 在文件中的任何位置，您都可以按图2右下角的Show frame按钮. 这将加载预览窗口, 这使多种查看模式能够相互比较编码文件和源文件.

为了说明这一点， Figure 3 is a screen grab of the Preview window that shows encoder 1, 如图2中的红线所示, on the left, and encoder 3, 右边的橙/黄线. As you can see in Figure 2, encoder 1 was a poor performer; encoder 3 among the best. 如果单击并查看图3, 你会在左边的草地和球员制服中看到更多的人为因素，这证实了你可以在两个窗口的右上角看到的低指标得分.

查看低帧

Figure 3. 查看低帧，以确定分数是否代表观众会注意到的实际质的差异.

重要的是，这些“低帧”视图是可访问的，因为有时度量评分差异并不代表视频文件中实际可见的差异. Remember that metrics are predictors of how human viewers will rate the video files, 所以你总是想要验证低帧区域，以确保它们代表明显的差异. With the Result plot and Show frame button, VQMT makes it exceptionally simple to do so. 正如你在左下角看到的 Figure 6, 您还可以导出每个分析的坏帧, an automated way to verify and save the low-frame scores.

回到预览窗口, 您可以查看原始编码质量的帧(图3)或每像素度量值的映射.

I’ve configured the map in black and white, but you can choose many other color values. 在视图中显示 Figure 4’s 地图的每像素度量值, the frame with more white space is the higher quality frame. 在图4的左下角, you see the frame views you can choose for the left side of the screen; the drop-down list on the right is for configuring the right-side view.

逐像素度量值

Figure 4. This shows the 逐像素度量值 in black and white; lighter is better.

After choosing the encoded files and the display (frame or pixel map), 您可以并排放置输出, 如图3和图4所示, or top and bottom, 或者在带有滑块的视图中，您可以向左或向右拖动滑块以显示一个帧并隐藏另一个帧. 您还可以完整地查看每一帧，然后通过图4左下角的下拉列表或所示的Ctrl+控件切换下一帧. 总的来说，这是探索由度量值报告的质量缺陷的绝佳机制.

Each time you analyze a file or files, VQMT produces a CSV file with the data shown in Figure 5, which I’ve imported into Google Sheets to better illustrate the contents. This is the VMAF analysis; for each file, you see the mean, harmonic mean, min/max values, 最小/最大帧数, 还有标准差和方差, 都是质量可变性的度量. Below those summary metrics are the individual frame scores for each file.

VQMT报告的数值结果

Figure 5. VQMT报道的数值结果

Overall, VMQT lets you easily compute over 20 highly configurable metrics, 可视化结果, 查看实际帧, 并将数值结果导入电子表格. It’s a program that I use on multiple computers in my office nearly every day.

以下是对版本14的主要补充.

比较两个以上的文件在GUI

到目前为止，我对VQMT最常见的失望可能是无法同时比较两个以上的文件. This is gone; now you can compare an unlimited number of files, subject to memory limitations. 我测试了10个，没有任何问题.

You see this in Figure 6, where I compare five HEVC encoded files to the same source, which is the analysis that produced all the results presented above. 添加其他文件, you click the small blue Plus button on the extreme lower right; to delete any processed slots click the Minus button to the right of the slot.

VQMT无限文件处理

Figure 6. VQMT现在可以处理看似无限的文件.

一旦超过5或6个文件，图形就会变得过于繁忙，难以解释，CSV也会变得过于拥挤，无法进行数据外推. Still, it’s remarkable how many times you want to compare 2–5 files and 可视化结果, and the ability to do so is a great enhancment.

Python Interface

版本14中增加的另一个主要特性是.1) is a Python interface for simplified VQMT scripting that you can read about here. 正如那一页所描述的, the package is a wrapper over VQMT that will “load the library as an instance of msu_vqmt.SharedInterface类并初始化VQMT. Using this object, you can run any number of measurements without reinitialization.”

这里提供了多个示例, 包括安装, activating, 运行度量(如图所示 Figure 7), +故障排除, 运行异步, 交互式抓取值, 并使用numpy矩阵作为输入帧的来源. In general, 与命令行脚本相比，Python是运行VQMT和处理结果的更灵活、更强大的工具, Python接口将使程序对那些具有Python编程能力或访问相同功能的人更有用.

Figure 7. 使用Python运行MSU VQMT

其他新功能:测量历史

眼尖的VQMT用户可能已经注意到，在左下方的开始图标右侧有一个新的多行蓝色图标 Figure 6. 这将打开Measurement History窗口 Figure 8, which conveniently stores previous measurements made by the program. Not only can you immediately grab the scoring data from the table; you can also recall the measurement and re-display it in the Result plot. So long as all the files are still in the same location, 你可以充分利用结果图, including the ability to view compressed frames as detailed above.

Figure 8. Recalling a previous measurement from the Measurement History window to the Result window

Depending upon the duration of your files and the selected metric, 计算指标可能需要几个小时, and processing time will only increase as you add additional files. Previously, 关闭Result图后, 分析没有了, 如果你想重新审视视觉结果, 你必须重新运行它. The new Measurement History window will prove incredibly useful in practice.

另外，在Measurement History窗口之外还可以手动保存结果文件, 稍后可以检索并重新显示. 这是对Measurement History的一个很好的补充，它允许您将结果文件存储在一个单独的文件夹中，以便与其他项目相关的文件一起保存.

其他新功能

Other new features include super-fast SSIM and MS SSIM modes that, in my tests, cut the processing time of a 73-minute video file from 9:22 (min:sec) to 3:20. I also computed the scores using the precise model and show all results in Table 1. 虽然分数各不相同, the range is tiny, and the respective quality conclusions would be the same with all results. 对于那些依赖SSIM和MS SSIM的人来说，快速和超高速模式看起来都是有用的补充.

SSIM和MS SSIM的超高速模式

Table 1. 用于SSIM和MS SSIM的新超高速模式将处理时间减少了191%，对度量值的影响最小.

如果你处理的文件特别长, 你会发现密歇根州立大学加速了所有的缩放, sliding, 以及结果图中的其他导航运动. I evaluated this enhancement after measuring VMAF on a 105-minute movie. With version 13, 在结果图中移动是不稳定的, with slight delays as I navigated until the GUI caught up. With version 14, 在结果图中导航是即时的, no different than working with a 10-second test file.

功能更强大的免费版本

Finally, version 14 contains a much more useful free version than previous versions, which were all limited to files below 720p in resolution. 新的免费版本现在可以让你分析高达8K分辨率的文件，但缺乏显示的功能 Table 2. Note that Table 2 is a truncated version of the table 显示在MSU网站上 that we cut for spacing reasons; the free version has many other features not shown in our table. All MSU asks is that you use the free version for personal work, not for business. All told, the free version is now a much more compelling try-before-you-buy option.

4个VQMT版本的功能和定价