pyiqa.archs.clipiqa_arch

CLIP-IQA metric, proposed by

Exploring CLIP for Assessing the Look and Feel of Images. Jianyi Wang, Kelvin C.K. Chan, Chen Change Loy. AAAI 2023.

Ref url: https://github.com/IceClear/CLIP-IQA Re-implemented by: Chaofeng Chen (https://github.com/chaofengc) with the following modification:

  • We assemble multiple prompts to improve the results of clipiqa model.

Module Contents

pyiqa.archs.clipiqa_arch.default_model_urls[source]
class pyiqa.archs.clipiqa_arch.PromptLearner(clip_model, n_ctx=16)[source]

Bases: torch.nn.Module

PromptLearner class for learning prompts for CLIP-IQA.

Disclaimer:

This implementation follows exactly the official codes in: https://github.com/IceClear/CLIP-IQA. We have no idea why some tricks are implemented like this, which include:

  1. Using n_ctx prefix characters “X”

  2. Appending extra “.” at the end

  3. Insert the original text embedding at the middle

get_prompts_with_middle_class()[source]

Get prompts with the original text embedding inserted in the middle.

Returns:

The generated prompts.

Return type:

torch.Tensor

forward(clip_model)[source]

Forward pass for the PromptLearner.

Parameters:

clip_model (nn.Module) – The CLIP model.

Returns:

The output features.

Return type:

torch.Tensor

class pyiqa.archs.clipiqa_arch.CLIPIQA(model_type='clipiqa', backbone='RN50', pretrained=True, pos_embedding=False)[source]

Bases: torch.nn.Module

CLIPIQA metric class.

Parameters:

  • model_type (str) – The type of the model. Default is ‘clipiqa’.

  • backbone (str) – The backbone model. Default is ‘RN50’.

  • pretrained (bool) – Whether to load pretrained weights. Default is True.

  • pos_embedding (bool) – Whether to use positional embedding. Default is False.

forward(x)[source]

Forward pass for the CLIPIQA model.

Parameters:

x (torch.Tensor) – Input tensor with shape (N, C, H, W).

Returns:

The output probabilities.

Return type:

torch.Tensor