pyiqa.archs.q_align.cmp_modelling_mplug_owl2

Module Contents

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.IGNORE_INDEX = -100

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.IMAGE_TOKEN_INDEX = -200

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.DEFAULT_IMAGE_TOKEN = '<|image|>'

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.tokenizer_image_token(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None)

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.expand2square(pil_img, background_color)

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.norm_cdf(x)

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.optimize_score_map_pytorch_cuda(c, seed=0, original_seed=20020, num_iterations=100)

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.softmax(logits)

pyiqa.archs.q_align.cmp_modelling_mplug_owl2.update_matrix(anchor_matrix, scores, indices)

class pyiqa.archs.q_align.cmp_modelling_mplug_owl2.MPLUGOwl2MetaModel(config)

get_vision_tower()

get_visual_abstractor()

class pyiqa.archs.q_align.cmp_modelling_mplug_owl2.MPLUGOwl2MetaForCausalLM

Bases: abc.ABC

Helper class that provides a standard way to create an ABC using inheritance.

abstractmethod get_model()

encode_images(images)

prepare_inputs_labels_for_multimodal(input_ids, attention_mask, past_key_values, labels, images)

class pyiqa.archs.q_align.cmp_modelling_mplug_owl2.MPLUGOwl2LlamaModel(config: pyiqa.archs.q_align.configuration_mplug_owl2.MPLUGOwl2Config)

Bases: MPLUGOwl2MetaModel, pyiqa.archs.q_align.modeling_llama2.LlamaModel

Transformer decoder consisting of config.num_hidden_layers layers. Each layer is a [LlamaDecoderLayer]

Parameters:

config – LlamaConfig

class pyiqa.archs.q_align.cmp_modelling_mplug_owl2.MPLUGOwl2LlamaForCausalLM(config)

Bases: pyiqa.archs.q_align.modeling_llama2.LlamaForCausalLM, MPLUGOwl2MetaForCausalLM

Helper class that provides a standard way to create an ABC using inheritance.

get_model()

score(image)

forward(input_ids: torch.LongTensor = None, attention_mask: torch.Tensor | None = None, past_key_values: List[torch.FloatTensor] | None = None, inputs_embeds: torch.FloatTensor | None = None, labels: torch.LongTensor | None = None, use_cache: bool | None = None, output_attentions: bool | None = None, output_hidden_states: bool | None = None, images: torch.FloatTensor | None = None, return_dict: bool | None = None) → Tuple | transformers.modeling_outputs.CausalLMOutputWithPast

Parameters:

labels (torch.LongTensor of shape (batch_size, sequence_length), optional) – Labels for computing the masked language modeling loss. Indices should either be in [0, …, config.vocab_size] or -100 (see input_ids docstring). Tokens with indices set to -100 are ignored (masked), the loss is only computed for the tokens with labels in [0, …, config.vocab_size].

Returns:

Example:

```python >>> from transformers import AutoTokenizer, LlamaForCausalLM

>>> model = LlamaForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
>>> tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZERS)

>>> prompt = "Hey, are you conscious? Can you answer me?"
>>> inputs = tokenizer(prompt, return_tensors="pt")

>>> # Generate
>>> generate_ids = model.generate(inputs.input_ids, max_length=30)
>>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
"Hey, are you conscious? Can you answer me?\nI'm not sure if I'm conscious, but I can answer you."
```

prepare_inputs_for_generation(input_ids, past_key_values=None, attention_mask=None, inputs_embeds=None, **kwargs)