pyiqa.archs.q_align.cmp_modelling_mplug_owl2
============================================

.. py:module:: pyiqa.archs.q_align.cmp_modelling_mplug_owl2








Module Contents
---------------

.. py:data:: IGNORE_INDEX
   :value: -100


.. py:data:: IMAGE_TOKEN_INDEX
   :value: -200


.. py:data:: DEFAULT_IMAGE_TOKEN
   :value: '<|image|>'


.. py:function:: tokenizer_image_token(prompt, tokenizer, image_token_index=IMAGE_TOKEN_INDEX, return_tensors=None)

.. py:function:: expand2square(pil_img, background_color)

.. py:function:: norm_cdf(x)

.. py:function:: optimize_score_map_pytorch_cuda(c, seed=0, original_seed=20020, num_iterations=100)

.. py:function:: softmax(logits)

.. py:function:: update_matrix(anchor_matrix, scores, indices)

.. py:class:: MPLUGOwl2MetaModel(config)

   .. py:method:: get_vision_tower()


   .. py:method:: get_visual_abstractor()


.. py:class:: MPLUGOwl2MetaForCausalLM

   Bases: :py:obj:`abc.ABC`


   Helper class that provides a standard way to create an ABC using
   inheritance.


   .. py:method:: get_model()
      :abstractmethod:



   .. py:method:: encode_images(images)


   .. py:method:: prepare_inputs_labels_for_multimodal(input_ids, attention_mask, past_key_values, labels, images)


.. py:class:: MPLUGOwl2LlamaModel(config: pyiqa.archs.q_align.configuration_mplug_owl2.MPLUGOwl2Config)

   Bases: :py:obj:`MPLUGOwl2MetaModel`, :py:obj:`pyiqa.archs.q_align.modeling_llama2.LlamaModel`


   Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`LlamaDecoderLayer`]

   :param config: LlamaConfig


.. py:class:: MPLUGOwl2LlamaForCausalLM(config)

   Bases: :py:obj:`pyiqa.archs.q_align.modeling_llama2.LlamaForCausalLM`, :py:obj:`MPLUGOwl2MetaForCausalLM`


   Helper class that provides a standard way to create an ABC using
   inheritance.


   .. py:method:: get_model()


   .. py:method:: score(image)


   .. py:method:: forward(input_ids: torch.LongTensor = None, attention_mask: Optional[torch.Tensor] = None, past_key_values: Optional[List[torch.FloatTensor]] = None, inputs_embeds: Optional[torch.FloatTensor] = None, labels: Optional[torch.LongTensor] = None, use_cache: Optional[bool] = None, output_attentions: Optional[bool] = None, output_hidden_states: Optional[bool] = None, images: Optional[torch.FloatTensor] = None, return_dict: Optional[bool] = None) -> Union[Tuple, transformers.modeling_outputs.CausalLMOutputWithPast]

      :param labels: Labels for computing the masked language modeling loss. Indices should either be in `[0, ...,
                     config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored
                     (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`.
      :type labels: `torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*

      Returns:

      Example:

      ```python
      >>> from transformers import AutoTokenizer, LlamaForCausalLM

      >>> model = LlamaForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
      >>> tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZERS)

      >>> prompt = "Hey, are you conscious? Can you answer me?"
      >>> inputs = tokenizer(prompt, return_tensors="pt")

      >>> # Generate
      >>> generate_ids = model.generate(inputs.input_ids, max_length=30)
      >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
      "Hey, are you conscious? Can you answer me?\nI'm not sure if I'm conscious, but I can answer you."
      ```



   .. py:method:: prepare_inputs_for_generation(input_ids, past_key_values=None, attention_mask=None, inputs_embeds=None, **kwargs)