Ошибка создания экземпляра Google Colab Bert с использованием Tensorflow

Я пытаюсь построить модель Берта, используя Tensorflow в Colab. Этот код отлично работал несколько недель назад. Теперь, если я попытаюсь создать экземпляр модели, я получу следующую ошибку:

Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFBertModel: ['cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.weight']
- This IS expected if you are initializing TFBertModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFBertModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFBertModel were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFBertModel for predictions without further training.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-14-b0e769ef7890> in <cell line: 7>()
      5 SC_mask_layer = Input(shape=(max_seq_length,), dtype=tf.int32, name = "attention_mask")
      6 SC_bert_model = TFBertModel.from_pretrained("bert-base-uncased")
----> 7 SC_pooler_output = SC_bert_model(SC_input_layer, attention_mask=SC_mask_layer)[1]  # Estrai il secondo output, che è il pooler_output
      8 
      9 # Aggiungi un layer di Dropout

36 frames
/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/type_spec.py in type_spec_from_value(value)
   1002         3, "Failed to convert %r to tensor: %s" % (type(value).__name__, e))
   1003 
-> 1004   raise TypeError(f"Could not build a TypeSpec for {value} of "
   1005                   f"unsupported type {type(value)}.")
   1006 

TypeError: Exception encountered when calling layer 'embeddings' (type TFBertEmbeddings).

Could not build a TypeSpec for name: "tf.debugging.assert_less_5/assert_less/Assert/Assert"
op: "Assert"
input: "tf.debugging.assert_less_5/assert_less/All"
input: "tf.debugging.assert_less_5/assert_less/Assert/Assert/data_0"
input: "tf.debugging.assert_less_5/assert_less/Assert/Assert/data_1"
input: "tf.debugging.assert_less_5/assert_less/Assert/Assert/data_2"
input: "Placeholder"
input: "tf.debugging.assert_less_5/assert_less/Assert/Assert/data_4"
input: "tf.debugging.assert_less_5/assert_less/y"
attr {
  key: "summarize"
  value {
    i: 3
  }
}
attr {
  key: "T"
  value {
    list {
      type: DT_STRING
      type: DT_STRING
      type: DT_STRING
      type: DT_INT32
      type: DT_STRING
      type: DT_INT32
    }
  }
}
 of unsupported type <class 'tensorflow.python.framework.ops.Operation'>.

Call arguments received by layer 'embeddings' (type TFBertEmbeddings):
  • input_ids=<KerasTensor: shape=(None, 128) dtype=int32 (created by layer 'input_ids')>
  • position_ids=None
  • token_type_ids=<KerasTensor: shape=(None, 128) dtype=int32 (created by layer 'tf.fill_5')>
  • inputs_embeds=None
  • past_key_values_length=0
  • training=False

Код модели:

SC_input_layer = Input(shape=(max_seq_length,), dtype=tf.int32, name = "input_ids")
SC_mask_layer = Input(shape=(max_seq_length,), dtype=tf.int32, name = "attention_mask")
SC_bert_model = TFBertModel.from_pretrained("bert-base-uncased")
SC_pooler_output = SC_bert_model(SC_input_layer, attention_mask=SC_mask_layer)[1]  

# Aggiungi un layer di Dropout
SC_dropout_layer = Dropout(dropout_rate)(SC_pooler_output)
SC_output_layer = Dense(6, activation='sigmoid')(SC_dropout_layer)
SC_model = Model(inputs=[SC_input_layer, SC_mask_layer], outputs=SC_output_layer)

Я обнаружил, что установка tensorflow 2.10.0 работает, но при использовании Google Colab у меня возникают проблемы с версией CUDA, а при использовании tensorflow 2.10 он не распознает графический процессор. Этот код работал несколько недель назад, у кого-нибудь есть решение?

Обновлено: та же ошибка появляется в Kaggle.