PEANUT996

TypeError:__init__() missing 2 required positional arguments:'op' and 'message'

Solution

  • Downgrade to tfds v3.1.0
  • Upgrade to tfds v3.2.1

If you on tfds v3.2.0 now, you can close the GCS:

# Line 40 in datasets/tensorflow_datasets/core/utils/gcs_utils.py  
_is_gcs_disabled = False

v3.2.1 has fixed this bug.

More information can be found here.

About

今天跟着TensorFlow官网的tutorials跑代码的时候遇到了这个问题,吐槽一下,官方的中文文档也太久没更新了,下面这个SplitAPI都已经被废弃了:

# 将训练集按照 6:4 的比例进行切割,从而最终我们将得到 15,000
# 个训练样本, 10,000 个验证样本以及 25,000 个测试样本

# deprecated
train_validation_split = tfds.Split.TRAIN.subsplit([6, 4])

(train_data, validation_data), test_data = tfds.load(
    name="imdb_reviews",
    split=(train_validation_split, tfds.Split.TEST),
    as_supervised=True)

在英文版的tutorials的代码修正如下:

# Split the training set into 60% and 40%, so we'll end up with 15,000 examples
# for training, 10,000 examples for validation and 25,000 examples for testing.
train_data, validation_data, test_data = tfds.load(
    name="imdb_reviews",
    split=('train[:60%]', 'train[60%:]', 'test'),
    as_supervised=True)

在下载IMDB的Datasets时遇到了这个问题:

2020-07-15 13:13:39.319324: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_101.dll
Traceback (most recent call last):
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\utils\py_utils.py", line 399, in try_reraise
    yield
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\registered.py", line 244,
in builder
    return builder_cls(name)(**builder_kwargs)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\api_utils.py", line 69, in disallow_positional_args_dec
    return fn(*args, **kwargs)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line
206, in __init__
    self.info.initialize_from_bucket()
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\dataset_info.py", line 423, in initialize_from_bucket
    data_files = gcs_utils.gcs_dataset_info_files(self.full_name)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\utils\gcs_utils.py", line
70, in gcs_dataset_info_files
    return gcs_listdir(posixpath.join(GCS_DATASET_INFO_DIR, dataset_dir))
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\utils\gcs_utils.py", line
63, in gcs_listdir
    if _is_gcs_disabled or not tf.io.gfile.exists(root_dir):
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 267, in
file_exists_v2
    _pywrap_file_io.FileExists(compat.as_bytes(path))
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'gs' not implemented (file: 'gs://tfds-data/dataset_info/imdb_reviews/plain_text/1.0.0')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".\text_classification_with_hub.py", line 18, in <module>
    as_supervised=True)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\api_utils.py", line 69, in disallow_positional_args_dec
    return fn(*args, **kwargs)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\registered.py", line 368,
in load
    dbuilder = builder(name, data_dir=data_dir, **builder_kwargs)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\registered.py", line 244,
in builder
    return builder_cls(name)(**builder_kwargs)
  File "C:\Program Files\Python37\lib\contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\utils\py_utils.py", line 401, in try_reraise
    reraise(*args, **kwargs)
  File "D:\Workspace\Python\venv\lib\site-packages\tensorflow_datasets\core\utils\py_utils.py", line 392, in reraise
    six.reraise(exc_type, exc_type(msg), exc_traceback)
TypeError: __init__() missing 2 required positional arguments: 'op' and 'message'

逛了一圈发现这个Bug也就前两天的事情,原因也很简单, 因为引入了tf.io.gfile导致的报错(#2190)