add Logic Inference Dataset
diff --git a/data/datasets/__init__.py b/data/datasets/__init__.py
index fb573c7d6c..f337023897 100644
--- a/data/datasets/__init__.py
+++ b/data/datasets/__init__.py
@@ -20,6 +20,7 @@
     "tell_a_joke": "mikegarts/oa_tell_a_joke_20000",
     "oa_wiki_qa_bart_10000row": "michaelthwan/oa_wiki_qa_bart_10000row",
     "oa_leet10k": "ehartford/oa_leet10k",
+    "LogicInference_OA": "KK04/LogicInference_OA",
 }
 
 SAFETY_DATASETS = {
diff --git a/data/datasets/logicreference_OA/README.md b/data/datasets/logicreference_OA/README.md
new file mode 100644
index 0000000000..ee639f09c6
--- /dev/null
+++ b/data/datasets/logicreference_OA/README.md
@@ -0,0 +1,97 @@
+# LogicInference Dataset
+
+This repository contains the Python code used to generate the `LogicInference`
+dataset. `LogicInference` is a dataset designed to evaluate the ability of
+models to perform logical inference. The dataset focuses on inference using
+propositional logic and a small subset of first-order logic, represented both in
+semi-formal logical notation, and in natural language. `LogicInference` has two
+main long-term goals: (1) to evaluate the ability of models to perform logical
+inference, and the degree to which inference chains are real or hallucinated,
+and (2) to assess whether learning logical inference abilities in the abstract
+(e.g., getting better in this dataset) would then transfer to other real-world
+tasks.
+
+**Note: to run this code you also need the other files from the original
+LogicInference project
+[here](https://github.com/google-research/google-research/tree/master/logic_inference_dataset).
+The generate_dataset script in this directory is a drop-in replacement for the
+original generate_dataset script which outputs data in Open Assistant instruct
+format.**
+
+For a detailed description of the dataset, please check the following paper:
+https://openreview.net/pdf?id=HAGeIS_Lcg9 (arXiv preprint:
+https://arxiv.org/abs/2203.15099 )
+
+Please cite as:
+
+```
+@inproceedings{ontanon2022logicinference,
+  url = {https://openreview.net/pdf?id=HAGeIS_Lcg9},
+  author = {Onta\~{n}\'{o}n, Santiago and Ainslie, Joshua and Cvicek, Vaclav and Fisher, Zachary},
+  title = {{LogicInference}: A New Dataset for Teaching Logical Inference to seq2seq Models},
+  booktitle={Proceedings of ICLR 2022 workshop on Objects, Structure and Causality},
+  year={2022}
+}
+```
+
+This is an re-produce of the dataset from LogicInference Dataset in paper:
+https://openreview.net/pdf?id=HAGeIS_Lcg9.
+
+The github page of LogicInference Dataset:
+https://github.com/google-research/google-research/tree/master/logic_inference_dataset.
+
+This dataset is aimed to offer more dataset for Open Assistant project,
+depending on their demands, there three columns: INSTRUCTION, RESPONSE, SOURCE.
+
+The results in this dataset is a little different from which was introduced in
+the original paper:
+
+1.For all three splits (IID/OOD/length), only IID is used. In the original
+paper, it seems that model can reach better performance with data generated by
+this split method.
+
+2.In the original paper, there are two form of responses:
+LOGICINFERENCE<sub>b</sub> (with the answer at the beginning) and
+LOGICINFERENCE<sub>e</sub> (with the answer at the end). This dataset uses
+LOGICINFERENCE<sub>e</sub>, that means: for all questions, the model will first
+do logic inference, and give the final answer at the end.
+
+3.The original paper, some parameters in generate_dataset.py are:
+
+N_INFERENCE_PROBLEMS = 5000
+
+N_VARIATIONS = 25
+
+N_EXAMPLES = 200000
+
+TRAIN_RATIO = 0.9
+
+LENGTH_SPLIT_THRESHOLD = 4
+
+RANDOM_SEED = 0
+
+I choose some new parameters:
+
+N_INFERENCE_PROBLEMS = 10000
+
+N_VARIATIONS = 25
+
+N_EXAMPLES = 55000
+
+TRAIN_RATIO = 1
+
+LENGTH_SPLIT_THRESHOLD = 4
+
+RANDOM_SEED = 1111
+
+The original script generated 4814 different inference problems and extended all
+those inference problems to around 200,000 Q-A pairs. My settings generated 5491
+different inference problems and extended them to around 54,607
+Instruction-Response pairs. I think for Open Assistant projects, maybe the
+number of different inference problems is more important, and generated many
+similar Instruction-Response pairs will only add training time and doesn't make
+much sense.
+
+4.I only keep the generate_dataset.py file in this directory, because the coding
+format of the original project does not fit OA project which need flake8 format.
+I only change the coding format of generate_dateset.py.
diff --git a/data/datasets/logicreference_OA/generate_dataset.py b/data/datasets/logicreference_OA/generate_dataset.py
new file mode 100644
index 0000000000..59ff45ff39
--- /dev/null
+++ b/data/datasets/logicreference_OA/generate_dataset.py
@@ -0,0 +1,129 @@
+# Copyright 2023 The Google Research Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Driver file that generates IID/OOD/length splits.
+"""
+
+
+import os
+import random
+
+import rules
+import splits
+import tensorflow as tf
+from absl import app
+
+# Generation parameters:
+# TARGET_FOLDER = "/path/to/generate/dataset/"
+TARGET_FOLDER = "./e_txt/"
+ANSWER_AT_THE_END = True
+LENGTH_DISTRIBUTION = [0.425, 0.3, 0.2, 0.05, 0.025]
+N_INFERENCE_PROBLEMS = 10000
+N_VARIATIONS = 25
+N_EXAMPLES = 55000
+TRAIN_RATIO = 1
+LENGTH_SPLIT_THRESHOLD = 4
+RANDOM_SEED = 1111
+
+
+def create_string_feature(values):
+    """Creates TensorFlow string features.
+
+    Args:
+      values: A sequence of unicode strings.
+
+    Returns:
+      An entry of int tf.train.Feature.
+    """
+    # Converts to `str` (in Python 2) and `bytes` (in Python 3) as
+    # `tf.train.Feature` only takes bytes.
+    values = [value.encode("utf-8") for value in values]
+
+    feature = tf.train.Feature(bytes_list=tf.train.BytesList(value=values))
+    return feature
+
+
+def generate_t5_split(path, file_name, examples):
+    print(f"Generating split of size {len(examples)} at {path}")
+    os.makedirs(path, exist_ok=True)
+    with open(os.path.join(path, file_name), "w") as f:
+        for example in examples:
+            f.write(f"INSTRUCTION: {example.inputs}\n")
+            f.write(f"RESPONSE: {example.targets}\n")
+            f.write("SOURCE: LogicInference Dataset e\n\n")
+
+
+def main(_):
+    rules.precompute_rules()
+
+    suffix = ""
+    if ANSWER_AT_THE_END:
+        suffix = "_e"
+    folder_iid_name = "logic_inference_iid" + suffix
+
+    # Generate each of the splits:
+    print("IID:")
+    random.seed(RANDOM_SEED)
+    (train_examples, test_examples) = splits.generate_training_and_test_sets_iid(
+        N_INFERENCE_PROBLEMS,
+        N_VARIATIONS,
+        N_EXAMPLES,
+        TRAIN_RATIO,
+        length_distribution=LENGTH_DISTRIBUTION,
+        answer_at_the_end=ANSWER_AT_THE_END,
+    )
+    generate_t5_split(
+        os.path.join(TARGET_FOLDER, folder_iid_name),
+        f"{folder_iid_name}-train_tf_examples-00000-of-00001",
+        train_examples,
+    )
+    generate_t5_split(
+        os.path.join(TARGET_FOLDER, folder_iid_name),
+        f"{folder_iid_name}-test_tf_examples-00000-of-00001",
+        test_examples,
+    )
+
+    # print("OOD:")
+    # random.seed(RANDOM_SEED)
+    # (train_examples, test_examples) = splits.generate_training_and_test_sets_ood(
+    #     N_INFERENCE_PROBLEMS, N_VARIATIONS, N_EXAMPLES, TRAIN_RATIO,
+    #     length_distribution=LENGTH_DISTRIBUTION,
+    #     answer_at_the_end=ANSWER_AT_THE_END)
+    # generate_t5_split(os.path.join(TARGET_FOLDER, folder_ood_name),
+    #                   f"{folder_ood_name}-train_tf_examples-00000-of-00001",
+    #                   train_examples)
+    # generate_t5_split(os.path.join(TARGET_FOLDER, folder_ood_name),
+    #                   f"{folder_ood_name}-test_tf_examples-00000-of-00001",
+    #                   test_examples)
+    #
+    # print("Length:")
+    # random.seed(RANDOM_SEED)
+    # (train_examples,
+    #  test_examples) = splits.generate_training_and_test_sets_length(
+    #      N_INFERENCE_PROBLEMS,
+    #      N_VARIATIONS,
+    #      N_EXAMPLES,
+    #      LENGTH_SPLIT_THRESHOLD,
+    #      length_distribution=LENGTH_DISTRIBUTION,
+    #      answer_at_the_end=ANSWER_AT_THE_END)
+    # generate_t5_split(
+    #     os.path.join(TARGET_FOLDER, folder_length_name),
+    #     f"{folder_length_name}-train_tf_examples-00000-of-00001", train_examples)
+    # generate_t5_split(
+    #     os.path.join(TARGET_FOLDER, folder_length_name),
+    #     f"{folder_length_name}-test_tf_examples-00000-of-00001", test_examples)
+
+
+if __name__ == "__main__":
+    app.run(main)
diff --git a/data/datasets/logicreference_OA/requirements.txt b/data/datasets/logicreference_OA/requirements.txt
new file mode 100644
index 0000000000..5261ed7c0e
--- /dev/null
+++ b/data/datasets/logicreference_OA/requirements.txt
@@ -0,0 +1,2 @@
+absl_py>=0.13.0
+tensorflow>=2.6.0

Regarding https://github.com/LAION-AI/Open-Assistant/issues/261.

This is an re-produce of the dataset from LogicInference Dataset in paper: https://openreview.net/pdf?id=HAGeIS_Lcg9. I think it will helpful for improving logic inference ability of the model.

The github page of LogicInference Dataset: https://github.com/google-research/google-research/tree/master/logic_inference_dataset.

This dataset is aimed to offer more dataset for Open Assistant project, depending on their demands, there three columns: INSTRUCTION, RESPONSE, SOURCE.

The results in this dataset is a little different from which was introduced in the original paper:

1.For all three splits (IID/OOD/length), only IID is used. In the original paper, it seems that model can reach better performance with data generated by this split method.

2.In the original paper, there are two form of responses: LOGICINFERENCEb (with the answer at the beginning) and LOGICINFERENCEe (with the answer at the end). This dataset uses LOGICINFERENCEe, that means: for all questions, the model will first do logic inference, and give the final answer at the end.

3.The original paper, some parameters in generate_dataset.py are:

N_INFERENCE_PROBLEMS = 5000

N_VARIATIONS = 25

N_EXAMPLES = 200000

TRAIN_RATIO = 0.9

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 0

I choose some new parameters:

N_INFERENCE_PROBLEMS = 10000

N_VARIATIONS = 25

N_EXAMPLES = 55000

TRAIN_RATIO = 1

LENGTH_SPLIT_THRESHOLD = 4

RANDOM_SEED = 1111

The original script generated 4814 different inference problems and extended all those inference problems to around 200,000 Q-A pairs. My settings generated 5491 different inference problems and extended them to around 54,607 Instruction-Response pairs. I think for Open Assistant projects, maybe the number of different inference problems is more important, and generated many similar Instruction-Response pairs will only add training time and doesn't make much sense.