is purely based on manual efforts and obviously not scalable.

Prior arts. Schema evolution for data that was managed

transform the old schema to the new schema [9], [25], [30],

[51], [61], [66]. (2) To ask the database administrators (DBAs)

or application developers to use a domain-specific language

(DSL) to describe the schema transformation process [11],

[17], [31]–[33], [41], [53], [55], [56], [64].

data, including open data such as publicly available CSV,

JSON, HTML, or text files that can be downloaded from a

URL, and transient data that is directly collected from sensor

devices or machines in real-time and discarded after being

integrated. That’s because the history of schema changes for

amount of time and human resources in manually handling the

system downtimes incurred by schema changes.

A deep learning approach. In this work, we argue for

a new data integration pipeline that uses deep learning to

avoid interruptions caused by the schema changes. In the

past few years, deep learning (DL) has become the most

popular direction in machine learning and artificial intel-

ligence [46], [65], and has transformed a lot of research

areas, such as image recognition, computer vision, speech

recognition, natural language processing, etc.. In recent years,

data integration are usually tolerable as long as most of the

data is correct, which is another motivation of our work. To the

best of our knowledge, we are the first to apply deep learning

representation. Step 2, obfuscations will be automatically

injected to the training data to make the model robust to

various schema changes. Step 3, different model architectures

will be chosen to train the predictive model that will perform

the data integration task depending on the model deploying

environment. Due to the space limitation, this paper will focus

on Step 2 and 3, while we will also discuss Step 1 as well

(1) As to our best knowledge, we are the first to systematically

investigate the application of deep learning and adversarial

training techniques to automatically handle schema changes

occurring in the data sources.

(2) We propose an effective formulation of the data integration

problem into a prediction task as well as flexible feature

these types of schema changes. (Sec. III)

(4) We compare and evaluate various trade-offs made by

different model architectures, including both simple sequence

model and complex transformer model for two different data

integration tasks involving semi-structured and non-structured

data respectively. (Sec. IV and Sec. VI)

converts a set of source datasets into one target dataset, each

row in a CSV file, a JSON object, a time series record, or a file,

there are a few candidate representations for formulating the

predictive task, including dataset-level representation, object-

level representation, attribute-level representation, cell-level

representation, and our proposed super cell based represen-

tation. The coarser-grained of the representations, the fewer

times of inferences are required, and the more efficient of

the prediction. However, a coarser-grained representation also

indicates that the prediction task is more complex and harder

are illustrated in Fig. 2. We find it almost impossible to

assume the target dataset is always tabular and propose a label

representation that includes the tuple identifier (i.e., key of

the target table) of the super cell, the target attribute name

of each cell, and an aggregation mode to specify how values

that are mapped to the same position should be aggregated

into one value, e.g., add, avg, max, min, count, replace old,

discard new, etc..

Super cell formulation. Suppose there are m source datasets,

represented as D = {d

keys shared by each cell in the super cell, attribute names of

each cell, and values of each cell, respectively, represented

as s

a super set to describe the current state of the entire data

repository: S = {s

{(

to one position that is indexed by the target keys

by all cells in the super cell, as well as attribute name of each

cell in the super cell, denoted as

represents the k-th attribute in the target table, as well

is broken down into multiple tuples with each describes the

statistics for a county. Such changes cannot be easily handled

by conventional data integration methodologies.

(3) Attribute name and ordering change. For example, in the

first CSV file added to the JHU COVID-19 daily report data

repository on Jan 22nd, 2020, the third column name is “Last

Update”. But in the CSV file added to the same repository

(4) Value type/format change. For example, in the daily

COVID-19 file created on Jan 22nd, the ”Last Update” has

development costs [79], [84].

(5) Addition or removal of non-key attributes. For example,

the JHU COVID-19 global daily report has changed from six

attributes initially to 14 attributes after a few months. This

change may not make much influence, if the affected attributes

are not used by users’ data integration workloads. On the

contrary, if a required column is totally removed, there is no

way to handle such a situation without interruption, even if

using a deep learning approach, so we mainly focus on the

first four types of schema changes.

Perturbation based on schema changes. First, our super

and reformatting of cell values, are similar to adversarial

attacks, which confuse the pre-trained models by adding

noises to the expected testing samples. Adding perturbations

to training data is an effective way of training robust models

against adversarial attacks [28]. Inspired by this analogy,

we add specially designed perturbations to training data to

handle these parts of schema changes. If we see each super

cell representation as a sequence of words (i.e., a sentence),

the training data is a corpus of sentences. Then we can

fastText [5], which maps related words to vectors that are close

to each other so that the model can recognize the similarity of

such words. We observe through experiments that character-

small model and the BERT base model are freezed, and only

the parameters of the frontend will be updated.

ensure the independent identical distribution (i.e., IID) to avoid

introducing bias.

from all dirty works of wrangling with schema changes.

Therefore it’s critical to reduce the human efforts required

in training data preparation, such as parsing and annotating

data. We propose to automate the training data creation by

utilizing conventional Python code developed for integrating

(2) Higher-order lambda composition functions such as binary

operators: &&, ||, &, |, <,>, ==, +, -,

operator like condition? on_true:on_false; etc..

be represented as Pandas dataframes. Then by traversing the IR

graph, the system can understand the keys and the super cell

mapping relationship. The super cell based processor of each

of atomic computations transforms each super cell represen-

tation accordingly. For example, map operator that transforms

a date cell “2020-10-06” to “Oct 6, 2020” as an example, the

processor takes a super cell {“keys”: [“2020-10-06”, “AZ”,

“US”], “attributes”: [“Date”], “cells”:[“2020-10-06”]} as

input, and outputs {“keys”: [“2020-10-06”, “AZ”, “US”],

“attributes”: [“Date”], “cells”:[“Oct 6, 2020”]} so that

“Arizona”, “United States”], “attributes”:[“datetime”]}}. In

this way, we can obtain training data automatically.

work if the input object is totally nested and opaque and cannot

be represented as a set of cells like Pandas dataframes or Spark

dataframes. For example, a corpus of totally unstructured

text files, unavoidably requires human pre-processing efforts.

Thereby, we design following approaches to further alleviate

the problem: model reusing and crowdsourcing.

unstructured dataset, e.g., a set of opaque and nested objects or

a set of unstructured texts, into a Pandas dataframe or similar

structures, the code generation approach maybe applicable to

and identify the keys. All these tasks are hard to automate. We

consider crowdsourcing as a potential approach to alleviate the

burden from the data scientists or domain experts for these

tasks. However, based on our experiments of crowdsourcing

are not in the domain, and usually datasets are not shipped

with detailed explanations for each attribute. Second, for large

datasets, it is impossible for a person who are not familiar

Model Training For the training process of both scenarios, Bi-

LSTM is relatively slower in converging, requiring around 50

epochs; while the models leveraging pre-trained GPT-2 small

and BERT base are much faster to converge, requiring only

around 5 epochs, as illustrated in Fig. 4.

applied at the inference stage. The accuracy of the data

integration model is defined as the ratio of the number of super

cells that has been predicted with correct target positions and

aggregation actions to the total number of super cells in the

testing data.

Hardware Platform. For all experiments, if without specifi-

cation, we use one NVIDIA Tesla V100 GPU from Google

Colab. All running times (e.g., training time, inference time)

are measured as the average of multiple repeated runs.

Perturbations. We add perturbations such as random changes

to attribute names and values, and replacing attribute names