第36名方案

首先，我要感谢 @muhakabartay 在每次比赛中不断激励我们进步。我们的解决方案基于 5 个不同的模型。

预处理

我们使用了以下内核中的预处理方法。

1. Ulrich GOUE 的内核：https://www.kaggle.com/ulrich07/do-not-write-off-rnn-cnn
2. Ulrich GOUE 的内核：https://www.kaggle.com/ulrich07/quantile-regression-with-keras

建模

模型 1 架构

x = L.GRU(400, return_sequences=True, name="d1")(x)
x = L.Dropout(0.5)(x)
x = L.Concatenate(name="m1")([x, context])
x = L.GRU(400, return_sequences=True, name="d2")(x)
x = L.Dropout(0.5)(x)
x = L.Concatenate(name="m2")([x, context])
x = L.GRU(400, return_sequences=True, name="d3")(x)
preds = L.Dense(9, activation="linear", name="preds")(x)
model = M.Model(inp, preds, name="M1")
model.compile(loss=wqloss, optimizer="adam")
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.001)

模型 2 架构

x = L.GRU(410, return_sequences=True, name="d1")(x)
x = L.Dropout(0.5)(x)
x = L.Concatenate(name="m1")([x, context])
x = L.GRU(410, return_sequences=True, name="d2")(x)
x = L.Dropout(0.5)(x)
x = L.Concatenate(name="m2")([x, context])
x = L.GRU(410, return_sequences=True, name="d3")(x)
preds = L.Dense(9, activation="linear", name="preds")(x)
model = M.Model(inp, preds, name="M1")
model.compile(loss=wqloss, optimizer="adam")
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2, patience=5, min_lr=0.01)

模型 3 架构

x = L.Concatenate(name="x1")([context, num])
x = L.Dense(500, activation="relu", name="d1")(x)
x = L.Dropout(0.2)(x) # 0.3
x = L.Concatenate(name="m1")([x, context])
x = L.Dense(500, activation="relu", name="d2")(x)
x = L.Dropout(0.2)(x) # 0.3
x = L.Concatenate(name="m2")([x, context])
x = L.Dense(500, activation="relu", name="d3")(x)
preds = L.Dense(9, activation="linear", name="preds")(x)
model = M.Model(inp, preds, name="M1")
model.compile(loss=qloss, optimizer="adam")
return model
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2,
                              patience=5, min_lr=0.001)

模型 4 架构

x = L.Concatenate(name="x1")([context, num])
x = L.Dense(400, activation="relu", name="d1")(x)
x = L.Dropout(0.25)(x) # 0.3
x = L.Concatenate(name="m1")([x, context])
x = L.Dense(400, activation="relu", name="d2")(x)
x = L.Dropout(0.25)(x) # 0.3
x = L.Concatenate(name="m2")([x, context])
x = L.Dense(400, activation="relu", name="d3")(x)
preds = L.Dense(9, activation="linear", name="preds")(x)
model = M.Model(inp, preds, name="M1")
model.compile(loss=qloss, optimizer="adam")
return model
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2,
                              patience=5, min_lr=0.01)

模型 5

基线模型 https://www.kaggle.com/ulrich