From: https://www.kaggle.com/h4211819/leaderboard-analysis
Author: 30CrMnSiA
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import os
print(os.listdir("../input"))
lb = pd.read_csv('../input/imet-leaderboard/imet-2019-fgvc6-publicleaderboard.csv',parse_dates=['SubmissionDate'])
lb.SubmissionDate.max()
The leaderboard-sheet was downloaded in 2019-06-04 13:37:01
team655 = (lb[lb.Score>0.655].TeamId.unique())
print(len(team655))
There are 19 teams break 0.655 when the leaderboard-sheet was downloaded.
lb = lb[lb.Score>0.6]
lb['score_diff'] = lb['Score'] - lb.groupby('TeamId')['Score'].shift(1)
lb_team655 = lb[lb.TeamId.isin(team655)]
lb_team655 = lb_team655.sort_values(by='score_diff',ascending=False)
lb_team655.groupby('TeamId').head(1)
In the 19 teams , there are 6 teams which have a score jump >0.04 after they had got 0.6+ score.They are all from ods.ai
In the 6 teams, there are 5 teams which jump after june 1st(deadline is june 4). They are all from X5.
lb_top_50 = pd.read_csv('../input/imet-leaderboard/lb_score_top50.csv',index_col=[0])
lb_top_50.sort_values(by='diff',ascending=False)
the top 5 teams drop in the private are all from X5.
Why do those five teams in X5 behave the same way?
They said the overfit public , how can they overfit public by one jump submission ?