Pandas Melt Function

melt gets you part way there.

In [29]: m = pd.melt(df, id_vars=['Year'], var_name="Name")

This has everything except Group. To get that, we need to reshape d a bit as well.

In [30]: d2 = {}

In [31]: for k, v in d.items():
    for item in v:
        d2[item] = k
   ....:

In [32]: d2
Out[32]: {'Amy': 'A', 'Ben': 'B', 'Bob': 'B', 'Carl': 'C', 'Chris': 'C'}

In [34]: m['Group'] = m['Name'].map(d2)

In [35]: m
Out[35]:
    Year   Name  value Group
0   2013    Amy      2     A
1   2014    Amy      9     A
2   2013    Bob      4     B
3   2014    Bob      2     B
4   2013   Carl      7     C
..   ...    ...    ...   ...
7   2014  Chris      5     C
8   2013    Ben      1     B
9   2014    Ben      5     B
10  2013  Other      3   NaN
11  2014  Other      6   NaN

[12 rows x 4 columns]

And moving ‘Other’ from Name to Group

In [8]: mask = m['Name'] == 'Other'

In [9]: m.loc[mask, 'Name'] = ''

In [10]: m.loc[mask, 'Group'] = 'Other'

In [11]: m
Out[11]:
    Year   Name  value  Group
0   2013    Amy      2      A
1   2014    Amy      9      A
2   2013    Bob      4      B
3   2014    Bob      2      B
4   2013   Carl      7      C
..   ...    ...    ...    ...
7   2014  Chris      5      C
8   2013    Ben      1      B
9   2014    Ben      5      B
10  2013             3  Other
11  2014             6  Other

[12 rows x 4 columns]